docs: a beads-merge skill, and the export the driver test regenerated

The skill covers the four things that bit us today: the per-clone git config
the driver needs (.gitattributes names it, but the driver lives in git config,
which a checkout does not carry -- without it git silently line-merges), how to
recover a file that already has markers in it, where genuine same-issue
divergence is reconciled (bd conflicts, against Dolt -- not reconstructed from
the JSONL), and the autostash trap that writes markers without anyone running
`git stash`.

Driver verified against a real conflict: plain text merge produced 3 marker
lines and 3 invalid JSONL lines; the driver produced 470 rows, 0 invalid, and
left the database byte-identical (470 rows before and after).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-21 14:22:58 -07:00
3 changed files with 515 additions and 0 deletions
+95
View File
@@ -0,0 +1,95 @@
---
name: beads-merge
description: Resolve or prevent git conflicts in .beads/issues.jsonl, and reconcile genuinely divergent issues. Use when a merge, rebase, cherry-pick or stash pop conflicts on that file, when auto-export reports "invalid character", or when setting up a fresh clone.
---
# Merging `.beads/issues.jsonl`
**The database is the truth; the file is derived.** `.beads/issues.jsonl` is a
passive export of the local Dolt DB, which syncs over `refs/dolt/data`. Nothing
here ever merges the *file* — it regenerates it.
## The one rule
**Never hand-edit it, never `git checkout` it** (that fires the post-checkout
hook and desynchronises `export-state.json`), and never resolve its conflict
markers by editing. Use `bd` and regenerate.
## Setup — required once per clone
`.gitattributes` names the driver; the driver itself lives in git config, which
is per-clone and **not** carried by a checkout. Without this, git silently falls
back to line-merging and writes conflict markers into the JSONL.
```sh
git config merge.beads.name "beads issues.jsonl: import both sides, regenerate from the database"
git config merge.beads.driver "$PWD/scripts/beads-merge-driver.sh %O %A %B %P"
git check-attr merge -- .beads/issues.jsonl # must print "merge: beads"
```
A worktree shares `.git/config` with its parent, so a worktree needs no separate
setup — but a fresh `git clone` does.
## What the driver does, and why it needs no merge logic
`scripts/beads-merge-driver.sh` imports **both** sides and re-exports. It writes
no merge logic of its own because `bd import` already has the semantics:
- an id the DB has not seen is **created**
- a row whose `updated_at` is **strictly newer** than the local one **wins**
- an older row is **skipped** (reported as `stale_skipped_ids`)
- a tie keeps **every local column** — `updated_at` has second granularity, so a
same-second tie can be two distinct updates, and local wins
So neither import can move the database backwards, which is why importing *ours*
as well as *theirs* is safe: it restores a database that has lost rows without
overwriting one that is ahead.
Set `BEADS_MERGE_PULL=1` to `bd dolt pull` first (remote via `BEADS_MERGE_REMOTE`,
default `gh`). Off by default: a merge driver that reaches the network fails in
ways a merge cannot report.
## Reconciling a genuinely divergent issue
The driver resolves by recency, which is right for the common case (two sides
filing different issues) and **wrong when both sides meaningfully edited the same
issue** — the older edit is skipped silently.
Do **not** reconstruct that from the JSONL. `bd` already does it field by field
against the Dolt working set, which is the real sync path:
```sh
bd conflicts list # which issues are conflicted
bd conflicts show improvise-XXXX # base / ours / theirs, per field
bd conflicts resolve improvise-XXXX --ours # or --theirs
bd conflicts resolve --all --theirs
```
If the driver already took the newer side and you need the other, the losing
content is in git history: `git show <their-commit>:.beads/issues.jsonl`, find
the row, and apply the fields with `bd update`. Never import a whole stale file
to recover one issue — that is what `--allow-stale` does, and it overwrites every
newer local row.
## Recovering a corrupted file
If auto-export reports `failed to compare existing JSONL against local store:
invalid character '<'`, the file has conflict markers in it and **auto-export has
been silently skipping** — possibly for many commits.
```sh
grep -nE '^(<<<<<<<|=======|>>>>>>>)' .beads/issues.jsonl # confirm
bd export -o .beads/issues.jsonl # regenerate from the DB
```
Check whether the markers reached a commit (`git log -S '<<<<<<<' -- .beads/issues.jsonl`).
They are usually working-tree only.
## The trap that caused this
A **`git stash pop`** conflict writes `<<<<<<< Updated upstream` /
`>>>>>>> Stashed changes` into the file. Git's autostash during a dirty-tree
merge or rebase does this **without anyone running `stash` explicitly**, and the
orphaned entry stays in `git stash list`. The merge driver covers stash pops too,
but the habit that avoids it is: **commit before merging**, and treat a stray
`autostash` in `git stash list` as a live hazard.
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/beads-merge
+419
View File
@@ -0,0 +1,419 @@
//! Report-only bug-hunt reproductions: assert the intended behavior, without fixes.
//! Run with `cargo test --test bug_hunt_reproductions -- --ignored --nocapture`.
use improvise_io::import::csv_parser::parse_csv_bytes_with_warnings;
use improvise_io::{
formula::AggFunc,
persistence::{format_md, parse_md},
workbook::Workbook,
};
#[test]
#[ignore = "improvise-3yv3: known bug reproduction"]
fn grouped_item_order_survives_save() {
let mut w = Workbook::new("test");
w.add_category("C").unwrap();
w.add_item_in_group("C", "first", "G").unwrap();
w.add_item("C", "second").unwrap();
let r = parse_md(&format_md(&w)).unwrap();
assert_eq!(
r.model.category("C").unwrap().ordered_item_names(),
w.model.category("C").unwrap().ordered_item_names()
);
}
#[test]
#[ignore = "improvise-ppru: known bug reproduction"]
fn aggregation_survives_save() {
let mut w = Workbook::new("test");
w.add_item("_Measure", "Price").unwrap();
w.model.measure_agg.insert("Price".into(), AggFunc::Avg);
let r = parse_md(&format_md(&w)).unwrap();
assert!(
matches!(r.model.measure_agg.get("Price"), Some(AggFunc::Avg)),
"loaded aggregation: {:?}",
r.model.measure_agg
);
}
#[test]
#[ignore = "improvise-m4rv: known bug reproduction"]
fn deleted_default_stays_deleted() {
let mut w = Workbook::new("test");
w.create_view("Custom");
w.delete_view("Default").unwrap();
let r = parse_md(&format_md(&w)).unwrap();
assert_eq!(
r.views.keys().collect::<Vec<_>>(),
w.views.keys().collect::<Vec<_>>()
);
}
#[test]
#[ignore = "improvise-jc95: known bug reproduction"]
fn csv_nonfinite_is_not_zero() {
for token in ["NaN", "inf", "-inf", "1e999"] {
let (r, _) = parse_csv_bytes_with_warnings(format!("Value\n{token}\n").as_bytes()).unwrap();
assert_ne!(r[0]["Value"].as_f64(), Some(0.0), "{token} became zero");
}
}
#[test]
#[ignore = "improvise-5hfh: known bug reproduction"]
fn duplicate_csv_headers_not_silent() {
let result = parse_csv_bytes_with_warnings(b"A,A\n1,2\n");
assert!(
match result {
Err(error) => error.to_string().to_lowercase().contains("duplicate"),
Ok((_, warnings)) => warnings.iter().any(|warning| {
warning.to_lowercase().contains("duplicate") && warning.contains("A")
}),
},
"duplicate headers silently overwrite a field"
);
}
#[test]
#[ignore = "improvise-l1f5: known bug reproduction"]
fn blank_measure_still_infers_numeric() {
use improvise_io::import::{analyzer::FieldKind, wizard::ImportPipeline};
let (records, _) = parse_csv_bytes_with_warnings(b"Region,Revenue\nEast,10\nWest,\n").unwrap();
let p = ImportPipeline::new(serde_json::Value::Array(records));
let proposal = p.proposals.iter().find(|p| p.field == "Revenue").unwrap();
assert_eq!(proposal.kind, FieldKind::Measure);
}
#[test]
#[ignore = "improvise-2cw9: known bug reproduction"]
fn repeated_import_coordinates_not_silently_overwritten() {
use improvise_io::import::wizard::ImportPipeline;
let (records, _) =
parse_csv_bytes_with_warnings(b"Region,Revenue\nEast,10\nEast,20\n").unwrap();
let (wb, warnings) = ImportPipeline::new(serde_json::Value::Array(records))
.build_model_with_warnings()
.unwrap();
let total: f64 = wb
.model
.data
.iter_cells()
.filter_map(|(_, v)| v.as_f64())
.sum();
assert!(
total == 30.0
|| warnings.iter().any(|warning| {
let warning = warning.to_lowercase();
(warning.contains("duplicate")
|| warning.contains("overwrit")
|| warning.contains("overwrot"))
&& warning.contains("east")
&& warning.contains("revenue")
}),
"total {total}, no warning {warnings:?}"
);
}
#[test]
#[ignore = "improvise-55v2: known bug reproduction"]
fn nonfinite_cell_value_survives_wire() {
use improvise::protocol::{CellValue, Command};
let cmd = Command::SetYanked(Some(CellValue::Number(f64::INFINITY)));
let json = serde_json::to_string(&cmd).unwrap();
assert!(
serde_json::from_str::<Command>(&json).is_ok(),
"unreadable wire: {json}"
);
}
#[test]
#[ignore = "improvise-8l0c: known bug reproduction"]
fn formula_panel_unicode_does_not_panic() {
use improvise::{
protocol::FormulaCache,
ui::{app::AppMode, formula_panel::FormulaContent, panel::Panel},
};
use ratatui::{buffer::Buffer, layout::Rect, widgets::Widget};
let cache = FormulaCache {
formulas: vec!["|é| = 1".into()],
};
let area = Rect::new(0, 0, 7, 5);
let mut buffer = Buffer::empty(area);
Panel::new(
FormulaContent::new(&cache, &AppMode::Normal),
&AppMode::Normal,
0,
)
.render(area, &mut buffer);
}
#[test]
#[ignore = "improvise-vw47: known bug reproduction"]
fn new_view_after_delete_does_not_overwrite_survivor() {
use improvise::{
command::{
cmd::{Cmd, panel::CreateAndSwitchView},
key::{KeyCode, KeyModifiers},
},
ui::app::App,
};
let mut w = Workbook::new("test");
w.create_view("View 2");
w.create_view("View 3").number_format = ".4".into();
w.delete_view("View 2").unwrap();
let mut app = App::new(w, None);
let effects =
CreateAndSwitchView.execute(&app.cmd_context(KeyCode::Char('n'), KeyModifiers::NONE));
app.apply_effects(effects);
assert_eq!(app.model_state.workbook.views["View 3"].number_format, ".4");
assert_eq!(app.model_state.workbook.views.len(), 3);
}
#[test]
#[ignore = "improvise-cj4u: known bug reproduction"]
fn label_stays_off_axes_in_new_view() {
use improvise_io::view::Axis;
let mut w = Workbook::new("test");
w.add_label_category("Description").unwrap();
assert_eq!(w.active_view().axis_of("Description"), Axis::None);
assert_eq!(w.create_view("Other").axis_of("Description"), Axis::None);
}
#[test]
#[ignore = "improvise-k8i: known bug reproduction"]
fn long_csv_row_is_not_silently_dropped() {
let r = parse_csv_bytes_with_warnings(b"A,B\n1,2,3\n");
assert!(
match r {
Err(error) => {
let message = error.to_string().to_lowercase();
message.contains("3")
&& message.contains("2")
&& (message.contains("field") || message.contains("column"))
}
Ok((_, warnings)) => warnings.iter().any(|warning| {
let warning = warning.to_lowercase();
warning.contains("3")
&& warning.contains("2")
&& (warning.contains("field") || warning.contains("column"))
}),
},
"extra value 3 dropped silently"
);
}
// Passing control: embedded carriage returns already survive persistence.
#[test]
fn literal_cr_name_survives_roundtrip() {
let mut w = Workbook::new("test");
w.add_category("C").unwrap();
w.add_item("C", "x\ry").unwrap();
let text = format_md(&w);
let r = parse_md(&text).unwrap();
assert_eq!(
r.model.category("C").unwrap().ordered_item_names(),
vec!["x\ry"]
);
}
#[test]
#[ignore = "improvise-pnf5: known bug reproduction"]
fn scalar_cell_survives_roundtrip() {
use improvise_io::model::cell::{CellKey, CellValue};
let mut w = Workbook::new("test");
w.model
.set_cell(CellKey::new(vec![]), CellValue::Number(42.0));
let text = format_md(&w);
assert!(
parse_md(&text).is_ok(),
"writer emitted unreadable document: {text}"
);
}
fn revenue_book() -> Workbook {
use improvise_io::model::cell::{CellKey, CellValue};
let mut w = Workbook::new("T");
w.add_category("Region").unwrap();
w.add_item("Region", "East").unwrap();
w.add_item("_Measure", "Revenue").unwrap();
w.model.set_cell(
CellKey::new(vec![
("Region".into(), "East".into()),
("_Measure".into(), "Revenue".into()),
]),
CellValue::Number(10.0),
);
w
}
#[test]
#[ignore = "improvise-s2w3: known bug reproduction"]
fn sum_of_derived_measure_agrees_with_identity_expression() {
use improvise_io::{formula::parse_formula, model::cell::CellKey};
let mut w = revenue_book();
for raw in [
"Double = Revenue * 2",
"A = SUM(Double)",
"B = SUM(Double + 0)",
] {
w.model.add_formula(parse_formula(raw, "_Measure").unwrap());
}
let key = CellKey::new(vec![("Region".into(), "East".into())]);
let a = w.model.evaluate(&key.clone().with("_Measure", "A"));
let b = w.model.evaluate(&key.with("_Measure", "B"));
assert_eq!(a, b);
}
#[test]
#[ignore = "improvise-s08t: known bug reproduction"]
fn unknown_aggregate_reference_is_not_all_measures() {
use improvise_io::{
formula::parse_formula,
model::cell::{CellKey, CellValue},
};
let mut w = revenue_book();
w.model
.add_formula(parse_formula("Total = SUM(Typo)", "_Measure").unwrap());
let key = CellKey::new(vec![
("Region".into(), "East".into()),
("_Measure".into(), "Total".into()),
]);
assert!(
matches!(w.model.evaluate(&key), Some(CellValue::Error(_))),
"actual {:?}",
w.model.evaluate(&key)
);
}
#[test]
#[ignore = "improvise-3b2b: known bug reproduction"]
fn category_panel_cannot_delete_required_virtual() {
use improvise::{
command::{
cmd::{Cmd, panel::DeleteCategoryAtCursor},
key::{KeyCode, KeyModifiers},
},
ui::app::App,
};
let mut app = App::new(revenue_book(), None);
app.view_state.cat_panel_cursor = 0;
let fx =
DeleteCategoryAtCursor.execute(&app.cmd_context(KeyCode::Char('d'), KeyModifiers::NONE));
app.apply_effects(fx);
let fx = improvise::command::cmd::grid::ToggleRecordsMode
.execute(&app.cmd_context(KeyCode::Char('r'), KeyModifiers::NONE));
app.apply_effects(fx);
assert!(
app.layout.is_records_mode(),
"deleting _Index prevents records mode; status {}",
app.view_state.status_msg
);
}
#[test]
#[ignore = "improvise-8l0c: known bug reproduction"]
fn formula_panel_handles_zero_inner_width() {
use improvise::{
protocol::FormulaCache,
ui::{app::AppMode, formula_panel::FormulaContent, panel::Panel},
};
use ratatui::{buffer::Buffer, layout::Rect, widgets::Widget};
let cache = FormulaCache {
formulas: vec!["A = 1".into()],
};
let area = Rect::new(0, 0, 2, 5);
let mut buffer = Buffer::empty(area);
Panel::new(
FormulaContent::new(&cache, &AppMode::Normal),
&AppMode::Normal,
0,
)
.render(area, &mut buffer);
}
#[test]
#[ignore = "improvise-pnf5: known bug reproduction"]
fn save_with_unfinished_record_is_readable() {
use improvise::{
command::{
cmd::{
Cmd,
grid::{AddRecordRow, ToggleRecordsMode},
},
key::{KeyCode, KeyModifiers},
},
ui::app::App,
};
let mut app = App::new(revenue_book(), None);
let fx = ToggleRecordsMode.execute(&app.cmd_context(KeyCode::Char('r'), KeyModifiers::NONE));
app.apply_effects(fx);
assert!(app.layout.is_records_mode());
let fx = AddRecordRow.execute(&app.cmd_context(KeyCode::Char('o'), KeyModifiers::NONE));
app.apply_effects(fx);
let text = format_md(&app.model_state.workbook);
assert!(parse_md(&text).is_ok(), "saved unparseable file: {text}");
}
#[test]
#[ignore = "improvise-7iww: known bug reproduction"]
fn circular_formula_not_a_plausible_cached_number() {
use improvise_io::{
formula::parse_formula,
model::{
Aggregation,
cell::{CellKey, CellValue},
},
};
let mut w = revenue_book();
w.model
.add_formula(parse_formula("Revenue = Revenue + 1", "_Measure").unwrap());
let key = CellKey::new(vec![
("Region".into(), "East".into()),
("_Measure".into(), "Revenue".into()),
]);
assert!(matches!(w.model.evaluate(&key), Some(CellValue::Error(_))));
w.model
.recompute_formulas(&["_Index".into(), "_Dim".into()]);
let actual = w.model.evaluate_aggregated(&key, Aggregation::OverHidden);
assert!(
matches!(actual, Some(CellValue::Error(_))),
"circular formula became {actual:?}"
);
}
#[test]
#[ignore = "improvise-yae4: known bug reproduction"]
fn wide_cell_clamps_without_overflow() {
use improvise::{
model::cell::{CellKey, CellValue},
ui::app::App,
};
let mut w = revenue_book();
w.model.set_cell(
CellKey::new(vec![
("Region".into(), "East".into()),
("_Measure".into(), "Revenue".into()),
]),
CellValue::Text("x".repeat(65535)),
);
let app = App::new(w, None);
let widths = improvise::ui::viewport::compute_col_widths(
&app.model_state.workbook.model,
&app.layout,
false,
0,
);
assert_eq!(
widths,
vec![improvise::ui::viewport::MAX_COL_WIDTH],
"the Revenue column must reach the width cap"
);
}
#[test]
fn ordinary_controls() {
use improvise_io::import::{analyzer::FieldKind, wizard::ImportPipeline};
let (records, warnings) =
parse_csv_bytes_with_warnings(b"Region,Revenue\nEast,10\nWest,20\n").unwrap();
assert!(warnings.is_empty());
let p = ImportPipeline::new(serde_json::Value::Array(records));
assert_eq!(
p.proposals
.iter()
.find(|p| p.field == "Revenue")
.unwrap()
.kind,
FieldKind::Measure
);
let (w, warnings) = p.build_model_with_warnings().unwrap();
assert!(warnings.is_empty());
assert_eq!(
w.model
.data
.iter_cells()
.filter_map(|(_, v)| v.as_f64())
.sum::<f64>(),
30.0
);
assert!(parse_md(&format_md(&w)).is_ok());
}