fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

chore: update repo-map

Browse Source
This commit is contained in:
Ed L
2026-04-16 16:05:42 -07:00
parent 6362078032
commit 47313c4e80
2 changed files with 214 additions and 500 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.beads/issues.jsonl
View File

@ -1,3 +1,7 @@
{"id":"improvise-4yc","title":"Malformed numbers in .improv file silently become 0.0","description":"crates/improvise-io/src/persistence/mod.rs:434 uses value_pair.as_str().parse().unwrap_or(0.0). Malformed number literals (e.g. '3.14.15', '1e999', '--5') become CellValue::Number(0.0) with no error. This is the only production unwrap/unwrap_or in the parse path and silently corrupts data. Fix: return a parse error with line context, or at minimum log a warning and use CellValue::Error.","status":"open","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:17Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:17Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-k8i","title":"CSV import silently drops columns when row widths mismatch","description":"crates/improvise-io/src/import/csv_parser.rs:14-20 uses ReaderBuilder with .flexible(true). Rows with fewer columns than the header are truncated with no warning. User data is silently lost on import. Fix: either enforce .flexible(false) for strict RFC 4180 parsing, or detect mismatched row widths and surface a warning to the import wizard.","status":"open","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:15Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:15Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-6kj","title":"Pest parse errors lack file:line context for users","description":"persistence/mod.rs:294 wraps pest errors as 'Parse error: {e}' which shows Pair offsets like 'Pair(6..12), expected \"=\", found \",\"' instead of a line preview. Users debugging a malformed .improv file have no way to find the offending line. Fix: extract (line, col) from pest::error::Error via line_col(), include the line contents and surrounding context in the wrapped error.","status":"open","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:13Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:13Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-dqn","title":"CyclePanelFocus does not actually cycle between panels","description":"The CyclePanelFocus command at src/command/cmd/panel.rs:299-313 is documented as 'Tab through open panels' but the if/else if chain always resolves to the first open panel (formula \u003e category \u003e view). Pressing the bound key repeatedly never rotates. Test at panel.rs:84 only asserts a panel mode is entered, not rotation. Fix: either implement real rotation using ctx.mode to determine 'next' open panel, or rename to FocusFirstOpenPanel. Also replace the three bool fields with an enum of open panel states.","status":"open","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:07Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:07Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-hmu","title":"TAB/Enter on bottom-right of records view should insert record below (TAB→first cell, Enter→same column); o inserts below cursor","description":"Current records view data entry is awkward. On last row + rightmost cell: TAB should add record below + go to first cell of new row + edit. Enter should add below + stay in same column + edit. 'o' should insert below cursor like Vim (not always at end). Red-green-refactor: add failing test first, then minimal implementation in layout.rs + related command/effect code, then refactor duplication. Builds on improvise-rbv and improvise-c5v.","status":"closed","priority":1,"issue_type":"feature","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-15T10:37:52Z","created_by":"Edward Langley","updated_at":"2026-04-16T05:08:00Z","closed_at":"2026-04-16T05:08:00Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-fpo","title":"Dynamic _Measure: formula targets auto-included","description":"_Measure should dynamically include all formula targets without add_formula manually adding items. Formula refs should resolve against Model::formulas as fallback. CommitFormula should not need to pick a target category for _Measure.","status":"closed","priority":1,"issue_type":"bug","assignee":"spot","owner":"el-github@elangley.org","created_at":"2026-04-09T10:14:14Z","created_by":"spot","updated_at":"2026-04-09T20:52:01Z","closed_at":"2026-04-09T20:52:01Z","close_reason":"Dynamic _Measure, optional target_category, atomic formula eval on load, skip virtual category persistence","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-h4r","title":"Replace ad-hoc parser with pest grammar","description":"The pest grammar at src/persistence/improv.pest defines the .improv format spec. Replace the hand-written line-scanner in parse_md with a pest-derived parser that walks the grammar's parse tree. This eliminates drift between spec and implementation.","status":"closed","priority":1,"issue_type":"task","assignee":"spot","owner":"el-github@elangley.org","created_at":"2026-04-09T09:03:53Z","created_by":"spot","updated_at":"2026-04-09T09:32:38Z","closed_at":"2026-04-09T09:32:38Z","close_reason":"Replaced ad-hoc line scanner with pest-derived parser. Grammar at improv.pest is now the single source of truth for both parsing and test generation. Grammar-walking generator uses pest_meta to read the grammar AST at test time and produce random valid files as proptest entropy.","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -10,7 +14,20 @@
{"id":"improvise-km8","title":"1.2 Fix Cargo.toml metadata","description":"Add missing package fields: description, repository, homepage, documentation, readme, keywords, categories, license. Description: 'Terminal pivot-table modeling in the spirit of Lotus Improv'. Keywords: tui, pivot, spreadsheet, data, improv. Categories: command-line-utilities, visualization.","status":"closed","priority":1,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:06:48Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:20:17Z","closed_at":"2026-04-09T07:20:17Z","close_reason":"Metadata already present: description, license, repository, homepage, readme, keywords, categories all populated.","dependency_count":0,"dependent_count":2,"comment_count":0}
{"id":"improvise-902","title":"1.1 Audit and remove or update context/SPEC.md","description":"Compare context/SPEC.md against actual code. If spec contradicts code significantly, delete it. Optionally move salvageable conceptual content to docs/design-notes.md with a staleness disclaimer. Do not try to bring spec into sync.","status":"closed","priority":1,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:06:46Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:10:05Z","closed_at":"2026-04-09T07:10:05Z","close_reason":"Audited SPEC.md against code. Found minor inaccuracies (storage internals, wizard step count, search mode representation) but no major architectural contradictions. Deleted SPEC.md since it was largely redundant with repo-map.md and design-principles.md. Salvaged product vision and non-goals into docs/design-notes.md with staleness disclaimer.","dependency_count":0,"dependent_count":1,"comment_count":0}
{"id":"improvise-tik","title":"Phase 1: Repository hygiene","description":"Raise the quality floor of what a stranger sees when they land on the repo. Subtasks cover spec audit, Cargo.toml metadata, publish-readiness, CSV audit, and example files.","status":"closed","priority":1,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-09T04:05:45Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:22:34Z","closed_at":"2026-04-09T07:22:34Z","close_reason":"All 5 subtasks complete: SPEC.md audited, Cargo.toml metadata fixed, publish dry-run clean, CSV audit done, example files created.","dependencies":[{"issue_id":"improvise-tik","depends_on_id":"improvise-2fr","type":"blocks","created_at":"2026-04-08T23:37:31Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-902","type":"blocks","created_at":"2026-04-08T23:37:30Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-bv1","type":"blocks","created_at":"2026-04-08T23:37:32Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-km8","type":"blocks","created_at":"2026-04-08T23:37:30Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-n1h","type":"blocks","created_at":"2026-04-08T23:37:32Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"id":"improvise-dwe","title":"Split App into AppState + App wrapper (in-place, no crate change)","description":"Prerequisite to improvise-3mm (extract improvise-command). Refactor App into two parts without moving files across crates: (a) AppState, the pure semantic state that effects mutate (workbook, mode, status_msg, dirty, buffers, expanded_cats, yanked, drill_state, search_query, search_mode, transient_keymap, view_back/forward_stack, panel cursors + open flags, help_page, wizard, file_path); (b) App, a thin wrapper in ui/app.rs that owns { state: AppState, term_width, term_height, layout: GridLayout, last_autosave, keymap_set }. Change Effect::apply's signature from \u0026mut App to \u0026mut AppState. For the small number of effects that today read App-level state at apply time (EnterEditAtCursor reads display_value from layout, etc.), pre-compute the derived value in the command's Cmd::execute (where CmdContext has layout access) and pass it through the effect struct's own fields (self). Apply bodies become pure AppState writes.","design":"AppState holds everything currently mutated by Effect::apply, minus rendering caches. App keeps term_width, term_height, layout, last_autosave. apply_effects loop: for e in effects { e.apply(\u0026mut self.state); } then self.rebuild_layout(). cmd_context() is built from both App (for visible_rows/cols via layout + term dims) and AppState (for model/view/mode/buffers). For effects that need layout-derived values at apply time: compute in execute (ctx has layout), bake into self. Example: EnterEditAtCursor { target_mode, initial_value: String } — initial_value is computed pre-effect from ctx.layout and baked in; apply does state.buffers.insert(\"edit\", self.initial_value.clone()); state.mode = self.target_mode.clone();","acceptance_criteria":"(1) AppState struct exists; App wraps it. (2) Effect::apply takes \u0026mut AppState, not \u0026mut App. (3) CmdContext.workbook still resolves to \u0026AppState.workbook. (4) rebuild_layout / term-dim tracking remain on App. (5) All existing tests pass. (6) cargo clippy --workspace --tests clean. (7) No behavior change.","notes":"Do this in-place BEFORE the crate-split (improvise-3mm). Keeps the diff isolated from packaging churn. Audit every impl Effect for Foo — most already only touch fields that belong in AppState; the ones that don't need pre-compute refactors in their associated Cmd.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T06:28:47Z","created_by":"spot","updated_at":"2026-04-16T06:28:47Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-tvt","title":"Route mutations through Workbook facade instead of app.workbook.model chains","description":"40+ sites follow the pattern 'if let Some(cat) = app.workbook.model.category_mut(\u0026name) { cat.add_item(\u0026item); }' (src/ui/effect.rs:38, 54, 99 and many test setups in src/command/cmd/mod.rs, src/ui/grid.rs).\\n\\nSimilarly .views.get().unwrap().axis_of() appears in 11 sites (workbook.rs:167-177, model/types.rs:1868-1890, persistence:943-946,1165) even though Workbook::active_view() / active_view_mut() already exist and are used correctly in some places.\\n\\nFix: add facade methods on Workbook: add_item(cat, item), remove_item(cat, item), view_axis(view_name, cat), sort_cells(), cell_count(). Migrate the existing chains to use them.\\n\\nCorrectness win: (a) Workbook becomes a real facade with the option to enforce invariants (e.g., notify views when items change); (b) Tests and effects stop reaching through three layers; (c) future refactors to Model/DataStore internals don't cascade into 40+ call sites.\\n\\n~60 LOC in production + simplification of many tests.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:02:18Z","created_by":"Ed L","updated_at":"2026-04-16T19:02:18Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-2hi","title":"Extract AppMode dispatch methods (mode_label, mode_style, mode_key)","description":"AppMode is matched in ~52 sites across 8 files to derive UI labels, styles, and mode keys. Each match arm is a small piece of per-variant logic that should live on the variant itself.\\n\\nHot spots:\\n- src/draw.rs:119-149 — mode_name() and mode_style() match blocks (16+ arms)\\n- src/command/keymap.rs:69-89 — ModeKey::from_app_mode match (16 arms)\\n- src/ui/{formula,category,view}_panel.rs — matches!(mode, AppMode::X) checks\\n- src/ui/tile_bar.rs, src/ui/effect.rs:898-900, src/command/cmd/registry.rs:376\\n- src/ui/app.rs: 13 scattered matches!() checks\\n\\nFix: add methods on AppMode — mode_label() -\u003e \u0026'static str, mode_style() -\u003e Style, mode_key() -\u003e Option\u003cModeKey\u003e, panel_mode() -\u003e Option\u003cPanel\u003e, is_text_entry_mode() -\u003e bool (partially present via minibuffer()). Also extend the existing minibuffer() with buffer_key() convenience.\\n\\nCorrectness win: adding a new AppMode variant requires updating the enum's methods (one file), not chasing 8 files with match blocks. Exhaustive match inside the method body still forces the compiler to flag every new variant.\\n\\n~80 LOC collapsed and 7 files depend on fewer concerns of AppMode.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:02:09Z","created_by":"Ed L","updated_at":"2026-04-16T19:02:09Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"id":"improvise-2lh","title":"Migrate name-string comparisons to CategoryKind enum checks","description":"8 sites compare category names as strings against virtual-category prefixes ('_Measure', '_Index', '_Dim') instead of checking the typed CategoryKind enum. Adding a new CategoryKind variant won't produce a compile error at any of these sites.\\n\\nSites:\\n- crates/improvise-core/src/model/types.rs:215, 226, 429, 636 (target_category == '_Measure', cat_name == '_Measure')\\n- crates/improvise-io/src/persistence/mod.rs:213, 226 (formula target comparison)\\n- crates/improvise-core/src/view/layout.rs:108 (c == '_Index' \u0026\u0026 c == '_Dim')\\n- src/command/cmd/grid.rs:306 (target_category == '_Measure')\\n\\nFix: add methods on CategoryKind: is_virtual(), is_measure(), is_persisted(), is_index_or_dim(). Look up the Category by name once, dispatch on its .kind.\\n\\nCorrectness win: the compiler flags every site when a new variant is added. Also removes the hardcoded-name dependency — if the underscore prefix convention ever changes, one place to edit.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:01:53Z","created_by":"Ed L","updated_at":"2026-04-16T19:01:53Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-6sq","title":"Merge EditOrDrill and EnterEditAtCursorCmd into EnterEdit{mode}","description":"command/cmd/mode.rs:254-307 has two commands (EditOrDrill, EnterEditAtCursorCmd) that each thread an edit_mode parameter. The drill-vs-edit decision is mode-independent; merging eliminates the fork.\\n\\nAbstraction: EnterEdit { mode: AppMode } — single command that checks aggregation state, pre-fills the buffer if editing, then enters the supplied mode.\\n\\nMatches the parameterization precedent from commit 30383f2 ('parameterize mode-related commands and effects'). ~50 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:39Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:39Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-6t3","title":"Unify Commit{Formula,CategoryAdd,ItemAdd} into CommitBuffer(kind)","description":"command/cmd/commit.rs:349-439 has three commands with identical shape: read buffer named X, if empty abort, else emit AddY effect, mark dirty, optionally exit mode. Only the buffer name and the target effect type differ.\\n\\nAbstraction: CommitBuffer { kind: CommitKind } where CommitKind is Formula | Category | Item.\\n\\nCorrectness win: minibuffer commit semantics live in one place. New minibuffer modes (rename-view, rename-item) get correct semantics for free. Pairs with improvise-{text-entry keymap template} — once the commit is generic, the keymap template composes Sequence [commit-buffer X, clear-buffer X, enter-mode Normal] uniformly across all text-entry modes.\\n\\n~40 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:33Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:33Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"id":"improvise-k8h","title":"Consolidate minibuffer AppMode constructors","description":"ui/app.rs:54-185 has 7 AppMode minibuffer variants (Editing, RecordsEditing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode) that each hand-build MinibufferConfig { buffer_key, prompt, color, .. }. The buffer_key string must match what AppendChar/PopChar/commit commands look up in ctx.buffers — nothing enforces this.\\n\\nAbstraction: impl AppMode { fn editing() -\u003e Self; fn command_mode() -\u003e Self; ... } with buffer_key drawn from a shared constant or enum.\\n\\nNote: full enum refactor to AppMode::Minibuffer { kind, config } is not worth the churn — most of the 35 match sites use { .. } and don't care about the payload; the constructor consolidation captures the correctness win (single buffer_key source) without the enum refactor.\\n\\n~40 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:25Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:25Z","dependencies":[{"issue_id":"improvise-k8h","depends_on_id":"improvise-2hi","type":"blocks","created_at":"2026-04-16T12:02:40Z","created_by":"Ed L","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"id":"improvise-4pg","title":"Extract panel mode keymap template","description":"command/keymap.rs:554-713 has three nearly-identical panel keymaps (FormulaPanel, CategoryPanel, ViewPanel) sharing Esc/Tab/j/k/F/C/V/: bindings. Only panel name varies.\\n\\nAbstraction: fn panel_keymap(panel: Panel) -\u003e Keymap\\n\\nCorrectness win: adding a 4th panel becomes a one-line call; prevents per-panel binding drift. Also helps keep CyclePanelFocus keybinding consistent once improvise-dqn is fixed.\\n\\n~80 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:24Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:24Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-9cn","title":"Introduce register_cmd! macro to prevent name drift","description":"src/command/cmd/registry.rs has 20+ register_pure(\u0026SomeCmd(vec![]), SomeCmd::parse) sites where the registered name string must match SomeCmd::name(). Nothing enforces this match — a typo produces a silent dispatch failure.\\n\\nAbstraction: register_cmd!(r, SomeCmd) macro that derives the name from the Cmd impl at compile time.\\n\\nCorrectness win: compile-time guarantee that the registered name matches Cmd::name(). Subsumes improvise-61f (the runtime invariant test); keeping 61f as a belt-and-braces check is fine but may be unnecessary once this lands.\\n\\n~40 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:22Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:22Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"id":"improvise-w3q","title":"Extract text-entry keymap template for minibuffer modes","description":"command/keymap.rs:770-966 hand-binds Enter/Esc/Tab/Backspace/AnyChar for 8 modes (Editing, RecordsEditing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode, SearchMode). Only buffer name, commit command, and exit mode vary between them.\\n\\nAbstraction: fn text_entry_keymap(buffer: \u0026str, commit: \u0026str, exit: AppMode, parent: Option\u003cArc\u003cKeymap\u003e\u003e) -\u003e Keymap\\n\\nCorrectness win: impossible to omit clear-buffer in one mode's Esc sequence (the exact class of bug fixed in a recent refactor). New text modes inherit semantics for free. Pairs naturally with generic CommitBuffer(kind) — once the commit primitive is unified, the template composes Sequence [commit-buffer X, clear-buffer X, enter-mode Normal] uniformly.\\n\\n~150 LOC collapsed.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:14Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:14Z","dependencies":[{"issue_id":"improvise-w3q","depends_on_id":"improvise-6t3","type":"blocks","created_at":"2026-04-16T11:54:32Z","created_by":"Ed L","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"id":"improvise-0bf","title":"Inconsistent float-equality tolerance in formula eval","description":"crates/improvise-core/src/model/types.rs uses different float-equality strategies: division checks 'rv == 0.0' exactly (:674), while = and != comparisons use '1e-10' epsilon (:555, :749). A formula like IF(x = 0, ..., y/x) with x = 1e-11 treats x as zero in the condition but divides without error. Either unify on a single FLOAT_EQ_EPSILON constant or document the design choice explicitly in comments.","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:35Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:35Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-kdp","title":"Duplicated aggregation dispatch block","description":"crates/improvise-core/src/model/types.rs:305-312 and :598-604 contain identical Sum/Avg/Min/Max/Count match blocks against AggFunc. If business logic changes (NaN handling, default agg), updates must be made in both sites. Fix: extract fn apply_agg(values: \u0026[f64], func: \u0026AggFunc) -\u003e f64 as a pure helper.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:29Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:29Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-nrs","title":"Duplicated find_item_category helper in eval paths","description":"crates/improvise-core/src/model/types.rs has identical nested find_item_category helpers at lines 420 and 627 (inside eval_formula_with_cache and eval_formula_depth). Both search all categories for an item, then fall back to formula targets. Fix: extract as Model::find_item_category(\u0026self, item: \u0026str) -\u003e Option\u003c\u0026str\u003e so bug fixes apply in one place.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:26Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:26Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-6os","title":"O(n^2) stem deduplication in recompute_formulas","description":"crates/improvise-core/src/model/types.rs:346-359 accumulates unique formula stems using 'if !stems.contains(\u0026stripped) { stems.push(stripped); }'. This is O(n^2) over every cell for every formula category. recompute_formulas runs at startup and after every data mutation. Large workbooks will show visible slowdown. Fix: collect into HashSet\u003cCellKey\u003e, convert to Vec once for iteration order.","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:24Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:24Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-dwe","title":"Split App into AppState + App wrapper (in-place, no crate change)","description":"Prerequisite to improvise-3mm (extract improvise-command). Refactor App into two parts without moving files across crates: (a) AppState, the pure semantic state that effects mutate (workbook, mode, status_msg, dirty, buffers, expanded_cats, yanked, drill_state, search_query, search_mode, transient_keymap, view_back/forward_stack, panel cursors + open flags, help_page, wizard, file_path); (b) App, a thin wrapper in ui/app.rs that owns { state: AppState, term_width, term_height, layout: GridLayout, last_autosave, keymap_set }. Change Effect::apply's signature from \u0026mut App to \u0026mut AppState. For the small number of effects that today read App-level state at apply time (EnterEditAtCursor reads display_value from layout, etc.), pre-compute the derived value in the command's Cmd::execute (where CmdContext has layout access) and pass it through the effect struct's own fields (self). Apply bodies become pure AppState writes.","design":"AppState holds everything currently mutated by Effect::apply, minus rendering caches. App keeps term_width, term_height, layout, last_autosave. apply_effects loop: for e in effects { e.apply(\u0026mut self.state); } then self.rebuild_layout(). cmd_context() is built from both App (for visible_rows/cols via layout + term dims) and AppState (for model/view/mode/buffers). For effects that need layout-derived values at apply time: compute in execute (ctx has layout), bake into self. Example: EnterEditAtCursor { target_mode, initial_value: String } — initial_value is computed pre-effect from ctx.layout and baked in; apply does state.buffers.insert(\"edit\", self.initial_value.clone()); state.mode = self.target_mode.clone();","acceptance_criteria":"(1) AppState struct exists; App wraps it. (2) Effect::apply takes \u0026mut AppState, not \u0026mut App. (3) CmdContext.workbook still resolves to \u0026AppState.workbook. (4) rebuild_layout / term-dim tracking remain on App. (5) All existing tests pass. (6) cargo clippy --workspace --tests clean. (7) No behavior change.","notes":"Do this in-place BEFORE the crate-split (improvise-3mm). Keeps the diff isolated from packaging churn. Audit every impl Effect for Foo — most already only touch fields that belong in AppState; the ones that don't need pre-compute refactors in their associated Cmd.","status":"closed","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T06:28:47Z","created_by":"spot","updated_at":"2026-04-16T23:02:32Z","closed_at":"2026-04-16T23:02:32Z","dependencies":[{"issue_id":"improvise-dwe","depends_on_id":"improvise-vb4","type":"supersedes","created_at":"2026-04-16T16:02:31Z","created_by":"Ed L","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-3zq","title":"Records-mode Enter/Tab at bottom-right: persist empty record, re-enter editing","description":"Two related bugs at the bottom-right cell of records view:\n\n1. Enter on bottom-right cell re-enters editing at the same cell (because EnterAdvance stays put at bottom-right and EnterEditAtCursor then re-edits). Expected: exit edit mode.\n\n2. Tab on bottom-right cell in records mode calls AddRecordRow, which SetCells with an empty CellKey when no page filters are active (the default records view has _Measure/foo on None axis, not Page). This writes a nonsense ' = 0' cell to the model that gets persisted (see sample.improv line 26: ' = 0'). Re-toggling records mode (RR) shows the empty record at the top of the table.\n\nRoot causes:\n- CommitAndAdvance (Down) always appends EnterEditAtCursor, even when no forward motion is possible.\n- AddRecordRow::execute unconditionally produces SetCell(empty_key, 0) when coords vec is empty, rather than bailing out.\n\nTests should be added in crates/improvise-persistence (or wherever) that demonstrate ' = 0' is not persisted, plus command-level tests that:\n- CommitAndAdvance(Down) at bottom-right produces a non-editing mode change.\n- AddRecordRow with no page filters produces no SetCell effect.","acceptance_criteria":"1. AddRecordRow with empty coords emits only a status message, no SetCell.\n2. commit-cell-edit (Enter) at bottom-right exits editing (lands in Normal/RecordsNormal).\n3. Tab at bottom-right of records view does not create an empty cell in the model.\n4. All new tests fail on main before the fix; pass after.","notes":"Bug #1 (Enter at bottom-right re-enters editing) — fixed via effect-chain abort mechanism:\n- Added App::abort_effects flag; apply_effects short-circuits and resets per batch\n- New AbortChain effect sets the flag\n- EnterAdvance at bottom-right now emits only AbortChain\n- CommitAndAdvance pushes change_mode(exit_mode_for(edit_mode)) BEFORE advance; trailing EnterEditAtCursor lifts back to editing only if advance succeeded\n\nBug #2 (Tab persists empty record across RR) — fixed by cleanup-on-leave:\n- New CleanEmptyRecords effect removes cells with empty CellKey\n- ToggleRecordsMode on leave emits [CleanEmptyRecords, ViewBack, status] — inverse of the SortData on entry\n\n5 new tests (2 integration demonstrating the bugs, 3 unit for the new primitives). 366 tests pass; clippy clean.","status":"closed","priority":2,"issue_type":"bug","assignee":"fido","owner":"el-github@elangley.org","created_at":"2026-04-16T06:01:20Z","created_by":"fido","updated_at":"2026-04-16T06:26:44Z","closed_at":"2026-04-16T06:26:44Z","close_reason":"Both bugs fixed; see notes for implementation summary. All 5 new tests pass; 366 workspace tests green; clippy clean.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-4ju","title":"Make EditOrDrill mode-agnostic; delete EnterEditMode","description":"EnterEditMode::execute currently checks ctx.mode.is_records() to decide between editing() and records_editing(). Per the keymap design principle (commands/effects must be mode-agnostic; modes switch keymaps), this is wrong. EditOrDrill (the only caller) should take an edit_mode parameter, and the records-mode keymap should pass records-editing while the normal keymap passes editing. Delete EnterEditMode; replace its registration with a parameterized binding for edit-or-drill.","notes":"Refactored both:\n- EditOrDrill (i/a) — now takes edit_mode: AppMode; keymap supplies it\n- EnterEditAtCursor effect + Cmd — now take target_mode: AppMode\n- CommitAndAdvance (Enter/Tab in editing modes) — now takes edit_mode: AppMode\n- Records-normal keymap overrides i/a to pass records-editing\n- Records-editing keymap overrides Enter/Tab to pass records-editing\n- Records-normal o sequence now passes records-editing\n- Shared parse_mode_name helper in registry.rs\n\nNo is_records() checks remain in any of these commands/effects.\nTests added for each parameterization (5 new tests).\n551 tests pass; clippy clean.","status":"closed","priority":2,"issue_type":"task","assignee":"fido","owner":"el-github@elangley.org","created_at":"2026-04-16T05:04:39Z","created_by":"fido","updated_at":"2026-04-16T05:35:11Z","closed_at":"2026-04-16T05:35:11Z","close_reason":"EditOrDrill, EnterEditAtCursor (effect + cmd), and CommitAndAdvance all parameterized with mode args from keymap; no runtime is_records() checks remain in those paths; 5 new tests cover the contract.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-kos","title":"_Dim and _Index should work as pivot axes outside Records/Drill view; must default to None axis on non-records views including empty models","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-15T11:15:40Z","created_by":"Edward Langley","updated_at":"2026-04-15T11:37:10Z","closed_at":"2026-04-15T11:37:10Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -31,14 +48,14 @@
{"id":"improvise-6mq","title":"Introduce Storage/VFS abstraction in persistence layer","description":"Refactor persistence/mod.rs to go through a Storage trait instead of std::fs directly. Native build wires PhysicalFS; later issues wire OPFS for wasm. Behavior-preserving refactor — .improv files still save and load identically on native. Foundation for the standalone web deployment and for alternative formats (sqlite, parquet) that can target any backend.","design":"Option A: adopt the vfs crate directly (vfs::VfsPath, vfs::PhysicalFS). Pros: batteries included, filesystem-path-based API, open-source implementations. Cons: sync-only; wasm backends may need async. Option B: define our own narrow Storage trait with read_bytes/write_bytes/list/stat methods, sync for native and async-wrapped for wasm. Pros: tailored to our needs. Cons: more code to maintain. Recommend starting with Option A (vfs crate) and falling back to Option B if OPFS's async-only nature forces the issue. Refactor persistence::{save_md, load_md, save_json, load_json} to take \u0026dyn Storage (or VfsPath) as the target parameter. Autosave path resolution (currently uses the dirs crate) goes behind the abstraction too — native resolves to a dirs path, wasm resolves to a fixed OPFS path.","acceptance_criteria":"(1) Persistence functions take a storage parameter rather than reading/writing files directly. (2) Native build wires PhysicalFS (or equivalent) and all existing persistence tests pass unchanged. (3) dirs crate usage is isolated behind a function that can be stubbed for wasm. (4) Round-trip tests still pass. (5) A memory-backed test fixture exists for unit testing persistence without touching disk.","notes":"Depends on crate-split epic step 3 (improvise-8zh — improvise-io extraction) so the refactor happens in the right crate. Valuable independently of wasm: enables testing persistence without tempfile, simplifies fuzzing.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:33:47Z","created_by":"spot","updated_at":"2026-04-14T07:33:47Z","dependencies":[{"issue_id":"improvise-6mq","depends_on_id":"improvise-8zh","type":"blocks","created_at":"2026-04-14T00:40:08Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":5,"comment_count":0}
{"id":"improvise-tm6","title":"Epic: Standalone static-web deployment via wasm + VFS storage","description":"Enable improvise to run entirely in the browser with no backend, suitable for GitHub Pages or any static host. Full App compiles to wasm; ratatui is stripped (automatically, by not depending on improvise-tui); persistence goes through a VFS/Storage abstraction backed by OPFS (or IndexedDB fallback) in the browser and by PhysicalFS on native. Shares the DOM renderer and Command protocol with the thin-client epic (improvise-6jk) — same code, different transport (in-process channel instead of websocket).","design":"Three deployment modes share code: (1) native TUI = today, (2) thin client over websocket = epic 6jk, (3) static web standalone = this epic. Mode 3 bundles the full App (improvise-core + improvise-formula + improvise-command + improvise-io's persistence half) into wasm. Same Command protocol and DOM renderer as mode 2; transport is an in-process command bus. Persistence abstracted behind a Storage trait (or the vfs crate) with PhysicalFS backing for native and OPFS backing for wasm. This makes sqlite/parquet formats a later orthogonal win — VFS handles bytes→medium, new format modules handle data→bytes. DOM renderer still reads ViewState + render cache even in standalone mode; an in-process projection layer updates the cache as App state changes. Ratatui falls out for free once crate-split epic completes: wasm build simply doesn't depend on improvise-tui.","notes":"Depends on crate-split epic's steps 3 (improvise-io) and 5 (improvise-command) being complete — both become critical-path under this epic. Shares protocol crate (improvise-cqi) and DOM renderer (improvise-cr3) with thin-client epic. Bundle size estimate: 1-3 MB compressed; biggest contributors are pest (formula parser) and chrono. Optimization (wee_alloc, wasm-opt, pest error trimming) is follow-on.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:33:45Z","created_by":"spot","updated_at":"2026-04-14T07:33:45Z","dependencies":[{"issue_id":"improvise-tm6","depends_on_id":"improvise-6mq","type":"blocks","created_at":"2026-04-14T00:40:28Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-9x6","type":"blocks","created_at":"2026-04-14T00:40:31Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-bck","type":"blocks","created_at":"2026-04-14T00:40:33Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-d31","type":"blocks","created_at":"2026-04-14T00:40:33Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-djm","type":"blocks","created_at":"2026-04-14T00:40:32Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-i34","type":"blocks","created_at":"2026-04-14T00:40:30Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-tm6","depends_on_id":"improvise-ywd","type":"blocks","created_at":"2026-04-14T00:40:29Z","created_by":"spot","metadata":"{}"}],"dependency_count":7,"dependent_count":0,"comment_count":0}
{"id":"improvise-rbv","title":"Include _Measure in _Dim columns for records mode","description":"In records mode, _Dim columns are generated from regular (non-virtual) categories plus a synthetic 'Value' column. _Measure is excluded because it starts with '_'. This means records show a generic 'Value' column instead of showing which measure (Revenue, Cost, etc.) each record belongs to. _Measure items should appear as _Dim columns in records mode so the measure name is visible per-record.","status":"closed","priority":2,"issue_type":"bug","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-14T07:30:40Z","created_by":"spot","updated_at":"2026-04-15T11:16:12Z","closed_at":"2026-04-15T11:16:12Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-dz8","title":"Add Axis::Filter for drill-fixed coordinates","description":"Drill mode sets fixed coordinates (e.g. Product=Widgets, Region=North) as Axis::Page, but these should be immutable constraints, not user-navigable page dimensions. Add Axis::Filter variant that: (1) excludes categories from tile bar in tile select mode, (2) excludes from page cycling via [ and ], (3) visually distinguished from Page in the UI. Drill uses Filter instead of Page for its fixed coordinates.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:30:04Z","created_by":"spot","updated_at":"2026-04-14T07:30:04Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-dz8","title":"Add Axis::Filter for drill-fixed coordinates","description":"Drill mode sets fixed coordinates (e.g. Product=Widgets, Region=North) as Axis::Page, but these should be immutable constraints, not user-navigable page dimensions. Add Axis::Filter variant that: (1) excludes categories from tile bar in tile select mode, (2) excludes from page cycling via [ and ], (3) visually distinguished from Page in the UI. Drill uses Filter instead of Page for its fixed coordinates.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:30:04Z","created_by":"spot","updated_at":"2026-04-14T07:30:04Z","dependencies":[{"issue_id":"improvise-dz8","depends_on_id":"improvise-rml","type":"blocks","created_at":"2026-04-16T16:02:39Z","created_by":"Ed L","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"id":"improvise-1ey","title":"Browser frontend MVP: end-to-end working demo","description":"Wire everything together into a working browser demo: start the ws-server, open the HTML shell in a browser, it loads the wasm client, connects to the websocket, receives the initial snapshot, renders the grid, accepts keystrokes, and round-trips commands to the server. This is the milestone where the architecture proves itself end-to-end. Scope: open an existing .improv file server-side, view it in the browser, type a number into a cell, see the server-side value update and the browser's projection arrive with the new value.","design":"HTML shell: minimal index.html with a \u003cdiv\u003e for the grid and a \u003cscript\u003e tag bootstrapping the wasm module. JS bootstrap: loads wasm, opens websocket connection, wires on_message from the socket into wasm.on_message, wires wasm.on_key_event output into the socket send. Reconnection: on socket close, show a status indicator; on reconnect, request a fresh snapshot. No auth, no TLS, localhost only. Test by pointing at bank-info.improv or any existing test file.","acceptance_criteria":"(1) ws-server serves a file from CLI arg, accepts a websocket connection. (2) Browser loads page, wasm initializes, grid renders. (3) Arrow keys move the cursor locally (view effect) with no round-trip. (4) Typing a number + Enter in a cell round-trips to the server and the new value appears. (5) Scrolling fetches cells for the new viewport. (6) Reload recovers cleanly from current server state. (7) Video or screenshot of the demo attached to the issue.","notes":"Final milestone of the browser epic. Depends on projection layer (improvise-cqq), ws-server (improvise-q08), wasm client (wasm-client id), DOM renderer (dom-renderer id). No new logic — just glue and testing.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:24:44Z","created_by":"spot","updated_at":"2026-04-14T07:24:44Z","dependencies":[{"issue_id":"improvise-1ey","depends_on_id":"improvise-cr3","type":"blocks","created_at":"2026-04-14T00:25:43Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-1ey","depends_on_id":"improvise-q08","type":"blocks","created_at":"2026-04-14T00:25:43Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":1,"comment_count":0}
{"id":"improvise-gsw","title":"improvise-wasm-client crate (keymap + reduce_view in wasm)","description":"Create the wasm crate that runs in the browser. Depends on improvise-protocol (Command, ViewState, reduce_view) and on the extracted improvise-command crate for the keymap. Exposes wasm-bindgen entry points: init, on_key_event, on_message (incoming command from server), current_view_state (for the DOM renderer to read). Resolves browser keyboard events locally via the keymap, dispatches the resulting command through reduce_view, serializes the command for sending upstream.","design":"crates/improvise-wasm-client/. wasm-bindgen + web-sys. Entry points: #[wasm_bindgen] fn init() -\u003e Handle (allocates ViewState + RenderCache); fn on_key_event(handle, browser_key_event) -\u003e Option\u003cString\u003e (resolves via keymap, dispatches via reduce_view, returns serialized Command to send upstream or None if no-op); fn on_message(handle, serialized_command) (deserializes incoming Command, applies via reduce_view, marks render dirty); fn render_state(handle) -\u003e JsValue (exposes ViewState + RenderCache for the DOM renderer). Key conversion: browser KeyboardEvent → improvise neutral Key enum via a From impl. The keymap is the same Keymap type used server-side, but with no ratatui dependency (from improvise-command after crate epic step 5).","acceptance_criteria":"(1) Crate compiles to wasm32-unknown-unknown. (2) Size budget: under 500 KB compressed. (3) on_key_event correctly resolves common keys (arrows, letters, Enter, Esc) via the keymap and dispatches through reduce_view. (4) on_message correctly deserializes and applies projection commands. (5) A small JS test harness can drive the crate end-to-end without a real websocket.","notes":"Transport-agnostic at the Rust layer: on_message accepts serialized commands regardless of source, and outgoing commands are returned to the JS bootstrap which routes them to a WebSocket (thin-client epic 6jk) or a worker MessageChannel (standalone epic tm6). The same wasm artifact is reused literally in both deployments — only the JS bootstrap differs. Depends on improvise-cqi (protocol crate) and crate-epic step 5 (improvise-3mm) so Keymap is importable without ratatui. Does not include the DOM renderer itself — that is improvise-cr3, which consumes the state this crate exposes.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:24:37Z","created_by":"spot","updated_at":"2026-04-14T07:39:34Z","dependencies":[{"issue_id":"improvise-gsw","depends_on_id":"improvise-3mm","type":"blocks","created_at":"2026-04-14T00:25:41Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-gsw","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:41Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0}
{"id":"improvise-q08","title":"improvise-ws-server binary (tokio + tungstenite session wrapper)","description":"New binary crate that wraps today's App behind a websocket. Accepts connections, creates a ClientSession per connection, receives Command messages from clients, dispatches them into the App, drains the outbound projection queue, sends projection commands back down. One App instance per session for the MVP (no shared authoritative state across sessions yet).","design":"crates/improvise-ws-server/. Tokio runtime, tokio-tungstenite for websocket. On connect: create App + ClientSession, send initial Snapshot (full ViewState + render cache filled from current viewport). Main loop: recv Command from client → feed into reduce_full (today's App::handle_key path, but command-keyed instead of key-keyed) → drain projection queue → send Commands down. On disconnect: drop session. Single session per connection for MVP. No auth, no collab, localhost only. Wire format: serde_json to start (easy to debug), bincode/postcard later if size matters.","acceptance_criteria":"(1) Binary crate builds and accepts websocket connections on a configurable port. (2) Initial snapshot is sent on connect. (3) Command round-trip works: client Command in → model updated → projection Command out. (4) Disconnection cleanly drops session state. (5) Integration test: spawn server, connect a test client, send a SetCell, verify a CellsUpdated comes back.","notes":"Depends on improvise-protocol (wire types) and improvise-mae/vb4/projection-emission for the server-side logic. The server itself is a thin transport wrapper — all the real logic lives in the App and projection layer.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:14Z","created_by":"spot","updated_at":"2026-04-14T07:23:14Z","dependencies":[{"issue_id":"improvise-q08","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:39Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-q08","depends_on_id":"improvise-cqq","type":"blocks","created_at":"2026-04-14T00:25:40Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0}
{"id":"improvise-cqq","title":"Server-side projection emission layer + per-client viewport tracking","description":"After each command is applied on the server, walk the resulting effect list, identify model-touching effects, compute which connected clients have the affected cells in their viewport, and enqueue projection commands (CellsUpdated, ColumnLabelsChanged, etc.) for dispatch to those clients. Server must track per-client viewport state (visible row/col range) and mirror each client's ViewState for this to work.","design":"Server holds HashMap\u003cSessionId, ClientSession\u003e where ClientSession { view: ViewState, visible_region: (row_range, col_range) }. When a client dispatches a command, update its session's view state via the shared reduce_view. When the command affects Model, walk the effect list: for each ModelTouching effect, compute which cells changed, intersect with each client session's visible_region, emit CellsUpdated commands into that session's outbound queue. ScrollTo commands update visible_region and trigger a fresh CellsUpdated for the newly-visible region. Viewport intersection logic: a cell is visible if its row/col position (computed from layout) lies within the client's visible range.","acceptance_criteria":"(1) Server struct ClientSession tracks view state and visible region per connected client. (2) After effect application, a projection step computes Vec\u003c(SessionId, Command)\u003e to dispatch. (3) ScrollTo updates visible region and triggers a fresh render-cache fill. (4) Unit tests: dispatching a SetCell produces CellsUpdated only for clients whose viewport includes the cell. (5) Unit tests: ScrollTo produces a CellsUpdated covering the new visible region.","notes":"Depends on improvise-mae (effect classification) and improvise-protocol (shared types). Does not yet wire up the actual websocket transport — that's a separate issue. This is the logical projection layer.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:12Z","created_by":"spot","updated_at":"2026-04-14T07:23:12Z","dependencies":[{"issue_id":"improvise-cqq","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:39Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-gxi","type":"blocks","created_at":"2026-04-14T00:53:45Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-mae","type":"blocks","created_at":"2026-04-14T00:25:38Z","created_by":"spot","metadata":"{}"}],"dependency_count":3,"dependent_count":3,"comment_count":0}
{"id":"improvise-cqi","title":"Extract improvise-protocol crate (Command, ViewState, reduce_view)","description":"Create a new crate that holds the types shared between the server and the wasm client: Command enum (the wire format), ViewState struct, render cache types (CellDisplay, visible region), and the reduce_view function. Depends on improvise-core so it can reference CellKey, CellValue, AppMode. Serde on everything that crosses the wire. This is the load-bearing crate for the browser epic.","design":"pub enum Command { /* user-initiated */ MoveCursor(Delta), EnterMode(AppMode), AppendMinibufferChar(char), ScrollTo { row, col }, SetCell { key: CellKey, value: CellValue }, AddCategory(String), ... /* server→client projections */ CellsUpdated { changes: Vec\u003c(CellKey, CellDisplay)\u003e }, ColumnLabelsChanged { labels: Vec\u003cString\u003e }, ColumnWidthsChanged { widths: Vec\u003cu16\u003e }, RowLabelsChanged { labels: Vec\u003cString\u003e }, ViewportInvalidated, ... }. pub fn reduce_view(vs: \u0026mut ViewState, cache: \u0026mut RenderCache, cmd: \u0026Command) — pattern-matches the command and applies view-slice effects + render-cache updates. Client imports this directly. Server imports Command + ViewState so it can serialize wire messages and maintain a mirror of each client's view state for projection computation.","acceptance_criteria":"(1) New crate crates/improvise-protocol/ exists. (2) Depends only on improvise-core, improvise-formula (if needed for anything), and serde. No ratatui, no crossterm. (3) Command, ViewState, RenderCache, CellDisplay all implement Serialize+Deserialize. (4) reduce_view has unit tests for view effect application and cache updates. (5) Crate builds for both host target and wasm32-unknown-unknown.","notes":"Depends on improvise-vb4 (ViewState must exist as a distinct type first) and on crate-epic step 2 (improvise-36h — improvise-core must be extracted so protocol can depend on it without pulling ratatui).","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:10Z","created_by":"spot","updated_at":"2026-04-14T07:23:10Z","dependencies":[{"issue_id":"improvise-cqi","depends_on_id":"improvise-36h","type":"blocks","created_at":"2026-04-14T00:25:38Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqi","depends_on_id":"improvise-vb4","type":"blocks","created_at":"2026-04-14T00:25:37Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":5,"comment_count":0}
{"id":"improvise-mae","title":"Tag existing effects as model / view / projection-emitting","description":"Add a classification to every Effect so the server can decide which effects trigger projection-command emission to connected clients. Pure view effects (cursor move, mode change) don't project. Model effects (SetCell, AddCategory, AddFormula) do project — they emit CellsUpdated or similar commands to clients whose viewport includes the change. Prerequisite for the server-side projection emission layer.","design":"Add a method to Effect like fn kind(\u0026self) -\u003e EffectKind where EffectKind is { ModelTouching, ViewOnly }. If Step 4 of crate epic (Effect→enum) has landed, this is a const match on variant. If not, each struct impls a kind() method. ModelTouching effects are further classified by what they change, so projection computation knows what kind of command to emit: { CellData, CategoryShape, ColumnLabels, RowLabels, FormulaResult, LayoutGeometry }.","acceptance_criteria":"(1) Every existing Effect has a kind classification. (2) A const/fn on Effect returns the kind without running apply. (3) Unit tests verify classification for each effect. (4) Audit notes list every effect and its kind.","notes":"Does not depend on Effect-as-enum refactor (crate epic step 4) but is much cleaner if that's landed. Depends on the AppState split so effects that cross slices are clearly cross-cutting.","status":"open","priority":2,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-14T07:22:02Z","created_by":"spot","updated_at":"2026-04-14T07:22:02Z","dependencies":[{"issue_id":"improvise-mae","depends_on_id":"improvise-vb4","type":"blocks","created_at":"2026-04-14T00:25:36Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
{"id":"improvise-vb4","title":"Split AppState into ModelState + ViewState (standalone refactor)","description":"Refactor the current App struct to make the model/view slice boundary explicit in types. Today App mixes Model with session state (mode, cursor, scroll, buffers, etc.); split them so that ModelState holds document state and ViewState holds per-session UI state. Server continues to own both, but as distinct fields. Valuable independently of the browser epic: cleaner persistence (only model slice serializes), cleaner undo (undo walks model effects only), better test boundaries, and it's the foundational slice separation every downstream issue depends on.","design":"pub struct ModelState { model: Model, file_path: Option\u003cPathBuf\u003e, dirty: bool }. pub struct ViewState { mode: AppMode, selected, row_offset, col_offset, search_query, search_mode, yanked, buffers, expanded_cats, help_page, drill_state_session_part, tile_cursor, panel_cursors }. App becomes { model_state: ModelState, view_state: ViewState, layout: GridLayout, ... }. Audit every App field and assign it. Gray zones: Model.views stays in Model (document state). drill_state staging map is session (View); commit flushes to Model. yanked is View. file_path/dirty are Model.","acceptance_criteria":"(1) App contains ModelState and ViewState as typed subfields. (2) No App field lives outside exactly one slice (or is explicitly called out as derived, like layout cache). (3) All existing tests pass. (4) Effect apply methods updated to take \u0026mut the correct slice where obvious, or \u0026mut App where cross-cutting. (5) Audit report in issue notes: every field classified.","notes":"Can start immediately — no dependencies. Does not require the crate-split epic. Purely a refactor of src/ui/app.rs + touchpoints in effects and commands.","status":"in_progress","priority":2,"issue_type":"feature","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-14T07:21:59Z","created_by":"spot","updated_at":"2026-04-15T10:24:54Z","dependency_count":0,"dependent_count":3,"comment_count":0}
{"id":"improvise-vb4","title":"Split AppState into ModelState + ViewState (standalone refactor)","description":"Refactor the current App struct to make the model/view slice boundary explicit in types. Today App mixes Model with session state (mode, cursor, scroll, buffers, etc.); split them so that ModelState holds document state and ViewState holds per-session UI state. Server continues to own both, but as distinct fields. Valuable independently of the browser epic: cleaner persistence (only model slice serializes), cleaner undo (undo walks model effects only), better test boundaries, and it's the foundational slice separation every downstream issue depends on.","design":"pub struct ModelState { model: Model, file_path: Option\u003cPathBuf\u003e, dirty: bool }. pub struct ViewState { mode: AppMode, selected, row_offset, col_offset, search_query, search_mode, yanked, buffers, expanded_cats, help_page, drill_state_session_part, tile_cursor, panel_cursors }. App becomes { model_state: ModelState, view_state: ViewState, layout: GridLayout, ... }. Audit every App field and assign it. Gray zones: Model.views stays in Model (document state). drill_state staging map is session (View); commit flushes to Model. yanked is View. file_path/dirty are Model.","acceptance_criteria":"(1) App contains ModelState and ViewState as typed subfields. (2) No App field lives outside exactly one slice (or is explicitly called out as derived, like layout cache). (3) All existing tests pass. (4) Effect apply methods updated to take \u0026mut the correct slice where obvious, or \u0026mut App where cross-cutting. (5) Audit report in issue notes: every field classified.","notes":"Additional implementation note merged from superseded improvise-dwe: a small number of effects today read App-level/layout-derived state at apply time (e.g. EnterEditAtCursor reads display_value from layout). For these, pre-compute the derived value in the command's Cmd::execute (where CmdContext has layout access) and bake it into the effect struct's own fields. Apply bodies then become pure model/view state writes. Example: EnterEditAtCursor { target_mode, initial_value: String } with initial_value computed in execute and consumed by apply.","status":"in_progress","priority":2,"issue_type":"feature","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-14T07:21:59Z","created_by":"spot","updated_at":"2026-04-16T23:02:18Z","dependency_count":0,"dependent_count":3,"comment_count":0}
{"id":"improvise-6jk","title":"Epic: Browser frontend via synchronized Redux-style stores","description":"Build a browser frontend for improvise by treating the client as a thin Redux-style peer of the existing server. Commands are the wire format; each side runs a reducer over its own state slice. Server is structurally unchanged (still runs today's full App with ratatui session, effect pipeline, formula eval). Client is a new thin peer: ViewState + render cache + reduce_view + keymap + DOM renderer. Commands flow both directions — upstream (user intent resolved client-side) and downstream (projection commands emitted by the server after model-touching effects). See child issues for sequenced steps.","design":"Client holds ViewState (mode, cursor, scroll, minibuffer, search, yanked, expanded cats, drill buffer, panel cursors) plus a render cache (visible cells, labels, col widths). Server holds full App as today. Wire type: Command enum, serde over websocket. Upstream: user-initiated commands resolved by client-side keymap. Downstream: projection commands (CellsUpdated, ColumnLabelsChanged, etc.) emitted by server after effect application. Client reduce_view pattern-matches commands and applies view-slice effects; client never runs formula eval or touches Model. Server emits projections by walking the effect list after each command and computing per-viewport deltas per connected client. Effect split (model vs view) is a server-internal tag driving projection emission — never crosses the wire. Key→command resolution is client-side (keymap in wasm bundle) for local responsiveness.","notes":"Prereqs from crate-split epic (improvise-xgl): step 2 (improvise-core, improvise-36h) for shared types; step 5 (improvise-command split, improvise-3mm) so wasm client can import keymap + Command + reduce_view without ratatui. Step 4 (Effect enum, improvise-45v) useful but not strictly required. Target bundle: 200-500 KB compressed — formula layer stays server-side.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:21:21Z","created_by":"spot","updated_at":"2026-04-14T07:21:21Z","dependencies":[{"issue_id":"improvise-6jk","depends_on_id":"improvise-1ey","type":"blocks","created_at":"2026-04-14T00:26:00Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:57Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-cqq","type":"blocks","created_at":"2026-04-14T00:25:58Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-cr3","type":"blocks","created_at":"2026-04-14T00:26:00Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-gsw","type":"blocks","created_at":"2026-04-14T00:25:59Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-mae","type":"blocks","created_at":"2026-04-14T00:25:57Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-q08","type":"blocks","created_at":"2026-04-14T00:25:58Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-6jk","depends_on_id":"improvise-vb4","type":"blocks","created_at":"2026-04-14T00:25:56Z","created_by":"spot","metadata":"{}"}],"dependency_count":8,"dependent_count":0,"comment_count":0}
{"id":"improvise-3mm","title":"Step 5: Extract improvise-command crate below improvise-tui","description":"Move src/command/ into an improvise-command crate that depends on improvise-core (+ formula) but NOT on ui/. The tui crate (with App, draw.rs, widgets) depends on improvise-command. This is the boundary most worth enforcing — it's where the current Cmd/Effect/App architecture is conceptually clean but mechanically unenforced. Should be mostly straightforward after step 4 (Effect-as-enum) because commands no longer transitively depend on App.","design":"AppMode and ModeKey must move down into improvise-command (both are pure enums used by commands for dispatch). One complication: AppMode::Editing { minibuf: MinibufferConfig } embeds ui-layer state in a mode enum. Decide: (a) pull MinibufferConfig down into command (if it's pure data, probably fine), or (b) split AppMode into a lean 'dispatch mode' enum in command and a richer per-mode state struct in tui. Recommendation: (a) if MinibufferConfig is pure data, (b) if it touches rendering. Also move command/keymap.rs (it imports AppMode and Effect — both will be in this crate).","acceptance_criteria":"(1) crates/improvise-command/ contains command/ plus AppMode, ModeKey. (2) No use crate::ui::* imports in improvise-command. (3) improvise-tui depends on improvise-command and contains ui/, draw.rs, main.rs. (4) cargo check in improvise-command succeeds without compiling ratatui or crossterm. (5) All tests pass. (6) cargo test -p improvise-command runs command/effect tests in isolation.","notes":"IMPORTANT UPDATE: App (or the equivalent server-side state struct) must live in improvise-command, NOT in improvise-tui. This is because the unified reduce_full(\u0026mut App, \u0026Command) function (improvise-gxi) needs to be imported by ws-server and worker-server — neither of which should pull in ratatui. If App stays in improvise-tui, reduce_full can only live in improvise-tui, which defeats the whole unified-reducer plan. Therefore: improvise-command hosts App + reduce_full + CmdRegistry + AppMode + Keymap + Cmd trait. improvise-tui becomes a thin rendering crate that depends on improvise-command and renders \u0026App (or after migration, renders from ViewModels per improvise-edp). Complication: App currently holds wizard: Option\u003cImportWizard\u003e. ImportWizard must either move to improvise-command (likely correct since it's session state) or be held behind an opaque handle. Resolve at implementation time.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T06:53:59Z","created_by":"spot","updated_at":"2026-04-14T07:54:16Z","dependencies":[{"issue_id":"improvise-3mm","depends_on_id":"improvise-45v","type":"blocks","created_at":"2026-04-13T23:54:40Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
{"id":"improvise-45v","title":"Step 4: Convert Effect trait to an enum (in-place, no crate change)","description":"Refactor Effect from a trait (Box\u003cdyn Effect\u003e with apply(\u0026self, \u0026mut App)) into a data enum, with a single apply(app: \u0026mut App, effect: \u0026Effect) function in the tui layer. This is the key semantic change that unblocks splitting command from ui: once effects are pure data, the command layer no longer transitively depends on App. Do this step in-place before any crate split so the diff stays isolated from packaging churn.","design":"Replace trait Effect with pub enum Effect { SetCursor { row, col }, SetMode(AppMode), AppendChar(char), ... } — one variant per current Effect struct (~50 variants). The existing Effect structs' fields become variant payloads. Move all apply() bodies into one apply_effect(app: \u0026mut App, e: \u0026Effect) function that matches exhaustively. Commands still return Vec\u003cEffect\u003e (no Box). Rationale: (1) design-principles.md §1 already claims 'effects can be logged, replayed, or composed' — an enum actually delivers that, a trait object does not. (2) matches the enum-heavy style used elsewhere (Binding, Axis, CategoryKind, BinOp). (3) exhaustive matching catches missed cases at compile time when adding new effects. Note: changes_mode() becomes a const match on the enum variant.","acceptance_criteria":"(1) trait Effect is gone. (2) Effect is a pub enum with one variant per previous struct. (3) apply_effect(app, effect) lives in ui/effect.rs (or similar) and is exhaustive. (4) Vec\u003cBox\u003cdyn Effect\u003e\u003e is replaced with Vec\u003cEffect\u003e in Cmd::execute signature and throughout command/. (5) All 568 existing tests pass. (6) No behavior change.","notes":"Biggest single semantic change in the epic. Do it in a branch, keep the commit history clean (one commit per submodule of effects migrated is fine). Watch for effects with nontrivial apply bodies — e.g., drill reconciliation and import wizard effects — those may need helper functions rather than inline match arms to keep apply_effect readable.","status":"closed","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T06:36:11Z","created_by":"spot","updated_at":"2026-04-16T06:26:50Z","closed_at":"2026-04-16T06:26:50Z","close_reason":"Superseded. Effect stays as a trait — the original blocker (command transitively depending on App via Effect::apply(\u0026mut App)) is resolved by splitting App into AppState (semantic state, in improvise-command) + App (rendering state wrapper, in improvise-tui). Effect::apply takes \u0026mut AppState; the few effects that currently read App-level state (layout, display_value) pre-compute those values in their Cmd::execute and pass them via the effect struct's own fields (self). No enum conversion, no dual trait, no noise. See improvise-3mm for the revised design.","dependencies":[{"issue_id":"improvise-45v","depends_on_id":"improvise-8zh","type":"blocks","created_at":"2026-04-13T23:54:40Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
@ -60,6 +77,22 @@
{"id":"improvise-pby","title":"2.2 Write the README","description":"Replace existing README.md with new one under 250 lines. Sections in order: Title, one-sentence pitch, inline demo GIF, why this exists (Lotus Improv reference), quick start, key bindings, installation (Nix/crates.io/prebuilt - no Homebrew), codebase overview, expectations disclaimer, license. No badges, TOC, or contributing guide.","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:07:32Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:02:52Z","closed_at":"2026-04-09T07:02:52Z","close_reason":"README written with all 10 required sections, committed. License updated to Apache-2.0 per user request.","dependency_count":0,"dependent_count":2,"comment_count":0}
{"id":"improvise-ihv","title":"2.1 Add Nix tooling for asciinema, VHS, and cargo-dist","description":"Modify flake.nix to add pkgs.asciinema and pkgs.vhs to dev shell nativeBuildInputs. Add cargo-dist if packaged in nixpkgs. Add runtime deps for VHS (ttyd, ffmpeg) if needed. Verify tools work via nix develop --command. Add nix run app or shell script for demo recording workflow.","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:07:30Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:30:19Z","closed_at":"2026-04-09T07:30:19Z","close_reason":"asciinema, vhs, cargo-dist added to flake.nix dev shell and verified. cargo-dist wrapper enables 'cargo dist' subcommand. scripts/record-demo.sh added.","dependency_count":0,"dependent_count":3,"comment_count":0}
{"id":"improvise-yk4","title":"Phase 2: README and demo artifacts","description":"The main launch work. README is 80% of the launch, demo artifacts are 20%. Covers Nix tooling for asciinema/VHS, writing the README, creating demo GIF and asciinema casts, and verifying all artifacts.","status":"closed","priority":2,"issue_type":"feature","assignee":"spot","owner":"el-github@elangley.org","created_at":"2026-04-09T04:05:49Z","created_by":"Edward Langley","updated_at":"2026-04-14T07:47:43Z","closed_at":"2026-04-14T07:47:43Z","close_reason":"All 5 subtasks closed; artifacts verified present and tracked in git (README.md, docs/demo.gif, docs/demo.tape, docs/casts/*.cast).","dependencies":[{"issue_id":"improvise-yk4","depends_on_id":"improvise-abz","type":"blocks","created_at":"2026-04-08T23:37:37Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-d4w","type":"blocks","created_at":"2026-04-08T23:37:36Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-ihv","type":"blocks","created_at":"2026-04-08T23:37:34Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-odx","type":"blocks","created_at":"2026-04-08T23:37:35Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-pby","type":"blocks","created_at":"2026-04-08T23:37:36Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"id":"improvise-ont","title":"Migrate existing callers of workbook.views.get() to Workbook::active_view","description":"Workbook already exposes active_view() and active_view_mut() (crates/improvise-core/src/workbook.rs:87-97) but many callers still do workbook.views.get(\u0026workbook.active_view).unwrap().X or wb.views.get('view_name').unwrap().axis_of(...).\\n\\n11 lookup sites identified:\\n- crates/improvise-core/src/workbook.rs:167-168, 176-177 (within workbook.rs itself — these may be the implementations and be fine)\\n- crates/improvise-core/src/model/types.rs:1868, 1871, 1873, 1886, 1890 (tests)\\n- crates/improvise-io/src/persistence/mod.rs:943, 946, 1165\\n\\nFix: audit each site; migrate to Workbook::active_view() / Workbook::view(name) / add Workbook::view_axis(view_name, cat_name) -\u003e Option\u003cAxis\u003e to absorb the common chain.\\n\\nMostly a cleanup task but reduces future refactor exposure. ~15 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:02:37Z","created_by":"Ed L","updated_at":"2026-04-16T19:02:37Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-ete","title":"Add Model/Category getter methods to absorb .unwrap().items/groups chains","description":"Several callers reach through Model and Category to get item names, groups, or counts — each chain includes an .unwrap() that panics if the category is missing:\\n\\n- persistence/mod.rs:1635-1639, 2006-2007: model.category(name).unwrap().items.values().map(|i| i.name.clone()).collect() — 4 sites\\n- persistence/mod.rs:831: \u0026m2.model.category('Month').unwrap().groups — 1 site\\n- src/ui/cat_tree.rs:32: cat.map(|c| c.items.len()) — item count\\n\\nAdd:\\n- Model::item_names(cat: \u0026str) -\u003e Option\u003cVec\u003cString\u003e\u003e\\n- Model::group_names(cat: \u0026str) -\u003e Option\u003cVec\u003cString\u003e\u003e\\n- Category::item_count() -\u003e usize\\n\\nCorrectness win: (a) prevents panics if the category lookup fails; (b) hides the items/groups IndexMap/HashMap from callers — internal-rep changes don't cascade; (c) the Option return type forces explicit handling.\\n\\n~15 LOC plus cleaner call sites.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:02:34Z","created_by":"Ed L","updated_at":"2026-04-16T19:02:34Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-rml","title":"Consolidate Axis display into a method on Axis","description":"Axis display logic is duplicated:\\n- src/ui/tile_bar.rs:30-36 — axis_display() match (4 arms)\\n- src/ui/category_panel.rs:14-19 — exact copy of the same match\\n- src/ui/view_panel.rs:40-44 — inline match\\n- src/command/cmd/tile.rs:112, 131, 143 — axis cycling and conversion matches\\n\\nAdd methods on Axis (crates/improvise-core/src/view/axis.rs):\\n- display_short() -\u003e (\u0026'static str, Color)\\n- display_long() -\u003e \u0026'static str\\n- next() -\u003e Self (cycle Row -\u003e Col -\u003e Page -\u003e None -\u003e Row)\\n- is_data_axis() -\u003e bool\\n\\nCorrectness win: eliminates the tile_bar/category_panel copy-paste — currently a change to axis colors or short labels must be made in both places.\\n\\n~25 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T19:02:21Z","created_by":"Ed L","updated_at":"2026-04-16T19:02:21Z","dependency_count":0,"dependent_count":1,"comment_count":0}
{"id":"improvise-1ud","title":"Document RecordsEditing variant in repo-map.md","description":"context/repo-map.md currently lists 15 AppMode variants but does not include RecordsEditing, which is referenced at src/draw.rs:142 and src/ui/app.rs:442. Add the variant to the AppMode list in the 'Key Types' section so coding agents see it.\\n\\nTrivial doc fix.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:54:16Z","created_by":"Ed L","updated_at":"2026-04-16T18:54:16Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-8fy","title":"Introduce simple_effect! macro for single-field setter effects","description":"src/ui/effect.rs has ~15 simple setter effects (SetSelected, SetRowOffset, SetColOffset, SetYanked, SetSearchQuery, SetSearchMode, SetStatus, etc.) that each implement the same 4-line apply: pub struct SetX(pub T); impl Effect { fn apply(\u0026self, app: \u0026mut App) { app.field = self.0.clone(); } }.\\n\\nAbstraction: simple_effect!(SetRowOffset, usize, row_offset) macro generates struct + Effect impl.\\n\\nCorrectness win: provides a single hook for cross-cutting concerns. If layout-affecting setters need to mark dirty or rebuild layout, one macro body covers all of them. Currently each setter independently decides whether to mark_dirty(), and silent omissions are plausible. ~50 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:54:14Z","created_by":"Ed L","updated_at":"2026-04-16T18:54:14Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-1wu","title":"Extract axis label helper in view/layout.rs","description":"crates/improvise-core/src/view/layout.rs:323-336 (row_label) and :338-351 (col_label) are character-identical except for which items list they read. Small syntactic duplication but trivial to consolidate.\\n\\nFix: fn axis_label(items: \u0026[AxisEntry], idx: usize) -\u003e String, call from both. ~14 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:54:13Z","created_by":"Ed L","updated_at":"2026-04-16T18:54:13Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-669","title":"Extract axis pruning helper in view/layout.rs","description":"crates/improvise-core/src/view/layout.rs:245-262 (rows) and :265-284 (cols) contain near-identical pruning loops that track data_idx separately from iteration order against a keep_row/keep_col bool vector. The data_idx tracking was flagged as brittle in the deep review.\\n\\nAbstraction: fn prune_axis_items(items: \u0026mut Vec\u003cAxisEntry\u003e, keep: \u0026[bool]) -\u003e Vec\u003cAxisEntry\u003e\\n\\nCorrectness win: centralizes the invariant (one place to assert data_idx == keep.len() at loop exit). A bug in the row prune can't hide while the col prune stays correct. ~20 LOC collapsed.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:54:12Z","created_by":"Ed L","updated_at":"2026-04-16T18:54:12Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-aaw","title":"Replace CreateAndSwitchView with Binding::Sequence","description":"command/cmd/panel.rs:546-560 bundles create-view + switch-view + change-mode. Both primitives already exist via effect_cmds.\\n\\nFix: bind Sequence [(create-view nix-shell-env), (switch-view nix-shell-env), (enter-mode normal)] in the keymap. If the auto-generated view name is load-bearing, either thread through a minibuffer (matches other add flows) or keep a single create-view primitive that both creates and switches by default.\\n\\n~12 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:54Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:54Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-git","title":"Replace SaveAndQuit with Binding::Sequence in keymap","description":"command/cmd/mode.rs:232-241 is an 8-line command chaining Save + ChangeMode(Quit). Both primitives already exist.\\n\\nFix: delete the struct; bind via Sequence [(w, []), (force-quit, [])] in the keymap.\\n\\nTrivial application of the 'if the keybinding is A then B, the binding should be a Sequence, not a command named AAndB' principle.\\n\\n~8 LOC.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:48Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:48Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-s13","title":"Delete mode-entry shim commands; use enter-mode with arg","description":"command/cmd/mode.rs:311-319 has EnterExportPrompt and similar shims (EnterSearchMode, EnterTileSelect, EnterFormulaEdit) that are each ~9-line wrappers around change_mode(AppMode::foo()). The generic EnterMode already exists with parse_mode_name() in the registry.\\n\\nFix: delete the shims; bind 'enter-mode' with the mode name as an arg in the keymap.\\n\\nCorrectness win: one code path for mode transitions. A new invariant on mode enter (e.g., 'always rebuild layout') needs to be added in one place instead of N shims.\\n\\n~40 LOC across all shims.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:53:44Z","created_by":"Ed L","updated_at":"2026-04-16T18:53:44Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-7yx","title":"Close test gaps flagged by deep review","description":"Deep review identified several test-coverage gaps worth closing:\\n\\n1. Keymap fallback chain: command/keymap.rs has 22 tests but no case for 'transient keymap exists but key does not match' (app.rs:396-406).\\n2. Ambiguous date detection: import/analyzer.rs:49-77 first-match heuristic not tested against ['01/02/2025', '02/03/2025'] style ambiguity.\\n3. CSV adversarial inputs: no tests for BOM, CRLF, empty file, single-column file.\\n4. Records missing a category field mid-import: wizard.rs:160-229 has the valid=false/break path but it is not covered by build_model_cells_match_source_data or similar.\\n5. Gzip corruption error path: save_and_load_roundtrip_gzip tests happy path only; add a test that load() on a truncated .gz file surfaces an error.\\n\\nLow-effort, high-value additions. Batch into a single test-coverage PR.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:19:07Z","created_by":"Ed L","updated_at":"2026-04-16T18:19:07Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-61f","title":"Add invariant test for command registry name mapping","description":"src/command/cmd/registry.rs is 586 lines with 0 tests. Each registration manually duplicates the command name as a string alongside the Cmd::name() implementation on the struct. Nothing verifies they match. Fix: add a test that iterates all registered entries and asserts registry_name == command.name() after construction. Low effort, locks down a class of typo bugs.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:55Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:55Z","dependencies":[{"issue_id":"improvise-61f","depends_on_id":"improvise-9cn","type":"blocks","created_at":"2026-04-16T11:54:34Z","created_by":"Ed L","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"id":"improvise-nr5","title":"Split persistence/mod.rs into focused sub-modules","description":"crates/improvise-io/src/persistence/mod.rs is 2402 lines mixing four cohesive units: pipe quoting (lines ~20-82), format_md (~155-282), parse_md (~285-580), export_csv (~581-630). Split into persistence/{quoting,format,parse,export}.rs behind the existing pub facade; no public-API change. Improves navigability and makes adding new format features easier to review.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:52Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:52Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-t8s","title":"View::axis_of panics on missing category","description":"crates/improvise-core/src/view/types.rs:118 uses expect('axis_of called for category not registered with this view'). The invariant is that Workbook::add_category registers every category with every view, but the panic surface is fragile for new callers. Fix: either audit all callers and document the invariant clearly, or change signature to fn axis_of(\u0026self, cat: \u0026str) -\u003e Option\u003cAxis\u003e and let callers handle None.","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:48Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:48Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-oaq","title":"Redundant file re-read in ImportJsonHeadless effect","description":"src/ui/effect.rs:846 chains 'serde_json::from_str(\u0026std::fs::read_to_string(\u0026self.path).unwrap_or_default()).unwrap_or(serde_json::Value::Array(records.clone()))'. This re-reads and re-parses the JSON file after records were already parsed earlier (lines 799-812). Dead code path from earlier refactor. Fix: remove the redundant read, use the already-parsed records array.","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:45Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:45Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-mzv","title":"Import wizard does not pre-check 12-category limit","description":"crates/improvise-io/src/import/wizard.rs:84-150 build_model() fails when \u003e12 categories are proposed, but the ImportWizard UI does not pre-check before allowing the user to confirm. A 20-column CSV imported as 20 categories fails late with 'Category limit exceeded' after the user commits. Fix: validate category count in the UI step and require the user to mark some as Measure/Skip before proceeding. Also add a test for the \u003e12 case.","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-04-16T18:18:44Z","created_by":"Ed L","updated_at":"2026-04-16T18:18:44Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-0hb","title":"Make Sequence bindings transactional at command level","description":"Currently Keymap::dispatch (src/command/keymap.rs:261-267) calls each Sequence step's execute(ctx) with the same pre-dispatch ctx. Effects accumulate in order and apply atomically as a batch via App::apply_effects, but later commands in a Sequence cannot observe earlier commands' effects through ctx — only effects-after-effects can see prior mutations.\n\nFor example, the records 'o' sequence [add-record-row, enter-edit-at-cursor records-editing] works only because the EnterEditAtCursor effect itself calls app.rebuild_layout() before reading display_value. Effects can self-heal mid-batch, but commands cannot.\n\nTrue command-level transactionality would require, between Sequence steps: apply effects-so-far, rebuild layout, build a fresh ctx, then dispatch the next command. This would change the semantics of every existing Sequence and is worth its own design pass.\n\nAcceptance: design doc + implementation; existing Sequences keep working; new tests prove a later step sees prior steps' model mutations.","notes":"Surfaced while parameterizing EnterEditAtCursor / CommitAndAdvance to remove is_records() runtime checks (improvise-4ju).","status":"open","priority":3,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-16T05:45:49Z","created_by":"fido","updated_at":"2026-04-16T05:45:49Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"id":"improvise-m91","title":"(Stretch) Command log + native undo/replay","description":"Once Commands are the universal unit of intent, log them per-session and support undo/redo via inverse commands or periodic snapshots. Enabled by the unified reducer; valuable for all three deployment modes but especially natural in the native TUI. Also enables debugging and reproducing bugs: a command log is a deterministic replay.","design":"Two approaches: (1) Inverse commands — each command definition includes how to undo it; undo-stack walks the log in reverse applying inverses. Clean but requires per-command inverse logic. (2) Snapshot-and-replay — periodic snapshots of ModelState + current log index; undo rolls back to nearest snapshot and replays up to target index. Simpler, works for any command, costs memory + replay time. Start with (2) for simplicity. The log is per-session (ClientSession.command_log) and survives only within a session. Persistence to disk as a .improv-journal companion file is a further enhancement. Replay use case: load a file, enable log recording, hit a bug, save the log, bug-report-as-replay.","acceptance_criteria":"(1) Every dispatched Command is logged in the ClientSession (or equivalent server-side state). (2) An Undo Command pops the log and restores state via snapshot+replay. (3) Redo re-applies. (4) Native TUI bindings for Ctrl-Z / Ctrl-Shift-Z. (5) Replay test: record a sequence, reset, replay, assert final state equals record-end state.","notes":"Stretch goal — valuable across all deployment modes but not part of the core architectural unification. Depends on issue improvise-gxi (reduce_full) and ideally on a persistence mechanism (improvise-6mq VFS abstraction) if logs should survive restart.","status":"open","priority":3,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:53:02Z","created_by":"spot","updated_at":"2026-04-14T07:53:02Z","dependencies":[{"issue_id":"improvise-m91","depends_on_id":"improvise-gxi","type":"blocks","created_at":"2026-04-14T00:53:44Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"id":"improvise-uq7","title":"(Stretch) Hybrid mode: native TUI with remote browser observer","description":"Once the native binary has the in-process client/server split (issue improvise-e0u), allow a second subscriber to attach via a local ws-server thread. Result: a developer running the native TUI can share their session with a teammate in a browser, or have their own browser tab observing while they work in the terminal. Both subscribers receive projection commands from the same in-process server; commands from either side flow through the same reduce_full.","design":"Native binary gains a --serve PORT flag (or similar) that, when set, spawns a ws-server thread alongside the native client. The ws-server is exactly improvise-q08's server logic, attached to the same in-process App as the native client. Two ClientSessions: one for the local ratatui client (zero-transport), one (or more) for remote websocket clients. Each has its own ViewState + render cache + subscription set. The projection layer fans out effect-driven updates to every session. The two subscribers see independent cursor/scroll/mode state (because view state is per-session) but the same model state (because model is shared). This is the deployment mode that fully justifies per-session view state tracking on the server.","acceptance_criteria":"(1) Native binary has an optional --serve flag that spawns a ws-server in the same process. (2) A remote browser can connect and see the native user's current session (shared model, independent view). (3) Edits from either side show up on the other after a round trip. (4) Graceful disconnect/reconnect of remote clients while native keeps running.","notes":"Stretch goal — not required for the core unification. Depends on issue e0u (binary split done) and improvise-q08 (ws-server exists). Orthogonal to the browser and standalone epics. The architectural investment in per-session view state and subscription-based projections is what makes this possible with minimal additional code.","status":"open","priority":3,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:52:57Z","created_by":"spot","updated_at":"2026-04-14T07:52:57Z","dependencies":[{"issue_id":"improvise-uq7","depends_on_id":"improvise-e0u","type":"blocks","created_at":"2026-04-14T00:53:42Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-uq7","depends_on_id":"improvise-q08","type":"blocks","created_at":"2026-04-14T00:53:43Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
@ -80,10 +113,10 @@
{"id":"improvise-3gy","title":"4.2 Enable GitHub Pages","description":"Enable Pages in GitHub repo settings: source=main branch, folder=/docs. Verify site is live at the GitHub Pages URL.","status":"open","priority":4,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-09T04:08:10Z","created_by":"Edward Langley","updated_at":"2026-04-09T06:38:03Z","dependencies":[{"issue_id":"improvise-3gy","depends_on_id":"improvise-e61","type":"blocks","created_at":"2026-04-08T21:09:32Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"id":"improvise-e61","title":"4.1 Create docs/index.html landing page","description":"Vanilla HTML, single file, under 200 lines. Dark background, monospace headings. Embed asciinema-player from jsdelivr CDN. Sections: title/tagline, pivot cast, drill cast, formulas cast, import cast, install commands + GitHub link. Player config: rows 30, cols 100, theme monokai, autoPlay false, loop true.","status":"open","priority":4,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-04-09T04:08:07Z","created_by":"Edward Langley","updated_at":"2026-04-09T06:38:02Z","dependencies":[{"issue_id":"improvise-e61","depends_on_id":"improvise-d4w","type":"blocks","created_at":"2026-04-08T21:09:31Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
{"id":"improvise-kh8","title":"Phase 4: Landing page (optional)","description":"Create docs/index.html with embedded asciinema casts and enable GitHub Pages. Optional but recommended for launch.","status":"open","priority":4,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-09T04:05:53Z","created_by":"Edward Langley","updated_at":"2026-04-09T06:38:02Z","dependencies":[{"issue_id":"improvise-kh8","depends_on_id":"improvise-3gy","type":"blocks","created_at":"2026-04-08T23:37:42Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-kh8","depends_on_id":"improvise-e61","type":"blocks","created_at":"2026-04-08T23:37:41Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
{"_type":"memory","key":"keymap-parent-inheritance-added-linked-list-of-keymaps","value":"Keymap parent inheritance added (linked list of keymaps). All minibuffer modes use Binding::Sequence for Enter and Esc to include clear-buffer command. ClearBufferCmd registered in registry."}
{"_type":"memory","key":"drillintocell-strips-measure-coordinate-from-drill-key-when","value":"DrillIntoCell strips _Measure coordinate from drill key when it matches a formula target, so matching_cells finds raw data records instead of returning empty."}
{"_type":"memory","key":"gen-grammar-example-generates-random-valid-improv-content","value":"gen-grammar example: generates random valid .improv content from pest grammar rules. Uses word pools for realistic output. pretty-print example: parses stdin and prints formatted output. Both in examples/ directory."}
{"_type":"memory","key":"persistence-index-and-dim-categories-are-never-written","value":"Persistence: _Index and _Dim categories are never written to .improv files. _Measure only persists non-formula items. Formulas targeting _Measure omit the [_Measure] suffix (it's the default). Parser defaults to _Measure when no [Category] suffix present."}
{"_type":"memory","key":"app-new-calls-recompute-formulas-before-building-initial","value":"App::new calls recompute_formulas before building initial layout so formula values appear on first frame. render() test helper also calls recompute_formulas."}
{"_type":"memory","key":"dynamic-measure-formula-targets-are-dynamically-included-via","value":"Dynamic _Measure: formula targets are dynamically included via Model::measure_item_names() and effective_item_names(). add_formula no longer adds items to _Measure category. find_item_category falls back to formula targets. CommitFormula defaults target_category to _Measure."}
{"_type":"memory","key":"keymap-parent-inheritance-added-linked-list-of-keymaps","value":"Keymap parent inheritance added (linked list of keymaps). All minibuffer modes use Binding::Sequence for Enter and Esc to include clear-buffer command. ClearBufferCmd registered in registry."}
{"_type":"memory","key":"persistence-index-and-dim-categories-are-never-written","value":"Persistence: _Index and _Dim categories are never written to .improv files. _Measure only persists non-formula items. Formulas targeting _Measure omit the [_Measure] suffix (it's the default). Parser defaults to _Measure when no [Category] suffix present."}
{"_type":"memory","key":"drillintocell-strips-measure-coordinate-from-drill-key-when","value":"DrillIntoCell strips _Measure coordinate from drill key when it matches a formula target, so matching_cells finds raw data records instead of returning empty."}
{"_type":"memory","key":"gen-grammar-example-generates-random-valid-improv-content","value":"gen-grammar example: generates random valid .improv content from pest grammar rules. Uses word pools for realistic output. pretty-print example: parses stdin and prints formatted output. Both in examples/ directory."}
{"_type":"memory","key":"lib-rs-created-to-enable-examples-to-import","value":"lib.rs created to enable examples to import from improvise crate. main.rs uses 'use improvise::*' instead of mod declarations."}

667
context/repo-map.md
View File

@ -1,292 +1,97 @@
# Repository Map (LLM Reference)
Terminal pivot-table modeling app. Rust, Ratatui TUI, command/effect architecture.
Cargo workspace, Apache-2.0, edition 2024. Root package `improvise` v0.1.0-rc2.
Library + binary crate: `src/lib.rs` re-exports public modules (many from sub-crates), `src/main.rs` is the CLI entry.
Sub-crates live under `crates/`:
- `crates/improvise-core/` — pure-data core: `Model`, `View`, `Workbook`, and number formatting. Depends on `improvise-formula`. Re-exported from the main crate so `crate::model`, `crate::view`, `crate::workbook`, `crate::format` still resolve everywhere. Has no awareness of UI, I/O, or commands — builds standalone via `cargo build -p improvise-core`.
- `crates/improvise-formula/` — formula parser, AST (`Expr`, `BinOp`, `AggFunc`, `Formula`, `Filter`), `parse_formula`. Re-exported as `crate::formula` from the main crate via `pub use improvise_formula as formula;`.
- `crates/improvise-io/` — `.improv` persistence (parse/format, save/load, CSV export) and import pipeline (CSV/JSON wizard, field analyzer). Depends on `improvise-core` and `improvise-formula`; has no UI or command code. Re-exported from the main crate so `crate::persistence` and `crate::import` still resolve everywhere. Builds standalone via `cargo build -p improvise-io`.
Terminal pivot-table app. Rust 2024, Ratatui TUI, command/effect architecture.
Apache-2.0. Root binary+lib `improvise`; sub-crates under `crates/`:
- `improvise-core` — `Model`, `View`, `Workbook`, number formatting. No UI/IO.
- `improvise-formula` — formula parser, AST, `parse_formula`.
- `improvise-io` — `.improv` save/load, CSV/JSON import. No UI/commands.
The main crate re-exports each as `crate::{model, view, workbook, format, formula, persistence, import}` so consumer paths stay stable when crates shuffle.
Architectural intent lives in `context/design-principles.md` — read that for the "why". This doc is the "where".
---
## How to Find Things
| I need to... | Look in |
|---------------------------------------|----------------------------------------------|
| Add a new keybinding | `command/keymap.rs` → `default_keymaps()` |
| Add a new user-facing command | `command/cmd/` → implement `Cmd` in the relevant submodule, register in `registry.rs` |
| Add a new state mutation | `ui/effect.rs` → implement `Effect` |
| Change formula evaluation | `model/types.rs` → `eval_formula()`, `eval_expr()` |
| Change how cells are stored/queried | `model/cell.rs` → `DataStore` |
| Change category/item behavior | `model/category.rs` → `Category` |
| Change view axis logic | `view/types.rs` → `View` |
| Change grid layout computation | `view/layout.rs` → `GridLayout` |
| Change .improv file format | `persistence/improv.pest` (grammar), `persistence/mod.rs` → `format_md()`, `parse_md()` |
| Change number display formatting | `format.rs` → `format_f64()` |
| Change CLI arguments | `main.rs` → clap structs |
| Change import wizard logic | `import/wizard.rs` → `ImportPipeline` |
| Change grid rendering | `ui/grid.rs` → `GridWidget` |
| Change TUI frame layout | `draw.rs` → `draw()` |
| Change app state / mode transitions | `ui/app.rs` → `App`, `AppMode` |
| Write a test for model logic | `model/types.rs` → `mod tests` / `mod formula_tests` |
| Write a test for a command | `command/cmd/<module>.rs` → colocated `mod tests` |
| I need to... | Look in |
|-------------------------------------|-------------------------------------------------|
| Add a keybinding | `command/keymap.rs` → `default_keymaps()` |
| Add a user command | `command/cmd/<submodule>.rs`, register in `registry.rs` |
| Add a state mutation | `ui/effect.rs` → implement `Effect` |
| Change formula eval | `model/types.rs` → `eval_formula` / `eval_expr` |
| Change cell storage / lookup | `model/cell.rs` → `DataStore` |
| Change category/item behavior | `model/category.rs` → `Category` |
| Change view axis logic | `view/types.rs` → `View` |
| Change grid layout | `view/layout.rs` → `GridLayout` |
| Change `.improv` format | `persistence/improv.pest` + `persistence/mod.rs` |
| Change number display | `format.rs` → `format_f64` |
| Change CLI args | `main.rs` (clap) |
| Change import logic | `import/wizard.rs` → `ImportPipeline` |
| Change frame layout | `draw.rs` → `draw()` |
| Change app state / modes | `ui/app.rs` → `App`, `AppMode` |
| Write a test for model logic | `model/types.rs` → `mod tests` / `formula_tests` |
| Write a test for a command | `command/cmd/<module>.rs` → colocated `mod tests`; helpers in `cmd/mod.rs::test_helpers` |
---
## Core Types and Traits
### Command/Effect Pipeline (the central architecture pattern)
## Central Pattern: Cmd → Effect
```
User keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
(immutable) (pure, read-only) (state mutations)
keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
(pure, read-only ctx) (list of mutations) (only mutation site)
```
```rust
// src/command/cmd/core.rs
pub trait Cmd: Debug + Send + Sync {
fn name(&self) -> &'static str;
fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
}
pub struct CmdContext<'a> {
pub model: &'a Model, // immutable
pub layout: &'a GridLayout, // immutable
pub registry: &'a CmdRegistry,
pub mode: &'a AppMode,
pub selected: (usize, usize), // (row, col) cursor
pub row_offset: usize,
pub col_offset: usize,
pub search_query: &'a str,
pub search_mode: bool,
pub yanked: &'a Option<CellValue>,
pub key_code: KeyCode, // the key that triggered this command
pub buffers: &'a HashMap<String, String>,
pub expanded_cats: &'a HashSet<String>,
// panel cursors, tile cursor, visible dimensions...
}
// src/ui/effect.rs
pub trait Effect: Debug {
fn apply(&self, app: &mut App);
fn changes_mode(&self) -> bool { false } // override if effect changes AppMode
fn changes_mode(&self) -> bool { false } // override when Effect swaps AppMode
}
```
**To add a command**: implement `Cmd` in the appropriate `command/cmd/` submodule, then register in `command/cmd/registry.rs`. Use the `effect_cmd!` macro (in `effect_cmds.rs`) for simple effect-wrapping commands. Bind it in `default_keymaps()`.
`CmdContext` (see `command/cmd/core.rs`) holds immutable refs to `Model`, `GridLayout`, `AppMode`, cursor/offsets, buffers, yanked cell, key code, expanded cats, and panel/tile cursors.
**To add an effect**: implement `Effect` in `effect.rs`, add a constructor function.
**Add a command**: implement `Cmd` in a `command/cmd/` submodule, register in `registry.rs`, bind in `default_keymaps()`. Simple wrappers go through the `effect_cmd!` macro in `effect_cmds.rs`.
### Data Model
```rust
// src/model/types.rs
pub struct Model {
pub name: String,
pub categories: IndexMap<String, Category>, // ordered
pub data: DataStore,
pub formulas: Vec<Formula>,
pub views: IndexMap<String, View>,
pub active_view: String,
pub measure_agg: HashMap<String, AggFunc>, // per-measure aggregation override
}
// Key methods:
// add_category(&mut self, name) -> Result<CategoryId> [max 12 regular]
// category(&self, name) -> Option<&Category>
// category_mut(&mut self, name) -> Option<&mut Category>
// set_cell(&mut self, key: CellKey, value: CellValue)
// evaluate(&self, key: &CellKey) -> Option<CellValue> [formulas + raw data]
// evaluate_aggregated(&self, key, none_cats) -> Option<CellValue> [sums over hidden dims]
// recompute_formulas(&mut self, none_cats) [fixed-point formula cache]
// add_formula(&mut self, formula: Formula) [replaces same target+category]
// remove_formula(&mut self, target, category)
// measure_item_names(&self) -> Vec<String> [_Measure items + formula targets]
// effective_item_names(&self, cat) -> Vec<String> [_Measure dynamic, others ordered_item_names]
// category_names(&self) -> Vec<&str> [includes virtual]
// regular_category_names(&self) -> Vec<&str> [excludes _Index, _Dim, _Measure]
const MAX_CATEGORIES: usize = 12; // virtual categories don't count
```
```rust
// src/model/cell.rs
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct CellKey(pub Vec<(String, String)>); // always sorted by category name
// CellKey::new(coords) — sorts on construction, enforcing canonical form
// CellKey::with(cat, item) -> Self — returns new key with coord added/replaced
// CellKey::without(cat) -> Self — returns new key with coord removed
// CellKey::get(cat) -> Option<&str>
#[derive(Clone, PartialEq)]
pub enum CellValue {
Number(f64),
Text(String),
Error(String), // formula evaluation error (circular ref, div/0, etc.)
}
// CellValue::as_f64() -> Option<f64>
// CellValue::is_error() -> bool
pub struct DataStore {
cells: HashMap<InternedKey, CellValue>,
pub symbols: SymbolTable,
index: HashMap<(Symbol, Symbol), HashSet<InternedKey>>, // secondary index
}
// DataStore::set(&mut self, key: &CellKey, value: CellValue)
// DataStore::get(&self, key: &CellKey) -> Option<&CellValue>
// DataStore::matching_values(&self, partial: &[(String,String)]) -> Vec<CellValue>
// DataStore::matching_cells(&self, partial) -> Vec<(CellKey, CellValue)>
```
```rust
// src/model/category.rs
pub struct Category {
pub id: CategoryId, // usize
pub name: String,
pub kind: CategoryKind,
pub items: IndexMap<String, Item>, // ordered
pub groups: IndexMap<String, Group>,
next_item_id: ItemId, // private, auto-increment
}
// Category::add_item(&mut self, name) -> ItemId [deduplicates by name]
// Category::ordered_item_names(&self) -> Vec<&str> [respects group order]
pub enum CategoryKind { Regular, VirtualIndex, VirtualDim, VirtualMeasure, Label }
```
### Formula System
```rust
// crates/improvise-formula/src/ast.rs
pub enum Expr {
Number(f64),
Ref(String), // reference to an item name
BinOp(BinOp, Box<Expr>, Box<Expr>),
UnaryMinus(Box<Expr>),
Agg(AggFunc, Box<Expr>, Option<Filter>),
If(Box<Expr>, Box<Expr>, Box<Expr>),
}
pub enum BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }
pub enum AggFunc { Sum, Avg, Min, Max, Count }
pub struct Formula {
pub raw: String, // "Profit = Revenue - Cost"
pub target: String, // "Profit"
pub target_category: String, // "Measure"
pub expr: Expr,
pub filter: Option<Filter>, // WHERE clause
}
// crates/improvise-formula/src/parser.rs
pub fn parse_formula(raw: &str, target_category: &str) -> Result<Formula>
```
Formula evaluation is in `model/types.rs` → `eval_formula()` / `eval_expr()`. Operates at full f64 precision. Display rounding in `format.rs` is view-only.
### View and Layout
```rust
// src/view/axis.rs
pub enum Axis { Row, Column, Page, None }
// src/view/types.rs
pub struct View {
pub name: String,
pub category_axes: IndexMap<String, Axis>,
pub page_selections: HashMap<String, String>,
pub hidden_items: HashMap<String, HashSet<String>>,
pub collapsed_groups: HashMap<String, HashSet<String>>,
pub number_format: String, // e.g. ",.0" or ",.2f"
pub prune_empty: bool,
// scroll/selection state...
}
// View::set_axis(&mut self, cat, axis)
// View::axis_of(&self, cat) -> Axis
// View::cycle_axis(&mut self, cat) [Row→Column→Page→None→Row]
// View::transpose(&mut self) [swap Row↔Column]
// View::categories_on(&self, axis) -> Vec<&str>
// View::on_category_added(&mut self, cat) [auto-assigns axis]
// src/view/layout.rs
pub struct GridLayout { /* computed from Model + View */ }
// GridLayout::new(model, view) -> Self
// GridLayout::cell_key(row, col) -> Option<CellKey>
// GridLayout::cell_value(row, col) -> Option<CellValue>
// GridLayout::row_label(row) -> &str
// GridLayout::col_label(col) -> &str
// GridLayout::drill_records(row, col) -> Vec<(CellKey, CellValue)>
// Records mode: auto-detected when _Index on Row + _Dim on Column
```
### App State
```rust
// src/ui/app.rs
pub enum AppMode {
Normal,
Editing { minibuf: MinibufferConfig },
FormulaEdit { minibuf: MinibufferConfig },
FormulaPanel,
CategoryPanel,
ViewPanel,
TileSelect,
CategoryAdd { minibuf: MinibufferConfig },
ItemAdd { minibuf: MinibufferConfig },
ExportPrompt { minibuf: MinibufferConfig },
CommandMode { minibuf: MinibufferConfig },
ImportWizard,
Help,
Quit,
}
// Note: SearchMode is Normal + search_mode:bool flag, not a separate variant.
pub struct App {
pub model: Model,
pub mode: AppMode,
pub file_path: Option<PathBuf>,
pub dirty: bool,
pub help_page: usize,
pub transient_keymap: Option<Arc<Keymap>>, // for prefix keys
// layout cache, drill_state, wizard, buffers, panel cursors, etc.
}
// App::handle_key(&mut self, KeyEvent) -> Result<()> [main input dispatch]
// App::rebuild_layout(&mut self)
// App::is_empty_model(&self) -> bool [true when only virtual categories exist]
```
### Keymap System
```rust
// src/command/keymap.rs
pub enum KeyPattern { Key(KeyCode, KeyModifiers), AnyChar, Any }
pub enum Binding {
Cmd { name: &'static str, args: Vec<String> },
Prefix(Arc<Keymap>), // Emacs-style sub-keymap
Sequence(Vec<(&'static str, Vec<String>)>), // multi-command chain
}
pub enum ModeKey {
Normal, Help, FormulaPanel, CategoryPanel, ViewPanel, TileSelect,
Editing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode,
SearchMode, ImportWizard,
}
// Keymap::with_parent(parent: Arc<Keymap>) -> Self [Emacs-style inheritance]
// Keymap::lookup(&self, key, mods) -> Option<&Binding>
// Fallback chain: exact(key,mods) → Char with NONE mods → AnyChar → Any → parent
// Minibuffer modes: Enter and Esc use Binding::Sequence to include clear-buffer
// KeymapSet::default_keymaps() -> Self [builds all 14 mode keymaps]
// KeymapSet::dispatch(&self, ctx, key, mods) -> Option<Vec<Box<dyn Effect>>>
```
**Add an effect**: implement `Effect` in `ui/effect.rs`, add a constructor fn if it helps composition.
---
## File Format (.improv)
## Key Types (skim; read the source for fields/methods)
Plain-text markdown-like, defined by a PEG grammar (`persistence/improv.pest`).
Parsed by pest; the grammar is the single source of truth for both the parser
and the grammar-walking test generator.
**Model** (`model/types.rs`): `categories: IndexMap<String, Category>`, `data: DataStore`, `formulas: Vec<Formula>`, `views: IndexMap<String, View>`, `active_view`, `measure_agg`. `MAX_CATEGORIES = 12` (regular only). Virtual categories `_Index`, `_Dim`, `_Measure` always exist. Use `regular_category_names()` for user-facing pickers.
**Not JSON** (JSON is legacy, auto-detected by `{` prefix).
**CellKey** (`model/cell.rs`): `Vec<(cat, item)>` — **always sorted by category**. Build with `CellKey::new(coords)` / `with(cat, item)` / `without(cat)`. Never construct the inner `Vec` directly.
**CellValue**: `Number(f64) | Text(String) | Error(String)`. Errors surface from formula eval (circular, div/0, etc.).
**DataStore** (`model/cell.rs`): interned symbols + secondary index `(Symbol, Symbol) → {InternedKey}`. Hot-path lookups go through `matching_values` / `matching_cells`.
**Formula AST** (`improvise-formula/src/ast.rs`):
- `Expr { Number, Ref, BinOp, UnaryMinus, Agg, If }`
- `BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }`
- `AggFunc { Sum, Avg, Min, Max, Count }`
- `Formula { raw, target, target_category, expr, filter }`
**View** (`view/types.rs`): `category_axes: IndexMap<cat, Axis>`, page selections, hidden items, collapsed groups, `number_format`, `prune_empty`. `Axis { Row, Column, Page, None }`. `cycle_axis` rotates Row → Column → Page → None.
**GridLayout** (`view/layout.rs`): pure function of `Model + View`. `cell_key(r,c)`, `cell_value(r,c)`, `drill_records(r,c)`. **Records mode** auto-detects when `_Index` is on Row and `_Dim` is on Column.
**AppMode** (`ui/app.rs`): 15 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, CommandMode, ImportWizard, Help, Quit). `SearchMode` is Normal + `search_mode: bool`, not its own variant.
**Keymap** (`command/keymap.rs`): `Binding { Cmd | Prefix(Arc<Keymap>) | Sequence(Vec<…>) }`. Lookup fallback: `exact(key,mods) → Char(NONE) → AnyChar → Any → parent`. 14 mode keymaps built by `KeymapSet::default_keymaps()`; mode resolved via `ModeKey::from_app_mode()`.
---
## `.improv` File Format
Plain-text, markdown-like, defined by `persistence/improv.pest`. Parsed by pest — the grammar is the single source of truth (the grammar-walking test generator reads it via `pest_meta`).
```
v2025-04-09
@ -296,49 +101,126 @@ Initial View: Default
## View: Default
Region: row
Measure: column
|Time Period|: page, Q1 ← pipe-quoted name, page with selection
|Time Period|: page, Q1
hidden: Region/Internal
collapsed: |Time Period|/|2024|
format: ,.2f
## Formulas
- Profit = Revenue - Cost ← defaults to [_Measure]
- Tax = Revenue * 0.1 [CustomCat] ← explicit [TargetCategory] for non-_Measure
- Profit = Revenue - Cost # defaults to [_Measure]
- Tax = Revenue * 0.1 [CustomCat]
## Category: Region
- North, South, East, West ← bare items, comma-separated
- Coastal_East[Coastal] ← grouped item (one per line)
- Coastal_West[Coastal]
> Coastal ← group definition
## Category: Measure
- Revenue, Cost, Profit
- North, South, East, West # bare items, comma-separated
- Coastal_East[Coastal] # grouped item, one per line
> Coastal # group definition
## Data
Region=East, Measure=Revenue = 1200
Region=East, Measure=Cost = 800
Region=West, Measure=Revenue = |pending| ← pipe-quoted text value
Region=West, Measure=Revenue = |pending| # pipe-quoted text
```
### Name quoting
- **Name quoting**: bare `[A-Za-z_][A-Za-z0-9_-]*`, else CL-style `|…|` with escapes `\|`, `\\`, `\n`. Same convention in the formula tokenizer.
- **Write order**: Views → Formulas → Categories → Data. Parser tolerates any order.
- **Gzip**: `.improv.gz` (same text, gzipped).
- **Legacy JSON**: auto-detected by leading `{`; never written.
- **Virtual categories**: `_Index`/`_Dim` never persist; `_Measure` persists only non-formula items (formula targets are rebuilt from the `## Formulas` section).
Bare names match `[A-Za-z_][A-Za-z0-9_-]*`. Everything else uses CL-style
pipe quoting: `|Income, Gross|`, `|2025|`, `|Name with spaces|`.
Escapes inside pipes: `\|` (literal pipe), `\\` (backslash), `\n` (newline).
---
### Section order
## Gotchas (read before editing)
`format_md` writes Views → Formulas → Categories → Data (smallest to largest).
The parser accepts sections in any order.
1. **Commands never mutate.** `&CmdContext` is read-only; return `Vec<Box<dyn Effect>>`. If you want to touch `&mut App`, add or extend an Effect.
2. **`CellKey` is always sorted.** Use the constructors; equality and hashing rely on canonical form.
3. **No `Model::add_item`.** Go through the category: `m.category_mut("Region").unwrap().add_item("East")`.
4. **Virtual categories.** `_Measure` items = explicit items ∪ formula targets. Use `measure_item_names()` / `effective_item_names("_Measure")`. `add_formula` does *not* add items to `_Measure`. Formula-target lookup also falls back into `_Measure` via `find_item_category`.
5. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full `f64`. Never feed a rounded value back into eval.
6. **Formula eval is fixed-point.** `recompute_formulas(none_cats)` iterates until stable, bounded by `MAX_EVAL_DEPTH`; circular refs converge to `CellValue::Error("circular")`. `App::new` runs it before the first frame so formula cells render on startup.
7. **Drill into formula cells** strips the `_Measure` coordinate when it names a formula target, so `matching_cells` returns the raw records that feed the formula instead of returning empty.
8. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the mode; Normal + `search_mode=true` maps to `SearchMode`. Minibuffer modes bind Enter/Esc as `Binding::Sequence` so `clear-buffer` fires alongside commit/cancel.
9. **`effect_cmd!` macro** wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written `Cmd` once there's real decision logic.
10. **Float equality inside formula eval is currently mixed** — `=`/`!=` use a `1e-10` epsilon but the div-by-zero guard checks `rv == 0.0` exactly. Don't assume one or the other; tracked for cleanup.
11. **`IndexMap`** preserves insertion order for categories, views, and items. Persistence relies on this.
12. **Commit paths for cell edits** split across `commit_cell_value` → `commit_regular_cell_value` / `commit_plain_records_edit` in `command/cmd/commit.rs`. Keep the synthetic-records branch in sync with the plain branch when changing behavior.
### Key design choices
---
- Version line is exact match (`v2025-04-09`) — grammar enforces valid versions only.
- `Initial View:` is a top-level header, not embedded in view sections.
- Text cell values are always pipe-quoted to distinguish from numbers.
- Bare items are comma-separated on one line; grouped items get one line each.
## File Inventory
Gzip variant: `.improv.gz` (same content, gzipped). Persistence code: `persistence/mod.rs`.
Line counts are static; test counts are informational — run `cargo test --workspace` for live numbers. Files under 100 lines and render-only widgets omitted.
### `improvise-core` (`crates/improvise-core/src/`)
```
model/types.rs 2062 / 70t Model, formula eval (fixed-point), CRUD
model/cell.rs 650 / 28t CellKey (sorted), CellValue, DataStore (interned + index)
model/category.rs 222 / 6t Category, Item, Group, CategoryKind
model/symbol.rs 79 / 3t Symbol interning
view/layout.rs 1140 / 24t GridLayout, drill, records mode
view/types.rs 531 / 28t View config (axes, pages, hidden, collapsed, format)
view/axis.rs 21 Axis enum
workbook.rs 259 / 11t Workbook: Model + cross-view ops
format.rs 229 / 29t format_f64, parse_number_format (display only)
```
### `improvise-formula` (`crates/improvise-formula/src/`)
```
parser.rs 776 / 65t pest grammar + tokenizer → Formula AST
ast.rs 77 Expr, BinOp, AggFunc, Formula, Filter
```
### `improvise-io` (`crates/improvise-io/src/`)
```
persistence/improv.pest 124 PEG grammar — single source of truth
persistence/mod.rs 2410 / 83t save/load/gzip/legacy-JSON, CSV export
import/wizard.rs 1117 / 38t ImportPipeline + ImportWizard
import/analyzer.rs 292 / 9t Field kind detection (Category/Measure/Time/Skip)
import/csv_parser.rs 300 / 8t CSV parsing, multi-file merge
```
### Command layer (`src/command/`)
```
cmd/core.rs 297 / 2t Cmd trait, CmdContext, CmdRegistry, parse helpers
cmd/registry.rs 586 / 0t default_registry() — all registrations (no tests yet)
cmd/navigation.rs 475 / 10t Move, EnterAdvance, Page*
cmd/cell.rs 198 / 6t ClearCell, YankCell, PasteCell, TransposeAxes, SaveCmd
cmd/commit.rs 330 / 7t CommitFormula, CommitCategoryAdd/ItemAdd, CommitExport
cmd/effect_cmds.rs 437 / 5t effect_cmd! macro, 25+ simple wrappers
cmd/grid.rs 409 / 7t ToggleGroup, ViewNavigate, DrillIntoCell, TogglePruneEmpty
cmd/mode.rs 308 / 8t EnterMode, Quit, EditOrDrill, EnterTileSelect
cmd/panel.rs 587 / 13t Panel toggle/cycle/cursor, formula/category/view panels
cmd/search.rs 202 / 4t SearchNavigate, SearchOrCategoryAdd, ExitSearchMode
cmd/text_buffer.rs 256 / 7t AppendChar, PopChar, CommandModeBackspace, ExecuteCommand
cmd/tile.rs 160 / 5t MoveTileCursor, TileAxisOp
keymap.rs 1066 / 22t KeyPattern, Binding, Keymap, ModeKey, 14 mode keymaps
parse.rs 236 / 19t Script/command-line parser (prefix syntax)
```
### UI, draw, main (`src/ui/`, `src/draw.rs`, `src/main.rs`)
```
ui/effect.rs 942 / 41t Effect trait, 50+ effect types
ui/app.rs 914 / 30t App state, AppMode (15), handle_key, autosave
ui/grid.rs 1036 / 13t GridWidget (ratatui), column widths
ui/help.rs 617 5-page help overlay (render only)
ui/import_wizard_ui.rs 347 Import wizard rendering
ui/cat_tree.rs 165 / 6t Category tree flattener for panel
draw.rs 400 TUI event loop, frame composition
main.rs 391 CLI entry (clap): open, import, cmd, script
# other ui/*.rs are small panel renderers — skip unless changing layout/style
```
### Examples
```
examples/gen-grammar.rs Grammar-walking random .improv generator (pest_meta)
examples/pretty-print.rs Parse stdin, print formatted .improv
```
### Context docs
```
context/design-principles.md Architectural principles & testing doctrine
context/plan.md Show HN launch plan
context/repo-map.md This file
docs/design-notes.md Product vision & non-goals
```
---
@ -355,219 +237,18 @@ Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axi
---
## Key Dependencies
## Testing — the short version
| Crate | Purpose |
|-------|---------|
| ratatui 0.30 | TUI framework |
| crossterm 0.28 | Terminal backend |
| clap 4.6 (derive) | CLI parsing |
| serde + serde_json | Serialization |
| indexmap 2 | Ordered maps (categories, views) |
| anyhow | Error handling |
| chrono 0.4 | Date parsing in import |
| pest + pest_derive | PEG parser for .improv format |
| flate2 | Gzip for .improv.gz |
| csv | CSV parsing |
| enum_dispatch | CLI subcommand dispatch |
| **dev:** proptest, tempfile, pest_meta | Property testing, temp dirs, grammar AST for test generator |
Full guidance lives in `context/design-principles.md` §6. Quick reminders:
- Suite runs in <2s. Don't let that drift.
- Commands: build a `CmdContext`, call `execute`, assert on returned effects. No terminal needed.
- Property tests (`proptest`, default 256 cases) cover invariants: CellKey sort, axis consistency, save/load roundtrip, `parse(format(parse(generate())))` stability.
- **Bug-fix workflow**: write a failing test *before* the fix (regression guard). Document the bug in the test's doc-comment (see `model/types.rs::formula_tests`).
- Coverage target ~80% line/branch on logic code; skip ratatui render paths.
---
## File Inventory
## Key dependencies
Lines / tests / path — grouped by layer.
### Core crate layers (sub-crate `improvise-core` under `crates/`)
All of `model/`, `view/`, `workbook.rs`, `format.rs` now live under
`crates/improvise-core/src/`. Paths below are relative to that root.
The main crate re-exports each as `crate::model`, `crate::view`,
`crate::workbook`, `crate::format` via `src/lib.rs`, so every
consumer path stays unchanged.
#### Model layer
```
2062 / 70t model/types.rs Model struct, formula eval, CRUD, MAX_CATEGORIES=12
650 / 28t model/cell.rs CellKey (canonical sort), CellValue, DataStore (interned, sort_by_key)
222 / 6t model/category.rs Category, Item, Group, CategoryKind
79 / 3t model/symbol.rs Symbol interning (SymbolTable)
6 / 0t model/mod.rs
```
#### View layer
```
1140 / 24t view/layout.rs GridLayout (pure fn of Model+View), records mode, drill
531 / 28t view/types.rs View config (axes, pages, hidden, collapsed, format), none_cats()
21 / 0t view/axis.rs Axis enum {Row, Column, Page, None}
7 / 0t view/mod.rs
```
#### Workbook + format (top-level in improvise-core)
```
259 / 11t workbook.rs Workbook wraps Model + views; cross-slice category/view ops
229 / 29t format.rs format_f64, parse_number_format (display-only rounding)
12 / 0t lib.rs Module declarations + `pub use improvise_formula as formula;`
```
### Formula layer (sub-crate `improvise-formula` under `crates/`)
```
776 / 65t crates/improvise-formula/src/parser.rs PEG parser (pest) → Formula AST
77 / 0t crates/improvise-formula/src/ast.rs Expr, BinOp, AggFunc, Formula, Filter (data only)
5 / 0t crates/improvise-formula/src/lib.rs
```
### Command layer
```
command/cmd/ Cmd trait, CmdContext, CmdRegistry, 40+ commands
297 / 2t core.rs Cmd trait, CmdContext, CmdRegistry, parse helpers
586 / 0t registry.rs default_registry() — all command registrations
475 / 10t navigation.rs Move, EnterAdvance, PageNext/Prev
198 / 6t cell.rs ClearCell, YankCell, PasteCell, TransposeAxes, SaveCmd
330 / 7t commit.rs CommitFormula, CommitCategoryAdd/ItemAdd, CommitExport
437 / 5t effect_cmds.rs effect_cmd! macro, 25+ parseable effect-wrapper commands
409 / 7t grid.rs ToggleGroup, ViewNavigate, DrillIntoCell, TogglePruneEmpty
308 / 8t mode.rs EnterMode, Quit, EditOrDrill, EnterTileSelect, etc.
587 / 13t panel.rs Panel toggle/cycle/cursor, formula/category/view panel cmds
202 / 4t search.rs SearchNavigate, SearchOrCategoryAdd, ExitSearchMode
256 / 7t text_buffer.rs AppendChar, PopChar, CommandModeBackspace, ExecuteCommand
160 / 5t tile.rs MoveTileCursor, TileAxisOp
121 / 0t mod.rs Module declarations, re-exports, test helpers
1066 / 22t command/keymap.rs KeyPattern, Binding, Keymap, ModeKey, 14 mode keymaps
236 / 19t command/parse.rs Script/command-line parser (prefix syntax)
12 / 0t command/mod.rs
```
### UI layer
```
942 / 41t ui/effect.rs Effect trait, 50+ effect types (all state mutations)
914 / 30t ui/app.rs App state, AppMode (15 variants), handle_key, autosave
1036 / 13t ui/grid.rs GridWidget (ratatui), col widths, rendering
617 / 0t ui/help.rs 5-page help overlay, HELP_PAGE_COUNT=5
347 / 0t ui/import_wizard_ui.rs Import wizard overlay rendering
165 / 6t ui/cat_tree.rs Category tree flattener for panel
113 / 0t ui/view_panel.rs View list panel
107 / 0t ui/category_panel.rs Category tree panel
95 / 0t ui/tile_bar.rs Tile bar (axis assignment tiles)
87 / 0t ui/panel.rs Generic panel frame widget
81 / 0t ui/formula_panel.rs Formula list panel
67 / 0t ui/which_key.rs Prefix-key hint popup
12 / 0t ui/mod.rs
```
### I/O crate layers (sub-crate `improvise-io` under `crates/`)
All of `persistence/` and `import/` now live under `crates/improvise-io/src/`.
Paths below are relative to that root. The main crate re-exports each
as `crate::persistence` and `crate::import` via `src/lib.rs`, so every
consumer path stays unchanged.
#### Persistence
```
124 / 0t persistence/improv.pest PEG grammar — single source of truth for .improv format
2410 / 83t persistence/mod.rs .improv save/load (pest parser + format + gzip + legacy JSON), export_csv
```
#### Import
```
1117 / 38t import/wizard.rs ImportPipeline + ImportWizard
292 / 9t import/analyzer.rs Field kind detection (Category/Measure/Time/Skip)
300 / 8t import/csv_parser.rs CSV parsing, multi-file merge
3 / 0t import/mod.rs
16 / 0t lib.rs Module decls + re-exports of improvise-core/formula modules
```
### Top-level
```
400 / 0t draw.rs TUI event loop (run_tui), frame composition
391 / 0t main.rs CLI entry (clap): open, import, cmd, script
10 / 0t lib.rs Public module re-exports (routes to sub-crates)
```
### Examples
```
examples/gen-grammar.rs Grammar-walking random file generator (pest_meta)
examples/pretty-print.rs Parse stdin, print formatted .improv to stdout
```
### Context docs
```
context/design-principles.md Architectural principles
context/plan.md Show HN launch plan
context/repo-map.md This file
docs/design-notes.md Product vision & non-goals (salvaged from former SPEC.md)
```
**Total: ~22,000 lines, 572 tests.**
---
## Testing Guidelines
### Coverage target
Aim for **~80% line and branch coverage** on logic code. This is a quality floor, not a
ceiling — go higher where the code warrants it, but don't chase 100% on rendering
widgets or write tests that just exercise trivial getters. Coverage should be run with
`cargo llvm-cov` (available via `nix develop`).
### What to test and how
| Layer | Approach | Notes |
|-------|----------|-------|
| **Model** (types, cell, category, symbol) | Unit tests + **proptest** | The data model is the foundation. Property tests catch invariant violations that hand-picked cases miss (see CellKey sort invariant, axis consistency). |
| **Formula** (parser, eval) | Unit tests per operator/construct | Cover each BinOp, AggFunc, IF, WHERE, unary minus, chained formulas, error cases (div-by-zero, missing ref). Ensure eval uses full f64 precision — never display-rounded values. |
| **View** (types, layout) | Unit tests + **proptest** | Property tests for axis assignment invariants (each category on exactly one axis, transpose is involutive, etc.). Unit tests for layout computation, records mode detection, drill. |
| **Command** (cmd, keymap, parse) | Unit tests | Test command execution by building a `CmdContext` and asserting on returned effects. Test keymap lookup fallback chain. Test script parser with edge cases (quoting, comments, dots). |
| **Persistence** | Round-trip + grammar-generated | `save → load → save` must be identical. Grammar-walking generator produces random valid files from the pest AST; proptests verify `parse(generate())` and `parse(format(parse(generate())))`. Cover groups, formulas, views, hidden items, pipe quoting edge cases. |
| **Format** | Unit tests | Boundary cases: comma placement at 3/4/7 digits, negative numbers, rounding half-away-from-zero (not banker's), zero, small fractions. |
| **Import** (analyzer, csv, wizard) | Unit tests | Field classification heuristics, CSV quoting (RFC 4180), multi-file merge, date extraction. |
| **UI rendering** (grid, panels, draw, help) | Generally skip | Ratatui widgets are hard to unit-test and change frequently. Test the *logic* they consume (layout, cat_tree, format) rather than the rendering itself. |
| **Effects** | Test indirectly | Effects are thin `apply` methods. Test via integration: send a key through `App::handle_key` and assert on resulting app state. The complex ones (drill reconciliation, import) deserve targeted unit tests. |
### Property tests (proptest)
Use property tests for **invariants that must hold across all inputs**, not as a
substitute for example-based tests. Good candidates:
- Structural invariants: CellKey always sorted, each category on exactly one axis,
toggle-collapse is involutive, hide/show roundtrips.
- Serialization roundtrips: save/load identity.
- Determinism: `evaluate` returns the same value for the same inputs.
Keep proptest case counts reasonable. The defaults (256 cases) are fine for most
properties. Don't crank them up to thousands — the test suite should complete in
under 2 seconds. If a property needs more cases to feel confident, that's a sign
the input space should be constrained with better strategies, not brute-forced.
### Bug-fix workflow
Per CLAUDE.md: **write a test that demonstrates the bug before fixing it.** Prefer
a small unit test targeting the specific function over an integration test. The test
should fail on the current code, then pass after the fix. Mark regression tests
with a doc-comment explaining the bug (see `model/types.rs` `formula_tests` for
examples).
### What not to test
- Trivial struct constructors and enum definitions (`ast.rs`, `axis.rs`).
- Ratatui `Widget::render` implementations — these are pure drawing code.
- Module re-export files (`mod.rs`).
- One-line delegation methods.
---
## Patterns to Know
1. **Commands never mutate.** They receive `&CmdContext` (read-only) and return `Vec<Box<dyn Effect>>`.
2. **CellKey is always sorted.** Use `CellKey::new()` — never construct the inner Vec directly.
3. **`category_mut()` for adding items.** `Model` has no `add_item` method; get the category first: `m.category_mut("Region").unwrap().add_item("East")`.
4. **Virtual categories** `_Index`, `_Dim`, and `_Measure` always exist. `is_empty_model()` checks whether any *non-virtual* categories exist. `_Measure` items come from two sources: explicit data items (in category) + formula targets (dynamically via `measure_item_names()`). `add_formula` does NOT add items to `_Measure` — use `effective_item_names("_Measure")` to get the full list. `_Index` and `_Dim` are never persisted to `.improv` files; `_Measure` only persists non-formula items.
5. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full f64.
5b. **Formula evaluation is fixed-point.** `recompute_formulas(none_cats)` iterates formula evaluation until values stabilize, using a cache. `evaluate_aggregated` checks the cache for formula results. Circular refs produce `CellValue::Error("circular")`.
6. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the current mode, then the corresponding `Keymap` is looked up. Normal + `search_mode=true` maps to `SearchMode`.
7. **`effect_cmd!` macro** generates a command struct that just produces effects. Use for simple commands without complex logic.
8. **`.improv` format is defined by a PEG grammar** (`persistence/improv.pest`). Parsed by pest. Names use CL-style `|...|` pipe quoting when they aren't valid bare identifiers. JSON is legacy only.
9. **`IndexMap`** is used for categories and views to preserve insertion order.
10. **`MAX_CATEGORIES = 12`** applies only to `CategoryKind::Regular`. Virtual/Label categories are exempt.
11. **Drill into formula cells** strips the `_Measure` coordinate from the drill key when it names a formula target, so `matching_cells` finds the raw data records that feed the formula instead of returning empty.
12. **`App::new` calls `recompute_formulas`** before building the initial layout, so formula values appear on the first rendered frame.
13. **Minibuffer buffer clearing** is handled by `Binding::Sequence` in keymaps: Enter and Esc sequences include `clear-buffer` to reset the text buffer. The `clear-buffer` command is registered in the registry.
ratatui 0.30, crossterm 0.28, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest + pest_derive, flate2 (gzip), csv, enum_dispatch. Dev: proptest, tempfile, pest_meta.