improvise/context/repo-map.md

Repository Map (LLM Reference)

Terminal pivot-table app. Rust 2024, Ratatui TUI, command/effect architecture. Apache-2.0. Root binary+lib improvise; sub-crates under crates/:

• improvise-core — Model, View, Workbook, number formatting. No UI/IO.
• improvise-formula — formula parser, AST, parse_formula.
• improvise-io — .improv save/load, CSV/JSON import, Storage backends. No UI/commands.
• improvise-protocol — the wire vocabulary and the portable session slice: Command, Projection, ViewState, the per-widget caches, reduce_view, key types. Depends only on improvise-core + serde. No IO, no command layer, no terminal.
• improvise-command — command/effect layer + semantic App state (app.rs, effect.rs, cat_tree.rs, viewport.rs). Depends on improvise-protocol. No ratatui/crossterm.
• gp-tree — problem-agnostic genetic-programming regression trees (Tree, Spec, breed, simplify, render). Knows nothing about improvise.
• improvise-gp — the one seam between the two: emits a fitted gp_tree::Tree as improvise formula text (formula_text / fitted_formula). Depends on gp-tree, improvise-formula and improvise-core — dataset.rs builds a feature matrix from a View's records (improvise-4s8) and fitted_formula takes the &Model it must resolve against (improvise-8xhy). Nothing depends on it yet.
• improvise-ws-server — the host: tokio + tungstenite, a Registry of Hosts keyed by DocumentId (multi-document, ADR 001), Sessions fan-out, and the socket framing (wire.rs). Native only.
• improvise-wasm-client — the browser client: the fold, the key classifier, the socket framing it mirrors (socket.rs), and the render model the DOM paints (render.rs). Non-dev deps are improvise-protocol + serde + wasm-bindgen and nothing else. The page itself is web/.

The main crate (and improvise-command) re-export the sub-crates as crate::{model, view, workbook, format, formula, persistence, import} so consumer paths stay stable when crates shuffle. The main crate also re-exports improvise_command::command as crate::command and improvise_command::ui::{app, cat_tree, effect, viewport} under crate::ui. ui::app re-exports the protocol's AppMode, ViewState, ViewCursor, ViewFrame, Panel, MinibufferConfig, MinibufColor and HELP_PAGE_COUNT, and command::key re-exports the protocol's key types, so every pre-existing path still resolves.

Architectural intent lives in context/design-principles.md — read that for the "why". This doc is the "where".

---

How to Find Things

I need to...	Look in
Add a keybinding	command/keymap.rs → default_keymaps()
Add a user command	command/cmd/<submodule>.rs, register in registry.rs
Add a state mutation	ui/effect.rs → implement Effect
Change formula eval	model/types.rs → eval_formula / eval_expr
Change cell storage / lookup	model/cell.rs → DataStore
Change category/item behavior	model/category.rs → Category
Change view axis logic	view/types.rs → View
Change an axis label, color or cycle	view/axis.rs → impl Axis (colors resolve in draw.rs::axis_fg)
Ask "what data is visible" vs "what can the user page"	View::filtering_cats() vs categories_on(Axis::Page)
Change grid layout	view/layout.rs → GridLayout
Change .improv format	persistence/improv.pest + persistence/{format,parse}.rs
Change where documents are stored	storage.rs → Storage impls (opfs.rs for wasm)
Change number display	format.rs → format_f64
Change CLI args	main.rs (clap)
Change import logic	import/wizard.rs → ImportPipeline
Change how a gp-tree fit searches	gp-tree/src/fit.rs → FitConfig / fit
Change gp-tree CSV preprocessing	gp-tree/src/data.rs → preprocess
Change how a name is quoted anywhere	improvise-formula/src/quote.rs — the one home, both halves
Change import-wizard keys/steps	command/cmd/wizard.rs (keys) + import/wizard.rs (steps)
Change frame layout	draw.rs → draw()
Change what the browser page draws	improvise-wasm-client/src/render.rs → RenderModel
Change a browser↔server frame	improvise-wasm-client/src/socket.rs and improvise-ws-server/src/wire.rs — both, they are mirrors
Change the browser page itself	web/app.js (thin), web/index.html, web/style.css
Host another document	improvise-ws-server/src/registry.rs → Registry, ClientId
Change app state / modes	ui/app.rs → App, AppMode
Write a test for model logic	model/types.rs → mod tests / formula_tests
Write a test for a command	command/cmd/<module>.rs → colocated mod tests; helpers in cmd/mod.rs::test_helpers

---

Central Pattern: Cmd → Effect

keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
                            (pure, read-only ctx)       (list of mutations)    (only mutation site)

pub trait Cmd: Debug + Send + Sync {
    fn name(&self) -> &'static str;
    fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
}

pub trait Effect: Debug {
    fn apply(&self, app: &mut App);
    fn kind(&self) -> EffectKind;              // NO default — the compiler makes you classify
}

// EffectKind::{ViewOnly, Model(ModelChange)}
// ModelChange::{CellData, LayoutGeometry, WholeWorkbook}

CmdContext (see command/cmd/core.rs) holds immutable refs to Model, GridLayout, AppMode, cursor/offsets, buffers, yanked cell, key code, expanded cats, and panel/tile cursors.

Add a command: implement Cmd in a command/cmd/ submodule, register in registry.rs, bind in default_keymaps(). Simple wrappers go through the effect_cmd! macro in effect_cmds.rs.

Add an effect: implement Effect in ui/effect.rs, add a constructor fn if it helps composition.

---

Key Types (skim; read the source for fields/methods)

Model (model/types.rs): categories: IndexMap<String, Category>, data: DataStore, formulas: Vec<Formula>, views: IndexMap<String, View>, active_view, measure_agg. MAX_CATEGORIES = 12 (regular only). Virtual categories _Index, _Dim, _Measure always exist. Use regular_category_names() for user-facing pickers.

CellKey (model/cell.rs): Vec<(cat, item)> — always sorted by category. Build with CellKey::new(coords) / with(cat, item) / without(cat). Never construct the inner Vec directly.

CellValue: Number(f64) | Text(String) | Error(String). Errors surface from formula eval (circular, div/0, etc.).

DataStore (model/cell.rs): interned symbols + secondary index (Symbol, Symbol) → {InternedKey}. Hot-path lookups go through matching_values / matching_cells.

Formula AST (improvise-formula/src/ast.rs):

• Expr { Number, Ref, BinOp, UnaryMinus, Agg, If }
• BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }
• AggFunc { Sum, Avg, Min, Max, Count }
• Formula { raw, target, target_category, expr, filter }

View (view/types.rs): category_axes: IndexMap<cat, Axis>, page selections, hidden items, collapsed groups, number_format, prune_empty. It carries no cursor — selected/row_offset/col_offset moved to ViewState::cursors (improvise-8uz). Axis { Row, Column, Page, Filter, None }. cycle_axis rotates Row → Column → Page → None; Filter is outside the rotation (see gotcha 27).

GridLayout (view/layout.rs): pure function of Model + View. cell_key(r,c), cell_value(r,c), drill_records(r,c). Records mode auto-detects when _Index is on Row and _Dim is on Column.

AppMode (ui/app.rs): 16 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, CommandMode, ImportWizard, Help, Quit, RecordsNormal, RecordsEditing). Text-entry variants carry a MinibufferConfig. SearchMode is Normal + search_mode: bool, not its own variant. Per-variant UI facts live on the enum as exhaustive-match methods — label(), hint_text(), panel(), minibuffer(), is_editing(), is_records() (improvise-2hi); add a variant and the compiler names every one you must fill in. mode_style() stays in src/draw.rs because Style is a ratatui type. App state is split into ModelState (workbook, file_path, dirty) and ViewState (mode, cursor/offsets, buffers, panels, search) — see improvise-x2c/ew0. wizard and drill_state live on App, not ViewState (improvise-h0b) — see gotcha 24.

Keymap (command/keymap.rs): Binding { Cmd | Prefix(Arc<Keymap>) | Sequence(Vec<…>) }. Lookup fallback: exact(key,mods) → Char(NONE) → AnyChar → Any → parent. 16 mode keymaps built by KeymapSet::default_keymaps(); mode resolved via ModeKey::from_app_mode().

---

.improv File Format

Plain-text, markdown-like, defined by persistence/improv.pest. Parsed by pest — the grammar is the single source of truth (the grammar-walking test generator reads it via pest_meta).

v2025-04-09
# Model Name
Initial View: Default

## View: Default
Region: row
Measure: column
|Time Period|: page, Q1
hidden: Region/Internal
collapsed: |Time Period|/|2024|
format: ,.2f

## Formulas
- Profit = Revenue - Cost               # defaults to [_Measure]
- Tax = Revenue * 0.1 [CustomCat]

## Category: Region
- North, South, East, West              # bare items, comma-separated
- Coastal_East[Coastal]                 # grouped item, one per line
> Coastal                               # group definition

## Data
Region=East, Measure=Revenue = 1200
Region=West, Measure=Revenue = |pending|   # pipe-quoted text

• Name quoting: bare [A-Za-z_][A-Za-z0-9_-]*, else CL-style |…| with escapes \|, \\, \n. Same convention in the formula tokenizer.
• Write order: Views → Formulas → Categories → Data. Parser tolerates any order.
• Gzip: .improv.gz (same text, gzipped).
• Legacy JSON: auto-detected by leading {; never written.
• Axis keywords: row, column, page, filter, none; page/filter carry the selected item after a comma.
• Virtual categories: _Index/_Dim never persist; _Measure persists only non-formula items (formula targets are rebuilt from the ## Formulas section).
• Virtual views: _Records/_Drill (view::VIRTUAL_VIEW_NAMES) are never written and are dropped on load if an older file carries them; saving from records/drill mode writes the first real view as Initial View. Their commands rebuild them on demand.

---

Gotchas (read before editing)

1. Records mode is wide format. A row is one coordinate tuple — the cells sharing everything but _Measure — and each raw measure gets its own editable column, followed by a + column that creates a new measure (improvise-c5v). Consequences: two measures on one coordinate are ONE row; editing a category column moves every measure cell in the row, not just the one under the cursor; _Dim in the synthetic key carries the column index, not a label, because a category and a measure may share a name — resolve it with GridLayout::record_col into a RecordCol. Formula targets are not columns (nothing to type into them). Drill pending edits are keyed by (row, RecordCol). prune_axis_items takes a predicate, not a flag vector. prune_empty hands it impl Fn(usize) -> bool over the data index (group headers consume no index). The domain is derived from items itself, so there is no parallel length a caller could get wrong — the earlier &[bool] + debug_assert pair could disagree between debug and release if a third caller ever sized keep from something other than row_count()/col_count() (improvise-n3wt).

2. Effects never touch std::fs. Every document read or write goes through App::storage: Box<dyn Storage> (improvise-fuj). The host picks the backend: App::new supplies PhysicalFs, App::with_storage takes any other — a browser worker will supply OPFS (improvise-i34), and tests use MemoryFs. improvise-io keeps path-taking wrappers (save, load, export_csv, parse_csv_with_warnings) for the native CLI, each delegating to a _to/_from/_bytes variant that takes the backend or the bytes.

3. Every effect declares an EffectKind, and there is no default. ViewOnly never reaches other clients; Model(CellData | LayoutGeometry | WholeWorkbook) does (improvise-mae). A new effect that forgets to classify itself fails to compile — that is deliberate, since the failure mode is silent: a mis-declared ViewOnly edit would simply never reach connected clients. Watch two traps: the View lives inside model_state.workbook, so "touches model_state" does NOT mean "is a model change" (axis/page/format are LayoutGeometry, cursor and scroll are ViewOnly — see improvise-8uz); and Save/SaveAs/MarkDirty touch model_state for bookkeeping only and are ViewOnly. AddCategory/RemoveCategory are the third trap and go the other way: they look like data edits, but both go through Workbook, whose on_category_added/on_category_removed rewrite every view's axis map, so they are LayoutGeometry. The two "an effect declaring X actually changes X" tests cannot separate them — a category change moves model data and view config, so it clears either bar; structural_effects_declare_their_kind pins the declaration itself. A third trap, now guarded: EffectKind is too coarse to check a mapping. ui/reduce.rs::model_effects sends each of 28 ModelCommand variants to one effect, and every inverse pair — HideItem/ShowItem, AddItem/RemoveItem, CreateView/DeleteView, SetCell/ClearCell — declares the same kind, so swapping three of them at once left the suite at 918 passing while three wire messages did the exact inverse of what they said. The mapping is name-for-name by construction, so every_model_command_names_the_effect_of_the_same_name asserts that, reading the type name off the effect's derived Debug (Effect is a trait object, so type_name is out of reach). A variant whose effect must be named something else is a variant that stopped being derived — it should fail there and be argued about. Their reach is wider than the name suggests: Model::remove_category also drops every formula targeting the category, which is why Dirty::of(LayoutGeometry) reports formulas: true — see gotcha 35.

4. Commands never mutate. &CmdContext is read-only; return Vec<Box<dyn Effect>>. If you want to touch &mut App, add or extend an Effect.

5. CellKey is always sorted. Use the constructors; equality and hashing rely on canonical form.

6. No Model::add_item. With a Workbook in hand use the facade: wb.add_item("Region", "East")? / add_item_in_group / remove_item (improvise-tvt). With only a Model, go through the category: m.category_mut("Region").unwrap().add_item("East").

7. Virtual categories. _Measure items = explicit items ∪ formula targets. Use measure_item_names() / effective_item_names("_Measure"). add_formula does not add items to _Measure. Formula-target lookup also falls back into _Measure via Model::find_item_category. Cell aggregation goes through apply_agg, formula SUM(…)/AVG(…) through eval_agg — they differ on empty input (absent vs. A third rule joins them for non-numeric values: aggregate_cell_values reduces text by unanimity — one matching value, or several copies of the same value, is that value; differing values, or text among numbers, is CellValue::Error("mixed") (improvise-l4qx). The all-numeric branch is checked first, so Count of a lone number is still 1 and not the number — that ordering is the whole difference between this rule and "a single value passes through". What the rule forbids is dropping: filter_map(as_f64) used to, and since _Index/_Dim are on Axis::None in every default view that made every text cell in every document render blank, including the pipe-quoted text .improv carries as a first-class value. evaluate_aggregated and aggregate_raw share the helper, so a formula ref over mixed values now errors "mixed" rather than silently summing the numeric ones. 0 for Sum/Count, error for Avg/Min/Max).

8. Display rounding is view-only. format_f64 (half-away-from-zero) is only called in rendering. Formula eval uses full f64. Never feed a rounded value back into eval.

9. Formula eval is fixed-point. recompute_formulas(none_cats) iterates until stable, bounded by MAX_EVAL_DEPTH; circular refs converge to CellValue::Error("circular"). App::new runs it before the first frame so formula cells render on startup.

10. Drill into formula cells strips the _Measure coordinate when it names a formula target, so matching_cells returns the raw records that feed the formula instead of returning empty.

11. Minibuffer keymaps come from TextEntry. The seven text-entry modes are built by the TextEntry builder in keymap.rs, which binds the shared spine (AnyChar→append-char <buf>, Backspace→pop-char <buf>, Esc→[clear-buffer <buf>, enter-mode <exit>]) and welds every commit key to its clear-buffer via .commit(key, cmd, args) (improvise-w3q). TextEntry::new takes the AppMode and reads <buf> off its MinibufferConfig, so the buffer the prompt names and the buffer the keys edit are the same string by construction (improvise-k8h) — the keymap never spells a buffer name. .rebind exists only for CommandMode's Backspace. SearchMode is hand-written on purpose — no commit key, and exit-search-mode owns the query. EnterMode clears the incoming mode's minibuffer through write_buffer too — building SetBuffer directly there was unreachable only because no shipped AppMode is keyed "search". MinibufferConfig::buffer_key is a String, not &'static str: a borrowed-'static field makes Deserialize unimplementable, and every text-entry AppMode variant carries a MinibufferConfig (improvise-a2x). A buffer named at runtime is read with cmd::core::read_buffer and written with its mirror write_buffer — "search" is not in the buffers map but its own ViewState field, and those two functions are the only code that knows it (improvise-gdl).

12. Panel keymaps come from PanelKeymap. The three panel modes share a spine (Esc/Tab→Normal, arrows and j/k→move-panel-cursor <panel>, every panel's toggle key, :→command mode) built by PanelKeymap::new(panel); the keys that act on the panel's contents arrive through .bind/.bind_args/.delete because they vary in shape, not just command name (improvise-4pg). Panel::ALL/name()/key() in ui/app.rs drive the toggle keys for Normal and all three panels, so a fourth panel reaches every keymap from one ALL entry. name() must spell what parse_panel accepts.

13. Prefix keys are stored as an address, not a keymap. ViewState::transient_prefix holds the (KeyCode, KeyModifiers) that opened a prefix; KeymapSet::prefix_keymap resolves it back to the sub-keymap, walking parents so RecordsNormal reaches Normal's g/y/Z (improvise-jc8). ViewState must stay free of improvise-command types — it is destined for the protocol crate, which improvise-command will depend on. Resolution assumes prefixes are flat; prefix_keymaps_are_flat fails the build if a nested one is added.

14. Keybindings are per-mode. ModeKey::from_app_mode() resolves the mode; Normal + search_mode=true maps to SearchMode. Minibuffer modes bind Enter/Esc as Binding::Sequence so clear-buffer fires alongside commit/cancel. Hints are checked against the keymaps: every_hint_key_resolves_in_its_mode (command/keymap.rs) splits each AppMode::hint_text() and requires every key it names to resolve in that mode's map, because only_quit_has_no_hint asserts only that a hint is present. It checks which keys are named, not what they do. Its tokenizer knows the hint dialect explicitly (named keys, Ctrl-<c>, F/C/V alternatives, character runs like hjkl/[:], the em-dash prose tail, CommandMode's :q ex-commands) and panics on a token it cannot classify rather than skipping it.

15. Registry names come from prototypes, not strings. Every register* call reads the name off a prototype Cmd instance, so a typo is impossible (improvise-9cn) — there is no name literal in registry.rs. What types can't reach: four commands name themselves off a field (ViewNavigate, TileAxisOp, ToggleGroupAtCursor, CommitAndAdvance), so a prototype can name the wrong variant and still compile. That shows up as a duplicate registration, not a mismatch — the wrong-variant entry collides with its sibling and the sibling's name vanishes. registered_names_are_unique catches it; every_bound_command_name_is_registered catches the keymap side, and checks both halves of dispatch's registry.interactive(name, args, ctx).ok()? — the name resolves and the args parse — because improvise-s13 moved enter-mode ["export-prompt"]/["formula-edit"] out of the nullary class, where a typo'd mode name compiles into a dead key. It builds the CmdContext in each keymap's own AppMode, since commit-buffer item reads ctx.mode.

16. effect_cmd! macro wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written Cmd once there's real decision logic. Its counterpart on the effect side is simple_effect! in ui/effect.rs, which declares a one-field newtype that writes one place on the App. Its kind argument is mandatory — simple_effect!(Name(T) => |app| place, EffectKind::ViewOnly) — even though every call site so far answers ViewOnly: hard-coding it reintroduced exactly the default gotcha 3 does without, since $place is an unconstrained expression and a one-line active_view_mut().number_format setter would have reported ViewOnly with the suite green (improvise-8fy). Likewise, mode entry has no per-mode shim commands — bind enter-mode <mode-name> (improvise-s13); the name is decoded by parse_mode_name in registry.rs, and EnterMode clears the mode's minibuffer on the way in.

17. Float equality inside formula eval is currently mixed — =/!= use a 1e-10 epsilon but the div-by-zero guard checks rv == 0.0 exactly. Don't assume one or the other; tracked for cleanup.

18. IndexMap preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do DataStore::matching_cells/matching_values: the secondary index narrows the candidates, then matching_positions resolves each to its position in the primary IndexMap, sorts, and reads back. An index whose values are HashSets is for O(1) lookup, never for iteration. Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; SUM/AVG drifted in the last bits because f64 addition is not associative (Min/Max/Count and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the drill snapshot froze its record list in hash order while drill_state.pending_edits are keyed by row index (gotcha 1), so the same drill opened in a second process addressed different records — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on matching_values, and matching_cells 7% faster, sorted positions resolving CellKeys with better locality. The alternatives both move the cost — an ordered index value type pays on every set, scanning the primary is O(store) per query. DataStore::live_records still walks a HashMap and is the next instance; it has no callers yet, so its order is an open choice (improvise-2yc2) rather than a bug.

19. Commit paths for cell edits split across commit_cell_value → commit_regular_cell_value / commit_plain_records_edit in command/cmd/commit.rs. Keep the synthetic-records branch in sync with the plain branch when changing behavior.

20. Minibuffer commits go through one command. commit-buffer <formula|category|item> (CommitBuffer { kind: CommitKind }) owns buffer→trim→bail-on-empty→dirty→status for all three; the varying parts are methods on CommitKind. The add effect is emitted last so its failure message wins over the command's optimistic status (improvise-6t3). CommitKind::Item carries its target category, resolved from AppMode::ItemAdd by parse_commit_kind at registry time — so commit-buffer item is not constructible from a script line.

21. A failed Save aborts the batch. The Save effect sets abort_effects when there's no file path or the write fails (improvise-8tk), so anything chained after it — the Quit in wq/ZZ — is skipped and the error stays on screen. A Binding::Sequence flattens into one effect vec, so the abort carries across sequence steps too.

22. The import wizard's back path is a history stack, not a reverse table. advance pushes the step it leaves onto ImportWizard::history; previous pops it, so back always retraces the path actually taken through the conditional transitions (improvise-907). Preconditions live in validation_error(), checked on the step where the user can still fix them — never deferred to build_model. NameModel is terminal for advance: Enter there calls build(), which parks the workbook in built on Done, and AcceptWizardModel takes it from there. Going back from Done discards built.

23. app.layout goes stale mid-batch. apply_effects rebuilds only after the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a RebuildLayout effect in front of it — the command decides the order (see EnterEditAtCursorCmd, CommitAndAdvance). Read the cursor's cell text via App::display_value_at_cursor(); don't build a CmdContext inside apply (it recomputes column widths).

24. The cursor is per-client and keyed by view. ViewState::cursors: HashMap<String, ViewCursor> holds selected/row_offset/col_offset, keyed by view name; an absent key means the origin. Reach it through App::cursor() (a Copy snapshot), App::cursor_mut(), and App::reset_cursor() — never off View, which no longer has the fields (improvise-8uz). Two consequences. Changing the grid's shape must reset the cursor explicitly: View::transpose_axes/cycle_axis used to do it themselves, and now TransposeAxes/CycleAxis call app.reset_cursor() — a new geometry effect that renumbers rows or columns has to do the same. Replacing the document must clear the map: LoadModel and AcceptWizardModel call cursors.clear(), because the incoming workbook's view names have nothing to do with the outgoing one's. That clearing replaces Workbook::normalize_view_state, which is gone — a View can no longer carry a stale offset. The same clearing owes the session mirrors: install_workbook bumps App::document_generation, and ClientSession::project reseeds any mirror seeded at an earlier one (improvise-cqq). An effect cannot reach Sessions, so the generation is the mechanism and Sessions::rebase is only the client-facing half. The same is true of a geometry change, and it owes the mirrors both halves: App::cursor_epoch is bumped by reset_cursor and watched by Sessions::adopt_cursor_epoch, which resets every mirror and returns Broadcast::ToAll(Command::ResetCursor) (improvise-gvnb). Doing only the first is strictly worse than doing nothing — the diff then believes every client already holds origin-positioned content and sends nothing, so they stay stale permanently rather than until their next scroll. The trigger is the epoch moving, not the command variant, for the reason gotcha 39 watches document_generation instead of matching on LoadModel: which effects renumber the grid is a per-effect decision, so a table of variants would be a second description of it, free to disagree. What is broadcast is an invalidation, never a position — another client's transpose must not move where I am looking (that is UI and it is mine, improvise-g0f), only declare that where I was looking has stopped meaning anything. And the observation is only as complete as the reset_cursor calls. Nothing declares which effects renumber the grid — deliberately — so a geometry effect that forgets reset_cursor is invisible in both directions: the host's own cursor stays stale and no client is told. Today only TransposeAxes and CycleAxis call it; SetAxis, HideItem/ShowItem, ToggleGroup, TogglePruneEmpty, AddCategory/RemoveCategory and AddItem/RemoveItem all renumber and do not (improvise-9mo9). SetAxis is the sharpest — CycleAxis is SetAxis to Axis::next(), and one resets while the other does not. The negative arm is real and must stay tested (SetNumberFormat, a LayoutGeometry effect that must not reset): without it, "reset on every model command" would pass the positive test.

25. ViewState and its field types are Serialize/Deserialize. It is the wire payload of the client Snapshot (improvise-q08), so AppMode, MinibufferConfig, MinibufColor, ViewFrame, KeyCode and KeyModifiers all derive both (improvise-a2x), and improvise-command depends on serde for it. The constraint this puts on new state: no borrowed-'static fields anywhere ViewState can reach — Deserialize cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. view_state_survives_a_json_round_trip guards it. ViewState and Snapshot also derive Clone and PartialEq, so the bootstrap message can be compared in a round-trip test.

26. Axis presentation lives on Axis, colors resolve in draw.rs. Four vocabularies hang off the enum in view/axis.rs — plain_label() ("Row"/"Col"/"Page"/"None"), glyph() (which Display appends, giving "Row ↕"), short_label() (three columns for the tile bar; None is "·", not a truncation), and initial() (one letter for the view panel; None is empty). Plus color() -> AxisColor and next() for the Row→Column→Page→None rotation. AxisColor is a semantic palette: improvise-core must not name a ratatui type, so draw.rs::axis_fg does the one mapping to a terminal color, exactly as minibuf_fg does for MinibufColor (improvise-rml). The tile bar and category panel each carried their own copy of the color table before, with nothing keeping them in agreement. Not covered here: persistence/format.rs maps Axis to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer.

27. Axis::Filter constrains without being navigable. Drilling fixes its coordinates on Filter, not Page (improvise-dz8): both restrict which records are visible, but only Page is something the user may move or page through. The split shows up as two different questions, and picking the wrong one is the trap: "what data is visible" is View::filtering_cats() (Page ∪ Filter — used by GridLayout and by add-record-row when it builds a key from the current constraints), while "what can the user act on" is categories_on(Axis::Page) (used by [/] page cycling) or View::tileable_cats() (used by all three tile-select sites). Axis::next() maps Filter to itself and no navigable axis rotates into it, so tile select cannot hand one out. The .improv grammar accepts filter and round-trips a selection with it — a Filter axis only ever occurs in the never-persisted _Drill view, but the alternative was writing it out as some other axis.

28. The browser backend is sync on purpose, with an async seam. OpfsFs (improvise-io/src/opfs.rs, wasm32 only) implements the same synchronous Storage trait as PhysicalFs, because FileSystemFileHandle.createSyncAccessHandle() yields a handle whose read/write/truncate/flush are blocking — available only inside a dedicated Web Worker, which is where improvise-9x6 puts the app. So Storage never became async. What is still async splits out into four explicit methods the worker awaits from its message loop: open(), attach(path) (must precede any read/write of that document — acquiring a handle is async even though using it is not), refresh() (repopulates the index list answers from), and commit_deletes(). Consequence: delete is visible immediately but lands in OPFS at the end of the batch — it drops the name from the index and queues a removeEntry. Handles are exclusive per file, so attach closes any handle it replaces and detach releases the lock. storage_contract in storage.rs is the behaviour every backend owes; it runs against both host backends and is the written-down spec OpfsFs must satisfy in a browser, where the harness cannot reach.

29. "Which view is active" is stored twice, and the copies must move together. Workbook::active_view is the document's; ViewState::active_view is the client's, and it is what keys ViewState::cursors. The second copy exists because reduce_view runs on a client that holds no Workbook (improvise-cqi). Only two writers are allowed to touch either — App::switch_to_view and App::install_workbook — and the_two_active_view_names_never_diverge guards it. Calling workbook.switch_view() directly desyncs them and the cursor map is then read under the wrong key; that is exactly how the 8uz characterization test broke during the extraction. Whether the two are allowed to diverge — two clients on different views — is improvise-g0f, and the answer is yes, eventually: the model is shared, the UI is not, so Workbook::active_view becomes the document's default (what a fresh client opens) and ViewState::active_view is authoritative per client. improvise-jytm owns the split; a "follow mode" binding two UIs together is deliberately not ruled out. Until then Sessions::apply keeps them in lockstep by routing SwitchView through App::switch_to_view — see gotcha 36.

30. Two tests gate ViewState membership. ViewState is the portable session slice — the one thing serialized into the Snapshot a newly-connected client gets (improvise-q08). A field may live there only if it is portable (plain data from the allowlisted crates, no Rc/Arc) and snapshot-safe (a fresh client inheriting the value is correct). wizard fails both — ImportWizard is an improvise-io type, and a half-open dialog is not something to inherit — and drill_state fails both — it holds an Rc, and pending_edits are uncommitted writes belonging to whoever staged them. Both live on App under the session-local residue tag (improvise-h0b). The ViewState: Send static assertion in app.rs's tests is the compile-time proxy for portability: Rc is not Send, so a re-added shared pointer fails the build (the Deserialize derive from gotcha 25 now rejects it too, serde's rc feature being off). Send does not catch the origin-crate or snapshot-safety halves — check those by hand.

31. : commands settle their mode by observation, not prediction. ExecuteCommand ends every batch with ReturnToNormalUnlessMoved(ctx.mode), which drops to Normal only if the mode is still what the batch started in. There is no Effect::changes_mode() any more: it was a per-type constant, and ViewBack/ViewForward/StartImportWizard swap the mode only on some runs, so no up-front answer was right — :import opened the wizard and was snapped straight back out (improvise-toc). A new effect that changes the mode needs no annotation; just don't return the batch's starting mode from a success path.

32. There is no aggregate render cache — each widget folds its own stream. improvise-protocol::render holds one cache type per widget (GridCache, CategoryTreeCache, FormulaCache, ViewListCache, TileBarCache), each implementing WidgetCache with its own Update type, and a Projection is one such update addressed to one widget (improvise-rkeq). reduce_view folds only ViewState now — it no longer takes a cache, and Command::Project passes through it untouched. Two consequences. The overlap between caches is deliberate: the tile bar and the category panel both carry (category, axis) and are updated by separate projections, so neither is woken by the other's changes; do not "fix" it by sharing a variant. Bootstrap is a compaction, not a replay: Snapshot { view, seed: Vec<Projection> } seeds a new client with a batch of the same projections the steady stream carries, so there is no whole-state type that could disagree with the incremental one. The seed is the server's current state, not the head of the history — GridCache::seed() builds it, and the property that earns it is GridCache::folded(&cache.seed()) == *cache. Nothing is defined for the help overlay (its content is static and help_page is already in ViewState) or for the import wizard, which now has a shape and deliberately no projection: WizardCache is built by project::wizard_cache and drawn locally, because a half-open dialogue is addressed to precisely the session that opened it, never to every client (improvise-764 — the rendering-side twin of the argument that removed AcceptWizardModel from ModelCommand, gotcha 38). A cache without a Projection is the shape session-local residue takes when it still has to be drawn by two different front ends. It lives in improvise-protocol rather than improvise-command for the flatten_cat_tree reason: when the wasm client grows a wizard it must draw these rows, not a second derivation.

33. The grid's labels frame its cells, and apply enforces it — the fold is not commutative. Three consequences in improvise-protocol/src/render.rs. (a) A GridUpdate::Cells that overtakes the RowLabels/ColLabels naming its positions is dropped, because fit_to_labels truncates to the labels it has; a client must not reorder the stream, and a_cell_update_that_overtakes_its_labels_is_lost pins it. (b) A label change that renames a position blanks the cell under it: fit_to_labels preserves by POSITION, which is only right where the label did not change, so apply calls invalidate_relabelled itself rather than trusting a producer to send GridUpdate::Invalidated first. Invalidated therefore now means only "contents stale, frame unchanged". (c) every arm that touches labels or widths re-fits — col_widths is re-fitted with the columns — a new column is drawn at its label's own character count until ColWidths catches up — because a renderer zips widths with labels and a length mismatch indexes off the end. Fitting on the label arms alone is what let a late ColWidths put the mismatch straight back — a short update keeps each unheard-of column at its label's width, and a long one is dropped as having overtaken the frame that names it, exactly as (a) holds Cells. GridCache also carries totals and page_coords (improvise-n10). totals is framed by col_labels on the same terms as the cells and widths and re-fits with them, with one asymmetry: an empty strip means "this grid has no totals" (records mode, or an empty grid) and is therefore not padded, where a non-empty one is — padding it would draw a blank total row under a records grid that has none. Invalidated blanks the totals too: they are contents, and a stale sum under fresh cells is the one number a user takes at face value. page_coords is framed by nothing (it names categories, not positions) and survives a reshape. Note the fold tests: asserting fold(prefix)+fold(rest) == fold(whole) is vacuous (it is the definition of a left fold), so the tests that earn their place are the order-sensitivity one and the compaction-equivalence one — see doctrine(fold_tests_must_test_order_not_the_axiom).

34. A formula target is unquoted, and pipe quoting has one home. parse_formula strips the pipes off a target exactly as it does off an Expr::Ref — the target names a _Measure item, so a target that kept its delimiters made add_formula create an item literally named |Total Revenue|: a ghost column the user's real item never matched, and Expr::Ref("Total Revenue") never resolved to. Both the TUI formula minibuffer (AddFormula) and .improv load go through that one call, so both were affected. The writer emits Formula::raw, not target, so file text is unaffected either way — which is exactly why the round-trip tests never saw it, and why the fix repairs existing files on next load with no migration. Pipe quoting has exactly one home: improvise_formula::quote, owning both halves — quote_name/is_bare_name, pipe_quote (unconditional, for a .improv text cell that must not read back as a number) and unescape_pipe_body. improvise-gp re-exports it, improvise-io calls it from persistence/{format,parse}.rs, parser.rs::unquote_pipe is a shim over it, and io's own copy is deleted (improvise-i2m5). The keyword guard is canonical — user-confirmed, so do not revert it as over-caution: an item literally named SUM is written |SUM|. Bare keywords do parse, but only because agg_call needs a ( and PEG backtracking falls through to ref_expr — rule ordering, not a stated rule — and a ## Formulas line carries formula text verbatim, so a name format_md writes is read back by parse_formula: one namespace, one spelling rule. One thing the guard does not cover, and one that used to be an exception and no longer is. A section header takes its name as the whole line, totally (improvise-pczv): header_name = { pipe_quoted ~ &NEWLINE | line_name } — the pipe-quoted arm needs the lookahead, or |A|B would parse as A and silently drop B, and line_name catches everything else, so a header name can no longer fail to parse. Four rules share it (model_name, initial_view, category_section, view_section), which is why it is a rule and not four copies. Note a header is still written raw where a reference is quoted — the grammar accepts both arms, so the quoting asymmetry between a header and a reference is a writer convention, not a format one. And the two former unquoting copies differed only on a trailing lone backslash, unreachable from the grammar, where io returned it and parser.rs panicked — the surviving unescape_pipe_body is total.

35. Dirty over-reports on purpose, and its doc comment is the derivation. Dirty::of maps an EffectKind to the widgets that kind can reach, and the two directions do not cost the same: over-reporting pays one redundant Projection, under-reporting leaves a client's cache wrong forever, with nothing in the stream to correct it. LayoutGeometry therefore includes formulas, because RemoveCategory wears it and Model::remove_category ends by dropping every formula targeting the category — a client's formula panel otherwise kept showing a formula the document no longer had. It now coincides with Dirty::EVERYTHING and is still spelled out, because the two are derived from different effects and need not stay equal. The doc comment on Dirty::of is the only record of that derivation, and it went stale once already — claiming AddCategory was CellData after it was reclassified — so reclassifying an effect means re-reading it.

36. A client's command has one door, and the door takes the App. Sessions::apply(app, id, cmd) is the only way a connected client's Command reaches the server; ClientSession::fold and the mirror's ViewState are private, so a transport cannot reach the mirror at all. The door takes the authoritative App because some commands have a document half: SwitchView routes through apply_command/App::switch_to_view (gotcha 29), and every mirror then takes what the document accepted, not what was asked — so a switch to a view that does not exist leaves the mirrors put. Not just the asker's: grid_of builds every client's grid from workbook.active_view(), so a bystander left naming the outgoing view has its cursor read under a stale key and is projected the new view's rows positioned by the old view's scroll — gotcha 24's mismatch arriving through somebody else's keystroke. apply_switch therefore takes no SessionId; the switch is addressed to no one session and that absence is the statement. A client's own slice lives on the client and stays in step only by folding the same command, so Sessions::apply returns Applied { dirty, broadcast }, where broadcast is a Vec<Broadcast> and each item names its own audience (improvise-gvnb). The audience follows from whether the asker folded. Broadcast::ToOthers is a correction of something the asker already applied optimistically; Broadcast::ToAll is a consequence the asker is owed like everybody else. A Command::Model is the one kind a client cannot fold optimistically — it cannot predict what the document will accept — so every relay originating in apply_model is ToAll: the ResetCursor from adopt_cursor_epoch, and the SwitchView that ViewBack/ViewForward move. apply_switch is the exception, and only because its asker really did fold. Copying ToOthers from apply_switch into apply_model is the trap, and it is invisible from the bystander's side — adopt_active_view moves the asker's mirror either way, so the drift is between the server's picture of the asker and the asker's own slice, and a test asserting through a "what do the others hear" helper looks away from it by construction. (apply_switch has the mirror-image hole on its refused path, where the asker folded something the document turned down and is the one client not told: improvise-6c6q.) A Vec rather than an Option because one command can owe two relays with different audiences, and an Option would have had to drop one silently. Everything else is that client's alone and never touches the App's own view slice: the model is shared, the UI is not. Folding SwitchView into the mirror alone was the bug — viewport() then read the cursor under the new view's key and the next project() built that client a grid out of the other view.

37. gp-tree returns data and never prints, and a fit takes two things to reproduce. Two rules with one root — the crate is a library a TUI, a wasm worker or a CLI may each host. (a) A decision the user should hear about comes back as data: Dataset::warnings carries the columns preprocess dropped for being entirely missing (improvise-d44s), and fit.rs hands back pooled (prediction, actual) pairs and a score per feature rather than tables. examples/fit_csv.rs owns every println!, and that is the only reason its output stayed byte-identical across these moves — adding an eprintln! to either module is the regression. (b) The seed is necessary and not sufficient: data::Model records seed: Option<u64> and config: Option<FitConfig>, because the same seed under a different population/generations/elite/penalty lands on a different tree (improvise-ksst). Both are Option for one reason — every FitConfig value and the seed 0 are legal, so nothing can double as "unknown". Note #[serde(default)] on an Option field is documentation, not the mechanism: serde defaults it anyway, so deleting the attribute fails no test, and a backward-compatibility test that only deletes it is not discriminating. Aim such a test at an absent field loading as the default instead of as unknown. The GA loop itself is fit::fit (improvise-0ai); breed remains one fitness-agnostic generation for callers whose fitness is not "predict this column". (c) A scoring function takes one slice of rows. Row<T> pairs a feature vector with what it predicts, so the parallel x/y lengths that zip used to truncate silently cannot be built (improvise-dkqn, doctrine(an_invariant_is_made_unrepresentable_not_validated) — the same answer prune_axis_items got, and the reason no Result entered five signatures). The pairing is made at the boundary that builds the data: preprocess walks one row index and the rows live inside Target, because a Dataset carrying rows and target separately just moves the zip up one level into every caller. The empty row set is one shape, answered once in mean_over. Changing any of this needs bit-level evidence: tests/scoring_known_answers.rs pins every path as f32 bit patterns plus a next_u64 witness that catches a changed RNG call order, because the CLI prints three or four decimals and byte-identical output alone cannot see a moved accumulation. (d) A fit is storable only when the document can name its features. preprocess invents Region=North, Cost.missing, when.year; those are encoding artefacts, not items of the document, and Expr::Ref resolves through Model::find_item_category — so a tree splitting on one emitted a well-formed formula whose every cell was Error("ref:…"), silently (improvise-8xhy). fitted_formula therefore takes the &Model and refuses, listing what it could not resolve; formula_text still spells it, being the tree's own text. The check asks the document, never the name's shape, because a measure literally named Region=North is a legal document and the two are indistinguishable as strings — which also means .missing and the date parts are covered with no second copy of preprocess's expansion rules to drift from. This is an interim (improvise-4zl7): a WHERE predicate needs a scalar coordinate predicate the formula language does not have — Filter gates a formula or an aggregate, it is not an Expr — would still leave .missing and the date parts, and must first answer what such a predicate means where context.get(cat) is None, since Formula::filter falls through to aggregate_raw and a categorical fit would silently degrade in every view that aggregates over the category. The test that missed all this is the shape to remember: improvise_eval fabricates a _Measure item per feature name, so it asserted the emitter and gp_tree::eval agree against a model that could not exist. It is kept, with its scope now stated, and the property it could not state is asserted separately against a real document.

38. The wire vocabulary is derived twice, by the same recipe. Command's view half is the EffectKind::ViewOnly effects targeting the portable slice (decision(improvise-cqi, command_set_derived_not_invented)); its model half, ModelCommand, is the EffectKind::Model(..) effects — 29 of them, one variant apiece, minus SwitchView, which Command already names and routes (gotcha 36), leaving 28 (improvise-kmgs). Note the obvious awk extraction reports 30: it picks up HelpPageSet, which is ViewOnly, because that is the last pub struct before #[cfg(test)] and the test module mentions ModelChange:: — brace-match the impl Effect blocks instead (doctrine(derive_then_verify)). Three things follow. A ModelCommand does not mutate: ui/reduce.rs::model_effects maps it back to the effect it came from and apply_command runs that through App::apply_effects, so effects stay the only mutation site and the Dirty is the effect's own EffectKind, never a second classification beside gotcha 35's table. Two model effects are deliberately off the wire. ApplyAndClearDrill and AcceptWizardModel were once kept on the grounds that the command is portable even where the operand is not; that was the wrong criterion (improvise-3x3u). A Command::Model is routed with no SessionId, which is right exactly when the change is addressed to no one session — and these are addressed to precisely one, never the sender: their operands are drill_state.pending_edits and the built wizard workbook, session-local residue (gotcha 30) private to whoever staged them (improvise-tom1). Unaddressed, B's ApplyAndClearDrill commits A's uncommitted typing and B's AcceptWizardModel swaps the document out of a dialogue B never opened — both demonstrated against Sessions::apply before removal. So 29 − SwitchView − these two = 26. The effects are untouched and still reached by the key path. Not permanent: multi-client drill wants the addressed form, which is where ADR 001 layer 2 goes anyway. Guarded by no_model_command_commits_another_clients_session_local_state, plus a companion asserting both effects still exist and are still model changes so the first cannot pass by naming nothing. Sessions::apply routes the model half to apply_model, which owes the same lockstep as apply_switch: ViewBack, ViewForward and a document-replacing LoadModel all end in App::switch_to_view, so every mirror takes the accepted active view. Its broadcast is conditional where apply_switch's is not, because no client optimistically folded an active view here. A keymap lookup still cannot return a Command, and not for want of vocabulary: a binding names an intent resolved against CmdContext (move-cursor down) while a Command is the resolved mutation (SetSelected { row, col }) — improvise-gxi's AC 4 asks for something the split forbids.

39. Applied is an enum because a document swap is not a diff. Sessions::apply returns Applied::Diff { dirty, broadcast }, Applied::Rebased { snapshots } or — since ADR 001 layer 1 — Applied::Rejected { reason }, and the enum is the mechanism: a host that wants the Dirty must match, and matching names the case where a Snapshot is owed. The server's mirrors self-heal — ClientSession::project reseeds on the document_generation bump — but ViewState::cursors lives on the client, keyed by the outgoing document's view names (gotcha 24), and only a fresh Snapshot replaces it. Command::Model put the three WholeWorkbook mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. apply_model therefore calls rebase itself rather than instructing the host to, exactly as apply_switch adopts the accepted view itself. The trigger is the generation moving, not the command variant: a LoadModel that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a failed load — a_load_that_fails_is_not_a_rebase guards that arm. A rebase: bool beside dirty was rejected, because the host that forgets to call rebase is the host that forgets to read the flag. The third case is a normal outcome, not an error, and it is in the enum for the same reason the other two are: a host that wants the Dirty must match, and matching names it. ServerMsg::Rejected is the matching frame, addressed to the asker alone — nothing happened, so nobody else is owed anything — and the client keeps it as a Rejection rather than a sentence, because force needs to know which record and which kind (gotcha 43). Fixture warning. When testing a document swap, the incoming document's active view must have the same name as the outgoing one. A rename makes apply_model broadcast a SwitchView; the client folds it, and the stale cursor is then read under a key nothing has written — right by accident, with the bug invisible. other_document() renames, and a first version of this test passed because of it; a_document_shaped_differently_under_the_same_view_name() keeps the name and changes only the shape, so there is nothing to hide behind.

40. A batch's basis is three tiers, and only two of them can be read off the effect. App carries document_generation (bumped by install_workbook), layout_version (per ModelChange::LayoutGeometry effect) and record_versions (per record) — ADR 001's stale-basis check, and shared document state, so never on ViewState (gotcha 30). The unit is the record: record_of(key) is the coordinate tuple minus _Measure (record_row_identity), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. Both tiers are read in apply_effects off the same EffectKind the Dirty is: one classification, two readers, no second table to drift from gotcha 35's. layout_version bumps inside the loop for the reason Dirty accumulates there — an aborted batch must not claim effects it never ran (gotcha 21). record_versions cannot, because an EffectKind says that a batch writes cells, never which; so the records are observed, via a per-record XOR fingerprint of the raw data taken when the batch declares a CellData effect and diffed after it. Three consequences. A write landing the value already there bumps nothing, which is correct — no basis was invalidated. A LayoutGeometry effect that drops cells will bump their records when it shares a batch with a CellData one: an over-bump costing at most an unnecessary rejection, the same direction Dirty::of is deliberately sloppy in. And only raw data is fingerprinted — formula results live in Model::formula_cache, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a set: chaining before.keys() with after.keys() counts any rewritten record twice, a bug that survived one test run.

41. Storage is not a VFS, and that closes the SQLite question. The trait is whole-document read/write bytes (gotcha 2, invariant(storage_is_two_ops)) — no offsets, seeking, handles or locking — so an sqlite3_vfs, which needs xRead(offset,len)/xWrite/xTruncate/xLock, has nothing to bind to. SQLite can still sit behind it, as an in-memory database serialized to a blob (rusqlite serialize/deserialize_read_exact, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is 22% larger than .improv, 4.7× larger gzipped, ~160× larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. A second serialization format is a second implementation of the grammar, obliged to track improv.pest forever; and its own round-trip test cannot detect drift from .improv, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way export_sqlite beside export_csv, native-only and feature-gated: libsqlite3-sys' C build fails under this repo's nix cc-wrapper for wasm32-unknown-unknown, which would break the wasm CI gate the day it landed.

42. The browser client's render decisions are made in Rust, and the frames it speaks are nobody's but the protocol's. improvise-wasm-client/src/render.rs builds a RenderModel in which every question with a silent wrong answer is already answered, because the DOM half (web/app.js) is untested JavaScript and a wrong answer there shows up as a wrong pixel rather than a failing test. Three such answers. editable is a fact about the CellId variant — Record only, per ADR 002 — and the trap is the third case: None means "no projected cell", not "aggregate", so conflating them either hides a cell or offers a write the type system exists to make unconstructible. The selection is placed by reconciling the scroll offset, because GridCache is viewport-indexed while ViewState::selected is absolute. render.rs does it by adding — r + cursor.row_offset == sel_row — rather than subtracting, which is why there is no underflow path and why a selection above the window simply never matches. Omitting the offset entirely looks right until the first scroll, so the test that earns its place uses nonzero and unequal offsets: at selected=(5,1), row_offset=4, col_offset=1 it separates "reconciles" from "ignores" and from "swaps the two offsets". The category panel is viewmodel::flatten_cat_tree, never a second walk of CategoryTreeCache — that second walk was improvise-edp. The page is four string calls: Client::viewport (the handshake — the Snapshot seeds the visible cells, so one built at extent zero is an empty grid and a resize is a fan-out with nothing dirty), on_frame (one frame, one fold, arrival order — gotcha 33 holds inside a batch, so never coalesce), render_model, typed_cell.

    One envelope pair, two deployments (improvise-xks6). improvise_protocol::frame owns ClientFrame { Viewport, Command, Key } and ServerFrame { Snapshot(Box<_>), Commands(Vec<_>), Rejected }; improvise-ws-server and improvise-wasm-client each pub use them and neither defines one. There used to be three pairs for two deployments — the socket's server side, a hand-kept mirror of it in the browser bundle, and a third written for the worker (improvise-tm6) before the projection architecture landed. The question that had to be answered before any of it moved was whether the worker and the socket want the same vocabulary, and the answer is yes: every difference was a date rather than a decision, and the test applied to each was whether the argument for it names the thing they differ in. None did. Viewport is deployment-independent — the extent is not in ViewState and no effect produces it, so no derivation reaches it, and a worker hosting the same Sessions owes set_extent the same answer; it is the handshake on both because a Snapshot seeds the visible cells. Key is deployment-independent too — a binding names an intent resolved against a CmdContext a thin client does not hold (gotcha 38), which says nothing about what the bytes travel over; the socket lacked it because key forwarding was never built on it (improvise-bpq2), and Host::handle now answers ClientFrame::Key with an explicit empty vec that says so, where the frame used to just fail to parse in next_client_msg. And Commands(Vec<_>) batches where a singleton cannot, because a fan-out must arrive whole and in sequence (gotcha 33) — a property of the fold, not of the transport; an in-process postMessage interleaves exactly as a socket does. The worker pair was not merely different but incoherent by omission: written before ADR 001, it could carry Command::Model { basis, .. } and had no Rejected, so a client speaking it composed a claim that could never be answered. That incoherence — not the difference — is the tell for staleness.

    The mirror is gone, and so are the two tests that held it honest; that deletion was the ticket's measure of success. the_socket_framing_matches_the_server round-tripped hand-listed fixtures through the other crate's types to catch a rename; every_server_variant_is_one_this_crate_mirrors matched exhaustively to catch a variant added on the server and never mirrored. With one definition both are type identity: a rename is a compile error at every call site in both crates, and the un-mirrored variant is not a constructible state. That is stronger than either test, and it is the shape to insist on — if a consolidation leaves one of its compatibility tests alive, the consolidation is not finished. What survives is the_retired_spellings_do_not_parse in frame.rs, aimed at the one thing still possible (the retired Bootstrap/Command spellings returning as serde aliases, which would be two homes again wearing one type), and it keeps the injected-fault discipline this paragraph earned the hard way: assert the mutated input is still well-formed before feeding it in, and assert the error kind — serde's unknown variant — not merely that an error occurred, because a rewrite that also breaks the JSON makes the check pass against an enum that really does have the variant. End-to-end compatibility is now tests/browser_round_trip.rs, which hands the wasm client's own serialized frame to a real Host::handle; improvise-ws-server stays a dev-dependency of the browser bundle for exactly that.

    What no test in this repo can see: web/app.js has never been rendered. There is no browser and no headless DOM here; it is syntax-checked and every call it makes is confirmed present in the generated .d.ts with the right arity, and that is all. Click-to-edit, the ResizeObserver extent arithmetic and the CSS are unexercised. What a careful read did establish: the observer cannot feed back into itself (#grid is a flex child with overflow: auto, so replaceChildren cannot resize it), and ROW_PX/COL_PX under-estimate, so the worst case is a row of over-fetch. What it cannot establish, and is filed as improvise-hwpi: a malformed frame is quiet, not loud — on_frame returns a Result, wasm-bindgen throws it, and it is thrown inside an unguarded message listener, so it reaches the devtools console and nothing else; the socket has no error listener; close() has no re-entry guard; and no keydown is registered at all, so on_key_event and the whole keys/lockstep machinery are unreached from the browser. What is verified end to end is the Rust half: tests/browser_round_trip.rs drives improvise-1ey's stated scope against a real Host over a real App touching the client only through the four string calls. One deployment note that is not a bug: examples/demo.improv leaves Customer/Date on the None axis, so every cell is an aggregate and nothing is editable — correct behaviour, bad demo. Pick a model with no regular category on None.

43. A batch carries its basis, and the server had to publish it first. Command::Model { basis, cmd } is a struct variant so the basis cannot be forgotten, and Sessions::apply_model checks it before applying anything — a stale batch is never applied, not even partially (improvise-xwi6/fkfg, ADR 001 layer 1). Four things to know. Basis has no Default and no public constructor: a zero-valued basis is not "no claim", it is the claim that the document is fresh, which in a new App is true — so a default would be a forgeable bypass rather than an absence. BasisCache::claim is the only door a client has and it returns None until the server has published one. Note what that does and does not buy: Basis derives Deserialize because it must cross the wire, and its fields are public, so a hand-rolled client can state any tiers it likes. This is an anti-staleness coordination check, not a security boundary — RecordUnstated closes omission, not invention, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having: it makes the honest path the easy one. Which records are checked is the server's decision — reduce.rs::records_named, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the client's claim would be a force by omission: the record tier skipped, with no flag to remove and nothing in the message saying so. A claim that omits a record the command writes gets Rejection::RecordUnstated, a case of its own because "you lost a race" and "you were never in one" are different things to tell a user. Rejection keeps structural and data staleness apart, which is the whole reason ADR 002 put a maintained RecordHash index in DataStore: RecordGone means nothing resolves and there is nothing to force (force never creates structure), RecordMoved means it resolves at another version and force is one keystroke — which is also why the wasm client stores the Rejection itself rather than collapsing it into status_msg, since a string cannot carry which record or which kind. And the ADR omitted the publication. It said a batch carries the basis it was composed against and never said how a client learns one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. Projection::Basis + BasisCache close that — the server publishes the two document tiers plus the counters for the records this client's grid names, diffed against what that client was last told and seeded in the bootstrap — and it is emitted after the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown, the mirror image of gotcha 33. That ordering was documented as load-bearing in three places and held up only by the order of two statements until a_published_basis_follows_the_grid_updates_it_describes pinned it (proved by injecting the swap) — the same job a_cell_update_that_overtakes_its_labels_is_lost does for the mirror-image rule. Two things are unchecked and deliberately so: Command::SwitchView carries no basis (it sits outside Command::Model; the workbook refuses a view it does not have, so it cannot corrupt, but it is a hole in the rule), and apply_command/reduce_full do not check at all — the check is a property of the session door, not of the mutation path, which is right because a host holding the App is the truth rather than a claimant. But that second one is a door, not merely an exemption (improvise-ondj): both take a wire Command and its Command::Model { cmd, .. } arm discards the basis with .., and ui/reduce.rs's own module doc points every future host — including the browser worker — at exactly those functions. The struct variant made the basis impossible to omit from the message; it did nothing to make it impossible to ignore on arrival, and .. is the syntax for ignoring it with no compile error. That is also why ui::reduce::caught_up(&App, cmd) is not a bypass: it takes &App, so nothing reachable from a wire can call it.

44. The TUI grid draws from its cache, but the cache does not frame it. Since improvise-n10 GridWidget holds no &Model: cell text, aggregate-ness (CellId::Aggregate, ADR 002 — the widget used to walk none_cats × kind.is_regular(), a second hand-rolled copy of project::cell_id), column widths, the page bar and the totals strip all come off GridCache, and the TUI builds that cache over the whole grid (offsets zero), so a cache position is an absolute position — the browser's Viewport is a real window, the TUI's is not, and the two clients therefore use it differently. Column totals must be projected rather than client-derived, because the sum runs over every row including the ones a viewport scrolled past. Six facts still do not come off the cache: row_cats/col_cats (how many header levels), row_items (multi-level label tuples and inline group headers), col_group_for (column group spans), is_group_collapsed (▼/▶), and record_col — the last of which is correct to keep off, being session-local and what keys pending drill edits (gotcha 1). The other five are a gap, and a visible one: GridCache::row_labels is GridLayout::row_label, which slash-joins a multi-level tuple — the exact rendering two_row_categories_produce_cross_product_labels asserts must not appear — so the browser grid shows Food/Alice where the TUI shows two header columns, with no groups and no collapse indicators. Closing it changes the type of row_labels/col_labels, which improvise-wasm-client consumes, so it is a co-ordinated change, not an additive one; adding a row_levels beside row_labels is the wrong fix, because two answers to "what are the labels" is the disease (improvise-edp). Also still derived in the widget, and so a second rule waiting for a browser to invent: the row-header width (compute_col_widths moved to ui::viewport; its sibling did not). And project::grid_of now evaluates every cell per frame where the widget evaluated only the visible ones — a widening of an existing whole-grid cost, unbenchmarked.

---

File Inventory

Line counts are static; test counts are informational — run cargo test --workspace for live numbers. Files under 100 lines and render-only widgets omitted.

improvise-core (crates/improvise-core/src/)

model/types.rs         2185 / 76t   Model, formula eval (fixed-point), CRUD
model/cell.rs           650 / 28t   CellKey (sorted), CellValue, DataStore (interned + index)
model/category.rs       222 /  6t   Category, Item, Group, CategoryKind
model/symbol.rs          79 /  3t   Symbol interning
view/layout.rs         1174 / 25t   GridLayout, drill, records mode
view/types.rs           627 / 33t   View config (axes, pages, hidden, collapsed, format)
view/axis.rs            205 / 5t   Axis enum + label/color/next methods, AxisColor
workbook.rs             345 / 17t   Workbook: Model + cross-view ops, item facade
format.rs               229 / 29t   format_f64, parse_number_format (display only)

improvise-formula (crates/improvise-formula/src/)

parser.rs              1109 / 67t   pest grammar + tokenizer → Formula AST
quote.rs                115 /  5t   quote_name / is_bare_name: CL-style |…| quoting, the
                                    writing half of the grammar's bare_ident/pipe_quoted.
                                    improvise-gp re-exports it; improvise-io still has its
                                    own unguarded copy in persistence/quoting.rs
ast.rs                   77         Expr, BinOp, AggFunc, Formula, Filter

improvise-io (crates/improvise-io/src/)

storage.rs               288 / 12t   Storage trait (read/write/list/delete), PhysicalFs, MemoryFs
                                     injected into App as `storage`; effects use it, never std::fs
opfs.rs                  253 /  0t   OpfsFs: Storage over the browser's Origin Private File
                                     System. wasm32 only. Compile-checked, not browser-tested.
persistence/improv.pest 124         PEG grammar — single source of truth
persistence/mod.rs      259 /  7t   document lifecycle: save/load/gzip/legacy-JSON over Storage;
                                     facade re-exporting format_md/parse_md/export_csv*
persistence/format.rs   350 / 11t   format_md: Workbook → .improv text
persistence/parse.rs   1378 / 42t   parse_md: pest tree-walk → Workbook (+ grammar generator)
persistence/export.rs   156 /  3t   export_csv / render_csv
persistence/roundtrip_tests.rs
                        635 / 34t   format_md ↔ parse_md characterization (tests only)
import/wizard.rs       1121 / 38t   ImportPipeline + ImportWizard
import/analyzer.rs      292 /  9t   Field kind detection (Category/Measure/Time/Skip)
import/csv_parser.rs    300 /  8t   CSV parsing, multi-file merge

improvise-protocol (crates/improvise-protocol/src/)

lib.rs                   483 /  1t  ViewState, ViewCursor, ViewFrame, AppMode (16), Panel,
                                    MinibufferConfig, MinibufColor, HELP_PAGE_COUNT
reduce.rs                509 /  9t  Command + ModelCommand (the wire format), HelpPage, reduce_view
render.rs               1188 / 22t  CellDisplay, WidgetCache trait + one cache per widget
                                    (GridCache/CategoryTreeCache/FormulaCache/ViewListCache/
                                    TileBarCache), GridCache::seed (the bootstrap
                                    compaction), Projection, bootstrap-only Snapshot
key.rs                   101 /  3t  KeyCode, KeyModifiers, KeyEvent (moved from command/)
frame.rs                 215 /  3t  ClientFrame/ServerFrame — the ONE envelope pair, both
                                    deployments (improvise-xks6). Nothing else defines one
basis.rs                 279 /  5t  Basis (the three tiers), Rejection, BasisCache — ADR 001
                                    layer 1. Basis has no Default and no public constructor

gp-tree / improvise-gp (crates/gp-tree/, crates/improvise-gp/)

gp-tree/src/lib.rs          669 /  8t  Tree, Spec, eval, simplify, render, breed, Population,
                                       SplitMix64 (vendored seeded Rng; seed() keeps the seed so
                                       a fit can be shown and recorded, pinned to the published
                                       reference vectors)
gp-tree/src/data.rs         937 / 19t  raw string rows -> f32 feature vectors: per-column type
                                       inference, one-hot, mean-imputation + col.missing flag,
                                       ColumnSpec/encode_row, Dataset (incl. warnings),
                                       Model = trees + plan + seed + FitConfig
gp-tree/src/fit.rs          561 / 11t  the search: FitConfig (Default = the CLI's numbers), fit()
                                       -> simplified champion, cv_regress/cv_classify,
                                       permutation_importance, folds/standardize/mse.
                                       Returns data; never prints
gp-tree/examples/fit_csv.rs 601        CLI over the above: flags, demo dataset, all printing
improvise-gp/src/lib.rs     730 / 17t  formula_text / fitted_formula: a Tree as IF(...) formula
                                       text, validated back through parse_formula. A threshold is
                                       the SHORTEST decimal that reads back as the same f64 (not
                                       the exact expansion), so a row on a split boundary routes
                                       the same in f64 as in f32 eval; non-finite values panic

Command layer (crates/improvise-command/src/command/)

cmd/core.rs             319 /  2t   Cmd trait, CmdContext, CmdRegistry, parse helpers
cmd/registry.rs         758 /  3t   default_registry() — all registrations
cmd/navigation.rs       534 / 12t   Move, EnterAdvance, Page*
cmd/cell.rs             198 /  6t   ClearCell, YankCell, PasteCell, TransposeAxes, SaveCmd
cmd/commit.rs           517 /  9t   CommitAndAdvance, CommitBuffer(CommitKind), CommitExport
cmd/effect_cmds.rs      466 /  5t   effect_cmd! macro, 25+ simple wrappers
cmd/grid.rs             519 /  9t   ToggleGroup, ViewNavigate, DrillIntoCell, TogglePruneEmpty
cmd/mode.rs             394 / 11t   EnterMode, Quit, EditOrDrill, EnterTileSelect
cmd/panel.rs            587 / 13t   Panel toggle/cycle/cursor, formula/category/view panels
cmd/search.rs           202 /  4t   SearchNavigate, SearchOrCategoryAdd, ExitSearchMode
cmd/text_buffer.rs      240 /  7t   AppendChar, PopChar, CommandModeBackspace, ExecuteCommand
cmd/tile.rs             160 /  5t   MoveTileCursor, TileAxisOp
cmd/wizard.rs           199 /  4t   HandleWizardKey — per-step import-wizard key tables
keymap.rs              1747 / 33t   KeyPattern, Binding, Keymap, TextEntry, ModeKey, 16 mode keymaps
parse.rs                236 / 19t   Script/command-line parser (prefix syntax)

Semantic UI state (crates/improvise-command/src/ui/)

ui/effect.rs           2644 / 63t   Effect trait, 50+ effect types
ui/app.rs              2528 / 78t   App, ModelState, ViewState, AppMode (16) + dispatch methods, Panel, handle_key, dispatch_key, document_generation, autosave, HELP_PAGE_COUNT
ui/project.rs          3083 / 46t   Dirty, Viewport, grid_of/category_rows/formula_texts/
                                    view_summaries/tile_entries, grid_updates (the diff),
                                    ClientSession + Sessions (connect/apply/rebase/project_all)
ui/reduce.rs            942 / 15t   apply_command / model_effects / reduce_full / reduce_key — one door
                                    from a Command or a keypress to the authoritative App
ui/cat_tree.rs          156 /  6t   Category tree flattener for panel
ui/viewport.rs          128         compute_col_widths / row_header_width / visible_cols (pure geometry)

TUI rendering, draw, main (src/ui/, src/draw.rs, src/main.rs)

ui/grid.rs             1075 / 17t   GridWidget (ratatui); geometry helpers live in improvise-command viewport.rs
ui/help.rs              603         5-page help overlay (render only)
ui/import_wizard_ui.rs  695 / 3t         Import wizard rendering
draw.rs                 422         TUI event loop, frame composition
main.rs                 388         CLI entry (clap): open, import, cmd, script
# other ui/*.rs are small panel renderers — skip unless changing layout/style

Browser host and client (crates/improvise-ws-server/src/, crates/improvise-wasm-client/src/, web/)

ws-server/registry.rs    698 / 14t  Registry of Hosts keyed by DocumentId; ClientId {document,
                                    session} — a SessionId is unique only within its document.
                                    A document is never evicted while the server runs (ADR 001)
ws-server/host.rs        631 / 13t  Host: one App + one Sessions; apply, project_all, rebase
ws-server/transport.rs   286 /  1t  tokio + tungstenite plumbing
ws-server/main.rs        171 /  4t  CLI entry: serve a file on a port
wasm-client/lib.rs       477 /  9t  Client: the wasm-bindgen surface (init, on_message, on_frame,
                                    viewport, render_model, set_cell_at, typed_cell)
wasm-client/keys.rs      459 / 12t  BrowserKey → KeyOutcome (Local | Upstream | Ignored)
wasm-client/render.rs    388 /  7t  RenderModel: editable / selected / cat tree, decided here
wasm-client/socket.rs     366 /  7t  the fold (fold_frame/handle_frame), the two compositions
                                    and typed() text→CellValue. Defines no enums
wasm-client/state.rs     269 /  5t  ClientState: ViewState + one cache per widget, the fold
wasm-client/tests/browser_round_trip.rs  238 / 3t  improvise-1ey's scope against a real Host
wasm-client/tests/lockstep.rs            200 / 5t  host and client agree on a keypress
web/app.js               167         four string calls, no protocol knowledge, no framework.
                                     NEVER RENDERED — see gotcha 42
web/index.html, style.css            the page; legibility only
web/build.sh                         wasm-pack build → web/pkg (gitignored). Needs
                                     wasm-opt = ["-O", "--enable-bulk-memory"] or the bundled
                                     wasm-opt rejects LLVM's memory.copy

Examples

examples/gen-grammar.rs             Grammar-walking random .improv generator (pest_meta)
examples/pretty-print.rs            Parse stdin, print formatted .improv

Context docs

.claude/skills/code-review/         Review skill: checks a change against the doctrine in these docs + kb.pl
scripts/nix-seed-flake-inputs.sh    Seeds locked flake inputs over git so `nix develop` works in cloud sessions
prolog-sessions/kb.pl               Executable knowledge base: invariants, doctrine, decisions (CI-checked)
context/adr/003-live-backed-models.md A model may be backed by a live query source —
                                    improvise as a DB client. Accepted, NOT a priority.
context/adr/002-naming-a-cell.md     How a client names a cell: an opaque surrogate on
                                    CellDisplay, whose absence means the cell is an
                                    aggregate. Records why _Index cannot serve.
context/adr/001-concurrent-edits.md  Concurrent edits between clients: record-level atomicity,
                                    optimistic basis check per batch, advisory reservations.
                                    Records why a causal clock was rejected.
context/cloud-agents.md             Cloud-session environment: tracker sync, bd pitfalls, what is blocked
context/design-principles.md        Architectural principles & testing doctrine
context/plan.md                     Show HN launch plan
context/repo-map.md                 This file
docs/design-notes.md                Product vision & non-goals

---

Wasm

The browser bundle is improvise-core, improvise-formula, improvise-io, improvise-protocol, improvise-command and improvise-wasm-client. All six cargo check --target wasm32-unknown-unknown clean with no feature flags (improvise-ywd), gated by the wasm job in .github/workflows/ci.yml. improvise-ws-server is the host and is native only — tokio, tungstenite and Box<dyn Storage> — which is why improvise-wasm-client mirrors its framing instead of depending on it, and takes it as a dev-dependency so the mirror is compatibility-tested (gotcha 42). The root improvise crate is the terminal binary and does not build for wasm — it depends on crossterm; that is correct, not a gap.

cargo check passing is not the same as running. Two known runtime hazards it cannot see: Instant::now() panics on wasm (hence web-time in improvise-command), and any std::fs call compiles but fails in a browser — see improvise-fuj for the two effects that still bypass Storage.

CLI

improvise [model.improv]                          # open TUI (default)
improvise import data.csv [--no-wizard] [-o out]  # import CSV/JSON
improvise cmd 'add-cat Region' -f model.improv    # headless command(s)
improvise script setup.txt -f model.improv        # run script file

Import flags: --category, --measure, --time, --skip, --extract, --axis, --formula, --name.

---

Testing — the short version

Full guidance lives in context/design-principles.md §6. Quick reminders:

• Suite runs in <2s. Don't let that drift.
• Commands: build a CmdContext, call execute, assert on returned effects. No terminal needed.
• Property tests (proptest, default 256 cases) cover invariants: CellKey sort, axis consistency, save/load roundtrip, parse(format(parse(generate()))) stability.
• Bug-fix workflow: write a failing test before the fix (regression guard). Document the bug in the test's doc-comment (see model/types.rs::formula_tests).
• Coverage target ~80% line/branch on logic code; skip ratatui render paths.

---

Key dependencies

ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest + pest_derive, flate2 (gzip), csv, enum_dispatch. Dev: proptest, tempfile, pest_meta.