improvise/context/repo-map.md

# Repository Map (LLM Reference)

Terminal pivot-table app. Rust 2024, Ratatui TUI, command/effect architecture.
Apache-2.0. Root binary+lib `improvise`; sub-crates under `crates/`:

- `improvise-core` — `Model`, `View`, `Workbook`, number formatting. No UI/IO.
- `improvise-formula` — formula parser, AST, `parse_formula`.
- `improvise-io` — `.improv` save/load, CSV/JSON import, `Storage` backends. No UI/commands.
- `improvise-protocol` — the wire vocabulary and the portable session slice: `Command`, `Projection`, `ViewState`, the per-widget caches, `reduce_view`, key types, and `delivery/{wire,notice,exchange,client}.rs` (notices, watermarks, corrections). Depends on `improvise-core`, `zset`, `numtower` + serde. No IO, no command layer, no terminal.
- `improvise-command` — command/effect layer + semantic `App` state (app.rs, effect.rs, cat_tree.rs, viewport.rs, delivery.rs). Depends on `improvise-protocol`, `improvise-gp`, `zset`, `improvise-incremental`. No ratatui/crossterm.
- `numtower` — exact decimal arithmetic tower (`Num`, canonical text), ADR 005. Pure Rust, no workspace deps; `improvise-protocol` uses it for the wire form. Long form `context/code/numtower.md`.
- `zset` — std-only Z-sets (non-zero checked `i64` weights, panic on overflow) plus `Circuit`, operators, `Trace`, `Rebuild` (ADR 009 clause 1). No dependencies. `context/code/zset.md`.
- `zset-ffi` — C ABI over `zset` (`cdylib`/`staticlib`/`rlib`) for CFFI; `include/zset_ffi.h`. Native only, **not** in `wasmCrates`. `context/code/zset-ffi.md`.
- `improvise-incremental` — the `Derived` trait, `Reading`, `Eager`, `differential_in`/`drawn_probes`, and `cell_delta`/`DeltaValue`/`moved_records` (the store's `CellChange` log to a `ZSet`). Depends on `improvise-core` + `zset`; `core` never depends on `zset`. `context/code/improvise-incremental.md`.
- `gp-tree` — problem-agnostic genetic-programming regression trees (`Tree`, `Spec`, `breed`, `simplify`, `render`). Knows nothing about improvise.
- `improvise-gp` — the one seam between the two: emits a fitted `gp_tree::Tree` as improvise formula text (`formula_text` / `fitted_formula`). Depends on `gp-tree`, `improvise-formula` **and `improvise-core`** — `dataset.rs` builds a feature matrix from a `View`'s records (improvise-4s8) and `fitted_formula` takes the `&Model` it must resolve against (improvise-8xhy). Nothing depends on it yet.
- `improvise-ws-server` — the host: tokio + tungstenite, a `Registry` of `Host`s keyed by `DocumentId` (multi-document, ADR 001), `Sessions` fan-out, and the socket framing (`wire.rs`). Native only.
- `improvise-worker-server` — the *other* host: the same `App`, `Sessions` and `Host` as the socket server, behind `self.onmessage`/`self.postMessage` with OPFS underneath. wasm32 only in anger; `worker.rs`/`vfs.rs` are ordinary Rust and are where every decision and every test is. Nothing depends on it.
- `improvise-wasm-client` — the browser client: the fold, the key classifier, the socket framing it mirrors (`socket.rs`), and the render model the DOM paints (`render.rs`). Non-dev deps are `improvise-protocol` + serde + wasm-bindgen and nothing else. The page itself is `web/`.

The main crate (and `improvise-command`) re-export the sub-crates as `crate::{model, view, workbook, format, formula, persistence, import}` so consumer paths stay stable when crates shuffle. The main crate also re-exports `improvise_command::command` as `crate::command` and `improvise_command::ui::{app, cat_tree, effect, viewport}` under `crate::ui`. `ui::app` re-exports the protocol's `AppMode`, `ViewState`, `ViewCursor`, `ViewFrame`, `Panel`, `MinibufferConfig`, `MinibufColor` and `HELP_PAGE_COUNT`, and `command::key` re-exports the protocol's key types, so every pre-existing path still resolves.

Architectural intent lives in `context/design-principles.md` — read that for the "why". This doc is the "where".

---

## How to Find Things

| I need to...                        | Look in                                         |
|-------------------------------------|-------------------------------------------------|
| Add a keybinding                    | `improvise-protocol/src/keymap.rs` → `Keymaps::default_keymaps()` (the table; the browser looks up the same bindings). Resolving a bound *name* into effects is `improvise-command/src/command/keymap.rs`. |
| Add a user command                  | `command/cmd/<submodule>.rs`, register in `registry.rs` |
| Add a state mutation                | `ui/effect.rs` → implement `Effect`             |
| Change formula eval                 | `model/types.rs` → `eval_formula` / `eval_expr` |
| Change cell storage / lookup        | `model/cell.rs` → `DataStore`                   |
| Change category/item behavior       | `model/category.rs` → `Category`                |
| Change view axis logic              | `view/types.rs` → `View`                        |
| Change an axis label or cycle | `view/axis.rs` → `impl Axis`. An axis's **colour** is `Face::Axis(axis)` in `improvise-protocol/src/face.rs`, resolved by the `Theme` and turned into a style by `draw.rs::face_style` (ADR 004) |
| Ask "what data is visible" vs "what can the user page" | `View::filtering_cats()` vs `categories_on(Axis::Page)` |
| Change grid layout                  | `view/layout.rs` → `GridLayout`                 |
| Change `.improv` format             | `persistence/improv.pest` + `persistence/{format,parse}.rs` |
| Change where documents are stored   | `storage.rs` → `Storage` impls (`opfs.rs` for wasm) |
| Change number display               | `format.rs` → `format_f64`                      |
| Change CLI args                     | `main.rs` (clap)                                |
| Change import logic                 | `import/wizard.rs` → `ImportPipeline`           |
| Change how a gp-tree fit searches   | `gp-tree/src/fit.rs` → `FitConfig` / `fit`      |
| Change gp-tree CSV preprocessing    | `gp-tree/src/data.rs` → `preprocess`            |
| Change how a name is quoted anywhere | `improvise-formula/src/quote.rs` — the one home, both halves |
| Change import-wizard keys/steps      | `command/cmd/wizard.rs` (keys) + `import/wizard.rs` (steps) |
| Change frame layout                 | `draw.rs` → `draw()`                            |
| Change an incremental operator or the circuit | `crates/zset/src/{ops,circuit}.rs`    |
| Change the C ABI                    | `crates/zset-ffi/src/lib.rs` + `crates/zset-ffi/include/zset_ffi.h` |
| Change the wire form of a value or delta | `improvise-protocol/src/delivery/wire.rs`  |
| Change what the host publishes/answers | `improvise-command/src/ui/delivery.rs`      |
| Change what counts as a document or record write | `improvise-command/src/ui/app.rs::apply_effects` (reads the batch cell delta) |
| Ask "does this grid draw column totals" | `GridLayout::draws_totals` (`view/layout.rs`) — the one condition |
| Change how wide the TUI grid is beside a side panel | `improvise-command/src/ui/viewport.rs` → `grid_width` (reached through `App::grid_width`); `draw.rs::draw_content` splits on it and `App::cmd_context` counts `visible_cols` on it (improvise-gtn7.1) |
| Change what the browser page draws   | `improvise-wasm-client/src/render.rs` → `RenderModel` |
| Change a browser↔server frame        | `improvise-protocol/src/frame.rs` → `ClientFrame`/`ServerFrame`. The **one** pair; both ends `pub use` it and neither defines one (improvise-xks6) |
| Change the worker's message loop     | `improvise-worker-server/src/worker.rs` → `Worker::on_frame` |
| Change the session loop (either host) | `improvise-command/src/ui/host.rs` → `Host::handle` |
| Change how the browser opens or seeds a document | `improvise-worker-server/src/vfs.rs` → `open` / `documents` |
| Change what a browser resolves without the host | `improvise-protocol/src/resolve.rs` → `resolve`. **Not** `wasm-client/src/keys.rs`, which holds only `BrowserKey` (`KeyboardEvent` → improvise key) — this row named it until 2026-09-21 and that pointer is how a false claim reached `context/` (improvise-e5pf) |
| Change the browser page itself       | `web/app.js` (thin), `web/index.html`, `web/style.css` |
| Host another document                | `improvise-ws-server/src/registry.rs` → `Registry`, `ClientId` |
| Change app state / modes            | `ui/app.rs` → `App`, `AppMode`                  |
| Write a test for model logic        | `model/types.rs` → `mod tests` / `formula_tests` |
| Write a test for a command          | `command/cmd/<module>.rs` → colocated `mod tests`; helpers in `cmd/mod.rs::test_helpers` |

---

## Central Pattern: Cmd → Effect

```
keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
                            (pure, read-only ctx)       (list of mutations)    (only mutation site)
```

```rust
pub trait Cmd: Debug + Send + Sync {
    fn name(&self) -> &'static str;
    fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
}

pub trait Effect: Debug {
    fn apply(&self, app: &mut App);
    fn kind(&self) -> EffectKind;              // NO default — the compiler makes you classify
}

// EffectKind::{ViewOnly, Model(ModelChange)}
// ModelChange::{CellData, LayoutGeometry, WholeWorkbook}
```

`CmdContext` (see `command/cmd/core.rs`) holds immutable refs to `Model`, `Workbook`, `View`, `GridLayout`, the `CmdRegistry`, `AppMode`, cursor/offsets, buffers, yanked cell, key code, expanded cats, panel/tile cursors, the opened editor, the view back/forward stacks, `visible_rows`/`visible_cols` and the cursor cell's `display_value`.

**Add a command**: implement `Cmd` in a `command/cmd/` submodule, register in `registry.rs`, bind in `default_keymaps()`. Simple wrappers go through the `effect_cmd!` macro in `effect_cmds.rs`.

**Add an effect**: implement `Effect` in `ui/effect.rs`, add a constructor fn if it helps composition.

---

## Key Types (skim; read the source for fields/methods)

**Model** (`model/types.rs`): `categories: IndexMap<String, Category>`, `data: DataStore`, `formulas: Vec<Formula>`, `views: IndexMap<String, View>`, `active_view`, `measure_agg`. `MAX_CATEGORIES = 24` (regular only) — `pub`, re-exported as `improvise_core::model::MAX_CATEGORIES`, and the one home: the import wizard reads it rather than keeping a copy (improvise-felo.1). Virtual categories `_Index`, `_Dim`, `_Measure` always exist. Use `regular_category_names()` for user-facing pickers.

**CellKey** (`model/cell.rs`): `Vec<(cat, item)>` — **always sorted by category**. Build with `CellKey::new(coords)` / `with(cat, item)` / `without(cat)`. Never construct the inner `Vec` directly.

**CellValue**: `Number(f64) | Text(String) | Error(String)`. Errors surface from formula eval (circular, div/0, etc.).

**DataStore** (`model/cell.rs`): interned symbols + secondary index `(Symbol, Symbol) → {InternedKey}`. Hot-path lookups go through `matching_values` / `matching_cells`.

**Formula AST** (`improvise-formula/src/ast.rs`):
- `Expr { Number, Ref, BinOp, UnaryMinus, Agg, If }`
- `BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }`
- `AggFunc { Sum, Avg, Min, Max, Count }`
- `Formula { raw, target, target_category, expr, filter }`

**View** (`view/types.rs`): `category_axes: IndexMap<cat, Axis>`, page selections, hidden items, collapsed groups, `number_format`, `prune_empty`. It carries **no cursor** — `selected`/`row_offset`/`col_offset` moved to `ViewState::cursors` (improvise-8uz). `Axis { Row, Column, Page, Filter, None }`. `cycle_axis` rotates Row → Column → Page → None; `Filter` is outside the rotation (see gotcha 27).

**GridLayout** (`view/layout.rs`): pure function of `Model + View`. `cell_key(r,c)`, `cell_value(r,c)`, `drill_records(r,c)`. **Records mode** auto-detects when `_Index` is on Row and `_Dim` is on Column.

**AppMode** (`ui/app.rs`): 17 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, SavePrompt, CommandMode, ImportWizard, Help, Quit, RecordsNormal, RecordsEditing). Eight text-entry variants carry a `MinibufferConfig`. `SearchMode` is Normal + `search_mode: bool`, not its own variant. Per-variant UI facts live on the enum as exhaustive-match methods — `label()`, `hint_text()`, `panel()`, `minibuffer()`, `is_editing()`, `is_records()` (improvise-2hi); add a variant and the compiler names every one you must fill in. A mode's colour is a `Face` (`Face::Mode*`, `Face::of_minibuffer`), turned into a ratatui `Style` only by `src/draw.rs::face_style`. `App` state is split into `ModelState` (workbook, file_path, dirty) and `ViewState` (mode, cursor/offsets, buffers, panels, search) — see improvise-x2c/ew0. `wizard` and `drill_state` live on `App`, not `ViewState` (improvise-h0b) — see gotcha 24.

**Keymap** (`improvise-protocol/src/keymap.rs`; `dispatch` and `KeymapSet` in `improvise-command/src/command/keymap.rs`): `Binding { Cmd | Prefix(Arc<Keymap>) | Sequence(Vec<…>) }`. Lookup fallback: `exact(key,mods) → Char(NONE) → AnyChar → Any → parent`. 17 mode keymaps, one per `ModeKey`, built by `Keymaps::default_keymaps()`; mode resolved via `ModeKey::from_app_mode()`. `ModeKey` and `AppMode` are two different enumerations that happen to be the same size — `SearchMode` is a `ModeKey` and not an `AppMode`; `Quit` is an `AppMode` with no keymap — so do not "fix" one count to match the other.

---

## `.improv` File Format

Plain-text, markdown-like, defined by `persistence/improv.pest`. Parsed by pest — the grammar is the single source of truth (the grammar-walking test generator reads it via `pest_meta`).

```
v2025-04-09
# Model Name
Initial View: Default

## View: Default
Region: row
Measure: column
|Time Period|: page, Q1
hidden: Region/Internal
collapsed: |Time Period|/|2024|
format: ,.2f

## Formulas
- Profit = Revenue - Cost               # defaults to [_Measure]
- Tax = Revenue * 0.1 [CustomCat]

## Category: Region
- North, South, East, West              # bare items, comma-separated
- Coastal_East[Coastal]                 # grouped item, one per line
> Coastal                               # group definition

## Data
Region=East, Measure=Revenue = 1200
Region=West, Measure=Revenue = |pending|   # pipe-quoted text
```

- **Name quoting**: bare `[A-Za-z_][A-Za-z0-9_-]*`, else CL-style `|…|` with escapes `\|`, `\\`, `\n`. Same convention in the formula tokenizer.
- **Write order**: Views → Formulas → Categories → Data. Parser tolerates any order.
- **Gzip**: `.improv.gz` (same text, gzipped).
- **Legacy JSON**: auto-detected by leading `{`; never written.
- **Axis keywords**: `row`, `column`, `page`, `filter`, `none`; `page`/`filter` carry the selected item after a comma.
- **Virtual categories**: `_Index`/`_Dim` never persist; `_Measure` persists only non-formula items (formula targets are rebuilt from the `## Formulas` section).
- **Virtual views**: `_Records`/`_Drill` (`view::VIRTUAL_VIEW_NAMES`) are never written and are dropped on load if an older file carries them; saving from records/drill mode writes the first real view as `Initial View`. Their commands rebuild them on demand.

---

## Gotchas (digest — read before editing)

One or two lines each: the rule, and the trap. **The numbers are stable and cited across the repo; a
citation of `gotcha N` means entry N in `context/gotchas.md`, which holds the full argument,
the incidents and the ticket ids.** Read the long form of every gotcha your diff touches. Counts and
"closed/open" claims in the long form are dated snapshots.

1. **Records mode is wide format.** A row is a coordinate tuple; columns are `[coords…, +cat, raw measures…, +]`. Editing a category column moves every measure cell in the row. `_Dim` carries a column *index*, resolved by `GridLayout::record_col`. `GridLayout::new` does **not** prune; `with_frozen_records(m, v, None)` does — production code wants the latter. `prune_axis_items` takes a predicate and re-marks survivors.
2. **Effects never touch `std::fs`.** Documents go through `App::storage: Box<dyn Storage>`. Autosave and the headless final write still bypass it (improvise-31oe.11).
3. **Every effect declares an `EffectKind`; there is no default.** `ViewOnly` never reaches other clients; `Model(CellData | LayoutGeometry | WholeWorkbook)` does. The view lives inside the workbook, so axis/page/format are `LayoutGeometry` and cursor/scroll/`SwitchView` are `ViewOnly`; `Add/RemoveCategory` are `LayoutGeometry`. `EffectKind` is too coarse to check the `ModelCommand` mapping — `every_model_command_names_the_effect_of_the_same_name` does.
4. **Commands never mutate.** `&CmdContext` is read-only; return effects.
5. **`CellKey` is always sorted.** Use the constructors.
6. **No `Model::add_item`.** Use the `Workbook` facade (`wb.add_item`) or go through the category.
7. **Virtual categories.** `_Measure` items = explicit ∪ formula targets (`measure_item_names()`). Cell aggregation (`apply_agg`) and formula `SUM/AVG` (`eval_agg`) differ on empty input; non-numeric values reduce by unanimity, else `Error("mixed")` — never dropped. `evaluate_aggregated` takes `Aggregation::{Exact, OverHidden}`.
8. **Display rounding is view-only.** `format_f64` only in rendering; never feed a rounded value back into eval.
9. **Formula eval is fixed-point.** `recompute_formulas` iterates to stability; cycles become `Error("circular")`.
10. **Drilling a formula cell** strips the `_Measure` coordinate so the raw feeding records come back.
11. **Minibuffer keymaps come from `TextEntry`** (eight text-entry modes); the buffer a prompt names and the keys edit are one string by construction. Read/write a runtime-named buffer with `read_buffer`/`write_buffer`. `SearchMode` is hand-written.
12. **Panel keymaps come from `PanelKeymap`**; `Panel::ALL`/`name()`/`key()` drive the toggle keys.
13. **A prefix key is stored as an address** (`ViewState::transient_prefix`), resolved by `Keymaps::prefix_keymap`; prefixes must stay flat (`prefix_keymaps_are_flat`). `ViewState` stays free of `improvise-command` types.
14. **Keybindings are per-mode** (`ModeKey::from_app_mode`); minibuffer Enter/Esc are `Binding::Sequence`. `every_hint_key_resolves_in_its_mode` checks which keys a hint names, not what they do.
15. **Registry names come from prototypes, not strings** — except the `Move` literals and four aliases, guarded by tests; a wrong-variant prototype shows up as a *duplicate* registration.
16. **`effect_cmd!` / `simple_effect!`** wrap one effect; `simple_effect!`'s kind argument is mandatory. Six wrappers return `unless_unknown(...)` (gotcha 51). Mode entry is `enter-mode <name>`; no per-mode shim commands.
17. **One float epsilon** (`FLOAT_EQ_EPSILON = 1e-10`) for `=`/`!=` *and* the div/0 guard, in both evaluators.
18. **Order is data.** `IndexMap` order is persisted; an index of `HashSet`s is for lookup, never iteration (`DataStore::matching_*` sort to store order; `analyze_records` is first-seen order). A group's fold reads through the same rule (`spanned_positions`).
19. **Cell-edit commit paths split** (`commit_regular_cell_value` / `commit_plain_records_edit`); keep both in sync. `cmd::core::uneditable` is the one decline, asked at open *and* commit. `CellId::NonItem` is a stored value naming no item.
20. **Minibuffer commits go through `commit-buffer <formula|category|item>`**; the add effect is emitted last so its failure message wins.
21. **An abort stops the chain, never the bookkeeping.** `Save`/`SaveAs`/`AddCategory` set `abort_effects`; tidy-up (mode restore, buffer clear) goes at the *head* of a batch. An abort is not a rollback (gotcha 47).
22. **The import wizard's back path is a history stack.** Preconditions live in `validation_error()`; `ResolveDuplicates` appears only when `collisions()` is non-empty; headless defaults to `DuplicatePolicy::Refuse`. A collision is a cell written by two records.
23. **`app.layout` goes stale mid-batch** — put a `RebuildLayout` effect before any effect that reads the layout after a grid change. The rebuild is skipped when nothing it reads moved (felo.13); a writer bypassing effects must call `rebuild_layout()`.
24. **The cursor is per-client, keyed by view** (`ViewState::cursors`; use `App::cursor()/cursor_mut()`). Replacing the document clears the map and bumps `document_generation`. Cursor invalidation is *observed* from the grid frame (`cursor_epoch`), not called by effects; two arms (`LayoutGeometry` hashes the active view, `CellData` every view).
25. **`ViewState` and its field types derive `Serialize`/`Deserialize`** — no borrowed-`'static` fields anywhere it reaches.
26. **Axis presentation lives on `Axis`; colour is a `Face`** resolved by the `Theme`; `draw.rs::face_style` is the one translation to ratatui.
27. **`Axis::Filter` constrains without being navigable.** Visible data = `filtering_cats()`; user-actionable = `categories_on(Page)` / `tileable_cats()`. What a virtual view must be is model-side (`records_steps`, `drill_steps`).
28. **The browser backend is sync on purpose, with an async seam** (`OpfsFs`; `open/attach/refresh/commit_deletes`). It needs a *dedicated* worker — `createSyncAccessHandle` is absent in a `SharedWorker`.
29. **"Active view" is stored twice and may differ:** `Workbook::active_view` is the default, `ViewState::active_view` is authoritative per client. `App::resolve_view` is the one fallback; `Sessions::adopt_deleted_views` handles a deleted view.
30. **Two tests gate `ViewState` membership:** portable (no `Rc`/`Arc`) and snapshot-safe. `wizard` and `drill_state` live on `App` as session-local residue.
31. **`:` commands settle their mode by observation** (`ReturnToNormalUnlessMoved`), not by a per-effect constant.
32. **There is no aggregate render cache.** One cache per widget (`WidgetCache`), a `Projection` is one update to one widget; bootstrap is a *compaction* (`Snapshot { view, seed }`). The wizard has a cache but deliberately no projection.
33. **The grid's labels frame its cells; the fold is not commutative.** A `Cells` update that overtakes its frame is dropped; `apply` invalidates relabelled positions itself; every arm touching labels or widths re-fits. Don't coalesce a batch.
34. **A formula target is unquoted; pipe quoting has one home** (`improvise_formula::quote`). The keyword guard is canonical. `header_name` is total; duplicate `## Category:` sections are one category; `WHERE` needs `sp+`.
35. **`Dirty` over-reports on purpose** — a redundant projection is cheap, a missed one leaves a client wrong forever. `LayoutGeometry` includes `formulas`.
36. **A client's command has one door:** `Sessions::apply_checked(app, id, Checked)`. Assert the `Broadcast` itself, and seat a bystander where a wrong relay can move it.
37. **`gp-tree` returns data and never prints; a fit needs seed *and* `FitConfig`.** A scorer takes `Row<T>` slices. `fitted_formula` refuses features the document cannot name; categories are forced `Categorical` (zv8t).
38. **The wire vocabulary is derived twice** (`Command` view half, `ModelCommand` model half). Completeness is checked (`vocabulary_is_complete.rs`). `ApplyAndClearDrill` and `AcceptWizardModel` are deliberately off the wire (session-local operands). Recount, don't cite.
39. **`Applied` is an enum (`Diff | Rebased | Rejected`)** because a document swap is not a diff; the trigger is the generation moving, not the variant. A swap fixture must keep the view *name*.
40. **A batch's basis is three tiers** (`document_generation`, `layout_version`, `record_versions`); the unit is the record; lower tiers are *observed*, not read off the effect. The record tier is the consolidated batch cell delta (`cell_delta` over the store's change log, `moved_records`); `record_fingerprints` is gone.
41. **`Storage` is not a VFS;** SQLite was declined on measurement (a second serialization is a second grammar).
42. **Browser render decisions are made in Rust** (`RenderModel`: `editable` from the `CellId` variant, selection reconciled with offsets, `flatten_cat_tree`). `improvise_protocol::frame` is the one envelope pair for both deployments.
43. **A batch carries its basis, and the server publishes it** (`Projection::Basis`). `Basis` has no `Default` or public constructor; `Checked`/`CheckedBatch` make "ignoring the basis" unrepresentable; `Rejection` keeps `RecordGone` / `RecordMoved` / `RecordUnstated` apart. Anti-staleness, not security. The check is a property of the session door (`apply_command` does not check), and only two of the three tiers hoist per frame — the record tier is asked per command (`records_named`) against the frame's basis. Read gotcha 43 before touching it.
44. **The TUI grid draws from `GridCache`**, which frames it too (`AxisFrame`; width rule `header_level_widths`). Use the one-pass accessors, not per-position scans. `RecordHash` is document-scoped, `AggregateHash` layout-scoped.
45. **One session loop (`Host`), two transports.** Saving is a request frame (`ClientFrame::Save`); dirtiness is *declared* for `WholeWorkbook` and *observed* for `CellData`/`LayoutGeometry` (`MarkDirty` is gone); for `CellData` the flag is each effect's own delta (drained after every `CellData` effect, so it keeps its order with `Save`) plus reorders, formulas and config, not a document fingerprint; writes that cancel across effects still leave it set. `impl Storage for Rc<T>` lives in `improvise-io`. `harness/` runs `OpfsFs` for real; Safari is unseen.
46. **The page picks a peer:** `?ws=` → socket, else a module `Worker`. `#status` composes two facts; the first-frame watchdog is per-peer-agnostic. The demo model is built so every cell is a `Record`.
47. **Keys become commands in the browser;** the server must not know the keymap. `GridExtent`, `set_viewport`, `Commands(Vec)`, `resolve::NEEDS_THE_HOST` and the five client rules (`projected_at`, `opened`, `refused_here`, `Step::Declined`, `notify`).
48. **The browser JS is partly executed** (`harness/tests/shell.spec.js`); changes to `web/*.js` belong there. Serialize frames on one tail promise; disarm, don't pre-empt, a watchdog. Evidence headings may be module-qualified (and must be when the leaf collides).
49. **A websocket handshake is exempt from same-origin policy** — `src/origin.rs` refuses the upgrade (loopback and absent allowed, two refused). Not authentication.
50. **A composed TUI row measures in cells and names which side yields** (`fill_line` + `Keeps`). Keep whichever key table is *derived from what it describes*. Scrolls are measured where drawn (`grid_width`).
51. **The command language reads pipes, refuses surplus, and checks names when applied** (`only_when_omitted`; `RequireKnown`/`unless_unknown`).
52. **A date column imports as one hierarchical category;** `--split-date` restores flat. `Plan::spelling_clash` refuses what would silently merge; `--strict` refuses partial skips.
53. **Delivery: a watermark is a `Stamp`; a correction is `target - held`, applied only at the watermark it starts from.** `through` is not `at` (the watermark advances only to `through`), a refused frame restores `Delivery` with the app snapshot, and a lost last notice is found only by `poll`. Axis changes refetch the window until ADR 008's render model. Long form `context/code/delivery.md`.

---

## File Map

Per-crate entry points. Per-file notes, test counts and a dated line-count snapshot are in
`context/file-inventory.md`; re-derive sizes with `wc -l`.

| Area | Where | Entry points |
|---|---|---|
| Core model | `crates/improvise-core/src/` | `model/{types,cell,record,category,id}.rs`, `view/{layout,types,axis}.rs`, `workbook.rs`, `format.rs` |
| Formula | `crates/improvise-formula/src/` | `parser.rs`, `formula.pest`, `ast.rs`, `quote.rs` |
| I/O | `crates/improvise-io/src/` | `storage.rs`, `opfs.rs`, `persistence/{improv.pest,format,parse,export,mod}.rs`, `import/{wizard,analyzer,csv_parser}.rs` |
| Protocol | `crates/improvise-protocol/src/` | `lib.rs` (ViewState), `reduce.rs`, `render.rs`, `keymap.rs`, `face.rs`, `frame.rs`, `basis.rs`, `resolve.rs`, `client.rs`, `viewmodel.rs`, `help.rs` |
| Command layer | `crates/improvise-command/src/command/` | `cmd/{core,registry,navigation,cell,commit,effect_cmds,fit,grid,mode,panel,search,text_buffer,tile,wizard}.rs`, `keymap.rs`, `parse.rs` |
| Semantic UI state | `crates/improvise-command/src/ui/` | `effect.rs`, `app.rs`, `project.rs`, `reduce.rs`, `host.rs`, `viewport.rs`, `cat_tree.rs` |
| TUI + CLI | `src/` | `draw.rs`, `main.rs`, `ui/{grid,help,import_wizard_ui,text}.rs`; integration tests in `tests/` |
| GP | `crates/{gp-tree,improvise-gp}/` | `gp-tree/src/{lib,data,fit}.rs`, `improvise-gp/src/{lib,dataset,fit}.rs` |
| Socket host | `crates/improvise-ws-server/src/` | `registry.rs`, `origin.rs`, `transport.rs`, `main.rs`; `host.rs` is a re-export shim |
| Worker host | `crates/improvise-worker-server/src/` | `worker.rs`, `vfs.rs`, `bindings.rs` (wasm32 shim) |
| Browser client | `crates/improvise-wasm-client/src/`, `web/` | `lib.rs`, `keys.rs`, `render.rs`, `socket.rs`; `web/{app,shell,worker,serial}.js`, `index.html`, `demo.improv`, `build.sh`, `assemble.sh` |
| Browser harness | `harness/` | Playwright (`tests/shell.spec.js`, `tests/opfs.spec.js`), `opfs-contract/`, `build.sh` — not a workspace member; its own CI job |
| Examples | `examples/`, `crates/*/examples/` | `gen-grammar.rs`, `pretty-print.rs`, `category_width_bench.rs`, `fit_csv.rs` |
| Scripts | `scripts/` | `checkers.sh` (every `*-test.sh` suite), `check-comment-runs.py`, `check-ci-graph.py`, `bd-audit-ids`, `kb`, `pr-watch`, `pr-reviews`, `build-doc-site.mjs` |
| Context docs | `context/` | `design-principles.md`, `gotchas.md`, `file-inventory.md`, `code/<area>.md` (arguments), `test-evidence/` (bug evidence by test name), `adr/` (001–005, 007, 009; no 006/008 file yet), `cloud-agents.md`; `prolog-sessions/kb.pl` is the KB; `docs/design-notes.md` is product vision |

---

## Wasm

The browser bundle is `improvise-{core,formula,io,protocol,command,wasm-client,worker-server,incremental}`
plus `zset` and `numtower`; each `cargo check --target wasm32-unknown-unknown` clean with no feature flags
(`wasmCrates` in `nix/ci.nix`, which also gates `improvise-gp`, a candidate). `improvise-ws-server`,
`zset-ffi` and the root crate are native-only.
Check new bundle dependencies with `cargo tree --edges normal` (the default output includes dev edges),
per crate. `cargo check` cannot see runtime hazards: `Instant::now()` panics on wasm (`web-time`), and
`std::fs` compiles but fails in a browser. Long form: `context/file-inventory.md`, appendix.

## CLI

```
improvise [model.improv]                          # open TUI (default)
improvise import data.csv [--no-wizard] [-o out]  # import CSV/JSON
improvise cmd 'add-cat Region' -f model.improv    # headless command(s)
improvise script setup.txt -f model.improv        # run script file
```

Import flags: `--category --measure --time --skip --extract --axis --formula --name`, `--split-date <field>`
(flat date split; default is one hierarchical category), `--strict` (refuse any skipped record), and
`--on-duplicate refuse|sum|separate` (default `refuse`). Details: `context/code/cli.md`, gotchas 51–52.

## Testing — the short version

Full guidance: `context/design-principles.md` §7.

- **No single test binary over 2 seconds**; the workspace wall clock is the sum. Re-measure with `cargo test --workspace` (a `finished in` per binary) and compare the slowest binary to the next few — a ratio, not seconds, and not a ratio to the median (it is zero).
- Commands: build a `CmdContext`, call `execute`, assert on the returned effects. No terminal.
- Property tests (`proptest`, default 256 cases) for invariants and round-trips; a grammar-walking strategy must read its pest grammar once (`OnceLock`).
- **Bug-fix workflow:** write the failing test first. Record the bug in `context/test-evidence/`, keyed by test name — not in a doc comment (two-line cap, `scripts/check-comment-runs.py`).
- Coverage target ~80% on logic code; skip ratatui render paths.

## Key dependencies

ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest, flate2, csv, enum_dispatch. Dev: proptest, tempfile, pest_meta.