diff --git a/context/code/doc-site.md b/context/code/doc-site.md index cfd4b734..09a9eb01 100644 --- a/context/code/doc-site.md +++ b/context/code/doc-site.md @@ -43,7 +43,7 @@ Not published: will meet, and nobody visiting a website is about to read a PR thread. None of the three is referenced by any published document except as a row in -`repo-map.md`'s file inventory, which is inside a code fence, so leaving them +the file inventory (`file-inventory.md`), which is inside a code fence, so leaving them out creates no dangling link. That was checked rather than assumed. **And "not published" is now derived rather than assumed** (improvise-res9). @@ -233,11 +233,13 @@ recorded as a decision rather than left to be rediscovered as a bug. ## 6. Navigation -`repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and -splitting it per gotcha was rejected: the gotchas cross-reference each other -constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader -following those wants a scroll, not 48 page loads. It stays one page, with an -id on every list item and a compact strip of numbered links under the heading. +`repo-map.md` carries a digest of the numbered gotchas and `gotchas.md` the long +form; the map was condensed because it was too long to put in every new agent's +context. Splitting per gotcha was rejected: the gotchas cross-reference each +other constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a +reader following those wants a scroll, not one page load per number. Each of the +two stays one page, with an id on every list item and a compact strip of +numbered links under the heading. The ids come from the list item's **own number**, not from a counter, so an inserted gotcha renumbers the anchors exactly as it renumbers the prose that cites them. diff --git a/context/file-inventory.md b/context/file-inventory.md new file mode 100644 index 00000000..6ba97b8b --- /dev/null +++ b/context/file-inventory.md @@ -0,0 +1,428 @@ +# File Inventory (the long form) + +The per-file listing that `context/repo-map.md` summarises. Moved here unchanged. + +Line counts are a **dated snapshot**, not a maintained fact: measured 2026-09-24 at `c1772ca9`, and **70 of 105 had drifted past 10%** in the three days since the 2026-09-21 refresh — eleven had doubled or more (`client.rs` 255 → 943, `lockstep.rs` 1126 → 4064). Treat them as "is this file big", never as a figure to cite; re-derive with `wc -l`. The `ws-server/` and `wasm-client/` entries use a shorthand prefix, so a refresh must map it to `crates/improvise-/{src,}/` by hand or they drift silently. Test counts are informational — run `cargo test --workspace` for live numbers. Files under 100 lines and render-only widgets omitted. + +### `improvise-core` (`crates/improvise-core/src/`) +``` +model/types.rs 3560 / 112t Model, formula eval (fixed-point), CRUD +model/cell.rs 1358 / 47t CellKey (sorted), CellValue, DataStore (interned + index), + live_records (insertion order — gotcha 18) +model/record.rs 226 / 8t RecordHash / CellId surrogates: the opaque identities a + thin client names a cell by (ADR 002) +model/category.rs 310 / 9t Category, Item, Group, CategoryKind +model/symbol.rs 89 / 3t Symbol interning +view/layout.rs 2604 / 57t GridLayout, drill, records mode, records_columns; a private + per-axis position index (`row_pos`/`col_pos`, rebuilt by `reindex`) + so a data position resolves in O(1) (improvise-felo.13) +view/types.rs 817 / 43t View config (axes, pages, hidden, collapsed, format) +model/id.rs 290 slugify + IdAllocator: how a generated id is made (ADR 007 + clause 4, core.md §20). Nothing outside the file calls it yet +view/axis.rs 190 / 5t Axis enum + label/next methods (no colour: that is a Face) +workbook.rs 413 / 22t Workbook: Model + cross-view ops, item facade +format.rs 519 / 40t format_f64, parse_number_format (display only) +``` + +### `improvise-formula` (`crates/improvise-formula/src/`) +``` +parser.rs 1394 / 79t pest grammar + tokenizer → Formula AST +quote.rs 160 / 7t quote_name / is_bare_name / pipe_quote / unescape_pipe_body: + the one home, both halves (improvise-i2m5). improvise-gp + re-exports it; improvise-io calls it from + persistence/{format,parse}.rs +formula.pest 108 PEG grammar for formula text +ast.rs 130 Expr, BinOp, AggFunc, Formula, Filter +``` + +### `improvise-io` (`crates/improvise-io/src/`) +``` +storage.rs 334 / 13t Storage trait (read/write/list/delete), PhysicalFs, MemoryFs, + `impl Storage for Rc` (improvise-hmws), and + contract::storage_contract — public, not cfg(test), because + a browser must run that sequence (harness/) +opfs.rs 289 / 0t OpfsFs: Storage over the browser's Origin Private File + System. wasm32 only. Run by harness/, not by cargo test +persistence/improv.pest 179 PEG grammar — single source of truth +persistence/mod.rs 441 / 11t document lifecycle: save/load/gzip/legacy-JSON over Storage; + facade re-exporting format_md/parse_md/export_csv* +persistence/format.rs 401 / 11t format_md: Workbook → .improv text +persistence/parse.rs 1684 / 46t parse_md: pest tree-walk → Workbook (+ grammar generator) +persistence/export.rs 243 / 5t export_csv / render_csv. Builds its layout with + with_frozen_records, NOT new — `new` skips prune_empty + (improvise-iot6) +persistence/roundtrip_tests.rs + 1236 / 62t format_md ↔ parse_md characterization (tests only) +import/wizard.rs 1967 / 62t ImportPipeline + ImportWizard. `Plan::place` is the one + placement pass, shared by `collisions()` and the build so + the two cannot disagree; `DuplicatePolicy` / `Collision` + (improvise-cgqt) +import/analyzer.rs 434 / 17t Field kind detection (Category/Measure/Time/Skip); + analyze_records is first-seen order, not HashSet (gotcha 18) +import/csv_parser.rs 590 / 26t CSV parsing, multi-file merge +``` + +### `improvise-protocol` (`crates/improvise-protocol/src/`) +``` +lib.rs 619 / 2t ViewState, ViewCursor, ViewFrame, AppMode (17), Panel, + MinibufferConfig, MinibufColor, HELP_PAGE_COUNT +reduce.rs 749 / 10t Command + ModelCommand (the wire format), HelpPage, reduce_view +render.rs 2157 / 46t CellDisplay, WidgetCache trait + one cache per widget + (GridCache/CategoryTreeCache/FormulaCache/ViewListCache/ + TileBarCache), AxisFrame/FrameEntry/FrameGroup and + header_level_widths (the grid frame, gotcha 44), + GridCache::seed (the bootstrap compaction), Projection, + bootstrap-only Snapshot +keymap.rs 975 / 0t KeyPattern, Binding, Keymap, ModeKey (17), TextEntry, + PanelKeymap, Keymaps::default_keymaps/prefix_keymap — the + shared table both front ends look up. Tests live with + dispatch in improvise-command/src/command/keymap.rs +viewmodel.rs 170 / 5t flatten_cat_tree — derivations between a cache and a + renderer, called by both front ends (improvise-edp) +face.rs 682 Face (a role: Axis, WizardField, Mode*, MinibufferOf…), Theme, + AnsiSlot — appearance as data a front end applies (ADR 004, + protocol.md §10); draw.rs::face_style is the TUI's one translation +key.rs 96 / 3t KeyCode, KeyModifiers, KeyEvent (moved from command/) +frame.rs 156 / 3t ClientFrame/ServerFrame — the ONE envelope pair, both + deployments (improvise-xks6). Nothing else defines one. + ClientFrame::Save is a request, not a Command (gotcha 45) +basis.rs 257 / 6t Basis (the three tiers), Rejection, BasisCache — ADR 001 + layer 1. Basis has no Default and no public constructor +help.rs 1241 / 14t HelpPage content and Hint { keys: Vec<(KeyCode, KeyModifiers)>, + command, caption }. `compact` drops a chord by its MODIFIERS + (anything beyond shift), never by parsing a label it just + formatted (improvise-thpe) — the guard is the signature +resolve.rs 4883 / 88t the browser's key resolvers: a BrowserKey to the Commands it + composes, mirroring each Cmd line for line INCLUDING where + siblings disagree (gotcha 47). The largest file in the crate +client.rs 943 / 15t ClientState: ViewState + one cache per widget, and the fold; + in_flight, `opened` (OpenedCell), notify, settle_status + (gotcha 47). EditorSession is in lib.rs beside ViewState +``` + +### `gp-tree` / `improvise-gp` (`crates/gp-tree/`, `crates/improvise-gp/`) +``` +gp-tree/src/lib.rs 575 / 8t Tree, Spec, eval, simplify, render, breed, Population, + SplitMix64 (vendored seeded Rng; seed() keeps the seed so + a fit can be shown and recorded, pinned to the published + reference vectors) +gp-tree/src/data.rs 1000 / 21t raw string rows -> f32 feature vectors: per-column type + inference, one-hot, mean-imputation + col.missing flag, + ColumnSpec/encode_row, Dataset (incl. warnings), + Model = trees + plan + seed + FitConfig +gp-tree/src/fit.rs 961 / 32t the search: FitConfig (Default = the CLI's numbers), fit() + -> simplified champion, cv_regress/cv_classify, + permutation_importance, folds/standardize/mse. + Returns data; never prints +gp-tree/examples/fit_csv.rs 583 CLI over the above: flags, demo dataset, all printing +gp-tree/tests/scoring_known_answers.rs + 250 / 6t f32 bit patterns + a next_u64 witness: the evidence any + change to the scoring paths needs (gotcha 37c) +improvise-gp/src/lib.rs 978 / 25t formula_text / fitted_formula: a Tree as IF(...) formula + text, validated back through parse_formula. A threshold is + the SHORTEST decimal that reads back as the same f64 (not + the exact expansion), so a row on a split boundary routes + the same in f64 as in f32 eval; non-finite values panic +improvise-gp/src/dataset.rs 506 / 15t a View's records -> the feature matrix a fit consumes — + the improvise-side counterpart of preprocess. Records mode + is already a feature matrix (gotcha 1) +improvise-gp/src/fit.rs 1778 / 35t one fit end to end: a View's records in, an improvise + Formula and the evidence for believing it out. Not in a + command, because none of it is terminal-specific +``` + +### Command layer (`crates/improvise-command/src/command/`) +``` +cmd/core.rs 475 / 2t Cmd trait, CmdContext, CmdRegistry, parse helpers +cmd/registry.rs 903 / 13t default_registry() — all registrations +cmd/navigation.rs 507 / 12t Move, EnterAdvance, Page* +cmd/cell.rs 724 / 20t ClearCellCommand, YankCell, PasteCell, TransposeAxes, SaveCmd +cmd/commit.rs 1304 / 27t CommitAndAdvance, CommitBuffer(CommitKind), CommitExport, + CommitSave +cmd/effect_cmds.rs 519 / 6t effect_cmd! macro, 25+ simple wrappers, `unless_unknown` (gotcha 51) +cmd/fit.rs 293 / 8t fit-measure: evolve a tree over the active view and store it + as an ordinary `## Formulas` line — nothing GP-specific is + left in the document afterwards +cmd/grid.rs 988 / 22t ToggleGroupAtCursor, ViewNavigate, DrillIntoCell, TogglePruneEmpty +cmd/mode.rs 469 / 13t EnterMode, Quit, EditOrDrill, EnterTileSelect +cmd/panel.rs 936 / 24t Panel toggle/cycle/cursor, formula/category/view panels +cmd/search.rs 202 / 4t SearchNavigate, SearchOrCategoryAdd, ExitSearchMode +cmd/text_buffer.rs 399 / 14t AppendChar, PopChar, CommandModeBackspace, ExecuteCommand +cmd/tile.rs 170 / 6t MoveTileCursor, TileAxisOp +cmd/wizard.rs 244 / 6t HandleWizardKey — per-step import-wizard key tables +keymap.rs 1353 / 44t KeymapSet, Keymap::dispatch/dispatch_transient, effects_of, + SetTransientPrefix — what a thin client cannot have. The + table itself is improvise-protocol/src/keymap.rs +parse.rs 350 / 23t Script/command-line parser (prefix syntax) +``` + +### Semantic UI state (`crates/improvise-command/src/ui/`) +``` +ui/effect.rs 4020 / 99t Effect trait. 63 effects: 53 `impl Effect for` blocks, 9 + simple_effect! expansions, and SetTransientPrefix in + command/keymap.rs. The 10 outside the brace-match are all + ViewOnly; 28 of the 53 are Model(_) (gotcha 38, h3y7) +ui/app.rs 5558 / 134t App, ModelState, ViewState + dispatch methods, Panel, handle_key, dispatch_key, document_generation, autosave; apply_effects declares `dirty` for WholeWorkbook and observes it for CellData/LayoutGeometry (gotcha 45) +ui/project.rs 6383 / 118t Dirty, Viewport, grid_of/category_rows/formula_texts/ + view_summaries/tile_entries, grid_updates (the diff), + ClientSession + Sessions (connect/apply/rebase/project_all) +ui/reduce.rs 1217 / 24t Checked / CheckedBatch / apply_command / model_effects / + reduce_key — one door from a Command or a keypress to the + authoritative App. `reduce_full` is GONE (improvise-h3y7) +ui/host.rs 3410 / 67t Host: one App + one Sessions; handle, commands, project_all, + rebase. Lives here, not in ws-server (gotcha 45) +ui/cat_tree.rs 174 / 8t Category tree flattener for panel +ui/viewport.rs 259 compute_col_widths / compute_row_header_width / visible_cols + (pure geometry; the width rule itself is the protocol's); + grid_width, SIDE_PANEL_WIDTH, MIN_GRID_WIDTH (improvise-gtn7.1); + GridSummary / grid_summary / grid_window (improvise-felo.13) +``` + +### TUI rendering, draw, main (`src/ui/`, `src/draw.rs`, `src/main.rs`) +``` +ui/grid.rs 905 / 14t GridWidget (ratatui): no &Model, no &View — the frame comes + off GridCache (gotcha 44). Its module doc is the best-written + statement of that rule in the repo +ui/help.rs 277 5-page help overlay (render only) +ui/import_wizard_ui.rs 829 / 9t Import wizard rendering. `Panel` is the one writer, and + every write is clipped to the panel's rows and width +ui/text.rs 169 truncate and wrap, both measured in terminal cells, never bytes +draw.rs 1122 / 20t TUI event loop, frame composition; `content_tests` draws a + whole frame to check the selection beside a side panel +main.rs 428 CLI entry (clap): open, import, cmd, script +# other ui/*.rs are small panel renderers — skip unless changing layout/style +tests/grid_frame_agreement.rs + 479 / 6t the two front ends draw the same grid frame (improvise-anr9) +tests/records_wide_smoke.rs 67 / 1t +tests/shipped_documents_load.rs 333 / 8t every TRACKED .improv/.improv.gz loads, and every record + it projects is writable. "Shipped" is read off `git ls-files`, + not a .gitignore reading (improvise-c0aj), and the walk's git + is scrubbed of every hostile GIT_* setting the caller may carry +tests/web_demo_model.rs 83 / 2t every visible cell in web/demo.improv is a Record +``` + +### Browser host and client (`crates/improvise-ws-server/src/`, `crates/improvise-wasm-client/src/`, `web/`) +``` +ws-server/registry.rs 616 / 14t Registry of Hosts keyed by DocumentId; ClientId {document, + session} — a SessionId is unique only within its document. + A document is never evicted while the server runs (ADR 001) +ws-server/host.rs 4 / 0t a re-export shim only. The session loop is + improvise-command/src/ui/host.rs (gotcha 45) +ws-server/origin.rs 132 / 6t OriginPolicy: which origins may open a socket. Loopback by + default, --allow-origin widens, absent allowed, two refused + (improvise-kc7n). The only decision transport::admitted makes +ws-server/transport.rs 330 / 4t tokio + tungstenite plumbing, plus the Origin check at the + handshake — a refused upgrade, not an ignored socket +ws-server/main.rs 190 / 5t CLI entry: serve a file on a port; --bind, --allow-origin +ws-server/tests/socket.rs 389 / 6t the one test that uses a real socket: handshake, JSON + framing, SessionId routing, which document a URL asked for +wasm-client/lib.rs 455 / 8t Client: the wasm-bindgen surface (init, on_message, on_frame, + viewport/set_viewport, render_model, set_cell_at, typed_cell) +wasm-client/keys.rs 187 / 5t BrowserKey ONLY — a KeyboardEvent flattened to an improvise + key. It does NOT resolve: `resolve` and KeyOutcome live in + improvise-protocol/src/resolve.rs, shared by both front ends +wasm-client/render.rs 1249 / 21t RenderModel: editable / selected / cat tree, decided here +wasm-client/socket.rs 547 / 13t the fold (fold_frame/handle_frame), the two compositions + and typed() text→CellValue. Defines no enums +# state.rs is GONE — ClientState moved to improvise-protocol/src/client.rs +wasm-client/tests/browser_round_trip.rs 2136 / 34t improvise-1ey's scope against a real Host +wasm-client/tests/lockstep.rs 4064 / 78t host and client agree on a keypress +web/app.js 816 string calls only, no protocol knowledge, no framework. + Run by harness/tests/shell.spec.js (Chromium; the standalone half on Firefox too) — see + gotcha 48 for what that does and does not reach +web/worker.js 86 three string calls, no protocol knowledge. RUN by + shell.spec.js's no-?ws path, with bindings.rs (gotcha 45) +web/shell.js 399 everything app.js DECIDES that is not painting: no DOM, + no browser API, one `node --test` away. Owns socketUrl, + gridPlan, frameCostOf, documentTitle — so a chunk of the + browser's render decisions is NOT in the file gotcha 42 + and gotcha 46 name. Reasoning: context/code/clients.md +web/serial.js 19 the one rule worker.js exists to keep: one frame at a + time, in arrival order (gotcha 48's async-listener race), + extracted so it is testable without worker globals +web/index.html, style.css the page; legibility only +web/build.sh 38 wasm-pack build → web/pkg (gitignored). Needs + wasm-opt = ["-O", "--enable-bulk-memory"] or the bundled + wasm-opt rejects LLVM's memory.copy +web/assemble.sh 112 assembles the deployable site (copies web/*.improv) +web/demo.improv 65 the document the site ships and seeds from. Every cell a + Record — asserted (gotcha 46) +``` + +### Browser worker host (`crates/improvise-worker-server/`) +``` +improvise-worker-server/src/worker.rs 765 / 17t Worker: one port, one client; handshake, Addressed + filter, persist trigger (model_state.dirty). Every decision, every test +improvise-worker-server/src/vfs.rs 348 / 8t documents/open: seeding is a Storage::write +improvise-worker-server/src/bindings.rs 185 / 0t wasm32 shim: OpfsFs awaits + postMessage. No Rust test + reaches it, but harness/tests/shell.spec.js DOES: the + no-?ws path runs start()/OpfsFs::open under Chromium + and Firefox. + Called NEVER RUN here for months: see gotcha 45 +improvise-worker-server/src/lib.rs 51 / 1t the module doc that states exactly what is and is not + covered — the one correct copy of that fact +improvise-worker-server/tests/post_message.rs 407 / 8t main-thread traffic, strings only +``` + +### Browser harness (`harness/`) — not a workspace member + +``` +harness/playwright.config.js 50 projects: chromium (every spec), firefox (opfs, plus + shell.spec's standalone half: grepInvert @mocked-peer), + webkit (opfs only, which skips on a runtime probe: no + OPFS). CI runs `--project=chromium --project=firefox` only: + nixpkgs' WebKit aborts inside the runner VMs (ci.yml) +harness/tests/shell.spec.js 345 the first execution of web/app.js. A change to web/*.js + belongs here (gotcha 48). Playwright serves harness/serve/, + which harness/build.sh ASSEMBLES: edit web/*.js, rebuild, + or the specs run the old page +harness/tests/opfs.spec.js 103 drives the OPFS contract page and reads back its result +harness/opfs-contract/src/lib.rs 470 runs improvise_io::storage::contract::storage_contract + against a real OpfsFs inside a real Web Worker +harness/www/opfs/{index.html,worker.js} the page and worker that host it +harness/build.sh 50 assembles harness/serve/ (web/assemble.sh, the fixtures, + and the contract crate's wasm-pack build) — gitignored +# gated by the `browser` job in .github/workflows/ci.yml, which can fail the build +``` + +### Examples +``` +examples/gen-grammar.rs Grammar-walking random .improv generator (pest_meta) +examples/pretty-print.rs Parse stdin, print formatted .improv +crates/improvise-core/examples/category_width_bench.rs + The same records at 12 and at 24 regular categories + (improvise-felo.1; context/code/core.md) +``` + +### Context docs +``` +.claude/skills/code-review/ Review skill: checks a change against the doctrine in these docs + kb.pl +scripts/check-comment-runs.py The two-line comment cap AND --check-evidence. The flag is + ADDITIVE since improvise-ijql — every invocation that can + exit 0 answers the cap — guarded by a subprocess self-test, + the only level a dispatch bug is visible at. Reasoning: + context/code/tooling.md §2 +scripts/check-ci-graph.py pages-deploy must `needs` every job in ci.yml, and every + suite `checkers.sh --list` knows must run in some CI job + (improvise-dak5). It REFUSES what it cannot read rather than + parsing YAML: the values it accepts are an allowlist + (tooling.md). --self-test; run in the `test` job. README + states the rule and lists no jobs (improvise-x7cq) +scripts/checkers.sh The one entry point for every *-test.sh suite: scrubs the + env to an allowlist, runs under the bash that runs it (no + version floor since #297). `ci_suites` is the table of which + suites each CI job runs (`--ci `); ci.yml names none +scripts/pr-watch Waits on a PR's checks (the pr-watch skill). Exit 5: the PR + is already MERGED or CLOSED, so there is nothing to wait for +scripts/assemble-stamp.sh The cache-bust stamp gate web/assemble.sh calls, extracted + so its refusals can be exercised without a web/pkg +scripts/assemble-stamp-test.sh Its check_the_checker, run first in the `build` CI job +context/code/tooling.md The repository's own checkers: why an evidence heading's + names come from cargo rather than from the source, and + why a leaf name is not an identity +context/test-evidence/tooling.md Evidence for the gates in scripts/ and for the tests that + hold CI's shape +scripts/nix-seed-flake-inputs.sh Seeds locked flake inputs over git so `nix develop` works in cloud sessions +prolog-sessions/kb.pl Executable knowledge base: invariants, doctrine, decisions (CI-checked) +context/adr/007-stable-identities.md Stable identities: ids unique across the document are the + keys and names are display-only; gensym suffixes apply to + generated ids only, and hand-written ids are canonical. + Accepted 2026-09-25. (There is no ADR 005 or 006 file: + improvise-l2b8 and improvise-ilvl claim those numbers.) +context/adr/004-faces-and-themes.md Faces: appearance as UI-agnostic data a front end + applies, emacs-style. Accepted 2026-09-22. +context/adr/003-live-backed-models.md A model may be backed by a live query source — + improvise as a DB client. Accepted, NOT a priority. +context/adr/002-naming-a-cell.md How a client names a cell: an opaque surrogate on + CellDisplay, whose absence means the cell is an + aggregate. Records why _Index cannot serve. +context/adr/001-concurrent-edits.md Concurrent edits between clients: record-level atomicity, + optimistic basis check per batch, advisory reservations. + Records why a causal clock was rejected. +context/cloud-agents.md Cloud-session environment: tracker sync, bd pitfalls, what is blocked +context/design-principles.md Architectural principles & testing doctrine +context/plan.md Show HN launch plan +context/repo-map.md This file +docs/design-notes.md Product vision & non-goals +``` + +--- + +--- + +# Wasm, CLI, testing and dependencies (long form) + +The three repo-map sections of these names, as they stood before the map was condensed. + +## Wasm + +The browser bundle is `improvise-core`, `improvise-formula`, `improvise-io`, +`improvise-protocol`, `improvise-command`, `improvise-wasm-client` and +`improvise-worker-server`. All seven +`cargo check --target wasm32-unknown-unknown` clean with no feature flags +(improvise-ywd), gated by the `wasm` job in `.github/workflows/ci.yml`. That +job gates **eight** crates, not seven: `improvise-gp` was added (improvise-z5eg) +explicitly *not* as a bundle member but as a candidate, because staying +wasm-clean is free now and expensive to retrofit — so the bundle is seven and +the gate is eight, and neither number is the other one's typo. +**A feature that adds a dependency to a bundle crate must be checked with `cargo tree --edges normal`** — the default output includes **dev** edges, so a plain `cargo tree -p improvise-protocol --target wasm32-unknown-unknown` reports a dev-only `serde_json` as *present* and reads exactly like a leak. Check per-crate too, because the `wasm` CI job invokes each crate separately and feature unification cannot reach across that (improvise-26dm added a `test-support` feature on `improvise-protocol` on those terms; it is off in every shipped build). + +`improvise-ws-server` is the host and is native only — tokio, tungstenite and +`Box` — so `improvise-wasm-client` cannot depend on it in anger. It +no longer needs to: both ends `pub use` `improvise_protocol::frame` and neither +defines a pair (gotcha 42). The **dev**-dependency remains, for +`tests/browser_round_trip.rs`, which hands the client's own serialized frame to +a real `Host::handle`. The root `improvise` crate is +the terminal binary and does **not** build for wasm — it depends on +crossterm; that is correct, not a gap. + +`cargo check` passing is not the same as running. Two known runtime hazards +it cannot see: `Instant::now()` panics on wasm (hence `web-time` in +`improvise-command`), and any `std::fs` call compiles but fails in a browser +— improvise-fuj closed the effects that bypassed `Storage`; autosave and the headless final write still do (improvise-31oe.11). + +## CLI + +``` +improvise [model.improv] # open TUI (default) +improvise import data.csv [--no-wizard] [-o out] # import CSV/JSON +improvise cmd 'add-cat Region' -f model.improv # headless command(s) +improvise script setup.txt -f model.improv # run script file +``` + +Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`, `--split-date ` (a date column imports as ONE hierarchical category by default; this restores the flat split, and `--extract` still picks the components — improvise-8jk6.3), `--strict` (with `--no-wizard`, any skipped record is a refusal and nothing is written, where the default writes and warns with a count — improvise-8jk6.6), and `--on-duplicate refuse|sum|separate` (default `refuse`; applies with `--no-wizard`, where the wizard would otherwise ask — improvise-cgqt). A headless import also refuses a `--split-date` that names a non-date field and an `--axis` that names a category the import did not build (exit 1). + +--- + +## Testing — the short version + +Full guidance lives in `context/design-principles.md` §7. Quick reminders: + +- **Two numbers, not one** — the ambiguity between them is what let the old + "suite runs in <2s" drift: **no single test binary over 2 seconds**, and the + workspace wall clock is the sum. Measured at `63a5c13`: **1464 tests over 35 + binaries** (1 ignored), longest single binary **0.6–1.0s across two runs on one machine** — + the spread being its own argument for reading your own run. Excludes compilation, + and excludes `harness/` (Playwright, minutes, its own CI job). *Re-measure* rather + than re-assert — `cargo test --workspace` prints a `finished in` per binary, and + **the portable signal is the ratio, not the seconds**. **Not the ratio to the + median, though: it is degenerate.** Most binaries print `0.00s` at two decimals, + so the median is zero and every ratio to it is undefined (improvise-i5z6, 39 + binaries). Compare the slowest binary to the next few slowest instead: an outlier + is one that sits well clear of them *in every run*. That is how i5z6 was read — + `improvise-formula`'s lib longest of 39 in all three runs, at 0.91–1.95s against a + runner-up's 0.50–0.92s. Both outliers so far had one cause: a generator re-parsing + its pest grammar per case. `improvise-io` (improvise-8shu, 1.42s → 0.21s) and + `improvise-formula` (improvise-i5z6, 0.84–0.98s → 0.06–0.07s) each read it once through a + `OnceLock` now. A new grammar-walking strategy should do the same from the start. + The full argument is `context/design-principles.md` §7. +- Commands: build a `CmdContext`, call `execute`, assert on returned effects. No terminal needed. +- Property tests (`proptest`, default 256 cases) cover invariants: CellKey sort, axis consistency, save/load roundtrip, `parse(format(parse(generate())))` stability. +- **Bug-fix workflow**: write a failing test *before* the fix (regression guard). Record the bug in `context/test-evidence/`, **keyed by test name** — *not* in the test's doc comment, which the two-line comment cap forbids and `scripts/check-comment-runs.py` enforces. A heading may be module-qualified and must be when its leaf is not unique. +- Coverage target ~80% line/branch on logic code; skip ratatui render paths. + +--- + +## Key dependencies + +ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest + pest_derive, flate2 (gzip), csv, enum_dispatch. Dev: proptest, tempfile, pest_meta. diff --git a/context/gotchas.md b/context/gotchas.md new file mode 100644 index 00000000..d98edc42 --- /dev/null +++ b/context/gotchas.md @@ -0,0 +1,139 @@ +# Gotchas (the long form) + +The numbered gotchas that `context/repo-map.md` summarises in a line or two each. +**The numbers are stable and are cited across the repo (`gotcha 24`, `repo-map gotcha 35`) +— a citation means the entry below.** Read the ones your diff touches; the repo map's +digest says which those are. Every entry is the original text, moved here unchanged. + +Dated claims and counts inside an entry are snapshots: recount rather than cite. + +1. **Records mode is wide format.** A row is one *coordinate tuple* — the cells sharing everything but `_Measure` — and the columns are `[coordinates…, **`+cat`**, raw measures…, `+`]`: a column per regular category, then the column that creates a **new category** (improvise-2zq, sitting where the column it creates will appear), then a column per raw measure, then the column that creates a new measure (improvise-c5v). Consequences: two measures on one coordinate are ONE row; editing a category column moves **every** measure cell in the row, not just the one under the cursor; `_Dim` in the synthetic key carries the column **index**, not a label, because a category and a measure may share a name — resolve it with `GridLayout::record_col` into a `RecordCol`. Formula targets are not columns (nothing to type into them). Drill pending edits are keyed by `(row, RecordCol)`. + **`prune_axis_items` takes a predicate, not a flag vector.** `prune_empty` hands it `impl Fn(usize) -> bool` over the *data* index (group headers consume no index). The domain is derived from `items` itself, so there is no parallel length a caller could get wrong — the earlier `&[bool]` + `debug_assert` pair could disagree between debug and release if a third caller ever sized `keep` from something other than `row_count()`/`col_count()` (improvise-n3wt). **It re-marks the survivors rather than filtering the markers** (improvise-e265): runs nest, so dropping an item can leave a header with nothing under it or a `GroupEnd` closing a run that was never reopened, and no per-marker filter can tell. `remark` instead emits, before each kept item, exactly the headers and `GroupEnd`s that make the output's open runs equal the input's at every level — each survivor keeps exactly the runs it was in. Reasoning: `context/code/core.md` §6. **`prune_empty` shares one `GridEval`** across the whole row×col emptiness test, and **a grid evaluates only through `GridLayout::evaluator`**, which builds the fold memo from that layout's own `none_cats` and hidden items — so no other grid's memo can reach its cells or totals (improvise-metf, improvise-opbo; the memo itself is improvise-h6z1, `core.md` §14). + **`GridLayout::new` does not prune; `with_frozen_records` does** — and that split has now produced **four** bugs: `view_frame`, `project::grid_in` (PR #78), and `persistence/export.rs::render_csv` (improvise-iot6), which wrote rows *and columns* the grid does not draw for as long as CSV export has existed. Every remaining `GridLayout::new` call site in the workspace is test code — verified — so a new production caller almost certainly wants `with_frozen_records(m, v, None)`. + +2. **Effects never touch `std::fs`.** Every document read or write goes through `App::storage: Box` (improvise-fuj, closed). Two writes still go around it: `App`'s autosave and the headless final write call the path-taking `persistence::save` (improvise-31oe.11, open). The host picks the backend: `App::new` supplies `PhysicalFs`, `App::with_storage` takes any other — the browser worker supplies OPFS (improvise-i34, closed), and tests use `MemoryFs`. `improvise-io` keeps path-taking wrappers (`save`, `load`, `export_csv`, `parse_csv_with_warnings`) for the native CLI, each delegating to a `_to`/`_from`/`_bytes` variant that takes the backend or the bytes. +3. **Every effect declares an `EffectKind`, and there is no default.** `ViewOnly` never reaches other clients; `Model(CellData | LayoutGeometry | WholeWorkbook)` does (improvise-mae). A new effect that forgets to classify itself fails to compile — that is deliberate, since the failure mode is silent: a mis-declared `ViewOnly` edit would simply never reach connected clients. Watch two traps: the *View* lives inside `model_state.workbook`, so "touches model_state" does NOT mean "is a model change" (axis/page/format are `LayoutGeometry`, cursor and scroll are `ViewOnly` — see improvise-8uz; and `SwitchView`, `ViewBack` and `ViewForward` moved from `LayoutGeometry` to `ViewOnly` with improvise-jytm, writing `view_state` and nothing else, which is what took the latter two off the wire — but a sound derivation is not a guard: only `SwitchView` had one, and review showed reclassifying either sibling back was silent across the whole suite); and `Save`/`SaveAs` touch `model_state` for bookkeeping only and are `ViewOnly`. `AddCategory`/`RemoveCategory` are the third trap and go the other way: they look like data edits, but both go through `Workbook`, whose `on_category_added`/`on_category_removed` rewrite **every** view's axis map, so they are `LayoutGeometry`. The two "an effect declaring X actually changes X" tests cannot separate them — a category change moves model data *and* view config, so it clears either bar; `structural_effects_declare_their_kind` pins the declaration itself. A third trap, now guarded: **`EffectKind` is too coarse to check a *mapping*.** `ui/reduce.rs::model_effects` sends each of the 26 `ModelCommand` variants (counted on `c1772ca9`; gotcha 38) to one effect, and every inverse pair — `HideItem`/`ShowItem`, `AddItem`/`RemoveItem`, `CreateView`/`DeleteView`, `SetCell`/`ClearCell` — declares the *same* kind, so swapping three of them at once left the suite at 918 passing while three wire messages did the exact inverse of what they said. The mapping is name-for-name by construction, so `every_model_command_names_the_effect_of_the_same_name` asserts that, reading the type name off the effect's derived `Debug` (`Effect` is a trait object, so `type_name` is out of reach). A variant whose effect must be named something else is a variant that stopped being derived — it should fail there and be argued about. Their reach is wider than the name suggests: `Model::remove_category` also drops every formula targeting the category, which is why `Dirty::of(LayoutGeometry)` reports `formulas: true` — see gotcha 35. +4. **Commands never mutate.** `&CmdContext` is read-only; return `Vec>`. If you want to touch `&mut App`, add or extend an Effect. +5. **`CellKey` is always sorted.** Use the constructors; equality and hashing rely on canonical form. +6. **No `Model::add_item`.** With a `Workbook` in hand use the facade: `wb.add_item("Region", "East")?` / `add_item_in_group` / `remove_item` (improvise-tvt). With only a `Model`, go through the category: `m.category_mut("Region").unwrap().add_item("East")`. +7. **Virtual categories.** `_Measure` items = explicit items ∪ formula targets. Use `measure_item_names()` / `effective_item_names("_Measure")`. `add_formula` does *not* add items to `_Measure`. Formula-target lookup also falls back into `_Measure` via `Model::find_item_category`. Cell aggregation goes through `apply_agg`, formula `SUM(…)`/`AVG(…)` through `eval_agg` — they differ on empty input (absent vs. A third rule joins them for **non-numeric** values: `aggregate_cell_values` reduces text by **unanimity** — one matching value, or several copies of the same value, is that value; differing values, or text among numbers, is `CellValue::Error("mixed")` (improvise-l4qx). The **all-numeric branch is checked first**, so `Count` of a lone number is still `1` and not the number — that ordering is the whole difference between this rule and "a single value passes through". What the rule forbids is *dropping*: `filter_map(as_f64)` used to, and since `_Index`/`_Dim` are on `Axis::None` in every default view that made **every text cell in every document render blank**, including the pipe-quoted text `.improv` carries as a first-class value. `evaluate_aggregated` and `aggregate_raw` share the helper, so a formula ref over mixed values now errors `"mixed"` rather than silently summing the numeric ones. `0` for Sum/Count, error for Avg/Min/Max). `Model::evaluate_aggregated` takes `Aggregation::{Exact, OverHidden}` rather than `none_cats: &[String]` (improvise-5jm5) — the list was only ever tested for emptiness, and the guard against that shape returning is the **signature**. `recompute_formulas`/`aggregate_raw` keep the list because they walk it. +8. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full `f64`. Never feed a rounded value back into eval. +9. **Formula eval is fixed-point.** `recompute_formulas(none_cats)` iterates until stable, bounded by `MAX_EVAL_DEPTH`; circular refs converge to `CellValue::Error("circular")`. `App::new` runs it before the first frame so formula cells render on startup. +10. **Drill into formula cells** strips the `_Measure` coordinate when it names a formula target, so `matching_cells` returns the raw records that feed the formula instead of returning empty. +11. **Minibuffer keymaps come from `TextEntry`.** The eight text-entry modes are built by the `TextEntry` builder in `improvise-protocol/src/keymap.rs` — seven by `TextEntry::new`, and `RecordsEditing` by `TextEntry::with_parent` off `Editing`'s map, which is why a count taken off the `::new` call sites reads seven. It binds the shared spine (AnyChar→`append-char `, Backspace→`pop-char `, Esc→`[clear-buffer , enter-mode ]`) and welds every commit key to its `clear-buffer` via `.commit(key, cmd, args)` (improvise-w3q). `TextEntry::new` takes the `AppMode` and reads `` off its `MinibufferConfig`, so the buffer the prompt names and the buffer the keys edit are the same string by construction (improvise-k8h) — the keymap never spells a buffer name. `.rebind` exists only for CommandMode's Backspace. `SearchMode` is hand-written on purpose — no commit key, and `exit-search-mode` owns the query. `EnterMode` clears the incoming mode's minibuffer through `write_buffer` too — building `SetBuffer` directly there was unreachable only because no shipped `AppMode` is keyed `"search"`. `MinibufferConfig::buffer_key` is a `String`, not `&'static str`: a borrowed-'static field makes `Deserialize` unimplementable, and every text-entry `AppMode` variant carries a `MinibufferConfig` (improvise-a2x). A buffer named at runtime is read with `cmd::core::read_buffer` and written with its mirror `write_buffer` — "search" is not in the buffers map but its own `ViewState` field, and those two functions are the only code that knows it (improvise-gdl). +12. **Panel keymaps come from `PanelKeymap`.** The three panel modes share a spine (Esc/Tab→Normal, arrows and j/k→`move-panel-cursor `, every panel's toggle key, `:`→command mode) built by `PanelKeymap::new(panel)`; the keys that act on the panel's contents arrive through `.bind`/`.bind_args`/`.delete` because they vary in shape, not just command name (improvise-4pg). `Panel::ALL`/`name()`/`key()` in `ui/app.rs` drive the toggle keys for Normal and all three panels, so a fourth panel reaches every keymap from one `ALL` entry. `name()` must spell what `parse_panel` accepts. +13. **Prefix keys are stored as an address, not a keymap.** `ViewState::transient_prefix` holds the `(KeyCode, KeyModifiers)` that opened a prefix; `KeymapSet::prefix_keymap` resolves it back to the sub-keymap, walking parents so `RecordsNormal` reaches Normal's `g`/`y`/`Z` (improvise-jc8) — that is the command-layer wrapper; the resolution itself is `Keymaps::prefix_keymap` in `improvise-protocol/src/keymap.rs`. `ViewState` must stay free of `improvise-command` types — it is destined for the protocol crate, which `improvise-command` will depend on. Resolution assumes prefixes are **flat**; `prefix_keymaps_are_flat` fails the build if a nested one is added. +14. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the mode; Normal + `search_mode=true` maps to `SearchMode`. Minibuffer modes bind Enter/Esc as `Binding::Sequence` so `clear-buffer` fires alongside commit/cancel. Hints are checked against the keymaps: `every_hint_key_resolves_in_its_mode` (`command/keymap.rs`) splits each `AppMode::hint_text()` and requires every key it names to resolve in that mode's map, because `only_quit_has_no_hint` asserts only that a hint is *present*. It checks which keys are named, not what they do. Its tokenizer knows the hint dialect explicitly (named keys, `Ctrl-`, `F/C/V` alternatives, character runs like `hjkl`/`[:]`, the em-dash prose tail, CommandMode's `:q` ex-commands) and panics on a token it cannot classify rather than skipping it. +15. **Registry names come from prototypes, not strings.** Every `register*` call reads the name off a prototype `Cmd` instance, so a typo is impossible (improvise-9cn) — so no `register*` call spells a name. Name literals do remain in `registry.rs`, and only tests guard them — the keymap test `every_bound_command_name_is_registered` for the `Move` literals (all bound; `registered_names_are_unique` covers them too), and for the four aliases `registry.rs::every_alias_resolves_to_a_registered_command` (a hand-copied list, not the table) plus the `parse.rs` tests `alias_add_cat_resolves_to_add_category`, `alias_formula_resolves_to_add_formula`, `alias_add_view_resolves_to_create_view` and `alias_q_bang_resolves_to_force_quit` — the `Move` prototypes carry `cmd_name: "move-selection"` / `"scroll-rows"` / `"page-scroll"` (the four `jump-*` names go through `reg_jump!`), and four `r.alias` calls spell an alias and its target (`add-cat`, `formula`, `add-view`, `q!`). What types can't reach: five commands name themselves off a field (`Move` off `cmd_name`, `ViewNavigate`, `TileAxisOp`, `ToggleGroupAtCursor`, `CommitAndAdvance`), so a prototype can name the wrong *variant* and still compile. That shows up as a **duplicate** registration, not a mismatch — the wrong-variant entry collides with its sibling and the sibling's name vanishes. `registered_names_are_unique` catches it; `every_bound_command_name_is_registered` catches the keymap side, and checks **both** halves of `dispatch`'s `registry.interactive(name, args, ctx).ok()?` — the name resolves *and* the args parse — because improvise-s13 moved `enter-mode ["export-prompt"]`/`["formula-edit"]` out of the nullary class, where a typo'd mode name compiles into a dead key. It builds the `CmdContext` in each keymap's own `AppMode`, since `commit-buffer item` reads `ctx.mode`. +16. **`effect_cmd!` macro** wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written `Cmd` once there's real decision logic. Its counterpart on the effect side is `simple_effect!` in `ui/effect.rs`, which declares a one-field newtype that writes one place on the `App`. Its **kind argument is mandatory** — `simple_effect!(Name(T) => |app| place, EffectKind::ViewOnly)` — even though every call site so far answers `ViewOnly`: hard-coding it reintroduced exactly the default gotcha 3 does without, since `$place` is an unconstrained expression and a one-line `active_view_mut().number_format` setter would have reported `ViewOnly` with the suite green (improvise-8fy). Exactly six wrappers return `unless_unknown(needs, effect)` rather than the bare effect (gotcha 51): `set-axis`, `set-page`, `toggle-group`, `hide-item`, `show-item` and `switch-view`. The rest return bare effects — `delete-view` (whose `DeleteView` effect refuses an unknown view itself), `add-item`, `add-items`, `set-cell`, `create-view` — and `remove-formula` checks in `execute`. Likewise, mode entry has no per-mode shim commands — bind `enter-mode ` (improvise-s13); the name is decoded by `parse_mode_name` in `registry.rs`, and `EnterMode` clears the mode's minibuffer on the way in. +17. **Float equality inside formula eval has one epsilon, and the div/0 guard shares it.** `FLOAT_EQ_EPSILON = 1e-10` in `model/types.rs` is used by `=`/`!=` **and** by the division-by-zero guard (`rv.abs() < FLOAT_EQ_EPSILON`), in both `eval_expr` and its aggregated twin — six uses across the two evaluators (a div/0 guard plus the `Eq` and `Ne` arms in each), one constant. This gotcha previously recorded the opposite (`=`/`!=` epsilon, div/0 exact) and was stale; the mix is gone. Why an epsilon at all and why this one: `context/code/core.md` §4. +18. **`IndexMap`** preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do `DataStore::matching_cells`/`matching_values`: the secondary index **narrows** the candidates, then `matching_positions` resolves each to its position in the primary `IndexMap`, sorts, and reads back. **An index whose values are `HashSet`s is for O(1) lookup, never for iteration.** Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; `SUM`/`AVG` drifted in the last bits because `f64` addition is not associative (`Min`/`Max`/`Count` and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the **drill snapshot** froze its record list in hash order while `drill_state.pending_edits` are keyed by *row index* (gotcha 1), so the same drill opened in a second process addressed **different records** — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on `matching_values`, and `matching_cells` 7% *faster*, sorted positions resolving `CellKey`s with better locality. `DataStore::live_records` was the open choice and is now decided (improvise-2yc2): it walks `cells` and projects insertion order onto records, deliberately paying O(cells) rather than O(records), because `cells`' order is already the answer `matching_positions` gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on `RecordEntry`, since a stored position is a copy and a record deleted and re-created would hold its old one. Note what is *not* on offer: an unordered return — every iterator has an order, and a type that merely disclaims one still hands the caller hash order, which is the guard living one level away from the hazard and the shape this whole class of bug returns through. **The second real instance was `import::analyzer::analyze_records`** (improvise-7yx), which built its distinct values through a `HashSet`: `build_model` adds items in exactly that order and `IndexMap` preserves it, so the *same CSV imported twice* produced two different `.improv` files and two different row orders. Now first-seen order, pinned by a five-iteration test because each `HashSet` gets its own `RandomState`. Also answered here since: an **empty `partial`** matches every cell vacuously (improvise-hkt6) — it used to index `sets[0]` and panic, unreachable only because both callers guarded. **A group's fold reads through the same rule.** `Model::fold_stored` folds a collapsed group's stored values once, over every record the group covers, through `DataStore::spanned_values` → `spanned_positions` (improvise-yb4b) — one scan, not a lookup per member. `spanned_positions` chooses **by index-set size** between narrowing once by the base key and narrowing once per covered item of the smallest span, whichever visits fewer (improvise-jq0m), and **both arms return store order** — sorted positions, as `matching_positions` does. That is load-bearing, not tidy: a fold that returned hash or per-item order would reintroduce the `SUM` drift above, and the choice between the arms would then be visible in the last bits of a total. **A partial as wide as the widest stored cell needs no narrowing** (improvise-felo.13): `DataStore` keeps `arities` (how many stored cells have each width), and `exact_positions` answers a strictly-sorted partial at least that wide with one probe of `cells`, since only an equal cell can match; anything else falls through to the index. The answer is the same sorted positions. +19. **Commit paths for cell edits** split across `commit_cell_value` → `commit_regular_cell_value` / `commit_plain_records_edit` in `command/cmd/commit.rs`. Keep the synthetic-records branch in sync with the plain branch when changing behavior. **Which cells refuse an editor is one function, asked at both ends of an edit**: `cmd::core::uneditable(model, layout, row, col)` is the shared decline for **opening** an editor (`EnterEditAtCursor`) and for **committing** one (`CmdContext::committable_cell_key`) — a fold or other aggregate, a formula's result, or a `CellId::NonItem` (improvise-enz6, improvise-gk51). It reads the cell's `CellId` through `project::cell_id`, the same derivation the projection uses, and the browser's `resolve::declined_at` matches on the projected `CellId`, so the two front ends refuse the same cells. Asking only at open is the bug enz6 was: Enter-to-advance moved onto a collapsed group's row and committed a write to the group coordinate. **`CellId::NonItem { category, item }` is a stored value whose coordinate names no item of its category** — records mode can draw one, and no write can address it, so it is projected as its own variant rather than as a `Record` that the host then refuses (gk51 was exactly that mismatch). Its one wording is `improvise_protocol::non_item_refusal`. +20. **Minibuffer commits go through one command.** `commit-buffer ` (`CommitBuffer { kind: CommitKind }`) owns buffer→trim→bail-on-empty→dirty→status for all three; the varying parts are methods on `CommitKind`. The add effect is emitted **last** so its failure message wins over the command's optimistic status (improvise-6t3). `CommitKind::Item` carries its target category, resolved from `AppMode::ItemAdd` by `parse_commit_kind` at registry time — so `commit-buffer item` is not constructible from a script line. +21. **An abort stops the chain, never the bookkeeping — and ordering is what separates them.** Three effects set `abort_effects`, and they are not all saves: `Save` and `SaveAs` on a failed write or a missing path (improvise-8tk — `SaveAs` was the arm left out of the original fix, so `:w /bad/path . q` quit on a failed write), and `AddCategory` when `Model::add_category` refuses, because what follows assumes the category exists — from the `+cat` column (improvise-2zq) the rest of the batch would advance the cursor and re-open an editor on a column that was never created. A `Binding::Sequence` flattens into one effect vec, so the abort carries across sequence steps; that is a property of the *flattening*, and it is why improvise-0hb's proposal to apply effects between steps was rejected — `apply_effects` resets the flag per batch, so step 2 in a fresh batch would land the `Quit` after a failed `Save`, silently reopening a P1. `a_sequence_whose_first_step_aborts_does_not_run_the_second` is the guard. **The rule the aborts taught, which is the part to carry:** an abort at the *tail* of a batch also ate the mode restoration and the buffer clearing, stranding the user in `CommandMode`/`Editing`/`SavePrompt` — where `draw_bottom_bar` renders the minibuffer *instead of* the status line, so the error the effect had just written was never drawn and the operation appeared to do nothing at all. So **tidy-up goes at the head, where an abort cannot reach it**: `ExecuteCommand` fronts its batch with `ReturnToNormalUnlessMoved`, `CommitAndAdvance` with the mode change, `CommitSave` with the mode change (found by Copilot on PR #27). Ordering answers it without giving any effect a "runs anyway" flag, and it does not weaken gotcha 31's observed-not-predicted mode settling: whatever a later effect sets, it sets *after*, so the mode a `:` line ends in is unchanged in every case — the only difference is what happens when the batch never reaches the end. **An abort is not a rollback**; that is the hole clause 4 leaves (gotcha 47). +22. **The import wizard's back path is a history stack, not a reverse table.** `advance` pushes the step it leaves onto `ImportWizard::history`; `previous` pops it, so back always retraces the path actually taken through the conditional transitions (improvise-907). Preconditions live in `validation_error()`, checked on the step where the user can still fix them — never deferred to `build_model`. `NameModel` is terminal for `advance`: Enter there calls `build()`, which parks the workbook in `built` on `Done`, and `AcceptWizardModel` takes it from there. Going back from `Done` discards `built`. **`ResolveDuplicates` appears only when `ImportPipeline::collisions()` is non-empty**: `after_fields` routes there from `ReviewProposals` or `ConfigureDates`, so it comes after the field and date steps and before `DefineFormulas`, with `NameDuplicateCategory` between them when the answer is Separate. Enter there is `WizardOp::ChooseDuplicate` (sum, keep separate, or go back). `ImportPipeline::on_duplicate` defaults to `DuplicatePolicy::Refuse`, so a path that never asked — the headless `--no-wizard` import included — errors listing the collisions rather than summing them. **A collision is a cell written by two records** (a tuple *and* a measure), not a shared tuple: records with disjoint measures on one tuple write different cells and collide with nothing (improvise-9bit). Reasoning: `context/code/formula-and-io.md`, "Colliding records are a question, not a default". +23. **`app.layout` goes stale mid-batch.** `apply_effects` rebuilds only *after* the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a `RebuildLayout` effect in front of it — the command decides the order (see `EnterEditAtCursorCmd`, `CommitAndAdvance`). Read the cursor's cell text via `App::display_value_at_cursor()`; don't build a `CmdContext` inside `apply` (it reads `layout`, the memoised `grid_geometry()` and the cursor, and a stale layout there is the bug). **Since improvise-felo.13 the rebuild is skipped when nothing it reads has moved**: `App::refresh_layout` rebuilds only when `layout_stale` (any `EffectKind::Model(_)` in the batch, or `install_workbook`), the drawn view's name, or the drill snapshot's pointer (`LayoutBasis`) changed, so a cursor key rebuilds nothing, and the column widths and row-header width are measured once per layout (`App::grid_geometry`, dropped with the layout). Two of the three inputs are **observed** (the view name and the drill pointer); `layout_stale` is **declared** from `EffectKind::Model(_)` (`touches_model`), so gotcha 3's classification hole — a `ViewOnly` effect that changes what the layout reads — now also leaves `app.layout` stale, where before felo.13 every batch rebuilt. It is still not a hand-kept list of effects. A writer that bypasses effects and changes what the layout reads must call `rebuild_layout()` itself. Measurements: `context/test-evidence/layout-cost.md`. + +24. **The cursor is per-client and keyed by view.** `ViewState::cursors: HashMap` holds `selected`/`row_offset`/`col_offset`, keyed by view name; an absent key means the origin. Reach it through `App::cursor()` (a `Copy` snapshot), `App::cursor_mut()`, and `App::reset_cursor()` — never off `View`, which no longer has the fields (improvise-8uz). Two consequences. **Changing the grid's shape must reset the cursor explicitly**: `View::transpose_axes`/`cycle_axis` used to do it themselves, and now `TransposeAxes`/`CycleAxis` call `app.reset_cursor()` — a new geometry effect that renumbers rows or columns has to do the same. **Replacing the document must clear the map**: `LoadModel` and `AcceptWizardModel` call `cursors.clear()`, because the incoming workbook's view names have nothing to do with the outgoing one's. That clearing replaces `Workbook::normalize_view_state`, which is gone — a `View` can no longer carry a stale offset. The same clearing owes the **session mirrors**: `install_workbook` bumps `App::document_generation`, and `ClientSession::project` reseeds any mirror seeded at an earlier one (improvise-cqq). An effect cannot reach `Sessions`, so the generation is the mechanism and `Sessions::rebase` is only the client-facing half. The same is true of a **geometry** change, and it owes the mirrors *both halves*: `App::cursor_epoch` is bumped by `reset_cursor` and watched by `Sessions::adopt_cursor_epoch`, which resets every mirror **and** returns `Broadcast::ToAll(Command::ResetCursor)` (improvise-gvnb). Doing only the first is **strictly worse than doing nothing** — the diff then believes every client already holds origin-positioned content and sends nothing, so they stay stale *permanently* rather than until their next scroll. The trigger is the **epoch moving**, not the command variant, for the reason gotcha 39 watches `document_generation` instead of matching on `LoadModel`: which effects renumber the grid is a per-effect decision, so a table of variants would be a second description of it, free to disagree. What is broadcast is an **invalidation, never a position** — another client's transpose must not move where I am looking (that is UI and it is mine, improvise-g0f), only declare that where I was looking has stopped meaning anything. **Since improvise-ni6k no effect calls `reset_cursor` at all — the cursor is invalidated by OBSERVING the grid's frame.** `apply_effects` diffs the frame after every model effect, **inside the loop**, because a batch that reframes and then places the cursor loses the placement otherwise (four records-mode tests pin that position). There is no list of effects anywhere, so the mechanism is total over `EffectKind::Model(_)` by construction — a genuinely stronger shape than a hand-kept table, and the reason the old "a geometry effect that forgets `reset_cursor` is invisible in both directions" hazard is closed. + + **Two arms, and the asymmetry is the whole design (improvise-mcq4).** `LayoutGeometry` hashes only the **active** view — it is the held-key path, and other views cannot reframe there since `view_mut()` writes only the active view, with the category and item effects already covered by `view_shapes`. `CellData` hashes **every** view, because a cell write can prune a row out of a view the host is not on; narrowing that arm to one view measured as *free* and silently lost a multi-client invalidation, which is gotcha 24's own strictly-worse-than-nothing case. Cost of the wide arm: 0% on the held-key path, +5.7% on an 18k two-view `SetCell`, linear in view count, accepted. State the absolute as well as the ratio: the observation roughly doubles held-key latency on large pruned documents (3.04 → 5.69 ms at 18k) and 56 ms at 90,000 cells is 3.4 frames. + + **Totality is conditional on classification**, and nothing makes classification total: a `ViewOnly` effect that renumbers the grid is invisible to all of this (gotcha 3's hole, unchanged). And two known gaps are filed rather than fixed — a geometry effect immediately after a mid-batch `SwitchView` skips the check entirely, because there is no basis for the new view yet (improvise-9c9o; under-reporting, which gotcha 35 says is the expensive direction), and `Model(WholeWorkbook)` matches neither arm, which is benign only because all three such effects go through `install_workbook` → `rebase` → a fresh `Snapshot`. + +25. **`ViewState` and its field types are `Serialize`/`Deserialize`.** It is the wire payload of the client Snapshot (improvise-q08), so `AppMode`, `MinibufferConfig`, `MinibufColor`, `ViewFrame`, `KeyCode` and `KeyModifiers` all derive both (improvise-a2x), and `improvise-command` depends on `serde` for it. The constraint this puts on new state: **no borrowed-`'static` fields anywhere ViewState can reach** — `Deserialize` cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. `view_state_survives_a_json_round_trip` guards it. `ViewState` and `Snapshot` also derive `Clone` and `PartialEq`, so the bootstrap message can be compared in a round-trip test. + +26. **Axis presentation lives on `Axis`; its colour is a `Face`.** Four vocabularies hang off the enum in `view/axis.rs` — `plain_label()` ("Row"/"Col"/"Page"/"None"), `glyph()` (which `Display` appends, giving "Row ↕"), `short_label()` (three columns for the tile bar; `None` is "·", *not* a truncation), and `initial()` (one letter for the view panel; `None` is empty). Plus `next()` for the Row→Column→Page→None rotation. **`Axis` carries no colour**: `AxisColor` and `Axis::color()` are gone. A colour is a `Face::Axis(axis)` in `improvise-protocol/src/face.rs` (ADR 004, `context/code/protocol.md` §10), resolved by the `Theme`, and `draw.rs::face_style` is the one place a face becomes a ratatui `Style` — the tile bar and the category panel both call it, so neither keeps a table. `improvise-core` still names no ratatui type. Not covered here: `persistence/format.rs` maps `Axis` to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer. + +27. **`Axis::Filter` constrains without being navigable.** Drilling fixes its coordinates on `Filter`, not `Page` (improvise-dz8): both restrict which records are visible, but only `Page` is something the user may move or page through. The split shows up as two different questions, and picking the wrong one is the trap: **"what data is visible" is `View::filtering_cats()`** (Page ∪ Filter — used by `GridLayout` and by `add-record-row` when it builds a key from the current constraints), while **"what can the user act on" is `categories_on(Axis::Page)`** (used by `[`/`]` page cycling) or `View::tileable_cats()` (used by all three tile-select sites). `Axis::next()` maps `Filter` to itself and no navigable axis rotates into it, so tile select cannot hand one out. The `.improv` grammar accepts `filter` and round-trips a selection with it — a Filter axis only ever occurs in the never-persisted `_Drill` view, but the alternative was writing it out as some other axis. **What a virtual view must *be* is model-side and has one home** (improvise-gd28): `view::{records_spine, records_steps, drill_steps}` return `Vec` — pure data naming no `Effect`, no `App` and no front end — and `cmd/grid.rs::steps_into_active_view` is the whole of *how the TUI gets there*, twenty lines a browser answers its own way. Before the split, the drill's filter placement and the records spine were spelled out twice in `cmd/grid.rs` with nothing keeping them in agreement. `DrillState` deliberately did **not** move: it holds an `Rc` and its `pending_edits` are uncommitted writes, so gotcha 30's session-local tag already places it. Reasoning: `context/code/core.md` §11. + +28. **The browser backend is sync on purpose, with an async seam.** `OpfsFs` (`improvise-io/src/opfs.rs`, wasm32 only) implements the same synchronous `Storage` trait as `PhysicalFs`, because `FileSystemFileHandle.createSyncAccessHandle()` yields a handle whose read/write/truncate/flush **are** blocking — available only inside a dedicated Web Worker, which is where improvise-9x6 puts the app. So `Storage` never became async. What is still async splits out into four explicit methods the worker awaits from its message loop: `open()`, `attach(path)` (must precede any read/write of that document — acquiring a handle is async even though using it is not), `refresh()` (repopulates the index `list` answers from), and `commit_deletes()`. Consequence: **`delete` is visible immediately but lands in OPFS at the end of the batch** — it drops the name from the index and queues a `removeEntry`. Handles are exclusive per file, so `attach` closes any handle it replaces and `detach` releases the lock. `storage_contract` in `storage.rs` is the behaviour every backend owes; it runs against both host backends and is the written-down spec `OpfsFs` must satisfy in a browser, where the harness cannot reach. **And the premise is dedicated-worker-only, which is narrower than this paragraph used to read.** `createSyncAccessHandle` is spec'd `[Exposed=DedicatedWorker]`, and measured in a module `SharedWorker` on Chromium and Firefox it is **absent** — not rejected, not flagged, not transient: `getDirectory()` and `getFileHandle()` both succeed and the method is simply not on the object (improvise-12ha). So `OpfsFs` cannot be hosted by a `SharedWorker` as written, and "a browser worker" in this paragraph means a *dedicated* one everywhere it appears. `OpfsFs::open`'s own guard does not enforce that — it `dyn_into`s `WorkerGlobalScope`, which a `SharedWorkerGlobalScope` **satisfies**, so a shared worker sails past a check whose message names the right requirement and fails later as a raw `TypeError` deep in the glue (improvise-7gip). The consequence for multi-tab is that the machinery everyone reaches for first is the storage seam, not `Sessions`: `Sessions`, the `Addressed` filter and the one `frame` pair are all ready, and none of that helps, because the no is entirely on the handle. + +29. **"Which view is active" is stored twice, and the copies are now allowed to differ.** `Workbook::active_view` is the document's **default** — what a freshly connected client opens, and what `.improv` writes as `Initial View`. `ViewState::active_view` is **authoritative for that client**, and it is what keys `ViewState::cursors`. The second copy exists because `reduce_view` runs on a client that holds no `Workbook` (improvise-cqi); the *split* is improvise-jytm answering improvise-g0f, on the user's ruling that **the model is shared and the UI is not**. Two clients may sit on different views of one document. Three things follow. **`App::resolve_view` is the one answer** to what a view name names in this document, falling back to the default so a slice naming a deleted view cannot panic in `rebuild_layout`; `view()`, `view_mut()`, `view_name()`, `project::grid_in` and `ClientSession::view_in` all go through it, and a second copy of that fallback was the first thing review found. **The guard was replaced, not deleted** — `the_two_active_view_names_never_diverge` became `the_documents_default_view_outlives_this_clients_switches` plus `a_fresh_session_opens_the_documents_default_view`, because a deliberate split with no guard is how one silently regresses into the other. And **a deleted view is the case the split creates**: `DeleteView` can take away the view another client is on, so `Sessions::adopt_deleted_views` moves that mirror to the default and tells it, and `effect::DeleteView` does the same for the host — observed ("does this mirror still name a view the document has"), never predicted from the command variant. **Testing any of it needs a bystander that can be moved**: both audience tests first sat theirs on the default, where a wrongly-addressed relay folds to a no-op, and review demonstrated the vacuity by injecting `ToAll` twice. One gap remains: `Initial View` no longer follows the TUI user (improvise-azhb). The other — a view-scoped `ModelCommand` naming no view — is **closed** (improvise-lltt), and this paragraph described it as an open gap for a day after: every view-scoped variant carries `view: String` and `model_effects` maps it to `ViewTarget::Named`, so a connected client's `SetAxis` lands on the view it names. `ViewTarget::Active` resolves through `App::view_mut()` and is therefore the *host's* view — but **the wire never constructs one**, which is what closed it. Follow mode is deliberately not foreclosed. + +30. **Two tests gate `ViewState` membership.** `ViewState` is the *portable* session slice — the one thing serialized into the Snapshot a newly-connected client gets (improvise-q08). A field may live there only if it is **portable** (plain data from the allowlisted crates, no `Rc`/`Arc`) and **snapshot-safe** (a fresh client inheriting the value is correct). `wizard` fails both — `ImportWizard` is an `improvise-io` type, and a half-open dialog is not something to inherit — and `drill_state` fails both — it holds an `Rc`, and `pending_edits` are uncommitted writes belonging to whoever staged them. Both live on `App` under the **session-local** residue tag (improvise-h0b). The `ViewState: Send` static assertion in `app.rs`'s tests is the compile-time proxy for portability: `Rc` is not `Send`, so a re-added shared pointer fails the build (the `Deserialize` derive from gotcha 25 now rejects it too, serde's `rc` feature being off). `Send` does *not* catch the origin-crate or snapshot-safety halves — check those by hand. + +31. **`:` commands settle their mode by observation, not prediction.** `ExecuteCommand` ends every batch with `ReturnToNormalUnlessMoved(ctx.mode)`, which drops to `Normal` only if the mode is still what the batch started in. There is no `Effect::changes_mode()` any more: it was a per-type constant, and `ViewBack`/`ViewForward`/`StartImportWizard` swap the mode only on some runs, so no up-front answer was right — `:import` opened the wizard and was snapped straight back out (improvise-toc). A new effect that changes the mode needs no annotation; just don't return the batch's starting mode from a success path. + +32. **There is no aggregate render cache — each widget folds its own stream.** `improvise-protocol::render` holds one cache type per widget (`GridCache`, `CategoryTreeCache`, `FormulaCache`, `ViewListCache`, `TileBarCache`), each implementing `WidgetCache` with its own `Update` type, and a `Projection` is one such update addressed to one widget (improvise-rkeq). `reduce_view` folds only `ViewState` now — it no longer takes a cache, and `Command::Project` passes through it untouched. Two consequences. **The overlap between caches is deliberate**: the tile bar and the category panel both carry `(category, axis)` and are updated by separate projections, so neither is woken by the other's changes; do not "fix" it by sharing a variant. **Bootstrap is a *compaction*, not a replay**: `Snapshot { view, seed: Vec }` seeds a new client with a batch of the same projections the steady stream carries, so there is no whole-state type that could disagree with the incremental one. The seed is the server's *current* state, not the head of the history — `GridCache::seed()` builds it, and the property that earns it is `GridCache::folded(&cache.seed()) == *cache`. Nothing is defined for the help overlay (its content is static and `help_page` is already in `ViewState`) or for the import wizard, which now has a shape and **deliberately no projection**: `WizardCache` is built by `project::wizard_cache` and drawn locally, because a half-open dialogue is addressed to precisely the session that opened it, never to every client (improvise-764 — the rendering-side twin of the argument that removed `AcceptWizardModel` from `ModelCommand`, gotcha 38). **A cache without a `Projection` is the shape session-local residue takes when it still has to be drawn by two different front ends.** It lives in `improvise-protocol` rather than `improvise-command` for the `flatten_cat_tree` reason: when the wasm client grows a wizard it must draw *these* rows, not a second derivation. + +33. **The grid's labels frame its cells, and `apply` enforces it — the fold is not commutative.** Three consequences in `improvise-protocol/src/render.rs`. (a) A `GridUpdate::Cells` that overtakes the `RowFrame`/`ColFrame` naming its positions is **dropped**, because `fit_to_labels` truncates to the labels it has; a client must not reorder the stream, and `a_cell_update_that_overtakes_its_labels_is_lost` pins it. (b) A label change that *renames* a position blanks the cell under it: `fit_to_labels` preserves by POSITION, which is only right where the label did not change, so `apply` calls `invalidate_relabelled` itself rather than trusting a producer to send `GridUpdate::Invalidated` first. `Invalidated` therefore now means only "contents stale, frame unchanged". (c) **every arm that touches labels *or* widths re-fits** — `col_widths` is re-fitted with the columns — a new column is drawn at its label's own character count until `ColWidths` catches up — because a renderer zips widths with labels and a length mismatch indexes off the end. Fitting on the label arms alone is what let a late `ColWidths` put the mismatch straight back — a short update keeps each unheard-of column at its label's width, and a long one is dropped as having overtaken the frame that names it, exactly as (a) holds `Cells`. `GridCache` also carries `totals` and `page_coords` (improvise-n10). `totals` is framed by `col_frame` on the same terms as the cells and widths and re-fits with them, with one asymmetry: an **empty** strip means "this grid has no totals" (records mode, or an empty grid) and is therefore *not* padded, where a non-empty one is — padding it would draw a blank total row under a records grid that has none. `Invalidated` blanks the totals too: they are contents, and a stale sum under fresh cells is the one number a user takes at face value. `page_coords` is framed by nothing (it names categories, not positions) and survives a reshape. Note the fold tests: asserting `fold(prefix)+fold(rest) == fold(whole)` is vacuous (it is the definition of a left fold), so the tests that earn their place are the order-sensitivity one and the compaction-equivalence one — see `doctrine(fold_tests_must_test_order_not_the_axiom)`. + +34. **A formula target is unquoted, and pipe quoting has one home.** `parse_formula` strips the pipes off a target exactly as it does off an `Expr::Ref` — the target names a `_Measure` item, so a target that kept its delimiters made `add_formula` create an item literally named `|Total Revenue|`: a ghost column the user's real item never matched, and `Expr::Ref("Total Revenue")` never resolved to. Both the TUI formula minibuffer (`AddFormula`) and `.improv` load go through that one call, so both were affected. The writer emits `Formula::raw`, not `target`, so file text is unaffected either way — which is exactly why the round-trip tests never saw it, and why the fix repairs existing files on next load with no migration. Pipe quoting has exactly **one home**: `improvise_formula::quote`, owning both halves — `quote_name`/`is_bare_name`, `pipe_quote` (unconditional, for a `.improv` text cell that must not read back as a number) and `unescape_pipe_body`. `improvise-gp` re-exports it, `improvise-io` calls it from `persistence/{format,parse}.rs`, `parser.rs::unquote_pipe` is a shim over it, and io's own copy is **deleted** (improvise-i2m5). **The keyword guard is canonical — user-confirmed, so do not revert it as over-caution**: an item literally named `SUM` is written `|SUM|`. Bare keywords do parse, but only because `agg_call` needs a `(` and PEG backtracking falls through to `ref_expr` — rule ordering, not a stated rule — and a `## Formulas` line carries formula text verbatim, so a name `format_md` writes is read back by `parse_formula`: one namespace, one spelling rule. One thing the guard does not cover, and one that used to be an exception and no longer is. **A section header takes its name as the whole line, totally** (improvise-pczv): `header_name = { pipe_quoted ~ &NEWLINE | line_name }` — the pipe-quoted arm needs the lookahead, or `|A|B` would parse as `A` and silently drop `B`, and `line_name` catches everything else, so a header name can no longer fail to parse. Four rules share it (`model_name`, `initial_view`, `category_section`, `view_section`), which is why it is a rule and not four copies. Note a header is still *written* raw where a reference is quoted — the grammar accepts both arms, so the quoting asymmetry between a header and a reference is a writer convention, not a format one. And the two former *unquoting* copies differed only on a trailing lone backslash, unreachable from the grammar, where io returned it and `parser.rs` panicked — the surviving `unescape_pipe_body` is total. Two `## Category:` sections naming one category are **one** category, concatenated by `ParsedFile::push_category` in **pass 1** (improvise-349m) — merging in pass 2 was not idempotent, because `build_category` creates its own section's item-groups before its own `>` lines while `format_md` writes every item before every `>` line. No grammar change was needed: this is the tree-walker, not the format, and duplicate headers stay legal. And **`WHERE` now carries `IS`'s word boundary** — both are compound-atomic with `sp+` (improvise-r08u), so `SUM(Rev WHEREAS x = y)` is a syntax error rather than a `WHERE` on a category `AS`. The guard is `sp+` and **not** the `!bare_char` lookahead the ticket proposed, because the grammar-walking generator reads the grammar through `pest_meta` and **skips predicates**, emitting `WHEREa=""` — a rule whose only guard is a lookahead is a rule the generator can violate. It costs one spelling: a category may no longer abut the keyword. + +35. **`Dirty` over-reports on purpose, and the asymmetry is the reason.** `Dirty::of` maps an `EffectKind` to the widgets that kind can reach, and the two directions do not cost the same: over-reporting pays one redundant `Projection`, under-reporting leaves a client's cache wrong **forever**, with nothing in the stream to correct it. `LayoutGeometry` therefore includes `formulas`, because `RemoveCategory` wears it and `Model::remove_category` ends by dropping every formula targeting the category — a client's formula panel otherwise kept showing a formula the document no longer had. It now coincides with `Dirty::EVERYTHING` and is still spelled out, because the two are derived from different effects and need not stay equal. The derivation now lives in `context/code/command.md`, and the doc comment on `Dirty::of` points there rather than restating it; the reason `LayoutGeometry` is spelled out rather than written as `EVERYTHING` is a two-line comment inside the match body. It went stale once already — claiming `AddCategory` was `CellData` after it was reclassified — so reclassifying an effect means re-reading the prose, not just the comment. + +36. **A client's command has one door, and the door takes the `App`.** `Sessions::apply_checked(app, id, Checked<'_>)` is the only way a connected client's `Command` reaches the server — `Host` calls *that* (improvise-ahdf). `Sessions::apply` is not `pub`, delegates to `apply_checked` after `Checked::against`, and has no production caller: its call sites are tests, which is why it takes a raw `&Command`. `ClientSession::fold` and the mirror's `ViewState` are private, so a transport cannot reach the mirror at all. `apply_switch` now **takes a `SessionId`** — it is addressed to exactly one session, which is the opposite of what this gotcha used to say — moves only that session's mirror, and dirties nothing. The one thing it still asks the document is whether the view exists, because a mirror keys `ViewState::cursors` and is read by `grid_in`. The audience is conditional: an accepted switch relays **nothing** (the asker folded it, and no other client's view moves), a refused one is `Broadcast::To(asker, …)` — improvise-6c6q, where `ToOthers` told every client but the one that was wrong. `adopt_active_view` and `Broadcast::ToOthers` are **deleted**; the variants are `ToAll` and `To(SessionId, _)`, the second because the deleted-view case needs a one-session audience that is *not* the asker. The paragraph's warning about the "what do the others hear" helper survives and is why the 6c6q test drives the asker's own `ViewState` — that helper is itself deleted. What survives unchanged: the model is shared and the UI is not, and `Broadcast` is a `Vec` because one command can owe two relays with different audiences. **The trap this paragraph used to describe is gone with `ToOthers`, and the lesson it taught is not.** It was copying `ToOthers` from `apply_switch` into `apply_model`, invisible from the bystander's side because `adopt_active_view` moved the asker's mirror either way — so the drift sat between the server's picture of the asker and the asker's own slice, where a test written through a "what do the others hear" helper looks away by construction. improvise-6c6q closed the mirror-image hole on the refused path. Two rules survive the mechanism: **assert the `Broadcast` itself**, since the audience is the message rather than something to infer from folded state, and **seat a bystander somewhere a wrong relay can move it from** — a bystander already on the relayed value cannot see a wrongly *widened* audience either, because the wrong relay folds to a no-op. Review demonstrated that vacuity twice on PR #48. A `Vec` rather than an `Option` because one command can owe two relays with different audiences, and an `Option` would have had to drop one silently. Everything else is that client's alone and never touches the `App`'s own view slice: **the model is shared, the UI is not.** Folding `SwitchView` into the mirror alone was the bug — `viewport()` then read the cursor under the new view's key and the next `project()` built that client a grid out of the *other* view. + +37. **`gp-tree` returns data and never prints, and a fit takes two things to reproduce.** Two rules with one root — the crate is a library a TUI, a wasm worker or a CLI may each host. (a) A decision the user should hear about comes back as **data**: `Dataset::warnings` carries the columns `preprocess` dropped for being entirely missing (improvise-d44s), and `fit.rs` hands back pooled `(prediction, actual)` pairs and a score per feature rather than tables. `examples/fit_csv.rs` owns every `println!`, and that is the only reason its output stayed byte-identical across these moves — adding an `eprintln!` to either module is the regression. (b) **The seed is necessary and not sufficient**: `data::Model` records `seed: Option` *and* `config: Option`, because the same seed under a different population/generations/elite/penalty lands on a different tree (improvise-ksst). Both are `Option` for one reason — every `FitConfig` value and the seed `0` are legal, so nothing can double as "unknown". Note `#[serde(default)]` on an `Option` field is **documentation, not the mechanism**: serde defaults it anyway, so deleting the attribute fails no test, and a backward-compatibility test that only deletes it is not discriminating. Aim such a test at an absent field loading as the *default* instead of as unknown. The GA loop itself is `fit::fit` (improvise-0ai); `breed` remains one fitness-agnostic generation for callers whose fitness is not "predict this column". (c) **A scoring function takes one slice of rows.** `Row` pairs a feature vector with what it predicts, so the parallel `x`/`y` lengths that `zip` used to truncate silently cannot be built (improvise-dkqn, `doctrine(an_invariant_is_made_unrepresentable_not_validated)` — the same answer `prune_axis_items` got, and the reason no `Result` entered five signatures). The pairing is made at the boundary that *builds* the data: `preprocess` walks one row index and the rows live inside `Target`, because a `Dataset` carrying `rows` and `target` separately just moves the zip up one level into every caller. The empty row set is one shape, answered once in `mean_over`. Changing any of this needs **bit-level** evidence: `tests/scoring_known_answers.rs` pins every path as f32 bit patterns plus a `next_u64` witness that catches a changed RNG call order, because the CLI prints three or four decimals and byte-identical output alone cannot see a moved accumulation. (d) **A fit is storable only when the document can *name* its features.** `preprocess` invents `Region=North`, `Cost.missing`, `when.year`; those are **encoding artefacts, not items of the document**, and `Expr::Ref` resolves through `Model::find_item_category` — so a tree splitting on one emitted a well-formed formula whose every cell was `Error("ref:…")`, silently (improvise-8xhy). `fitted_formula` therefore takes the `&Model` and refuses, listing what it could not resolve; `formula_text` still spells it, being the tree's own text. **The check asks the document, never the name's shape**, because a measure literally named `Region=North` is a legal document and the two are indistinguishable as strings — which also means `.missing` and the date parts are covered with no second copy of `preprocess`'s expansion rules to drift from. **This is an interim** (improvise-4zl7): a `WHERE` predicate needs a *scalar coordinate predicate* the formula language does not have — `Filter` gates a formula or an aggregate, it is not an `Expr` — would still leave `.missing` and the date parts, and must first answer what such a predicate means where `context.get(cat)` is `None`, since `Formula::filter` falls through to `aggregate_raw` and a categorical fit would silently degrade in every view that aggregates over the category. **improvise-zv8t closed the categorical half of the accident, and the close cost a document.** `fit_measure` handed `preprocess` an `Overrides::default()`, so it never said which header columns are document categories *although the very next line told `Features::of`* — and `preprocess` infers `Numeric → Date → Categorical` from the values, so a `Year` category with items `2024`/`2025` was a *numeric column* and a category of ISO dates a *date column*, both refused with `Year IS 2024` spellable all along. It now forces the same `header[..category_columns]` slice to `Forced::Categorical`. Two things follow. The **date decline is narrower than it read** — it covers a genuine date *measure*, never a category misread as one. And the forcing **widens the encoding**: a hundred-item numeric category one-hots past `MAX_FEATURES` and refuses a fit that previously succeeded by never splitting on it (improvise-e8no — and **not** to be closed by making the forcing conditional on cardinality, which is inference by another name and makes the *encoding* depend on a budget rather than on what the document says). The `Artefact` arms stay: `crate::dataset` is `pub` and takes the caller's `Overrides`, so the roles are still reachable. **Forcing by name while budgeting by index defeats the budget** — a category and a measure may legally share a name, and `no_ambiguous_column_names` ran *after* `preprocess`, so the expansion the preflight exists to prevent was allocated and only then refused; `as_categories` now returns `Result`, so the ordering is a compile error rather than a convention. The test that missed all this is the shape to remember: `improvise_eval` **fabricates** a `_Measure` item per feature name, so it asserted the emitter and `gp_tree::eval` agree against a model that could not exist. It is kept, with its scope now stated, and the property it could not state is asserted separately against a real document. + +38. **The wire vocabulary is derived twice, by the same recipe.** `Command`'s view half is the `EffectKind::ViewOnly` effects targeting the portable slice (`decision(improvise-cqi, command_set_derived_not_invented)`); its model half, `ModelCommand`, is the `EffectKind::Model(..)` effects — **28** of them, brace-matched on `c1772ca9` (2026-09-24) — minus `ApplyAndClearDrill` and `AcceptWizardModel`, leaving **26** variants (improvise-kmgs). The two added since this paragraph counted 26 and 24 are `ReplaceFormula` (improvise-km71) and `RenameRecordCoordinate` (improvise-fdk1); `SwitchView`, `ViewBack` and `ViewForward` left the model half with improvise-jytm. Every count in the rest of this paragraph is dated, so recount rather than cite it. **Completeness is now checked, not counted**: `ui/reduce/vocabulary_is_complete.rs` requires every effect to be on the wire under its own name or listed in `NOT_ON_THE_WIRE` with a verdict — `Command`, `Frame`, `Off` with its argument, or `Gap` naming the ticket that owns it (`SaveAs` and `ExportCsv` are gaps, improvise-2mqz). It is the forward twin of `vocabulary_is_composed.rs`, and it is what the enumeration below lacked. `SwitchView` no longer needs subtracting separately: it is not a model effect any more. `ViewBack`/`ViewForward` left for a reason this paragraph **stated wrongly until 2026-09-21**: it said they went for the *same* reason as the other two exclusions, session-local residue. That is false, and the falseness mattered. `ViewState::view_back_stack` **is in `ViewState`**, which is the portable slice — so by the view half's own rule (`decision(improvise-cqi)`: the `ViewOnly` effects targeting the portable slice) they were always `Command`s and were simply **missing**. The cost of the mis-statement was a live divergence nobody could see: `effect::SwitchView` pushes the leaving frame and clears the forward stack, while `reduce_view`'s arm set `active_view` **and nothing else** — so **no client ever had a back history at all**, and that, not the drill commit the `declined()` comment blames, is why `view-back` was dead in a browser (improvise-gd28, PR #115). **The lesson is about derivations, not about these two variants**: a derivation stated as a rule is only as good as the last time somebody enumerated it *against* the rule. This one had been checked for correctness — every listed variant belongs — and never for **completeness**. Note the obvious `awk` extraction reports **30**: it picks up `HelpPageSet`, which is `ViewOnly`, because that is the last `pub struct` before `#[cfg(test)]` and the test module mentions `ModelChange::` — brace-match the `impl Effect` blocks instead (`doctrine(derive_then_verify)`). Three things follow. **A `ModelCommand` does not mutate**: `ui/reduce.rs::model_effects` maps it back to the effect it came from and `apply_command` runs that through `App::apply_effects`, so effects stay the only mutation site and the `Dirty` is the effect's own `EffectKind`, never a second classification beside gotcha 35's table. **Two model effects are deliberately off the wire.** `ApplyAndClearDrill` and `AcceptWizardModel` were once kept on the grounds that the *command* is portable even where the operand is not; **that was the wrong criterion** (improvise-3x3u). A `Command::Model` is routed with no `SessionId`, which is right exactly when the change is addressed to *no one* session — and these are addressed to precisely one, never the sender: their operands are `drill_state.pending_edits` and the built wizard workbook, session-local residue (gotcha 30) private to whoever staged them (improvise-tom1). Unaddressed, B's `ApplyAndClearDrill` commits A's uncommitted typing and B's `AcceptWizardModel` swaps the document out of a dialogue B never opened — both demonstrated against `Sessions::apply` before removal. So **28** − these two = **26** on `c1772ca9` (it was 26 − 2 = 24 before km71 and fdk1), and the older spelling of this sum (`29 − SwitchView − these two = 26`) was itself a survival of the same staleness this paragraph documents: it subtracted `SwitchView` and forgot its two siblings, so it contradicted the count at the head of the paragraph and the enum’s own 24 variants (found by ChatGPT Codex on PR #116). The effects are untouched and still reached by the key path. Not permanent: multi-client drill wants the *addressed* form, which is where ADR 001 layer 2 goes anyway. Guarded by `no_model_command_commits_another_clients_session_local_state`, plus a companion asserting both effects still exist and are still model changes so the first cannot pass by naming nothing. **`Sessions::apply_checked` routes the model half to `apply_model`**, which relays `adopt_cursor_epoch`'s `ResetCursor` and `adopt_deleted_views`'s per-client switch; a document-replacing `LoadModel` bumps the generation and lands in the rebase arm. The former claim here — that `ViewBack`, `ViewForward` and `LoadModel` all end in `App::switch_to_view` so every mirror takes the accepted active view — was wrong on all three clauses after improvise-jytm and is deleted. A keymap lookup still cannot return a `Command`, and not for want of vocabulary: a binding names an *intent* resolved against `CmdContext` (`move-cursor down`) while a `Command` is the *resolved mutation* (`SetSelected { row, col }`) — improvise-gxi's AC 4 asks for something the split forbids. + +39. **`Applied` is an enum because a document swap is not a diff.** `Sessions::apply_checked` returns `Applied::Diff { dirty, broadcast }`, `Applied::Rebased { snapshots }` or — since ADR 001 layer 1 — `Applied::Rejected { reason }`, and the enum *is* the mechanism: a host that wants the `Dirty` must match, and matching names the case where a `Snapshot` is owed. The server's mirrors self-heal — `ClientSession::project` reseeds on the `document_generation` bump — but `ViewState::cursors` lives on the **client**, keyed by the outgoing document's view names (gotcha 24), and only a fresh `Snapshot` replaces it. `Command::Model` put the three `WholeWorkbook` mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. `apply_model` therefore calls `rebase` itself rather than instructing the host to, exactly as `apply_switch` adopts the accepted view itself. **The trigger is the generation moving, not the command variant**: a `LoadModel` that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a *failed* load — `a_load_that_fails_is_not_a_rebase` guards that arm. A `rebase: bool` beside `dirty` was rejected, because the host that forgets to call `rebase` is the host that forgets to read the flag. The **third** case is a *normal outcome*, not an error, and it is in the enum for the same reason the other two are: a host that wants the `Dirty` must match, and matching names it. `ServerMsg::Rejected` is the matching frame, addressed to **the asker alone** — nothing happened, so nobody else is owed anything — and the client keeps it as a `Rejection` rather than a sentence, because force needs to know *which* record and *which* kind (gotcha 43). + **Fixture warning.** When testing a document swap, the incoming document's active view must have the **same name** as the outgoing one. A rename makes `apply_model` broadcast a `SwitchView`; the client folds it, and the stale cursor is then read under a key nothing has written — right *by accident*, with the bug invisible. `other_document()` renames, and a first version of this test passed because of it; `a_document_shaped_differently_under_the_same_view_name()` keeps the name and changes only the shape, so there is nothing to hide behind. + +40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per *view shape*) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. **Only the `Dirty` is read off the `EffectKind`; both lower tiers are *observed* around the loop** — the source says so at `apply_effects` and this paragraph said the opposite until improvise-ilvl's audit. The `EffectKind` decides only *whether to look*: `touches_model` arms the shape snapshot, `writes_cells` the record fingerprints. `record_versions` **must** be observed, because an `EffectKind` says *that* a batch writes cells, never *which*; `layout_version` is observed for a better reason than necessity — `observe_reshape` diffs `view_shapes` before and after and bumps only where a view's shape actually moved (improvise-pdvm/nynk), so it is a **fact about the document** rather than a claim about the batch. Two consequences the declared form did not have. An **idempotent** `set-axis` bumps nothing, and batching two real changes into one line or two gives the same `2`. And the gotcha 21 argument is subsumed rather than restated: an aborted batch moved no shape, so there is nothing to not-claim — which is why observing once **after** the loop is correct where accumulating `Dirty` inside it is. **`view_shape` hashes what is drawn, not occupancy** (its doc comment: "Drawn, not configured") — axes, the drawn item names minus hidden ones, folds, page selections and the `prune_empty` *flag* — so an occupancy write does **not** bump `layout_version`: a `CellData`-only batch that prunes a row out of a view moves the cursor epoch (the frame observation) and not `layout_version`, as `context/code/command.md` records (the question that raised was improvise-6nsd, closed). The arming is wider than the tier (`touches_model` takes the shape snapshot), but a bump needs a shape hash to have moved. Three further consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run. + +41. **`Storage` is not a VFS, and that closes the SQLite question.** The trait is whole-document `read`/`write` bytes (gotcha 2, `invariant(storage_is_two_ops)`) — no offsets, seeking, handles or locking — so an `sqlite3_vfs`, which needs `xRead(offset,len)`/`xWrite`/`xTruncate`/`xLock`, has nothing to bind to. SQLite *can* still sit behind it, as an in-memory database serialized to a blob (`rusqlite` `serialize`/`deserialize_read_exact`, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is **22% larger** than `.improv`, **4.7× larger gzipped**, ~**160×** larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. **A second serialization format is a second implementation of the grammar**, obliged to track `improv.pest` forever; and **its own round-trip test cannot detect drift from `.improv`**, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way `export_sqlite` beside `export_csv`, native-only and feature-gated: `libsqlite3-sys`' C build fails under this repo's nix cc-wrapper for `wasm32-unknown-unknown`, which would break the `wasm` CI gate the day it landed. + +42. **The browser client's render decisions are made in Rust, and the frames it speaks are nobody's but the protocol's.** `improvise-wasm-client/src/render.rs` builds a `RenderModel` in which every question with a *silent* wrong answer is already answered, because the DOM half (`web/app.js`) is untested JavaScript and a wrong answer there shows up as a wrong pixel rather than a failing test. Three such answers. **`editable` is a fact about the `CellId` variant** — `Record` only, per ADR 002 — and the trap is the third case: `None` means "no projected cell", **not** "aggregate", so conflating them either hides a cell or offers a write the type system exists to make unconstructible. **The selection is placed by reconciling the scroll offset**, because `GridCache` is viewport-indexed while `ViewState::selected` is absolute. `render.rs` does it by *adding* — `r + cursor.row_offset == sel_row` — rather than subtracting, which is why there is no underflow path and why a selection above the window simply never matches. Omitting the offset entirely looks right until the first scroll, so the test that earns its place uses **nonzero and unequal** offsets: at `selected=(5,1)`, `row_offset=4`, `col_offset=1` it separates "reconciles" from "ignores" *and* from "swaps the two offsets". **The category panel is `viewmodel::flatten_cat_tree`**, never a second walk of `CategoryTreeCache` — that second walk was improvise-edp. The page is four string calls: `Client::viewport` (the handshake — the `Snapshot` seeds the *visible* cells, so one built at extent zero is an empty grid and a resize is a fan-out with nothing dirty), `on_frame` (one frame, one fold, arrival order — gotcha 33 holds *inside* a batch, so never coalesce), `render_model`, `typed_cell`. + + **One envelope pair, two deployments (improvise-xks6).** `improvise_protocol::frame` owns `ClientFrame { Viewport, Commands(Vec<_>), Save, Key }` and `ServerFrame { Snapshot(Box<_>), Commands(Vec<_>), Rejected, Accepted }` (as of `c1772ca9`; this line named a singleton `Command` and no `Save`/`Accepted` for some time after both changed); `improvise-ws-server` and `improvise-wasm-client` each `pub use` them and **neither defines one**. There used to be **three** pairs for two deployments — the socket's server side, a hand-kept *mirror* of it in the browser bundle, and a third written for the worker (improvise-tm6) before the projection architecture landed. The question that had to be answered before any of it moved was whether the **worker** and the **socket** want the same vocabulary, and the answer is **yes**: every difference was a *date* rather than a decision, and the test applied to each was whether the **argument** for it names the thing they differ in. None did. `Viewport` is deployment-independent — the extent is not in `ViewState` and no effect produces it, so no derivation reaches it, and a worker hosting the same `Sessions` owes `set_extent` the same answer; it is the handshake on both because a `Snapshot` seeds the *visible* cells. `Key` is deployment-independent too — a binding names an *intent* resolved against a `CmdContext` a thin client does not hold (gotcha 38), which says nothing about what the bytes travel over; the socket lacked it because key forwarding was never built on it (improvise-bpq2), and `Host::handle` now answers `ClientFrame::Key` with an explicit empty vec that **says so**, where the frame used to just fail to parse in `next_client_msg`. And `Commands(Vec<_>)` batches where a singleton cannot, because a fan-out must arrive whole and in sequence (gotcha 33) — a property of the **fold**, not of the transport; an in-process `postMessage` interleaves exactly as a socket does. The worker pair was not merely different but **incoherent by omission**: written before ADR 001, it could carry `Command::Model { basis, .. }` and had no `Rejected`, so a client speaking it composed a claim that could never be answered. That incoherence — not the difference — is the tell for staleness. + + **The mirror is gone, and so are the two tests that held it honest**; that deletion was the ticket's measure of success. `the_socket_framing_matches_the_server` round-tripped hand-listed fixtures through the other crate's types to catch a **rename**; `every_server_variant_is_one_this_crate_mirrors` matched exhaustively to catch a **variant added on the server and never mirrored**. With one definition both are **type identity**: a rename is a compile error at every call site in both crates, and the un-mirrored variant is not a constructible state. That is stronger than either test, and it is the shape to insist on — **if a consolidation leaves one of its compatibility tests alive, the consolidation is not finished.** What survives is `the_retired_spellings_do_not_parse` in `frame.rs`, aimed at the one thing still possible (the retired `Bootstrap`/`Command` spellings returning as serde **aliases**, which would be two homes again wearing one type), and it keeps the injected-fault discipline this paragraph earned the hard way: **assert the mutated input is still well-formed** before feeding it in, and **assert the error kind** — serde's `unknown variant` — not merely that an error occurred, because a rewrite that also breaks the JSON makes the check pass against an enum that really does have the variant. End-to-end compatibility is now `tests/browser_round_trip.rs`, which hands the wasm client's own serialized frame to a real `Host::handle`; `improvise-ws-server` stays a **dev**-dependency of the browser bundle for exactly that. + + **What the harness does and does not reach is gotcha 48's subject** — `web/app.js` is no longer unrun (`harness/tests/shell.spec.js` executes it under Chromium, and its standalone half under Firefox), improvise-hwpi is closed, and the quiet-failure and never-reads-a-key claims that used to stand here went with it. What remains unexercised is named there: the grouped and multi-level DOM, the CSS and the `ResizeObserver` extent arithmetic. What a careful read *did* establish about the last of these: the observer cannot feed back into itself (`#grid` is a flex child with `overflow: auto`, so `replaceChildren` cannot resize it), and `ROW_PX`/`COL_PX` under-estimate, so the worst case is a row of over-fetch. What *is* verified end to end is the Rust half: `tests/browser_round_trip.rs` drives improvise-1ey's stated scope against a real `Host` over a real `App` touching the client only through the four string calls. One deployment note that is not a bug: `examples/demo.improv` leaves `Customer`/`Date` on the None axis, so **every** cell is an aggregate and nothing is editable — correct behaviour, bad demo. Pick a model with no regular category on None. + +43. **A batch carries its basis, and the server had to publish it first.** `Command::Model { basis, cmd }` is a **struct variant** so the basis cannot be forgotten, and `Sessions::apply_model` checks it *before* applying anything — a stale batch is never applied, not even partially (improvise-xwi6/fkfg, ADR 001 layer 1). Four things to know. **`Basis` has no `Default` and no public constructor**: a zero-valued basis is not "no claim", it is the claim that *the document is fresh*, which in a new `App` is true — so a default would be a forgeable bypass rather than an absence. `BasisCache::claim` is the only door a client has and it returns `None` until the server has published one. Note what that does and does not buy: `Basis` derives `Deserialize` because it must cross the wire, and its fields are public, so **a hand-rolled client can state any tiers it likes**. This is an *anti-staleness coordination check, not a security boundary* — `RecordUnstated` closes **omission**, not **invention**, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having: it makes the honest path the easy one. **Which records are checked is the server's decision** — `reduce.rs::records_named`, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the *client's* claim would be a **force by omission**: the record tier skipped, with no flag to remove and nothing in the message saying so. A claim that omits a record the command writes gets `Rejection::RecordUnstated`, a case of its own because "you lost a race" and "you were never in one" are different things to tell a user. **`Rejection` keeps structural and data staleness apart**, which is the whole reason ADR 002 put a maintained `RecordHash` index in `DataStore`: `RecordGone` means nothing resolves and there is nothing to force (**force never creates structure**), `RecordMoved` means it resolves at another version and force is one keystroke — which is also why the wasm client stores the `Rejection` itself rather than collapsing it into `status_msg`, since a string cannot carry which record or which kind. **And the ADR omitted the publication.** It said a batch carries the basis it was composed against and never said how a client learns one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. `Projection::Basis` + `BasisCache` close that — the server publishes the two document tiers plus the counters for the records *this client's grid names*, diffed against what that client was last told and seeded in the bootstrap — and it is emitted **after** the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown, the mirror image of gotcha 33. That ordering was documented as load-bearing in three places and **held up only by the order of two statements** until `a_published_basis_follows_the_grid_updates_it_describes` pinned it (proved by injecting the swap) — the same job `a_cell_update_that_overtakes_its_labels_is_lost` does for the mirror-image rule. Two things are unchecked and deliberately so: `Command::SwitchView` carries no basis (it sits outside `Command::Model`; the workbook refuses a view it does not have, so it cannot corrupt, but it is a hole in the rule), and `apply_command` does not check at all — **the check is a property of the session door, not of the mutation path**, which is right because a host holding the `App` is the truth rather than a claimant. That second one **was** a door rather than an exemption, and needed a type to stop being one (improvise-ondj): both took a wire `Command` and their `Command::Model { cmd, .. }` arm **discarded the basis with `..`**, while `ui/reduce.rs`'s own module doc pointed every future host — the browser worker included — at exactly those two names. **The struct variant closed omission *from the message*; it did nothing to close ignoring *on arrival*, which is the half a struct variant is usually reached for.** So both now take `Checked<'_>`, a private-field newtype over a `&Command` whose only constructor is `Checked::against(&App, &Command) -> Result<_, Rejection>` — also the only caller of `stale_basis`. `Sessions::apply_checked` receives the proof and `Checked::against` is answered **once, before the dispatch**, so `apply_switch`/`apply_model` receive the proof rather than each remembering to ask: an invariant made unrepresentable rather than validated, the same answer `prune_axis_items` and `gp-tree`'s `Row` got. Visibility was the weaker option and was rejected for a concrete reason — `pub(crate)` closes only the out-of-crate path, and `reduce_full` had **no production caller anywhere in the workspace**, so it would have become dead code. It has since been **deleted** — which is the argument’s own conclusion arriving, not a contradiction of it; the surviving sibling is `reduce_key`. This paragraph named it in the present tense for some time after, so read every function name here as a claim to check. **The regression guard is therefore the signature**: the test that reproduced the bug no longer compiles, and what remains asserts the constructor refuses and that refusing leaves the document untouched. One residual hole, documented on the type and filed: a `Checked` does not name *which* `App` it was checked against, so a multi-document host could in principle check one and apply to another. That is also why `ui::reduce::caught_up(&App, cmd)` is not a bypass: it takes `&App`, so nothing reachable from a wire can call it. **Since improvise-955n the frame, not the command, is what is answered — and only two of the three tiers hoist.** `CheckedBatch::against` runs before anything in a `ClientFrame::Commands` is applied. `document_generation` and `layout_version` describe the *document*, so answering them once for the frame is exactly right, and a first version hoisted all three — which was a lost update: the record tier asks "is the client's picture of **this record** current", and which records are at stake is a property of the *command* (`records_named`). Skipping it for later commands let `[SetCell(North), SetCell(South)]` overwrite a South somebody else had moved, silently, with the frame reported accepted, and invisible to the other two tiers because a `SetCell` is `CellData` and moves neither. So every mutation answers for its own records **against the frame's basis** — the one the first basis-carrying command stated, never each command's own field, or a later command could claim a fresher picture than the frame was composed under. The one exemption is stated over a *property* rather than a list of commands: a record that **neither resolves in the document nor is claimed by the basis** is one the frame is about to create (`[AddItem(East), SetCell(East)]`, which `BasisCache::claim` cannot possibly claim for), and only that combination is exempt; a record that does not resolve but *is* claimed is still `RecordGone`. That the `Basis` inside a *later* `Command::Model` is ignored rather than compared is an honest interim: clause 1 says the **frame** states one basis, and moving the field from `Command::Model` to `ClientFrame::Commands` would make that unrepresentable-otherwise instead of a convention — a protocol change that belongs with clause 4. Two things worth keeping from the rejected design: **arity is not a property of a variant** (a future `Command::RunScript` is one variant performing unboundedly many mutations), so no count over the vocabulary can enforce "one mutation per frame"; and **the transaction boundary is not the frame**, because one command may already be a chain, which is why clause 4's mechanism belongs at `App::apply_effects`. + +44. **The TUI grid draws from its cache, and since improvise-anr9 the cache frames it too.** `GridWidget` holds no `&Model` and no `&View`: cell text, aggregate-ness (`CellId::Aggregate`, ADR 002), column widths, the page bar, the totals strip and now the **frame** all come off `GridCache`. The TUI still builds that cache over the **whole** grid (offsets zero), so a cache position is an absolute position — the browser's `Viewport` is a real window, the TUI's is not, and the two clients therefore use it differently; column totals **must** be projected rather than client-derived, because the sum runs over every row *including the ones a viewport scrolled past*. The frame is an `AxisFrame` per axis — `cats` (the categories, outermost first) plus `entries: Vec` interleaving `Group(FrameGroup)` headers and `GroupEnd { level }` markers with `Item { tuple, collapsed }` label *tuples*, exactly as `GridLayout::row_items` interleaves them. That replaced a `Vec` built by `GridLayout::row_label`, which slash-joined the tuple — the exact rendering `two_row_categories_produce_cross_product_labels` says must not appear — so the browser drew `Food/Alice` where the TUI drew two header columns, with no groups and no collapse indicators. It is **one** type and not a `labels` plus a `levels` beside it: two answers to "what are the labels" is the disease (improvise-edp), and this is the second cure, which cost a *type* change rather than an addition. Three things to know before touching it. **`leaf_labels()` is not "the labels"** — it is the innermost label, what a one-line summary calls a row by; a draw loop wants `header_labels(first_rendered)`, `groups()` and `shares_ancestry(sel)`, each of which is one pass, because `item`/`label`/`shows_level`/`group_of` scan `entries` from the start and asking them per position is quadratic in the frame *and grows with the TUI's absolute scroll offset* (found three times on anr9; the per-position forms survive for one-off lookups and say so). **The width rule has one home**, `improvise_protocol::header_level_widths` — widest label per level, plus a gap, clamped — reached by `AxisFrame::header_widths` and by `ui::viewport::compute_row_header_width`, which borrows level tuples off a `GridLayout` rather than building a frame on the keypress path. **A `cats` change re-identifies every position**: a cell's identity is `cats` zipped with its tuple, so `invalidate_relabelled` blanks the whole axis when the categories move, not merely the positions whose tuples differ, and blanks the **totals** on a recategorization of *either* axis — a column total is a sum over the rows, so re-identifying the rows leaves a number answering coordinates the grid no longer has (gotcha 33(b): the rule lives in `apply`, not in a producer's memory). What is still **not** on the cache is `record_col`, and that is correct: it keys pending drill edits and is session-local (gotcha 1). `body_from(idx)` takes an argument only because the two front ends scroll differently — a browser's frame is already windowed and passes `0`, the TUI's covers the whole grid and passes its row offset — not because the rule differs. `group_of` answers group **membership**, and since improvise-e265 membership is **the innermost open run**: `FrameGroup` and `FrameEntry::GroupEnd` each carry a `level` (an index into `cats`), runs **nest**, a header at level L replaces L and closes every deeper run, and a `GroupEnd { level }` closes L and deeper. Before e265 it was the nearest preceding header unless a level-less `GroupEnd` intervened (improvise-gd83, PR #77), which could not express one run inside another. `GroupEnd` draws nothing and exists only so membership is not inherited. `AxisFrame` deserialises through `UncheckedAxisFrame` and **refuses** a `level >= cats.len()` — a level naming no category is a malformed frame, not a clamp. And `FrameEntry::Item` is `{ tuple, collapsed }`: `collapsed` is the **layout's own fold flag**, read by `AxisFrame::folds`, so a client asks the frame whether a position folds rather than re-deriving it from the groups (improvise-4k1j). Both of the gaps this sentence used to list are **closed**, and it named them as open for a day after: `group_runs` coalesces by **header identity** (`std::ptr::eq`), not by name, so two runs under two headers equal as values are two runs (improvise-h0iv); and `AxisFrame::first` carries **no** `serde(default)` — its doc comment says why, since zero is a claim and not an absence (improvise-1wl0). `doctrine(a_stale_list_of_what_is_missing_is_worse_than_no_list)` is about exactly this paragraph, twice now. And `project::grid_of` used to evaluate every cell per frame where the widget evaluated only the visible ones; since improvise-felo.13 the host's own layout keeps a `GridSummary` (whole-grid widths, column formats and totals, `ui/viewport.rs::grid_summary`, held in `App::grid_geometry()`), and `grid_in` evaluates only a `grid_window` over it. The browser's window therefore costs its own cells. A session on another view, or a rederived model, still takes the whole-grid `grid_body` path. `grid_window` equals `grid_body` where the summary is current. Since improvise-jytm `grid_in` may additionally **derive a `GridLayout`** for a session on a view the host is not on; the host's layout is reused where the names agree, which is the common case and the only one that can carry a drill snapshot. + + **`RecordHash` is document-scoped; `AggregateHash` is layout-scoped and cannot be otherwise** (improvise-gd28). A `RecordHash` resolves through `DataStore`'s maintained index — but **only when the command names a view the document has** (improvise-wpxr; Codex P2 on #281). `SetCell`/`ClearCell` carry `view`, and the lookup is **strict**: a view the document lacks drew nothing, stored or not, so it is `Rejection::RecordGone` rather than a `resolve_view` fallback. An **empty** record — one the store holds no cell for, which the grid still drew — resolves against **that named view's layout**, the only place it exists. `RenameRecordCoordinate` is the exception and resolves in the **store alone**, since only a record holding cells can be renamed: `protocol::named_records` answers each command's records as `NamedRecord::Drawn { view, record }` or `NamedRecord::Stored(record)`, one exhaustive match. Every one of these questions goes through `App::grids()`, a `Grids` borrow that memoises **one derived `GridLayout` per view for as long as the borrow lives** (improvise-kimw) — hold one across a check, never across a mutation, since the layouts it holds describe the model it borrowed. An aggregate has no existence outside the layout that derives it, so there is no index to fall back to and none can be built — `App::resolve_aggregate` therefore answers only for the **host's** active view, which since improvise-jytm is not the client's. **That, and not ADR 001 layer 2, is what blocks browser drill.** The ticket's blocking argument was right about the *commit* (`ApplyAndClearDrill` is addressed to one session, gotcha 38) and wrong about the *entry*: entering a drill needs no commit, and `CreateView`/`SetAxis`/`SetPageSelection` are already unaddressed `ModelCommand`s that name their view. The gap is one level down from where it was looked for — **the mutation and its operand have different scopes**: the mutation configures the shared `_Drill` and is addressed to nobody, while the operand exists only inside one session's layout. So the fix is not a new mechanism but the `view: String` convention every other view-scoped variant already follows, resolved against a layout derived for that name (improvise-obtu). Two things travel with it: editing *inside* a drill grid from a browser does not follow from it, because `edit_or_drill` declines records-mode editing out loud before reaching a cell (improvise-fdk1, since closed, is the separate records-coordinate rename on the wire — `RenameRecordCoordinate`, gotcha 38 — and did not unblock drill editing); and closing it makes improvise-t219 live, `_Drill` being one shared view in `Workbook::views` that two clients drilling would contend for — the same masking relationship improvise-6uva has with improvise-up8g. Guarded by `an_aggregate_a_session_was_shown_does_not_resolve_against_the_hosts_view`, whose **control** is the load-bearing half — the same surrogate resolves once the host is moved onto that view, so a blanket `None` cannot pass. + +45. **One session loop, two transports — and the browser saves itself.** `Host` is `App` + `Sessions` + `handle(id, ClientFrame) -> Vec` and lives in `improvise_command::ui::host`, **not** in `improvise-ws-server`, which re-exports it. It moved for improvise-9x6: the worker hosts the same authoritative `App` over a `MessagePort` and cannot depend on tokio, and a second copy would have been exactly the mirror improvise-xks6 had just deleted for the frames — a drifting `Applied`/`Broadcast` match is invisible until a client sees it. `registry` stayed on the server, because a worker hosts **one** document. What the worker adds is only what a `MessagePort` differs in: **one client by construction** (the `Addressed` filter is honoured anyway, against a future `SharedWorker`), and **a handshake with no close** — `ClientFrame::Viewport` must still come first, but a pre-handshake frame is *dropped* rather than closing a connection that does not exist. + + **Saving is a frame, and dirtiness is read off the classification the effect already carries.** `effect::Save` is `ViewOnly` (gotcha 3) — correctly, since it writes *this host's* `file_path` through *this host's* `Storage` — so the derivation that builds `ModelCommand` from the model effects cannot name it, and for a while that meant no client on any transport could ask its host to save. `ClientFrame::Save` closes it (improvise-368h): a **request** carrying no path, because a client cannot see the host's storage, answered `ServerFrame::Rejected(NoDocumentPath | SaveFailed)` to the asker alone and **silently on success** — nothing moved, so no widget is dirty. It is a frame and not a `Command` for one concrete reason: as a `Command` it put a runtime-fallible operation inside a batch `ClientFrame::Commands` promises is all-or-nothing, so `[Save, SetCell]` reported a failed write and then applied the edit. It also fails the test every `Command` passes — it mutates no shared state and no client's slice, so `reduce_view` has nothing to fold and `Checked` has no basis to answer for. **And `MarkDirty` is gone.** It was a separate effect the *command* layer composed beside each mutation — reasonable on the key path, unreachable from anywhere else, because improvise-kmgs derived `ModelCommand` from the effects **one at a time** and so put `SetCell` on the wire without its companion (improvise-gv9r): a derivation correct per element and wrong for a set, whose symptom was that "save if dirty" never fired. `App::apply_effects` now raises the flag from the effect's `EffectKind`, which makes omitting it unrepresentable rather than validated — **declared** for `WholeWorkbook` (a replaced document differs from the file its path names, so a `LoadModel` that cannot read its path still raises it, costing one redundant write) and **observed** for `CellData` and `LayoutGeometry`, where `document_content`/`document_config` fingerprint the document before and after, so a write that leaves it as it was raises nothing (improvise-748f). The observation is read inside the loop, and `Save` clears the flag in loop order, so a batch that edits and then saves ends clean. The worker's persist trigger is therefore the **flag**, not a table of command variants: an observation rather than a second description of which commands change the document, for the reason `cursor_epoch` beats matching on `CycleAxis` (gotcha 24) and `document_generation` beats matching on `LoadModel` (gotcha 39). It is also strictly tighter — a refused batch raises nothing, and neither does a write that changes nothing, so the "was anything rejected" guard went with it. A browser still never *sends* `ClientFrame::Save`, and that is the answer rather than a gap: there is no file the user owns and no Ctrl-S semantics, the document *is* the OPFS entry, so the worker persists on its own behalf through the one existing `Save` effect. The cost that remains: `Storage` is whole-document (gotcha 41), so OPFS rewrites the file per accepted model command. + + **The async seam does not survive being boxed.** `OpfsFs::open`/`attach`/`refresh`/`commit_deletes` are on the struct, not on `Storage`. `App::storage` is a `Box` and a trait object can name none of the four, so **handing the backend to the `App` puts the async seam permanently out of reach**; the host must hold the concrete backend *and* give the `App` a `Storage`, which means sharing one. That is `impl Storage for Rc`, and it lives in **`improvise-io`** rather than in each worker (improvise-hmws): `impl Storage for Rc` is two foreign types in any other crate, so improvise-9x6 wrote a forwarding newtype that every future worker would have written again. `Rc` and not `Arc` because the one caller is a Web Worker and wasm32 has no other thread. **The flat root is no longer load-bearing**: `OpfsFs::list` joins `dir` like the other two backends, so `storage_contract` takes `dir` as a parameter and the three agree at *every* directory rather than only at `Path::new("")` where joining is the identity (improvise-vhcz). What the index holds is **documents, not entries**, and the split between its three writers is the rule: `attach` indexes nothing (what it creates is zero bytes), `Storage::write` indexes the name and un-queues any pending delete of it, and `refresh` rebuilds from OPFS skipping zero-byte entries — counting an entry it cannot measure as a document, since the one cause of that is another tab holding the exclusive handle (improvise-vvl3). + + **What no test here can see** has shrunk, and precisely. `harness/` (improvise-vvl3/xf1) runs `improvise_io::storage::contract::storage_contract` against a real `OpfsFs` inside a real dedicated Web Worker under Chromium and Firefox, and the `browser` job in CI can fail the build — so **`OpfsFs` is no longer compile-checked-and-never-run**, and two bugs fell out of its first execution. That is why `storage_contract` stopped being a `#[cfg(test)]` helper and became a public function returning `Result`: a browser must run *that* sequence, not a second transcription of it, and a failure has to travel out of a worker as a string. Still unseen: **Safari** (Playwright's WebKit has no OPFS, so xf1's criterion is met for two engines of three; `opfs.spec.js` skips on a runtime probe and an unexpected skip on any other engine is a failure). **`bindings.rs` is NOT in that list, and said it was for months.** `shell.spec.js`'s "the standalone deployment" loads the app with no `?ws=`, which spawns the worker, which imports the glue, which calls `bindings.rs::start` and `OpfsFs::open` — and the spec then asserts a grid draws, so the whole chain has run in CI since improvise-vvl3 (Chromium, and Firefox since 2026-09-20). The claim survived in **five** places (this line, the File Inventory, two `kb.pl` facts and improvise-bck's grooming note) because each was written before the harness landed and none was re-read after; the spec's own comment says the chain plainly. It is the reason to distrust a "never run" written in prose: nothing checks it, and `kb.pl`'s CI gate checks consistency rather than truth. What was already verified stays: `tests/post_message.rs` drives a real `Worker` over a real `App` on nothing but the JSON a `MessagePort` carries. Build note unchanged: the worker bundle needs `--enable-nontrapping-float-to-int` **as well as** `--enable-bulk-memory`, because it carries the whole `App` and so chrono's date arithmetic. + +46. **The page picks a peer, and the demo was built against an assertion.** `web/app.js` holds two transports and no protocol knowledge: `?ws=` selects `improvise-ws-server`, its absence spawns `new Worker('./worker.js?doc=&seed=', { type: "module" })` (improvise-djm). `type: "module"` is mandatory — `worker.js` `import`s the bundle — and there is **no `MessageChannel`**: a dedicated worker's own `postMessage`/`onmessage` *is* the port, so the ticket's channel wording was a date rather than a decision. The websocket path is **kept**, because the ws-server is how a TUI-hosted document is shared and nothing else in the browser can reach it; the cost is contained because the two differ only in *where strings come from* — handshake, fold, repaint and keyboard are all above the seam — and **the presence of the parameter is the whole selection mechanism**, a second knob being a second thing to get wrong. `#status` composes **two** facts, the transport's and the document's, because a single source let a repaint silently erase a connection error. **A worker that rejects in `start()` fires no `error` on the main thread** — `worker.js` catches its own — so the guard is a **first-frame watchdog**, deliberately *not* per-transport: "the peer never answered the handshake" is the one failure that means the same thing over a socket and over a port, and a check aimed at each peer's own failure modes would have missed exactly this one. + + **The demo is a model built to satisfy a check, not an axis flipped and hoped for.** `examples/demo.improv` leaves `Customer`/`Date` on None, so every cell is an `Aggregate` and nothing is editable — and the naive fix does not work either, since `Product × Region` by `Date_Month × Measure` still aggregates. `web/demo.improv` puts **exactly one category on each visible axis** (`Product` row, `_Measure` column, `Quarter` page, nothing regular on None) and carries `Profit = Revenue - Cost`, so the payoff is on one screen with no keystroke. **Settability is asked of `grid_of` as a `CellId` variant**, never re-derived from the axes — a second derivation could agree with itself while disagreeing with `render.rs::editable`, which is the whole point of ADR 002 — and the assertion was proved discriminating by pointing it at `examples/demo.improv`, where it names the aggregate at (0,0). **One category per axis is a property of the demo, not of the format**: when this was written `GridCache` slash-joined a multi-level tuple, so a second category on an axis put that known gap on the front page — improvise-anr9 has since closed it (gotcha 44), so what remains is the demo's own simplicity rather than a rendering limit. `web/*.improv` is un-ignored in `.gitignore` (`*.improv` is ignored repo-wide) and the CI assemble step copies it: leaving it out is an empty grid, not a 404 the page could report. One wart it now puts on the front page: a formula cell **is** a `Record`, so it is offered as editable and a write to it reverts on recompute — pre-existing and matching the TUI, newly visible. + +47. **Keys become commands in the browser, and the server must not know the keymap.** The user's ruling on improvise-bpq2, following from "the model is shared, the UI is not" — and it buys two clients different bindings over one document. What made it urgent is **not** latency. `a_key_resolved_on_the_host_moves_state_no_client_can_hear` shows nothing in the projection stream carries the host's own `ViewState`, so host-side resolution **diverges**; and `Host::handle` answers `ClientFrame::Key` with an explicit empty vec, so **an upstream key is a black hole, not a slow path** — every `KeyOutcome::Upstream` names a binding that does nothing at all in a browser. That test still holds and was not flipped: bpq2 closes the divergence by *avoiding* it. **The method is the ticket's** — enumerate the **bindings**, not the commands (`doctrine(binding_dump_diff)`) — and where a client cannot compose the `Command`, **enrich the projection** rather than moving resolution back. + + Three things that cost. **`GridExtent` on `GridCache`** — the whole grid, of which the labels are one window — because a viewport-indexed cache cannot tell "the last visible row" from "the last row". It is deliberately **not framed by the labels** (`fit_to_labels` leaves it alone) and its batch position is not load-bearing the way `RowFrame`-before-`Cells` is, since it addresses no position; the producer reads `GridLayout::row_count()`, and reading it off the projected ranges is **invisible in the TUI** (which builds at offsets zero, gotcha 44) and is exactly the browser bug. **The window is the client's own fact**, recorded by `Client::set_viewport` from what it announced — *not derivable*, since the row frame's own length is the window **clipped to the grid** and so differs precisely at the bottom edge where the scroll arithmetic matters. `Client::viewport` (associated, no recording) still exists and is the one that quietly turns cursor keys off; `web/app.js` does call `set_viewport`. **`ClientFrame::Commands(Vec)`** replaces the singleton, because one cursor key is `SetSelected` plus the offsets `viewport_effects` returns and a host that folds the selection but not the offset projects the wrong window. It is **answered whole, then applied whole** (improvise-955n): `CheckedBatch::against` checks the frame before anything runs, so there is no longer a moment at which part of a frame has landed and the rest has been refused, and `Applied::Rejected` is unreachable through `Host::commands` — still matched, because a host that wants the `Dirty` must name every case (gotcha 39). The contract is four clauses and **all four are now delivered** — clause 4 by improvise-yqpd on 2026-09-21, whose mechanism is `App::snapshot`/`restore` plus a `Sessions` clone taken in `Host::mutating_frame` and put back by `undo_frame`, deliberately *not* inside `apply_effects`, which runs on every keypress. **This paragraph said it was undelivered for a day after it landed**, and an orchestrator brief repeated that from here, so read the clause list as a claim to check. A delivered clause 4 is also what makes the CLIENT's rollback well-defined (improvise-nxah): a refusal means the host applied none of the frame and restored every mirror, so a client keeping its optimistic fold is the only thing left out of step. What rolls back there is `ViewState`, never a cache — a `Command::Model` is a no-op in `reduce_view`, so the optimistic fold never reached one. The old text, kept because the hazard it names is still the right shape to fear: one basis per frame, checked once, applied in order with each command seeing the last one's state, and all-or-nothing. was that the fourth was open because `App::apply_effects` aborts without undoing (gotcha 21 — an abort is not a rollback), so `[RemoveItem("North"), SetCell(the North record)]` would land the removal, resolve the write to nothing, and report the frame accepted having applied half of itself — partial *and* silent. That is what `yqpd` closed, and it closed it **outside** `apply_effects` for the reason the old text implies: the abort lives on the keypress path and must stay cheap. The old note that it was *unreachable* from any client here, the only multi-command producer being `keys::commit`, still reads correctly as a caution — **reachability is a reason not to rush a fix and not a reason not to notice one**, the lost update `CheckedBatch` fixed having had the same reachability argument and none of the documentation, which is how it shipped. + + Two corrections to the ticket's own gap list, both worth more than the code. **Panel cursor movement was never a gap**: improvise-edp had already made `viewmodel::flatten_cat_tree` the one home both ends call, so it is the *shared* derivation, not a second one — worth re-checking the "known gaps" in other old tickets against what has landed since. And the "unmodified characters only" rule was **mis-scoped, not wrong**: it was about `KeyPattern::AnyChar`, which is bound only in the text-entry modes, so `Ctrl-d` in Normal has an exact binding and mirroring it is a faithful subset. Each resolver mirrors its `Cmd` **line for line, including where siblings disagree** — `Move` and `MoveTileCursor` emit even when they clamp, `MovePanelCursor` returns nothing when the cursor does not move. Tidying that into one rule would make the client a second derivation, which is what `tests/lockstep.rs` exists to rule out. What still needs the host is **one list**, `resolve::NEEDS_THE_HOST` in `improvise-protocol/src/resolve.rs` (improvise-uln5) — this line named a list in `keys.rs` and a test called `the_normal_mode_keys_that_still_need_the_host_are_these` for some time after both had gone. Three tests hold it, one per direction: `the_command_names_that_still_need_the_host_are_these` (no **false** entry: each name is bound and really is silent), `a_help_row_the_browser_cannot_run_is_drawn_marked` in `wasm-client/render.rs` (the help overlay **marks** those rows, improvise-uln5), and `every_silent_binding_is_on_the_list` (every binding of **every mode a browser can reach** acts, declines out loud, or is on the list — improvise-oyaa, improvise-s2gb). The last replaced a check read off the help rows, which could not see a binding no help row names. Closing a gap therefore means deleting its entry, and the first test fails until you do. + + **What the browser client decides before a frame leaves it** (all in `improvise-protocol`, so both ends share it). Five rules, each bought by a bug: + - **Every client cache read goes through `projected_at`**, which subtracts the frame's own origin (`AxisFrame::first`) before indexing. A position outside the window the cache holds is **unknown**, never the cell at the same cache index (improvise-7mro) — once the client scrolled ahead of its cache, an index read answered a different cell's value. `unprojected` names that state, and a cell op on it waits rather than guessing. + - **The keyboard commit writes `ClientState::opened`**, not the cursor. `OpenedCell` captures the record, the `OpenedTarget` (`Value { measure }` or `Coordinate { category }`), the record's version, the `document_generation` and the view **at open**, so a commit lands on the cell the editor opened on however the cursor moved since (improvise-8xq0), and claims the basis it was **opened** against rather than one read at commit, which silently overwrote a value another client wrote while the editor was open (improvise-xz2z). A retry after a crossing `LayoutMoved` must match the **whole** `OpenedTarget`, not the record alone, or it wrote whatever cell was under the cursor by then (improvise-8phw for values, improvise-9ww8 for renames). + - **`refused_here` exempts a captured editor** (improvise-3h9w): with `opened` set, `commit-cell-edit` is answered by the capture, so the cursor's current cell cannot refuse a commit aimed elsewhere. + - **A decline that must stop a binding is `Step::Declined`**, not a `SetStatus` beside the binding's other steps: Enter's welded `clear-buffer` otherwise still ran and threw away the typed text (improvise-joed), and a touched editor over a declined cell went `Upstream` with no word at all (improvise-ik6l). + - **`ClientState::notify` queues a client-side notice at the tail of `in_flight`**, as if it were a sent frame, so a rebase that rebuilds the view from `settled` and replays `in_flight` replays the notice too instead of erasing it (improvise-n0pb). `settle_status` runs on **every** path that rebuilds the view (bootstrap, accept, refuse, relay), and assigns a message the editor open as it arrives, so an old message loses its owner once that editor closes (improvise-0dbk). `EditorSession`, the key it is owned by, lives in `improvise-protocol/src/lib.rs` beside `ViewState` so both front ends ask it (improvise-su4u, improvise-1x9h). + +48. **The browser JavaScript is argued *and now partly executed*, and the two rules it can break are both about *when*.** `web/app.js` and `web/worker.js` were syntax-checked, `.d.ts`-checked for call arity and read — never run — and every bug in them was found by reading. `harness/tests/shell.spec.js` is the first execution of either: it runs `app.js` under Chromium against a mocked peer, and it exists because PR #21 declined to add regression tests for its own close-listener fix on the ground that there was nowhere to put them. That decline is only honest because the assertions live there now, and **a change to `web/*.js` belongs in that spec directory** rather than in a second ad-hoc surface beside it. Playwright rather than a headless-DOM shim because what goes wrong in these files is a real Worker, a real WebSocket, real OPFS and real event ordering, none of which jsdom has. Two classes of bug still follow from *when*, both found by review. **An `async` `message` listener does not serialize.** `worker.js`'s `handle()` awaits `server.flush()`, so the event loop may start the next message while an earlier one is suspended, interleaving two `on_message`/`flush` pairs and letting a later frame's replies **overtake** an earlier frame's — which gotcha 33 answers by *dropping* the overtaken update, not by replaying it. Every frame is therefore chained onto **one tail promise** and the listener is deliberately not `async`; that also turns "frames that arrived before `start()` finished" into an ordinary case of the ordering rule instead of a drain loop, which had the same race twice over because `server` was assigned *before* the loop ran. **A watchdog must be disarmed, not pre-empted.** The handshake watchdog exists for one condition — the peer never answered — and every *loud* failure (worker `error`/`messageerror`, socket `error`) has already written something more precise; the timer then fired 8s later and replaced it, so `setShellFailure` clears it. Its neighbour, the socket's `close` listener overwriting a precise `error` with "disconnected", **is fixed** (improvise-taxx, PR #21) — and the ticket's own proposed mechanism was wrong: `readyState` cannot discriminate, an `opened` flag is the only thing that can. PR #21's *other* half is the one that keeps getting lost: it routes **both** close arms through `setShellFailure`, which disarms the watchdog, so a genuine mid-session disconnect is **not** overwritten eight seconds later. A bug report said it was, measured it honestly against a pre-#21 tree, and then "confirmed" it survived #21 by simulating the `opened` flag **and not the routing** — a simulated fix that is not the fix you named reproduces a bug the real code does not have, and reads exactly like a confirmation. The ticket was filed and closed invalid; the property is now asserted by `a socket that opened and then dropped reads as disconnected, and stays that way`, which runs past the 8s window, rather than argued in a comment that was wrong twice. + + **What the harness still cannot see**, and why the argued-not-tested discipline stays: the grouped and multi-level DOM, the CSS, and the `ResizeObserver` extent arithmetic. What a careful read established about the last of these — the observer cannot feed back into itself (`#grid` is a flex child with `overflow: auto`, so `replaceChildren` cannot resize it) and `ROW_PX`/`COL_PX` under-estimate, so the worst case is a row of over-fetch — is still a read and not a test. + + **A third thing no test here can see, which is why the docs must be read as code.** Three comments in one range said "not built yet" about work that had landed *in that same range* — `README.md`'s browser section, `app.js`'s keyboard note, and `Host::handle`'s `Key` arm — each because a follow-up commit updated **one copy of an argument and not its siblings**. When a fact is stated in more than one place, changing it is a grep, not an edit. + + **An evidence heading may be module-qualified, and must be when its leaf is not unique.** `scripts/check-comment-runs.py --check-evidence` takes its Rust names from `cargo test --workspace -- --list` minus the `--ignored` twin, as **full module paths**, and **refuses** an ambiguous leaf naming every candidate rather than warning. Six leaf names collide in this workspace. Note neither half suffices alone: rename one of a colliding pair and the leaf is unique again and resolves silently to the survivor, so qualification is the fix and the refusal is only what forces someone to write it (improvise-rm4s). The residual, filed: `--list` does not name the binary, so two crates sharing a module path are indistinguishable (improvise-sxsf) — **measured latent** (1444 names, zero collisions) and the proposed per-package fix **declined twice over**: 18× the cost (45.2s against 2.4s, and `--check-evidence` needs the `--ignored` twin, so 22 cargo invocations), and *a package is not a binary* — `improvise` alone has five integration test binaries beside its lib, and an integration test's top-level `fn` lists with **no module path at all** (60 of 1444 names are bare), so `-p improvise` prefixes all six identically. Per-**target** is the only shape that would close it. + +49. **A websocket handshake is exempt from the same-origin policy, and that is a server-side problem.** There is no CORS preflight, so any page a user visits can open a socket to `improvise-ws-server` *from that user's own allowed address* and read and rewrite every document it holds — cross-site websocket hijacking (improvise-kc7n). **No network control can see it**: an IP allow-list, a private network and loopback binding all answer "may this **client** reach the endpoint", and the honest answer is yes; only `Origin` says what is **driving** the client. **Loopback is the least protected case, not the safest** — no proxy in front of it, and loopback is not a boundary a browser enforces. `src/origin.rs` is the whole decision and `transport::admitted` its one caller; it refuses the **upgrade** with a 403 rather than accepting a socket and ignoring it, and `serve` **takes** the policy so a host cannot forget to apply it. Three arms are decided rather than defaulted: **loopback allowed** (the shipped local recipe serves its page from `http://localhost:8080`, so the one case worth optimising for must need no flag), **absent allowed** (no browser omits `Origin`, so absent is a non-browser client an address check already answers — and refusing it breaks `tests/socket.rs`, the repo's own instrument), and **two refused** (`HeaderMap::get` is first-wins, so a proxy that *appends* rather than sets leaves two, and answering from the first lets the second be anything). The deployed Pages origin is deliberately **not** hard-coded: it can only reach a plain-`ws://` server through a TLS proxy anyway, so one flag is an honest cost on an already multi-step path where a baked-in default would be a permanent hole for every other deployment. The residual hole is **stated rather than guarded** — a proxy that *strips* `Origin` turns the check off, which is why the proxy's own allow-list stays in `README.md` as defence in depth. **None of this is authentication**: it answers what page is driving a client, never who is connecting, and improvise-kpnr is still open. It correctly does **not** move to `improvise-command` beside `Host` (gotcha 45) — a `MessagePort` has no handshake and no headers, and a worker and its page are the same origin by construction, so this is genuinely transport-only and not a mirror waiting to drift. + + +50. **A composed TUI row measures in cells and names which side yields.** `draw.rs::fill_line` takes a `Keeps`, because the three rows that share a line genuinely disagree about it: the title bar keeps the **right** (the `?:help :q quit` escape hatch outranks an unbounded document name — improvise-t14d), while `minibuffer_line` and `draw_status` keep the **left**, which is what they already did, since right-priority there would let a long status message eat the text the user is editing. So the ticket's "give the right segment priority" was correct only for the title bar, and a global rule would have been the wrong shape. Widths go through `ui::text::truncate`, never `str::len` — and the ticket framed *that* half wrongly too: the title's own `·` separator is two bytes and one cell, so a byte count mis-padded **every** document, not only one with a non-ASCII name. + + **One key table on screen, and it is the one derived from the keymap** (improvise-rr8n). `TileBar` now advertises no keys at all: its literal read `Ctrl+↑↓←→ to move tiles` where those keys are bound to `enter-tile-select`, and the derived status line is a strict superset (`Space`, `h`/`l`, `Tab` appear only there). The ticket asked for the opposite — rewrite the literal, suppress the derived line — and was inverted deliberately, because suppressing a derived superset to preserve a literal that had already rotted keeps the stale table and deletes information. The **import wizard goes the other way**, and the asymmetry is the rule rather than an exception: its status line *is* derived, but from the **mode** keymap while the real keys are per-step inside `HandleWizardKey`, so there the derived table is the one that lies and `draw_status` suppresses the fallback for that one mode. **Keep whichever table is derived from the thing it describes.** Two blind spots make this class invisible: `every_hint_key_resolves_in_its_mode` **cannot see a string literal in a widget** (the same gap improvise-wk18 names for `help.rs`), and the widget's own characterization test renders at **80 columns**, where the hint does not fit and is never drawn — so the one test pinning that output was structurally incapable of seeing either hint. + + **A scroll is measured where it is drawn** (improvise-gtn7.1). The keypress path counted `visible_cols` from `term_width` while `draw_content` gave the grid `term_width` minus the 32-column side panel, so with any panel open the cursor scrolled to a column never drawn. Now `viewport::grid_width(term_width, side_panel_open)` is the one answer, reached through `App::grid_width`, and both `draw_content` and `App::cmd_context` read it; `App::side_panel_open` is the one answer to whether a panel took its column. The tile bar had the same shape: `first_tile` chose its scroll against a width that began one or two columns before where the tiles were drawn, and both now start at `tiles_left(start)`. The grid bug was latent from `b5418f2e`, when `compute_visible_cols` took `term_width`, and was exposed by `50e4a372`, which opened the category panel by default. Still open: columns of mixed width, where a `visible_cols` count taken at the old offset is applied at the new one (improvise-gtn7.2). + + +51. **The command language reads pipes, refuses surplus, and checks names when it applies (improvise-31oe.2, 31oe.3).** Reasoning: `context/code/cli.md` §14 and §15. (a) A `:` argument is read through `improvise_formula::quote`, so `create-view |By area|` names `By area`; a **surplus argument is refused** rather than ignored, and `cmd::core::only_when_omitted` gates the *cursor fallbacks* of `clear-cell`, `yank`, `paste` and `drill-into-cell` so a coordinate typed by the user is never overridden by the cursor. (b) `set-axis`, `set-page`, `toggle-group`, `hide-item`, `show-item` and `switch-view` return `RequireKnown(Needs)` (`effect.rs`) ahead of the real effect, built by `effect_cmds.rs::unless_unknown`. It is `ViewOnly`, writes nothing, is **off the wire** (listed in `vocabulary_is_complete.rs`) and checks the **document at apply time** — a `:` line builds every segment before any runs, so a check in `execute` refused `create-view Second . switch-view Second` — then refuses and aborts the batch (gotcha 21). (c) A refused name therefore stops the rest of a `:` line, while headless `improvise cmd` rebuilds the context per segment and runs each as its own batch (improvise-31oe.14, open). + +52. **A date column imports as one hierarchical category, and the import refuses what it cannot keep (improvise-8jk6.3, 8jk6.6, d0n6).** Reasoning: `context/code/formula-and-io.md` ("Import: …") and `cli.md` §13. The default `DateShape` nests Year > Quarter > Month (the chosen levels) in ONE category named after the column and opens folded to its years; `DateShape::Split` is the old flat split, chosen by `--split-date ` headless or `s` on the wizard's date step. `Plan::spelling_clash` refuses a raw, unparsed date spelled like a generated item or group (`2024`, `2024-01`) because the two would silently merge — asked at the wizard's ConfigureDates step **and** again at build time, so a headless import cannot reach it either. Headless, `--strict` turns a partial skip into a refusal before the build, and a folder whose CSVs are all header-only (no records) is refused as an import that kept no record. + +--- diff --git a/context/repo-map.md b/context/repo-map.md index b2888ace..b967a21f 100644 --- a/context/repo-map.md +++ b/context/repo-map.md @@ -156,513 +156,101 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text --- -## Gotchas (read before editing) +## Gotchas (digest — read before editing) -1. **Records mode is wide format.** A row is one *coordinate tuple* — the cells sharing everything but `_Measure` — and the columns are `[coordinates…, **`+cat`**, raw measures…, `+`]`: a column per regular category, then the column that creates a **new category** (improvise-2zq, sitting where the column it creates will appear), then a column per raw measure, then the column that creates a new measure (improvise-c5v). Consequences: two measures on one coordinate are ONE row; editing a category column moves **every** measure cell in the row, not just the one under the cursor; `_Dim` in the synthetic key carries the column **index**, not a label, because a category and a measure may share a name — resolve it with `GridLayout::record_col` into a `RecordCol`. Formula targets are not columns (nothing to type into them). Drill pending edits are keyed by `(row, RecordCol)`. - **`prune_axis_items` takes a predicate, not a flag vector.** `prune_empty` hands it `impl Fn(usize) -> bool` over the *data* index (group headers consume no index). The domain is derived from `items` itself, so there is no parallel length a caller could get wrong — the earlier `&[bool]` + `debug_assert` pair could disagree between debug and release if a third caller ever sized `keep` from something other than `row_count()`/`col_count()` (improvise-n3wt). **It re-marks the survivors rather than filtering the markers** (improvise-e265): runs nest, so dropping an item can leave a header with nothing under it or a `GroupEnd` closing a run that was never reopened, and no per-marker filter can tell. `remark` instead emits, before each kept item, exactly the headers and `GroupEnd`s that make the output's open runs equal the input's at every level — each survivor keeps exactly the runs it was in. Reasoning: `context/code/core.md` §6. **`prune_empty` shares one `GridEval`** across the whole row×col emptiness test, and **a grid evaluates only through `GridLayout::evaluator`**, which builds the fold memo from that layout's own `none_cats` and hidden items — so no other grid's memo can reach its cells or totals (improvise-metf, improvise-opbo; the memo itself is improvise-h6z1, `core.md` §14). - **`GridLayout::new` does not prune; `with_frozen_records` does** — and that split has now produced **four** bugs: `view_frame`, `project::grid_in` (PR #78), and `persistence/export.rs::render_csv` (improvise-iot6), which wrote rows *and columns* the grid does not draw for as long as CSV export has existed. Every remaining `GridLayout::new` call site in the workspace is test code — verified — so a new production caller almost certainly wants `with_frozen_records(m, v, None)`. +One or two lines each: the rule, and the trap. **The numbers are stable and cited across the repo; a +citation of `gotcha N` means entry N in `context/gotchas.md`, which holds the full argument, +the incidents and the ticket ids.** Read the long form of every gotcha your diff touches. Counts and +"closed/open" claims in the long form are dated snapshots. -2. **Effects never touch `std::fs`.** Every document read or write goes through `App::storage: Box` (improvise-fuj, closed). Two writes still go around it: `App`'s autosave and the headless final write call the path-taking `persistence::save` (improvise-31oe.11, open). The host picks the backend: `App::new` supplies `PhysicalFs`, `App::with_storage` takes any other — the browser worker supplies OPFS (improvise-i34, closed), and tests use `MemoryFs`. `improvise-io` keeps path-taking wrappers (`save`, `load`, `export_csv`, `parse_csv_with_warnings`) for the native CLI, each delegating to a `_to`/`_from`/`_bytes` variant that takes the backend or the bytes. -3. **Every effect declares an `EffectKind`, and there is no default.** `ViewOnly` never reaches other clients; `Model(CellData | LayoutGeometry | WholeWorkbook)` does (improvise-mae). A new effect that forgets to classify itself fails to compile — that is deliberate, since the failure mode is silent: a mis-declared `ViewOnly` edit would simply never reach connected clients. Watch two traps: the *View* lives inside `model_state.workbook`, so "touches model_state" does NOT mean "is a model change" (axis/page/format are `LayoutGeometry`, cursor and scroll are `ViewOnly` — see improvise-8uz; and `SwitchView`, `ViewBack` and `ViewForward` moved from `LayoutGeometry` to `ViewOnly` with improvise-jytm, writing `view_state` and nothing else, which is what took the latter two off the wire — but a sound derivation is not a guard: only `SwitchView` had one, and review showed reclassifying either sibling back was silent across the whole suite); and `Save`/`SaveAs` touch `model_state` for bookkeeping only and are `ViewOnly`. `AddCategory`/`RemoveCategory` are the third trap and go the other way: they look like data edits, but both go through `Workbook`, whose `on_category_added`/`on_category_removed` rewrite **every** view's axis map, so they are `LayoutGeometry`. The two "an effect declaring X actually changes X" tests cannot separate them — a category change moves model data *and* view config, so it clears either bar; `structural_effects_declare_their_kind` pins the declaration itself. A third trap, now guarded: **`EffectKind` is too coarse to check a *mapping*.** `ui/reduce.rs::model_effects` sends each of the 26 `ModelCommand` variants (counted on `c1772ca9`; gotcha 38) to one effect, and every inverse pair — `HideItem`/`ShowItem`, `AddItem`/`RemoveItem`, `CreateView`/`DeleteView`, `SetCell`/`ClearCell` — declares the *same* kind, so swapping three of them at once left the suite at 918 passing while three wire messages did the exact inverse of what they said. The mapping is name-for-name by construction, so `every_model_command_names_the_effect_of_the_same_name` asserts that, reading the type name off the effect's derived `Debug` (`Effect` is a trait object, so `type_name` is out of reach). A variant whose effect must be named something else is a variant that stopped being derived — it should fail there and be argued about. Their reach is wider than the name suggests: `Model::remove_category` also drops every formula targeting the category, which is why `Dirty::of(LayoutGeometry)` reports `formulas: true` — see gotcha 35. -4. **Commands never mutate.** `&CmdContext` is read-only; return `Vec>`. If you want to touch `&mut App`, add or extend an Effect. -5. **`CellKey` is always sorted.** Use the constructors; equality and hashing rely on canonical form. -6. **No `Model::add_item`.** With a `Workbook` in hand use the facade: `wb.add_item("Region", "East")?` / `add_item_in_group` / `remove_item` (improvise-tvt). With only a `Model`, go through the category: `m.category_mut("Region").unwrap().add_item("East")`. -7. **Virtual categories.** `_Measure` items = explicit items ∪ formula targets. Use `measure_item_names()` / `effective_item_names("_Measure")`. `add_formula` does *not* add items to `_Measure`. Formula-target lookup also falls back into `_Measure` via `Model::find_item_category`. Cell aggregation goes through `apply_agg`, formula `SUM(…)`/`AVG(…)` through `eval_agg` — they differ on empty input (absent vs. A third rule joins them for **non-numeric** values: `aggregate_cell_values` reduces text by **unanimity** — one matching value, or several copies of the same value, is that value; differing values, or text among numbers, is `CellValue::Error("mixed")` (improvise-l4qx). The **all-numeric branch is checked first**, so `Count` of a lone number is still `1` and not the number — that ordering is the whole difference between this rule and "a single value passes through". What the rule forbids is *dropping*: `filter_map(as_f64)` used to, and since `_Index`/`_Dim` are on `Axis::None` in every default view that made **every text cell in every document render blank**, including the pipe-quoted text `.improv` carries as a first-class value. `evaluate_aggregated` and `aggregate_raw` share the helper, so a formula ref over mixed values now errors `"mixed"` rather than silently summing the numeric ones. `0` for Sum/Count, error for Avg/Min/Max). `Model::evaluate_aggregated` takes `Aggregation::{Exact, OverHidden}` rather than `none_cats: &[String]` (improvise-5jm5) — the list was only ever tested for emptiness, and the guard against that shape returning is the **signature**. `recompute_formulas`/`aggregate_raw` keep the list because they walk it. -8. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full `f64`. Never feed a rounded value back into eval. -9. **Formula eval is fixed-point.** `recompute_formulas(none_cats)` iterates until stable, bounded by `MAX_EVAL_DEPTH`; circular refs converge to `CellValue::Error("circular")`. `App::new` runs it before the first frame so formula cells render on startup. -10. **Drill into formula cells** strips the `_Measure` coordinate when it names a formula target, so `matching_cells` returns the raw records that feed the formula instead of returning empty. -11. **Minibuffer keymaps come from `TextEntry`.** The eight text-entry modes are built by the `TextEntry` builder in `improvise-protocol/src/keymap.rs` — seven by `TextEntry::new`, and `RecordsEditing` by `TextEntry::with_parent` off `Editing`'s map, which is why a count taken off the `::new` call sites reads seven. It binds the shared spine (AnyChar→`append-char `, Backspace→`pop-char `, Esc→`[clear-buffer , enter-mode ]`) and welds every commit key to its `clear-buffer` via `.commit(key, cmd, args)` (improvise-w3q). `TextEntry::new` takes the `AppMode` and reads `` off its `MinibufferConfig`, so the buffer the prompt names and the buffer the keys edit are the same string by construction (improvise-k8h) — the keymap never spells a buffer name. `.rebind` exists only for CommandMode's Backspace. `SearchMode` is hand-written on purpose — no commit key, and `exit-search-mode` owns the query. `EnterMode` clears the incoming mode's minibuffer through `write_buffer` too — building `SetBuffer` directly there was unreachable only because no shipped `AppMode` is keyed `"search"`. `MinibufferConfig::buffer_key` is a `String`, not `&'static str`: a borrowed-'static field makes `Deserialize` unimplementable, and every text-entry `AppMode` variant carries a `MinibufferConfig` (improvise-a2x). A buffer named at runtime is read with `cmd::core::read_buffer` and written with its mirror `write_buffer` — "search" is not in the buffers map but its own `ViewState` field, and those two functions are the only code that knows it (improvise-gdl). -12. **Panel keymaps come from `PanelKeymap`.** The three panel modes share a spine (Esc/Tab→Normal, arrows and j/k→`move-panel-cursor `, every panel's toggle key, `:`→command mode) built by `PanelKeymap::new(panel)`; the keys that act on the panel's contents arrive through `.bind`/`.bind_args`/`.delete` because they vary in shape, not just command name (improvise-4pg). `Panel::ALL`/`name()`/`key()` in `ui/app.rs` drive the toggle keys for Normal and all three panels, so a fourth panel reaches every keymap from one `ALL` entry. `name()` must spell what `parse_panel` accepts. -13. **Prefix keys are stored as an address, not a keymap.** `ViewState::transient_prefix` holds the `(KeyCode, KeyModifiers)` that opened a prefix; `KeymapSet::prefix_keymap` resolves it back to the sub-keymap, walking parents so `RecordsNormal` reaches Normal's `g`/`y`/`Z` (improvise-jc8) — that is the command-layer wrapper; the resolution itself is `Keymaps::prefix_keymap` in `improvise-protocol/src/keymap.rs`. `ViewState` must stay free of `improvise-command` types — it is destined for the protocol crate, which `improvise-command` will depend on. Resolution assumes prefixes are **flat**; `prefix_keymaps_are_flat` fails the build if a nested one is added. -14. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the mode; Normal + `search_mode=true` maps to `SearchMode`. Minibuffer modes bind Enter/Esc as `Binding::Sequence` so `clear-buffer` fires alongside commit/cancel. Hints are checked against the keymaps: `every_hint_key_resolves_in_its_mode` (`command/keymap.rs`) splits each `AppMode::hint_text()` and requires every key it names to resolve in that mode's map, because `only_quit_has_no_hint` asserts only that a hint is *present*. It checks which keys are named, not what they do. Its tokenizer knows the hint dialect explicitly (named keys, `Ctrl-`, `F/C/V` alternatives, character runs like `hjkl`/`[:]`, the em-dash prose tail, CommandMode's `:q` ex-commands) and panics on a token it cannot classify rather than skipping it. -15. **Registry names come from prototypes, not strings.** Every `register*` call reads the name off a prototype `Cmd` instance, so a typo is impossible (improvise-9cn) — so no `register*` call spells a name. Name literals do remain in `registry.rs`, and only tests guard them — the keymap test `every_bound_command_name_is_registered` for the `Move` literals (all bound; `registered_names_are_unique` covers them too), and for the four aliases `registry.rs::every_alias_resolves_to_a_registered_command` (a hand-copied list, not the table) plus the `parse.rs` tests `alias_add_cat_resolves_to_add_category`, `alias_formula_resolves_to_add_formula`, `alias_add_view_resolves_to_create_view` and `alias_q_bang_resolves_to_force_quit` — the `Move` prototypes carry `cmd_name: "move-selection"` / `"scroll-rows"` / `"page-scroll"` (the four `jump-*` names go through `reg_jump!`), and four `r.alias` calls spell an alias and its target (`add-cat`, `formula`, `add-view`, `q!`). What types can't reach: five commands name themselves off a field (`Move` off `cmd_name`, `ViewNavigate`, `TileAxisOp`, `ToggleGroupAtCursor`, `CommitAndAdvance`), so a prototype can name the wrong *variant* and still compile. That shows up as a **duplicate** registration, not a mismatch — the wrong-variant entry collides with its sibling and the sibling's name vanishes. `registered_names_are_unique` catches it; `every_bound_command_name_is_registered` catches the keymap side, and checks **both** halves of `dispatch`'s `registry.interactive(name, args, ctx).ok()?` — the name resolves *and* the args parse — because improvise-s13 moved `enter-mode ["export-prompt"]`/`["formula-edit"]` out of the nullary class, where a typo'd mode name compiles into a dead key. It builds the `CmdContext` in each keymap's own `AppMode`, since `commit-buffer item` reads `ctx.mode`. -16. **`effect_cmd!` macro** wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written `Cmd` once there's real decision logic. Its counterpart on the effect side is `simple_effect!` in `ui/effect.rs`, which declares a one-field newtype that writes one place on the `App`. Its **kind argument is mandatory** — `simple_effect!(Name(T) => |app| place, EffectKind::ViewOnly)` — even though every call site so far answers `ViewOnly`: hard-coding it reintroduced exactly the default gotcha 3 does without, since `$place` is an unconstrained expression and a one-line `active_view_mut().number_format` setter would have reported `ViewOnly` with the suite green (improvise-8fy). Exactly six wrappers return `unless_unknown(needs, effect)` rather than the bare effect (gotcha 51): `set-axis`, `set-page`, `toggle-group`, `hide-item`, `show-item` and `switch-view`. The rest return bare effects — `delete-view` (whose `DeleteView` effect refuses an unknown view itself), `add-item`, `add-items`, `set-cell`, `create-view` — and `remove-formula` checks in `execute`. Likewise, mode entry has no per-mode shim commands — bind `enter-mode ` (improvise-s13); the name is decoded by `parse_mode_name` in `registry.rs`, and `EnterMode` clears the mode's minibuffer on the way in. -17. **Float equality inside formula eval has one epsilon, and the div/0 guard shares it.** `FLOAT_EQ_EPSILON = 1e-10` in `model/types.rs` is used by `=`/`!=` **and** by the division-by-zero guard (`rv.abs() < FLOAT_EQ_EPSILON`), in both `eval_expr` and its aggregated twin — six uses across the two evaluators (a div/0 guard plus the `Eq` and `Ne` arms in each), one constant. This gotcha previously recorded the opposite (`=`/`!=` epsilon, div/0 exact) and was stale; the mix is gone. Why an epsilon at all and why this one: `context/code/core.md` §4. -18. **`IndexMap`** preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do `DataStore::matching_cells`/`matching_values`: the secondary index **narrows** the candidates, then `matching_positions` resolves each to its position in the primary `IndexMap`, sorts, and reads back. **An index whose values are `HashSet`s is for O(1) lookup, never for iteration.** Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; `SUM`/`AVG` drifted in the last bits because `f64` addition is not associative (`Min`/`Max`/`Count` and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the **drill snapshot** froze its record list in hash order while `drill_state.pending_edits` are keyed by *row index* (gotcha 1), so the same drill opened in a second process addressed **different records** — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on `matching_values`, and `matching_cells` 7% *faster*, sorted positions resolving `CellKey`s with better locality. `DataStore::live_records` was the open choice and is now decided (improvise-2yc2): it walks `cells` and projects insertion order onto records, deliberately paying O(cells) rather than O(records), because `cells`' order is already the answer `matching_positions` gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on `RecordEntry`, since a stored position is a copy and a record deleted and re-created would hold its old one. Note what is *not* on offer: an unordered return — every iterator has an order, and a type that merely disclaims one still hands the caller hash order, which is the guard living one level away from the hazard and the shape this whole class of bug returns through. **The second real instance was `import::analyzer::analyze_records`** (improvise-7yx), which built its distinct values through a `HashSet`: `build_model` adds items in exactly that order and `IndexMap` preserves it, so the *same CSV imported twice* produced two different `.improv` files and two different row orders. Now first-seen order, pinned by a five-iteration test because each `HashSet` gets its own `RandomState`. Also answered here since: an **empty `partial`** matches every cell vacuously (improvise-hkt6) — it used to index `sets[0]` and panic, unreachable only because both callers guarded. **A group's fold reads through the same rule.** `Model::fold_stored` folds a collapsed group's stored values once, over every record the group covers, through `DataStore::spanned_values` → `spanned_positions` (improvise-yb4b) — one scan, not a lookup per member. `spanned_positions` chooses **by index-set size** between narrowing once by the base key and narrowing once per covered item of the smallest span, whichever visits fewer (improvise-jq0m), and **both arms return store order** — sorted positions, as `matching_positions` does. That is load-bearing, not tidy: a fold that returned hash or per-item order would reintroduce the `SUM` drift above, and the choice between the arms would then be visible in the last bits of a total. **A partial as wide as the widest stored cell needs no narrowing** (improvise-felo.13): `DataStore` keeps `arities` (how many stored cells have each width), and `exact_positions` answers a strictly-sorted partial at least that wide with one probe of `cells`, since only an equal cell can match; anything else falls through to the index. The answer is the same sorted positions. -19. **Commit paths for cell edits** split across `commit_cell_value` → `commit_regular_cell_value` / `commit_plain_records_edit` in `command/cmd/commit.rs`. Keep the synthetic-records branch in sync with the plain branch when changing behavior. **Which cells refuse an editor is one function, asked at both ends of an edit**: `cmd::core::uneditable(model, layout, row, col)` is the shared decline for **opening** an editor (`EnterEditAtCursor`) and for **committing** one (`CmdContext::committable_cell_key`) — a fold or other aggregate, a formula's result, or a `CellId::NonItem` (improvise-enz6, improvise-gk51). It reads the cell's `CellId` through `project::cell_id`, the same derivation the projection uses, and the browser's `resolve::declined_at` matches on the projected `CellId`, so the two front ends refuse the same cells. Asking only at open is the bug enz6 was: Enter-to-advance moved onto a collapsed group's row and committed a write to the group coordinate. **`CellId::NonItem { category, item }` is a stored value whose coordinate names no item of its category** — records mode can draw one, and no write can address it, so it is projected as its own variant rather than as a `Record` that the host then refuses (gk51 was exactly that mismatch). Its one wording is `improvise_protocol::non_item_refusal`. -20. **Minibuffer commits go through one command.** `commit-buffer ` (`CommitBuffer { kind: CommitKind }`) owns buffer→trim→bail-on-empty→dirty→status for all three; the varying parts are methods on `CommitKind`. The add effect is emitted **last** so its failure message wins over the command's optimistic status (improvise-6t3). `CommitKind::Item` carries its target category, resolved from `AppMode::ItemAdd` by `parse_commit_kind` at registry time — so `commit-buffer item` is not constructible from a script line. -21. **An abort stops the chain, never the bookkeeping — and ordering is what separates them.** Three effects set `abort_effects`, and they are not all saves: `Save` and `SaveAs` on a failed write or a missing path (improvise-8tk — `SaveAs` was the arm left out of the original fix, so `:w /bad/path . q` quit on a failed write), and `AddCategory` when `Model::add_category` refuses, because what follows assumes the category exists — from the `+cat` column (improvise-2zq) the rest of the batch would advance the cursor and re-open an editor on a column that was never created. A `Binding::Sequence` flattens into one effect vec, so the abort carries across sequence steps; that is a property of the *flattening*, and it is why improvise-0hb's proposal to apply effects between steps was rejected — `apply_effects` resets the flag per batch, so step 2 in a fresh batch would land the `Quit` after a failed `Save`, silently reopening a P1. `a_sequence_whose_first_step_aborts_does_not_run_the_second` is the guard. **The rule the aborts taught, which is the part to carry:** an abort at the *tail* of a batch also ate the mode restoration and the buffer clearing, stranding the user in `CommandMode`/`Editing`/`SavePrompt` — where `draw_bottom_bar` renders the minibuffer *instead of* the status line, so the error the effect had just written was never drawn and the operation appeared to do nothing at all. So **tidy-up goes at the head, where an abort cannot reach it**: `ExecuteCommand` fronts its batch with `ReturnToNormalUnlessMoved`, `CommitAndAdvance` with the mode change, `CommitSave` with the mode change (found by Copilot on PR #27). Ordering answers it without giving any effect a "runs anyway" flag, and it does not weaken gotcha 31's observed-not-predicted mode settling: whatever a later effect sets, it sets *after*, so the mode a `:` line ends in is unchanged in every case — the only difference is what happens when the batch never reaches the end. **An abort is not a rollback**; that is the hole clause 4 leaves (gotcha 47). -22. **The import wizard's back path is a history stack, not a reverse table.** `advance` pushes the step it leaves onto `ImportWizard::history`; `previous` pops it, so back always retraces the path actually taken through the conditional transitions (improvise-907). Preconditions live in `validation_error()`, checked on the step where the user can still fix them — never deferred to `build_model`. `NameModel` is terminal for `advance`: Enter there calls `build()`, which parks the workbook in `built` on `Done`, and `AcceptWizardModel` takes it from there. Going back from `Done` discards `built`. **`ResolveDuplicates` appears only when `ImportPipeline::collisions()` is non-empty**: `after_fields` routes there from `ReviewProposals` or `ConfigureDates`, so it comes after the field and date steps and before `DefineFormulas`, with `NameDuplicateCategory` between them when the answer is Separate. Enter there is `WizardOp::ChooseDuplicate` (sum, keep separate, or go back). `ImportPipeline::on_duplicate` defaults to `DuplicatePolicy::Refuse`, so a path that never asked — the headless `--no-wizard` import included — errors listing the collisions rather than summing them. **A collision is a cell written by two records** (a tuple *and* a measure), not a shared tuple: records with disjoint measures on one tuple write different cells and collide with nothing (improvise-9bit). Reasoning: `context/code/formula-and-io.md`, "Colliding records are a question, not a default". -23. **`app.layout` goes stale mid-batch.** `apply_effects` rebuilds only *after* the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a `RebuildLayout` effect in front of it — the command decides the order (see `EnterEditAtCursorCmd`, `CommitAndAdvance`). Read the cursor's cell text via `App::display_value_at_cursor()`; don't build a `CmdContext` inside `apply` (it reads `layout`, the memoised `grid_geometry()` and the cursor, and a stale layout there is the bug). **Since improvise-felo.13 the rebuild is skipped when nothing it reads has moved**: `App::refresh_layout` rebuilds only when `layout_stale` (any `EffectKind::Model(_)` in the batch, or `install_workbook`), the drawn view's name, or the drill snapshot's pointer (`LayoutBasis`) changed, so a cursor key rebuilds nothing, and the column widths and row-header width are measured once per layout (`App::grid_geometry`, dropped with the layout). Two of the three inputs are **observed** (the view name and the drill pointer); `layout_stale` is **declared** from `EffectKind::Model(_)` (`touches_model`), so gotcha 3's classification hole — a `ViewOnly` effect that changes what the layout reads — now also leaves `app.layout` stale, where before felo.13 every batch rebuilt. It is still not a hand-kept list of effects. A writer that bypasses effects and changes what the layout reads must call `rebuild_layout()` itself. Measurements: `context/test-evidence/layout-cost.md`. - -24. **The cursor is per-client and keyed by view.** `ViewState::cursors: HashMap` holds `selected`/`row_offset`/`col_offset`, keyed by view name; an absent key means the origin. Reach it through `App::cursor()` (a `Copy` snapshot), `App::cursor_mut()`, and `App::reset_cursor()` — never off `View`, which no longer has the fields (improvise-8uz). Two consequences. **Changing the grid's shape must reset the cursor explicitly**: `View::transpose_axes`/`cycle_axis` used to do it themselves, and now `TransposeAxes`/`CycleAxis` call `app.reset_cursor()` — a new geometry effect that renumbers rows or columns has to do the same. **Replacing the document must clear the map**: `LoadModel` and `AcceptWizardModel` call `cursors.clear()`, because the incoming workbook's view names have nothing to do with the outgoing one's. That clearing replaces `Workbook::normalize_view_state`, which is gone — a `View` can no longer carry a stale offset. The same clearing owes the **session mirrors**: `install_workbook` bumps `App::document_generation`, and `ClientSession::project` reseeds any mirror seeded at an earlier one (improvise-cqq). An effect cannot reach `Sessions`, so the generation is the mechanism and `Sessions::rebase` is only the client-facing half. The same is true of a **geometry** change, and it owes the mirrors *both halves*: `App::cursor_epoch` is bumped by `reset_cursor` and watched by `Sessions::adopt_cursor_epoch`, which resets every mirror **and** returns `Broadcast::ToAll(Command::ResetCursor)` (improvise-gvnb). Doing only the first is **strictly worse than doing nothing** — the diff then believes every client already holds origin-positioned content and sends nothing, so they stay stale *permanently* rather than until their next scroll. The trigger is the **epoch moving**, not the command variant, for the reason gotcha 39 watches `document_generation` instead of matching on `LoadModel`: which effects renumber the grid is a per-effect decision, so a table of variants would be a second description of it, free to disagree. What is broadcast is an **invalidation, never a position** — another client's transpose must not move where I am looking (that is UI and it is mine, improvise-g0f), only declare that where I was looking has stopped meaning anything. **Since improvise-ni6k no effect calls `reset_cursor` at all — the cursor is invalidated by OBSERVING the grid's frame.** `apply_effects` diffs the frame after every model effect, **inside the loop**, because a batch that reframes and then places the cursor loses the placement otherwise (four records-mode tests pin that position). There is no list of effects anywhere, so the mechanism is total over `EffectKind::Model(_)` by construction — a genuinely stronger shape than a hand-kept table, and the reason the old "a geometry effect that forgets `reset_cursor` is invisible in both directions" hazard is closed. - - **Two arms, and the asymmetry is the whole design (improvise-mcq4).** `LayoutGeometry` hashes only the **active** view — it is the held-key path, and other views cannot reframe there since `view_mut()` writes only the active view, with the category and item effects already covered by `view_shapes`. `CellData` hashes **every** view, because a cell write can prune a row out of a view the host is not on; narrowing that arm to one view measured as *free* and silently lost a multi-client invalidation, which is gotcha 24's own strictly-worse-than-nothing case. Cost of the wide arm: 0% on the held-key path, +5.7% on an 18k two-view `SetCell`, linear in view count, accepted. State the absolute as well as the ratio: the observation roughly doubles held-key latency on large pruned documents (3.04 → 5.69 ms at 18k) and 56 ms at 90,000 cells is 3.4 frames. - - **Totality is conditional on classification**, and nothing makes classification total: a `ViewOnly` effect that renumbers the grid is invisible to all of this (gotcha 3's hole, unchanged). And two known gaps are filed rather than fixed — a geometry effect immediately after a mid-batch `SwitchView` skips the check entirely, because there is no basis for the new view yet (improvise-9c9o; under-reporting, which gotcha 35 says is the expensive direction), and `Model(WholeWorkbook)` matches neither arm, which is benign only because all three such effects go through `install_workbook` → `rebase` → a fresh `Snapshot`. - -25. **`ViewState` and its field types are `Serialize`/`Deserialize`.** It is the wire payload of the client Snapshot (improvise-q08), so `AppMode`, `MinibufferConfig`, `MinibufColor`, `ViewFrame`, `KeyCode` and `KeyModifiers` all derive both (improvise-a2x), and `improvise-command` depends on `serde` for it. The constraint this puts on new state: **no borrowed-`'static` fields anywhere ViewState can reach** — `Deserialize` cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. `view_state_survives_a_json_round_trip` guards it. `ViewState` and `Snapshot` also derive `Clone` and `PartialEq`, so the bootstrap message can be compared in a round-trip test. - -26. **Axis presentation lives on `Axis`; its colour is a `Face`.** Four vocabularies hang off the enum in `view/axis.rs` — `plain_label()` ("Row"/"Col"/"Page"/"None"), `glyph()` (which `Display` appends, giving "Row ↕"), `short_label()` (three columns for the tile bar; `None` is "·", *not* a truncation), and `initial()` (one letter for the view panel; `None` is empty). Plus `next()` for the Row→Column→Page→None rotation. **`Axis` carries no colour**: `AxisColor` and `Axis::color()` are gone. A colour is a `Face::Axis(axis)` in `improvise-protocol/src/face.rs` (ADR 004, `context/code/protocol.md` §10), resolved by the `Theme`, and `draw.rs::face_style` is the one place a face becomes a ratatui `Style` — the tile bar and the category panel both call it, so neither keeps a table. `improvise-core` still names no ratatui type. Not covered here: `persistence/format.rs` maps `Axis` to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer. - -27. **`Axis::Filter` constrains without being navigable.** Drilling fixes its coordinates on `Filter`, not `Page` (improvise-dz8): both restrict which records are visible, but only `Page` is something the user may move or page through. The split shows up as two different questions, and picking the wrong one is the trap: **"what data is visible" is `View::filtering_cats()`** (Page ∪ Filter — used by `GridLayout` and by `add-record-row` when it builds a key from the current constraints), while **"what can the user act on" is `categories_on(Axis::Page)`** (used by `[`/`]` page cycling) or `View::tileable_cats()` (used by all three tile-select sites). `Axis::next()` maps `Filter` to itself and no navigable axis rotates into it, so tile select cannot hand one out. The `.improv` grammar accepts `filter` and round-trips a selection with it — a Filter axis only ever occurs in the never-persisted `_Drill` view, but the alternative was writing it out as some other axis. **What a virtual view must *be* is model-side and has one home** (improvise-gd28): `view::{records_spine, records_steps, drill_steps}` return `Vec` — pure data naming no `Effect`, no `App` and no front end — and `cmd/grid.rs::steps_into_active_view` is the whole of *how the TUI gets there*, twenty lines a browser answers its own way. Before the split, the drill's filter placement and the records spine were spelled out twice in `cmd/grid.rs` with nothing keeping them in agreement. `DrillState` deliberately did **not** move: it holds an `Rc` and its `pending_edits` are uncommitted writes, so gotcha 30's session-local tag already places it. Reasoning: `context/code/core.md` §11. - -28. **The browser backend is sync on purpose, with an async seam.** `OpfsFs` (`improvise-io/src/opfs.rs`, wasm32 only) implements the same synchronous `Storage` trait as `PhysicalFs`, because `FileSystemFileHandle.createSyncAccessHandle()` yields a handle whose read/write/truncate/flush **are** blocking — available only inside a dedicated Web Worker, which is where improvise-9x6 puts the app. So `Storage` never became async. What is still async splits out into four explicit methods the worker awaits from its message loop: `open()`, `attach(path)` (must precede any read/write of that document — acquiring a handle is async even though using it is not), `refresh()` (repopulates the index `list` answers from), and `commit_deletes()`. Consequence: **`delete` is visible immediately but lands in OPFS at the end of the batch** — it drops the name from the index and queues a `removeEntry`. Handles are exclusive per file, so `attach` closes any handle it replaces and `detach` releases the lock. `storage_contract` in `storage.rs` is the behaviour every backend owes; it runs against both host backends and is the written-down spec `OpfsFs` must satisfy in a browser, where the harness cannot reach. **And the premise is dedicated-worker-only, which is narrower than this paragraph used to read.** `createSyncAccessHandle` is spec'd `[Exposed=DedicatedWorker]`, and measured in a module `SharedWorker` on Chromium and Firefox it is **absent** — not rejected, not flagged, not transient: `getDirectory()` and `getFileHandle()` both succeed and the method is simply not on the object (improvise-12ha). So `OpfsFs` cannot be hosted by a `SharedWorker` as written, and "a browser worker" in this paragraph means a *dedicated* one everywhere it appears. `OpfsFs::open`'s own guard does not enforce that — it `dyn_into`s `WorkerGlobalScope`, which a `SharedWorkerGlobalScope` **satisfies**, so a shared worker sails past a check whose message names the right requirement and fails later as a raw `TypeError` deep in the glue (improvise-7gip). The consequence for multi-tab is that the machinery everyone reaches for first is the storage seam, not `Sessions`: `Sessions`, the `Addressed` filter and the one `frame` pair are all ready, and none of that helps, because the no is entirely on the handle. - -29. **"Which view is active" is stored twice, and the copies are now allowed to differ.** `Workbook::active_view` is the document's **default** — what a freshly connected client opens, and what `.improv` writes as `Initial View`. `ViewState::active_view` is **authoritative for that client**, and it is what keys `ViewState::cursors`. The second copy exists because `reduce_view` runs on a client that holds no `Workbook` (improvise-cqi); the *split* is improvise-jytm answering improvise-g0f, on the user's ruling that **the model is shared and the UI is not**. Two clients may sit on different views of one document. Three things follow. **`App::resolve_view` is the one answer** to what a view name names in this document, falling back to the default so a slice naming a deleted view cannot panic in `rebuild_layout`; `view()`, `view_mut()`, `view_name()`, `project::grid_in` and `ClientSession::view_in` all go through it, and a second copy of that fallback was the first thing review found. **The guard was replaced, not deleted** — `the_two_active_view_names_never_diverge` became `the_documents_default_view_outlives_this_clients_switches` plus `a_fresh_session_opens_the_documents_default_view`, because a deliberate split with no guard is how one silently regresses into the other. And **a deleted view is the case the split creates**: `DeleteView` can take away the view another client is on, so `Sessions::adopt_deleted_views` moves that mirror to the default and tells it, and `effect::DeleteView` does the same for the host — observed ("does this mirror still name a view the document has"), never predicted from the command variant. **Testing any of it needs a bystander that can be moved**: both audience tests first sat theirs on the default, where a wrongly-addressed relay folds to a no-op, and review demonstrated the vacuity by injecting `ToAll` twice. One gap remains: `Initial View` no longer follows the TUI user (improvise-azhb). The other — a view-scoped `ModelCommand` naming no view — is **closed** (improvise-lltt), and this paragraph described it as an open gap for a day after: every view-scoped variant carries `view: String` and `model_effects` maps it to `ViewTarget::Named`, so a connected client's `SetAxis` lands on the view it names. `ViewTarget::Active` resolves through `App::view_mut()` and is therefore the *host's* view — but **the wire never constructs one**, which is what closed it. Follow mode is deliberately not foreclosed. - -30. **Two tests gate `ViewState` membership.** `ViewState` is the *portable* session slice — the one thing serialized into the Snapshot a newly-connected client gets (improvise-q08). A field may live there only if it is **portable** (plain data from the allowlisted crates, no `Rc`/`Arc`) and **snapshot-safe** (a fresh client inheriting the value is correct). `wizard` fails both — `ImportWizard` is an `improvise-io` type, and a half-open dialog is not something to inherit — and `drill_state` fails both — it holds an `Rc`, and `pending_edits` are uncommitted writes belonging to whoever staged them. Both live on `App` under the **session-local** residue tag (improvise-h0b). The `ViewState: Send` static assertion in `app.rs`'s tests is the compile-time proxy for portability: `Rc` is not `Send`, so a re-added shared pointer fails the build (the `Deserialize` derive from gotcha 25 now rejects it too, serde's `rc` feature being off). `Send` does *not* catch the origin-crate or snapshot-safety halves — check those by hand. - -31. **`:` commands settle their mode by observation, not prediction.** `ExecuteCommand` ends every batch with `ReturnToNormalUnlessMoved(ctx.mode)`, which drops to `Normal` only if the mode is still what the batch started in. There is no `Effect::changes_mode()` any more: it was a per-type constant, and `ViewBack`/`ViewForward`/`StartImportWizard` swap the mode only on some runs, so no up-front answer was right — `:import` opened the wizard and was snapped straight back out (improvise-toc). A new effect that changes the mode needs no annotation; just don't return the batch's starting mode from a success path. - -32. **There is no aggregate render cache — each widget folds its own stream.** `improvise-protocol::render` holds one cache type per widget (`GridCache`, `CategoryTreeCache`, `FormulaCache`, `ViewListCache`, `TileBarCache`), each implementing `WidgetCache` with its own `Update` type, and a `Projection` is one such update addressed to one widget (improvise-rkeq). `reduce_view` folds only `ViewState` now — it no longer takes a cache, and `Command::Project` passes through it untouched. Two consequences. **The overlap between caches is deliberate**: the tile bar and the category panel both carry `(category, axis)` and are updated by separate projections, so neither is woken by the other's changes; do not "fix" it by sharing a variant. **Bootstrap is a *compaction*, not a replay**: `Snapshot { view, seed: Vec }` seeds a new client with a batch of the same projections the steady stream carries, so there is no whole-state type that could disagree with the incremental one. The seed is the server's *current* state, not the head of the history — `GridCache::seed()` builds it, and the property that earns it is `GridCache::folded(&cache.seed()) == *cache`. Nothing is defined for the help overlay (its content is static and `help_page` is already in `ViewState`) or for the import wizard, which now has a shape and **deliberately no projection**: `WizardCache` is built by `project::wizard_cache` and drawn locally, because a half-open dialogue is addressed to precisely the session that opened it, never to every client (improvise-764 — the rendering-side twin of the argument that removed `AcceptWizardModel` from `ModelCommand`, gotcha 38). **A cache without a `Projection` is the shape session-local residue takes when it still has to be drawn by two different front ends.** It lives in `improvise-protocol` rather than `improvise-command` for the `flatten_cat_tree` reason: when the wasm client grows a wizard it must draw *these* rows, not a second derivation. - -33. **The grid's labels frame its cells, and `apply` enforces it — the fold is not commutative.** Three consequences in `improvise-protocol/src/render.rs`. (a) A `GridUpdate::Cells` that overtakes the `RowFrame`/`ColFrame` naming its positions is **dropped**, because `fit_to_labels` truncates to the labels it has; a client must not reorder the stream, and `a_cell_update_that_overtakes_its_labels_is_lost` pins it. (b) A label change that *renames* a position blanks the cell under it: `fit_to_labels` preserves by POSITION, which is only right where the label did not change, so `apply` calls `invalidate_relabelled` itself rather than trusting a producer to send `GridUpdate::Invalidated` first. `Invalidated` therefore now means only "contents stale, frame unchanged". (c) **every arm that touches labels *or* widths re-fits** — `col_widths` is re-fitted with the columns — a new column is drawn at its label's own character count until `ColWidths` catches up — because a renderer zips widths with labels and a length mismatch indexes off the end. Fitting on the label arms alone is what let a late `ColWidths` put the mismatch straight back — a short update keeps each unheard-of column at its label's width, and a long one is dropped as having overtaken the frame that names it, exactly as (a) holds `Cells`. `GridCache` also carries `totals` and `page_coords` (improvise-n10). `totals` is framed by `col_frame` on the same terms as the cells and widths and re-fits with them, with one asymmetry: an **empty** strip means "this grid has no totals" (records mode, or an empty grid) and is therefore *not* padded, where a non-empty one is — padding it would draw a blank total row under a records grid that has none. `Invalidated` blanks the totals too: they are contents, and a stale sum under fresh cells is the one number a user takes at face value. `page_coords` is framed by nothing (it names categories, not positions) and survives a reshape. Note the fold tests: asserting `fold(prefix)+fold(rest) == fold(whole)` is vacuous (it is the definition of a left fold), so the tests that earn their place are the order-sensitivity one and the compaction-equivalence one — see `doctrine(fold_tests_must_test_order_not_the_axiom)`. - -34. **A formula target is unquoted, and pipe quoting has one home.** `parse_formula` strips the pipes off a target exactly as it does off an `Expr::Ref` — the target names a `_Measure` item, so a target that kept its delimiters made `add_formula` create an item literally named `|Total Revenue|`: a ghost column the user's real item never matched, and `Expr::Ref("Total Revenue")` never resolved to. Both the TUI formula minibuffer (`AddFormula`) and `.improv` load go through that one call, so both were affected. The writer emits `Formula::raw`, not `target`, so file text is unaffected either way — which is exactly why the round-trip tests never saw it, and why the fix repairs existing files on next load with no migration. Pipe quoting has exactly **one home**: `improvise_formula::quote`, owning both halves — `quote_name`/`is_bare_name`, `pipe_quote` (unconditional, for a `.improv` text cell that must not read back as a number) and `unescape_pipe_body`. `improvise-gp` re-exports it, `improvise-io` calls it from `persistence/{format,parse}.rs`, `parser.rs::unquote_pipe` is a shim over it, and io's own copy is **deleted** (improvise-i2m5). **The keyword guard is canonical — user-confirmed, so do not revert it as over-caution**: an item literally named `SUM` is written `|SUM|`. Bare keywords do parse, but only because `agg_call` needs a `(` and PEG backtracking falls through to `ref_expr` — rule ordering, not a stated rule — and a `## Formulas` line carries formula text verbatim, so a name `format_md` writes is read back by `parse_formula`: one namespace, one spelling rule. One thing the guard does not cover, and one that used to be an exception and no longer is. **A section header takes its name as the whole line, totally** (improvise-pczv): `header_name = { pipe_quoted ~ &NEWLINE | line_name }` — the pipe-quoted arm needs the lookahead, or `|A|B` would parse as `A` and silently drop `B`, and `line_name` catches everything else, so a header name can no longer fail to parse. Four rules share it (`model_name`, `initial_view`, `category_section`, `view_section`), which is why it is a rule and not four copies. Note a header is still *written* raw where a reference is quoted — the grammar accepts both arms, so the quoting asymmetry between a header and a reference is a writer convention, not a format one. And the two former *unquoting* copies differed only on a trailing lone backslash, unreachable from the grammar, where io returned it and `parser.rs` panicked — the surviving `unescape_pipe_body` is total. Two `## Category:` sections naming one category are **one** category, concatenated by `ParsedFile::push_category` in **pass 1** (improvise-349m) — merging in pass 2 was not idempotent, because `build_category` creates its own section's item-groups before its own `>` lines while `format_md` writes every item before every `>` line. No grammar change was needed: this is the tree-walker, not the format, and duplicate headers stay legal. And **`WHERE` now carries `IS`'s word boundary** — both are compound-atomic with `sp+` (improvise-r08u), so `SUM(Rev WHEREAS x = y)` is a syntax error rather than a `WHERE` on a category `AS`. The guard is `sp+` and **not** the `!bare_char` lookahead the ticket proposed, because the grammar-walking generator reads the grammar through `pest_meta` and **skips predicates**, emitting `WHEREa=""` — a rule whose only guard is a lookahead is a rule the generator can violate. It costs one spelling: a category may no longer abut the keyword. - -35. **`Dirty` over-reports on purpose, and the asymmetry is the reason.** `Dirty::of` maps an `EffectKind` to the widgets that kind can reach, and the two directions do not cost the same: over-reporting pays one redundant `Projection`, under-reporting leaves a client's cache wrong **forever**, with nothing in the stream to correct it. `LayoutGeometry` therefore includes `formulas`, because `RemoveCategory` wears it and `Model::remove_category` ends by dropping every formula targeting the category — a client's formula panel otherwise kept showing a formula the document no longer had. It now coincides with `Dirty::EVERYTHING` and is still spelled out, because the two are derived from different effects and need not stay equal. The derivation now lives in `context/code/command.md`, and the doc comment on `Dirty::of` points there rather than restating it; the reason `LayoutGeometry` is spelled out rather than written as `EVERYTHING` is a two-line comment inside the match body. It went stale once already — claiming `AddCategory` was `CellData` after it was reclassified — so reclassifying an effect means re-reading the prose, not just the comment. - -36. **A client's command has one door, and the door takes the `App`.** `Sessions::apply_checked(app, id, Checked<'_>)` is the only way a connected client's `Command` reaches the server — `Host` calls *that* (improvise-ahdf). `Sessions::apply` is not `pub`, delegates to `apply_checked` after `Checked::against`, and has no production caller: its call sites are tests, which is why it takes a raw `&Command`. `ClientSession::fold` and the mirror's `ViewState` are private, so a transport cannot reach the mirror at all. `apply_switch` now **takes a `SessionId`** — it is addressed to exactly one session, which is the opposite of what this gotcha used to say — moves only that session's mirror, and dirties nothing. The one thing it still asks the document is whether the view exists, because a mirror keys `ViewState::cursors` and is read by `grid_in`. The audience is conditional: an accepted switch relays **nothing** (the asker folded it, and no other client's view moves), a refused one is `Broadcast::To(asker, …)` — improvise-6c6q, where `ToOthers` told every client but the one that was wrong. `adopt_active_view` and `Broadcast::ToOthers` are **deleted**; the variants are `ToAll` and `To(SessionId, _)`, the second because the deleted-view case needs a one-session audience that is *not* the asker. The paragraph's warning about the "what do the others hear" helper survives and is why the 6c6q test drives the asker's own `ViewState` — that helper is itself deleted. What survives unchanged: the model is shared and the UI is not, and `Broadcast` is a `Vec` because one command can owe two relays with different audiences. **The trap this paragraph used to describe is gone with `ToOthers`, and the lesson it taught is not.** It was copying `ToOthers` from `apply_switch` into `apply_model`, invisible from the bystander's side because `adopt_active_view` moved the asker's mirror either way — so the drift sat between the server's picture of the asker and the asker's own slice, where a test written through a "what do the others hear" helper looks away by construction. improvise-6c6q closed the mirror-image hole on the refused path. Two rules survive the mechanism: **assert the `Broadcast` itself**, since the audience is the message rather than something to infer from folded state, and **seat a bystander somewhere a wrong relay can move it from** — a bystander already on the relayed value cannot see a wrongly *widened* audience either, because the wrong relay folds to a no-op. Review demonstrated that vacuity twice on PR #48. A `Vec` rather than an `Option` because one command can owe two relays with different audiences, and an `Option` would have had to drop one silently. Everything else is that client's alone and never touches the `App`'s own view slice: **the model is shared, the UI is not.** Folding `SwitchView` into the mirror alone was the bug — `viewport()` then read the cursor under the new view's key and the next `project()` built that client a grid out of the *other* view. - -37. **`gp-tree` returns data and never prints, and a fit takes two things to reproduce.** Two rules with one root — the crate is a library a TUI, a wasm worker or a CLI may each host. (a) A decision the user should hear about comes back as **data**: `Dataset::warnings` carries the columns `preprocess` dropped for being entirely missing (improvise-d44s), and `fit.rs` hands back pooled `(prediction, actual)` pairs and a score per feature rather than tables. `examples/fit_csv.rs` owns every `println!`, and that is the only reason its output stayed byte-identical across these moves — adding an `eprintln!` to either module is the regression. (b) **The seed is necessary and not sufficient**: `data::Model` records `seed: Option` *and* `config: Option`, because the same seed under a different population/generations/elite/penalty lands on a different tree (improvise-ksst). Both are `Option` for one reason — every `FitConfig` value and the seed `0` are legal, so nothing can double as "unknown". Note `#[serde(default)]` on an `Option` field is **documentation, not the mechanism**: serde defaults it anyway, so deleting the attribute fails no test, and a backward-compatibility test that only deletes it is not discriminating. Aim such a test at an absent field loading as the *default* instead of as unknown. The GA loop itself is `fit::fit` (improvise-0ai); `breed` remains one fitness-agnostic generation for callers whose fitness is not "predict this column". (c) **A scoring function takes one slice of rows.** `Row` pairs a feature vector with what it predicts, so the parallel `x`/`y` lengths that `zip` used to truncate silently cannot be built (improvise-dkqn, `doctrine(an_invariant_is_made_unrepresentable_not_validated)` — the same answer `prune_axis_items` got, and the reason no `Result` entered five signatures). The pairing is made at the boundary that *builds* the data: `preprocess` walks one row index and the rows live inside `Target`, because a `Dataset` carrying `rows` and `target` separately just moves the zip up one level into every caller. The empty row set is one shape, answered once in `mean_over`. Changing any of this needs **bit-level** evidence: `tests/scoring_known_answers.rs` pins every path as f32 bit patterns plus a `next_u64` witness that catches a changed RNG call order, because the CLI prints three or four decimals and byte-identical output alone cannot see a moved accumulation. (d) **A fit is storable only when the document can *name* its features.** `preprocess` invents `Region=North`, `Cost.missing`, `when.year`; those are **encoding artefacts, not items of the document**, and `Expr::Ref` resolves through `Model::find_item_category` — so a tree splitting on one emitted a well-formed formula whose every cell was `Error("ref:…")`, silently (improvise-8xhy). `fitted_formula` therefore takes the `&Model` and refuses, listing what it could not resolve; `formula_text` still spells it, being the tree's own text. **The check asks the document, never the name's shape**, because a measure literally named `Region=North` is a legal document and the two are indistinguishable as strings — which also means `.missing` and the date parts are covered with no second copy of `preprocess`'s expansion rules to drift from. **This is an interim** (improvise-4zl7): a `WHERE` predicate needs a *scalar coordinate predicate* the formula language does not have — `Filter` gates a formula or an aggregate, it is not an `Expr` — would still leave `.missing` and the date parts, and must first answer what such a predicate means where `context.get(cat)` is `None`, since `Formula::filter` falls through to `aggregate_raw` and a categorical fit would silently degrade in every view that aggregates over the category. **improvise-zv8t closed the categorical half of the accident, and the close cost a document.** `fit_measure` handed `preprocess` an `Overrides::default()`, so it never said which header columns are document categories *although the very next line told `Features::of`* — and `preprocess` infers `Numeric → Date → Categorical` from the values, so a `Year` category with items `2024`/`2025` was a *numeric column* and a category of ISO dates a *date column*, both refused with `Year IS 2024` spellable all along. It now forces the same `header[..category_columns]` slice to `Forced::Categorical`. Two things follow. The **date decline is narrower than it read** — it covers a genuine date *measure*, never a category misread as one. And the forcing **widens the encoding**: a hundred-item numeric category one-hots past `MAX_FEATURES` and refuses a fit that previously succeeded by never splitting on it (improvise-e8no — and **not** to be closed by making the forcing conditional on cardinality, which is inference by another name and makes the *encoding* depend on a budget rather than on what the document says). The `Artefact` arms stay: `crate::dataset` is `pub` and takes the caller's `Overrides`, so the roles are still reachable. **Forcing by name while budgeting by index defeats the budget** — a category and a measure may legally share a name, and `no_ambiguous_column_names` ran *after* `preprocess`, so the expansion the preflight exists to prevent was allocated and only then refused; `as_categories` now returns `Result`, so the ordering is a compile error rather than a convention. The test that missed all this is the shape to remember: `improvise_eval` **fabricates** a `_Measure` item per feature name, so it asserted the emitter and `gp_tree::eval` agree against a model that could not exist. It is kept, with its scope now stated, and the property it could not state is asserted separately against a real document. - -38. **The wire vocabulary is derived twice, by the same recipe.** `Command`'s view half is the `EffectKind::ViewOnly` effects targeting the portable slice (`decision(improvise-cqi, command_set_derived_not_invented)`); its model half, `ModelCommand`, is the `EffectKind::Model(..)` effects — **28** of them, brace-matched on `c1772ca9` (2026-09-24) — minus `ApplyAndClearDrill` and `AcceptWizardModel`, leaving **26** variants (improvise-kmgs). The two added since this paragraph counted 26 and 24 are `ReplaceFormula` (improvise-km71) and `RenameRecordCoordinate` (improvise-fdk1); `SwitchView`, `ViewBack` and `ViewForward` left the model half with improvise-jytm. Every count in the rest of this paragraph is dated, so recount rather than cite it. **Completeness is now checked, not counted**: `ui/reduce/vocabulary_is_complete.rs` requires every effect to be on the wire under its own name or listed in `NOT_ON_THE_WIRE` with a verdict — `Command`, `Frame`, `Off` with its argument, or `Gap` naming the ticket that owns it (`SaveAs` and `ExportCsv` are gaps, improvise-2mqz). It is the forward twin of `vocabulary_is_composed.rs`, and it is what the enumeration below lacked. `SwitchView` no longer needs subtracting separately: it is not a model effect any more. `ViewBack`/`ViewForward` left for a reason this paragraph **stated wrongly until 2026-09-21**: it said they went for the *same* reason as the other two exclusions, session-local residue. That is false, and the falseness mattered. `ViewState::view_back_stack` **is in `ViewState`**, which is the portable slice — so by the view half's own rule (`decision(improvise-cqi)`: the `ViewOnly` effects targeting the portable slice) they were always `Command`s and were simply **missing**. The cost of the mis-statement was a live divergence nobody could see: `effect::SwitchView` pushes the leaving frame and clears the forward stack, while `reduce_view`'s arm set `active_view` **and nothing else** — so **no client ever had a back history at all**, and that, not the drill commit the `declined()` comment blames, is why `view-back` was dead in a browser (improvise-gd28, PR #115). **The lesson is about derivations, not about these two variants**: a derivation stated as a rule is only as good as the last time somebody enumerated it *against* the rule. This one had been checked for correctness — every listed variant belongs — and never for **completeness**. Note the obvious `awk` extraction reports **30**: it picks up `HelpPageSet`, which is `ViewOnly`, because that is the last `pub struct` before `#[cfg(test)]` and the test module mentions `ModelChange::` — brace-match the `impl Effect` blocks instead (`doctrine(derive_then_verify)`). Three things follow. **A `ModelCommand` does not mutate**: `ui/reduce.rs::model_effects` maps it back to the effect it came from and `apply_command` runs that through `App::apply_effects`, so effects stay the only mutation site and the `Dirty` is the effect's own `EffectKind`, never a second classification beside gotcha 35's table. **Two model effects are deliberately off the wire.** `ApplyAndClearDrill` and `AcceptWizardModel` were once kept on the grounds that the *command* is portable even where the operand is not; **that was the wrong criterion** (improvise-3x3u). A `Command::Model` is routed with no `SessionId`, which is right exactly when the change is addressed to *no one* session — and these are addressed to precisely one, never the sender: their operands are `drill_state.pending_edits` and the built wizard workbook, session-local residue (gotcha 30) private to whoever staged them (improvise-tom1). Unaddressed, B's `ApplyAndClearDrill` commits A's uncommitted typing and B's `AcceptWizardModel` swaps the document out of a dialogue B never opened — both demonstrated against `Sessions::apply` before removal. So **28** − these two = **26** on `c1772ca9` (it was 26 − 2 = 24 before km71 and fdk1), and the older spelling of this sum (`29 − SwitchView − these two = 26`) was itself a survival of the same staleness this paragraph documents: it subtracted `SwitchView` and forgot its two siblings, so it contradicted the count at the head of the paragraph and the enum’s own 24 variants (found by ChatGPT Codex on PR #116). The effects are untouched and still reached by the key path. Not permanent: multi-client drill wants the *addressed* form, which is where ADR 001 layer 2 goes anyway. Guarded by `no_model_command_commits_another_clients_session_local_state`, plus a companion asserting both effects still exist and are still model changes so the first cannot pass by naming nothing. **`Sessions::apply_checked` routes the model half to `apply_model`**, which relays `adopt_cursor_epoch`'s `ResetCursor` and `adopt_deleted_views`'s per-client switch; a document-replacing `LoadModel` bumps the generation and lands in the rebase arm. The former claim here — that `ViewBack`, `ViewForward` and `LoadModel` all end in `App::switch_to_view` so every mirror takes the accepted active view — was wrong on all three clauses after improvise-jytm and is deleted. A keymap lookup still cannot return a `Command`, and not for want of vocabulary: a binding names an *intent* resolved against `CmdContext` (`move-cursor down`) while a `Command` is the *resolved mutation* (`SetSelected { row, col }`) — improvise-gxi's AC 4 asks for something the split forbids. - -39. **`Applied` is an enum because a document swap is not a diff.** `Sessions::apply_checked` returns `Applied::Diff { dirty, broadcast }`, `Applied::Rebased { snapshots }` or — since ADR 001 layer 1 — `Applied::Rejected { reason }`, and the enum *is* the mechanism: a host that wants the `Dirty` must match, and matching names the case where a `Snapshot` is owed. The server's mirrors self-heal — `ClientSession::project` reseeds on the `document_generation` bump — but `ViewState::cursors` lives on the **client**, keyed by the outgoing document's view names (gotcha 24), and only a fresh `Snapshot` replaces it. `Command::Model` put the three `WholeWorkbook` mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. `apply_model` therefore calls `rebase` itself rather than instructing the host to, exactly as `apply_switch` adopts the accepted view itself. **The trigger is the generation moving, not the command variant**: a `LoadModel` that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a *failed* load — `a_load_that_fails_is_not_a_rebase` guards that arm. A `rebase: bool` beside `dirty` was rejected, because the host that forgets to call `rebase` is the host that forgets to read the flag. The **third** case is a *normal outcome*, not an error, and it is in the enum for the same reason the other two are: a host that wants the `Dirty` must match, and matching names it. `ServerMsg::Rejected` is the matching frame, addressed to **the asker alone** — nothing happened, so nobody else is owed anything — and the client keeps it as a `Rejection` rather than a sentence, because force needs to know *which* record and *which* kind (gotcha 43). - **Fixture warning.** When testing a document swap, the incoming document's active view must have the **same name** as the outgoing one. A rename makes `apply_model` broadcast a `SwitchView`; the client folds it, and the stale cursor is then read under a key nothing has written — right *by accident*, with the bug invisible. `other_document()` renames, and a first version of this test passed because of it; `a_document_shaped_differently_under_the_same_view_name()` keeps the name and changes only the shape, so there is nothing to hide behind. - -40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per *view shape*) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. **Only the `Dirty` is read off the `EffectKind`; both lower tiers are *observed* around the loop** — the source says so at `apply_effects` and this paragraph said the opposite until improvise-ilvl's audit. The `EffectKind` decides only *whether to look*: `touches_model` arms the shape snapshot, `writes_cells` the record fingerprints. `record_versions` **must** be observed, because an `EffectKind` says *that* a batch writes cells, never *which*; `layout_version` is observed for a better reason than necessity — `observe_reshape` diffs `view_shapes` before and after and bumps only where a view's shape actually moved (improvise-pdvm/nynk), so it is a **fact about the document** rather than a claim about the batch. Two consequences the declared form did not have. An **idempotent** `set-axis` bumps nothing, and batching two real changes into one line or two gives the same `2`. And the gotcha 21 argument is subsumed rather than restated: an aborted batch moved no shape, so there is nothing to not-claim — which is why observing once **after** the loop is correct where accumulating `Dirty` inside it is. **`view_shape` hashes what is drawn, not occupancy** (its doc comment: "Drawn, not configured") — axes, the drawn item names minus hidden ones, folds, page selections and the `prune_empty` *flag* — so an occupancy write does **not** bump `layout_version`: a `CellData`-only batch that prunes a row out of a view moves the cursor epoch (the frame observation) and not `layout_version`, as `context/code/command.md` records (the question that raised was improvise-6nsd, closed). The arming is wider than the tier (`touches_model` takes the shape snapshot), but a bump needs a shape hash to have moved. Three further consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run. - -41. **`Storage` is not a VFS, and that closes the SQLite question.** The trait is whole-document `read`/`write` bytes (gotcha 2, `invariant(storage_is_two_ops)`) — no offsets, seeking, handles or locking — so an `sqlite3_vfs`, which needs `xRead(offset,len)`/`xWrite`/`xTruncate`/`xLock`, has nothing to bind to. SQLite *can* still sit behind it, as an in-memory database serialized to a blob (`rusqlite` `serialize`/`deserialize_read_exact`, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is **22% larger** than `.improv`, **4.7× larger gzipped**, ~**160×** larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. **A second serialization format is a second implementation of the grammar**, obliged to track `improv.pest` forever; and **its own round-trip test cannot detect drift from `.improv`**, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way `export_sqlite` beside `export_csv`, native-only and feature-gated: `libsqlite3-sys`' C build fails under this repo's nix cc-wrapper for `wasm32-unknown-unknown`, which would break the `wasm` CI gate the day it landed. - -42. **The browser client's render decisions are made in Rust, and the frames it speaks are nobody's but the protocol's.** `improvise-wasm-client/src/render.rs` builds a `RenderModel` in which every question with a *silent* wrong answer is already answered, because the DOM half (`web/app.js`) is untested JavaScript and a wrong answer there shows up as a wrong pixel rather than a failing test. Three such answers. **`editable` is a fact about the `CellId` variant** — `Record` only, per ADR 002 — and the trap is the third case: `None` means "no projected cell", **not** "aggregate", so conflating them either hides a cell or offers a write the type system exists to make unconstructible. **The selection is placed by reconciling the scroll offset**, because `GridCache` is viewport-indexed while `ViewState::selected` is absolute. `render.rs` does it by *adding* — `r + cursor.row_offset == sel_row` — rather than subtracting, which is why there is no underflow path and why a selection above the window simply never matches. Omitting the offset entirely looks right until the first scroll, so the test that earns its place uses **nonzero and unequal** offsets: at `selected=(5,1)`, `row_offset=4`, `col_offset=1` it separates "reconciles" from "ignores" *and* from "swaps the two offsets". **The category panel is `viewmodel::flatten_cat_tree`**, never a second walk of `CategoryTreeCache` — that second walk was improvise-edp. The page is four string calls: `Client::viewport` (the handshake — the `Snapshot` seeds the *visible* cells, so one built at extent zero is an empty grid and a resize is a fan-out with nothing dirty), `on_frame` (one frame, one fold, arrival order — gotcha 33 holds *inside* a batch, so never coalesce), `render_model`, `typed_cell`. - - **One envelope pair, two deployments (improvise-xks6).** `improvise_protocol::frame` owns `ClientFrame { Viewport, Commands(Vec<_>), Save, Key }` and `ServerFrame { Snapshot(Box<_>), Commands(Vec<_>), Rejected, Accepted }` (as of `c1772ca9`; this line named a singleton `Command` and no `Save`/`Accepted` for some time after both changed); `improvise-ws-server` and `improvise-wasm-client` each `pub use` them and **neither defines one**. There used to be **three** pairs for two deployments — the socket's server side, a hand-kept *mirror* of it in the browser bundle, and a third written for the worker (improvise-tm6) before the projection architecture landed. The question that had to be answered before any of it moved was whether the **worker** and the **socket** want the same vocabulary, and the answer is **yes**: every difference was a *date* rather than a decision, and the test applied to each was whether the **argument** for it names the thing they differ in. None did. `Viewport` is deployment-independent — the extent is not in `ViewState` and no effect produces it, so no derivation reaches it, and a worker hosting the same `Sessions` owes `set_extent` the same answer; it is the handshake on both because a `Snapshot` seeds the *visible* cells. `Key` is deployment-independent too — a binding names an *intent* resolved against a `CmdContext` a thin client does not hold (gotcha 38), which says nothing about what the bytes travel over; the socket lacked it because key forwarding was never built on it (improvise-bpq2), and `Host::handle` now answers `ClientFrame::Key` with an explicit empty vec that **says so**, where the frame used to just fail to parse in `next_client_msg`. And `Commands(Vec<_>)` batches where a singleton cannot, because a fan-out must arrive whole and in sequence (gotcha 33) — a property of the **fold**, not of the transport; an in-process `postMessage` interleaves exactly as a socket does. The worker pair was not merely different but **incoherent by omission**: written before ADR 001, it could carry `Command::Model { basis, .. }` and had no `Rejected`, so a client speaking it composed a claim that could never be answered. That incoherence — not the difference — is the tell for staleness. - - **The mirror is gone, and so are the two tests that held it honest**; that deletion was the ticket's measure of success. `the_socket_framing_matches_the_server` round-tripped hand-listed fixtures through the other crate's types to catch a **rename**; `every_server_variant_is_one_this_crate_mirrors` matched exhaustively to catch a **variant added on the server and never mirrored**. With one definition both are **type identity**: a rename is a compile error at every call site in both crates, and the un-mirrored variant is not a constructible state. That is stronger than either test, and it is the shape to insist on — **if a consolidation leaves one of its compatibility tests alive, the consolidation is not finished.** What survives is `the_retired_spellings_do_not_parse` in `frame.rs`, aimed at the one thing still possible (the retired `Bootstrap`/`Command` spellings returning as serde **aliases**, which would be two homes again wearing one type), and it keeps the injected-fault discipline this paragraph earned the hard way: **assert the mutated input is still well-formed** before feeding it in, and **assert the error kind** — serde's `unknown variant` — not merely that an error occurred, because a rewrite that also breaks the JSON makes the check pass against an enum that really does have the variant. End-to-end compatibility is now `tests/browser_round_trip.rs`, which hands the wasm client's own serialized frame to a real `Host::handle`; `improvise-ws-server` stays a **dev**-dependency of the browser bundle for exactly that. - - **What the harness does and does not reach is gotcha 48's subject** — `web/app.js` is no longer unrun (`harness/tests/shell.spec.js` executes it under Chromium, and its standalone half under Firefox), improvise-hwpi is closed, and the quiet-failure and never-reads-a-key claims that used to stand here went with it. What remains unexercised is named there: the grouped and multi-level DOM, the CSS and the `ResizeObserver` extent arithmetic. What a careful read *did* establish about the last of these: the observer cannot feed back into itself (`#grid` is a flex child with `overflow: auto`, so `replaceChildren` cannot resize it), and `ROW_PX`/`COL_PX` under-estimate, so the worst case is a row of over-fetch. What *is* verified end to end is the Rust half: `tests/browser_round_trip.rs` drives improvise-1ey's stated scope against a real `Host` over a real `App` touching the client only through the four string calls. One deployment note that is not a bug: `examples/demo.improv` leaves `Customer`/`Date` on the None axis, so **every** cell is an aggregate and nothing is editable — correct behaviour, bad demo. Pick a model with no regular category on None. - -43. **A batch carries its basis, and the server had to publish it first.** `Command::Model { basis, cmd }` is a **struct variant** so the basis cannot be forgotten, and `Sessions::apply_model` checks it *before* applying anything — a stale batch is never applied, not even partially (improvise-xwi6/fkfg, ADR 001 layer 1). Four things to know. **`Basis` has no `Default` and no public constructor**: a zero-valued basis is not "no claim", it is the claim that *the document is fresh*, which in a new `App` is true — so a default would be a forgeable bypass rather than an absence. `BasisCache::claim` is the only door a client has and it returns `None` until the server has published one. Note what that does and does not buy: `Basis` derives `Deserialize` because it must cross the wire, and its fields are public, so **a hand-rolled client can state any tiers it likes**. This is an *anti-staleness coordination check, not a security boundary* — `RecordUnstated` closes **omission**, not **invention**, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having: it makes the honest path the easy one. **Which records are checked is the server's decision** — `reduce.rs::records_named`, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the *client's* claim would be a **force by omission**: the record tier skipped, with no flag to remove and nothing in the message saying so. A claim that omits a record the command writes gets `Rejection::RecordUnstated`, a case of its own because "you lost a race" and "you were never in one" are different things to tell a user. **`Rejection` keeps structural and data staleness apart**, which is the whole reason ADR 002 put a maintained `RecordHash` index in `DataStore`: `RecordGone` means nothing resolves and there is nothing to force (**force never creates structure**), `RecordMoved` means it resolves at another version and force is one keystroke — which is also why the wasm client stores the `Rejection` itself rather than collapsing it into `status_msg`, since a string cannot carry which record or which kind. **And the ADR omitted the publication.** It said a batch carries the basis it was composed against and never said how a client learns one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. `Projection::Basis` + `BasisCache` close that — the server publishes the two document tiers plus the counters for the records *this client's grid names*, diffed against what that client was last told and seeded in the bootstrap — and it is emitted **after** the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown, the mirror image of gotcha 33. That ordering was documented as load-bearing in three places and **held up only by the order of two statements** until `a_published_basis_follows_the_grid_updates_it_describes` pinned it (proved by injecting the swap) — the same job `a_cell_update_that_overtakes_its_labels_is_lost` does for the mirror-image rule. Two things are unchecked and deliberately so: `Command::SwitchView` carries no basis (it sits outside `Command::Model`; the workbook refuses a view it does not have, so it cannot corrupt, but it is a hole in the rule), and `apply_command` does not check at all — **the check is a property of the session door, not of the mutation path**, which is right because a host holding the `App` is the truth rather than a claimant. That second one **was** a door rather than an exemption, and needed a type to stop being one (improvise-ondj): both took a wire `Command` and their `Command::Model { cmd, .. }` arm **discarded the basis with `..`**, while `ui/reduce.rs`'s own module doc pointed every future host — the browser worker included — at exactly those two names. **The struct variant closed omission *from the message*; it did nothing to close ignoring *on arrival*, which is the half a struct variant is usually reached for.** So both now take `Checked<'_>`, a private-field newtype over a `&Command` whose only constructor is `Checked::against(&App, &Command) -> Result<_, Rejection>` — also the only caller of `stale_basis`. `Sessions::apply_checked` receives the proof and `Checked::against` is answered **once, before the dispatch**, so `apply_switch`/`apply_model` receive the proof rather than each remembering to ask: an invariant made unrepresentable rather than validated, the same answer `prune_axis_items` and `gp-tree`'s `Row` got. Visibility was the weaker option and was rejected for a concrete reason — `pub(crate)` closes only the out-of-crate path, and `reduce_full` had **no production caller anywhere in the workspace**, so it would have become dead code. It has since been **deleted** — which is the argument’s own conclusion arriving, not a contradiction of it; the surviving sibling is `reduce_key`. This paragraph named it in the present tense for some time after, so read every function name here as a claim to check. **The regression guard is therefore the signature**: the test that reproduced the bug no longer compiles, and what remains asserts the constructor refuses and that refusing leaves the document untouched. One residual hole, documented on the type and filed: a `Checked` does not name *which* `App` it was checked against, so a multi-document host could in principle check one and apply to another. That is also why `ui::reduce::caught_up(&App, cmd)` is not a bypass: it takes `&App`, so nothing reachable from a wire can call it. **Since improvise-955n the frame, not the command, is what is answered — and only two of the three tiers hoist.** `CheckedBatch::against` runs before anything in a `ClientFrame::Commands` is applied. `document_generation` and `layout_version` describe the *document*, so answering them once for the frame is exactly right, and a first version hoisted all three — which was a lost update: the record tier asks "is the client's picture of **this record** current", and which records are at stake is a property of the *command* (`records_named`). Skipping it for later commands let `[SetCell(North), SetCell(South)]` overwrite a South somebody else had moved, silently, with the frame reported accepted, and invisible to the other two tiers because a `SetCell` is `CellData` and moves neither. So every mutation answers for its own records **against the frame's basis** — the one the first basis-carrying command stated, never each command's own field, or a later command could claim a fresher picture than the frame was composed under. The one exemption is stated over a *property* rather than a list of commands: a record that **neither resolves in the document nor is claimed by the basis** is one the frame is about to create (`[AddItem(East), SetCell(East)]`, which `BasisCache::claim` cannot possibly claim for), and only that combination is exempt; a record that does not resolve but *is* claimed is still `RecordGone`. That the `Basis` inside a *later* `Command::Model` is ignored rather than compared is an honest interim: clause 1 says the **frame** states one basis, and moving the field from `Command::Model` to `ClientFrame::Commands` would make that unrepresentable-otherwise instead of a convention — a protocol change that belongs with clause 4. Two things worth keeping from the rejected design: **arity is not a property of a variant** (a future `Command::RunScript` is one variant performing unboundedly many mutations), so no count over the vocabulary can enforce "one mutation per frame"; and **the transaction boundary is not the frame**, because one command may already be a chain, which is why clause 4's mechanism belongs at `App::apply_effects`. - -44. **The TUI grid draws from its cache, and since improvise-anr9 the cache frames it too.** `GridWidget` holds no `&Model` and no `&View`: cell text, aggregate-ness (`CellId::Aggregate`, ADR 002), column widths, the page bar, the totals strip and now the **frame** all come off `GridCache`. The TUI still builds that cache over the **whole** grid (offsets zero), so a cache position is an absolute position — the browser's `Viewport` is a real window, the TUI's is not, and the two clients therefore use it differently; column totals **must** be projected rather than client-derived, because the sum runs over every row *including the ones a viewport scrolled past*. The frame is an `AxisFrame` per axis — `cats` (the categories, outermost first) plus `entries: Vec` interleaving `Group(FrameGroup)` headers and `GroupEnd { level }` markers with `Item { tuple, collapsed }` label *tuples*, exactly as `GridLayout::row_items` interleaves them. That replaced a `Vec` built by `GridLayout::row_label`, which slash-joined the tuple — the exact rendering `two_row_categories_produce_cross_product_labels` says must not appear — so the browser drew `Food/Alice` where the TUI drew two header columns, with no groups and no collapse indicators. It is **one** type and not a `labels` plus a `levels` beside it: two answers to "what are the labels" is the disease (improvise-edp), and this is the second cure, which cost a *type* change rather than an addition. Three things to know before touching it. **`leaf_labels()` is not "the labels"** — it is the innermost label, what a one-line summary calls a row by; a draw loop wants `header_labels(first_rendered)`, `groups()` and `shares_ancestry(sel)`, each of which is one pass, because `item`/`label`/`shows_level`/`group_of` scan `entries` from the start and asking them per position is quadratic in the frame *and grows with the TUI's absolute scroll offset* (found three times on anr9; the per-position forms survive for one-off lookups and say so). **The width rule has one home**, `improvise_protocol::header_level_widths` — widest label per level, plus a gap, clamped — reached by `AxisFrame::header_widths` and by `ui::viewport::compute_row_header_width`, which borrows level tuples off a `GridLayout` rather than building a frame on the keypress path. **A `cats` change re-identifies every position**: a cell's identity is `cats` zipped with its tuple, so `invalidate_relabelled` blanks the whole axis when the categories move, not merely the positions whose tuples differ, and blanks the **totals** on a recategorization of *either* axis — a column total is a sum over the rows, so re-identifying the rows leaves a number answering coordinates the grid no longer has (gotcha 33(b): the rule lives in `apply`, not in a producer's memory). What is still **not** on the cache is `record_col`, and that is correct: it keys pending drill edits and is session-local (gotcha 1). `body_from(idx)` takes an argument only because the two front ends scroll differently — a browser's frame is already windowed and passes `0`, the TUI's covers the whole grid and passes its row offset — not because the rule differs. `group_of` answers group **membership**, and since improvise-e265 membership is **the innermost open run**: `FrameGroup` and `FrameEntry::GroupEnd` each carry a `level` (an index into `cats`), runs **nest**, a header at level L replaces L and closes every deeper run, and a `GroupEnd { level }` closes L and deeper. Before e265 it was the nearest preceding header unless a level-less `GroupEnd` intervened (improvise-gd83, PR #77), which could not express one run inside another. `GroupEnd` draws nothing and exists only so membership is not inherited. `AxisFrame` deserialises through `UncheckedAxisFrame` and **refuses** a `level >= cats.len()` — a level naming no category is a malformed frame, not a clamp. And `FrameEntry::Item` is `{ tuple, collapsed }`: `collapsed` is the **layout's own fold flag**, read by `AxisFrame::folds`, so a client asks the frame whether a position folds rather than re-deriving it from the groups (improvise-4k1j). Both of the gaps this sentence used to list are **closed**, and it named them as open for a day after: `group_runs` coalesces by **header identity** (`std::ptr::eq`), not by name, so two runs under two headers equal as values are two runs (improvise-h0iv); and `AxisFrame::first` carries **no** `serde(default)` — its doc comment says why, since zero is a claim and not an absence (improvise-1wl0). `doctrine(a_stale_list_of_what_is_missing_is_worse_than_no_list)` is about exactly this paragraph, twice now. And `project::grid_of` used to evaluate every cell per frame where the widget evaluated only the visible ones; since improvise-felo.13 the host's own layout keeps a `GridSummary` (whole-grid widths, column formats and totals, `ui/viewport.rs::grid_summary`, held in `App::grid_geometry()`), and `grid_in` evaluates only a `grid_window` over it. The browser's window therefore costs its own cells. A session on another view, or a rederived model, still takes the whole-grid `grid_body` path. `grid_window` equals `grid_body` where the summary is current. Since improvise-jytm `grid_in` may additionally **derive a `GridLayout`** for a session on a view the host is not on; the host's layout is reused where the names agree, which is the common case and the only one that can carry a drill snapshot. - - **`RecordHash` is document-scoped; `AggregateHash` is layout-scoped and cannot be otherwise** (improvise-gd28). A `RecordHash` resolves through `DataStore`'s maintained index — but **only when the command names a view the document has** (improvise-wpxr; Codex P2 on #281). `SetCell`/`ClearCell` carry `view`, and the lookup is **strict**: a view the document lacks drew nothing, stored or not, so it is `Rejection::RecordGone` rather than a `resolve_view` fallback. An **empty** record — one the store holds no cell for, which the grid still drew — resolves against **that named view's layout**, the only place it exists. `RenameRecordCoordinate` is the exception and resolves in the **store alone**, since only a record holding cells can be renamed: `protocol::named_records` answers each command's records as `NamedRecord::Drawn { view, record }` or `NamedRecord::Stored(record)`, one exhaustive match. Every one of these questions goes through `App::grids()`, a `Grids` borrow that memoises **one derived `GridLayout` per view for as long as the borrow lives** (improvise-kimw) — hold one across a check, never across a mutation, since the layouts it holds describe the model it borrowed. An aggregate has no existence outside the layout that derives it, so there is no index to fall back to and none can be built — `App::resolve_aggregate` therefore answers only for the **host's** active view, which since improvise-jytm is not the client's. **That, and not ADR 001 layer 2, is what blocks browser drill.** The ticket's blocking argument was right about the *commit* (`ApplyAndClearDrill` is addressed to one session, gotcha 38) and wrong about the *entry*: entering a drill needs no commit, and `CreateView`/`SetAxis`/`SetPageSelection` are already unaddressed `ModelCommand`s that name their view. The gap is one level down from where it was looked for — **the mutation and its operand have different scopes**: the mutation configures the shared `_Drill` and is addressed to nobody, while the operand exists only inside one session's layout. So the fix is not a new mechanism but the `view: String` convention every other view-scoped variant already follows, resolved against a layout derived for that name (improvise-obtu). Two things travel with it: editing *inside* a drill grid from a browser does not follow from it, because `edit_or_drill` declines records-mode editing out loud before reaching a cell (improvise-fdk1, since closed, is the separate records-coordinate rename on the wire — `RenameRecordCoordinate`, gotcha 38 — and did not unblock drill editing); and closing it makes improvise-t219 live, `_Drill` being one shared view in `Workbook::views` that two clients drilling would contend for — the same masking relationship improvise-6uva has with improvise-up8g. Guarded by `an_aggregate_a_session_was_shown_does_not_resolve_against_the_hosts_view`, whose **control** is the load-bearing half — the same surrogate resolves once the host is moved onto that view, so a blanket `None` cannot pass. - -45. **One session loop, two transports — and the browser saves itself.** `Host` is `App` + `Sessions` + `handle(id, ClientFrame) -> Vec` and lives in `improvise_command::ui::host`, **not** in `improvise-ws-server`, which re-exports it. It moved for improvise-9x6: the worker hosts the same authoritative `App` over a `MessagePort` and cannot depend on tokio, and a second copy would have been exactly the mirror improvise-xks6 had just deleted for the frames — a drifting `Applied`/`Broadcast` match is invisible until a client sees it. `registry` stayed on the server, because a worker hosts **one** document. What the worker adds is only what a `MessagePort` differs in: **one client by construction** (the `Addressed` filter is honoured anyway, against a future `SharedWorker`), and **a handshake with no close** — `ClientFrame::Viewport` must still come first, but a pre-handshake frame is *dropped* rather than closing a connection that does not exist. - - **Saving is a frame, and dirtiness is read off the classification the effect already carries.** `effect::Save` is `ViewOnly` (gotcha 3) — correctly, since it writes *this host's* `file_path` through *this host's* `Storage` — so the derivation that builds `ModelCommand` from the model effects cannot name it, and for a while that meant no client on any transport could ask its host to save. `ClientFrame::Save` closes it (improvise-368h): a **request** carrying no path, because a client cannot see the host's storage, answered `ServerFrame::Rejected(NoDocumentPath | SaveFailed)` to the asker alone and **silently on success** — nothing moved, so no widget is dirty. It is a frame and not a `Command` for one concrete reason: as a `Command` it put a runtime-fallible operation inside a batch `ClientFrame::Commands` promises is all-or-nothing, so `[Save, SetCell]` reported a failed write and then applied the edit. It also fails the test every `Command` passes — it mutates no shared state and no client's slice, so `reduce_view` has nothing to fold and `Checked` has no basis to answer for. **And `MarkDirty` is gone.** It was a separate effect the *command* layer composed beside each mutation — reasonable on the key path, unreachable from anywhere else, because improvise-kmgs derived `ModelCommand` from the effects **one at a time** and so put `SetCell` on the wire without its companion (improvise-gv9r): a derivation correct per element and wrong for a set, whose symptom was that "save if dirty" never fired. `App::apply_effects` now raises the flag from the effect's `EffectKind`, which makes omitting it unrepresentable rather than validated — **declared** for `WholeWorkbook` (a replaced document differs from the file its path names, so a `LoadModel` that cannot read its path still raises it, costing one redundant write) and **observed** for `CellData` and `LayoutGeometry`, where `document_content`/`document_config` fingerprint the document before and after, so a write that leaves it as it was raises nothing (improvise-748f). The observation is read inside the loop, and `Save` clears the flag in loop order, so a batch that edits and then saves ends clean. The worker's persist trigger is therefore the **flag**, not a table of command variants: an observation rather than a second description of which commands change the document, for the reason `cursor_epoch` beats matching on `CycleAxis` (gotcha 24) and `document_generation` beats matching on `LoadModel` (gotcha 39). It is also strictly tighter — a refused batch raises nothing, and neither does a write that changes nothing, so the "was anything rejected" guard went with it. A browser still never *sends* `ClientFrame::Save`, and that is the answer rather than a gap: there is no file the user owns and no Ctrl-S semantics, the document *is* the OPFS entry, so the worker persists on its own behalf through the one existing `Save` effect. The cost that remains: `Storage` is whole-document (gotcha 41), so OPFS rewrites the file per accepted model command. - - **The async seam does not survive being boxed.** `OpfsFs::open`/`attach`/`refresh`/`commit_deletes` are on the struct, not on `Storage`. `App::storage` is a `Box` and a trait object can name none of the four, so **handing the backend to the `App` puts the async seam permanently out of reach**; the host must hold the concrete backend *and* give the `App` a `Storage`, which means sharing one. That is `impl Storage for Rc`, and it lives in **`improvise-io`** rather than in each worker (improvise-hmws): `impl Storage for Rc` is two foreign types in any other crate, so improvise-9x6 wrote a forwarding newtype that every future worker would have written again. `Rc` and not `Arc` because the one caller is a Web Worker and wasm32 has no other thread. **The flat root is no longer load-bearing**: `OpfsFs::list` joins `dir` like the other two backends, so `storage_contract` takes `dir` as a parameter and the three agree at *every* directory rather than only at `Path::new("")` where joining is the identity (improvise-vhcz). What the index holds is **documents, not entries**, and the split between its three writers is the rule: `attach` indexes nothing (what it creates is zero bytes), `Storage::write` indexes the name and un-queues any pending delete of it, and `refresh` rebuilds from OPFS skipping zero-byte entries — counting an entry it cannot measure as a document, since the one cause of that is another tab holding the exclusive handle (improvise-vvl3). - - **What no test here can see** has shrunk, and precisely. `harness/` (improvise-vvl3/xf1) runs `improvise_io::storage::contract::storage_contract` against a real `OpfsFs` inside a real dedicated Web Worker under Chromium and Firefox, and the `browser` job in CI can fail the build — so **`OpfsFs` is no longer compile-checked-and-never-run**, and two bugs fell out of its first execution. That is why `storage_contract` stopped being a `#[cfg(test)]` helper and became a public function returning `Result`: a browser must run *that* sequence, not a second transcription of it, and a failure has to travel out of a worker as a string. Still unseen: **Safari** (Playwright's WebKit has no OPFS, so xf1's criterion is met for two engines of three; `opfs.spec.js` skips on a runtime probe and an unexpected skip on any other engine is a failure). **`bindings.rs` is NOT in that list, and said it was for months.** `shell.spec.js`'s "the standalone deployment" loads the app with no `?ws=`, which spawns the worker, which imports the glue, which calls `bindings.rs::start` and `OpfsFs::open` — and the spec then asserts a grid draws, so the whole chain has run in CI since improvise-vvl3 (Chromium, and Firefox since 2026-09-20). The claim survived in **five** places (this line, the File Inventory, two `kb.pl` facts and improvise-bck's grooming note) because each was written before the harness landed and none was re-read after; the spec's own comment says the chain plainly. It is the reason to distrust a "never run" written in prose: nothing checks it, and `kb.pl`'s CI gate checks consistency rather than truth. What was already verified stays: `tests/post_message.rs` drives a real `Worker` over a real `App` on nothing but the JSON a `MessagePort` carries. Build note unchanged: the worker bundle needs `--enable-nontrapping-float-to-int` **as well as** `--enable-bulk-memory`, because it carries the whole `App` and so chrono's date arithmetic. - -46. **The page picks a peer, and the demo was built against an assertion.** `web/app.js` holds two transports and no protocol knowledge: `?ws=` selects `improvise-ws-server`, its absence spawns `new Worker('./worker.js?doc=&seed=', { type: "module" })` (improvise-djm). `type: "module"` is mandatory — `worker.js` `import`s the bundle — and there is **no `MessageChannel`**: a dedicated worker's own `postMessage`/`onmessage` *is* the port, so the ticket's channel wording was a date rather than a decision. The websocket path is **kept**, because the ws-server is how a TUI-hosted document is shared and nothing else in the browser can reach it; the cost is contained because the two differ only in *where strings come from* — handshake, fold, repaint and keyboard are all above the seam — and **the presence of the parameter is the whole selection mechanism**, a second knob being a second thing to get wrong. `#status` composes **two** facts, the transport's and the document's, because a single source let a repaint silently erase a connection error. **A worker that rejects in `start()` fires no `error` on the main thread** — `worker.js` catches its own — so the guard is a **first-frame watchdog**, deliberately *not* per-transport: "the peer never answered the handshake" is the one failure that means the same thing over a socket and over a port, and a check aimed at each peer's own failure modes would have missed exactly this one. - - **The demo is a model built to satisfy a check, not an axis flipped and hoped for.** `examples/demo.improv` leaves `Customer`/`Date` on None, so every cell is an `Aggregate` and nothing is editable — and the naive fix does not work either, since `Product × Region` by `Date_Month × Measure` still aggregates. `web/demo.improv` puts **exactly one category on each visible axis** (`Product` row, `_Measure` column, `Quarter` page, nothing regular on None) and carries `Profit = Revenue - Cost`, so the payoff is on one screen with no keystroke. **Settability is asked of `grid_of` as a `CellId` variant**, never re-derived from the axes — a second derivation could agree with itself while disagreeing with `render.rs::editable`, which is the whole point of ADR 002 — and the assertion was proved discriminating by pointing it at `examples/demo.improv`, where it names the aggregate at (0,0). **One category per axis is a property of the demo, not of the format**: when this was written `GridCache` slash-joined a multi-level tuple, so a second category on an axis put that known gap on the front page — improvise-anr9 has since closed it (gotcha 44), so what remains is the demo's own simplicity rather than a rendering limit. `web/*.improv` is un-ignored in `.gitignore` (`*.improv` is ignored repo-wide) and the CI assemble step copies it: leaving it out is an empty grid, not a 404 the page could report. One wart it now puts on the front page: a formula cell **is** a `Record`, so it is offered as editable and a write to it reverts on recompute — pre-existing and matching the TUI, newly visible. - -47. **Keys become commands in the browser, and the server must not know the keymap.** The user's ruling on improvise-bpq2, following from "the model is shared, the UI is not" — and it buys two clients different bindings over one document. What made it urgent is **not** latency. `a_key_resolved_on_the_host_moves_state_no_client_can_hear` shows nothing in the projection stream carries the host's own `ViewState`, so host-side resolution **diverges**; and `Host::handle` answers `ClientFrame::Key` with an explicit empty vec, so **an upstream key is a black hole, not a slow path** — every `KeyOutcome::Upstream` names a binding that does nothing at all in a browser. That test still holds and was not flipped: bpq2 closes the divergence by *avoiding* it. **The method is the ticket's** — enumerate the **bindings**, not the commands (`doctrine(binding_dump_diff)`) — and where a client cannot compose the `Command`, **enrich the projection** rather than moving resolution back. - - Three things that cost. **`GridExtent` on `GridCache`** — the whole grid, of which the labels are one window — because a viewport-indexed cache cannot tell "the last visible row" from "the last row". It is deliberately **not framed by the labels** (`fit_to_labels` leaves it alone) and its batch position is not load-bearing the way `RowFrame`-before-`Cells` is, since it addresses no position; the producer reads `GridLayout::row_count()`, and reading it off the projected ranges is **invisible in the TUI** (which builds at offsets zero, gotcha 44) and is exactly the browser bug. **The window is the client's own fact**, recorded by `Client::set_viewport` from what it announced — *not derivable*, since the row frame's own length is the window **clipped to the grid** and so differs precisely at the bottom edge where the scroll arithmetic matters. `Client::viewport` (associated, no recording) still exists and is the one that quietly turns cursor keys off; `web/app.js` does call `set_viewport`. **`ClientFrame::Commands(Vec)`** replaces the singleton, because one cursor key is `SetSelected` plus the offsets `viewport_effects` returns and a host that folds the selection but not the offset projects the wrong window. It is **answered whole, then applied whole** (improvise-955n): `CheckedBatch::against` checks the frame before anything runs, so there is no longer a moment at which part of a frame has landed and the rest has been refused, and `Applied::Rejected` is unreachable through `Host::commands` — still matched, because a host that wants the `Dirty` must name every case (gotcha 39). The contract is four clauses and **all four are now delivered** — clause 4 by improvise-yqpd on 2026-09-21, whose mechanism is `App::snapshot`/`restore` plus a `Sessions` clone taken in `Host::mutating_frame` and put back by `undo_frame`, deliberately *not* inside `apply_effects`, which runs on every keypress. **This paragraph said it was undelivered for a day after it landed**, and an orchestrator brief repeated that from here, so read the clause list as a claim to check. A delivered clause 4 is also what makes the CLIENT's rollback well-defined (improvise-nxah): a refusal means the host applied none of the frame and restored every mirror, so a client keeping its optimistic fold is the only thing left out of step. What rolls back there is `ViewState`, never a cache — a `Command::Model` is a no-op in `reduce_view`, so the optimistic fold never reached one. The old text, kept because the hazard it names is still the right shape to fear: one basis per frame, checked once, applied in order with each command seeing the last one's state, and all-or-nothing. was that the fourth was open because `App::apply_effects` aborts without undoing (gotcha 21 — an abort is not a rollback), so `[RemoveItem("North"), SetCell(the North record)]` would land the removal, resolve the write to nothing, and report the frame accepted having applied half of itself — partial *and* silent. That is what `yqpd` closed, and it closed it **outside** `apply_effects` for the reason the old text implies: the abort lives on the keypress path and must stay cheap. The old note that it was *unreachable* from any client here, the only multi-command producer being `keys::commit`, still reads correctly as a caution — **reachability is a reason not to rush a fix and not a reason not to notice one**, the lost update `CheckedBatch` fixed having had the same reachability argument and none of the documentation, which is how it shipped. - - Two corrections to the ticket's own gap list, both worth more than the code. **Panel cursor movement was never a gap**: improvise-edp had already made `viewmodel::flatten_cat_tree` the one home both ends call, so it is the *shared* derivation, not a second one — worth re-checking the "known gaps" in other old tickets against what has landed since. And the "unmodified characters only" rule was **mis-scoped, not wrong**: it was about `KeyPattern::AnyChar`, which is bound only in the text-entry modes, so `Ctrl-d` in Normal has an exact binding and mirroring it is a faithful subset. Each resolver mirrors its `Cmd` **line for line, including where siblings disagree** — `Move` and `MoveTileCursor` emit even when they clamp, `MovePanelCursor` returns nothing when the cursor does not move. Tidying that into one rule would make the client a second derivation, which is what `tests/lockstep.rs` exists to rule out. What still needs the host is **one list**, `resolve::NEEDS_THE_HOST` in `improvise-protocol/src/resolve.rs` (improvise-uln5) — this line named a list in `keys.rs` and a test called `the_normal_mode_keys_that_still_need_the_host_are_these` for some time after both had gone. Three tests hold it, one per direction: `the_command_names_that_still_need_the_host_are_these` (no **false** entry: each name is bound and really is silent), `a_help_row_the_browser_cannot_run_is_drawn_marked` in `wasm-client/render.rs` (the help overlay **marks** those rows, improvise-uln5), and `every_silent_binding_is_on_the_list` (every binding of **every mode a browser can reach** acts, declines out loud, or is on the list — improvise-oyaa, improvise-s2gb). The last replaced a check read off the help rows, which could not see a binding no help row names. Closing a gap therefore means deleting its entry, and the first test fails until you do. - - **What the browser client decides before a frame leaves it** (all in `improvise-protocol`, so both ends share it). Five rules, each bought by a bug: - - **Every client cache read goes through `projected_at`**, which subtracts the frame's own origin (`AxisFrame::first`) before indexing. A position outside the window the cache holds is **unknown**, never the cell at the same cache index (improvise-7mro) — once the client scrolled ahead of its cache, an index read answered a different cell's value. `unprojected` names that state, and a cell op on it waits rather than guessing. - - **The keyboard commit writes `ClientState::opened`**, not the cursor. `OpenedCell` captures the record, the `OpenedTarget` (`Value { measure }` or `Coordinate { category }`), the record's version, the `document_generation` and the view **at open**, so a commit lands on the cell the editor opened on however the cursor moved since (improvise-8xq0), and claims the basis it was **opened** against rather than one read at commit, which silently overwrote a value another client wrote while the editor was open (improvise-xz2z). A retry after a crossing `LayoutMoved` must match the **whole** `OpenedTarget`, not the record alone, or it wrote whatever cell was under the cursor by then (improvise-8phw for values, improvise-9ww8 for renames). - - **`refused_here` exempts a captured editor** (improvise-3h9w): with `opened` set, `commit-cell-edit` is answered by the capture, so the cursor's current cell cannot refuse a commit aimed elsewhere. - - **A decline that must stop a binding is `Step::Declined`**, not a `SetStatus` beside the binding's other steps: Enter's welded `clear-buffer` otherwise still ran and threw away the typed text (improvise-joed), and a touched editor over a declined cell went `Upstream` with no word at all (improvise-ik6l). - - **`ClientState::notify` queues a client-side notice at the tail of `in_flight`**, as if it were a sent frame, so a rebase that rebuilds the view from `settled` and replays `in_flight` replays the notice too instead of erasing it (improvise-n0pb). `settle_status` runs on **every** path that rebuilds the view (bootstrap, accept, refuse, relay), and assigns a message the editor open as it arrives, so an old message loses its owner once that editor closes (improvise-0dbk). `EditorSession`, the key it is owned by, lives in `improvise-protocol/src/lib.rs` beside `ViewState` so both front ends ask it (improvise-su4u, improvise-1x9h). - -48. **The browser JavaScript is argued *and now partly executed*, and the two rules it can break are both about *when*.** `web/app.js` and `web/worker.js` were syntax-checked, `.d.ts`-checked for call arity and read — never run — and every bug in them was found by reading. `harness/tests/shell.spec.js` is the first execution of either: it runs `app.js` under Chromium against a mocked peer, and it exists because PR #21 declined to add regression tests for its own close-listener fix on the ground that there was nowhere to put them. That decline is only honest because the assertions live there now, and **a change to `web/*.js` belongs in that spec directory** rather than in a second ad-hoc surface beside it. Playwright rather than a headless-DOM shim because what goes wrong in these files is a real Worker, a real WebSocket, real OPFS and real event ordering, none of which jsdom has. Two classes of bug still follow from *when*, both found by review. **An `async` `message` listener does not serialize.** `worker.js`'s `handle()` awaits `server.flush()`, so the event loop may start the next message while an earlier one is suspended, interleaving two `on_message`/`flush` pairs and letting a later frame's replies **overtake** an earlier frame's — which gotcha 33 answers by *dropping* the overtaken update, not by replaying it. Every frame is therefore chained onto **one tail promise** and the listener is deliberately not `async`; that also turns "frames that arrived before `start()` finished" into an ordinary case of the ordering rule instead of a drain loop, which had the same race twice over because `server` was assigned *before* the loop ran. **A watchdog must be disarmed, not pre-empted.** The handshake watchdog exists for one condition — the peer never answered — and every *loud* failure (worker `error`/`messageerror`, socket `error`) has already written something more precise; the timer then fired 8s later and replaced it, so `setShellFailure` clears it. Its neighbour, the socket's `close` listener overwriting a precise `error` with "disconnected", **is fixed** (improvise-taxx, PR #21) — and the ticket's own proposed mechanism was wrong: `readyState` cannot discriminate, an `opened` flag is the only thing that can. PR #21's *other* half is the one that keeps getting lost: it routes **both** close arms through `setShellFailure`, which disarms the watchdog, so a genuine mid-session disconnect is **not** overwritten eight seconds later. A bug report said it was, measured it honestly against a pre-#21 tree, and then "confirmed" it survived #21 by simulating the `opened` flag **and not the routing** — a simulated fix that is not the fix you named reproduces a bug the real code does not have, and reads exactly like a confirmation. The ticket was filed and closed invalid; the property is now asserted by `a socket that opened and then dropped reads as disconnected, and stays that way`, which runs past the 8s window, rather than argued in a comment that was wrong twice. - - **What the harness still cannot see**, and why the argued-not-tested discipline stays: the grouped and multi-level DOM, the CSS, and the `ResizeObserver` extent arithmetic. What a careful read established about the last of these — the observer cannot feed back into itself (`#grid` is a flex child with `overflow: auto`, so `replaceChildren` cannot resize it) and `ROW_PX`/`COL_PX` under-estimate, so the worst case is a row of over-fetch — is still a read and not a test. - - **A third thing no test here can see, which is why the docs must be read as code.** Three comments in one range said "not built yet" about work that had landed *in that same range* — `README.md`'s browser section, `app.js`'s keyboard note, and `Host::handle`'s `Key` arm — each because a follow-up commit updated **one copy of an argument and not its siblings**. When a fact is stated in more than one place, changing it is a grep, not an edit. - - **An evidence heading may be module-qualified, and must be when its leaf is not unique.** `scripts/check-comment-runs.py --check-evidence` takes its Rust names from `cargo test --workspace -- --list` minus the `--ignored` twin, as **full module paths**, and **refuses** an ambiguous leaf naming every candidate rather than warning. Six leaf names collide in this workspace. Note neither half suffices alone: rename one of a colliding pair and the leaf is unique again and resolves silently to the survivor, so qualification is the fix and the refusal is only what forces someone to write it (improvise-rm4s). The residual, filed: `--list` does not name the binary, so two crates sharing a module path are indistinguishable (improvise-sxsf) — **measured latent** (1444 names, zero collisions) and the proposed per-package fix **declined twice over**: 18× the cost (45.2s against 2.4s, and `--check-evidence` needs the `--ignored` twin, so 22 cargo invocations), and *a package is not a binary* — `improvise` alone has five integration test binaries beside its lib, and an integration test's top-level `fn` lists with **no module path at all** (60 of 1444 names are bare), so `-p improvise` prefixes all six identically. Per-**target** is the only shape that would close it. - -49. **A websocket handshake is exempt from the same-origin policy, and that is a server-side problem.** There is no CORS preflight, so any page a user visits can open a socket to `improvise-ws-server` *from that user's own allowed address* and read and rewrite every document it holds — cross-site websocket hijacking (improvise-kc7n). **No network control can see it**: an IP allow-list, a private network and loopback binding all answer "may this **client** reach the endpoint", and the honest answer is yes; only `Origin` says what is **driving** the client. **Loopback is the least protected case, not the safest** — no proxy in front of it, and loopback is not a boundary a browser enforces. `src/origin.rs` is the whole decision and `transport::admitted` its one caller; it refuses the **upgrade** with a 403 rather than accepting a socket and ignoring it, and `serve` **takes** the policy so a host cannot forget to apply it. Three arms are decided rather than defaulted: **loopback allowed** (the shipped local recipe serves its page from `http://localhost:8080`, so the one case worth optimising for must need no flag), **absent allowed** (no browser omits `Origin`, so absent is a non-browser client an address check already answers — and refusing it breaks `tests/socket.rs`, the repo's own instrument), and **two refused** (`HeaderMap::get` is first-wins, so a proxy that *appends* rather than sets leaves two, and answering from the first lets the second be anything). The deployed Pages origin is deliberately **not** hard-coded: it can only reach a plain-`ws://` server through a TLS proxy anyway, so one flag is an honest cost on an already multi-step path where a baked-in default would be a permanent hole for every other deployment. The residual hole is **stated rather than guarded** — a proxy that *strips* `Origin` turns the check off, which is why the proxy's own allow-list stays in `README.md` as defence in depth. **None of this is authentication**: it answers what page is driving a client, never who is connecting, and improvise-kpnr is still open. It correctly does **not** move to `improvise-command` beside `Host` (gotcha 45) — a `MessagePort` has no handshake and no headers, and a worker and its page are the same origin by construction, so this is genuinely transport-only and not a mirror waiting to drift. - - -50. **A composed TUI row measures in cells and names which side yields.** `draw.rs::fill_line` takes a `Keeps`, because the three rows that share a line genuinely disagree about it: the title bar keeps the **right** (the `?:help :q quit` escape hatch outranks an unbounded document name — improvise-t14d), while `minibuffer_line` and `draw_status` keep the **left**, which is what they already did, since right-priority there would let a long status message eat the text the user is editing. So the ticket's "give the right segment priority" was correct only for the title bar, and a global rule would have been the wrong shape. Widths go through `ui::text::truncate`, never `str::len` — and the ticket framed *that* half wrongly too: the title's own `·` separator is two bytes and one cell, so a byte count mis-padded **every** document, not only one with a non-ASCII name. - - **One key table on screen, and it is the one derived from the keymap** (improvise-rr8n). `TileBar` now advertises no keys at all: its literal read `Ctrl+↑↓←→ to move tiles` where those keys are bound to `enter-tile-select`, and the derived status line is a strict superset (`Space`, `h`/`l`, `Tab` appear only there). The ticket asked for the opposite — rewrite the literal, suppress the derived line — and was inverted deliberately, because suppressing a derived superset to preserve a literal that had already rotted keeps the stale table and deletes information. The **import wizard goes the other way**, and the asymmetry is the rule rather than an exception: its status line *is* derived, but from the **mode** keymap while the real keys are per-step inside `HandleWizardKey`, so there the derived table is the one that lies and `draw_status` suppresses the fallback for that one mode. **Keep whichever table is derived from the thing it describes.** Two blind spots make this class invisible: `every_hint_key_resolves_in_its_mode` **cannot see a string literal in a widget** (the same gap improvise-wk18 names for `help.rs`), and the widget's own characterization test renders at **80 columns**, where the hint does not fit and is never drawn — so the one test pinning that output was structurally incapable of seeing either hint. - - **A scroll is measured where it is drawn** (improvise-gtn7.1). The keypress path counted `visible_cols` from `term_width` while `draw_content` gave the grid `term_width` minus the 32-column side panel, so with any panel open the cursor scrolled to a column never drawn. Now `viewport::grid_width(term_width, side_panel_open)` is the one answer, reached through `App::grid_width`, and both `draw_content` and `App::cmd_context` read it; `App::side_panel_open` is the one answer to whether a panel took its column. The tile bar had the same shape: `first_tile` chose its scroll against a width that began one or two columns before where the tiles were drawn, and both now start at `tiles_left(start)`. The grid bug was latent from `b5418f2e`, when `compute_visible_cols` took `term_width`, and was exposed by `50e4a372`, which opened the category panel by default. Still open: columns of mixed width, where a `visible_cols` count taken at the old offset is applied at the new one (improvise-gtn7.2). - - -51. **The command language reads pipes, refuses surplus, and checks names when it applies (improvise-31oe.2, 31oe.3).** Reasoning: `context/code/cli.md` §14 and §15. (a) A `:` argument is read through `improvise_formula::quote`, so `create-view |By area|` names `By area`; a **surplus argument is refused** rather than ignored, and `cmd::core::only_when_omitted` gates the *cursor fallbacks* of `clear-cell`, `yank`, `paste` and `drill-into-cell` so a coordinate typed by the user is never overridden by the cursor. (b) `set-axis`, `set-page`, `toggle-group`, `hide-item`, `show-item` and `switch-view` return `RequireKnown(Needs)` (`effect.rs`) ahead of the real effect, built by `effect_cmds.rs::unless_unknown`. It is `ViewOnly`, writes nothing, is **off the wire** (listed in `vocabulary_is_complete.rs`) and checks the **document at apply time** — a `:` line builds every segment before any runs, so a check in `execute` refused `create-view Second . switch-view Second` — then refuses and aborts the batch (gotcha 21). (c) A refused name therefore stops the rest of a `:` line, while headless `improvise cmd` rebuilds the context per segment and runs each as its own batch (improvise-31oe.14, open). - -52. **A date column imports as one hierarchical category, and the import refuses what it cannot keep (improvise-8jk6.3, 8jk6.6, d0n6).** Reasoning: `context/code/formula-and-io.md` ("Import: …") and `cli.md` §13. The default `DateShape` nests Year > Quarter > Month (the chosen levels) in ONE category named after the column and opens folded to its years; `DateShape::Split` is the old flat split, chosen by `--split-date ` headless or `s` on the wizard's date step. `Plan::spelling_clash` refuses a raw, unparsed date spelled like a generated item or group (`2024`, `2024-01`) because the two would silently merge — asked at the wizard's ConfigureDates step **and** again at build time, so a headless import cannot reach it either. Headless, `--strict` turns a partial skip into a refusal before the build, and a folder whose CSVs are all header-only (no records) is refused as an import that kept no record. +1. **Records mode is wide format.** A row is a coordinate tuple; columns are `[coords…, +cat, raw measures…, +]`. Editing a category column moves every measure cell in the row. `_Dim` carries a column *index*, resolved by `GridLayout::record_col`. `GridLayout::new` does **not** prune; `with_frozen_records(m, v, None)` does — production code wants the latter. `prune_axis_items` takes a predicate and re-marks survivors. +2. **Effects never touch `std::fs`.** Documents go through `App::storage: Box`. Autosave and the headless final write still bypass it (improvise-31oe.11). +3. **Every effect declares an `EffectKind`; there is no default.** `ViewOnly` never reaches other clients; `Model(CellData | LayoutGeometry | WholeWorkbook)` does. The view lives inside the workbook, so axis/page/format are `LayoutGeometry` and cursor/scroll/`SwitchView` are `ViewOnly`; `Add/RemoveCategory` are `LayoutGeometry`. `EffectKind` is too coarse to check the `ModelCommand` mapping — `every_model_command_names_the_effect_of_the_same_name` does. +4. **Commands never mutate.** `&CmdContext` is read-only; return effects. +5. **`CellKey` is always sorted.** Use the constructors. +6. **No `Model::add_item`.** Use the `Workbook` facade (`wb.add_item`) or go through the category. +7. **Virtual categories.** `_Measure` items = explicit ∪ formula targets (`measure_item_names()`). Cell aggregation (`apply_agg`) and formula `SUM/AVG` (`eval_agg`) differ on empty input; non-numeric values reduce by unanimity, else `Error("mixed")` — never dropped. `evaluate_aggregated` takes `Aggregation::{Exact, OverHidden}`. +8. **Display rounding is view-only.** `format_f64` only in rendering; never feed a rounded value back into eval. +9. **Formula eval is fixed-point.** `recompute_formulas` iterates to stability; cycles become `Error("circular")`. +10. **Drilling a formula cell** strips the `_Measure` coordinate so the raw feeding records come back. +11. **Minibuffer keymaps come from `TextEntry`** (eight text-entry modes); the buffer a prompt names and the keys edit are one string by construction. Read/write a runtime-named buffer with `read_buffer`/`write_buffer`. `SearchMode` is hand-written. +12. **Panel keymaps come from `PanelKeymap`**; `Panel::ALL`/`name()`/`key()` drive the toggle keys. +13. **A prefix key is stored as an address** (`ViewState::transient_prefix`), resolved by `Keymaps::prefix_keymap`; prefixes must stay flat (`prefix_keymaps_are_flat`). `ViewState` stays free of `improvise-command` types. +14. **Keybindings are per-mode** (`ModeKey::from_app_mode`); minibuffer Enter/Esc are `Binding::Sequence`. `every_hint_key_resolves_in_its_mode` checks which keys a hint names, not what they do. +15. **Registry names come from prototypes, not strings** — except the `Move` literals and four aliases, guarded by tests; a wrong-variant prototype shows up as a *duplicate* registration. +16. **`effect_cmd!` / `simple_effect!`** wrap one effect; `simple_effect!`'s kind argument is mandatory. Six wrappers return `unless_unknown(...)` (gotcha 51). Mode entry is `enter-mode `; no per-mode shim commands. +17. **One float epsilon** (`FLOAT_EQ_EPSILON = 1e-10`) for `=`/`!=` *and* the div/0 guard, in both evaluators. +18. **Order is data.** `IndexMap` order is persisted; an index of `HashSet`s is for lookup, never iteration (`DataStore::matching_*` sort to store order; `analyze_records` is first-seen order). A group's fold reads through the same rule (`spanned_positions`). +19. **Cell-edit commit paths split** (`commit_regular_cell_value` / `commit_plain_records_edit`); keep both in sync. `cmd::core::uneditable` is the one decline, asked at open *and* commit. `CellId::NonItem` is a stored value naming no item. +20. **Minibuffer commits go through `commit-buffer `**; the add effect is emitted last so its failure message wins. +21. **An abort stops the chain, never the bookkeeping.** `Save`/`SaveAs`/`AddCategory` set `abort_effects`; tidy-up (mode restore, buffer clear) goes at the *head* of a batch. An abort is not a rollback (gotcha 47). +22. **The import wizard's back path is a history stack.** Preconditions live in `validation_error()`; `ResolveDuplicates` appears only when `collisions()` is non-empty; headless defaults to `DuplicatePolicy::Refuse`. A collision is a cell written by two records. +23. **`app.layout` goes stale mid-batch** — put a `RebuildLayout` effect before any effect that reads the layout after a grid change. The rebuild is skipped when nothing it reads moved (felo.13); a writer bypassing effects must call `rebuild_layout()`. +24. **The cursor is per-client, keyed by view** (`ViewState::cursors`; use `App::cursor()/cursor_mut()`). Replacing the document clears the map and bumps `document_generation`. Cursor invalidation is *observed* from the grid frame (`cursor_epoch`), not called by effects; two arms (`LayoutGeometry` hashes the active view, `CellData` every view). +25. **`ViewState` and its field types derive `Serialize`/`Deserialize`** — no borrowed-`'static` fields anywhere it reaches. +26. **Axis presentation lives on `Axis`; colour is a `Face`** resolved by the `Theme`; `draw.rs::face_style` is the one translation to ratatui. +27. **`Axis::Filter` constrains without being navigable.** Visible data = `filtering_cats()`; user-actionable = `categories_on(Page)` / `tileable_cats()`. What a virtual view must be is model-side (`records_steps`, `drill_steps`). +28. **The browser backend is sync on purpose, with an async seam** (`OpfsFs`; `open/attach/refresh/commit_deletes`). It needs a *dedicated* worker — `createSyncAccessHandle` is absent in a `SharedWorker`. +29. **"Active view" is stored twice and may differ:** `Workbook::active_view` is the default, `ViewState::active_view` is authoritative per client. `App::resolve_view` is the one fallback; `Sessions::adopt_deleted_views` handles a deleted view. +30. **Two tests gate `ViewState` membership:** portable (no `Rc`/`Arc`) and snapshot-safe. `wizard` and `drill_state` live on `App` as session-local residue. +31. **`:` commands settle their mode by observation** (`ReturnToNormalUnlessMoved`), not by a per-effect constant. +32. **There is no aggregate render cache.** One cache per widget (`WidgetCache`), a `Projection` is one update to one widget; bootstrap is a *compaction* (`Snapshot { view, seed }`). The wizard has a cache but deliberately no projection. +33. **The grid's labels frame its cells; the fold is not commutative.** A `Cells` update that overtakes its frame is dropped; `apply` invalidates relabelled positions itself; every arm touching labels or widths re-fits. Don't coalesce a batch. +34. **A formula target is unquoted; pipe quoting has one home** (`improvise_formula::quote`). The keyword guard is canonical. `header_name` is total; duplicate `## Category:` sections are one category; `WHERE` needs `sp+`. +35. **`Dirty` over-reports on purpose** — a redundant projection is cheap, a missed one leaves a client wrong forever. `LayoutGeometry` includes `formulas`. +36. **A client's command has one door:** `Sessions::apply_checked(app, id, Checked)`. Assert the `Broadcast` itself, and seat a bystander where a wrong relay can move it. +37. **`gp-tree` returns data and never prints; a fit needs seed *and* `FitConfig`.** A scorer takes `Row` slices. `fitted_formula` refuses features the document cannot name; categories are forced `Categorical` (zv8t). +38. **The wire vocabulary is derived twice** (`Command` view half, `ModelCommand` model half). Completeness is checked (`vocabulary_is_complete.rs`). `ApplyAndClearDrill` and `AcceptWizardModel` are deliberately off the wire (session-local operands). Recount, don't cite. +39. **`Applied` is an enum (`Diff | Rebased | Rejected`)** because a document swap is not a diff; the trigger is the generation moving, not the variant. A swap fixture must keep the view *name*. +40. **A batch's basis is three tiers** (`document_generation`, `layout_version`, `record_versions`); the unit is the record; lower tiers are *observed*, not read off the effect. +41. **`Storage` is not a VFS;** SQLite was declined on measurement (a second serialization is a second grammar). +42. **Browser render decisions are made in Rust** (`RenderModel`: `editable` from the `CellId` variant, selection reconciled with offsets, `flatten_cat_tree`). `improvise_protocol::frame` is the one envelope pair for both deployments. +43. **A batch carries its basis, and the server publishes it** (`Projection::Basis`). `Basis` has no `Default` or public constructor; `Checked`/`CheckedBatch` make "ignoring the basis" unrepresentable; `Rejection` keeps `RecordGone` / `RecordMoved` / `RecordUnstated` apart. Anti-staleness, not security. +44. **The TUI grid draws from `GridCache`**, which frames it too (`AxisFrame`; width rule `header_level_widths`). Use the one-pass accessors, not per-position scans. `RecordHash` is document-scoped, `AggregateHash` layout-scoped. +45. **One session loop (`Host`), two transports.** Saving is a request frame (`ClientFrame::Save`); dirtiness is raised from the `EffectKind`, observed for data. `impl Storage for Rc` lives in `improvise-io`. `harness/` runs `OpfsFs` for real; Safari is unseen. +46. **The page picks a peer:** `?ws=` → socket, else a module `Worker`. `#status` composes two facts; the first-frame watchdog is per-peer-agnostic. The demo model is built so every cell is a `Record`. +47. **Keys become commands in the browser;** the server must not know the keymap. `GridExtent`, `set_viewport`, `Commands(Vec)`, `resolve::NEEDS_THE_HOST` and the five client rules (`projected_at`, `opened`, `refused_here`, `Step::Declined`, `notify`). +48. **The browser JS is partly executed** (`harness/tests/shell.spec.js`); changes to `web/*.js` belong there. Serialize frames on one tail promise; disarm, don't pre-empt, a watchdog. Evidence headings may be module-qualified (and must be when the leaf collides). +49. **A websocket handshake is exempt from same-origin policy** — `src/origin.rs` refuses the upgrade (loopback and absent allowed, two refused). Not authentication. +50. **A composed TUI row measures in cells and names which side yields** (`fill_line` + `Keeps`). Keep whichever key table is *derived from what it describes*. Scrolls are measured where drawn (`grid_width`). +51. **The command language reads pipes, refuses surplus, and checks names when applied** (`only_when_omitted`; `RequireKnown`/`unless_unknown`). +52. **A date column imports as one hierarchical category;** `--split-date` restores flat. `Plan::spelling_clash` refuses what would silently merge; `--strict` refuses partial skips. --- -## File Inventory +## File Map -Line counts are a **dated snapshot**, not a maintained fact: measured 2026-09-24 at `c1772ca9`, and **70 of 105 had drifted past 10%** in the three days since the 2026-09-21 refresh — eleven had doubled or more (`client.rs` 255 → 943, `lockstep.rs` 1126 → 4064). Treat them as "is this file big", never as a figure to cite; re-derive with `wc -l`. The `ws-server/` and `wasm-client/` entries use a shorthand prefix, so a refresh must map it to `crates/improvise-/{src,}/` by hand or they drift silently. Test counts are informational — run `cargo test --workspace` for live numbers. Files under 100 lines and render-only widgets omitted. +Per-crate entry points. Per-file notes, test counts and a dated line-count snapshot are in +`context/file-inventory.md`; re-derive sizes with `wc -l`. -### `improvise-core` (`crates/improvise-core/src/`) -``` -model/types.rs 3560 / 112t Model, formula eval (fixed-point), CRUD -model/cell.rs 1358 / 47t CellKey (sorted), CellValue, DataStore (interned + index), - live_records (insertion order — gotcha 18) -model/record.rs 226 / 8t RecordHash / CellId surrogates: the opaque identities a - thin client names a cell by (ADR 002) -model/category.rs 310 / 9t Category, Item, Group, CategoryKind -model/symbol.rs 89 / 3t Symbol interning -view/layout.rs 2604 / 57t GridLayout, drill, records mode, records_columns; a private - per-axis position index (`row_pos`/`col_pos`, rebuilt by `reindex`) - so a data position resolves in O(1) (improvise-felo.13) -view/types.rs 817 / 43t View config (axes, pages, hidden, collapsed, format) -model/id.rs 290 slugify + IdAllocator: how a generated id is made (ADR 007 - clause 4, core.md §20). Nothing outside the file calls it yet -view/axis.rs 190 / 5t Axis enum + label/next methods (no colour: that is a Face) -workbook.rs 413 / 22t Workbook: Model + cross-view ops, item facade -format.rs 519 / 40t format_f64, parse_number_format (display only) -``` - -### `improvise-formula` (`crates/improvise-formula/src/`) -``` -parser.rs 1394 / 79t pest grammar + tokenizer → Formula AST -quote.rs 160 / 7t quote_name / is_bare_name / pipe_quote / unescape_pipe_body: - the one home, both halves (improvise-i2m5). improvise-gp - re-exports it; improvise-io calls it from - persistence/{format,parse}.rs -formula.pest 108 PEG grammar for formula text -ast.rs 130 Expr, BinOp, AggFunc, Formula, Filter -``` - -### `improvise-io` (`crates/improvise-io/src/`) -``` -storage.rs 334 / 13t Storage trait (read/write/list/delete), PhysicalFs, MemoryFs, - `impl Storage for Rc` (improvise-hmws), and - contract::storage_contract — public, not cfg(test), because - a browser must run that sequence (harness/) -opfs.rs 289 / 0t OpfsFs: Storage over the browser's Origin Private File - System. wasm32 only. Run by harness/, not by cargo test -persistence/improv.pest 179 PEG grammar — single source of truth -persistence/mod.rs 441 / 11t document lifecycle: save/load/gzip/legacy-JSON over Storage; - facade re-exporting format_md/parse_md/export_csv* -persistence/format.rs 401 / 11t format_md: Workbook → .improv text -persistence/parse.rs 1684 / 46t parse_md: pest tree-walk → Workbook (+ grammar generator) -persistence/export.rs 243 / 5t export_csv / render_csv. Builds its layout with - with_frozen_records, NOT new — `new` skips prune_empty - (improvise-iot6) -persistence/roundtrip_tests.rs - 1236 / 62t format_md ↔ parse_md characterization (tests only) -import/wizard.rs 1967 / 62t ImportPipeline + ImportWizard. `Plan::place` is the one - placement pass, shared by `collisions()` and the build so - the two cannot disagree; `DuplicatePolicy` / `Collision` - (improvise-cgqt) -import/analyzer.rs 434 / 17t Field kind detection (Category/Measure/Time/Skip); - analyze_records is first-seen order, not HashSet (gotcha 18) -import/csv_parser.rs 590 / 26t CSV parsing, multi-file merge -``` - -### `improvise-protocol` (`crates/improvise-protocol/src/`) -``` -lib.rs 619 / 2t ViewState, ViewCursor, ViewFrame, AppMode (17), Panel, - MinibufferConfig, MinibufColor, HELP_PAGE_COUNT -reduce.rs 749 / 10t Command + ModelCommand (the wire format), HelpPage, reduce_view -render.rs 2157 / 46t CellDisplay, WidgetCache trait + one cache per widget - (GridCache/CategoryTreeCache/FormulaCache/ViewListCache/ - TileBarCache), AxisFrame/FrameEntry/FrameGroup and - header_level_widths (the grid frame, gotcha 44), - GridCache::seed (the bootstrap compaction), Projection, - bootstrap-only Snapshot -keymap.rs 975 / 0t KeyPattern, Binding, Keymap, ModeKey (17), TextEntry, - PanelKeymap, Keymaps::default_keymaps/prefix_keymap — the - shared table both front ends look up. Tests live with - dispatch in improvise-command/src/command/keymap.rs -viewmodel.rs 170 / 5t flatten_cat_tree — derivations between a cache and a - renderer, called by both front ends (improvise-edp) -face.rs 682 Face (a role: Axis, WizardField, Mode*, MinibufferOf…), Theme, - AnsiSlot — appearance as data a front end applies (ADR 004, - protocol.md §10); draw.rs::face_style is the TUI's one translation -key.rs 96 / 3t KeyCode, KeyModifiers, KeyEvent (moved from command/) -frame.rs 156 / 3t ClientFrame/ServerFrame — the ONE envelope pair, both - deployments (improvise-xks6). Nothing else defines one. - ClientFrame::Save is a request, not a Command (gotcha 45) -basis.rs 257 / 6t Basis (the three tiers), Rejection, BasisCache — ADR 001 - layer 1. Basis has no Default and no public constructor -help.rs 1241 / 14t HelpPage content and Hint { keys: Vec<(KeyCode, KeyModifiers)>, - command, caption }. `compact` drops a chord by its MODIFIERS - (anything beyond shift), never by parsing a label it just - formatted (improvise-thpe) — the guard is the signature -resolve.rs 4883 / 88t the browser's key resolvers: a BrowserKey to the Commands it - composes, mirroring each Cmd line for line INCLUDING where - siblings disagree (gotcha 47). The largest file in the crate -client.rs 943 / 15t ClientState: ViewState + one cache per widget, and the fold; - in_flight, `opened` (OpenedCell), notify, settle_status - (gotcha 47). EditorSession is in lib.rs beside ViewState -``` - -### `gp-tree` / `improvise-gp` (`crates/gp-tree/`, `crates/improvise-gp/`) -``` -gp-tree/src/lib.rs 575 / 8t Tree, Spec, eval, simplify, render, breed, Population, - SplitMix64 (vendored seeded Rng; seed() keeps the seed so - a fit can be shown and recorded, pinned to the published - reference vectors) -gp-tree/src/data.rs 1000 / 21t raw string rows -> f32 feature vectors: per-column type - inference, one-hot, mean-imputation + col.missing flag, - ColumnSpec/encode_row, Dataset (incl. warnings), - Model = trees + plan + seed + FitConfig -gp-tree/src/fit.rs 961 / 32t the search: FitConfig (Default = the CLI's numbers), fit() - -> simplified champion, cv_regress/cv_classify, - permutation_importance, folds/standardize/mse. - Returns data; never prints -gp-tree/examples/fit_csv.rs 583 CLI over the above: flags, demo dataset, all printing -gp-tree/tests/scoring_known_answers.rs - 250 / 6t f32 bit patterns + a next_u64 witness: the evidence any - change to the scoring paths needs (gotcha 37c) -improvise-gp/src/lib.rs 978 / 25t formula_text / fitted_formula: a Tree as IF(...) formula - text, validated back through parse_formula. A threshold is - the SHORTEST decimal that reads back as the same f64 (not - the exact expansion), so a row on a split boundary routes - the same in f64 as in f32 eval; non-finite values panic -improvise-gp/src/dataset.rs 506 / 15t a View's records -> the feature matrix a fit consumes — - the improvise-side counterpart of preprocess. Records mode - is already a feature matrix (gotcha 1) -improvise-gp/src/fit.rs 1778 / 35t one fit end to end: a View's records in, an improvise - Formula and the evidence for believing it out. Not in a - command, because none of it is terminal-specific -``` - -### Command layer (`crates/improvise-command/src/command/`) -``` -cmd/core.rs 475 / 2t Cmd trait, CmdContext, CmdRegistry, parse helpers -cmd/registry.rs 903 / 13t default_registry() — all registrations -cmd/navigation.rs 507 / 12t Move, EnterAdvance, Page* -cmd/cell.rs 724 / 20t ClearCellCommand, YankCell, PasteCell, TransposeAxes, SaveCmd -cmd/commit.rs 1304 / 27t CommitAndAdvance, CommitBuffer(CommitKind), CommitExport, - CommitSave -cmd/effect_cmds.rs 519 / 6t effect_cmd! macro, 25+ simple wrappers, `unless_unknown` (gotcha 51) -cmd/fit.rs 293 / 8t fit-measure: evolve a tree over the active view and store it - as an ordinary `## Formulas` line — nothing GP-specific is - left in the document afterwards -cmd/grid.rs 988 / 22t ToggleGroupAtCursor, ViewNavigate, DrillIntoCell, TogglePruneEmpty -cmd/mode.rs 469 / 13t EnterMode, Quit, EditOrDrill, EnterTileSelect -cmd/panel.rs 936 / 24t Panel toggle/cycle/cursor, formula/category/view panels -cmd/search.rs 202 / 4t SearchNavigate, SearchOrCategoryAdd, ExitSearchMode -cmd/text_buffer.rs 399 / 14t AppendChar, PopChar, CommandModeBackspace, ExecuteCommand -cmd/tile.rs 170 / 6t MoveTileCursor, TileAxisOp -cmd/wizard.rs 244 / 6t HandleWizardKey — per-step import-wizard key tables -keymap.rs 1353 / 44t KeymapSet, Keymap::dispatch/dispatch_transient, effects_of, - SetTransientPrefix — what a thin client cannot have. The - table itself is improvise-protocol/src/keymap.rs -parse.rs 350 / 23t Script/command-line parser (prefix syntax) -``` - -### Semantic UI state (`crates/improvise-command/src/ui/`) -``` -ui/effect.rs 4020 / 99t Effect trait. 63 effects: 53 `impl Effect for` blocks, 9 - simple_effect! expansions, and SetTransientPrefix in - command/keymap.rs. The 10 outside the brace-match are all - ViewOnly; 28 of the 53 are Model(_) (gotcha 38, h3y7) -ui/app.rs 5558 / 134t App, ModelState, ViewState + dispatch methods, Panel, handle_key, dispatch_key, document_generation, autosave; apply_effects declares `dirty` for WholeWorkbook and observes it for CellData/LayoutGeometry (gotcha 45) -ui/project.rs 6383 / 118t Dirty, Viewport, grid_of/category_rows/formula_texts/ - view_summaries/tile_entries, grid_updates (the diff), - ClientSession + Sessions (connect/apply/rebase/project_all) -ui/reduce.rs 1217 / 24t Checked / CheckedBatch / apply_command / model_effects / - reduce_key — one door from a Command or a keypress to the - authoritative App. `reduce_full` is GONE (improvise-h3y7) -ui/host.rs 3410 / 67t Host: one App + one Sessions; handle, commands, project_all, - rebase. Lives here, not in ws-server (gotcha 45) -ui/cat_tree.rs 174 / 8t Category tree flattener for panel -ui/viewport.rs 259 compute_col_widths / compute_row_header_width / visible_cols - (pure geometry; the width rule itself is the protocol's); - grid_width, SIDE_PANEL_WIDTH, MIN_GRID_WIDTH (improvise-gtn7.1); - GridSummary / grid_summary / grid_window (improvise-felo.13) -``` - -### TUI rendering, draw, main (`src/ui/`, `src/draw.rs`, `src/main.rs`) -``` -ui/grid.rs 905 / 14t GridWidget (ratatui): no &Model, no &View — the frame comes - off GridCache (gotcha 44). Its module doc is the best-written - statement of that rule in the repo -ui/help.rs 277 5-page help overlay (render only) -ui/import_wizard_ui.rs 829 / 9t Import wizard rendering. `Panel` is the one writer, and - every write is clipped to the panel's rows and width -ui/text.rs 169 truncate and wrap, both measured in terminal cells, never bytes -draw.rs 1122 / 20t TUI event loop, frame composition; `content_tests` draws a - whole frame to check the selection beside a side panel -main.rs 428 CLI entry (clap): open, import, cmd, script -# other ui/*.rs are small panel renderers — skip unless changing layout/style -tests/grid_frame_agreement.rs - 479 / 6t the two front ends draw the same grid frame (improvise-anr9) -tests/records_wide_smoke.rs 67 / 1t -tests/shipped_documents_load.rs 333 / 8t every TRACKED .improv/.improv.gz loads, and every record - it projects is writable. "Shipped" is read off `git ls-files`, - not a .gitignore reading (improvise-c0aj), and the walk's git - is scrubbed of every hostile GIT_* setting the caller may carry -tests/web_demo_model.rs 83 / 2t every visible cell in web/demo.improv is a Record -``` - -### Browser host and client (`crates/improvise-ws-server/src/`, `crates/improvise-wasm-client/src/`, `web/`) -``` -ws-server/registry.rs 616 / 14t Registry of Hosts keyed by DocumentId; ClientId {document, - session} — a SessionId is unique only within its document. - A document is never evicted while the server runs (ADR 001) -ws-server/host.rs 4 / 0t a re-export shim only. The session loop is - improvise-command/src/ui/host.rs (gotcha 45) -ws-server/origin.rs 132 / 6t OriginPolicy: which origins may open a socket. Loopback by - default, --allow-origin widens, absent allowed, two refused - (improvise-kc7n). The only decision transport::admitted makes -ws-server/transport.rs 330 / 4t tokio + tungstenite plumbing, plus the Origin check at the - handshake — a refused upgrade, not an ignored socket -ws-server/main.rs 190 / 5t CLI entry: serve a file on a port; --bind, --allow-origin -ws-server/tests/socket.rs 389 / 6t the one test that uses a real socket: handshake, JSON - framing, SessionId routing, which document a URL asked for -wasm-client/lib.rs 455 / 8t Client: the wasm-bindgen surface (init, on_message, on_frame, - viewport/set_viewport, render_model, set_cell_at, typed_cell) -wasm-client/keys.rs 187 / 5t BrowserKey ONLY — a KeyboardEvent flattened to an improvise - key. It does NOT resolve: `resolve` and KeyOutcome live in - improvise-protocol/src/resolve.rs, shared by both front ends -wasm-client/render.rs 1249 / 21t RenderModel: editable / selected / cat tree, decided here -wasm-client/socket.rs 547 / 13t the fold (fold_frame/handle_frame), the two compositions - and typed() text→CellValue. Defines no enums -# state.rs is GONE — ClientState moved to improvise-protocol/src/client.rs -wasm-client/tests/browser_round_trip.rs 2136 / 34t improvise-1ey's scope against a real Host -wasm-client/tests/lockstep.rs 4064 / 78t host and client agree on a keypress -web/app.js 816 string calls only, no protocol knowledge, no framework. - Run by harness/tests/shell.spec.js (Chromium; the standalone half on Firefox too) — see - gotcha 48 for what that does and does not reach -web/worker.js 86 three string calls, no protocol knowledge. RUN by - shell.spec.js's no-?ws path, with bindings.rs (gotcha 45) -web/shell.js 399 everything app.js DECIDES that is not painting: no DOM, - no browser API, one `node --test` away. Owns socketUrl, - gridPlan, frameCostOf, documentTitle — so a chunk of the - browser's render decisions is NOT in the file gotcha 42 - and gotcha 46 name. Reasoning: context/code/clients.md -web/serial.js 19 the one rule worker.js exists to keep: one frame at a - time, in arrival order (gotcha 48's async-listener race), - extracted so it is testable without worker globals -web/index.html, style.css the page; legibility only -web/build.sh 38 wasm-pack build → web/pkg (gitignored). Needs - wasm-opt = ["-O", "--enable-bulk-memory"] or the bundled - wasm-opt rejects LLVM's memory.copy -web/assemble.sh 112 assembles the deployable site (copies web/*.improv) -web/demo.improv 65 the document the site ships and seeds from. Every cell a - Record — asserted (gotcha 46) -``` - -### Browser worker host (`crates/improvise-worker-server/`) -``` -improvise-worker-server/src/worker.rs 765 / 17t Worker: one port, one client; handshake, Addressed - filter, persist trigger (model_state.dirty). Every decision, every test -improvise-worker-server/src/vfs.rs 348 / 8t documents/open: seeding is a Storage::write -improvise-worker-server/src/bindings.rs 185 / 0t wasm32 shim: OpfsFs awaits + postMessage. No Rust test - reaches it, but harness/tests/shell.spec.js DOES: the - no-?ws path runs start()/OpfsFs::open under Chromium - and Firefox. - Called NEVER RUN here for months: see gotcha 45 -improvise-worker-server/src/lib.rs 51 / 1t the module doc that states exactly what is and is not - covered — the one correct copy of that fact -improvise-worker-server/tests/post_message.rs 407 / 8t main-thread traffic, strings only -``` - -### Browser harness (`harness/`) — not a workspace member - -``` -harness/playwright.config.js 50 projects: chromium (every spec), firefox (opfs, plus - shell.spec's standalone half: grepInvert @mocked-peer), - webkit (opfs only, which skips on a runtime probe: no - OPFS). CI runs `--project=chromium --project=firefox` only: - nixpkgs' WebKit aborts inside the runner VMs (ci.yml) -harness/tests/shell.spec.js 345 the first execution of web/app.js. A change to web/*.js - belongs here (gotcha 48). Playwright serves harness/serve/, - which harness/build.sh ASSEMBLES: edit web/*.js, rebuild, - or the specs run the old page -harness/tests/opfs.spec.js 103 drives the OPFS contract page and reads back its result -harness/opfs-contract/src/lib.rs 470 runs improvise_io::storage::contract::storage_contract - against a real OpfsFs inside a real Web Worker -harness/www/opfs/{index.html,worker.js} the page and worker that host it -harness/build.sh 50 assembles harness/serve/ (web/assemble.sh, the fixtures, - and the contract crate's wasm-pack build) — gitignored -# gated by the `browser` job in .github/workflows/ci.yml, which can fail the build -``` - -### Examples -``` -examples/gen-grammar.rs Grammar-walking random .improv generator (pest_meta) -examples/pretty-print.rs Parse stdin, print formatted .improv -crates/improvise-core/examples/category_width_bench.rs - The same records at 12 and at 24 regular categories - (improvise-felo.1; context/code/core.md) -``` - -### Context docs -``` -.claude/skills/code-review/ Review skill: checks a change against the doctrine in these docs + kb.pl -scripts/check-comment-runs.py The two-line comment cap AND --check-evidence. The flag is - ADDITIVE since improvise-ijql — every invocation that can - exit 0 answers the cap — guarded by a subprocess self-test, - the only level a dispatch bug is visible at. Reasoning: - context/code/tooling.md §2 -scripts/check-ci-graph.py pages-deploy must `needs` every job in ci.yml, and every - suite `checkers.sh --list` knows must run in some CI job - (improvise-dak5). It REFUSES what it cannot read rather than - parsing YAML: the values it accepts are an allowlist - (tooling.md). --self-test; run in the `test` job. README - states the rule and lists no jobs (improvise-x7cq) -scripts/checkers.sh The one entry point for every *-test.sh suite: scrubs the - env to an allowlist, runs under the bash that runs it (no - version floor since #297). `ci_suites` is the table of which - suites each CI job runs (`--ci `); ci.yml names none -scripts/pr-watch Waits on a PR's checks (the pr-watch skill). Exit 5: the PR - is already MERGED or CLOSED, so there is nothing to wait for -scripts/assemble-stamp.sh The cache-bust stamp gate web/assemble.sh calls, extracted - so its refusals can be exercised without a web/pkg -scripts/assemble-stamp-test.sh Its check_the_checker, run first in the `build` CI job -context/code/tooling.md The repository's own checkers: why an evidence heading's - names come from cargo rather than from the source, and - why a leaf name is not an identity -context/test-evidence/tooling.md Evidence for the gates in scripts/ and for the tests that - hold CI's shape -scripts/nix-seed-flake-inputs.sh Seeds locked flake inputs over git so `nix develop` works in cloud sessions -prolog-sessions/kb.pl Executable knowledge base: invariants, doctrine, decisions (CI-checked) -context/adr/007-stable-identities.md Stable identities: ids unique across the document are the - keys and names are display-only; gensym suffixes apply to - generated ids only, and hand-written ids are canonical. - Accepted 2026-09-25. (There is no ADR 005 or 006 file: - improvise-l2b8 and improvise-ilvl claim those numbers.) -context/adr/004-faces-and-themes.md Faces: appearance as UI-agnostic data a front end - applies, emacs-style. Accepted 2026-09-22. -context/adr/003-live-backed-models.md A model may be backed by a live query source — - improvise as a DB client. Accepted, NOT a priority. -context/adr/002-naming-a-cell.md How a client names a cell: an opaque surrogate on - CellDisplay, whose absence means the cell is an - aggregate. Records why _Index cannot serve. -context/adr/001-concurrent-edits.md Concurrent edits between clients: record-level atomicity, - optimistic basis check per batch, advisory reservations. - Records why a causal clock was rejected. -context/cloud-agents.md Cloud-session environment: tracker sync, bd pitfalls, what is blocked -context/design-principles.md Architectural principles & testing doctrine -context/plan.md Show HN launch plan -context/repo-map.md This file -docs/design-notes.md Product vision & non-goals -``` +| Area | Where | Entry points | +|---|---|---| +| Core model | `crates/improvise-core/src/` | `model/{types,cell,record,category,id}.rs`, `view/{layout,types,axis}.rs`, `workbook.rs`, `format.rs` | +| Formula | `crates/improvise-formula/src/` | `parser.rs`, `formula.pest`, `ast.rs`, `quote.rs` | +| I/O | `crates/improvise-io/src/` | `storage.rs`, `opfs.rs`, `persistence/{improv.pest,format,parse,export,mod}.rs`, `import/{wizard,analyzer,csv_parser}.rs` | +| Protocol | `crates/improvise-protocol/src/` | `lib.rs` (ViewState), `reduce.rs`, `render.rs`, `keymap.rs`, `face.rs`, `frame.rs`, `basis.rs`, `resolve.rs`, `client.rs`, `viewmodel.rs`, `help.rs` | +| Command layer | `crates/improvise-command/src/command/` | `cmd/{core,registry,navigation,cell,commit,effect_cmds,fit,grid,mode,panel,search,text_buffer,tile,wizard}.rs`, `keymap.rs`, `parse.rs` | +| Semantic UI state | `crates/improvise-command/src/ui/` | `effect.rs`, `app.rs`, `project.rs`, `reduce.rs`, `host.rs`, `viewport.rs`, `cat_tree.rs` | +| TUI + CLI | `src/` | `draw.rs`, `main.rs`, `ui/{grid,help,import_wizard_ui,text}.rs`; integration tests in `tests/` | +| GP | `crates/{gp-tree,improvise-gp}/` | `gp-tree/src/{lib,data,fit}.rs`, `improvise-gp/src/{lib,dataset,fit}.rs` | +| Socket host | `crates/improvise-ws-server/src/` | `registry.rs`, `origin.rs`, `transport.rs`, `main.rs`; `host.rs` is a re-export shim | +| Worker host | `crates/improvise-worker-server/src/` | `worker.rs`, `vfs.rs`, `bindings.rs` (wasm32 shim) | +| Browser client | `crates/improvise-wasm-client/src/`, `web/` | `lib.rs`, `keys.rs`, `render.rs`, `socket.rs`; `web/{app,shell,worker,serial}.js`, `index.html`, `demo.improv`, `build.sh`, `assemble.sh` | +| Browser harness | `harness/` | Playwright (`tests/shell.spec.js`, `tests/opfs.spec.js`), `opfs-contract/`, `build.sh` — not a workspace member; its own CI job | +| Examples | `examples/`, `crates/*/examples/` | `gen-grammar.rs`, `pretty-print.rs`, `category_width_bench.rs`, `fit_csv.rs` | +| Scripts | `scripts/` | `checkers.sh` (every `*-test.sh` suite), `check-comment-runs.py`, `check-ci-graph.py`, `bd-audit-ids`, `kb`, `pr-watch`, `pr-reviews`, `build-doc-site.mjs` | +| Context docs | `context/` | `design-principles.md`, `gotchas.md`, `file-inventory.md`, `code/.md` (arguments), `test-evidence/` (bug evidence by test name), `adr/` (001–004, 007), `cloud-agents.md`; `prolog-sessions/kb.pl` is the KB; `docs/design-notes.md` is product vision | --- ## Wasm -The browser bundle is `improvise-core`, `improvise-formula`, `improvise-io`, -`improvise-protocol`, `improvise-command`, `improvise-wasm-client` and -`improvise-worker-server`. All seven -`cargo check --target wasm32-unknown-unknown` clean with no feature flags -(improvise-ywd), gated by the `wasm` job in `.github/workflows/ci.yml`. That -job gates **eight** crates, not seven: `improvise-gp` was added (improvise-z5eg) -explicitly *not* as a bundle member but as a candidate, because staying -wasm-clean is free now and expensive to retrofit — so the bundle is seven and -the gate is eight, and neither number is the other one's typo. -**A feature that adds a dependency to a bundle crate must be checked with `cargo tree --edges normal`** — the default output includes **dev** edges, so a plain `cargo tree -p improvise-protocol --target wasm32-unknown-unknown` reports a dev-only `serde_json` as *present* and reads exactly like a leak. Check per-crate too, because the `wasm` CI job invokes each crate separately and feature unification cannot reach across that (improvise-26dm added a `test-support` feature on `improvise-protocol` on those terms; it is off in every shipped build). - -`improvise-ws-server` is the host and is native only — tokio, tungstenite and -`Box` — so `improvise-wasm-client` cannot depend on it in anger. It -no longer needs to: both ends `pub use` `improvise_protocol::frame` and neither -defines a pair (gotcha 42). The **dev**-dependency remains, for -`tests/browser_round_trip.rs`, which hands the client's own serialized frame to -a real `Host::handle`. The root `improvise` crate is -the terminal binary and does **not** build for wasm — it depends on -crossterm; that is correct, not a gap. - -`cargo check` passing is not the same as running. Two known runtime hazards -it cannot see: `Instant::now()` panics on wasm (hence `web-time` in -`improvise-command`), and any `std::fs` call compiles but fails in a browser -— improvise-fuj closed the effects that bypassed `Storage`; autosave and the headless final write still do (improvise-31oe.11). +The browser bundle is `improvise-{core,formula,io,protocol,command,wasm-client,worker-server}`; all seven +`cargo check --target wasm32-unknown-unknown` clean with no feature flags, and the `wasm` CI job also gates +`improvise-gp` (a candidate, not a member). `improvise-ws-server` and the root crate are native-only. +Check new bundle dependencies with `cargo tree --edges normal` (the default output includes dev edges), +per crate. `cargo check` cannot see runtime hazards: `Instant::now()` panics on wasm (`web-time`), and +`std::fs` compiles but fails in a browser. Long form: `context/file-inventory.md`, appendix. ## CLI @@ -673,39 +261,20 @@ improvise cmd 'add-cat Region' -f model.improv # headless command(s) improvise script setup.txt -f model.improv # run script file ``` -Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`, `--split-date ` (a date column imports as ONE hierarchical category by default; this restores the flat split, and `--extract` still picks the components — improvise-8jk6.3), `--strict` (with `--no-wizard`, any skipped record is a refusal and nothing is written, where the default writes and warns with a count — improvise-8jk6.6), and `--on-duplicate refuse|sum|separate` (default `refuse`; applies with `--no-wizard`, where the wizard would otherwise ask — improvise-cgqt). A headless import also refuses a `--split-date` that names a non-date field and an `--axis` that names a category the import did not build (exit 1). - ---- +Import flags: `--category --measure --time --skip --extract --axis --formula --name`, `--split-date ` +(flat date split; default is one hierarchical category), `--strict` (refuse any skipped record), and +`--on-duplicate refuse|sum|separate` (default `refuse`). Details: `context/code/cli.md`, gotchas 51–52. ## Testing — the short version -Full guidance lives in `context/design-principles.md` §7. Quick reminders: +Full guidance: `context/design-principles.md` §7. -- **Two numbers, not one** — the ambiguity between them is what let the old - "suite runs in <2s" drift: **no single test binary over 2 seconds**, and the - workspace wall clock is the sum. Measured at `63a5c13`: **1464 tests over 35 - binaries** (1 ignored), longest single binary **0.6–1.0s across two runs on one machine** — - the spread being its own argument for reading your own run. Excludes compilation, - and excludes `harness/` (Playwright, minutes, its own CI job). *Re-measure* rather - than re-assert — `cargo test --workspace` prints a `finished in` per binary, and - **the portable signal is the ratio, not the seconds**. **Not the ratio to the - median, though: it is degenerate.** Most binaries print `0.00s` at two decimals, - so the median is zero and every ratio to it is undefined (improvise-i5z6, 39 - binaries). Compare the slowest binary to the next few slowest instead: an outlier - is one that sits well clear of them *in every run*. That is how i5z6 was read — - `improvise-formula`'s lib longest of 39 in all three runs, at 0.91–1.95s against a - runner-up's 0.50–0.92s. Both outliers so far had one cause: a generator re-parsing - its pest grammar per case. `improvise-io` (improvise-8shu, 1.42s → 0.21s) and - `improvise-formula` (improvise-i5z6, 0.84–0.98s → 0.06–0.07s) each read it once through a - `OnceLock` now. A new grammar-walking strategy should do the same from the start. - The full argument is `context/design-principles.md` §7. -- Commands: build a `CmdContext`, call `execute`, assert on returned effects. No terminal needed. -- Property tests (`proptest`, default 256 cases) cover invariants: CellKey sort, axis consistency, save/load roundtrip, `parse(format(parse(generate())))` stability. -- **Bug-fix workflow**: write a failing test *before* the fix (regression guard). Record the bug in `context/test-evidence/`, **keyed by test name** — *not* in the test's doc comment, which the two-line comment cap forbids and `scripts/check-comment-runs.py` enforces. A heading may be module-qualified and must be when its leaf is not unique. -- Coverage target ~80% line/branch on logic code; skip ratatui render paths. - ---- +- **No single test binary over 2 seconds**; the workspace wall clock is the sum. Re-measure with `cargo test --workspace` (a `finished in` per binary) and compare the slowest binary to the next few — a ratio, not seconds, and not a ratio to the median (it is zero). +- Commands: build a `CmdContext`, call `execute`, assert on the returned effects. No terminal. +- Property tests (`proptest`, default 256 cases) for invariants and round-trips; a grammar-walking strategy must read its pest grammar once (`OnceLock`). +- **Bug-fix workflow:** write the failing test first. Record the bug in `context/test-evidence/`, keyed by test name — not in a doc comment (two-line cap, `scripts/check-comment-runs.py`). +- Coverage target ~80% on logic code; skip ratatui render paths. ## Key dependencies -ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest + pest_derive, flate2 (gzip), csv, enum_dispatch. Dev: proptest, tempfile, pest_meta. +ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest, flate2, csv, enum_dispatch. Dev: proptest, tempfile, pest_meta. diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index cede9610..2a464b56 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -34,6 +34,8 @@ const SECTIONS = [ ["docs/design-notes.md", "design-notes", "Design notes"], ["context/design-principles.md", "design-principles", "Design principles"], ["context/repo-map.md", "repo-map", "Repository map"], + ["context/gotchas.md", "gotchas", "Gotchas (long form)"], + ["context/file-inventory.md", "file-inventory", "File inventory (long form)"], ], }, { @@ -613,7 +615,7 @@ for (const section of SECTIONS) { for (const [src, slug, title] of section.docs) { const depth = slug.split("/").length - 1; const ctx = { depth, cited, unresolved, anchors: new Set() }; - if (slug === "repo-map") ctx.listItemId = (n) => (n >= 1 && n <= 200 ? `gotcha-${n}` : null); + if (slug === "repo-map" || slug === "gotchas") ctx.listItemId = (n) => (n >= 1 && n <= 200 ? `gotcha-${n}` : null); const md = readFileSync(join(root, src), "utf8"); const { html, toc } = renderMarkdown(md, ctx);