diff --git a/context/repo-map.md b/context/repo-map.md index 21d17824..63d5b880 100644 --- a/context/repo-map.md +++ b/context/repo-map.md @@ -31,7 +31,7 @@ Architectural intent lives in `context/design-principles.md` — read that for t | Change cell storage / lookup | `model/cell.rs` → `DataStore` | | Change category/item behavior | `model/category.rs` → `Category` | | Change view axis logic | `view/types.rs` → `View` | -| Change an axis label, color or cycle | `view/axis.rs` → `impl Axis` (colors resolve in `draw.rs::axis_fg`) | +| Change an axis label or cycle | `view/axis.rs` → `impl Axis`. An axis's **colour** is `Face::Axis(axis)` in `improvise-protocol/src/face.rs`, resolved by the `Theme` and turned into a style by `draw.rs::face_style` (ADR 004) | | Ask "what data is visible" vs "what can the user page" | `View::filtering_cats()` vs `categories_on(Axis::Page)` | | Change grid layout | `view/layout.rs` → `GridLayout` | | Change `.improv` format | `persistence/improv.pest` + `persistence/{format,parse}.rs` | @@ -109,7 +109,7 @@ pub trait Effect: Debug { **GridLayout** (`view/layout.rs`): pure function of `Model + View`. `cell_key(r,c)`, `cell_value(r,c)`, `drill_records(r,c)`. **Records mode** auto-detects when `_Index` is on Row and `_Dim` is on Column. -**AppMode** (`ui/app.rs`): 17 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, SavePrompt, CommandMode, ImportWizard, Help, Quit, RecordsNormal, RecordsEditing). Eight text-entry variants carry a `MinibufferConfig`. `SearchMode` is Normal + `search_mode: bool`, not its own variant. Per-variant UI facts live on the enum as exhaustive-match methods — `label()`, `hint_text()`, `panel()`, `minibuffer()`, `is_editing()`, `is_records()` (improvise-2hi); add a variant and the compiler names every one you must fill in. `mode_style()` stays in `src/draw.rs` because `Style` is a ratatui type. `App` state is split into `ModelState` (workbook, file_path, dirty) and `ViewState` (mode, cursor/offsets, buffers, panels, search) — see improvise-x2c/ew0. `wizard` and `drill_state` live on `App`, not `ViewState` (improvise-h0b) — see gotcha 24. +**AppMode** (`ui/app.rs`): 17 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, SavePrompt, CommandMode, ImportWizard, Help, Quit, RecordsNormal, RecordsEditing). Eight text-entry variants carry a `MinibufferConfig`. `SearchMode` is Normal + `search_mode: bool`, not its own variant. Per-variant UI facts live on the enum as exhaustive-match methods — `label()`, `hint_text()`, `panel()`, `minibuffer()`, `is_editing()`, `is_records()` (improvise-2hi); add a variant and the compiler names every one you must fill in. A mode's colour is a `Face` (`Face::Mode*`, `Face::of_minibuffer`), turned into a ratatui `Style` only by `src/draw.rs::face_style`. `App` state is split into `ModelState` (workbook, file_path, dirty) and `ViewState` (mode, cursor/offsets, buffers, panels, search) — see improvise-x2c/ew0. `wizard` and `drill_state` live on `App`, not `ViewState` (improvise-h0b) — see gotcha 24. **Keymap** (`improvise-protocol/src/keymap.rs`; `dispatch` and `KeymapSet` in `improvise-command/src/command/keymap.rs`): `Binding { Cmd | Prefix(Arc) | Sequence(Vec<…>) }`. Lookup fallback: `exact(key,mods) → Char(NONE) → AnyChar → Any → parent`. 17 mode keymaps, one per `ModeKey`, built by `Keymaps::default_keymaps()`; mode resolved via `ModeKey::from_app_mode()`. `ModeKey` and `AppMode` are two different enumerations that happen to be the same size — `SearchMode` is a `ModeKey` and not an `AppMode`; `Quit` is an `AppMode` with no keymap — so do not "fix" one count to match the other. @@ -162,7 +162,7 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text **`prune_axis_items` takes a predicate, not a flag vector.** `prune_empty` hands it `impl Fn(usize) -> bool` over the *data* index (group headers consume no index). The domain is derived from `items` itself, so there is no parallel length a caller could get wrong — the earlier `&[bool]` + `debug_assert` pair could disagree between debug and release if a third caller ever sized `keep` from something other than `row_count()`/`col_count()` (improvise-n3wt). **It re-marks the survivors rather than filtering the markers** (improvise-e265): runs nest, so dropping an item can leave a header with nothing under it or a `GroupEnd` closing a run that was never reopened, and no per-marker filter can tell. `remark` instead emits, before each kept item, exactly the headers and `GroupEnd`s that make the output's open runs equal the input's at every level — each survivor keeps exactly the runs it was in. Reasoning: `context/code/core.md` §6. **`prune_empty` shares one `GridEval`** across the whole row×col emptiness test, and **a grid evaluates only through `GridLayout::evaluator`**, which builds the fold memo from that layout's own `none_cats` and hidden items — so no other grid's memo can reach its cells or totals (improvise-metf, improvise-opbo; the memo itself is improvise-h6z1, `core.md` §14). **`GridLayout::new` does not prune; `with_frozen_records` does** — and that split has now produced **four** bugs: `view_frame`, `project::grid_in` (PR #78), and `persistence/export.rs::render_csv` (improvise-iot6), which wrote rows *and columns* the grid does not draw for as long as CSV export has existed. Every remaining `GridLayout::new` call site in the workspace is test code — verified — so a new production caller almost certainly wants `with_frozen_records(m, v, None)`. -2. **Effects never touch `std::fs`.** Every document read or write goes through `App::storage: Box` (improvise-fuj). The host picks the backend: `App::new` supplies `PhysicalFs`, `App::with_storage` takes any other — a browser worker will supply OPFS (improvise-i34), and tests use `MemoryFs`. `improvise-io` keeps path-taking wrappers (`save`, `load`, `export_csv`, `parse_csv_with_warnings`) for the native CLI, each delegating to a `_to`/`_from`/`_bytes` variant that takes the backend or the bytes. +2. **Effects never touch `std::fs`.** Every document read or write goes through `App::storage: Box` (improvise-fuj, closed). Two writes still go around it: `App`'s autosave and the headless final write call the path-taking `persistence::save` (improvise-31oe.11, open). The host picks the backend: `App::new` supplies `PhysicalFs`, `App::with_storage` takes any other — a browser worker will supply OPFS (improvise-i34), and tests use `MemoryFs`. `improvise-io` keeps path-taking wrappers (`save`, `load`, `export_csv`, `parse_csv_with_warnings`) for the native CLI, each delegating to a `_to`/`_from`/`_bytes` variant that takes the backend or the bytes. 3. **Every effect declares an `EffectKind`, and there is no default.** `ViewOnly` never reaches other clients; `Model(CellData | LayoutGeometry | WholeWorkbook)` does (improvise-mae). A new effect that forgets to classify itself fails to compile — that is deliberate, since the failure mode is silent: a mis-declared `ViewOnly` edit would simply never reach connected clients. Watch two traps: the *View* lives inside `model_state.workbook`, so "touches model_state" does NOT mean "is a model change" (axis/page/format are `LayoutGeometry`, cursor and scroll are `ViewOnly` — see improvise-8uz; and `SwitchView`, `ViewBack` and `ViewForward` moved from `LayoutGeometry` to `ViewOnly` with improvise-jytm, writing `view_state` and nothing else, which is what took the latter two off the wire — but a sound derivation is not a guard: only `SwitchView` had one, and review showed reclassifying either sibling back was silent across the whole suite); and `Save`/`SaveAs`/`MarkDirty` touch `model_state` for bookkeeping only and are `ViewOnly`. `AddCategory`/`RemoveCategory` are the third trap and go the other way: they look like data edits, but both go through `Workbook`, whose `on_category_added`/`on_category_removed` rewrite **every** view's axis map, so they are `LayoutGeometry`. The two "an effect declaring X actually changes X" tests cannot separate them — a category change moves model data *and* view config, so it clears either bar; `structural_effects_declare_their_kind` pins the declaration itself. A third trap, now guarded: **`EffectKind` is too coarse to check a *mapping*.** `ui/reduce.rs::model_effects` sends each of the 26 `ModelCommand` variants (counted on `c1772ca9`; gotcha 38) to one effect, and every inverse pair — `HideItem`/`ShowItem`, `AddItem`/`RemoveItem`, `CreateView`/`DeleteView`, `SetCell`/`ClearCell` — declares the *same* kind, so swapping three of them at once left the suite at 918 passing while three wire messages did the exact inverse of what they said. The mapping is name-for-name by construction, so `every_model_command_names_the_effect_of_the_same_name` asserts that, reading the type name off the effect's derived `Debug` (`Effect` is a trait object, so `type_name` is out of reach). A variant whose effect must be named something else is a variant that stopped being derived — it should fail there and be argued about. Their reach is wider than the name suggests: `Model::remove_category` also drops every formula targeting the category, which is why `Dirty::of(LayoutGeometry)` reports `formulas: true` — see gotcha 35. 4. **Commands never mutate.** `&CmdContext` is read-only; return `Vec>`. If you want to touch `&mut App`, add or extend an Effect. 5. **`CellKey` is always sorted.** Use the constructors; equality and hashing rely on canonical form. @@ -178,12 +178,12 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text 15. **Registry names come from prototypes, not strings.** Every `register*` call reads the name off a prototype `Cmd` instance, so a typo is impossible (improvise-9cn) — there is no name literal in `registry.rs`. What types can't reach: four commands name themselves off a field (`ViewNavigate`, `TileAxisOp`, `ToggleGroupAtCursor`, `CommitAndAdvance`), so a prototype can name the wrong *variant* and still compile. That shows up as a **duplicate** registration, not a mismatch — the wrong-variant entry collides with its sibling and the sibling's name vanishes. `registered_names_are_unique` catches it; `every_bound_command_name_is_registered` catches the keymap side, and checks **both** halves of `dispatch`'s `registry.interactive(name, args, ctx).ok()?` — the name resolves *and* the args parse — because improvise-s13 moved `enter-mode ["export-prompt"]`/`["formula-edit"]` out of the nullary class, where a typo'd mode name compiles into a dead key. It builds the `CmdContext` in each keymap's own `AppMode`, since `commit-buffer item` reads `ctx.mode`. 16. **`effect_cmd!` macro** wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written `Cmd` once there's real decision logic. Its counterpart on the effect side is `simple_effect!` in `ui/effect.rs`, which declares a one-field newtype that writes one place on the `App`. Its **kind argument is mandatory** — `simple_effect!(Name(T) => |app| place, EffectKind::ViewOnly)` — even though every call site so far answers `ViewOnly`: hard-coding it reintroduced exactly the default gotcha 3 does without, since `$place` is an unconstrained expression and a one-line `active_view_mut().number_format` setter would have reported `ViewOnly` with the suite green (improvise-8fy). Likewise, mode entry has no per-mode shim commands — bind `enter-mode ` (improvise-s13); the name is decoded by `parse_mode_name` in `registry.rs`, and `EnterMode` clears the mode's minibuffer on the way in. 17. **Float equality inside formula eval has one epsilon, and the div/0 guard shares it.** `FLOAT_EQ_EPSILON = 1e-10` in `model/types.rs` is used by `=`/`!=` **and** by the division-by-zero guard (`rv.abs() < FLOAT_EQ_EPSILON`), in both `eval_expr` and its aggregated twin — six uses across the two evaluators (a div/0 guard plus the `Eq` and `Ne` arms in each), one constant. This gotcha previously recorded the opposite (`=`/`!=` epsilon, div/0 exact) and was stale; the mix is gone. Why an epsilon at all and why this one: `context/code/core.md` §4. -18. **`IndexMap`** preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do `DataStore::matching_cells`/`matching_values`: the secondary index **narrows** the candidates, then `matching_positions` resolves each to its position in the primary `IndexMap`, sorts, and reads back. **An index whose values are `HashSet`s is for O(1) lookup, never for iteration.** Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; `SUM`/`AVG` drifted in the last bits because `f64` addition is not associative (`Min`/`Max`/`Count` and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the **drill snapshot** froze its record list in hash order while `drill_state.pending_edits` are keyed by *row index* (gotcha 1), so the same drill opened in a second process addressed **different records** — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on `matching_values`, and `matching_cells` 7% *faster*, sorted positions resolving `CellKey`s with better locality. `DataStore::live_records` was the open choice and is now decided (improvise-2yc2): it walks `cells` and projects insertion order onto records, deliberately paying O(cells) rather than O(records), because `cells`' order is already the answer `matching_positions` gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on `RecordEntry`, since a stored position is a copy and a record deleted and re-created would hold its old one. Note what is *not* on offer: an unordered return — every iterator has an order, and a type that merely disclaims one still hands the caller hash order, which is the guard living one level away from the hazard and the shape this whole class of bug returns through. **The second real instance was `import::analyzer::analyze_records`** (improvise-7yx), which built its distinct values through a `HashSet`: `build_model` adds items in exactly that order and `IndexMap` preserves it, so the *same CSV imported twice* produced two different `.improv` files and two different row orders. Now first-seen order, pinned by a five-iteration test because each `HashSet` gets its own `RandomState`. Also answered here since: an **empty `partial`** matches every cell vacuously (improvise-hkt6) — it used to index `sets[0]` and panic, unreachable only because both callers guarded. **A group's fold reads through the same rule.** `Model::fold_stored` folds a collapsed group's stored values once, over every record the group covers, through `DataStore::spanned_values` → `spanned_positions` (improvise-yb4b) — one scan, not a lookup per member. `spanned_positions` chooses **by index-set size** between narrowing once by the base key and narrowing once per covered item of the smallest span, whichever visits fewer (improvise-jq0m), and **both arms return store order** — sorted positions, as `matching_positions` does. That is load-bearing, not tidy: a fold that returned hash or per-item order would reintroduce the `SUM` drift above, and the choice between the arms would then be visible in the last bits of a total. +18. **`IndexMap`** preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do `DataStore::matching_cells`/`matching_values`: the secondary index **narrows** the candidates, then `matching_positions` resolves each to its position in the primary `IndexMap`, sorts, and reads back. **An index whose values are `HashSet`s is for O(1) lookup, never for iteration.** Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; `SUM`/`AVG` drifted in the last bits because `f64` addition is not associative (`Min`/`Max`/`Count` and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the **drill snapshot** froze its record list in hash order while `drill_state.pending_edits` are keyed by *row index* (gotcha 1), so the same drill opened in a second process addressed **different records** — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on `matching_values`, and `matching_cells` 7% *faster*, sorted positions resolving `CellKey`s with better locality. `DataStore::live_records` was the open choice and is now decided (improvise-2yc2): it walks `cells` and projects insertion order onto records, deliberately paying O(cells) rather than O(records), because `cells`' order is already the answer `matching_positions` gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on `RecordEntry`, since a stored position is a copy and a record deleted and re-created would hold its old one. Note what is *not* on offer: an unordered return — every iterator has an order, and a type that merely disclaims one still hands the caller hash order, which is the guard living one level away from the hazard and the shape this whole class of bug returns through. **The second real instance was `import::analyzer::analyze_records`** (improvise-7yx), which built its distinct values through a `HashSet`: `build_model` adds items in exactly that order and `IndexMap` preserves it, so the *same CSV imported twice* produced two different `.improv` files and two different row orders. Now first-seen order, pinned by a five-iteration test because each `HashSet` gets its own `RandomState`. Also answered here since: an **empty `partial`** matches every cell vacuously (improvise-hkt6) — it used to index `sets[0]` and panic, unreachable only because both callers guarded. **A group's fold reads through the same rule.** `Model::fold_stored` folds a collapsed group's stored values once, over every record the group covers, through `DataStore::spanned_values` → `spanned_positions` (improvise-yb4b) — one scan, not a lookup per member. `spanned_positions` chooses **by index-set size** between narrowing once by the base key and narrowing once per covered item of the smallest span, whichever visits fewer (improvise-jq0m), and **both arms return store order** — sorted positions, as `matching_positions` does. That is load-bearing, not tidy: a fold that returned hash or per-item order would reintroduce the `SUM` drift above, and the choice between the arms would then be visible in the last bits of a total. **A partial as wide as the widest stored cell needs no narrowing** (improvise-felo.13): `DataStore` keeps `arities` (how many stored cells have each width), and `exact_positions` answers a strictly-sorted partial at least that wide with one probe of `cells`, since only an equal cell can match; anything else falls through to the index. The answer is the same sorted positions. 19. **Commit paths for cell edits** split across `commit_cell_value` → `commit_regular_cell_value` / `commit_plain_records_edit` in `command/cmd/commit.rs`. Keep the synthetic-records branch in sync with the plain branch when changing behavior. **Which cells refuse an editor is one function, asked at both ends of an edit**: `cmd::core::uneditable(model, layout, row, col)` is the shared decline for **opening** an editor (`EnterEditAtCursor`) and for **committing** one (`CmdContext::committable_cell_key`) — a fold or other aggregate, a formula's result, or a `CellId::NonItem` (improvise-enz6, improvise-gk51). It reads the cell's `CellId` through `project::cell_id`, the same derivation the projection uses, and the browser's `resolve::declined_at` matches on the projected `CellId`, so the two front ends refuse the same cells. Asking only at open is the bug enz6 was: Enter-to-advance moved onto a collapsed group's row and committed a write to the group coordinate. **`CellId::NonItem { category, item }` is a stored value whose coordinate names no item of its category** — records mode can draw one, and no write can address it, so it is projected as its own variant rather than as a `Record` that the host then refuses (gk51 was exactly that mismatch). Its one wording is `improvise_protocol::non_item_refusal`. 20. **Minibuffer commits go through one command.** `commit-buffer ` (`CommitBuffer { kind: CommitKind }`) owns buffer→trim→bail-on-empty→dirty→status for all three; the varying parts are methods on `CommitKind`. The add effect is emitted **last** so its failure message wins over the command's optimistic status (improvise-6t3). `CommitKind::Item` carries its target category, resolved from `AppMode::ItemAdd` by `parse_commit_kind` at registry time — so `commit-buffer item` is not constructible from a script line. 21. **An abort stops the chain, never the bookkeeping — and ordering is what separates them.** Three effects set `abort_effects`, and they are not all saves: `Save` and `SaveAs` on a failed write or a missing path (improvise-8tk — `SaveAs` was the arm left out of the original fix, so `:w /bad/path . q` quit on a failed write), and `AddCategory` when `Model::add_category` refuses, because what follows assumes the category exists — from the `+cat` column (improvise-2zq) the rest of the batch would advance the cursor and re-open an editor on a column that was never created. A `Binding::Sequence` flattens into one effect vec, so the abort carries across sequence steps; that is a property of the *flattening*, and it is why improvise-0hb's proposal to apply effects between steps was rejected — `apply_effects` resets the flag per batch, so step 2 in a fresh batch would land the `Quit` after a failed `Save`, silently reopening a P1. `a_sequence_whose_first_step_aborts_does_not_run_the_second` is the guard. **The rule the aborts taught, which is the part to carry:** an abort at the *tail* of a batch also ate the mode restoration and the buffer clearing, stranding the user in `CommandMode`/`Editing`/`SavePrompt` — where `draw_bottom_bar` renders the minibuffer *instead of* the status line, so the error the effect had just written was never drawn and the operation appeared to do nothing at all. So **tidy-up goes at the head, where an abort cannot reach it**: `ExecuteCommand` fronts its batch with `ReturnToNormalUnlessMoved`, `CommitAndAdvance` with the mode change, `CommitSave` with the mode change (found by Copilot on PR #27). Ordering answers it without giving any effect a "runs anyway" flag, and it does not weaken gotcha 31's observed-not-predicted mode settling: whatever a later effect sets, it sets *after*, so the mode a `:` line ends in is unchanged in every case — the only difference is what happens when the batch never reaches the end. **An abort is not a rollback**; that is the hole clause 4 leaves (gotcha 47). 22. **The import wizard's back path is a history stack, not a reverse table.** `advance` pushes the step it leaves onto `ImportWizard::history`; `previous` pops it, so back always retraces the path actually taken through the conditional transitions (improvise-907). Preconditions live in `validation_error()`, checked on the step where the user can still fix them — never deferred to `build_model`. `NameModel` is terminal for `advance`: Enter there calls `build()`, which parks the workbook in `built` on `Done`, and `AcceptWizardModel` takes it from there. Going back from `Done` discards `built`. **`ResolveDuplicates` appears only when `ImportPipeline::collisions()` is non-empty**: `after_fields` routes there from `ReviewProposals` or `ConfigureDates`, so it comes after the field and date steps and before `DefineFormulas`, with `NameDuplicateCategory` between them when the answer is Separate. Enter there is `WizardOp::ChooseDuplicate` (sum, keep separate, or go back). `ImportPipeline::on_duplicate` defaults to `DuplicatePolicy::Refuse`, so a path that never asked — the headless `--no-wizard` import included — errors listing the collisions rather than summing them. **A collision is a cell written by two records** (a tuple *and* a measure), not a shared tuple: records with disjoint measures on one tuple write different cells and collide with nothing (improvise-9bit). Reasoning: `context/code/formula-and-io.md`, "Colliding records are a question, not a default". -23. **`app.layout` goes stale mid-batch.** `apply_effects` rebuilds only *after* the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a `RebuildLayout` effect in front of it — the command decides the order (see `EnterEditAtCursorCmd`, `CommitAndAdvance`). Read the cursor's cell text via `App::display_value_at_cursor()`; don't build a `CmdContext` inside `apply` (it recomputes column widths). +23. **`app.layout` goes stale mid-batch.** `apply_effects` rebuilds only *after* the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a `RebuildLayout` effect in front of it — the command decides the order (see `EnterEditAtCursorCmd`, `CommitAndAdvance`). Read the cursor's cell text via `App::display_value_at_cursor()`; don't build a `CmdContext` inside `apply` (it recomputes column widths). **Since improvise-felo.13 the rebuild is skipped when nothing it reads has moved**: `App::refresh_layout` rebuilds only when `layout_stale` (any `EffectKind::Model(_)` in the batch, or `install_workbook`), the drawn view's name, or the drill snapshot's pointer (`LayoutBasis`) changed, so a cursor key rebuilds nothing, and the column widths and row-header width are measured once per layout (`App::grid_geometry`, dropped with the layout). The skip is keyed on those **observed inputs**, not on a list of effect kinds. A writer that bypasses effects and changes what the layout reads must call `rebuild_layout()` itself. Measurements: `context/test-evidence/layout-cost.md`. 24. **The cursor is per-client and keyed by view.** `ViewState::cursors: HashMap` holds `selected`/`row_offset`/`col_offset`, keyed by view name; an absent key means the origin. Reach it through `App::cursor()` (a `Copy` snapshot), `App::cursor_mut()`, and `App::reset_cursor()` — never off `View`, which no longer has the fields (improvise-8uz). Two consequences. **Changing the grid's shape must reset the cursor explicitly**: `View::transpose_axes`/`cycle_axis` used to do it themselves, and now `TransposeAxes`/`CycleAxis` call `app.reset_cursor()` — a new geometry effect that renumbers rows or columns has to do the same. **Replacing the document must clear the map**: `LoadModel` and `AcceptWizardModel` call `cursors.clear()`, because the incoming workbook's view names have nothing to do with the outgoing one's. That clearing replaces `Workbook::normalize_view_state`, which is gone — a `View` can no longer carry a stale offset. The same clearing owes the **session mirrors**: `install_workbook` bumps `App::document_generation`, and `ClientSession::project` reseeds any mirror seeded at an earlier one (improvise-cqq). An effect cannot reach `Sessions`, so the generation is the mechanism and `Sessions::rebase` is only the client-facing half. The same is true of a **geometry** change, and it owes the mirrors *both halves*: `App::cursor_epoch` is bumped by `reset_cursor` and watched by `Sessions::adopt_cursor_epoch`, which resets every mirror **and** returns `Broadcast::ToAll(Command::ResetCursor)` (improvise-gvnb). Doing only the first is **strictly worse than doing nothing** — the diff then believes every client already holds origin-positioned content and sends nothing, so they stay stale *permanently* rather than until their next scroll. The trigger is the **epoch moving**, not the command variant, for the reason gotcha 39 watches `document_generation` instead of matching on `LoadModel`: which effects renumber the grid is a per-effect decision, so a table of variants would be a second description of it, free to disagree. What is broadcast is an **invalidation, never a position** — another client's transpose must not move where I am looking (that is UI and it is mine, improvise-g0f), only declare that where I was looking has stopped meaning anything. **Since improvise-ni6k no effect calls `reset_cursor` at all — the cursor is invalidated by OBSERVING the grid's frame.** `apply_effects` diffs the frame after every model effect, **inside the loop**, because a batch that reframes and then places the cursor loses the placement otherwise (four records-mode tests pin that position). There is no list of effects anywhere, so the mechanism is total over `EffectKind::Model(_)` by construction — a genuinely stronger shape than a hand-kept table, and the reason the old "a geometry effect that forgets `reset_cursor` is invisible in both directions" hazard is closed. @@ -193,7 +193,7 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text 25. **`ViewState` and its field types are `Serialize`/`Deserialize`.** It is the wire payload of the client Snapshot (improvise-q08), so `AppMode`, `MinibufferConfig`, `MinibufColor`, `ViewFrame`, `KeyCode` and `KeyModifiers` all derive both (improvise-a2x), and `improvise-command` depends on `serde` for it. The constraint this puts on new state: **no borrowed-`'static` fields anywhere ViewState can reach** — `Deserialize` cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. `view_state_survives_a_json_round_trip` guards it. `ViewState` and `Snapshot` also derive `Clone` and `PartialEq`, so the bootstrap message can be compared in a round-trip test. -26. **Axis presentation lives on `Axis`, colors resolve in `draw.rs`.** Four vocabularies hang off the enum in `view/axis.rs` — `plain_label()` ("Row"/"Col"/"Page"/"None"), `glyph()` (which `Display` appends, giving "Row ↕"), `short_label()` (three columns for the tile bar; `None` is "·", *not* a truncation), and `initial()` (one letter for the view panel; `None` is empty). Plus `color() -> AxisColor` and `next()` for the Row→Column→Page→None rotation. `AxisColor` is a semantic palette: `improvise-core` must not name a ratatui type, so `draw.rs::axis_fg` does the one mapping to a terminal color, exactly as `minibuf_fg` does for `MinibufColor` (improvise-rml). The tile bar and category panel each carried their own copy of the color table before, with nothing keeping them in agreement. Not covered here: `persistence/format.rs` maps `Axis` to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer. +26. **Axis presentation lives on `Axis`; its colour is a `Face`.** Four vocabularies hang off the enum in `view/axis.rs` — `plain_label()` ("Row"/"Col"/"Page"/"None"), `glyph()` (which `Display` appends, giving "Row ↕"), `short_label()` (three columns for the tile bar; `None` is "·", *not* a truncation), and `initial()` (one letter for the view panel; `None` is empty). Plus `next()` for the Row→Column→Page→None rotation. **`Axis` carries no colour**: `AxisColor` and `Axis::color()` are gone. A colour is a `Face::Axis(axis)` in `improvise-protocol/src/face.rs` (ADR 004, `context/code/protocol.md` §10), resolved by the `Theme`, and `draw.rs::face_style` is the one place a face becomes a ratatui `Style` — the tile bar and the category panel both call it, so neither keeps a table. `improvise-core` still names no ratatui type. Not covered here: `persistence/format.rs` maps `Axis` to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer. 27. **`Axis::Filter` constrains without being navigable.** Drilling fixes its coordinates on `Filter`, not `Page` (improvise-dz8): both restrict which records are visible, but only `Page` is something the user may move or page through. The split shows up as two different questions, and picking the wrong one is the trap: **"what data is visible" is `View::filtering_cats()`** (Page ∪ Filter — used by `GridLayout` and by `add-record-row` when it builds a key from the current constraints), while **"what can the user act on" is `categories_on(Axis::Page)`** (used by `[`/`]` page cycling) or `View::tileable_cats()` (used by all three tile-select sites). `Axis::next()` maps `Filter` to itself and no navigable axis rotates into it, so tile select cannot hand one out. The `.improv` grammar accepts `filter` and round-trips a selection with it — a Filter axis only ever occurs in the never-persisted `_Drill` view, but the alternative was writing it out as some other axis. **What a virtual view must *be* is model-side and has one home** (improvise-gd28): `view::{records_spine, records_steps, drill_steps}` return `Vec` — pure data naming no `Effect`, no `App` and no front end — and `cmd/grid.rs::steps_into_active_view` is the whole of *how the TUI gets there*, twenty lines a browser answers its own way. Before the split, the drill's filter placement and the records spine were spelled out twice in `cmd/grid.rs` with nothing keeping them in agreement. `DrillState` deliberately did **not** move: it holds an `Rc` and its `pending_edits` are uncommitted writes, so gotcha 30's session-local tag already places it. Reasoning: `context/code/core.md` §11. @@ -222,7 +222,7 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text 39. **`Applied` is an enum because a document swap is not a diff.** `Sessions::apply_checked` returns `Applied::Diff { dirty, broadcast }`, `Applied::Rebased { snapshots }` or — since ADR 001 layer 1 — `Applied::Rejected { reason }`, and the enum *is* the mechanism: a host that wants the `Dirty` must match, and matching names the case where a `Snapshot` is owed. The server's mirrors self-heal — `ClientSession::project` reseeds on the `document_generation` bump — but `ViewState::cursors` lives on the **client**, keyed by the outgoing document's view names (gotcha 24), and only a fresh `Snapshot` replaces it. `Command::Model` put the three `WholeWorkbook` mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. `apply_model` therefore calls `rebase` itself rather than instructing the host to, exactly as `apply_switch` adopts the accepted view itself. **The trigger is the generation moving, not the command variant**: a `LoadModel` that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a *failed* load — `a_load_that_fails_is_not_a_rebase` guards that arm. A `rebase: bool` beside `dirty` was rejected, because the host that forgets to call `rebase` is the host that forgets to read the flag. The **third** case is a *normal outcome*, not an error, and it is in the enum for the same reason the other two are: a host that wants the `Dirty` must match, and matching names it. `ServerMsg::Rejected` is the matching frame, addressed to **the asker alone** — nothing happened, so nobody else is owed anything — and the client keeps it as a `Rejection` rather than a sentence, because force needs to know *which* record and *which* kind (gotcha 43). **Fixture warning.** When testing a document swap, the incoming document's active view must have the **same name** as the outgoing one. A rename makes `apply_model` broadcast a `SwitchView`; the client folds it, and the stale cursor is then read under a key nothing has written — right *by accident*, with the bug invisible. `other_document()` renames, and a first version of this test passed because of it; `a_document_shaped_differently_under_the_same_view_name()` keeps the name and changes only the shape, so there is nothing to hide behind. -40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per *view shape*) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. **Only the `Dirty` is read off the `EffectKind`; both lower tiers are *observed* around the loop** — the source says so at `apply_effects` and this paragraph said the opposite until improvise-ilvl's audit. The `EffectKind` decides only *whether to look*: `touches_model` arms the shape snapshot, `writes_cells` the record fingerprints. `record_versions` **must** be observed, because an `EffectKind` says *that* a batch writes cells, never *which*; `layout_version` is observed for a better reason than necessity — `observe_reshape` diffs `view_shapes` before and after and bumps only where a view's shape actually moved (improvise-pdvm/nynk), so it is a **fact about the document** rather than a claim about the batch. Two consequences the declared form did not have. An **idempotent** `set-axis` bumps nothing, and batching two real changes into one line or two gives the same `2`. And the gotcha 21 argument is subsumed rather than restated: an aborted batch moved no shape, so there is nothing to not-claim — which is why observing once **after** the loop is correct where accumulating `Dirty` inside it is. Note the arming is deliberately wider than the tier: a `CellData`-only batch that prunes a row out of a view **does** bump `layout_version`, because the shape moved. Three further consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run. +40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per *view shape*) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. **Only the `Dirty` is read off the `EffectKind`; both lower tiers are *observed* around the loop** — the source says so at `apply_effects` and this paragraph said the opposite until improvise-ilvl's audit. The `EffectKind` decides only *whether to look*: `touches_model` arms the shape snapshot, `writes_cells` the record fingerprints. `record_versions` **must** be observed, because an `EffectKind` says *that* a batch writes cells, never *which*; `layout_version` is observed for a better reason than necessity — `observe_reshape` diffs `view_shapes` before and after and bumps only where a view's shape actually moved (improvise-pdvm/nynk), so it is a **fact about the document** rather than a claim about the batch. Two consequences the declared form did not have. An **idempotent** `set-axis` bumps nothing, and batching two real changes into one line or two gives the same `2`. And the gotcha 21 argument is subsumed rather than restated: an aborted batch moved no shape, so there is nothing to not-claim — which is why observing once **after** the loop is correct where accumulating `Dirty` inside it is. **`view_shape` is configuration-only** — axes, drawn item names, folds, page selections and the `prune_empty` *flag* — so an occupancy write does **not** bump `layout_version`: a `CellData`-only batch that prunes a row out of a view moves the cursor epoch (the frame observation) and not `layout_version`, as `context/code/command.md` records (the question that raised was improvise-6nsd, closed). The arming is wider than the tier (`touches_model` takes the shape snapshot), but a bump needs a shape hash to have moved. Three further consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run. 41. **`Storage` is not a VFS, and that closes the SQLite question.** The trait is whole-document `read`/`write` bytes (gotcha 2, `invariant(storage_is_two_ops)`) — no offsets, seeking, handles or locking — so an `sqlite3_vfs`, which needs `xRead(offset,len)`/`xWrite`/`xTruncate`/`xLock`, has nothing to bind to. SQLite *can* still sit behind it, as an in-memory database serialized to a blob (`rusqlite` `serialize`/`deserialize_read_exact`, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is **22% larger** than `.improv`, **4.7× larger gzipped**, ~**160×** larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. **A second serialization format is a second implementation of the grammar**, obliged to track `improv.pest` forever; and **its own round-trip test cannot detect drift from `.improv`**, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way `export_sqlite` beside `export_csv`, native-only and feature-gated: `libsqlite3-sys`' C build fails under this repo's nix cc-wrapper for `wasm32-unknown-unknown`, which would break the `wasm` CI gate the day it landed. @@ -236,13 +236,13 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text 43. **A batch carries its basis, and the server had to publish it first.** `Command::Model { basis, cmd }` is a **struct variant** so the basis cannot be forgotten, and `Sessions::apply_model` checks it *before* applying anything — a stale batch is never applied, not even partially (improvise-xwi6/fkfg, ADR 001 layer 1). Four things to know. **`Basis` has no `Default` and no public constructor**: a zero-valued basis is not "no claim", it is the claim that *the document is fresh*, which in a new `App` is true — so a default would be a forgeable bypass rather than an absence. `BasisCache::claim` is the only door a client has and it returns `None` until the server has published one. Note what that does and does not buy: `Basis` derives `Deserialize` because it must cross the wire, and its fields are public, so **a hand-rolled client can state any tiers it likes**. This is an *anti-staleness coordination check, not a security boundary* — `RecordUnstated` closes **omission**, not **invention**, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having: it makes the honest path the easy one. **Which records are checked is the server's decision** — `reduce.rs::records_named`, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the *client's* claim would be a **force by omission**: the record tier skipped, with no flag to remove and nothing in the message saying so. A claim that omits a record the command writes gets `Rejection::RecordUnstated`, a case of its own because "you lost a race" and "you were never in one" are different things to tell a user. **`Rejection` keeps structural and data staleness apart**, which is the whole reason ADR 002 put a maintained `RecordHash` index in `DataStore`: `RecordGone` means nothing resolves and there is nothing to force (**force never creates structure**), `RecordMoved` means it resolves at another version and force is one keystroke — which is also why the wasm client stores the `Rejection` itself rather than collapsing it into `status_msg`, since a string cannot carry which record or which kind. **And the ADR omitted the publication.** It said a batch carries the basis it was composed against and never said how a client learns one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. `Projection::Basis` + `BasisCache` close that — the server publishes the two document tiers plus the counters for the records *this client's grid names*, diffed against what that client was last told and seeded in the bootstrap — and it is emitted **after** the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown, the mirror image of gotcha 33. That ordering was documented as load-bearing in three places and **held up only by the order of two statements** until `a_published_basis_follows_the_grid_updates_it_describes` pinned it (proved by injecting the swap) — the same job `a_cell_update_that_overtakes_its_labels_is_lost` does for the mirror-image rule. Two things are unchecked and deliberately so: `Command::SwitchView` carries no basis (it sits outside `Command::Model`; the workbook refuses a view it does not have, so it cannot corrupt, but it is a hole in the rule), and `apply_command` does not check at all — **the check is a property of the session door, not of the mutation path**, which is right because a host holding the `App` is the truth rather than a claimant. That second one **was** a door rather than an exemption, and needed a type to stop being one (improvise-ondj): both took a wire `Command` and their `Command::Model { cmd, .. }` arm **discarded the basis with `..`**, while `ui/reduce.rs`'s own module doc pointed every future host — the browser worker included — at exactly those two names. **The struct variant closed omission *from the message*; it did nothing to close ignoring *on arrival*, which is the half a struct variant is usually reached for.** So both now take `Checked<'_>`, a private-field newtype over a `&Command` whose only constructor is `Checked::against(&App, &Command) -> Result<_, Rejection>` — also the only caller of `stale_basis`. `Sessions::apply_checked` receives the proof and `Checked::against` is answered **once, before the dispatch**, so `apply_switch`/`apply_model` receive the proof rather than each remembering to ask: an invariant made unrepresentable rather than validated, the same answer `prune_axis_items` and `gp-tree`'s `Row` got. Visibility was the weaker option and was rejected for a concrete reason — `pub(crate)` closes only the out-of-crate path, and `reduce_full` had **no production caller anywhere in the workspace**, so it would have become dead code. It has since been **deleted** — which is the argument’s own conclusion arriving, not a contradiction of it; the surviving sibling is `reduce_key`. This paragraph named it in the present tense for some time after, so read every function name here as a claim to check. **The regression guard is therefore the signature**: the test that reproduced the bug no longer compiles, and what remains asserts the constructor refuses and that refusing leaves the document untouched. One residual hole, documented on the type and filed: a `Checked` does not name *which* `App` it was checked against, so a multi-document host could in principle check one and apply to another. That is also why `ui::reduce::caught_up(&App, cmd)` is not a bypass: it takes `&App`, so nothing reachable from a wire can call it. **Since improvise-955n the frame, not the command, is what is answered — and only two of the three tiers hoist.** `CheckedBatch::against` runs before anything in a `ClientFrame::Commands` is applied. `document_generation` and `layout_version` describe the *document*, so answering them once for the frame is exactly right, and a first version hoisted all three — which was a lost update: the record tier asks "is the client's picture of **this record** current", and which records are at stake is a property of the *command* (`records_named`). Skipping it for later commands let `[SetCell(North), SetCell(South)]` overwrite a South somebody else had moved, silently, with the frame reported accepted, and invisible to the other two tiers because a `SetCell` is `CellData` and moves neither. So every mutation answers for its own records **against the frame's basis** — the one the first basis-carrying command stated, never each command's own field, or a later command could claim a fresher picture than the frame was composed under. The one exemption is stated over a *property* rather than a list of commands: a record that **neither resolves in the document nor is claimed by the basis** is one the frame is about to create (`[AddItem(East), SetCell(East)]`, which `BasisCache::claim` cannot possibly claim for), and only that combination is exempt; a record that does not resolve but *is* claimed is still `RecordGone`. That the `Basis` inside a *later* `Command::Model` is ignored rather than compared is an honest interim: clause 1 says the **frame** states one basis, and moving the field from `Command::Model` to `ClientFrame::Commands` would make that unrepresentable-otherwise instead of a convention — a protocol change that belongs with clause 4. Two things worth keeping from the rejected design: **arity is not a property of a variant** (a future `Command::RunScript` is one variant performing unboundedly many mutations), so no count over the vocabulary can enforce "one mutation per frame"; and **the transaction boundary is not the frame**, because one command may already be a chain, which is why clause 4's mechanism belongs at `App::apply_effects`. -44. **The TUI grid draws from its cache, and since improvise-anr9 the cache frames it too.** `GridWidget` holds no `&Model` and no `&View`: cell text, aggregate-ness (`CellId::Aggregate`, ADR 002), column widths, the page bar, the totals strip and now the **frame** all come off `GridCache`. The TUI still builds that cache over the **whole** grid (offsets zero), so a cache position is an absolute position — the browser's `Viewport` is a real window, the TUI's is not, and the two clients therefore use it differently; column totals **must** be projected rather than client-derived, because the sum runs over every row *including the ones a viewport scrolled past*. The frame is an `AxisFrame` per axis — `cats` (the categories, outermost first) plus `entries: Vec` interleaving `Group(FrameGroup)` headers and `GroupEnd { level }` markers with `Item { tuple, collapsed }` label *tuples*, exactly as `GridLayout::row_items` interleaves them. That replaced a `Vec` built by `GridLayout::row_label`, which slash-joined the tuple — the exact rendering `two_row_categories_produce_cross_product_labels` says must not appear — so the browser drew `Food/Alice` where the TUI drew two header columns, with no groups and no collapse indicators. It is **one** type and not a `labels` plus a `levels` beside it: two answers to "what are the labels" is the disease (improvise-edp), and this is the second cure, which cost a *type* change rather than an addition. Three things to know before touching it. **`leaf_labels()` is not "the labels"** — it is the innermost label, what a one-line summary calls a row by; a draw loop wants `header_labels(first_rendered)`, `groups()` and `shares_ancestry(sel)`, each of which is one pass, because `item`/`label`/`shows_level`/`group_of` scan `entries` from the start and asking them per position is quadratic in the frame *and grows with the TUI's absolute scroll offset* (found three times on anr9; the per-position forms survive for one-off lookups and say so). **The width rule has one home**, `improvise_protocol::header_level_widths` — widest label per level, plus a gap, clamped — reached by `AxisFrame::header_widths` and by `ui::viewport::compute_row_header_width`, which borrows level tuples off a `GridLayout` rather than building a frame on the keypress path. **A `cats` change re-identifies every position**: a cell's identity is `cats` zipped with its tuple, so `invalidate_relabelled` blanks the whole axis when the categories move, not merely the positions whose tuples differ, and blanks the **totals** on a recategorization of *either* axis — a column total is a sum over the rows, so re-identifying the rows leaves a number answering coordinates the grid no longer has (gotcha 33(b): the rule lives in `apply`, not in a producer's memory). What is still **not** on the cache is `record_col`, and that is correct: it keys pending drill edits and is session-local (gotcha 1). `body_from(idx)` takes an argument only because the two front ends scroll differently — a browser's frame is already windowed and passes `0`, the TUI's covers the whole grid and passes its row offset — not because the rule differs. `group_of` answers group **membership**, and since improvise-e265 membership is **the innermost open run**: `FrameGroup` and `FrameEntry::GroupEnd` each carry a `level` (an index into `cats`), runs **nest**, a header at level L replaces L and closes every deeper run, and a `GroupEnd { level }` closes L and deeper. Before e265 it was the nearest preceding header unless a level-less `GroupEnd` intervened (improvise-gd83, PR #77), which could not express one run inside another. `GroupEnd` draws nothing and exists only so membership is not inherited. `AxisFrame` deserialises through `UncheckedAxisFrame` and **refuses** a `level >= cats.len()` — a level naming no category is a malformed frame, not a clamp. And `FrameEntry::Item` is `{ tuple, collapsed }`: `collapsed` is the **layout's own fold flag**, read by `AxisFrame::folds`, so a client asks the frame whether a position folds rather than re-deriving it from the groups (improvise-4k1j). Both of the gaps this sentence used to list are **closed**, and it named them as open for a day after: `group_runs` coalesces by **header identity** (`std::ptr::eq`), not by name, so two runs under two headers equal as values are two runs (improvise-h0iv); and `AxisFrame::first` carries **no** `serde(default)` — its doc comment says why, since zero is a claim and not an absence (improvise-1wl0). `doctrine(a_stale_list_of_what_is_missing_is_worse_than_no_list)` is about exactly this paragraph, twice now. And `project::grid_of` evaluates every cell per frame where the widget evaluated only the visible ones — a widening of an existing whole-grid cost. Since improvise-jytm `grid_in` may additionally **derive a `GridLayout`** for a session on a view the host is not on; the host's layout is reused where the names agree, which is the common case and the only one that can carry a drill snapshot. +44. **The TUI grid draws from its cache, and since improvise-anr9 the cache frames it too.** `GridWidget` holds no `&Model` and no `&View`: cell text, aggregate-ness (`CellId::Aggregate`, ADR 002), column widths, the page bar, the totals strip and now the **frame** all come off `GridCache`. The TUI still builds that cache over the **whole** grid (offsets zero), so a cache position is an absolute position — the browser's `Viewport` is a real window, the TUI's is not, and the two clients therefore use it differently; column totals **must** be projected rather than client-derived, because the sum runs over every row *including the ones a viewport scrolled past*. The frame is an `AxisFrame` per axis — `cats` (the categories, outermost first) plus `entries: Vec` interleaving `Group(FrameGroup)` headers and `GroupEnd { level }` markers with `Item { tuple, collapsed }` label *tuples*, exactly as `GridLayout::row_items` interleaves them. That replaced a `Vec` built by `GridLayout::row_label`, which slash-joined the tuple — the exact rendering `two_row_categories_produce_cross_product_labels` says must not appear — so the browser drew `Food/Alice` where the TUI drew two header columns, with no groups and no collapse indicators. It is **one** type and not a `labels` plus a `levels` beside it: two answers to "what are the labels" is the disease (improvise-edp), and this is the second cure, which cost a *type* change rather than an addition. Three things to know before touching it. **`leaf_labels()` is not "the labels"** — it is the innermost label, what a one-line summary calls a row by; a draw loop wants `header_labels(first_rendered)`, `groups()` and `shares_ancestry(sel)`, each of which is one pass, because `item`/`label`/`shows_level`/`group_of` scan `entries` from the start and asking them per position is quadratic in the frame *and grows with the TUI's absolute scroll offset* (found three times on anr9; the per-position forms survive for one-off lookups and say so). **The width rule has one home**, `improvise_protocol::header_level_widths` — widest label per level, plus a gap, clamped — reached by `AxisFrame::header_widths` and by `ui::viewport::compute_row_header_width`, which borrows level tuples off a `GridLayout` rather than building a frame on the keypress path. **A `cats` change re-identifies every position**: a cell's identity is `cats` zipped with its tuple, so `invalidate_relabelled` blanks the whole axis when the categories move, not merely the positions whose tuples differ, and blanks the **totals** on a recategorization of *either* axis — a column total is a sum over the rows, so re-identifying the rows leaves a number answering coordinates the grid no longer has (gotcha 33(b): the rule lives in `apply`, not in a producer's memory). What is still **not** on the cache is `record_col`, and that is correct: it keys pending drill edits and is session-local (gotcha 1). `body_from(idx)` takes an argument only because the two front ends scroll differently — a browser's frame is already windowed and passes `0`, the TUI's covers the whole grid and passes its row offset — not because the rule differs. `group_of` answers group **membership**, and since improvise-e265 membership is **the innermost open run**: `FrameGroup` and `FrameEntry::GroupEnd` each carry a `level` (an index into `cats`), runs **nest**, a header at level L replaces L and closes every deeper run, and a `GroupEnd { level }` closes L and deeper. Before e265 it was the nearest preceding header unless a level-less `GroupEnd` intervened (improvise-gd83, PR #77), which could not express one run inside another. `GroupEnd` draws nothing and exists only so membership is not inherited. `AxisFrame` deserialises through `UncheckedAxisFrame` and **refuses** a `level >= cats.len()` — a level naming no category is a malformed frame, not a clamp. And `FrameEntry::Item` is `{ tuple, collapsed }`: `collapsed` is the **layout's own fold flag**, read by `AxisFrame::folds`, so a client asks the frame whether a position folds rather than re-deriving it from the groups (improvise-4k1j). Both of the gaps this sentence used to list are **closed**, and it named them as open for a day after: `group_runs` coalesces by **header identity** (`std::ptr::eq`), not by name, so two runs under two headers equal as values are two runs (improvise-h0iv); and `AxisFrame::first` carries **no** `serde(default)` — its doc comment says why, since zero is a claim and not an absence (improvise-1wl0). `doctrine(a_stale_list_of_what_is_missing_is_worse_than_no_list)` is about exactly this paragraph, twice now. And `project::grid_of` used to evaluate every cell per frame where the widget evaluated only the visible ones; since improvise-felo.13 the host's own layout keeps a `GridSummary` (whole-grid widths, column formats and totals, `ui/viewport.rs::grid_summary`, held in `App::grid_geometry()`), and `grid_in` evaluates only a `grid_window` over it. The browser's window therefore costs its own cells. A session on another view, or a rederived model, still takes the whole-grid `grid_body` path. `grid_window` equals `grid_body` where the summary is current. Since improvise-jytm `grid_in` may additionally **derive a `GridLayout`** for a session on a view the host is not on; the host's layout is reused where the names agree, which is the common case and the only one that can carry a drill snapshot. **`RecordHash` is document-scoped; `AggregateHash` is layout-scoped and cannot be otherwise** (improvise-gd28). A `RecordHash` resolves through `DataStore`'s maintained index — but **only when the command names a view the document has** (improvise-wpxr; Codex P2 on #281). `SetCell`/`ClearCell` carry `view`, and the lookup is **strict**: a view the document lacks drew nothing, stored or not, so it is `Rejection::RecordGone` rather than a `resolve_view` fallback. An **empty** record — one the store holds no cell for, which the grid still drew — resolves against **that named view's layout**, the only place it exists. `RenameRecordCoordinate` is the exception and resolves in the **store alone**, since only a record holding cells can be renamed: `protocol::named_records` answers each command's records as `NamedRecord::Drawn { view, record }` or `NamedRecord::Stored(record)`, one exhaustive match. Every one of these questions goes through `App::grids()`, a `Grids` borrow that memoises **one derived `GridLayout` per view for as long as the borrow lives** (improvise-kimw) — hold one across a check, never across a mutation, since the layouts it holds describe the model it borrowed. An aggregate has no existence outside the layout that derives it, so there is no index to fall back to and none can be built — `App::resolve_aggregate` therefore answers only for the **host's** active view, which since improvise-jytm is not the client's. **That, and not ADR 001 layer 2, is what blocks browser drill.** The ticket's blocking argument was right about the *commit* (`ApplyAndClearDrill` is addressed to one session, gotcha 38) and wrong about the *entry*: entering a drill needs no commit, and `CreateView`/`SetAxis`/`SetPageSelection` are already unaddressed `ModelCommand`s that name their view. The gap is one level down from where it was looked for — **the mutation and its operand have different scopes**: the mutation configures the shared `_Drill` and is addressed to nobody, while the operand exists only inside one session's layout. So the fix is not a new mechanism but the `view: String` convention every other view-scoped variant already follows, resolved against a layout derived for that name (improvise-obtu). Two things travel with it: editing *inside* a drill grid from a browser does not follow from it, because `edit_or_drill` declines records-mode editing out loud before reaching a cell (improvise-fdk1, since closed, is the separate records-coordinate rename on the wire — `RenameRecordCoordinate`, gotcha 38 — and did not unblock drill editing); and closing it makes improvise-t219 live, `_Drill` being one shared view in `Workbook::views` that two clients drilling would contend for — the same masking relationship improvise-6uva has with improvise-up8g. Guarded by `an_aggregate_a_session_was_shown_does_not_resolve_against_the_hosts_view`, whose **control** is the load-bearing half — the same surrogate resolves once the host is moved onto that view, so a blanket `None` cannot pass. 45. **One session loop, two transports — and the browser saves itself.** `Host` is `App` + `Sessions` + `handle(id, ClientFrame) -> Vec` and lives in `improvise_command::ui::host`, **not** in `improvise-ws-server`, which re-exports it. It moved for improvise-9x6: the worker hosts the same authoritative `App` over a `MessagePort` and cannot depend on tokio, and a second copy would have been exactly the mirror improvise-xks6 had just deleted for the frames — a drifting `Applied`/`Broadcast` match is invisible until a client sees it. `registry` stayed on the server, because a worker hosts **one** document. What the worker adds is only what a `MessagePort` differs in: **one client by construction** (the `Addressed` filter is honoured anyway, against a future `SharedWorker`), and **a handshake with no close** — `ClientFrame::Viewport` must still come first, but a pre-handshake frame is *dropped* rather than closing a connection that does not exist. - **Saving is a frame, and dirtiness is read off the classification the effect already carries.** `effect::Save` is `ViewOnly` (gotcha 3) — correctly, since it writes *this host's* `file_path` through *this host's* `Storage` — so the derivation that builds `ModelCommand` from the model effects cannot name it, and for a while that meant no client on any transport could ask its host to save. `ClientFrame::Save` closes it (improvise-368h): a **request** carrying no path, because a client cannot see the host's storage, answered `ServerFrame::Rejected(NoDocumentPath | SaveFailed)` to the asker alone and **silently on success** — nothing moved, so no widget is dirty. It is a frame and not a `Command` for one concrete reason: as a `Command` it put a runtime-fallible operation inside a batch `ClientFrame::Commands` promises is all-or-nothing, so `[Save, SetCell]` reported a failed write and then applied the edit. It also fails the test every `Command` passes — it mutates no shared state and no client's slice, so `reduce_view` has nothing to fold and `Checked` has no basis to answer for. **And `MarkDirty` is gone.** It was a separate effect the *command* layer composed beside each mutation — reasonable on the key path, unreachable from anywhere else, because improvise-kmgs derived `ModelCommand` from the effects **one at a time** and so put `SetCell` on the wire without its companion (improvise-gv9r): a derivation correct per element and wrong for a set, whose symptom was that "save if dirty" never fired. `App::apply_effects` now raises the flag for any `EffectKind::Model(_)`, all three `ModelChange` levels included, which makes omitting it unrepresentable rather than validated. It over-approximates in the same direction `layout_version` does — a `LoadModel` that cannot read its path leaves the flag raised, costing one redundant write — and `Save` clears it in loop order, so a batch that edits and then saves ends clean. The worker's persist trigger is therefore the **flag**, not a table of command variants: an observation rather than a second description of which commands change the document, for the reason `cursor_epoch` beats matching on `CycleAxis` (gotcha 24) and `document_generation` beats matching on `LoadModel` (gotcha 39). It is also strictly tighter — a refused batch raises nothing, so the "was anything rejected" guard went with it. A browser still never *sends* `ClientFrame::Save`, and that is the answer rather than a gap: there is no file the user owns and no Ctrl-S semantics, the document *is* the OPFS entry, so the worker persists on its own behalf through the one existing `Save` effect. The cost that remains: `Storage` is whole-document (gotcha 41), so OPFS rewrites the file per accepted model command. + **Saving is a frame, and dirtiness is read off the classification the effect already carries.** `effect::Save` is `ViewOnly` (gotcha 3) — correctly, since it writes *this host's* `file_path` through *this host's* `Storage` — so the derivation that builds `ModelCommand` from the model effects cannot name it, and for a while that meant no client on any transport could ask its host to save. `ClientFrame::Save` closes it (improvise-368h): a **request** carrying no path, because a client cannot see the host's storage, answered `ServerFrame::Rejected(NoDocumentPath | SaveFailed)` to the asker alone and **silently on success** — nothing moved, so no widget is dirty. It is a frame and not a `Command` for one concrete reason: as a `Command` it put a runtime-fallible operation inside a batch `ClientFrame::Commands` promises is all-or-nothing, so `[Save, SetCell]` reported a failed write and then applied the edit. It also fails the test every `Command` passes — it mutates no shared state and no client's slice, so `reduce_view` has nothing to fold and `Checked` has no basis to answer for. **And `MarkDirty` is gone.** It was a separate effect the *command* layer composed beside each mutation — reasonable on the key path, unreachable from anywhere else, because improvise-kmgs derived `ModelCommand` from the effects **one at a time** and so put `SetCell` on the wire without its companion (improvise-gv9r): a derivation correct per element and wrong for a set, whose symptom was that "save if dirty" never fired. `App::apply_effects` now raises the flag from the effect's `EffectKind`, which makes omitting it unrepresentable rather than validated — **declared** for `WholeWorkbook` (a replaced document differs from the file its path names, so a `LoadModel` that cannot read its path still raises it, costing one redundant write) and **observed** for `CellData` and `LayoutGeometry`, where `document_content`/`document_config` fingerprint the document before and after, so a write that leaves it as it was raises nothing (improvise-748f). The observation is read inside the loop, and `Save` clears the flag in loop order, so a batch that edits and then saves ends clean. The worker's persist trigger is therefore the **flag**, not a table of command variants: an observation rather than a second description of which commands change the document, for the reason `cursor_epoch` beats matching on `CycleAxis` (gotcha 24) and `document_generation` beats matching on `LoadModel` (gotcha 39). It is also strictly tighter — a refused batch raises nothing, and neither does a write that changes nothing, so the "was anything rejected" guard went with it. A browser still never *sends* `ClientFrame::Save`, and that is the answer rather than a gap: there is no file the user owns and no Ctrl-S semantics, the document *is* the OPFS entry, so the worker persists on its own behalf through the one existing `Save` effect. The cost that remains: `Storage` is whole-document (gotcha 41), so OPFS rewrites the file per accepted model command. **The async seam does not survive being boxed.** `OpfsFs::open`/`attach`/`refresh`/`commit_deletes` are on the struct, not on `Storage`. `App::storage` is a `Box` and a trait object can name none of the four, so **handing the backend to the `App` puts the async seam permanently out of reach**; the host must hold the concrete backend *and* give the `App` a `Storage`, which means sharing one. That is `impl Storage for Rc`, and it lives in **`improvise-io`** rather than in each worker (improvise-hmws): `impl Storage for Rc` is two foreign types in any other crate, so improvise-9x6 wrote a forwarding newtype that every future worker would have written again. `Rc` and not `Arc` because the one caller is a Web Worker and wasm32 has no other thread. **The flat root is no longer load-bearing**: `OpfsFs::list` joins `dir` like the other two backends, so `storage_contract` takes `dir` as a parameter and the three agree at *every* directory rather than only at `Path::new("")` where joining is the identity (improvise-vhcz). What the index holds is **documents, not entries**, and the split between its three writers is the rule: `attach` indexes nothing (what it creates is zero bytes), `Storage::write` indexes the name and un-queues any pending delete of it, and `refresh` rebuilds from OPFS skipping zero-byte entries — counting an entry it cannot measure as a document, since the one cause of that is another tab holding the exclusive handle (improvise-vvl3). @@ -282,6 +282,11 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text **A scroll is measured where it is drawn** (improvise-gtn7.1). The keypress path counted `visible_cols` from `term_width` while `draw_content` gave the grid `term_width` minus the 32-column side panel, so with any panel open the cursor scrolled to a column never drawn. Now `viewport::grid_width(term_width, side_panel_open)` is the one answer, reached through `App::grid_width`, and both `draw_content` and `App::cmd_context` read it; `App::side_panel_open` is the one answer to whether a panel took its column. The tile bar had the same shape: `first_tile` chose its scroll against a width that began one or two columns before where the tiles were drawn, and both now start at `tiles_left(start)`. The grid bug was latent from `b5418f2e`, when `compute_visible_cols` took `term_width`, and was exposed by `50e4a372`, which opened the category panel by default. Still open: columns of mixed width, where a `visible_cols` count taken at the old offset is applied at the new one (improvise-gtn7.2). + +51. **The command language reads pipes, refuses surplus, and checks names when it applies (improvise-31oe.2, 31oe.3).** Reasoning: `context/code/cli.md` §14 and §15. (a) A `:` argument is read through `improvise_formula::quote`, so `create-view |By area|` names `By area`; a **surplus argument is refused** rather than ignored, and `cmd::core::only_when_omitted` gates the *cursor fallbacks* of `clear-cell`, `yank`, `paste` and `drill-into-cell` so a coordinate typed by the user is never overridden by the cursor. (b) `set-axis`, `set-page`, `toggle-group`, `hide-item`, `show-item` and `switch-view` return `RequireKnown(Needs)` (`effect.rs`) ahead of the real effect, built by `effect_cmds.rs::unless_unknown`. It is `ViewOnly`, writes nothing, is **off the wire** (listed in `vocabulary_is_complete.rs`) and checks the **document at apply time** — a `:` line builds every segment before any runs, so a check in `execute` refused `create-view Second . switch-view Second` — then refuses and aborts the batch (gotcha 21). (c) A refused name therefore stops the rest of a `:` line, while headless `improvise cmd` rebuilds the context per segment and runs each as its own batch (improvise-31oe.14, open). + +52. **A date column imports as one hierarchical category, and the import refuses what it cannot keep (improvise-8jk6.3, 8jk6.6, d0n6).** Reasoning: `context/code/formula-and-io.md` ("Import: …") and `cli.md` §13. The default `DateShape` nests Year > Quarter > Month (the chosen levels) in ONE category named after the column and opens folded to its years; `DateShape::Split` is the old flat split, chosen by `--split-date ` headless or `s` on the wizard's date step. `Plan::spelling_clash` refuses a raw, unparsed date spelled like a generated item or group (`2024`, `2024-01`) because the two would silently merge — asked at the wizard's ConfigureDates step **and** again at build time, so a headless import cannot reach it either. Headless, `--strict` turns a partial skip into a refusal before the build, and a folder whose CSVs are all header-only (no records) is refused as an import that kept no record. + --- ## File Inventory @@ -297,9 +302,13 @@ model/record.rs 226 / 8t RecordHash / CellId surrogates: the opaque i thin client names a cell by (ADR 002) model/category.rs 310 / 9t Category, Item, Group, CategoryKind model/symbol.rs 89 / 3t Symbol interning -view/layout.rs 2604 / 57t GridLayout, drill, records mode, records_columns +view/layout.rs 2604 / 57t GridLayout, drill, records mode, records_columns; a private + per-axis position index (`row_pos`/`col_pos`, rebuilt by `reindex`) + so a data position resolves in O(1) (improvise-felo.13) view/types.rs 817 / 43t View config (axes, pages, hidden, collapsed, format) -view/axis.rs 190 / 5t Axis enum + label/color/next methods, AxisColor +model/id.rs 290 slugify + IdAllocator: how a generated id is made (ADR 007 + clause 4, core.md §20). Nothing outside the file calls it yet +view/axis.rs 190 / 5t Axis enum + label/next methods (no colour: that is a Face) workbook.rs 413 / 22t Workbook: Model + cross-view ops, item facade format.rs 519 / 40t format_f64, parse_number_format (display only) ``` @@ -359,6 +368,9 @@ keymap.rs 975 / 0t KeyPattern, Binding, Keymap, ModeKey (17), T dispatch in improvise-command/src/command/keymap.rs viewmodel.rs 170 / 5t flatten_cat_tree — derivations between a cache and a renderer, called by both front ends (improvise-edp) +face.rs 682 Face (a role: Axis, WizardField, Mode*, MinibufferOf…), Theme, + AnsiSlot — appearance as data a front end applies (ADR 004, + protocol.md §10); draw.rs::face_style is the TUI's one translation key.rs 96 / 3t KeyCode, KeyModifiers, KeyEvent (moved from command/) frame.rs 156 / 3t ClientFrame/ServerFrame — the ONE envelope pair, both deployments (improvise-xks6). Nothing else defines one. @@ -451,7 +463,8 @@ ui/host.rs 3410 / 67t Host: one App + one Sessions; handle, comman ui/cat_tree.rs 174 / 8t Category tree flattener for panel ui/viewport.rs 259 compute_col_widths / compute_row_header_width / visible_cols (pure geometry; the width rule itself is the protocol's); - grid_width, SIDE_PANEL_WIDTH, MIN_GRID_WIDTH (improvise-gtn7.1) + grid_width, SIDE_PANEL_WIDTH, MIN_GRID_WIDTH (improvise-gtn7.1); + GridSummary / grid_summary / grid_window (improvise-felo.13) ``` ### TUI rendering, draw, main (`src/ui/`, `src/draw.rs`, `src/main.rs`) @@ -649,7 +662,7 @@ crossterm; that is correct, not a gap. `cargo check` passing is not the same as running. Two known runtime hazards it cannot see: `Instant::now()` panics on wasm (hence `web-time` in `improvise-command`), and any `std::fs` call compiles but fails in a browser -— see improvise-fuj for the two effects that still bypass `Storage`. +— improvise-fuj closed the effects that bypassed `Storage`; autosave and the headless final write still do (improvise-31oe.11). ## CLI @@ -660,7 +673,7 @@ improvise cmd 'add-cat Region' -f model.improv # headless command(s) improvise script setup.txt -f model.improv # run script file ``` -Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`, and `--on-duplicate refuse|sum|separate` (default `refuse`; applies with `--no-wizard`, where the wizard would otherwise ask — improvise-cgqt). +Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`, `--split-date ` (a date column imports as ONE hierarchical category by default; this restores the flat split, and `--extract` still picks the components — improvise-8jk6.3), `--strict` (with `--no-wizard`, any skipped record is a refusal and nothing is written, where the default writes and warns with a count — improvise-8jk6.6), and `--on-duplicate refuse|sum|separate` (default `refuse`; applies with `--no-wizard`, where the wizard would otherwise ask — improvise-cgqt). A headless import also refuses a `--split-date` that names a non-date field and an `--axis` that names a category the import did not build (exit 1). --- diff --git a/prolog-sessions/kb.pl b/prolog-sessions/kb.pl index e6865604..b17cf362 100644 --- a/prolog-sessions/kb.pl +++ b/prolog-sessions/kb.pl @@ -144,6 +144,7 @@ invariant(an_unprojected_cell_is_unknown,'Every client read of the grid cache go invariant(default_duplicate_policy_is_refuse,'ImportPipeline::on_duplicate defaults to DuplicatePolicy::Refuse, so any path that never asked the question, the headless CLI included (--on-duplicate defaults to refuse), errors naming the collisions rather than summing. On ResolveDuplicates, Refuse blocks advance with a message; the build under Refuse returns an error when collisions exist (improvise-cgqt).'). invariant(a_collision_is_a_cell_written_twice,'A collision is a cell -- a coordinate tuple AND a measure -- written by two or more records, never merely a shared tuple: records with disjoint measures on one tuple write different cells and collide with nothing, and only the records that share a cell are listed and numbered (improvise-9bit). Plan::place is the one placement pass, used by both collisions() and build_model_with_warnings, so the question asked and the model built cannot disagree.'). invariant(wizard_draws_are_clipped_to_the_panel,'Every import-wizard write goes through Panel in src/ui/import_wizard_ui.rs, which clips to the panel rows and width, so no step can draw past the buffer; text is measured in cells by ui::text::truncate and wrap, never in bytes (review B1/B2 and a Codex P2 on PR #304).'). +invariant(a_window_over_a_held_summary_equals_grid_body,'grid_window over the GridSummary of the same layout returns what the one-walk grid_body returns for that window, so the browser window and the TUI whole grid cannot disagree on a width, a format or a total. Pinned by a_window_over_a_held_summary_equals_the_one_walk_grid_body in improvise-command ui/viewport.rs; it holds only where the summary is current, which is why App::grid_geometry is dropped with the layout.'). doctrine(bug_first,'write a failing test demonstrating the bug before fixing it'). doctrine(unit_over_e2e,'prefer small unit tests over integration/e2e'). doctrine(suite_speed,'Two numbers, because the ambiguity between them is what let this drift: NO SINGLE TEST BINARY may exceed 2 seconds, and the WORKSPACE WALL CLOCK is the sum of them. Measured at 22d43b2: 1309 tests over 33 binaries, ~2.2s wall, longest single binary 1.42s -- so the workspace is already past 2s and the rule that bites is the per-binary one. Re-MEASURE rather than re-assert: cargo test --workspace prints a \"finished in\" per binary. Excludes compilation, and excludes harness/ (Playwright, minutes, its own CI job).'). @@ -448,6 +449,12 @@ work_done('improvise-cgqt',feature,'PR #304, merge 4f38a35e. The import wizard a work_done('improvise-9bit',bugfix,'PR #304 (commit f00ed3e9), merge 4f38a35e. The collision check counted records with disjoint measures on one tuple as colliding; collisions_among now keeps only records that write a measure another record on the tuple also writes. Rival: the tuple-only rule, which fails only_the_records_that_share_a_cell_are_listed_and_numbered.'). work_done('improvise-4klp',test,'PR #304 (commit f00ed3e9), merge 4f38a35e. the_name_check_covers_categories_derived_from_dates: a date-derived category name such as when_Year is refused as the occurrence category name. Green on arrival; its evidence is the reviewer rival that drops date-derived names from the taken names.'). work_done('improvise-felo.1',feature,'PR #307, merge 1719f2cf. MAX_CATEGORIES raised from 12 to 24 in improvise-core model/types.rs, pub and re-exported as improvise_core::model::MAX_CATEGORIES; the import wizard private copy is deleted and it reads the model constant. Fill-to-the-cap tests derive the number from the constant. crates/improvise-core/examples/category_width_bench.rs measures 12 against 24; the coupling inventory is context/code/core.md.'). +work_done('improvise-felo.13',bugfix,'PR #333, merge 8da5a310. GridLayout is no longer O(rows^2): a private per-axis position index (row_pos/col_pos, rebuilt by reindex) answers cell_key/row_tuple/column_key/folds/data_to_visual in O(1); App::refresh_layout rebuilds only when layout_stale, the drawn view name or the drill snapshot pointer moved, so a cursor key rebuilds nothing; the column widths and row-header width are measured once per layout (App::grid_geometry); GridSummary/grid_summary/grid_window in ui/viewport.rs let project::grid_in evaluate only a window; DataStore::exact_positions with arities answers a full-width sorted partial with one probe. Measurements: context/test-evidence/layout-cost.md and scale-bisect.md; harness benches/scale_bisect.rs and bench/*.py.'). +work_done('improvise-31oe.3',bugfix,'PR #316, merge 66931785. The command language reads |pipe quoting| through improvise_formula::quote, so create-view |By area| names By area; a surplus argument is refused; cmd::core::only_when_omitted gates the cursor fallbacks of clear-cell, yank, paste and drill-into-cell so a typed coordinate is never overridden. context/code/cli.md section 14.'). +work_done('improvise-31oe.2',bugfix,'PR #316, merge 66931785. set-axis, set-page, toggle-group, hide-item, show-item and switch-view check at APPLY time, through the ViewOnly off-wire effect RequireKnown(Needs) built by effect_cmds::unless_unknown, that the category, item, group or view exists, and refuse and abort the batch otherwise; a refused name stops the rest of a : line, while headless improvise cmd runs each segment as its own batch (improvise-31oe.14, open). context/code/cli.md section 15.'). +work_done('improvise-8jk6.3',feature,'PR #322, merge d571abbd. A date column imports as ONE hierarchical category (Year > Quarter > Month) by default and opens folded to its years; --split-date and s on the wizard date step restore the flat split (DateShape::Split); --split-date naming a non-date field and --axis naming a category the import did not build are refused with exit 1. context/code/formula-and-io.md, Import section.'). +work_done('improvise-d0n6',bugfix,'PR #322, merge d571abbd. Plan::spelling_clash refuses a raw unparsed date spelled like a generated month, quarter or year item, because the two would silently merge; asked at the wizard ConfigureDates step and again at build time so the headless import cannot reach it.'). +work_done('improvise-8jk6.6',feature,'PR #321, merge 7e879702. headless import --strict: with --no-wizard any skipped record is a refusal before the build and nothing is written; without it a partial skip writes the model and warns with a count. A folder whose CSVs are all header-only is refused as an import that kept no record. context/code/cli.md section 13.'). work_followup('improvise-8tk','improvise-lae','no-file-path UX: prompt for a path rather than only erroring. ExportPrompt is CSV-only, so a new SavePrompt TextEntry mode committing to SaveAs is the likely shape.'). work_followup('improvise-6sq','improvise-ct5','i on an empty records grid reports \'cannot drill\', leaking a pivot-mode concept into records mode'). work_followup('improvise-h0b','improvise-a2x','ViewState field types (AppMode, MinibufferConfig, ViewFrame, KeyCode, KeyModifiers) still derive no Serialize, and improvise-command depends only on serde_json. Deferred: nothing serializes ViewState until improvise-cqi builds a Snapshot, so the derives would have no consumer.'). @@ -573,6 +580,7 @@ decision('improvise-gtn7.1',scroll_measured_where_drawn,'A scroll offset is chos decision('improvise-cgqt',import_collisions_are_asked_not_defaulted,'PR #304, merge 4f38a35e. When records would write the same cell, the import wizard asks: ResolveDuplicates offers sum, keep separate under a new category, or go back, and appears only when ImportPipeline::collisions() is non-empty, after the field and date steps and before DefineFormulas. Silently summing was improvise-2cw9 fix for last-row-wins, and it too was a default the user never chose. The headless --no-wizard import takes the answer as --on-duplicate refuse|sum|separate. context/code/formula-and-io.md, Colliding records are a question, not a default.'). decision('improvise-cgqt',occurrence_category_goes_on_axis_none,'Under DuplicatePolicy::Separate the build adds the occurrence category with items 1..=deepest collision and sets it to Axis::None in the active view, so the default grid still shows each tuple total and nothing new is pivoted. A record that collides with nothing is occurrence 1 (improvise-cgqt, improvise-9bit).'). decision('improvise-felo.1',category_cap_has_one_home,'The regular-category cap is improvise_core::model::MAX_CATEGORIES and nothing else. Until felo.1 the import wizard kept a private const of its own, so raising the model number alone would have left the wizard refusing a 13th category the model accepts -- a derivation stated twice that agreed only by coincidence. The only place the literal 24 is spelled is the test add_category_max_limit; every other test that fills to the cap derives it. No recorded reason for 12 was found; the user raised it on 2026-09-25 (12 categories is feeling small, we should increase to 24).'). +decision('improvise-felo.13',skip_rebuild_keyed_on_observed_inputs_not_effect_kind,'App::refresh_layout skips the rebuild by comparing what the layout was built from (layout_stale, the drawn view name, the drill snapshot pointer) against the present, never by a list of effect kinds that need a rebuild: a table of kinds would be a second description of which effects reshape the grid, free to disagree with the first, the shape cursor_epoch and document_generation already avoid (repo-map gotchas 24 and 39). The cost is that a writer outside effects must call rebuild_layout itself (gotcha 23).'). wire_carries(command,both_directions,'user intent up, projection commands down'). wire_carries(snapshot,on_connect_only,'improvise-q08: full ViewState + render cache filled from current viewport'). wire_carries(command,both_directions,'user intent client->server, projections server->client').