docs(kb): gotcha 40 was stale in three sentences (improvise-ilvl)
CI / Build all bundles (push) Has been cancelled
CI / Test and lint (host) (push) Has been cancelled
CI / wasm32 check (browser bundle crates) (push) Has been cancelled
CI / Browser JS unit tests (node --test) (push) Has been cancelled
CI / Knowledge base consistency (push) Has been cancelled
CI / Lint the harness crate (push) Has been cancelled
CI / Browser harness (OPFS contract + shell) (push) Has been cancelled
CI / Deploy to GitHub Pages (push) Has been cancelled

The replay audit found `layout_version` documented as bumping "per
`ModelChange::LayoutGeometry` effect". It does not: `observe_reshape` diffs
`view_shapes` before and after the batch and bumps only where a view's shape
actually moved, so an idempotent `set-axis` bumps nothing. The source comment
at `apply_effects` has said so since improvise-pdvm; the repo map was the
stale copy, and it said the opposite in three sibling sentences — the
parenthetical, the "both tiers are read off the `EffectKind`" claim, and the
"bumps inside the loop" claim, which is also false (it runs once, after).

The observed form subsumes the gotcha 21 argument rather than restating it:
an aborted batch moved no shape, so there is nothing to not-claim.

Also files improvise-6uva (P1) and improvise-up8g, and records the audit on
improvise-ilvl. kb_consistent: TRUE, 783 facts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-22 18:27:02 -07:00
parent 3be1653385
commit db6593b1b2
3 changed files with 10 additions and 4 deletions
+5 -3
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -221,7 +221,7 @@ Region=West, Measure=Revenue = |pending| # pipe-quoted text
39. **`Applied` is an enum because a document swap is not a diff.** `Sessions::apply_checked` returns `Applied::Diff { dirty, broadcast }`, `Applied::Rebased { snapshots }` or — since ADR 001 layer 1 — `Applied::Rejected { reason }`, and the enum *is* the mechanism: a host that wants the `Dirty` must match, and matching names the case where a `Snapshot` is owed. The server's mirrors self-heal — `ClientSession::project` reseeds on the `document_generation` bump — but `ViewState::cursors` lives on the **client**, keyed by the outgoing document's view names (gotcha 24), and only a fresh `Snapshot` replaces it. `Command::Model` put the three `WholeWorkbook` mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. `apply_model` therefore calls `rebase` itself rather than instructing the host to, exactly as `apply_switch` adopts the accepted view itself. **The trigger is the generation moving, not the command variant**: a `LoadModel` that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a *failed* load — `a_load_that_fails_is_not_a_rebase` guards that arm. A `rebase: bool` beside `dirty` was rejected, because the host that forgets to call `rebase` is the host that forgets to read the flag. The **third** case is a *normal outcome*, not an error, and it is in the enum for the same reason the other two are: a host that wants the `Dirty` must match, and matching names it. `ServerMsg::Rejected` is the matching frame, addressed to **the asker alone** — nothing happened, so nobody else is owed anything — and the client keeps it as a `Rejection` rather than a sentence, because force needs to know *which* record and *which* kind (gotcha 43).
**Fixture warning.** When testing a document swap, the incoming document's active view must have the **same name** as the outgoing one. A rename makes `apply_model` broadcast a `SwitchView`; the client folds it, and the stale cursor is then read under a key nothing has written — right *by accident*, with the bug invisible. `other_document()` renames, and a first version of this test passed because of it; `a_document_shaped_differently_under_the_same_view_name()` keeps the name and changes only the shape, so there is nothing to hide behind.
40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per `ModelChange::LayoutGeometry` effect) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. Both tiers are read in `apply_effects` off the same `EffectKind` the `Dirty` is: one classification, two readers, no second table to drift from gotcha 35's. `layout_version` bumps **inside** the loop for the reason `Dirty` accumulates there — an aborted batch must not claim effects it never ran (gotcha 21). `record_versions` **cannot**, because an `EffectKind` says *that* a batch writes cells, never *which*; so the records are **observed**, via a per-record XOR fingerprint of the raw data taken when the batch declares a `CellData` effect and diffed after it. Three consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run.
40. **A batch's basis is three tiers, and only two of them can be read off the effect.** `App` carries `document_generation` (bumped by `install_workbook`), `layout_version` (per *view shape*) and `record_versions` (per *record*) — ADR 001's stale-basis check, and shared document state, so never on `ViewState` (gotcha 30). **The unit is the record**: `record_of(key)` is the coordinate tuple minus `_Measure` (`record_row_identity`), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. **Only the `Dirty` is read off the `EffectKind`; both lower tiers are *observed* around the loop** — the source says so at `apply_effects` and this paragraph said the opposite until improvise-ilvl's audit. The `EffectKind` decides only *whether to look*: `touches_model` arms the shape snapshot, `writes_cells` the record fingerprints. `record_versions` **must** be observed, because an `EffectKind` says *that* a batch writes cells, never *which*; `layout_version` is observed for a better reason than necessity — `observe_reshape` diffs `view_shapes` before and after and bumps only where a view's shape actually moved (improvise-pdvm/nynk), so it is a **fact about the document** rather than a claim about the batch. Two consequences the declared form did not have. An **idempotent** `set-axis` bumps nothing, and batching two real changes into one line or two gives the same `2`. And the gotcha 21 argument is subsumed rather than restated: an aborted batch moved no shape, so there is nothing to not-claim — which is why observing once **after** the loop is correct where accumulating `Dirty` inside it is. Note the arming is deliberately wider than the tier: a `CellData`-only batch that prunes a row out of a view **does** bump `layout_version`, because the shape moved. Three further consequences. A write landing the value already there **bumps nothing**, which is correct — no basis was invalidated. A `LayoutGeometry` effect that drops cells **will** bump their records when it shares a batch with a `CellData` one: an over-bump costing at most an unnecessary rejection, the same direction `Dirty::of` is deliberately sloppy in. And only *raw* data is fingerprinted — formula results live in `Model::formula_cache`, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a **set**: chaining `before.keys()` with `after.keys()` counts any rewritten record twice, a bug that survived one test run.
41. **`Storage` is not a VFS, and that closes the SQLite question.** The trait is whole-document `read`/`write` bytes (gotcha 2, `invariant(storage_is_two_ops)`) — no offsets, seeking, handles or locking — so an `sqlite3_vfs`, which needs `xRead(offset,len)`/`xWrite`/`xTruncate`/`xLock`, has nothing to bind to. SQLite *can* still sit behind it, as an in-memory database serialized to a blob (`rusqlite` `serialize`/`deserialize_read_exact`, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is **22% larger** than `.improv`, **4.7× larger gzipped**, ~**160×** larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. **A second serialization format is a second implementation of the grammar**, obliged to track `improv.pest` forever; and **its own round-trip test cannot detect drift from `.improv`**, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way `export_sqlite` beside `export_csv`, native-only and feature-gated: `libsqlite3-sys`' C build fails under this repo's nix cc-wrapper for `wasm32-unknown-unknown`, which would break the `wasm` CI gate the day it landed.
+4
View File
@@ -135,6 +135,7 @@ invariant(a_hidden_toast_keeps_its_text,'The toast element is the record of what
invariant(headless_cmd_and_script_are_one_function,'run_headless_script DELEGATES to run_headless_commands, so \`improvise cmd\` and \`improvise script\` cannot diverge in what they print, what they exit with, or when they write the document back. Asserted by a_script_reports_exactly_as_cmd_does rather than left to the delegation, because a future caller could inline it.').
invariant(the_creating_columns_insert_and_so_suppress_the_advance,'A +cat or +measure commit that NAMES something inserts a column at the cursor own index, so an advance computed from the PRE-commit layout is wrong for both -- CursorState::col_count is stale because apply_effects rebuilds only after the batch (gotcha 23). inserts_a_column covers both creating columns and takes the committed VALUE, because an empty name creates nothing and is genuinely at the right edge. created_name is the one home of that rule, read by add_measure, add_category and the predicate.').
invariant(a_reported_share_cannot_overflow_its_scale,'gp_tree::fit::importance_ranking sums and divides in f64 and narrows only the resulting share, which is bounded by 100 and therefore always finite (improvise-k844). TWO overflows, not one: 100.0*imp[j] associates left so a lone importance over f32::MAX/100 is inf before the divide that would have returned it to range; and the f32 TOTAL itself overflows when several are large, which reassociating to 100.0*(imp[j]/total) turns from NaN% into a SILENT 0% FOR EVERY FEATURE. f64 closes both, since no sum of finite f32s can overflow it. The residual guard answers only an importance that ARRIVES non-finite -- an infinite total apportions nothing, so the ranking is empty rather than NaN, and note NaN <= 0.0 is false so the pre-existing early return passed NaN straight through. Reachable PAST improvise-pv3g because both callers score importance in the target OWN units, making it an MSE difference that grows as the target square.').
invariant(viewport_extent_is_external_on_both_front_ends,'The viewport extent is an EXTERNAL INPUT that reaches the document, and it is not a browser problem. run_headless_commands reads crossterm::terminal::size() into app.term_{width,height} and CmdContext::{visible_rows,visible_cols} derive from it: 80x24 gives visible_rows 16, 200x60 gives 52. Measured on a 40-row grid, the SAME command sequence -- one PageDown then clear-cell at the cursor -- deleted R12 at 80x24 and R39 at 200x60. So Client::set_viewport is not a browser-introduced input; it is the browser SPELLING of one the TUI has always had, and gotcha 47 \"not derivable\" is true of both front ends. Any ADR treating extent as a browser problem produces a log that replays in a browser and not in a terminal. Cheapest of the five external inputs to close -- one pair of integers with an existing wire spelling, ClientFrame::Viewport -- but it must land in the SAME CHANGE as improvise-6uva, which currently masks it. A test must drive TWO sizes; one cannot see this.').
doctrine(bug_first,'write a failing test demonstrating the bug before fixing it').
doctrine(unit_over_e2e,'prefer small unit tests over integration/e2e').
doctrine(suite_speed,'Two numbers, because the ambiguity between them is what let this drift: NO SINGLE TEST BINARY may exceed 2 seconds, and the WORKSPACE WALL CLOCK is the sum of them. Measured at 22d43b2: 1309 tests over 33 binaries, ~2.2s wall, longest single binary 1.42s -- so the workspace is already past 2s and the rule that bites is the per-binary one. Re-MEASURE rather than re-assert: cargo test --workspace prints a \"finished in\" per binary. Excludes compilation, and excludes harness/ (Playwright, minutes, its own CI job).').
@@ -362,6 +363,8 @@ doctrine(never_hide_the_overflow_you_are_asserting_the_absence_of,'improvise-tg2
doctrine(a_deferral_cost_must_be_measured_not_named,'improvise-k844 sat open because context/code/gp.md section 7 deferred it on the claim that reordering the arithmetic \"moves the bit patterns misclass_rate_and_importance_are_bit_for_bit_what_they_were pins\". That test pins permutation_importance RAW SCORES; importance_ranking consumes them AFTERWARDS, so no path it witnesses runs through the share -- and scoring_known_answers.rs is byte-identical across the actual fix, verified by the orchestrator as absent from the diff. A COST PREDICTED FROM A TEST NAME RATHER THAN FROM WHAT IT CALLS is how a one-line fix stays filed, and it reads as diligence, which is why it survives review. Measure the claimed cost or state none. Pairs with an_invariant_is_made_unrepresentable_not_validated: the clamp improvise-pv3g added instead was a call-site check that left inf% in the data behind it.').
doctrine(a_reported_number_can_overflow_while_every_input_is_finite,'The gp crates learned this twice. standardize sums (t-mean)^2 in f32 (improvise-pv3g) and importance_ranking scaled a share by 100.0* in f32 (improvise-k844), and the second was REACHABLE PAST THE FIRST. Both are fixed by making the arithmetic incapable of it -- a Scaling that refuses a non-finite part, an f64 accumulator -- never by a call-site check, and pv3g own f32 clamp WAS that call-site check: it stopped the abort and left inf% in the data behind it. The silent twin is the half to look for: pv3g had cv_regress pooling a fold it could not standardize, k844 had an f32 total going inf with no panic to announce it. Ask what ELSE consumes the derived quantity.').
doctrine(a_dev_shell_is_a_third_environment,'.envrc puts nix bash 5.3 on PATH -- the RIGHT INTERPRETER, CI semantics -- and pkgs.mkShell ALSO exports about thirty generically-named lowercase variables the runner does not have: name, out, outputs, system, shell, builder, stdenv, phases, buildInputs, doCheck and more. A script reading one by accident under set -u is SILENT IN THE DEV SHELL AND FATAL IN CI. improvise-f9qf: pr-reviews-test.sh passed 19/19 under macOS bash 3.2 AND 19/19 under the dev shell bash 5.3, while failing 2/19 in CI -- two different reasons for green, one of them the correct interpreter, and unsetting \`name\` alone reproduced CI 2/17 exactly. Note nix shell does NOT export name and nix develop DOES (verified with env -u name as the control), so the dev shell is the one nix entry point that masks this class. The orchestrator own verification of this was itself confounded by inheriting name from a direnv-loaded shell -- it answered \"does name end up set\" where the question is \"does nix shell set it\".').
doctrine(an_observation_is_a_function_of_the_trajectory_not_the_state,'improvise-ilvl claimed \"an observation is a pure function of state\". It is not -- layout_version, cursor_epoch and model_state.dirty all differ between two routes to an IDENTICAL document (0/0/false against 2/2/true). The weaker true claim, that they are reproducible FROM THE LOG, is all replay needs. But the stronger false one is exactly what would justify SNAPSHOT-PLUS-SUFFIX LOG COMPACTION, and that is wrong silently in the direction that looks like it works: Basis puts document_generation and layout_version ON THE WIRE and clients compare them for EQUALITY, so a compacted replay is not basis-compatible with any client holding a claim, and dirty is the worker OPFS persist trigger so the same document reached by a different route persists a different number of times.').
doctrine(a_determinism_check_can_pass_because_the_divergent_path_is_unreachable,'Three identical replays of a 26-line script fixture proved the script path deterministic and proved NOTHING, because the fixture cursor moves were silent no-ops (improvise-6uva) -- the divergence had no way to occur. The defect surfaced only when clear-cell FAILED TO PARSE for want of coordinates. Sharper form of \"a green suite is not evidence\": a passing determinism or idempotence check must be shown to be capable of FAILING, by driving the input the check exists to discriminate. It is also how one defect MASKS another -- improvise-up8g viewport hazard is invisible in improvise script precisely because 6uva discards the cursor, so fixing 6uva makes up8g live and they must be closed together.').
work_done('improvise-8tk',bug_fix,'ZZ/:wq quit even when the save failed, silently discarding the model. Save now aborts the effect batch on failure; save_to_current_path/1 returns Result<String,String>; three tests in ui/effect.rs.').
work_done('improvise-907',bug_fix,'Import wizard step machine redesign: validation_error() gate at ReviewProposals (no-category and over-limit), history-stack previous() bound to Left on every step, and Done made a real confirmation step holding the built workbook. Superseded improvise-gfq, improvise-o1c, improvise-c6j.').
work_done('improvise-6sq',wont_do,'Merging EditOrDrill and EnterEditAtCursorCmd into one EnterEdit{mode} is not safe: they differ in WHEN the cell value is read, not in a mode parameter. Added ui/app.rs::tests::i_and_o_diverge_on_an_empty_records_grid as executable evidence.').
@@ -511,6 +514,7 @@ decision('improvise-nxah',a_refused_frame_is_rolled_back_by_the_fold_not_a_cache
decision('improvise-tg27',a_reserve_is_a_floor_and_must_reflow_not_widen,'header{flex-wrap:wrap} over a @media breakpoint: a flex item wraps exactly when the line cannot hold it, so the desktop bar is untouched BY CONSTRUCTION rather than by a chosen number -- no magic width to drift. Plus main{min-width:0}, 40vw caps on the side panels, and max-width caps so a reserve cannot exceed its own line. The reflow threshold is deliberately asserted NOWHERE, because #status composes the DOCUMENT NAME: the same 1024px window is one line under a short name and two under a long one. Only 1280 is pinned, being where improvise-n6t8 stability spec runs.').
decision('improvise-f9qf',one_checker_entry_point,'scripts/checkers.sh is the ONE entry point for every check_the_checker, called verbatim by CI and by the local gate so there is no second invocation to drift. It pins a bash >= 5 (ambient first, nix only when the ambient one is older, exit 3 when neither), SCRUBS THE ENVIRONMENT to an allowlist (env -i plus PATH/HOME/TMPDIR/NIX_*/GITHUB_*, extensible via CHECKERS_KEEP), and refuses any shell shebang in scripts/ that is not #!/usr/bin/env bash. The pinned bash goes on the FRONT of PATH so scripts the suite invokes get it too -- which is where the fault actually lived. CI logs \"checkers: bash 5.2 at /usr/bin/bash\": ambient, no nix, zero startup.').
decision('improvise-f9qf',nix_shebang_declined_on_availability_not_speed,'Measured warm, 20 invocations: ambient bash -c true 5.1ms; nix shell nixpkgs#bash --command 61.8ms; nix-shell -p bash --run, THE MANPAGE FORM THE USER POINTED AT, 675.2ms -- 132x, and that inside git merge machinery for beads-merge-driver.sh. But the DECIDER is availability: nix is installed on ONLY the kb job. build, test, wasm, js, harness-lint, browser and pages-deploy have none, so a nix shebang needs install-nix-action added to three more jobs, paid every run, to reach a bash those runners already ship. zsh declined too -- not a shebang change (word splitting, 1-based arrays, globbing), sixteen scripts to read, and no zsh on the runner either.').
decision('improvise-6uva',script_replay_lost_its_context_at_35946af,'improvise script and improvise cmd resolve every command through CmdRegistry::parse -- the PURE arm, which builds CursorState::default() (0,0,0,0) -- never the interactive arm that builds CursorState::from_ctx. So every Move command and enter-advance snapshots a ZEROED cursor AT PARSE TIME, clamps to (0,0) and emits SetSelected(0,0): worse than a no-op, it RESETS the cursor, and no cursor-reading command is reachable from a script. Broken by 35946af (2026-04-04), a performance refactor introducing the register/register_pure/register_nullary split, FIVE MONTHS BEFORE the first wasm-client commit -- so the web refactor is innocent. Both directions demonstrated: at the parent 649d80c a move-then-clear cleared row 2 and a bare clear cleared row 0; at HEAD both clear row 0. THE SHAPE: a registry with two construction arms silently demotes every caller of the weaker arm, and nothing type-checks which arm a caller deserves.').
wire_carries(command,both_directions,'user intent up, projection commands down').
wire_carries(snapshot,on_connect_only,'improvise-q08: full ViewState + render cache filled from current viewport').
wire_carries(command,both_directions,'user intent client->server, projections server->client').