diff --git a/context/code/doc-site.md b/context/code/doc-site.md index 2304f3cf..92f8965d 100644 --- a/context/code/doc-site.md +++ b/context/code/doc-site.md @@ -35,8 +35,14 @@ Not published: cloud session — tracker hydration, `bd` pitfalls, what the sandbox blocks. It is true and it is useful and it is addressed to nobody who visits a website. +- **`context/code/pr-review.md`** is how an agent reads a PR's reviews — where + a ChatGPT Codex review appears, how to tell it from this repo's own agents' + comments, and why absence is not approval. It is about *this repository's + process*, not about the program, which is the `cloud-agents.md` line and not + the `tooling.md` one: `tooling.md` explains checkers a reader of the code + will meet, and nobody visiting a website is about to read a PR thread. -Neither is referenced by any published document except as a row in +None of the three is referenced by any published document except as a row in `repo-map.md`'s file inventory, which is inside a code fence, so leaving them out creates no dangling link. That was checked rather than assumed. diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index 67dd049c..1a847d4e 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -98,6 +98,7 @@ for (const s of SECTIONS) for (const [src, slug] of s.docs) PAGE_BY_SOURCE.set(s const OMITTED = new Map([ ["context/plan.md", "a prompt, not a plan; cites a context/SPEC.md that no longer exists"], ["context/cloud-agents.md", "operational notes for agents, not about the program"], + ["context/code/pr-review.md", "how an agent reads a PR's reviews; about this repo's process, not about the program"], ]); // Derived, because the manifest only ever grows and a doc added without a line diff --git a/site/docs/adr/001-concurrent-edits.html b/site/docs/adr/001-concurrent-edits.html new file mode 100644 index 00000000..7d1f4823 --- /dev/null +++ b/site/docs/adr/001-concurrent-edits.html @@ -0,0 +1,99 @@ + + + + + +ADR 001 — Concurrent edits — improvise docs + + + +
+ +
+

Source: context/adr/001-concurrent-edits.md

On this page

ADR 001 — Concurrent edits between clients

+

Status: accepted (2026-09-19); layer 1 implemented (improvise-xwi6/fkfg, 2026-09-19) · Supersedes: nothing · Decided by: the user

+

#Context

+

improvise-cqq made the client/server split real: one authoritative App, a ClientSession mirror per client, and projections diffed against those mirrors. improvise-kmgs then put the 28 model mutations on the wire. Two clients can now edit the same document, and nothing decides what happens when they edit the same part of it.

+

#There is already a total order

+

Every ModelCommand reaches the document through one path: Sessions::apply → apply_command → App::apply_effects, against a single App. So the server already serialises writes. We do not need to build linear history — it exists.

+

What we lack is a way to notice that a client composed an edit against a picture of the document that has since moved. This is a stale basis problem, not an ordering problem — the same shape as the stale cursor (gotcha 24) and the stale session mirror after a document swap (gotcha 39), one level up.

+

#Scope: conflicts range over one document

+

improvise is a multi-document app, and the server is expected to host several separate models. Sessions editing independent models are trivially non-conflicting, so everything below scopes to a single document.

+

That falls out of the existing shape rather than needing to be built: Sessions does not own the App — connect, apply and project_all all take it as a parameter — so a Sessions is already bound to one document's generation sequence, and multi-document hosting is a registry above it mapping a document to its (App, Sessions) pair. The version tiers this ADR introduces live on the App, so they scope per document for free.

+

Multi-document hosting landed as improvise-ws-server::registry (improvise-j31x): a Registry of Hosts keyed by DocumentId, addressed by ClientId { document, session } because a SessionId is minted per Sessions and so is unique only within its document. The version tiers scope per document as predicted. Note one lifecycle rule that exists to protect this ADR: a document is never evicted while the server runs, because document_generation and record_versions start at zero in a fresh App, so re-opening would accept a reconnecting client's stale basis as current.

+

#What actually conflicts

+
    +
  1. Two writers, same record. Survivable under last-write-wins: someone's number is lost, the document stays coherent.
  2. +
  3. Shape change vs. cell edit. A removes category Region while B commits an edit keyed by Region=East. B's CellKey names a coordinate that no longer exists. This is corruption, not a lost keystroke, and it is the class a cell-level lock does not cover — holding Region=East × Revenue does not stop anyone deleting Region.
  4. +
  5. Formula vs. data. A adds a formula targeting the measure B is typing into; B's raw value is shadowed by a derived one.
  6. +
+

#Machinery that already exists

+ +

#Machinery that did not exist, and the hole it left

+

Nothing in the projection stream carried any of the three tiers. This ADR says a batch carries the basis it was composed against and never says how a client comes to know one — so the check as originally specified was satisfiable only by a client forging a basis. That is the invariant deleted with extra steps, arriving through the front door rather than as a force flag.

+

Closed during implementation (improvise-fkfg) by Projection::Basis + BasisCache: the server publishes the two document tiers plus the write counters for the records this client's grid names — exactly the set it can address a write to, per ADR 002 — diffed against what that client was last told and seeded in the bootstrap. It is emitted after the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown. That is the mirror image of gotcha 33.

+

Read this as step (a′) of the implementation order below: publication comes before the check, because the check is unsatisfiable without it.

+

#Decision

+

Record-level atomicity with optimistic concurrency per batch.

+
    +
  1. The unit of atomicity is the record, i.e. the coordinate tuple excluding _Measure — the thing record_row_identity already names. Not the cell: gotcha 1 says editing a category column in records mode moves every measure cell in the row, so a cell-level unit would be violated by a single legal edit. The unit must be at least as coarse as the coarsest legal write.
  2. +
  3. The unit of commit is the batch — one Vec<Box<dyn Effect>> applied by one apply_effects. This is already atomic in practice: a Binding::Sequence flattens into one batch, and a failed Save aborts the rest (gotcha 21).
  4. +
  5. A batch carries the basis it was composed against, and the server rejects it if that basis moved. The basis is tiered by ModelChange, which is what makes class 2 fall out for free:

    +
    tierbumped bya batch is stale if
    document_generationWholeWorkbookit moved at all
    layout_versionLayoutGeometryit moved at all
    record_versionCellData on that recordany record the batch touches moved
    +

    A CellData batch is stale if layout moved, because the shape its keys hang off changed. That is class 2, detected without a lock.

  6. +
  7. Rejection is a normal outcome, not an error. The client already receives the new state through projections; it re-composes against the new basis. Applied grows a third case alongside Diff and Rebased — matching is what forces a host to handle it (gotcha 39's reasoning).
  8. +
+

#Rejection must be forceable, and force means re-compose

+

A rejected edit has to give the user a way through. But force is a re-composition, not a bypass — it never applies a batch whose basis is stale. A mechanism whose only purpose is to violate the invariant is not a feature, it is the invariant deleted with extra steps. Force re-composes the same intent against the current state and submits a fresh batch with a current basis.

+

That distinction is not pedantic, because what force can mean differs by class:

+
classcan force?what force does
1 — two writers, same recordyes, triviallyre-compose and apply; the user has seen their value and chooses to overwrite it
2 — shape change vs. cell editnothe intent no longer type-checks: the key names a coordinate that no longer exists
3 — formula shadows a valueyes, with a warningthe write lands; the displayed value is still derived
4 — the batch names a record its basis says nothing aboutno — and it is not a racenothing to re-compose against; the client was never in the running
+

Class 4 was not in the original draft and was added by the implementation (Rejection::RecordUnstated). It exists because "any record the batch touches" had to name whose list, and the answer is the server's: reading it off the client's claim would be a force by omission — a client that simply omits a record skips the record tier, with no flag to remove and nothing in the message saying so. reduce.rs::records_named is therefore an exhaustive read of the ModelCommand, with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. A well-formed client cannot produce class 4; it is a case of its own rather than folded into class 1 because "you lost a race" and "you were never in one" are different things to tell a user.

+

For class 1, force is one keystroke and should be. For class 2 there is nothing coherent to force: applying the batch anyway writes a CellKey naming a deleted category, which is the ghost-coordinate corruption this ADR exists to prevent. So:

+ +

Two consequences that pull work forward from layer 2:

+ +

#Optional second layer: advisory reservations

+

The staged-edit mechanism from drill, generalised: a client publishes the keys it has staged so others can grey them out and warn early. The values stay private, per improvise-tom1.

+

This layer is advisory only. If a reservation is lost, never arrives, or leaks from a crashed client, the system degrades to "you find out at commit" — never to corruption. That is what makes it safe to be sloppy about timeouts and disconnects, and it is why correctness must not be built on it.

+

#Why not a causal clock

+

Considered and rejected, for four reasons.

+
    +
  1. The server already serialises. Causal clocks earn their keep where no single node orders writes — peer-to-peer, or offline-capable clients merging on reconnect. With an authoritative App we need staleness detection (one counter per unit), not concurrency detection (a vector per client).
  2. +
  3. A Lamport clock is insufficient. It gives a total order but cannot tell you two edits were concurrent, which is exactly the question.
  4. +
  5. A vector clock is sufficient but does not answer the question that matters. It costs O(clients), needs entries reaped for departed clients, and when it reports "these were concurrent" you still need a policy for class 2 — no clock says whether B's edit into Region=East should survive A's deletion of Region. The clock tells you there is a decision to make; it does not make it.
  6. +
  7. CRDTs converge, but not necessarily to something meaningful. A LWW-register per cell is easy; the hard part is the shape — categories, items, formulas, axis assignments — where convergence can mean an orphaned cell or a resurrected category.
  8. +
+

If offline editing ever becomes a goal, this decision should be revisited: that is the requirement that makes a vector clock worth its cost, and it is a much larger commitment than multi-client-while-connected.

+

#Consequences

+

Good. No lock to acquire, so no timeout, no lease renewal, no disconnect handling, and no liveness risk — a crashed client cannot wedge anyone. Correctness does not depend on the advisory layer. The tiered basis reuses ModelChange, so there is no second classification to drift from gotcha 35's table. Class 2 is detected rather than prevented, which is cheaper and does not require predicting which shape changes matter.

+

Bad. A client can compose an edit and be told no, and the UI owes real work there: keep the typing, show what moved, and offer force where force is coherent. Class 2 is the expensive case — the honest answer is a small dialogue about what changed, not a button. Record-level granularity means two clients editing different measures of the same record serialise against each other even though their writes do not overlap — acceptable at the expected scale, and refinable to (record, measure) if contention shows up. A busy LayoutGeometry editor invalidates every in-flight CellData batch, which is correct but could feel aggressive.

+

Not solved here. Class 3 (formula shadows a raw value) is detected — the formula bumps layout_version — but the policy is unstated: the edit is rejected, and whether that is the right answer for the user is a separate question. Nor does this address a client that wants to keep editing while disconnected.

+

#Implementation order

+

Deliberately two tickets, so the correctness floor lands without waiting on the UX.

+
    +
  1. Basis check. Version tiers, ModelCommand carries its basis, Applied grows the rejected case. Includes the decision of where the version counters live — they are shared document state, so they belong beside document_generation on the authoritative App, not on ViewState.
  2. +
  3. Advisory reservations. Generalise drill staging; publish key sets, keep values private.
  4. +
+

The nullary wire variants ApplyAndClearDrill and AcceptWizardModel were to be resolved first or alongside. Already done: improvise-3x3u removed both from ModelCommand entirely, so they have no wire form to carry a basis or a SessionId, and layer 1 landed honestly without them (improvise-k2i4 closed as stale). What survives is the addressed form multi-client drill will want, which is layer 2 work.

+

#What the check is not

+

It is an anti-staleness coordination check, not a security boundary. Basis derives Deserialize because it must cross the wire, and every field is public, so any value at all can arrive as a claim — versions start at 0 and increment by 1, so guessing one is trivial. Rejection::RecordUnstated closes omission — a client that leaves a record out of its claim — which is the bypass that would otherwise sit in the message format with nothing saying so. It does not close invention, and nothing here tries to: a hostile client that forges a current basis is indistinguishable from one that is genuinely caught up, and defending against that is authentication's job.

+

BasisCache::claim being the only constructor is therefore a statement about clients written against this crate, not about the wire. It is still worth having: it makes the honest path the easy one, and it is what makes "a client that has folded nothing composes nothing" true by construction.

+

#Two places the check deliberately does not reach

+

Command::SwitchView carries no basis. It is LayoutGeometry but sits outside Command::Model, so apply_switch is unchecked. It cannot corrupt — the workbook refuses a view it does not have, and every mirror takes what was accepted (gotcha 36) — but it is a hole in "a batch carries its basis" and should be decided rather than left as an accident.

+

apply_command does not check at all (nor did reduce_full, which improvise-s17o has since deleted for having no caller), and the principle is correct: the check is a property of the session door, not of the mutation path, because a host holding the App is the truth and not a claimant. That is also why ui::reduce::caught_up(&App, cmd) is not a bypass — it takes &App, so nothing reachable from a wire can call it.

+

As implemented it was a second door, not an exemption — both functions took a wire Command whose Command::Model { cmd, .. } arm discarded the basis with .., while ui/reduce.rs's module doc pointed every future host at them by name. Closed by improvise-ondj as layer-1 cleanup: both now take a Checked<'_>, whose only constructor runs the check against the authoritative App and returns a Rejection. reduce_full was deleted outright shortly after (improvise-s17o): having no caller was the condition that let the defect hide, and a Checked signature on a function nobody calls fixes the type and not that. The exemption stands — a host's own ModelCommand still carries no basis — but it is no longer reachable as a door, and the unchecked path is unconstructible rather than merely discouraged.

+
+
+ + diff --git a/site/docs/adr/002-naming-a-cell.html b/site/docs/adr/002-naming-a-cell.html new file mode 100644 index 00000000..0c04a8a1 --- /dev/null +++ b/site/docs/adr/002-naming-a-cell.html @@ -0,0 +1,82 @@ + + + + + +ADR 002 — Naming a cell — improvise docs + + + +
+ +
+

Source: context/adr/002-naming-a-cell.md

On this page

ADR 002 — How a client names a cell

+

Status: accepted (2026-09-19), extended (2026-09-20) · Decided by: the user · Blocks: improvise-1ey · Extended by: improvise-ir7l, improvise-poia

+

#Context

+

improvise-q08 established that a thin client cannot compose SetCell, and it is the gap that blocks the browser MVP, whose stated scope is "type a number into a cell and see the value update".

+

A client sees a grid of label strings indexed by viewport position. A CellKey is the full sorted coordinate tuple — every Row and Column category, plus every Page and Filter selection (gotcha 27). Nothing in GridCache says which category a row label belongs to; nested row axes flatten several categories into one label; page and filter constraints are not projected at all; and records mode encodes a column index in _Dim (gotcha 1). There is no client-side derivation, and projecting the coordinate space would make every client learn conventions the server exists to own.

+

#Decision

+

Every projected cell carries an opaque surrogate identity, and the kind of identity says what may be done to it.

+
pub struct CellDisplay {
+    pub text: String,
+    pub numeric: bool,
+    pub error: bool,
+    pub id: CellId,
+}
+
+pub enum CellId {
+    /// A stored cell. `SetCell` takes this and nothing else.
+    Record(RecordHash),
+    /// An aggregate over stored cells. `Drill` takes this and nothing else.
+    Aggregate(AggregateHash),
+}
+

An aggregate's surrogate is a hash of its coordinate constraint — the row and column coordinates plus the page and filter selections that define which stored cells it sums. One hash per cell, the same width and the same index machinery as a record's, and the server resolves it back and runs drill_records when the client asks.

+

Three things follow, and each is the reason to prefer this over alternatives.

+

1. Both operations are unrepresentable-if-wrong, not checked. A client cannot compose SetCell for an aggregate, and cannot compose Drill for a stored cell, because neither carries an id of the other's type. That is design-principles §3 — make invalid states unrepresentable — and it is strictly stronger than a server-side rejection, which would be a runtime answer to a question the type can settle.

+

An earlier draft used Option<CellId>, absence meaning "aggregate". The enum is better for a reason worth keeping: an aggregate is not the absence of an identity, it is a different kind of identity. Modelling it as None made SetCell safe and left Drill with nothing to address — which is precisely how drill addressing came to be an open question. Two variants close it by construction.

+

2. The client never learns the coordinate space. No category names, no page/filter constraints, no _Dim column-index convention. The surrogate is a token to echo, not a structure to understand — so gotchas 1 and 27 stay knowledge the server owns, and a second client implementation cannot get them subtly wrong.

+

3. It mirrors a fork that already exists. EditOrDrill (cmd/mode.rs) already branches on exactly this in the TUI: a stored cell is edited, an aggregate is drilled. The wire is not inventing a distinction, it is projecting one the model already makes — the same discipline as decision(improvise-cqi, command_set_derived_not_invented).

+

It also follows the precedent CellDisplay::numeric set, whose doc already says the server decides "because it is the side that knows which it was". Settability is another such fact.

+

#Decisions made along the way, and the traps in them

+

Most of what started as open questions here got answered. They are kept with their reasoning because each rejected alternative is a thing someone will propose again.

+

What the surrogate is — DECIDED: a hash of the normalized record. Not of the cell: the record is the coordinate tuple minus _Measure, which is already ADR 001's unit of atomicity and already what record_versions is keyed by. A cell is then (record hash, measure).

+

The reason is that it makes stale edits cheap to identify, and it does so three ways at once. The surrogate a client already holds is the lookup key for its own version, so record_versions becomes HashMap<RecordHash, u64> — fixed-size, no allocation, no CellKey reconstruction on the hot path. A basis is then a list of (record hash, version) pairs, which is compact on the wire and O(1) to check. And a record that no longer exists simply fails to resolve, so that whole class of staleness needs no version comparison at all. One identity serves addressing and staleness instead of each ADR carrying its own.

+

It is well-defined because CellKey already has a canonical form: CellKey::new sorts coordinates by category name and equality and hashing rely on it (gotcha 5). "Normalized" means that canonical sorted form with _Measure removed — the normalization is a property the type already guarantees, not a new convention to maintain.

+

An interned id was rejected: it would need a table with a lifetime nobody has specified, and it buys only collision-freedom, which width buys more cheaply.

+

Three ways this silently breaks, all avoidable. They are listed because each produces a wrong write rather than an error, which is the failure mode this ADR exists to prevent:

+ +

_Index was considered and rejected — right ontology, unsafe representation. It is the coordinate that identifies a record, and a record is exactly the thing that is settable rather than aggregated, so the instinct is sound. But CategoryKind::VirtualIndex is documented as "Items are "0", "1", … N where N = number of matching cells" — it is positional over the currently matching set, not an identity. So _Index = "3" names a different record after a filter change, an insert, or a delete, and names a different one again in another view. A client echoing a stale _Index back would write to the wrong cell, silently — strictly worse than a surrogate that fails to resolve, and the same hazard class as the viewport-relative-vs-absolute trap and the stale cursor under a renamed key.

+

What _Index is pointing at is worth keeping, though: the surrogate should name the record, not the cell. A record is the coordinate tuple minus _Measure (record_row_identity), which is already ADR 001's unit of atomicity and already what record_versions is keyed by. If a cell is addressed as (record surrogate, measure), then one identity serves both addressing and staleness, and the two ADRs share a mechanism instead of each inventing one.

+

How the server resolves a surrogate back — DECIDED: a maintained index, in DataStore. A hash is one-way, so resolution needs a reverse map, and regenerating it from the layout on demand would be O(visible cells) per resolve and would only ever cover the visible window — a client may legitimately hold a surrogate for a record that has scrolled out of view.

+

DataStore is the right home because it already does exactly this. It interns every category and item name into a SymbolTable, and it already maintains a secondary index HashMap<(Symbol, Symbol), HashSet<InternedKey>> for hot-path lookups. A RecordHash → record index is the same shape, updated at the same points, and the existing (Symbol, Symbol) index is what makes invalidation tractable: when a category or item disappears, the records that mention it are findable without a scan.

+

Two consequences worth stating, because they fall out and are easy to miss.

+

The index and record_versions want to be one structure. Both are keyed by the record hash and both are invalidated at the same moments, so RecordHash → (record, version) gives ADR 001's basis check and this ADR's resolution in a single lookup.

+

Structural and data staleness separate cleanly, which is better than the regenerate-on-demand shape would have given:

+
incoming surrogatemeansanswer
does not resolvethe record is gone — a category or item it named was removedreject; there is nothing to write to
resolves, version matchesthe client's basis is currentapply
resolves, version differsthe record changed since the client read itreject as stale (ADR 001), offer force
+

Regenerating from the layout would have conflated the first two rows, because a record scrolled out of the viewport would have looked identical to a record that no longer exists.

+

Whether a forged or stale surrogate is an error or a rejection. Under the regenerate-from-layout shape it is simply unresolvable, which suggests the same Applied rejection path ADR 001 defines rather than a distinct error.

+

#Consequences

+

Good. Unblocks improvise-1ey. Keeps the client thin. Makes an invalid SetCell unconstructible rather than rejected. Composes with ADR 001 instead of duplicating it. Projects a distinction the model already makes, so there is one source of truth for what is settable.

+

Bad. CellDisplay grows a field, and it is on the hot path — every visible cell carries it, so the surrogate's width is a wire-size decision, not only a collision one. A client holding a surrogate across a layout change holds something that will fail; that is correct, but the UI must handle it as a normal outcome rather than an error (the same obligation ADR 001 already places on rejection).

+

Not addressed. The other composability blockers improvise-q08 listed — ToggleGroup, AddItemInGroup, RemoveFormula, LoadModel — which are separate projection gaps and are not solved by a cell surrogate.

+

#Extension (2026-09-20) — two more kinds, by the same reasoning

+

The decision above is unchanged. CellId has since grown two variants, both arrived at by applying clause 1 rather than departing from it: a cell whose permission differs gets a different kind of identity, so the wrong operation stays unconstructible instead of being refused at runtime.

+
pub enum CellId {
+    Record(RecordCell),
+    Aggregate(AggregateHash),
+    /// A cell a formula owns (improvise-ir7l).
+    Derived { target: String },
+    /// A records-mode creating column (improvise-poia).
+    Create(CreateKind),
+}
+

Derived — a formula target is a real stored coordinate, so the original two-variant enum called it a Record and every client offered an editor over it. The write landed, and was then shadowed forever by formula_cache. It carries the formula's target and no surrogate: there is nothing a client could spend on a SetCell, for the same reason Aggregate carries an AggregateHash.

+

Create — the records-mode + and +cat columns projected None, which this ADR defines as "no projected cell". That conflation is the one clause 1's rejected Option<CellId> draft was about, arriving from the other side: not an aggregate mistaken for an absence, but a column the user can type into mistaken for one. It carries only the kind, because what the user types into a creating column is a name, not a value — the operation is ModelCommand::AddItem / AddCategory, which already existed, so no new command was needed and the record under the cursor is not part of it.

+

What the original two clauses cost, and still cost, is unchanged: a client learns no coordinate space, and an invalid operation is unconstructible rather than rejected. What the extension adds is that the enum is now four answers to "what may be done here", so None finally means only what this ADR always said it meant — no projected cell at all.

+
+
+ + diff --git a/site/docs/adr/003-live-backed-models.html b/site/docs/adr/003-live-backed-models.html new file mode 100644 index 00000000..dcc68601 --- /dev/null +++ b/site/docs/adr/003-live-backed-models.html @@ -0,0 +1,51 @@ + + + + + +ADR 003 — Live-backed models — improvise docs + + + +
+ +
+

Source: context/adr/003-live-backed-models.md

On this page

ADR 003 — A model may be backed by a live query source

+

Status: accepted, not a priority (2026-09-19) · Decided by: the user · Supersedes: the read-only framing in improvise-rucf

+

The goal is to have the option of essentially using improvise as a DB client. Direction, not schedule: nothing here outranks the browser MVP (improvise-1ey), and the tickets are filed low. This is written down now so the option stays open — several decisions being made this week could foreclose it by accident.

+

#Context

+

improvise must handle arbitrary SQLite databases as models — a user points it at a database they already have and pivots over it.

+

Two readings were offered and the larger one was chosen. Import-once — read a query into a Model in RAM, exactly as CSV import does — is days of work behind an existing seam (csv_parser produces Vec<Value>, analyze_records classifies it). It was rejected because it makes the model a snapshot: source changes are invisible until re-import, and the data must fit in memory.

+

Live-backed was chosen: the model is a view over the source, and queries go to it as the user pivots.

+

This is not improvise-9ix. That ticket proposed SQLite as improvise's own persistence format and was declined on measurements — a second implementation of the pest grammar, 22% larger than .improv and 4.7× gzipped. Reading someone else's database has no round-trip obligation and no grammar duplication. What does carry over is the dependency constraint: improvise-io is in the wasm bundle, and libsqlite3-sys does not build for wasm32-unknown-unknown under this repo's nix cc-wrapper.

+

#What the model is today

+

Model holds categories, a DataStore, formulas, and views. DataStore interns every category and item name into a SymbolTable, stores IndexMap<InternedKey, CellValue>, and maintains a secondary index (Symbol, Symbol) → {InternedKey} for hot-path lookups. A Workbook loads entire — there is no partial-load path anywhere in the tree, a fact verified while declining improvise-9ix.

+

So this decision changes what Model is, not merely where its bytes come from.

+

#Decision

+

Model becomes a facade over a data source, of which the current in-RAM DataStore is one implementation and a SQL query engine is another.

+

Three things make this more tractable than it first looks, and they should shape the design rather than be discovered late.

+

1. Aggregation is already the interface, and it is already SQL. apply_agg computes Sum | Avg | Min | Max | Count over a matching cell set — those are exactly SQL's aggregates. The pushdown is not a translation layer to be invented; it is the operation the model already performs, expressed in the source's own language. GridLayout asks for a cell's value, not for the rows behind it.

+

2. Formula results are already separate from stored data. Evaluation is fixed-point (recompute_formulas iterates to stability, gotcha 9), and formula-derived values live in Model::formula_cache, not in DataStore. So the split is clean: raw data is queried, derived data stays in RAM. Formulas do not need to become SQL.

+

3. Clients are already insulated. The projection architecture means a client sees CellDisplays and never learns where a value came from — the server holds the App and therefore the connection. Live-backing and the browser MVP compose rather than compete, and a browser never needs SQLite at all.

+

#The consequence nobody asked for, and it is the important one

+

A live-backed document splits into our configuration and their data, and .improv does not currently distinguish them. Today a .improv file holds categories, views, formulas and a ## Data section. A live-backed model's categories and cell values are the source's; the views, formulas and axis assignments are improvise's.

+

So the format grows a source specification where a live model would otherwise have data — a connection, a query, and the column-to-field mapping. Categories become derived (SELECT DISTINCT) rather than declared. That is a grammar change, and improv.pest is the single source of truth, so it comes first (design-principles §3).

+

Read through the DB-client framing, that split stops being a complication and becomes the natural shape of the artifact: a live-backed .improv is a saved query plus a pivot configuration — which is precisely what a saved report is in any BI tool. The document stops being "my data" and becomes "my question", which is portable between databases in a way a data file never is, and shareable with someone who has the same source and not the same rows.

+

#Open questions, in the order they will bite

+

Writes — DECIDED: a live-backed model is read-only. Not a limitation accepted reluctantly: the user's framing is that for many use cases editing capabilities would be a downside. Pointing a pivot tool at a production database and being unable to write to it is a feature. So there is no write-through, no overlay, and no reconciliation.

+

Two things fall out, and both are simplifications rather than costs.

+

It dissolves the ADR 001 tension instead of requiring an amendment. ADR 001 governs concurrent edits; a read-only model has none, so the basis check simply does not apply to it. The guarantee is not voided, it is scoped to documents improvise owns — which is what it always meant. What remains is a smaller and different question: an external writer changes what clients see, so live-backed projections need a freshness story (polled, triggered, or explicitly stale-until-refreshed). That is projection freshness, not conflict resolution.

+

The machinery to express it already exists, built for something else. ADR 002 makes SetCell take a CellId::Record and nothing else, so a live-backed model that projects no Record ids makes a write unconstructible by the client — no runtime check, no error path, no "this model is read-only" dialog. The settability distinction built for aggregates carries read-only sources for free.

+

One design consequence for the source trait: writes should not be in the trait that a query source implements. If they were, a read-only source would have to fail at runtime for a method it structurally cannot serve. Keep reads (and aggregates) in the trait every source implements, and let writing be a capability only the owned in-RAM implementation has — so "read-only" is unrepresentable rather than enforced, the same discipline as CellId.

+

Interning and identity. DataStore interns names and ADR 002 hashes the coordinate tuple. Both still work — coordinates come from the query — but the RecordHash → record index that resolves a CellId becomes a cache with an invalidation story rather than a structure maintained alongside writes.

+

Which rows. A table name or a user-supplied SELECT. A query is strictly more useful (joins, filters, computed columns) and no harder to consume; a bare table name is sugar over it.

+

Type mapping. SQLite is dynamically typed per value, but columns carry declared types, which is better input than CSV's string sampling: detect_date_format currently guesses a date by pattern-matching text. Dates may be TEXT, INTEGER (epoch) or REAL (julian) — detect, do not assume. NULL is absence, not the string "null" (improvise-ych).

+

Where the boundary goes. improvise-core must not gain an IO dependency (forbidden(improvise_core, io)), so a query-backed source cannot simply be a DataStore variant inside core. The trait belongs in core and the SQLite implementation in improvise-io, feature-gated and native-only so the wasm gate keeps passing.

+

#Consequences

+

Good. Data larger than memory. A source that stays current. Aggregation pushed to an engine built for it. And the client/server split already built means none of this reaches a browser client.

+

Bad. It changes Model, which is the centre of the codebase — GridLayout, evaluation, persistence and the import pipeline all touch it. Latency enters a path that is currently a HashMap lookup, so the viewport-bounded projection stops being an optimisation and becomes load-bearing. And the .improv grammar grows a concept.

+

Worth doing first regardless of the eventual shape: the source trait and its in-RAM implementation, so that today's DataStore becomes an implementation rather than the implementation. That is a refactor with no behaviour change, characterization_survives_the_move applies, and it is the seam everything else needs — including, if it is ever wanted, the import-once path this ADR rejected as the only answer.

+
+
+ + diff --git a/site/docs/code/clients.html b/site/docs/code/clients.html new file mode 100644 index 00000000..af425470 --- /dev/null +++ b/site/docs/code/clients.html @@ -0,0 +1,362 @@ + + + + + +The clients — improvise docs + + + +
+ +
+

Source: context/code/clients.md

On this page

Clients: the browser client, the two hosts, and the harness

+

The arguments that used to live in the comment headers of improvise-wasm-client, improvise-ws-server, improvise-worker-server, web/ and harness/. Written for a reader with the code open. Bug narratives keyed by test name are in context/test-evidence/clients.md.

+

Repo-map gotchas 42, 44, 45, 46, 47 and 48 cover the same ground from the "where" side; this file is the "why", at the density the comments had.

+
+

#improvise-worker-server/src/worker.rs — the message loop

+

Worker is the whole browser host minus the two things a browser has to supply — self.onmessage/self.postMessage, and OPFS. Both are in crate::bindings, which is cfg(target_arch = "wasm32") and has no decisions in it. Everything that could be wrong about what the main thread is told is in worker.rs, where cargo test can drive it.

+

#It is the socket's Host, not a second one

+

Host is App + Sessions + handle(id, ClientFrame) -> Vec<Addressed>. Nothing in it is a socket, and improvise-9x6 is what made that load-bearing rather than tidy: the worker needs the same session loop and cannot depend on improvise-ws-server, which is tokio and tungstenite. So Host moved down to improvise_command::ui::host and both deployments name the one type. Re-implementing it here would have been a mirror, and improvise-xks6 had just finished deleting the last one: a second copy of a decision layer drifts, and the drift is invisible until a client sees it.

+

What the worker adds on top is only what a MessagePort differs in:

+ +

#ClientFrame::Key is answered with nothing, and that is not laziness

+

A worker holds the authoritative App, so ui::reduce::reduce_key really is callable here where it is not on a socket. Calling it would still be wrong, and the reason is not about transports at all: nothing in the projection stream carries the host's own ViewState. A client's view slice moves only by folding Commands, and a keypress resolved against the App moves the App's slice — its mode, its cursor, its minibuffer — with no Command produced that any client could fold. ClientSession::project emits document projections, a grid diff and a basis; none of the three is a mode change.

+

So a half-wired Key arm does not under-deliver, it diverges: the host would believe it is in AppMode::Editing while the client draws Normal, and ClientSession::viewport() reads the mirror's cursor, so the grid would not even scroll to follow.

+

improvise-bpq2 answered it, and not by deriving Commands from a ViewState delta. The user's ruling is that a keymap is UI and the server must not know it, so resolution moves to the client: a key the client can answer arrives here already resolved, as a ClientFrame::Commands batch it has folded itself, and this arm's emptiness stops being a gap and becomes the statement. What remains upstream is the set of bindings a client still cannot compose — prefix keys, anything naming a row's category or a group, anything touching Storage — and those are closed by enriching the projection, one at a time, not by wiring this arm up. The list lives in keys.rs (see below).

+

#Persistence is the worker's, whether or not anybody asks

+

There is no ModelCommand::Save, and that is not an oversight: effect::Save is EffectKind::ViewOnly (it writes the document to this host's file path and clears this host's dirty flag; it mutates no document state), so the derivation that produces ModelCommand from the model effects cannot reach it (decision(improvise-cqi, command_set_derived_not_invented)). Saving was therefore unreachable from any client on any transport until improvise-368h added ClientFrame::Save — a request to the host, framed rather than commanded precisely because it mutates nothing shared.

+

A browser still never sends it, and that is the right answer rather than a gap: there is no file the user owns and no Ctrl-S semantics to honour, so asking would be the wrong question — the document is the OPFS entry, and a page that loses an edit because nobody pressed a key has simply lost it. So the worker persists on its own behalf: Worker::persist runs the one existing Save effect through App::apply_effects, so effects remain the only mutation site and the save path is not duplicated. A worker with no file_path silently does nothing, which is what Save already does; a failed save leaves its message in the status and aborts the batch.

+

The trigger is an observation of the document, not a table of command variants. It used to be the latter, and the reason was honest: a wire ModelCommand::SetCell reached effect::SetCell, which did not raise model_state.dirty — that was a separate MarkDirty effect only the key path composed — so "save if dirty" was a trigger that never fired (improvise-gv9r). With dirtiness raised by App::apply_effects off the same EffectKind the Dirty is, the flag is the observation, and asking it is better than enumerating variants for the same reason the cursor reset watches cursor_epoch rather than matching on CycleAxis, and the rebase watches document_generation rather than matching on LoadModel: a table of variants is a second description of which commands change the document, free to disagree with the first. It is also strictly tighter — a refused batch changes nothing, so it raises nothing, and the "was anything rejected" special case that used to guard this went with it.

+

One cost remains and is stated rather than hidden: Storage is whole-document by construction, so OPFS rewrites the file per accepted model command — correct, not yet cheap. And a failed write leaves the flag raised, so the next frame retries: right for a transient failure, and harmless for a permanent one (a worker with no file_path) because the only thing a failed Save produces is a status line, and nothing in a worker renders one.

+

The one frame on_frame does not auto-persist after is ClientFrame::Save: a client that asked to save has already been answered, and a failed write leaves dirty set — so retrying would repeat the I/O and, on a transient failure, tell the client SaveFailed about a document that is now on disk. The automatic write exists for edits nobody asked to save.

+

So this is the browser's own answer, and ClientFrame::Save is the other one — for a socket client editing a TUI-hosted document, where there is a file somebody owns.

+

#Ordering

+

Worker::on_frame returns the frames for one inbound frame, in order, and a ServerFrame::Commands batch must reach the client whole and in sequence: a GridUpdate::Cells that overtakes the labels framing it is dropped, not replayed. That is a property of the fold, so a postMessage transport owes it exactly as a socket does — never coalesce two batches, never split one.

+

#Malformed frames are loud

+

Worker::on_message returns Err on a frame that will not parse rather than swallowing it. A worker that silently ignores a malformed frame leaves the client waiting forever for an answer that is never coming.

+

Worker is not Debug, for the reason Host is not: it owns the whole document.

+
+

#improvise-ws-server — the socket host

+

Three layers, and the split is the point — the frames are a fourth thing this crate speaks but does not own:

+ +

Native-only: tokio's networking does not build for wasm32-unknown-unknown, and it should not.

+

#The registry's grain

+

Sessions editing independent documents are trivially non-conflicting, which is why ADR 001's basis tiers — document_generation, layout_version, record_versions — live on the App and scope per document for free. A registry that let two documents share any of that would throw the free scoping away, so the registry owns no state of its own beyond the map: there is nothing here for two documents to share.

+

A SessionId is not unique server-wide. Sessions mints ids from a counter it owns, so every document's first client is SessionId(0). The server-wide address is therefore the pair, ClientId, and Registry::route is the one place a Host's SessionId acquires the document it came out of. That is the whole cross-document isolation argument, and it is structural rather than checked: a Host can only name its own sessions, and the only tag route can apply is the document whose Host produced the frames. In particular an Applied::Rebased fans a fresh Snapshot out to every client of that document and cannot reach further, because the ids it returns are meaningless anywhere else.

+

#Lifecycle

+

Open happens at startup, from the documents the host names. A client cannot ask the server to open a path it was not told about, and that is not an oversight: deciding which paths a client may open is access control, which improvise-j31x puts explicitly out of scope. Serving only what was named leaves the question unanswered rather than answering it badly. A stem collision is fatal rather than resolved: two files with the same stem are two documents a client could not tell apart, and silently serving one is worse than saying so.

+

Close does not happen when the last session leaves. A document stays resident for the life of the server. Evicting it was considered and rejected twice over:

+ +

An idle document therefore costs its App, and Registry::is_idle exists so a future eviction policy has somewhere to ask the question from — nothing calls it to evict, because nothing evicts.

+

#The transport's two rules

+

A client names its document in the URL (improvise-j31x): ws://host:port/quarterly asks for quarterly, and ws://host:port/ asks for whichever one there is when there is exactly one. The path was chosen over a new ClientFrame variant for two reasons: a document is a resource, which is what a URL path is for, and ClientFrame stays exactly as it was, so every client written against the single-document server keeps working unchanged. Which document a name resolves to is Registry::resolve's decision, not the transport's; all that happens there is path -> Option<&str>. It is not a path walker — everything after the leading slashes is the name.

+

The other rule is that the first frame of a connection reports the extent, because the bootstrap Snapshot cannot be built without it.

+

A refused connection is closed with a websocket close frame carrying the reason. No ServerFrame::Error was added: the frame vocabulary is derived from the mutations that need it, and "this connection is not happening" is a fact about the connection, which the protocol already has a frame for.

+

A Registry is not Send (a Host's App owns a Box<dyn Storage>), and must not be shared anyway: it holds the authoritative documents, and one owner is the point. So it sits in a single task on a LocalSet and every connection talks to it over a channel. That is also what serializes the clients — commands arrive in one order, and the projections each client gets are produced in that order, which matters because a GridUpdate::Cells that overtakes its labels is dropped. One task for the whole registry rather than one per document is a throughput choice and nothing more: the runtime is single-threaded either way, and no state is shared between documents by sharing a queue.

+

No auth, no TLS, localhost only. A deliberate MVP limit (improvise-q08), not an oversight: anyone who can open a socket to this port can rewrite every document it serves, so it binds a loopback address by default and serve says so. --bind exists so the limit is a choice you can see rather than one buried in the code. ADR 001 owns what multi-writer safety becomes.

+

#Origin

+

src/origin.rs is the handshake's one access control, and the reason it cannot be delegated: a websocket handshake is exempt from the same-origin policy. There is no CORS preflight, so any page a user visits can open a socket to the endpoint from that user's own address, and read and edit every document the server holds — cross-site websocket hijacking (improvise-kc7n). An address check cannot see it, because the request genuinely comes from an allowed address; an IP allow-list, a private network and loopback binding all answer "may this client reach the endpoint", and the attacker's answer is honestly yes. Only Origin says what is driving the client, which is doctrine(who_may_connect_is_two_questions). Loopback is the least protected case, not the safest: there is no proxy in front of it to check anything, and loopback is not a boundary a browser enforces, so while the server runs any page open in that browser can reach ws://127.0.0.1:8787.

+

The default allows loopback pages and nothing else, because a default that fails open is worth nothing and a default that breaks the shipped recipe is a different bug. The two shipped paths were the constraint. The local pairing recipe serves the page with python3 -m http.server 8080 --directory web, so its origin is http://localhost:8080 while the socket is on another loopback port — loopback is therefore the default, not a flag, or the one case worth optimising for would need configuring. The deployed Pages page sends https://fiddlerwoaroof.github.io and can only reach the server through a TLS proxy (improvise-ws-server speaks plain ws://), which is already a multi-step setup with an Origin allow-list in it; one --allow-origin is therefore an honest cost there and the project's own deployment origin is not hard-coded as a default. OriginPolicy::permits is the whole decision, and the transport's admitted is the only caller.

+

An absent Origin is allowed, decided rather than defaulted. Every browser sends one on a handshake, unconditionally — unlike fetch, there is no no-Origin mode to fall into — so an absent one is a non-browser client: curl, a native client, this crate's own tests/socket.rs. Refusing those would break the repo's own tests and every scripted client while buying nothing, since an attacker who can run a non-browser client already has direct network access, which is the question --bind and a proxy answer. The cost is stated rather than hidden: a proxy that strips Origin turns this check off, so the proxy's own Origin allow-list stays in the recipe as defence in depth.

+

Two shapes state no origin a policy can answer for, and both are refused rather than read as absent: an Origin that is not valid text, and two Origin headers. HeaderMap::get is first-wins, and the deployment this README recommends puts a proxy in front of the socket — one that appends rather than sets the header leaves two, and answering from the first lets the second be anything at all. No browser sends two.

+

What this does not buy is authentication. It answers "what page is driving this client", never "who is connecting"; anything that is not a browser page sends no Origin and is unaffected by any of it. improvise-kpnr is still open and is a different question.

+
+

#improvise-protocol/src/resolve.rs — one keypress → Commands

+

#It is not the browser's module any more (improvise-0yq8)

+

resolve/step and ClientState were improvise-wasm-client's, and nothing about either was a browser fact: both depend on ViewState, the protocol widget caches and the announced window, every one of which is already a protocol type. Being in the wasm crate meant a second front end could reach them only by depending on a cdylib built for a browser, so the move cost no new dependency and bought the rule design-principles §5 states — a new front end needs exactly two things: its own event → KeyEvent mapping, and painting. BrowserKey is the first of those two and is the only thing left in improvise-wasm-client/src/keys.rs; ClientState is now improvise_protocol::client, and typed_cell_command went with step, which calls it.

+

What did not become trivial is tests/lockstep.rs. It drives improvise_command::ui::reduce::reduce_key — the host's Cmd/Effect path — against resolve, and those are still two derivations of the same answer, in two crates, one of which cannot be reached from a browser. Moving resolve changed which crate it is spelled in, not that it is a mirror, so doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished) does not fire and the test stays.

+

#The keymap is not this module's

+

It used to be. improvise-bpq2 gave this file a hand-written classifier that decided, per key and per mode, what a keypress meant — a second spelling of default_keymaps(), kept in step by a lockstep test over a hand-written list. The user's ruling is that the web front end uses the same keymap as the TUI, not a mirror of it, and doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished) says why: two definitions held honest by a compatibility test are one definition waiting to be made.

+

So the table moved to improvise_protocol::keymap and this module reads it. Which key means what is now correct by construction: a rebinding in default_keymaps() reaches the browser with no change here, and a new binding for a command this module can already execute simply works.

+

#What is left here, and why it is the better boundary

+

A lookup yields a command name and arguments. Turning that into something that runs needs a CmdContext — a Model, a GridLayout, an App — which this client does not have. So key → intent is shared and intent → Command is ours, for the subset we can compose. step is that subset, and it is the list that replaces the old list of keys. That is a list of names rather than of keys, which is strictly better in two ways: it is enumerable, and it does not go stale when a binding moves.

+

What this client can execute, and what each one needed:

+ +

#The second tranche, and the three it could not reach (improvise-e24v)

+

Fifteen more command names resolve here: cycle-panel-focus; the category panel's cycle-axis-at-cursor, open-item-add-at-cursor, filter-to-item, toggle-cat-expand and delete-category-at-cursor; the view panel's switch-view-at-cursor, create-and-switch-view and delete-view-at-cursor; tile select's set-axis-for-tile and cycle-axis-for-tile; the help overlay's help-page-next/help-page-prev; commit-buffer in all three of its kinds; and the two cell writes clear-cell and paste.

+

Each is Cmd-for-Cmd, including the parts that look like noise: the status sentence a command writes beside its mutation is part of the answer, and a client that emitted the mutation alone would diverge on the status line and nowhere else — which is precisely the sort of divergence nobody notices. Two orderings are load-bearing and copied rather than rationalised. CommitBuffer writes its optimistic status first so the add's own failure message wins (improvise-6t3), and TileAxisOp writes the axis first and the status after.

+

Three of the tranche's own list turned out to need enrichment after all (improvise-4bwa), and the reasons are worth more than the code:

+ +

The lockstep harness these are tested through (both_ends in tests/lockstep.rs) is worth two sentences, because two of its choices were wrong first. It compares the document as .improv text, not as Debug: Debug prints the symbol table's HashMaps, so two byte-identical documents print differently per process (gotcha 18), and the first version failed on nothing but hash order. And it applies the client's composed commands to a second authoritative host rather than eyeballing them, restating the basis against that host with caught_up — the claim was published by the first host, and the basis is the only thing that legitimately differs between two hosts holding the same document.

+

How the client claims, and why there is nothing here to keep in step. A client must answer the same question the server does — which records does this ModelCommand write — to build its Basis. The first version of this answered it again, in two helpers (structural, claiming nothing, and record_write, claiming the cell's own record), and the adversarial review of PR #50 showed that nothing tested them: lockstep.rs's apply_on restates the basis against the twin with caught_up, which re-derives it with the server's records_named and throws the client's claim away before anything is compared. Emptying record_write's claim left the workspace green.

+

The fix is not a test. records_named is pure — &ModelCommand -> Vec<RecordHash>, no App — and RecordHash is an improvise-core type the protocol already re-exports, so it moved down into improvise-protocol beside the enum it reads, and improvise-command re-exports it. resolve.rs has one model() that claims records_named(&cmd). The client's claim is now right by construction, and the guard is the one that was always there: records_named is an exhaustive match, so a new record-writing variant fails the build.

+

What the old shape would have cost is worth recording, because it is not a compile error: structural accepted any variant and claimed [], so a new record-writing command would have compiled clean on the client with the wrong answer, and the symptom is Rejection::RecordUnstated — the browser binding becomes a dead key, refused, with nothing failing.

+

both_ends compares every cursor, not only the document, the mode and the status. Three of the resolvers' decisions are a cursor move and nothing else — delete_view_at_cursor's step-back is the sharpest — so a comparison that stopped at the document would agree while the panel cursor diverged.

+

#Mirror the Cmd line for line, including where siblings disagree

+

doctrine(mirror_the_cmd_line_for_line). Move emits SetSelected even when it clamps and MovePanelCursor returns nothing when the cursor does not move; MoveTileCursor emits regardless, like Move. Tidying that into one rule would make this a second derivation, which is exactly what tests/lockstep.rs exists to rule out. Which widgets say nothing about a move that goes nowhere is not a rule of this module — it is whatever the Cmd being mirrored does, and the three differ.

+

Two more places the mirroring is literal. panel_cursor clamps a cursor past the end before applying the delta, and derives the category panel's length from flatten_cat_tree — which depends on expanded_cats, per-client, which is why it is derived rather than sent and why the derivation lives in improvise-protocol where both ends reach it (improvise-edp). And commit mirrors change_mode(exit_mode_for(edit_mode)), then EnterAdvance — which aborts the rest of the batch at the bottom-right corner, so the re-entry is skipped and the user lands in normal mode — then EnterEditAtCursor, which seeds the edit buffer from the cell it landed on. The trailing clear-buffer the keymap welds to every commit then empties that buffer again. That ordering looks wrong and is exactly what the TUI does; mirroring it is the point.

+

GridMove is a hand-spelled mirror of navigation.rs's MoveKind, because the command crate is not on a browser's dependency list and never will be. The keys that reach it are no longer spelled twice. scrolled_to mirrors navigation.rs::viewport_effects over the window this client announced.

+

#The three outcomes

+

KeyOutcome::Local(Vec<Command>) — resolved here; the caller folds them locally and sends them, as one frame and in this order (the asker's own fold is its own; the host forwards only to the others). A Vec because one keypress is one binding but several Effects: a cursor move is SetSelected plus whichever scroll offsets follow, and a Binding::Sequence is several commands to begin with. Never empty — a keypress this client can answer with nothing is Ignored, and KeyOutcome::of is the one constructor so that contract is a property of the type's users rather than a rule each resolver remembers.

+

Upstream(KeyEvent) — bound to a command this client cannot compose. Ignored — nothing bound, or what is bound resolves to no change. Nothing bound is not "the host might know better": the host looks the key up in this same table. That is new; it used to go upstream into an empty vec.

+

A Binding::Sequence is all or nothing: half of Esc — clearing the buffer without leaving the mode — is worse than none of it. Each step is resolved against the unchanged state, because the host resolves every step against one immutable CmdContext; the commands are then folded in order.

+

#What still needs the host

+

the_command_names_that_still_need_the_host_are_these names them, and Host::handle answers a Key frame with an empty vec, so each one is a binding that does nothing at all in a browser (doctrine(an_upstream_key_is_a_black_hole)).

+

An upstream key that the app also ADVERTISES is a different severity, and it had its own class of defect (improvise-rmvx). A key nobody offers doing nothing is a gap; a key named in the mode's own hint line doing nothing reads as broken. The two are crossed by every_advertised_command_answers_in_its_own _mode, which asks help::advertised — a table of command names, so the cross-reference is exact and nothing tokenises the rendered string — against resolve in every mode, over a fixture furnished enough that an Upstream can only mean an unhandled name. The old guard asked only about Normal, which is why add-record-row and delete-formula-at-cursor were outside its reach entirely; view-back in RecordsNormal was a seventh instance nobody had noticed.

+

Ignored is a black hole too, and the guard watches both variants. KeyOutcome::of turning an empty Local into an honest Ignored is good design and also a laundering step — every wrong answer that composes nothing arrives wearing the variant that is usually innocent. A guard's discriminator must therefore be state (did anything the user can see change) and never the shape of the return value. Watching only Upstream let three of the six declines go completely silent with 1397 Rust, 19 harness and 26 node tests green, because no browser spec presses those three keys (PR #71 review).

+

nothing_declined_is_also_composed closes the other direction: step reaches declined first, so an arm in composed for a declined name is unreachable, and the browser would go on saying "not available yet" over working code. It is derived from declined itself over every bound (name, args) in every mode, so a seventh decline is covered without editing it. enter-mode was a real overlap on its first run; rather than move the check, both halves hold the rule — composed returns None for a stranded mode — so removing one fails towards Upstream rather than towards a dead room.

+

declined is the answer, and it is one list rather than an arm apiece: six command names, each with its own true subject. Wording matters here more than it looks — the > fix was nearly routed through a message about aggregates, which would have been true-sounding about the wrong thing and worse than silence, because a reader who is told something plausible stops looking. step checks declined first, so composed has no arm for any of them and the two lists cannot overlap.

+

A declining step takes its whole sequence with it: o in RecordsNormal is [add-record-row, enter-edit-at-cursor], and the second step's premise is false once the first has declined. Returning only the decline, rather than the decline plus whatever composed before it, is what keeps Binding::Sequence's "every step or none of them" true — an abort is not a rollback (gotcha 21), and this client has no way to undo a Command it has already put on the wire.

+

The classes that remain:

+ +

#The three improvise-e24v listed as free, and what each really costs

+

improvise-4bwa. Each of the three is short exactly one fact, and the three answers are different in kind — which is the point, because "needs enrichment" read as one bucket is what let them be listed as free.

+

view-back is declined, permanently as things stand, and it is not an enrichment at all. ViewNavigate { forward: false } emits ApplyAndClearDrill before ViewBack, and ApplyAndClearDrill is deliberately off the wire: its operand is drill_state.pending_edits, session- local residue belonging to whoever staged it, and a Command::Model is routed with no SessionId (gotcha 38, improvise-3x3u). A client that mirrored view-back by reconstructing the drill commit would commit somebody else's uncommitted typing — which is the exact thing 3x3u removed, and the reason it must not be "mirrored line for line" here despite that being the rule everywhere else in this module. So it stays upstream until ADR 001 layer 2 gives the addressed form. Note the sharper reading: mirror_the_cmd_line_for_line is a rule about resolvers, and a Cmd whose first effect is unaddressable is not a Cmd a thin client may mirror at all. The doctrine does not conflict with this; it simply does not reach it.

+

toggle-prune-empty needs one bool on a per-view projection. TogglePruneEmpty::execute reads ctx.view.prune_empty only to word its status — "Showing all rows/columns" against "Hiding empty rows/columns". The ModelCommand itself is composable today; it is the sentence beside it that a client cannot write, so mirroring it would put the two ends out of step on the status line in exactly half of all presses. View::prune_empty is on the View inside the Workbook, in no widget cache and not in ViewState.

+

delete-formula-at-cursor needs an element-type change, and the shape of the change is the whole decision. The effect is RemoveFormula { target, target_category }; FormulaCache carries the formula's raw text ("Profit = Revenue - Cost"), from which neither field is recoverable without reimplementing parse_formula in the client. Adding a targets vector beside formulas would be two answers to "what is in the formula panel", which is improvise-edp's disease and the mistake improvise-anr9 had to pay a type change to undo. So the element type grows the fields; it does not acquire a parallel array.

+

Both of those are ordinary projection additions and both live in improvise-protocol (render.rs, resolve.rs) and improvise-command (ui/project.rs) — not in this crate, which is why this section records the design and the work lands with whoever owns those files. Whatever lands, the named list the_command_names_that_still_need_the_host_are_these shrinks by exactly what closed.

+

#What a client that could parse would cost: measured

+

improvise-cl7y asks a client to stop claiming "Formula added" for text only the host can parse. Its own note says the two routes are a dependency decision or a status addressed back to the composer, and leaves the cost of the first unmeasured. It is measured now, by adding improvise-formula to this crate, calling parse_formula from an exported method so nothing is stripped, and building the bundle both ways:

+
improvise_wasm_client_bg.wasmgzipped
today503,854191,613
with improvise-formula634,515238,973
cost+130,661 (+25.9%)+47,360 (+24.7%)
+

The raw column is exact and byte-verified; the gzipped one is a gzip -9 proxy and not what a visitor downloads, since GitHub Pages serves brotli. The conclusion is unaffected — the raw figure alone carries it — but do not quote "+47,360 gzipped" as a transfer size.

+

A quarter of the download, for a status string. That is the evidence for the other route — the host addressing a status back to the composer, the way ClientFrame::Save's rejection already does — and it costs no dependency.

+

Note what the measurement does not settle, because it is the more interesting half: a client that could parse would be a second derivation of whether a formula is well formed, agreeing with the host only for as long as both call the same function. The status line is not the only thing that wants it, so the number above is a floor rather than the price of a feature.

+

#stranding: a prompt this client can enter but cannot commit

+

enter-mode is trivially resolvable here, and that is the trap: the user reaches the prompt, types a path, presses Enter — and the commit resolves to nothing, because Host::handle answers a Key frame with an empty vec. A dead room is worse than a locked door: the door costs one inert key, the room swallows work the user believes was done.

+

SavePrompt and ExportPrompt are host-only by construction, not by omission: effect::Save, SaveAs and ExportCsv are all EffectKind::ViewOnly, so the derivation that builds ModelCommand from the model effects cannot name them. Closing this is not a resolver to add; it is a wire operation that does not exist. In a browser it is also close to meaningless: the worker persists on its own behalf whenever the document is dirty, so the document is the OPFS entry and there is no Ctrl-S semantics to offer.

+

It returns the subject rather than a bool (improvise-rmvx): the refusal is right and the silence was not, so : now says the command line is unavailable instead of doing nothing at all. The predicate is named as a property rather than matched at the call site (doctrine(two_questions_one_enum)): "can this client finish what entering the mode starts" is not the same question as "is this a text-entry mode", and a third such prompt belongs here. Found by Copilot on PR #27 against the newly added SavePrompt; ExportPrompt had the identical trap already and was fixed with it — fixing one arm of a rule with two is the shape doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only) names.

+

CommandMode is dead by omission, and listed anyway. Copilot found it on the next round, and it was a false negative in the predicate's first version — which mattered more than its severity, because a_prompt_this_client_cannot_commit_is_never_entered used : as the negative arm that stops "refuse every mode" from passing. An arm asserting something false does not merely fail to guard; it certifies the behaviour it was added to rule out. : resolves locally into CommandMode and typing appends locally, but Enter is [execute-command, clear-buffer] and step has no execute-command arm, so resolve returns Upstream and the frame is dropped. Every ex-command behind : is inert for the same reason, so refusing the mode removes no working surface, only a room that swallows typing.

+

The distinction from the two prompts is worth keeping visible, because it says who removes that line. They are dead by construction: no ModelCommand can ever name Save/SaveAs/ExportCsv, so no resolver would help. CommandMode is dead by omission: execute-command is implementable client-side and improvise-e24v is scheduled to do it. Listing it is therefore also the named gap gotcha 47 asks for — closing e24v has to come to that line and delete it.

+

#Per-arm notes that would otherwise be re-derived wrongly

+ +

#Browser key → improvise key

+

to_key_event returns None for two cases the host must never see. A bare modifier press ("Shift", "Control", …) is not a keypress. And anything held with the platform meta key is left to the browser: KeyModifiers has no bit for Meta, so forwarding Cmd-R would arrive as a plain r and type into a cell instead of reloading the page.

+

Two conversions are not obvious. A terminal reports Shift-Tab as its own code and the keymaps bind that code; the browser reports Tab plus a shift flag. And shift is deliberately dropped for character keys: the browser already applied it ("A", not shift+"a"), and the shared keymap's lookup_local retries character keys without modifiers for that very reason, so keeping the flag would only make an exact binding miss.

+

#The pointer: Click, and the four rules it obeys

+

A click is resolved here for the same reason a keypress is — a front end paints and wires, it does not decide (design-principles §5). resolve::click takes a Click naming a widget and an index, and returns the Commands. web/app.js attaches listeners and knows nothing else; there is one entry point per widget on Client, which was the user's ruling on improvise-7ybz.

+

Rule 1 — a click navigates to what was clicked, and activates it where activation is something the client can express. Every arm composes the same shape: focus the panel, SetPanelCursor to the row, then whatever Enter does on that row. The activation is resolved against the clicked index, never against the cursor, because the SetPanelCursor ahead of it has not been folded when the resolver runs — the trap that reads as "the cursor lands on Plan and the document switches to Default", silent and wrong in the half a user does not watch.

+

The second clause of rule 1 is what makes the formula panel look inconsistent and is in fact the rule holding. ModeKey::ViewPanel binds Enter to switch-view-at-cursor; ModeKey::FormulaPanel binds a/n/o and d/Delete and no Enter at all. So a view-row click switches and a formula-row click only navigates. Opening the editor on a formula click would be an activation invented for the pointer that the keyboard does not have — a second derivation of what a formula row means, which is the disease §5 names. The guard for this must be aimed at the keymap, not at the click: asserting that the click emits no FormulaEdit only restates the author's reading, and a reviewer demonstrated the gap by adding Enter -> enter-mode formula-edit to FormulaPanel and watching the whole workspace stay green while the click and the keyboard disagreed. the_formula_panel_has_no_enter_to_mirror_and_the_view_panel_does asserts the binding's absence and names click_formula_row as what must change if it ever appears.

+

The browser assertion that PR #60 dropped stays dropped, and improvise-qf57 is the decision rather than the oversight. #grid input toHaveCount(0) was argued to be redundant with the #mode assertion on the grounds that toHaveText("FORMULAS") is exact and AppMode::FormulaEdit labels as "FORMULA" — a coincidence of two strings differing by one character, which would be a thin reason to leave it out. Measuring it gives a better one: it is not redundant, it is unreachable. web/app.js creates a grid <input> in exactly one place, the grid cell's own mousedown, gated on cell.editable; no repaint creates one and no panel handler can reach it. And an invented formula-row activation opens the minibuffer, not a grid editor. Injecting Command::ChangeMode(AppMode::formula_edit()) into click_formula_row and running the spec: #grid input toHaveCount(0) passes, and the run fails on the #mode assertion below it. Restoring it would add an assertion that cannot fail — the same defect this file already records one line down, where an explanatory assert! was removed for never being able to be the failing one.

+

Rule 2 — a click is all of its intents or none of them (improvise-hkr8). Each click_* returns Option<Vec<Command>> and click does the single unwrap_or_default. The case that matters is an activation the client cannot compose: filter_to_item_at goes through model, which goes through BasisCache::claim, which returns None until the server has published a basis (gotcha 43). Before the fix the None was swallowed per-intent, so the click kept its navigation prefix, dropped its activation, and reported success — while the keyboard handled the identical None correctly by answering KeyOutcome::Upstream. Note that None at the click level means something different and benign: the whole click composed nothing, which is the invariant rather than a breach of it.

+

Rule 3 — a click on an unselected tile never mutates the document. The user's ruling on the tile bar. The cost accepted is two clicks for the common action and a deliberate divergence from the keyboard, where Enter cycles directly — record it as chosen, not as an inconsistency to tidy, or someone will read the binding and "fix" the pointer to match. What it buys is worth more than the keystroke: a tile cycle is a LayoutGeometry change, which bumps cursor_epoch and therefore resets every connected client's cursor (gotcha 24), so a stray click used to cost every other viewer their position.

+

State the rule this precisely, because the looser wordings are false. The implemented rule is "clicking the already-selected tile cycles it; clicking any other selects it", where already selected is mode == TileSelect and tile_cat_idx == index. The tempting summaries — "a click never mutates on its own", "a second click cycles" — both have the same counterexample, found on review: T enters TileSelect leaving tile_cat_idx at 0, so a click on the leftmost tile is a click on the already-selected one and cycles on the first gesture. Two adjacent tiles therefore answer the identical first click differently, and that is correct rather than a hole — a click does what Enter does there, and Enter in TileSelect cycles the tile under the cursor. The resolver cannot see how the state arose and must not try to.

+

The mode conjunct is still load-bearing in the other direction: without it, tile_cat_idx == 0 in Normal mode would read tile 0 as already selected, and the very first click of a session on the leftmost tile would cycle it.

+

Rule 4 — the shared helper is the idiom, and breaking it is how the two arms drift. click_category_row calls filter_to_item_at, click_tile calls tile_axis_at, click_view_row calls switch_view_at — each is foo_at(index) with the keyboard's foo_at_cursor calling foo_at(cursor). click_view_row originally re-spelled SwitchView + ChangeMode(Normal) inline, so a host-side change to SwitchViewAtCursor was forced into the keyboard arm by tests/lockstep.rs and left the click behind silently. A helper that returns only the name rather than the whole activation is what invites that.

+

#The category panel opens at launch, in both front ends

+

ViewState::default() sets category_panel_open: true. The user ruled this twice: first that the sidebar be visible at launch, then — asked whether the terminal was collateral — that it applies to both. improvise model.improv opening with the sidebar showing is intended, not a side effect of sharing a default. It is the shared default for the reason wsaw gives: a browser-only default would be "the page overrides the flag", which is the same defect as "the page ignores the flag" wearing different clothes.

+

A launch default disarms every test that asserted the thing it now supplies, and that is the expensive part. Three tests asserted category_panel_open was true after doing something to make it true; all three then passed without the code under them working. The worst was set_panel_open_and_cursor: with it vacuous, effect::SetPanelOpen's Category arm could be replaced by a no-op — C doing nothing in the terminal — with the whole workspace green. The rule this leaves: assert the direction the default cannot supply. Closing the panel is the discriminating move; opening it proves nothing.

+

Note the two mappings that had to be repaired separately. effect::SetPanelOpen and reduce_view's Command::SetPanelOpen are two three-way matches in two crates — the TUI folds one and the browser the other — so a guard on either says nothing about the other. That is a derivation done twice (§5), pre-existing and filed rather than fixed here.

+

Two tests, because either alone passes against the bug wsaw fixed: the panel is visible with no keypress, and C still hides it. Injecting "the page ignores the flag" fails the second; injecting "the default is false" fails the first. A test that only asserts the sidebar is visible cannot tell the flag being honoured from the flag being ignored.

+
+

#improvise-wasm-client — the crate, the fold, the render model

+

#What the crate is, structurally

+

One wasm32 artifact, reused literally in both deployments: the thin client talking to a server over a WebSocket and the standalone app talking to a worker that hosts the App over the worker's own port. Only the JS bootstrap differs — nothing in the crate knows what the transport is.

+

It is a fold and a key classifier, and nothing else. It holds a ViewState plus one widget cache each — there is no aggregate render cache — and it moves them by folding the same Command stream the host folds. The fold is state.rs; the key classifier is keys.rs.

+

The argument that shaped it: a keymap binding names an intent resolved against a CmdContext; a Command is the resolved mutation. A thin client holds no Model and no GridLayout, so it cannot do that resolution. (Since improvise-bpq2 the table is shared and the resolution of the composable subset happens here; what is left upstream is the subset keys.rs names.)

+

That is why the command crate is a dev-dependency and not a dependency: its keymap could not be used here even if the bundle could afford App, Workbook, persistence and the import wizard. What the dev-dependency buys is tests/lockstep.rs, which drives the authoritative host and this client over the same keypress and asserts the two ViewStates agree — the correspondence is checked rather than asserted in prose.

+

The boundary is JSON, so the JS side needs no generated structural bindings and the same strings go on a socket unchanged. No std::fs (it compiles for wasm and fails in a browser) and no clock: nothing here calls Instant::now(), so the web-time shim the command crate needs is not needed. Both are structural, not disciplined — this crate depends only on improvise-protocol, serde, serde_json and wasm-bindgen, none of which reaches a filesystem or a clock.

+

Ordering. The JS bootstrap must deliver frames to Client::on_frame in arrival order, one at a time. The grid fold is deliberately not commutative — a GridUpdate::Cells that overtakes the labels framing it is dropped — so batching or reordering silently loses cells. For the same reason a locally-resolved command and a forwarded keypress must go out on the same ordered channel, in the order Client::on_key_event returned them.

+

Why the wasm_bindgen surface is a thin wrapper over a plain-Rust one. wasm_bindgen methods cannot take or return the protocol's types, so there is one code path serialized at the edge. The split is not only ergonomic: JsError::new panics off wasm, so every fallible path has to be reachable without it or its failure arm cannot be tested at all.

+

Client::viewport vs Client::set_viewport. viewport is associated rather than a method, because a page must send the handshake before it has anything to fold. But it builds the frame and forgets it, so a client driven only through it never learns its own window and therefore resolves no cursor key locally — a truthful subset (those keys go upstream and the host answers nothing) but a whole feature quietly off. set_viewport is the one a page should call. The window is not derivable from anything the host sends: the row frame's own length is min(window, grid − offset), so it agrees with the window except near the bottom edge, which is precisely where the scroll arithmetic matters. viewport is kept because web/app.js already spells it and retiring it is a co-ordinated change.

+

#improvise-protocol/src/client.rs — the two properties this dispatch owes

+

Both are tested in state.rs rather than assumed from the protocol crate's own tests, because the dispatch between them is this crate's.

+ +

window is deliberately not reset by bootstrap: the browser window did not change size because the document was replaced, and clearing it would silently stop this client resolving its own cursor keys until the next resize.

+

last_rejection is kept as a value, not a sentence. A rejection is the input to force, and force is a re-composition against the republished basis — which needs to know which record and which kind. RecordGone cannot be forced at all; RecordMoved can be forced with one keystroke. A string cannot carry that. It is session-local: it belongs to whoever composed the refused batch, so it is not part of the portable slice and does not survive a bootstrap.

+

window is session-local and not portable for the same family of reasons: it is a fact about a browser window, which is why no Command carries it and why it is not on ViewState. (0, 0) means this client has not announced one yet, and a client in that state resolves no cursor key — it has been sent no grid either.

+

#socket.rs — the frames, the wording, and the one composition

+

Where the frames live: not here. ClientFrame/ServerFrame are improvise-protocol's, and the same pair frames both deployments. This crate used to carry two envelope pairs of its own: one mirroring improvise-ws-server, because a native crate cannot be a dependency of a wasm bundle, and one written for the worker before the projection architecture landed. The mirror was held honest by two tests round-tripping every variant through the server's own types; with one definition there is nothing left for them to compare, which is what improvise-xks6 was for.

+

rejection_message is spelled here rather than on Rejection for the reason AxisColor is not a ratatui colour: the protocol carries the fact, and each front end words it. The wording tracks ADR 001's force table, because "try again" and "this cannot be forced" are the two different things a user must be able to tell apart — a RecordGone has nothing to write to, and offering force there would promise something the design refuses to deliver (force never creates structure). Two wordings are load-bearing: DocumentReplaced must not say "reloading", because nothing here reloads, and the two save failures are not staleness so neither offers a retry — retrying changes nothing about the host's storage.

+

typed is a rule that genuinely lives in two places. It is the same three-way split commit_regular_cell_value makes — empty clears, a parseable number is a number, anything else is text. It is duplicated here for the reason the whole client exists: the browser has no App to ask, and ModelCommand::SetCell takes a CellValue, so somebody on this side must decide. Putting it in Rust rather than in the page's JavaScript is what makes it testable and keeps one spelling per language boundary instead of two.

+

This is the only answer the client gives to "what does a typed string mean" (improvise-r4lo), and it holds on every deployment including the worker: a host's App sits behind the frame boundary and is handed a value, not a string, so no host ever classifies a string that arrived over the wire. The careful half: a host does classify its own local keystrokes, which is why this is a duplicate of that split rather than a call to it — the command crate is deliberately not a dependency, and reaching it would drag App, Workbook and persistence into the browser bundle.

+

So a_typed_string_is_classified_as_the_host_classifies_it states the obligation without discharging it: it asserts the three outcomes by hand rather than comparing with the host's function, so the two could drift and stay green. That is doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished) still open, one layer below what improvise-r4lo closed — the real fix is a shared split in improvise-protocol, which both crates already depend on. Filed rather than done.

+

typed's None means "clear this cell", which is a different command, not a different value — hence the Option rather than a CellValue::Text("").

+

typed_cell_frame is the composition improvise-q08 found a thin client could not do. It holds no Model, so it cannot build a CellKey — but it does not need to: the id on the cell it is already drawing is the address, and it is echoed back unread (ADR 002). It takes the user's string, not a CellValue, for the reason above; there is no deployment for which a pre-classified entry point would be right. A sibling taking a serialized CellValue used to sit beside it and claim the opposite; it was deleted rather than re-documented, because it could not express ClearCell and was therefore a strictly weaker duplicate.

+

None on four shapes, each a different fact the client already holds: the position is outside the grid; the position holds no settable cell (id: None — padding, or a records-mode column that renames a coordinate rather than writing one); the cell is an aggregate, which is the pattern failing to bind rather than a check; or this client has not been told the document's basis and so has nothing truthful to say about what it composed against (BasisCache::claim withholds rather than invents).

+

Nothing is folded there. The new value comes back as a GridUpdate::Cells from the host, which is the side that formats it; guessing it locally is how the two ends drift apart.

+

typed_cell_command is the one composition shared by the mouse path (typed_cell_frame) and the keyboard path (keys::commit). Writing it twice is how a browser commit by Enter and a browser commit by click would come to disagree about what an empty string means.

+

#render.rs — the four decisions that are silent when wrong

+

The DOM half of this client is JavaScript. The split is drawn to make that acceptable rather than comfortable: every question with a wrong answer is answered here, in Rust, under test, and what crosses to the page is a list of rows of cells that each already know whether they are editable, numeric, erroneous and selected. The page's job is document.createElement.

+ +

Nothing in RenderModel is on the wire. It is a projection of a projection — the client's own caches shaped for one renderer — so it may change freely without a protocol conversation.

+

Four smaller decisions that would be re-derived wrongly:

+ +

RenderCell::edit_text is the stored value, not the formatted one: seeding an editor from text and committing turns 1200 into the string "1,200". The numeric/error flags are the server's answers, echoed — the client must not re-derive them from the text, which is how the two ends drift.

+

render_model builds header_labels(0) once for the whole frame, because the per-position accessors scan and asking them inside the row loop is quadratic.

+
+

#render.rs and the help overlay (improvise-ktv1)

+

RenderModel carries help, help_page and help_titles, and the decision the page must not make is whether the overlay is open: an empty help is what says it is closed. That is the shape col_groups already uses for "this grid has no column groups" — an absent list rather than a flag the page has to interpret alongside a mode it would then have to understand.

+

The rows themselves are improvise_protocol::help::pages(), the same ones src/ui/help.rs draws. RenderHelpRow carries a kind string (heading, keys, accent, …) which is a semantic name the page maps to a CSS class — the AxisColor → axis_fg shape again, and the reason the page can style a heading without knowing what makes a line a heading. Where the content comes from and what is derived from the keymap is context/code/protocol.md §8.

+

#web/app.js — the page

+

The browser half of improvise-cr3, kept as thin as it can be made, now talking to either of two peers (improvise-djm). The boundary is JSON strings in and out, so this file holds no protocol knowledge at all: it never inspects a Command, never builds a CellId, never decides whether a cell is editable or whether a typed string is a number. Five calls are the whole harness:

+
callwhat it is
client.set_viewport(rows, cols)the handshake frame, and every resize; also records the window so keys resolve here
client.on_frame(json)fold one frame, in arrival order
client.render_model()everything to paint, already decided
client.typed_cell(r, c, s)the frame for "the user typed s here"
client.on_key_event(k,c,a,s,m)the frame for a keypress, folded locally first where this client may answer it
+

The two rules this file must not break are ordering and framing: frames are folded one at a time in the order they arrive (a cell update that overtakes its labels is dropped, not replayed), and the first frame out must be the viewport, because the server builds the snapshot at it.

+

Everything in this file that is a decision rather than a paint now lives in web/shell.js — socketUrl, the #status composition, the watchdog, the extent arithmetic and gridPlan — so that node --test can reach it without a browser. The sections below describe those rules; the code is one file over.

+

#Two transports, one client

+

The wasm bundle is the SAME artifact either way (improvise-xks6: one envelope pair, two deployments). Only where the strings come from differs, so the difference is confined to openTransport and nothing below it knows which peer it has:

+ +

Neither is reachable silently. Every transport writes what it is doing to #status — "starting", "connected", "cannot reach …", "disconnected" — so a peer that never arrives reads differently from one that went away. "disconnected" means exactly the second: a socket that opened and then closed. A socket that never opened keeps saying "cannot reach …", because the close event there is the tail of the failure and not news.

+

#status carries two independent facts — what the transport is doing and what the document last said — so it composes them rather than letting a repaint quietly erase a connection error.

+

#socketUrl: wss:// is the default

+

A bare host:port gets wss://, because the address that works from where this page is actually served is the one it should be easiest to write. The page ships to GitHub Pages, which is https, and an https origin refuses a ws:// address as mixed content — so a ws:// default would hand the browser a URL that cannot work and report it as a connection failure, which reads like "the server is not running".

+

An explicit ws:// is honoured only where it can function: a page served over plain http, which is the local python3 -m http.server case that pairs with improvise-ws-server (which speaks plain ws:// only — it has no TLS, so reaching it from https means terminating TLS in front of it). From an https page an explicit ws:// is upgraded rather than passed through, since passing it through can only produce a failure the user cannot act on.

+

#set_viewport, and the twin that is gone

+

Same frame, but it also records the window. A client learns its window only from what it announced — the row frame's own length is the window clipped to the grid, so the two differ exactly at the bottom edge, which is where the scroll arithmetic matters. Without it the client cannot clamp, so every cursor key goes upstream into Host::handle's empty vec and the arrow keys silently do nothing (improvise-bpq2).

+

There used to be a frame-only twin, Client::viewport, exported beside it and called by nobody. Its doc said it was kept because web/app.js already spelled it — and web/app.js called set_viewport in the same wave, so the reason was false before it was read (improvise-seik). That matters more than an ordinary unused function, because the two are one letter apart and the wrong one fails silently: it sends an identical frame and simply forgets the window, which turns every cursor key into the black hole above. It is deleted, along with the exported init() free function that duplicated new Client().

+

A deletion is not the guard, though — the next export with no caller arrives the same way. web/tests/bundle/wasm_surface.test.js now reads the generated .d.ts and requires every declared method to be one app.js calls, or to be named in an UNCALLED table with its reason. free and initSync are wasm-bindgen's own; revision is there because on_frame returns it. Proved by injecting a spare static viewport(...) declaration into the built .d.ts and watching it fail by name.

+

#frameCost: what the grid's own frame costs, in whole cells

+

One header row and one header column was right until improvise-anr9: the column header is now one row PER LEVEL plus an optional group row, the row header is row_levels columns wide, and inline group headers take a row each. Reserving one of each announces a viewport the page cannot show, and #grid has overflow: auto, so the DOM would acquire a scroll position of its own while the client's cursor offsets were computed from the larger window — putting the selected cell off screen.

+

It is measured off the painted model rather than predicted, and defaults to the old one-and-one before the first paint, which is the only moment there is nothing to measure.

+

Inline group ROWS are deliberately NOT reserved. They take real height, but how many are in view depends on how many rows the window asked for, so every way of counting them is either a feedback loop (count the current ones: a smaller window crosses fewer groups and asks for a larger one) or a ratchet (keep the maximum: a group-dense grid stores an oversized cost on its first oversized projection and never lets it fall — one group header per row clamps the grid to a single row, permanently). Both were tried. What is left is an over-request bounded by the groups in view: a grouped grid asks for a few rows more than the table shows, which matters only for a selection near the bottom edge — and that gap is not new and not specific to groups, since nothing in this page calls scrollIntoView at all. Filed as its own ticket rather than answered with a heuristic in a file the harness only partly executes.

+

#The watchdog, and the two close arms

+

The handshake watchdog is about the first frame, not about either peer's own failure modes. The socket has an error event for a peer that never answers; the worker has none that fires, because worker.js catches its own start() rejection — so a browser with no OPFS, or a worker bundle that failed to load, would leave the page saying "in-page worker" over an empty grid forever. "The peer never answered the handshake" is the one condition that means the same thing in both deployments, and a check aimed at each peer's own failure modes would have missed exactly this one.

+

A peer that failed loudly has already said something more precise, so the watchdog must be DISARMED and not merely pre-empted: it fires later and would overwrite the better message with "did not answer" (PR #13 review). The watchdog is for silence only, and setShellFailure clears the timer.

+

Did this socket ever open? The page has to record that itself, because nothing on the close event carries it: readyState is already CLOSED by the time a close listener runs, whichever happened, and a refused connection and a mid-session network drop are BOTH code 1006 with wasClean: false. So "never connected" and "was connected and went away" are distinguishable only by remembering — an opened flag is the only discriminator.

+

A socket that never opened did not "disconnect": it failed to connect, and the close event is the tail of that failure rather than a second, vaguer thing that happened (improvise-taxx). Writing "disconnected" there unconditionally replaced the error listener's precise message with the least informative of the two facts.

+

That is the same family as the watchdog bug and NOT the same fix. The watchdog is disarmed because it is for silence only, whereas suppressing close outright would hide a genuine mid-session disconnect, which is the one thing that listener is really for. Hence the split rather than a guard. Both arms go through setShellFailure, which disarms the watchdog — and that half is the one that keeps getting lost. It matters on the opened path: the watchdog is armed after transport.ready resolves, i.e. on open, so a socket that opened and then dropped before its first frame reaches the close listener with the timer still pending, and "disconnected" is more precise than the guess it would otherwise fire 8s later. On the never-opened path there is no timer yet and the clear is a no-op. Neither arm leaves one armed.

+

#Painting

+ +

#The selection scrolls itself into view

+

web/app.js called scrollIntoView nowhere at all, so a selection driven past the bottom of #grid stayed there, invisible, with no way back but scrolling by hand (improvise-jq8l). Two things about it are easy to get wrong.

+

It is not a projection bug and not a row_offset bug. The row had been sent: #grid has overflow: auto and the page announces an extent from ROW_PX/ COL_PX, which under-estimate — a real row is 28.6px against the assumed 26 — and inline group rows are deliberately not charged for at all. So the page renders more rows than its own box can show, and the overflow is the DOM's, not the window's. Measured on harness/fixtures/tall.improv: 105px of overflow, the selected cell 96px below the visible bottom, scrollTop still 0.

+

block: "nearest" is the whole option. It scrolls only when the element is actually out of view, and it scrolls the nearest scroll container rather than the document, which is what keeps the page itself from moving sideways.

+

The reason this was untestable until now is the fixture: web/demo.improv is four rows and harness/fixtures/grouped.improv is eight, and neither overflows a 484px box. tall.improv is thirty rows in three groups, and the spec asserts the overflow exists before asserting anything about the scroll — without that line the whole test passes on a page that never scrolls.

+

#The pointer takes cells on mousedown

+

A cell's handler is mousedown with preventDefault(), not click, and the reason is an ordering trap worth the paragraph (improvise-0x9u). From an editing state a click never arrived at all: the mousedown moved focus, the <input>'s blur ran close, close repainted, replaceChildren destroyed the <td> the mousedown had landed on, and no click was ever dispatched. The pointer was dead until the user pressed Escape or clicked outside the grid.

+

Three things had to move together, and each is load-bearing:

+ +

The state the old spec could not see is the whole point: it pressed Escape before clicking the second cell, so it only ever clicked from a non-editing page — this repo's characteristic defect, a test that passes while the thing it names is broken in a neighbouring state.

+

#The keyboard

+

Client::on_key_event classifies the event and folds it locally where this client may answer it — text entry, and since improvise-bpq2 the grid, panel and tile cursors too, which is why sendViewport must call set_viewport and not Client.viewport. Anything it cannot answer becomes a ClientFrame::Key, which Host::handle answers with an empty vec: those keys are sent and do nothing. That is settled rather than unfinished — the host must not know the keymap — and closing one means teaching the client, not that arm.

+

The page does not claim a key it did not answer: a frame means the page claimed it, so the browser must not also act on it — an arrow that both moves the cursor and scrolls the page is the worse of the two errors. Combinations under Ctrl or Alt are left alone regardless: those are the browser's own shortcuts, and the client resolves nothing under them. BrowserKey drops anything held with the platform meta key precisely so Cmd-R does not arrive as a bare r; the page does not claim it either.

+
+

#web/worker.js — the worker half

+

It hosts the authoritative App. Everything it decides is decided in Rust — the file only moves strings between self.onmessage/self.postMessage and WorkerServer::on_message, which is the same shape web/app.js has on the main thread and for the same reason: the less JavaScript there is, the less can be silently wrong.

+

Two rules it must not break:

+
    +
  1. Post every reply, in the order the Rust returned them, one message per frame. A GridUpdate::Cells that overtakes the labels framing it is DROPPED by the client's fold, not replayed. Never batch two calls' replies together and never reorder them.
  2. +
  3. Await flush() after each message. Storage::delete is visible to read/list immediately but only reaches OPFS there.
  4. +
+

Rule 1 is about frames within one call's replies AND about the calls themselves, which is why every message is chained onto one tail promise rather than handled by an async listener: an async listener does not serialize. The event loop is free to start the next message while an earlier one is suspended at an await, so two on_message/flush pairs interleave and a later frame's replies can be posted before an earlier frame's. The chain is also what makes "frames that arrived before start() finished" an ordinary case of the same rule instead of a second code path with its own drain loop — and that loop had the race in it twice over, since server was set before it ran.

+

The chaining itself is web/serial.js::serialize, which is promise plumbing and nothing else, so web/tests/serial.test.js can force the interleaving a browser cannot be asked to produce on demand — including the negative arm, which runs the same two handlers without the chain and shows them coming back swapped.

+

The document is named in the worker's own URL — new Worker('worker.js?doc=demo.improv') — the analogue of the socket server naming it in the ws:// path. It is deliberately not a ClientFrame: the envelope pair is shared with the socket deployment and stays exactly as it is (improvise-xks6). The seed is fetched and handed to Rust, which writes it through the VFS and then reads it back like any other document — not injected past storage. A missing seed is not an error: with none, the worker creates an empty model under DOC.

+

An unhandled rejection inside a message listener reaches the devtools console and nothing else, so it is caught there — naming which frame it was, which the main thread cannot see and has no way to ask — rather than left on the chain, where one bad frame would poison every later one.

+
+

#docs/ — the landing page and its casts

+

#The casts are recorded by a script, not by asciinema rec

+

docs/record-casts.py drives improvise on a pty it owns and writes asciicast v2 itself. asciinema rec records a shell, and the shell it records is the recorder's own: every previous cast opened and closed on (0) Mac:<hostname> ~gf/u/edwlan/improvise, on what improvise-kh8 had since made the site root, and formulas.cast and import.cast showed that prompt as their visible opening frame (improvise-2oxt).

+

The repair is structural rather than careful: improvise is the pty's only child, so there is no shell for a prompt to leak from — nothing to remember to clean, and nothing that comes back when somebody re-records on a different machine. Three other things stopped being accidents at the same time. The geometry is an input (120×37, the numbers docs/index.html already spelled, which is why they were not changed — a 30-row player over a 37-row cast clips the tile bar and the status line, exactly where the output is). The pacing is a per-step table rather than typing speed. And the header carries no env and no timestamp: the first held $SHELL, the second the recording machine's clock, and neither is anything a visitor is owed.

+

Two decisions inside the scripts that are not obvious and cost a re-record each:

+ +

That last point is the method, and it is reusable: a cast can be replayed through a small ANSI reconstruction and read as text, so "what does frame N actually show" is a question with an answer rather than a guess. It is how the +cat and + columns in drill.cast — improvise-2oxt's substantive complaint, since the old recording predated both — were confirmed present.

+

What the harness can check, and does: landing.spec.js asserts over the cast files that no header carries env or timestamp, that the geometry is the one the page configures, and that no chunk contains an OSC window-title sequence, which is the leak's signature — improvise emits none and a shell emits one per prompt, carrying the cwd. Asserted over the file rather than the rendering because that is where the leak lives: a player draws a prompt as ordinary text, and a screenshot assertion would have to know what the hostname had been.

+
+

#harness/ — the browser instrument

+

#Why it exists, and what belongs in it

+

Before harness/ there was no JavaScript test infrastructure in the repository at all — no package.json, no runner, nothing in CI running a single line of web/app.js or web/worker.js — so every bug in them was found by reading and the bug-first rule could not be honoured for a change to either. PR #21 declined to add regression tests for its close-listener fix on exactly that ground, and that decline is only honest because those assertions live here instead.

+

Playwright rather than a headless-DOM runner because what goes wrong in these files is a real Worker, a real WebSocket, real OPFS and real event ordering, none of which a DOM shim has. That still holds, and it is why no shim was added; what changed is the scope — see the section below, which is now the answer to "where does a change to web/*.js belong". It is deliberately small (improvise-vvl3): the point is to be able to run storage_contract against OpfsFs and to read #status out of a real page, not to grow a second test framework beside cargo test. The server root is harness/serve, built by build.sh — the assembled site plus the harness's own page; a module script will not load from file://, so an http server is not optional.

+

#Engine coverage

+

improvise-xf1's acceptance criterion names Chrome, Firefox and Safari, and the OPFS spec is the reason: createSyncAccessHandle is the one API the whole design rests on, its sync read/write/truncate signatures were added later than the handle itself, and each engine implemented it separately. Chromium alone would be one engine's opinion.

+

Result today: Chromium and Firefox run the whole battery and pass. Playwright's WebKit provides no OPFS at all — getDirectory() itself rejects with UnknownError — so Safari is NOT covered and xf1's criterion is met for two engines of three. The project is kept rather than deleted so that it starts running the day that build gains OPFS; opfs.spec.js skips on a runtime probe, not on the browser name, so if Chromium or Firefox ever stops providing OPFS it fails rather than skipping. An unexpected skip on any other engine is a failure — a silently skipped check is exactly what this harness exists to prevent.

+

The shell spec's mocked-peer block is Chromium-only on purpose: it is about web/app.js's own logic, which is engine-independent, and routeWebSocket is slower and flakier to no benefit across three. That argument is about those four tests and was never made about the two beside them: the standalone block mocks nothing, it is the deployment improvise-bck's criterion (1) names in so many words ("Chrome and Firefox"), and the engine's OPFS is already exercised by opfs.spec.js, so the path is known to be there. It runs on Firefox too since improvise-zkkw — testMatch widened, grepInvert: /@mocked-peer/ — measured at 2.0s for the pair and 10/10 green under --repeat-each=5. The split is a tag rather than a third copy of the file, and widening the whole file and then quieting what flaked would be adopting a reason nobody gave.

+

Every other spec is Chromium-only, including the three added by improvise-plcx, 78v6 and bgtf: a fixture document, a help overlay and a landing page decide nothing per engine.

+

#harness/opfs-contract

+

A harness crate, not part of the application: it ships in no bundle and is not a workspace member. OpfsFs had been compile-checked and never run; this is the instrument that changed that (improvise-vvl3, absorbing improvise-xf1).

+

Why a worker. FileSystemFileHandle.createSyncAccessHandle() exists only in a dedicated Web Worker, and it is the whole reason Storage could stay synchronous. A check that ran on the main thread would fail for a reason that has nothing to do with OpfsFs.

+

Why a report rather than a panic. Every check runs and its outcome is reported, so a failure names which step and the rest still run. More importantly it is what lets the spec check the checker: Playwright asserts the report contains exactly the checks it expects, so a worker that never started, or a build that dropped a check, is a failure rather than a silent pass. The list of expected check names is spelled in opfs.spec.js as well as in Rust on purpose: a check deleted on one side has to be deleted on both, which is what stops the battery quietly shrinking.

+

wipe uses only the backend's own vocabulary — refresh, delete, commit_deletes — rather than reaching for removeEntry directly, so a broken delete shows up there as well as in the contract. It therefore cannot clear a zero-byte entry, because list no longer names one. That is deliberate and costs nothing: attach recreates and write fills whatever a previous run left behind, and a leftover empty entry is invisible to every assertion for the same reason it is invisible to a document picker.

+

opfs_root/make_directory exist only in the harness: OpfsFs has no directory vocabulary at all — it stores one flat entry per document — so making a directory, to check that refresh skips one, has to happen outside it.

+
+

#Where a change to web/*.js is tested

+

E2E and integration tests are rare and pin boundary contracts; unit tests assume those contracts and test between them. (User ruling, 2026-09-20.) So the question is not "can this assertion be made cheaper" but "which boundary is it about", and the browser half has seven:

+
boundarythe contractwhere it is pinned
RenderModel → DOMevery field the render model projects reaches an elementweb/tests/render_surface.test.js (statically) and harness/tests/contract.spec.js (on the screen)
JS → wasmthe calls the page makes exist, with the arity the bundle declares, and their answers are paintedweb/tests/bundle/wasm_surface.test.js (the surface) and contract.spec.js's pointer test (the wiring)
page → peerthe handshake goes out first, a frame is answered whole, a failure is named, and the page routes each failure through the call that disarms the watchdogharness/tests/shell.spec.js, harness/tests/drill.spec.js
key → screena key the mode's hint line ADVERTISES produces something the user can see, and a text-entry mode paints what is typed into itharness/tests/keys.spec.js
help::pages() → DOMthe overlay's rows are the shared content — the keys off the keymap and the words beside them — and not the right number of blank rowsharness/tests/help.spec.js
worker → OPFSstorage_contract, run against a real OpfsFsharness/tests/opfs.spec.js
AxisFrame → DOMthe grid's frame — header levels, group spans, the collapse indicator and the group row that carries no cells — is drawn as the frame says, not slash-joinedharness/tests/grouped.spec.js, over harness/fixtures/grouped.improv; web/tests/grid_plan.test.js pins the plan it draws from
the site → the appthe landing page's casts are really fetched and really play, and its front door reaches ./app/harness/tests/landing.spec.js
+

A unit test of a rule does not test the wiring that reaches it, and the sharpest case in this file is the watchdog. shell.test.js proves that createShell.fail() disarms rather than pre-empts; it cannot prove that app.js's socket close and error listeners call fail() rather than setShell(). That routing is PR #21's other half, the one gotcha 48 says keeps getting lost — and PR #61 lost it a third time by keeping the unit test and dropping the browser assertion, with 26/26 and 13/13 green. Restored, and the rule generalises: when a unit test replaces a boundary test, ask which of the two the regression would have been in. Here it is the routing, so the guard stays in Playwright.

+

The faked clock is faithful outside the window and silent inside it. shell.spec.js walks past the 8s handshake watchdog with page.clock, and the question that owes an answer is whether collapsing ten seconds into one synchronous step flattens an ordering rather than merely skipping the wait. Probed for the sharpest case — a peer that opens, stays silent past the window and is closed afterwards — the #status transcript is byte-identical under the fake clock and under a real nine-second wait: four states, same order, same text. What that does not cover is an event landing strictly inside the collapsed window, which a synchronous runFor gives no opportunity to deliver; that is a race whose outcome the page does not promise, so there is nothing to assert. State it rather than re-derive it.

+

A negative assertion must demonstrate the denied event is reachable on that exact code path — not on a sibling path, and not in the rule's own unit test. This file's worked example cost four review rounds. shell.arm runs only after await transport.ready, so on a socket that never opens the watchdog is never armed; a test denying "sent no document" there denies nothing, and passed against both the routing revert and the disarm regression. outlastTheWatchdog now asks the page timesArmed() > 0 first, which turns that review finding into a failure by name. Note ever armed and not armed now: a correct page has already disarmed by the time the disconnect test jumps — that is the property — so the obvious spelling would have denied the passing case.

+

And a constant a test must exceed has one home. HANDSHAKE_TIMEOUT_MS lived in web/app.js and again in the spec, under a comment saying a silent disagreement "would pass for the wrong reason". It did: the spec at 1000 against the page's 8000 made the suite green over the reverted routing, because the jump landed inside the window. It is exported from web/shell.js and imported by both, so the disagreement is unconstructible rather than discouraged.

+

Everything between those seams is node --test web/tests, and it may assume the boundary holds rather than re-establishing it: transport selection from ?ws=, the two facts #status composes, the watchdog's arm and disarm, the frameCost/extent arithmetic, the collapse glyph, the grid's plan, and the worker's tail-promise ordering. None of it needs a DOM, so none of it is Playwright's.

+

The seam that makes this possible is web/shell.js and web/serial.js: pure modules that return a description rather than take a DOM double. gridPlan(model) answers every question gridTable used to answer while it was creating elements — which row index a click carries when group headers are interleaved, what a group header spans, whether a null group label is a name, whether an empty edit buffer means an editor is open — and app.js is then a loop that creates elements from it. Painting stays untested by Node; the decision about what to paint stopped being untestable.

+

#Why the four widgets were one bug

+

The tile bar, the view list, the formula panel and the category sidebar ignoring category_panel_open were reported as four broken features and fixed as four. They were one failure at one boundary: a RenderModel field with no element painting it. Four per-feature specs cost four page loads and catch the fifth only if somebody writes a fifth spec; one assertion that every projected field is painted catches it by construction. That assertion is cheap enough to make twice, from different sides:

+ +

And the static half is weaker than it reads, which is worth knowing before leaning on it. render_surface.test.js asks whether the string model.<field> occurs anywhere in the page's code — so a field mentioned by a function nothing ever calls passes. That happened on improvise-fdt6: the new minibuffer field had a paintMinibuffer reading it, and the call site and the getElementById were both missing, so the node suite went green over a page that still painted nothing. keys.spec.js failed on its first run, in Chromium, naming the hidden element. The static check is a completeness check — it tells you a field was not forgotten — and never an execution check; only harness/ is that (gotcha 48).

+

What is deliberately not in Playwright any more is what each key or click means: improvise_protocol::resolve decides that and its tests are where the meaning is pinned. One instance of "a resolved change repaints" is kept, because the repaint is the boundary; the other seven were the same instance again.

+

#Two numbers, and neither is the other

+

Measured on one machine, in one session, on a private IMPROVISE_HARNESS_PORT, with both trees clean — 1af51c9 reached by a full checkout rather than by restoring paths over a working branch (see the trap below). Two runs each; the pair is the variance.

+
suitechromiumall projectsexecutionwhole job
Playwright, before (1af51c9)2326 passed + 3 skipped36.2s / 36.3s~4m30s in CI
Playwright, after1417 passed + 3 skipped15.1s / 14.9s~4m30s in CI
Playwright, + wave 42126 passed + 3 skipped34.4s / 33.7s~4m30s in CI
node --test web/tests—260.12sseconds
+

The last row is improvise-78v6/igxp/plcx/bgtf/zkkw: seven more Chromium tests and two Firefox ones. Most of the 19 seconds is one test — the landing page's four asciinema casts are played until each terminal draws, which is the only assertion that can tell a cast from a 404 (evidence file). The Firefox pair is 2.0s of it.

+

Two columns because one number was ambiguous three times. Firefox runs opfs.spec.js and the standalone half of shell.spec.js and every other spec is Chromium-only, so a count taken off a run is five larger than the chromium count, with three more skipped (WebKit's OPFS battery). The columns must also agree: chromium + 3 is the passed total, and a row where they do not is a row built from two different trees. npx playwright test --list prints the per-project truth; quote which one, and this is the same "two numbers, not one" the Rust suite learned in design-principles.md §7.

+

The trap that made the first before figure wrong. It was taken by git checkout 1af51c9 -- harness web inside a working branch — which restores tracked files and leaves untracked ones where they are, so the branch's own contract.spec.js stayed and the "baseline" ran main's 23 tests plus three of the branch's. A baseline taken by checking paths out over your own work measures a tree that never existed. The full checkout above is the repair.

+

What gave it away was the count, not the clock — and that is the lesson. The contaminated run read 29 passed where a clean 1af51c9 reads 26, because three extra tests really were there; but it read 36.9s against the clean tree's 36.2s, because those three cost under a second between them. A wall clock absorbs a contaminated tree without a trace; an arithmetic identity does not. The same error survived three separate corrections — it reached PR #61's body as "26 chromium" (23 + the 3), survived a fix that corrected the chromium column and left the run count beside it, and was caught each time only by someone checking that the columns add up.

+

Where the saving actually came from, since the obvious answer is wrong. The eleven per-widget tests that were deleted cost 2.7 seconds of test time between them — restored verbatim and re-measured — against 1.55s for the three contract tests replacing them. Those are the per-test sums, not the two wall clocks, which read 3.3s and 3.4s and flattered the replacements: they were separate npx playwright test invocations, so each paid its own cold start (~0.6s and ~1.9s) and a near-tie appeared between two things not measured the same way. Warm inside a full suite those three cost under a second, which is why the contaminated baseline's clock barely moved.

+

Eleven tests at 2.7s cannot buy a twenty-second saving, and they did not. Essentially all of 36.2s → 15s was two shell.spec.js tests that waited out the 8s watchdog in real time, and one of those two was the routing guard this section explains should never have gone. So the deletion of the eleven bought shape, not speed; the speed came from a deletion that cost coverage. Both are back at ~75ms each, because page.clock fakes the page's timers — the honest version of the trade: the wait was the cost, not the test.

+

Cutting Playwright tests barely moves the CI number in any case, because roughly four minutes of that job is three wasm-pack builds, npm ci and a browser download — not test execution. The feedback-time win is the js job, which needs no Rust toolchain and no wasm bundle at all, and that is a constraint rather than an accident: the moment a web/tests/*.test.js file needs web/pkg/, it inherits the four-minute build and the argument for it evaporates. That is why the one test that does need the bundle lives in web/tests/bundle/ and runs in the browser job instead.

+

Until improvise-4ktt the mechanism was a directory convention and a glob, and nothing checked it. bundle_freedom.test.js does now: it reads its own sibling files and refuses one that names the bundle directory, and its companion asserts web/tests/bundle/ still holds a test that does — a scoping rule is uninteresting if nothing falls outside it. Evidence in context/test-evidence/tooling.md.

+
+
+ + diff --git a/site/docs/code/command.html b/site/docs/code/command.html new file mode 100644 index 00000000..72f49689 --- /dev/null +++ b/site/docs/code/command.html @@ -0,0 +1,622 @@ + + + + + +improvise-command — improvise docs + + + +
+ +
+

Source: context/code/command.md

On this page

improvise-command — the arguments

+

The reasoning that used to live in this crate's comments (improvise-9c8c). Read it with the code open; it is written for someone about to change one of these files, not as an overview. context/repo-map.md is the "where", context/design-principles.md is the "why in general"; this is the "why here".

+

Test bug-narratives live in context/test-evidence/command.md, keyed by test name.

+
+

#ui/reduce.rs — the one door from a command to the authoritative state

+

#What the module is (improvise-gxi, corrected by improvise-s17o)

+

This module used to say that every deployment mode — the native TUI, a websocket server, a browser worker — reaches the model through it. That was aspirational, and it is what let improvise-ondj's missing basis check sit unnoticed. The sentence described a door nobody walked: reduce_full had no production caller anywhere in the workspace, so nothing exercised its Command::Model arm and nothing could notice it skipped the check. A door advertised by documentation and used only by its own tests is exactly the shape that hides a defect until the next host arrives — which is the general form, not an anecdote about one bug.

+

So reduce_full is gone, and the argument for deleting rather than wiring it up is in its own last doc comment: "Per-client grid updates need a viewport and come from Sessions::project_all." It returned half of what a host needs. The browser worker was the natural candidate caller and was being built at the time; it reaches the model through Host::handle → Sessions::apply_checked, because that is the half that has the viewport. A function that hands a host half its answer is not a host API, and the test helper that was its only caller now spells the two lines it was.

+

What remains, and what each is for:

+ +

What is not two is the mutation: model_effects maps a ModelCommand back to the effect it was derived from, and both paths run it through App::apply_effects. Effects remain the only place the App is written.

+

apply_command's remaining arms no longer fold. They used to route a wire view command into app.view_state — the authoritative slice — which nothing in production did and nothing should: Sessions::apply_checked folds it into that client's mirror, because the model is shared and the UI is not (principle 5, gotcha 36). A client's SetStatus moving the host's status line is not a feature the module was missing a caller for; it is behaviour that was safe only because the arm was dead. a_view_slice_command_does_not_move_the_authoritative_view_state now asserts the inverse of what the old test asserted.

+

A wire command arrives here through Checked, and nowhere else (improvise-ondj, ADR 001). The basis check is a property of the session door rather than of the mutation path — a host holding the App is the truth and not a claimant — but "the mutation path does not check" was implemented as a second door rather than as an exemption: both functions used to take a &Command and their Command::Model { cmd, .. } arm discarded the basis with .., which is the syntax for ignoring a field and costs no compile error. So the answer is not a guard but a type: the only way to obtain a Checked over a Command::Model is Checked::against, which runs stale_basis against the authoritative App. A host that wants to apply a client's batch must face the Rejection, and Sessions::apply_checked is the door a host goes through.

+

Host::commands answers the whole frame first with CheckedBatch::against, so Sessions::apply — which answers one command and then dispatches — is not a door a host may use: checking per command is the lost update improvise-955n fixed. It is now #[cfg(test)] (improvise-s17o), which makes "no production caller" a fact the compiler keeps rather than a note verified at one commit, and leaves apply_checked as the one door.

+

#Checked — a wire Command the authoritative state has answered for

+

The one thing it makes unrepresentable: a Command::Model applied without its basis being checked (improvise-ondj). The inner field is private and Checked::against is the only constructor, so there is no spelling of "apply this batch and ignore what it claims" — the .. that used to discard the basis is now a Rejection the caller must handle.

+

It is not a claim about which App was asked: a host with two documents could check against one and apply to the other. That is not a hole this closes, because a Sessions is already bound to one document's generation sequence (ADR 001) and a Registry hands out one pair at a time.

+

#Why the lifetime does not close it (improvise-7v62)

+

The obvious answer — make the lifetime the &App's, so the borrow checker forbids checking one document and applying to another — was tried, and it is not merely inconvenient: it is contradictory with what the type is for. A Checked is a proof discharged before a mutation, and its whole job is to be carried across that mutation; a borrow of the App is a claim that must stay live during it. The two cannot both hold. Tying them makes every call site an E0502, starting with the canonical one:

+
error[E0502]: cannot borrow `*app` as mutable because it is also borrowed as immutable
+    match Checked::against(app, cmd) {
+                           --- immutable borrow occurs here
+        Ok(checked) => self.apply_checked(app, id, checked),
+                       ^^^^^ mutable borrow occurs here
+

A generative brand would express it without borrowing, but that needs a closure-scoped API and would distort every host. The remaining option is carrying a document identity, and DocumentId lives in the server's registry: an App does not have one and does not want one, precisely because the unit of authority is one document per App. So this is a deliberate won't-fix, not an oversight. What keeps it unreachable is stated above and is load-bearing — improvise-j31x made this a multi-document server on purpose, and ADR 001 never evicts a document so that a reconnecting client cannot have a stale basis accepted as current.

+

#CheckedBatch — a whole frame, answered before any of it is applied

+

(improvise-955n / improvise-08wb, ADR 001.)

+

Checked alone was not enough, and the gap was exactly one level up from the one improvise-ondj closed. A Checked answers for one command against the App as it stands; a frame is a sequence of them, applied in order, so a frame whose second command was refused had already applied its first. ADR 001 says a stale batch is never applied, not even partially, and that was true of a single command and false of a batch.

+

#One basis, checked once, and then the chain runs

+

The contract, as the user stated it: "a sequence of commands sent to the server should assume that it's played in order from start to end and that each command sees the state after the previous command runs", sent as "a single transaction from a given base". Four clauses:

+
    +
  1. A frame states one basis, describing the base the client composed against.
  2. +
  3. That basis is checked once, against that base, before anything is applied.
  4. +
  5. Commands are applied in order, and command N sees the state command N-1 left. Data dependence between them is expected, not an error.
  6. +
  7. All or nothing. If any command fails, the frame leaves the document exactly as the base described.
  8. +
+

Clauses 1–3 are implemented. Clause 4 is not — see the hole below and the design note on PR #26.

+

#Two of the three tiers hoist, and the third does not

+

Clause 2 hoists the check, but not every tier is a frame-level fact, and a first version hoisted all three. That was a lost update: document_generation and layout_version describe the document, so answering them once for the frame is exactly right — but the record tier asks "is the client's picture of this record current", and which records are at stake is a property of the command. That is why records_named is an exhaustive match over ModelCommand and not something a frame could summarise (gotcha 43).

+

Skipping it for later commands let [SetCell(North), SetCell(South)] overwrite a South that somebody else had moved, silently, with the frame reported accepted — the precise loss ADR 001's record tier exists to prevent, and invisible to the other two tiers because a SetCell is CellData and moves neither of them. So every mutation answers for its own records, against the frame's basis: the one the first basis-carrying command stated, not each command's own field, so a later command cannot state a fresher claim than the frame was composed under.

+
#Except the records the frame is creating
+

The argument that motivated clause 3 is true, and it is true of a narrower thing than it was first applied to. A client cannot claim a counter for a record its own chain is about to create, because BasisCache::claim filters to records the cache has been told about — so [AddItem(East), SetCell(East)] would be refused by a naive per-command check however it is ordered. But that argument was then stretched to records the frame merely writes, and those are the ones the tier is for.

+

So the exemption is stated over the property rather than over a list of commands: a record that neither resolves in the document nor is claimed by the basis is one the frame is about to create, and is not answered for. Every other combination still is — including a record that does not resolve but is claimed, which is a client believing in something that has been removed, and is RecordGone as before. See will_be_created_by_the_frame.

+

What the base means is therefore: what the client saw. The transaction promises the whole chain lands on top of it or none of it does. The chain's internal coherence is the transaction's job, not the basis check's.

+

#The hole clause 4 leaves, stated rather than hidden

+

App::apply_effects aborts without undoing (gotcha 21 — an abort is not a rollback), so a frame whose second mutation fails leaves the first applied. [RemoveItem("North"), SetCell(the North record)] is the example: the removal lands, the write then resolves to nothing and model_effects returns an empty batch, so the frame is reported accepted having applied half of itself. Both halves of that are open — the partial application and the silence.

+

The create exemption has a residue of the same kind and it is the same hole: a later command naming a record that is neither present nor claimed is presumed to be one the frame creates, so if the frame does not in fact create it, the write resolves to nothing and silently does not happen. That is model_effects' empty batch again, not a second defect, and it closes when clause 4 makes a mid-chain failure reportable.

+

Both are unreachable from any client in this workspace: the only producer of a multi-command frame is improvise-wasm-client::keys::commit, which emits one mutation among view commands. Reachability is a reason not to rush a fix and not a reason not to notice one — the lost update above had the same reachability argument as this hole and none of its documentation, which is how it shipped. That is what makes it safe to document rather than rush, and it is why an earlier "at most one document mutation per frame" rule — which closed the hole by forbidding the capability — was rejected.

+

#Two things the rule that was rejected got wrong, worth keeping

+

Arity is not a property of a variant. A future Command::RunScript is one variant performing an unbounded number of mutations, so no count over the vocabulary can be right: a classifier can honestly answer whether a command changes the document, never how many times. A mutates_the_document predicate written for the rejected rule is deleted with it rather than kept unused — but it was made an exhaustive match on review, because a matches! answers false for a future variant in silence, and clause 4 will need that classifier back. Write it exhaustive when it returns.

+

The transaction boundary is not the frame. Because one command may already be a chain, clause 4 is a property the system needs inside a command as much as across a frame of them, which is why the design note puts the mechanism at App::apply_effects rather than at the batch.

+

#CheckedBatch::against

+

Answer for every command in a frame, against the state the frame was composed against — which is the state before any of them runs. The refusal is the frame's answer and nothing is applied. It is a Result rather than a per-command verdict for the same reason Checked is a type rather than a guard: the only enforceable version of "answer before applying" is not being able to reach the commands without having done so.

+

The two document tiers are answered once, on the first command that carries a basis — clause 2 — and every later command is admitted without re-asking them, which is clause 3. The record tier is asked of every mutation, against that same frame basis, because it is not a frame-level fact. A frame with no mutation answers trivially.

+

The Basis on a later Command::Model is ignored rather than compared, and that is an honest interim rather than the end state: clause 1 says the frame states one basis, but the wire still carries a Basis inside each Command::Model — right when one command was one transaction, and now one field per command describing something that belongs to the frame. Moving it to ClientFrame::Commands would make "one basis per frame" unrepresentable-otherwise instead of a convention, and that protocol change belongs with clause 4.

+

Inside the loop, frame_basis is the one its first mutation stated; later commands' own Basis fields are deliberately not read, for the reason above.

+

#Checked::against

+

Total over the vocabulary, and only Command::Model can be refused: everything else is either this client's own view slice or a SwitchView, which carries no basis and cannot corrupt — the workbook simply refuses a view it does not have (ADR 001 names that second hole and leaves it open deliberately).

+

Saving is not in this vocabulary and deliberately so: it is ClientFrame::Save, answered by the host, because it mutates no shared state and has no basis to answer for (improvise-368h).

+

There is no argument that suppresses the answer. ADR 001's force is a fresh batch under a fresh basis, composed by the client; it is not a way past this constructor.

+

#Checked::within_a_frame

+

A command in a frame whose document tiers have already been answered (clause 2), answering only for its own records. The two document tiers are not re-asked, which is clause 3: they describe a document this command is not running against, since its predecessors have moved it. The record tier is asked, because it is a fact about the command rather than the frame.

+

Private, and that is what keeps it from being a bypass: from outside the module the only doors are still Checked::against and CheckedBatch::against, and the only caller is the loop that has just answered the frame's basis.

+

#Checked::command

+

Hands back a &'a Command rather than one borrowed from &self, so a caller can dispatch on the variant and still move the proof into the function it dispatched to. Crate-private: a proof that could be unwrapped from outside would be a proof nobody had to obtain.

+

#apply_command

+

Command::SwitchView does not go through reduce_view alone, but not for the reason it once did. Since improvise-jytm the document's copy is the default — what a fresh client opens — and the client's slice is authoritative, so a switch moves no document state at all. What it still has that reduce_view has not is the view history: effect::SwitchView pushes the frame effect::ViewBack pops, and that stack is this client's. Everything else is pure view slice.

+

The SwitchView arm specifically: the mutation is effect::SwitchView's, not a second copy of it. The effect records the view history the key path leaves behind, and effect::ViewBack pops exactly that stack. Switching by calling App::switch_to_view here instead left the stack empty, so a wire client could not go back where a keyboard one could — see a_wire_view_switch_records_history_the_way_a_key_switch_does. apply_effects rebuilds the layout.

+

The Dirty that arm returns is not Dirty::of of anything, and since improvise-jytm that is the point rather than a narrowing. effect::SwitchView is ViewOnly, so its Dirty is NOTHING; what this arm returns is what the host's own widgets must repaint, which is a different question from what the document changed. The view list is left out for the old reason (improvise-pca): its names and axes belong to the document, and which view is active is not in ViewListCache at all. A connected client never reaches this arm — Sessions::apply_switch moves that client's mirror and ClientSession::project observes the move.

+

An inbound Command::Project is ignored. Projections travel server→client; a client that sends one is describing state it does not own.

+

The Command::Model arm needs no basis check of its own: the basis was answered for by Checked::against, and there is no other way to have got here with that variant.

+

#model_effects

+

The other half of the derivation: a ModelCommand back to the effect it was derived from (improvise-kmgs). This is the seam that keeps a command from becoming a second way to mutate. A ModelCommand names a mutation; it does not perform one. The effect it names is the same value the keymap path builds, and it is run through the same App::apply_effects, so both doors go through the one mutation site and both get their Dirty from the effect's own EffectKind rather than from a second classification that could drift.

+

The match is exhaustive on purpose: a new ModelCommand variant fails the build here until it names an effect, exactly as a new effect fails the build until it classifies itself (gotcha 3).

+

It takes the App because two variants name their operand by surrogate rather than by coordinates (ADR 002), and resolving one is a lookup against the authoritative state. Resolution failing is not an error path bolted on: it is one of the two arms of staleness — the record is gone, a category or item it named was removed — and the answer is an empty batch, which applies nothing. Sessions::apply reaches that same conclusion first and typed, as Rejection::RecordGone; this stays as the floor under the key path and under any host that bypasses the session door.

+

#records_named — which records a ModelCommand writes

+

Decided by the server (ADR 001, improvise-fkfg). This is deliberately not "whichever records the client's basis mentions". A client that simply left a record out of its claim would otherwise have a force by omission — the record tier skipped, with no flag to remove and nothing in the message saying so — and force is supposed to be a re-composition against the current basis, never a bypass of the check. The server decides what is checked; the client only supplies what it saw.

+

The match is exhaustive, and the long list of "names no record" is the point rather than noise: a new variant that writes a record fails the build here until it says so. That matters because the failure mode is silent — an unlisted record-writing variant would simply never be checked against its own tier, and the suite would stay green.

+

Only CellData variants that carry a RecordCell name a record. The item and formula commands are CellData too but name no record: they move the axes a record's coordinates are drawn from, which is what the layout tier is for.

+

#stale_basis

+

None means apply it. The three tiers are checked in order of how much they invalidate, so the answer a client is given names the largest thing that moved rather than an arbitrary one: a document swap is not usefully reported as "a record moved".

+

The record tier separates structural from data staleness, and keeping them apart is the whole reason ADR 002 put a maintained RecordHash index in DataStore rather than regenerating one from the layout. A record that does not resolve is gone — there is nothing to write to and nothing coherent to force, because force never creates structure. A record that resolves at a different version merely changed, and force there is one keystroke.

+

Nothing in it mutates, and there is no argument that makes it return None regardless: a stale batch is never applied, and ADR 001's force is a fresh batch under a fresh basis rather than a way past this function.

+

#stale_records_in_a_frame

+

The record tier for a command inside a frame whose document tiers are already answered. It reuses stale_record unchanged — one home for what each of the four combinations means — and differs only in which records it asks about: the ones the frame is about to create are skipped.

+

#will_be_created_by_the_frame

+

A property, not a list of commands. The document does not have it and the basis claims nothing about it — which is precisely what "I am creating this" looks like from here, because BasisCache::claim can only state a counter for a record the client has been told about.

+

The near miss is the other half of that conjunction: a record that does not resolve but is claimed is a client believing in something that has been removed, and stays Rejection::RecordGone. Only the pair exempts.

+

It is asked only inside a frame, never of a lone command: a first command has no predecessor that could have created anything, so Checked::against still refuses a write to a record that is not there.

+

#caught_up

+

The command a client that is fully caught up would compose. It reads the tiers straight off the App, which looks like forging and is not: it takes &App, so only the side that is the authoritative state can call it, and for that side "the current basis" is not a claim to be checked but a fact. A remote client has no App and no other constructor — BasisCache::claim withholds until the server has spoken — so nothing reachable from a wire can get here.

+

It exists for two callers: a host that resolves a command locally and wants to push it through the same door a client's would take, and the tests whose subject is not staleness. The tests whose subject is staleness build the Basis by hand, so neither kind can pass by accident through the other's convenience.

+

#reduce_key

+

Resolve a keypress through the keymap, run the commands, apply the effects, and report what to send onward. The returned commands are the document-scoped projections; per-client grid updates need a session's viewport and come from Sessions::project_all, driven by the Dirty that apply_command returns.

+

It used to have a wire-command twin, reduce_full, described right here as "the signature the three hosts share". None of them shared it; the module section above says why it is gone (improvise-s17o).

+
+

#ui/effect.rs — the only mutation site

+

#EffectKind has no default, deliberately (improvise-mae, gotcha 3)

+

EffectKind::ViewOnly changes only the acting client's own view — cursor, scroll, mode, panels, minibuffer text, drill staging, status line — and is never projected. EffectKind::Model(ModelChange) changes something every client renders.

+

Cursor and scroll live in the shared View (they are persisted in .improv), but they are a per-client concern: one client scrolling must not move another's viewport. See improvise-8uz.

+

Effect::kind deliberately has no default: a new effect that forgot to classify itself would silently never reach connected clients, so the compiler asks. The three traps a classifier walks into are gotcha 3's and are not restated here.

+

#Effect::apply stays on &mut App — declined, with counts (improvise-drg)

+

improvise-drg proposed narrowing each apply to the slice it touches: &mut ModelState or &mut ViewState where an effect only needs one, &mut App for the rest. The win it claimed is type-level documentation and finer-grained tests, explicitly not line count. Declined, on four grounds.

+

The narrow set is shrinking, for an architectural reason. Counted over all 57 effects — 49 impl Effect blocks plus the 8 simple_effect! invocations, which also expand to one — 16 touch only view_state, 9 only model_state, and 32 cross-cut. At the wave's base commit it was 16 / 16 / 25, so the narrow set went from 32 to 25, and the seven movers are SetAxis, HideItem, ShowItem, ToggleGroup, TogglePruneEmpty, RemoveItem and RemoveCategory — every one of them an effect improvise-9mo9 gave a reset_cursor.

+

These are the corrected figures, and the first ones were wrong twice. They said "25 to 17 of 49": they omitted the simple_effect! invocations, all of which are view-only, and they misclassified AddItem at the base commit because the extraction regex could not see app followed by a newline and .model_state. That is doctrine(derive_then_verify) catching the author of this paragraph with the exact failure it was written about — the kb entry names multi-line method chains and three effects they had already mis-sorted. The correction weakens this bullet and is recorded rather than quietly restated: a narrow set of 25 of 57 is 44%, not the 35% first claimed, and the fall is 22% rather than 32%. The direction survives; the headline was overstated.

+

The reason the direction is what matters: a grid reshape lives in model_state.workbook and the cursor it invalidates lives in ViewState, so the effects that matter most are exactly the ones that reconcile the two. Gotcha 3 already records the same asymmetry from the other side — the View lives inside model_state.workbook, so touching model_state does not mean being a model change. The App is where the slices meet, and narrowing bets against the direction the code moves in.

+

The type-level documentation exists and is checked. Effect::kind() has no default, so the compiler demands the classification (gotcha 3), and apply_effects reads that one answer for the Dirty, and for the CellData and WholeWorkbook half of model_state.dirty (gotchas 40, 45; the shape tiers are observed rather than declared — see apply_effects below). A narrowed signature would be a second statement of what an effect touches, free to disagree with the first. Two answers to one question is the defect this repo names, not the cure.

+

Every mechanism the ticket offered weakens a guard. Its (A), three traits, needs Vec<Box<dyn Effect>> to become three queues or an enum wrapper: three queues destroy batch order, which gotcha 21's abort, gotcha 23's stale layout and gotcha 43's frame contract all lean on, and an enum wrapper is a fourth classification beside EffectKind. Its (C), default methods falling through to apply(&mut App), gives apply a default — the precise thing kind() does without, and for the precise reason: an effect that overrode apply_view but needed the model would compile and write the wrong slice silently, and one that overrode neither would compile as a no-op. Its (B) rebinds inside the body and does not meet the ticket's own criterion.

+

Finer-grained tests are not blocked today. effect.rs's 64 tests already call Effect.apply(&mut app) against an App::new(wb, None) built in a line; narrowing the signature would not shrink that fixture.

+

The 17 that could narrow may still be worth reading as narrow, which is approach (B) — rebinding the slice at the top of the body. That is a style choice inside a function, not a trait change, and it is not being mandated.

+

#simple_effect! — the kind argument is mandatory (improvise-8fy)

+

Declare a setter effect: a one-field newtype whose apply writes that field into one place on the App.

+

The kind is a mandatory argument, even though every call site so far answers ViewOnly. Hard-coding it reintroduced exactly the default that Effect::kind deliberately does without (gotcha 3): $place is an unconstrained expression, so

+
simple_effect!(X(String) => |app| app.model_state.workbook.active_view_mut().number_format, …)
+

is a textbook LayoutGeometry write that the macro would have reported as ViewOnly, silently, with the whole suite green. Writing the answer per call site is the same reasoning that gave kind() no default.

+

Within that, the macro is still the single hook for anything these setters should all do — today none of them marks the document dirty or rebuilds the layout, and the macro makes that one decision instead of fifteen silent ones.

+

Clone::clone rather than .clone(): the field is usize or bool as often as it is a String, and the call form keeps clippy::clone_on_copy off the generated code.

+

#AddCategory aborts the batch on refusal

+

A refusal aborts the batch, for the same reason a failed save does (gotcha 21): what follows assumes the category exists. From the +cat column (improvise-2zq) the rest of the batch would advance the cursor and re-open an editor on a column that was never created — and re-opening the minibuffer is also what would hide this message, since draw_bottom_bar renders the minibuffer instead of the status line.

+

MAX_CATEGORIES is a reachable refusal on that path rather than a theoretical one, and the limit is checked in exactly one place — Model::add_category — so nothing in the effect restates it.

+

#AddFormula and non-_Measure targets

+

For non-_Measure targets the item is added to the category so it appears in the grid. _Measure targets are dynamically included via Model::measure_item_names() (gotcha 7).

+

#RebuildLayout (improvise-ehz)

+

apply_effects rebuilds the layout once after the whole batch, so any effect that must read the layout after an earlier effect changed it needs a RebuildLayout in front of it. That is gotcha 23; the command decides the order.

+

#EnterEditAtCursor — the mode is the caller's, never app.view_state.mode

+

Re-enter edit mode by reading the cell value at the current cursor; used after commit+advance to continue data entry. It reads the current layout, so emit RebuildLayout first when an earlier effect in the batch moved the cursor or changed the grid.

+

target_mode is supplied by the caller (a keymap binding via EnterEditAtCursorCmd, or CommitAndAdvance from its own edit_mode field). The effect itself never inspects app.view_state.mode: the mode is decided statically by whoever invoked it. An earlier version branched on app.view_state.mode.is_records() and got it wrong mid-batch.

+

#Which effects reset the cursor, and why it is decided one at a time

+

Swapping the axes renumbers every row and column, so the old indices name different cells. That reset used to live inside View::transpose_axes; it moved here with the cursor (improvise-8uz, gotcha 24).

+

Nothing declares which effects renumber the grid, deliberately — so for a long while only TransposeAxes and CycleAxis reset, while eight siblings that renumber just as thoroughly did not (improvise-9mo9). SetAxis was the sharp case: CycleAxis is SetAxis to Axis::next(), so one of a pair that differ by an argument reset and the other did not. The gap was invisible in both directions, because reset_cursor is also the only thing that bumps cursor_epoch, which is the only thing Sessions::adopt_cursor_epoch watches: a forgotten reset leaves the host's own cursor stale and tells no client.

+

Ten effects now reset, and the test that holds them there (every_reshaping_model_command_invalidates_every_cursor) asserts of each case that the grid's row_items/col_items really changed before asserting anything about a cursor, so a case that has stopped reshaping fails rather than passing vacuously. The per-effect arguments:

+ +

What does not reset, and why:

+ +

Drilling is the case this ordering has to survive, and it is tested rather than argued (drilling_resets_only_the_drill_views_cursor). DrillIntoCell and ToggleRecordsMode each issue several SetAxis effects, so each now resets several times — but both push SwitchView first, and reset_cursor writes the active view's cursor, which by then is the freshly created _Drill/_Records one. The view drilled from keeps its entry in ViewState::cursors. What is still only argued is the cursor_epoch: a drill bumps it once per SetAxis, Sessions::adopt_cursor_epoch compares before and after per command so it relays once rather than eight times, and DrillIntoCell is reachable only from the key path today — Host::handle answers ClientFrame::Key with an empty vec — so no client sees it at all. That changes when drill becomes addressable.

+

The one caller that depended on a missing reset is CommitAndAdvance's inserts_a_column branch: typing into the +cat column creates a category and then deliberately stays put, so the cursor lands on the column it just created (improvise-2zq). That worked because nothing moved the cursor. Now AddCategory does, so the branch ends with an explicit SetSelected back to where it was — which states the intent instead of inheriting it from an absence, and still sits after AddCategory's abort so a refused category leaves the batch unchanged.

+

#ReturnToNormalUnlessMoved — observation, not prediction (improvise-toc)

+

Return to Normal unless something earlier in this batch already moved the mode away from from — the mode the batch started in.

+

ExecuteCommand used to decide this up front by asking each effect a per-type changes_mode(). That cannot describe ViewBack, ViewForward or StartImportWizard, which swap the mode only on some runs: answering "no" snapped a successful :import straight back out of the wizard, while answering "yes" would have stranded a failed one in CommandMode. Reading the mode here, after the batch has run, needs no prediction. That is gotcha 31.

+

#App::normal_mode — what Normal means on this grid (improvise-sihr)

+

AppMode::Normal and AppMode::RecordsNormal are the same mode over two layouts, and every minibuffer mode opened from a records grid used to exit to the first: the exit was a constant, spelled in TextEntry::new(.., "normal") and again in each commit command, and nothing carried where the user came from. Three arms — :, Ctrl-E and Ctrl-W — and the class predates the last of them, which is why Copilot's finding against SavePrompt alone was declined on PR #27.

+

It is resolved once, at the end of the batch. App::apply_effects calls settle_normal_mode after rebuild_layout, and that is the only caller: Normal and RecordsNormal are one mode over two layouts, so the mode a batch ends in must match the grid it ends on. No effect resolves anything — ChangeMode is an ordinary simple_effect! again, and ReturnToNormalUnlessMoved and CloseWizard write plain Normal. The ticket proposed the opposite — an exit mode beside MinibufferConfig — and it was declined for a reason beyond doctrine: carried generically it also returns CategoryAdd to CategoryPanel, which nobody asked for, and carried for three modes it is the one-arm shape the ticket itself argues against.

+

Resolving per-effect was tried first and was wrong, in a new direction. Set at the moment the mode is written, the resolution reads the batch's opening layout (gotcha 23), so a batch that crosses — :toggle-records-mode . enter-mode normal — settled against the grid it started on and left a records mode over a pivot grid. One edit, after the rebuild, has no crossing case to get wrong, and it also removes the per-edit coverage question three separate writers created.

+

The guard is two-armed and the crossing test could only see one arm (improvise-4dpj). settle_normal_mode matches Normal | RecordsNormal, and replacing that with a one-way Normal left the whole workspace green: the : crossing drives the layout both ways, but both of its branches end in enter-mode normal, so the mode handed to the settle is AppMode::Normal either way. It discriminates "settles at all" and "settles against the ending layout", and not "settles from either mode" — three questions that look like one until an injection separates them. The RecordsNormal arm is live: SetAxis { "_Index", Axis::None } applied from a records grid leaves is_records_mode() false with the mode still RecordsNormal.

+

And records-ness is auto-detected from the axes (gotcha 1), so a view switch crosses it too. SwitchViewAtCursor emits [SwitchView(target), ChangeMode(Normal)], which is a genuinely different path from the toggle the : test drives — nothing in it says "records", the crossing falls out of the target view's axis map. Moving the settle back before rebuild_layout is the injection for both, and it fails each by name.

+

This fixes the TUI and not the browser, and that is LIVE (improvise-e5pf). Command::ChangeMode(AppMode::Normal) is on the wire and reduce.rs folds it as view.mode = mode.clone() with no resolution, so the browser front end still has the improvise-sihr bug. The live browser key resolver is improvise_protocol::resolve — re-exported as improvise_wasm_client::resolve — and it emits that command in five places. An earlier version of this paragraph said the gap was latent because "keys.rs emits no ChangeMode", which is true and irrelevant: keys.rs holds only BrowserKey. Nothing checks a claim in context/, which is exactly why one must not be written from a grep of the wrong file (design-principles §6).

+

It compounds, and that is the part worth knowing: once the client's mode reads plain Normal, its only records-ness fact — state.view.mode.is_records() — is false, so a later commit over a records grid takes the non-records branch.

+

It is still not closeable here. By the time Esc arrives from a panel the client's mode no longer says records, and a client holds a GridCache rather than a GridLayout, so records-ness is not a fact it has at all. Closing it means enriching the projection, the shape gotcha 47 names, never moving resolution back into the host.

+

:toggle-records-mode is not the case that proves any of this, and an earlier version of this paragraph claimed it was. It works, but not for the reason given: ToggleRecordsMode emits change_mode(RecordsNormal) explicitly on entry, and on exit ViewBack restores the mode SwitchView pushed, so any resolution's answer is overwritten in both directions and the case exercises nothing. a_batch_that_leaves_records_and_then_sets_normal_lands_in_normal drives the crossing in both directions instead, because a careful comment standing in for a test is what let the per-effect version ship.

+

#SetBuffer — "search" is not a buffer (improvise-gdl)

+

Every name goes into the buffers map. The search query is not a buffer — it has its own ViewState field and its own SetSearchQuery effect — and commands reach the right one through cmd::core::write_buffer, next to the read_buffer it mirrors.

+

#ApplyAndClearDrill

+

RecordCol::NewMeasure / NewCategory are never staged: commit_cell_value applies a new measure or category immediately, since a schema change is not a cell edit and has nothing to roll back with the drill.

+

There is deliberately no dirty = true in it: it declares Model(CellData) and App::apply_effects raises the flag off that (improvise-gv9r, gotcha 45). The line that used to be there was the last survivor of MarkDirty — harmless, and a second home for the one thing that commit set out to give exactly one.

+

#Save / SaveAs abort the batch (improvise-8tk / improvise-lae)

+

A failed save aborts the rest of the batch, so the Quit that wq/ZZ chains after it never lands and the error stays on screen. SaveAs is the other arm of that rule and was left out of the original fix, so :w /bad/path . q quit on a failed write. Gotcha 21 is the general statement.

+

#WizardOp

+

One discrete mutation of the import wizard's own state. Which op a key means in a given step is decided by HandleWizardKey; the enum is just the verb (improvise-2wm).

+

#AcceptWizardModel

+

Commit the wizard: build a workbook from its proposals and install it. On failure the wizard stays open carrying the error so the user can go back and fix their field choices.

+

#json_import_records / ImportJsonHeadless (improvise-oaq, improvise-k8i)

+

json_import_records extracts the records array from an already-parsed JSON import value. Precedence: explicit array_path → root array → first auto-detected array path. It is pure, which is what lets ImportJsonHeadless parse the file exactly once and reuse the parsed value as the pipeline's raw. CSV warnings (short rows, improvise-k8i) are threaded into the status message rather than written to stderr, which is invisible inside the TUI.

+

#AbortChain

+

Signals App::apply_effects to skip the remaining effects in the batch. The flag is reset at the start of every apply_effects call, so each dispatch starts clean. Use it when a sequence's premise no longer holds — "advance to next cell" at bottom-right — and later effects ("re-enter editing there") should be short-circuited.

+

#CleanEmptyRecords

+

Removes cells whose CellKey has no coordinates. These are meaningless records that can only be produced by AddRecordRow when no page filters are set. Pushed by ToggleRecordsMode when leaving records mode, as the inverse of the SortData that runs on entry.

+
+

#ui/app.rs — the authoritative state, and the three tiers read off one classification

+

#The slice boundary, and the residue

+

App is two named slices plus a residue of fields that belong to neither:

+ +

wizard is session-local on both counts: ImportWizard is an improvise-io type, outside the allowlist a serialized ViewState may draw on, and a client that connects mid-import must not open onto someone else's half-finished dialog. drill_state likewise: it holds an Rc, so ViewState would not even be Send, and pending_edits are uncommitted writes belonging to whoever staged them. Its frozen records list is what records-mode layouts iterate, so records do not disappear when filters would change.

+

storage is where documents are read and written; the host chooses, and every effect that touches a document goes through it rather than std::fs, so the command layer has no filesystem of its own (improvise-fuj, gotcha 2).

+

DrillState::pending_edits is keyed by (record_idx, RecordCol) — the column as a RecordCol rather than an index, so applying them does not depend on the layout still being the drill layout.

+

#record_of and RecordHash — the unit of atomicity

+

record_of is a cell's coordinate tuple with _Measure dropped, re-exported from improvise-core where it lives beside the RecordHash it feeds: the surrogate is defined as the hash of exactly this, so two definitions of "the record" would be two things to keep in step (ADR 002).

+

It is the unit ADR 001 settles on, and the reason it is not the cell is gotcha 1: renaming a category coordinate in wide records mode moves every measure cell in the row, so one legal edit spans a whole record. A unit finer than the coarsest legal write cannot be maintained.

+

#record_fingerprints — a process-local XOR fold

+

A fingerprint per record of the raw cells it holds, keyed by the same surrogate the index and the version counter are keyed by. One u64 folded over each cell's key and value. The fold is XOR, so it does not care what order iter_cells yields — and it cannot cancel a pair against itself, because a key appears at most once in the store.

+

This hasher is not the surrogate's: it is process-local, compared only against a fingerprint taken moments earlier in the same process, and never crosses a wire. RecordHash is the opposite case and is specified accordingly (improvise_core::model::record).

+

The value hash is exhaustive and by bits rather than by Debug: two values that print alike must not fingerprint alike, and it runs once per cell per editing batch.

+

Only raw data is fingerprinted. Formula results live in the model's separate formula_cache, and a record whose derived value moved because someone edited a different record has not itself been written.

+

#document_generation, layout_version, cursor_epoch

+

document_generation — how many times the whole document has been replaced. Bumped by install_workbook and by nothing else, so a client mirror can tell that the document it was attached to is gone without the effect that replaced it having to reach the session list (improvise-cqq). Not part of ViewState: it describes the document's identity over this process's lifetime, not a client's view.

+

layout_version — shared document state: how many times the grid's shape has moved, bumped once per batch that is observed to move it, never off a declaration (ADR 001, improvise-xwi6; improvise-pdvm). The middle tier of the basis: a CellData batch composed before this moved is stale even if no record it names was touched, because the keys it carries hang off a shape that has changed under them.

+

cursor_epoch — shared document state: how many times a shape change has declared a cursor into this document invalid. Bumped by reset_cursor and nothing else (improvise-gvnb). It is a counter rather than a list of the commands that reset the cursor, for the reason gotcha 39 gives for watching document_generation instead of matching on LoadModel: which effects renumber the grid is a per-effect decision (gotcha 24 asks a new geometry effect to make it), so a table of variants here would be a second description of that decision, free to disagree with it. Observing the write is not.

+

#batch_aborted — the outcome, where abort_effects is the mechanism

+

abort_effects is transient by design — it is reset the moment the loop breaks, so nothing outside apply_effects can ever observe it — and that is right for the mechanism but leaves the outcome unreadable. A caller that ran a one-effect batch and needs to know whether that effect refused has no other way to ask: Effect::apply returns () on purpose, so an effect reports by writing the App.

+

Its one consumer is Host::save, where the batch is literally [Save] and so "the batch stopped" and "the save failed" are the same fact (gotcha 21, improvise-368h). Naming the real call site matters because that identity is what makes the flag readable at all: over a longer batch it says that something refused, never which. App::last_batch_aborted is the accessor and carries the same caveat.

+

#App::new recomputes formulas before the first layout

+

The formula cache is rebuilt before building the initial layout so formula-derived values are available on the first frame. The cache is keyed by the active view's None-axis categories, so the caller gathers them explicitly (gotcha 9).

+

#install_workbook

+

Replace the whole document. The client's view slice is scoped to the old one — its cursor map is keyed by that document's view names — so it is reset to the incoming document's active view rather than carried over (improvise-8uz, improvise-cqi).

+

Every connected client's mirror is scoped to the outgoing document in the same way, and the effect that got here cannot reach the session list. Bumping document_generation is how the reset reaches them: ClientSession::project compares it and reseeds.

+

#adopt_cursor_epoch — the epoch is one trigger of two, the view is the audience

+

reset_cursor resets the host's own slice; every connected client owes the same invalidation, and cursor_epoch is how the reset reaches them. The trigger stays observed — which effects renumber the grid is a per-effect decision, so a table of variants here would be a second description of it (gotcha 24).

+

The epoch is the host's own cursor, so it is blind to a view the host is not on, and since improvise-lltt a view-configuration command names its view, so such a write is reachable. The second trigger is App::last_reframed_elsewhere — the non-active views this batch was observed to reframe — and it is a precise set where last_reframed_views is deliberately a union with the reshaped set. Two consequences. A batch that reframes only elsewhere relays against that precise set, because the union would invalidate a client whose view merely reshaped (a page selection that prunes nothing), which paging_with_nothing_pruned_invalidates_no_cursor forbids. And the opening frames it diffs against are taken lazily, at App::view_mut_of — the one door a non-active view can be written through — so the held-key path still hashes one view and pays nothing for the wider question (the premise improvise-mcq4 narrowed that arm on was view_mut() writing only the active view, and that premise is gone).

+

The audience is not everybody, and has not been since improvise-jytm made "which view a client is on" per-client: a reshape of view A threw away the cursor of a client sitting on view B, which nothing reshaped. cursor_invalidated asks App::last_reshaped_views — the observed set, so AddCategory, which seats itself on an axis in every view, still reaches everybody — and the relay is one Broadcast::To(id, ResetCursor) per affected client rather than one ToAll.

+

An empty reshaped set means "not known", never "nobody": a batch that moved the cursor with no view observed to change shape gets the safe answer, everybody. Telling too few is strictly worse than telling too many — the diff then believes each client already holds origin-positioned content and sends nothing, so they stay stale permanently rather than until their next scroll. That is the half-fix gotcha 24 warns about, and cursor_invalidated is a named function with both arms tested so the fail-safe cannot be tidied away.

+

#record_version / resolve_record / resolve_aggregate

+

record_version takes a cell key or the record's own key indifferently — RecordHash::of normalizes. It is zero for a record nothing has written, which is deliberately also what a record that does not exist reports: a basis composed against an absent record and one composed against an untouched record are the same claim. The counters live in the DataStore beside the index that resolves a surrogate, because both are keyed by the record and both are invalidated at the same moment (ADR 002). They are still shared document state, never ViewState (gotcha 30); what moved is where on the document they sit, not whose they are.

+

resolve_record maps a client's record surrogate back to the cell it names (ADR 002). None means reject: there is nothing to write to. Two sources, in this order, and the order is the point.

+ +

resolve_aggregate goes against the layout and only the layout — not because that is cheaper but because an aggregate has no existence outside the layout that derives it. There is nothing stored to index. A layout change therefore makes an old aggregate id unresolvable, which is the right answer: the cell it named is not there any more.

+

layout_cell_keys excludes records mode: its cell_key is a synthetic (_Index, _Dim) pair carrying positions, which is the positional-identity trap ADR 002 rejected _Index for. A records-mode cell is projected from RecordRow::coords, which is already in the store, so the index answers it.

+

#switch_to_view and reset_cursor

+

switch_to_view points the view slice at name and switches the workbook to it. The two copies of "which view is active" must move together: the workbook's is the document's, ViewState's is the client's, and the client's is what keys the cursor map (improvise-cqi, gotcha 29).

+

reset_cursor sends the active view's cursor back to the origin, called when the grid's shape changes underneath it. It resets this slice, which on a server is the host's own and nobody's client. Every connected client owes the same invalidation, and cursor_epoch is how the reset reaches them: the effect that got here cannot see the session list, so Sessions::apply watches the counter instead — the same shape as document_generation and for the same reason (improvise-gvnb, gotcha 24).

+

#display_value_at_cursor / display_value_of

+

display_value_at_cursor is editable text for the cell under the cursor: a staged drill edit if one exists, else the layout's resolved value for synthetic record cells, else the stored cell; empty when the cursor is off-grid. It reads self.layout, so callers that have mutated the grid must rebuild_layout() (or emit RebuildLayout) first (gotcha 23).

+

display_value_of is the value an editor opened on key starts with — unformatted, and so not CellDisplay::text (design-principles §4: display rounding is view-only and must not feed back into the model). Crate-visible so project::cell_display can put it on the wire as CellDisplay::edit_text; a browser that un-formatted the drawn text instead would be a second derivation, and a lossy one.

+

#apply_effects — one classification, three readers

+

Apply a batch of effects and report which widgets it could have changed, so a host with connected clients knows what to project (improvise-cqq). The Dirty accumulates inside the loop: an aborted batch must not claim the effects it never ran (gotcha 21).

+

layout_version and model_state.dirty are no longer read off the EffectKind (improvise-75ld, improvise-pdvm). A kind says what an effect means to do, and both counters were believing it. Two failures, in opposite directions, from the one assumption:

+ +

So a counter is bumped on an observed change, not on a declared intent — the answer record_versions already had, and the reason cursor_epoch is a counter rather than a table of variants. Two fingerprints are taken around the batch, both gated on the batch containing any EffectKind::Model(_):

+

view_shape is one view's drawn shape: the axis map, the effective item names minus this view's hidden ones, the collapsed groups that have something drawn under them, the page selection, prune_empty. Drawn, not configured — hide_item records a name whether or not the model has one, and a hidden ghost moves no row. view_shapes is that per view name, and it answers two questions: did anything move (layout_version), and which views moved (Sessions::adopt_cursor_epoch, improvise-nynk). layout_version is therefore bumped once per batch rather than once per effect; nothing reads it but "did it move".

+

#Two observations, not one (improvise-ni6k)

+

view_shape above is not the only fingerprint taken per batch, and the reason is the part worth keeping: the cursor's question and layout_version's question are different questions, and a single hash cannot answer both.

+

layout_version asks are the coordinates a client's keys hang off still the ones it composed against. The cursor asks does position (r, c) still name what it named. AddCategory of a category with no items separates them exactly: no row and no column moves, so no cursor is invalid — but every CellKey now needs a coordinate it did not before, so the basis tier must move. Unifying the two broke tests in both directions, which is how the distinction was found rather than reasoned to; do not re-merge them.

+

So view_frame sits beside view_shape, and last_reframed_views beside last_reshaped_views. view_frame hashes the drawn frame — row_cats, col_cats, and the AxisEntry sequence of both axes in order, with each group header's collapse state. Four things about it are load-bearing:

+ +

No effect declares any of this. improvise-9mo9's twelve reset_cursor calls are gone, along with the six inline no-op predicates (moved, drawn, real, existed) that existed only to stop those calls firing on a no-op, and the two helpers effective_items and item_is_drawn that served them. That table was the shape gotcha 24 names as the disease, and it had a hole (improvise-7kvw) the day it was written; six more were enumerated on the ticket before it was fixed.

+

Is every reframe also a reshape? No — measured, and the answer matters (improvise-9v0j). The relation between the two observations was separated by example and never stated as a property. One direction is the known asymmetry and the reason there are two: AddCategory of an empty category reshapes without reframing. The other direction was open, and the ticket guessed "probably yes, and if not, layout_version under-reports, which per gotcha 35 leaves a client wrong forever".

+

Probed 2026-09-21 by driving each ModelCommand through apply_command and reading cursor_epoch() and layout_version(): every command in reshaping_commands() moves both, but a ClearCell that prunes a row out of a pruning view moves the cursor epoch and not layout_version. That is not a defect in the observation — it is improvise-mcq4 working exactly as designed, since a cell write changes the data and not the view's configuration, and view_shape hashes configuration. So the candidate property test 9v0j proposes would fail, and writing it would be wrong.

+

What is left open is narrower and worth stating as a question rather than answered here: after such a write the grid has renumbered while layout_version has not moved, so a client still claiming the old layout tier is accepted. The record tier covers a write to the pruned record; it says nothing about a write to a surviving one whose row index moved under it. Whether that is a hole in ADR 001's middle tier or correctly delegated to the record tier is an ADR question, not a test to land quietly, and is filed as improvise-6nsd. Note the probe needed the right fixture to say anything: run against app_with_two_pruning_views, seven of the twelve commands are no-ops because that document has no Product category, and a table of false/false rows reads exactly like a finding.

+

Cost, measured rather than asserted (benches/reshape_observation.rs, the fixtures grid_projection uses): one SetCell batch goes from 172µs to 209µs at 400 cells, 7.09ms to 7.97ms at 18,000, and 35.2ms to 38.0ms at 90,000 — +8% to +22%, the largest share on the smallest fixture where the batch is cheapest. The batch is dominated by rebuild_layout and recompute_formulas, not by the fingerprint. The ticket required this measurement before landing, because "much cheaper than grid_of" is exactly the kind of claim that criterion exists to refuse.

+

The cost interacts with improvise-748f, and the fix is where the surprise was. 748f split the geometry fingerprint so a batch writing no cell stays O(config) off the held-key path. view_frame builds through with_frozen_records, which applies prune_empty, which walks cells — so for a pruned view the walk came back, four builds per batch. Measured, one CycleAxis batch with pruning on:

+
cellsfirst versionnowfingerprint off
400283.2µs167.0µs93.6µs
9,6007.34ms3.879ms1.967ms
18,00012.41ms5.752ms3.043ms
90,00091.69ms56.54ms28.52ms
+

4.1× down to 1.9× at the canonical size, back inside the 16.7ms budget.

+

The obvious fix bought nothing, and then bought a bug. Narrowing the fingerprint from every view to the active one looks like the answer and is not: a second view is usually a fresh one with everything on Axis::None, which costs nothing to build. The per-view multiplier was never the cost — two expensive builds were. What worked instead is that the opening basis is free: at batch start self.layout already is the active view's pruned layout, so opening_view_frame hashes it rather than building a second. It is guarded on drill_state.is_none(), because a drill freezes the record list and the held layout is then not what view_frame would build, and it is opening-use only because self.layout goes stale mid-batch (gotcha 23).

+

The narrowing was kept anyway, and that was wrong. It reads as the honest shape — the cursor's question is about one view — but a CellData write can prune a row out of a view nobody is looking at, and view_shapes cannot see it (config-only: it hashes the prune_empty flag, not which rows survive). So a bystander was left positioned on a grid that had lost a row, permanently, which is gotcha 24's worst case. A measurement showing a change is free is not evidence that it is correct, and the two questions were answered in the wrong order.

+

The arms are now split by the thing that differs: the LayoutGeometry arm keeps the cheap one-view frame, because view_mut() writes only the active view and the structural cases are already covered by view_shapes; the CellData arm pays for every view. Measured: 0% on the held-key path, +5.7% on an 18k-cell two-view SetCell, scaling linearly in view count.

+

The frame basis is keyed by view name, and it follows active_view. A batch that switches and then runs a model effect was comparing the new view's hash against the old view's — a near-certain mismatch, so a spurious reset. The first fix guarded the comparison on the names agreeing, which exempted the first geometry effect after a mid-batch switch from the check altogether (improvise-9c9o): under-reporting, gotcha 35's expensive side. So the re-base happens at the switch — in the loop, as soon as the name moves — and by the time the geometry arm runs the name and the basis agree by construction. Note what this did not need: SwitchView stays ViewOnly, because the basis is App's own bookkeeping and none of it is on the wire.

+

It costs one extra view_frame per batch that both switches and moves geometry — the re-base builds the incoming view's layout and the geometry arm builds it again. DrillIntoCell already pays one per SetAxis, so this is the batch that pays most, and the direction is gotcha 35's cheap one.

+

And writing the test for all this found the bug that was hiding it.

+

#The elsewhere path is only as good as the inputs it can rebuild (improvise-ac2z)

+

grid_in on a view the host is not on rebuilds a GridLayout from the Model and that view's config. Everything the host derived and cached stays the host's, and Model::formula_cache is exactly that: rebuild_layout fills it with recompute_formulas(&none_cats) for the host's view. A client on a view whose none_cats differ therefore reads formula values keyed by somebody else's coordinates — and since prune_empty drops what resolves to nothing, its formula columns vanish rather than showing wrong numbers.

+

prune_empty (improvise-t6db) was the first instance of this shape and was one line. This is the second, and it is not local: grid_in takes &App while recompute_formulas needs &mut Model, so it cannot be fixed where it is observed. Filed rather than bodged; candidates are on the ticket.

+

How it was found is the part to carry. The PR #78 hand-back reported this hazard as "argued and not observed", from a fixture whose stated setup — a category on Axis::None in one view and on Column in another — was absent: add_category defaults a new category to Column and nothing moved it. A negative result from a setup lacking its own condition reads identically to a real one, and is worse than silence, because it stops the next person looking. Setting the axis explicitly failed the property on the first run.

+

project::grid_in built a non-active view with GridLayout::new, which does not apply prune_empty — so a client on another view was projected rows its own config prunes, and its grid never lost a row, so the staleness was unobservable. Third instance of the new-versus-with_frozen_records trap. Fixed, and it broke no other test.

+

Two fixtures nearly hid all of this, and both are the same mistake. The SetCell benchmark shows the fingerprint at +2–5%, because 748f's own document_content walk dominates that arm. And the geometry benchmark, written first against the existing fixture, had prune_empty off — where with_frozen_records does not walk cells at all and the batch reads 17–176µs. A fixture that cannot reach the expensive branch measures the cheap one and reports it as the cost.

+

What is still guarded in one direction only, and is filed: nothing asserts that a batch which reframes a view is always one that view_shape also moves, or the converse. The separation is argued here and tested by example, not by property.

+

document_config is everything a LayoutGeometry effect can write that the file would carry: the categories with their items and groups, and every view's configuration whether it draws or not. It is what raises model_state.dirty, and it is a different hash from view_shape on purpose — the number format and a hidden ghost belong to the bytes on disk and not to the shape. The cell data is absent from both, because record_fingerprints answers for it and a whole-document hash per keystroke is not affordable (gotcha 41: Storage is whole-document).

+

ModelChange::CellData and ModelChange::WholeWorkbook still raise dirty off the declaration, because they write things no structural fingerprint can see: a formula, a sort order, a replaced document.

+

Both halves are read inside the loop, and the position is the property. The observed half compares document_config after each LayoutGeometry effect against the value it held after the previous one, rather than once around the whole batch. An after-the-loop comparison is correct about whether the document changed and still wrong, because it re-raises a flag Save cleared earlier in the same batch: [SetNumberFormat, Save] — reachable from :format ,.2f . w and from any Binding::Sequence pairing a geometry effect with a save — then reported a successful save and a dirty document, so the TUI drew "Saved to …" while :q refused, and the worker paid a whole-document OPFS rewrite. That is the failure class this whole section exists to close, reintroduced one level up. Guarded by a_batch_that_changes_the_layout_and_then_saves_ends_clean, over both arms. It still over-approximates there — a LoadModel that cannot read its path leaves the flag raised, costing one redundant write, where the other direction costs an edit. It used to be a separate MarkDirty effect that each command composed alongside its mutation, which improvise-kmgs's one-at-a-time derivation put SetCell on the wire without (improvise-gv9r): a derivation correct per element and wrong for a set.

+

Save is ViewOnly and clears the flag itself, so a batch that edits and then saves ends clean: the order in the loop is the order the effects were composed in. That is why the observation is in the loop and not after it.

+

layout_version is bumped after the loop, and that is not the same hazard: nothing clears it, and an aborted batch's effects never ran, so a comparison taken at the end sees only what did run. Neither arm of that is tested today; filed with the orchestrator rather than cited here, because an id that was never read back is worse than no id.

+

What the split does not reach, and is not a regression because the declared scheme did not reach it either: prune_empty makes a view's shape depend on the data, so a cell edit that empties a row prunes it without moving view_shape. The record tier covers the write; the shape tier does not see the reshape.

+

Both fingerprints sort every HashMap/HashSet they touch. That is not tidiness: an unsorted iteration order is per-process (gotcha 18), and a fingerprint that re-randomises every run would bump both counters on every batch.

+

record_versions cannot be read off the kind, because an EffectKind says that a batch writes cells, never which — so the records are read off the data itself, fingerprinted around the batch. Two consequences worth stating. A write that lands the value already there bumps nothing, which is the right answer: nobody's basis was invalidated by it. And a LayoutGeometry effect that drops cells will bump their records when it shares a batch with a CellData one — an over-bump, costing a rejection that need not have happened, which is the same asymmetry Dirty::of is deliberately sloppy in.

+

The case list is checked for completeness, not for length (improvise-26dm). layout_geometry_effects_actually_change_document_config walked a hand-built list under an assert_eq!(cases.len(), 13), and a count catches a case being deleted and never one being absent — the two are written identically. The adversarial review of PR #78 added a fourteenth pub struct declaring LayoutGeometry and the suite stayed green. doctrine(a_list_checked_for_length_reads_like_a_list_checked_for_completeness).

+

improvise-5ubs's answer — read serde's derive-generated variant list out of its unknown variant … expected one of message — cannot reach this one: an effect is a struct implementing a trait, so there is no derived enumeration of effects at all. But gotcha 38 says ModelCommand is derived from the model effects, with model_effects mapping each variant back to the effect it came from, so the list is now diffed against the LayoutGeometry effects reachable through ModelCommand, and the failure names the missing effect rather than a number. The two lists agree exactly today, which is itself the evidence the substitution lost nothing.

+

The right-hand side had to be bounded before it could be believed (PR #96, found independently by this branch's review and by ChatGPT Codex). every_model_command()'s doc comment claimed the enumeration was compiler-forced — variant_name matches exhaustively, so a new variant fails the build. That inference is invalid: the build fails in variant_name, and nothing obliged the value list to gain an entry. Deleting one from it failed this check and nothing else in the workspace.

+

So the list is now itself checked, by improvise-5ubs's mechanism: the_enumeration_names_every_variant_the_derive_declares diffs it against variants::declared::<ModelCommand>(), serde's derive-generated variant list. That makes the chain derive → every_model_command() → model_effects → case list, every link checked, and the whole thing is what the ticket asked for. variants was #[cfg(test)] pub(crate), so reaching it costs a test-support feature on improvise-protocol gating an optional serde_json — off in every shipped build, which is why the wasm gate is unaffected. The assertion message still names both directions, because a short right-hand side reads as a spurious case and invites deleting the left, degrading both sides together — the failure the ticket exists to close.

+

Its bound is stated rather than assumed: this closes the effects on the wire, and an effect declaring LayoutGeometry that no ModelCommand names is still invisible — which is precisely the probe variant PR #78 injected. That is an honest residual and not a fix, and it is the reason the ticket asked for a compiler-maintained enumeration of effects: none exists without a proc macro or inventory-style registration, and a fourth hand-rolled list in a fourth place is the disease. A second […; 13] array survives one function away in every_layout_geometry_effect_is_visible_to_the_narrow_fingerprint and is filed, not smuggled in.

+

On abort, the loop resets abort_effects so the next dispatch starts clean and records batch_aborted where a caller can still read it.

+

#bump_changed_records

+

Bump every record whose raw cells read differently than they did at before. Both directions count: a record that gained cells, lost them, or had one rewritten has moved under anybody holding an older basis.

+

The diff collects into a set, not a list: a record present in both maps is named twice by the chain, and a rewritten record would otherwise count as two writes. (That bug survived one test run — gotcha 40.)

+

#dispatch_key

+

Resolve one keypress through the keymap and apply what it produces, reporting which widgets changed. The whole key path, with the projection half factored out: a host with no connected clients (the native TUI) drops the Dirty, and one with clients hands it to Sessions::project_all. ui::reduce is the named door onto this (improvise-gxi).

+
+

#ui/project.rs — turning what the server did into what each client is told

+

#grid_of's whole-grid cost, measured (improvise-i5tk)

+

The TUI builds its GridCache over the whole grid at offsets zero, so a cache position is an absolute position (gotcha 44). improvise-i5tk proposed narrowing that to the viewport, as the browser client already does, and made the benchmark acceptance criterion 1 — because "this can dominate TUI rendering" is exactly as unevidenced as "it is fine". benches/grid_projection.rs is that benchmark and it says no. One grid_of call, dense pivot grids, this machine:

+
typical                  100 x   4 =    400 cells  whole-grid  1.8-1.9ms   ratio 1.2x      col_widths 45%
+large                   1200 x   8 =   9600 cells  whole-grid  175-177ms   ratio 1.4x      col_widths 46-49%
+18k cells (gotcha 41)   3000 x   6 =  18000 cells  whole-grid  318-328ms   ratio 1.5x      col_widths 45-46%
+pathological            9000 x  10 =  90000 cells  whole-grid  4.8-4.9s    ratio 1.3-1.4x  col_widths 45-47%
+

Those are min-of-N figures, and the first version of this harness could not reproduce its own. It took the mean over a handful of iterations, which on a loaded machine is dominated by whatever else is running: review measured the same col_widths share at 8% and at 131% — a part taking longer than the whole containing it, printed flat because nothing checked. It also passed (comma, decimals) = (false, 2) where grid_of passes the active view's (true, 0), so it timed a call that never happens and reported it as a share of one that does. Both are fixed: the arguments come from parse_number_format on the view, the estimator is min-of-N, and a share above 100% prints UNSTABLE rather than a number that reads like a measurement. A benchmark that cannot detect its own impossible output is not evidence — the standard doctrine(check_the_checker) sets for a test, applied to a measurement.

+

ratio is whole-grid over a 45×12 terminal window: 1.2–1.5x, against a fix whose stated cost is teaching GridWidget that a cache index means something else and putting every scroll site in scope. That is the wrong trade, so i5tk is closed on the measurement rather than on the reasoning either way.

+

The reason the ratio is so low is the finding, and it is bigger than the ticket was. Two of the three whole-grid passes a frame performs are viewport- independent by construction, so no amount of windowing reaches them: ui/viewport.rs::compute_col_widths takes no viewport and walks every (row, col) twice — once for cell text, once for the column totals it must also size — and a column total has to sum every row including the ones a viewport scrolled past (gotcha 44). grid_of's own cell_display loop is the third pass and the only one narrowing would touch. So the whole grid is evaluated about three times per frame, every result discarded, and at 18,000 cells a keypress costs four tenths of a second. That is improvise-jmjz, and its fix is caching or one fused pass — not windowing, which this benchmark has now ruled out.

+

#One walk, and what it did and did not buy (improvise-jmjz)

+

ui/viewport.rs::grid_body is now the one walk. It evaluates each cell once and derives all three answers from that value: widen the column, add to the column's sum, and — if the position is inside the requested window — keep the CellValue for cell_display to dress. compute_col_widths survives as grid_body with an empty window, because the keypress path (App::apply_effects) wants widths and no cells, and two width rules would be the disease improvise-edp names.

+

Measured on the orchestrator's machine, 2026-09-21, before and after, with cargo bench --bench grid_projection:

+
casebeforeafter
typical (400 cells)1.840 ms0.687 ms
large (9,600)172.6 ms50.0 ms
18k (gotcha 41)294.4 ms95.2 ms
pathological (90,000)4.481 s1.320 s
+

About 3x, which is the number of passes removed — the shape the acceptance criterion asked for, rather than a speedup argued from the structure.

+

Three things to carry, none of which the ticket predicted.

+

The col_widths column of that benchmark now measures something else. It read 45–47% of the frame and now reads 64–82%, which looks like a regression and is the opposite: compute_col_widths is the fused walk, so its share rose while the frame it is a share of fell by two thirds. A ratio whose denominator moved is not comparable across the change, and the benchmark's own comment now says so — otherwise the next reader concludes the fix made widths worse.

+

The windowing ratio did not improve, and that is correct. It stays 1.2–1.4x, because the pass that remains is exactly the one a viewport cannot narrow: the widths and the totals are whole-grid by definition (gotcha 44). improvise-i5tk's decline stands, and the two findings are now cleanly separated — windowing was never the fix, and removing the redundancy never made windowing worth it.

+

The keypress path got the same fix for free. compute_col_widths is the walk now rather than two of them, so App::apply_effects' own width computation roughly halved on the same fixtures (134 ms to 68 ms at 18,000 cells). That is the 748f held-key path, and it was not the ticket's subject.

+

It is not solved, only three times cheaper. 18,000 cells is 95 ms, still 5.7 frames at 60fps, and the remaining cost is one genuine evaluation per cell. The next lever is caching across frames, which needs an invalidation story the record_versions/layout_version tiers could plausibly supply.

+

The guard is a_window_narrows_the_cells_and_neither_the_totals_nor_the_widths: with one walk serving three consumers, a window that leaked into the other two would silently make a column total the sum of the visible rows, or size a column by what happens to be on screen. Two faults, then, and its first fixture could see only one — see context/test-evidence/command.md, because the way it was blind is more instructive than the fixture that fixed it. Briefly: the widths half was asserted by comparing grid_body against compute_col_widths, which is grid_body with an empty window, so windowing degrades both sides identically. The same function twice is not a check, and it reads exactly like one.

+

The benchmark carries test = false: it takes seconds by design, and the suite's rule is that no single test binary exceeds two of them.

+

#What the module is (improvise-cqq)

+

The producer half of the per-widget event architecture (improvise-rkeq): improvise-protocol says how a client folds a Projection, and this module says which projections the server emits. Nothing here renders, and nothing here holds a transport — a websocket or a worker port calls Sessions::project_all and writes the result out.

+

Three decisions shape it.

+

What changed is decided by diff, not by the effect that changed it. An effect knows its EffectKind but not which cells it touched, and asking every effect to report its cell set would be a second, silently divergent description of what it did. So the server rebuilds the desired state of each widget and compares it with the mirror of what that client was last told. The effect kinds still earn their keep: Dirty turns a batch's kinds into the set of widgets worth rebuilding, so a ViewOnly batch costs nothing, and a cell edit outside a client's viewport produces no traffic for that client because the desired viewport is unchanged.

+

Only the grid gets a mirror. The other widgets' updates replace their cache wholesale (Projection::Formulas is the whole list), so "has it changed" and "send it" collapse into one decision, which Dirty already made. The grid's Cells update is positional and incremental, so it needs to know what the client already holds.

+

A viewport is the client's report, not the server's guess. The scroll offsets are already in the command stream and reach the server's mirror through reduce_view, so the server reads them off the mirrored ViewState. The extent — how many rows and columns fit — is a fact about a terminal or a browser window that no Command carries, and inventing one would break decision(improvise-cqi, command_set_derived_not_invented). It is set on the session by the host instead (ClientSession::set_extent).

+

The emission order is load-bearing: a GridUpdate::Cells that overtakes the labels naming its positions is dropped by the consumer, so grid_updates emits labels first and diffs the cells against the frame those labels will leave behind (gotcha 33).

+

#Dirty — a gate, not a description

+

Which widgets a batch of effects could have changed. The projections themselves are computed by comparing current state with what the client holds, so a Dirty that over-reports costs work and never correctness. Under-reporting is a correctness bug, which is why Dirty::of matches exhaustively on EffectKind — a new ModelChange variant fails the build.

+

#Dirty::of — the derivation (gotcha 35)

+

This section is the record of the derivation that the doc comment used to be. Re-read it when an effect is reclassified. It has gone stale once already, claiming AddCategory was CellData after it had been moved.

+

Read off the effects that carry each kind, not guessed:

+ +

The formula list is the one entry where over- and under-reporting are not symmetric in cost: a LayoutGeometry batch that leaves the formulas alone pays one redundant Projection::Formulas, while one that dropped a formula and said nothing leaves the client's FormulaCache wrong forever.

+

The LayoutGeometry arm is spelled out rather than written as EVERYTHING: the two coincide today, but they are derived from different effects and an entry added for one must not silently arrive for the other.

+

#grid_in's elsewhere path and the inputs it must rebuild (improvise-ac2z)

+

grid_in projects a view the host may not be on, and doctrine(the_elsewhere_path_is_only_as_correct_as_the_inputs_it_can_rebuild) says to enumerate the inputs and ask of each whether it was derived for the caller or for the subject. Two have been found that way. prune_empty was the first (improvise-t6db): GridLayout::new does not apply it, so the fix was with_frozen_records. Model::formula_cache is the second, and it was worse, because it cannot be fixed where it is observed — grid_in takes &App and recompute_formulas needs &mut Model.

+

rederived_for answers it by copying: where the target view's none_cats differ from the host's and the document has formulas, it clones the model and recomputes for that view. The three conditions are what keep the clone rare — the common case is a client on the host's own view, which pays nothing, and a formula-free document pays nothing ever. It is an interim: keying formula_cache by none_cats, so one Model holds more than one view's derived values, is the durable answer and is an improvise-core change.

+

The list is not closed. Anything else rebuild_layout computes for the host's view and grid_in then reads is the next instance, and nothing enumerates them — which is the doctrine's whole point.

+

#grid_of — the extent is the whole grid

+

GridExtent carries the whole grid, not the window: it is what a client clamps its own cursor against (improvise-bpq2, gotcha 47). It is read off the layout rather than off the projected ranges, which are already clipped.

+

#axis_frame — one axis's frame, windowed

+

Three edge rules, and every one of them is about a group header, because a header occupies a row without consuming a data index (gotcha 1's cousin: the data index counts items, the visual position counts entries).

+ +

#column_totals

+

Each visible column summed over every row of the grid, not just the rows in the viewport. That is why it is projected rather than left to the client, and it is the same sum the TUI's total row has always drawn. Empty in records mode, where a column is a measure or a coordinate and a column sum means nothing, and empty for a grid with no rows or no columns — an empty strip says "no totals", which is a different statement from a strip of blanks (gotcha 33).

+

#cell_display

+

One cell as the client draws it. The server is the side that knows whether a value was a number, so it says so rather than letting the client re-derive it from the text — and by the same argument it is the side that knows whether the cell is settable, so it says that too (ADR 002, cell_id).

+

Records mode resolves through the frozen snapshot, which yields display text and nothing else; numeric-ness is read back off the text, the one place this module infers rather than knows.

+

edit_text is the unformatted value, which is what the TUI's own display_value_at_cursor seeds its edit buffer from. Projecting it rather than letting a client un-format text is the rule from gotcha 47: where a client cannot compose the command, enrich the projection.

+

#cell_id — stored cell or aggregate

+

What the cell at (row, col) is, expressed as the surrogate the client echoes back. The fork is not invented here. EditOrDrill (cmd/mode.rs) already makes exactly this decision on every keypress in the TUI, and asks it the same way: a cell is aggregated when a regular category sits on the None axis. Virtual categories are always on None in pivot mode and imply nothing, and a records-mode cell is never aggregated whatever the axes say — which is a property of the layout, not of a mode flag.

+

In records mode it uses the row's coordinates, never the synthetic (_Index, _Dim) key cell_key returns there: that one carries positions, and a position is what ADR 002 rejected _Index for.

+

RecordCol::Category, NewMeasure and NewCategory are not cells to write into and get no id. Editing a category column renames a coordinate across every measure cell in the row (gotcha 1) and the +/+cat columns create a measure or a category; all are different intents from SetCell, so none gets an id it could be composed with.

+

#category_rows

+

The categories the category panel draws: every category, virtual ones included, with their items and their axis in the active view.

+

category_names, not regular_category_names (improvise-vjms): the TUI panel has always listed _Index/_Dim/_Measure and the browser panel is to list the same rows, so the filter belonged in neither.

+

effective_item_names, not category(name).items: _Measure's items are explicit items ∪ formula targets (gotcha 7), and the raw list omits every target. That is the whole reason this is one function and not two.

+

try_axis_of, not axis_of: a view that has never seen a category has no entry for it and axis_of panics. Unseen is None, which is what the view will show it as.

+

#The four whole-cache accessors, for a host that holds the App

+

A host that renders in the same process as the authoritative App — the ratatui binary today — has no transport to fold a projection stream out of, but must draw the same widgets from the same data as a remote client. These four give it each widget's cache outright. They are the same derivation document_projections sends, spelled once: a projection is built from the cache, so an in-process renderer and a remote one cannot disagree about what a widget holds.

+

There is deliberately no function returning all of them together, and no struct to hold them: that aggregate is the shape improvise-rkeq rejected, and a caller that wants several calls several.

+

Their use by the TUI is temporary. The native binary calls these once per frame, rebuilding from the App rather than folding updates, so that widgets can migrate off &Model/&View one at a time while the binary split is in flight. The per-frame rebuild is what goes away when the TUI becomes a client; the functions themselves stay, because the server needs them. Nothing in src/ should come to depend on when they are called.

+

#document_projections

+

The projections for every widget whose content does not depend on a client's viewport. Each of these updates replaces its cache wholesale, so dirty is the whole decision.

+

#wizard_of / wizard_cache (improvise-764)

+

The import wizard as the widget draws it, or None when no import is open. This is where the wizard's improvise-io types stop. FieldKind, DateComponent, ImportPipeline and the built Workbook are all read here and none of them leaves; what comes out is WizardCache, which names only strings, booleans and its own two enums.

+

Unlike every other *_of in the module it feeds no Projection: the wizard is session-local (gotcha 30) and broadcasting it would draw one client's dialogue on every client — the rendering-side twin of the bug improvise-tom1 removed from ModelCommand.

+

wizard_cache is split out from wizard_of so it can be exercised against a wizard driven through its steps, with no App in the way. The App half is one Option::map; this half is where every improvise-io type is spent.

+

DATE_COMPONENTS lists the date components the wizard offers, in the order it offers them. The order is load-bearing: ImportWizard::date_config_at_cursor counts these same rows, so the cursor the cache carries indexes this list.

+

summarize_workbook is the one-line description of the workbook awaiting confirmation. It lived in the widget, which is how a renderer came to be calling regular_category_names() and measure_item_names() on a Model.

+

#grid_updates — the diff, and what it diffs against

+

The updates that carry mirror to desired. Labels come first and cells last, because the consumer's fold is not commutative: a Cells update that overtakes the labels naming its positions is dropped. The cells are therefore compared not against mirror but against the frame the label updates will leave — which is also what makes the result exact rather than merely sufficient, since a relabelled grid keeps the cells that still fit.

+

The totals are diffed against that same framed mirror (improvise-hkt6, found by Copilot). apply blanks the total under a relabelled column (gotcha 33(b)), so comparing against mirror missed exactly the case where the label moved and the number did not: no Totals went out, the client blanked its own, and ClientSession::project_diff's self.grid = desired left the server believing it was still there. A client cache wrong forever, which is the direction Dirty::of is deliberately sloppy to avoid (gotcha 35).

+

Only the totals. The widths stay on mirror: folding the labels re-fits them too, so diffing widths against the fold would suppress the ColWidths that a_client_on_a_socket_is_seeded_and_gets_its_edit_back asserts a reframe carries — redundant, but it is gotcha 33(c)'s standing evidence and not this ticket's to spend.

+

#ClientSession

+

One connected client as the server sees it: the server's mirror of that client's ViewState — maintained by folding the same commands the client folds, never by shipping state — and the grid cache the client is believed to hold.

+

basis is the basis this client was last told (ADR 001, improvise-fkfg). It is mirrored for the same reason the grid is: it is diffed, so a client hears about it exactly when it moves. Without publication the check would be unsatisfiable — nothing else in the stream carries any of the three tiers, so a client could only ever forge one, and ADR 001 does not say where a client gets its basis from. That omission is the one place the ADR met the code and was wrong.

+

fold folds a client→server command into this client's mirrored view slice — the same function the client runs, so the two cannot drift. It is private, and reached only through Sessions::apply: a command with a document half cannot be honoured from inside one session, and a door that takes every Command while honouring only some of them is a convention every transport author would have to be told about.

+

project is everything this client must be told, given what the last batch changed. The grid is emitted when the document dirtied it or when this client's own viewport moved — a scroll is ViewOnly and reaches no other client, but it still changes what this one can see. A mirror seeded against a document that has since been replaced is reseeded instead of diffed: its cursor map is keyed by the outgoing document's view names (gotcha 24), so every position in it means nothing here. The check lives on this path rather than on a hook because the effect that installs a workbook cannot reach the session list, and a hook nothing can call is a convention, not a mechanism.

+

basis_update is the basis to publish, if it has moved since this client last heard. It is emitted after the grid updates in the same batch, and that order is load-bearing: the record counters describe the cells those updates carry, so a basis arriving first would let a client claim a version for a value it had not yet been shown. It is the mirror image of gotcha 33's rule that labels precede the cells they frame. The records published are the ones this client's grid names, which is exactly the set it can address a write to (ADR 002); publishing the whole document's counters would be both larger and no more useful, since a surrogate the client has never been projected is one it cannot compose a SetCell for.

+

reseed points the mirror at the app's current document, holding nothing, and returns the projections that carry it back — a compaction of the stream so far, not a replay of it. That is what connect does for a new client, and what a document swap does for every existing one.

+

#Broadcast — the audience is part of the message

+

The two cases mean different things and a host cannot infer which from the command. Making it a field of the thing being sent is the same move as Applied being an enum: a host must name the audience to get at the command (doctrine(an_invariant_is_made_unrepresentable_not_validated)).

+ +

#Applied — an enum because a document swap is not a diff

+

A command that replaced the document obliges the host to do something categorically different, and an obligation stated in a field is one a host can read past. Matching is how it is made unavoidable: a host that wants the Dirty has to name the other cases to get at it.

+

The distinction is not cosmetic. The server's own mirrors self-heal — ClientSession::project reseeds the moment the generation moves — but ViewState::cursors lives on the client, keyed by the outgoing document's view names, and only a fresh Snapshot replaces it (gotcha 24). Before Command::Model existed, a document swap could only come from this host's own keypress; now another client can ask for one, and the host that must send the snapshots is not the host that asked.

+ +

#Sessions::connect

+

Attach a client and hand it the bootstrap Snapshot: a fresh view slice pointed at the document's active view, plus the seed it folds to catch up. The snapshot's view is the mirror's, cloned — not a second value built to the same recipe. reseed is what decides what a freshly seeded mirror holds, and a client told anything else starts out disagreeing with the server's picture of it.

+

#Sessions::apply — the one door (gotcha 36)

+

It takes the authoritative App because some commands are not one client's business alone. Command::Model is a document mutation by definition and goes to apply_model, never to a mirror: the model is shared, so what the other clients are told is the Dirty it produced, not the command itself. SwitchView has a document half — the workbook's active view and the client's must move together, and App::switch_to_view is one of the only two writers allowed to touch either (gotcha 29) — so it is routed through apply_command rather than folded into the mirror. Folding it into the mirror alone left the session naming a view the workbook had never switched to, and the next project() built that client a grid out of the other view.

+

Everything else has no document half and stays this client's: it never touches the App's own view slice, because the model is shared and the UI is not.

+

Lockstep on the active view is today's answer, not the last one: improvise-g0f allows two clients to sit on different views, and improvise-jytm owns that split. What this shape fixes survives it — the mirror and the state the projection is built from move in one call, so whichever way g0f lands there is one place to change.

+

Lockstep is a property of every session, not of the one that asked. grid_of builds every client's grid from the workbook's active view, so a session left naming the outgoing one is read under a stale cursor key: it is projected the new view's rows positioned by the old view's scroll, which is gotcha 24's cursor-vs-geometry mismatch arriving through another client. So the accepted switch reaches every mirror, and Applied::broadcast carries it on to the clients whose own slices must fold it too.

+

The basis check is the first thing, before the dispatch, and it is a guard rather than a branch inside the apply: a batch whose basis moved is never applied, not even partially (ADR 001). It is Checked::against that runs it, because the answer is what buys the Checked every mutation below needs — the check is not a step this function could forget (improvise-ondj).

+

Note: at this commit Sessions::apply has no production caller; hosts go through apply_checked. See the ui/reduce.rs section.

+

#Sessions::apply_checked

+

The same door, for a command whose answer is already in hand. A frame is a sequence of commands and ADR 001's atomicity is the frame's, so Host::commands answers for all of them with a CheckedBatch and then applies them through here — re-asking per command would reject the later ones for staleness their own siblings caused (improvise-955n). It cannot return Applied::Rejected: the refusal happened, if it happened, before the batch was built.

+

#apply_switch

+

Which view a client is on is that client's (improvise-jytm, improvise-g0f): the model is shared and the UI is not. So this takes a SessionId — it is addressed to exactly one session, which is the opposite of what it once was — moves only that session's mirror, and dirties nothing.

+

The one thing it still asks the document is whether the view exists. A slice keys ViewState::cursors and is read by grid_in, so a mirror naming a view the document lacks would index nothing; what is adopted is therefore what was accepted, which on refusal is the view the session was already on.

+

The audience follows from that, and it is conditional. On the accepted path the asker folded the right thing and nobody else is affected, so nothing is relayed. On the refused path the asker — and only the asker — folded a name the document turned down, so it gets Broadcast::To(id, …). ToOthers was the pre-jytm shape and had the mirror-image hole: it told every client but the one that was wrong. A blanket ToAll is equally wrong here, and for a reason that survives jytm: the asker really did fold, so ToAll double-applies on the common path.

+

Note what is not here any more. adopt_active_view pushed the accepted view into every mirror; under a per-client active view that is precisely the bug, so it is gone along with the Broadcast::ToOthers variant it was the only producer of. A mechanism whose reason has gone is deleted rather than left.

+

#apply_model (improvise-kmgs)

+

It goes through the same apply_command, which runs the effect the command was derived from and reads the Dirty off that effect's own EffectKind — there is no second classification here that could disagree with gotcha 35's table.

+

It takes no SessionId, and since improvise-jytm that is the whole distinction from apply_switch: a document change is addressed to no one session, a view switch to exactly one. No model mutation moves anybody's active view any more — ViewBack/ViewForward left ModelCommand with jytm, because the stack they pop is one client's history (the ApplyAndClearDrill argument, gotcha 38), and a LoadModel that installs a different document bumps the generation and so lands in the rebase arm, where every client gets a whole fresh slice.

+

Two things are left to relay. adopt_cursor_epoch's reset is Broadcast::ToAll: nothing was folded optimistically, so the asker is not a client being corrected but one being told, exactly like the others. And adopt_deleted_views is the case a per-client active view creates — see below.

+

#adopt_deleted_views (improvise-jytm)

+

A DeleteView can take away the view a different client is sitting on. Its mirror then names nothing, App::resolve_view falls back to the document's default, and that client is projected the default's grid while its own slice still names the view that is gone — the server and the client disagreeing about which view keys the cursor map, which is gotcha 24's mismatch arriving through somebody else's keystroke.

+

Before the split this could not happen: Workbook::delete_view moved the document's active view and adopt_active_view pushed the result into every mirror. The split removed both, so the case has to be answered on its own.

+

It is observed, not predicted — the check is "does this mirror still name a view the document has", not "was this command a DeleteView" — for the reason cursor_epoch beats matching on CycleAxis (gotcha 24) and document_generation beats matching on LoadModel (gotcha 39): a table of which commands can remove a view would be a second description of Workbook, free to disagree with it.

+

Each affected client is told with Broadcast::To, and the others are not: a client on another view lost nothing.

+

The basis was answered for by Checked::against before the dispatch that reached here. That it is a type and not a line at the top of the function is what stops a second mutation path from being written without it (improvise-ondj). stale_basis is the only decision, and it has no argument that suppresses it — ADR 001's force is a fresh batch under a fresh basis, composed by the client, not a flag that walks past it.

+

Rebase is decided by the generation, not by the variant. The three WholeWorkbook commands can land here, and whether they actually replaced the document is not a property of the variant: a LoadModel that fails to read leaves the document alone, and an AcceptWizardModel with nothing built does nothing at all. So the question asked is the decisive one — did the generation move (gotcha 24) — and the rebase is done here rather than handed to the host as an instruction, for the same reason apply_switch adopts the accepted view itself: the only enforceable version of "you must also do X" is doing X.

+

#adopt_cursor_epoch (improvise-gvnb)

+

If the batch renumbered the grid, invalidate every mirror's cursor and say so on the wire.

+

Two halves, and neither works alone. Resetting only the mirrors would leave the diff believing each client already holds origin-positioned content, so nothing would be sent and every client would stay stale permanently rather than until its next scroll: a partial fix strictly worse than none. Broadcasting only would leave the server projecting each client a window it is no longer looking at.

+

The audience is Broadcast::ToAll, and the asker is not an exception. Its mirror was reset here like everyone else's, and it folded nothing optimistically, because a Command::Model is not something a client can apply to itself.

+

What is not broadcast is a position: another client's transpose must not move where I am looking — that is UI and it is mine (improvise-g0f). It may only declare that where I was looking has stopped meaning anything, which is a fact about the shared geometry.

+

#Sessions::rebase

+

Re-attach every session to a document that has just replaced the one they were connected to, and hand each client the bootstrap it needs. The server's own mirrors are reset by ClientSession::project the moment the generation moves, so this is not what keeps the server honest. It is the other half: a client's view slice lives on the client, and only a fresh Snapshot tells it to drop a cursor that names a view of a document that no longer exists.

+

apply_model calls it itself when a command moved the generation, and hands the result back as Applied::Rebased. It stays public because a host also replaces the document by its own keypress path, where no Applied is produced.

+

#fresh_view

+

The view slice a newly-connected client starts from: empty, but pointed at the document's active view, because that is what keys its cursor map (improvise-8uz, gotcha 29).

+
+

#ui/host.rs — the session loop

+

#What the module is

+

Messages in, addressed messages out. A websocket is awkward to test, so nothing that decides anything lives in one. Host is a pure-ish state machine over the authoritative App — it takes one client's frame and returns the frames that owes every client — and crate::transport is the shell that moves bytes in and out of it. Every test in this crate drives Host directly; the socket is exercised by running the binary.

+

One App per document, many clients per App. The ticket (improvise-q08) asked for one App per connection, "no shared authoritative state across sessions yet". That premise no longer holds (doctrine(check_the_ticket_premise)): Sessions exists precisely to hold many ClientSessions against one authoritative App, apply_model invalidates every mirror's cursor rather than the asker's (doctrine(lockstep_binds_every_session_not_the_asker), now scoped to the document half since improvise-jytm made the active view per-client), and Applied::Diff::broadcast exists only because other clients must hear what this one did. An App per connection would make all of that dead code and give two browser tabs on the same document two documents.

+

Nor is it one App per server. improvise is a multi-document app, and the unit of authority is the document: a Host is one App, one Sessions, and the clients attached to them. That grain is what Sessions was already built for — it never owns the App, every method takes it as a parameter — and it is what makes concurrency scope for free: sessions editing independent models cannot conflict, and the basis versions ADR 001 reasons about live on the App, one sequence per document.

+

So a second document is a second Host, and hosting several means a registry of them keyed by document id. That registry is improvise_ws_server::registry (improvise-j31x), and it cost this file nothing: a Host was always a value rather than a singleton, and handle is already addressed to one document by virtue of being a method. What the registry adds is the only thing a Host cannot supply — a server-wide address, since SessionIds are minted per Sessions and every document's first client is SessionId(0).

+

What sharing one document costs is honest and stated: there is no per-client permission, no conflict resolution beyond last-writer-wins, and a client that can connect can rewrite the document. ADR 001 owns where that goes next; this host is the localhost single-user case that ADR describes the starting point of.

+

#Addressed

+

One outbound frame and the connection it goes to. The host addresses by SessionId and knows nothing about sockets; the transport owns the map from a session to a connection.

+

#Host

+

One authoritative document and every client attached to it — the unit a multi-document server keys its registry by. A Host is a value, never a global, so hosting several is a map of these and not a change to anything inside one.

+

It is deliberately not Debug: App is not, and a Host printing the whole document into a log line would not be wanted even if it were.

+

#Host::app_mut

+

The authoritative App, for work the host itself initiates. Deliberately narrow in intent, and not a second door: a client's Command must go through Host::handle, which routes it into Sessions::apply — the one place a basis is checked and the mirrors are kept in lockstep (gotchas 36, 43). What this is for is the work a host does on nobody's behalf, where there is no session to address and no claim to check: the browser worker writes the document back through the Save effect after a batch dirties it, because saving is EffectKind::ViewOnly and so reaches no wire vocabulary.

+

#Host::connect

+

Attach a client that has reported an extent of rows x cols, and hand back its bootstrap. Total: there is no way for a connection to fail here, which is why the transport reads the extent off the first frame before calling. Nothing is owed to the other clients — a new connection changes no document state and moves no mirror but its own.

+

#ClientFrame::Key answers nothing, and that is settled (improvise-bpq2)

+

Resolving a keypress here needs reduce_key against the authoritative App, which emits no Command a client could fold — so the two would diverge, which improvise-worker-server's a_key_resolved_on_the_host_moves_state_no_client_can_hear demonstrates. bpq2 closed that by moving resolution to the client, so a key a client can answer never arrives as a Key frame at all, and one that does is a binding nothing on this side could answer without diverging. Growing the set is a matter of enriching the projection until the client can compose the command itself — see improvise-wasm-client::keys, which carries the list of what is still upstream. (Gotcha 47.)

+

#Host::save (improvise-368h)

+

Write the document back, because a client asked.

+

No session state moves and nobody else is told, which is why it does not go through Sessions at all: saving changes no document state and no client's view slice, so there is no mirror to adopt and nothing to broadcast. What moved is bytes in this host's storage — the same fact that makes effect::Save EffectKind::ViewOnly and the reason the ModelCommand derivation could never reach it (gotchas 3, 38). Success is silent, on the same terms as any accepted command that dirties no widget.

+

Both refusals are the host's to answer, not Checked's: neither is about a basis. The missing path is decided before anything runs, because a save that cannot land should be refused rather than attempted and then reported. The failed write can only be found by trying, and Save signals it by aborting its batch (gotcha 21) — the batch here is exactly [Save], so an abort is that failure and nothing else. Reporting a failed write as success is the thing this exists to stop.

+

#Host::resize

+

A resize dirties no document state at all — but it moves this client's viewport, and ClientSession::project emits the grid when the viewport moved even under Dirty::NOTHING. So the fan-out is the same call as everywhere else, with nothing dirty. A frame for a session that has already gone is dropped: there is no connection left to answer.

+

#Host::commands — answered whole, then applied whole (improvise-955n)

+

Order is why a batch exists at all: one keypress resolved on a client is SetSelected plus the scroll offsets that follow it, and a host that folded the selection but not the offset projects that client the wrong window.

+

It used to apply in order and abandon at the first rejection, which left a hole ADR 001's "not even partially" does not have: a refused frame's earlier commands stayed applied. The check has moved from the command to the frame, so there is no longer a moment at which part of a frame has landed and the rest has been refused — the refusal, if any, happens before anything is applied, and Sessions::apply_checked cannot produce one. CheckedBatch carries what a batch's basis means and the one residue this does not close (clause 4).

+

#Host::command

+

The one door (gotcha 36), and the match that follows it is the mechanism (gotcha 39): asking for the Dirty means naming the case where a Snapshot is owed instead.

+ +

#Host::relay / Host::project

+

Who hears a relayed command is the relay's own business, not the host's guess: Broadcast names its audience and the match is what makes the host honour it (improvise-gvnb).

+

project drains the outbound projections — the ticket's "projection queue", which turned out to be a call rather than a queue. Clients with nothing to hear are omitted by project_all and so get no frame.

+
+

#ui/viewport.rs and ui/cat_tree.rs — the residue of two shared derivations

+

#viewport.rs

+

MAX_ROW_HEADER_W/MIN_ROW_HEADER_W moved to improvise-protocol with the rule that clamps to them (improvise-1125): a browser sizing its own row header must reach the same numbers, and it cannot depend on this crate.

+

compute_col_widths computes adaptive column widths for pivot mode (header labels

+ +

joined multi-level string, matching how the grid renderer draws each level on its own row with repeat-suppression.

+

compute_row_header_width is the total row-header width for a layout. The rule itself is improvise_protocol::header_level_widths, which is where both front ends reach it (improvise-1125). This is the one caller that holds a GridLayout rather than a projected frame — App::visible_cols, deciding how many columns fit — and it is on the keypress path, so it hands the rule borrowed tuples rather than building a frame and cloning every row. It does not reach for a View either, because a collapsed flag cannot change a width.

+

#cat_tree.rs

+

The derivation itself moved to improvise_protocol::viewmodel (improvise-edp), because the browser client needs it and can only reach the protocol crate. What is left here is the path that call sites in this crate and in the TUI already spell, and the one convenience that needs an App: building the cache before flattening it.

+

build_cat_tree builds the flattened tree straight from the model, for a host that holds the authoritative state rather than a folded cache — the same shortcut project::category_tree_of is for the widgets.

+
+

#command/keymap.rs — the dispatch/registry half

+

#What is here and what is not

+

The table — KeyPattern, Binding, Keymap, ModeKey, TextEntry, PanelKeymap and Keymaps::default_keymaps() — lives in improvise_protocol::keymap, so the browser client looks up the same bindings the terminal does rather than mirroring them. Read that module's docs for why.

+

What is here is what a thin client cannot have: resolving a bound name into Effects through a CmdRegistry, and the SetTransientPrefix effect that arms a prefix key. This file is the authority on that split — the doc review named it as such — so a claim about where the keymap lives belongs here and in improvise-protocol::keymap, and nowhere else.

+

#SetTransientPrefix (improvise-jc8)

+

Arms a prefix key, so the next keypress is dispatched against that prefix's sub-keymap. It records the address of the prefix — the key that opened it — rather than the sub-keymap itself. KeymapSet already owns every keymap, so holding an Arc<Keymap> in ViewState would only put a non-portable type into what is meant to become portable, shared state (gotcha 13).

+

#effects_of — why a Sequence flattens, and why it stays that way (improvise-0hb)

+

The one place a keymap lookup becomes a mutation, and the reason Binding itself can be registry-free data.

+

Every step is executed against the same pre-dispatch ctx, and the effects are concatenated into one vec that App::apply_effects runs as one batch. So a later command cannot observe an earlier one's effects — only a later effect can. improvise-0hb proposed closing that: apply effects so far between steps, rebuild the layout, build a fresh CmdContext, dispatch the next. The design pass says no, on three findings.

+

It would silently reopen improvise-8tk, a P1 data loss. apply_effects resets abort_effects at entry and again when it breaks, because the flag is per batch. Applying between steps therefore dispatches step 2 into a fresh batch with the flag cleared, so the Quit after a failed Save would land — the exact silent discard 8tk fixed. Gotcha 21's claim that the abort "carries across sequence steps" is true because of this flattening, not in spite of it. It was also untested until recently: no shipped Sequence has a first step that can abort, and ZZ is a prefix onto the single wq command. app.rs::a_sequence_whose_first_step_aborts_does_not_run_the_second is the guard, and it fails when the abort is made per-step.

+

Almost nothing would benefit. Of the shipped Sequence bindings, all but one come from TextEntry and are either [<commit>, clear-buffer <buf>] or [clear-buffer <buf>, enter-mode <exit>] — clear-buffer and enter-mode take their operand as a literal arg and read nothing from ctx, so a refreshed ctx would change no outcome. The single genuine composition is records o = [add-record-row, enter-edit-at-cursor], and it already works, because EnterEditAtCursorCmd fronts its effect with RebuildLayout and the effect re-reads the layout itself (gotcha 23). Every Esc and every commit keypress would pay a layout rebuild and a CmdContext construction — which recomputes column widths — to buy that nothing.

+

And the ticket's motivating rewrite does not typecheck as a binding anyway. Its [(w, []), (force-quit, [])] cannot dispatch: w is register_pure, whose interactive constructor rejects an empty arg list, so run returns None and the whole binding is dead. The nullary spelling is save. every_bound_command_name_is_registered checks exactly this (gotcha 15) and would have caught it.

+

If command-level transactionality is ever wanted, it wants its own campaign: the abort must become a property of the sequence rather than of the batch before any step is applied separately.

+
+

#command/ — the command layer

+

#lib.rs / command/mod.rs / command/parse.rs

+

The crate is the Cmd/Effect pipeline, the keymaps and the App state machine, with no terminal dependency; rendering lives in the root improvise crate. All model mutations go through the command layer so they can be replayed, scripted and tested without the TUI: commands are trait objects (dyn Cmd) producing effects (dyn Effect), and the headless CLI (--cmd / --script) parses quasi-lisp text into effects and applies them directly.

+

improvise_protocol is re-exported as crate::protocol so a host — the ratatui binary, a wasm worker — can name a widget cache without taking its own improvise-protocol dependency, and so the cache types a host renders are the same ones ui::project builds (improvise-rkeq).

+

command::key re-exports improvise_protocol::key (improvise-cqi): the key types ride in ViewState::transient_prefix, so they must live where the wire types do, and the re-export keeps crate::command::key::… paths working.

+

parse.rs is the quasi-lisp prefix parser: word arg1 arg2 …, several commands per line separated by ., coordinate pairs written Category/Item, and quoted strings ("Profit = Revenue - Cost").

+

#cmd/core.rs

+

CmdContext is the read-only context commands decide against. Commands receive a &Model (pure data) and a &View (the active view); the full &Workbook is also available for the rare command that needs to enumerate all views, such as the view panel.

+

InteractiveFn constructs a Cmd from the interactive context (keymap dispatch). It receives both the keymap args and the interactive context, so commands can combine text arguments (a panel name) with runtime state (whether that panel is currently open). register_pure registers a command that does not need interactive context: called interactively with args it delegates to parse, and without args it errors.

+

write_buffer is the effect that writes a named buffer — the mirror of read_buffer, and the only place that knows "search" is not an entry in the buffers map but its own ViewState field (improvise-gdl, gotcha 11). That knowledge used to sit inside the SetBuffer effect, where nothing paired it with the read side and every caller had to know that passing "search" meant something else than it said.

+

#cmd/cell.rs / cmd/navigation.rs — the explicit-state convention

+

All cell commands take an explicit CellKey; the interactive spec fills it from ctx.cell_key() and the parser fills it from Cat/Item coordinate args. Navigation commands take explicit cursor state the same way: the interactive spec fills position and bounds from context, and the parser accepts them as args.

+

page_cat_data gathers (cat_name, items, current_idx) for page-axis categories, deliberately using categories_on(Axis::Page) and not filtering_cats(): Filter coordinates constrain the data but are not navigable, so [ and ] must not page them (improvise-dz8, gotcha 27).

+

At bottom-right EnterAdvance has no place to go, so it emits a single AbortChain — the advance premise no longer holds, and the caller's trailing effects (a CommitAndAdvance's EnterEditAtCursor) are skipped.

+

#cmd/commit.rs — the two commit paths (gotcha 19)

+

commit_regular_cell_value commits a cell value: for synthetic records keys it stages in drill pending edits while in drill mode, or applies directly in plain records mode; for real keys it writes to the model.

+

rename_record_coordinate renames one coordinate of a records row. In wide format a row is a group of cells — one per measure — so the rename has to move every one of them, not just the cell under the cursor (improvise-c5v, gotcha 1). Every cell in the row moves, and a row seeded by add-record-row has no measure cells yet: its coordinates live on a lone measure-less cell, which must move too or the row loses its identity.

+

add_measure: typing a name into the trailing column creates that measure, which gives it a real column to type values into. Adding a measure is a schema change, so it applies immediately even inside a drill.

+

add_category (improvise-2zq): typing a name into the +cat column creates that category, which gives the row a coordinate column to type into. It creates the column and does not set the coordinate on the current row — the deliberate answer to the ticket's third question, for two reasons. It is symmetric with the measure +, which creates an empty column and writes no value. And setting it would move every measure cell in the row onto the new coordinate (gotcha 1): a data migration across N cells attached to the same Enter that made a schema change, in an app with no undo. The follow-up edit is one l away in the column that just appeared. Like add_measure it is a schema change, so it applies immediately rather than being staged.

+

inserts_a_column asks whether committing key inserts a column, invalidating every column index to its right. It is asked of the layout — record_col resolves the synthetic _Dim position into what the column actually is — rather than re-derived from the value or the cursor, so there is one answer to "what is this column" (gotcha 1).

+

exit_mode_for returns the normal-mode counterpart of an editing mode, used by CommitAndAdvance to compute the mode to land in if the advance aborts.

+

CommitAndAdvance commits a cell edit, advances the cursor and re-enters edit mode, subsuming the old CommitCellEdit (Down) and CommitAndAdvanceRight (Right). edit_mode is the editing mode to re-enter after advancing, supplied by the keymap binding — the editing-mode keymap passes editing and the records-editing keymap passes records-editing. The command itself never inspects ctx.mode.

+

Two orderings inside it are load-bearing.

+

The pre-emptive drop to normal goes first. It drops to the normal counterpart of edit_mode; if the advance succeeds the trailing EnterEditAtCursor lifts back into editing on the new cell, and if the advance aborts (already at bottom-right on Enter) that effect is skipped and the user lands in normal mode — the desired "Enter at bottom-right commits and exits" behaviour. It goes before the commit, not after (improvise-2zq): the commit can abort the batch itself — AddCategory does so at MAX_CATEGORIES — and an abort before this ran left the app in the editing mode, where draw_bottom_bar renders the minibuffer instead of the status line and the refusal the effect had just written was never drawn. Tidy-up goes where an abort cannot reach it; the same rule as ExecuteCommand's (gotcha 21).

+

A schema change that inserts a column suppresses the advance. Inserting moves every index to its right, and the advance is computed from ctx — the pre-commit layout, because apply_effects rebuilds only after the whole batch (gotcha 23). On the last row Enter wraps to col + 1, which after the insert is +cat again, so the editor reopened category creation and a user naming two categories in a row never reached the column they had made. Tab reached it the same way. Not advancing is also the useful answer rather than merely the safe one: add_category inserts the new column exactly where the cursor already is, so staying put lands on it. The measure + shares none of this, and not by luck — it is the last column, so there is no column to its right for an index to shift into, and both advances are already special-cased at the bottom-right edge.

+

CommitKind is what a minibuffer commit produces. The variants differ only in the model mutation they build and in where they leave the user — reading the buffer, trimming it, bailing out when it is empty, marking the model dirty and reporting are shared by CommitBuffer (gotcha 20). Item carries its target category, resolved from AppMode::ItemAdd when the command is constructed, so execute stays a pure function of the kind and its buffer. parse_commit_kind resolves the argument against the current mode: Item is the only kind that needs anything from the mode, so a script line (which has no live mode) can build the other two.

+

CommitKind::on_commit says where a successful commit lands the user. None keeps the minibuffer open for the next entry — categories and items are entered in batches, formulas are not.

+

CommitBuffer commits the contents of a minibuffer as a formula, a category or an item; buffer clearing is the keymap's (Enter → [commit, clear-buffer]). The add effect goes last on purpose: it is the only party that knows whether it worked, so the optimistic status must not clobber the error it reports (improvise-6t3).

+

CommitSave commits a save path from the save buffer. It is the sibling of CommitExport and deliberately not the same command: the export prompt writes CSV, this one writes the model through SaveAs, which also adopts the path so a later plain Save has one (improvise-lae). An empty path bails rather than reaching SaveAs, which would otherwise ask the storage backend to write to "" and report its error instead of the one the user can act on.

+

Its mode change comes first, and that ordering is the whole reason the user sees a failed save (found by Copilot on PR #27). SaveAs aborts the batch when the write fails (improvise-8tk's other arm), so anything after it is skipped. With the order reversed the batch stopped while still in SavePrompt — and draw_bottom_bar renders the minibuffer instead of the status line, so the "Save error:" set by the effect was never drawn. Leading with the mode change means the error lands in Normal, where the status bar is what the bottom row draws. It costs the typed path, which the welded clear-buffer would have taken on a success anyway and which EnterMode clears on the next Ctrl-W regardless (gotcha 11) — so nothing durable is lost, and the convention that the side effect runs last and its message wins is the one gotcha 20 already states.

+

#cmd/mode.rs

+

EnterMode clears the corresponding buffer when entering a text-entry mode, through write_buffer — the only code that knows a runtime-named buffer may not be an entry in the buffers map (improvise-gdl).

+

SaveAndQuit saves then quits; the Quit only lands if the Save succeeds, because a failed save aborts the effect chain, so wq/ZZ leaves the user in the app with the error visible rather than discarding the model (improvise-8tk).

+

It looks like an AAndB command and is not one. wq is the user's name for the composition, not an invented internal one: it is registered, so :wq reaches it, and Quit's unsaved-changes message advertises that spelling. A keymap Binding::Sequence cannot be reached from the : line, so replacing this with one would either delete the ex-command or leave two independently-written spellings of "save then quit". One definition behind two surfaces is what this already is. Pinned by text_buffer::tests::colon_wq_saves_then_quits.

+

EditOrDrill is the smart dispatch for i/a: if the cursor is on an aggregated pivot cell (categories on Axis::None and the cell is not a synthetic records-mode row) it drills instead of editing; otherwise it pre-fills the edit buffer with the displayed cell value and enters edit_mode. edit_mode is supplied by the keymap binding, so the command itself is mode-agnostic. Only regular (non-virtual, non-label) categories on None count as true aggregation: virtuals like _Index/_Dim are always None in pivot mode and imply nothing.

+

EditOrDrill::nothing_at_cursor is what to say when there is no cell under the cursor at all. improvise-ct5: it used to say "cannot drill — no cell at cursor" in both modes. Drilling is a pivot-mode concept, and in records mode — where every regular category sits on Axis::None, so this branch is the one an empty grid always takes — the user's model is "a table I type into", with o one key away to add a row. Naming drilling there described a mechanism the user is not using instead of the key that would help. The choice reads edit_mode, the argument the keymap supplies, rather than asking ctx what mode it is in, so the command stays as mode-agnostic as it claims and no runtime is_records() check comes back (improvise-4ju).

+

EnterEditAtCursorCmd is a thin command wrapper around the EnterEditAtCursor effect so it can participate in Binding::Sequence; target_mode is the keymap binding's argument.

+

#cmd/grid.rs

+

DrillIntoCell creates a _Drill view with _Index on Row and _Dim on Column (records/long-format view); fixed coordinates from the drilled cell become filters.

+

If drilling into a formula cell, the formula target is stripped from the key so matching_cells finds the underlying raw data records instead of returning nothing (gotcha 10).

+

Fixed coords go to Filter, an immutable constraint of this drill rather than a dimension to page through (improvise-dz8, gotcha 27). Previously-aggregated categories (none_cats) stay on Axis::None so they do not filter records; they appear as columns in records mode. Virtual categories are skipped, _Index/_Dim having been set already.

+

ToggleRecordsMode toggles between records and pivot mode using the view stack: entering creates a _Records view and switches to it, leaving navigates back. Leaving first cleans up any records with empty CellKeys — produced by AddRecordRow when no page filters are set — which is the inverse of the SortData that runs on entry.

+

#cmd/panel.rs

+

CyclePanelFocus tabs through open panels in order Formula → Category → View, wrapping. The starting point is the panel currently focused (ctx.mode); from a non-panel mode focus goes to the first open panel. With a single open panel it self-transitions back to it; with none open it is a no-op.

+

DeleteCategoryAtCursor deletes the category or item at the panel cursor: on a category header the whole category, on an item row just that item.

+

CreateAndSwitchView creates a new view, switches to it and returns to Normal. Not a shim, despite the name (improvise-aaw). The new view's name is derived from ctx — format!("View {}", views.len() + 1) — and a keymap Binding::Sequence can only carry literal args, so the proposed [create-view X, switch-view X, enter-mode normal] would hard-code one name and the second n in the view panel would collide with the first. The derivation is the decision that makes this a command.

+

The ticket's two fallbacks are both behaviour changes wanting a product decision, not refactors: prompting for a name is a new feature, and making create-view switch by default changes :create-view for scripts and duplicates what grid.rs does by hand for _Drill/_Records.

+

#cmd/registry.rs

+

parse_mode_name decodes a mode-name string (as supplied by enter-mode/edit-or-drill keymap bindings) into an AppMode. The table itself is AppMode::from_name, beside AppMode in the protocol crate: the spellings are keymap vocabulary and the keymap is shared, so the browser client decodes enter-mode from the same list (gotcha 16).

+

The commit registrations take a mode-name arg ("editing", "records-editing") as args[0]; the keymap supplies it and the command never inspects ctx.mode.

+

#cmd/text_buffer.rs

+

ExecuteCommand executes the : command buffer by delegating to the command registry — the : prompt is just another front end to the scripting language, using the same parser as improvise script.

+

Its batch leads with ReturnToNormalUnlessMoved, before anything that can fail runs. The mode is still decided at apply time rather than up front, because ViewBack, ViewForward and StartImportWizard move the mode only on some runs and no up-front answer is right (improvise-toc, gotcha 31). Running it at the head does not weaken that: whatever a later effect sets, it sets after this, so the mode a : line ends in is unchanged in every case — the only difference is what happens when the batch never reaches the end.

+

Which is the point. A failed Save/SaveAs sets abort_effects (gotcha 21), and an abort at the tail ate this restoration too, stranding the user in CommandMode with the stale command line drawn — where draw_bottom_bar renders the minibuffer instead of the status line, so the error the effect had just set was invisible. The abort must stop the chain (:w bad . q must not quit) and must not stop the bookkeeping, and ordering answers that without giving any effect a "runs anyway" flag: tidy-up goes where an abort cannot reach it.

+

#cmd/tile.rs

+

TileAxisOp cycles or sets the axis for the category at the tile cursor and stays in TileSelect mode so the user can adjust several tiles. axis: None cycles; axis: Some(a) sets.

+

#cmd/effect_cmds.rs

+

The parseable model-mutation commands are thin Cmd wrappers around effects, constructible from string args, sharing the same execution path as keymap-dispatched commands (gotcha 16).

+

#cmd/wizard.rs (improvise-2wm, improvise-907)

+

Import-wizard key handling. The wizard is a small step machine, and which key means what depends on the step it is in. That decision is pure, so it lives here: the command reads the wizard out of CmdContext and emits the discrete effects (WizardApply, CloseWizard, AcceptWizardModel) that carry it out.

+

done_keys is the confirmation step: the workbook is built and waiting, Enter installs it, and both Esc and Left return to NameModel — unlike every other step, Esc here does not throw the import away.

+

back_key: Left steps back to the previous step, on every step. It is the one key free everywhere — Backspace edits the name and the formula buffer, and h is a printable character the name step needs.

+
+

#command/cmd/fit.rs — fit-measure

+

Evolve a decision tree over what the active view shows and store it as an ordinary improvise formula. After this command there is nothing GP-specific left in the document — the rule is a ## Formulas line like any other, recomputed by the fixed-point evaluator, editable in the formula panel, drillable, and saved with no new grammar (improvise-21d's enabling fact, and improvise-7na's answer).

+

#The fit runs inside execute, and that is the architecture rather than a shortcut

+

A Cmd is pure over &CmdContext and returns effects; an Effect is the only mutation site. A fit is decision logic — a pure function of (Model, View, target, seed, FitConfig) that mutates nothing — so it belongs on the command side, and what the effects carry is its result. Putting the search inside an Effect::apply would have mixed an expensive stochastic computation into the one place the app mutates, which is the split's whole point.

+

What this does not buy is asynchrony. The command runs on the thread that draws the next frame, so a fit is bounded rather than cancellable: FitRequest::default is a deliberately smaller search than the CLI's (improvise-vx3's AC 4 asks for "cancellable or bounded"). Moving it off the main loop is a change to draw.rs's event loop, not to this file, and nothing here forecloses it — the fit is already a pure function that could be handed to a worker and have its FitOutcome posted back as the same effects.

+

#What a refusal is for

+

improvise_gp::fit_measure fails rather than storing something misleading — most sharply when the champion splits on an encoding artefact like Region=North, which is not an item of anyone's document, so every cell of the stored measure would be Error("ref:…") (improvise-8xhy, gotcha 37d). This command is where a user actually meets that restriction, so the refusal goes to the status line whole: it names the features that could not be resolved, because "the fit failed" is not something a user can act on and "it split on Region, which is a category rather than a measure" is.

+

Both the success and the failure lines are flattened to one line: several refusals interpolate a document name, and a name may legally contain a newline (Copilot on PR #25). A failure is when a user is most reading the status bar, so it is the last place to let one split. The report goes first and AddFormula last, so a formula the emitter spelled but the parser will not take leaves its message on screen rather than this command's optimistic one (gotcha 20).

+

#What registering it costs the browser, measured

+

improvise-command is in the worker bundle, and the path from App::new → default_keymaps → default_registry → this command's prototype is live, so gp-tree links into it whether or not a browser ever invokes fit-measure. kb.pl names keeping GP out of the wasm bundle as improvise-gp's reason to exist, so this is a cost to state rather than to discover: built with wasm-pack --release, improvise_worker_server_bg.wasm goes 1,341,130 → 1,420,470 bytes (+79 KB, +5.9%), or 498 → 531 KB gzipped (+6.5%). improvise-wasm-client — the main-thread bundle — does not depend on improvise-command at all and is unchanged.

+

Accepted rather than feature-gated, for two reasons. The worker already carries the whole App, persistence and the import wizard, so this is a 6% addition to something an order of magnitude larger; and a browser fit is a wanted end state, not dead weight — improvise_gp::fit exists where it does precisely so a second host can run the same sequence. If the bundle ever needs shrinking, a default-on cargo feature that web/build.sh turns off is the lever, and these are the numbers to beat.

+

#parse_seed and the seed argument

+

A seed as the status line spells it (0x…, either case) or in decimal. The hex arm is not a convenience: FitOutcome::status_line prints {:#x} and tells the user that is what reproduces the fit, so accepting only decimal made the advertised round trip fail on the advertised value.

+

The seed is an argument rather than a clock read for the reason the CLI fixes its own default: two fits of the same document should agree, and a user who wants a different search asks for one (gotcha 37b).

+

#The active view is split (improvise-jytm)

+

Workbook::active_view is the document's default — what a freshly connected client opens, and what .improv writes as Initial View. ViewState::active_view is authoritative for the client that holds it. Two clients may sit on different views of one document: the model is shared and the UI is not (improvise-g0f, user ruling 2026-09-19).

+

The old guard, the_two_active_view_names_never_diverge, is replaced rather than deleted, by the_documents_default_view_outlives_this_clients_switches and by a_fresh_session_opens_the_documents_default_view — a deliberate split with no guard is how this silently regresses.

+

#What the split moved

+

App::switch_to_view writes only view_state.active_view, and only for a view the document has. App::resolve_view is the one answer to "what does this name name in this document", falling back to the default so a slice naming a deleted view cannot panic in rebuild_layout; view(), view_mut(), view_name() and project::grid_in all go through it rather than each testing views.contains_key. Two copies of that fallback would be a derivation done twice — they would agree until one of them learned about _Drill. Every effect that used to write workbook.active_view_mut() now writes app.view_mut() — the View is still document state shared by everyone on it; what changed is which one this client means.

+

effect::SwitchView, effect::ViewBack and effect::ViewForward are therefore EffectKind::ViewOnly. Gotcha 3 says a mis-declared kind is silent, so this was re-derived rather than assumed: after the split each of the three writes view_state and nothing else, and a ViewOnly effect is exactly one that reaches no other client. A sound derivation is not a guard, though, and review found that only SwitchView had one — putting either sibling back to Model(LayoutGeometry) was invisible across the whole suite. walking_the_view_history_moves_no_document_state closes that, and it asserts the consequences rather than the declaration because those are what a reader can price: a mis-declared view-back raises model_state.dirty (so the browser worker rewrites the whole OPFS document, gotchas 41 and 45) and bumps layout_version (so every other client's next Command::Model is refused LayoutMoved, ADR 001) for a move only one client made. That re-classification is also what takes ViewBack/ViewForward off the wire, since ModelCommand is derived from the Model(..) effects — and it is the right answer for a second reason, the one that removed ApplyAndClearDrill: their operand is ViewState::view_back_stack, session-local residue belonging to whoever navigated (gotchas 30, 38).

+

#What projects per client

+

Every projection is view-dependent except the formula list and the view list, so grid_in, category_tree_in and tile_bar_in take the view and ClientSession::project_diff passes its own. All three need guarding, not just the grid: review found that only grid_in had a check, and pointing ClientSession::view_in at the host's view left the whole suite green. The one that matters most is the tile bar — a category's axis is what that widget is, so a client on another view would be shown the wrong axis in the control it reassigns axes with, beside a grid that was right. grid_in reuses App::layout when the session names the view the host is on — which is the common case and the only one that can carry a drill snapshot — and derives a GridLayout otherwise.

+

A session on a view that is deleted is the other half, and it is answered in adopt_deleted_views rather than here: project can only describe a view, it cannot move a client off one.

+

Testing any of this needs a bystander that can be moved. Both the refused-switch and the deleted-view tests first sat their bystander on the document default, where a relay addressed to everybody folds to a no-op — so the assertion read as coverage and caught nothing, which review demonstrated by injecting ToAll twice. Both now put the bystander on a view it can be moved off, and both read the audience off the Broadcast itself rather than inferring it from folded state. The audience is the message; that is what makes it the thing to assert.

+

A switch is ViewOnly, so Dirty cannot carry it. ClientSession therefore observes its own move: projected_view records the view the mirror's grid was built from, and a mismatch projects Dirty::EVERYTHING for that session alone. This is the cursor_epoch shape (gotcha 24) — watch the state move rather than keep a second table of which commands move it.

+

#Two gaps this opens, both filed rather than smuggled

+

A view-scoped ModelCommand names no view (improvise-lltt), and it is deferred on a condition, not a judgement: it is unreachable only because no shipped client emits Command::SwitchView, while the browser already emits TransposeAxes. SetAxis, CycleAxis, TogglePruneEmpty, SetNumberFormat and their siblings act on "the active view", which is now ambiguous: apply_model runs them against the App's own slice, so a connected client on another view moves the host's view rather than its own. That was consistent before the split, because every client was on the same view. The honest fix is for those variants to name their view, which is a protocol change.

+

Initial View no longer follows the TUI user (improvise-azhb). A switch moves no document state, so opening a document, switching view and saving writes the view it was loaded with. Whether a host's own switch should also set the document's default is a product question, not something to decide inside this ticket.

+

#"What text is in this cell" has one answer, plus an overlay (improvise-tom1)

+

The user has ruled: a remote client must not see another client's staged drill edits. They are uncommitted writes belonging to whoever staged them, which is why gotcha 30 keeps drill_state off ViewState and why gotcha 38 took ApplyAndClearDrill off the wire. They become everyone's business at ApplyAndClearDrill, which is CellData and already reaches every client. So the invisibility is correct and was never the defect.

+

The defect was that three places answered the question. Two of them are now one, and the third turns out to have stopped being an answer.

+

GridLayout::records_value is the resolution. It returns the CellValue, and records_display is one to_string over it, so the text can never disagree with the value it is a spelling of. project::cell_display used to call records_display and then guess numeric-ness with text.parse::<f64>() — the one place the producer guessed rather than knew, and it was wrong in a way nothing caught: a coordinate column holds an item name, so a Year category with items 2024/2025 projected a coordinate the browser right-aligned as a number. It now reads the value and matches.

+

A coordinate is CellValue::Text by construction, not by a check, which is why the fix is in records_value rather than in a guard at the producer: there is no spelling of an item name that could route it to the numeric branch.

+

src/ui/grid.rs was not the third answer. Since gotcha 44 the TUI grid takes its text off GridCache and then overlays drill_state.pending_edits — that is the one derivation plus a session-local overlay, which is exactly the shape the ruling wants, and App::display_value_of does the same for the editor. The ticket's "three answers, two of which agree" was written before anr9 landed and is worth correcting rather than repeating.

+

What must stay true, and is now asserted: the session that staged an edit sees it through display_value_of, and grid_of projects the committed value to everyone, including that session's own grid. The producer never reads drill_state, and teaching it to would put one client's uncommitted typing on another's screen.

+
+
+ + diff --git a/site/docs/code/core.html b/site/docs/code/core.html new file mode 100644 index 00000000..14637987 --- /dev/null +++ b/site/docs/code/core.html @@ -0,0 +1,159 @@ + + + + + +improvise-core — improvise docs + + + +
+ +
+

Source: context/code/core.md

On this page

improvise-core — the arguments

+

Written for a reader with the code open. This is where the reasoning that used to live in crates/improvise-core/**'s comments now lives; the comments themselves are capped at two lines (design-principles §6). Bug narratives keyed by test name are in context/test-evidence/core.md.

+

The crate is pure data: Model, View, GridLayout, Workbook, number formatting. It depends on improvise-formula for AST types and knows nothing about UI, IO or commands.

+
+

#1. CellKey is canonical by construction

+

CellKey(Vec<(category, item)>) is always sorted by category name, and the constructors are the only way to get one: CellKey::new(coords) sorts, with(cat, item) replaces-or-inserts-and-re-sorts, without(cat) filters and so preserves the order. Equality, hashing and storage all rely on the canonical form, so two keys naming the same intersection in different orders are the same key. Never build the inner Vec directly (gotcha 5).

+

That without preserves the sort is what lets record_of be a one-liner rather than a normalisation step — see §5.

+

#2. Aggregating a cell's raw values

+

Three functions, deliberately with three different emptiness rules, and the differences are the whole content of the design (improvise-kdp):

+ +

#evaluate_aggregated takes Aggregation, not a list it cannot read

+

The parameter was none_cats: &[String], and the body consulted it exactly once, for emptiness — a boolean wearing a list's type, since the aggregation itself runs over key.0 through matching_values. It never filtered by that list, in any revision: git log -S back to the commit that introduced Axis::None shows the same is_empty() and nothing else. So the name promised a filter the body has never performed, and the next caller to build a careful list would have threaded it into a parameter that cannot use it.

+

Aggregation::{Exact, OverHidden} admits only the two states the body distinguishes, and Aggregation::of(none_cats) performs the reduction once, at the boundary that holds the list (improvise-5jm5, doctrine(an_invariant_is_made_unrepresentable_not_validated) — the same answer prune_axis_items and gp-tree's Row<T> got). It is an enum rather than a bool because the two states have names worth reading at the call site, and the regression guard is the signature: the thing the ticket describes — two different lists, same emptiness, same answer — is no longer constructible.

+

Note what does not change: recompute_formulas and aggregate_raw keep &[String], because they walk the elements (stripping each hidden coordinate off a stem). Two parameters that looked alike were doing two different jobs.

+

#Non-numeric values reduce by unanimity, and the branch order is the rule

+

aggregate_cell_values checks the all-numeric branch first. If every value is a number it reduces through apply_agg; otherwise, if every value equals the first, the result is that value; otherwise it is CellValue::Error("mixed").

+

That ordering is the entire difference between this rule and "a single value passes through". Count is the only function that can tell them apart — every other function is the identity on a one-element set — so Count of a lone number must still be 1 and not the number itself. a_lone_number_still_counts_ as_one_rather_than_passing_through is the pin.

+

What the rule forbids is dropping. The old code did filter_map(as_f64), and since _Index/_Dim sit on Axis::None in every default view, none_cats is non-empty almost always and the aggregating path was almost always taken — so every CellValue::Text in every document rendered blank, including the pipe-quoted text .improv carries as a first-class value (Region=West, Measure=Revenue = |pending|). A user could type a text value, save it, load it, and never see it again (improvise-l4qx).

+

Both evaluate_aggregated and aggregate_raw share the helper, so a formula ref over mixed values now errors "mixed" rather than silently summing the numeric ones and reading as authoritative while under-reporting.

+

#3. Formula evaluation

+

Fixed point. recompute_formulas(none_cats) clears formula_cache, collects the distinct coordinate stems (each data key with the formula's target category and every none_cat stripped), and iterates stem × formula target until nothing changes, bounded by MAX_EVAL_DEPTH = 16. Chained formulas (Margin = Profit / Revenue over Profit = Revenue - Cost) converge naturally; circular references converge to CellValue::Error("circular") rather than overflowing the stack. The stems are an IndexSet: dedupe is O(1) per cell and insertion order is preserved (improvise-6os replaced an O(n²) Vec::contains).

+

There are two evaluators, and they must be changed together:

+ +

A rule stated in one and not the other is a live bug; improvise-0bf was exactly that (see §4).

+

find_item_category searches real categories first and then formula targets, because add_formula never inserts its target into _Measure (improvise-nrs). Without the fallback, a reference to a computed measure fails to resolve.

+

Display rounding never reaches eval. format_f64 is called only in rendering paths. Feeding a rounded value back into a formula is the bug formula_chain_preserves_full_precision exists to catch.

+

#4. Float equality has one epsilon, and the div/0 guard shares it

+

FLOAT_EQ_EPSILON = 1e-10 is used by =/!= and by the division-by-zero guard, in both evaluators. The two must agree or IF(x = 0, a, y / x) is incoherent: the condition calls x zero and the else branch divides by it anyway. With x = 1e-11 the old exact rv == 0.0 guard returned 1e13 (improvise-0bf). 1e-10 is far below typical data magnitudes but absorbs accumulated f64 rounding noise from chained formulas.

+

Note the repo map's gotcha 17 (which called this "currently mixed") describes the pre-fix state; the guard and the comparison share the constant now.

+

#5. Surrogate identities — model/record.rs (ADR 002)

+

A client sees a grid of label strings indexed by viewport position. It holds no Model, no category names and no page/filter constraints, so it cannot build a CellKey — which is why improvise-q08 found a thin client cannot compose SetCell. The answer is an opaque surrogate the server projects per visible cell and the client echoes back.

+

Two surrogates, because two kinds of thing are nameable and only one is settable. RecordHash names a record — the coordinate tuple minus _Measure (record_of). That unit is not chosen here: it is already ADR 001's unit of atomicity and already what the write counters are keyed by, because renaming a category coordinate in wide records mode moves every measure cell in the row (§6). One identity therefore serves addressing and staleness. AggregateHash names an aggregate's coordinate constraint — row and column coordinates plus the page and filter selections that decide which stored cells it sums (View::filtering_cats(), §7). An aggregate is not the absence of an identity; it is a different kind of identity, which is what makes Drill addressable at all. It is deliberately not normalised: a measure on Row or Column is part of what the aggregate sums, so dropping _Measure would give two visibly different totals one name.

+

The hash is specified here, not inherited. Three ways it breaks silently, each producing a wrong write rather than an error:

+
    +
  1. Not DefaultHasher, not RandomState. std's default HashMap hasher is randomly seeded per process, so a surrogate would not survive a restart or agree between two processes. This is SipHash-2-4 under a constant key.
  2. +
  3. Not #[derive(Hash)]. Rust gives no stability guarantee for a derived Hash across compiler versions or platforms. What is hashed is the explicit byte serialization in hash_coords, so the surrogate is defined by that format rather than by whatever the compiler did this week.
  4. +
  5. 128 bits, argued rather than defaulted. A collision is a silent write to the wrong record — precisely what got _Index rejected as a surrogate — and no later check catches it: a colliding surrogate resolves successfully to a record the client never saw. 64 bits reaches birthday risk around 2^32 records, which a pivot tool will not see by live record count; but the risk is over every record a long-lived document ever names, and the consequence is unbounded and undetectable. The other choice costs 8 more bytes per visible cell — 8 KB on a full 50×20 projection and far less on the diffs that carry steady-state traffic. Bounded cost against silent corruption is not close.
  6. +
+

Serialization details that are load-bearing. The byte format is tag ‖ count ‖ (len(category) ‖ category ‖ len(item) ‖ item)*, every length a little-endian u64. The lengths are what make it injective: without them ("ab","c") and ("a","bc") serialize alike. Only Hasher::write is used, never write_u64/write_u8, which are free to feed the hasher in native byte order. The SipHash key constants are written as big-endian readings of b"improvis"/b"e-record" so the constant is the same on every platform. A domain tag byte (0x01 record, 0x02 aggregate) is hashed first, because a record and an aggregate may be built from the same coordinates — an aggregate over a single record is the degenerate case — and they must not hash alike.

+

Hex on the wire, not a JSON number. serde_json writes a u128 as a bare number and a JavaScript host parses every JSON number as f64, so anything above 2^53 comes back changed — which for a surrogate means resolving to a different record or to none. The wire form is a fixed 32-character lowercase hex string.

+

#6. Records mode is wide format (view/layout.rs)

+

A row is one coordinate tuple — the cells sharing everything but _Measure — and each raw measure gets its own editable column. The columns are [coordinates…, +cat, measures…, +]: one per regular category, then the column that creates a new category (improvise-2zq), then one per raw measure, then the column that creates a new measure (improvise-c5v). Each + sits adjacent to what it adds to; two trailing columns would have made the user read a label to tell them apart, and position says it instead. Formula targets are not columns — they are computed, so there is nothing to type into them.

+

Consequences: two measures on one coordinate are one row; editing a category column moves every measure cell in the row, not just the one under the cursor; and _Dim in the synthetic key carries the column index, not a label, because a category and a measure item may share a name and a label could not say which column an edit landed on. Resolve the index through GridLayout::record_col into a RecordCol. Drill pending edits are keyed by (row, RecordCol).

+

RecordRow::group collapses loose cells into wide rows in first-appearance order, so the result follows the store's order rather than a hash walk (§8). A cell with no _Measure coordinate still anchors a row — that is how add-record-row seeds one — but contributes no column value.

+

Records mode is a property of the view's axes, and View owns the one definition. View::is_records_layout() (_Index on Row and _Dim on Column) is what GridLayout::new asks and what on_category_added asks; two derivations would drift (improvise-edp, the disease this codebase keeps curing).

+

#prune_axis_items takes a predicate, not a flag vector

+

prune_empty hands it impl Fn(usize) -> bool over the data index — the position a row/col argument means, which group headers do not consume. The predicate's domain is derived from items itself, so there is no parallel length a caller can get wrong and debug and release builds cannot disagree about an out-of-range flag. The earlier &[bool] + debug_assert pair could have, had a third caller ever sized keep from something other than row_count()/col_count() (improvise-n3wt, doctrine(an_invariant_is_made_unrepresentable_not_validated)).

+

A group header survives only when a data item that follows it survives. In prune_empty the row pass does not touch col_items or has_value, so the column pass still reads the original grid. Records mode is exempt entirely — the user drilled in to see all the raw data.

+

#7. Axes: Page and Filter are two different questions

+

Axis is Row | Column | Page | Filter | None. Filter names a coordinate the drill fixed (improvise-dz8): like Page it constrains which records are visible, but unlike Page the user cannot navigate it. Axis::next() maps Filter to itself and no navigable axis rotates into it, so tile select cannot hand one out, and is_user_assignable() is false for it.

+

Picking the wrong question is the trap:

+ +

Axis also owns four presentation vocabularies — plain_label(), glyph() (which Display appends, giving "Row ↕"), short_label() (three columns for the tile bar; None is "·", not a truncation of "None") and initial() (one letter for the view panel; None is empty) — plus color() -> AxisColor. AxisColor is a semantic palette because improvise-core must not name a ratatui type; draw.rs::axis_fg does the one mapping, exactly as minibuf_fg does for MinibufColor (improvise-rml). The tile bar and the category panel each carried a copy of the colour table before, with nothing keeping them in agreement.

+

#on_category_added: a virtual on an axis is a configuration, not a vacancy

+

Virtual (underscore) categories seat on Axis::None. A regular category takes the first slot no regular category holds — Row, then Column, then Page.

+

This used to bump a virtual off Row or Column to make room, assuming a virtual sitting there was a placeholder. It never is, and the assumption cost two live bugs (improvise-2zq):

+ +

So a virtual is never moved. A records view seats the new category on Axis::None, which is where records mode keeps every regular category and where records_columns — which filters on kind.is_regular() and never looks at an axis — picks it up as a column immediately. Elsewhere the new category simply joins whatever virtual is already there, nesting a header rather than destroying one.

+

The negative arm is load-bearing and must stay tested: without a_pivot_view_still_seats_categories_row_then_column_then_page, "always seat on None" would pass every assertion about records mode. Probing confirmed the asymmetry — seating everything on None fails 12 tests, while removing the virtual bump failed none at all, which is how the bump came to be wrong in two directions with nothing noticing.

+

#Known defect: two answers to "is this name a virtual"

+

on_category_added tests cat_name.starts_with('_') (twice) while is_records_layout() four lines away asks CategoryKind::of_name. That is improvise-bidq and it is open; do not assume either spelling is the intended one when editing near it.

+

#8. Ordering is correctness, not tidiness (model/cell.rs)

+

DataStore holds cells in an IndexMap keyed by InternedKey, plus a secondary index HashMap<(Symbol, Symbol), HashSet<InternedKey>>.

+

An index whose values are HashSets is for O(1) lookup, never for iteration. matching_cells/matching_values used to walk the index's HashSet directly, so their order was hash order — different in every process, and different between two stores in one process, because RandomState keys each HashMap afresh. That defeated the whole reason cells is an IndexMap, and it was not tidiness (improvise-shyt). It caused, in ascending order of seriousness:

+ +

matching_positions is the fix: the index still narrows the candidates, and the primary map orders them — each surviving candidate resolves to its position with one hash lookup and the positions are sorted. Shape stays O(smallest set × |partial|) plus O(m log m) over the matches, not a scan. Measured at 18,000 cells: +20% worst case on matching_values, and matching_cells 7% faster, sorted positions resolving CellKeys with better locality. The alternatives both move the cost — an ordered index value type pays on every set; scanning the primary is O(store) per query.

+

An empty partial matches every cell, vacuously, and matching_positions is where that is answered (improvise-hkt6). It used to index sets[0] and panic; both public callers happened to guard, so it was unreachable — but the guard lived one level away from the indexing, which is how this class of bug returns. The guards could not simply be pushed down as vec![]: "no coordinates constrain anything" means all positions, and answering "none" here would have made removing a caller's fast path change behaviour silently. The callers keep theirs as a fast path, not as a guard.

+

live_records has a decided order, not an inherited one (improvise-2yc2). It walks cells and projects insertion order onto records: a record sits where its first cell sits. It deliberately pays O(cells) rather than O(records), because cells' order is already the answer matching_positions gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on RecordEntry, since a stored position is a copy and a record deleted and re-created would hold its old one while its cells sit at the end of the store. Note what is not on offer: an unordered return. Every iterator has an order, and a type that merely disclaims one still hands the caller hash order — the guard living one level away from the hazard, which is the shape this whole class of bug returns through. The function had no callers when this was decided, so the order was still free to decide rather than to discover; with no caller to benchmark against, correctness is the defensible default, and a hot-path caller can argue for a memoised position with numbers.

+

#The record index: RecordEntry

+

DataStore::records: HashMap<RecordHash, RecordEntry> answers both of a client surrogate's questions — "which record do you mean?" (ADR 002) and "is your basis stale?" (ADR 001) — from one entry in one lookup, because both are keyed by the record and both are invalidated at the same moment.

+

It lives in the store because this is where the same job is already done: the store already interns every name and already maintains a secondary index updated at exactly these two points (set, remove), and it is that existing (Symbol, Symbol) index which makes invalidation tractable — when a category or item disappears, the cells naming it are found without a scan, so the records built from them are too.

+

It is deliberately not regenerated from the layout on demand. That would be O(visible cells) per resolve and would only ever cover the visible window, so a record scrolled out of view would look identical to a record that no longer exists — conflating "reject, nothing to write to" with "apply".

+

It is not serialized: Deserialize rebuilds it through set, and every version starts at zero, which is correct because a freshly loaded document is a new document_generation and no client holds a basis against it.

+

cells == 0 is a tombstone, not a reason to forget the entry. A record that is deleted and then re-created must not hand a client holding the old basis its own version number back; dropping the entry would reset the counter to zero and make a stale basis look current, which is the silent accept the basis check exists to prevent. The counter tracks liveness, not writes — set bumps cells only for a genuinely new cell, and version is moved by the observed diff in App::apply_effects, never from set.

+

resolve_record returning None means the record is gone — never stored, or every cell removed, which is what happens when a category or item it named is deleted. That is one arm of staleness and needs no version comparison: there is nothing to write to. record_version reporting 0 for an unknown surrogate is deliberate: a basis composed against an absent record and one composed against an untouched record are the same claim.

+

#9. Categories

+

CategoryKind is Regular | VirtualIndex | VirtualDim | VirtualMeasure | Label. Business rules match on the enum rather than sniffing name prefixes.

+

CategoryKind::category_name() is the one place the name↔kind mapping is written down, so adding a variant forces a decision there (improvise-2lh). of_name(name) is safe to use in place of looking the Category up, because Model::add_category hands back the existing virtual category rather than shadowing a reserved name — stored_virtual_categories_match_their_names is what makes that sound. Label categories are user-named and so read as Regular from of_name; consult the stored Category::kind when the distinction matters.

+

VirtualMeasure items come from two sources: numeric data fields listed in the file, and formula targets added implicitly. They are virtual because formula-derived items are implied by the formula definitions — listing them explicitly would be redundant in the file format and confusing in the UI. add_formula does not add items to _Measure; use measure_item_names() / effective_item_names("_Measure").

+

Label is a high-cardinality per-row field (description, id, note): stored alongside the data so it shows up in record/drill views, but defaulting to Axis::None and excluded from the MAX_CATEGORIES = 12 pivot limit and from auto Row/Column assignment.

+

Category::add_item deduplicates by name and assigns IDs from a private counter, so external code cannot fabricate an ItemId.

+

#10. Workbook exists to break the Model ↔ View cycle

+

Model knows nothing about views; View depends on Model, one direction. Cross-slice operations — adding or removing a category must update both the model's categories and every view's axis map — live on Workbook, so Model stays pure data and improvise-core can be extracted without pulling view code along. That is also why AddCategory/RemoveCategory are LayoutGeometry effects rather than CellData ones (gotcha 3).

+

Workbook::new seeds a Default view, registers the three virtual categories through on_category_added (so all three land on Axis::None), then bumps _Measure to Axis::Page so aggregated pivot views show a single measure at a time (improvise-kos, commit 709f2df). Leaving _Index/_Dim on None keeps pivot mode the default — records mode activates only when the user moves both onto axes.

+

Workbook::add_item errors on an unknown category so callers do not reach through model.category_mut(..) and invent their own handling (improvise-tvt). There is no Model::add_item.

+

#11. Virtual views

+

_Records and _Drill (view::VIRTUAL_VIEW_NAMES) are scratch views the app builds on demand. Both put the virtual _Index/_Dim categories on axes, and those categories are never persisted — so these views are rebuilt from scratch each time rather than saved (improvise-60z).

+

#12. axis_of panics, try_axis_of does not

+

Registration is guaranteed for every model category: Workbook::new, add_category, add_label_category and create_view all call View::on_category_added for every view. So any name obtained from Model::categories or from the view's own category_axes is safe for axis_of. Use try_axis_of when the name comes from anywhere else — cycle_axis does, and no-ops on an unknown category to match set_axis (improvise-t8s).

+

#record_cell_key is a match, and the match is the guard (improvise-ir7l)

+

GridLayout::record_cell_key(row, col) is the one mapping from a records position to a model cell, and it answers Some for RecordCol::Measure alone. The other three arms are spelled out rather than swept into a _, and that exhaustive match is the whole guard: a new RecordCol variant fails the build until someone decides whether it names a stored cell.

+

There is no test for the Category arm, and the reason is not that the wrong answer would be harmless. It would be a real fault, and a subtle one worth naming so nobody re-derives it wrongly. RecordCol::Category(name) carries a category name, while RecordRow::cell_key(arg) puts its argument under _Measure. So answering Some there yields coords + _Measure=<category name>, and Model::formula_for would match it whenever a _Measure item happens to share a name with a regular category — projecting CellId::Derived for a coordinate column, which is never a formula target and which the pivot path does not have at all. The two front ends would then disagree, which is the design-principles §5 defect this ticket exists to close. RecordCol exists for exactly that collision: its doc says the enum resolves a column from its index "so a category and a measure sharing a name stay distinguishable".

+

What earns the decision to leave it untested is narrower: the reachable version of that fault is a new variant answered carelessly, and the exhaustive match makes that unrepresentable rather than merely unlikely (doctrine(an_invariant_is_made_unrepresentable_not_validated)). A test would pin behaviour the type system already pins, and would not survive the arm being renamed. Found in review, and the first recorded reason for declining it was wrong — it claimed the injected behaviour was defensible by analogy with a formula targeting a regular category, which is a different key entirely.

+

#13. Route then aggregate: SUM(<expr>) folds over the member records

+

improvise-4zl7, and the one place the evaluator reaches past the cell it was asked about.

+

The problem. A formula is evaluated at a CellKey. In an aggregating cell evaluate_aggregated aggregates each Expr::Ref first and evaluates the expression once on the totals. That is exactly right for a linear formula — SUM(Revenue) - SUM(Cost) and SUM(Revenue - Cost) are the same number — and exactly wrong for anything that routes. IF(Cost < 50, 1, 10) over records costing 10 and 100 answers 1 + 10 = 11 per record and 10 on the total, because aggregate-then-route is not route-then-aggregate. Every fitted tree (improvise-8xhy, improvise-vx3) is such a formula, so it meant one thing on a record row and something else one row up. The user's ruling was to aggregate the rule over the members.

+

The mechanism, and why it needed no new vocabulary. SUM(e) already means "fold e over the members". It simply did not: eval_agg specialised Expr::Ref and, for every other inner expression, summed the raw cells matching the partial key and ignored the expression entirely. It now folds — one entry per member record, the expression evaluated at that record's own coordinates:

+
partial   = context.without(target_category)    (plus any inline WHERE)
+records   = distinct(matching_cells(partial).map(key.without("_Measure")))
+values    = records.map(|r| eval(inner, context.with(every coord of r)))
+

Three consequences worth stating.

+

The rule is written once, and so is the evaluator it folds with. eval_agg is shared by eval_expr and eval_expr_cached, and both hand it the same eval_at: the record-level eval_expr. That is not a convenience, it is the correctness. A member key is a record, and formula_cache is keyed by stems with none_cats stripped — so the cached evaluator's Expr::Ref can never hit the cache at a member key, falls through to aggregate_raw, finds no raw cell behind a formula target, and answers Err("ref:…"). A first version did exactly that, and the two evaluators then disagreed on any routing formula reading a derived measure: Number(4.0) uncached, Error("ref:Profit") cached, which is the very divergence this section exists to remove (§3: "a rule stated in one and not the other is a live bug").

+

And making the lookup "work" would have been worse than the error. The only way to hit the cache from a member key is to strip none_cats off it — which returns the group's value for that formula and hands it to every member. That is aggregate-then-route again, laundered through a cache lookup, and it would have been silent. The cache cannot answer a per-record question, so the fold does not ask it. Only Expr::Is needed an arm in each evaluator, and since the fold routes through eval_expr, the cached arm is now reached only by a bare Is outside a SUM — which is what the_cached_evaluator_has_its_own_working_coordinate_predicate pins.

+

The Expr::Ref fast path stays, and it is the same fold specialised: a record's Ref is that record's own cell, so summing them is summing the matching values. It differs in exactly one place — a record with no such cell is skipped there and an error in the fold. That is deliberate: a bare SUM(Revenue) asks for the values that exist, while SUM(<expr>) asks the expression of each member, and an expression that cannot be evaluated is an error rather than a zero. Silently dropping members is how a routed sum would quietly answer for a subset.

+

The asymmetry's blast radius is wider than it sounds, and worth stating: gotcha 7 makes a pipe-quoted text cell a first-class .improv value, so one text cell anywhere in a member set turns the whole aggregated fitted cell into Error("ref:Cost"), where the bare-ref path would have dropped it and returned a number. That is the loud failure this repo prefers to a quietly wrong sum, but it is a real behaviour difference and not a detail.

+

Records come out in DataStore insertion order, deduplicated by an IndexSet, for §8's reason — hash order here would make a fitted cell drift in its last bits between processes.

+

Interaction with the fixed point (§3) and formula_cache. None of this is new state. The fold reads raw data through the same evaluate_depth / aggregate_raw the non-folding path uses, so a formula cell still depends only on raw cells and on other formula cells' cached values; recompute_formulas iterates the same stems to the same fixed point, and formula_cache still holds exactly the aggregated formula cells it held before. The fold does not cache and does not memoise: it is O(members) per evaluation per pass.

+

Measured, because the cost was a blocker and not a footnote (improvise-9iyv; crates/improvise-core/examples/fold_bench.rs, five runs per point, best–worst). At 18,050 cells with one routing formula and Tag on Axis::None, recompute_formulas takes 18.1–19.2 ms release and 132.9–135.1 ms debug, against 7.1–9.0 ms / 71.7–74.6 ms for a linear Rev - Cost. That is 2.3× a linear formula and linear in cells: 400 → 0.4 ms, 2,000 → 2.0 ms, 18,050 → 18.1 ms, so 45× the cells costs 45× the time.

+

It was not always linear. Routing the fold through eval_expr_cached — the first version, above — made every member's Ref miss the cache and fall into aggregate_raw, one poorly-narrowed scan per member: 51.1–53.3 ms at the same 18,050 cells and superlinear with it (45× the cells cost 85× the time). So the divergence and the stall were one defect, and fixing the first fixed the second.

+

And "18,050 cells" was not a sufficient description of that workload. At the same cell count the old code cost 51 ms at a 25×361 shape and 244 ms at 95×95 — a 4.8× spread from group count alone, which is what a per-member scan predicts and a constant factor does not. After the fix the two shapes are 16.9–17.2 ms and 16.4–17.5 ms: indistinguishable. The shape-dependence being gone is the confirmation of the cause, more than the absolute number is. Re-measuring this means stating the shape, not just the cells.

+

The fixed point runs exactly 2 passes (measured, not assumed), so hoisting the member set out of the loop has a 2× ceiling on the fold's share and was declined: a memo is state, and a memo keyed by anything less than the whole partial is a second answer to "what does this cell depend on".

+

One asymmetry not to tidy: the cached path hands eval_expr a fresh MAX_EVAL_DEPTH at each Agg, while the uncached Agg threads its own depth through. The second is what bounds a chain of folded formulas; making them agree by resetting both would unbound it.

+

Expr::Is(Filter) — a coordinate predicate as a number. Cat IS Item is 1.0 when the cell's own Cat coordinate is Item and 0.0 otherwise. It keeps the evaluator f64-only — no string comparison enters eval_expr — and it reuses the primitive Formula::filter already had, context.get(&cat), including its unwrap_or(false): a cell with no Cat coordinate is not that item. Under route-then-aggregate that arm is reached only where a record genuinely lacks the coordinate, because the fold hands the expression a record key. Evaluated bare, outside a SUM, it degrades exactly as Formula::filter does — which is why a fitted tree is always emitted wrapped.

+

#PageAxis — what constrains, and what the user may turn (improvise-p1ym)

+

GridLayout::page_coords answers two questions that gotcha 27 says must not be confused. The bar at the top of a view names every category constraining the visible records, which is View::filtering_cats() — Page ∪ Filter — because a drill fixes its coordinates on Filter and the user should still see them. [/] cycle only what the user may act on, which is categories_on(Axis::Page).

+

Those were two walks before: GridLayout did the first, navigation.rs:: page_cat_data did the second, and each spelled the Page-vs-Filter rule itself. Two spellings of one rule is the shape design-principles §5 calls the disease, and here it had already leaked into the protocol — GridCache::page_coords was a verbatim copy of the layout's list, so a browser handed it would have tried to page a Filter axis.

+

So page_coords is now Vec<PageAxis>, and items is Some exactly when the axis is navigable. The rule lives in the type: a Filter axis carries no item list, so no caller can compose a cycle for one, and PageAxis::cycle() is the one place that pairs a list with the index its selection sits at. page_cat_data and the browser's page_cycle both read it rather than deriving it.

+

Two consequences worth stating. The list is Model::effective_item_names, the same call that already chose the default selection, so the selection and the list it indexes into come from one place by construction — page_cat_data used ordered_item_names before, which for a _Measure page axis meant the default selection could be a formula target [/] could not reach. That incoherence is gone; the cost is that [/] over a _Measure page axis now visits formula targets, which is what the column list shows anyway.

+

And the list is not filtered to visible items. Hidden items are still cycled, because that is what the TUI has always done, and making the browser differ would be the divergence rather than the fix. Changing it is a change to both, together.

+

#The number format — format.rs (improvise-711n)

+

A view's number_format is a printf-style picture, and only that. A , anywhere turns on thousands grouping; a . introduces the precision, which is the leading digit run after the last .; anything following those digits — a conversion letter — is ignored. So ,.0, ,.2, .2, ,.2f and %.2f all name what they look like they name.

+

Why the trailing f is accepted rather than refused. It round-trips into documents today and always has. improv.pest spells the line format_line = { "format: " ~ rest_of_line ~ NEWLINE ~ blank_lines } — the grammar takes the format string as an opaque line, so format: ,.2f is a document the grammar accepts; :set-number-format stores whatever the user typed verbatim (cmd/effect_cmds.rs), and format_md writes view.number_format back out unchanged. Refusing the spelling at load would therefore reject documents this repo's own grammar admits and its own .improv example spells, to no benefit: by gotcha 8 the format is display-only, so misreading it costs a wrong pixel and can never cost a wrong number. A refusal here buys nothing and costs a document.

+

Why printf and not Excel. The two conventions disagree on exactly the spelling that distinguishes them: .00 is precision zero to printf and two places to Excel, so a parser that tried to honour both would have to guess. One is picked, and it is the one every spelling in this repository already uses. Consequences worth stating rather than discovering: 0.00 reads as (false, 0), and an Excel picture such as #,##0.00 reads as (true, 0) — grouped, no decimals.

+

Nothing is validated, deliberately. A spelling the rule cannot read names no precision and formats with none; there is no error channel and no Result, because the reader is display-only and four call sites would have to thread one.

+

What the bug was, and why six tests sat on top of it. The precision was read as fmt[i+1..].parse::<u8>() over the whole tail, so "2f" failed to parse and fell through to unwrap_or(0) — ,.2f displayed zero decimals, in the TUI, in the browser projection and in CSV export. ",.2f" and its ,.3f/,.4f siblings are what this workspace's own fixtures reach for, in about ten tests, and none could see it: they assert the stored string round-trips or use it as an opaque value to dirty a widget, or — in export_csv_quotes_commas_per_rfc4180 — they assert the rendered value starts_with("1,200"), which 1,200 and 1,200.00 both satisfy. A prefix check over a value whose suffix is the thing in question answers a neighbouring question, which is why that one is now an exact equality.

+

Only two of the three consumers are guarded. Reverting the fix fails format.rs's own unit tests and export_csv_quotes_commas_per_rfc4180, and nothing else in the workspace: ui/project.rs::grid_of (the browser projection) and ui/app.rs both call parse_number_format and neither has a test that renders a cell and asserts its digits. That gap is named rather than closed — those files are improvise-command, which was another wave's ground when this landed — and it is the more useful half of this entry, because it says where the next regression would go unseen.

+
+
+ + diff --git a/site/docs/code/formula-and-io.html b/site/docs/code/formula-and-io.html new file mode 100644 index 00000000..5a07ef62 --- /dev/null +++ b/site/docs/code/formula-and-io.html @@ -0,0 +1,137 @@ + + + + + +improvise-formula and improvise-io — improvise docs + + + +
+ +
+

Source: context/code/formula-and-io.md

On this page

improvise-formula and improvise-io

+

The arguments that used to live in the comments of these two crates. Written for a reader with the code open; the code says what, this says why, and why the obvious alternative is wrong. Bug narratives keyed by test name are in context/test-evidence/formula-and-io.md.

+
+

#Pipe quoting has exactly one home

+

improvise_formula::quote owns both halves of the CL-style |…| convention — writing and reading — for both grammars. formula.pest's bare_ident/pipe_quoted and improv.pest's bare_name/pipe_quoted mirror each other character for character, and quote.rs is the one Rust statement of that rule.

+

It lives in improvise-formula because that crate owns the grammar that says what a bare name is, and because every emitter needs it: a name that came from a user's data has to be spelled so the parser hands it back unchanged. improvise-gp re-exports it, improvise-io calls it from persistence/{format,parse}.rs, and parser.rs::unquote_pipe is a thin shim over unescape_pipe_body.

+

Before improvise-i2m5 there were three copies and they had drifted, over the keyword rule below. improvise-io no longer has one — its persistence/quoting.rs is deleted. context/repo-map.md's file inventory kept claiming io "still has its own unguarded copy" long after the deletion — it was corrected in wave 1, but any other doc or comment still saying so is wrong, and that is the shape this particular fact goes stale in.

+

Three entry points, because the callers do not want the same thing:

+ +

#The keyword guard is canonical, user-confirmed — do not revert it

+

An item literally named SUM is written |SUM|. is_bare_name is deliberately stricter than the grammar strictly requires: bare keywords (WHERE SUM AVG MIN MAX COUNT IF IS) do parse today, but only because agg_call needs a ( and PEG backtracking then falls through to ref_expr — an emergent property of rule ordering, not a stated rule. A pipe-quoted name is an identifier in every position, with nothing to fall through, so a name colliding with a keyword is quoted and stops depending on that ordering.

+

IS joined the list with is_expr (improvise-4zl7), and it is the one keyword whose omission would have been silently wrong rather than merely fragile: a category literally named IS written bare turns IS IS East into a predicate on a category called East. Adding a production to formula.pest means adding its keyword here — the two are one change.

+

The cost is worth paying because a ## Formulas line carries formula text verbatim: a name format_md writes is later read by parse_formula. One namespace, one spelling rule.

+

#The escape table, and the one input outside both grammars

+

\| → |, \\ → \, \n → newline; any other \X is preserved verbatim. The trailing arms of unescape_pipe_body are unreachable from either grammar (pipe_inner is ("\\" ~ ANY | !"|" ~ ANY)*, so a backslash is always followed by a character). They are written as a total function rather than a panic so a caller arriving by some other route gets its bytes back. The two pre-i2m5 unquoting copies differed only on a trailing lone backslash, where io returned it and parser.rs panicked; the survivor returns it.

+
+

#Cat IS Item: a coordinate predicate in the expression grammar

+

is_expr = { identifier ~ ^"IS" ~ filter_value } sits in primary before ref_expr, which would otherwise consume the category name and leave IS Item unparseable. A bare name still parses as a reference: is_expr needs the keyword, fails without it, and PEG backtracks into ref_expr.

+

It reuses where_clause's two operands — an identifier for the category and a filter_value for the item — and therefore build_filter, which already reads exactly that pair. The ^"IS" literal is a string in the grammar rather than a rule, so it contributes no pair and into_inner() yields the two operands and nothing else. One builder, one quoting rule, both halves through improvise_formula::quote.

+

The grammar-walking generator needs no new case: it reads formula.pest through pest_meta at test time, so a production added to primary is generated and round-tripped from the day it exists.

+

What the evaluator does with it, and why a fitted tree is always wrapped in a SUM, is context/code/core.md §13.

+

#WHERE carries the same boundary, and the mechanism has to be generatable

+

where_clause and inline_where are compound-atomic with sp+ after the keyword, exactly as is_expr is (improvise-r08u). Non-atomic, pest auto-skipped WHITESPACE after ^"WHERE", so SUM(Rev WHEREAS x = y) read as a WHERE on a category AS — a silently wrong filter where it should have been a syntax error, the same class of defect IS had.

+

Two things decided the shape, and the second is the one worth carrying. A negative lookahead !bare_char with sp* would have been strictly more permissive — it keeps WHERE|Region|=East, which the old rule accepted — but the grammar-walking generator cannot honour a predicate: it walks formula.pest through pest_meta and emits WHEREa="", text the grammar then refuses, so generated_formula_parses goes red. A rule whose only guard is a lookahead is a rule the generator can violate. sp+ is generatable, matches IS, and costs exactly one accepted spelling: a category may no longer abut the keyword, so WHERE |Region| needs its space.

+

Note compound-atomicity alone closes the ticket's own reproducer, because AS x cannot be one identifier once implicit whitespace is gone. sp+ is what closes the residue, WHEREAS=y, and the negative test names both.

+

#A formula target is unquoted, exactly as a reference is

+

parse_formula strips the pipes off the target the same way it does off an Expr::Ref, because the target names a _Measure item. A target that kept its delimiters made Model::add_formula create an item literally called |Total Revenue| — a ghost column the user's real item never matched, and that Expr::Ref("Total Revenue") never resolved to.

+

Both the TUI formula minibuffer (AddFormula) and .improv load go through that one call, so both were affected. The writer emits Formula::raw, never target, so the file text is unaffected either way — which is exactly why the round-trip tests never saw it, and why the fix repairs existing files on next load with no migration.

+
+

#The .improv grammar is the single source of truth

+

persistence/improv.pest defines the format; parse.rs is a tree-walker over pest's parse tree, not a line scanner. Adding a format feature means changing the grammar first.

+

The test generator (parse.rs::generator, and its twin in parser.rs::generator for formula.pest) reads the grammar's AST at test time via pest_meta and emits random valid input. A new production therefore needs generator coverage, or the proptests quietly stop describing the format: an Expr variant the emitter does not handle produces nothing, and generated_file_parses then passes over input that never exercised the new rule. formula.pest's generator is explicit about this — lookaheads emit nothing on purpose, and any unused Expr variant falls through the _ => {} arm; if the grammar starts using one, generated formulas fail to parse and the gap surfaces.

+

Two generator details worth not re-deriving. The atomic flag in the formula generator controls whether a Seq inserts a space between children, so non-atomic rules produce the whitespace-separated tokens pest's implicit WHITESPACE accepts. And Choice is 50/50 between two branches, so rules with three or more alternatives (nested Choices) pick deeper branches less often — good enough for random generation, not a uniform distribution.

+

#The tree walkers are infallible by construction

+

Every build_* in parser.rs operates on a Pair pest has already validated. "A formula has exactly one target", "a comparison has lhs, op and rhs" — pest guarantees these before the walker sees the pair. So every .expect(GRAMMAR_INVARIANT) is a bug marker, not a runtime condition: if one fires, the grammar and the walker have diverged. The same reasoning makes the final else arm of each dispatch unlabelled-but-safe (ref_expr is the only remaining alternative of primary, COUNT the only remaining agg_func, >= the only remaining cmp_op).

+

parse.rs's walkers are the opposite and deliberately so: they return Result and name the section in the error, because the .improv file is hand-editable and a failure there is a user's problem, not a programmer's. parse_tree renders pest's error as a located message (line number plus the offending line), because "Parse error: {pest dump}" is unusable to somebody fixing a file by hand.

+

#Two passes, because sections may appear in any order

+

Pass 1 turns each ## section into a plain collector (PCategory, PView) or a flat list; pass 2 builds the Workbook. That split is what makes the format order-independent — a ## View: may name a category defined further down. PCategory::items is a Vec in file order because IndexMap insertion order is load-bearing (gotcha 18), so pass 2 must replay the list as written.

+

Two ## Category: sections naming one category are one category, and the merge is a concatenation performed in pass 1 (ParsedFile::push_category, improvise-349m). Merging in pass 2 — building each section in turn — was not idempotent. A group is created either by the item that names it or by a > line, and build_category creates its own section's item-groups before its own > lines; so the first section's > groups landed ahead of the second section's item-groups, while format_md writes one section with all the items before all the > lines. The reparse then built the other order, and the two format_md outputs differed. Concatenating first makes the built order the written order by construction. Refusing a duplicate header was the alternative and would have made a legal hand-written file an error for no gain.

+

#A section header takes its name as the whole line, totally

+

header_name = { pipe_quoted ~ &NEWLINE | line_name } (improvise-pczv). The &NEWLINE lookahead is load-bearing: without it, ## Category: |A|B parses as A and silently drops B. line_name catches everything else, so a header name can no longer fail to parse. Four rules share it — model_name, initial_view, category_section, view_section — which is why it is one rule and not four copies.

+

The writing half is format.rs::quote_header, and it is deliberately not quote_name: that rule is for a name embedded among other tokens, and applying it to a header would pipe-quote every ordinary spaced title the line carries perfectly well, rewriting the header of every existing document for no correctness gain. quote_header quotes exactly four things, each one thing the reader does:

+ +

So a header is written raw where a reference is quoted. The grammar accepts both arms, so that asymmetry is a writer convention, not a format rule.

+
+

#Storage is whole-document read/write, and that closed the SQLite question

+

Saving or loading a workbook only ever touches one whole document, so a backend needs no seeking, metadata or partial reads (improvise-6mq). It does need to enumerate and remove documents, because a browser build has no OS file manager behind it — hence list and delete beside read and write (improvise-i34).

+

Paths are opaque document names, not a directory tree: a backend may treat them as filesystem paths, OPFS keys or map entries. The trait doc says "backends that are not trees treat dir as a name prefix" — keep that sentence reachable. It is the sentence that turned out to be right while the OpfsFs impl was wrong, and the bug was found by reading it.

+

Because the trait is two ops over whole documents, an sqlite3_vfs — which needs xRead(offset,len), xWrite, xTruncate, xLock — has nothing to bind to. See gotcha 41 for the measurements that declined improvise-9ix.

+

#storage_contract is a public function, not a test helper

+

It reads like a test helper and it was one: #[cfg(test)] in storage.rs, run against MemoryFs and PhysicalFs so they could not drift. The one backend that most needed it could never be handed to it — OpfsFs is cfg(target_arch = "wasm32") and its sync access handles exist in no host harness — so for that backend the contract lived only in a doc comment.

+

Moving it out of cfg(test) is what lets a browser run this sequence rather than a second transcription of it (improvise-vvl3/xf1). A second copy of a contract is the disease this codebase keeps curing. It returns Result rather than asserting because a failure has to travel out of a Web Worker as a string; the host tests .unwrap() it and get the same message.

+

Two bugs fell out of its first actual execution — see the dir parameter and the index rules below.

+

dir is a parameter, not a constant, because the three backends must agree at every dir, not only at the flat root the browser happens to use (improvise-vhcz). A backend whose list answered in a different path shape passes at Path::new("") — where joining is the identity — and fails at anything else.

+

#impl Storage for Rc<T> lives in improvise-io because of the orphan rule

+

OpfsFs keeps four genuinely async operations — open, attach, refresh, commit_deletes — off the trait, so Storage can stay synchronous (gotcha 28). But App::storage is a Box<dyn Storage>, and a trait object can name none of the four: handing the backend to the App puts the async seam permanently out of reach. The host must hold the concrete backend and give the App a Storage, which means sharing one.

+

impl Storage for Rc<OpfsFs> is two foreign types in any other crate, so improvise-9x6 wrote a forwarding newtype in the worker and every future worker would have written it again. Here the trait is local, so the impl is allowed, and it is blanket rather than OpfsFs-specific because nothing about it is OPFS-shaped: it is the general statement that a shared handle to a backend is usable as that backend (improvise-hmws). Rc and not Arc because the one caller is a Web Worker — a single thread, and wasm32-unknown-unknown has no others. An Arc impl would be speculative.

+
+

#OpfsFs: why a browser backend can be synchronous, and what the index holds

+

OPFS has two APIs. The promise-based one (getFileHandle, createWritable) is async, but FileSystemFileHandle::createSyncAccessHandle() yields a handle whose read/write/truncate/flush/close are ordinary blocking calls. It exists so wasm can do blocking file I/O — it is what SQLite's wasm build uses — and is available only inside a dedicated Web Worker, which is where improvise-9x6 puts the whole app. So Storage never had to become async.

+

Acquiring a handle is still async, as are directory listing and removal, so the backend splits in two:

+ +

So a delete is visible to read and list immediately and lands in OPFS at the end of the batch. That ordering is the price of a synchronous trait and is invisible to a single worker. write therefore un-queues any pending delete of the same name: otherwise delete-then-write would be listed and readable and then removed by commit_deletes at the end of the batch — the one ordering the deferral can get wrong.

+

#The index holds documents, not entries

+

Storage::list promises document names, and documents and entries stopped being the same thing the moment attach had to create a file to get a handle for a document that does not exist yet (improvise-vvl3). Three writers, and the split between them is the whole rule:

+ +

holds_bytes is where the last clause lives, and its two failure modes get opposite answers, which is the whole function:

+ +

Handles are exclusive per file, so attach closes any handle it replaces and detach releases the lock.

+

#list joins dir like the other backends

+

The join is not cosmetic (improvise-vhcz). MemoryFs and PhysicalFs answer with dir-joined paths — paths a caller can hand straight back to read — and OpfsFs used to answer with bare names. At Path::new("") the two shapes coincide, which is the only dir the worker ever passes, so the disagreement was invisible: a worker that organised documents under docs/ would have got a full picker in every host test and an empty one in a browser, with nothing saying why.

+

dir is decoration rather than a filter, because this backend is flat by construction: entry_name reduces every path to its file name, so docs/a.improv and a.improv are the same OPFS entry and an entry name may not contain a separator at all. That is the trait's own "treat dir as a name prefix" — the doc was right and the impl was wrong.

+
+

#Loading: decompression and decoding are two failures with two answers

+

load_from decompresses, then decodes UTF-8, with a separate Context on each. read_to_string would fuse them, and a valid gzip stream holding invalid UTF-8 would then be reported as unreadable gzip — a wrong diagnosis of a document that decompressed perfectly well. Decoding once, afterwards, also leaves one answer to "is this UTF-8" rather than one per branch.

+

The .gz suffix selects gzip regardless of backend: that is a property of the document name, not of the filesystem (improvise-6mq). Legacy JSON is detected by a leading { and is never written.

+

parse_number_cell returns CellValue::Error naming the offending text rather than a fabricated 0.0. No token the current grammar admits actually fails f64::from_str (overflow saturates to ±inf), but if the two ever drift the mismatch must surface — never silently store a wrong number.

+
+

#Persistence write order and the virtual views

+

format_md writes Views → Formulas → Categories → Data: views first because they are typically small and orient the reader, data last because it is the largest section. The parser tolerates any order (see the two-pass note above).

+

_Records and _Drill are scratch views the app rebuilds on demand. They put _Index/_Dim on axes, and those categories are never written — so persisting the views would produce axis lines naming categories the loader will not find. They are skipped on write, and dropped on load if an older file carries them, rather than reinstating a view the app would rebuild anyway; Initial View then falls through to the default, since the name it pointed at is gone. persisted_initial_view is the writing half: saving from records or drill mode falls back to the first real view rather than naming one the file will not contain (improvise-60z).

+

_Measure persists only its non-formula items — formula targets are rebuilt from the ## Formulas section, so format_md collects the targets and excludes them.

+

Page and Filter both carry the selected item after the axis keyword; the other axes have nothing to select. Axis::Filter round-trips even though it occurs only inside the never-written _Drill view, because the alternative was silently writing a Filter axis as some other axis (improvise-dz8, gotcha 27).

+
+

#Import: order is a correctness property, not tidiness

+

analyzer::analyze_records emits distinct values in first-seen order, never HashSet order (improvise-7yx, gotcha 18). build_model adds items in exactly that order and IndexMap preserves it, so a hash order made the same CSV import to a different document on every run — different .improv bytes, different row order on screen, and a different detected date format, since detect_date_format only inspects the first ten samples and which ten it saw varied. The HashSet in analyze_records is there to deduplicate and never to iterate.

+

#The ambiguous-date heuristic reads American-first, deliberately

+

01/02/2025 parses under both %m/%d/%Y and %d/%m/%Y and nothing in the data can tell them apart. detect_date_format is first-match over DATE_FORMATS, so the American reading wins. That is a defensible answer and not obviously the right one: a European CSV imports with days and months swapped, silently, and the derived _Month/_Quarter categories inherit the error. What makes it tolerable is that the wizard shows the detected format and lets the user change it — the same argument two-digit years already rested on. It is pinned by a test so that changing the answer is a decision somebody makes rather than a side effect of reordering the table.

+

Ruled a known limit (improvise-l8rh, the user, 2026-09-20). Two things the ruling rests on. Changing the default to day-first only moves which half of the world imports silently wrong; and the honest fix is larger than the heuristic — an ambiguous column should surface its ambiguity rather than resolve it, which changes FieldProposal's shape and the wizard step that displays it, the way Dataset::warnings returns a decision the user should hear about as data. Worth doing deliberately, not as a patch.

+

What makes the limit reasonable to live with is that it is now deterministic: analyze_records iterated a HashSet until improvise-7yx, so the ten samples detect_date_format inspects were a different ten every run and a column whose eleventh value ruled out %m/%d/%Y detected a different format run to run. The ten-sample window itself is unchanged and is part of the limit.

+

#CSV parsing

+

Short rows keep every header column, missing trailing fields filled with Value::Null, and each affected column is reported as a warning with a row count. The reader is .flexible(true), so without this a short row parsed cleanly and the trailing columns were simply absent from the record map — no error, no warning, and skewed value counts in field-kind analysis (improvise-k8i).

+

A header name becomes a category name and survives into the .improv file, so anything riding along invisibly in one — a byte-order mark, a stray carriage return — is silent corruption rather than cosmetics. That is why the adversarial inputs are tested with controls: a check that cannot see a BOM where there is one proves nothing.

+

parse_csv_bytes_with_warnings exists so CSV import can come from any Storage backend rather than only a file (improvise-fuj); the path-taking wrappers remain for the native CLI.

+

#The wizard's back path is a history stack

+

advance pushes the step it leaves; previous pops it. Keeping the path actually taken, rather than re-deriving the reverse of each conditional transition, means back can never disagree with how the user really got here (improvise-907).

+

Preconditions live in validation_error() and are checked on the step where the user can still fix them — never deferred to build_model after the final confirm. Two gates earn that: too many categories (improvise-mzv) and no category at all (improvise-gfq); both are fixable with c at ReviewProposals, and both used to refuse only at the last keystroke. MAX_CATEGORIES here mirrors the model's private limit and counts regular categories only.

+

NameModel is terminal for advance: leaving it is a build, not a step change. build() parks the workbook in built on Done; a build failure leaves the user on NameModel with the error. Going back from Done discards built, so accepting an import always reflects the latest field decisions.

+

scalar_name decides what a JSON scalar contributes as an item name. Value::Null — which is also what the CSV parser produces for a blank field — and a blank string are absences, not names; stringifying them used to invent an item literally called "null" (improvise-ych). Every other value keeps its printed form.

+

A record missing a value for an accepted Category field has no place in the grid and is dropped; build_model_with_warnings counts it against the first category it is missing and reports the total, because without a warning that loss is invisible in a thousand-row import (improvise-5j9).

+
+

#CSV export

+

render_csv is separated from writing so the bytes can go to any Storage backend, not just a file (improvise-fuj). It goes through the csv crate rather than joining with commas, and it honours the view's number format — a value of 1200 under a comma-grouping format renders 1,200, and a label like Smith, Jr. splits into two fields, corrupting every row (improvise-1cz).

+
+
+ + diff --git a/site/docs/code/gp.html b/site/docs/code/gp.html new file mode 100644 index 00000000..40ca6783 --- /dev/null +++ b/site/docs/code/gp.html @@ -0,0 +1,188 @@ + + + + + +gp-tree and improvise-gp — improvise docs + + + +
+ +
+

Source: context/code/gp.md

On this page

gp-tree and improvise-gp

+

The reasoning that used to live in the doc comments of crates/gp-tree/** and crates/improvise-gp/**. Written for a reader with the code open: the code says what happens, this says why, and why the obvious alternative is wrong.

+

Bug narratives keyed by test name are in context/test-evidence/gp.md. The condensed version is repo-map gotcha 37.

+
+

#1. Two crates, one seam

+

gp-tree is problem-agnostic: a Tree is a regression tree over a fixed-width f32 feature vector, and the crate owns the representation, the GP operators, simplify, render, one generation (breed), the search (fit), and the optional raw-table bridge (data). It knows nothing about what the features mean or how a candidate is scored — the caller supplies a Spec (per-feature ranges and names) and, outside fit, its own fitness. Determinism is the caller's too: the operators take an Rng trait implemented over a seeded generator.

+

improvise-gp is the one seam between that crate and improvise. It cannot live in gp-tree — that inverts the dependency and drags improvise into a generic GP crate — and it cannot live in improvise-formula, which must not know GP exists. It holds three things:

+ +

fit.rs is here rather than in a Cmd because none of it is terminal-specific. A browser fitting the same measure over the same document must run exactly these steps in exactly this order, or it gets a different tree from the same seed. improvise-command/src/command/cmd/fit.rs is the thin terminal wrapper over it.

+

#2. gp-tree returns data and never prints

+

A decision the user should hear about comes back as data: Dataset::warnings carries the columns preprocess dropped for being entirely missing, fit.rs hands back pooled (prediction, actual) pairs and a score per feature rather than tables. examples/fit_csv.rs owns every println! and every eprintln!, and that is the only reason its output stayed byte-identical across these moves. Adding an eprintln! to fit.rs or data.rs is the regression, not a convenience.

+

The reason is not tidiness: a library does not own its caller's stderr. A TUI writing to an alternate screen, a wasm build, or a caller that wants the notice in its own status line can none of them suppress, redirect or translate an eprintln!. preprocess used to do exactly that, which was correct while this code was examples/fit_csv.rs.

+

improvise-gp inherits the rule. FitOutcome::status_line is a rendering of the outcome data, offered so a second front end does not compose its own — the same argument that put flatten_cat_tree in the shared layer — and it is a returned String, never a println!.

+

#3. The vendored RNG, and why its constants are a promise

+

SplitMix64 is ~20 lines and no dependency. The Rng trait exists so a caller can bring its own generator, and every caller so far wrote out this same one.

+

A fit is not a throwaway: it writes a rule into a user's document, so re-running it on the same data must produce the same rule, and a bug report must be reconstructible from its seed alone. That is worth vendoring rather than depending — the point is reproducibility, not statistical quality, and SplitMix64's is ample for picking nodes and thresholds.

+

SplitMix64::seed() keeps the seed the run started from, unchanged as the generator advances, so a caller can show it and record it. A generator that has advanced past its seed cannot answer that question.

+

next_u64 is public because splitmix64_matches_the_reference_vectors pins it. The constants are the algorithm, not an implementation detail: every seed already printed by a fit and every data::Model seed already saved is a promise that these numbers come back.

+

#4. Spec::ranges is a soundness precondition

+

ranges[i] must be a superset of the values feature i can actually take. A too-narrow range makes simplify prune a branch that is reachable, silently changing the tree's outputs. preprocess derives the ranges from the columns themselves, which is what keeps this true; every_training_value_lies_inside_its_feature_range pins that it stays derived.

+

simplify seeds each feature's feasible interval with its real range rather than ±∞, so an out-of-range threshold prunes. The interval is half-open [lo, hi) but a feature can hit its max, so hi is nudged by EPS = 1e-3 and a split at the max (x < max) stays live. EPS is safely below typical threshold granularity.

+

#5. Row<T>: an invariant made unrepresentable

+

Row<T> pairs a feature vector with what it is supposed to predict. The target is generic because the two supervised shapes differ only in it — Row<f32> for regression, Row<usize> for a class code — and every scoring function then reads one slice instead of two.

+

That is the point of the type. Parallel x/y slices could be handed in at different lengths, and zip answered by silently scoring the shorter prefix while a denominator taken from the longer one made the mean too small. There is no length to get wrong here, so there is nothing to validate and no Result to thread through five signatures (improvise-dkqn, doctrine(an_invariant_is_made_unrepresentable_not_validated) — the same answer prune_axis_items got).

+

The pairing is made at the boundary that builds the data. preprocess walks one row index and the rows live inside Target; a Dataset carrying rows and target as two collections would move the mismatched length one level up, into every caller.

+

mean_over(total, n) is the one place the empty row set is answered. Every mean in fit.rs used to reach its own 0.0 / 0.0 on an empty slice: a NaN that propagates through FitConfig::cost, sorts unpredictably against real costs, and reaches a caller as a number rather than as an error. Scoring no rows finds no error, so the answer is zero, stated once instead of in mse, misclass_rate and standardize separately.

+

Changing any of this needs bit-level evidence. tests/scoring_known_answers.rs pins every scoring path as f32 bit patterns plus a next_u64 witness that catches a changed RNG call order, because the CLI prints three or four decimals and byte-identical output alone cannot see a moved accumulation. See context/test-evidence/gp.md.

+

#6. folds and cv_folds answer different questions

+

folds(n, k) splits 0..n into k pieces after a deterministic shuffle. k arrives from a host — a CLI flag, in this crate's only caller — so zero is a number to answer, not an impossible state: out[pos % 0] is a public library function taking its host down with it. One fold is the answer the shape already gives, and fit spells the same max(1) over restarts.

+

cv_folds(n, k) yields nothing below two. A partition into one piece is still a partition, but a cross-validation over one fold is not a cross-validation: the single fold holds every index, the training complement is empty, and both cross-validators would fit a tree on nothing and then let it predict every row. Yielding no folds makes the pooled pairs empty, which is the true count of such predictions — and no fold is drawn, so the shuffle never touches the caller's generator either (improvise-ry89).

+

complement(n, test) builds a bool mask rather than asking test.contains(i) per row, which was a scan of the fold per row and so quadratic in the row count (improvise-t3ux). It yields the same indices in the same order, which is what every downstream draw from the generator depends on.

+

#7. standardize / denormalize: the affine map must be folded back

+

The crate's leaf scores live in a ±5 range, so a target in the thousands cannot be reached by leaf mutation and standardize is not optional. Its consequence is that the champion predicts standard deviations.

+

fit_csv answers that by printing ≈ score × std + mean beside the tree. A document has nowhere to put that, so improvise-gp::fit_measure folds the affine map back into the leaves with denormalize before a single character of formula text is spelled. Omitting it emits a perfectly well-formed formula whose every cell is out by an affine map — wrong in the quiet way, since small numbers in the right shape look like an answer.

+

denormalize touches leaves only, and that is the whole reason it is sound: standardize rescales the target, so a threshold is already in its feature's own units and re-scaling it would be the bug.

+

#8. RegressionQuality carries its own baseline

+

A tree with a plausible-looking RMSE that is no better than always predicting the mean is not a weak rule, it is no rule — and it is worse than no rule at all, because it is one a user will believe. So the comparison is part of the answer rather than something a caller may forget to compute. r2 is not clamped: negative means the fit does worse than the mean, which is a real outcome, and clamping would make "learned nothing" and "actively misleading" read alike.

+

importance_ranking reports each feature's share of the total importance. The all-zero case returns early rather than flooring the divisor: fit_csv used .max(1e-9), which is right for the only input it ever had and wrong as a contract — a lone importance of 1e-10 then divides by the floor rather than by itself and reports 10%, where the one feature the model uses carries all of it. Importances this small are ordinary, being error differences over a standardized target, so the shares have to sum to 100% at every scale.

+

#9. A fit takes two things to reproduce — and here, four

+

data::Model records seed: Option<u64> and config: Option<FitConfig>, because the same seed under a different population/generations/elite/penalty lands on a different tree (improvise-ksst). Both are Option for one reason: every FitConfig value and the seed 0 are legal, so nothing can double as "unknown", and anything reporting a sentinel would state a seed that never ran.

+

Note #[serde(default)] on an Option field is documentation, not the mechanism — serde defaults it anyway, so deleting the attribute fails no test. A backward-compatibility test that only deletes the attribute is not discriminating; aim it at an absent field loading as the default instead.

+

improvise-gp::FitOutcome carries the whole FitRequest, not the pair. FitRequest is { seed, config, cv_folds, importance_repeats, max_records }, and every stochastic step draws from one generator in a fixed order — cross-validate, then fit, then explain. So cv_folds decides how far that generator has advanced before fit draws from it, and therefore changes the champion and not merely the evidence for it. Recording two of four fields documented the reproducibility contract without meeting it.

+

#10. The emitter: quoting, and the threshold's exact decimal

+

A regression tree is a nested conditional, and improvise formulas already have IF(cond, then, else), so a fit can be written straight into a user's model as an ordinary formula with nothing GP-specific left in the file:

+
Score = SUM(IF(humidity < 77.5, 1.25, IF(outlook IS rainy < 0.5, -0.5, 2)))
+

(An earlier form of that sample sat in improvise-gp/src/lib.rs's module doc inside a ``text fence. rustdoc does not execute a text` fence, so it was prose wearing a fence rather than a doctest, and the two-line cap applies to it like anything else.)

+

Two things in it are improvise-4zl7's, and both are load-bearing. The outer SUM is the route-then-aggregate rule made syntax: a tree is fitted on records, and SUM(e) is improvise's existing word for "fold e over the member records" (context/code/core.md §13). Without it the formula routes on the totals one row up and means something else there; with it, a record row is a one-member fold and answers the rule itself. outlook IS rainy is the one-hot outlook=rainy said as a coordinate predicate rather than as a reference to an item no document has — see §11 for which features get which spelling.

+

Two properties the emitter owes its caller:

+

It parses. Feature names come from the caller's data, so they carry spaces, operators, pipes and keyword collisions. Every name goes through quote_name — improvise_formula::quote, the one home for pipe quoting (gotcha 34); improvise-gp re-exports it because it was part of this crate's public surface. A formula target is unquoted on the way back in: it names a _Measure item, so a target keeping its delimiters writes the fit into an item literally named |Total Revenue|.

+

It routes identically. number_text emits the shortest decimal that reads back as the same f64 the f32 widens to — not the f32's full decimal expansion, which runs to 149 fractional digits, and not a shortest-round-trip-to-f32 rendering. Widening an f32 to f64 is exact, so text that reads back as that f64 makes feature < thresh give the same answer in f64 formula evaluation as in f32 gp_tree::eval. A short rendering (0.1 for 0.1f32) sits up to half an ULP away, and a row whose feature value equals the threshold exactly then falls on the other side of the split.

+

number_text panics on a non-finite value. The grammar spells neither, and both fail quietly rather than loudly: "inf" parses as an f64, so the round-trip loop would return text no formula parser accepts, and NaN matches no width at all. Trees built by this crate's own operators over a finite Spec are finite, so this is a precondition rather than a runtime condition — but see §12, because a .improv cell holding inf makes it reachable from a document.

+

expr_text and mark_features share one precondition and spell it the same way (improvise-hkt6): a bare used[*feat] fails with index out of bounds, which names neither the tree nor the spec. Two sites, one failure, one sentence. used_features is the single walk both the document check and the plan check read, because walking the tree twice two ways is how those answers drift apart.

+

#11. A fit is storable only when the document can name its features

+

preprocess invents Region=North for a one-hot, Cost.missing for a missing indicator, when.year for a date part. Those are encoding artefacts, not items of the document, and Expr::Ref resolves through Model::find_item_category — so a tree splitting on one emitted a perfectly well-formed formula whose every cell then evaluated to Error("ref:Region=North"), with nothing between the fit and the file to say so (improvise-8xhy). improvise-4zl7 closed the categorical third of that by giving the emitter a vocabulary for a coordinate, and by giving the evaluator a rule that makes such a formula mean the same thing at every level of aggregation (context/code/core.md §13).

+

#Features: one entry per feature index, read off the plan

+

Features::of(&Dataset, categories) walks gp_tree::data::feature_roles — the one statement of preprocess's expansion order, which encode_row drives off too — and pairs each role with the name preprocess gave it. A tree's feat indexes Features directly, so nothing anywhere reconstructs index → role from the feature names. Spec::names is &[&str] and could not have carried this; the earlier check matched names against the raw plan instead and said in this section that it was avoiding a second implementation of the expansion. Now there is no second implementation to avoid.

+

Each feature gets one of three Spellings, decided by role and by column, never by the shape of the name:

+
rolecolumnspelling
Numerica measureExpr::Ref(measure)
OneHot { value }a category, value != "(missing)"Cat IS Item
Numerica categoryArtefact(NumericCategory)
DatePartArtefact(DatePart)
MissingArtefact(MissingFlag)
OneHota measureArtefact(ValueOfAMeasure)
OneHot { "(missing)" }a categoryArtefact(AbsentCoordinate)
+

The last five arms were one _ => Artefact until improvise-wd6n, and the wildcard was not merely untidy. The refusal it fed read "a col.missing flag or a date part is an encoding artefact", so three of the five families were refused in words naming something they are not — and a future FeatureRole variant would have joined them silently, declined under whichever family the message happened to mention. The match is now exhaustive, so a sixth role is a compile error rather than a sixth thing that message misdescribes.

+

The two exclusions are not fussiness. A one-hot over a measure column names no coordinate at all — the record's value for that measure is text, and Cat IS Item asks about coordinates. (missing) is the absence of a coordinate, which Expr::Is reads as 0 by §13's unwrap_or(false), so the one spelling that would be wrong is exactly the one omitted. A Numeric role over a category column is the trap the previous version of this section called "the hole a name cannot close": a category whose items all parse as numbers produces a feature named after the category, and Expr::Ref would resolve it to a _Measure item of the same name — a different column. Both halves now fall out of is_category, which is a lookup in the header's own first category_columns entries.

+

Duplicate header names are still refused outright, by no_ambiguous_column_names and before anything else: is_category decides by name, so a name that means two columns is a spelling with two meanings.

+

Dataset::columns is not index-aligned with the header — preprocess continues past an entirely-missing column without pushing a ColumnSpec, so one empty category shifts every later spec left. FeatureRole::column() indexes columns, not the header, so that shift is carried rather than tripped over.

+

#What fitted_formula still asks the document

+

Two refusals, in order. An Artefact is refused with no reference to the document at all — nothing could make Cost.missing spellable. Everything else is asked of the document: a Measure must be find_item_category-resolvable and a Coordinate must name a category the model has. A measure literally named Region=North is a legal document and is accepted, because the check never looks at the name's shape — and that is also why the date parts and the .missing flags stay covered without a second copy of preprocess's naming rules.

+

formula_text still spells whatever it is handed, being the tree's own text; an Artefact reaching it is a precondition failure and panics, the same split number_text makes for a non-finite value.

+

#The two families improvise-wd6n declines, separately

+

Both stay Artefact and both stay refused. The decline is written here rather than left open, because each needs a formula-language change and neither needs an emitter change — and they are answered separately because the vocabulary each wants is different. What changed with the decline is that the refusal now says which, in that family's own words.

+

col.missing is declined because the language has no word for absence. The flag means "this record has no value for measure X". Expr::Ref over an absent cell is an Error, not a zero, so there is no existing spelling to reuse and no way to phrase the question as an expression. Closing it means deciding what an absence is in a language whose aggregates already treat it as "no rows" — and that decision reaches apply_agg, eval_agg and the text-unanimity rule, not just this emitter. It is not worth that for a split a fit happens to like.

+

The date sub-features are declined because improvise has no dates. when.epoch_day, .year, .month, .day, .dow, and with a time component .hour, .minute — gp_tree::data::build_date emits them. A .improv cell holding a date is CellValue::Text; nothing in eval_expr reads one, and there is no date type, no date literal and no date arithmetic anywhere in the crate. Closing this is a type addition to the value language, and the sub-features are the smallest part of it: a user who could split on when.year would immediately want to filter and aggregate by it.

+

Both are visible to the user only as a refusal naming the feature and the reason, which is the right interim: a fit that cannot be stored says so, rather than storing a formula whose every cell is a reference error (improvise-8xhy). improvise-4zl7's route-then-aggregate rule (context/code/core.md §13) is untouched — nothing new is emitted, so there is nothing new for it to hold over.

+

Two of the other three are declined permanently: a one-hot over a measure and the (missing) pseudo-item name no coordinate by construction, so no vocabulary closes them. They are refusals rather than gaps.

+

NumericCategory was not, and improvise-zv8t has closed it. That it was called one is the more useful half of improvise-wd6n: prose declaring something a permanent refusal is what stops anyone filing the ticket, and nothing but a reader could have caught it — the suite was green either way.

+

The accident was one argument. fit_measure handed preprocess an Overrides::default(), so it never said which header columns are document categories, although the very next line told Features::of. preprocess then inferred Numeric → Date → Categorical from the cell values, so a Year category with items 2024/2025 became a numeric column and a category of ISO dates became a date column — and both were then refused as encoding artefacts with Year IS 2024 spellable all along. fit_measure now builds its Overrides from the same header[..category_columns] slice it already hands Features::of, forcing each to Forced::Categorical, so the one place the document's own answer was known is now the only place either caller reads it from.

+

Three things follow, and the last is a cost rather than a win.

+

The date decline above now means what it says. A DatePart could arrive over a category column by exactly this accident, so §11's date family was narrower than it read. It now covers a genuine date measure and nothing else, which is the scope the decline was always argued for.

+

The Artefact arms stay, and stay correct. crate::dataset is pub and takes the caller’s Overrides, and Features::of reads roles off whatever Dataset it is handed — so a caller that builds one without forcing still produces a Numeric role over a category column, and fitted_formula still has to refuse it. The arm is unreachable from fit_measure, which is the only entry point a document goes through; it is not unreachable code, and deleting it would be deriving the classification from one caller’s habits rather than from the roles.

+

Forcing widens the encoding, and that costs a document. A Year category with two items was one numeric feature and is now two one-hots; with a hundred items it is a hundred, and MAX_FEATURES refuses the fit outright.

+

The first draft of this paragraph claimed "no document that fitted before stops fitting", on the reasoning that a tree splitting on such a column was refused as an artefact anyway. That is false, and the review that caught it was of this prose rather than of the diff. The refusal is conditioned on used_features, so a fit that simply never split on the numeric category succeeded — and it is precisely the wide categories that a fit has least reason to split on. Measured, on a hundred records with a hundred-item numeric Year beside a Cost that predicts Score on its own: before, the fit lands on IF(Cost < 48.0…) and Year is never touched; after, the preflight refuses at 101 features. a_wide_numeric_category_now_refuses_a_fit_that_used_to_ignore_it pins both halves, including that the repair the message names really works.

+

It is still the right trade, for a reason that is not "both refuse". The old success was accidental: Year was encoded as a single continuous number over 1900–1999, an encoding that means nothing and that the fit could only use by producing a formula it would then be refused for. The document now gets a refusal that names the repair — mark the category a Label — and marking it restores the pre-zv8t feature set exactly. What is lost is the case where the user wanted neither: a wide numeric category left alone while the fit uses the measures. That is filed as improvise-e8no; the honest fix is a budget that can decline a column rather than the whole table, which is a change to what encoded_width returns and not to this decision.

+

The preflight encoded_width had to learn the same slice, or it counted such a column as one feature and handed the refusal to the exact check after materialising the matrix — which is the side of the cost the preflight exists to stay on (§14).

+

And the refusal's own words must not name a column kind the arm does not know. FeatureRole::Missing is not conditioned on is_category — unlike its Numeric and OneHot neighbours — and dataset.rs writes "" for a record with no coordinate for a category, so a .missing flag over a category is reachable. Its clause therefore says "that column" and not "that measure": naming the wrong kind there would be this section's own charge — declined in words naming something they are not — surviving one level down.

+

#12. A fit is persisted as its formula, and nothing else (improvise-7na)

+

.improv stores the emitted formula and no serialized gp_tree::data::Model. The grammar is unchanged by this decision, deliberately and not by omission.

+

The ticket offered three options: (a) formula only, refit to score new data; (b) the data::Model as a JSON blob; (c) seed + config so the fit reproduces. The answer is (a), with (c) already available in gp-tree for whoever wants provenance later.

+

Against (b), two arguments. The general one is the second half of gotcha 41: a blob is a second copy of information the document already carries — the tree is the formula — and nothing keeps the two in agreement. A user who edits the formula by hand, or a cell it was fitted on, leaves a blob that disagrees with the file it sits in, and a round-trip test over the blob cannot see that, because it never compares the two. (The first half of gotcha 41 does not transfer: an opaque blob inside one new grammar production is not a second implementation of the grammar.) The specific one is layering: Tree and FitConfig are gp-tree types, so improvise-io would gain a dependency on gp-tree and .improv would be obliged to track a third crate's serde layout forever — inverting exactly the dependency improvise-gp exists to keep out of the core.

+

What (b) was for, and why it is not needed. The encoding plan exists so fit_csv --predict can score a row that arrives as untyped CSV text. improvise has no such row: a new row is a record in the same document, and recompute_formulas scores it through the stored formula from its own measures.

+

Why (c) is not written to the file either. data::Model already records seed and config, so reproducing a fit is solved wherever that value is in hand; and unlike the CSV case the training data is the document. What is missing is a reader — writing provenance now would add a grammar production and a round-trip obligation for metadata nothing consumes. Provenance is the fit command's decision, alongside how it words the fit for the user.

+

A premise the ticket got wrong, and it is the bigger finding. "The emitted formula is enough to recompute existing cells" holds only for a fit over numeric measures — §11 is that defect, and it is a defect in what a fit may emit, not an argument for persisting the model.

+

#13. dataset.rs: what a view shows, as a feature matrix

+

The shape is not invented here. Records mode is already a feature matrix: a row is one coordinate tuple — every cell sharing all coordinates but _Measure — and each measure is its own column (gotcha 1). That is exactly "a sample, and its features". So the module enumerates the same rows records mode does, lays them out as the string table preprocess takes, and lets preprocess own type inference, one-hot expansion, imputation and the ranges.

+

What it does not reuse is GridLayout: reaching the records path means putting _Index on Row and _Dim on Column, which is a different view from the one the user is looking at. RecordRow::group is public, so the traversal here is the same two steps the layout takes, against the user's own view.

+

Four selection rules, each of which could reasonably have gone the other way:

+ +

_Measure is a filtering category and must not filter a fit. That exception in filter_coords is not a special case, it is the difference between a grid and a feature matrix: _Measure is the coordinate the columns are spread across, so pinning it to one item is not "which records am I looking at" but "throw away every column except this one". Workbook::new's default view puts it on Axis::Page, so that was the behaviour on a freshly opened document (improvise-vx3).

+

Two smaller decisions. Records are sorted by their coordinate tuple. That began as a workaround for matching_cells walking a HashSet; since improvise-shyt it yields insertion order, so the sort is no longer what buys stability. It is kept because a fit's row order should be a property of the data rather than of the typing history that entered it. And a Number cell is rendered with f64::to_string, never format_f64: display rounding is view-only and must not reach a fit (gotcha 8). An Error cell has no value, so it is spelled as missing and picks up the same imputation and .missing indicator an absent cell would.

+

#14. improvise-gp::fit: budgets, and the enum that refuses to guess

+

#Two budgets, and why the preflight is advisory

+

FitConfig bounds the search; nothing bounded the width, which preprocess can multiply without limit (one column per distinct value of a categorical column). Permutation importance alone is repeats × rows × features tree evaluations, on the thread that draws the next frame.

+

MAX_FEATURES = 64 is comfortably above any hand-built pivot table — this repo's own 18,000-cell benchmark encodes far fewer — and comfortably below the one-column-per-row shape that is really a mis-typed category. The message names the fix (Label), because that is what the user actually wants.

+

encoded_width is the advisory preflight, run on the raw table before preprocess materialises anything. Checking width afterwards guards the wrong side of the cost: a category with a distinct value per record allocates a rows × rows one-hot during preprocessing, so the refusal arrives after the memory and the O(rows²) work have been spent. It approximates preprocess's type inference rather than sharing it. Copying the predicate is the lesser evil; the alternative is exporting the inference and calling this a second implementation of it.

+

The debt that copy carries is that the preflight owes EVERY branch preprocess can take, and it paid two of three. preprocess infers Numeric → Date → Categorical, in that order, and the preflight had no date arm at all: a column whose cells parse as neither numbers nor a forced category fell through to the categorical count. So a measure column holding dates — one distinct timestamp in every row — promised 1 and materialised 5, because build_date emits epoch_day/year/month/day/dow, and 7 once any cell carries a time (improvise-x1ns, second instance, found by ChatGPT Codex on #97). That is an under-count of four or six on a single column, against a budget of 64.

+

The two instances are the same defect one inference-branch apart, which is why the shape is now structural rather than remembered. inferred_kind reproduces preprocess's match arm for arm and returns a Forced, and encoded_width matches that with no wildcard — so a fourth Forced variant is a compile error here instead of silently taking the categorical arm, which is exactly how Date was missed. The .missing flag is added once outside the match, because every branch owes it on the same terms.

+

The count each arm owes:

+
preprocess buildsfeatures
build_numeric1
build_date5, or 7 if any present cell has a time
build_categoricaldistinct present values, +1 for the reserved (missing)
+

plus col.missing wherever any cell is missing, and clamped to at least 1.

+

The two directions of a drift do not cost the same, and this paragraph said they did (improvise-x1ns). It read: the cost of the copy drifting is bounded in both directions by the exact check — over-counting costs a needless refusal, under-counting "only lets the exact check do its job". That is wrong, and it is wrong against the argument two paragraphs above it: the exact check runs on Dataset::feature_names.len(), which exists only after preprocess has materialised the matrix. So it bounds the error message, never the cost — which is precisely the thing the preflight was introduced to bound. Over-counting is cheap; under-counting spends the memory and the O(rows²) work the preflight exists to prevent, and then reports it politely.

+

The live instance was the (missing) column: the categorical arm counted present values only, so a forced category with N distinct present values and one record lacking that coordinate promised N and materialised N+1. has_missing was already computed and used in the numeric arm alone — two arms of one match disagreeing about what a column costs, which is improvise-zv8t's forcing-by-name / budgeting-by-index defect one level in. That one was repaired by making the ordering a data dependency and so a compile error; this one has no such shape available, so the guard is a test that asks preprocess what it actually built rather than asserting a number (the_preflight_counts_the_missing_category_a_forced_column_materialises).

+

The previous wave's review saw a version of this and reasoned it away as "read, not injected" and "bounded by the exact check". Both halves of that are the failure mode this file exists to catch: the first declined to run the experiment, and the second repeated the sentence above instead of checking it. An argument that a cost is bounded is checkable, and this one was checkable the day it was written: encoded_width landed 2026-09-19 and the sentence 2026-09-20, so the code it described was one commit old and one file away.

+

encoded_width remains advisory in the direction that is safe: it may over-count, and a column whose cells are entirely missing is counted as one where preprocess skips it altogether. It must never under-count.

+

That sentence was written as a proposed kb.pl invariant while the tree held a counter-example, in the same change that fixed the categorical one — the date branch was missing the whole time and nothing in the hand-back noticed. An invariant proposed from the defect you just fixed generalises the fix, not the code; the thing that would have caught it is the question "what else does preprocess do that I have not mirrored", which is a question about the other function and not about the bug.

+

MAX_RECORDS = 20_000 bounds the work, which the width bound does not: a one-feature table with a million records still runs cross-validation, the final search and permutation importance over all of them. Measured at 105 ms for 2,000 records and roughly linear, so 20,000 is about a second — where a pause stops being a pause. Refusing is honest, and the limit is the caller's FitRequest::max_records rather than a constant, so a host with somewhere else to run a fit can raise it.

+

FitRequest::default() is the interactive budget: FitConfig::default() is the CLI's 80 members over 600 generations, which is right for a batch tool and wrong for a synchronous command. Measured, release build, on the shape this module's tests use: 6 ms at 12 records, 12 ms at 200, 105 ms at 2,000 — the whole sequence, five-fold CV included. There is deliberately no timing test: a wall-clock assertion is flaky on shared CI and would fail for a reason that is not this code's. Re-measure after changing the numbers.

+

#Forcing a numeric category is a representability choice, not a budget one

+

improvise-zv8t made fit_measure force every document category to Forced::Categorical, and improvise-e8no records the cost: a hundred-item numeric category one-hots to a hundred columns, past MAX_FEATURES, and is refused where inference used to read it as one numeric column and the fit succeeded. improvise-e8no is declined, and the refusal is correct.

+

The ticket rules out making the forcing conditional on cardinality, and rightly: that is inference by another name, and it makes the encoding depend on a budget rather than on what the document says. But the stronger reason to decline is that the alternatives are not merely distasteful — they are unstorable, and the code already says so.

+

A coordinate becomes an Expr only by passing through IS: is_expr is identifier ~ sp+ ~ ^"IS" ~ sp+ ~ filter_value, a single-value equality, and there is no IN and no ordering over a category's items. (where_clause and inline_where are coordinate predicates too, and also single-value equalities — but they gate a formula or an aggregate rather than being an Expr, which is gotcha 37(d)'s distinction and is why IS is the one that matters here.) So whatever a tree says about a category, the emitted formula names its items one at a time.

+

A first draft of this paragraph said that made one-hot the only spellable encoding. That is false, and an adversarial review disproved it by running parse_formula rather than reading the grammar: is_expr is a primary, so it is an Expr yielding 1/0, and cmp_op's operands are full exprs — so an ordinal split is spellable, as a weighted indicator sum (IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, …)). The claim is recorded here rather than quietly deleted because it is the shape this file keeps catching: a grammar argument made by reading the rule that looked relevant instead of running the parser.

+

So ordinal is not ruled out by spellability, and is ruled out anyway, for two reasons that survive. It invents a metric the document does not assert — a category's items carry insertion order, not magnitude, so Year < 3.5 means "the first four items as they happen to be ordered" and stops meaning that when someone reorders them. And the width it saves does not stay saved: the encoding would be one feature, but the formula still spells one IS per item, so it trades a MAX_FEATURES problem for a formula-size problem while adding a false order. What the current emitter does with it is narrower and is not the argument: spell() maps FeatureRole::Numeric over a category column to Artefact::NumericCategory, because a bare reference to the category's name would resolve to a _Measure item of the same name (improvise-8xhy) — a fact about the bare-ref spelling, not about ordinals in general. Hashing is ruled out outright: a bucket has no name in the document, so it cannot even be refused informatively.

+

And this is what makes the decline clean rather than resigned. The "fit that previously succeeded" was one of two things, never a third. Either the tree never split on the numeric category — in which case the column contributed nothing to the rule, and marking it a Label gives a fit over the same remaining features, inside the budget, which is exactly what the refusal already advises. Or the tree did split on it, and fitted_formula refused the result as Artefact::NumericCategory at storage time. So no fit is lost that was both storable and used the column.

+

Two things that claim less than a first draft of this paragraph did, both worth keeping. Marking the category a Label does not yield the same fit: the feature count changes, so the RNG draw sequence in random_tree/sample_thresh changes, so the champion for a given seed changes — gotcha 37(b)'s own rule, applied to this paragraph. "The same fit" is a checkable cost claim and it is false; what is true is a fit over the same remaining features. And the dichotomy above is argued, from spell()'s exhaustive match over FeatureRole, not asserted: what a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored asserts is one instance of its second branch, over one three-row fixture. That is the most a test can do here, and saying so is the difference between evidence and a flourish.

+

The test still had no predecessor. NumericCategory had no coverage before it — and it was one of three, not the only one: grepping each Artefact variant and each variant's reason() text, MissingFlag and DatePart are covered by the_missing_flag_and_the_date_sub_features_are_refused_too, and ValueOfAMeasure and AbsentCoordinate have no test at all, before this change or after it. Two families remain uncovered; that is filed, not fixed here.

+

What the decline does cost, stated rather than waved at: a user whose Year category really is a quantity they want fitted as a number has no way to say so through fit_measure. The document says Year is a category of items, and that is what is honoured. The remedy is a document change — Year belongs in _Measure if it is a measurement — and crate::dataset is pub and takes the caller's Overrides, so a host that wants to offer the choice can (gotcha 37d). Offering it inside fit_measure would be a new user-facing control, and nobody has asked for one.

+

The refusal's wording was narrower than the rule and is now aligned in both places that carry it: it said a category with a distinct value per record becomes one column per record, which diagnoses the runaway shape and not improvise-e8no's — a category of modest cardinality that is simply wider than the budget. Both now say a category becomes one column per distinct item, whether its items read as numbers or not, since that clause is the whole of what changed under zv8t.

+

#Quality is an enum, not an Option

+

cv_regress pools no pairs below two folds, and scoring an empty pool gives an R² of exactly zero — which reads as "this rule is worth nothing" when what happened is that nobody checked. Quality::Measured carrying an all-zero RegressionQuality means "this tree is no better than the mean", the loudest thing a fit can say, and it must not be reachable by simply not having asked. So the enum, and a host that wants the numbers has to name the case where there are none.

+

The fold cap is rows / 2, not rows. fit_csv clamps to the row count, which lets two rows become two folds of one: each tree is then fit on a single row — a constant — and the numbers come back meaning nothing while wearing the Measured label. That is the confusion Quality exists to prevent, moved from one row to two rather than removed.

+

#fitted_measure_name — a fit is a rule about the data

+

"{target} (fit)", unless the caller names another. Storing it under target itself would make the formula shadow the very cells it was fitted to: gotcha 7's _Measure items are explicit items ∪ formula targets, so the two would be one item with a computed value, and the user's typed data would stop being visible.

+

The destination is refused when it names any existing item, not only an existing measure. Expr::Ref resolves through Model::find_item_category, which finds a regular category's item before it falls back into _Measure (gotcha 7) — so naming the fit after an item of some category installs cleanly and then makes every later formula that references it read the category coordinate instead. Asking the document the same question Expr::Ref will ask is both stronger than a measure-only check and simpler.

+

#one_line is public because the refusals need it too

+

.improv pipe quoting makes \n legal inside a category or item name, so a document can name a measure that splits the TUI status bar in two. The whole composed status line is flattened once at the end rather than each part being sanitised and one being forgotten.

+

The refusals need it as well, and they are composed by the host: several interpolate a document name, and a failure is when a user is most reading the bar. Flattening only the success path left the failure path able to split it.

+

A related trap with no mechanism guarding it: a rustfmt pass over a multi-line string literal leaves runs of ten spaces mid-sentence in a message a user actually sees — invisible in the source and in every test that only asks whether a substring is present. a_name_containing_a_newline_cannot_split_the_status_line asserts the one-line rule for that reason.

+

#15. Known gaps, filed rather than fixed

+

Closed: aggregate-then-route is not route-then-aggregate (improvise-4zl7). Kept here because the shape recurs. A tree is fitted on individual records, and an improvise formula evaluated at an aggregating CellKey used to aggregate each Expr::Ref first and evaluate the IF once on the totals — the same answer for a linear formula, a different one for anything that routes, so the stored rule meant one thing on a record cell and another one row up. The emitter now wraps every tree in SUM(…) and eval_agg folds a non-Ref inner expression over the member records: context/code/core.md §13. What remains of it is a cost, not a defect — the fold is O(members) per evaluation per fixed-point pass and is unbenchmarked (improvise-9iyv).

+

Two artefact families are still unspellable (improvise-wd6n). col.missing and the date sub-features stay Spelling::Artefact and are still refused. §11 says what each would need; they are independent of each other.

+

A mean-imputed feature is read raw by the formula. preprocess fills a missing numeric cell with the column mean, so a champion may split on a value the training rows had and the document does not. The stored formula reads the document, so that record evaluates to something other than what the fit predicted for it — quietly, since the measure name resolves and only the cell is absent. fitted_formula's check asks whether the document can name the feature, a different question from whether every record has one. Closing it means choosing between refusing any feature with a missing cell — which would refuse most real documents, imputation being why preprocess imputes — and teaching the formula language an IFMISSING-shaped primitive. A product decision of the same kind as improvise-4zl7.

+

A boundary value can route differently in the formula than in the fit. Training narrows each f64 cell to f32; the stored formula compares the original f64 against the threshold. number_text guarantees the threshold reads back as the same f64, so the threshold is exact — but a cell value just below a threshold that f32 rounds to it takes hi in gp_tree::eval and lo in formula evaluation. Pre-existing in the emitter (improvise-izk); the fix, one numeric representation end to end, would rewrite the bit-level characterization tests that are the only evidence the routing argument holds.

+

Quality::Measured is transductive rather than fully held out. Preprocessing runs over all rows before cv_regress picks its folds, so an imputation mean and a categorical vocabulary are learned partly from the rows they are then validated against. And it is wider than imputation: Spec::ranges is computed over all rows and the same spec goes into every training fold, so the thresholds the search samples and the branches simplify prunes are bounded by a held-out minimum or maximum. That reaches a fit over complete numeric measures too — the only shape fit_measure currently stores — so "the case that stores leaks nothing" is wrong. Closing it means building the ranges and the preprocessing plan per training fold and encoding each validation fold with that plan, which cv_regress cannot express: it takes one spec and rows that are already encoded. That is an API change in gp-tree, and it is independent of improvise-4zl7, which turned out to need no change to how cv_regress is fed.

+

Which features a champion picks is a property of the search, which used to mean the same command refused or succeeded depending on the seed. For the categorical family that is gone: a one-hot over a real category is spellable whichever way the search goes. It survives for col.missing and the date parts (improvise-wd6n), and it is the reason those are worth closing rather than living with — a refusal that depends on a seed is not a contract.

+

#16. preprocess: what each column kind expands into

+

Real data is not a matrix of floats. Columns are numeric, categorical or dated; cells go missing; a category is unordered and a < thresh split over its arbitrary codes would be meaningless. So each raw column is inferred (or forced through Overrides) to a ColumnKind and expanded:

+ +

Any column that had a missing cell also emits a col.missing 0/1 indicator, so absence itself is something the tree can split on rather than a silent substitution. A clean column emits none, or every fit would carry a constant feature.

+

The expansion is not reproducible from a fresh table: imputation means and the one-hot value list are properties of the training rows, and a later table may not even contain some of them. preprocess therefore records a ColumnSpec per raw column and encode_row replays it; that pairing of fitted trees with their encoding plan is data::Model. A value never seen in training is simply all-zeros, not a new column.

+

Note for §14's preflight: build_categorical pushes the range (0.0, 1.0) for every one-hot whatever the column contains, so a constant one-hot is not pruned by simplify — it cannot know the column is constant. A test fixture that relies on a degenerate range being pruned is relying on seed luck.

+

#17. fit_csv: the CLI's own documentation

+

examples/fit_csv.rs is the crate's second use case (the first was a game bot), a wholly different domain — the point being that gp-tree knows nothing about features or fitness. Almost none of what it does is in that file: preprocessing is gp_tree::data, the search is gp_tree::fit, and what is left is flag parsing, the demo dataset, and the printing.

+
cargo run -p gp-tree --example fit_csv -- path/to/data.csv
+cargo run -p gp-tree --example fit_csv -- --save model.json path/to/data.csv
+cargo run -p gp-tree --example fit_csv -- --predict model.json path/to/new.csv
+

With no path a built-in categorical dataset (the classic "play tennis" decision-tree example) is fitted, so it runs standalone.

+

Shape. The header names the columns; the LAST column is the target, the rest are features. A numeric target is a regression (one tree, minimising MSE); a categorical target is a classification — one tree per class fit to a 0/1 indicator (one-vs-rest), predicting by argmax.

+

What a fit reports, in order: a generalization metric first — k-fold cross-validation against a trivial baseline (predict-the-mean / majority-class), with R²/RMSE for regression and a confusion matrix plus per-class precision/recall for classification — then the final tree(s) fit on all rows, whose train error is optimistic, then a ranked permutation feature-importance table.

+

Tuning flags. --cv <k> (folds, < 2 disables; default 5), --seeds <n> (keep the best of n GA restarts; default 1), --penalty <λ> (per-node parsimony cost keeping trees small and general; default 0.01), --seed <n> (the RNG seed, decimal or 0x-hex; printed by every fit and saved with --save).

+

Column directives. --ignore a,b drops columns (ids, free text); --numeric / --categorical / --date a,b force a column's type — a numeric-looking zip code that is really a category, say.

+

Messy CSVs. Missing cells (blank or NA/null/?/…) are mean-imputed for numeric and date columns, with a col.missing indicator so absence is itself learnable, and become a (missing) level for categorical columns. Rows missing the target are dropped up front, with a count on stderr.

+

--save <path> additionally writes a JSON data::Model: the fitted tree(s) plus enough of the preprocessing plan to re-encode a fresh CSV's raw cells into the exact same feature layout, and the seed and the FitConfig those flags produced — because the seed alone does not reproduce a fit (§9).

+

--predict <model.json> <data.csv> loads that model and scores each row of a CSV of raw feature columns (same names as training, any order, target column optional and ignored), printing one prediction per row — plus a softmax confidence column for classification.

+
+
+ + diff --git a/site/docs/code/kb-tool.html b/site/docs/code/kb-tool.html new file mode 100644 index 00000000..5ac19681 --- /dev/null +++ b/site/docs/code/kb-tool.html @@ -0,0 +1,40 @@ + + + + + +The kb tool — improvise docs + + + +
+ +
+

Source: context/code/kb-tool.md

On this page

Maintaining the knowledge base

+

prolog-sessions/kb.pl is executable knowledge, checked by CI. This is the argument for how it is written, and why the Prolog MCP stopped being the way.

+

#Why the MCP was retired

+

loadProgram replaces a whole predicate, so adding one clause means re-sending every existing clause of it through an agent's context window — and a short payload silently drops the rest. Three further blockers: saveSession re-adds the MCP's own host_rpc shims and reserializes the whole file, and assertz raises permission_error(modify, static_procedure) against a consulted predicate, so there is no incremental route through it at all.

+

#What replaced it

+

scripts/kb, a shell entry point over scripts/kb-merge.pl.

+

Clauses are read as DATA, never consulted. read_term/3 in a loop yields a list of terms; merging is list manipulation; portray_clause/1 writes them back. Nothing is ever asserted, so the static-procedure wall is not reached and the cost of adding a clause is the clause.

+

Canonical form is read_term in, portray_clause out. This was not designed, it was measured: all 495 facts in the existing file — including every escaped quote — round-trip byte-identically, and the form is a fixed point in one pass. The only change on first use is that rules move from the MCP's two-line layout to one line, which is why the reformat lands with this tool rather than before it: the MCP would re-expand them on its next save and the two would thrash.

+

trealla comes from the dev shell. Reaching it through nix shell per invocation cost more than the work: the tool's first real use timed out past two minutes on a write that takes 0.42 seconds with tpl on PATH.

+

#The guards, and why each exists

+

Every one of these was added because injecting its fault showed the tool accepting something it should refuse. None was found by reading.

+

The round-trip check. After writing, the file is read back and required to be the clauses intended. A serializer bug is the wipe-the-knowledge-base failure with a new owner, and this is the only check that sees it. Note it compares variant equality via numbervars over a copy — reading a term back yields fresh variables, so == compares variable identity and reports every rule as a difference.

+

kb_consistent runs on the CANDIDATE. The file is replaced only after the check passes, so a delta that breaks the knowledge base leaves it untouched rather than requiring a revert.

+

Parse errors are caught explicitly. An uncaught syntax error drops trealla into its REPL, which hangs with a terminal and exits 0 without one — a refusal that reads as acceptance. scripts/kb also closes stdin on every call as a backstop.

+

Every run reaches a halt/1. tpl exits 0 unless a halt is reached: measured on 3.10.3, -g "fail" and -g "throw(boom)" both exit 0, and so do fail,halt and throw(boom),halt, because a goal that fails or throws never reaches the halt beside it. halt(2) alone exits 2, so halt works — the default is the hazard. The goal used is therefore

+
(catch(main,E,(write(user_error,E),nl(user_error),halt(2))) -> halt(0) ; halt(1))
+

which gives 0, 2 and 1 for success, throw and failure. Without it, if ! merge ... reads a crashed merge as a clean one and replaces the knowledge base.

+

Terms must be callable. 42. parses, is not a clause, and reached the knowledge base as a fact whose head is a number. kb_consistent has no opinion about stray terms, so nothing objected.

+

A clause can be retired. kb rm takes keys and refuses if any of them is absent, because a typo'd key would otherwise report success and leave the clause in place. Without it no correction could ever drop a fact — the first thing it was used for was retiring kb_is_maintained_through_the_mcp, whose name was the claim that went false.

+

Duplicate keys are refused by default. A key is the functor plus every argument but the last, so doctrine(Name,_) keys on the name and decision(Id,Key,_) on the pair. replace is the door a correction goes through, and it overwrites in place — a count that grew would be two clauses under one name.

+

#What no test here reaches

+

The key rule is looser than functional/1 for crate/3, which keys on the first argument alone; kb_consistent is the backstop for that case rather than the merge.

+

A clause name cited inside another clause's string is unchecked — dropping or renaming a clause silently dangles every clause referring to it (improvise-2f97).

+

A shebang would let kb-merge.pl run directly and collapse the two files into one. It does not work on trealla 3.10.3: #! is a syntax error that aborts the consult, so main/0 is never defined, and tpl wants -- args after the file, which a shebang cannot inject. Worth re-testing if trealla gains script support.

+
+
+ + diff --git a/site/docs/code/protocol.html b/site/docs/code/protocol.html new file mode 100644 index 00000000..34b4540d --- /dev/null +++ b/site/docs/code/protocol.html @@ -0,0 +1,362 @@ + + + + + +improvise-protocol — improvise docs + + + +
+ +
+

Source: context/code/protocol.md

On this page

improvise-protocol — the wire vocabulary and the portable session slice

+

The arguments that used to live in this crate's comments (improvise-9c8c). Written for a reader who has the code open. Nothing here restates what the code plainly says; everything here is something somebody got wrong first.

+

The crate crosses a process boundary, so it depends on improvise-core and serde and nothing else — no terminal backend, no command layer, no IO. improvise-command depends on this, never the reverse, and that direction is what lets a client hold the view half without pulling in the host. boundary_tests::the_wire_types_need_nothing_but_core_and_serde is the part the compiler can hold; the CI wasm job is the other half, since a crate that pulled in crossterm would stop building for wasm32.

+

Three things live here and it is worth being precise about which travels:

+ +
+

#1. lib.rs — the portable slice

+

#What may live on ViewState

+

Two properties gate membership, and both are load-bearing (improvise-h0b):

+ +

State failing either test lives on App instead, under the session-local residue tag: wizard (an improvise-io type, and a half-open dialog is not something to inherit) and drill_state (holds an Rc, and pending_edits are uncommitted writes belonging to whoever staged them).

+

The constraint this puts on new fields: no borrowed-'static fields anywhere ViewState can reach — Deserialize cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. That is why MinibufferConfig::buffer_key is a String and not a &'static str (improvise-a2x).

+

#ViewCursor and the cursors map

+

Per-client (improvise-8uz). It used to live on View, inside the shared workbook, which meant one client scrolling moved everyone's viewport. View no longer carries it at all — nothing persisted it, so there was no saved-position behaviour to preserve. The map is keyed by view name so that switching views restores where you were in each one; an absent key means the origin.

+

#ViewState::active_view is the second copy, deliberately

+

Workbook::active_view is the document's; ViewState::active_view is the client's, and it is what keys cursors. The second copy exists because reduce_view runs on a client that holds no Workbook (improvise-cqi). The host keeps the two in step; whether they are allowed to diverge — two clients on different views — is improvise-g0f. See repo-map gotchas 29/36.

+

#AppMode::from_name — not every variant is nameable

+

The list is exactly what improvise-command's registry accepted before the move, carried across unchanged, because changing it here changes what enter-mode accepts in a script. It lives beside AppMode rather than in the registry because the names are part of the keymap vocabulary and the keymap is shared: a browser resolving enter-mode must decode the same spellings, and two tables would be the mirror that move existed to delete.

+

Three variants are absent, for two reasons. ItemAdd resolves its target category from the runtime mode and never from a script line (gotcha 20), so naming it would produce a mode with nothing to add to. Quit and ImportWizard are entered by their own commands — quit and start-import-wizard — which do work beyond changing the mode, so enter-mode would be a way to reach half of each.

+

#Panel

+

Panel::ALL/name()/key() drive the toggle keys for Normal and all three panel keymaps, so a fourth panel reaches every keymap from one ALL entry. name() must spell what parse_panel accepts. Panel::mode() is the inverse of AppMode::panel() for the three top-level panel modes only — the sub-modes (FormulaEdit, CategoryAdd, ItemAdd) map back to their panel but not forwards, which is what keeps a panel's styling active while the user types into its minibuffer.

+

AppMode::hint_text() is checked against the keymaps by every_hint_key_resolves_in_its_mode in improvise-command; Quit alone shows nothing, because it is never rendered.

+

#all() walks a cycle, and its totality is tested rather than structural

+

AppMode::all() and Rejection::all() walk a total, cyclic after(). Both doc comments used to claim a new variant could not be omitted, because after() has no None arm. That is half true and the wrong half. Answering after() for a new variant is compiler-forced; pointing an existing arm at it is not, so a variant nothing names is simply never walked — all().len() came back 17 where the enum had 18, with the suite green (improvise-5ubs). The general statement is doctrine(a_list_checked_for_length_reads_like_a_list_checked_for_completeness): an assert_eq!(cases.len(), 13) catches a case being deleted from a list and never one being absent from it, and the two are written identically.

+

Closing it needs an enumeration independent of after(), and without one the ticket's own first option restates the problem. There is no such enumeration in stable Rust for an enum with data — variant_count is nightly, a derive macro means a new proc-macro crate, and strum's EnumIter cannot construct these variants at all, since AppMode::ItemAdd carries a category name and Editing a whole MinibufferConfig. What there is is serde: the Deserialize derive already builds the complete variant-name list, from the enum definition, maintained by the compiler, and publishes it in its unknown variant … expected one of … message. variants.rs reads it back and diffs it against the names the cycle actually reached.

+

Three things this costs, and they are the reason the module is defensive rather than three lines. The message is parsed, so a serde whose wording changes must fail loudly instead of reporting an empty list that passes every cycle — hence the unknown variant prefix assertion and the names.len() > 1 floor. Both live in parse_expected, a seam separated from the deserialization because a should_panic pointed at a real type reaches only one of the three refusals and all three then shared one panic message, so expected = … could not say which had fired (PR #92 review). Each now carries its own wording and is fed a hand-written mutated message; a happy-path case keeps the three refusals from being the only evidence the parser runs. A variant is identified by its own serialization rather than by Debug, because an externally tagged unit variant is a string and a struct variant a one-key object, and reading a name off Debug would depend on the field list. And it checks only that every declared variant is reached: an after() that is not injective — two arms naming one successor — is a different defect this does not see, and would show up as a shorter cycle only if it also orphaned something.

+

What it does not reach: the fourth instance of this shape, layout_geometry_effects_actually_change_document_config in improvise-command/src/ui/effect.rs, whose assert_eq!(cases.len(), 13) is over a hand-built list of effects, not enum variants, so serde has nothing to say about it. Filed as improvise-26dm rather than bodged, because a fourth hand-rolled attempt in a fourth place is the disease.

+
+

#2. reduce.rs — the wire vocabulary

+

#Derived, not invented

+

Command is not a fresh invention (doctrine(derive_the_enum_from_the_effects)):

+ +

The count is 29 model effects, minus three: SwitchView, which Command already names and routes, and ApplyAndClearDrill/AcceptWizardModel, which are not addressable (improvise-3x3u). That leaves 26. A variant with no data mirrors a nullary effect whose operand is document state the authoritative App already holds.

+

Two derivation traps worth keeping:

+ +

#Why two model effects are key-path only

+

A command on this wire is routed by Sessions::apply with no SessionId, which is correct precisely when the change is addressed to no one session (gotcha 36). ApplyAndClearDrill and AcceptWizardModel fail that test: their operands — drill_state.pending_edits and the built wizard workbook — are session-local residue, private to whoever staged them (improvise-tom1). Unaddressed, client B's ApplyAndClearDrill commits client A's uncommitted typing, and B's AcceptWizardModel replaces the document out of a dialogue B never opened. Both were demonstrated against Sessions::apply before removal.

+

Portability of the command was the criterion when they were kept; addressability is the criterion that matters. Not permanent: multi-client drill wants the addressed form (the variant carries a SessionId, the staging map moves onto the session), which is where ADR 001 layer 2 goes anyway.

+

#Command::SwitchView has a foot in both halves

+

The SwitchView effect wears EffectKind::Model(LayoutGeometry), so the model-half derivation lands on it too — and ModelCommand deliberately does not name it a second time (improvise-kmgs). ModelCommand::ViewBack and ViewForward also move the active view and so carry the same document half.

+

#Command::ResetCursor is an invalidation, never a position

+

Derived from App::reset_cursor, the write TransposeAxes and CycleAxis make after changing the grid's shape (gotcha 24). It is a variant rather than a SetSelected plus two offsets for a reason that is not brevity: what the server says is "your cursor is no longer valid", not "be here". Where a client then puts its cursor is its own business — the model is shared, the UI is not — so broadcasting one client's position to everyone would be the wrong message even though origin happens to be where each of them lands today (improvise-gvnb).

+

#Command::Model is a struct variant

+

So the basis cannot be forgotten. ADR 001's check is worth nothing if a batch may arrive without one, and the only enforceable version of "you must also state X" is not being able to build the message without X. There is no Basis::default() for the same reason — see §5.

+

Note what the struct variant does not buy: it closed omission from the message and did nothing to close ignoring on arrival, which is the half a struct variant is usually reached for. That hole needed a type — Checked/CheckedBatch in improvise-command (improvise-ondj).

+

#ModelCommand::SetCell takes a RecordCell and nothing else

+

A CellKey here would require the client to know the coordinate space — every row and column category plus the page and filter selections (gotcha 27), and in records mode the _Dim column-index convention (gotcha 1) — which is precisely the knowledge the server exists to own. The surrogate is a token to echo back; and because it is a record identity, a client holding one for an aggregate cell cannot build this variant at all (ADR 002).

+

LoadModel and ImportJsonHeadless name paths in the host's Storage, never the client's filesystem (gotcha 2).

+

The variants are grouped by the ModelChange level their effect declares, because that level is what Dirty::of turns into projections (gotcha 35). Adding a variant here means classifying its effect there.

+

#reduce_view

+

The same function runs on both ends — the client folds it to draw, the server folds it over its mirror — so the two cannot drift (improvise-cqq, improvise-m91). Total and deterministic: no IO, no clock, no failure.

+

Command::Project and Command::Model are both no-ops in it, for different reasons. A projection is routed to one widget's WidgetCache instead. A model command is the document's business: what a client must fold arrives as the projections it dirtied, plus — when the active view moved — the SwitchView the host broadcasts.

+

A ModelCommand does not mutate anything by itself: improvise-command's model_effects maps it back to the effect it came from and runs it through the same App::apply_effects the key path uses. Effects stay the only mutation site, and the Dirty is the effect's own EffectKind — never a second classification beside gotcha 35's table.

+
+

#3. render.rs — the fold

+

#There is no aggregate render cache

+

Each widget owns its own cache type and folds its own stream of projections (improvise-rkeq): the grid does not learn that a formula was deleted, and the formula panel does not learn that a cell changed. A Projection is one such update, addressed to exactly one widget, and WidgetCache::apply is the fold.

+

The overlap between TileBarCache and CategoryTreeCache — both carry (category, axis) — is deliberate, so neither is woken by the other's changes. Do not "fix" it by sharing a variant.

+

WidgetCache::apply is total and deterministic, like reduce_view: an update naming something the cache does not have is dropped, never a panic. Projections race on the wire, and a client that crashes on a stale index is a client that crashes.

+

Projection::Basis is not a widget — nothing draws it — but it is folded on exactly the same terms; see §5.

+

#DocumentCache — a widget, not a ViewState field (improvise-z754)

+

draw_title read workbook.model.name, model_state.dirty and model_state.file_path. None was projected, so a browser could not draw a title bar at all.

+

A projection rather than a ViewState field, and the argument is gotcha 30's membership test read backwards. ViewState is the portable session slice: per-client UI a client folds its own commands into. These three are facts about the document, shared by every client and writable only by the server — on ViewState they would be document state wearing a session slice's clothes, and reduce_view would need commands nothing composes. The per-widget cache is exactly the shape for "a local copy of what a widget draws, replaced by a Projection".

+

It is the one widget no EffectKind can gate. dirty is raised by any EffectKind::Model(_) in apply_effects and cleared by effect::Save, which is ViewOnly (gotcha 3) and correctly so. SaveAs moves file_path and is also ViewOnly. So Dirty::of reports NOTHING for exactly the effects that change this widget most visibly, and gating it on Dirty would leave every client showing a [+] for a document that has been saved — the forever-wrong half of gotcha 35. ClientSession::document_update therefore observes: it builds the desired cache and diffs it against the mirror, the same answer record_versions got for the same reason (gotcha 40), and the same reason cursor_epoch beats matching on CycleAxis. The diff, not a flag, is what keeps an ungated widget from projecting on every batch.

+

A save is projected, not silent. Host::save ends with project_all() and the worker re-projects after its own persist(), because both clear model_state.dirty outside any batch Dirty describes. gotcha 45's "answered silently on success — nothing moved, so no widget is dirty" was true until this cache existed and is now exactly wrong: the title bar moved. A test that reaches for Sessions::project_all by hand proves nothing here, because that is the call the production paths omitted.

+

stored_as is a name, never a path. The TUI already drew only file_name(), and a socket server projecting file_path would hand a remote client the host's directory layout — useless to it and a leak. What a client can act on is whether the document has a home at all, which is the fact ClientFrame::Save fails on (NoDocumentPath, gotcha 45), so None carries it. A browser's answer is its OPFS entry name; neither end has to know which.

+

#CellId: the kind of identity is the permission

+

SetCell/ClearCell take a RecordCell and nothing else, Drill takes an AggregateHash and nothing else. A client cannot compose a write against an aggregate or a drill against a stored cell — not because either is checked, but because neither carries an id of the other's type (design-principles §3). It projects a fork the model already makes: EditOrDrill branches on exactly this in the TUI today.

+

An earlier draft made this Option<CellId>, absence meaning aggregate. The enum is better: an aggregate is not the absence of an identity, it is a different kind of identity — and modelling it as None left Drill with nothing to address.

+

A RecordCell is (record hash, measure) because the surrogate names the record, not the cell (ADR 002). That is what lets one identity serve both addressing and staleness: record_versions is keyed by the same hash, so a client's own token is the lookup key for its own version, with no CellKey rebuilt on the hot path. measure is absent for a record carrying no _Measure coordinate at all — the shape add-record-row seeds.

+

#CellId::Derived — a formula owns the cell (improvise-ir7l)

+

A formula target is a real stored coordinate, so the record branch claimed it and every client offered an editor over it. The write landed and was then shadowed forever, because evaluate_aggregated reads formula_cache before raw data — not a revert, a permanent silent shadow.

+

The fix keeps ADR 002's discipline rather than adding a second answer beside it: a formula-owned cell projects a different kind of identity, so the write is unconstructible instead of flagged. Derived carries the formula's target and no surrogate, so there is nothing a client could spend on a SetCell; that is the same reason Aggregate carries an AggregateHash and not a RecordHash.

+

In records mode the question is asked of the record's real coordinate. The synthetic (_Index, _Dim) key names no measure, so formula_for can never match it; GridLayout::record_cell_key is the one mapping from a records position to a model cell, and both cell_id and CmdContext::owning_formula go through it. This is not defensive: a measure can be both an explicit _Measure item and a formula target, and the records + column (improvise-poia) is a one-keystroke way to build one.

+

Two orderings matter in project::cell_id. Derived is asked before the record branch, since the coordinate is stored and the record branch would otherwise win. It is asked after the aggregate branch, because a formula cell that also aggregates is still drillable (gotcha 10) — Aggregate and Derived are not alternatives for the same cell, and reversing the two turns a drillable cell into a dead one.

+

The predicate itself is Model::formula_for(&CellKey) in improvise-core, the one home of "is this cell derived". It already existed three times inside model/types.rs as an open-coded loop; the TUI's EditOrDrill and the producer now call the same function rather than each spelling the rule. The TUI had to change in the same breath: it re-derives from the Model it holds, which is legitimate (it is the authority), but it must give the same answer or the two front ends diverge — design-principles §5.

+

#CellId::Create — the column writes no cell (improvise-poia)

+

The two records-mode creating columns — + and +cat (gotcha 1) — projected None, which means "no projected cell". So a client could not tell a column that creates something from a padded slot, and gotcha 42's conflation had a third instance.

+

The ticket offered two designs and both assumed a cell write. They do not happen: commit.rs::add_measure emits exactly one effect, AddItem { category: "_Measure", item: name }, and add_category emits AddCategory(name) and deliberately writes no coordinate on the current row (improvise-2zq). The record under the cursor is not part of the operation, and neither is any value — what the user types is a name. So there is no "distinct command carrying the record surrogate plus the new measure name" to invent, and no two-command flow to sequence: one existing ModelCommand does it, and the address it needs is the name.

+

What was missing was therefore never an address but a recognition. Create carries the kind and nothing else, because the kind plus the typed name is the whole operand. It is not a SetCell target, so render.rs::editable stays Record-only and record_at_cursor still answers None — both by matching, which the compiler now forces.

+

No client composes it yet, and that is one gap rather than this one: keys.rs::commit returns early for every records-mode commit, because a category column renames every measure cell in the row and that fork needs record_col. The creating columns do not need it, so the early return is broader than its reason — improvise-gcvb, which also records the two things that make it more than a one-liner: an inserted +cat column means the TUI does not advance, and AddCategory can abort.

+

#CellDisplay::id — None is not "aggregate"

+

That is Some(CellId::Aggregate(..)), and the distinction is the whole point of the enum. None means the slot holds no projected cell: a GridCache position padded by fit_to_labels, blanked by a reframing, or a records-mode column that is not a cell to write into at all (a category column renames a coordinate across the whole row, gotcha 1, which is a different intent from SetCell). Conflating the two either hides a cell or offers a write the type system exists to make unconstructible.

+

#CellDisplay::edit_text — the difference corrupts data

+

What an editor opened on this cell should start with: the value as the user last typed it, not as it is drawn. text has been through format_f64, so a stored 1200 is drawn 1,200 under a comma-grouping format; seeding an editor with that and committing it parses as CellValue::Text("1,200"), and the number is gone. Display rounding is view-only and must never be fed back into the model (design-principles §4) — and an editor's seed is exactly a feedback path.

+

It is the projection of App::display_value_at_cursor, which is what EditOrDrill seeds the TUI's buffer from, so both front ends start from one string rather than each deriving one. None where there is nothing to edit: the same set of positions id is None for, plus cells whose value is absent. An error cell carries none, because its text is a diagnostic and not a value.

+

A browser found this: tests/lockstep.rs drove i through the host and the client over the same grid, and they disagreed by a comma.

+

numeric and error are decided by the server for the same reason: it is the side that knows which it was, and re-deriving client-side is how the two ends drift apart. CellDisplay is deliberately not a CellValue — by the time a cell reaches a client the number formatting has already happened.

+

#AxisFrame — the grid's frame, and why it is one type

+

AxisFrame (improvise-anr9/s1xw) replaced a Vec<String> of labels that GridLayout::row_label built by slash-joining a multi-level tuple — the exact rendering two_row_categories_produce_cross_product_labels asserts must not appear. The protocol carried a form of the labels the TUI itself considered wrong, so the browser drew Food/Alice where the TUI drew two header columns, with no groups and no collapse indicators (gotcha 44).

+

It is one type rather than a labels plus a levels beside it, and that is the whole point: two answers to "what are the labels" is the disease this projection architecture exists to cure (improvise-edp). A renderer asks this type and there is nothing else to ask. Note what this cure cost that flatten_cat_tree did not: a Vec<String> had to change type, and the shared derivation could not be added beside the old one.

+

cats is the categories outermost-first; entries interleaves Group(FrameGroup) headers with Item(Vec<String>) label tuples exactly as GridLayout::row_items interleaves them. On the row axis the TUI paints a Group as its own row; on the column axis it paints the runs as spans above the headers. That difference is painting, and it is the only thing that differs.

+

cats may be empty — an axis carrying no category is a real layout (one nameless row or column), not a gap, hence levels() returning at least one. AxisFrame::flat leaves cats empty on purpose, so it is not a way to smuggle a multi-level axis in flattened.

+

FrameGroup::collapsed is on the wire rather than left to the client because it is a fact about the view, which only the server holds. FrameGroup::cat travels so a client can echo it back in ToggleGroup without ever working out which category a label came from — which slash-joined labels made impossible.

+

#AxisFrame: four accessors that are not for a draw loop

+

item, label, shows_level and group_of each scan entries from the start, so asking them per position is quadratic in the frame length and, in the TUI, grows with the absolute scroll offset. That defect was found three times on improvise-anr9. The per-position forms survive for one-off lookups only, and their two-line warnings are the thing standing between a future renderer and a measured regression.

+ +

The one-pass forms a draw loop wants:

+ +

leaf_labels() is not "the labels". It is the innermost label of each item: the single most specific name a position has, what a test or a one-line summary calls a row by. A renderer reaching for it on a multi-level axis is drawing the wrong header, not a shorter one.

+

first_rendered is the first data index actually on screen. That entry always shows its full tuple, or a scrolled viewport loses its group context at the top edge (improvise-jjx).

+

body_from(idx) drops the group header that owns the first drawn item: a frame opening mid-group carries that header so group_of can answer for it, but it is above the first drawn row and must not be painted as one. This is the same cut GridLayout::data_row_to_visual has always made.

+

Two things about its argument, and the first of them used to be the whole story and was wrong. It exists partly because the two front ends scroll differently — a browser's frame is already windowed and passes 0, the TUI's covers the whole grid and passes its row offset (gotcha 44). But idx alone cannot say whether the frame leads the axis, and at the top there is no header "above" to drop: it is content, and a collapsed first group has no other affordance. That is AxisFrame::first and improvise-0o6x — §3a.

+

group_of answers membership, and the paragraph that stood here said the opposite: that it answers "nearest preceding header", deliberately, because that is what the TUI had always painted, and that encoding an end boundary would mean changing AxisEntry in improvise-core and was "filed rather than smuggled into" improvise-anr9. All of that was true when written. The filed ticket is improvise-gd83 and it has landed — AxisEntry::GroupEnd closes a run, groups() clears on it, and the TUI's painted span changed with it. The mechanism and its two emission arms are §3a.

+

That correction is here rather than only in §3a on purpose. This paragraph forbade, in terms, the change §3a describes; a reader arriving at the stale copy first would conclude the rule had been smuggled past its own prohibition. When a fact is stated twice, changing it is a grep (gotcha 48) — and the two copies being in one file is what made this worth saying out loud.

+

#The width rule has one home

+

header_level_widths (improvise-1125): per level, the widest label in display cells plus a gap, clamped to [MIN_ROW_HEADER_W, MAX_ROW_HEADER_W]. It lived twice before — inline in src/ui/grid.rs::render_grid and again in improvise-command::ui::viewport::compute_row_header_width — and a browser sizing its own row header would have been a third.

+

It takes an iterator of borrowed tuples rather than an AxisFrame because its other caller holds a GridLayout and is on the keypress path (reached from App::cmd_context); building a frame there to ask the question would clone every row tuple on every key (raised by Copilot on PR #23). AxisFrame::header_widths is the frame-shaped entry point, derived rather than projected as a field because the frame is the whole input — nothing about the model is needed, so a shared function cannot fall out of step with the data the way a separately-sent field could.

+

#GridCache is viewport-indexed, and GridExtent is not

+

A cache position is a position within the viewport: the client does not know the model's coordinates, and the server is free to change what a row means as long as it says so.

+

GridExtent is how big the whole grid is, of which the frames are one window (improvise-bpq2). It is the one fact a client needs to move its own cursor: row_frame.len() is what this client was sent, so a client holding only its window cannot tell "the last visible row" from "the last row" and cannot clamp — a cursor clamped against the window stops at the bottom of the screen. It is deliberately not framed by the frames, unlike cells, col_widths and totals: it is the size of the thing they are a window onto, so fit_to_labels must leave it alone.

+

For a client whose window covers the whole grid — the TUI, which builds its cache at offsets zero — it equals the frame counts, and that coincidence is exactly what makes a derived version look correct there and fail in a browser. It is a named pair rather than two usize on GridCache, because beside row_frame/col_frame two bare counts read as "how many rows are in the frame", which is the confusion the field exists to remove.

+

GridUpdate::Extent's batch position is not load-bearing the way RowFrame-before-Cells is: it addresses no position, so it cannot overtake or be overtaken. It is emitted first only because outermost-first reads better.

+

#GridCache::totals — empty means "no totals", not "not caught up"

+

Framed by col_frame exactly as cells and col_widths are, with one asymmetry: an empty strip means this grid has no totals to draw (records mode, or an empty grid) and is therefore not padded, where a non-empty one is. Padding an empty one would draw a blank total row under a records-mode grid that has none.

+

The emptiness is the server's answer rather than a client's re-derivation, and the totals must be projected rather than client-derived: the sum runs over every row including the ones this viewport has scrolled past, so a client holding only its window could not compute it.

+

page_coords is framed by nothing — it names categories, not positions — so it survives a reshape that drops every row and column. It carries Page and Filter (View::filtering_cats, gotcha 27): both restrict what is visible, and the bar states what is being looked at, not what the user may page through.

+

#CategoryTreeUpdate::Items and ::Axes have no producer, deliberately

+

document_projections only ever sends Categories(..), which replaces the whole panel. The other two arms are derived rather than invented: the effects that move a category tree come in three shapes — the list reorders (AddCategory/RemoveCategory), one category's items move (AddItem/ RemoveItem), several axes move at once (SetAxis, and CycleAxis through it) — and doctrine(derive_the_enum_from_the_effects) is the same recipe that built ModelCommand. An arm the derivation produces is not speculation just because the producer has not reached for it.

+

What the producer reaches for instead is the coarse arm, on purpose: category_tree_of walks every category once, the panel is small, and Dirty::of cannot say which category moved anyway — an EffectKind says that a batch changed the tree, never which part, exactly as it cannot say which records a CellData batch wrote (gotcha 40). Emitting Items/Axes therefore needs the tree observed and diffed, the way document_update observes the title bar, and that is a bandwidth optimisation for a widget that costs a few hundred bytes. It is not worth doing until category-tree traffic is measured to matter.

+

The contrast with the deleted Invalidated is the point: these are unused, and Invalidated was redundant — its meaning already lived in running code. Unused-but-derived earns a note; redundant earns a deletion.

+

#The fold is not commutative — three rules in apply

+

(a) A Cells update that overtakes its frames is dropped, not guessed at. fit_to_labels truncates to the frames it has. A client must not reorder the stream.

+

(b) A frame change that renames a position blanks the cell under it. fit_to_labels preserves by position, which is only right where the label did not change, so apply calls invalidate_relabelled itself rather than trusting a producer to declare it. It blanks the totals too, because they are contents and a stale sum under fresh cells is the one number a user takes at face value.

+

There used to be a GridUpdate::Invalidated arm meaning "contents stale, frame unchanged", and it is deleted (improvise-n89z). It never had a producer: grid_updates diffs, so it always knows which cells moved and sends Cells, and GridCache::seed compacts. There is no state in which a server knows the contents are stale and not which ones — and if there ever were, the answer is a fresh Snapshot through rebase, not a blanking the client cannot act on. Meanwhile its stated meaning had migrated into running code: invalidate_relabelled blanks exactly what a reframing invalidates, totals included. So it was redundant, not merely unused, and three documents described it as load-bearing, which is worse than silence — a reader reasoning about the fold's guarantees assumed something produced it. the_retired_invalidated_spelling_does_not_parse keeps it from returning as a serde alias.

+

(c) Every arm that touches a frame or the widths re-fits. A renderer zips widths with labels, and a length mismatch indexes off the end. Fitting on the frame arms alone is what let a late ColWidths put the mismatch straight back: a short update keeps each unheard-of column at its own widest level label until ColWidths catches up, and a long one is dropped as having overtaken the frame that names it, exactly as (a) holds Cells.

+

The fallback width is measured in display cells, the same unit header_widths and the authoritative ColWidths use: chars().count() would draw 日本 two cells wide where it needs four, so a cache caught between ColFrame and ColWidths would render the new column too narrow. The materialization inside fit_to_labels is deliberate — AxisFrame::item scans, so asking it per column would make a re-fit quadratic.

+

#invalidate_relabelled — a cats change re-identifies everything

+

A cell's identity is cats zipped with its item tuple, so a change of cats re-identifies every position at once — even the positions whose labels are byte-identical. Region = [North] becoming Department = [North] leaves items equal and names a different cell, so the tuples alone cannot answer this and every surviving position is stale.

+

The sharpest half is invisible: a preserved CellDisplay carries an editable CellId::Record (ADR 002), so a client would go on offering an edit that addresses the record the old coordinate named — a misdirected write, in the same family as gotcha 18's drill snapshot, and one no rendering shows.

+

Otherwise the comparison is on the item tuples, not the entries: a group header appearing or collapsing changes the frame's shape without renaming any surviving position, and blanking on that would throw away cells the very next Cells update would have to resend.

+

Two asymmetries about totals:

+ +

This is what obliges the producer to diff the totals against the frame the label updates will leave rather than against the mirror they change: a column renamed while its total stayed the same number is blanked here, and a producer comparing mirror.totals would find them equal and send nothing. grid_updates compares framed.totals for exactly that reason. The blanking stays anyway, because rule (b) exists to stop apply trusting a producer to remember — the rule lives on both sides or on neither.

+

LabelAxis is private so the row and column cases share one invalidation rule rather than two copies that can disagree.

+

#Bootstrap is a compaction, not a replay

+

A server does not replay history to a late client — it holds the current state and expresses it as the same updates the steady stream carries, so bootstrap is not a second code path and there is no whole-state shape that could disagree with the incremental one. The property that earns it is GridCache::folded(&cache.seed()) == *cache, and it is the ordering that earns it: the frames frame the cells, so they go first.

+

Snapshot { view, seed } is the only thing on the wire that is not an event, and view is the one point at which ViewState is serialized (improvise-q08).

+

Note the fold tests: asserting fold(prefix) + fold(rest) == fold(whole) is vacuous — it is the definition of a left fold — so the tests that earn their place are the order-sensitivity one and the compaction-equivalence one (doctrine(fold_tests_must_test_order_not_the_axiom)).

+

#CategoryTreeCache carries every category, _-prefixed included

+

improvise-vjms: the TUI panel has always listed them, and a browser panel that quietly did not would be a second, divergent answer to "what are the categories". So the _-prefixed convention does reach the client — it is presentation, and both ends render the same rows from this one source.

+

The trap is _Measure, whose items are derived: explicit items ∪ formula targets (gotcha 7). Whoever builds these rows must go through Model::effective_item_names; reading category("_Measure").items compiles, looks right, and silently omits every formula target.

+

Three mutations reach this panel because the effects are three shapes: adding or removing a category reorders the whole list, adding or removing an item touches one category, and a transpose moves several axes at once without touching membership. Which categories are expanded is ViewState::expanded_cats, per-client, so the flattening is a client-side concern and does not belong on the wire.

+

FormulaCache and ViewListCache take one whole-list update, because add and delete are the only effects that touch them and both are cheap to send whole. Which view is active is not in ViewListCache: that is ViewState::active_view, carried by Command::SwitchView, because it is per-client (improvise-8uz). Likewise which tile the cursor is on is ViewState::tile_cat_idx.

+

#WizardCache has a shape and deliberately no Projection

+

improvise-764, and that absence is the ticket's answer to "where does the wizard state live". A Projection is broadcast to every connected client's widget, which is right exactly when the change is addressed to no one session. A half-open import dialogue is the opposite: it belongs to precisely the session that opened it. That is already settled doctrine elsewhere — ImportWizard lives on App and not on ViewState, and AcceptWizardModel was removed from ModelCommand for this same reason (improvise-tom1) after B was shown swapping the document out of a dialogue B never opened. Putting the wizard on the broadcast stream would have re-created that bug on the rendering side: every client drawing a dialogue one of them opened.

+

So it is a cache in shape and a local render model in fact, built from the authoritative App by the host that holds the wizard. A cache without a Projection is the shape session-local residue takes when it still has to be drawn by two different front ends. It lives here rather than in improvise-command for the flatten_cat_tree reason: the browser client depends on improvise-protocol alone, and when it grows a wizard it must draw these rows, not a second derivation. When multi-client import arrives it wants the addressed form, which is where ADR 001 layer 2 goes anyway.

+

Every field is already render-ready — the widget that used to build these reached through ImportWizard into ImportPipeline, FieldKind, DateComponent and a whole Workbook; none of those cross now. Two substitutions are made here so both ends spell them the same way: an undetected date format reads "?", and the root array path reads "(root)" rather than empty. WizardProposal::kind_label is carried rather than re-derived from kind, so the two ends cannot disagree about what "Measure (numeric)" is called.

+

WizardStepKind mirrors improvise_io::import::WizardStep, spelled again rather than re-exported, because this crate must not depend on improvise-io (the wizard is an io type, gotcha 30) and a client that draws the wizard must still be able to name the step it is on. WizardFieldKind is the semantic half of a colour decision — the terminal mapping lives in draw.rs, exactly as AxisColor/axis_fg does (gotcha 26).

+

#Projection carries no multi-widget variant

+

The envelope a widget subscribes by: it folds the payload of the variant that names it and ignores the rest. There is deliberately no variant carrying several widgets' updates together — that would be the aggregate cache in another form.

+

Nothing is defined for the help overlay: its content is static and help_page is already in ViewState.

+
+

#ColWidths stays diffed against the mirror, and the ticket's reason was wrong

+

improvise-44ad asked whether grid_updates should diff col_widths against the label-folded cache rather than against the mirror, as improvise-hkt6 already obliged for totals. Folding the labels re-fits the widths, so under a fold-relative diff the ColWidths a reframe currently carries becomes redundant and is suppressed. Declined, and the two things learned are worth more than the change.

+

The stated cost does not exist. The ticket says the move breaks a_client_on_a_socket_is_seeded_and_gets_its_edit_back, and that that assertion is §3(c)'s standing evidence — so rewriting it inside a change §3(c) guards would be the improvise-q0vb / PR #12 shape. Measured 2026-09-21 by deleting the re-fit from fit_to_labels: three existing unit tests fail (a_col_widths_update_is_fitted_to_the_labels_it_is_framed_by, a_column_without_its_width_yet_is_measured_in_display_cells, col_widths_always_span_the_column_labels) and the socket test is not one of them. What it actually asserts is that a reframe carried widths — a statement about the producer's diff, not about the consumer's re-fit. §3(c) is triply guarded elsewhere, so the evidence argument was about the wrong test.

+

The real reason to decline is coupling. The widths a label-fold invents are each column's own label width, which is an approximation the consumer applies until the authority catches up. Diffing the producer's output against it makes what the server sends depend on a fallback rule defined in the client. Today both are in this crate and the dependency is invisible; the day a second client implements the fold, the producer's message set silently depends on that client's approximation. The redundant ColWidths per reframe is what buys out of that, and it is the direction Dirty::of is deliberately sloppy in (§5, repo-map gotcha 35): over-sending costs one projection, under-sending leaves a client wrong with nothing in the stream to correct it.

+

So the asymmetry between totals and col_widths is stated rather than discovered. totals is fold-relative because hkt6 found a real hole — a rename that leaves the number equal sends no Totals and the client blanks its own. col_widths has no such hole, because the re-fit is a safe fallback rather than a stale value.

+

What was landed is the half the ticket calls "the better test either way": a_client_never_holds_widths_of_a_different_length_from_its_labels asserts the invariant §3(c) is really about, at every step of a fold that widens before its widths arrive and narrows before they shrink — not merely at the end, which is where a message-count assertion looks.

+

#GridCache::records — what kind of grid this is (improvise-e5pf)

+

The host answers "is this the wide records layout" with GridLayout::is_records_mode(), which reads the axes. A thin client has no GridLayout, and it was answering the neighbouring question instead — ViewState::mode.is_records() — at three decision points in resolve.rs. The two agree only while the mode and the layout agree, and Command::ChangeMode(AppMode::Normal) is precisely the message that separates them: the host settles Normal against the layout the batch ends on (App::settle_normal_mode), and reduce_view folds the variant it was handed.

+

So the fact is projected, not derived: GridCache::records, carried by GridUpdate::Records. Deriving it on the client from row_frame.cats containing _Index would have been a second implementation of is_records_mode — improvise-edp's disease — and deriving it from the mode is the bug itself.

+

Like GridUpdate::Extent it frames nothing, so apply does not re-fit and its position in a batch is not load-bearing: it describes what kind of grid the frames describe, not any position in one.

+

The settling is observed, never composed — and the first version of this fix got that wrong. It settled at each composition site, in resolve::changed_mode, against the grid the client held at the moment it resolved the key. ChatGPT Codex found the hole on PR #98: switch_view_at composes [SwitchView(target), ChangeMode(Normal)], so the settling read the outgoing view's grid and records → pivot emitted RecordsNormal while pivot → records emitted Normal. The drift the fix existed to close, by a second door.

+

The unreachability argument the first version leaned on covered the toggle route only — R is a stranding decline in the browser — and records-ness is auto-detected from the axes (gotcha 1), so a view switch changes it with nothing in the batch saying "records". doctrine(an_unreachability_claim_is_only_as_wide_as_the_routes_enumerated).

+

So changed_mode is deleted, every site emits the plain AppMode::Normal, and ClientState::fold settles after every fold against the grid it holds: self.view.mode = settled_mode(&self.view.mode, self.grid.records). That is route-independent — it needs no enumeration of composition sites, which is the part nothing checks — and it is the same shape the repo already prefers elsewhere: cursor_epoch over matching on CycleAxis (gotcha 24), document_generation over matching on LoadModel (gotcha 39), the dirty flag over a table of command variants (gotcha 45).

+

settled_mode is the one rule, in improvise-protocol, and all three ends call it: ClientState::fold, ClientSession::fold (the server's mirror, so the two cannot diverge in a field nobody currently reads) and App::settle_normal_mode. App::normal_mode is gone — it was that function's host-side duplicate. It settles the Normal pair and nothing else: Editing/RecordsEditing pass through untouched, which is exactly why a records decision asks the grid and never the mode.

+

There remains a window, and it is inherent: between the local optimistic fold of a SwitchView and the arrival of the target view's projection, the client holds the outgoing grid and cannot know the target's records-ness. The mode settles the moment the projection lands. No design removes that window — read-time derivation has exactly the same exposure, since it would read the same stale grid — so the honest statement is that the mode settles when the grid does, on both ends.

+

The three decision points ask state.grid.records. What that buys concretely: the commit at a records cell declines rather than composing typed_cell_command against a grid whose columns are [coords…, +cat, measures…, +] (gotcha 1), which was a wrong write and strictly worse than the black hole it was written to produce.

+

#§3a. The frame's group boundaries and its position in the axis

+

Two facts AxisFrame gained after improvise-anr9, both about group runs.

+

FrameEntry::GroupEnd closes a run. group_of used to answer "nearest preceding header", which is not membership: expand_category emitted a header on entering a group and nothing on leaving one, so an ungrouped item after a grouped run inherited it (improvise-gd83). The §3 paragraph that recorded the old rule — and forbade this change — is corrected there, not merely superseded here. AxisEntry::GroupEnd is the boundary, carried onto the wire as FrameEntry::GroupEnd; groups() clears current on it and group_of stops walking. It is emitted only on Some -> None, never between two groups — the next header already ends the previous run — and at the end of an expansion whose last item was grouped, which is the across-prefix arm cross_product creates.

+

Nothing draws a GroupEnd. Both front ends skip it and it consumes no data index, so it is invisible except through group_of/groups.

+

AxisFrame::first is the absolute data index of the frame's first item. The windowing rule carries the nearest header above a scrolled window so group_of can answer, and body_from drops it on the way to the screen. At the top of an axis there is no "above", so that same leading header is content — a collapsed first group has no other affordance (improvise-0o6x). idx alone cannot tell the two apart, because the browser's frame is already windowed and always passes 0; first + idx == 0 can.

+

The TUI builds over the whole grid so first is always 0 there, and the browser's comes from the projected window. A frame built by hand in a test must set it, which is why it is a plain field rather than derived.

+

It carried #[serde(default)], which was a defect rather than a decision and is now gone (improvise-1wl0): first: 0 is not an absence but the claim "this frame leads the axis", which is the claim body_from acts on — so a frame that lost the field deserialized as leading, and a scrolled browser drew the carried header above its first row. Same argument as Basis having no Default: a zero-valued claim that happens to be true in the common case is a forgeable bypass rather than an absence.

+

Dropping the attribute was preferred to Option<usize> or a two-variant enum, and the reason is about the wire rather than the type. There is exactly one production producer (project.rs::axis_frame, which reads window.start), so a frame is never partially constructed and "unknown" is not a state any honest peer can be in — making it representable would oblige body_from to answer for it, which is the same question one level down. A missing field is now a loud parse failure at the transport, where a wire-format skew belongs. No skew ships today, both ends being one workspace at one commit; this is about the day there is one, and about the attribute reading as a considered compatibility decision when it was not.

+

The Default derive still yields first: 0, and that is not the same hole: an AxisFrame::default() has no entries, so body_from returns an empty slice on either arm and the claim is vacuous. The hazard was specifically a populated frame acquiring the claim by omission.

+

group_runs / group_run_heads are the coalescing rule's one home. The TUI and render.rs::col_groups each coalesced adjacent columns sharing a group, kept in step by a comment (improvise-8qlr). The TUI's one divergence is that it labels its first visible column even mid-run, so a scrolled grid does not draw an unlabelled group line.

+

A run is opened by a HEADER, not by a name (improvise-h0iv). Both prior copies coalesced by name alone, and improvise-8qlr preserved that because it was a consolidation: changing the rule in the commit that moved it would have left the agreement test unable to tell "both ends agree" from "both ends changed together". With one home that argument is spent, and the rule can be changed once for both ends — which is the shape to notice, since a consolidation's frozen behaviour reads as decided afterwards and nothing says it was only frozen for the duration of the move.

+

The ticket's own proposed fix does not work, and that is worth more than the fix. h0iv suggested comparing the (cat, name) pair, FrameGroup carrying cat precisely so a client need not work out which category a label came from. But its own reachability case is one category: Month grouped into Q1, nested under Region, giving North/Q1 and South/Q1 adjacent — the two headers are (Month, Q1) and (Month, Q1), equal as values, so the pair separates nothing. the_two_headers_are_equal_as_values_so_comparing_cat_and_name_cannot_split_them pins that, so the next reader does not re-propose it. What separates them is that they are two different FrameEntry::Group entries, and groups() already borrows each from self.entries — so the comparison is std::ptr::eq on the header, which is the literal statement of "the same header opened both".

+

The case that settled the product question was not the span but the indicator. Two adjacent same-named runs might well be one span a user wants spanned, which is why h0iv was filed rather than fixed. What is not arguable is column_groups_coalesce_by_name_as_the_tui_does, whose fixture merged Month/Q1 (expanded) with Fiscal/Q1 (collapsed) and whose own message said the merged span is "drawn with the FIRST one's indicator" — a collapsed group rendered as expanded, with no affordance saying otherwise. A span boundary is a matter of taste; a wrong collapse glyph is a wrong fact. So: two runs. Reversing it is one comparison, and the test is renamed column_groups_coalesce_per_header_as_the_tui_does rather than deleted.

+

#4. frame.rs — one envelope pair, two deployments

+

#The framing adds as little as it can get away with

+

doctrine(derive_the_enum_from_the_effects): inventing wire variants ahead of the mutations that need them produces a protocol nobody can implement.

+

#There were three pairs for two deployments (improvise-xks6)

+

The socket's server side, a hand-kept mirror of it in the browser bundle, and a third pair written for the worker (improvise-tm6) before the projection architecture landed. The question that had to be answered before any of it moved was whether the worker and the socket want the same vocabulary. They do, and the test applied to each difference was whether the argument for it names the thing they differ in. None did — every difference was a date rather than a decision:

+ +

The worker pair was not merely different but incoherent by omission: written before ADR 001, it could carry Command::Model { basis, .. } and had no Rejected, so a client speaking it composed a claim that could never be answered. That incoherence — not the difference — is the tell for staleness.

+

#The mirror is gone, and so are the two tests that held it honest

+

That deletion was the ticket's measure of success. the_socket_framing_matches_the_server round-tripped hand-listed fixtures through the other crate's types to catch a rename; every_server_variant_is_one_this_crate_mirrors matched exhaustively to catch a variant added on the server and never mirrored. With one definition both are type identity: a rename is a compile error at every call site in both crates, and the un-mirrored variant is not a constructible state. That is stronger than either test, and it is the shape to insist on — if a consolidation leaves one of its compatibility tests alive, the consolidation is not finished (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)).

+

What survives is the_retired_spellings_do_not_parse, aimed at the one thing still possible; see context/test-evidence/protocol.md.

+

#ClientFrame::Commands — the frame is the unit of atomicity

+

Not the command (ADR 001, improvise-955n). The host answers for every command in the frame before applying any, so a frame is never applied partially because of a refusal. It used to apply in order and abandon at the first rejection, which left a hole ADR 001's "never applied, not even partially" does not have.

+

The frame states one basis, and it is the only claim a client can construct: a basis is published between batches and never during one, so no client can hold one describing the document as it would be half-way through its own frame. Commands then run in order, each against what its predecessor left, and data dependence between them is expected — a chain may create a record and then write it.

+

But only two of the three tiers are answered once. document_generation and layout_version describe the document, so answering them on the frame's first mutation is right. The record tier is asked of every mutation, because which records are at stake is a property of the command rather than of the frame — skipping it for later commands let one client's chain overwrite a record another had moved, silently. The exemption that keeps data-dependent chains working is narrower than "later commands are not checked": a record that neither resolves nor is claimed is one the frame is about to create, and only that combination is exempt. A record that resolves is answered for however the claim is written. CheckedBatch in improvise-command carries the derivation.

+

All-or-nothing is the contract and clause 4 is not yet delivered. That is the other way a frame stops half-way, and it is not a refusal: a chain whose later command fails at apply time leaves its predecessors applied, because App::apply_effects aborts without undoing (gotcha 21 — an abort is not a rollback).

+

#ClientFrame::Save is a request, not a Command

+

improvise-368h. It carries no path, because a client cannot see the host's storage: effect::Save writes to this host's file_path through this host's Storage (gotcha 2), and which document a host is hosting is the host's own fact. A client naming a path would be naming a namespace it cannot read — the same mistake ModelCommand::SetCell avoids by taking a surrogate rather than a CellKey.

+

It is a frame and not a Command, and the reason is atomicity. It began as a Command, which put a runtime-fallible operation inside a batch Commands guarantees is applied whole or not at all — so [Save, SetCell] reported a failed write and then applied the edit anyway, and a client reading "refused" could not tell which. Saving also fails the test every Command passes: it mutates no shared state and no client's slice, so reduce_view has nothing to fold and Checked has no basis to answer for. Here it cannot be composed with a mutation at all — the batching question stops being a rule and becomes unconstructible — and it sits beside Viewport, the other session input no Command carries.

+

Success is silent: the document did not move, so no widget is dirty and nobody else is owed anything.

+

#ClientFrame::Key — and a correction

+

The doc used to say "the host runs it through the keymap against the authoritative App". It does not. Host::handle answers this arm with an explicit empty vec, because a key resolved on the host moves state no client can hear (a_key_resolved_on_the_host_moves_state_no_client_can_hear): the projection stream carries nothing of the host's own ViewState, so host-side resolution diverges. improvise-bpq2 closes that by avoiding it — the browser resolves keys against the shared keymap (§6) — so an upstream key is a black hole, not a slow path, and every KeyOutcome::Upstream names a binding that does nothing at all in a browser. The variant is kept because the list of keys that still need the host is named in improvise-wasm-client/src/keys.rs, so closing one has to come there and say so.

+

It carries (KeyCode, KeyModifiers) rather than a KeyEvent because KeyEvent is not Serialize — the same pair Command::SetTransientPrefix already puts on the wire.

+

#ServerFrame has three variants because Sessions::apply has three cases

+

Applied::Diff, Applied::Rebased, Applied::Rejected (gotcha 39).

+

Snapshot is boxed: it carries a whole ViewState plus the seed that redraws the document, and every Commands frame would otherwise be allocated at that size for a variant sent once per connection. Serde sees through the box, so the JSON is unchanged. It is sent again to every client whenever the document is replaced, because a client's cursors are keyed by the outgoing document's view names and only a fresh snapshot replaces them (gotcha 24).

+

ServerFrame::Commands carries both of the things a client folds, because on the wire they are the same thing: a Command::Project is a widget update, and any other Command is a peer's accepted move that this client's own view slice must fold too (gotcha 36). Order is load-bearing, so a transport must deliver a batch whole and in sequence.

+

Rejected is addressed to the asker alone — nothing happened, so nobody else is owed anything — and it carries no new state, because the client already holds what is current through the projections it has been folding all along. That is what makes rejection cheap enough to be a normal outcome rather than an error.

+
+

#5. basis.rs — what a batch was composed against

+

#The problem is not ordering

+

The server already serialises every write — one App, one apply_effects — so there is a total order and nothing here is about ordering. What was missing is noticing that a client composed an edit against a picture of the document that has since moved: the same stale-basis shape as the cursor under a reshaped grid (gotcha 24) and the session mirror after a document swap (gotcha 39), one level up.

+

#Three tiers, and the tiering is the whole economy

+
tiermoved bya batch is stale if
document_generationa WholeWorkbook effectit moved at all
layout_versionan observed change to any view's drawn shapeit moved at all
recordsa CellData effect on that recordany record the batch names moved
+

A cell edit is stale when the layout moved, even though no record it names was touched, because the shape its keys hang off changed underneath it. That is ADR 001's class 2 — A removes the category Region while B commits an edit keyed by Region=East — and it is corruption rather than a lost keystroke. Detecting it falls out of the tiering with no lock anywhere: a cell-level lock would not have covered it, because holding Region=East × Revenue does not stop anyone deleting Region.

+

#The client understands none of this

+

A Basis is three opaque tokens — nothing in it is a structure to interpret, the same discipline as the cell surrogate it is keyed by (ADR 002). The server publishes the current one as a Projection, the client keeps it in a BasisCache and hands back claim() when it composes a write.

+

#Basis has no Default and no public constructor

+

A zero-valued basis is not "no claim", it is the claim that the document is fresh — which in a brand-new App is true, so a default would be a forgeable bypass rather than an absence. BasisCache::claim is the only door a client has, and it returns None until the server has published one.

+

Note what that does and does not buy. Basis derives Deserialize because it must cross the wire, and its fields are public, so a hand-rolled client can state any tiers it likes. This is an anti-staleness coordination check, not a security boundary: it closes omission, not invention, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having — it makes the honest path the easy one.

+

There is deliberately no flag that skips any of this, because "a mechanism whose only purpose is to violate the invariant is not a feature, it is the invariant deleted with extra steps". A force re-composes the same intent against the current basis and submits a fresh batch; it never applies a stale one.

+

#claim omits rather than invents

+

A record the cache has never heard of is left out, not defaulted to zero. Defaulting would be a claim the client cannot support, and the server would read it as "untouched" — the exact forged-basis case RecordUnstated exists to catch.

+

#The publication is what ADR 001 omitted

+

The ADR says "a batch carries the basis it was composed against" and never says how a client comes to know one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. That is the one place that document met the code and was wrong. Projection::Basis + BasisCache close it.

+

It is folded like any other widget (improvise-rkeq) — not a widget, nothing draws it, but it subscribes on exactly the same terms, and the alternative was a second server→client channel with its own ordering rules for a payload that changes at precisely the moments a projection already does. The update replaces wholesale, as Projection::Formulas does, so "has it changed" and "send it" collapse into one decision.

+

It is emitted after the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown — the mirror image of the fold's frame-before-cells rule.

+

#Which records are checked is the server's decision

+

reduce.rs::records_named in improvise-command, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the client's claim would be a force by omission: the record tier skipped, with no flag to remove and nothing in the message saying so.

+

#published() takes a BTreeMap

+

So the record list is deduplicated and in a stable order. The server diffs the published basis against what it last sent this client, and a list whose order followed grid traversal would report a change every time a row moved.

+

#Rejection keeps structural and data staleness apart

+

Which is the whole reason ADR 002 put a maintained RecordHash index in DataStore: a regenerate-on-demand scheme could not tell a record that had scrolled out of view from one that no longer existed.

+ +

This is also why the wasm client stores the Rejection itself rather than collapsing it into status_msg: a string cannot carry which record and which kind.

+

#Two classes live in one type

+

Rejection is a refusal of a frame, not of a command — since improvise-368h a ClientFrame::Save the host cannot honour is answered here too, and that is not a refused command at all.

+
    +
  1. Staleness (ADR 001) — DocumentReplaced, LayoutMoved and the three record variants. The only class where force means anything.
  2. +
  3. A request the host cannot honour — NoDocumentPath and SaveFailed (improvise-368h). Nothing went stale; there is nothing to re-compose.
  4. +
+

They share this type rather than getting refusal channels of their own, for the reason the type exists: a refusal is "nothing happened, here is why", addressed to the asker alone, and a second channel would be a second set of ordering and addressing rules for one idea. A third class is coming and is not here yet: a transaction rolled back because a later command in a chain could not be applied (clause 4 of the frame contract). Until rollback exists there is nothing to report, because nothing is undone.

+

NoDocumentPath is decided before anything runs, by the same constructor that answers for a basis, so a save that cannot land is refused rather than attempted. What the asker should do about it differs by front end: a TUI prompts for a path (improvise-lae), a browser worker whose document is its storage entry never sees this at all. SaveFailed carries no message because Rejection is Copy and an I/O error's text is not — the host keeps the text in its own status line, and what crosses the wire is the one fact a client can act on: the document it is editing is not on disk. Reporting a failed write as success is the thing this exists to stop.

+
+

#6. keymap.rs — one table, both ends

+

This used to live in improvise-command, where only the terminal could reach it, and the browser client carried a hand-written classifier answering the same question for the subset it could resolve. That is two definitions of one vocabulary kept in step by a lockstep test over a hand-written list of keys — and doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished) says such a pair is one definition waiting to be made.

+

The move is the one gotcha 13 already anticipated: ViewState::transient_prefix stores a prefix as an address rather than an Arc<Keymap> precisely so ViewState could stay free of improvise-command types while the keymap came the other way.

+

#The boundary that did not move

+

A lookup yields a command name and arguments. Turning that into something that runs needs a CmdContext — a Model, a GridLayout, an App — which a thin client does not have, so intent → mutation stays where it was: improvise-command resolves a name through its CmdRegistry into Effects, and a browser client resolves the subset it can compose into Commands.

+

That is a better seam than the one it replaces. The drift risk used to be "which key does what" — ~170 bindings, no direct test, invisible when wrong. It is now "which command names can I execute", which is a list.

+

What stayed in improvise-command, and why each had to:

+ +

The keymap's own tests live with dispatch in improvise-command/src/command/keymap.rs, for the same reason: they need a registry to check a bound name against.

+

#ModeKey and AppMode are two different enumerations

+

They happen to be the same size (17). SearchMode is a ModeKey and not an AppMode — it is Normal plus search_mode: bool; Quit is an AppMode with no keymap. Do not "fix" one count to match the other. ModeKey::from_app_mode is exhaustive on purpose: a new AppMode without a keymap silently swallows every keypress, so the compiler must flag it.

+

#Lookup fallback order

+

exact(key, mods) → Char with NONE (shift is implicit in the char) → AnyChar → Any → parent. Keymap::with_parent is Emacs-style inheritance, and it is what lets RecordsNormal reach Normal's g/y/Z prefixes.

+

Keymaps::prefix_keymap deliberately goes through lookup, which walks parents, for exactly that reason. Resolution assumes prefixes are flat; prefix_keymaps_are_flat fails the build if a nested one is added.

+

#TextEntry — the spine, welded

+

Every text-entry mode has the same spine: characters append to one named buffer, Backspace pops from it, Esc clears it before leaving. Only the buffer, the exit mode and the commit keys differ. Building them through here means a new text mode cannot accidentally type into one buffer while clearing another, or drop the clear entirely — which would leave the next entry pre-filled with the last one.

+

TextEntry::new takes the AppMode and reads the buffer name off its MinibufferConfig, so the buffer the prompt describes and the buffer the keys edit are the same string by construction rather than by agreement (improvise-k8h) — the keymap never spells a buffer name.

+

commit(key, cmd, args) welds the command to its clear-buffer so no caller can bind a commit without its clear (improvise-w3q). rebind exists for exactly one case: CommandMode's Backspace, which exits the mode on an empty : line instead of popping a character that isn't there.

+

Eight AppMode variants carry a MinibufferConfig, and buffer_of's callers are all eight. Seven are built by TextEntry::new; RecordsEditing is built by TextEntry::with_parent off Editing's map, so a count taken off the ::new call sites reads seven and is not the number of text-entry modes. (See repo-map gotcha 11 — this exact count has already gone stale once.)

+

SearchMode is hand-written on purpose and does not use TextEntry: it has no commit key, and its query is owned by exit-search-mode rather than by clear-buffer. "search" is not in the buffers map but its own ViewState field, and cmd::core::read_buffer/write_buffer are the only code that knows it (improvise-gdl).

+

Category and item add stay open after a commit, so Tab and Enter do the same thing: file this one and start the next. ItemAdd's keymap is built with an empty category name because the category only shapes the prompt text, never the buffer.

+

#PanelKeymap

+

The three panels share a spine — Esc/Tab leave for Normal, arrows and j/k move that panel's cursor, every panel's toggle key is bound so focus can move between them, and : opens command mode. The keys that act on the panel's contents differ in shape, not just in command name (Category's n takes an argument, View has no a, Formula has no Enter), so they arrive through chained methods rather than a fixed-arity signature (improvise-4pg). delete() binds d and Delete together so a panel cannot offer one without the other.

+

bind_panel_toggles is called by PanelKeymap::new and by Normal, so a new panel reaches all four keymaps from one Panel::ALL entry.

+

#Normal-mode bindings worth a note

+

i/a pass their target editing mode as an argument to edit-or-drill, so the command itself stays mode-agnostic; RecordsNormal overrides just those two bindings to say records-editing. Ctrl-S saves to the known path and Ctrl-W asks for one — the pair a user with no file path needs, and what the "No file path" status message points at (improvise-lae).

+
+

#7. viewmodel.rs — the residue between a cache and a renderer

+

improvise-edp asked for a ViewModel layer, a SurfaceViewModel paired with every SurfaceCache. Most of that layer turned out not to exist: after improvise-35e each widget already takes exactly one cache plus the mode, and a cache built per-widget is already render-shaped, so there is nothing left to adapt. What this module holds is the residue — derivations genuinely between a cache and a renderer, because they mix a cache with per-client ViewState.

+

Today that is exactly one: flatten_cat_tree. It lives here rather than in improvise-command because two ends need it and only one of them can reach that crate — the browser client depends on improvise-protocol alone (deliberately; see its Cargo.toml), and move-panel-cursor walks this flattening, so resolving that key client-side (improvise-bpq2) would mean re-deriving it. Two derivations of one flattening is how the two ends drift.

+

CatTreeEntry carries the axis outright, so a renderer needs neither a View nor the cache the entry came from: the entry list is what the panel draws. expanded is ViewState::expanded_cats — per-client, which is why this is derived here and not sent — and the index into the result is ViewState::cat_panel_cursor.

+
+

#8. key.rs

+

These mirror the subset of the terminal backend's key API the command, keymap and effect layers actually use, so those layers carry no terminal-backend dependency. Variant names match crossterm's so call sites only need an import change. The only place both type families may appear is the TUI boundary (src/draw.rs), which converts incoming backend events into these before calling App::handle_key.

+

KeyCode::Null stands in for "no key" (headless command execution) and for any backend key code this crate does not model. KeyEvent is not Serialize — ClientFrame::Key and Command::SetTransientPrefix both carry the (KeyCode, KeyModifiers) pair instead.

+
+

#9. Corrections made while relocating this prose (2026-09-20)

+

Four comments in this crate were wrong about the code and were fixed rather than moved. Recorded because the campaign's own evidence is that a stale argument reads exactly like a live one:

+
    +
  1. frame.rs, ClientFrame::Key said "the host runs it through the keymap against the authoritative App". Host::handle returns an empty vec for that arm. This is the fourth sibling of gotcha 48's "three comments in one range said not built yet" — improvise-bpq2 moved resolution to the client and updated one copy of the argument, not all of them.
  2. +
  3. reduce.rs, every_command_round_trips_through_json said the model half brings "four payload types the view half never carried — CellKey, CellValue, Axis and PathBuf". ModelCommand carries no CellKey at all since ADR 002 (SetCell takes a RecordCell), and CellValue is in the view half, as Command::SetYanked.
  4. +
  5. render.rs, identified_grid carried three paragraphs of doc that describe a different test — "improvise-hkt6 … three columns and a rename in the middle" is a_relabelled_column_drops_its_total, which had no doc of its own, while identified_grid is a two-column fixture. The paragraphs had been merged onto the wrong item.
  6. +
  7. keymap.rs, buffer_of said "the seven modes that carry a MinibufferConfig". Eight do, since SavePrompt (improvise-lae); seven is the number of TextEntry::new call sites, which is the exact confusion repo-map gotcha 11 already warns about.
  8. +
+

Also stale and corrected, without being wrong enough to list above: key.rs's module doc called these "crate-local key types for the command layer" — they are the protocol's, and both front ends use them.

+

records_named lives here (moved from improvise-command::ui::reduce by the review of PR #50). It is &ModelCommand -> Vec<RecordHash> with no App in it, and both ends need the same answer: the server to decide which records to check, a client to decide which to claim. Two copies made the client's half untestable — lockstep.rs restates the basis against its twin host, so it compares the server's answer with itself. One home makes the claim right by construction, and the exhaustive match is the guard it always was: a new record-writing variant fails the build.

+
+

#§8 help.rs — where UI strings live relative to a wire enum

+

Three tickets asked one question three times — improvise-x7sw (which-key hints have no protocol home), improvise-ktv1 (the browser has no help overlay) and improvise-k099 (AppMode::hint_text/label put terminal UI strings on a serde enum). Answering it three times is the disease. The answer:

+

A UI string is keyed by a command name, never by a key, and never hangs off the enum that crosses the wire. AppMode::label and AppMode::hint_text are gone; help::mode_label and help::hint_line replace them. The enum a client deserializes now carries no English at all.

+

And wherever a string names a key, the key is looked up rather than spelled. Hint { keys, command, caption } is the unit: keys come off Keymaps::default_keymaps(), command is the semantic value a localized or non-terminal front end keys off, and caption is the default English — exactly the AxisColor/draw.rs::axis_fg shape, with the decision shared and only the last translation local (invariant(semantic_palette_at_the_core_boundary)).

+

#What that bought, and what it cost

+

It deleted every_hint_key_resolves_in_its_mode and its 60-line tokenizer. That test existed to hold a hand-written string honest against the keymap; the string is derived now, so the property is true by construction. A consolidation that keeps its compatibility test is unfinished (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)), and this is the deletion that measures this one. What replaced it, every_advertised_hint_names_a_binding_that_exists, checks the other direction — a mode advertising a command nothing binds — which was never checkable before.

+

It also deleted Keymap::binding_hints: help::which_key is the one walker, and the binding dump reads it too rather than a second one.

+

The cost is that the hints changed shape. They are longer, they name arrow keys beside hjkl, and two sub-lists in the help pages lost their visual indent. That is the price of the keys being true.

+

#The three rules the rendering follows

+
    +
  1. hint_line drops chorded keys — any key held with a modifier beyond shift, asked of the modifiers and never of the label (improvise-thpe). Hint::keys is (KeyCode, KeyModifiers) and compact takes the pairs, so the string it used to parse is not in scope to parse; Hint::labels and the help pages format last, through the one spelling in keymap::key_label. T advertises tile select; its four Ctrl-arrow aliases would push the rest of the line off the screen and teach nobody anything. A client wanting all of them reads hints(), which keeps them.
  2. +
  3. It runs single-character keys together (hjkl) and slashes the rest (Esc/Tab), which is the idiom the hand-written hints already used.
  4. +
  5. A help-page row whose command resolves to no keys is dropped, so a page can never advertise a key that does not exist. keys() returning Option is what makes that unrepresentable rather than validated.
  6. +
+

#What is not derived, and why

+

The wizard's. AppMode::ImportWizard's whole keymap is one Any binding into HandleWizardKey, which decides per step inside the command — so there is nothing in the table to look up, and its hint is a hand-written tail. That is a real hole and it is where it is visible, not papered over: a keymap cannot describe keys a command resolves privately.

+

gg, yy, ZZ and the ex-commands. A prefix sequence is two addresses and not one binding (gotcha 13), and :q/:w are parsed text rather than keys. Both are HelpLine::Literal, which says out loud that the row is spelled rather than derived.

+

The captions. i:edit where i entered records mode would still pass every test here. A caption is English beside a command name and only a reader judges it — the same limit gotcha 14 recorded for the old hint test.

+

#The browser half

+

RenderModel carries help, help_page and help_titles, and an empty help is what says the overlay is closed — the page asks no question about the mode, the same shape col_groups uses for "no groups". web/app.js paints rows it does not author; src/ui/help.rs keeps its ratatui styles and lost ~310 lines of content that is now shared. Neither end restates what the keys are.

+
+
+ + diff --git a/site/docs/code/tooling.html b/site/docs/code/tooling.html new file mode 100644 index 00000000..286620ec --- /dev/null +++ b/site/docs/code/tooling.html @@ -0,0 +1,79 @@ + + + + + +The repository's own checkers — improvise docs + + + +
+ +
+

Source: context/code/tooling.md

On this page

The repository's own checkers

+

scripts/ holds the gates this repo runs on itself. They are code, and they are held to the rule they exist to enforce: a check nobody has seen fail is indistinguishable from one that always passes, so each of them carries a self-test whose cases are injected faults (doctrine(check_the_checker)).

+

Four docstrings in scripts/check-comment-runs.py point here. This is the file they mean.

+
+

#1. The comment cap

+

check-comment-runs.py fails a .rs file carrying more than two consecutive comment lines. The rule is context/design-principles.md §6: reasoning lives in context/ prose, where it can be read, corrected and audited, not in a comment that nothing checks against the code.

+

The whole difficulty is deciding what is a comment line. Ten silent successes have been found in this one function, every one of them failing in the direction that looks like passing — a file that should have been flagged and was not:

+ +

Two rules that are decisions rather than accidents. A fence **suspends** the run rather than ending it, so two lines above a fence and two below is a run of four — a reader meets it as one uninterrupted block. And only a *runnable* fence earns the exemption: rustdoc executes a bare or rust and does not execute text, ignore or no_run, so the rest is prose wearing a fence.

+

#2. The evidence check

+

--check-evidence holds context/test-evidence/ and the tests it describes to each other, in two directions:

+ +

Neither direction is "every test has evidence". Most tests have none and should not be forced to have one; what is checkable is a claim, and both directions are claims. The failure they close is the same one a dead ticket ID is: a pointer that reads as a promise the thing exists.

+

#A leaf is not an identity

+

The names cargo lists are module paths, and the checker keeps them whole. Six leaf names collide in this workspace — two of them cited in evidence — so reducing a path to its last segment made a heading satisfiable by a test in another module answering another question. Renaming ui::project::tests::a_load_that_fails_is_not_a_rebase, the one its entry actually describes, left the checker at 579 entries, all name a test that runs, exit 0: the surviving ui::host::tests namesake answered for it.

+

So a bare leaf resolves only when it is unique, and an ambiguous one is a refusal naming the candidates — not a warning, because a warning re-creates the problem for anyone who does not read it. The heading is then written with its module path, which is also what makes the entry say which test it is about.

+

Note the two halves are both needed and neither is sufficient. The refusal forces qualification but cannot survive a rename on its own: rename one of a colliding pair and the leaf becomes unique, silently resolving to the survivor. Qualification is what closes that, and the refusal is what makes somebody write it.

+

What stays open is collision across binaries: --list prints module paths with no crate and no binary, so two crates sharing one could not be told apart (improvise-sxsf). Measured 2026-09-21 and still latent — 1444 names, zero duplicate full paths, and zero duplicate (crate, path) pairs.

+

The fix the ticket proposes — --list per package with the crate prefixed — was measured and is declined twice over. It costs 45.2s against 2.4s for the one workspace call (11 members, warm), 18× for an unreachable case, and --check-evidence needs the --ignored twin too, so the real figure is 22 invocations. More to the point it does not close the class it names: a package is not a binary. improvise alone carries five integration test binaries beside its lib, and an integration test's top-level fn lists with no module path at all — 60 of the 1444 names are bare — so -p improvise prefixes all six identically and tests/a.rs::foo still cannot be told from tests/b.rs::foo. A fix that ever becomes worth having is per-target (--lib, --test <name>), not per-package, and the ticket's proposal would have bought 18× for a partial cure.

+

#The names come from cargo, not from the source

+

A heading is diffed against cargo test --workspace -- --list, minus the names that --list --ignored reports. It used to be scraped out of .rs text, and four shapes defeated that, all of them "evidence for a check that never runs" (improvise-rm4s):

+
shapewhy the scrape accepted it
#[test] fn x inside a raw string literalthe scanner reads inside strings
#[cfg(feature = "…")] #[test]the feature is never enabled
#[cfg(target_arch = "wasm32")] #[test]cargo test --workspace is native
#[test] #[ignore]it compiles, and never runs
+

The third is the realistic one here: this repo has real wasm32-only code and a wasm CI job that only runs cargo check. Adopting the stricter source cost nothing — all 539 headings resolved against a real listing on the commit that made the change — which is the moment to tighten a gate rather than the moment to argue about it.

+

Demonstrated before the fix, by injecting all four shapes into src/draw.rs and context/test-evidence/tui.md at once: the previous checker printed 543 entries, all name a test that exists, exit 0. The current one names all four.

+

What this costs: --check-evidence now needs a toolchain and a compiled workspace, about three seconds warm. A missing cargo says so rather than raising. The comment-cap check is unaffected and needs neither.

+

--rust-tests FILE takes a saved listing for a caller that has one, and it requires --rust-tests-ignored FILE: a first version took one file, called cargo_test_names with one argument, and so skipped the ignored rule entirely while printing the same success line as the honest path. A flag that is strictly more permissive and indistinguishable in the log is a bypass, not a convenience — so the pair is mandatory and the success line names its source. CI passes neither and runs cargo itself.

+

A missing context/test-evidence/ is not an early return either. It used to be, and it exited 0 while 170 tests cited a directory that was not there; the walk now yields nothing and every citation is reported unstated, which is loud.

+

Note what the self-test can and cannot carry here. The listing reader is testable from fixtures — a leaf name taken off a module path, an --ignored name subtracted, cargo's own prose not mistaken for a test — and those cases are in SELF_TEST_LISTING. The four source shapes are not: once defined comes from cargo, a fixture asserting "a heading with an empty listing is an orphan" is the same case however the Rust beside it is spelled, and writing four of them would be four copies of one assertion dressed as coverage. The evidence for those is the injection above, against the real tree.

+

What it does not close: citing is still scraped, so a citation inside a string literal would be reported as a test owing an entry. That is a false positive in the safe direction — loud, not silent — and it has no instance.

+

#A demotion is a reviewer's job, not a gate's

+

A heading is classified by shape: a bare identifier is a Rust test, a quoted string a JS test, anything else is prose about the section (Helper — x, Note — y, a file's own scope). So any orphaned heading can be retired by rewriting it with a space and a dash, and nothing can check whether the prose is honest — no mechanism could, because the question is whether a sentence is true (improvise-9za9).

+

Two things make that tolerable. The unsafe direction is the one that fails loudly: prose that looks like an identifier is classified as a Rust claim and reported as an orphan, so the error you can make by accident is the noisy one. And the mechanism is git diff context/test-evidence/ — a demotion shows up there as a ### identifier line becoming prose, in the diff every reviewer already reads. The live instance, PR #69 demoting a heading while PR #72 restored its coverage under another name, is what says a reviewer rather than a gate has to catch it.

+

The printed count of prose headings is a convenience, not a guard: nothing stores the previous count and nobody diffs two CI logs, so a retirement moves a number no one is watching. Saying otherwise would claim a detectability it cannot support, which is the class of defect §7's adjacent-check section is about.

+

#A flag adds a question; it never selects one away

+

--check-evidence used to early-return, so the flagged run answered the evidence question and said nothing at all about the comment-run cap. Three agents hit that independently in one campaign, and in one case a local gate went green while CI went red on a three-line doc comment (improvise-ijql). It is now additive: main runs check_cap first, then check_evidence, and returns the worse of the two.

+

The ticket offered a cheaper shape — leave the early return and print one line saying the cap did not run — on the precedent of PR #83 making --rust-tests name its source. That precedent does not reach this case, and the distinction is worth keeping. A provenance line and a disclaimer are not the same object. Naming which listing the names came from qualifies an answer the run actually gave; saying "the cap did not run" qualifies an answer it did not give, while still exiting 0. Exit 0 is what CI branches on and what a human reads as clean, so a disclaimer leaves the gate exactly as skippable and merely documents the skip. The rule the fix is written to is therefore stated over the exit code: a tree with a long comment run must not pass any invocation that can exit 0.

+

Which invocations those are is a decision, not a sweep. --self-test and --list-doctests are excluded and neither is an exception to the rule: the first scans no tree, and the second is a listing that renders no verdict. Only the two that answer "is this tree clean" are in SELF_TEST_DISPATCH.

+

That table is checked by running the script as a subprocess against a temporary tree, which is the only level at which this bug is visible — every unit in this file was correct, and the defect lived in main's dispatch. Both arms are needed and both were demonstrated: restoring the early return fails DISPATCH: --check-evidence: exited 0 over a four-line comment run, and forcing the cap to always report a violation fails the clean arm for both invocations, which is what stops an always-fail from satisfying the first. The temporary tree also has no context/test-evidence/ and no JS roots, so the evidence half is vacuously clean there and the cap is the only thing that can decide the exit code — the isolation is what makes the assertion name one question.

+

#Two headings, one test

+

An orphan is a heading whose test nobody can find; the mirror image is a test that two headings answer for, and it is invisible to every rule above — each copy names a test that runs, so each copy passes. PR #92 shipped two entries under one heading where the older described a fixture the test no longer had and repeated a claim an adversarial review had measured false. A reader meeting the older one first has no way to know a successor exists two thousand lines down, so the stale copy reads as current. That is worse than an orphan, which at least announces itself as broken.

+

It is a refusal and not a warning, on the precedent improvise-rm4s set for the ambiguous leaf: the two are the same defect seen from opposite ends — one name answered by two tests, and one test answered by two names — so they are answered the same way. The refusal names the file and every line, in file order.

+

The key is the resolved test, not the spelling. ### f beside ### m::tests::f is two headings for one test, and grouping by the string would miss it precisely because qualification is the repair rm4s asks for; a heading qualified in one file and bare in another would have been the first way this came back. Resolution reuses the same leaf table the ambiguity rule uses, so a leaf that is genuinely ambiguous canonicalises to itself and is reported as ambiguous rather than as a duplicate — two different tests sharing a leaf are not two entries for one test, and conflating them would file the wrong repair. The check is over the whole directory rather than per file, because a heading keys a test name and a test name is unique workspace-wide once qualified.

+

What it still cannot see, and neither can any of the others: an entry that exists, names a live test, is the only one that does — and describes it backwards.

+

#3. The others

+

scripts/kb is the only writer of prolog-sessions/kb.pl; its own reasoning is in context/code/kb-tool.md. scripts/node-test.sh, scripts/kb-test.sh, scripts/pr-watch-test.sh and scripts/assemble-stamp-test.sh are the same shape as this file's subject: a guard, plus a suite that makes the guard refuse what it claims to refuse. Each is wired into .github/workflows/ci.yml in the same job as the thing it guards, because a check that only runs locally is a convention.

+

scripts/check-ci-graph.py is the workflow's own gate: pages-deploy must needs every other job in ci.yml. The rule "every check job" was written down (improvise-x7cq) and a job added during review of PR #83 was still ungated with nothing noticing, which is what moved it from a sentence to a check. It reads the job keys and the needs: list off indentation, tracking which job a needs: belongs to — taking the first one seen would read another job's list as the deploy's and pass. README.md therefore states the rule and deliberately does not list the jobs: a second copy with nothing holding it to the first is what went stale, twice.

+

#Why the Pages deploy fails on "Found 0 artifact(s)" (improvise-864a)

+

actions/deploy-pages@v5 intermittently fails a push to main with No artifacts named "github-pages" were found for this workflow run, while the upload step was green in that same run. The ticket framed this as concurrent merges racing on one Pages environment, and PR #46 proposed skipping the deploy unless the commit is the tip of main. Both the framing and the guard are wrong, and measurement says so.

+

What was measured. All 100 push-to-main CI runs from 2026-09-19T22:02Z to 2026-09-21T05:02Z, from the run, job, step and log APIs (job metadata and logs outlive artifacts, which expire in a day and are all 404 now — so this is reproducible where the ticket's own artifact-count table is not):

+
deployswith another run's SUCCESSFUL deploy completing inside this run's own upload→deploy window
12 artifact-not-found failures12 (100%)
56 successes2 (3.6%)
4 other failures0 — all four are the unrelated Create Pages site failed: Resource not accessible by integration, from before enablement: false (improvise-3gy)
+

The direction is settled, and it is neither of the two the ticket offered. The oldest run of the 17:47Z batch failed at 17:47:23Z — before the next deploy in the batch had started (17:47:31Z) and before any Pages deployment existed at all, since deploy-pages fetches the artifact metadata before it creates a deployment. So the failure cannot have been caused by this run's own deploy, by supersession, or by a concurrent deploy. The artifact was already gone. The reaper is another run's successful deployment.

+

It is scoped by artifact NAME, not by run. In run 35560726456 the build job finalized three artifacts seconds apart — harness-serve, wasm-bundles and github-pages, each with an id — and 2m21s later that run's own lookup reported Found 2 artifact(s) and no github-pages. Same run, same token, same call, same retention: the two siblings survived and only github-pages was missing. That one observation rules out the whole family of general explanations — service flakiness, eventual consistency, retention, a token scope, a v3/v4 backend mismatch — because any of them would have taken the siblings too.

+

Four things this kills. Queue depth is not the mechanism: the three deploys of the 17:47Z batch never queued, running 9s and 17s apart with each failure taking one second, so the pages group serialized nothing. Retention is not it either — all three shared retention-days: 1, and since the newest-finalized expires last, expiry can produce 0/1/1 or 0/0/1 but never the ticket's 0/1/0. Nothing in deploy-pages deletes an artifact (an upstream feature request asks for exactly that capability, which is only open because nothing does). And the action-version theory dies on the interleaving: successes and failures alternate minute by minute on identically-resolved action SHAs.

+

One clause of the ticket's own correction is falsified, and the conclusion still holds. The correction argued the tip-of-main guard would have skipped the one deploy that published "and let 52507a6 deploy and fail anyway" — which assumed 52507a6's artifact loss was independent of its neighbour. It was not: 5f871f7's deploy completed at 17:47:40Z, four seconds before 52507a6's began. Fewer successful deploys really is the right lever, so PR #46 aimed at the right quantity for the wrong reason. It should still not be shipped, because the association is not a mechanism and the mechanism is not deterministic: 5f871f7 had a foreign deploy land inside its own window too and survived. A guard measured at nothing, against a probabilistic effect, is doctrine(measure_the_obvious_fix_before_reporting_it_as_the_fix) exactly.

+

What the evidence does license. The risk lives entirely in the upload→deploy window — every victim had a foreign deploy land inside its own — so shortening that window is the one lever with evidence behind it, and consolidating the build already did some of that. Closing it properly means no two runs holding a github-pages artifact across a deployment, which is a real change to the job graph with real costs and is not attempted here. A retry is not a fix: the artifact is gone rather than transiently unlistable, so a retry cannot find it.

+
+
+ + diff --git a/site/docs/code/tui.html b/site/docs/code/tui.html new file mode 100644 index 00000000..b12071b8 --- /dev/null +++ b/site/docs/code/tui.html @@ -0,0 +1,93 @@ + + + + + +The TUI — improvise docs + + + +
+ +
+

Source: context/code/tui.md

On this page

The terminal front end (src/)

+

The root crate is the ratatui binary: src/draw.rs (event loop and frame composition), src/ui/*.rs (the widgets), src/main.rs (clap). It is one of two front ends, and design-principles §5 is the rule it is judged by — it paints and wires, and every decision with a silently wrong answer is made in improvise-protocol so the browser cannot answer it differently.

+

Test narratives live in context/test-evidence/tui.md. Facts about GridCache, AxisFrame, Projection and the Command/Effect layer belong to context/code/protocol.md and context/code/command.md; this file cites them and does not restate them.

+
+

#src/ui/grid.rs — GridWidget

+

The grid is drawn from a GridCache (improvise-n10).

+

No &Model. Every value the grid paints — each cell's text, whether it is an aggregate, the column widths, the page bar and the totals strip — arrives already decided, on the cache. A widget that cannot reach the model cannot quietly answer a question differently from the way the browser client answers it, and that divergence is the whole thing projecting exists to prevent.

+

The frame comes off the cache too (improvise-anr9). GridCache carries an AxisFrame per axis — the header levels, the label tuples and the groups with their collapse state — so the multi-level headers, the repeat suppression, the group spans and the pieces below are all read rather than derived, and a browser painting the same frame paints the same thing. It used to be derived here from GridLayout + View::is_group_collapsed while the protocol carried GridLayout::row_label's slash-joined form — the rendering two_row_categories_produce_cross_product_labels asserts must not appear — so the browser drew Food/Alice.

+

Two things are still not on the cache and are passed separately, each a different kind of absence:

+ +

The cache is expected to cover the whole grid — offsets zero, every row and every column — so a position in it is an absolute position and the widget's scrolling arithmetic is unchanged.

+

#Inside render_grid

+

GridWidget::cursor is this client's cursor in the active view (improvise-8uz), passed in rather than read off View, which no longer carries per-client state.

+

Everything about the frame is the cache's answer (improvise-anr9): how many header levels, what each level's label is, which group a position sits in and whether that group is collapsed. Nothing in render_grid reads GridLayout or View, so a browser folding the same AxisFrame draws the same header.

+

The bulk forms are built once per frame, and that is not a micro-optimisation. header_labels(offset), groups() and shares_ancestry(sel_col) each make one pass. The per-position accessors (item, label, shows_level, group_of, same_prefix) scan the frame from the start, so asking them inside the draw loops is quadratic in the grid and, at a deep scroll offset, grows with the offset. col_lines transposes header_labels to level-major because the column header is drawn one line per level while header_labels answers per column.

+

The underline on the column header marks columns sharing sel_col's ancestor group through each level: at the bottom level that is only sel_col, at higher levels it spans all the sub-columns.

+

Pending drill edits shadow the projected value. They are this session's uncommitted typing and by construction are not on the cache (gotcha 30).

+

Aggregate cells are drawn in italic to signal "drill to edit", and the fact is not re-derived here: the projection already named what each cell is, and CellId::Aggregate is exactly that fact (ADR 002), decided once by the side that holds the model. The widget used to walk none_cats × kind.is_regular() — a second hand-rolled copy of project::cell_id.

+

Whether there is a total row at all is the cache's answer. An empty totals strip means "no totals" (records mode, or an empty grid), which is why the widget does not read is_records_mode a second time.

+

The collapse glyph (indicator) is the one thing that is legitimately local: the fact — whether the group is collapsed — is on the frame; only its spelling is the terminal's (design-principles §5).

+
+

#src/draw.rs

+

#The key-event boundary

+

convert_key_event / convert_key_code / convert_modifiers are the only place crossterm's key types and the crate's own command::key types may both appear. Three rules:

+ +

#Semantic-to-terminal colour, and the one that is wrong

+

This file holds the mapping design-principles §5 names as the shape everything else should imitate: the decision lives in the shared layer as a semantic enum, and only the last translation is local.

+ +

mode_style(&AppMode) -> Style stays here for the same reason (Style and Color are ratatui types and improvise-command, where AppMode lives, carries no rendering-framework dependency), and since improvise-zx8t it is exhaustive: the fourteen variants that take the default chrome are spelled out rather than swept up by a _ => wildcard, so adding an AppMode fails the build here until someone chooses its colour.

+

It is still not quite the same shape as axis_fg and its two siblings, and the difference is deliberate rather than pending. Those three map a semantic enum the shared layer owns; mode_style matches AppMode itself, because there is no ModeColor. improvise-zx8t offered making one, and it was declined here for two reasons: the colour is the only thing at stake, and the ticket says to decide it together with improvise-k099, which asks the same question of hint_text/label and is not in this unit. Exhaustiveness is what the repo map's "the compiler names every one you must fill in" promise needed; a semantic enum is a separate, larger answer and is still open.

+

The guard is the compiler, not a test. Nothing asserts the colours, and a test enumerating the variants would not fail when a fifteenth is added — only the missing match arm does. Verified by deleting the AppMode::Help arm: error[E0004]: non-exhaustive patterns: &AppMode::Help not covered.

+

#The minibuffer shares the bottom row with the status line

+

draw_bottom_bar used to render the minibuffer instead of draw_status, so a status message written while any text-entry mode was open was never drawn (improvise-mai9, and the stranding gotcha 21 describes). minibuffer_line composes both: the prompt and buffer on the left, any pending status_msg right-aligned. The alternatives and why they lost are in context/test-evidence/tui.md.

+

#The caches are built here, per frame

+

project::formulas_of, category_tree_of, view_list_of, tile_bar_of, grid_of and wizard_of are all called from draw/draw_content against the authoritative App. This is the temporary half of improvise-35e: when the TUI becomes a client it will hold each cache and fold the matching Projection into it, and none of the widgets change either way.

+

Two consequences today:

+ +

draw renders overlays last so they appear on top. The which-key popup's armed prefix is stored as the key that opened it and resolved back to a sub-keymap by the keymap set (improvise-jc8, gotcha 13).

+
+

#src/ui/import_wizard_ui.rs — ImportWizardWidget

+

Drawn from a WizardCache (improvise-764).

+

No ImportWizard, no ImportPipeline, no Workbook. The widget used to hold the live wizard and read through it into improvise-io types and into the model — it called regular_category_names() and measure_item_names() on the built Workbook to write its one-line confirmation summary. Everything it draws now arrives already decided, from project::wizard_of.

+

The cache is not a Projection, and that is the ticket's answer to where wizard state lives: a half-open import dialogue belongs to the session that opened it, not to every connected client. WizardCache's own documentation carries the argument.

+

Smaller facts the drawing depends on: the root array path is already spelled (root) by the projection, so both ends say it the same way; the date components arrive in the order the wizard offers them, already paired with whether they are on, so the cursor index counts exactly those rows; and on Done the workbook is built but not yet installed, so the step shows what the user is about to accept, warnings included (improvise-907).

+

#set_bottom_line — one writer for the bottom row

+

That row has two writers — each step's key hint, and the wizard message that overrides it while present — and they write in that order to the same y. Buffer::set_string writes exactly as many cells as the string has and leaves the rest alone, so a message shorter than the hint it replaced left the hint's tail sitting beside it: something went wrongcel, the cel from Esc: cancel (improvise-q0vb).

+

Padding here rather than at each site is the point. The bug is not that one of the five writers forgot; it is that the row had five writers and no one of them owned clearing it.

+

The padding is measured in terminal cells, not characters. Buffer lays a string out by display width, so a char count over-pads a message carrying a wide character — enough to run past the panel and overwrite the border — and under-pads a combining mark, which is how a sliver of the hint would be left behind. src/ui/grid.rs measures the same way.

+

#truncate / prefix — bytes, deliberately

+

truncate cuts by byte length. prefix returns the longest prefix at most max bytes that ends on a char boundary, because every string truncate cuts can carry a field name, a formula or a message the user typed — and a multi-byte character straddling the cut used to panic the whole TUI on a slice at a non-boundary index. Walking down to the nearest boundary is the smallest change that cannot: for ASCII, which is what every fixture in this file holds, max is already a boundary and the result is byte-identical, so the characterization tests keep saying what they said.

+

It is bytes and not columns deliberately. Byte length is the measure truncate has always used, and changing it to display width would move where every line wraps — a different question from not crashing, and one that belongs with the hint-line padding it travels with. Bytes are never fewer than cells, so truncate cutting by bytes and set_bottom_line padding by cells cannot together overrun.

+

The blind spot, stated because the fixtures cannot state it: the eight exact-frame characterization fixtures in this file are entirely ASCII. They are therefore blind to every question about display width — a change to either measure is invisible to them, and the evidence for those has to be built directly (see set_bottom_line_clears_every_cell_it_covers_and_no_more).

+
+

#src/ui/help.rs

+

~600 lines of mostly-static content: five pages of styled (key, description) lines, a tab bar and a footer. improvise-ktv1 proposes moving that content into the shared layer so a browser help overlay is not a second copy of it. Until then it is TUI-only and help_page is already in ViewState, which is why gotcha 32 says the overlay needs no Projection.

+

#The panel widgets

+

src/ui/panel.rs holds the PanelContent trait and the generic Panel widget. The one non-obvious member is panel(): each impl declares its own identity, and the widget compares it against AppMode::panel() to decide whether to draw itself as active, instead of restating the mode match per panel.

+

category_panel.rs, formula_panel.rs, view_panel.rs and tile_bar.rs each draw one cache and nothing else. Two facts worth keeping:

+ +

tile_bar.rs scrolls by whole tiles: it finds the smallest start index at which the selected tile still fits in the available width, and draws ‹/› overflow indicators. That is the file's only piece of real arithmetic and it is local geometry, not a shared derivation.

+
+
+ + diff --git a/site/docs/design-notes.html b/site/docs/design-notes.html new file mode 100644 index 00000000..7f2442d1 --- /dev/null +++ b/site/docs/design-notes.html @@ -0,0 +1,30 @@ + + + + + +Design notes — improvise docs + + + +
+ +
+

Source: docs/design-notes.md

On this page

Design Notes

+

Staleness warning: This document captures conceptual framing from the original specification. It is not kept in sync with the code. For current architecture and types, see context/repo-map.md and context/design-principles.md.

+

#Product Vision

+

Traditional spreadsheets conflate data, formulas, and presentation into a single flat grid addressed by opaque cell references (A1, B7). This makes models fragile, hard to audit, and impossible to rearrange without rewriting formulas.

+

Improvise treats data as a multi-dimensional, semantically labeled structure -- separating data, computation, and views into independent layers. Formulas reference meaningful names, views can be rearranged instantly, and the same dataset can be explored from multiple perspectives simultaneously.

+

The application compiles to a single static binary and provides a rich TUI experience.

+

#Non-Goals (v1)

+ +
+
+ + diff --git a/site/docs/design-principles.html b/site/docs/design-principles.html new file mode 100644 index 00000000..f1fea959 --- /dev/null +++ b/site/docs/design-principles.html @@ -0,0 +1,193 @@ + + + + + +Design principles — improvise docs + + + +
+ +
+

Source: context/design-principles.md

On this page

Improvise Design Principles

+

#1. Functional-First Architecture

+

#Commands Are Pure, Effects Are Side-Effectful

+

Every user action flows through a two-phase pipeline:

+
    +
  1. Command (Cmd trait) — reads immutable context, returns a list of effects. The CmdContext is a read-only snapshot: model, layout, mode, cursor position. Commands never touch &mut App. All decision logic is pure.
  2. +
  3. Effect (Effect trait) — a small struct with an apply(&self, app: &mut App) method. Each effect is one discrete, debuggable state change. The app applies them in order.
  4. +
+

This separation means:

+ +

#Prefer Transformations to Mutation

+

Where possible, build new values rather than mutating in place:

+ +

#Compose Small Pieces

+

Commands compose via Binding::Sequence — a keymap entry can chain multiple commands, each contributing effects independently. The o key (add row + begin editing) is two commands composed at the binding level, not a monolithic handler.

+

#Decompose rather than early return

+

Early Returns usually are a signal of mixed responsibilities: if an early return would clarify a function, consider how the function could be decomposed for the same effect without the early return.

+
+

#2. Polymorphism Over Conditionals

+

#Dispatch Through Traits and Registries, Not Match Blocks

+ +

#Use Enums to Make Invalid States Unrepresentable

+ +

#When You Add a Variant, the Compiler Finds Every Call Site

+

Prefer exhaustive match over if let or _ => wildcards. When a new Axis variant or AppMode is added, non-exhaustive matches produce compile errors that guide you to every place that needs updating.

+
+

#3. Correctness by Construction

+

#Canonical Forms Prevent Equivalence Bugs

+

CellKey::new() sorts coordinates by category name. Two keys that name the same intersection but in different order are identical after construction. Equality, hashing, and storage all work correctly without callers needing to remember to sort. Property tests verify this invariant.

+

#Smart Constructors Enforce Invariants

+ +

#Type-Safe Identifiers

+

CategoryId and ItemId are typed aliases. While they are usize underneath, using named types signals intent and prevents accidentally passing an item count where an item ID is expected.

+

#Symbol Interning for Data Integrity

+

DataStore interns category and item names into Symbol values (small copyable handles). This means:

+ +

#Parse-Time Validation

+

Formulas are parsed into a typed AST (Expr enum) at entry time. If the syntax is invalid, the user gets an error immediately. The evaluator only sees well-formed trees — it does not need to handle malformed input.

+

#Grammar-Defined File Format

+

The .improv file format is defined by a PEG grammar (persistence/improv.pest) and parsed by pest. The grammar is the single source of truth — the parser is a tree-walker over the grammar's parse tree, not an ad-hoc line scanner. This means:

+ +

#CL-Style Pipe Quoting for Names

+

Names in the .improv format use CL-style |...| pipe quoting. A name is bare if it matches [A-Za-z_][A-Za-z0-9_-]*; everything else must be pipe-quoted. Escapes inside pipes: \| (literal pipe), \\ (backslash), \n (newline). This convention is shared between the .improv persistence format and the formula parser's identifier syntax.

+

#Formula Identifiers and Quoting

+

Bare identifiers are (ASCII_ALPHA | "_") ~ (ASCII_ALPHANUMERIC | "_" | "-")* — no spaces, the same shape improv.pest's bare_name has. A multi-word name such as Total Revenue must be pipe-quoted.

+

Keywords need a word boundary after them, not before. IS, WHERE and the aggregate names are spelled ^"…" in formula.pest, and is_expr/where_clause/ inline_where are compound-atomic with sp+ so that Region ISLAND and SUM(Rev WHEREAS x = y) are parse errors rather than silently-wrong filters. A SUM/IF without parens falls through to ref_expr, which is why a bare keyword still names an item; the writer quotes it anyway (quote::quote_name).

+

Pipe-quoted identifiers (|...|) allow any characters — including spaces, keywords, and operators — inside the delimiters. Use pipes when a category or item name collides with a keyword or contains special characters:

+
|WHERE|             — category named "WHERE"
+|Revenue (USD)|     — name with parens
+|Cost + Tax|        — name with operator chars
+SUM(|Net Revenue| WHERE |Region Name| = |East Coast|)
+

identifier is pipe_quoted | bare_ident, so a pipe-quoted name works everywhere a bare one does: expressions, aggregate arguments, and a WHERE clause's category name. A double-quoted string is a filter_value only — the right-hand side of a WHERE or an IS, never an expression.

+
+

#4. Separation of Concerns

+

#Four Layers

+
LayerDirectoryResponsibility
Modelsrc/model/Categories, items, groups, cell data, formulas. Pure data, no rendering.
Viewsrc/view/Axis assignments, page selection, hidden items, layout computation. Derived from model.
Command / Effectsrc/command/, src/ui/effect.rsIntent (commands) and state mutation (effects). Bridges user input to model changes.
Renderingsrc/draw.rs, src/ui/Terminal drawing. Reads model + view, writes pixels. No mutation.
+

#Formulas Are Data, Not Code

+

A formula is a serializable struct: raw text, target name, category, AST, optional filter. It is stored in the model alongside regular data. The evaluator walks the AST at read time. Formulas never become closures or runtime-generated code.

+

#Formula Evaluation Is Fixed-Point

+

recompute_formulas(none_cats) iterates formula evaluation until values stabilize. Each pass evaluates all formula cells using the current cache (for formula-derived values) and raw data aggregation (for data values). This avoids recursive evaluation through evaluate_aggregated and naturally handles chained formulas (Margin = Profit / Revenue where Profit = Revenue - Cost). Circular references converge to CellValue::Error("circular") after MAX_EVAL_DEPTH iterations.

+

#Display Rounding Is View-Only

+

Number formatting (format_f64) rounds for display. Formula evaluation always operates on full f64 precision. The rounding function is only called in rendering paths — never in eval_formula or aggregation.

+

#Drill State Isolates Edits

+

When editing aggregated (drill-down) cells, a DrillState snapshot freezes the current cell set. Pending edits accumulate in a staging map. On commit, ApplyAndClearDrill writes them all atomically. On cancel, the snapshot is discarded. No partial writes reach the model.

+
+

#5. A Front End Displays State; It Does Not Reimplement Behaviour

+

The goal is to minimise the amount of new code a new UI framework needs. Logic is centralised in a redux-like store, so a second or third front end is mostly a function from state to pixels plus a wire for events — not a second implementation of how the application behaves.

+

The pieces already exist and are worth naming as what they are:

+
reduxhere
actionCommand / ModelCommand (improvise-protocol)
reducerreduce_view, and WidgetCache::apply per widget
storeViewState + the per-widget caches
dispatchSessions::apply_checked — the one door (gotcha 36)
+

So the question to ask of any code in a front end is: would a third front end have to write this again? If yes, it is behaviour and it belongs in the shared layer. Three things make this concrete rather than aspirational:

+

The measure is lines of framework-specific code, and it should trend to "paint and wire". web/app.js is the current benchmark: a few hundred lines that know no protocol, reaching the application through a handful of string calls. A front end that needs to understand CellKey coordinates, or which category a row label belongs to, or what a keypress means, is one that has been handed a job the store should have done.

+

A derivation done twice is the defect, even when both copies are correct today. Two front ends deriving the same fact will diverge, and the divergence is invisible until someone compares them side by side. improvise-edp is the worked example — the category tree was flattened twice, two different ways, and the fix put viewmodel::flatten_cat_tree where both ends call it. Gotcha 44 was the same defect and is now closed the same way (improvise-anr9): the grid's frame — header levels, label tuples, group runs and their collapse state — is improvise_protocol::AxisFrame, and both src/ui/grid.rs and improvise-wasm-client/src/render.rs ask it rather than deriving one. Note what the second cure cost that the first did not: a Vec<String> of labels had to change type, and a shared derivation could not be added beside the old one — row_levels next to row_labels would have been two answers to "what are the labels", which is the disease and not the cure.

+

Sharing the table beats testing two copies of it. Where a front end must turn input into actions, share the mapping rather than mirroring it: two hand-kept copies held in step by a test are one definition waiting to be made (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)). That is why the keymap is shared rather than reimplemented per front end, and why doctrine(mirror_the_cmd_line_for_line) is a rule about resolvers — the narrow, unavoidable residue — and never a licence to restate a table.

+

What legitimately stays per-framework. Painting, event wiring, and the mapping from a semantic value to a framework type: AxisColor → ratatui::Color in draw.rs::axis_fg, MinibufColor → minibuf_fg. That mapping is the shape to imitate — the decision is in the shared layer as a semantic enum, and only the last translation is local. Anything larger than a lookup table is a candidate for the store.

+
+

#6. Guidelines for New Code

+ +
+

#7. Testing

+

#Coverage and ambition

+

Aim for ~80% line and branch coverage on logic code. This is a quality floor — go higher where the code is tricky or load-bearing, but don't pad coverage by testing trivial getters or chasing 100% on rendering widgets. The test suite should remain fast, and the rule is two numbers because the ambiguity between them is what let the old single "under 2 seconds" drift: no single test binary may exceed 2 seconds, and the workspace wall clock is the sum of them. Excludes compilation, and excludes harness/, which is Playwright and takes minutes in its own CI job. Slow tests erode the habit of running them.

+

The threshold is a tripwire, and the seconds are not portable. The same commit measured improvise-io's lib binary at 1.4s on one machine and 3.7–5.2s on another (improvise-8ec3), so a PR body citing "longest single binary 1.47s" is checkable and false for the reader who checks it, which is worse than no claim at all. Three consequences:

+ +

improvise-8shu is closed, and its diagnosis was wrong — which is the more useful half. The ticket blamed an unbounded grammar walk and asked for a constrained strategy. Instrumenting the two halves separately: load_grammar() ×1000 took 2.50s against a 15.7ms walk, so the walk was 0.6% of the cost and improv_file() was re-parsing improv.pest once per generated case. Caching it in a OnceLock took improvise-io's lib from 1.42s to 0.21s with the strategy's input space unchanged; constraining it would have moved 15.7ms and shipped as "the fix". Measured at f152d10 on the orchestrator's machine: 1445 tests over 34 binaries, longest single binary 0.6–1.0s across two runs — the spread on one machine in one sitting being its own argument for reading your own run. improvise-io is no longer the outlier. Lowering the case count below 256 is still not on the table (see below).

+

#Demonstrate bugs before fixing them

+

Write a test that fails on the current code before writing the fix. Prefer a small unit test targeting the broken function over an end-to-end test. After the fix, the test stays as a regression guard.

+

Record the bug in context/test-evidence/, keyed by test name — what broke, what fault was injected, and why the test can fail. It does not go in the test's doc-comment: comment runs are capped at two lines (see below), and the evidence is longer than that in every case worth having. The cost of the split is that a renamed test orphans its entry. scripts/check-comment-runs.py --check-evidence catches that, in two directions. Every ###/#### heading shaped like a test name must name a test that runs — the names are diffed against cargo test --workspace -- --list minus its --ignored twin, never scraped out of the source, because text that looks like a test and a test are indistinguishable to a scanner (improvise-rm4s). And a test whose doc comment cites context/test-evidence/ must have an entry there, or the pointer reads as a promise the explanation exists.

+

Neither direction is "every test has evidence", deliberately: most tests have none and should not be forced to have one. The reasoning and the residual holes are in context/code/tooling.md.

+

#A check that looks like the answer, answering a neighbouring question

+

The characteristic defect in this repo is not a wrong line; it is a check that is adjacent to the thing it names. Three instances landed in one campaign (PRs #59/#60), in three different layers, which is why it is worth stating as a shape rather than three gotchas:

+ +

Each has the same remedy shape: name the question you actually mean, then find an input that separates it from its neighbour.

+ +

This generalises doctrine(fold_tests_must_test_order_not_the_axiom) — asserting fold(prefix)+fold(rest) == fold(whole) is the definition of a left fold, so it answers a neighbouring question too. What the campaign adds is that the pattern recurs in test repairs and in merge hygiene, not only in production code and not only in fold tests.

+

#E2E pins the boundaries; unit tests work between them

+

E2E and integration tests are rare and pin boundary contracts. Unit tests assume those contracts and test between the boundaries (user, 2026-09-20).

+

So the question to ask of a slow test is never "what feature does this cover" — it is "which boundary is this, and what is the contract at it?" The answer changes what you write. Four browser widgets had never worked — the tile bar, the view panel, the formula panel, and the category sidebar ignoring category_panel_open — and all four were one boundary failure: a RenderModel field with no element painting it. Four per-widget Playwright specs cost four slow tests and would still have missed the fifth; a single contract assertion that every field the render model exposes is painted catches all four and the next one. harness/tests/opfs.spec.js is the cleanest instance of the right shape already in the tree: it runs the storage contract, not a feature.

+

Three corollaries, in the order they bite:

+ +

#Use property tests judiciously

+

Property tests (proptest) are for invariants that must hold across all inputs — not a replacement for example-based tests. Good candidates:

+ +

Keep case counts at the default (256). Don't crank them to thousands — if a property needs more cases to feel safe, constrain the input space with better strategies rather than brute-forcing. Property tests that take hundreds of milliseconds each are a sign something is wrong.

+

#What to test

+ +

#What not to test

+ +

#Test the property, not the implementation

+

A test like "calling set_axis(cat, Row) sets the internal map entry to Row" is brittle — it mirrors the implementation and breaks if the storage changes. Instead test the observable contract: "after set_axis(cat, Row), axis_of(cat) returns Row and categories_on(Row) includes cat." This style survives refactoring and catches real bugs.

+
+
+ + diff --git a/site/docs/doc-site.html b/site/docs/doc-site.html new file mode 100644 index 00000000..75dd3fcd --- /dev/null +++ b/site/docs/doc-site.html @@ -0,0 +1,69 @@ + + + + + +About this site — improvise docs + + + +
+ +
+

Source: context/code/doc-site.md

On this page

The documentation site — the arguments

+

scripts/build-doc-site.mjs turns the prose in context/ into the static site served at /docs/ on GitHub Pages. This is the reasoning behind what it publishes, what it deliberately does not, and why it is a hundred-odd lines of node rather than a static site generator (improvise-djoo).

+
+

#1. What is published, and what is not

+

The manifest is the SECTIONS array at the top of the script, and a document is published only by being named there. That is the point: a glob over context/** would publish the next working file somebody drops in, and the decision about what a stranger should read is not one to make by accident.

+

Published, in four sections:

+
SectionContentsWhy
Start heredocs/design-notes.md, context/design-principles.md, context/repo-map.mdWhat the project is for, the doctrine, and the single best explanation of how it is built.
Decision recordscontext/adr/*.mdAn ADR is written for an outside reader by construction — it states the options that were rejected, which is the part that is worth reading and the part that is nowhere else.
The arguments, by areacontext/code/*.mdThis is the reasoning that used to live in code comments and was moved out under the two-line cap (design-principles §6). It is the most valuable prose in the repository and it is addressed to exactly the reader a documentation site has.
Test evidencecontext/test-evidence/*.md501 entries, one per test whose narrative is longer than its code.
+

Not published:

+ +

None of the three is referenced by any published document except as a row in repo-map.md's file inventory, which is inside a code fence, so leaving them out creates no dangling link. That was checked rather than assumed.

+

And "not published" is now derived rather than assumed (improvise-res9). The generator walks context/**/*.md plus docs/design-notes.md, subtracts the manifest, and fails unless the remainder is exactly the OMITTED map in the script — which carries the reason beside the name, so this section and the code cannot drift into disagreeing. An allowlist and not a count, because an omission is a decision that should have to be argued in code; and the map is checked in both directions, because an entry naming a file nobody can find asserts a reason for nothing, which is the failure a dead ticket id is.

+

The check was latent when it was filed and live by the time it was written: context/code/kb-tool.md had been added after this table was drafted and was publishing nothing, with no argument for its absence anywhere. The table above says context/code/*.md is published; the manifest disagreed and nothing could see it. That is the whole failure mode in one file.

+

Test evidence is the one judgement call worth stating. The argument for leaving it out is that 501 entries about individual tests are noise to a reader who wanted to know what a pivot table is. The argument for keeping it is stronger: every one of the seven context/code/*.md documents points at its evidence file by name, so omitting it would break the published prose in exactly the way this site is otherwise careful not to; and the evidence file is the most distinctive artefact the repository has — a written record of the injected fault behind each load-bearing test is not a thing most projects can show. It is therefore published as its own section, last, described as a contributor reference rather than a tour, so nobody wanders into it expecting a narrative.

+

#2. Ticket ids resolve to a page, never to a URL that does not exist

+

The prose cites bead ids roughly 190 times — this paragraph deliberately spells none, because a document that gives one as an example makes itself a source of audit noise, and a placeholder is worse than a real id: it is ID-shaped and resolves to nothing. On a public site those citations are meaningless strings, and the obvious repair — link them to an issue tracker — is unavailable: beads keeps its issues in a Dolt database on a git ref, and there is no public issue URL to point at. Inventing one would produce 190 links that 404, which is worse than plain text because a link is a promise.

+

So the ids resolve inward. tickets.html is generated from .beads/issues.jsonl — already committed, already public — and carries id, status and title, nothing else; descriptions are internal working notes and are not the site's business. Each citation in the prose becomes a link to that page's anchor, with the title in a title= attribute so a hover answers the question without a navigation.

+

issues.jsonl is a passive export and lags the database, which is not a defect to work around but a fact to render honestly: an id the export does not carry is emitted as plain text with a dotted underline and no link, and the ticket page lists it as "not exported". The site therefore never links to an anchor it did not generate, whatever state the export is in. A generator that instead linked every id it saw would produce a page that is right today and broken the first time somebody cites a fresh ticket.

+

The page publishes every cited id, open bugs and their titles included, and that is a decision rather than a default (user ruling). The data is already public in .beads/issues.jsonl in a repository anyone can clone; the page only makes it legible. An honest open-bug list is a normal thing for an open-source project to have, and filtering one while the page's own lede links the raw JSONL would be theatre. Note the second-order reason to keep it unfiltered: a citation whose row was suppressed would be a link into a page that does not answer it, which is the failure this whole section exists to avoid.

+

Titles are tracker data and are escaped as attribute values. Five shipped bead titles contain a double quote; interpolated into title="…" by an esc that handled only &, < and >, they broke out of the attribute and the browser parsed the remains as three junk attributes. esc now escapes " and ' as well. Today's corpus being friendly is not a reason to interpolate untrusted-shaped text unescaped — the tracker is written by whoever files a bug.

+ +

A code span whose entire content is a repo path becomes a link — to the generated page when the path is one of the published documents, to GitHub otherwise. Two constraints make that safe. The whole span must be the path, so a sentence that merely mentions a directory is untouched; and the path must exist on disk at generation time, so a citation that has gone stale degrades to plain text rather than to a 404. The GitHub ref is the cache-bust token web/assemble.sh already receives, used only when it is a 40-hex sha and falling back to main otherwise — a local dev build must not emit blob/dev/….

+ +

The generator collects every anchor it emits and every internal href it writes, resolves them against each other, and exits non-zero if any href names a page it did not generate or a fragment that page does not carry. web/assemble.sh runs under set -eu, so a broken cross-reference fails the assembly rather than shipping.

+

This is not ceremony. It caught two real defects on its first two runs: a fence-close regex that compared a three-character marker against a one-character string, which swallowed every document from its first code block onward; and a list parser that ended the gotcha list at the first blank line, so 47 of the 48 gotcha anchors were never emitted while the "jump to gotcha N" strip linked to all of them. Both were invisible in a rendered page that otherwise looked fine.

+

It is proved by injection rather than asserted: renaming the emitted anchor ids makes it report 48 missing anchors, and pointing the ticket links at a page name that is never generated makes it report 763 missing pages — two different diagnoses, because "the page is absent" and "the page is there but the anchor is not" call for different repairs.

+

Links that leave the docs/ subtree — ../index.html, ../app/ — name files this script does not generate, so they are resolved against the filesystem instead. That is the whole reason assemble.sh runs the generator last: its siblings have to exist before they can be checked. An earlier version skipped those 48 links while still counting them as checked, behind a comment promising they were "checked below" when there was no below.

+

That ordering is itself checked now (improvise-res9). The generator reads web/assemble.sh and refuses to run unless its own invocation is that script's last command. PR #63 appended a loop and it happened to land above the generator; one landing below would have made the outward-link check pass against a half-built tree, silently and in the one direction this section says must not go unnoticed.

+

And the link check is not sufficient, which is the more important lesson. It answers one question — does every href resolve — and a defect outside that question is invisible to it however carefully it is run. The worked example is a nested <a>: a TOC entry wrapping a heading whose rendered text already contained a link produced <a href="#anchor"><a href="github…">…</a></a>, and the HTML parser closes the outer anchor early, so the entry became an empty unclickable stub and clicking it navigated to GitHub. Both hrefs resolved perfectly. It shipped past the link check, past the whole-site link check, and past a screenshot pass, and was caught by a reviewer clicking a link. The repair is therefore two things, not one: the TOC keeps a tag-free variant of each heading, and a separate check asserts no <a> appears inside another <a> — proved by reverting the first half and watching the second report 66.

+

The general form: when a check misses a defect, add the check that could have seen it rather than only fixing the defect, and ask what question the existing check was answering.

+

#5. Why a node script and not a generator

+

The landing page is hand-written HTML with no build step and one pinned CDN script; the same posture applies here. A static site generator would add a dependency tree, a lockfile and a toolchain to a repository whose site is currently four files, in exchange for features this corpus does not use — the docs contain no front matter, no shortcodes and, as it happens, not a single markdown link. Every cross-reference in them is prose naming a path or an id, which is precisely why §2 and §3 are the interesting parts and Markdown rendering is the boring part.

+

So the renderer is a deliberately small subset covering exactly what the corpus contains: ATX headings, fenced code, pipe tables, blockquotes, ordered and unordered lists with indented continuations, thematic breaks, and inline code, strong, emphasis and links. Anything it does not recognise becomes paragraph text, so no input line is ever dropped — the failure mode is an ugly paragraph, never a missing one.

+

Two details in it are load-bearing and easy to undo. Code spans are replaced by placeholders before emphasis is matched, because this corpus routinely opens ** on one side of a code span and closes it on the other; matching emphasis per segment silently drops that bold, which is a fidelity loss with no visible symptom. And strong is matched before emphasis, with a body that may contain a single * pair, because [^*]+ cannot match **a *b* c** and the leftover * then pairs across the bold — which is how a single sentence in gotcha 33 came out as three italic paragraphs.

+

A third, added later and for the same reason: a table cell boundary is an unescaped pipe outside a code span, so the row is walked rather than split("|"). This corpus documents a file format whose own quoting syntax is |Time Period|, and a sentence containing one inside a table cell would have been shredded into extra columns — a row that still parses, so no column count could detect it. It was latent when fixed, which is the point: the trigger is one ordinary sentence away in a document whose subject is pipe quoting.

+

One deliberate divergence from GFM inside that rule: a \| inside a code span in a table cell keeps its backslash, where GFM would unescape it to a bare pipe. Splitting is suppressed inside code spans, so the escape is never needed there and never consumed. The corpus contains no instance; it is recorded as a decision rather than left to be rediscovered as a bug.

+

#6. Navigation

+

repo-map.md is one long document whose gotcha list is numbered 1 to 48, and splitting it per gotcha was rejected: the gotchas cross-reference each other constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader following those wants a scroll, not 48 page loads. It stays one page, with an id on every list item and a compact strip of numbered links under the heading. The ids come from the list item's own number, not from a counter, so an inserted gotcha renumbers the anchors exactly as it renumbers the prose that cites them.

+

The strip's contents are derived from the anchors actually emitted, never from a count. A hardcoded 48 failed in only one direction: deleting a gotcha left a link to a missing anchor, which the link check catches, while adding one produced an unlinked gotcha and a clean build. The list only ever grows, so the hardcoded form was silent in precisely the direction it would be used — the same shape as an exclusion list that over-matches, failing in the direction that looks like success.

+

Long unbreakable headings need overflow-wrap. The test-evidence pages are hundreds of snake_case test names as headings; at phone width seven of them overflowed the viewport by up to 92%. The diagnosis is worth keeping because the first guess was wrong: getBoundingClientRect reported every element inside the viewport, because the text overflows its box without widening it. Only scrollWidth per element names the culprit. Measure the property that actually overflowed rather than the one that usually does.

+

#7. The phone nav is a checkbox, and that is not arbitrary

+

At phone width the sidebar's 25 links stacked above the content, so every page opened on roughly 1500px of nav. The wall was not inherited: all of this CSS is new, so it shipped for the first time with the site, which is the test for whether a defect is pre-existing and the one most easily got wrong in the direction of declining work you introduced.

+

<details> is the obvious control and it is wrong here, for a reason no spec-level reading predicts. The sidebar layout has to force the disclosure open at wide widths, and a closed <details> hides its content through ::details-content — which display cannot override. Measured in Chromium, the desktop sidebar came out collapsed to a single link with its toggle also hidden: unusable, and in the half nobody would think to re-check after a phone-only change. A hidden checkbox plus a <label> puts the hiding in our media query, so forcing it open is ordinary cascade. Anyone reading the markup will see a checkbox where <details> obviously belongs; this paragraph is why it must not be tidied into one.

+

And the first probe of the fix could not see the difference. Counting nav links whose getBoundingClientRect().height > 0 reported 26 visible in both the collapsed and the expanded state, because Chrome reports a height for content-visibility-hidden children — so it would have passed the broken <details> build. checkVisibility({contentVisibilityAuto: true, visibilityProperty: true}) discriminates.

+

That is the same instrument failing twice: §6's overflow fix also measured clean by getBoundingClientRect while the page still scrolled. A measurement that cannot distinguish the two states is not a measurement — the repo's signature defect, a test that passes while the thing it names is broken, arriving through the instrument rather than through the assertion. Before trusting a browser measurement, injure the thing being measured and check the number moves.

+

And the fix's own probe had the same shape one level up. Counting visible nav links answers "does operating the toggle reveal the links" — correct and complete for a mouse — and is silent on whether any non-mouse route exists. The first version of the checkbox carried hidden, which is display:none, which removes it from the tab order and from the accessibility tree: the label is not focusable on its own, so at ≤900px the navigation could be clicked and reached no other way, and the accessibility snapshot contained neither the checkbox nor any of the 25 links. A visibility count cannot see that, however sharp the visibility instrument. The toggle is therefore visually hidden but focusable at narrow widths and display:none at wide ones, and the check is tab order, .focus() and the accessibility snapshot — measured at 390, 900, 901 and 1280, because the regression lives on one side of a media query.

+

So the rule from §6 has a second half: a measurement of the wrong property is not a measurement either, however precise it is. checkVisibility fixed the visibility instrument; it does not turn a visibility count into an operability check. Ask what the defect would do to a user, then pick the instrument that observes that.

+

A correction, and the risk the original argument missed. The commit that added content: "\2630 " / "" said Firefox ignores the alt-text form and falls back to reading the content, so the change was "an improvement in two engines and a no-op in the third". That mechanism is wrong — Firefox shipped the syntax in 133, and all three engines parse it (measured: Chrome 141, Firefox 142, WebKit 26 each compute the declaration intact). The conclusion held; the reason given for it did not, and the reason was the load-bearing part, because it was what made the change look like a one-way door. It is one because every engine shipped to supports the syntax — not because the unsupported path is harmless. That commit message is superseded by this paragraph (design-principles §6: an argument must not live only in a commit message).

+

The unsupported path is not harmless, and that is the risk neither the change nor its first review named. / alt in an engine that does not know it is a parse error for the whole content declaration, so the glyph would not lose its alt text — it would vanish. A silent visual regression, the opposite of the assumed failure mode. The glyph is therefore verified painted in all three engines by pixel diff against ::before { content: none }, with the text starting 20px further right in each.

+

That check also produced this file's fourth instance of the wrong-property trap, and the sharpest, because it produced a confident false positive: the first probe measured the label's width with and without ::before and reported the glyph missing in all three engines. The label is display: block, so its width comes from the container and cannot see inline content at all. Right question, wrong property, and this time the wrong instrument accused working code rather than clearing broken code.

+
+
+ + diff --git a/site/docs/index.html b/site/docs/index.html new file mode 100644 index 00000000..ff7687b7 --- /dev/null +++ b/site/docs/index.html @@ -0,0 +1,22 @@ + + + + + +Documentation — improvise docs + + + +
+ +
+

improvise — documentation

+

These are the working documents of the improvise repository, published as written. They are addressed to someone reading or changing the code: the arguments behind the architecture, the decisions that were made and the ones that were rejected, and the record of what each load-bearing test is guarding against.

+

Nothing here is a tutorial. For what the program does, see the project home or run the browser demo.

+

#Start here

What the project is for, and the principles the code is held to.

#Decision records

Decisions with consequences, including the options that were rejected.

#The arguments, by area

The reasoning that used to live in code comments. Written for a reader with the source open.

#Test evidence

For each test whose narrative is longer than its code: what broke, what fault was injected, and why the test can fail. Contributor reference, not a tour.

#About this site

What is published here, what is not, and why.

+

#Conventions

+

Ids like improvise-9c8c are tracker issues and link to the ticket index below. Paths like crates/improvise-core/src/model/types.rs link to the source on GitHub at main.

+
+
+ + diff --git a/site/docs/repo-map.html b/site/docs/repo-map.html new file mode 100644 index 00000000..d6593f51 --- /dev/null +++ b/site/docs/repo-map.html @@ -0,0 +1,448 @@ + + + + + +Repository map — improvise docs + + + +
+ +
+

Source: context/repo-map.md

On this page

Gotchas: 12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849

Repository Map (LLM Reference)

+

Terminal pivot-table app. Rust 2024, Ratatui TUI, command/effect architecture. Apache-2.0. Root binary+lib improvise; sub-crates under crates/:

+ +

The main crate (and improvise-command) re-export the sub-crates as crate::{model, view, workbook, format, formula, persistence, import} so consumer paths stay stable when crates shuffle. The main crate also re-exports improvise_command::command as crate::command and improvise_command::ui::{app, cat_tree, effect, viewport} under crate::ui. ui::app re-exports the protocol's AppMode, ViewState, ViewCursor, ViewFrame, Panel, MinibufferConfig, MinibufColor and HELP_PAGE_COUNT, and command::key re-exports the protocol's key types, so every pre-existing path still resolves.

+

Architectural intent lives in context/design-principles.md — read that for the "why". This doc is the "where".

+
+

#How to Find Things

+
I need to...Look in
Add a keybindingimprovise-protocol/src/keymap.rs → Keymaps::default_keymaps() (the table; the browser looks up the same bindings). Resolving a bound name into effects is improvise-command/src/command/keymap.rs.
Add a user commandcommand/cmd/<submodule>.rs, register in registry.rs
Add a state mutationui/effect.rs → implement Effect
Change formula evalmodel/types.rs → eval_formula / eval_expr
Change cell storage / lookupmodel/cell.rs → DataStore
Change category/item behaviormodel/category.rs → Category
Change view axis logicview/types.rs → View
Change an axis label, color or cycleview/axis.rs → impl Axis (colors resolve in draw.rs::axis_fg)
Ask "what data is visible" vs "what can the user page"View::filtering_cats() vs categories_on(Axis::Page)
Change grid layoutview/layout.rs → GridLayout
Change .improv formatpersistence/improv.pest + persistence/{format,parse}.rs
Change where documents are storedstorage.rs → Storage impls (opfs.rs for wasm)
Change number displayformat.rs → format_f64
Change CLI argsmain.rs (clap)
Change import logicimport/wizard.rs → ImportPipeline
Change how a gp-tree fit searchesgp-tree/src/fit.rs → FitConfig / fit
Change gp-tree CSV preprocessinggp-tree/src/data.rs → preprocess
Change how a name is quoted anywhereimprovise-formula/src/quote.rs — the one home, both halves
Change import-wizard keys/stepscommand/cmd/wizard.rs (keys) + import/wizard.rs (steps)
Change frame layoutdraw.rs → draw()
Change what the browser page drawsimprovise-wasm-client/src/render.rs → RenderModel
Change a browser↔server frameimprovise-protocol/src/frame.rs → ClientFrame/ServerFrame. The one pair; both ends pub use it and neither defines one (improvise-xks6)
Change the worker's message loopimprovise-worker-server/src/worker.rs → Worker::on_frame
Change the session loop (either host)improvise-command/src/ui/host.rs → Host::handle
Change how the browser opens or seeds a documentimprovise-worker-server/src/vfs.rs → open / documents
Change what a browser resolves without the hostimprovise-protocol/src/resolve.rs → resolve. Not wasm-client/src/keys.rs, which holds only BrowserKey (KeyboardEvent → improvise key) — this row named it until 2026-09-21 and that pointer is how a false claim reached context/ (improvise-e5pf)
Change the browser page itselfweb/app.js (thin), web/index.html, web/style.css
Host another documentimprovise-ws-server/src/registry.rs → Registry, ClientId
Change app state / modesui/app.rs → App, AppMode
Write a test for model logicmodel/types.rs → mod tests / formula_tests
Write a test for a commandcommand/cmd/<module>.rs → colocated mod tests; helpers in cmd/mod.rs::test_helpers
+
+

#Central Pattern: Cmd → Effect

+
keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
+                            (pure, read-only ctx)       (list of mutations)    (only mutation site)
+
pub trait Cmd: Debug + Send + Sync {
+    fn name(&self) -> &'static str;
+    fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
+}
+
+pub trait Effect: Debug {
+    fn apply(&self, app: &mut App);
+    fn kind(&self) -> EffectKind;              // NO default — the compiler makes you classify
+}
+
+// EffectKind::{ViewOnly, Model(ModelChange)}
+// ModelChange::{CellData, LayoutGeometry, WholeWorkbook}
+

CmdContext (see command/cmd/core.rs) holds immutable refs to Model, GridLayout, AppMode, cursor/offsets, buffers, yanked cell, key code, expanded cats, and panel/tile cursors.

+

Add a command: implement Cmd in a command/cmd/ submodule, register in registry.rs, bind in default_keymaps(). Simple wrappers go through the effect_cmd! macro in effect_cmds.rs.

+

Add an effect: implement Effect in ui/effect.rs, add a constructor fn if it helps composition.

+
+

#Key Types (skim; read the source for fields/methods)

+

Model (model/types.rs): categories: IndexMap<String, Category>, data: DataStore, formulas: Vec<Formula>, views: IndexMap<String, View>, active_view, measure_agg. MAX_CATEGORIES = 12 (regular only). Virtual categories _Index, _Dim, _Measure always exist. Use regular_category_names() for user-facing pickers.

+

CellKey (model/cell.rs): Vec<(cat, item)> — always sorted by category. Build with CellKey::new(coords) / with(cat, item) / without(cat). Never construct the inner Vec directly.

+

CellValue: Number(f64) | Text(String) | Error(String). Errors surface from formula eval (circular, div/0, etc.).

+

DataStore (model/cell.rs): interned symbols + secondary index (Symbol, Symbol) → {InternedKey}. Hot-path lookups go through matching_values / matching_cells.

+

Formula AST (improvise-formula/src/ast.rs):

+ +

View (view/types.rs): category_axes: IndexMap<cat, Axis>, page selections, hidden items, collapsed groups, number_format, prune_empty. It carries no cursor — selected/row_offset/col_offset moved to ViewState::cursors (improvise-8uz). Axis { Row, Column, Page, Filter, None }. cycle_axis rotates Row → Column → Page → None; Filter is outside the rotation (see gotcha 27).

+

GridLayout (view/layout.rs): pure function of Model + View. cell_key(r,c), cell_value(r,c), drill_records(r,c). Records mode auto-detects when _Index is on Row and _Dim is on Column.

+

AppMode (ui/app.rs): 17 variants (Normal, Editing, FormulaEdit, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, CategoryAdd, ItemAdd, ExportPrompt, SavePrompt, CommandMode, ImportWizard, Help, Quit, RecordsNormal, RecordsEditing). Eight text-entry variants carry a MinibufferConfig. SearchMode is Normal + search_mode: bool, not its own variant. Per-variant UI facts live on the enum as exhaustive-match methods — label(), hint_text(), panel(), minibuffer(), is_editing(), is_records() (improvise-2hi); add a variant and the compiler names every one you must fill in. mode_style() stays in src/draw.rs because Style is a ratatui type. App state is split into ModelState (workbook, file_path, dirty) and ViewState (mode, cursor/offsets, buffers, panels, search) — see improvise-x2c/ew0. wizard and drill_state live on App, not ViewState (improvise-h0b) — see gotcha 24.

+

Keymap (improvise-protocol/src/keymap.rs; dispatch and KeymapSet in improvise-command/src/command/keymap.rs): Binding { Cmd | Prefix(Arc<Keymap>) | Sequence(Vec<…>) }. Lookup fallback: exact(key,mods) → Char(NONE) → AnyChar → Any → parent. 17 mode keymaps, one per ModeKey, built by Keymaps::default_keymaps(); mode resolved via ModeKey::from_app_mode(). ModeKey and AppMode are two different enumerations that happen to be the same size — SearchMode is a ModeKey and not an AppMode; Quit is an AppMode with no keymap — so do not "fix" one count to match the other.

+
+

#.improv File Format

+

Plain-text, markdown-like, defined by persistence/improv.pest. Parsed by pest — the grammar is the single source of truth (the grammar-walking test generator reads it via pest_meta).

+
v2025-04-09
+# Model Name
+Initial View: Default
+
+## View: Default
+Region: row
+Measure: column
+|Time Period|: page, Q1
+hidden: Region/Internal
+collapsed: |Time Period|/|2024|
+format: ,.2f
+
+## Formulas
+- Profit = Revenue - Cost               # defaults to [_Measure]
+- Tax = Revenue * 0.1 [CustomCat]
+
+## Category: Region
+- North, South, East, West              # bare items, comma-separated
+- Coastal_East[Coastal]                 # grouped item, one per line
+> Coastal                               # group definition
+
+## Data
+Region=East, Measure=Revenue = 1200
+Region=West, Measure=Revenue = |pending|   # pipe-quoted text
+ +
+

#Gotchas (read before editing)

+
    +
  1. Records mode is wide format. A row is one coordinate tuple — the cells sharing everything but _Measure — and the columns are [coordinates…, **+cat**, raw measures…, +]: a column per regular category, then the column that creates a new category (improvise-2zq, sitting where the column it creates will appear), then a column per raw measure, then the column that creates a new measure (improvise-c5v). Consequences: two measures on one coordinate are ONE row; editing a category column moves every measure cell in the row, not just the one under the cursor; _Dim in the synthetic key carries the column index, not a label, because a category and a measure may share a name — resolve it with GridLayout::record_col into a RecordCol. Formula targets are not columns (nothing to type into them). Drill pending edits are keyed by (row, RecordCol). prune_axis_items takes a predicate, not a flag vector. prune_empty hands it impl Fn(usize) -> bool over the data index (group headers consume no index). The domain is derived from items itself, so there is no parallel length a caller could get wrong — the earlier &[bool] + debug_assert pair could disagree between debug and release if a third caller ever sized keep from something other than row_count()/col_count() (improvise-n3wt). GridLayout::new does not prune; with_frozen_records does — and that split has now produced four bugs: view_frame, project::grid_in (PR #78), and persistence/export.rs::render_csv (improvise-iot6), which wrote rows and columns the grid does not draw for as long as CSV export has existed. Every remaining GridLayout::new call site in the workspace is test code — verified — so a new production caller almost certainly wants with_frozen_records(m, v, None).
  2. +
  3. Effects never touch std::fs. Every document read or write goes through App::storage: Box<dyn Storage> (improvise-fuj). The host picks the backend: App::new supplies PhysicalFs, App::with_storage takes any other — a browser worker will supply OPFS (improvise-i34), and tests use MemoryFs. improvise-io keeps path-taking wrappers (save, load, export_csv, parse_csv_with_warnings) for the native CLI, each delegating to a _to/_from/_bytes variant that takes the backend or the bytes.
  4. +
  5. Every effect declares an EffectKind, and there is no default. ViewOnly never reaches other clients; Model(CellData | LayoutGeometry | WholeWorkbook) does (improvise-mae). A new effect that forgets to classify itself fails to compile — that is deliberate, since the failure mode is silent: a mis-declared ViewOnly edit would simply never reach connected clients. Watch two traps: the View lives inside model_state.workbook, so "touches model_state" does NOT mean "is a model change" (axis/page/format are LayoutGeometry, cursor and scroll are ViewOnly — see improvise-8uz; and SwitchView, ViewBack and ViewForward moved from LayoutGeometry to ViewOnly with improvise-jytm, writing view_state and nothing else, which is what took the latter two off the wire — but a sound derivation is not a guard: only SwitchView had one, and review showed reclassifying either sibling back was silent across the whole suite); and Save/SaveAs/MarkDirty touch model_state for bookkeeping only and are ViewOnly. AddCategory/RemoveCategory are the third trap and go the other way: they look like data edits, but both go through Workbook, whose on_category_added/on_category_removed rewrite every view's axis map, so they are LayoutGeometry. The two "an effect declaring X actually changes X" tests cannot separate them — a category change moves model data and view config, so it clears either bar; structural_effects_declare_their_kind pins the declaration itself. A third trap, now guarded: EffectKind is too coarse to check a mapping. ui/reduce.rs::model_effects sends each of 28 ModelCommand variants to one effect, and every inverse pair — HideItem/ShowItem, AddItem/RemoveItem, CreateView/DeleteView, SetCell/ClearCell — declares the same kind, so swapping three of them at once left the suite at 918 passing while three wire messages did the exact inverse of what they said. The mapping is name-for-name by construction, so every_model_command_names_the_effect_of_the_same_name asserts that, reading the type name off the effect's derived Debug (Effect is a trait object, so type_name is out of reach). A variant whose effect must be named something else is a variant that stopped being derived — it should fail there and be argued about. Their reach is wider than the name suggests: Model::remove_category also drops every formula targeting the category, which is why Dirty::of(LayoutGeometry) reports formulas: true — see gotcha 35.
  6. +
  7. Commands never mutate. &CmdContext is read-only; return Vec<Box<dyn Effect>>. If you want to touch &mut App, add or extend an Effect.
  8. +
  9. CellKey is always sorted. Use the constructors; equality and hashing rely on canonical form.
  10. +
  11. No Model::add_item. With a Workbook in hand use the facade: wb.add_item("Region", "East")? / add_item_in_group / remove_item (improvise-tvt). With only a Model, go through the category: m.category_mut("Region").unwrap().add_item("East").
  12. +
  13. Virtual categories. _Measure items = explicit items ∪ formula targets. Use measure_item_names() / effective_item_names("_Measure"). add_formula does not add items to _Measure. Formula-target lookup also falls back into _Measure via Model::find_item_category. Cell aggregation goes through apply_agg, formula SUM(…)/AVG(…) through eval_agg — they differ on empty input (absent vs. A third rule joins them for non-numeric values: aggregate_cell_values reduces text by unanimity — one matching value, or several copies of the same value, is that value; differing values, or text among numbers, is CellValue::Error("mixed") (improvise-l4qx). The all-numeric branch is checked first, so Count of a lone number is still 1 and not the number — that ordering is the whole difference between this rule and "a single value passes through". What the rule forbids is dropping: filter_map(as_f64) used to, and since _Index/_Dim are on Axis::None in every default view that made every text cell in every document render blank, including the pipe-quoted text .improv carries as a first-class value. evaluate_aggregated and aggregate_raw share the helper, so a formula ref over mixed values now errors "mixed" rather than silently summing the numeric ones. 0 for Sum/Count, error for Avg/Min/Max). Model::evaluate_aggregated takes Aggregation::{Exact, OverHidden} rather than none_cats: &[String] (improvise-5jm5) — the list was only ever tested for emptiness, and the guard against that shape returning is the signature. recompute_formulas/aggregate_raw keep the list because they walk it.
  14. +
  15. Display rounding is view-only. format_f64 (half-away-from-zero) is only called in rendering. Formula eval uses full f64. Never feed a rounded value back into eval.
  16. +
  17. Formula eval is fixed-point. recompute_formulas(none_cats) iterates until stable, bounded by MAX_EVAL_DEPTH; circular refs converge to CellValue::Error("circular"). App::new runs it before the first frame so formula cells render on startup.
  18. +
  19. Drill into formula cells strips the _Measure coordinate when it names a formula target, so matching_cells returns the raw records that feed the formula instead of returning empty.
  20. +
  21. Minibuffer keymaps come from TextEntry. The eight text-entry modes are built by the TextEntry builder in improvise-protocol/src/keymap.rs — seven by TextEntry::new, and RecordsEditing by TextEntry::with_parent off Editing's map, which is why a count taken off the ::new call sites reads seven. It binds the shared spine (AnyChar→append-char <buf>, Backspace→pop-char <buf>, Esc→[clear-buffer <buf>, enter-mode <exit>]) and welds every commit key to its clear-buffer via .commit(key, cmd, args) (improvise-w3q). TextEntry::new takes the AppMode and reads <buf> off its MinibufferConfig, so the buffer the prompt names and the buffer the keys edit are the same string by construction (improvise-k8h) — the keymap never spells a buffer name. .rebind exists only for CommandMode's Backspace. SearchMode is hand-written on purpose — no commit key, and exit-search-mode owns the query. EnterMode clears the incoming mode's minibuffer through write_buffer too — building SetBuffer directly there was unreachable only because no shipped AppMode is keyed "search". MinibufferConfig::buffer_key is a String, not &'static str: a borrowed-'static field makes Deserialize unimplementable, and every text-entry AppMode variant carries a MinibufferConfig (improvise-a2x). A buffer named at runtime is read with cmd::core::read_buffer and written with its mirror write_buffer — "search" is not in the buffers map but its own ViewState field, and those two functions are the only code that knows it (improvise-gdl).
  22. +
  23. Panel keymaps come from PanelKeymap. The three panel modes share a spine (Esc/Tab→Normal, arrows and j/k→move-panel-cursor <panel>, every panel's toggle key, :→command mode) built by PanelKeymap::new(panel); the keys that act on the panel's contents arrive through .bind/.bind_args/.delete because they vary in shape, not just command name (improvise-4pg). Panel::ALL/name()/key() in ui/app.rs drive the toggle keys for Normal and all three panels, so a fourth panel reaches every keymap from one ALL entry. name() must spell what parse_panel accepts.
  24. +
  25. Prefix keys are stored as an address, not a keymap. ViewState::transient_prefix holds the (KeyCode, KeyModifiers) that opened a prefix; KeymapSet::prefix_keymap resolves it back to the sub-keymap, walking parents so RecordsNormal reaches Normal's g/y/Z (improvise-jc8) — that is the command-layer wrapper; the resolution itself is Keymaps::prefix_keymap in improvise-protocol/src/keymap.rs. ViewState must stay free of improvise-command types — it is destined for the protocol crate, which improvise-command will depend on. Resolution assumes prefixes are flat; prefix_keymaps_are_flat fails the build if a nested one is added.
  26. +
  27. Keybindings are per-mode. ModeKey::from_app_mode() resolves the mode; Normal + search_mode=true maps to SearchMode. Minibuffer modes bind Enter/Esc as Binding::Sequence so clear-buffer fires alongside commit/cancel. Hints are checked against the keymaps: every_hint_key_resolves_in_its_mode (command/keymap.rs) splits each AppMode::hint_text() and requires every key it names to resolve in that mode's map, because only_quit_has_no_hint asserts only that a hint is present. It checks which keys are named, not what they do. Its tokenizer knows the hint dialect explicitly (named keys, Ctrl-<c>, F/C/V alternatives, character runs like hjkl/[:], the em-dash prose tail, CommandMode's :q ex-commands) and panics on a token it cannot classify rather than skipping it.
  28. +
  29. Registry names come from prototypes, not strings. Every register* call reads the name off a prototype Cmd instance, so a typo is impossible (improvise-9cn) — there is no name literal in registry.rs. What types can't reach: four commands name themselves off a field (ViewNavigate, TileAxisOp, ToggleGroupAtCursor, CommitAndAdvance), so a prototype can name the wrong variant and still compile. That shows up as a duplicate registration, not a mismatch — the wrong-variant entry collides with its sibling and the sibling's name vanishes. registered_names_are_unique catches it; every_bound_command_name_is_registered catches the keymap side, and checks both halves of dispatch's registry.interactive(name, args, ctx).ok()? — the name resolves and the args parse — because improvise-s13 moved enter-mode ["export-prompt"]/["formula-edit"] out of the nullary class, where a typo'd mode name compiles into a dead key. It builds the CmdContext in each keymap's own AppMode, since commit-buffer item reads ctx.mode.
  30. +
  31. effect_cmd! macro wraps a pre-built effect as a parseable command. Use it for simple wrappers; reach for a hand-written Cmd once there's real decision logic. Its counterpart on the effect side is simple_effect! in ui/effect.rs, which declares a one-field newtype that writes one place on the App. Its kind argument is mandatory — simple_effect!(Name(T) => |app| place, EffectKind::ViewOnly) — even though every call site so far answers ViewOnly: hard-coding it reintroduced exactly the default gotcha 3 does without, since $place is an unconstrained expression and a one-line active_view_mut().number_format setter would have reported ViewOnly with the suite green (improvise-8fy). Likewise, mode entry has no per-mode shim commands — bind enter-mode <mode-name> (improvise-s13); the name is decoded by parse_mode_name in registry.rs, and EnterMode clears the mode's minibuffer on the way in.
  32. +
  33. Float equality inside formula eval has one epsilon, and the div/0 guard shares it. FLOAT_EQ_EPSILON = 1e-10 in model/types.rs is used by =/!= and by the division-by-zero guard (rv.abs() < FLOAT_EQ_EPSILON), in both eval_expr and its aggregated twin — six uses across the two evaluators (a div/0 guard plus the Eq and Ne arms in each), one constant. This gotcha previously recorded the opposite (=/!= epsilon, div/0 exact) and was stale; the mix is gone. Why an epsilon at all and why this one: context/code/core.md §4.
  34. +
  35. IndexMap preserves insertion order for categories, views, and items. Persistence relies on this — and so, since improvise-shyt, do DataStore::matching_cells/matching_values: the secondary index narrows the candidates, then matching_positions resolves each to its position in the primary IndexMap, sorts, and reads back. An index whose values are HashSets is for O(1) lookup, never for iteration. Walking it gave a different order in every process, which was not tidiness. Records-mode rows reordered between sessions; SUM/AVG drifted in the last bits because f64 addition is not associative (Min/Max/Count and the text-unanimity rule of gotcha 7 are order-free, so they were never affected); and worst, the drill snapshot froze its record list in hash order while drill_state.pending_edits are keyed by row index (gotcha 1), so the same drill opened in a second process addressed different records — silent misdirected writes, and nothing in the tracker knew. Measured at 18,000 cells: +20% worst case on matching_values, and matching_cells 7% faster, sorted positions resolving CellKeys with better locality. DataStore::live_records was the open choice and is now decided (improvise-2yc2): it walks cells and projects insertion order onto records, deliberately paying O(cells) rather than O(records), because cells' order is already the answer matching_positions gives and a second answer to "what order is this store in" is the disease. It is derived per call rather than memoised on RecordEntry, since a stored position is a copy and a record deleted and re-created would hold its old one. Note what is not on offer: an unordered return — every iterator has an order, and a type that merely disclaims one still hands the caller hash order, which is the guard living one level away from the hazard and the shape this whole class of bug returns through. The second real instance was import::analyzer::analyze_records (improvise-7yx), which built its distinct values through a HashSet: build_model adds items in exactly that order and IndexMap preserves it, so the same CSV imported twice produced two different .improv files and two different row orders. Now first-seen order, pinned by a five-iteration test because each HashSet gets its own RandomState. Also answered here since: an empty partial matches every cell vacuously (improvise-hkt6) — it used to index sets[0] and panic, unreachable only because both callers guarded.
  36. +
  37. Commit paths for cell edits split across commit_cell_value → commit_regular_cell_value / commit_plain_records_edit in command/cmd/commit.rs. Keep the synthetic-records branch in sync with the plain branch when changing behavior.
  38. +
  39. Minibuffer commits go through one command. commit-buffer <formula|category|item> (CommitBuffer { kind: CommitKind }) owns buffer→trim→bail-on-empty→dirty→status for all three; the varying parts are methods on CommitKind. The add effect is emitted last so its failure message wins over the command's optimistic status (improvise-6t3). CommitKind::Item carries its target category, resolved from AppMode::ItemAdd by parse_commit_kind at registry time — so commit-buffer item is not constructible from a script line.
  40. +
  41. An abort stops the chain, never the bookkeeping — and ordering is what separates them. Three effects set abort_effects, and they are not all saves: Save and SaveAs on a failed write or a missing path (improvise-8tk — SaveAs was the arm left out of the original fix, so :w /bad/path . q quit on a failed write), and AddCategory when Model::add_category refuses, because what follows assumes the category exists — from the +cat column (improvise-2zq) the rest of the batch would advance the cursor and re-open an editor on a column that was never created. A Binding::Sequence flattens into one effect vec, so the abort carries across sequence steps; that is a property of the flattening, and it is why improvise-0hb's proposal to apply effects between steps was rejected — apply_effects resets the flag per batch, so step 2 in a fresh batch would land the Quit after a failed Save, silently reopening a P1. a_sequence_whose_first_step_aborts_does_not_run_the_second is the guard. The rule the aborts taught, which is the part to carry: an abort at the tail of a batch also ate the mode restoration and the buffer clearing, stranding the user in CommandMode/Editing/SavePrompt — where draw_bottom_bar renders the minibuffer instead of the status line, so the error the effect had just written was never drawn and the operation appeared to do nothing at all. So tidy-up goes at the head, where an abort cannot reach it: ExecuteCommand fronts its batch with ReturnToNormalUnlessMoved, CommitAndAdvance with the mode change, CommitSave with the mode change (found by Copilot on PR #27). Ordering answers it without giving any effect a "runs anyway" flag, and it does not weaken gotcha 31's observed-not-predicted mode settling: whatever a later effect sets, it sets after, so the mode a : line ends in is unchanged in every case — the only difference is what happens when the batch never reaches the end. An abort is not a rollback; that is the hole clause 4 leaves (gotcha 47).
  42. +
  43. The import wizard's back path is a history stack, not a reverse table. advance pushes the step it leaves onto ImportWizard::history; previous pops it, so back always retraces the path actually taken through the conditional transitions (improvise-907). Preconditions live in validation_error(), checked on the step where the user can still fix them — never deferred to build_model. NameModel is terminal for advance: Enter there calls build(), which parks the workbook in built on Done, and AcceptWizardModel takes it from there. Going back from Done discards built.
  44. +
  45. app.layout goes stale mid-batch. apply_effects rebuilds only after the whole batch, so an effect that reads the layout after an earlier effect changed the grid needs a RebuildLayout effect in front of it — the command decides the order (see EnterEditAtCursorCmd, CommitAndAdvance). Read the cursor's cell text via App::display_value_at_cursor(); don't build a CmdContext inside apply (it recomputes column widths).
  46. +
  47. The cursor is per-client and keyed by view. ViewState::cursors: HashMap<String, ViewCursor> holds selected/row_offset/col_offset, keyed by view name; an absent key means the origin. Reach it through App::cursor() (a Copy snapshot), App::cursor_mut(), and App::reset_cursor() — never off View, which no longer has the fields (improvise-8uz). Two consequences. Changing the grid's shape must reset the cursor explicitly: View::transpose_axes/cycle_axis used to do it themselves, and now TransposeAxes/CycleAxis call app.reset_cursor() — a new geometry effect that renumbers rows or columns has to do the same. Replacing the document must clear the map: LoadModel and AcceptWizardModel call cursors.clear(), because the incoming workbook's view names have nothing to do with the outgoing one's. That clearing replaces Workbook::normalize_view_state, which is gone — a View can no longer carry a stale offset. The same clearing owes the session mirrors: install_workbook bumps App::document_generation, and ClientSession::project reseeds any mirror seeded at an earlier one (improvise-cqq). An effect cannot reach Sessions, so the generation is the mechanism and Sessions::rebase is only the client-facing half. The same is true of a geometry change, and it owes the mirrors both halves: App::cursor_epoch is bumped by reset_cursor and watched by Sessions::adopt_cursor_epoch, which resets every mirror and returns Broadcast::ToAll(Command::ResetCursor) (improvise-gvnb). Doing only the first is strictly worse than doing nothing — the diff then believes every client already holds origin-positioned content and sends nothing, so they stay stale permanently rather than until their next scroll. The trigger is the epoch moving, not the command variant, for the reason gotcha 39 watches document_generation instead of matching on LoadModel: which effects renumber the grid is a per-effect decision, so a table of variants would be a second description of it, free to disagree. What is broadcast is an invalidation, never a position — another client's transpose must not move where I am looking (that is UI and it is mine, improvise-g0f), only declare that where I was looking has stopped meaning anything. Since improvise-ni6k no effect calls reset_cursor at all — the cursor is invalidated by OBSERVING the grid's frame. apply_effects diffs the frame after every model effect, inside the loop, because a batch that reframes and then places the cursor loses the placement otherwise (four records-mode tests pin that position). There is no list of effects anywhere, so the mechanism is total over EffectKind::Model(_) by construction — a genuinely stronger shape than a hand-kept table, and the reason the old "a geometry effect that forgets reset_cursor is invisible in both directions" hazard is closed.

    +

    Two arms, and the asymmetry is the whole design (improvise-mcq4). LayoutGeometry hashes only the active view — it is the held-key path, and other views cannot reframe there since view_mut() writes only the active view, with the category and item effects already covered by view_shapes. CellData hashes every view, because a cell write can prune a row out of a view the host is not on; narrowing that arm to one view measured as free and silently lost a multi-client invalidation, which is gotcha 24's own strictly-worse-than-nothing case. Cost of the wide arm: 0% on the held-key path, +5.7% on an 18k two-view SetCell, linear in view count, accepted. State the absolute as well as the ratio: the observation roughly doubles held-key latency on large pruned documents (3.04 → 5.69 ms at 18k) and 56 ms at 90,000 cells is 3.4 frames.

    +

    Totality is conditional on classification, and nothing makes classification total: a ViewOnly effect that renumbers the grid is invisible to all of this (gotcha 3's hole, unchanged). And two known gaps are filed rather than fixed — a geometry effect immediately after a mid-batch SwitchView skips the check entirely, because there is no basis for the new view yet (improvise-9c9o; under-reporting, which gotcha 35 says is the expensive direction), and Model(WholeWorkbook) matches neither arm, which is benign only because all three such effects go through install_workbook → rebase → a fresh Snapshot.

  48. +
  49. ViewState and its field types are Serialize/Deserialize. It is the wire payload of the client Snapshot (improvise-q08), so AppMode, MinibufferConfig, MinibufColor, ViewFrame, KeyCode and KeyModifiers all derive both (improvise-a2x), and improvise-command depends on serde for it. The constraint this puts on new state: no borrowed-'static fields anywhere ViewState can reach — Deserialize cannot produce one, and the error surfaces on every enclosing variant rather than at the offending field. view_state_survives_a_json_round_trip guards it. ViewState and Snapshot also derive Clone and PartialEq, so the bootstrap message can be compared in a round-trip test.
  50. +
  51. Axis presentation lives on Axis, colors resolve in draw.rs. Four vocabularies hang off the enum in view/axis.rs — plain_label() ("Row"/"Col"/"Page"/"None"), glyph() (which Display appends, giving "Row ↕"), short_label() (three columns for the tile bar; None is "·", not a truncation), and initial() (one letter for the view panel; None is empty). Plus color() -> AxisColor and next() for the Row→Column→Page→None rotation. AxisColor is a semantic palette: improvise-core must not name a ratatui type, so draw.rs::axis_fg does the one mapping to a terminal color, exactly as minibuf_fg does for MinibufColor (improvise-rml). The tile bar and category panel each carried their own copy of the color table before, with nothing keeping them in agreement. Not covered here: persistence/format.rs maps Axis to the on-disk keywords ("row"/"column"/"page"), which is a format concern and stays in the io layer.
  52. +
  53. Axis::Filter constrains without being navigable. Drilling fixes its coordinates on Filter, not Page (improvise-dz8): both restrict which records are visible, but only Page is something the user may move or page through. The split shows up as two different questions, and picking the wrong one is the trap: "what data is visible" is View::filtering_cats() (Page ∪ Filter — used by GridLayout and by add-record-row when it builds a key from the current constraints), while "what can the user act on" is categories_on(Axis::Page) (used by [/] page cycling) or View::tileable_cats() (used by all three tile-select sites). Axis::next() maps Filter to itself and no navigable axis rotates into it, so tile select cannot hand one out. The .improv grammar accepts filter and round-trips a selection with it — a Filter axis only ever occurs in the never-persisted _Drill view, but the alternative was writing it out as some other axis.
  54. +
  55. The browser backend is sync on purpose, with an async seam. OpfsFs (improvise-io/src/opfs.rs, wasm32 only) implements the same synchronous Storage trait as PhysicalFs, because FileSystemFileHandle.createSyncAccessHandle() yields a handle whose read/write/truncate/flush are blocking — available only inside a dedicated Web Worker, which is where improvise-9x6 puts the app. So Storage never became async. What is still async splits out into four explicit methods the worker awaits from its message loop: open(), attach(path) (must precede any read/write of that document — acquiring a handle is async even though using it is not), refresh() (repopulates the index list answers from), and commit_deletes(). Consequence: delete is visible immediately but lands in OPFS at the end of the batch — it drops the name from the index and queues a removeEntry. Handles are exclusive per file, so attach closes any handle it replaces and detach releases the lock. storage_contract in storage.rs is the behaviour every backend owes; it runs against both host backends and is the written-down spec OpfsFs must satisfy in a browser, where the harness cannot reach.
  56. +
  57. "Which view is active" is stored twice, and the copies are now allowed to differ. Workbook::active_view is the document's default — what a freshly connected client opens, and what .improv writes as Initial View. ViewState::active_view is authoritative for that client, and it is what keys ViewState::cursors. The second copy exists because reduce_view runs on a client that holds no Workbook (improvise-cqi); the split is improvise-jytm answering improvise-g0f, on the user's ruling that the model is shared and the UI is not. Two clients may sit on different views of one document. Three things follow. App::resolve_view is the one answer to what a view name names in this document, falling back to the default so a slice naming a deleted view cannot panic in rebuild_layout; view(), view_mut(), view_name(), project::grid_in and ClientSession::view_in all go through it, and a second copy of that fallback was the first thing review found. The guard was replaced, not deleted — the_two_active_view_names_never_diverge became the_documents_default_view_outlives_this_clients_switches plus a_fresh_session_opens_the_documents_default_view, because a deliberate split with no guard is how one silently regresses into the other. And a deleted view is the case the split creates: DeleteView can take away the view another client is on, so Sessions::adopt_deleted_views moves that mirror to the default and tells it, and effect::DeleteView does the same for the host — observed ("does this mirror still name a view the document has"), never predicted from the command variant. Testing any of it needs a bystander that can be moved: both audience tests first sat theirs on the default, where a wrongly-addressed relay folds to a no-op, and review demonstrated the vacuity by injecting ToAll twice. Two gaps remain, both filed: a view-scoped ModelCommand names no view, so a connected client's SetAxis lands on the host's view (improvise-lltt — deferred on a condition, that no shipped client emits SwitchView, while the browser already emits TransposeAxes), and Initial View no longer follows the TUI user (improvise-azhb). Follow mode is deliberately not foreclosed.
  58. +
  59. Two tests gate ViewState membership. ViewState is the portable session slice — the one thing serialized into the Snapshot a newly-connected client gets (improvise-q08). A field may live there only if it is portable (plain data from the allowlisted crates, no Rc/Arc) and snapshot-safe (a fresh client inheriting the value is correct). wizard fails both — ImportWizard is an improvise-io type, and a half-open dialog is not something to inherit — and drill_state fails both — it holds an Rc, and pending_edits are uncommitted writes belonging to whoever staged them. Both live on App under the session-local residue tag (improvise-h0b). The ViewState: Send static assertion in app.rs's tests is the compile-time proxy for portability: Rc is not Send, so a re-added shared pointer fails the build (the Deserialize derive from gotcha 25 now rejects it too, serde's rc feature being off). Send does not catch the origin-crate or snapshot-safety halves — check those by hand.
  60. +
  61. : commands settle their mode by observation, not prediction. ExecuteCommand ends every batch with ReturnToNormalUnlessMoved(ctx.mode), which drops to Normal only if the mode is still what the batch started in. There is no Effect::changes_mode() any more: it was a per-type constant, and ViewBack/ViewForward/StartImportWizard swap the mode only on some runs, so no up-front answer was right — :import opened the wizard and was snapped straight back out (improvise-toc). A new effect that changes the mode needs no annotation; just don't return the batch's starting mode from a success path.
  62. +
  63. There is no aggregate render cache — each widget folds its own stream. improvise-protocol::render holds one cache type per widget (GridCache, CategoryTreeCache, FormulaCache, ViewListCache, TileBarCache), each implementing WidgetCache with its own Update type, and a Projection is one such update addressed to one widget (improvise-rkeq). reduce_view folds only ViewState now — it no longer takes a cache, and Command::Project passes through it untouched. Two consequences. The overlap between caches is deliberate: the tile bar and the category panel both carry (category, axis) and are updated by separate projections, so neither is woken by the other's changes; do not "fix" it by sharing a variant. Bootstrap is a compaction, not a replay: Snapshot { view, seed: Vec<Projection> } seeds a new client with a batch of the same projections the steady stream carries, so there is no whole-state type that could disagree with the incremental one. The seed is the server's current state, not the head of the history — GridCache::seed() builds it, and the property that earns it is GridCache::folded(&cache.seed()) == *cache. Nothing is defined for the help overlay (its content is static and help_page is already in ViewState) or for the import wizard, which now has a shape and deliberately no projection: WizardCache is built by project::wizard_cache and drawn locally, because a half-open dialogue is addressed to precisely the session that opened it, never to every client (improvise-764 — the rendering-side twin of the argument that removed AcceptWizardModel from ModelCommand, gotcha 38). A cache without a Projection is the shape session-local residue takes when it still has to be drawn by two different front ends. It lives in improvise-protocol rather than improvise-command for the flatten_cat_tree reason: when the wasm client grows a wizard it must draw these rows, not a second derivation.
  64. +
  65. The grid's labels frame its cells, and apply enforces it — the fold is not commutative. Three consequences in improvise-protocol/src/render.rs. (a) A GridUpdate::Cells that overtakes the RowFrame/ColFrame naming its positions is dropped, because fit_to_labels truncates to the labels it has; a client must not reorder the stream, and a_cell_update_that_overtakes_its_labels_is_lost pins it. (b) A label change that renames a position blanks the cell under it: fit_to_labels preserves by POSITION, which is only right where the label did not change, so apply calls invalidate_relabelled itself rather than trusting a producer to send GridUpdate::Invalidated first. Invalidated therefore now means only "contents stale, frame unchanged". (c) every arm that touches labels or widths re-fits — col_widths is re-fitted with the columns — a new column is drawn at its label's own character count until ColWidths catches up — because a renderer zips widths with labels and a length mismatch indexes off the end. Fitting on the label arms alone is what let a late ColWidths put the mismatch straight back — a short update keeps each unheard-of column at its label's width, and a long one is dropped as having overtaken the frame that names it, exactly as (a) holds Cells. GridCache also carries totals and page_coords (improvise-n10). totals is framed by col_frame on the same terms as the cells and widths and re-fits with them, with one asymmetry: an empty strip means "this grid has no totals" (records mode, or an empty grid) and is therefore not padded, where a non-empty one is — padding it would draw a blank total row under a records grid that has none. Invalidated blanks the totals too: they are contents, and a stale sum under fresh cells is the one number a user takes at face value. page_coords is framed by nothing (it names categories, not positions) and survives a reshape. Note the fold tests: asserting fold(prefix)+fold(rest) == fold(whole) is vacuous (it is the definition of a left fold), so the tests that earn their place are the order-sensitivity one and the compaction-equivalence one — see doctrine(fold_tests_must_test_order_not_the_axiom).
  66. +
  67. A formula target is unquoted, and pipe quoting has one home. parse_formula strips the pipes off a target exactly as it does off an Expr::Ref — the target names a _Measure item, so a target that kept its delimiters made add_formula create an item literally named |Total Revenue|: a ghost column the user's real item never matched, and Expr::Ref("Total Revenue") never resolved to. Both the TUI formula minibuffer (AddFormula) and .improv load go through that one call, so both were affected. The writer emits Formula::raw, not target, so file text is unaffected either way — which is exactly why the round-trip tests never saw it, and why the fix repairs existing files on next load with no migration. Pipe quoting has exactly one home: improvise_formula::quote, owning both halves — quote_name/is_bare_name, pipe_quote (unconditional, for a .improv text cell that must not read back as a number) and unescape_pipe_body. improvise-gp re-exports it, improvise-io calls it from persistence/{format,parse}.rs, parser.rs::unquote_pipe is a shim over it, and io's own copy is deleted (improvise-i2m5). The keyword guard is canonical — user-confirmed, so do not revert it as over-caution: an item literally named SUM is written |SUM|. Bare keywords do parse, but only because agg_call needs a ( and PEG backtracking falls through to ref_expr — rule ordering, not a stated rule — and a ## Formulas line carries formula text verbatim, so a name format_md writes is read back by parse_formula: one namespace, one spelling rule. One thing the guard does not cover, and one that used to be an exception and no longer is. A section header takes its name as the whole line, totally (improvise-pczv): header_name = { pipe_quoted ~ &NEWLINE | line_name } — the pipe-quoted arm needs the lookahead, or |A|B would parse as A and silently drop B, and line_name catches everything else, so a header name can no longer fail to parse. Four rules share it (model_name, initial_view, category_section, view_section), which is why it is a rule and not four copies. Note a header is still written raw where a reference is quoted — the grammar accepts both arms, so the quoting asymmetry between a header and a reference is a writer convention, not a format one. And the two former unquoting copies differed only on a trailing lone backslash, unreachable from the grammar, where io returned it and parser.rs panicked — the surviving unescape_pipe_body is total. Two ## Category: sections naming one category are one category, concatenated by ParsedFile::push_category in pass 1 (improvise-349m) — merging in pass 2 was not idempotent, because build_category creates its own section's item-groups before its own > lines while format_md writes every item before every > line. No grammar change was needed: this is the tree-walker, not the format, and duplicate headers stay legal. And WHERE now carries IS's word boundary — both are compound-atomic with sp+ (improvise-r08u), so SUM(Rev WHEREAS x = y) is a syntax error rather than a WHERE on a category AS. The guard is sp+ and not the !bare_char lookahead the ticket proposed, because the grammar-walking generator reads the grammar through pest_meta and skips predicates, emitting WHEREa="" — a rule whose only guard is a lookahead is a rule the generator can violate. It costs one spelling: a category may no longer abut the keyword.
  68. +
  69. Dirty over-reports on purpose, and the asymmetry is the reason. Dirty::of maps an EffectKind to the widgets that kind can reach, and the two directions do not cost the same: over-reporting pays one redundant Projection, under-reporting leaves a client's cache wrong forever, with nothing in the stream to correct it. LayoutGeometry therefore includes formulas, because RemoveCategory wears it and Model::remove_category ends by dropping every formula targeting the category — a client's formula panel otherwise kept showing a formula the document no longer had. It now coincides with Dirty::EVERYTHING and is still spelled out, because the two are derived from different effects and need not stay equal. The derivation now lives in context/code/command.md, and the doc comment on Dirty::of points there rather than restating it; the reason LayoutGeometry is spelled out rather than written as EVERYTHING is a two-line comment inside the match body. It went stale once already — claiming AddCategory was CellData after it was reclassified — so reclassifying an effect means re-reading the prose, not just the comment.
  70. +
  71. A client's command has one door, and the door takes the App. Sessions::apply_checked(app, id, Checked<'_>) is the only way a connected client's Command reaches the server — Host calls that (improvise-ahdf). Sessions::apply is not pub, delegates to apply_checked after Checked::against, and has no production caller: its call sites are tests, which is why it takes a raw &Command. ClientSession::fold and the mirror's ViewState are private, so a transport cannot reach the mirror at all. apply_switch now takes a SessionId — it is addressed to exactly one session, which is the opposite of what this gotcha used to say — moves only that session's mirror, and dirties nothing. The one thing it still asks the document is whether the view exists, because a mirror keys ViewState::cursors and is read by grid_in. The audience is conditional: an accepted switch relays nothing (the asker folded it, and no other client's view moves), a refused one is Broadcast::To(asker, …) — improvise-6c6q, where ToOthers told every client but the one that was wrong. adopt_active_view and Broadcast::ToOthers are deleted; the variants are ToAll and To(SessionId, _), the second because the deleted-view case needs a one-session audience that is not the asker. The paragraph's warning about the "what do the others hear" helper survives and is why the 6c6q test drives the asker's own ViewState — that helper is itself deleted. What survives unchanged: the model is shared and the UI is not, and Broadcast is a Vec because one command can owe two relays with different audiences. The trap this paragraph used to describe is gone with ToOthers, and the lesson it taught is not. It was copying ToOthers from apply_switch into apply_model, invisible from the bystander's side because adopt_active_view moved the asker's mirror either way — so the drift sat between the server's picture of the asker and the asker's own slice, where a test written through a "what do the others hear" helper looks away by construction. improvise-6c6q closed the mirror-image hole on the refused path. Two rules survive the mechanism: assert the Broadcast itself, since the audience is the message rather than something to infer from folded state, and seat a bystander somewhere a wrong relay can move it from — a bystander already on the relayed value cannot see a wrongly widened audience either, because the wrong relay folds to a no-op. Review demonstrated that vacuity twice on PR #48. A Vec rather than an Option because one command can owe two relays with different audiences, and an Option would have had to drop one silently. Everything else is that client's alone and never touches the App's own view slice: the model is shared, the UI is not. Folding SwitchView into the mirror alone was the bug — viewport() then read the cursor under the new view's key and the next project() built that client a grid out of the other view.
  72. +
  73. gp-tree returns data and never prints, and a fit takes two things to reproduce. Two rules with one root — the crate is a library a TUI, a wasm worker or a CLI may each host. (a) A decision the user should hear about comes back as data: Dataset::warnings carries the columns preprocess dropped for being entirely missing (improvise-d44s), and fit.rs hands back pooled (prediction, actual) pairs and a score per feature rather than tables. examples/fit_csv.rs owns every println!, and that is the only reason its output stayed byte-identical across these moves — adding an eprintln! to either module is the regression. (b) The seed is necessary and not sufficient: data::Model records seed: Option<u64> and config: Option<FitConfig>, because the same seed under a different population/generations/elite/penalty lands on a different tree (improvise-ksst). Both are Option for one reason — every FitConfig value and the seed 0 are legal, so nothing can double as "unknown". Note #[serde(default)] on an Option field is documentation, not the mechanism: serde defaults it anyway, so deleting the attribute fails no test, and a backward-compatibility test that only deletes it is not discriminating. Aim such a test at an absent field loading as the default instead of as unknown. The GA loop itself is fit::fit (improvise-0ai); breed remains one fitness-agnostic generation for callers whose fitness is not "predict this column". (c) A scoring function takes one slice of rows. Row<T> pairs a feature vector with what it predicts, so the parallel x/y lengths that zip used to truncate silently cannot be built (improvise-dkqn, doctrine(an_invariant_is_made_unrepresentable_not_validated) — the same answer prune_axis_items got, and the reason no Result entered five signatures). The pairing is made at the boundary that builds the data: preprocess walks one row index and the rows live inside Target, because a Dataset carrying rows and target separately just moves the zip up one level into every caller. The empty row set is one shape, answered once in mean_over. Changing any of this needs bit-level evidence: tests/scoring_known_answers.rs pins every path as f32 bit patterns plus a next_u64 witness that catches a changed RNG call order, because the CLI prints three or four decimals and byte-identical output alone cannot see a moved accumulation. (d) A fit is storable only when the document can name its features. preprocess invents Region=North, Cost.missing, when.year; those are encoding artefacts, not items of the document, and Expr::Ref resolves through Model::find_item_category — so a tree splitting on one emitted a well-formed formula whose every cell was Error("ref:…"), silently (improvise-8xhy). fitted_formula therefore takes the &Model and refuses, listing what it could not resolve; formula_text still spells it, being the tree's own text. The check asks the document, never the name's shape, because a measure literally named Region=North is a legal document and the two are indistinguishable as strings — which also means .missing and the date parts are covered with no second copy of preprocess's expansion rules to drift from. This is an interim (improvise-4zl7): a WHERE predicate needs a scalar coordinate predicate the formula language does not have — Filter gates a formula or an aggregate, it is not an Expr — would still leave .missing and the date parts, and must first answer what such a predicate means where context.get(cat) is None, since Formula::filter falls through to aggregate_raw and a categorical fit would silently degrade in every view that aggregates over the category. improvise-zv8t closed the categorical half of the accident, and the close cost a document. fit_measure handed preprocess an Overrides::default(), so it never said which header columns are document categories although the very next line told Features::of — and preprocess infers Numeric → Date → Categorical from the values, so a Year category with items 2024/2025 was a numeric column and a category of ISO dates a date column, both refused with Year IS 2024 spellable all along. It now forces the same header[..category_columns] slice to Forced::Categorical. Two things follow. The date decline is narrower than it read — it covers a genuine date measure, never a category misread as one. And the forcing widens the encoding: a hundred-item numeric category one-hots past MAX_FEATURES and refuses a fit that previously succeeded by never splitting on it (improvise-e8no — and not to be closed by making the forcing conditional on cardinality, which is inference by another name and makes the encoding depend on a budget rather than on what the document says). The Artefact arms stay: crate::dataset is pub and takes the caller's Overrides, so the roles are still reachable. Forcing by name while budgeting by index defeats the budget — a category and a measure may legally share a name, and no_ambiguous_column_names ran after preprocess, so the expansion the preflight exists to prevent was allocated and only then refused; as_categories now returns Result<Overrides>, so the ordering is a compile error rather than a convention. The test that missed all this is the shape to remember: improvise_eval fabricates a _Measure item per feature name, so it asserted the emitter and gp_tree::eval agree against a model that could not exist. It is kept, with its scope now stated, and the property it could not state is asserted separately against a real document.
  74. +
  75. The wire vocabulary is derived twice, by the same recipe. Command's view half is the EffectKind::ViewOnly effects targeting the portable slice (decision(improvise-cqi, command_set_derived_not_invented)); its model half, ModelCommand, is the EffectKind::Model(..) effects — 26 of them since improvise-jytm made SwitchView, ViewBack and ViewForward ViewOnly — minus ApplyAndClearDrill and AcceptWizardModel, leaving 24 variants (improvise-kmgs). SwitchView no longer needs subtracting separately: it is not a model effect any more. ViewBack/ViewForward left for the same reason as the other two exclusions — their operand is ViewState::view_back_stack, session-local residue belonging to whoever navigated. Note the obvious awk extraction reports 30: it picks up HelpPageSet, which is ViewOnly, because that is the last pub struct before #[cfg(test)] and the test module mentions ModelChange:: — brace-match the impl Effect blocks instead (doctrine(derive_then_verify)). Three things follow. A ModelCommand does not mutate: ui/reduce.rs::model_effects maps it back to the effect it came from and apply_command runs that through App::apply_effects, so effects stay the only mutation site and the Dirty is the effect's own EffectKind, never a second classification beside gotcha 35's table. Two model effects are deliberately off the wire. ApplyAndClearDrill and AcceptWizardModel were once kept on the grounds that the command is portable even where the operand is not; that was the wrong criterion (improvise-3x3u). A Command::Model is routed with no SessionId, which is right exactly when the change is addressed to no one session — and these are addressed to precisely one, never the sender: their operands are drill_state.pending_edits and the built wizard workbook, session-local residue (gotcha 30) private to whoever staged them (improvise-tom1). Unaddressed, B's ApplyAndClearDrill commits A's uncommitted typing and B's AcceptWizardModel swaps the document out of a dialogue B never opened — both demonstrated against Sessions::apply before removal. So 29 − SwitchView − these two = 26. The effects are untouched and still reached by the key path. Not permanent: multi-client drill wants the addressed form, which is where ADR 001 layer 2 goes anyway. Guarded by no_model_command_commits_another_clients_session_local_state, plus a companion asserting both effects still exist and are still model changes so the first cannot pass by naming nothing. Sessions::apply_checked routes the model half to apply_model, which relays adopt_cursor_epoch's ResetCursor and adopt_deleted_views's per-client switch; a document-replacing LoadModel bumps the generation and lands in the rebase arm. The former claim here — that ViewBack, ViewForward and LoadModel all end in App::switch_to_view so every mirror takes the accepted active view — was wrong on all three clauses after improvise-jytm and is deleted. A keymap lookup still cannot return a Command, and not for want of vocabulary: a binding names an intent resolved against CmdContext (move-cursor down) while a Command is the resolved mutation (SetSelected { row, col }) — improvise-gxi's AC 4 asks for something the split forbids.
  76. +
  77. Applied is an enum because a document swap is not a diff. Sessions::apply_checked returns Applied::Diff { dirty, broadcast }, Applied::Rebased { snapshots } or — since ADR 001 layer 1 — Applied::Rejected { reason }, and the enum is the mechanism: a host that wants the Dirty must match, and matching names the case where a Snapshot is owed. The server's mirrors self-heal — ClientSession::project reseeds on the document_generation bump — but ViewState::cursors lives on the client, keyed by the outgoing document's view names (gotcha 24), and only a fresh Snapshot replaces it. Command::Model put the three WholeWorkbook mutations on the wire (gotcha 38), so the host that must send the snapshots is no longer the host that asked. apply_model therefore calls rebase itself rather than instructing the host to, exactly as apply_switch adopts the accepted view itself. The trigger is the generation moving, not the command variant: a LoadModel that cannot read its path leaves the document alone, and reseeding on the variant would cost a client its cursors for a failed load — a_load_that_fails_is_not_a_rebase guards that arm. A rebase: bool beside dirty was rejected, because the host that forgets to call rebase is the host that forgets to read the flag. The third case is a normal outcome, not an error, and it is in the enum for the same reason the other two are: a host that wants the Dirty must match, and matching names it. ServerMsg::Rejected is the matching frame, addressed to the asker alone — nothing happened, so nobody else is owed anything — and the client keeps it as a Rejection rather than a sentence, because force needs to know which record and which kind (gotcha 43). Fixture warning. When testing a document swap, the incoming document's active view must have the same name as the outgoing one. A rename makes apply_model broadcast a SwitchView; the client folds it, and the stale cursor is then read under a key nothing has written — right by accident, with the bug invisible. other_document() renames, and a first version of this test passed because of it; a_document_shaped_differently_under_the_same_view_name() keeps the name and changes only the shape, so there is nothing to hide behind.
  78. +
  79. A batch's basis is three tiers, and only two of them can be read off the effect. App carries document_generation (bumped by install_workbook), layout_version (per ModelChange::LayoutGeometry effect) and record_versions (per record) — ADR 001's stale-basis check, and shared document state, so never on ViewState (gotcha 30). The unit is the record: record_of(key) is the coordinate tuple minus _Measure (record_row_identity), because renaming a category coordinate in wide records mode moves every measure cell in the row (gotcha 1) — a cell-level unit is violated by one legal edit, so the unit must be at least as coarse as the coarsest legal write. Both tiers are read in apply_effects off the same EffectKind the Dirty is: one classification, two readers, no second table to drift from gotcha 35's. layout_version bumps inside the loop for the reason Dirty accumulates there — an aborted batch must not claim effects it never ran (gotcha 21). record_versions cannot, because an EffectKind says that a batch writes cells, never which; so the records are observed, via a per-record XOR fingerprint of the raw data taken when the batch declares a CellData effect and diffed after it. Three consequences. A write landing the value already there bumps nothing, which is correct — no basis was invalidated. A LayoutGeometry effect that drops cells will bump their records when it shares a batch with a CellData one: an over-bump costing at most an unnecessary rejection, the same direction Dirty::of is deliberately sloppy in. And only raw data is fingerprinted — formula results live in Model::formula_cache, and a record whose derived value moved because someone edited elsewhere has not itself been written. The diff collects into a set: chaining before.keys() with after.keys() counts any rewritten record twice, a bug that survived one test run.
  80. +
  81. Storage is not a VFS, and that closes the SQLite question. The trait is whole-document read/write bytes (gotcha 2, invariant(storage_is_two_ops)) — no offsets, seeking, handles or locking — so an sqlite3_vfs, which needs xRead(offset,len)/xWrite/xTruncate/xLock, has nothing to bind to. SQLite can still sit behind it, as an in-memory database serialized to a blob (rusqlite serialize/deserialize_read_exact, proven with a probe before declining — declining on a capability that actually exists would have been the wrong decline). It is worth nothing: measured on 18,000 cells the SQLite container is 22% larger than .improv, 4.7× larger gzipped, ~160× larger at the floor (24 KB against ~150 bytes), and still rewritten whole on every save, so the one dimension SQLite normally wins — incremental page writes — is unavailable. improvise-9ix was declined on that evidence. Two things generalise past SQLite. A second serialization format is a second implementation of the grammar, obliged to track improv.pest forever; and its own round-trip test cannot detect drift from .improv, because it never compares the two — that missing cross-format equality test is what made the ticket look cheap. If a structured-query story is ever wanted it is a one-way export_sqlite beside export_csv, native-only and feature-gated: libsqlite3-sys' C build fails under this repo's nix cc-wrapper for wasm32-unknown-unknown, which would break the wasm CI gate the day it landed.
  82. +
  83. The browser client's render decisions are made in Rust, and the frames it speaks are nobody's but the protocol's. improvise-wasm-client/src/render.rs builds a RenderModel in which every question with a silent wrong answer is already answered, because the DOM half (web/app.js) is untested JavaScript and a wrong answer there shows up as a wrong pixel rather than a failing test. Three such answers. editable is a fact about the CellId variant — Record only, per ADR 002 — and the trap is the third case: None means "no projected cell", not "aggregate", so conflating them either hides a cell or offers a write the type system exists to make unconstructible. The selection is placed by reconciling the scroll offset, because GridCache is viewport-indexed while ViewState::selected is absolute. render.rs does it by adding — r + cursor.row_offset == sel_row — rather than subtracting, which is why there is no underflow path and why a selection above the window simply never matches. Omitting the offset entirely looks right until the first scroll, so the test that earns its place uses nonzero and unequal offsets: at selected=(5,1), row_offset=4, col_offset=1 it separates "reconciles" from "ignores" and from "swaps the two offsets". The category panel is viewmodel::flatten_cat_tree, never a second walk of CategoryTreeCache — that second walk was improvise-edp. The page is four string calls: Client::viewport (the handshake — the Snapshot seeds the visible cells, so one built at extent zero is an empty grid and a resize is a fan-out with nothing dirty), on_frame (one frame, one fold, arrival order — gotcha 33 holds inside a batch, so never coalesce), render_model, typed_cell.

    +

    One envelope pair, two deployments (improvise-xks6). improvise_protocol::frame owns ClientFrame { Viewport, Command, Key } and ServerFrame { Snapshot(Box<_>), Commands(Vec<_>), Rejected }; improvise-ws-server and improvise-wasm-client each pub use them and neither defines one. There used to be three pairs for two deployments — the socket's server side, a hand-kept mirror of it in the browser bundle, and a third written for the worker (improvise-tm6) before the projection architecture landed. The question that had to be answered before any of it moved was whether the worker and the socket want the same vocabulary, and the answer is yes: every difference was a date rather than a decision, and the test applied to each was whether the argument for it names the thing they differ in. None did. Viewport is deployment-independent — the extent is not in ViewState and no effect produces it, so no derivation reaches it, and a worker hosting the same Sessions owes set_extent the same answer; it is the handshake on both because a Snapshot seeds the visible cells. Key is deployment-independent too — a binding names an intent resolved against a CmdContext a thin client does not hold (gotcha 38), which says nothing about what the bytes travel over; the socket lacked it because key forwarding was never built on it (improvise-bpq2), and Host::handle now answers ClientFrame::Key with an explicit empty vec that says so, where the frame used to just fail to parse in next_client_msg. And Commands(Vec<_>) batches where a singleton cannot, because a fan-out must arrive whole and in sequence (gotcha 33) — a property of the fold, not of the transport; an in-process postMessage interleaves exactly as a socket does. The worker pair was not merely different but incoherent by omission: written before ADR 001, it could carry Command::Model { basis, .. } and had no Rejected, so a client speaking it composed a claim that could never be answered. That incoherence — not the difference — is the tell for staleness.

    +

    The mirror is gone, and so are the two tests that held it honest; that deletion was the ticket's measure of success. the_socket_framing_matches_the_server round-tripped hand-listed fixtures through the other crate's types to catch a rename; every_server_variant_is_one_this_crate_mirrors matched exhaustively to catch a variant added on the server and never mirrored. With one definition both are type identity: a rename is a compile error at every call site in both crates, and the un-mirrored variant is not a constructible state. That is stronger than either test, and it is the shape to insist on — if a consolidation leaves one of its compatibility tests alive, the consolidation is not finished. What survives is the_retired_spellings_do_not_parse in frame.rs, aimed at the one thing still possible (the retired Bootstrap/Command spellings returning as serde aliases, which would be two homes again wearing one type), and it keeps the injected-fault discipline this paragraph earned the hard way: assert the mutated input is still well-formed before feeding it in, and assert the error kind — serde's unknown variant — not merely that an error occurred, because a rewrite that also breaks the JSON makes the check pass against an enum that really does have the variant. End-to-end compatibility is now tests/browser_round_trip.rs, which hands the wasm client's own serialized frame to a real Host::handle; improvise-ws-server stays a dev-dependency of the browser bundle for exactly that.

    +

    What the harness does and does not reach is gotcha 48's subject — web/app.js is no longer unrun (harness/tests/shell.spec.js executes it under Chromium), improvise-hwpi is closed, and the quiet-failure and never-reads-a-key claims that used to stand here went with it. What remains unexercised is named there: the grouped and multi-level DOM, the CSS and the ResizeObserver extent arithmetic. What a careful read did establish about the last of these: the observer cannot feed back into itself (#grid is a flex child with overflow: auto, so replaceChildren cannot resize it), and ROW_PX/COL_PX under-estimate, so the worst case is a row of over-fetch. What is verified end to end is the Rust half: tests/browser_round_trip.rs drives improvise-1ey's stated scope against a real Host over a real App touching the client only through the four string calls. One deployment note that is not a bug: examples/demo.improv leaves Customer/Date on the None axis, so every cell is an aggregate and nothing is editable — correct behaviour, bad demo. Pick a model with no regular category on None.

  84. +
  85. A batch carries its basis, and the server had to publish it first. Command::Model { basis, cmd } is a struct variant so the basis cannot be forgotten, and Sessions::apply_model checks it before applying anything — a stale batch is never applied, not even partially (improvise-xwi6/fkfg, ADR 001 layer 1). Four things to know. Basis has no Default and no public constructor: a zero-valued basis is not "no claim", it is the claim that the document is fresh, which in a new App is true — so a default would be a forgeable bypass rather than an absence. BasisCache::claim is the only door a client has and it returns None until the server has published one. Note what that does and does not buy: Basis derives Deserialize because it must cross the wire, and its fields are public, so a hand-rolled client can state any tiers it likes. This is an anti-staleness coordination check, not a security boundary — RecordUnstated closes omission, not invention, and nothing here tries to close invention, because a client that forges a current basis is indistinguishable from one genuinely caught up. "The only constructor" is a statement about clients written against this crate, which is still worth having: it makes the honest path the easy one. Which records are checked is the server's decision — reduce.rs::records_named, an exhaustive match with one or-pattern for the variants that name no record, so a new record-writing variant fails the build. Reading the list off the client's claim would be a force by omission: the record tier skipped, with no flag to remove and nothing in the message saying so. A claim that omits a record the command writes gets Rejection::RecordUnstated, a case of its own because "you lost a race" and "you were never in one" are different things to tell a user. Rejection keeps structural and data staleness apart, which is the whole reason ADR 002 put a maintained RecordHash index in DataStore: RecordGone means nothing resolves and there is nothing to force (force never creates structure), RecordMoved means it resolves at another version and force is one keystroke — which is also why the wasm client stores the Rejection itself rather than collapsing it into status_msg, since a string cannot carry which record or which kind. And the ADR omitted the publication. It said a batch carries the basis it was composed against and never said how a client learns one; nothing in the projection stream carried any of the three tiers, so the check as written was satisfiable only by forging. Projection::Basis + BasisCache close that — the server publishes the two document tiers plus the counters for the records this client's grid names, diffed against what that client was last told and seeded in the bootstrap — and it is emitted after the grid updates in a batch, because the counters describe the cells those updates carry: a basis arriving first would let a client claim a version for a value it had not been shown, the mirror image of gotcha 33. That ordering was documented as load-bearing in three places and held up only by the order of two statements until a_published_basis_follows_the_grid_updates_it_describes pinned it (proved by injecting the swap) — the same job a_cell_update_that_overtakes_its_labels_is_lost does for the mirror-image rule. Two things are unchecked and deliberately so: Command::SwitchView carries no basis (it sits outside Command::Model; the workbook refuses a view it does not have, so it cannot corrupt, but it is a hole in the rule), and apply_command/reduce_full do not check at all — the check is a property of the session door, not of the mutation path, which is right because a host holding the App is the truth rather than a claimant. That second one was a door rather than an exemption, and needed a type to stop being one (improvise-ondj): both took a wire Command and their Command::Model { cmd, .. } arm discarded the basis with .., while ui/reduce.rs's own module doc pointed every future host — the browser worker included — at exactly those two names. The struct variant closed omission from the message; it did nothing to close ignoring on arrival, which is the half a struct variant is usually reached for. So both now take Checked<'_>, a private-field newtype over a &Command whose only constructor is Checked::against(&App, &Command) -> Result<_, Rejection> — also the only caller of stale_basis. Sessions::apply_checked receives the proof and Checked::against is answered once, before the dispatch, so apply_switch/apply_model receive the proof rather than each remembering to ask: an invariant made unrepresentable rather than validated, the same answer prune_axis_items and gp-tree's Row<T> got. Visibility was the weaker option and was rejected for a concrete reason — pub(crate) closes only the out-of-crate path, and reduce_full has no production caller anywhere in the workspace, so it would have become dead code. The regression guard is therefore the signature: the test that reproduced the bug no longer compiles, and what remains asserts the constructor refuses and that refusing leaves the document untouched. One residual hole, documented on the type and filed: a Checked does not name which App it was checked against, so a multi-document host could in principle check one and apply to another. That is also why ui::reduce::caught_up(&App, cmd) is not a bypass: it takes &App, so nothing reachable from a wire can call it. Since improvise-955n the frame, not the command, is what is answered — and only two of the three tiers hoist. CheckedBatch::against runs before anything in a ClientFrame::Commands is applied. document_generation and layout_version describe the document, so answering them once for the frame is exactly right, and a first version hoisted all three — which was a lost update: the record tier asks "is the client's picture of this record current", and which records are at stake is a property of the command (records_named). Skipping it for later commands let [SetCell(North), SetCell(South)] overwrite a South somebody else had moved, silently, with the frame reported accepted, and invisible to the other two tiers because a SetCell is CellData and moves neither. So every mutation answers for its own records against the frame's basis — the one the first basis-carrying command stated, never each command's own field, or a later command could claim a fresher picture than the frame was composed under. The one exemption is stated over a property rather than a list of commands: a record that neither resolves in the document nor is claimed by the basis is one the frame is about to create ([AddItem(East), SetCell(East)], which BasisCache::claim cannot possibly claim for), and only that combination is exempt; a record that does not resolve but is claimed is still RecordGone. That the Basis inside a later Command::Model is ignored rather than compared is an honest interim: clause 1 says the frame states one basis, and moving the field from Command::Model to ClientFrame::Commands would make that unrepresentable-otherwise instead of a convention — a protocol change that belongs with clause 4. Two things worth keeping from the rejected design: arity is not a property of a variant (a future Command::RunScript is one variant performing unboundedly many mutations), so no count over the vocabulary can enforce "one mutation per frame"; and the transaction boundary is not the frame, because one command may already be a chain, which is why clause 4's mechanism belongs at App::apply_effects.
  86. +
  87. The TUI grid draws from its cache, and since improvise-anr9 the cache frames it too. GridWidget holds no &Model and no &View: cell text, aggregate-ness (CellId::Aggregate, ADR 002), column widths, the page bar, the totals strip and now the frame all come off GridCache. The TUI still builds that cache over the whole grid (offsets zero), so a cache position is an absolute position — the browser's Viewport is a real window, the TUI's is not, and the two clients therefore use it differently; column totals must be projected rather than client-derived, because the sum runs over every row including the ones a viewport scrolled past. The frame is an AxisFrame per axis — cats (the categories, outermost first) plus entries: Vec<FrameEntry> interleaving Group(FrameGroup) headers with Item(Vec<String>) label tuples, exactly as GridLayout::row_items interleaves them. That replaced a Vec<String> built by GridLayout::row_label, which slash-joined the tuple — the exact rendering two_row_categories_produce_cross_product_labels says must not appear — so the browser drew Food/Alice where the TUI drew two header columns, with no groups and no collapse indicators. It is one type and not a labels plus a levels beside it: two answers to "what are the labels" is the disease (improvise-edp), and this is the second cure, which cost a type change rather than an addition. Three things to know before touching it. leaf_labels() is not "the labels" — it is the innermost label, what a one-line summary calls a row by; a draw loop wants header_labels(first_rendered), groups() and shares_ancestry(sel), each of which is one pass, because item/label/shows_level/group_of scan entries from the start and asking them per position is quadratic in the frame and grows with the TUI's absolute scroll offset (found three times on anr9; the per-position forms survive for one-off lookups and say so). The width rule has one home, improvise_protocol::header_level_widths — widest label per level, plus a gap, clamped — reached by AxisFrame::header_widths and by ui::viewport::compute_row_header_width, which borrows level tuples off a GridLayout rather than building a frame on the keypress path. A cats change re-identifies every position: a cell's identity is cats zipped with its tuple, so invalidate_relabelled blanks the whole axis when the categories move, not merely the positions whose tuples differ, and blanks the totals on a recategorization of either axis — a column total is a sum over the rows, so re-identifying the rows leaves a number answering coordinates the grid no longer has (gotcha 33(b): the rule lives in apply, not in a producer's memory). What is still not on the cache is record_col, and that is correct: it keys pending drill edits and is session-local (gotcha 1). body_from(idx) takes an argument only because the two front ends scroll differently — a browser's frame is already windowed and passes 0, the TUI's covers the whole grid and passes its row offset — not because the rule differs. group_of answers group membership: the nearest preceding header unless an AxisEntry::GroupEnd closed the run first (improvise-gd83, landed in PR #77). It previously answered "nearest preceding header" unconditionally, which gave an ungrouped item the preceding group. GroupEnd draws nothing and exists only so membership is not inherited. Still open beside it: group_runs coalesces by name where FrameGroup carries a cat, so two runs under different headers merge (improvise-h0iv), and AxisFrame::first is serde(default), so a frame that loses the field claims it leads the axis (improvise-1wl0). And project::grid_of evaluates every cell per frame where the widget evaluated only the visible ones — a widening of an existing whole-grid cost. Since improvise-jytm grid_in may additionally derive a GridLayout for a session on a view the host is not on; the host's layout is reused where the names agree, which is the common case and the only one that can carry a drill snapshot.
  88. +
  89. One session loop, two transports — and the browser saves itself. Host is App + Sessions + handle(id, ClientFrame) -> Vec<Addressed> and lives in improvise_command::ui::host, not in improvise-ws-server, which re-exports it. It moved for improvise-9x6: the worker hosts the same authoritative App over a MessagePort and cannot depend on tokio, and a second copy would have been exactly the mirror improvise-xks6 had just deleted for the frames — a drifting Applied/Broadcast match is invisible until a client sees it. registry stayed on the server, because a worker hosts one document. What the worker adds is only what a MessagePort differs in: one client by construction (the Addressed filter is honoured anyway, against a future SharedWorker), and a handshake with no close — ClientFrame::Viewport must still come first, but a pre-handshake frame is dropped rather than closing a connection that does not exist.

    +

    Saving is a frame, and dirtiness is read off the classification the effect already carries. effect::Save is ViewOnly (gotcha 3) — correctly, since it writes this host's file_path through this host's Storage — so the derivation that builds ModelCommand from the model effects cannot name it, and for a while that meant no client on any transport could ask its host to save. ClientFrame::Save closes it (improvise-368h): a request carrying no path, because a client cannot see the host's storage, answered ServerFrame::Rejected(NoDocumentPath | SaveFailed) to the asker alone and silently on success — nothing moved, so no widget is dirty. It is a frame and not a Command for one concrete reason: as a Command it put a runtime-fallible operation inside a batch ClientFrame::Commands promises is all-or-nothing, so [Save, SetCell] reported a failed write and then applied the edit. It also fails the test every Command passes — it mutates no shared state and no client's slice, so reduce_view has nothing to fold and Checked has no basis to answer for. And MarkDirty is gone. It was a separate effect the command layer composed beside each mutation — reasonable on the key path, unreachable from anywhere else, because improvise-kmgs derived ModelCommand from the effects one at a time and so put SetCell on the wire without its companion (improvise-gv9r): a derivation correct per element and wrong for a set, whose symptom was that "save if dirty" never fired. App::apply_effects now raises the flag for any EffectKind::Model(_), all three ModelChange levels included, which makes omitting it unrepresentable rather than validated. It over-approximates in the same direction layout_version does — a LoadModel that cannot read its path leaves the flag raised, costing one redundant write — and Save clears it in loop order, so a batch that edits and then saves ends clean. The worker's persist trigger is therefore the flag, not a table of command variants: an observation rather than a second description of which commands change the document, for the reason cursor_epoch beats matching on CycleAxis (gotcha 24) and document_generation beats matching on LoadModel (gotcha 39). It is also strictly tighter — a refused batch raises nothing, so the "was anything rejected" guard went with it. A browser still never sends ClientFrame::Save, and that is the answer rather than a gap: there is no file the user owns and no Ctrl-S semantics, the document is the OPFS entry, so the worker persists on its own behalf through the one existing Save effect. The cost that remains: Storage is whole-document (gotcha 41), so OPFS rewrites the file per accepted model command.

    +

    The async seam does not survive being boxed. OpfsFs::open/attach/refresh/commit_deletes are on the struct, not on Storage. App::storage is a Box<dyn Storage> and a trait object can name none of the four, so handing the backend to the App puts the async seam permanently out of reach; the host must hold the concrete backend and give the App a Storage, which means sharing one. That is impl<T: Storage + ?Sized> Storage for Rc<T>, and it lives in improvise-io rather than in each worker (improvise-hmws): impl Storage for Rc<OpfsFs> is two foreign types in any other crate, so improvise-9x6 wrote a forwarding newtype that every future worker would have written again. Rc and not Arc because the one caller is a Web Worker and wasm32 has no other thread. The flat root is no longer load-bearing: OpfsFs::list joins dir like the other two backends, so storage_contract takes dir as a parameter and the three agree at every directory rather than only at Path::new("") where joining is the identity (improvise-vhcz). What the index holds is documents, not entries, and the split between its three writers is the rule: attach indexes nothing (what it creates is zero bytes), Storage::write indexes the name and un-queues any pending delete of it, and refresh rebuilds from OPFS skipping zero-byte entries — counting an entry it cannot measure as a document, since the one cause of that is another tab holding the exclusive handle (improvise-vvl3).

    +

    What no test here can see has shrunk, and precisely. harness/ (improvise-vvl3/xf1) runs improvise_io::storage::contract::storage_contract against a real OpfsFs inside a real dedicated Web Worker under Chromium and Firefox, and the browser job in CI can fail the build — so OpfsFs is no longer compile-checked-and-never-run, and two bugs fell out of its first execution. That is why storage_contract stopped being a #[cfg(test)] helper and became a public function returning Result: a browser must run that sequence, not a second transcription of it, and a failure has to travel out of a worker as a string. Still unseen: Safari (Playwright's WebKit has no OPFS, so xf1's criterion is met for two engines of three; opfs.spec.js skips on a runtime probe and an unexpected skip on any other engine is a failure). bindings.rs is NOT in that list, and said it was for months. shell.spec.js's "the standalone deployment" loads the app with no ?ws=, which spawns the worker, which imports the glue, which calls bindings.rs::start and OpfsFs::open — and the spec then asserts a grid draws, so the whole chain has run under Chromium in CI since improvise-vvl3. The claim survived in five places (this line, the File Inventory, two kb.pl facts and improvise-bck's grooming note) because each was written before the harness landed and none was re-read after; the spec's own comment says the chain plainly. It is the reason to distrust a "never run" written in prose: nothing checks it, and kb.pl's CI gate checks consistency rather than truth. What was already verified stays: tests/post_message.rs drives a real Worker over a real App on nothing but the JSON a MessagePort carries. Build note unchanged: the worker bundle needs --enable-nontrapping-float-to-int as well as --enable-bulk-memory, because it carries the whole App and so chrono's date arithmetic.

  90. +
  91. The page picks a peer, and the demo was built against an assertion. web/app.js holds two transports and no protocol knowledge: ?ws= selects improvise-ws-server, its absence spawns new Worker('./worker.js?doc=&seed=', { type: "module" }) (improvise-djm). type: "module" is mandatory — worker.js imports the bundle — and there is no MessageChannel: a dedicated worker's own postMessage/onmessage is the port, so the ticket's channel wording was a date rather than a decision. The websocket path is kept, because the ws-server is how a TUI-hosted document is shared and nothing else in the browser can reach it; the cost is contained because the two differ only in where strings come from — handshake, fold, repaint and keyboard are all above the seam — and the presence of the parameter is the whole selection mechanism, a second knob being a second thing to get wrong. #status composes two facts, the transport's and the document's, because a single source let a repaint silently erase a connection error. A worker that rejects in start() fires no error on the main thread — worker.js catches its own — so the guard is a first-frame watchdog, deliberately not per-transport: "the peer never answered the handshake" is the one failure that means the same thing over a socket and over a port, and a check aimed at each peer's own failure modes would have missed exactly this one.

    +

    The demo is a model built to satisfy a check, not an axis flipped and hoped for. examples/demo.improv leaves Customer/Date on None, so every cell is an Aggregate and nothing is editable — and the naive fix does not work either, since Product × Region by Date_Month × Measure still aggregates. web/demo.improv puts exactly one category on each visible axis (Product row, _Measure column, Quarter page, nothing regular on None) and carries Profit = Revenue - Cost, so the payoff is on one screen with no keystroke. Settability is asked of grid_of as a CellId variant, never re-derived from the axes — a second derivation could agree with itself while disagreeing with render.rs::editable, which is the whole point of ADR 002 — and the assertion was proved discriminating by pointing it at examples/demo.improv, where it names the aggregate at (0,0). One category per axis is a property of the demo, not of the format: when this was written GridCache slash-joined a multi-level tuple, so a second category on an axis put that known gap on the front page — improvise-anr9 has since closed it (gotcha 44), so what remains is the demo's own simplicity rather than a rendering limit. web/*.improv is un-ignored in .gitignore (*.improv is ignored repo-wide) and the CI assemble step copies it: leaving it out is an empty grid, not a 404 the page could report. One wart it now puts on the front page: a formula cell is a Record, so it is offered as editable and a write to it reverts on recompute — pre-existing and matching the TUI, newly visible.

  92. +
  93. Keys become commands in the browser, and the server must not know the keymap. The user's ruling on improvise-bpq2, following from "the model is shared, the UI is not" — and it buys two clients different bindings over one document. What made it urgent is not latency. a_key_resolved_on_the_host_moves_state_no_client_can_hear shows nothing in the projection stream carries the host's own ViewState, so host-side resolution diverges; and Host::handle answers ClientFrame::Key with an explicit empty vec, so an upstream key is a black hole, not a slow path — every KeyOutcome::Upstream names a binding that does nothing at all in a browser. That test still holds and was not flipped: bpq2 closes the divergence by avoiding it. The method is the ticket's — enumerate the bindings, not the commands (doctrine(binding_dump_diff)) — and where a client cannot compose the Command, enrich the projection rather than moving resolution back.

    +

    Three things that cost. GridExtent on GridCache — the whole grid, of which the labels are one window — because a viewport-indexed cache cannot tell "the last visible row" from "the last row". It is deliberately not framed by the labels (fit_to_labels leaves it alone) and its batch position is not load-bearing the way RowFrame-before-Cells is, since it addresses no position; the producer reads GridLayout::row_count(), and reading it off the projected ranges is invisible in the TUI (which builds at offsets zero, gotcha 44) and is exactly the browser bug. The window is the client's own fact, recorded by Client::set_viewport from what it announced — not derivable, since the row frame's own length is the window clipped to the grid and so differs precisely at the bottom edge where the scroll arithmetic matters. Client::viewport (associated, no recording) still exists and is the one that quietly turns cursor keys off; web/app.js does call set_viewport. ClientFrame::Commands(Vec<Command>) replaces the singleton, because one cursor key is SetSelected plus the offsets viewport_effects returns and a host that folds the selection but not the offset projects the wrong window. It is answered whole, then applied whole (improvise-955n): CheckedBatch::against checks the frame before anything runs, so there is no longer a moment at which part of a frame has landed and the rest has been refused, and Applied::Rejected is unreachable through Host::commands — still matched, because a host that wants the Dirty must name every case (gotcha 39). The contract is four clauses and clause 4 is not delivered: one basis per frame, checked once, applied in order with each command seeing the last one's state, and all-or-nothing. The fourth is open because App::apply_effects aborts without undoing (gotcha 21 — an abort is not a rollback), so [RemoveItem("North"), SetCell(the North record)] lands the removal, resolves the write to nothing, and reports the frame accepted having applied half of itself — partial and silent. It is unreachable from any client in this workspace, the only multi-command producer being keys::commit; reachability is a reason not to rush a fix and not a reason not to notice one — the lost update CheckedBatch fixed had the same reachability argument and none of the documentation, which is how it shipped.

    +

    Two corrections to the ticket's own gap list, both worth more than the code. Panel cursor movement was never a gap: improvise-edp had already made viewmodel::flatten_cat_tree the one home both ends call, so it is the shared derivation, not a second one — worth re-checking the "known gaps" in other old tickets against what has landed since. And the "unmodified characters only" rule was mis-scoped, not wrong: it was about KeyPattern::AnyChar, which is bound only in the text-entry modes, so Ctrl-d in Normal has an exact binding and mirroring it is a faithful subset. Each resolver mirrors its Cmd line for line, including where siblings disagree — Move and MoveTileCursor emit even when they clamp, MovePanelCursor returns nothing when the cursor does not move. Tidying that into one rule would make the client a second derivation, which is what tests/lockstep.rs exists to rule out. What still needs the host is a named list in keys.rs and in the_normal_mode_keys_that_still_need_the_host_are_these, so closing one has to come there and say so.

  94. +
  95. The browser JavaScript is argued and now partly executed, and the two rules it can break are both about when. web/app.js and web/worker.js were syntax-checked, .d.ts-checked for call arity and read — never run — and every bug in them was found by reading. harness/tests/shell.spec.js is the first execution of either: it runs app.js under Chromium against a mocked peer, and it exists because PR #21 declined to add regression tests for its own close-listener fix on the ground that there was nowhere to put them. That decline is only honest because the assertions live there now, and a change to web/*.js belongs in that spec directory rather than in a second ad-hoc surface beside it. Playwright rather than a headless-DOM shim because what goes wrong in these files is a real Worker, a real WebSocket, real OPFS and real event ordering, none of which jsdom has. Two classes of bug still follow from when, both found by review. An async message listener does not serialize. worker.js's handle() awaits server.flush(), so the event loop may start the next message while an earlier one is suspended, interleaving two on_message/flush pairs and letting a later frame's replies overtake an earlier frame's — which gotcha 33 answers by dropping the overtaken update, not by replaying it. Every frame is therefore chained onto one tail promise and the listener is deliberately not async; that also turns "frames that arrived before start() finished" into an ordinary case of the ordering rule instead of a drain loop, which had the same race twice over because server was assigned before the loop ran. A watchdog must be disarmed, not pre-empted. The handshake watchdog exists for one condition — the peer never answered — and every loud failure (worker error/messageerror, socket error) has already written something more precise; the timer then fired 8s later and replaced it, so setShellFailure clears it. Its neighbour, the socket's close listener overwriting a precise error with "disconnected", is fixed (improvise-taxx, PR #21) — and the ticket's own proposed mechanism was wrong: readyState cannot discriminate, an opened flag is the only thing that can. PR #21's other half is the one that keeps getting lost: it routes both close arms through setShellFailure, which disarms the watchdog, so a genuine mid-session disconnect is not overwritten eight seconds later. A bug report said it was, measured it honestly against a pre-#21 tree, and then "confirmed" it survived #21 by simulating the opened flag and not the routing — a simulated fix that is not the fix you named reproduces a bug the real code does not have, and reads exactly like a confirmation. The ticket was filed and closed invalid; the property is now asserted by a socket that opened and then dropped reads as disconnected, and stays that way, which runs past the 8s window, rather than argued in a comment that was wrong twice.

    +

    What the harness still cannot see, and why the argued-not-tested discipline stays: the grouped and multi-level DOM, the CSS, and the ResizeObserver extent arithmetic. What a careful read established about the last of these — the observer cannot feed back into itself (#grid is a flex child with overflow: auto, so replaceChildren cannot resize it) and ROW_PX/COL_PX under-estimate, so the worst case is a row of over-fetch — is still a read and not a test.

    +

    A third thing no test here can see, which is why the docs must be read as code. Three comments in one range said "not built yet" about work that had landed in that same range — README.md's browser section, app.js's keyboard note, and Host::handle's Key arm — each because a follow-up commit updated one copy of an argument and not its siblings. When a fact is stated in more than one place, changing it is a grep, not an edit.

    +

    An evidence heading may be module-qualified, and must be when its leaf is not unique. scripts/check-comment-runs.py --check-evidence takes its Rust names from cargo test --workspace -- --list minus the --ignored twin, as full module paths, and refuses an ambiguous leaf naming every candidate rather than warning. Six leaf names collide in this workspace. Note neither half suffices alone: rename one of a colliding pair and the leaf is unique again and resolves silently to the survivor, so qualification is the fix and the refusal is only what forces someone to write it (improvise-rm4s). The residual, filed: --list does not name the binary, so two crates sharing a module path are indistinguishable (improvise-sxsf) — measured latent (1444 names, zero collisions) and the proposed per-package fix declined twice over: 18× the cost (45.2s against 2.4s, and --check-evidence needs the --ignored twin, so 22 cargo invocations), and a package is not a binary — improvise alone has five integration test binaries beside its lib, and an integration test's top-level fn lists with no module path at all (60 of 1444 names are bare), so -p improvise prefixes all six identically. Per-target is the only shape that would close it.

  96. +
  97. A websocket handshake is exempt from the same-origin policy, and that is a server-side problem. There is no CORS preflight, so any page a user visits can open a socket to improvise-ws-server from that user's own allowed address and read and rewrite every document it holds — cross-site websocket hijacking (improvise-kc7n). No network control can see it: an IP allow-list, a private network and loopback binding all answer "may this client reach the endpoint", and the honest answer is yes; only Origin says what is driving the client. Loopback is the least protected case, not the safest — no proxy in front of it, and loopback is not a boundary a browser enforces. src/origin.rs is the whole decision and transport::admitted its one caller; it refuses the upgrade with a 403 rather than accepting a socket and ignoring it, and serve takes the policy so a host cannot forget to apply it. Three arms are decided rather than defaulted: loopback allowed (the shipped local recipe serves its page from http://localhost:8080, so the one case worth optimising for must need no flag), absent allowed (no browser omits Origin, so absent is a non-browser client an address check already answers — and refusing it breaks tests/socket.rs, the repo's own instrument), and two refused (HeaderMap::get is first-wins, so a proxy that appends rather than sets leaves two, and answering from the first lets the second be anything). The deployed Pages origin is deliberately not hard-coded: it can only reach a plain-ws:// server through a TLS proxy anyway, so one flag is an honest cost on an already multi-step path where a baked-in default would be a permanent hole for every other deployment. The residual hole is stated rather than guarded — a proxy that strips Origin turns the check off, which is why the proxy's own allow-list stays in README.md as defence in depth. None of this is authentication: it answers what page is driving a client, never who is connecting, and improvise-kpnr is still open. It correctly does not move to improvise-command beside Host (gotcha 45) — a MessagePort has no handshake and no headers, and a worker and its page are the same origin by construction, so this is genuinely transport-only and not a mirror waiting to drift.
  98. +
+
+

#File Inventory

+

Line counts are a dated snapshot, not a maintained fact: measured 2026-09-21, and 26 of 61 had drifted past 10% before that refresh — several by a third. Treat them as "is this file big", never as a figure to cite; re-derive with wc -l. Note the ws-server/ and improvise-worker-server/ entries use a shorthand prefix that no path-walking refresh resolves, so they drift silently and must be checked by hand. Test counts are informational — run cargo test --workspace for live numbers. Files under 100 lines and render-only widgets omitted.

+

#improvise-core (crates/improvise-core/src/)

+
model/types.rs         2352 / 74t   Model, formula eval (fixed-point), CRUD
+model/cell.rs          1234 / 43t   CellKey (sorted), CellValue, DataStore (interned + index),
+                                    live_records (insertion order — gotcha 18)
+model/record.rs         326 /  8t   RecordHash / CellId surrogates: the opaque identities a
+                                    thin client names a cell by (ADR 002)
+model/category.rs       316 /  9t   Category, Item, Group, CategoryKind
+model/symbol.rs          79 /  3t   Symbol interning
+view/layout.rs         1251 / 30t   GridLayout, drill, records mode, records_columns
+view/types.rs           711 / 36t   View config (axes, pages, hidden, collapsed, format)
+view/axis.rs            205 /  5t   Axis enum + label/color/next methods, AxisColor
+workbook.rs             345 / 17t   Workbook: Model + cross-view ops, item facade
+format.rs               229 / 29t   format_f64, parse_number_format (display only)
+

#improvise-formula (crates/improvise-formula/src/)

+
parser.rs              1087 / 67t   pest grammar + tokenizer → Formula AST
+quote.rs                192 /  7t   quote_name / is_bare_name / pipe_quote / unescape_pipe_body:
+                                    the one home, both halves (improvise-i2m5). improvise-gp
+                                    re-exports it; improvise-io calls it from
+                                    persistence/{format,parse}.rs
+formula.pest            103         PEG grammar for formula text
+ast.rs                   77         Expr, BinOp, AggFunc, Formula, Filter
+

#improvise-io (crates/improvise-io/src/)

+
storage.rs               388 / 13t   Storage trait (read/write/list/delete), PhysicalFs, MemoryFs,
+                                     `impl Storage for Rc<T>` (improvise-hmws), and
+                                     contract::storage_contract — public, not cfg(test), because
+                                     a browser must run that sequence (harness/)
+opfs.rs                  367 /  0t   OpfsFs: Storage over the browser's Origin Private File
+                                     System. wasm32 only. Run by harness/, not by cargo test
+persistence/improv.pest 153         PEG grammar — single source of truth
+persistence/mod.rs      403 / 10t   document lifecycle: save/load/gzip/legacy-JSON over Storage;
+                                     facade re-exporting format_md/parse_md/export_csv*
+persistence/format.rs   385 / 11t   format_md: Workbook → .improv text
+persistence/parse.rs   1519 / 43t   parse_md: pest tree-walk → Workbook (+ grammar generator)
+persistence/export.rs   219 /  4t   export_csv / render_csv. Builds its layout with
+                                    with_frozen_records, NOT new — `new` skips prune_empty
+                                    (improvise-iot6)
+persistence/roundtrip_tests.rs
+                        965 / 45t   format_md ↔ parse_md characterization (tests only)
+import/wizard.rs       1714 / 55t   ImportPipeline + ImportWizard
+import/analyzer.rs      403 / 13t   Field kind detection (Category/Measure/Time/Skip);
+                                    analyze_records is first-seen order, not HashSet (gotcha 18)
+import/csv_parser.rs    495 / 20t   CSV parsing, multi-file merge
+

#improvise-protocol (crates/improvise-protocol/src/)

+
lib.rs                   561 /  1t  ViewState, ViewCursor, ViewFrame, AppMode (17), Panel,
+                                    MinibufferConfig, MinibufColor, HELP_PAGE_COUNT
+reduce.rs                569 /  9t  Command + ModelCommand (the wire format), HelpPage, reduce_view
+render.rs               2008 / 35t  CellDisplay, WidgetCache trait + one cache per widget
+                                    (GridCache/CategoryTreeCache/FormulaCache/ViewListCache/
+                                    TileBarCache), AxisFrame/FrameEntry/FrameGroup and
+                                    header_level_widths (the grid frame, gotcha 44),
+                                    GridCache::seed (the bootstrap compaction), Projection,
+                                    bootstrap-only Snapshot
+keymap.rs                957 /  0t  KeyPattern, Binding, Keymap, ModeKey (17), TextEntry,
+                                    PanelKeymap, Keymaps::default_keymaps/prefix_keymap — the
+                                    shared table both front ends look up. Tests live with
+                                    dispatch in improvise-command/src/command/keymap.rs
+viewmodel.rs             168 /  5t  flatten_cat_tree — derivations between a cache and a
+                                    renderer, called by both front ends (improvise-edp)
+key.rs                   101 /  3t  KeyCode, KeyModifiers, KeyEvent (moved from command/)
+frame.rs                 330 /  3t  ClientFrame/ServerFrame — the ONE envelope pair, both
+                                    deployments (improvise-xks6). Nothing else defines one.
+                                    ClientFrame::Save is a request, not a Command (gotcha 45)
+basis.rs                 332 /  5t  Basis (the three tiers), Rejection, BasisCache — ADR 001
+                                    layer 1. Basis has no Default and no public constructor
+help.rs                  925 /  7t  HelpPage content and Hint { keys: Vec<(KeyCode, KeyModifiers)>,
+                                    command, caption }. `compact` drops a chord by its MODIFIERS
+                                    (anything beyond shift), never by parsing a label it just
+                                    formatted (improvise-thpe) — the guard is the signature
+resolve.rs              2477 / 46t  the browser's key resolvers: a BrowserKey to the Commands it
+                                    composes, mirroring each Cmd line for line INCLUDING where
+                                    siblings disagree (gotcha 47). The largest file in the crate
+client.rs                255 /  5t  ClientState: ViewState + one cache per widget, and the fold
+

#gp-tree / improvise-gp (crates/gp-tree/, crates/improvise-gp/)

+
gp-tree/src/lib.rs          669 /  8t  Tree, Spec, eval, simplify, render, breed, Population,
+                                       SplitMix64 (vendored seeded Rng; seed() keeps the seed so
+                                       a fit can be shown and recorded, pinned to the published
+                                       reference vectors)
+gp-tree/src/data.rs         938 / 19t  raw string rows -> f32 feature vectors: per-column type
+                                       inference, one-hot, mean-imputation + col.missing flag,
+                                       ColumnSpec/encode_row, Dataset (incl. warnings),
+                                       Model = trees + plan + seed + FitConfig
+gp-tree/src/fit.rs          970 / 25t  the search: FitConfig (Default = the CLI's numbers), fit()
+                                       -> simplified champion, cv_regress/cv_classify,
+                                       permutation_importance, folds/standardize/mse.
+                                       Returns data; never prints
+gp-tree/examples/fit_csv.rs 585        CLI over the above: flags, demo dataset, all printing
+gp-tree/tests/scoring_known_answers.rs
+                            285 /  6t  f32 bit patterns + a next_u64 witness: the evidence any
+                                       change to the scoring paths needs (gotcha 37c)
+improvise-gp/src/lib.rs     844 / 21t  formula_text / fitted_formula: a Tree as IF(...) formula
+                                       text, validated back through parse_formula. A threshold is
+                                       the SHORTEST decimal that reads back as the same f64 (not
+                                       the exact expansion), so a row on a split boundary routes
+                                       the same in f64 as in f32 eval; non-finite values panic
+improvise-gp/src/dataset.rs 632 / 15t  a View's records -> the feature matrix a fit consumes —
+                                       the improvise-side counterpart of preprocess. Records mode
+                                       is already a feature matrix (gotcha 1)
+improvise-gp/src/fit.rs    1426 / 23t  one fit end to end: a View's records in, an improvise
+                                       Formula and the evidence for believing it out. Not in a
+                                       command, because none of it is terminal-specific
+

#Command layer (crates/improvise-command/src/command/)

+
cmd/core.rs             333 /  2t   Cmd trait, CmdContext, CmdRegistry, parse helpers
+cmd/registry.rs         822 /  3t   default_registry() — all registrations
+cmd/navigation.rs       502 / 12t   Move, EnterAdvance, Page*
+cmd/cell.rs             194 /  6t   ClearCell, YankCell, PasteCell, TransposeAxes, SaveCmd
+cmd/commit.rs           739 / 11t   CommitAndAdvance, CommitBuffer(CommitKind), CommitExport,
+                                    CommitSave
+cmd/effect_cmds.rs      459 /  5t   effect_cmd! macro, 25+ simple wrappers
+cmd/fit.rs              280 /  7t   fit-measure: evolve a tree over the active view and store it
+                                    as an ordinary `## Formulas` line — nothing GP-specific is
+                                    left in the document afterwards
+cmd/grid.rs             540 /  9t   ToggleGroup, ViewNavigate, DrillIntoCell, TogglePruneEmpty
+cmd/mode.rs             414 / 11t   EnterMode, Quit, EditOrDrill, EnterTileSelect
+cmd/panel.rs            688 / 17t   Panel toggle/cycle/cursor, formula/category/view panels
+cmd/search.rs           206 /  4t   SearchNavigate, SearchOrCategoryAdd, ExitSearchMode
+cmd/text_buffer.rs      382 / 14t   AppendChar, PopChar, CommandModeBackspace, ExecuteCommand
+cmd/tile.rs             138 /  5t   MoveTileCursor, TileAxisOp
+cmd/wizard.rs           244 /  6t   HandleWizardKey — per-step import-wizard key tables
+keymap.rs               917 / 35t   KeymapSet, Keymap::dispatch/dispatch_transient, effects_of,
+                                    SetTransientPrefix — what a thin client cannot have. The
+                                    table itself is improvise-protocol/src/keymap.rs
+parse.rs                232 / 19t   Script/command-line parser (prefix syntax)
+

#Semantic UI state (crates/improvise-command/src/ui/)

+
ui/effect.rs           2731 / 64t   Effect trait, 50+ effect types
+ui/app.rs              3407 / 92t   App, ModelState, ViewState + dispatch methods, Panel, handle_key, dispatch_key, document_generation, autosave; apply_effects raises `dirty` off EffectKind::Model(_) (gotcha 45)
+ui/project.rs          3708 / 55t   Dirty, Viewport, grid_of/category_rows/formula_texts/
+                                    view_summaries/tile_entries, grid_updates (the diff),
+                                    ClientSession + Sessions (connect/apply/rebase/project_all)
+ui/reduce.rs           1411 / 18t   Checked / CheckedBatch / apply_command / model_effects /
+                                    reduce_full / reduce_key — one door from a Command or a
+                                    keypress to the authoritative App
+ui/host.rs             1493 / 24t   Host: one App + one Sessions; handle, commands, project_all,
+                                    rebase. Lives here, not in ws-server (gotcha 45)
+ui/cat_tree.rs          184 /  8t   Category tree flattener for panel
+ui/viewport.rs          124         compute_col_widths / compute_row_header_width / visible_cols
+                                    (pure geometry; the width rule itself is the protocol's)
+

#TUI rendering, draw, main (src/ui/, src/draw.rs, src/main.rs)

+
ui/grid.rs              985 / 14t   GridWidget (ratatui): no &Model, no &View — the frame comes
+                                    off GridCache (gotcha 44). Its module doc is the best-written
+                                    statement of that rule in the repo
+ui/help.rs              603         5-page help overlay (render only)
+ui/import_wizard_ui.rs  947 /  9t   Import wizard rendering
+draw.rs                 599 /  5t   TUI event loop, frame composition
+main.rs                 393         CLI entry (clap): open, import, cmd, script
+# other ui/*.rs are small panel renderers — skip unless changing layout/style
+tests/grid_frame_agreement.rs
+                        422 /  4t   the two front ends draw the same grid frame (improvise-anr9)
+tests/records_wide_smoke.rs   67 / 1t
+tests/shipped_documents_load.rs 42 / 1t
+tests/web_demo_model.rs       80 / 2t   every visible cell in web/demo.improv is a Record
+

#Browser host and client (crates/improvise-ws-server/src/, crates/improvise-wasm-client/src/, web/)

+
ws-server/registry.rs    595 / 14t  Registry of Hosts keyed by DocumentId; ClientId {document,
+                                    session} — a SessionId is unique only within its document.
+                                    A document is never evicted while the server runs (ADR 001)
+ws-server/host.rs         23 /  0t  a re-export shim only. The session loop is
+                                    improvise-command/src/ui/host.rs (gotcha 45)
+ws-server/origin.rs      132 /  6t  OriginPolicy: which origins may open a socket. Loopback by
+                                    default, --allow-origin widens, absent allowed, two refused
+                                    (improvise-kc7n). The only decision transport::admitted makes
+ws-server/transport.rs   324 /  4t  tokio + tungstenite plumbing, plus the Origin check at the
+                                    handshake — a refused upgrade, not an ignored socket
+ws-server/main.rs        190 /  5t  CLI entry: serve a file on a port; --bind, --allow-origin
+ws-server/tests/socket.rs 284 / 2t  the one test that uses a real socket: handshake, JSON
+                                    framing, SessionId routing, which document a URL asked for
+wasm-client/lib.rs       333 /  7t  Client: the wasm-bindgen surface (init, on_message, on_frame,
+                                    viewport/set_viewport, render_model, set_cell_at, typed_cell)
+wasm-client/keys.rs      187 /  5t  BrowserKey ONLY — a KeyboardEvent flattened to an improvise
+                                    key. It does NOT resolve: `resolve` and KeyOutcome live in
+                                    improvise-protocol/src/resolve.rs, shared by both front ends
+wasm-client/render.rs    888 / 12t  RenderModel: editable / selected / cat tree, decided here
+wasm-client/socket.rs    289 /  7t  the fold (fold_frame/handle_frame), the two compositions
+                                    and typed() text→CellValue. Defines no enums
+# state.rs is GONE — ClientState moved to improvise-protocol/src/client.rs
+wasm-client/tests/browser_round_trip.rs  384 / 5t  improvise-1ey's scope against a real Host
+wasm-client/tests/lockstep.rs           1126 / 23t host and client agree on a keypress
+web/app.js               496         string calls only, no protocol knowledge, no framework.
+                                     Run under Chromium by harness/tests/shell.spec.js — see
+                                     gotcha 48 for what that does and does not reach
+web/worker.js             64         three string calls, no protocol knowledge. RUN by
+                                     shell.spec.js's no-?ws path, with bindings.rs (gotcha 45)
+web/index.html, style.css            the page; legibility only
+web/build.sh              38         wasm-pack build → web/pkg (gitignored). Needs
+                                     wasm-opt = ["-O", "--enable-bulk-memory"] or the bundled
+                                     wasm-opt rejects LLVM's memory.copy
+web/assemble.sh           120         assembles the deployable site (copies web/*.improv)
+web/demo.improv           57         the document the site ships and seeds from. Every cell a
+                                     Record — asserted (gotcha 46)
+

#Browser worker host (crates/improvise-worker-server/)

+
improvise-worker-server/src/worker.rs           738 / 13t  Worker: one port, one client; handshake, Addressed
+                                        filter, persist trigger (model_state.dirty). Every decision, every test
+improvise-worker-server/src/vfs.rs              311 /  6t  documents/open: seeding is a Storage::write
+improvise-worker-server/src/bindings.rs         158 /  0t  wasm32 shim: OpfsFs awaits + postMessage. No Rust test
+                                        reaches it, but harness/tests/shell.spec.js DOES: the
+                                        no-?ws path runs start()/OpfsFs::open under Chromium.
+                                        Called NEVER RUN here for months: see gotcha 45
+improvise-worker-server/src/lib.rs              114 /  1t  the module doc that states exactly what is and is not
+                                        covered — the one correct copy of that fact
+improvise-worker-server/tests/post_message.rs   381 /  7t  main-thread traffic, strings only
+

#Browser harness (harness/) — not a workspace member

+
harness/playwright.config.js     50   shell.spec.js runs on CHROMIUM ONLY (engine-independent
+                                      logic; routeWebSocket is slower and flakier across three).
+                                      opfs.spec.js runs chromium + firefox + webkit, and webkit
+                                      provides NO OPFS at all, so it skips on a runtime probe
+harness/tests/shell.spec.js     165   the first execution of web/app.js. A change to web/*.js
+                                      belongs here (gotcha 48)
+harness/tests/opfs.spec.js       79   drives the OPFS contract page and reads back its result
+harness/opfs-contract/src/lib.rs 361  runs improvise_io::storage::contract::storage_contract
+                                      against a real OpfsFs inside a real Web Worker
+harness/www/opfs/{index.html,worker.js}  the page and worker that host it
+harness/build.sh                 35   wasm-pack build for the contract crate
+# gated by the `browser` job in .github/workflows/ci.yml, which can fail the build
+

#Examples

+
examples/gen-grammar.rs             Grammar-walking random .improv generator (pest_meta)
+examples/pretty-print.rs            Parse stdin, print formatted .improv
+

#Context docs

+
.claude/skills/code-review/         Review skill: checks a change against the doctrine in these docs + kb.pl
+scripts/check-comment-runs.py       The two-line comment cap AND --check-evidence. The flag is
+                                    ADDITIVE since improvise-ijql — every invocation that can
+                                    exit 0 answers the cap — guarded by a subprocess self-test,
+                                    the only level a dispatch bug is visible at. Reasoning:
+                                    context/code/tooling.md §2
+scripts/check-ci-graph.py           pages-deploy must `needs` every job in ci.yml. Carries its
+                                    own --self-test; run in the `test` job. README states the
+                                    rule and deliberately lists no jobs (improvise-x7cq)
+scripts/assemble-stamp.sh           The cache-bust stamp gate web/assemble.sh calls, extracted
+                                    so its refusals can be exercised without a web/pkg
+scripts/assemble-stamp-test.sh      Its check_the_checker, run first in the `build` CI job
+context/code/tooling.md             The repository's own checkers: why an evidence heading's
+                                    names come from cargo rather than from the source, and
+                                    why a leaf name is not an identity
+context/test-evidence/tooling.md    Evidence for the gates in scripts/ and for the tests that
+                                    hold CI's shape
+scripts/nix-seed-flake-inputs.sh    Seeds locked flake inputs over git so `nix develop` works in cloud sessions
+prolog-sessions/kb.pl               Executable knowledge base: invariants, doctrine, decisions (CI-checked)
+context/adr/003-live-backed-models.md A model may be backed by a live query source —
+                                    improvise as a DB client. Accepted, NOT a priority.
+context/adr/002-naming-a-cell.md     How a client names a cell: an opaque surrogate on
+                                    CellDisplay, whose absence means the cell is an
+                                    aggregate. Records why _Index cannot serve.
+context/adr/001-concurrent-edits.md  Concurrent edits between clients: record-level atomicity,
+                                    optimistic basis check per batch, advisory reservations.
+                                    Records why a causal clock was rejected.
+context/cloud-agents.md             Cloud-session environment: tracker sync, bd pitfalls, what is blocked
+context/design-principles.md        Architectural principles & testing doctrine
+context/plan.md                     Show HN launch plan
+context/repo-map.md                 This file
+docs/design-notes.md                Product vision & non-goals
+
+

#Wasm

+

The browser bundle is improvise-core, improvise-formula, improvise-io, improvise-protocol, improvise-command, improvise-wasm-client and improvise-worker-server. All seven cargo check --target wasm32-unknown-unknown clean with no feature flags (improvise-ywd), gated by the wasm job in .github/workflows/ci.yml. That job gates eight crates, not seven: improvise-gp was added (improvise-z5eg) explicitly not as a bundle member but as a candidate, because staying wasm-clean is free now and expensive to retrofit — so the bundle is seven and the gate is eight, and neither number is the other one's typo. A feature that adds a dependency to a bundle crate must be checked with cargo tree --edges normal — the default output includes dev edges, so a plain cargo tree -p improvise-protocol --target wasm32-unknown-unknown reports a dev-only serde_json as present and reads exactly like a leak. Check per-crate too, because the wasm CI job invokes each crate separately and feature unification cannot reach across that (improvise-26dm added a test-support feature on improvise-protocol on those terms; it is off in every shipped build).

+

improvise-ws-server is the host and is native only — tokio, tungstenite and Box<dyn Storage> — so improvise-wasm-client cannot depend on it in anger. It no longer needs to: both ends pub use improvise_protocol::frame and neither defines a pair (gotcha 42). The dev-dependency remains, for tests/browser_round_trip.rs, which hands the client's own serialized frame to a real Host::handle. The root improvise crate is the terminal binary and does not build for wasm — it depends on crossterm; that is correct, not a gap.

+

cargo check passing is not the same as running. Two known runtime hazards it cannot see: Instant::now() panics on wasm (hence web-time in improvise-command), and any std::fs call compiles but fails in a browser — see improvise-fuj for the two effects that still bypass Storage.

+

#CLI

+
improvise [model.improv]                          # open TUI (default)
+improvise import data.csv [--no-wizard] [-o out]  # import CSV/JSON
+improvise cmd 'add-cat Region' -f model.improv    # headless command(s)
+improvise script setup.txt -f model.improv        # run script file
+

Import flags: --category, --measure, --time, --skip, --extract, --axis, --formula, --name.

+
+

#Testing — the short version

+

Full guidance lives in context/design-principles.md §7. Quick reminders:

+ +
+

#Key dependencies

+

ratatui 0.30, crossterm 0.29, clap 4.6 (derive), serde/serde_json, indexmap 2, anyhow, chrono 0.4, pest + pest_derive, flate2 (gzip), csv, enum_dispatch. Dev: proptest, tempfile, pest_meta.

+
+
+ + diff --git a/site/docs/style.css b/site/docs/style.css new file mode 100644 index 00000000..4e8af207 --- /dev/null +++ b/site/docs/style.css @@ -0,0 +1,60 @@ +:root{--bg:#14161a;--panel:#1b1e24;--fg:#d7dae0;--dim:#8b93a1;--accent:#8ec07c;--rule:#2b2f37;--mono:ui-monospace,"SFMono-Regular","Menlo","Consolas",monospace} +*{box-sizing:border-box} +body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.65 system-ui,-apple-system,"Segoe UI",sans-serif} +a{color:var(--accent)} +.shell{display:flex;align-items:flex-start;max-width:1400px;margin:0 auto} +nav.side{position:sticky;top:0;flex:0 0 17rem;max-height:100vh;overflow-y:auto;padding:1.5rem 1rem 3rem;border-right:1px solid var(--rule);font-size:.9rem} +nav.side .brand{font-family:var(--mono);font-size:1.15rem;color:var(--accent);text-decoration:none;font-weight:600} +nav.side .navtoggle{display:none} +nav.side .navsummary{display:none;cursor:pointer;margin:.9rem 0 .2rem;padding:.4rem .55rem;border:1px solid var(--rule);border-radius:4px;font-family:var(--mono);font-size:.85rem;color:var(--dim)} +nav.side .navsummary::before{content:"\2630 " / ""} +nav.side .navtoggle:checked~.navsummary{border-color:var(--accent);color:var(--fg)} +nav.side h3{font-size:.72rem;letter-spacing:.09em;text-transform:uppercase;color:var(--dim);margin:1.6rem 0 .4rem} +nav.side ul{list-style:none;margin:0;padding:0} +nav.side li{margin:.15rem 0} +nav.side a{display:block;padding:.2rem .45rem;border-radius:4px;color:var(--fg);text-decoration:none} +nav.side a:hover{background:var(--panel)} +nav.side a.here{background:var(--panel);color:var(--accent);font-weight:600} +main{flex:1 1 auto;min-width:0;max-width:60rem;padding:2.5rem 2rem 6rem} +h1,h2,h3,h4,h5{font-family:var(--mono);font-weight:600;letter-spacing:-.01em;line-height:1.3;overflow-wrap:anywhere} +h1{font-size:2rem;margin:0 0 1.2rem;color:var(--accent)} +h2{font-size:1.25rem;margin:2.6rem 0 .6rem;padding-top:1.2rem;border-top:1px solid var(--rule)} +h3{font-size:1.02rem;margin:1.8rem 0 .4rem;color:#c8cdd6} +h4,h5{font-size:.95rem;margin:1.3rem 0 .3rem;color:var(--dim)} +.anchor{float:left;margin-left:-1.1em;padding-right:.35em;color:var(--rule);text-decoration:none;opacity:0} +h2:hover .anchor,h3:hover .anchor,h4:hover .anchor,h5:hover .anchor{opacity:1} +p{max-width:74ch} +li{max-width:74ch} +code{font-family:var(--mono);font-size:.88em;background:var(--panel);padding:.1em .3em;border-radius:3px;overflow-wrap:anywhere} +a code{background:#22262e} +pre{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:1rem;overflow-x:auto} +pre code{background:none;padding:0;font-size:.84rem;line-height:1.5;overflow-wrap:normal} +blockquote{margin:1rem 0;padding:.5rem .9rem;border-left:3px solid var(--accent);background:var(--panel)} +blockquote p{margin:.4rem 0} +table{border-collapse:collapse;margin:1rem 0;font-size:.92rem;display:block;overflow-x:auto} +th,td{border:1px solid var(--rule);padding:.4rem .6rem;text-align:left;vertical-align:top} +th{background:var(--panel)} +hr{border:0;border-top:1px solid var(--rule);margin:2rem 0} +.tid{font-family:var(--mono);font-size:.86em;white-space:nowrap} +.tid-unknown{color:var(--dim);border-bottom:1px dotted var(--rule)} +.toc{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.8rem 1.1rem;margin:0 0 2rem;font-size:.92rem} +.toc summary{cursor:pointer;color:var(--dim);font-family:var(--mono)} +.toc ul{list-style:none;margin:.6rem 0 0;padding:0;columns:2;column-gap:2rem} +.toc li{break-inside:avoid;margin:.12rem 0;overflow-wrap:anywhere} +.toc li.l3{padding-left:1rem;font-size:.92em} +.toc a{text-decoration:none;overflow-wrap:anywhere} +.toc a:hover{text-decoration:underline} +.jump{margin:0 0 2rem;font-size:.85rem;font-family:var(--mono)} +.jump a{display:inline-block;min-width:2.1rem;text-align:center;padding:.12rem .3rem;margin:.1rem;border:1px solid var(--rule);border-radius:4px;text-decoration:none} +.jump a:hover{border-color:var(--accent)} +.lede{color:var(--dim);max-width:70ch} +.src{font-size:.85rem;color:var(--dim);font-family:var(--mono);margin:-.6rem 0 1.8rem} +.cards{display:grid;gap:1rem;grid-template-columns:repeat(auto-fill,minmax(19rem,1fr));margin:1rem 0 2rem} +.card{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.9rem 1.1rem} +.card a{font-family:var(--mono);text-decoration:none} +.card p{color:var(--dim);font-size:.88rem;margin:.3rem 0 0} +.tickets{width:100%} +.tickets td:first-child{white-space:nowrap;font-family:var(--mono);font-size:.86rem} +.tickets td:last-child{overflow-wrap:anywhere} +.st{font-size:.78rem;color:var(--dim);text-transform:uppercase;letter-spacing:.05em} +@media(max-width:900px){.toc ul{max-height:45vh;overflow-y:auto}nav.side .navtoggle{display:block;position:absolute;width:1px;height:1px;margin:-1px;clip-path:inset(50%);overflow:hidden}nav.side .navtoggle:focus-visible~.navsummary{outline:2px solid var(--accent);outline-offset:2px}nav.side .navsummary{display:block}nav.side .navlinks{display:none}nav.side .navtoggle:checked~.navlinks{display:block}.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}} \ No newline at end of file diff --git a/site/docs/test-evidence/clients.html b/site/docs/test-evidence/clients.html new file mode 100644 index 00000000..53575fcd --- /dev/null +++ b/site/docs/test-evidence/clients.html @@ -0,0 +1,521 @@ + + + + + +The clients — improvise docs + + + +
+ +
+

Source: context/test-evidence/clients.md

On this page

Test evidence: browser client, hosts, harness

+

What each of these tests proves, what fault was injected to prove it can fail, and — where it applies — the bug it was written against. Keyed by exact test name; a renamed test orphans its entry, which scripts/check-comment-runs.py --check-evidence reports.

+

The design argument each of these belongs to is in context/code/clients.md.

+
+

#improvise-worker-server/src/worker.rs

+

#an_edit_is_persisted_without_the_client_asking_to_save

+

The persistence half of improvise-9x6's acceptance criterion 2, and the reason the worker saves on its own behalf: there is no ModelCommand::Save for a client to send, because effect::Save is ViewOnly and the wire vocabulary is derived from the model effects. So an edit must reach storage with the client having asked for nothing but the edit. The test writes one cell over the wire and reopens the document through the same VFS.

+

#a_partly_refused_batch_does_not_leave_disk_and_memory_disagreeing

+

improvise-955n's second-order effect, which nobody had written down until it was found while declining the ticket: on_frame skipped persist() when any frame in the answer was Rejected, so a batch that did apply its earlier command before being refused left the document moved in memory and not written back — disk and worker disagreeing, silently.

+

Two changes close it independently, which is why this is worth a test of its own rather than trusting either: the batch no longer applies partially at all (improvise-955n), and the persist trigger is now the document's own dirtiness rather than "was anything rejected" (improvise-gv9r), so a batch that did change something would be written whatever else the answer contained.

+

The fixture is a refused frame with a command behind the refusal, and the assertion is that the axes on disk and the axes in memory agree. Under the settled contract the refusal comes from the frame's basis — answered once, on the first command that carries one — so that is where the staleness is injected; a later command carries no answer of its own.

+

#an_explicit_save_is_not_retried_by_the_automatic_write

+

Found by Copilot on PR #26. A failed write leaves model_state.dirty set — deliberately, so the next frame retries — and the automatic trigger runs after every frame. A save frame therefore attempted the same write twice: extra I/O on a permanent failure, and on a transient one something worse, because the answer is composed from the first attempt, so the client is told SaveFailed about a document that is now on disk.

+

The fix is a per-frame question rather than a per-document one, and that is the right shape precisely because the automatic write exists for edits nobody asked to save; one that was asked for has already been answered.

+

The fixture is a Storage backend whose writes always fail and that counts them. Counting is the point: "it was refused" is true whether the write was attempted once or twice, so a test asserting only the refusal would pass against the bug. Failing is the point too: a successful save clears dirty, so the redundant second attempt never happens and the bug is invisible from the happy path.

+

#a_key_resolved_on_the_host_moves_state_no_client_can_hear

+

Why the Key arm answers nothing, demonstrated rather than asserted in prose.

+

A worker can call reduce_key — it holds the authoritative App, which is the thing a socket server lacks. What neither has is a way to tell a client that the host's own ViewState moved: a client's slice advances only by folding Commands, and resolving a keypress produces none. So the two slices diverge, and ClientSession::viewport goes on reading the mirror's cursor, which means the grid does not even follow. That is improvise-bpq2's real cost, and it is a ticket rather than plumbing. The test asserts all three: the authoritative cursor moved, the mirror did not, and nothing reduce_key emits would let a client catch up.

+
+

#improvise-worker-server/src/vfs.rs

+

#an_entry_that_holds_no_document_is_seeded_over

+

The first browser run of the deployed page failed here, and no host test could see it: attach(create: true) leaves a zero-byte entry behind, documents listed it, and open therefore took the seed as already applied and read an empty file. The precondition is asserted before the act, because a test that only opens cannot tell "the seed landed" from "there was nothing in the way". The rule it pins is the one gotcha 45 states for the index: what is indexed is a document, and an entry nobody has written bytes to is not one.

+

This entry exists because the test cites this file and --check-evidence now requires that (improvise-lyyc); it was written from the test, so read the test first if the two ever disagree.

+
+

#improvise-ws-server

+

#distinct_stems_both_open

+

doctrine(check_the_checker) for colliding_stems_are_fatal: inject the fault that test rules out — distinct stems — and both documents must open. Otherwise an open_all that failed on every second file would pass the collision test.

+

#a_bystander_on_the_same_document_is_projected_the_edit

+

doctrine(check_the_checker) for an_edit_is_projected_only_to_the_edited_documents_clients: put the bystander on the same document and the assertion must flip. Without it, an isolation test that passed because nobody is ever projected anything looks identical.

+

#the_other_document_would_also_have_a_cursor_to_invalidate

+

doctrine(check_the_checker) for a_broadcast_does_not_cross_a_document_boundary. The fault to rule out is the other document being incapable of producing the broadcast at all — then a server-wide relay would be silent over there for the wrong reason and the isolation test would pass without proving isolation. So the checker drives the same TransposeAxes against beta and asserts it does produce a ResetCursor there; beta's silence in the isolation test is the routing and nothing else.

+

Both tests moved from Command::SwitchView to ModelCommand::TransposeAxes with improvise-jytm: a view switch is now addressed to one session and relays nothing, so it can no longer carry an isolation test. ResetCursor is the surviving Broadcast::ToAll.

+

#a_load_that_fails_rebases_nobody

+

doctrine(check_the_checker) for a_rebase_stops_at_the_document_boundary: inject a load that cannot happen. The generation does not move, so nobody is rebased — which is also what a routing bug that sent Snapshot to nobody would look like, so the rebase test needs this one beside it to mean anything.

+

#a_reopened_document_would_have_lost_the_edit_and_the_basis

+

doctrine(check_the_checker) for the_last_session_leaving_does_not_close_the_document: inject the eviction — a document freshly opened from the same fixture, which is exactly what closing on last-disconnect would hand the next client. Both assertions in the lifecycle test fail against it, so neither passes for a reason other than the document having stayed put.

+

#a_rebase_stops_at_the_document_boundary

+

Fixture note, and it is gotcha 39's warning: the swapped-in document keeps the same active view name and changes only its shape. A rename would make apply_model broadcast a SwitchView, the client would fold it, and the stale cursor would then be read under a key nothing has written — right by accident, with the bug invisible.

+

#a_client_on_a_socket_is_seeded_and_gets_its_edit_back

+

(crates/improvise-ws-server/tests/socket.rs, and its neighbours in that file.) Everything the socket test asserts is already asserted in host.rs without a socket, and deliberately so: its subject is the shell, not the decisions. It earns its place by covering what the unit tests structurally cannot — the handshake rule, JSON framing in both directions, the routing from a SessionId back to the connection it came from, and the one fact that lives only in a URL: which document the connection asked for. If it ever starts being the place a behaviour is pinned, that behaviour belongs in host.rs or registry.rs.

+

Two disciplines inside it are load-bearing rather than stylistic. The write is composed from the surrogate the server projected (ADR 002) — a CellKey there would be the test knowing something a browser does not — and the basis is composed from what the server published, because a basis the test built itself would be a client claiming to have seen something it was never sent.

+
+

#improvise-wasm-client/src/keys.rs

+

#text_entry_modes (a fixture, not a test)

+

Not a test — a fixture, and its limitation is the evidence. The comment here used to claim a new mode "shows up as a missing line rather than as untested behaviour". It does not: this is a hand-written list and nothing checks it against AppMode, so a new text-entry mode is simply absent and silently uncovered — which is what happened to save_prompt (improvise-lae) until it was added by hand. There is a second hand-written copy in the command crate's keymap::tests::text_entry_modes_with_exits. Deriving both from one place is filed, not done.

+

#a_modified_character_now_types_because_the_fallback_is_the_shared_one

+

The old classifier sent a modified character upstream in a text-entry mode, because the host's AnyChar fallback matches under any modifier and a hand-written rule could only safely be a subset of it. Reading the same table removes the question: Ctrl-x falls through to AnyChar here exactly as it does on the host.

+

#search_typing_writes_the_query_and_not_a_buffer

+

Search typing resolves on the client for the first time, and only because the keymap is shared: SearchMode binds append-char ["search"], and the name "search" is ViewState::search_query rather than an entry in buffers (improvise-gdl). The old classifier sent every search key upstream into an empty vec. The negative half of the test matters too: Esc stays upstream, because exit-search-mode owns the query and is the host's.

+

#the_command_names_that_still_need_the_host_are_these

+

The command names this client cannot execute, one by one, so that closing one of these gaps has to come here and say so. This replaced a list of keys, and that is the whole gain of sharing the keymap: a rebinding no longer invalidates the list, and a new binding for an already-executable command works in a browser with no change to this crate. Each entry is driven through the key that reaches it, because a name that is not bound anywhere would otherwise sit in the list forever looking like a gap.

+

#a_prompt_this_client_cannot_commit_is_never_entered

+

The refusal list above is only half the property. Upstream says the key produced no Command; it does not say the client stayed put, and the bug being guarded is precisely entering a room with no exit.

+

Found by Copilot on PR #27. Without strands_the_user, Ctrl-W resolved enter-mode save-prompt locally — ChangeMode(SavePrompt), a perfectly good Command — and the user then typed a path into a prompt whose Enter goes nowhere. Ctrl-E did the same into ExportPrompt and had done since before that branch.

+

The negative arm has two halves, and both are needed. Half one: a mode that is not refused is still entered — the half that goes through strands_the_user, so the only thing that can fail when the predicate degenerates to "refuse everything". ? is honest here where : was not: Help has no commit at all, so nothing a user types into it can be swallowed, and Esc closes it locally. (Its page keys are upstream, which makes page navigation inert — but inert navigation loses no work, which is the line the predicate draws.) Half two: the property is about finishing, not entering, so something has to press Enter and observe. The first version of this test asserted only that : entered its mode, and : is itself a dead room — so that arm was certifying the very behaviour it existed to rule out.

+

#the_command_line_cannot_be_committed_by_this_client

+

The evidence behind listing CommandMode in strands_the_user, rather than the assertion that it is listed. : opens locally and typing appends locally, but Enter is [execute-command, clear-buffer] and step has no execute-command arm — so the whole sequence fails to resolve, Host::handle drops the frame, and every ex-command behind : is inert. That is what makes refusing the mode cost no working surface. When improvise-e24v gives this client execute-command, this is the test that should start failing, and the CommandMode arm of strands_the_user is what it points at.

+

#a_commit_in_the_last_cell_exits_editing_instead_of_advancing

+

At the bottom-right corner EnterAdvance returns AbortChain, so the re-entry is skipped and the commit leaves the user in normal mode. That abort is a fact about the host's effect chain and it has to be mirrored, or the browser would sit in an editor on a cell the terminal has left. The fixture is a one-by-one grid, so the only cell is the corner.

+

#a_move_that_goes_nowhere_follows_the_command_it_mirrors

+

Which widgets say nothing about a move that goes nowhere is not a rule of keys.rs — it is whatever the Cmd being mirrored does, and the three differ: Move emits even when it clamps, MovePanelCursor returns nothing when the cursor does not move, MoveTileCursor emits regardless. The grid arm carries its own check-the-checker inline (a j that does move), so "emits when clamped" cannot pass against a fixture that is simply inert.

+

#i_on_an_aggregate_declines_out_loud

+

i on an aggregate must not go upstream: Host::handle answers a Key frame with an empty vec, so that would be a keypress the user sees do nothing at all (doctrine(an_upstream_key_is_a_black_hole)).

+

#enter_commits_the_cell_advances_and_re_enters_editing

+

Enter in Editing is [commit-cell-edit editing, clear-buffer edit], and every part of that sequence shows up in what the client composes — including the trailing clear that empties the buffer EnterEditAtCursor has just seeded. That ordering looks wrong and is exactly what the TUI does; mirroring it line for line is the contract.

+
+

#improvise-wasm-client — lib, socket, state, render

+

#a_client_with_no_basis_composes_nothing

+

The fourth None, and the one ADR 001 adds: a client that has not been told the document's basis has nothing truthful to say about what it composed against, so it composes nothing. That is the property that makes the basis check unforgeable from a browser — BasisCache::claim is the only constructor, and it withholds.

+

The other three None shapes — an aggregate, id: None, and a position outside the grid — are pinned against the same composition in socket.rs's typing_into_a_cell_that_is_not_a_record_composes_nothing. They were asserted twice, once against each of two compositions, until improvise-r4lo left only one.

+

#a_refused_batch_is_kept_as_a_rejection_and_not_only_as_a_sentence

+

A refused batch must reach the person who composed it, and it must reach them as more than a sentence. ADR 001 makes rejection a normal outcome, and the obligation that comes with it is that the client keeps the typing and can re-compose. Force needs the Rejection itself — RecordGone cannot be forced at all where RecordMoved can — so collapsing it into status_msg would throw away the input to the very feature rejection exists to enable. Both halves are asserted, because setting the string alone is the plausible half-fix, and the revision bump is asserted too: a renderer polls the revision, and a rejection it cannot see is a rejection it cannot draw.

+

#every_rejection_message_says_what_it_refused

+

Found by the PR #12 Copilot review: DocumentReplaced read "the document was replaced — reloading", and nothing in fold_frame reloads anything. A message that names an action the code does not take is worse than a vague one, because the user waits for it.

+

Renamed and re-scoped by improvise-8uj8, which is the more interesting half. The test was called every_rejection_message_says_the_edit_did_not_land and its own doc said the fact "is asserted over all of them rather than one at a time" — while it drove a hand-written list of five, written before PR #26 added NoDocumentPath and SaveFailed. So two variants were silently exempted from a rule the test called universal, which is worse than no test: the claim was false about its own subject.

+

The re-scope is that the claim, not the messages, was wrong. A save failure did not fail to apply anything, so "the edit did not land" is the wrong sentence for it. Rejection::refused_an_edit (exhaustive) splits the two classes and Rejection::all (a cyclic walk, the same shape AppMode::all uses) drives them, so a new variant fails the build rather than slipping out of the list.

+

Fault injected: wording SaveFailed as "hmm" fails the test — the arm the old five-item list could not have seen. Adding a Rejection variant fails the build at basis.rs's refused_an_edit and after.

+

#a_typed_string_composes_the_frame_that_carries_it

+

Was a_typed_string_is_classified_as_the_host_classifies_it, a test that stated an obligation in its name and did not discharge it — it asserted three outcomes by hand rather than comparing against the host's function, so the two copies of the rule could drift and stay green. improvise-j7v6 removed the duplication rather than the drift: commit_regular_cell_value now calls improvise_protocol::typed, which the browser's compose path already used.

+

Per doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished) the test could not survive as a comparison — type identity replaced its claim, and a hand-written comparison beside the shared function would be the copy coming back. It is re-scoped to what this crate still owes, which nothing else covers: the frame a typed string composes — basis, cell and command.

+

The classification's own obligation moved to tests/typed_string_agreement.rs, where it is checked against the thing it could really disagree with: what .improv calls a number (gotcha 34).

+

#a_cell_projection_that_overtakes_its_labels_is_lost

+

Gotcha 33, carried across this crate's dispatch. The protocol crate pins the drop inside GridCache::apply; what is this crate's is that nothing between the socket and that cache buffers, sorts or retries — so the same early Cells is still lost after routing. The second half of the test feeds the same three updates in frame-first order and requires them to land, so "everything is dropped" cannot pass it.

+

#bootstrapping_from_a_compaction_lands_where_the_stream_lands

+

doctrine(fold_tests_must_test_order_not_the_axiom): a client that bootstraps from a compaction of the host's state must end up where a client that folded the whole stream is. The seed is the host's current state expressed as projections, not the head of the history.

+

#the_selection_is_placed_by_subtracting_the_scroll_offset

+

(render.rs.) The conversion that looks right until the first scroll: the grid is viewport-indexed, the cursor is not. The offsets in the fixture are nonzero and unequal, which is what separates "reconciles" from "ignores the offset" and from "swaps the two offsets".

+

#column_groups_coalesce_per_header_as_the_tui_does

+

(render.rs.) Was column_groups_coalesce_by_name_as_the_tui_does, and its justification was consumed by the consolidation it predates. It read: the run boundary must be the name alone, "because that is what src/ui/grid.rs uses (group_name != prev_group) and the two ends must not disagree". Since improvise-8qlr src/ui/grid.rs uses AxisFrame::group_run_heads, so there is no second copy to agree with — the rule can change once and both ends follow.

+

Its fixture is what settled improvise-h0iv. It merges Month/Q1 (expanded) with Fiscal/Q1 (collapsed), and its own message said the merged span is "drawn with the FIRST one's indicator". That is a collapsed group rendered as expanded, with nothing on screen saying otherwise. Whether two adjacent same-named runs are one span is a question of taste and is why h0iv was filed rather than fixed; a wrong collapse glyph is not. The expectation is now three runs, and the collapse flag of each is asserted — which is what makes this test see the fault at all, since the labels and spans alone cannot distinguish a merge that keeps the first indicator from a split.

+

#a_keyboard_edit_in_progress_is_part_of_the_render_model

+

(render.rs.) A keyboard edit has to be visible, and what is visible is decided in Rust rather than in the page. The failure this rules out is quiet: a client that resolved i locally would change its mode label, put the typing into buffers["edit"], and draw the cell's old value the whole time — so the user sees a keystroke do nothing, types over it, and commits something they never read. None outside an editing mode is the other half: a stale buffer must not draw an editor that is not open.

+
+

#improvise-wasm-client/tests/browser_round_trip.rs

+

The file's scope is improvise-1ey's, quoted: "open an existing document server-side, view it in the browser, type a number into a cell, see the server value update and the projection arrive with the new value." Everything in that sentence except the DOM is Rust on both sides, so the test drives the whole of it: a real Host over an authoritative App, and a real Client reached only through the string calls the page makes. The socket is the one thing left out, and it is covered by improvise-ws-server's own tests/socket.rs; what this adds is the half that crate cannot reach — that the browser client seeds, renders, composes and folds correctly against it.

+

The discipline that makes it evidence rather than decoration: the client is touched only through Client::viewport, Client::on_frame, Client::render_model and Client::typed_cell — strings in and out — and it learns the cell to write from its own render model, never from the fixture. A CellKey anywhere in it would be the test knowing something a browser does not (ADR 002).

+

#a_grouped_row_axis_does_not_shift_the_write

+

A group header is a row of the render model and not a row of the grid, and a write composed against the wrong one lands on a different record — silently, because both are usize. FirstEditable therefore returns two numbers.

+

A fixture with no groups makes them coincide, which is exactly how an earlier version of the first_editable helper handed a visual index to typed_cell and passed (Copilot, PR #23). This test is the fixture the main round trip cannot be: a Coastal header sits between the two rows, so for the second row visual_row and data_row differ. Injecting the confusion — passing at.visual_row to typed_cell — leaves East unwritten. Note the ordering of the fixture is load-bearing: West comes first and ungrouped, because a header before the first row is dropped by AxisFrame::body_from and the two coordinates would coincide again.

+

#a_commit_keypress_is_one_mutation_among_view_commands_and_is_accepted

+

improvise-955n: the one real producer of a multi-command frame must still be accepted. CheckedBatch::against answers a frame's basis before applying any of it, and every mutation answers for its own records. Getting that check right is the easy half; the expensive half is not tightening it until the browser stops working. A commit keypress is exactly the shape at risk — one SetCell plus the mode change, the scroll offsets and the re-seeded buffer around it — and it is produced by keys.rs::commit, which no test in the command crate can reach.

+

So it drives a real keypress through a real Host and asserts the frame is accepted and lands: ServerFrame::Rejected anywhere in the answer, or an unchanged cell, is the regression. Asserting only "no rejection" would pass against a host that dropped the frame on the floor. The "exactly one mutation" assertion is a characterization of what keys::commit emits, not a protocol constraint — a frame may carry more than one — so that if commit ever started batching two, the test would be exercising something else and should be re-read rather than silently re-scoped. And the fixture types 888 after i has seeded the buffer, so the assertion is on the appended value: 888 alone would also be produced by a client that ignored the seed.

+
+

#improvise-wasm-client/tests/lockstep.rs

+

src/keys.rs decides a small set of keypresses without the document. The claim that makes that safe is not "the rule looks the same" but the two ends land in the same ViewState, and that is checkable: the host's reduce_key runs the real keymap, registry and effect pipeline against an authoritative App, and ViewState is the protocol's own type on both sides. So every local outcome is driven through both and compared.

+

This is why the command crate is a dev-dependency. Dev-dependencies are not built for --target wasm32-unknown-unknown, so nothing the host owns reaches the browser bundle.

+

#a_locally_typed_character_lands_where_the_host_would_put_it

+

Driving one mode would say nothing about the others (doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only)), so every text-entry mode is driven, and each with a non-empty buffer so an append that silently replaced the buffer would be visible.

+

#a_forwarded_key_reaches_the_host_unchanged

+

An unresolvable key must reach the host as a keypress, and what the host then does to its copy of this client's view slice is what the client will be told. Pinned as the shape the transport owes: the message carries the code and modifiers, and they survive the round trip. Enter rather than an arrow key, because since improvise-bpq2 the arrows resolve locally and enter-advance is one of the movements deliberately left upstream — it aborts the rest of its batch at the bottom-right corner, which is a fact about the host's effect chain and not a cursor position.

+

#a_locally_resolved_cursor_move_lands_where_the_host_would_put_it

+

The headline claim for improvise-bpq2's first gap: a cursor key resolved on the client lands where the host would have put it — including the scroll that follows it, and including the clamp at the end of the grid.

+

The fixture is six rows in a three-row window, and the keys walk past the bottom of the window (so row_offset must move) and then past the bottom of the grid (so the clamp must hold). A client reading its bound off the row frame's length would stop at row 2 and never scroll; one that clamped against nothing would run off the end. Both are visible. Eight presses for six rows: the last three are the clamp.

+

The fixture's one column is deliberate. The two ends measure their windows differently: the host's visible_rows is term_height - 8 and its visible_cols is computed from column widths, while a browser announces both outright. term_height can be chosen to match the rows; nothing can be chosen to match the columns, so the fixture removes the question rather than guessing at an answer.

+

#a_locally_resolved_panel_cursor_agrees_with_the_host

+

improvise-bpq2's second named gap, which turned out not to be one: the category panel's flattening depends on expanded_cats, but the derivation already lives in improvise_protocol::viewmodel and both ends call it (improvise-edp). So this is not two derivations agreeing by luck — it is one derivation, and the test says so by expanding a category, which changes the flattened length the cursor clamps against. A client that counted categories would stop several rows early.

+

#the_buffer_i_seeds_is_the_value_the_host_would_seed

+

i seeds the edit buffer from the cell's display value, and the two ends must agree on what that is: the host reads App::display_value_at_cursor, the client reads the text the server projected into its GridCache. Those are two paths to one string, and this is the assertion that they are the same string.

+

#entering_typing_and_cancelling_an_edit_agrees_with_the_host

+

Esc rather than Enter for the cancel arm, because it is a Binding::Sequence and so exercises the all-or-nothing rule: half of it — a cleared buffer in a mode the user has not left — is worse than none.

+

#a_locally_committed_edit_leaves_both_ends_in_the_same_place

+

What is not compared is the document, and deliberately: the client answers with a Command::Model, which reduce_view drops, while reduce_key writes through the effect pipeline. The write's composition is asserted in keys.rs against typed_cell's own — the shared one — so what is left to prove here is the half a unit test cannot see, which is that the advance and the re-entry land in the same place.

+

#client_ready_to_edit (a fixture, not a test)

+

The published basis a real client folds before it is allowed to compose a write (ADR 001). The tiers are the ones lockstep never checks — the host applies the keypress directly rather than the client's Command::Model — but BasisCache::claim withholds until it has something, so a client without one resolves no commit at all.

+
+

#harness/ — the browser specs

+

These are Playwright tests, so the checker's --check-evidence pass (which scans Rust fn names) cannot see their names. They are recorded here under their exact test("…") titles anyway, because they are where the JavaScript evidence lives.

+

#harness/tests/shell.spec.js — the file's own scope

+

The three assertions improvise-hwpi could not write. That ticket fixed four things and said of itself that the fixes were "as unverified as the bugs", because three of them are one-assertion tests that needed a browser. These are those three, plus the watchdog — which was found by reading, and only because someone went looking. Until this file, web/app.js and web/worker.js had never been rendered or run.

+

Two fixtures are load-bearing. NO_WEBSOCKET_HERE is a peer that accepts the TCP connection and then refuses the WebSocket upgrade — the harness's own static file server — deterministic where a closed port is not: a refused port's close timing is a property of the host's network stack, and on the GitHub runner it did not arrive inside the window. The page cannot tell the two apart and neither can web/app.js: in both cases the socket never opened. HANDSHAKE_TIMEOUT_MS is imported from web/shell.js and restates nothing (PR #65), and one test outlasts it — this sentence said the opposite of both for a while, which is gotcha 48's third clause: a fact stated twice is changed by a grep, not an edit. Since improvise-669b the spec also compares the running page's window against the one it imported, because the single home is in the TREE and the page loads harness/serve/app/shell.js — see outlastTheWatchdog below. logStatus records every distinct #status, not just the last one, because several of these tests are about a message being replaced, which the final value cannot see.

+

#"a frame the client cannot fold says so on screen"

+

improvise-hwpi: on_frame returns a Result, which wasm-bindgen throws. Thrown in a bare listener it reached the devtools console and nothing else — the grid simply stopped updating with nothing on screen saying why. The fault is injected at the peer, not in the page — a mocked socket that answers the handshake with a frame the client cannot fold — so this exercises the real on_frame and the real catch.

+

#"a peer that cannot be reached is named, and not called a disconnect"

+

improvise-hwpi: without an error listener, a connection that never opened left #status blank, and "the server is not running" read exactly like "still connecting". This test no longer says anything about the watchdog, and the paragraph below is why it never could. shell.arm runs only after await transport.ready, which resolves on open — so on a socket that never opens the watchdog is never armed at all, and denying its message here denies an unreachable event.

+

That sentence was already in this file, five lines above an assertion that contradicted it. PR #65 added the watchdog clause and edited this very paragraph, inserting "runFor(10s) advances the watchdog" immediately before the sentence saying it is never armed. Reverting PR #21's routing, or stripping fail() of its sawFrame(), both left that clause passing. So a PR whose subject was "a deleted guard went unnoticed" planted the same shape one level up: three injections, three distinct test names, and two of them proving nothing. What the test guards now is the naming — close on a socket that never opened must not read as "disconnected" (improvise-taxx) — which is the half injection 3 really does exercise.

+

The mechanism, so it cannot come back. outlastTheWatchdog now asks the page timesArmed() > 0 before jumping, and a watchdog denial re-added to this path fails by name. It asks ever armed rather than armed now: by the time the disconnect test jumps, a correct page has already disarmed — that is the property — so armed() would have denied the passing case. Building that guard is what surfaced the distinction, which no amount of reading had.

+

#"a socket that opened and then dropped reads as disconnected, and stays that way"

+

The other arm, which must keep working when the first is fixed: a socket that really did open and then dropped IS a disconnect, and saying so is the whole reason close cannot simply be suppressed.

+

It runs past HANDSHAKE_TIMEOUT_MS on purpose — through page.clock, not through a real wait. On this arm the watchdog IS armed — armWatchdog runs after transport.ready resolves, i.e. on open — so "disconnected" is written while a timer is pending, and the only reason it survives is that close goes through setShellFailure, which clears it. That was argued in a comment and asserted nowhere, and the argument was got wrong from the outside: a bug report (improvise-xps8) claimed the timer fired 8s later and overwrote "disconnected" with "sent no document". It had been measured, honestly, but against a tree predating PR #21 — and the "confirmation" that it survived PR #21 simulated only that PR's opened flag and not its other half, routing BOTH close arms through setShellFailure. A simulated fix that is not the fix you named reproduces a bug the real code does not have, and reads exactly like a confirmation. The ticket was filed and closed invalid. Hence this assertion rather than another paragraph — proved discriminating by making setShellFailure skip its clearTimeout, which makes it fail. Same rule as doctrine(an_injected_fault_must_be_the_fault_you_named), one level up.

+

And it was lost a third time, by the PR that was arguing about exactly this. PR #61 split the browser tests into boundary contracts (Playwright) and unit tests (node --test), and merged these two tests into one that kept only the last-value assertions. shell.test.js's "a loud failure DISARMS the watchdog" covers createShell in isolation and cannot see whether app.js routes the close event through fail() rather than setShell() — which is the half PR #21 fixed and the half that keeps getting lost. Reverting both socket arms to setShell left the node suite 26/26 and Playwright 13/13 green. Restored under their original names, and re-proved with that exact revert: the two-arm revert and the close-arm-only revert each fail "…and stays that way", and dropping the opened flag fails "…the watchdog stays out of it". Three injections, three different tests, none of them the unit test.

+

And the helper guards its own premise. page.clock.runFor on a page where clock.install() was never called does nothing and throws nothing, so a test that forgot the install would assert "the watchdog never spoke" against a watchdog never given the chance — vacuous, and green. outlastTheWatchdog reads Date.now() inside the page either side of the jump and requires it to have moved. Demonstrated both ways: with the install removed the suite passed 6/6, and with the guard in place the same omission fails by name.

+

And it guards its second premise since improvise-669b: that the page runs the constant the spec imported. PR #65 single-homed the handshake window in web/shell.js, which closed the understated spec constant — but the home is single in the tree, not in what the page runs: the spec imports ../../web/shell.js and the page loads harness/serve/app/shell.js, the copy web/assemble.sh put there. A stale serve/ restores the exact vacuity, and review demonstrated it — shrink the constant in source only, revert both socket arms, 6/6 green. createShell now exposes timeoutMs() beside timesArmed() and outlastTheWatchdog reads the running page's value and requires it to equal the import. Injected with the ticket's own recipe (8000 → 100 in web/shell.js, no rebuild): the run goes to 5 passed / 1 failed, on "the page's handshake window is not the one this spec imported", where the same staleness was previously invisible.

+

Note what this does and does not reach: it guards the tests that call the helper. "a peer that accepts and then says nothing trips the watchdog" also names HANDSHAKE_TIMEOUT_MS, but as a Playwright timeout: margin rather than as a window to outlast — a stale value there costs patience, not evidence.

+

The window is single-homed, and that is the second half of the same finding. HANDSHAKE_TIMEOUT_MS lived twice — once in web/app.js, once in the spec — with a comment saying a silent disagreement "would pass for the wrong reason" and nothing checking it. Setting the spec's copy to 1000 against the page's 8000 made the whole suite green against the reverted routing: the jump landed inside the window, the watchdog had not yet fired, and the denial held vacuously. It is now exported from web/shell.js and imported by both, so the two cannot disagree; shrinking it in its one home still catches the routing revert, which is the demonstration that the guard did not go quiet.

+

The faked clock is what makes the restoration cheap without weakening it, and the injections are the proof of that too: if page.clock.runFor did not really advance the page's timers, a broken routing could not surface the watchdog's message and those injections would have passed. They failed. The test keeps the real page, the real socket, the real listeners and the real setTimeout call — only the passage of time is simulated, and 9 seconds of wall clock became 75 milliseconds.

+

The measurement that could not become a CI guard. Both of the above were once asserted as test.fail expected-failures and had to be removed: neither reproduced on the GitHub runner, where close does not follow error within the window. An expected-failure is green when the bug is present and red when it is absent, so one whose trigger is platform-dependent does not merely flake — it inverts CI's meaning, and it failed the build twice saying "expected to fail, but passed". The lesson stands even though the second bug did not: assert the PROPERTY THAT MUST HOLD, which is portable, rather than the fault, whose trigger is the host's network stack.

+

The first of the two, for the record, was real: the socket's close listener was unconditional, so on a connection that never opened it overwrote "cannot reach …" with "disconnected". FIXED by PR #21. An opened flag is the only discriminator — readyState at close time cannot tell the two apart, and neither can the CloseEvent: both cases are code 1006, wasClean: false.

+

#"a peer that accepts and then says nothing trips the watchdog"

+

The first-frame watchdog, found by reading. The condition is one no per-transport check would catch: the peer accepted the connection and then said nothing. The mock accepts and never answers.

+

#"the page draws the demo document out of the worker"

+

No ?ws=, so web/app.js spawns the in-page worker over OPFS: the whole application, with no server process. If this draws a grid, then worker.js, bindings.rs, OpfsFs, vfs::open and the seed fetch all ran.

+

#"committing an edit sends one frame and shows the host's value"

+

A commit round-trips through the host, and what lands on screen is the host's value rather than the string that was typed. 20000 is typed and 20,000 asserted: the number format is applied by the host and relayed back, so a page that echoed the input would fail.

+

This is deliberately NOT named for improvise-hwpi's third fix (the closed re-entry guard in edit), which Copilot asked for on PR #24. That guard is not observable from the DOM, measured rather than assumed: with it removed, close still runs exactly twice, the frame is still sent exactly once, #grid still takes exactly two childList mutations, and the final text is identical. The only difference is one extra entry into repaint() that produces no output — because the second close is close(false), so it does not re-send, and everything else it does is idempotent. That is precisely why hwpi called it "harmless before, but only by accident". Counting repaint() entries would need a test hook inside web/app.js, which would be a page that knows it is being tested. So the test name says what is checked and this entry says what is not. What is observable is asserted: the editor is gone and did not come back, so no stray <input> is left behind and the host's value is not overwritten by the original text the second close writes before repainting.

+

#"the harness can see a failure"

+

Check the checker, and it runs first: a green run against a worker that never started is the failure this repo has a doctrine about. run_canary drives the same reporting path every real check uses, and the spec asserts it is reported as a failure with its message intact.

+

#"every check in the battery runs and passes"

+

The battery is asserted complete before any of it is believed: the report's names must equal EXPECTED exactly, so a build that dropped a check fails rather than passing with fewer.

+

#"what OPFS held survives a reload, and a committed delete does not"

+

The one thing no in-page check can establish. A delete is visible immediately and lands in OPFS at the end of the batch — so "commit_deletes actually removed it" is only answered by a fresh page. This is also the storage half of improvise-bck's persistence criterion.

+

#The opfs-contract checks (harness/opfs-contract/src/lib.rs)

+ +

#a_locally_cycled_panel_focus_agrees_with_the_host

+

cycle-panel-focus is bound to Tab in Normal only: PanelKeymap's spine binds Tab to enter-mode normal inside every panel (gotcha 12), so the wrap arms of CyclePanelFocus — the ones that start from the focused panel — are unreachable by keypress. The test therefore asserts the one reachable path and says so, rather than looping three times and passing on a mode that never moved. The first draft did loop, and failed on step 1 with the mode back at Normal; that failure is the evidence for this paragraph.

+

#a_locally_resolved_tile_axis_change_agrees_with_the_host

+

Both of TileAxisOp's names in one walk, because the command names itself off its axis field: r takes the Some arm (SetAxis) and Enter the None arm (CycleAxis), and a test that drove only one would leave a whole Cmd unmirrored. Injecting parse_axis("row") => Axis::Column fails it.

+

#the_category_panel_bindings_agree_with_the_host

+

The fixture sits on a header row for Space/a and on an item row for Enter/d, and then on a header again for a second d. Both of FilterToItem's arms and both of DeleteCategoryAtCursor's arms are therefore reached. That is not decoration: the first version sat only on the item row, and injecting a wrong sentence into the category arm of delete_at_cat_cursor did not fail it. One fixture position, one arm — and a check that passes against an arm it never enters is the defect this repo keeps finding.

+

#the_view_panel_bindings_agree_with_the_host

+

Two views in the fixture, and the cursor moved to 1 before Enter, because view_at_cursor reading .get(0) is the natural wrong implementation and a single-view fixture cannot tell it from the right one. Injecting .get(0) fails it.

+

It presses d as well, and that was added after the fact: the first version said "create, switch and delete" in its doc comment and pressed only n and Enter, so delete_view_at_cursor — whose only decision is the cursor step-back it makes away from the top of the list — was never run. The adversarial review of PR #50 deleted that step-back and the workspace stayed at 1322 passing. d is pressed at index 1 so the decision is reached, and both_ends now compares every cursor (see below), without which the deletion would still be invisible.

+

#a_locally_committed_minibuffer_agrees_with_the_host

+

commit-buffer for all three kinds, driven as keystrokes so the TextEntry spine's clear-buffer weld is part of what is compared. CommitKind::Item carries its category, resolved from AppMode::ItemAdd — so the mode the test starts in is load-bearing, not scene-setting. Injecting AddCategory in place of AddItem fails it on the document comparison.

+

Formula was missing from the first version, and it is the kind that matters most: it is the only one whose on_commit() is Some, so a test of the other two cannot see the mode a commit returns to. The review of PR #50 broke both CommitKind::status's and on_commit's Formula arms and nothing failed. Its text is seeded rather than typed, and valid, for the reason the next entry gives.

+

#a_formula_the_client_cannot_parse_diverges_on_the_status_line

+

The one thing a client cannot mirror about commit-buffer formula, found by widening the test above and kept as a pinned divergence rather than a fix (improvise-cl7y).

+

CommitBuffer emits its optimistic status first and the add effect last, so on the host a parse failure overwrites "Formula added" with "Formula error: …" (improvise-6t3). The client reproduces the command line for line — which is right — but it cannot parse, so it folds "Formula added" and stops. The user sees a browser confidently report a formula it did not add.

+

This asserts both sides, so a fix has to come here and change it. That is the difference between a pinned divergence and a comment: a_locally_committed _minibuffer_agrees_with_the_host would otherwise have to use a valid formula forever with no record of why.

+

#panel_focus_cycles_from_the_focused_panel_not_the_first

+

CyclePanelFocus's wrap arms, driven directly rather than through a key. PanelKeymap's spine binds Tab to enter-mode normal inside every panel (gotcha 12), so no keypress reaches them — which is why the lockstep test asserts only the reachable path and why breaking the start index survived the review's injection pass. Mirroring a Cmd faithfully includes the arms the keymap cannot reach today; testing them takes a direct call.

+

The fixture leaves the category panel closed, so "the next open one" and "the next one" give different answers and a mirror that walked the array blindly would fail.

+

#a_locally_resolved_clear_and_paste_agree_with_the_host

+

The two writes ADR 002 lets a thin client name. Note the fixture's cell key has three coordinates: Workbook::add_category puts a new category on Axis::Column, so Product is part of the cell's address and a two-coordinate key names a cell the grid never shows. The first version asserted on the two-coordinate key, and passed the per-key comparisons while asserting against a cell nothing had touched.

+

#walking_off_the_end_of_the_help_pages_agrees_with_the_host

+

The bug this was written to demonstrate: effect::HelpPageNext clamps and HelpPagePrev saturates, while reduce_view's Command::HelpPage arm wrapped. Two derivations of one fact, in disagreement, invisible because h/l in the help overlay were upstream keys and no client had ever sent the command. It failed on the first press past the end — client 0, host 4 — and the wire was changed to match the effect. Whether both should wrap instead is improvise-trd3.

+

#"? opens help, and its rows carry the keys and the words, not empty frames"

+

This entry was demoted to prose as Retired when PR #61 deleted the spec. PR #72 restored the coverage under a new name, so the claim is live again and the heading now names the test that exists. What follows is the original evidence; improvise-igxp's stronger assertions are recorded separately below.

+

improvise-ktv1's acceptance in one place: a browser shows help. ? had entered AppMode::Help since the keymap was shared, and nothing appeared — the mode changed and the page had no overlay, which is worse than a key that does nothing because the user thinks they missed it.

+

It asserts the shape, not merely that something is visible: the active tab is the shared module's page title (Welcome, then Navigation after l, which also exercises Command::HelpPage folding on the client), and the body has a key row. Two faults were injected and each failed it. Dropping the paintHelp(model) call fails toBeVisible; removing the mode != Help guard in help_rows — so the overlay is never empty and therefore never hidden — fails toBeHidden. The second is the one worth having: an overlay that is always open passes any check that only ever opens it.

+

It runs against the assembled site like every other test here, so it covers render.rs, the serialized model, web/app.js and the CSS together.

+

It was gone for a release, and that is why the entry reads oddly. 36ade35 deleted it with the rest of harness/tests/widgets.spec.js and nothing replaced it, so the heading promised coverage that did not exist -- exactly the cost improvise-5ray names, and invisible because the evidence checker then read only ### headings and only Rust names. PR #69 taught it to see both; PR #72 restored the coverage under the current name, with the stronger content assertions improvise-igxp asked for.

+

The injections above are the original two and still describe the property. The present test is harness/tests/help.spec.js, and its own entry below records what igxp added: the rows are read as dt/dd pairs and asserted as text, so blanking keys.textContent fails it where toBeVisible passed.

+

#the_shipped_demo_seeds_a_worker_whose_raw_cells_are_settable

+

improvise-ir7l renamed this from ..._whose_every_cell_is_settable and narrowed what it claims. The old loop called MainThread::cell — which panics on anything but a stored cell — over every column including Profit, which web/demo.improv defines as Revenue - Cost. It passed only because a formula target projected CellId::Record, which is the bug the ticket names: a test whose name was false for a third of the grid it walked.

+

It now walks the two raw columns and asserts separately that the Profit column is CellId::Derived, so it fails if either half moves. That second assertion is what makes the narrowing honest rather than a retreat — dropping the column would have left the page free to offer it again with nothing noticing.

+

#the_page_keys_turn_the_odometer_as_the_command_does

+

improvise-p1ym. GridCache::page_coords carried (category, selected item) and nothing else, so [/] resolved to a command name the client held but could not compose an argument for. They reached Host::handle, which answers an empty vec — doctrine(an_upstream_key_is_a_black_hole) — so an independent smoke test found them inert against the live page, where Quarter sits on the page axis precisely because paging is a thing a visitor tries.

+

The test asserts the shape composed, not that something resolved: two page axes, the least significant at Q4, so one press must both wrap Quarter and carry into Year. A resolver that moved only the first axis, or carried the wrong way, fails it.

+

#a_filter_axis_is_projected_but_not_cycled

+

The gotcha-27 arm. Page and Filter both constrain which records are visible, and GridLayout::page_coords is deliberately filtering_cats() (Page ∪ Filter) because the layout does not care which the user can navigate. The projection copied that verbatim, so a client handed the list would have tried to cycle a drill's fixed coordinate.

+

PageAxis::items is Some only for Axis::Page, so the difference is in the type rather than in a rule each client remembers. The test adds a Filter entry to a client that is otherwise paging normally and asserts the two Page axes still move while the Filter one does not — an assertion that only the Filter axis is skipped, not that paging stopped.

+

#a_locally_resolved_page_turn_lands_where_the_host_would_put_it

+

The lockstep claim, and the first version of it was vacuous, which is worth recording because it looked exactly right. It rebuilt the client's cache from the host after every press and then compared the client's cache against that same host's document — two readings of one value. Reversing the odometer's significance in page_cycle left it green.

+

It now drives two independent documents: one Host fed the commands the client composed, one App fed the same keypress through reduce_key, compared on View::page_selection — the document, never a projection. Proved discriminating by that same injection (for i in (0..axes.len()).rev() → for i in 0..axes.len()), which now reports Q2 against Q3. The assert_ne! against a fresh fixture is the second half: without it, a resolver that emitted nothing at all would pass by moving neither.

+

#harness/tests/widgets.spec.js — the file's own scope

+

Both files were deleted by PR #61, which moved the pointer's and the widgets' browser halves into harness/tests/contract.spec.js and the rest into node --test web/tests. What follows is the history of how these assertions were arrived at; the assertions themselves now live there.

+

The file is gone; its subject is not. All four widgets it named were one failure at one boundary — a RenderModel field with no element painting it — and that is now asserted once, from both sides, by web/tests/render_surface.test.js and harness/tests/contract.spec.js. What is not carried over is each key's meaning, which improvise_protocol::resolve decides and tests. Everything below is the history of the bug, and still true.

+

A user reported three things as broken: "I can't expand categories, and the Tile + View UIs don't work." None of the three was a regression. Driving the assembled site under Chromium separated them, and the separation is the point: two of the three had every layer but the last one, and the layer they were missing was not the same in each case.

+

Category expand already worked, by keyboard. Shift-C, j, Enter folds Command::ToggleCatExpand client-side and the items appear. What does not work is the mouse: categoryList builds plain <li>s with no listener, so a click on the triangle does nothing at all. That half is improvise-7ybz, and it is a product decision about the whole page's pointer model rather than a defect in an implemented feature — #grid offers a click only on an editable cell, so there is no click-to-select either.

+

The tile bar and the view panel had the cache, the keymap, the resolver and the host round trip, and no DOM. Before the fix, render_state() in the browser carried five TileBarCache entries and both ViewListCache views; Shift-T entered TileSelect and ArrowRight moved tile_cat_idx; r/c/p reached the host and really did move a category between axes. RenderModel simply had no field for either, and web/index.html had no element. So the keys worked and the user was aiming them at nothing — which reads exactly like "the Tile UI doesn't work". The formula panel was the third of Panel::ALL in the same state and is fixed by the same mechanism.

+

The fault injection was the unfixed bundle itself. Run against the tree at a3bef6e, the category test passes and the other four fail — #tiles, #views and #formulas resolve to zero elements. That is the evidence the four tests are discriminating, and it is why the category test stays: it is the control that says the harness really is driving the keyboard.

+

Two assertions in the first draft were wrong, and both corrections are worth keeping. Axis::short_label() is three columns, so a Page tile reads Quarter Pag, not Quarter Page — a test asserting the long spelling would have forced the page to invent a label instead of echoing Axis's own. And switching views leaves the panel open: switch-view-at-cursor returns to Normal and never closes the panel, exactly as src/draw.rs keeps drawing it off view_panel_open alone. Asserting it hidden would have pinned the browser to behaviour the TUI does not have.

+

The gating rule itself — an empty list is what says a panel is closed, the same rule help follows — is asserted in Rust as well, by a_closed_panel_projects_no_rows_and_an_open_one_does and the_tile_bar_is_always_present_and_only_its_cursor_is_modal, because a browser test cannot distinguish "closed" from "the cache is empty" and those two tests fill the caches first.

+

What these tests do not cover: the pointer (improvise-7ybz), and the inconsistency that #views/#formulas honour their open flags while #categories is drawn unconditionally because app.js has never read category_panel_open (improvise-wsaw).

+

One assertion was added after the first green run and is worth naming: the tile test originally asserted only that the tile label changed to Product Col. That is the bar reporting itself, which is exactly the shape of test this repo keeps catching — it would pass against a tile bar that displayed an axis the document had not taken. It now also asserts the grid reshaped: Espresso moves from a tbody row header to a thead column header. .first() is needed there because Product then shares the column axis with _Measure, so the label appears once per header level — which is itself a small confirmation that gotcha 44's multi-level frame reaches the browser.

+

Three review findings, applied. The second is the one worth carrying.

+

The page had two spellings of the same triangle, inside the file that declares it has one. indicator() carries the comment "only this spelling is the page's (gotcha 26)" — singular — and categoryList did not call it: it inlined U+25B8/U+25BE (the SMALL triangles) where indicator(), the page's own grid group headers and src/ui/category_panel.rs all draw U+25B6/U+25BC. So the browser's category panel disagreed with the TUI's and with its own grid. categoryList now calls indicator(!row.expanded).

+

And the first version of this spec pinned the wrong spelling. It asserted /▸ Product/ and ▾ — the divergent pair — so the obvious cure, which the file's own comment implies should already be in place, would have failed a test this work added. That is this repo's characteristic defect exactly: not a wrong line, a test that cements a state the doctrine says is wrong, and it makes the cure more expensive the longer it sits. The spec now asserts ▶/▼, and the change was proved discriminating by running it against the pre-fix bundle, where it fails with Received string: "▸ Product (4) R".

+

Note what this does not settle: whether glyphs should move to the protocol the way short_label() did. That is improvise-h04z, not answered here — what is fixed is the page contradicting itself. viewRow still spells "▶ " as a literal duplicating src/ui/view_panel.rs:62; the two agree today, which is precisely the state categoryList was in before it drifted.

+

And the drift was already spreading, which is the evidence that ticket needs. harness/tests/pointer.spec.js, written on a different branch, had independently grown the same ▸/▾ assertions — so the corrected indicator() broke it too. Three files converging on a spelling nothing compares is the failure design-principles §5 predicts, observed rather than argued, and it is what answers "is one duplicated literal worth a protocol change" with data instead of doctrine.

+

el.tiles.hidden was a no-op, removed with its comment. #tiles declares display: flex, an author-origin rule that beats the UA sheet's [hidden] { display: none } at any specificity, so the guard never hid anything and "an empty tile list draws nothing at all" was false. #views, #formulas and #help declare no display and so are genuinely hidden by the attribute. Worth knowing before anyone "fixes" it: because #tiles is always laid out, its height never changes, so it feeds nothing back into the ResizeObserver — restoring the guard would introduce exactly the first-paint height change the observer argument assumes away.

+

The ResizeObserver loop is no longer "argued, not tested", and the proof is worth more than the measurement. It was measured: 820 combinations — four panel states (none / views / formulas / tile-select) × heights 360-440 in 2px steps × five widths straddling the COL_PX = 110 boundary, each counting #grid child replacements in a 500ms quiet window. Every size settled; nothing repainted twice. But the reason is structural, so the measurement is a check on it rather than the evidence itself: the frameCost dependency graph is acyclic. frameCost.cols is row_levels, the number of row-axis header levels, which does not depend on how many rows are projected. frameCost.rows derives from col_levels.length and col_groups, which depend on the projected column window, which depends on width and frameCost.cols. So cols → rows, and nothing depends on rows. There is no cycle to oscillate. A change that made either term depend on the projected row count would create one, and that — not the pixel arithmetic — is the thing to check.

+

A harness port collision can make a run test somebody else's tree, silently. playwright.config.js hard-coded 8099 with reuseExistingServer: true, so a python3 -m http.server left by another worktree is adopted, and the run then exercises that checkout with no warning. The port is now IMPROVISE_HARNESS_PORT, defaulting to 8099 so CI is unchanged. The tell is a result that disagrees with the file you just read: check lsof -a -p <pid> -d cwd on the listener before believing it.

+

It fires in both directions, and the dangerous one is the false PASS. Two agents hit it in one session. One got a false negative — a glyph fix reported absent, because the adopted tree did not have it — which is merely confusing, since a failure sends you looking. The other got widgets.spec.js reporting 5 passed against a tree it had content-verified as having no #tiles and no paintTiles at all, which is impossible and was caught only because that agent knew what the tree contained and refused to believe the number. A false pass prompts nobody. That is what makes a hard-coded port with reuseExistingServer a correctness bug rather than an inconvenience, and it is the same rule as the rest of this file: a green result is evidence only once you know what it measured.

+

#a_click_focuses_the_panel_moves_its_cursor_and_activates_the_row

+

improvise-7ybz's whole difficulty in one assertion, and the reason the click resolvers take an index rather than reading the cursor.

+

A click means three things at once — focus the panel, move its cursor to the row under the pointer, activate that row — and the third is resolved in the same pure call as the first two, before any of them has been folded. So filter_to_item_at(state, index) must be asked about index; asking filter_to_item(state) reads view.cat_panel_cursor, which is still wherever the keyboard last left it. That is why cat_tree_entry, toggle_cat_expand, filter_to_item and tile_axis each gained an _at twin and the cursor-reading form became a one-line call of it: one definition, two entry points.

+

Injected fault, run: commands.extend(filter_to_item_at(state, index)…) → filter_to_item(state). The test fails with ToggleCatExpand("Region") against ToggleCatExpand("Product") — the panel cursor lands on the row clicked while the expansion happens to the row that was selected before, which is the silent half. The fixture therefore has two categories and clicks the second: with one category, or with a click on the row already under the cursor, the bug is invisible.

+

The sibling a_click_in_an_already_focused_panel_only_moves_and_activates pins the other half: focus emits only what is not already true, so a click inside the focused panel is not a mode change on every click. Without it, "always emit SetPanelOpen + ChangeMode" passes the first test.

+

#a_cell_click_is_absolute_even_though_the_page_paints_a_window

+

The pointer's version of the_selection_is_placed_by_subtracting_the_scroll_offset, and it fails the same way: GridCache is viewport-indexed, ViewState::selected is absolute, so a click at painted (2,1) under offsets (12,3) is (14,4). Offsets are nonzero and unequal for the same reason as its sibling — equal offsets cannot separate "reconciles" from "swaps the two". The second half asserts a click past the grid's extent resolves to nothing, which is what stops a click on the padding below the last row selecting a cell that does not exist.

+

#a_click_past_the_last_row_resolves_to_nothing

+

doctrine(check_the_checker) for the two above: the empty-answer path. Without it, a click that returned Vec::new() unconditionally would pass every "resolves to nothing" expectation while failing silently everywhere else — so this one is only meaningful because its neighbours assert non-empty answers.

+

#harness/tests/pointer.spec.js — the file's own scope

+

Both files were deleted by PR #61, which moved the pointer's and the widgets' browser halves into harness/tests/contract.spec.js and the rest into node --test web/tests. What follows is the history of how these assertions were arrived at; the assertions themselves now live there.

+

The file is gone; one test carries it. contract.spec.js's "each clickable surface is wired to its own index" keeps what only a browser can say — the handler is on the element, it passes that element's index, and the answer is painted — against three surfaces. What each click means is improvise_protocol::resolve's, and its tests are named below. The six tests' injection stands as recorded.

+

The user's ruling on improvise-7ybz ("build it — one entry point per widget") and improvise-wsaw ("honour the flag"), executed in a real browser.

+

Six tests, and all six fail against the bundle built immediately before them — the tree that already had the three widgets drawn (PR #55) but no click handlers and no category_panel_open read. That is the injection: the sidebar is visible when it should be hidden, and every click() resolves to a <li> that has no listener, so nothing moves. The category test is the sharpest, because the element it clicks exists and looks interactive in the unfixed tree — which is exactly what the user hit.

+

Two properties here that the Rust tests structurally cannot reach. The pointer and the keyboard are one cursor: the keyboard continues from where the pointer left the cursor clicks a cell, closes the editor, presses ArrowLeft, and requires the move to start from the clicked cell — if the click had only opened an editor (the pre-fix behaviour) the arrow would move from the origin instead. And the tile click is asserted against the grid, not only against its own label: Espresso must appear as a thead header, because a tile bar reporting an axis the document did not take is the failure shape this repo keeps finding.

+

One fixture note worth keeping: openPanelThenLeave opens the panel with Shift-C and then presses Escape, which returns to Normal without closing the panel (PanelKeymap's Esc is enter-mode normal, nothing more). That is the only state in which the "focus" third of the click is observable — from a freshly opened panel the mode is already CategoryPanel and the assertion would pass without the focus commands existing at all.

+

#harness/tests/drill.spec.js — the file's own scope

+

The fourth widget the user reported, and the one with a different answer from the other three. The tile bar, view panel and formula panel were each one RenderModel field short — cheap. Drill is not: there is no wire vocabulary for it, which is improvise-gd28, and these tests document the refusal rather than pretending the gap is a bug to be fixed here.

+

Where it stops, by layer. Everything below the command exists and was verified in the browser, not read: project::grid_of emits CellId::Aggregate, the client receives it ({"Aggregate":"750ab1f5ffa96d5e…"} in render_state), render.rs::editable correctly answers false for it, and resolve.rs reaches the aggregate arm. What is missing is one link: reduce.rs contains no Drill variant at all — not Command, not ModelCommand. The seam is ready and the sentence is not sayable.

+

Why it was left out is a decision, not an oversight (gotcha 38): drilling creates the _Drill view and writes drill_state, which gotchas 30 and 1 rule session-local — pending_edits are keyed by (row, RecordCol) and belong to whoever staged them. A Command::Model is routed with no SessionId, which is right exactly when a change is addressed to no one session; drill is addressed to precisely one. That is the same argument that removed ApplyAndClearDrill from the wire in improvise-3x3u, so adding ModelCommand::Drill would reintroduce the bug that removal fixed. It needs the addressed form — ADR 001 layer 2.

+

The two entry points used to fail differently, and that asymmetry was the finding. i/a on an aggregate declines out loud: edit_or_drill answers the CellId::Aggregate arm with a status the user can read. But > (drill-into-cell) was a name resolve::step did not handle, so it became KeyOutcome::Upstream → ClientFrame::Key → Host::handle, whose Key arm is an explicit empty vec — a black hole, not a slow path (gotcha 47). Pressing > in the browser did nothing whatsoever: no status, no mode change, nothing. That is why this read as "never confirmed working" rather than as a refusal.

+

That silence is now closed, and the closing is what this file ships. step answers "drill-into-cell" with a SetStatus saying drill is not available in the browser yet — an interim the wording of which is load-bearing, since it must be true both today and on the day improvise-gd28 lands. The wire vocabulary is still absent; only the silence was fixed. Note for gd28: when the real drill arrives, a leftover "drill-into-cell" arm in step would shadow it with nothing failing, because a name that resolves never reaches the host.

+

The TUI is the control and it works: 12 drill tests pass across cmd/grid.rs, cmd/mode.rs, ui/effect.rs and ui/app.rs. So this is isolated to the client/wire, not a general regression.

+

A fixture fact that is itself part of the answer. web/demo.improv has no aggregate cell at all — gotcha 46 puts exactly one category on each visible axis precisely so every cell is a Record, and tests/web_demo_model.rs asserts it. So on the shipped demo a user cannot even reach the refusal without first changing an axis. anAggregateGrid therefore makes one, through the tile bar, the way a user would: Quarter cycles Page → None and every cell becomes an aggregate. The precondition is then asserted (td.editable count 0) rather than assumed, so neither test can pass against a grid of Records.

+

Both injections, run — against the tree before the interim, which is what made them two faults rather than one. Replacing the aggregate arm with None — silencing the honest refusal — failed both tests, and the second failed on its check_the_checker control, which is the point: with nothing answering, the control can no longer tell "> is silent" from "the page is dead", and the test correctly refuses to pass. Adding "drill-into-cell" => Some(vec![SetStatus(…)]) to step — making > resolve — failed only the second test and left the first green. Two faults, two different failure signatures, neither test passing for the other's reason. That second injection is now the shipped code, which is why the test it broke was rewritten rather than kept.

+

That control is load-bearing and not decoration: a test asserting nothing happened passes against a page that has stopped listening, so it presses i afterwards and requires #status to change. Without that, this file would be folklore with a green tick.

+

#the_drill_key_and_an_aggregate_edit_refuse_for_different_reasons

+

The Rust half of the interim, and the assertion that carries the whole weight. Two keys on the same aggregate cell must both refuse and must not refuse with the same sentence: > says drill is unavailable, i says drilling into an aggregate needs the host. Collapsing them into one message would be the tempting tidy-up and would lose the distinction between "this key does nothing here yet" and "this cell is not one you may write".

+

The injection, and why it is not the obvious one. Replacing >'s sentence with the aggregate sentence looks like it should fail on the first assertion — and does not: that sentence contains both "drill" and "browser", so it clears the substring check. What fires is the anti-aggregate assertion lower down. The claim that the second assertion is the load-bearing one was doubted on review, tested, and confirmed; record it here because the first assertion reads like the discriminating one and is not.

+

The harness sibling (drill.spec.js) asserts the same distinction in a browser, but its not.toContainText("aggregate") is decoration under the only injection that exists — Playwright throws at the positive assertion above it first, so that line is never reached. The Rust assertion is the guard; the harness one is end-to-end confirmation that the string survives the wire.

+

#a_view_row_click_focuses_moves_and_switches_to_the_row_clicked

+

improvise-iioi, and it is the same trap as the category click one level over: the activation must be resolved against the row clicked, never view_panel_cursor, because the SetPanelCursor ahead of it has not been folded when the resolver runs. Injecting exactly that — reading the name at the cursor instead of at index — yields SwitchView("Default") beside SetPanelCursor { cursor: 1 }, i.e. the cursor lands on Plan and the document switches to Default. Silent, and wrong in the half a user does not watch.

+

The fixture therefore holds two views and clicks the second. With one view, or with a click on the row already under the cursor, the bug is invisible — the same reason a_click_focuses_the_panel_moves_its_cursor_and_activates_the_row clicks Product rather than _Index.

+

#a_formula_row_click_navigates_and_activates_nothing

+

The negative arm, and it is a statement about the rule rather than about the formula panel. The rule is "go there, then do what Enter does there"; ModeKey::FormulaPanel's keymap binds a/n/o to enter-mode formula-edit and d/Delete to delete, and no Enter at all. So the rule yields navigation and nothing else, and that is the rule applying rather than the fix stopping half way.

+

This test asserts the exact command vector, which is the whole of what it can say. It once carried a second, explanatory assert! that no FormulaEdit mode change appears — which could never have been the failing assertion, because the assert_eq! above it pins the same three commands and aborts first. A reviewer demonstrated it by injecting a ChangeMode(FormulaEdit) and watching the panic land on the assert_eq! with the explanatory message never printed. It was a comment wearing an assert!, and the argument now lives in context/code/clients.md where a comment run cannot hold it.

+

#the_formula_panel_has_no_enter_to_mirror_and_the_view_panel_does

+

The guard that the removed assertion was pretending to be. The asymmetry — a view-row click activates, a formula-row click does not — is a fact about the keymap, and nothing tied the click to it. The reviewer proved the gap by adding Enter -> enter-mode formula-edit to ModeKey::FormulaPanel and running the whole workspace: 1381 passing, nothing failed, while click_formula_row still activated nothing. The click and the keyboard then disagree about what a formula row does, silently.

+

Injecting that same binding now fails this test and only this test. The failure message names click_formula_row as what must change, because the person who adds an Enter binding is not the person reading resolve.rs's click arms. The positive half (ViewPanel does bind Enter) is there so the test cannot pass by asserting that no panel binds anything.

+

#The view and formula row clicks in the browser (improvise-iioi)

+

These began as two tests in harness/tests/pointer.spec.js. That file no longer exists: PR #61 deleted it and moved the pointer's browser half into harness/tests/contract.spec.js, whose stated scope is the wiring — the handler is on the element and passes that element's index — while what a click MEANS is improvise_protocol::resolve's and tested there. The assertions were merged into that file's own click test rather than restored, because resurrecting a file a merge deleted is how a deliberate removal gets undone by reflex.

+

They leave the panel open but unfocused — the only state in which the focus third of the click is observable.

+

History, not current evidence: the original pair was proved against the bundle built immediately before the fix, both failing there while the pre-existing pointer assertions passed. That run was against harness/tests/pointer.spec.js, which no longer exists. What holds the relocated assertions honest is the injection below, run in their new home.

+

The view test asserts three things — #view-name, the active marker and the selected cursor — because switching ends in NORMAL and a test that only checked the mode would pass against a click that did nothing but leave the panel. The relocation initially dropped two of the three, and a reviewer caught it; they are restored.

+

And the fixture was vacuous in a way the prose had already warned about. Default is index 0 and holds view_panel_cursor by default, so clicking it — which both the old pointer.spec.js and the first relocated version did — makes index and the cursor the same number, and reading either gives the same name. A reviewer injected switch_view_at(state, state.view.view_panel_cursor) into click_view_row and the whole contract suite passed 3/3: the exact fault this test, the ticket and the Rust test all exist for was invisible to the browser half. The fixture now switches to Default by keyboard, leaving the cursor there, then clicks Plan — so the two readings name different rows and the outcomes differ. Re-running that injection now fails it.

+

The blindness was inherited rather than introduced: the deleted file clicked Default too. It became worth fixing because the relocated test is now the only browser assertion on that surface.

+

One fact the reviewer's report supplied and this fix leans on: shipping the cure for one panel and not its two siblings reproduces the original report. A user who has just learned category rows are clickable clicks a view row and reads the silence as broken — which is literally what "I can't expand categories" meant.

+

#The harness port: reuse is now opt-in

+

reuseExistingServer: !process.env.CI became === "1" on IMPROVISE_HARNESS_REUSE. A default-on reuse adopts a server another checkout left on the port and then tests that tree, which fired in both directions in one session — a false negative and, worse, a false positive. A path-hash default was considered and rejected: it still silently reuses when two agents share a worktree path, where an explicit opt-in cannot. CI is unaffected (it never reused) and a developer wanting the old speed sets the variable.

+

#a_click_that_cannot_compose_its_activation_composes_nothing

+

improvise-hkr8. click_category_row ended with filter_to_item_at(state, index).unwrap_or_default(), and that None arm is not "nothing to do" — it is BasisCache::claim answering that the server has published no basis yet (gotcha 43), so the document change cannot be composed. The click therefore kept its navigation prefix (focus the panel, move the cursor), dropped its activation, and reported success. The keyboard handles the identical None correctly, by mapping step(..) == None to KeyOutcome::Upstream.

+

The fixture has to do two things a careless one would not. It expands Region so that row 1 is an Item row — a header row activates via ToggleCatExpand, which needs no basis and so cannot show the bug — and it asserts both preconditions (cat_tree_entry_at is an Item, claim is None) rather than assuming them, because an empty ClientState makes every state-dependent refusal look like the one under test.

+

Injecting the old unwrap_or_default() fails this test and only this test.

+

#the_same_click_composes_everything_once_a_basis_is_published

+

The positive arm of the one above, and it is what stops the fix from being "return nothing more often". With a basis published, the same click composes all six commands: open, focus, move the cursor, two Command::Models and the status. Without this, deleting the activation entirely would pass hkr8's test.

+

#a_first_tile_click_selects_without_touching_the_document

+

The user's ruling: a click selects, a second click cycles. The fixture publishes a basis deliberately, so the absence of a Command::Model is a decision rather than a client that could not compose one — pointed at an unpublished basis this test would pass for hkr8's reason instead of its own.

+

Both halves are asserted, because "the axis did not change" passes against a page where nothing happened at all: the positive half requires SetTileCatIdx plus ChangeMode(TileSelect), the negative half requires no Command::Model.

+

Four injections, four distinct signatures. "First click also cycles" fails this test, the_first_click_on_tile_zero_selects_it_like_any_other and clicking_a_different_tile_selects_it_rather_than_cycling_it — and not a_second_click_on_the_selected_tile_cycles_its_axis. "Second click does not cycle" fails only that last one. "already_selected reads the index alone" fails only the tile-zero test. Reverting hkr8 fails only the hkr8 test. No injection passes for another's reason.

+

#t_then_a_click_on_the_leftmost_tile_cycles_it_immediately

+

Found on review, and it is a defect in the wording rather than the code. T enters TileSelect leaving tile_cat_idx at 0, so a click on the leftmost tile is a click on the already-selected one and cycles on the first gesture. Both loose summaries of the ruling — "a click never mutates on its own" and "a second click cycles" — are false against that probe.

+

The behaviour is right: a click does what Enter does there, and Enter in TileSelect cycles the tile under the cursor. The resolver cannot see how the state arose and must not try to. So the test exists to name the case rather than to change it, and it asserts the neighbouring tile answers the identical gesture by selecting — because the surprising half is that two adjacent tiles differ, and a reader who meets that in the wild should find it written down.

+

Note what this means for its sibling a_second_click_on_the_selected_tile_cycles_its_axis: that test sets the state directly rather than by clicking, so it already was the keyboard-entry case. Nothing was untested; the name and the argument were missing.

+

#enter_in_an_empty_view_panel_resolves_locally_rather_than_upstream

+

switch_view_at returns Option, and its two callers answer the None differently on purpose. From a click_*, None means "this gesture composed nothing" — a dynamic fact, and hkr8's rule. From step, None means "this client cannot execute this command name" — a static fact about the vocabulary, which the_command_names_that_still_need_the_host_are_these exists to keep static. Two questions sharing one type, which is why the asymmetry reads wrong at first. (delete_view_at_cursor already answered Some(Vec::new()) the same way, so this follows the file rather than inventing.)

+

Nothing held the keyboard side to it until this test. A reviewer removed the unwrap_or_default() so out-of-range propagates None — making Enter in an empty view panel KeyOutcome::Upstream, gotcha 47's black hole — and the whole workspace passed under --no-fail-fast. That would make membership of the host-needed list state-dependent, the one property that list denies. Injecting it now fails this test and only this test.

+

The assertion is Ignored, not Local(vec![]): KeyOutcome::of maps an empty vec to Ignored by construction, so Local is never empty. What matters is that it is not Upstream.

+

#the_first_click_on_tile_zero_selects_it_like_any_other

+

The trap in the ruling's own wording. "A second click on the already-selected tile" reads as an index comparison, and tile_cat_idx is 0 before anything has selected anything — so an index-only test would read tile 0 as already selected and cycle it on its very first click, making the non-mutation guarantee true of every tile but one. The condition is mode == TileSelect and tile_cat_idx == index.

+

#clicking_a_different_tile_selects_it_rather_than_cycling_it

+

The other half of the same trap, from the opposite side: without it, "a click does not mutate" would hold only for the first click of a session, since every click after the first is a "second click" in the loose reading. Clicking A then B must select B.

+

#How to know a launch default has stopped disarming things

+

The question this left open was how to know you have exhausted the class. Grep does not answer it: grep finds sites that name the flag, and the hazard is a test that merely starts from a default-built state and asserts something downstream, mentioning nothing. Three disarmed tests were found by grepping category_panel_open; that was luck, not method.

+

Mutation answers it, because it does not care what a test mentions. The recipe, which a reviewer ran across five sites in about twenty minutes:

+

For every production site that writes, propagates or reads the value, replace it with the constant the default supplies, and run --no-fail-fast. Sites where nothing fails are suspects. Then re-run each suspect with the default inverted. If it now fails something, the test was vacuous — the default disarmed it, and the fix is the fixture. If it still fails nothing, it was simply unguarded — a different problem, pre-existing, and a bead rather than a fixture.

+

The second step is the one that matters, and without it the two classes are indistinguishable: both look like "nothing failed". It is what established that the class is closed at two (set_panel_open_and_cursor, panel_commands_address_the_named_panel_only) and that two further silent sites are older problems this ruling did not cause:

+ +

And a repair needs its own probe, because a repair can be shaped like a fix. The two-step recipe above finds a disarmed test; it does not tell you whether the repair you then write is real. That gap is not hypothetical — the first repair of panel_commands_address_the_named_panel_only folded [open:false, open:true], read as a fix in review, and was still vacuous.

+

The probe that settles it is a one-directional fault: make the writer if open { set true }, so the flag can be raised but never lowered. A vacuous repair cannot see this, because a test that only ever asserts the true direction is satisfied by a writer that only ever writes true. Injected into both copies at once, it fails set_panel_open_and_cursor and panel_commands_address_the_named_panel_only, each in its own copy — which is what establishes both repairs are genuine rather than merely differently spelled. Prefer it to re-reading the diff: the diff contained open: false in the version that was vacuous.

+

Note resolve.rs's panel_open reader is well guarded by contrast: hardcoding it true fails two click tests. So the picture is not "everything touching this flag is untested"; it is specifically the propagators.

+

#the_terminal_constructor_inherits_the_shared_default_rather_than_setting_it

+

Prospective rather than present-tense, and the doc comment says so. With three panels, the_terminal_app_launches_with_the_category_panel_open and the_other_two_panels_are_shut_at_launch already cover every Panel::ALL member, so this test cannot currently fail without one of them failing too. A reviewer confirmed that by control.

+

It earns its place the way gotcha 12 describes Panel::ALL: a fourth panel reaches it from one ALL entry, and at that moment it becomes the only test asserting that App::new did not quietly give the TUI its own launch state — the divergence improvise-wsaw closed. Recorded here so nobody later reads a green tick as evidence of something it is not yet checking.

+

#the_terminal_app_launches_with_the_category_panel_open

+

The user ruled (2026-09-20) that the category sidebar is visible at launch in the TUI as well as the browser, so improvise model.improv opening with it showing is intended. This is the guard on the front end the ruling newly changes: a_fresh_session_opens_with_the_category_panel asserts ViewState::default() in the protocol crate, and nothing downstream of it.

+

Why not a rendered-frame test: design-principles §7 says not to test ratatui Widget::render, and there is no TestBackend anywhere in this repo. The honest target is the TUI's own constructor. src/draw.rs's one-line read of the flag stays argued-not-tested, as gotcha 48 has it.

+

Three injections. Putting the shared default back to false fails this test and a_fresh_session_opens_with_the_category_panel. Making App::new override the default (app.view_state.category_panel_open = false) fails this test and the_terminal_constructor_inherits_the_shared_default_rather_than_setting_it — which is the one that would catch a TUI-only launch state being reintroduced, the divergence improvise-wsaw closed. the_other_two_panels_are_shut_at_launch stops "open everything at launch" passing the first.

+

#set_panel_open_and_cursor

+

The launch default silently disarmed this test, and that is the finding. It did SetPanelOpen { Category, open: true } then asserted the flag was true — but test_app() builds from ViewState::default(), where it is already true. Its Formula and View siblings still default to false, so only the category arm went vacuous.

+

What that cost, demonstrated: replacing Panel::Category => app.view_state.category_panel_open = self.open with a no-op — so no effect could open or close the TUI's category sidebar at all — left the entire workspace green. C could be made to do nothing in the terminal with 1391 tests passing.

+

The fix is the fixture stating its own precondition: close the panel first and assert it closed, so the effect has to move the flag before the true direction is asserted. Injecting the no-op now fails this test.

+

Note the harness cannot see this one, and that is correct rather than a gap: the browser folds Command::SetPanelOpen through reduce_view, a different three-way match in a different crate from effect::SetPanelOpen's. Two derivations of one mapping — which is why the reduce_view guard needed the same repair independently. Since improvise-56ek there is one derivation, so that is history rather than a standing hazard; see below.

+

#a_panel_command_writes_that_panel_and_no_other

+

The reduce_view half of the one above, and the first repair of it was still vacuous — worth recording because the mistake is subtle. Folding [open: false, open: true] and asserting the final state is true passes against a reducer that folds nothing at all: the default is already true, so neither command needs to land. Only asserting the false direction discriminates, because that is the one the default cannot supply. Caught by injection rather than by reading — the no-op fold reported NONE failing after the first fix.

+

improvise-56ek then removed the second derivation entirely: ViewState::panel_open_mut and panel_cursor_mut (plus their read halves) are the one place a Panel becomes a field, and both effect::SetPanelOpen::apply and reduce_view call them. The cure is one home, not a test holding two copies in step (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)), so there is deliberately no "the two agree" test — a wrong field in the one home is a failure in every caller.

+

This test replaces panel_commands_address_the_named_panel_only, which drove Panel::Category only and so could not tell one panel from another. It keeps that test's lesson by zeroing every panel before opening one.

+

Injected fault: point Panel::Category's arm at formula_panel_open. Fails by name for every panel in Panel::ALL.

+

That injection alone was not enough, and the gap is gotcha 3's shape. The first version asserted only through panel_open/panel_cursor — the read halves of the mapping it was testing. A consistent permutation of all four accessors is a self-consistent bijection: injecting Formula↔Category into panel_open_mut, panel_open, panel_cursor_mut and panel_cursor together left the test passing, while two unrelated tests that read raw fields failed. It caught only a non-injective mapping, which is exactly the injection first recorded here. The test now reads the three field pairs by name once per panel, and the consistent permutation fails it: "Formula is not the field Formula names".

+

Also corrected here: the claim that consolidation left "no second copy to agree with" was false when written. resolve.rs carried a fourth copy of the same Panel → ViewState three-way match, in the same crate on the same type; it is deleted and its three call sites go through ViewState::panel_open. What remains is not the same mapping: registry.rs maps Panel to CmdContext fields, a different type whose values are copied from view_state, and resolve.rs/registry.rs each map Panel to a (cursor, collection length) pair, which is a second derivation of something else and is lockstep.rs's subject.

+

#a_fresh_session_opens_with_the_category_panel

+

The user's ruling that the category panel is visible at launch, implemented as the shared ViewState::default() rather than a browser special case — a browser-only default is "the page overrides the flag", which is the same defect as the "the page ignores the flag" bug improvise-wsaw fixed. The consequence, accepted deliberately, is that the TUI opens with the panel too.

+

The test also asserts the other two panels stay shut, so it cannot pass against a default that simply opens everything.

+

Two existing tests had to change, and both were pinning the old decision by inheriting it rather than stating it. panel_focus_cycles_from_the_focused_panel_not_the_first is about skipping a closed panel and took "the category panel is closed" from the default; it now spells that precondition. with_categories() is used by tests about a click opening the panel, so it now shuts it explicitly. Neither is the decision being re-made — the decision is re-made only in the harness, deliberately.

+

#The launch state and the tile ruling in the browser

+

Also merged into harness/tests/contract.spec.js rather than left in the deleted pointer.spec.js — see the note above.

+

Two states, one mechanism: the sidebar is visible with no keypress, and C still hides it. Proved by injection in a real browser: setting el.categories.hidden = false (the page ignoring the flag) fails on toBeHidden; setting the shared default back to false fails on toBeVisible. Different signatures, different halves.

+

Only the second is a guard, and an earlier wording here got that wrong. It claimed "either alone passes against the bug wsaw fixed" — but under the new default, visible-at-launch passes for correct code and for a sidebar drawn unconditionally. The launch assertion states the ruling; the toBeHidden after the toggle is the only thing that still catches wsaw.

+

Note the first C now focuses rather than opens, since the panel is already open — toggle-panel-and-focus closes only when the panel is both open and focused — so the test presses it twice and the hide is the second press.

+

The tile tests were rewritten rather than adjusted, because the previous one asserted the behaviour the ruling reverses; its failure against the new code was itself the injection. The first-click test waits out a timeout before asserting the axis did not move, since a cycle would arrive asynchronously and an instantaneous assertion would pass against a slow one. ---

+

#web/tests/ — the Node suite, and what was injected into it

+

No ### headings here on purpose: --check-evidence resolves a heading to a Rust fn, and a JavaScript test name under one would read as an orphan. These are keyed by file.

+

Every test below was proved able to fail by patching the code it names, checking the patch was on disk, running the suite and reading the failure — never by the absence of an error. Thirteen injections, thirteen failures:

+

What shell.test.js cannot reach, and the PR found out the hard way: it tests createShell in isolation, so it sees the watchdog RULE and never the app.js wiring that calls it. That distinction is now written into context/code/clients.md and guarded in shell.spec.js.

+

shell.test.js — socketUrl stops upgrading ws:// on an https page; composeStatus returns the model fact alone (which fails five tests, because the watchdog's messages are shell facts and vanish with it); fail() stops clearing the timer, so the watchdog pre-empts rather than being disarmed — the bug PR #13 fixed and PR #21 half-lost; extentOf stops subtracting the frame; frameCostOf stops counting the column-group line.

+

grid_plan.test.js — the data-row index is taken from the row's position in rows, so a group header consumes one (the click coordinate bug gotcha 1 is about); the group label is tested for truthiness, so a group legitimately named "" draws as "in no group"; typing is Boolean(model.editing), so an empty edit buffer reads as no editor open.

+

serial.test.js — the chain is replaced by Promise.resolve().then(...), which is what an async listener does: the slow first frame is overtaken. The suite also keeps that fault as a passing test in its own right, asserting the unchained pair really does come back swapped, so the ordering test cannot be green for want of a race.

+

render_surface.test.js — a pub sparkline: String added to RenderModel is reported unpainted; deleting paintTiles reports tiles unpainted; a getElementById that names an id index.html has not got is reported. The first is the shape of the bug the four widgets shared, and the list it checks against currently names doc_name/doc_dirty/doc_stored_as — improvise-5jna, which this test found rather than remembered.

+

bundle/wasm_surface.test.js — client.render_model() renamed to client.render_modal() fails both its tests. It reads the generated .d.ts, so a missing bundle is an assertion failure and never a skip.

+

web/assemble.sh — dropping web/shell.js from the copy list, with the page's own imports as the check. The first version of that guard anchored its sed at ^import, so it could not see the multi-line import that is the only one it exists for, and the injection passed: the guard was written, landed, and blind. That is what the injection was for.

+

#harness/tests/contract.spec.js — eight injections into the served page

+

Patched in harness/serve/app/app.js, which is what the browser loads, so no rebuild stands between the fault and the run. The sidebar ignoring category_panel_open; the tile bar, view panel, formula panel and help overlay each never drawn; click_cell sending a constant row; click_category sending a constant index; and the typed buffer drawn without its caret. All eight fail.

+

The last one did not fail at first: the test asserted the typing class and not the text, so a cell drawn blank passed. The assertion was strengthened until the injection failed, which is the only reason it is worth having — and it is the same lesson as the eleven tests above: a class is a flag, and a flag is not a pixel.

+

#every_advertised_command_answers_in_its_own_mode

+

improvise-rmvx. Gotcha 47 says an upstream key is a black hole, and the_command_names_that_still_need_the_host_are_these is the guard that was supposed to keep the list of them honest. It was built with in_mode(AppMode::Normal), so it only ever asked about Normal-mode bindings: add-record-row (RecordsNormal) and delete-formula-at-cursor (FormulaPanel) were outside its reach entirely and could never have failed it. A named list that silently scopes itself to one mode is a guard that looks broader than it is.

+

This asks it of every mode, and it asks it the only way that is not over-reporting. The naive version resolves every advertised name against an EMPTY ClientState and flags edit-or-drill, commit-cell-edit, create-and-switch-view and page-prev/page-next — but those are handled names refused by STATE, which is correct behaviour and not this bug. So the fixture is furnished: every widget has something in it, a basis is published, and a window is announced, so an Upstream can only mean the name is unhandled.

+

Two things it does NOT do, each because the first attempt did and was wrong. It does not call step with the advertised args: advertised carries None for "any binding of this command", and step's numeric and mode arguments are mandatory, so that reports move-selection, move-panel-cursor and move-tile-cursor as gaps in every mode. And it does not tokenise the rendered hint string: help::advertised is a table of command NAMES, so the cross-reference is exact and anything reading the rendered text is solving a harder problem than exists. The keys come off Keymap::keys_running, which labels_for now shares so "which keys" and "how are they spelled" cannot give two answers.

+

It watches BOTH silent variants, and the first version did not. KeyOutcome has two ways of saying nothing: Upstream, and Ignored — which KeyOutcome::of(vec![]) produces whenever a resolver composes an empty vector. To a user they are the same black hole. The first version asserted resolve(..) == Upstream(event), so three of the six commands this whole change exists to make audible could go completely silent with the entire evidence base green: degrading only add-record-row, delete-formula-at-cursor and view-back to Step::Declined(Vec::new()) left 1397 Rust tests, 19 harness tests and 26 node tests passing, because no browser spec presses those three keys. Found on review of PR #71. The check is now !matches!(.., Upstream(_) | Ignored).

+

The generalisation is worth more than the fix. KeyOutcome::of turning an empty Local into an honest Ignored is good design and it is also a laundering step: every wrong answer that composes nothing arrives wearing the variant that is usually innocent. So a guard's discriminator has to be state — did anything the user can see change — and never the shape of the return value. That is exactly why the +cat injection in a_locally_committed_creating_column_agrees_with_the_host failed (it compares documents and cursors) and the decline injection passed everywhere (nothing compared a screen).

+

Widening the check costs ten more entries, all one legitimate boundary case: move-panel-cursor mirrors MovePanelCursor, which returns nothing when the cursor does not move, so a one-row panel or a cursor on an edge answers Ignored for a reason that is not this test's subject. furnished therefore carries three of everything with all three panel cursors seated at index 1. Note what that rules out: seating them at 0 and calling the Up results expected would have hidden the real defect in the same list as the boundary case.

+

The injection: delete the "toggle-prune-empty" arm from declined. The test fails naming NORMAL: Char('P') and RECORDS: Char('P') — both modes, which is the widening the ticket was about, and a build that ran rather than a compile error. The review's own injection is also kept as a check: degrading the three unpressed declines to empty now fails naming FORMULAS: Char('d'), FORMULAS: Delete, RECORDS: Char('<') and RECORDS: Char('o').

+

What it found that the ticket did not predict: view-back is advertised in RecordsNormal (<, captioned "back"), so it was a seventh instance of the same class. The ticket had it filed under the Normal-mode host list instead, where being upstream is not a defect.

+

The five that remain are listed by name with a reason each. Three of them — commit-export, commit-save, execute-command — are reachable only from a mode declined refuses at the door, so a browser never stands in front of the hint; the other two are improvise-gcvb's records-cell commit, which needs record_col.

+

#a_locally_committed_creating_column_agrees_with_the_host

+

improvise-gcvb. resolve's commit returned None for the whole of records mode, and the reason given was sound for the coordinate and measure columns — a category column renames every measure cell in the row, which needs record_col (gotcha 1). The guard was broader than its reason: a cell at the cursor now projects CellId::Create, and committing one is a single existing ModelCommand over the typed NAME that reads no record_col and writes no cell.

+

Both halves are proved by injection, and the second is the one that matters. Putting the broad is_records() early return back makes the test fail at "the client must answer this key locally". Deleting the CreateKind::Category early return — so +cat advances like any other column — fails with the client in RecordsEditing where the host is in RecordsNormal: the TUI returns straight after a +cat commit with a RebuildLayout and no cursor move, because an inserted column renumbers every index to its right and the advance was computed from the pre-commit layout (gotcha 23). A client that advanced anyway would land one column off, silently.

+

both_ends compares the document as .improv text, the mode, the status and every cursor, so neither half can pass by agreeing on the wrong thing. The fixture drives both hosts into records mode with the R key rather than hand-building a _Records view, because twin_of copies only the session slice and a twin without the view would refuse the commit for the wrong reason.

+

#every_minibuffer_mode_paints_what_is_being_typed

+

improvise-fdt6. commit-buffer formula already resolved on the client — that half of the ticket was closed by an earlier wave — so the browser could enter FormulaEdit, type a formula and commit it. What did not exist was any way to SEE it: RenderModel::editing is populated only when mode.is_editing(), which is the two cell-editing modes, and it reads the "edit" buffer by name. Every character typed into FormulaEdit, CategoryAdd, ItemAdd, SavePrompt, ExportPrompt or CommandMode went into a buffer nothing painted. The mode opened, the user typed, and the screen did not move — which is exactly the dead-room shape the ticket names, arrived at from the rendering side rather than the resolution side.

+

The value is read with improvise_protocol::read_buffer rather than buffers.get, so the "search" special case (gotcha 11) keeps its one home.

+

The injection: .filter(|_| state.view.mode.is_editing()) on the new field, which is the pre-fix behaviour exactly. It fails naming "formula".

+

And the static check was not enough, which is the part worth keeping. web/tests/render_surface.test.js asserts that every pub field of RenderModel appears as model.<field> somewhere in app.js. It passed while the page was still broken: the string model.minibuffer was inside paintMinibuffer, and paintMinibuffer was never called and el.minibuffer was never looked up. A field can be mentioned by a function nothing invokes. harness/tests/keys.spec.js caught it on the first run, in the browser, which is the whole argument for that directory existing (gotcha 48).

+

#nothing_declined_is_also_composed

+

step is declined then composed, and until this test the claim that the two lists cannot overlap was a doc comment. The dangerous direction is not a typo: it is a future ticket implementing view-back (or toggle-prune-empty, which is one projection field away) in composed and not thinking to delete its declined arm. The browser then goes on saying "not available in the browser yet" while the working code sits unreachable, and nothing notices — the classic shape of a check passing while the thing it names is broken. Raised on review of PR #71.

+

It is derived from declined itself, not from a list beside it: it walks every (name, args) bound in every mode's keymap — parents and prefix sub-keymaps included, sequence steps included — and for each one declined answers, asserts composed does not. A seventh decline is covered without coming here, which is the property a hand-written list could not have.

+

The injection: add "view-back" => Some(vec![Command::SwitchView(..)]) to composed. It fails naming NORMAL: view-back [] and RECORDS: view-back [].

+

It also found a real overlap on its first run. enter-mode is answered by declined for the three stranding modes and by composed for every mode, so composed's arm would have opened SavePrompt, ExportPrompt or CommandMode if declined were ever bypassed. Rather than moving the check, both halves now hold the rule — composed returns None for a stranded mode — so the failure direction if one is removed is the pre-PR Upstream, not a dead room. ---

+

#harness/tests/help.spec.js — the overlay, asserted as words

+

PR #61 deleted "? opens help, and its rows are the shared ones" and nothing replaced it, so paintHelp, help, help_page and help_titles reached no assertion at all (improvise-78v6). What contract.spec.js carried in its place said only that a dt was VISIBLE — and an empty dt is a visible dt (improvise-igxp), which is the same shape as the caret bug above: a row is a frame, and a frame is not a word.

+

The injection both tickets name: blank keys.textContent in paintHelp, patched in harness/serve/app/app.js. Before this work it left all three contract tests passing. It now fails both tests in this file (the Welcome banner reads empty; the advertised key reads empty) and contract.spec.js, whose two help lines were strengthened from toBeVisible to the banner's text and the first dt.help-keys's text.

+

What the pairs helper is for. The overlay is a <dl>, and the association between a key row and its description is document order — dt then dd. Read as two lists they zip wrong, because a prose row emits no dd at all (deliberately: an empty one would leave a blank grid cell). Welcome is prose end to end, so dd count zero there is an assertion about that rule and not a coincidence.

+

#"a key the overlay advertises does what the overlay says it does"

+

The assertions above are still about a string on a screen; this one asks whether the string is true. The key for Jump to last row is read OUT OF THE DOM, the overlay closed, the key pressed, and the selection asserted onto the last row. A page drawing a plausible key it does not have — what a hand-spelled table decays into — fails here, and the spec holds no second copy of the keymap.

+

It presses the FIRST key the row names because a row is a compaction of every binding that reaches the command (h j k l ← ↑ → ↓ is eight).

+

#harness/tests/grouped.spec.js — the first browser run of improvise-anr9

+

web/demo.improv has no groups and one category per visible axis, deliberately (gotcha 46), so the multi-level header, the group span, the collapse indicator and the inline group row — exactly the code improvise-anr9 added — had never been rendered anywhere a test could see (improvise-plcx). The fixture is harness/fixtures/grouped.improv, served beside the shipped demo by harness/build.sh, and the shipped document is unchanged.

+

A missing seed is not an error: web/worker.js answers a fetch that 404s with an empty model. So a typo in the filename yields a page that draws a grid, and "a grid is drawn" would pass on it. Every assertion here is about labels the fixture really carries.

+

openGrouped waits with retrying assertions and the tests then read once. The head and the row labels are read in a single evaluate each, because what these tests assert is the RELATION between the lines — and a one-shot read cannot wait. The grid is painted from the first projection and reframed by the viewport the page then announces, so the helper gates on the head's shape, its first group and the first row's innermost label before anything is read.

+

Five injections, five distinct failures, patched in harness/serve/ so no rebuild stands between the fault and the run:

+ +

A bug this fixture surfaced rather than predicted. Coastal is the row frame's FIRST entry and its header is never drawn: the spec asserts two group rows where the document has three, with improvise-0o6x named on the line. That count is a record of an open bug, not an endorsement — when 0o6x lands it becomes 3 and this line is where the change must be argued.

+

#harness/tests/landing.spec.js — the front door

+

docs/index.html is copied to the site root by web/assemble.sh, so the harness has been serving it all along with nothing asserted about it (improvise-bgtf).

+

Counting the four .ap-player elements proves nothing, and that is the whole reason this is a spec rather than a one-liner: autoPlay is false, so asciinema-player fetches no cast until play is pressed. A page whose every cast path 404s constructs four players, throws no page error, and looks identical to a working one. Injected while the page was being written (./casts/ -> ./cats/) the construction-only check passed on the broken page.

+

Re-injected into harness/serve/index.html now, the spec fails at the first player: .ap-line elements exist only once there is a cast to lay out. The spec also asserts, BEFORE pressing anything, that no .cast has been requested — without which the network evidence below could be satisfied by a page that fetched them eagerly, which is not the page we ship.

+

Two assertions where one looks sufficient: the rendering says a cast played, and the recorded responses say four distinct casts answered 200. Three players showing the same recording satisfies the first and fails the second.

+

"the front door reaches the app it advertises" follows the footer link rather than asserting its text, because whether ./app/ resolves is a fact about the assembled tree. Injecting href="./aplication/" fails it.

+

#harness/tests/contract.spec.js — the ninth injection, and a lost guard

+

The lost guard. improvise-78v6 asks for the test-name diff over PR #61's range, and it finds a fourth loss beyond the help test: "a click on an item row filters, as Enter does". contract.spec.js asserts the CATEGORY arm of click_category_row (the row expands) and nothing asserts the ITEM arm, whose rule is "do what Enter does here" — filter. Only #status distinguishes them, so a click wired to the wrong arm passed the whole file. Restored inside the click test, and proved by injecting click_category(row.is_category ? index : 0), which fails that line alone.

+

The diff that finds this is two commands, and is worth running on any PR whose subject is a lost guard:

+
M=$(gh pr view 61 --json mergeCommit -q .mergeCommit.oid)
+for r in "$M^1" "$M"; do
+  git ls-tree -r --name-only "$r" -- harness/tests web/tests |
+    while read -r f; do git show "$r:$f" | grep -oE 'test\("[^"]*"'; done
+done
+

The other two losses are deliberate: pointer.spec.js's sidebar test was superseded by the open-at-launch ruling, and the widgets file was merged into this one.

+

#the_keys_that_compose_a_switch_view_are_these

+

improvise-lltt is deferred on a premise, not a judgement: its own note says it is "unreachable today ONLY because no shipped client emits Command::SwitchView", and asks for a guard asserting exactly that, so the silent failure becomes loud the moment the premise stops holding.

+

The guard failed on its first run. Two live paths in this client's own resolver (resolve.rs): switch-view-at-cursor and create-and-switch-view. The other half was already live — the same resolver composes ModelCommand::TransposeAxes. So the scenario lltt describes is reachable today from the shipped view panel, and the ticket was escalated P2 → P1 rather than fixed here, the honest fix being the protocol change it describes.

+

It is an assert_eq against those two names, not an assert-empty, and the list is not an exemption list. It pins what is true so a third live path fails, and its message says so — because a reader who found this passing would otherwise conclude the premise holds. When lltt is fixed the list empties and this test goes with it.

+

Why the enumeration and not a read: the bug was already written into a passing test's comment elsewhere ("Aimed at the host's own view… HideItem writes view_mut()") and had been read as a fixture note for as long as it had been there.

+

#"a click moves the selection from an editing state too"

+

improvise-0x9u, and the fault was injected by running this assertion against the bundle as it stood: cells.nth(4).locator("input") came back 0, with the selection still on cell 0, exactly as the bug report measured in Chromium.

+

Two things about it are worth more than the assertion. First, what it does not do: it never presses Escape between the two clicks. The sibling test does, and that single line is why the property it names ("a click moves the selection") was asserted for a whole wave while the neighbouring state was broken — the click path from an editing page was never exercised.

+

Second, the fix's own first attempt failed this same test, which is the part to remember. Moving the handler to mousedown is necessary and not sufficient: close still repainted, and removing a focused <input> fires blur synchronously, so close rebuilt the table from inside itself and detached the <td> the handler was about to open an editor on. The assertion could not tell that apart from the original bug — same symptom, different cause — so it is the debug run rather than the test that separated them. close therefore no longer repaints, and the blur arm checks closed before dismissing.

+

Third — and this is the edit the repair nearly shipped unguarded — the fix is three edits, not two, and the third is the one with the most careful comment: close restores td.textContent (the drawn value) instead of seed (the stored one), because close no longer repaints and repaint() early-returns for as long as editing is set. Reverting that one line to td.replaceChildren(original) left the whole suite green, and the defect was on the front page: web/demo.improv carries format: ,.0 over 18400, and this test's own two clicks — no Escape between them — leave cell 0 reading 18400 where the page had drawn 18,400, for as long as the second editor stays open. The assertion added for it is toHaveText("18,400") after the second click, and the injection is exactly that revert: it fails with unexpected value "18400", naming this test and no other.

+

The pattern is worth carrying past this entry. A repair of N edits whose N−1 guarded ones pass is not a guarded repair, and the unguarded edit is reliably the one the author understood best — which is why it carries the clearest comment and why it felt covered. For each distinct edit, name the test that fails when only that edit is reverted; a line with a comment and no answer to that question is the line to inject.

+

#"the widgets the demo document fills are drawn, in the state that fills them"

+

The #doc-name row is improvise-5jna, and the reason it belongs in a browser test rather than in render_surface.test.js is that the latter is a substring search over the page's source: doc_name mentioned by a function nothing calls reads as painted. That is how model.minibuffer once shipped dead, and how z754's three title-bar fields sat projected and unpainted for a wave while the static check was green. Proved by adding the assertion first: element(s) not found for #doc-name.

+

The [+] marker is deliberately not asserted here. The standalone deployment persists on every accepted batch and effect::Save clears the flag in the same round trip (gotcha 45), so the marker is transient in a browser and any timing assertion over it would be flaky rather than discriminating. Its spelling — the order name (stored_as) [+], the null stored_as writing no empty parens — is web/tests/shell.test.js, over documentTitle directly.

+

#"a selection driven below the fold scrolls itself into view"

+

improvise-jq8l, and the expensive half was the fixture, not the assertion. Neither shipped document can reach the state: web/demo.improv is four rows and harness/fixtures/grouped.improv is eight, and a probe measured both at scrollHeight === clientHeight in a 484px box — nothing to scroll, so the assertion would have passed against a page that never scrolls. That is the vacuity this entry exists to record, and it is why the spec asserts scrollHeight - clientHeight > 0 before it presses a key.

+

harness/fixtures/tall.improv is thirty rows in three groups, which overflows for a reason worth stating: ROW_PX is 26 and a real row measures 28.6px, and inline group rows are not charged for at all, so the page renders more rows than its own box can show. Measured against the unfixed page: 105px of overflow, the selected cell 96px below the visible bottom, scrollTop still 0 after twenty-five ArrowDowns.

+

Both directions are asserted, and neither alone is enough: the cell being inside #grid's rect would also hold if the selection had never moved, and scrollTop > 0 would also hold if the page scrolled to the wrong place.

+

#"the casts are legible on a phone, and the page still does not scroll sideways"

+

improvise-o31u, and the instrument is the finding. getBoundingClientRect reports a height for a content-visibility-hidden child, and heading text overflows its box without widening it, so neither "the element is present" nor a box measurement can see this defect at all. What a reader sees is the computed font size, and that is what is asserted.

+

Measured against the unfixed page, and it reproduces the ticket's figure to five digits: pivot.cast renders its terminal at 4.83258px on a 390px viewport. The arithmetic is not subtle — the player is told cols: 120, the container is ~356px inside the page's 16px gutters, and the player DERIVES its font size from the width it is given. After the fix: 11.9457px.

+

Two assertions guard the trade this fix must not make. The page's own scrollWidth - clientWidth must stay 0 — a document that scrolls sideways on a phone is a worse bug than one unreadable section — and #cast-pivot must be the thing that scrolls, which is what separates "contained the overflow" from "removed the overflow by shrinking something else".

+

#"a desktop viewport is unchanged by the phone fix"

+

The negative arm, and it is forward-looking: no fault available today makes it fail. The entry previously named two mechanisms and both were measured, at a 1280px viewport, against the shipped page:

+ +

The arithmetic is why, and it is worth stating rather than re-deriving: at 1280px #cast-pivot measures clientWidth === scrollWidth === 966, so an 880px floor is inert at desktop wherever it is placed. The floor would have to exceed the ~966px the player actually gets — or the content column would have to narrow — before either assertion here could move. That is a real future fault and the reason to keep the test: it pins 13.11px and "nothing to scroll" against a floor raised past the player's own width. It is not a reason to claim the arm currently discriminates between the two placements, because it does not.

+

Note also that the phone arm is stronger than this entry used to credit it with: the misplaced-min-width fault, attributed here to the desktop arm, is caught there. scrolls.page === 0 is the assertion that separates "contained the overflow" from "moved it onto the document".

+

#"no cast carries the recording machine's shell, clock or geometry"

+

improvise-2oxt, and both clauses were injected separately, which matters because the first would otherwise mask the second. Serving the pre-2oxt pivot.cast fails on env — Received: {"SHELL": "/bin/zsh", "TERM": "screen-256color"} — and stops there. Stripping env and timestamp from that same header and serving the body unchanged then fails on pivot.cast sets a window title, which is the clause that actually watches for the leak: an OSC window-title sequence, emitted by a shell once per prompt and carrying the cwd, and emitted by improvise never.

+

Asserted over the cast file rather than over the rendering, and that is the finding. A player draws a shell prompt as ordinary terminal text, so a screenshot or a toContainText assertion would have to know what the hostname and home directory had been — a check that can only be written by someone who has already seen the leak. The file-level signature can be written by anyone.

+

What no assertion here reaches, and how it was checked instead: whether each cast SHOWS what its section claims. That is not a property of the bytes. It was verified by replaying each recording through a small ANSI reconstruction and reading the frames as text, which found two scenarios that recorded cleanly and demonstrated nothing — a drill into a cell with no records behind it, and a c pressed on a category already on the column axis. Both are true frames about the wrong thing, which is the failure mode this repo keeps meeting, and neither was visible from the keystroke list.

+
+

#improvise-ws-server — the Origin check (improvise-kc7n)

+

The bug: grep -rni origin crates/improvise-ws-server/src/ returned nothing. The server inspected no Origin header and accepted every handshake, and a websocket handshake is exempt from the same-origin policy — no CORS preflight — so any page a user visited could open a socket from that user's own allowed address and read and edit every document served. An address check cannot see it; the argument is context/code/clients.md §Origin.

+

Five edits, five guards. Each was reverted alone and the named test failed; nothing here is covered only by a comment. Review added a sixth — see two_origins_state_none_this_policy_can_answer_for.

+

#a_handshake_from_a_foreign_origin_is_refused

+

The reproduction, written first and watched fail on 2afbf37: a handshake carrying Origin: https://evil.example connected, and the test's connect_async(...).err() was None. After the fix the upgrade is refused with a 403 and the assertion is on the status, not merely on "an error happened" — a refused upgrade and a dropped connection are different outcomes and only one of them is the fix.

+

Injection: replacing the handshake callback's admitted(&origins, req) with Ok(()) — the transport collecting a policy it never asks — fails this and a_named_origin_reaches_the_document, and nothing else in the workspace.

+

#a_handshake_with_no_origin_is_accepted

+

The case most likely to be wrong later, so it is named rather than left to a match arm. Non-browser clients send no Origin: curl, a native client, this file's own other tests. Injection: flipping OriginPolicy::permits's None arm to false fails this, an_absent_origin_is_allowed, the_handshake_refuses_what_the_policy_refuses, and both pre-existing socket tests — which is the point: the repo's own instrument is the non-browser client this arm is about.

+

#a_handshake_from_a_loopback_page_is_accepted

+

The shipped local recipe — the page served by python3 -m http.server 8080, the socket on another loopback port — must keep working with no flag, or the default is a different bug from the one being fixed. Injection: making is_loopback return false fails this and a_loopback_page_is_allowed_by_default and leaves the refusal tests green, which separates "allows loopback" from "allows everything".

+

#a_named_origin_reaches_the_document

+

--allow-origin end to end over a real socket, plus the negative arm in the same test: widening for one page does not widen for its neighbours. Without that second assertion the test passes against a policy that permits everything, which is the fault E1a injects.

+

#allow_origin_widens_the_policy_and_nothing_else_does

+

The wiring, which no socket test can reach: the other tests build an OriginPolicy directly, so a flag collected by clap and dropped on the floor is invisible to every one of them. This drives Args::parse_from and asserts both directions — bare refuses the deployed https origin and still allows loopback; widened allows the named origin and not its neighbours. Injection: making Args::origin_policy return OriginPolicy::default() fails this test alone.

+

#two_origins_state_none_this_policy_can_answer_for

+

Found by review rather than by the ticket. HeaderMap::get is first-wins, and the deployment this README recommends puts a reverse proxy in front of the socket — a proxy that appends Origin rather than setting it leaves two, and reading the first lets the second be anything at all. No browser sends two, so two is not a state any policy can answer for and the handshake is refused. Injection: restoring the first-wins read ((Some(value), Some(_)) answering from value) fails this test alone, across the whole workspace. The test also asserts the allowed origin by itself is still allowed, so the refusal is demonstrably the second header rather than the first.

+

#a_refusal_is_a_forbidden_response

+

The refusal has to be a refused upgrade, and the socket test asserts the Error::Http kind and the status rather than matching "403" in a Display string: a dropped connection and a refused handshake are different outcomes and only one of them is this fix. Injection: changing forbidden() to UNAUTHORIZED fails this and a_handshake_from_a_foreign_origin_is_refused, which is what shows the socket test reads the status rather than merely observing that connecting failed.

+

What the signature covers instead of a test: transport::serve takes the policy, so a host cannot forget to pass one — doctrine(an_invariant_is_made_ unrepresentable_not_validated). What remains uncovered and is stated rather than guarded: a proxy that strips Origin turns the check off, which is why the proxy's own Origin allow-list stays in README.md as defence in depth.

+

#the_fixtures_basis_tracks_a_host_whose_layout_has_moved

+

improvise-1l9o. furnished_client and client_ready_to_edit built their BasisCache by walking the client's own grid for Record cells and calling basis::published(0, 0, records) with every version hard-coded to zero — a small re-implementation of project.rs::basis_update's record selection, in a file whose entire purpose is to rule out re-implementations. Both now take the Projection::Basis out of a real Sessions::connect snapshot.

+

The reason the ticket calls it a nit is the reason it needed a test: a fresh App really is at generation 0, layout 0 and every record at version 0, so the fabrication and the publication are indistinguishable on every existing fixture. All 23 lockstep tests pass under either. So the change alone is structural and untested by construction, and this test is what makes it observable: it moves a tier first — a HideItem through apply_on, which observe_reshape sees as a shape change — and then asserts the fixture's claim names the host's current tiers.

+

Note SetNumberFormat does not work here and was the first attempt. It declares EffectKind::Model(ModelChange::LayoutGeometry), but layout_version is observed from view_shapes rather than read off the declaration (improvise-pdvm), and a number format is not part of a view's shape hash. A command being classified LayoutGeometry is therefore not a promise that it moves layout_version — which is the observed-not-declared design working as intended, and easy to misread as a bug.

+

Fault injected 2026-09-21: re-fabricating the basis inside published_basis (published(0, 0, real_records) — keeping the real record set, so only the two document tiers are forged) fails this test with left: (0, 0) against right: (0, 1), and fails nothing else in the workspace.

+

One residual, found in self-review and not closed. published_basis asks Sessions::connect for a window of window.0 x window.1, and a fresh session sits at offsets zero; client_of builds its grid at the cursor's offsets. Every fixture here has its cursor at the origin, so the two windows coincide and the published record set is the one the client's grid names. A future fixture that scrolls before building its client would get a basis naming records its grid does not show — which claim would then silently omit, and Rejection::RecordUnstated is what the server answers with. The fixtures no longer fabricate, which was the ticket; they do still assume a cursor at the origin.

+

#leaving_a_panel_in_records_mode_lands_where_the_host_lands

+

improvise-e5pf. AppMode::Normal is settled by the host against the layout the batch ends on; the wire carries Command::ChangeMode(AppMode::Normal) and reduce_view folds it verbatim. So a client that opened a panel from records mode and pressed Esc landed in Normal while its host landed in RecordsNormal — and RecordsNormal has its own keymap, so the divergence compounds on the next keypress.

+

The reachable path, which the ticket did not name: Tab in RecordsNormal resolves locally to ChangeMode(CategoryPanel), and Esc there resolves to ChangeMode(Normal). c/f/v are Ignored in RecordsNormal and R itself is a stranding decline, so the panel keys a first draft of this test used never separated anything — the test panicked on "the client must answer this key itself" rather than on the drift.

+

Why lockstep could not see it before. Its records coverage (typing_at, records_pair) set host.view_state.mode by assignment and seeded the client with client.view = app.view_state.clone(), so it started where the mode and the layout already agree, and every divergence here requires them to disagree — doctrine(a_guard_seeded_on_agreement_cannot_detect_drift). in_lockstep drives the key on both ends and then refreshes the client's grid from a real projection, never from the host's slice, which is the only seeding that can show the two apart.

+

Fault injected: resolve::changed_mode reduced to Command::ChangeMode(mode). The test fails on close: the client and the host disagree about the mode, Normal against RecordsNormal.

+

Two existing resolve.rs fixtures had the same shape and had to be corrected with the fix: furnished and a_records_mode_commit_is_the_hosts put the client in a records mode over a non-records grid, which is a state no host can produce. Both now project GridUpdate::Records as well. That they had to change is the evidence that the substitution was load-bearing rather than incidental.

+

#a_records_commit_is_declined_off_the_grid_and_not_off_the_mode

+

improvise-e5pf's damage half. resolve::commit is written to decline a records commit it cannot compose (it needs record_col, gotcha 1). It asked state.view.mode.is_records(); with the mode drifted to plain Editing over a records grid it did not decline, and composed typed_cell_command plus a Basis claim against a grid whose columns are [coords…, +cat, measures…, +]. A wrong write, strictly worse than the black hole it exists to produce.

+

The test states the drift rather than reaching it — reaching it takes the key sequence the test above drives — and asserts no Command::Model is composed.

+

The control is the point. A first version of this test passed against the broken code, because client_of publishes no basis and BasisCache::claim returns None until one arrives, so commit returned None for a reason that had nothing to do with records mode: doctrine(an_injection_that_does_not_fail_names_an_untested_path), caught by injecting the old spelling and watching the test still pass. records_client now publishes a basis naming every record the grid projects, and the test carries a positive control — the same Editing mode over a regular grid must still compose a write — so "this client never writes" cannot pass it.

+

Fault injected: the decline restored to state.view.mode.is_records(). The test fails on a records commit this client cannot compose was composed anyway.

+

#the_normal_pair_is_settled_in_both_directions

+

App::settle_normal_mode settles both Normal and RecordsNormal against the layout, so a RecordsNormal over an ordinary grid becomes Normal. The first version of the client's settling mapped only Normal → RecordsNormal, which left the mirror-image drift live. Found in review of this branch.

+

The test now aims at improvise_protocol::settled_mode, which is the one rule all three ends call — ClientState::fold, ClientSession::fold and App::settle_normal_mode.

+

Fault injected: the (Normal | RecordsNormal, false) arm narrowed to (Normal, false). The case records=false, holding RecordsNormal fails. The fourth row and the pass-through cases are the negative arms — without them a rule that answered RecordsNormal always, or Normal always, would pass, and the pass-through loop is what pins that Editing/RecordsEditing are not settled (the reason a records decision asks the grid and never the mode).

+

#switching_between_a_records_view_and_a_pivot_view_settles_both_ways

+

The P1 ChatGPT Codex found on PR #98, and the more useful half is why the first fix missed it. That fix settled the mode at each composition site (resolve::changed_mode), against the grid the client held when it resolved the key. switch_view_at composes [SwitchView(target), ChangeMode(Normal)], so it settled against the outgoing view: records → pivot emitted RecordsNormal and pivot → records emitted Normal, recreating the very drift the PR closed.

+

The hand-back had predicted this as follow-up 3 and concluded it was unreachable because R is a stranding decline in the browser. That reasoning is right about the toggle route and wrong about this one: records-ness is auto-detected from the axes (gotcha 1), so a view switch changes it with nothing in the batch saying "records". One door was enumerated and there were two — doctrine(an_unreachability_claim_is_only_as_wide_as_the_routes_enumerated). Codex had flagged the same path on PR #91, where the host side was fixed and the client side was not, so it was tracking one defect across three PRs.

+

The remedy is general rather than a patch to switch_view_at: changed_mode is deleted, every site emits the plain AppMode::Normal, and ClientState::fold settles after every fold against the grid it holds. Route-independent, so no enumeration of composition sites is load-bearing any more.

+

The fixture drives the real path: a Host in records mode, a session connected (which opens the document's default view, gotcha 29 — the first draft asserted the opposite and failed its own precondition), the composed pair folded locally and sent as a ClientFrame::Commands, and every reply frame delivered in arrival order. deliver takes the whole reply rather than a hand-picked projection.

+

The composed ChangeMode carries the STALE answer deliberately — settled_mode(&Normal, client.grid.records) off the outgoing grid — because that is what switch_view_at really emitted, and composing the friendly value instead would make the records → pivot direction pass under the fault.

+

Fault injected: ClientState::settle_mode reduced to a no-op. Both directions fail, each proved separately because the loop stops at the first:

+ +

Records-ness and the mode are asserted as two separate facts rather than one derived from the other — a first version computed the records precondition from the mode expectation, so relaxing the mode silently relaxed the precondition too and the test failed for the wrong reason.

+

It has a host-side twin, and the two are a pair rather than a duplicate. improvise-4dpj landed switching_views_across_records_ness_settles_against_the_view_switched_to in ui/app.rs while this was being written: the same defect, the same batch ([SwitchView(to), ChangeMode(Normal)]), the same crossing in both directions, and independently the same phrase in its message — "settled against the view it LEFT". One is the host settling after the batch against the layout it ends on; this one is a client settling after the fold against the grid it holds. Neither is redundant: nothing in the projection stream carries the host's ViewState (gotcha 47), so a host-side guard cannot see a client's mode and the reverse. Delete either and that side goes unwatched.

+

#the_mode_a_switch_lands_in_is_the_one_a_host_on_that_view_holds

+

The lockstep half of the entry above: it asserts the host itself answers differently per view (Normal on the pivot, RecordsNormal on the records view). Without it, "the client settles to X" is a claim about the client alone — a host that answered Normal everywhere would make the switch test's expectations arbitrary rather than lockstep.

+
+
+ + diff --git a/site/docs/test-evidence/command.html b/site/docs/test-evidence/command.html new file mode 100644 index 00000000..f6de07e2 --- /dev/null +++ b/site/docs/test-evidence/command.html @@ -0,0 +1,927 @@ + + + + + +improvise-command — improvise docs + + + +
+ +
+

Source: context/test-evidence/command.md

On this page

improvise-command — test evidence

+

Why each of these tests exists, what bug it reproduced, and what fault was injected to prove it can fail. Keyed by exact function name; a rename orphans the entry, and scripts/check-comment-runs.py --check-evidence reports that (improvise-9c8c, design-principles §7).

+

Entries cover tests and the helpers whose shape is the evidence.

+
+

#ui/reduce.rs

+

#a_view_slice_command_does_not_move_the_authoritative_view_state

+

This test used to assert the opposite, under the name view_slice_commands_land_in_the_authoritative_view_state, and it passed against a path no host walks (improvise-s17o): apply_command's fallback arm folded a wire view command into app.view_state, the authoritative slice. Nothing in production reached it — Sessions::apply_checked folds a view command into the asking client's mirror — so the behaviour it pinned was wrong and safe only because it was dead. The model is shared, the UI is not (principle 5, gotcha 36); a client's SetStatus must not move the host's status line.

+

The fixture writes "host" into the status line first, so the assertion distinguishes "the fold was removed" from "the field happened to be empty" — with the default empty string, deleting the fold and leaving it in place look the same for SetStatus. Restoring reduce_view(&mut app.view_state, other) fails on the first field.

+

#Helper — checked

+

Not a test — the fixture every reduce.rs test below reaches the door through.

+

For the tests whose subject is not staleness: a command the current state accepts, which every command built by caught_up is by construction. The tests whose subject is staleness call Checked::against themselves and assert on the Rejection, so neither kind can pass by accident through the other's convenience.

+

#switch_view_moves_the_client_and_leaves_the_documents_default

+

The successor to switch_view_moves_the_document_and_the_client_together, which asserted the pre-improvise-jytm lockstep. Which view a client is on is now that client's; Workbook::active_view is the default a fresh client opens and what .improv writes. The fault it rules out is the old behaviour returning by inertia — any writer that moves the workbook's copy from a client's switch fails it.

+

Fault injected 2026-09-20: restoring App::switch_to_view's workbook.switch_view(name) call fails this and the_documents_default_view_outlives_this_clients_switches.

+

#a_wire_view_switch_records_history_the_way_a_key_switch_does

+

A wire-driven view switch owes the same history a key-driven one leaves behind.

+

effect::ViewBack pops view_state.view_back_stack, and the only thing that ever pushes onto it is effect::SwitchView. While apply_command performed the switch itself — calling App::switch_to_view directly rather than running the effect — the stack stayed empty, so a client that switched over the wire could not go back, though the same two keypresses could. Wave 4 is what made that reachable: before ModelCommand there was no wire way to ask.

+

Nothing else in the suite could see it. switch_view_moves_the_client_and_leaves_the_documents_default checks where a switch landed; this checks what it recorded.

+

Bug demonstrated 2026-09-19 (Copilot, PR #11): the ViewBack left the document on "Second". Since improvise-jytm the step back is driven through effect::ViewBack rather than ModelCommand::ViewBack, which left the wire with that ticket — the stack it pops is one client's history, so a nullary wire command could not address it (gotcha 38). What the test asserts is unchanged.

+

#switching_views_projects_the_axis_widgets_only

+

Switching views changes what the axis-bearing widgets read, so they are told; the list of views itself did not change, so it is not.

+

#an_inbound_projection_is_ignored

+

Projections travel one way. A client that sends one back must not be able to write it into the server's state.

+

#a_keypress_that_changes_the_model_projects_it

+

The key path is the model's door, and it reports the widgets it touched so a host can project them.

+

#a_view_only_keypress_projects_nothing

+

A keypress that moves only this client's cursor reaches no other client (gotcha 3).

+

#every_model_command_names_an_effect_that_declares_a_model_change

+

The derivation, stated as a check. This is what "derived, not invented" means operationally (doctrine(derive_the_enum_from_the_effects)). A variant pointed at a ViewOnly effect would be a wire message that changed the document for the client that sent it and for nobody else — Dirty::of would report NOTHING, and no projection would ever correct the others (gotcha 35).

+

#Helper — model_app

+

Not a test — the fixture the three every_model_command walks run against.

+

app() with a cell actually stored, so a surrogate for it resolves through the index rather than only through the layout. The walks over every_model_command run against this: a SetCell whose record cannot be resolved produces no effects, and every one of those tests asserts on the effects, so the fixture has to name something real or they would all fail for a reason that is not theirs.

+

#no_model_command_commits_another_clients_session_local_state

+

improvise-3x3u: the wire vocabulary is the addressable model effects, not all of them.

+

Sessions::apply routes a Command::Model with no SessionId, which is right exactly when the change is addressed to no one session (gotcha 36). ApplyAndClearDrill and AcceptWizardModel are addressed to precisely one, and it is not the sender: an unaddressed ApplyAndClearDrill commits another client's private typing (improvise-tom1) and an unaddressed AcceptWizardModel swaps the document out of a dialogue the sender never opened. Both were demonstrated against Sessions::apply before the variants were removed.

+

Re-adding either variant means making it addressed — carrying a SessionId, with the staging map moved onto the session — not restoring the nullary form this test forbids.

+

SESSION_LOCAL_EFFECTS is the list of the two effect names.

+

#the_session_local_effects_are_real_model_effects

+

The guard above is only worth having while the effects it names still exist and are still model changes — otherwise it would pass by naming nothing (doctrine(check_the_checker)).

+

#the_effect_name_reader_reads_all_three_debug_shapes

+

The reader above is itself a check, so it gets the injection too (doctrine(check_the_checker)): one effect of each Debug shape, because the leading identifier is followed by a different delimiter in each — nothing, (, and {.

+

#Helper — effect_type_name

+

Not a test — the reader that test injects against, used by the two guards above.

+

The type name of an effect, read off the leading identifier of its derived Debug. Effect is a trait object, so std::any::type_name is not reachable through it; every effect derives Debug, and a derived Debug opens with the type's own name for a unit struct, a tuple struct and a braced struct alike. So the name is already on the value — this only reads it back.

+

#every_model_command_names_the_effect_of_the_same_name

+

The mapping is name-for-name by construction, so state that: every ModelCommand variant names the effect that carries its own name.

+

This is the check that separates an effect from its inverse. every_model_command_names_an_effect_that_declares_a_model_change cannot: HideItem and ShowItem, AddItem and RemoveItem, CreateView and DeleteView, SetCell and ClearCell are each two effects of the same EffectKind, so a swapped pair clears that bar while inverting what the wire message does. And a_model_command_lands_where_its_keypress_lands cannot reach them either — most variants have no single keypress to compare against, so it drives three of twenty-eight.

+

Proven by injection 2026-09-19: HideItem → effect::ShowItem, CreateView → effect::DeleteView and AddItem → effect::RemoveItem all left cargo test --workspace at 918 passing, 0 failing. This test fails on each.

+

The name-for-name rule is a design constraint, not a coincidence, and it is what makes the derivation checkable at all (doctrine(derive_the_enum_from_the_effects)): a variant whose effect must be named something else is a variant that stopped being derived, and it should fail here and be argued about rather than pass quietly.

+

#the_enumeration_names_each_variant_once

+

The every_model_command list covers the enum once. A duplicated entry would leave a variant untested while the count still looked right.

+

#Helper — every_model_command

+

Not a test — the enumeration that test, and the three walks above it, cover.

+

One value per ModelCommand variant. The enum is the derived set, so a test that walks "the set" has to walk all of it. variant_name is the completeness guard: it matches exhaustively, so a variant added to the protocol fails the build here until it is listed in this vector as well.

+

#a_model_command_lands_where_its_keypress_lands

+

The point of the whole ticket: a model Command applied authoritatively leaves the document exactly where the keypress bound to the same mutation leaves it. That equivalence is what lets the two entry points share one door.

+

The Dirty is compared too, because a host with connected clients projects off it: a command that produced the right document and the wrong Dirty would leave every other client stale.

+

Its pivoted() fixture has two categories on different axes, because the one-category fixture cannot tell a transpose from a cycle: with only Region on Row, swapping the axes and rotating Row→Column land on the same view, and an equivalence test that cannot distinguish the wrong effect from the right one proves nothing (doctrine(check_the_checker)).

+

#a_stale_wire_model_command_cannot_be_applied

+

A wire Command::Model cannot reach the document without its basis being checked (ADR 001, improvise-ondj).

+

Bug demonstrated 2026-09-19: reduce_full and apply_command — the functions this module's own doc points every future host at, the browser worker included — took a &Command and matched Command::Model { cmd, .. }, so a batch composed against a document that had since moved landed anyway. The .. discarded the basis with no compile error.

+

The regression guard is the signature, not this test. Both now take a Checked, whose only constructor runs stale_basis, so the version of this test that reproduced the bug — reduce_full(&mut app, &wire) with a stale wire — no longer compiles, and reduce_full itself no longer exists (improvise-s17o: having no caller was the condition that let the defect hide in the first place). What is left to assert is that the constructor refuses, and that refusing leaves the document alone: there is no half-applied batch, because the type is obtained before anything is applied.

+

#a_model_commands_dirty_comes_from_its_effects_kind

+

The Dirty a model command reports is the one its effect declares, not a second table kept alongside the enum (gotcha 35).

+

#a_wire_model_command_marks_the_document_dirty

+

improvise-gv9r. The bug: a client's edit left the authoritative document reporting clean, on every transport.

+

MarkDirty used to be a separate effect that the command layer composed alongside the mutation, and improvise-kmgs derived ModelCommand from the effects one at a time — so the wire got SetCell without its companion. A derivation correct per element and wrong for a set. "Save if dirty" was therefore a trigger that never fired, and a host built on it loses every client edit silently; the browser worker found that out the hard way and had to over-approximate off the command variant instead.

+

The fix is the unrepresentable one: dirtiness is raised by App::apply_effects off the same EffectKind the Dirty and the two basis tiers are read from (gotcha 40), so no path can compose a document change without it. There is nothing left to forget, and MarkDirty is gone rather than left as a second answer.

+

The three levels are asserted separately because they are three different effects: an effect that is only ever reached by one of them would otherwise pass on a sibling's evidence. The negative arm — that a ViewOnly effect leaves the document clean — is a_view_only_batch_leaves_the_document_clean in ui/app.rs, and it has to live there: a view-slice Command never reaches apply_effects at all (apply_command folds it straight into ViewState), so a negative written against this door would pass however unconditional the raise was.

+

#a_wire_view_switch_leaves_the_document_clean

+

The inversion of a_wire_view_switch_marks_the_document_dirty, which asserted the pre-improvise-jytm behaviour. A switch used to move Workbook::active_view, which persists as the Initial View line, so it really did put the document out of step with the file. It no longer moves any document state, so there is nothing to persist and the browser worker's persist trigger (improvise-gv9r) must not fire.

+

The direction matters: the worker asks the document's dirty flag rather than a table of variants, so a switch that dirtied spuriously would cost an OPFS rewrite of the whole document per keystroke (gotcha 41). This is the negative arm — the positive one, that a real model command does dirty, is the loop above it.

+

The consequence to keep in view is a product one, filed as improvise-azhb rather than decided here: a TUI user who switches view and saves now writes the Initial View the document was loaded with.

+
+

#ui/effect.rs

+

#the_document_surface_runs_on_a_non_filesystem_backend

+

improvise-fuj: every effect that touches a document goes through App::storage, so the whole save/load/import/export surface works on a backend that is not a filesystem. This is what a browser worker supplies (improvise-i34); here MemoryFs stands in for it, and the test would fail if any of these effects reached for std::fs.

+

#view_only_effects_leave_shared_state_alone

+

improvise-mae: a declared kind() is only worth having if it matches what the effect actually does. Rather than restate each declaration, this applies the effect and checks the shared state it claims not to touch is genuinely unchanged — a ViewOnly effect that quietly wrote a cell would never reach connected clients.

+

No exemptions: since improvise-8uz moved the cursor out of View, SetSelected/SetRowOffset/SetColOffset touch nothing shared, so every effect in the list must clear the same bar.

+

#structural_effects_declare_their_kind

+

The structural effects, each pinned to the kind it must declare.

+

The bug this guards: AddCategory and RemoveCategory declared CellData, but both go through Workbook, whose on_category_added / on_category_removed rewrite every view's axis map. That is a re-layout, so a client told only "cell data changed" would keep rendering the old grid shape. The two "actually change" tests beside it cannot see this — a category change moves model data and view config, so it passes either list. Only pinning the declaration does.

+

#add_category_past_the_limit_reports_the_failure

+

Bug (improvise-djn): AddCategory discarded the Result from Workbook::add_category, so pressing n in the category panel with the 12-regular-category limit already reached reported the optimistic Added category "X" its command had queued and added nothing. The effect runs last precisely so its failure can overwrite that message (improvise-6t3) — it just never wrote one.

+

#add_formula_adds_target_item_to_category

+

Regression: AddFormula must add the target item to the target category so the grid layout includes cells for it. Without this, a formula like Margin = Profit / Revenue would be registered but invisible in the grid.

+

#abort_chain_short_circuits_apply_effects

+

AbortChain must cause subsequent effects in the same apply_effects batch to be skipped, and the flag must reset so the next dispatch starts clean.

+

#save_without_file_path_aborts_the_chain

+

BUG (improvise-8tk): Save reported failure only through the status bar, so the Quit that wq/ZZ chains after it landed anyway and the whole session was discarded without the user seeing the error. A failed save must abort the rest of the batch.

+

#failed_save_as_aborts_the_chain

+

BUG (improvise-lae, found reviewing it): improvise-8tk fixed one arm of a rule with two. Save aborts the batch when the write fails; SaveAs reported the error to the status bar and let the batch continue, which is the same silent discard 8tk closed — the shape doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only) names, and the three tests above it are evidence about Save only.

+

It is reachable today: the : line composes with . , so :w /bad/path . q builds exactly this batch, and ExecuteCommand concatenates the two commands' effects into one. improvise-lae's save prompt also commits to SaveAs, so the arm was about to carry more traffic.

+

Note what the abort does and does not buy the prompt: the typed path survives for the user to correct, but the message does not reach the screen, because draw_bottom_bar draws the minibuffer instead of the status line while a prompt is open. That is pre-existing and reported separately; the data-loss half is what this fixes.

+

#successful_save_as_lets_the_chain_continue

+

The positive arm: a SaveAs that succeeds must not abort, or every :w <path> would swallow whatever followed it. Without this, "always abort" passes failed_save_as_aborts_the_chain.

+

#enter_edit_at_cursor_uses_target_mode_not_app_mode

+

EnterEditAtCursor must use its target_mode field, not whatever app.view_state.mode happens to be when applied. The previous implementation branched on app.view_state.mode.is_records(); the parameterized version trusts the caller (keymap or composing command).

+

#enter_edit_at_cursor_reads_post_batch_layout

+

improvise-ehz: the edit buffer must be loaded from layout state that already reflects earlier effects in the same batch. apply_effects only rebuilds after the batch, so a preceding RebuildLayout is what makes the cursor resolve to the post-mutation cell. Removing the row-0 item slides a different cell under the cursor; the buffer must show that cell's value, not the pre-batch one.

+

#set_buffer_does_not_special_case_the_search_name

+

SetBuffer no longer treats "search" as an alias for the search query (improvise-gdl): it is a plain buffer write whatever the name says, and the query is reached through SetSearchQuery. Commands pick between the two in cmd::core::write_buffer.

+

#apply_and_clear_drill_coord_rename_moves_every_measure_in_the_row

+

improvise-c5v: in wide format a records row groups one cell per measure, so renaming a coordinate has to move every one of them. The long-format code moved only the cell under the cursor, which would now split the row in half.

+

#import_headless_csv_short_rows_surface_warning_in_status

+

BUG (improvise-k8i): ImportJsonHeadless parsed CSVs with parse_csv, which sends short-row warnings to stderr — invisible inside the TUI. Short rows must surface in the status message, naming the affected column, never silently succeed.

+

#Note — the drill tests drive apply_effects, not apply

+

Several ApplyAndClearDrill tests call app.apply_effects(vec![...]) rather than effect.apply(&mut app). That is deliberate: the dirty flag is raised off the EffectKind now (improvise-gv9r, gotcha 45), so driving the effect directly would assert a flag nothing in production sets.

+

#every_file_writing_effect_aborts_the_batch_on_failure

+

improvise-z05r, the third instance of one rule: improvise-8tk fixed Save and not SaveAs; PR #27 found SaveAs; ExportCsv was the arm still missing, so a Binding::Sequence or a : line composing an export with anything after it continued past a failed write.

+

The fix is structural rather than a third patch: all three now report through one report_file_write(app, outcome), which writes the status message either way and sets abort_effects on Err. That is not sufficient by itself — a fourth writer could still be written without calling it — so the test names the three, and a fourth has to come here and be named. Naming is the mechanism; the shared function is what makes the naming cheap to honour.

+

Each case drives App::apply_effects with [writer, ChangeMode(Quit)] against a path under a directory that does not exist, and asserts both last_batch_aborted() and that the trailing mode change never landed.

+

Injected fault: drop app.abort_effects = true from report_file_write. Fails by name, along with the four older abort tests — which is the point: they and this one now share one mechanism.

+

Injected fault, narrower: route ExportCsv around the reporter and write its status directly. Only this test fails, naming ExportCsv, with 460 passing — the exact state the ticket describes.

+

#a_batch_that_writes_nothing_leaves_the_document_clean

+

improvise-748f. ViewNavigate { forward: false } emits ApplyAndClearDrill unconditionally, and that effect returns immediately when drill_state is None. It declares Model(CellData) regardless — Effect::kind takes no App, so the classification cannot be state-dependent — so apply_effects raised model_state.dirty on a document nothing had touched. Press view-back, press :q, and the quit is refused; autosave arms thirty seconds later.

+

The older test here, start_drill_and_apply_clear_drill_with_no_edits, pinned the wrong behaviour in an assertion message arguing it was the same over-approximation a failed LoadModel gets. It is not: a succeeding LoadModel leaves a document that really does differ from the file its path names, and that remains declared. This was a wrong answer to no divergence at all.

+

#a_cell_data_effect_no_raw_cell_records_still_dirties_the_document

+

The guard against the cure improvise-748f proposed, which is unsound. The ticket suggested raising dirty off bump_changed_records' per-record fingerprint of the raw cells, on the reading that CellData means cells. Five of the ten CellData effects change the document without writing a raw cell: AddItem, AddItemInGroup and SortData (which reorders — and .improv carries the order), and AddFormula and RemoveFormula, whose results live in Model::formula_cache and not in the store. Observing raw cells would have left every one of them reading clean, which is a lost edit — the opposite direction from the spurious prompt it was fixing.

+

So the observation is document_content — document_config plus the cells in iteration order and the formulas — and only the CellData arm reads it. The LayoutGeometry arm keeps the narrow document_config. A first version pointed both arms at the wide fingerprint, and review benchmarked what that costs at 18,000 cells in release over 200 batches after warm-up: one SetCell batch 8.07 ms → 10.21 ms (+27%, a constant factor on a path that was already O(cells) through record_fingerprints), but one SetNumberFormat batch 78 µs → 2.63 ms, 33× — an asymptotic change from O(config) to O(cells + config) on the held-key path that [/], axis cycle, hide/show, group toggle, transpose and prune toggle all take. ~380/sec where they had been ~12,800/sec. The split costs no correctness because all five blind effects are CellData; that every LayoutGeometry effect really is visible to the narrow fingerprint is asserted separately, and not left as a reading.

+

WholeWorkbook stays declared, and that is the remaining over-report: a LoadModel whose incoming content happens to be identical still marks the document dirty, and a failed one does too. Both cost one redundant write.

+

The SortData case needs two cells inserted out of sorted order, or it reorders nothing and its case asserts the opposite of what it names. The AddFormula case removes the fixture's formula first, or adding it is a no-op.

+

Injected fault: narrow document_content back to the configuration it hashed before. Fails naming SortData, with five other dirty-flag tests.

+

#every_layout_geometry_effect_is_visible_to_the_narrow_fingerprint

+

The guard that makes the two-fingerprint split safe rather than a trap. Splitting CellData (wide) from LayoutGeometry (narrow) is only sound while every effect wearing LayoutGeometry writes something document_config hashes; an effect it cannot see would leave the document reading clean forever, which is the same lost-edit failure the wide fingerprint exists to prevent, arriving by the other door.

+

The thirteen are enumerated by brace-matching the impl Effect blocks, not by grepping — the naive grep reports fourteen, picking up HelpPageSet, which is ViewOnly and is simply the last pub struct before #[cfg(test)] (gotcha 38 names this exact artefact). Each case asserts its effect really declares Model(LayoutGeometry) before asserting anything else, so a reclassified effect fails here rather than silently leaving the list unrepresentative.

+

Four cases carry a prepare step run directly against the workbook and not through an effect, because the interesting state is the precondition. What makes such a case honest is the no-op-fails property: without its precondition the effect writes nothing, an unwritten document leaves the fingerprint unmoved, and the case therefore fails rather than passing vacuously.

+

Measured, by deleting each prepare in turn, two of the four are load-bearing in exactly that way and two are not:

+ +

dirty is cleared after prepare and before the effect. That clear is defensive hygiene for a future prepare that goes through apply_effects, and it is not what makes any case honest today: dirty is raised only inside App::apply_effects, and no prepare in this test runs an effect, so no preparation can set it. Demonstrated by removing the clear and blinding document_config to 0 together: all thirteen cases still read dirty == false. An earlier version of this entry claimed the opposite — that without the clear "all four cases pass against any fingerprint" — and measurement contradicts it.

+

Injected fault: document_config returns 0. Fails, with five other dirty-flag tests across three files. Injected fault, narrower and the one that proves the test is pointed at the right thing: drop view.number_format from document_config. Fails naming SetNumberFormat, with only one other test failing anywhere.

+
+

#ui/app.rs

+

#model_state_and_view_state_are_constructible

+

improvise-3vr: ModelState and ViewState are the named slices of App state introduced by the vb4 refactor. Step 1 only requires that the types exist and are constructible; subsequent steps move fields in.

+

#Note — the ViewState: Send static assertion

+

improvise-h0b: ViewState is the portable slice — the one thing serialized into the Snapshot a newly-connected client receives (improvise-q08). Send is the cheapest compile-time proxy for "portable": a field that cannot cross a thread boundary certainly cannot cross a serialization one. Before h0b this failed to compile, because drill_state: Option<DrillState> holds an Rc<Vec<RecordRow>>. Session-local state lives on App instead. It is a const _: fn() in the test module, not a named test, so it has no heading of its own.

+

#app_view_state_owns_ui_session_fields

+

improvise-ew0: ViewState owns the UI session slice — mode, status, search, panels, navigation, drill, yanked, buffers. App accesses them through view_state.

+

#view_state_survives_a_json_round_trip

+

improvise-a2x: ViewState is what a newly-connected client receives as a Snapshot (improvise-q08), so it has to survive a round trip through the wire format. Populated with non-default values in every field family — a mode carrying a MinibufferConfig, a navigation frame, a yanked cell, a keyed buffer, an armed prefix — because a roundtrip of ViewState::default() would pass while proving nothing.

+

#the_documents_default_view_outlives_this_clients_switches

+

The replacement for the_two_active_view_names_never_diverge, and it is a replacement rather than a deletion on purpose: improvise-jytm makes the two copies diverge deliberately, and a deliberate split with no guard is how one silently regresses into the other.

+

What it pins, all four of which have a silent failure mode. A switch moves the client's copy and not the document's default. App::view() — what rebuild_layout and cmd_context read — follows the client, so the split is observable in the layout and not merely in a string. A switch to a view the document does not have moves nothing, because the slice keys ViewState::cursors and grid_in indexes the workbook by it. And install_workbook still seats this client on the incoming document's default and clears the cursor map, since the outgoing document's view names mean nothing in the new one (gotcha 24).

+

Fault injected 2026-09-20: App::switch_to_view calling workbook.switch_view(name) again fails the second assertion.

+

#moving_the_cursor_does_not_touch_the_shared_workbook

+

improvise-8uz: moving the cursor must leave the shared document untouched, so that one client scrolling cannot move another's viewport. Before the move these three effects wrote into model_state.workbook, and this assertion would have failed.

+

#app_owns_the_session_local_fields_not_view_state

+

improvise-h0b: the two session-local fields moved off ViewState and onto App. Each fails both membership tests: ImportWizard is an improvise-io type (not portable) and a half-open dialog (not snapshot-safe); DrillState holds an Rc (not portable — see the Send assertion) and stages uncommitted edits (not snapshot-safe).

+

#staged_drill_edits_are_unreachable_from_view_state

+

improvise-h0b: a fresh client must not inherit another session's staged, uncommitted drill edits. Those edits are unreachable from ViewState, so a Snapshot built from it cannot carry them.

+

#switching_views_restores_each_views_own_cursor

+

improvise-8uz: the cursor is per view. Written as a characterization test against the old View-owned fields and kept verbatim across the move to ViewState::cursors, because a refactor about where state lives must not change what the app does.

+

#the_improv_format_does_not_persist_the_cursor

+

improvise-8uz: the ticket asserted these fields were "PERSISTED in .improv". They were not — the grammar has no cursor production and the writer emitted none. That is what made the move free: there was no restore-where-you-left-off behaviour to lose, and no format change to make. The assertion still guards the format going forward.

+

#entering_records_mode_sorts_existing_data

+

Pressing R to enter records mode sorts existing data by CellKey, so display order is deterministic regardless of insertion order.

+

Note: this test's doc comment carried a stray first line from a neighbouring test ("Regression: pressing o in an empty records view should create the") that was never finished. It was dropped in the comment-cap wave; the sentence above is what the test actually asserts.

+

#the_new_category_column_creates_a_category_and_its_column

+

improvise-2zq step 2: the +cat column creates a category and its column, without leaving the grid.

+

Driven through real keys. The assertion that earns its place is the last one: the new column is usable immediately, which is the whole point — records_columns filters on kind.is_regular() and never requires items, so a category with no items is a column the moment it exists.

+

#enter_on_plus_cat_in_the_last_row_does_not_reopen_category_creation

+

improvise-2zq, found by Copilot on PR #27: creating a category from +cat shifted the grid under the advance that followed it.

+

add_category inserts the new coordinate column immediately before +cat, so every column index from +cat rightwards moves by one. CommitAndAdvance then computed its advance from ctx, whose layout is the pre-commit one (gotcha 23: apply_effects rebuilds only after the whole batch). On the last row Enter wraps to the next column — and the index it wrapped to was +cat again after the rebuild.

+

So the failure is not "lands one column off". It reopens category creation: a user naming two categories in a row never reaches the column they just made, and gets an unbounded +cat loop instead. The last row is the arm that wraps, so it is the arm tested.

+

#tab_on_plus_cat_does_not_reopen_category_creation

+

The same stale index reached Tab, which advances right rather than wrapping. Both directions read the pre-commit layout, so both are covered — one arm of a two-arm rule is the shape doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only) names.

+

#the_new_category_column_creates_the_column_but_writes_no_coordinate

+

Committing the +cat name creates the column and does not write the coordinate on the row the cursor was on.

+

The deliberate answer to the ticket's third question, and the reason is gotcha 1: setting a coordinate moves every measure cell in the row. That is a data migration across N cells, and attaching it to the same Enter that made a schema change — in an app with no undo — is a bigger single keystroke than the measure + asks for. The follow-up edit is one column away.

+

#the_new_category_column_refuses_past_the_limit_where_the_user_can_see_it

+

MAX_CATEGORIES is a reachable refusal on this path, and the refusal has to be visible.

+

draw_bottom_bar renders the minibuffer instead of the status line, so a message written while an editor is open is never drawn. The commit path therefore drops out of editing before running the add, and AddCategory aborts the batch on refusal so the trailing EnterEditAtCursor cannot re-open the editor over the message.

+

The limit itself is checked in exactly one place — Model::add_category — and nothing here restates it; the test fills the document to the limit by asking for categories until it refuses.

+

#naming_a_category_in_records_mode_does_not_eject_the_user

+

improvise-2zq step 1: naming a category in records mode used to eject the user from the grid they were standing in.

+

Records mode is _Index on Row and _Dim on Column, with every regular category on Axis::None. View::on_category_added seated a new regular category on the first axis no regular category held — Row — and bumped the virtual out of the way to get there. So N, a name, Enter moved _Index off Row, records detection stopped matching, and the grid silently became a pivot.

+

Driven through the real keys rather than through on_category_added, because that is how the bug was found and it is what makes this evidence rather than a restatement of the fix.

+

#i_and_o_diverge_on_an_empty_records_grid

+

improvise-6sq: i and o want opposite behaviour on an empty records grid, which is why edit-or-drill and enter-edit-at-cursor cannot be merged into one command. Entering records mode puts every regular category on Axis::None, so EditOrDrill's aggregation check is true unless the cursor is on a synthetic record — and on an empty grid there is none, so i reports "cannot drill". o must still open an edit there (see add_record_row_in_empty_records_view_creates_first_row), so routing it through the same drill check would break it.

+

improvise-ct5 changed the message and deliberately not the divergence. The outcome — i opens no editor where there is nothing to edit — is right, and o one key away is the affordance for adding a row. What was wrong is that the message said "cannot drill", a pivot-mode concept in a mode whose model is "a table I type into". Making i behave like o here (the ticket's option (b)) would collapse exactly the divergence improvise-6sq exists to record, so the assertion still pins that i does not enter editing; only the wording assertion moved, and it now also names what must not appear.

+

#the_empty_grid_message_is_not_the_records_one_in_pivot_mode

+

The negative arm of improvise-ct5, and the reason the wording is keyed off the mode the keymap supplies rather than reworded once.

+

Without this, "say the records sentence everywhere" passes i_and_o_diverge_on_an_empty_records_grid — and it would promise o, which adds no row in pivot mode, trading one misleading message for another.

+

The fixture is an empty pivot grid that still counts as aggregated: Region on Row with no items makes cell_key() None, and Extra cycled to Axis::None is the regular category that makes EditOrDrill take the drill branch.

+

#blank_model_bootstraps_its_first_measure_from_the_new_measure_column

+

A blank model has no categories and no measures, so wide format shows exactly the two creation columns. Typing a name into the new-measure one creates the measure, which gives it a real column to type values into — records mode stays self-sufficient for onboarding (improvise-c5v).

+

improvise-2zq widened that: a blank grid now offers +cat beside +, so both halves of a schema can be bootstrapped without leaving the grid. This test still pins the measure half; the category half is the_new_category_column_creates_a_category_and_its_column.

+

Inside it, the cursor lands at column 0, which in a blank grid is now +cat rather than + (improvise-2zq) — a record needs coordinates before it needs measures, so that order is deliberate. o leaves the test editing column 0, so the Esc before moving to the new-measure column is load-bearing.

+

#ctrl_w_prompts_for_a_path_and_a_failed_write_reports_it_where_it_shows

+

improvise-lae end to end: Ctrl-W, a path, Enter.

+

The failing case is the interesting one, and it pins an ordering (found by Copilot on PR #27). SaveAs aborts the batch on a failed write, so whatever CommitSave emits after it is skipped. A first version emitted [SaveAs, ChangeMode(Normal)], which left the app in SavePrompt — and draw_bottom_bar renders the minibuffer instead of the status line, so the "Save error:" the effect had just set was never drawn. The save appeared to do nothing whatsoever.

+

Leading with the mode change puts the error in Normal, where the bottom row is the status bar. This test therefore asserts the mode left the prompt, which is the opposite of what it asserted before — deliberately, and it is the only thing keeping the two effects in that order.

+

#every_minibuffer_mode_entered_from_records_returns_to_records_normal

+

Copilot round 2 reported that Ctrl-W from a records grid returns to Normal, losing the records keymap while the layout is still a records layout. It is right, and it is a class: every minibuffer mode reachable from RecordsNormal exited to Normal, because the exit was a constant baked into TextEntry::new(..., "normal") and into each commit command.

+

The class predates SavePrompt: : and Ctrl-E do exactly the same thing and always have. That is why the fix was not "make SavePrompt remember its origin" — that would leave two of the three arms broken, which is the shape doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only) names.

+

The binding table is not involved and was checked rather than reasoned about: an IMPROVISE_BINDING_DUMP diff shows RecordsNormal gains Ctrl-W and loses nothing, so with_parent inheritance (gotcha 13) is intact. What was lost is the runtime mode.

+

improvise-sihr fixes it by observation rather than memory, which is what took the exit out of the keymap entirely: App::apply_effects calls settle_normal_mode after rebuild_layout, so the mode a batch ends in matches the grid it ends on and nothing carries an entry mode. The ticket proposed the opposite — carrying the entry mode on MinibufferConfig — and that was declined for a concrete reason beyond doctrine: carried generically it would also return CategoryAdd to CategoryPanel, a behaviour change nobody asked for, while carried for only three modes it is the one-arm shape again.

+

This test still drives both exits — Esc and Enter — where the original drove only Esc, because they reach the mode through different effects (ChangeMode and ReturnToNormalUnlessMoved) even though one settle now covers both.

+

The first version of the fix resolved per-effect and was wrong in a new direction, which the adversarial review of PR #91 caught. Set at the moment the mode is written, the resolution reads the batch's opening layout (gotcha 23) — so a batch that crosses settled against the grid it started on. See a_batch_that_leaves_records_and_then_sets_normal_lands_in_normal.

+

Injected fault: settle_normal_mode()'s call site removed. Exactly three tests fail across the whole workspace — this one, the :-line sibling and the crossing test — which is the per-edit coverage question answered by there being one edit.

+

#a_colon_line_run_from_records_settles_back_into_records_normal

+

The sibling above cannot see ReturnToNormalUnlessMoved, and that was found by injection rather than by reading: forcing only that effect back to AppMode::Normal left the sibling green. An empty : line — which is what pressing Enter straight after : gives — takes ExecuteCommand's raw.is_empty() branch and settles through ChangeMode, so the effect that settles every line that actually runs was untested.

+

So this test types a line. page-next rather than yank, because the script parser requires coordinates for yank and an unparseable line takes the error branch — which settles through ChangeMode too, and would have made this test a second copy of its sibling while looking like a new case. The setup assertion that parse_line_with accepts the line is there for exactly that: a future rename of the command would otherwise turn this test vacuous silently.

+

Injected fault (against the per-effect version of the fix): ReturnToNormalUnlessMoved forced to AppMode::Normal. This test failed, left: Normal right: RecordsNormal, while the sibling stayed green — which is the point, and is how this test came to exist. Against the shipped end-of-batch settle the discriminating injection is removing settle_normal_mode()'s call site, which fails this test with the same values.

+

#a_batch_that_leaves_records_and_then_sets_normal_lands_in_normal

+

New wrong state introduced by PR #91's first attempt, found by the adversarial review and verified by reverting only ui/app.rs and ui/effect.rs to base and re-running the identical probe:

+
:toggle-records-mode . enter-mode normalbase#91 round 1
from a pivot gridNormal (the sihr bug)Normal still
from a records gridNormal (correct)RecordsNormal
+

The cause is gotcha 23: resolving at the moment the mode is written reads the batch's opening layout, so a batch that crosses settles against the grid it started on. :toggle-records-mode is the case the round-1 prose offered as proof that the staleness was a property — and it is not, because ToggleRecordsMode emits change_mode(RecordsNormal) explicitly on entry and ViewBack restores the pushed mode on exit, so the resolution's answer is overwritten in both directions and the case exercises nothing. That is a careful comment standing in for a test, in the PR that recorded doctrine(an_injection_that_does_not_fail_names_an_untested_path).

+

The fix moves the resolution to one place, settle_normal_mode after rebuild_layout, where there is no crossing case left to get wrong.

+

This test drives both directions deliberately. One direction alone is satisfied by a settle that only ever answers one way, and the pivot→records direction was also wrong in round 1 for the mirror-image reason.

+

#a_wizard_abandoned_over_a_records_grid_returns_to_records_normal

+

CloseWizard writes view_state.mode directly rather than through an effect that could resolve it, so under round 1's per-effect design it was a third writer needing its own edit — and reverting that edit alone left the whole workspace green, which the review filed. The end-of-batch settle removes the per-edit question entirely (there is one edit), and this test covers the path the review named: the wizard abandoned over a records grid.

+

It drives CloseWizard as an effect rather than typing :import, because the wizard needs a real CSV and the contract under test is one batch's settling, not the import pipeline.

+

#a_failed_colon_write_lands_in_normal_where_its_error_is_drawn

+

A failing : command must not strand the user in CommandMode.

+

Found by Copilot on PR #27, and it is the sharper form of gotcha 21. The abort is supposed to stop the chain — :w /bad . q must not quit. What it must not also stop is the bookkeeping: the mode restoration that ExecuteCommand appends. Aborting before that ran left the app in CommandMode with the stale command line still drawn, and draw_bottom_bar renders the minibuffer instead of the status line — so the "Save error:" set one effect earlier was invisible. A failed save looked exactly like a key that did nothing.

+

The answer is ordering, not classification: a batch puts its tidy-up where an abort cannot reach it, which is in front. No effect needs a new "runs anyway" flag, and the rule generalises to any command composing bookkeeping with something that can fail.

+

Both arms are here because they fail in different places: no path at all is Save's guard (aborting since improvise-8tk, so this arm was broken before that branch), and a bad path is SaveAs's write (aborting since f0f400d).

+

#a_failed_colon_write_still_refuses_to_quit_after_it

+

The other half of the same rule: the abort must still stop the chain. :w <bad> . q is improvise-8tk's data loss spelled on one : line, and moving the mode restoration to the front must not have bought the fix above at the price of that one.

+

#a_sequence_whose_first_step_aborts_does_not_run_the_second

+

Gotcha 21 claims a failed Save aborts across Binding::Sequence steps as well as within one command's effect list. Nothing tested it: no shipped Sequence has a first step that can abort, and ZZ is a prefix resolving to the single wq command, so prefix_z_then_z_reaches_save_and_quit exercises the within-a-command case only. The claim was structurally true and unguarded.

+

It is true because Keymap::dispatch flattens a Sequence into ONE effect vec and App::apply_effects runs that as one batch, breaking on abort_effects. It is load-bearing, because it is the reason a [w, force-quit] Sequence would be safe — the argument the grooming note unblocked on.

+

And it is exactly what improvise-0hb would remove. Command-level transactionality means applying effects between steps, and apply_effects resets abort_effects at entry and again on abort, so step 2 would be dispatched into a fresh batch with the flag cleared — the Quit after a failed save would land, silently reopening the improvise-8tk P1 data loss. This is the guard that would catch it.

+

#unmatched_key_after_prefix_is_swallowed

+

A prefix is consumed whether or not the next key matches, so a stray key after g does nothing rather than being treated as a fresh normal-mode binding.

+

#renaming_a_coordinate_moves_every_measure_in_the_row

+

improvise-c5v: a records row groups one cell per measure, so editing a category column has to move every one of them. The long-format code moved only the cell under the cursor, which in wide format would tear the row apart — half its measures under the old coordinate, half under the new one.

+

#leaving_records_mode_cleans_empty_key_cells

+

improvise-3zq (bug #2): AddRecordRow creates a cell with an empty CellKey when no Page-axis categories supply coords — that cell serialises as = 0 in .improv and re-appears on every records toggle. Leaving records mode must clean up any such meaningless records (the inverse of the SortData that runs on entry).

+

#enter_at_bottom_right_of_records_view_exits_editing

+

improvise-3zq (bug #1): Enter on the bottom-right cell of records view should commit and leave edit mode. Previously CommitAndAdvance pushed an EnterEditAtCursor effect unconditionally, so the cursor stayed put and editing re-opened on the same cell.

+

#Helper — records_model_with_two_rows

+

Not a test — the fixture for every Tab/Enter boundary test in this file.

+

A records-mode app with two data rows, for testing Tab/Enter behaviour at boundaries. In wide format a row is one coordinate tuple, so the two rows are two Region items, each carrying both measures (improvise-c5v).

+

#tab_on_bottom_right_of_records_inserts_below

+

improvise-hmu: Tab on the bottom-right cell of records view inserts a new record below and moves to the first cell of the new row in edit mode.

+

#Note — the wizard_key_tests module

+

Behavioural coverage for the import wizard's key handling (improvise-2wm). The wizard is driven entirely through handle_key, so these tests pin the user-visible contract independently of whether the step/key decisions live in an effect or in a command.

+

#name_model_typing_edits_the_name_and_enter_confirms_the_import

+

Enter at NameModel builds and parks on the Done confirmation; a second Enter there installs the workbook and closes the wizard (improvise-907, from improvise-c6j).

+

#review_proposals_blocks_when_no_category_is_accepted

+

BUG (improvise-907, from improvise-gfq): an import whose fields all classify as Measure used to walk the user through DefineFormulas and NameModel before build_model refused. The refusal now happens at ReviewProposals, where c can still change a field's kind.

+

#left_walks_back_through_the_steps_taken

+

BUG (improvise-907, from improvise-o1c): reaching a later step and wanting to change a field kind meant Esc, which discarded every decision. Left walks back through the path actually taken (gotcha 22).

+

#done_esc_returns_to_name_model_and_drops_the_built_model

+

Esc at Done goes back to NameModel rather than discarding the import, and the workbook built for confirmation is dropped so a later accept cannot install stale field decisions.

+

#Note — the basis_tests module fixture

+

The two lower tiers of a batch's basis (ADR 001, improvise-xwi6). Fixture note: two regular categories, two measures and two records, deliberately. A one-category fixture cannot tell TransposeAxes from CycleAxis, a one-record fixture cannot tell "the record it touched" from "every record", and a one-measure fixture cannot tell a record from a cell — the three distinctions these tests exist to draw.

+

#a_records_two_measures_share_one_record_key

+

The record is the coordinate tuple with _Measure gone, and both measures of one coordinate name the same record, which is the whole point of the unit.

+

#a_layout_change_moves_the_layout_version_and_no_record

+

Tier 2 moves, tier 3 does not. A shape change invalidates every in-flight CellData batch through layout_version alone; it must not also masquerade as a write to any record.

+

Fixture repaired under improvise-75ld: it asked for SetAxis { category: "Month", axis: Axis::Column } against a fixture where on_category_added had already seated Month on Column — so the "layout change" was a no-op, and the test passed only because the tier was bumped off the declaration. It now asserts the axis before and after, and asks for Axis::Row. This is the failure that turned "reclassify three effects" into "stop reading the tier off the classification". Injected fault: the same as both_shape_counters_move_for_every_reshaping_command — fails naming SetAxis.

+

#a_loaded_document_is_dirty_because_its_path_still_names_the_old_one

+

A loaded document is dirty, and that is a behaviour change with a reason (improvise-gv9r, raised by the independent review of PR #26).

+

LoadModel is Model(WholeWorkbook), so apply_effects raises the flag — where before that branch :e other.improv left the document reporting clean. Clean was the wrong answer: LoadModel does not move model_state.file_path, so after it the in-memory document is a different one from the file the path names, and the bytes on disk and the bytes in memory really have parted company. That is all dirty claims.

+

It is worth a test rather than a comment because of what it gates: autosave_if_needed and the :q refusal both read this flag, so a user who loads and quits is now asked rather than silently dropping a document whose path is stale. :q! still discards. The path-staleness itself is pre-existing and is reported separately; this pins only that the flag follows from it.

+

#a_view_only_batch_leaves_the_document_clean

+

The negative arm of improvise-gv9r's raise, and it must be aimed at apply_effects rather than at a Command: a view-slice command is folded straight into ViewState and never reaches this loop, so a negative written against the session door would pass however unconditional the raise was. It is a ViewOnly effect run through the loop that discriminates.

+

What it pins: a client's cursor, status line or panel does not put the document out of step with the bytes on disk, so it must not arm an unsaved-changes prompt or an autosave.

+

#a_cell_edit_moves_only_the_record_it_touched

+

Tier 3 moves for the record written and for no other, and tier 2 stays put: a cell edit does not invalidate batches that merely share the document.

+

#a_coordinate_rename_is_one_write_to_each_record_not_one_per_measure

+

The unit is the record, not the cell. Renaming a coordinate in wide records mode moves every measure cell in the row (gotcha 1), and that is ONE write to the record it left and ONE to the record it arrived at — not one per measure.

+

#an_aborted_batch_bumps_nothing_it_did_not_run

+

Gotcha 21: a batch that aborts halfway must bump nothing after the abort — in either tier. apply_effects accumulates the Dirty inside the loop for exactly this reason, and the versions are read off the same place.

+

#a_write_that_changes_nothing_moves_no_version

+

A write that lands the value already there invalidates nobody's basis, so it moves no version. This is a deliberate choice of semantics — observed change, not "a CellData effect ran" — and the safe direction, because the alternative rejects batches for nothing.

+

#the_version_tiers_are_not_in_the_view_state

+

The counters are shared document state on App, never on the portable session slice (gotcha 30): a client inheriting another's idea of how far the document has moved is exactly the stale basis this machinery exists to catch.

+

#a_page_flip_under_prune_empty_invalidates_the_cursor

+

improvise-7kvw, the case improvise-9mo9 decided the other nine and deliberately left open. SetPageSelection changes which records are visible and not which categories frame them, so a grid position normally keeps naming the same cell. Under View::prune_empty that is false: which rows survive pruning is a function of the page selection, so a page flip renumbers the grid while neither reset_cursor nor cursor_epoch moved — gotcha 24's failure in both directions at once, the host's own cursor stale and no client told.

+

9mo9 did not patch it because [/] paging is constant navigation and sending the cursor home on every flip is worse than the hole when prune_empty is off, which is the default. So the reset is conditional on prune_empty, and on the selection actually moving.

+

The fixture has three regions and two quarters whose non-empty rows differ, so position 0 names North on Q1 and South on Q2. The test asserts the row labels really differ before asserting the epoch moved; the negative arm asserts they really do not differ with pruning off, so neither case can pass vacuously.

+

Injected fault: drop the reset. The prune arm fails. Injected fault: make the reset unconditional. The no-prune arm fails.

+

#a_page_flip_without_prune_empty_leaves_the_cursor_alone

+

The negative arm of the above, and it is load-bearing for the same reason gotcha 24 gives for SetNumberFormat: without it, "reset on every page flip" passes the positive test and costs the user their position on every [/].

+

#selecting_the_page_already_selected_invalidates_no_cursor

+

The third arm, and the one the user actually hits. SetPageSelection resets the cursor only when the selection moved, and that guard shipped untested: deleting moved && left the whole workspace green at 1416, which review caught.

+

It is not an edge case. PageNext/PagePrev are odometer-style and emit a SetPageSelection for every page category on every keypress, so with more than one page category the non-carrying categories are always no-ops, and a single one-item category re-selects the same item every time. Unguarded, each such keystroke bumps cursor_epoch, sends the host's cursor to the origin and broadcasts Command::ResetCursor to every client on that view — improvise-75ld's "a no-op that refused everybody else", one tier down and aimed at the cursor instead of the basis.

+

The test asserts both preconditions before applying the effect — that the fixture really is on Q1, and that the grid really does prune — and that the row labels did not move before asserting the epoch stood still. So it cannot pass by re-selecting a page that was never selected, or by testing a grid that does not prune.

+

Both before placements are load-bearing, and both were wrong once. The page check used to run after the effect, which sets the page to Q1, so it could not see a fixture that started elsewhere: with the fixture moved to Q2 the test did fail, but at the row-labels assertion, under a message about renumbering. The sentence naming the property was not the assertion guarding it. The prune_empty check did not exist at all, and with pruning off the reset never fires whatever moved says — so paged_model(false) plus the deleted guard left the test green, proving nothing, with nothing saying so.

+

Both preconditions read App::view(), the view the effect writes through app.view_mut(), and not Workbook::active_view(), which is the document's default and is deliberately allowed to differ from it (gotcha 29). They agree in this single-client fixture, so the distinction is latent rather than a live bug — but an assertion about what an effect did must read the view that effect wrote.

+

Injected fault: delete moved &&. Fails by name, alone. Injected fault: paged_model(false) and delete moved &&. Fails on the prune_empty assertion; before that assertion existed, this combination passed. Injected fault: paged_model's fixture starts on Q2. Fails on the page-selection assertion, which now names it; before the move it failed on the row labels instead.

+

#a_foreign_layout_is_not_shown_the_hosts_staged_drill_text

+

improvise-pubt, filed with an argument for unreachability that the test disproves. project::cell_display fills edit_text from App::display_value_in(layout, key), and since improvise-jytm that layout may be another client's. drill_state.pending_edits are keyed by (row index, RecordCol) against the host's drill layout (gotcha 1), and a row index in one layout names nothing in another.

+

The ticket read the arm as unreachable because "a foreign layout built by GridLayout::new carries no frozen records, so it should not produce a synthetic key". That is wrong: records mode auto-detects from the axes (gotcha 1), so an ordinary saved view with _Index on Row and _Dim on Column gives GridLayout::new a records layout whose cell_key returns exactly the synthetic (_Index, _Dim) key the drill arm matches on. The test fails on the original code, naming the host's staged text read out of a layout it was never staged against.

+

The fix is App::staged_drill_edit, which consults drill_state only when std::ptr::eq(layout, &self.layout). A pointer comparison rather than a name comparison because grid_in reuses &app.layout by reference where the view names agree and derives a fresh one otherwise, so identity is the question being asked. Whether the host's pending drill text should reach any other client at all is improvise-tom1 and is untouched here.

+

The positive arm — the host's own layout still shows STAGED — is in the same test, or "never consult the drill" passes, which would blank the TUI's own drill editor. The drill fixture needs a real RecordRow, not an empty Vec: the host's layout iterates the frozen snapshot, so an empty one gives a zero-row grid and the positive arm cannot be built.

+

#a_default_that_names_nothing_says_which_invariant_broke

+

improvise-h074. App::resolve_view falls back to workbook.active_view for a name the document has not got and then indexes workbook.views[default]. Several of these call sites previously went through Workbook::active_view(), which carries .expect("active_view always names an existing view"); IndexMap's Index impl panics with no message at all, so a workbook whose default names nothing — reachable only by constructing the struct directly, since create_view/delete_view maintain it — failed with a bare index panic where it used to name the broken invariant.

+

Injected fault: restore &workbook.views[default]. The test fails on the expected substring.

+
+

#ui/project.rs

+

#a_window_reaching_the_last_row_carries_the_trailing_group_header

+

A collapsed group at the bottom of the grid has no items to hang off, and is still the only row that says those rows exist. A window that reaches the last row therefore carries it.

+

It must be seen; it is not yet actionable in a browser. web/app.js registers no listener on a group <th>, and toggle-group-under-cursor is still in the_command_names_that_still_need_the_host_are_these, so z resolves Upstream into Host::handle's empty vec — a black hole, not a slow path (doctrine(an_upstream_key_is_a_black_hole)). The TUI can expand it today; the browser can only show it.

+

#Helper — app_with_a_trailing_group

+

Not a test — the fixture for that test and for its stops-short arm.

+

Region on Row with East in a group that comes last, so the group header is the final entry on the axis and nothing follows it. That is the shape axis_frame's trailing rule is about.

+

#a_window_that_stops_short_does_not_carry_what_is_below_it

+

The other arm, and the one that would make a browser scrolled to the middle of a document draw the bottom of it: a window that stops short of the last row carries nothing below its last item.

+

#a_window_opening_inside_a_group_names_only_the_nearest

+

A window opening inside a group still names it — otherwise a browser could not say which group a column or row belongs to — while body_from declines to paint a header that sits above the window.

+

And only the nearest one comes. The fixture puts two headers above the window (Hidden is collapsed, so it emits a header and no items, and Late's follows it immediately) because that is the only way to tell "the nearest header comes" from "every header above comes". With one group above, deleting the compaction leaves this green — it is a cost property, O(the groups a deep scroll crossed), and correctness alone cannot see it: group_of still finds the nearest and body_from still starts at the first item either way.

+

#Helper — app_with_consecutive_headers

+

Not a test — the fixture for that test and for consecutive_headers_between_two_rows_all_come.

+

Two consecutive group headers between two rows: Hidden is collapsed, so it emits a header and no items, and Late's header follows it immediately.

+

#consecutive_headers_between_two_rows_all_come

+

Headers between two in-window rows all come, not just the nearest.

+

Only the nearest wins group_of, so a compaction that kept one would still answer every question about membership correctly — and would silently lose the Hidden header, which is the only thing on screen that says those rows exist at all, and the only thing that will be clickable once the browser can toggle a group. It cannot today: z is Upstream and answered with nothing.

+

Above the window the rule is the opposite, and that arm is a_window_opening_inside_a_group_names_it_without_painting_it.

+

#folding_what_the_server_emits_reproduces_what_the_server_has

+

The contract of the whole module: a client that folds what the server sends ends up holding what the server has. Not the fold axiom — the yardstick (server_truth) is built independently of the projections.

+

#Helper — server_truth

+

Not a test — the yardstick that test and its siblings measure a fold against.

+

What the server believes the client should be holding, built straight from the authoritative state — the yardstick every fold is measured against, and built independently of the projections so the fold tests are not vacuous (doctrine(fold_tests_must_test_order_not_the_axiom)).

+

#removing_a_category_empties_the_clients_formula_panel

+

Removing a category removes every formula that targets it — Model::remove_category ends with formulas.retain(|f| f.target_category != name). RemoveCategory is LayoutGeometry, and Dirty::of(LayoutGeometry) reported formulas: false, so the formula panel was never told: a client's FormulaCache kept showing a formula the document no longer has, indefinitely, with nothing to bring it back short of a reconnect. (Gotcha 35.)

+

#a_late_client_converges_with_one_that_followed_from_the_start

+

Bootstrap is a compaction, not a replay: a client that connects late and folds only the seed lands exactly where one that followed the whole stream is.

+

#the_projected_extent_is_the_whole_grid_not_the_window

+

The extent is the whole grid, not the window (improvise-bpq2).

+

A client resolving its own cursor keys clamps against this, so a producer that read it off the projected ranges would stop every browser cursor at the bottom of the screen — and would look right in the TUI, whose cache covers the whole grid at offsets zero (gotcha 44). The fixture is deliberately not square (3 rows, 2 columns) and the window is smaller in both directions, so it separates "the whole grid" from "the window" and from "rows and cols swapped".

+

#a_reshaped_grid_folds_to_the_desired_state

+

Reshaping is the case the ordering exists for. Growing is the half that bites: cells at positions the client does not have yet are dropped, so the labels that create those positions must go first. Shrinking is here for the other direction — cells that no longer fit must not be sent into a grid that has already been cut down.

+

#a_renamed_column_whose_total_did_not_change_still_gets_its_total_back

+

improvise-hkt6, found by Copilot on PR 20. The diff must carry the mirror to desired, and it stopped doing so for one shape: a column renamed while its total stayed the same number.

+

apply blanks the total under a relabelled column (gotcha 33(b)), so the client's total went blank; but the diff compared mirror.totals against desired.totals, found them equal, and sent no Totals. ClientSession::project_diff then assigns self.grid = desired, so the server believed the number was still on screen — a cache wrong forever, with nothing in the stream to correct it, which is precisely the direction gotcha 35 keeps Dirty::of sloppy to avoid.

+

The assertion is the one a_reshaped_grid_folds_to_the_desired_state makes — fold the updates onto the mirror and require the result to be desired — because "a Totals update was emitted" would pass for a producer that emitted the wrong totals.

+

The fixture renames the middle column and keeps every total equal, so it separates "re-sends what the relabel blanked" from "re-sends whenever the labels move at all".

+

#renaming_an_axis_category_resends_every_cell

+

The producer half of the category-rename rule, and the reason the apply half is not cosmetic: the two compose into "wrong forever" without it.

+

grid_updates diffs the cells against the framed mirror — the mirror with the frame updates already folded in. So an apply that kept the cells through a cats change made the producer agree they matched and send nothing, ClientSession::project_diff then assigned self.grid = desired, and the client held cells answering the old coordinates with no update in the stream that would ever correct them.

+

An axis swap is the case that bites: two categories can legally share an item name, so Region = [North] becoming Department = [North] leaves the labels identical and the CellIds valid and pointing at the other record.

+

#a_session_that_switches_views_is_projected_the_view_it_switched_to

+

The original bug: ClientSession::apply folded straight into the session's mirror, so a Command::SwitchView moved the mirrored active_view while the workbook's stayed put (gotcha 29). viewport() then read the cursor under the new view's key and the next project() built the grid from the other view.

+

improvise-jytm answers it the other way round — the mirror is now authoritative and grid_in is built from it — so the assertions moved with it: the asker's mirror names "Second", the document's default is still "Default", and the rows it is sent are the ones "Second" has. The fixture's two views are transposes of each other, so "was I sent the right grid" cannot be answered by accident.

+

Two faults injected 2026-09-20, both of which fail it. Pointing grid_in at app.view_name() regardless of the session reproduces the original defect exactly. Removing ClientSession's projected_view observation is the new way to reproduce it: a switch is ViewOnly, so nothing in Dirty carries it and the session must notice its own move or never rebuild its grid at all.

+

#Helper — app_with_two_views

+

Not a test — the fixture for every lockstep and rebase test in this file.

+

A second view whose axes are the first one's transposed, so the two grids cannot be confused for each other: "Default" rows are regions, "Second" rows are measures.

+

#a_view_switch_moves_only_the_switchers_mirror

+

The successor to a_view_switch_moves_every_sessions_mirror_not_just_the_switchers, which pinned the lockstep improvise-jytm removes. The model is shared and the UI is not, so a bystander keeps its view, its scroll and its grid.

+

Three assertions, because the old bug can come back through three different doors: the bystander's mirrored active_view, its row_offset (nonzero, so a mirror reset shows as a move rather than coinciding with the origin), and whether project_all names it at all. That third one is the sharpest — a bystander repainted for somebody else's switch is invisible in the first two, since it would be repainted with its own view's content and look right.

+

The scroll is drained through project_all before the switch, so the fan-out the test reads names only what the switch caused.

+

Fault injected 2026-09-20: restoring the adopt_active_view loop (push the accepted view into every mirror) fails it, as does removing the projected_view observation.

+

#a_refused_switch_corrects_the_asker_and_nobody_else

+

improvise-6c6q. apply_switch relayed Broadcast::ToOthers, so on the refused path every client but the one that was wrong was told the accepted name. The asker's own slice kept the name it had folded optimistically while its server-side mirror held the accepted one — gotcha 24's mismatch, arriving through the asker's own keystroke. apply_switch's doc comment claimed the hole was closed, which is worse than the hole.

+

The test drives the asker's own ViewState, folding the command the way a real client would before handing it to the door. That is the whole cost of the ticket: the sibling bug in apply_model shipped green because its test read a "what do the others hear" helper, which looks away from the asker by construction. There is deliberately no such helper in this test.

+

The bystander is asserted from a view it can be moved OFF. A first version left it on the default, where a wrongly-addressed relay folds to a no-op — so the assertion read as coverage and was vacuous, which review demonstrated by injecting ToAll and watching this test pass. It now sits on "Second", and the audience itself is asserted (Broadcast::To(asker, …)) rather than inferred from what somebody's slice folded to: the audience is the message (gotcha 36), so it is the thing to read.

+

Two faults injected 2026-09-20, both of which fail it: relaying ToOthers (the original improvise-6c6q bug) and relaying ToAll (the over-correction). The second also fails folding_the_relay_puts_a_refused_asker_back, which reads frames rather than folded state.

+

#deleting_the_view_this_client_is_on_moves_it_to_the_default

+

The host-side half of deleting_a_view_moves_the_client_that_was_on_it_and_tells_it, and a desync that predates improvise-jytm: Workbook::delete_view moved the document's active view and nothing moved ViewState::active_view, so the TUI's cursor was read under a key naming a view that was gone. It was invisible while draw_status read the workbook's copy; improvise-k69y makes it read the client's, which is what turns it into a wrong badge.

+

The negative arm is the discriminating one: deleting a different view must leave this client where it is. Without it, unconditionally seating the client on the default would pass the positive assertion. The fixture therefore carries three views, and the first delete is of the one nobody is on.

+

Fault injected 2026-09-20: dropping the view_state.active_view line from effect::DeleteView fails it.

+

#deleting_a_view_moves_the_client_that_was_on_it_and_tells_it

+

A hole improvise-jytm opens and must therefore close. Before the split, Workbook::delete_view moved the document's active view and adopt_active_view pushed the result into every mirror, so deleting the view a client was on moved that client. The split removes both, and without adopt_deleted_views the client is left naming a view the document has not got: App::resolve_view falls back to the default, so it is projected the default's grid while its slice says otherwise, and viewport() reads its cursor under a key nothing will ever write again.

+

The fixture carries three views and the bystander sits on the one the delete does not remove. That is the whole discrimination: a first version left the bystander on the default, where a relay addressed to everybody folds to a no-op, and review demonstrated the vacuity by replacing Broadcast::To with ToAll and watching the test pass. The evidence entry had even stated the risk in one sentence and claimed the assertion covered it in the next — the exact shape this file exists to stop.

+

So four assertions, over three different surfaces. The audience is read off the Broadcast itself, because the audience is the message (gotcha 36). The client that lost its view moved, in its mirror and in its own slice, the latter driven by folding what the host relayed. And the bystander is still on "Third" — in its slice and in its mirror — which only means something because there is somewhere for a wrong relay to have moved it from.

+

The deletion itself is asserted, because Workbook::delete_view refuses to remove the last view: a fixture with one view would pass every other line here while deleting nothing.

+

Two faults injected 2026-09-20, both of which fail it: dropping the adopt_deleted_views call from apply_model, and addressing its relay ToAll.

+

#a_session_on_another_view_is_projected_that_views_axes

+

The hole review found in improvise-jytm's first cut: only the grid half of "every projection follows the session's view" was checked. category_tree_in and tile_bar_in are reached through ClientSession::view_in, and replacing that function's body with app.view() — the host's view — left the whole suite green.

+

The harm is worse than a wrong pixel. A category's axis is what the tile bar is for: it is the control a user reassigns axes with. Under the fault a client on another view sees the wrong axis for every category, in the widget it would use to change them, while the grid beside it is correct — nothing to notice and nothing that fails.

+

app_with_two_views is what makes two assertions enough: "Second" is "Default" transposed, so reading the wrong view swaps Row and Column rather than blurring them, and the test names both categories rather than one.

+

Fault injected 2026-09-20: ClientSession::view_in returning app.view() fails it.

+

#walking_the_view_history_moves_no_document_state

+

The other hole review found: of the three effects improvise-jytm reclassified to ViewOnly, only SwitchView had a guard (a_wire_view_switch_leaves_the_document_clean). Putting ViewBack or ViewForward back to Model(LayoutGeometry) left all 1313 tests green — which is gotcha 3's whole point: a mis-declared kind is silent.

+

It asserts the consequences rather than the declaration, because those are what a reader can price. App::apply_effects raises model_state.dirty for any Model(_), and the browser worker persists on that flag — so a mis-declared view-back rewrites the entire OPFS document (gotcha 41: Storage is whole-document) for a move that changed nothing. And it bumps layout_version, so every other client's next Command::Model is refused LayoutMoved (ADR 001) because somebody pressed a navigation key.

+

The assert_ne! on the active view before each pair of assertions is the check-the-checker: an empty history stack makes ViewBack a no-op, and a no-op dirties nothing whatever its kind.

+

Both effects are driven, because they are separate kind() implementations and review injected them separately.

+

#a_fresh_session_opens_the_documents_default_view

+

The other half of improvise-jytm's contract, and the reason Workbook::active_view survives at all: it is what a freshly connected client opens. The fixture switches an existing session first, so a connect that read any client's view rather than the document's would be visible.

+

Fault injected 2026-09-20: pushing the accepted view into every mirror in apply_switch moves the document's default nowhere, but making apply_switch write workbook.active_view fails it.

+

#Helper — diff

+

Not a test — the accessor every diff-subject test in this file reads its answer through.

+

The two halves of an Applied::Diff, for the tests whose subject is not the rebase. It panics rather than defaulting, so a command that starts replacing the document is a loud failure here and not a silent Dirty::NOTHING.

+

Note: before the comment-cap wave this helper's doc comment was preceded by an orphaned paragraph ("One connected client, as its host really drives it…") describing a type that no longer exists at that site. Dropped.

+

Both a_model_command_that_moves_the_active_view_binds_every_mirror and a_model_commands_view_switch_reaches_the_asker_too were retired by improvise-jytm, along with the to_others helper whose blind spot the second one existed to cover. No ModelCommand moves anybody's active view any more: ViewBack/ViewForward left the wire because the stack they pop is one client's history, and a document swap lands in the rebase arm. The property the second test defended — a correction must reach the client that folded it — is now a_refused_switch_corrects_the_asker_and_nobody_else, which drives the asker's own slice rather than a helper.

+

#a_geometry_command_invalidates_every_clients_cursor_including_the_askers

+

TransposeAxes and CycleAxis call app.reset_cursor() because renumbering every row and column makes the old indices name different cells (gotcha 24) — but that reset only reached the authoritative App's own view slice. Each ClientSession keeps its own cursor in its mirrored ViewState, and nothing reset those, so every client was left indexing a grid that had changed shape under it: Viewport::window clips, so what a bystander actually saw was a blank grid, until it happened to scroll (improvise-gvnb).

+

The asker is checked alongside the bystander, and that is not symmetry for its own sake. A Command::Model is the one kind a client cannot fold optimistically, so the asker is in exactly the same position as everyone else, which is why the reset is Broadcast::ToAll and not the To(id, …) a refused SwitchView correction is.

+

#a_model_command_that_does_not_reshape_the_grid_leaves_cursors_alone

+

The other arm. A model command that leaves the grid's shape alone must not throw anyone's cursor away: where a client is looking is its own UI, and only a change to the shared geometry may invalidate it. Without this, "reset on every model command" would pass the test above.

+

#a_bystanders_own_slice_follows_the_invalidation

+

And the bystander really is shown the new grid from the origin, rather than a window clipped to nothing. This is the half the mirror reset alone cannot give: the client's own slice is driven by what the host relays, so the test drives it the way Host::command does.

+

#Helper — host_dispatch

+

Not a test — the relay stand-in that test and its siblings drive a client's own slice with.

+

A stand-in for Host::command's relay, driving one client's own slice. to is the client being driven and from the one that asked, because that pair is exactly what Broadcast's audience is about.

+

#a_model_command_that_leaves_the_active_view_alone_broadcasts_no_switch

+

Renamed in review: it was ..._broadcasts_nothing, and since improvise-9mo9 it asserts a ResetCursor broadcast, so the name read as a lie about its own assertion. No client folded an active view, so there is no SwitchView correction to make. What the others are told is the Dirty it produced — and, since improvise-9mo9, the ResetCursor an AddCategory owes, because a new category lands on an axis in every view. The assertion is the exact broadcast list rather than "contains no SwitchView": a weaker form would go on passing if the cursor invalidation were dropped again.

+

#every_reshaping_model_command_invalidates_every_cursor

+

Only TransposeAxes and CycleAxis reset the cursor, while SetAxis, HideItem, ShowItem, ToggleGroup, TogglePruneEmpty, AddCategory, RemoveCategory, AddItem, AddItemInGroup and RemoveItem renumbered the grid and did not (improvise-9mo9). SetAxis is the case that shows the shape of the bug: CycleAxis is SetAxis to Axis::next(), so a pair differing only by an argument disagreed. Because reset_cursor is also the only writer of cursor_epoch, every one of these left the host's own cursor stale and broadcast nothing, which gotcha 24 records as strictly worse than doing neither — the diff then believes each client already holds origin-positioned content and sends nothing, so they stay stale permanently rather than until the next scroll.

+

Two faults were injected. Deleting app.reset_cursor() from SetAxis fails the bystander-cursor assertion (left: 1, right: 0). Changing the fixture so the item ShowItem reveals was never hidden fails the first assertion — "ShowItem reshaped nothing, so this case is not evidence of anything" — which is the one that stops this from becoming a table of commands asserted against a grid that did not move. That guard matters most for AddItem: the fixture puts two categories on Row precisely so an append to the inner one inserts a row inside every outer block, because on a single-level axis an append lands at the end and moves nothing, and a one-level fixture would have made the reset look unearned.

+

#Helper — app_with_a_reshapeable_grid

+

Not a test. Two categories on Row, one of them grouped, one item hidden and one item left with no data — so ShowItem has something to reveal, ToggleGroup has a group to fold and TogglePruneEmpty has an empty row to prune. Every command in the table has something real to reshape, which is what the first assertion checks.

+

#drilling_resets_only_the_drill_views_cursor

+

improvise-9mo9 made SetAxis call app.reset_cursor(), and DrillIntoCell issues eight or so SetAxis effects to build the _Drill view. The PR claimed that was safe and did not test it — the claim was that SwitchView precedes the batch, so reset_cursor writes the active view's cursor, which by then is the freshly created _Drill one, while the view drilled from keeps its entry in ViewState::cursors. That is an ordering argument about effects inside a batch, which is exactly the class of claim gotcha 23 exists because people get wrong.

+

Two faults injected. Removing the SwitchView from DrillIntoCell's batch fails the first assertion — "the drill did not switch views, so this proves nothing about per-view cursors" — which is the guard that stops the test passing for the wrong reason if the drill ever stops switching. Changing reset_cursor to clear the whole cursors map rather than the active entry fails the third: left: (0, 0), right: (1, 1), the cursor in the origin view thrown away.

+

It asserts both sides on purpose. That the _Drill view opens at its origin is half the property; that the other view keeps (1, 1) and row_offset: 1 is the half that would have broken, and a test asserting only the first would pass against a map that had been emptied.

+

#an_accepted_no_op_invalidates_no_cursor

+

The third negative arm, found in review. Gating the reset on Ok is not the same as gating it on "something changed", and six commands the document accepts changed nothing and reset every cursor anyway: SetAxis to the axis already held, HideItem on an already-hidden item, ShowItem on a never-hidden one, AddItem of an existing item (Category::add_item dedupes and returns Ok), and RemoveItem/RemoveCategory of something absent.

+

The first assertion is the non-vacuity guard and it is doing real work here: it uses grid_shape to prove each case really is a no-op, so a fixture drifting into one where the command does reshape fails rather than passes.

+

Review then found five more cases the table could not reach by construction, because app_with_a_reshapeable_grid had no formula, no category on Axis::None and no unknown-name case — the one-category-fixture shape the doctrine names. The fixture now carries a Profit formula, a Channel category parked on Axis::None, and a Ghost entry hidden under a name the model never had, and the table covers: SetAxis on an unregistered category, HideItem/ShowItem on a name the grid never drew, ToggleGroup on a group that does not exist, and AddItem { _Measure, Profit }.

+

That last one is the one to remember. The guard it replaced read category.items, but _Measure's effective items are explicit ∪ formula targets (gotcha 7), so it called a drawn column "new". Two of the five injections initially failed to reproduce, and both times the injection was wrong rather than the guard: the ShowItem case needed a hidden ghost name, since is_hidden is false for a name nobody hid; and the AddItem case had to compute its predicate before the add, because afterwards the item really is in category.items. An injection that does not reproduce the original ordering tests nothing — the same shape as the simulated-fix confirmation in gotcha 48.

+

ToggleGroup's first guard was also wrong and this test caught it: add_item_in_group sets item.group without pushing to Category::groups, so asking groups said "no such group" about the fixture's real one.

+

Faults injected, two, both caught by name: removing SetAxis's moved check, and removing AddItem's fresh check. The TUI repro needs no test harness — TileSelect's r emits SetAxis with no comparison against the current axis.

+

#naming_a_measure_at_the_bottom_right_leaves_the_cursor_put

+

A regression this PR introduced and review caught. AddItem gained a reset_cursor, and CommitAndAdvance{Down} on the trailing + column routes RecordCol::NewMeasure through add_measure → effect::AddItem. inserts_a_column matches only NewCategory, so NewMeasure falls through to the advance — and EnterAdvance at the bottom-right returns AbortChain, emitting no SetSelected. Nothing restored the cursor. The SetSelected repair added for +cat covered that arm only.

+

The fix moves the restore before the advance, which is gotcha 21's rule (tidy-up goes where an abort cannot reach it) applied to the cursor rather than to the mode. Injecting the old ordering fails it: left: (0, 0), right: (0, 3).

+

#a_refused_model_command_invalidates_no_cursor

+

The second negative arm, and the one the first version of improvise-9mo9's fix got wrong. It was a loop of one until review: it named a class and tested AddItem alone, and injecting reset_cursor() into AddCategory's Err arm — the very bug it was written for, one effect over — left it green. It is now a table over all three refusable effects, and that injection fails it by name. AddItem and AddItemInGroup report a failure — an unknown category — through status_msg, and the reset was written after the if let Err, so a refused add bumped cursor_epoch and broadcast Command::ResetCursor to every client for a change the document never made. AddCategory had been gated on success in the same commit, which is what made the inconsistency visible.

+

SetNumberFormat's arm cannot see this: it tests a command that succeeds and reshapes nothing, where this tests one that reshapes nothing because it was refused. Injecting the fault — moving the reset back outside the match — fails it on the bystander's cursor. The first assertion is the non-vacuity guard: if the category ever became valid, the case would stop being about a refusal and the test would say so rather than pass.

+

#Helper — grid_shape

+

Not a test. The row_items/col_items of the layout, as text: which cell each position names. It is compared before and after rather than counted, because a cats change can re-identify every position without changing how many there are.

+

#a_clients_view_only_command_moves_only_that_clients_mirror

+

The other half of the door: a command that is nobody else's business must stay that way. A client's scroll must not move the authoritative App's own view slice — the model is shared, the UI is not.

+

#a_document_swap_reseeds_every_session

+

Replacing the document must clear the cursor map, because the incoming document's view names have nothing to do with the outgoing one's (gotcha 24). install_workbook does that for the App's own view slice, but nothing touched the session mirrors: a connected client went on naming the old document's view and scrolling by the old document's cursor, so it was projected a window past the end of a document it had never seen — an empty grid, with nothing that would ever bring it back.

+

#Helper — other_document

+

Not a test — that test's incoming document, and the one whose rename hides a bug.

+

A different document, with a differently-named active view, so a mirror still keyed to the outgoing one is visible rather than coincidentally right.

+

#rebase_hands_every_session_a_snapshot_of_the_new_document

+

The reseed is reached from the projection path itself, not from a hook a host must remember to call — Sessions is not reachable from the effect that installs a workbook, so a convention there would be unenforceable. rebase exists for the other half: the client's own view slice lives on the client, and only a fresh Snapshot resets it.

+

#a_document_swap_over_the_wire_rebases_every_client

+

A document swap asked for by one client must reach every other client's own view slice, and the host must not be able to miss it.

+

Command::Model put the WholeWorkbook mutations on the wire (improvise-kmgs), so LoadModel is now something another client can ask for. The server stayed honest — the generation bump reseeds every mirror on the next project — but a bystander went on holding cursors["Default"] = row_offset 3 from a document that no longer exists, and Applied { dirty, broadcast } said nothing that would tell its host to send a Snapshot. Sessions::rebase, the only thing that produces one, had no non-test caller in the tree.

+

That is gotcha 24's cursor-vs-geometry mismatch, arriving through another client's command — the same shape doctrine(lockstep_binds_every_session_not_the_asker) names, one layer out: the obligation was addressed to a host in prose instead of being carried by the value it gets back.

+

The server's own mirror self-heals off the generation bump (a_document_swap_reseeds_every_session covers that half); this is the other one, the client's slice, which only a Snapshot resets.

+

#Helper — a_document_shaped_differently_under_the_same_view_name

+

Not a test — the fixture that test needs instead of other_document.

+

A different document whose active view has the same name as the outgoing one, and a different shape: one row where the outgoing document had three.

+

other_document renames the active view, and that rename is what makes the bug below invisible: apply_model broadcasts a SwitchView when the name moves, a client folds it, and the stale cursor is then read under a key nothing has written yet — right by accident. Same name, different geometry is the case with nothing to hide behind, and it is the ordinary one: two revisions of a document both call their first view Default. (Gotcha 39's fixture warning.)

+

#Helper — app_over_a_loadable_document

+

Not a test — the App that makes a wire LoadModel in that test really replace the document.

+

An App whose storage already holds that document, so a Model(LoadModel) arriving over the wire really does replace the document rather than setting an error status.

+

#a_command_that_leaves_the_document_in_place_is_not_a_rebase

+

The other arm of the same rule, so the check is not one that always says "rebase" (doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only)). A model command that mutates the document in place, and a view-only one, are both diffs — the question apply_model asks is whether the generation moved, not which variant arrived.

+

#ui::project::tests::a_load_that_fails_is_not_a_rebase

+

Qualified because ui::host::tests has a test of the same name: this is the apply_model one, and a bare leaf was satisfiable by either.

+

A LoadModel naming a document the storage does not hold leaves the document alone, so it is a diff too. This is why apply_model asks the generation rather than the variant: "is this one of the three WholeWorkbook commands" is not the same question as "was the document replaced", and a client would be reseeded onto the document it was already holding, losing its cursors to a failed load.

+

#Note — the cell_identity module

+

Emitting and resolving the surrogate a client names a cell by (ADR 002). The two halves are tested together on purpose: an id that is emitted and never resolved, or resolved by re-deriving it the same way it was emitted, would agree with itself while agreeing with nothing else.

+

#the_two_measures_of_a_row_share_a_record_and_differ_by_measure

+

Gotcha 1's unit, projected: two measure columns of one row are one record and two cells. Both halves matter — a surrogate that also varied by measure would break record_versions, and one that did not vary by measure at all would make the two columns the same cell.

+

#a_projected_record_id_resolves_to_the_cell_it_was_projected_for

+

The round trip that makes the surrogate usable: what the server projected for a position resolves back to the key that position has. Every row is checked, because one row cannot tell "resolves correctly" from "resolves to the first record".

+

#Helper — raw_app

+

Not a test — the fixture for every stored-cell surrogate test in cell_identity.

+

Region on Row, _Measure on Column. Nothing regular sits on None, so every cell is a stored cell. Three rows and two measure columns, so "the record" cannot be confused with "the row" or with "the cell".

+

#a_projected_aggregate_id_resolves_to_the_same_constraint

+

An aggregate names a coordinate constraint, and it must come back as the same constraint — including the page and filter coordinates baked into the layout's key (gotcha 27), which is why this resolves against cell_key rather than against the row and column labels.

+

#Helper — aggregated_app

+

Not a test — the fixture for the aggregate half of cell_identity.

+

The same, with a second regular category left on None, so every visible cell sums over Product rather than being one stored cell. That is the fork EditOrDrill makes, and the one the wire projects.

+

#a_surrogate_for_a_removed_row_fails_to_resolve_and_the_others_do_not

+

The first arm of staleness. The identifiers are held constant and only the thing that makes the id stale is varied (doctrine(a_rename_can_hide_a_stale_key)): "East" is removed, so its surrogate must fail to resolve — while "North"'s, taken from the same projection, still resolves to the same cell. Without that second assertion a resolve_cell that returned None for everything would pass.

+

#an_empty_cell_can_be_written_through_its_projected_id

+

improvise-1ey is "type a number into a cell and see the value update", and the cell a user types into is usually empty — so it is named by the grid and by nothing in the store. The index cannot answer it, which is why resolution falls back to the layout, and this is the case that makes that fallback load-bearing rather than defensive.

+

#a_write_by_surrogate_lands_on_exactly_that_cell

+

Two rows and two measure columns, so a write that went to "the first cell", "the whole row" or "the whole column" is visible.

+

#a_write_through_a_dead_surrogate_changes_nothing

+

The second half of "reject, nothing to write to": a surrogate that does not resolve must apply nothing, not fall back to some other cell. The document is compared whole, because a write to the wrong place is exactly the failure being excluded.

+

It goes through model_effects rather than the wire form, because the session door reaches the same conclusion first and typed — Checked::against refuses this command with Rejection::RecordGone, so a Checked over it does not exist. What is asserted here is the floor underneath that door: the key path and any host resolving a command locally get an empty batch rather than a misdirected write.

+

#a_records_mode_cell_is_named_by_its_coordinates_not_by_its_position

+

The trap gotcha 1 sets. GridLayout::cell_key returns a synthetic (_Index, _Dim) key here, carrying positions — the very thing ADR 002 rejected _Index for, because position names a different record after a filter, an insert or a delete. The projected id must come from RecordRow::coords instead, which is what resolving it back to the real cell proves.

+

#Helper — records_app

+

Not a test — the fixture for every records-mode surrogate test.

+

Wide format: one row per coordinate tuple, a column per raw measure, the coordinate columns in front and a + column behind. Two records and two measures, so neither "the row" nor "the column" can stand in for the record by accident.

+

#the_measure_columns_of_a_record_row_share_its_record

+

Two measure columns of one row are two cells of one record, and the next row is a different record — the same shape the pivot case has, checked here because records mode reaches it by a different path.

+

#a_coordinate_column_is_projected_with_no_identity

+

A coordinate column is not a cell to write into: editing it renames that coordinate across every measure cell in the row (gotcha 1), which is a different intent from SetCell. So it is projected with no id at all, and a client cannot compose a write against it.

+

#a_write_moves_the_version_under_the_clients_own_surrogate

+

The version counter is keyed by the same surrogate the client holds, which is what lets one identity answer both ADR 002's question and ADR 001's. A write must move the counter the client would check.

+

#Helper — apply_caught_up

+

Not a test — and note it is #[cfg(test)] fn, not #[test] fn.

+

Sessions::apply for a client that is fully caught up, for the tests whose subject is not the basis. The staleness tests build the Basis by hand, so neither kind can pass by accident through the other's convenience.

+
+

#Note — the basis_check module

+

ADR 001's basis check: a batch composed against a picture of the document that has since moved is refused, and one composed against the current picture is not (improvise-xwi6, improvise-fkfg).

+

Both arms, per tier. A check nobody has watched reject is indistinguishable from one that always accepts, and a check nobody has watched accept is indistinguishable from one that always rejects (doctrine(check_the_checker)). So every tier appears twice.

+

The fixtures hold every identifier constant and vary only what makes the basis stale (doctrine(a_rename_can_hide_a_stale_key)): the record a batch names is projected once and reused, so a rejection can never be an artifact of the test naming something different the second time.

+

#a_cell_edit_is_rejected_when_the_layout_moved_under_it

+

The class-2 case, and the one test to keep if only one survives.

+

A CellData batch is stale when the layout moved, even though no record it names was written: the coordinates its key hangs off changed underneath it. This is A removing an item while B commits an edit keyed by it, and it is corruption rather than a lost keystroke — which is exactly what a cell-level lock would not have covered, because holding a cell does not stop anyone deleting the row.

+

#Helper — basis_for

+

Not a test — the claim every basis_check test moves exactly one tier of.

+

The basis a caught-up client holds for one record. Built by hand rather than through caught_up so each test can move exactly one tier of it and leave the rest true.

+

#a_cell_edit_composed_after_the_reshape_is_applied

+

The other arm of the same tier. A batch composed after the reshape is applied — so the rule is "the layout moved since you looked", not "the layout has ever moved".

+

#a_write_is_rejected_when_the_record_it_names_moved

+

Class 1: somebody wrote the record since this client read it. Data staleness — the record still resolves, so force is coherent and the reason says so. The identifiers do not move, only the value, so a rejection cannot be a renamed key in disguise.

+

#a_write_survives_another_record_moving

+

The arm that keeps the tier from being "reject whenever anything was written". A record moving does not invalidate a batch that names a different one — which is the whole reason the tier is per-record rather than a single counter.

+

#a_write_to_a_record_that_no_longer_exists_says_so

+

Class 2 by the other route, and the reason ADR 002 put a maintained index in DataStore rather than regenerating one from the layout: "the record is gone" and "the record changed" must not collapse into one answer. There is nothing to write to here, so nothing to force — force never creates structure.

+

The test brings the layout claim up to date so that, whatever kind RemoveItem wears, only the record tier is left to answer. It is CellData today — removing an item drops its cells and moves no view's axes — so that line is currently a no-op; it is kept because a reclassification to LayoutGeometry would otherwise reject on the tier above and hide the case this test is about, silently turning it into a second copy of a_cell_edit_is_rejected_when_the_layout_moved_under_it.

+

#a_claim_that_omits_the_record_it_writes_is_not_a_free_pass

+

The force-by-omission hole, closed. Which records are checked is the server's decision, read off the command, so a client that simply left its record out of the claim does not skip the tier — it is told it never stated one. Without this, ADR 001's "force is a re-composition, not a bypass" would have a bypass sitting in the message format.

+

#a_document_swap_invalidates_every_claim

+

The top tier invalidates everything, whatever the other two say. It is checked first so the answer names the largest thing that moved: a client told "a record moved" after a document swap would go looking for a record that is not there.

+

#a_rejected_batch_changes_nothing_anywhere

+

Rejection is a normal outcome: nothing is applied, nobody else hears about it, and no basis moves. A rejection that dirtied something would tell every other client to redraw for a change that did not happen.

+

#a_published_basis_follows_the_grid_updates_it_describes

+

The order ClientSession::basis_update claims is load-bearing, pinned.

+

The basis names record counters for the cells the grid updates in the same batch carry, so a Projection::Basis that overtook them would let a client claim a version for a value it had not been shown — the mirror image of gotcha 33's rule that labels precede the cells they frame, and unlike that rule it had nothing holding it but the order of two statements in project_diff. Proved by injecting the swap (gotcha 43).

+

#a_client_that_folds_the_stream_composes_an_accepted_batch

+

The check is only satisfiable because the server publishes the basis — which ADR 001 does not mention, and without which a client could only forge one. So: a client that folds what it is sent can compose a batch that is accepted, end to end, with no test-only constructor anywhere in the path.

+

#re_composing_against_the_republished_basis_is_accepted

+

And the other side of publication: after somebody else writes, the stream carries a new basis, so the client that folds it can re-compose and be accepted. That is ADR 001's force — a re-composition against the current basis — reachable without any mechanism that skips the check.

+

#ui/host.rs

+

#connecting_hands_that_client_a_snapshot_that_draws_the_document

+

AC 2: the bootstrap is sent on connect, addressed to the new client, and it is not empty — a Snapshot whose seed carried nothing would satisfy "a snapshot was sent" while leaving the client blank.

+

#the_reported_extent_bounds_the_snapshot

+

The extent is what the snapshot is built at, so a client that says it can see one row is not sent two. This is the property that makes Viewport the handshake rather than an afterthought.

+

#without_the_command_the_cell_is_empty

+

doctrine(check_the_checker): the preceding assertion would also pass if the seed happened to contain the value already. Inject the fault — never send the command — and it must fail.

+

#a_bystander_is_projected_another_clients_edit

+

A cell edit is the whole document's, so a client that did not ask for it is told about it too. The projection is per-session because the viewports differ, not because the change does.

+

#Helper — model

+

Not a test — and note it is #[cfg(test)] fn, not #[test] fn.

+

A model command from a client that is fully caught up, for the tests whose subject is not ADR 001's basis. It goes through handle like any other frame; only the basis is supplied rather than folded, and caught_up can do that because it holds the App (a client cannot). The staleness tests build their own Basis.

+

#Helper — addressed

+

Not a test — how every host.rs test names a cell the way a client would.

+

The same cell as a client names it: the opaque surrogate the server projected on CellDisplay::id, which is the only handle a client has (ADR 002). A CellKey is not constructible over there.

+

#a_scroll_reaches_only_the_client_that_scrolled

+

A view-only command reaches nobody else. The counterpart to the test above: if every command fanned out, that one would pass for the wrong reason.

+

#a_batch_of_commands_is_applied_whole_and_in_order

+

One keypress resolved on a client is up to three commands, and the host must take all of them (improvise-bpq2).

+

The fixture is the shape that actually arrives: a cursor moved past the bottom of a one-row window, so the selection and the offset that follows it travel together. The assertion is the offset, not the selection, because a host that applied only the first command would still look right on selected — the window is what goes wrong, and it goes wrong silently.

+

#a_batch_is_abandoned_at_the_first_rejection

+

A refused batch applies nothing after the refusal. The stronger half — nothing before it either — is a_refused_batch_leaves_the_commands_before_it_unapplied; this is kept because the two halves fail independently.

+

#a_refused_batch_leaves_the_commands_before_it_unapplied

+

improvise-955n / improvise-08wb. The bug: ADR 001 is emphatic that a stale batch is never applied, not even partially, and improvise-ondj went to the trouble of a type (Checked) to make the unchecked path unconstructible. But Checked is per command, and a ClientFrame::Commands is a sequence of them: Host::commands applied in order and abandoned at the first rejection, so a frame whose second command was refused had already applied its first. The guarantee was true of a single command and false of a batch.

+

The earlier command is a view command, which is not a concession: it is the shape the one real producer emits (keys::commit — one mutation among view commands), and it is still observable, because the host folds it into that client's mirror. A frame may carry more than one mutation; what it may not do is apply some of them and refuse the rest, which is clause 4 and open.

+

A refusal test needs an accepting one (doctrine(a_helper_that_builds_the_world_the_code_wants_tests_nothing): pair a refusal with an acceptance, or the refusal can pass by refusing everything). The second half runs the same leading SetSelected followed by a command whose basis is current, and requires both to land. Without it, a SetSelected that simply never worked would clear the first assertion and prove nothing at all.

+

Inside it, the stale claim is built from caught_up and then has a single tier bumped, so the refusal can only be about the tier that was moved — a hand-built basis that is stale in three ways at once cannot tell which check answered.

+

#a_later_write_in_a_chain_still_answers_for_its_own_record

+

improvise-955n. The regression 90ea26e introduced, and the reason clause 2 cannot be applied to all three tiers.

+

Hoisting the basis check to the frame hoisted all of it, so every mutation after the first was admitted with no record check at all. Two clients, one record: A composes [SetCell(North), SetCell(South)] against an honest current basis, B writes South, A's frame arrives — and A's write to South landed on top of B's silently, which is exactly the lost update ADR 001's record tier exists to prevent. At the base commit the second command was checked and the frame refused.

+

The per-command check was doing two jobs and the commit noticed one. "Is the client's picture of the document current" is a frame-level fact and hoists correctly. "Is the client's picture of this record current" is not: which records are at stake is a property of the command, which is why records_named is an exhaustive match over ModelCommand (gotcha 43). The incapability argument that motivated clause 3 is true of records a frame creates and was over-generalised to records it merely writes — and those are the ones the tier is for.

+

Three arms, because a fix that refuses everything would pass the first alone and a fix that checks nothing would pass the last two.

+

In arm 2, B moves South with a SetCell, which is CellData: neither the document nor the layout tier moves, so only the record tier can discriminate — which is what makes this a test of the record tier and not of the hoist in general.

+

Arm 3 — the same record twice in one chain. Legitimate and likely (two measures on one record is one record) and it works for a reason worth pinning rather than rediscovering: every check happens before any command is applied, so the second write is compared to the record's pre-frame counter, not to the one its own sibling just bumped. Were the checks interleaved with the applies, this frame would refuse itself with RecordMoved.

+

Read this before closing clause 4. Interleaving is tempting there, because it would close the silence: [RemoveItem(North), SetCell(North)] under interleaving is refused RecordGone instead of silently half-applying, which is half of what clause 4 owes. So check-before-apply is not a free win — it buys the legitimate double-write and sells the loud failure on a broken chain, and clause 4 has to pay that back some other way. Measured rather than argued: injecting the interleaving fails exactly two tests, this one and a_frame_whose_later_mutation_fails_is_not_yet_undone, and the second failing is the trade. Arm 3 is what stops someone taking the trade without noticing they made it.

+

#a_data_dependent_chain_lands_in_order

+

The capability the contract is for: a chain of data-dependent commands, applied in order, each seeing what the previous one left.

+

[AddItem("East"), SetCell(the East record)] is the archetype, and it is the case a per-command basis check cannot admit however it is ordered: a client cannot claim a counter for a record its own chain is about to create, because BasisCache::claim filters to records the cache has been told about. So the basis is answered once, on the first command that carries one, and the rest of the chain runs against what its predecessors left.

+

The second assertion is the one that earns the test. Asserting only that the item exists would pass against a host that applied the first command and dropped the second — which is exactly what the previous rule did, loudly, and what the pre-frame check did, silently.

+

#a_frame_whose_later_mutation_fails_is_not_yet_undone

+

Characterization of the open hole (clause 4), not an endorsement.

+

The contract says a frame is all-or-nothing. It is not yet: App::apply_effects aborts without undoing (gotcha 21 — an abort is not a rollback), so a chain whose later mutation fails leaves the earlier one standing. [RemoveItem("North"), SetCell(the North record)] is the shape: the removal lands, the write then resolves to nothing, model_effects returns an empty batch, and the frame is reported accepted having applied half of itself. Both the partial application and the silence are open.

+

This test did not pin that, and the way it failed is the repo's own signature defect sitting on the one test whose job is to fail loudly if anyone believes the hole is closed. It asserted that North was removed and that nothing was rejected — both true of any wholly successful frame. The independent reviewer retargeted the second command from the removed North to the live South, making the frame succeed completely, and the test still passed. It could not tell half-applied from fully-applied, so it would never have inverted when clause 4 landed; it would have gone on passing. The missing assertion is the one that names the hole: the second mutation did not land.

+

It is unreachable from any client here — the only producer of a multi-command frame is keys::commit, which emits one mutation among view commands — which is what makes it safe to document rather than rush.

+

This test must be inverted when clause 4 lands: all three assertions flip, because then nothing at all happens.

+

The frame claims North at its current counter, so it passes the check and fails at apply time — which is the only way to reach the hole. A frame refused by the check never applies anything and would be testing clause 2 instead.

+

#Helper — frame_basis

+

Not a test — the frame-level claim that test and the 955n tests state.

+

The basis a caught-up client would state for a frame naming regions: the two document tiers plus each named record's current counter.

+

reduce::caught_up cannot stand in for this, and the difference is exactly what a frame is: it builds the claim for one command, so a frame's records beyond the first are simply absent from it and come back RecordUnstated. A real client calls BasisCache::claim once with every record its frame names, which is what this reproduces.

+

Name only records that exist. claim omits a record the cache has never heard of; this helper always includes one, so pointed at a record that is not there it fabricates (hash, 0) — a claim no real client could compose, and precisely the input that flips the create exemption in CheckedBatch. For a record the frame is creating, leave it out, which is what a client would do.

+

#an_accepted_switch_is_relayed_to_nobody

+

The successor to a_bystander_is_sent_the_broadcast_and_the_asker_is_not, which pinned the pre-improvise-jytm relay (Copilot, PR #10). An accepted switch is now owed to nobody: the asker folded it, and no other client's view moves.

+

It asserts the bystander gets no frame at all, not merely no SwitchView. A relay that vanished while the projection fan-out still repainted every client would pass the narrower check and still be pushing one client's view change at the rest.

+

#a_geometry_command_relays_the_cursor_reset_to_the_asker_too

+

The other audience, and the reason Broadcast carries one rather than the relay deciding. A geometry change invalidates every cursor into the document (improvise-gvnb), and the asker's is not exempt: a Command::Model is the one kind a client cannot fold optimistically, so nothing has reset it locally and skipping it would leave the very client that asked positioned in a grid it no longer has.

+

#a_stale_batch_is_refused_to_the_asker_alone

+

A refused batch is answered to the asker and nobody else (ADR 001). Nothing happened, so there is nothing to tell the others — and the asker is told only which class of staleness it was, because it already holds what is current.

+

#folding_the_relay_puts_a_refused_asker_back

+

What Broadcast::To is for, asserted end to end through Host rather than through Sessions: a client that folded a view name the document refused is put back by the frame the host sends it, so the cursor its next grid is positioned by is read under a key the document has (gotcha 24).

+

The client slice is folded before the command is handed to the host, which is what makes the assertion mean anything: a test that never folded would see "Default" whatever the relay did. The bystander is asserted silent in the same test, because a ToAll correction would also put the asker back and would be wrong.

+

#a_document_swap_sends_every_client_a_fresh_snapshot

+

AC of gotcha 39: a command that replaced the document owes every client a fresh Snapshot, not a diff — including the client that did not ask.

+

#Helper — host_over_a_loadable_document

+

Not a test — the host that test and a_load_that_fails_is_not_a_rebase drive.

+

A host whose document can be replaced out from under its clients: a second document sits in the storage backend under a name LoadModel can reach.

+

#a_client_can_ask_its_host_to_save

+

improvise-368h. The bug: effect::Save is EffectKind::ViewOnly — correctly, since it writes this host's path — so the derivation that builds ModelCommand out of the EffectKind::Model effects could not reach it (gotcha 38). There was no save on the wire at all, so no client on any transport could cause its host to write: a websocket session edited the document in memory and the file on disk never moved unless a human was at a TUI that was not there.

+

The derivation was not wrong; it was complete over the mutations and saving is not one. So the variant is added rather than derived, and it carries no path: a client cannot see the host's storage, and which document a host is hosting is the host's own fact.

+

#a_save_with_nowhere_to_write_is_refused_and_told_only_to_the_asker

+

improvise-368h: the "what if there is no path" half. Not a panic and not a silent no-op — the two answers the ticket ruled out — but the vocabulary that already exists for "nothing happened and here is why", addressed to the asker alone (gotcha 39).

+

The second client is not decoration: a refusal that leaked to the bystanders would be telling them about a failure of somebody else's host-local request, and this asserts it does not.

+

#a_write_that_fails_is_refused_rather_than_reported_as_saved

+

improvise-368h: a write that was attempted and failed must not be reported as a success. Save signals failure by aborting its batch (gotcha 21), and the batch here is exactly [Save], so an abort is that failure and nothing else.

+

The fixture fails for a reason that is real rather than mocked: a PhysicalFs pointed at a directory that does not exist. It creates nothing, so the test leaves no trace on disk.

+

#every_frame_survives_a_json_round_trip

+

The wire is JSON to start with (the ticket's choice, kept), so both directions must survive a round trip through it — including the one point at which ViewState is serialized (gotcha 25).

+

The inbound half carries a payload-bearing variant on each side of the envelope (doctrine(populate_before_roundtrip)): a Viewport and a Command carrying a CellValue::Text, which is the arm the outbound half cannot reach — a text cell aggregates to nothing under a None-axis category, so it never becomes a CellDisplay.

+

Note: host.rs also carries a second a_load_that_fails_is_not_a_rebase (doctrine(check_the_checker): inject the fault the test targets — a load that does not happen — and the snapshot must not appear; a test that saw Snapshot on every command would pass a_document_swap_sends_every_client_a_fresh_snapshot). It shares its name with the ui/project.rs test of the same name, which is why there is one heading for both above.

+
+

#ui/cat_tree.rs

+

#the_tree_carries_the_virtual_categories_and_measure_includes_formula_targets

+

improvise-vjms: the panel lists the virtual categories alongside the regular ones, and _Measure's items are derived — explicit items ∪ formula targets (gotcha 7). An implementation reading category("_Measure").items passes every other assertion in the module and silently drops Profit, which is the failure improvise-ete was closed for proposing.

+
+

#command/keymap.rs

+

#f1_and_question_mark_both_open_help

+

improvise-th7m: F1 was bound twice on consecutive lines — first to a bare enter-mode with no args, then to enter-mode ["help"]. The first was dead, because Keymap::bind inserts and the later binding wins; had it ever been the surviving one it would have failed require_args at dispatch and made F1 do nothing.

+

The risk in deleting a line of a duplicated pair is deleting the wrong one, so this pins which binding F1 is meant to carry: the same one ? carries, which is what identified it as the intended survivor.

+

#every_bound_command_name_is_registered

+

improvise-9cn: Keymap::dispatch resolves a binding through registry.interactive(name, args, ctx).ok()?, so a binding is a silent no-op key in two ways, not one: the name may not be registered, or the args may not parse. Both walks live in this test.

+

The args half matters because improvise-s13 moved bindings out of the nullary class the name check covers: enter-mode ["export-prompt"] and enter-mode ["formula-edit"] carry a mode name that parse_mode_name must accept, and a typo like "export_prompt" compiles, registers, and produces a dead Ctrl-e. (Gotcha 15.)

+

It builds the context in the mode whose keymap it is walking, because a few parsers read ctx.mode — commit-buffer item resolves its target category off AppMode::ItemAdd.

+

#prefix_keymaps_are_flat

+

improvise-jc8: an armed prefix is stored as the key that opened it and resolved against the CURRENT MODE's keymap. That is only correct while prefixes are flat — a Prefix binding nested inside a sub-keymap would be resolved against the mode map, where it does not exist, and the second half of the sequence would silently do nothing. Adding a nested prefix must fail here rather than in the terminal.

+

#every_panel_keymap_shares_the_same_spine

+

improvise-4pg: the three panel keymaps were written out separately, so a spine binding could be changed in one and forgotten in the others. They share PanelKeymap now; this pins what "the spine" is, including that each panel's cursor keys move its own cursor. (Gotcha 12.)

+

#every_mode_with_panel_toggles_binds_all_of_them

+

Every panel's toggle key is bound in Normal and in all three panel modes, so focus can move between any two and the focused one closes. A fourth panel gets all four keymaps from one Panel::ALL entry.

+

#panel_toggle_keys_dispatch_to_real_effects

+

Panel::name() has to spell what the registry's panel parser accepts. A lookup-only assertion would not catch a mismatch — dispatch swallows an unresolvable command with .ok()? — so the toggle key must come back with real effects.

+

#every_text_entry_mode_types_into_the_buffer_its_prompt_names

+

improvise-k8h: the buffer name used to be written on both sides of the crate — AppMode's MinibufferConfig names the buffer the prompt describes, and the keymap names the buffer append-char edits. If those disagree the mode prompts over one buffer while every keystroke edits another, with nothing to notice. TextEntry now reads the name off the mode, and this pins the result for every text-entry mode. (Gotcha 11.)

+

#Helper — text_entry_modes_with_exits

+

Not a test — the list that test and the two below it walk.

+

The seven modes that carry a MinibufferConfig, each with the keys that commit or leave it. SearchMode is not among them: it is Normal plus a flag, so it has no AppMode variant and no minibuffer config to agree with.

+

#every_typing_keymap_belongs_to_a_text_entry_mode

+

The list in text_entry_modes_with_exits must stay complete: every keymap that types into a buffer belongs to a mode whose minibuffer names that same buffer. Only SearchMode, which has no AppMode variant, is exempt.

+

#text_entry_modes_clear_the_buffer_they_type_into

+

Every minibuffer mode types into exactly one buffer, and every key that commits or leaves must clear that same buffer on the way — a buffer left behind pre-fills the next entry with the last one. SearchMode is excluded on purpose: it has no commit and its query is owned by exit-search-mode, not by clear-buffer.

+

The buffer is derived from each mode rather than restated in the test — every_text_entry_mode_types_into_the_buffer_its_prompt_names owns pinning what it should be (improvise-k8h).

+

#minibuffer_enter_dispatches_to_a_registered_commit

+

improvise-6t3: the three commit commands collapsed into a single commit-buffer <kind>. dispatch swallows an unresolvable command name with .ok()?, so a lookup-only assertion would still pass had the rename missed the registry — Enter has to come back with real effects.

+

#dump_every_binding

+

The instrument doctrine(binding_dump_diff) asks for: every (mode, key, modifier) lookup across all the modes, written to the file named by IMPROVISE_BINDING_DUMP.

+

Most bindings have no direct test, so a green suite is not evidence that a keymap refactor preserved them. Dump before, dump after, diff. It is inert without the variable: a test that writes a file nobody asked for is a surprise, not a check.

+

This is what proved the keymap move to improvise-protocol changed nothing — 4021 resolved lookups, byte-identical either side — and the diff was shown to be discriminating by renaming one binding and watching eight lines move.

+
+

#command/cmd/commit.rs

+

#invalid_formula_reports_the_parse_error_not_success

+

improvise-6t3: the commit pushed set_status("Formula added") after AddFormula, so the parse error that AddFormula::apply writes to the status bar was immediately overwritten. An unparseable formula was dropped while the UI reported success.

+

The add effect is the only party that knows whether it worked, so it has to get the last word on the status line.

+

#commit_and_advance_threads_edit_mode_to_enter_edit_at_cursor

+

CommitAndAdvance must thread its edit_mode through to the trailing EnterEditAtCursor effect so the post-commit re-edit lands in the mode the keymap requested. The command never reads ctx.mode.

+

#commit_save_writes_the_model_to_the_typed_path

+

improvise-lae: the save prompt has to commit to SaveAs and not to ExportCsv, which is what makes it a separate mode from ExportPrompt rather than a second binding onto it — ExportCsv would write CSV under whatever .improv name the user typed.

+

SaveAs and not Save, because Save writes the path the document already has, and the whole reason this prompt exists is that there isn't one.

+

#commit_save_on_an_empty_path_writes_nothing

+

An empty prompt must not reach SaveAs: SaveAs(PathBuf::from("")) asks the backend to write to "", and the user then reads whatever I/O error that produces instead of "you typed nothing".

+

#commit_export_bails_on_an_empty_path

+

improvise-z05r's first arm. CommitExport did not bail on an empty path where CommitBuffer's three kinds all do (gotcha 20: buffer to trim to bail-on-empty to dirty to status is the shared spine), so Enter on an empty export prompt asked the backend to write to "" and the user read whatever I/O error that produced.

+

The same change reorders the batch: the mode change now comes first, because ExportCsv aborts on a failed write as of z05r's other arm, and an abort at the tail would strand the user in ExportPrompt — where draw_bottom_bar renders the minibuffer instead of the status line, so the export error is never drawn at all and the operation appears to do nothing (gotcha 21). That ordering is asserted in commit_export_produces_export_and_normal_mode, by index and not by presence.

+
+

#command/cmd/fit.rs

+

#the_command_installs_a_formula_the_document_evaluates

+

The end-to-end claim of improvise-vx3, asserted against a real document: a command fits, the formula lands through the ordinary add_formula path, and the new measure evaluates to numbers.

+

#Helper — workbook

+

Not a test — the document every fit.rs test fits against.

+

A document a tree can learn exactly — Score steps at Cost 50 — in the units a real document uses.

+

Every category is a Label, so the only feature is the numeric measure Cost and the champion cannot reach an unresolvable one. An earlier version left Region regular with one item and claimed simplify would prune the constant one-hot; it does not, because build_categorical records (0.0, 1.0) for every one-hot whatever the column holds — so these tests were passing on seed luck (Copilot on PR #25). See improvise_gp::fit's own fixture for the same correction.

+

#the_status_line_reports_quality_against_the_baseline

+

improvise-vx3 is emphatic that the rule alone is not the deliverable: a tidy tree that generalizes no better than the mean is worse than useless. So the status line carries the held-out comparison, what the rule leans on, and the seed that reproduces it.

+

#a_refusal_names_what_went_wrong_and_stores_nothing

+

A refusal reaches the user whole. "fit failed" is not actionable; naming the measure that does not exist is (improvise-8xhy's argument, arriving at the place a user meets it).

+ +

Gotcha 37b: the seed is half of what reproduces a fit, so it is an argument — and a different one really does search differently.

+

It asserts on the rule, never on the batch's debug string. The first version compared effects_debug over the whole vec, which was vacuous: status_line prints seed {:#x}, so the two strings differ because the seed was printed, and fit_measure could have ignored request.seed entirely with the test still green (doctrine(check_the_checker)).

+

AddFormula has exactly two fields — the formula text and the target category — and the seed reaches neither, so its Debug cannot differ for the reason the batch's did. (A substring guard was tried and is wrong: the seeds 1 and 99 occur inside perfectly ordinary thresholds.)

+

#the_seed_the_status_line_prints_is_a_seed_this_command_accepts

+

Copilot on PR #25. The status line advertises the seed as {:#x} and tells the user that is what reproduces the fit, so the one instruction it gives has to work on the value it just showed them — a decimal-only parser made copying it back an error.

+

Asserted as a round trip through the status line, not against a hand-written "0x…": a literal would still pass if status_line changed how it spells the number, which is the half that breaks.

+
+

#command/cmd/grid.rs

+

#drill_into_formula_cell_returns_data_records

+

Drilling into a formula cell (Profit = Revenue - Cost) must return the underlying data records, not an empty result set. The formula target coordinate is stripped from the drill key so that matching_cells finds the raw data backing the formula (gotcha 10).

+

#drilling_fixes_its_coordinates_on_filter_not_page

+

improvise-dz8: a drilled coordinate is a constraint of the drill, not a dimension the user may page through, so it lands on Filter. It used to land on Page, which put it in the tile bar and under [/] as if the user could move it (gotcha 27).

+
+

#command/cmd/mode.rs

+

#enter_mode_export_prompt_sets_mode_and_clears_its_buffer

+

The enter-export-prompt shim is gone (improvise-s13); the generic enter-mode carries the mode name instead, and clears the prompt's buffer on the way in.

+

#enter_mode_clears_a_search_keyed_minibuffer_through_write_buffer

+

improvise-gdl made write_buffer the single accessor for a runtime-named buffer, because "search" is not an entry in the buffers map but its own ViewState field. EnterMode built a SetBuffer directly, so a mode whose buffer_key is "search" would clear a map entry nobody reads and leave the previous query on screen — the exact desync gdl closed. No shipped AppMode names that buffer yet, so the mode here is constructed by hand; that is what makes the test a demonstration rather than a restatement.

+

#edit_or_drill_passes_records_editing_mode_through

+

EditOrDrill must trust its edit_mode parameter rather than checking ctx.mode — the records-normal keymap supplies records-editing, but the command itself never inspects the runtime mode. This is the parallel of the deleted enter_edit_mode_produces_editing_mode test for the records branch.

+

#enter_edit_at_cursor_cmd_passes_target_mode_to_effect

+

EnterEditAtCursorCmd must hand its target_mode straight through to the EnterEditAtCursor effect — the keymap (records o sequence or commit-and-advance) decides; the command never inspects ctx. It fronts the pair with RebuildLayout so the cursor is read against a layout that includes whatever ran earlier in the sequence (improvise-ehz, gotcha 23).

+
+

#command/cmd/navigation.rs

+

#page_cycling_skips_filter_coordinates

+

improvise-dz8: [ and ] page through Page-axis categories. A coordinate the drill fixed sits on Filter and must be invisible to them, or the user could page out of the drill they are inside.

+

#enter_advance_at_bottom_right_emits_abort_chain

+

At bottom-right EnterAdvance has no place to go, so it emits a single AbortChain effect. Trailing effects in a CommitAndAdvance batch (EnterEditAtCursor) are then skipped, which is how "Enter at bottom-right commits and exits editing" is realised.

+
+

#command/cmd/panel.rs

+

#cycle_panel_focus_advances_from_formula_to_category

+

Bug improvise-dqn: CyclePanelFocus always picked the first open panel, ignoring the currently focused panel. With formula + category open and focus on the formula panel, cycling must move to the category panel.

+

#create_and_switch_view_names_incrementally

+

improvise-aaw asked for this command to be replaced by a keymap Binding::Sequence [create-view X, switch-view X, enter-mode normal]. It cannot be: a Sequence carries literal args, and the name here is computed from ctx — so a Sequence would hard-code one name and the second n in the view panel would collide with the first.

+

The test was called ..._names_incrementally and asserted only that CreateView/SwitchView/Normal appeared — it never looked at the name, which is the one thing that makes this command a command and not a composition. Now it does, at two different view counts, because a single count cannot tell "derived from the workbook" from "constant".

+
+

#command/cmd/registry.rs

+

#every_entry_constructs_a_command_with_its_registered_name

+

improvise-9cn / improvise-61f: every registration derives its name from a prototype instance, so a typo in the name is impossible. What the types do not catch is the commands whose name() branches on a field — ViewNavigate, TileAxisOp, ToggleGroupAtCursor, CommitAndAdvance. Their prototype and the value their constructor builds are the same type, so registering ViewNavigate { forward: true } while building forward: false compiles cleanly and files the command under the wrong name, where dispatch silently misses. Constructing every entry both ways and comparing names is the only check that reaches that class. (Gotcha 15.)

+

Some entries are deliberately one-path — commit-and-advance-right reads the edit buffer and cursor, so its parse fn always errors — but every entry must be reachable by at least one path, or its name goes unchecked.

+

#registered_names_are_unique

+

A variant mix-up does not show up as a name mismatch — it shows up as a collision. Registering ViewNavigate { forward: false } where forward: true belongs files a second entry under "view-back" and removes "view-forward" from the registry entirely; parse takes the first match, so every surviving entry still agrees with its own name. Uniqueness is what catches it.

+

#the_context_reports_each_panels_own_open_flag

+

improvise-y4xg and improvise-qsmp, two shapes of one defect. registry.rs mapped Panel to a CmdContext field in two places, and App::cmd_context copies the three flags out of ViewState in a third. Both were demonstrated unguarded: swapping Formula and Category at the two registry sites left the workspace at 1400 passing, and hardcoding the category_panel_open propagator to true left it at 1394.

+

Nothing exercised either, because cmd/mod.rs::test_helpers sets the panel flags explicitly on the context it builds — the command tests construct the value they then assert on, so no test ever drove the real propagator.

+

The registry mapping is gone: both sites call CmdContext::panel_open(panel), the one mapping on this side of the wall. That leaves two permutable sites, and the test is aimed at both from a path neither controls — it writes view_state's three fields by name and reads back through the enum accessor on a context built by the real App::cmd_context. PR #70's review caught the weaker shape of this: a panel test that zeroed and read through the same four accessors passed a consistent permutation.

+

Injected fault: swap Panel::Formula and Panel::Category in panel_open. Fails. Injected fault: hardcode the category_panel_open propagator to true. Fails.

+

#toggle_panel_visibility_reads_the_panel_it_names

+

The behavioural half, and the one that reaches the registry. Its interactive arm passes currently_open straight through, so with exactly one panel open the command built for each of the three names carries its own panel's flag. Both injections above fail here too.

+

#a_focused_open_panel_closes_and_a_closed_one_opens

+

improvise-qsmp's own scenario: toggle-panel-and-focus decides close only when the panel is both open and focused, so the case that separates a right propagator from a stuck-true one needs the mode set to the panel's own. With the flag stuck true, pressing a panel's key on a closed panel decides to close it and the panel becomes unopenable by its own key — which is what this test's second arm names.

+

#Helper — open_only

+

Not a test. It spells formula_panel_open, category_panel_open and view_panel_open as fields rather than looping over Panel::ALL through an accessor, and that is the whole evidentiary value: a permutation in any mapping between the Panel enum and those three booleans has to disagree with this helper.

+
+

#command/cmd/text_buffer.rs

+

#writing_the_search_buffer_targets_the_search_query

+

Writing the "search" buffer means writing the search query, and write_buffer is the one place that knows it (improvise-gdl). Every buffer-writing command routes through it, so clear-buffer search clears the query rather than parking an unread entry in the map.

+

#colon_wq_saves_then_quits

+

improvise-git proposed deleting the SaveAndQuit command and binding ZZ to a Binding::Sequence [w, force-quit] instead — the repo's "do not name a command AAndB" principle.

+

That premise is incomplete, and this test is why. wq is not only a keybinding: it is a registered command name, so :wq reaches it through the registry, and Quit's own unsaved-changes message advertises it ("Use :q! to force quit or :wq to save+quit"). A keymap Sequence is not reachable from the : line, so deleting the command deletes the ex-command with it.

+

Keeping the command and adding the Sequence would leave two independently-written spellings of "save then quit", which is the duplication this codebase keeps curing rather than creating. So this pins the surface, and the conversion is declined on it.

+

#colon_enter_edit_at_cursor_is_not_snapped_back_to_normal

+

improvise-6sq: ExecuteCommand appends change_mode(Normal) unless some effect reports changes_mode(). EnterEditAtCursor sets view_state.mode but did not override it, so :enter-edit-at-cursor editing produced [RebuildLayout, EnterEditAtCursor(Editing), ChangeMode(Normal)] — the trailing snap-back undid the edit mode and the user never got a cursor in the minibuffer.

+

Asserting on changes_mode() alone is not enough here: the appended ChangeMode(Normal) reports true itself, so the assertion has to name the snap-back it is ruling out.

+

#colon_import_stays_in_the_wizard_on_success

+

improvise-toc: StartImportWizard swaps the mode only when the file parses, so a per-type changes_mode() cannot describe it. Reporting false made ExecuteCommand append a return-to-Normal on top of the success path: :import d.json opened the wizard and was immediately snapped back, leaving the wizard drawn under the Normal keymap. (Gotcha 31.)

+

#colon_import_falls_back_to_normal_on_failure

+

The other half of the same bug: reporting true unconditionally would strand the user in CommandMode when the import fails. A failed :import must still fall back to Normal.

+

#colon_view_back_respects_whether_history_exists

+

The ViewBack half of improvise-toc: it restores the stored frame's mode only when the back-stack is non-empty. With history present, :view-back must keep the mode it restored; with an empty stack it falls through to Normal like any other no-op command.

+
+

#command/cmd/wizard.rs

+

#name_model_enter_builds_the_model

+

Enter on the name step builds the workbook into the wizard rather than advancing the step machine; every other printable key edits the name (gotcha 22).

+

#done_confirms_on_enter_and_steps_back_on_esc

+

On the confirmation step Enter installs the built workbook and Esc steps back — it does not throw the import away, and no key closes the wizard outright (improvise-907, from improvise-c6j).

+

#a_formula_owned_cell_is_projected_as_derived_and_its_siblings_are_not

+

improvise-ir7l. project::cell_id decided Record vs Aggregate purely on whether a regular category sat on Axis::None; it never asked whether the _Measure coordinate named a formula target. So a formula cell was a Record, render.rs::editable said true, and a write to it was accepted, stored and then shadowed forever by formula_cache.

+

The fault injected: replacing the derived lookup with None, which is the pre-fix code exactly. The test then reports Record(RecordCell { measure: Some("Profit") }) where it wants Derived { target: "Profit" }. The siblings are asserted in the same test so a producer that answered Derived for every cell could not pass.

+

#an_aggregating_formula_cell_is_still_projected_as_an_aggregate

+

The negative arm of the same change, and the one a careless fix breaks: gotcha 10 says drilling into a formula cell is supported, so Aggregate must still win where the cell aggregates. Proved discriminating by returning derived before the aggregate branch — the cell then reads Derived and becomes undrillable, with nothing else in the suite noticing.

+

#edit_or_drill_refuses_a_formula_owned_cell

+

The TUI half. EditOrDrill never asked the question either, so the terminal offered the same editor over the same shadowed cell. Landing only the projection half would have made the two front ends disagree, which is the defect design-principles §5 is about. Fault injected by deleting the guard: the command returns [SetBuffer{edit}, ChangeMode(Editing)] again.

+

#a_save_clears_the_dirty_marker_on_a_client_although_nothing_is_dirty

+

improvise-z754, and the arm the whole design turns on. effect::Save is EffectKind::ViewOnly, so Dirty::of reports NOTHING — the test asserts that first, so it cannot quietly stop being the interesting case if Save is ever reclassified. A title bar gated on Dirty would therefore leave every client showing [+] for a document that has just been saved, forever, with nothing in the stream to correct it.

+

Fault injected: wrapping document_update in if !dirty.is_empty(), which is the obvious implementation and the wrong one. The test fails on the cleared arm. a_title_bar_that_did_not_move_is_not_projected is its counterweight — without it, "project the title bar every batch" would pass this one.

+

#the_title_bar_names_the_document_without_naming_the_path

+

The fixture's path is /home/someone/private/books.improv on purpose: a test whose fixture is a bare file name cannot tell file_name() from the whole path, and the failure mode is a socket server handing a remote client the host's directory layout.

+

#a_creating_column_is_named_rather_than_left_as_no_cell

+

improvise-poia. Both + and +cat projected None, which the protocol defines as "no projected cell" — the same conflation gotcha 42 names for aggregates, arriving a third time.

+

The test walks every column of a records grid and counts, rather than asserting the two creating columns alone. That matters in the direction a weaker test fails: a producer that answered Create for everything, or that made the Region coordinate column nameable, would pass a two-position check and fails this one. The fault injected — returning None for both creating columns, which is the pre-fix code — reports three nameless columns where one is allowed.

+

#a_records_coordinate_is_text_however_it_is_spelled

+

improvise-tom1. project::cell_display's records arm called records_display — which returns a String — and then inferred numeric-ness with text.parse::<f64>(). A records coordinate column holds an item name, so a Year category with items 2024/2025 projected a coordinate the browser right-aligns as a number.

+

Nothing in the suite could see it: every records fixture in the tree names its categories Region/East/North, so the guess happened to be right. The fixture here is the whole test — a category whose items are spelled like numbers — and the measure column beside it is asserted numeric in the same test, so a producer that answered false for everything would not pass.

+

Fault injected: the parse::<f64>() guess restored verbatim. It reports the coordinate as numeric.

+

#a_staged_drill_edit_reaches_its_own_session_and_no_projection

+

The user's ruling, pinned in both directions (improvise-tom1). A remote client must NOT see another client's staged drill edits, so grid_of must project the committed value; the session that staged one must still see its own typing, through App::display_value_of.

+

Asserting only the second half would pass against a producer that leaked, and asserting only the first would pass against one where staging did nothing at all — so both are here. The fault injected is the leak: teaching cell_display's records arm to prefer drill_state.pending_edits, which is the plausible "fix" this ticket exists to forbid. The projected cell then reads 999 instead of 1.

+

The fixture takes its DrillState::records from the layout the app really built, because an empty frozen snapshot silently empties the drill grid and the test then asserts nothing about a cell that does not exist — the first version did exactly that and panicked on expect.

+

#a_page_axis_carries_its_items_and_a_filter_axis_does_not

+

The producer half of improvise-p1ym, added during self-review because a_filter_axis_is_projected_but_not_cycled builds its own PageAxis by hand — a test that fabricates the very data the code under test is supposed to produce, which is this repo's characteristic defect. Only this one can say what grid_of actually emits.

+

The fixture puts Quarter on Axis::Page and Year on Axis::Filter, which is the shape a drill makes (gotcha 27), and asserts the whole projected list in one comparison so a producer that emitted items for everything or for nothing both fail. Fault injected: let navigable = true, which is filtering_cats() leaking into the protocol — the exact trap the ticket names. Year comes back carrying ["2024"].

+

#a_records_grid_has_no_column_for_a_formula_target

+

A records grid has no column for a formula target that is only a formula target, because GridLayout::records_columns builds its measure columns from Category::ordered_item_names — the explicit _Measure items — while add_formula adds none (gotcha 7). Fault injected: swapping ordered_item_names for measure_item_names in records_columns, which is a plausible tidy-up — a Profit column appears.

+

This test's original premise was FALSE, and the review caught it. It was written claiming the records branch of cell_id was safe because "a formula target lives only in measure_item_names". An item can be both an explicit _Measure item and a formula target, and improvise-poia — in the same PR — made the + column that builds exactly that reachable. So this test asserted the accident in the one configuration where the accident holds, and its own failure message named a repair for a case it could not reach. The real guards are a_records_column_a_formula_owns_is_derived_not_a_record and typing_into_a_records_formula_column_is_refused_on_the_key_path; this one now claims only what it can see.

+

#a_records_column_a_formula_owns_is_derived_not_a_record

+

The hole improvise-ir7l's first fix left, found by adversarial review. A measure can be both an explicit _Measure item and a formula target — the records + column creates precisely that — and cell_id's records branch asked formula_for about nothing, so it projected CellId::Record{measure:"Profit"}. A write there is accepted, stored, and then shadowed forever by formula_cache, which is the defect the ticket exists to close, surviving in records mode in both front ends since editable is Record-only.

+

The repair asks the document about the record's real coordinate — GridLayout::record_cell_key, the one mapping from a records position to a model cell — never the synthetic (_Index, _Dim) key, which names no measure and so can never match a formula. Fault injected: deleting the lookup, which is the pre-review code. Profit reads back as a Record. The Revenue column is asserted still writable in the same test, so a producer answering Derived for everything could not pass.

+

#typing_into_a_records_formula_column_is_refused_on_the_key_path

+

The same defect through App::handle_key and no test API at all: R, walk to +, type Profit, Enter, Esc, then i on the column just created. It exists because the projection half and the EditOrDrill half are separately breakable, and a unit test of either would have missed the other — proved by injecting each alone. Restoring cell_id while leaving CmdContext:: owning_formula reading the raw key fails on the mode assertion; the reverse fails on the identity assertion.

+

One fixture note that cost a debugging round: CommitAndAdvance re-enters editing on the cell it advances to, so the mode after Enter is RecordsEditing and an i there is a literal character. Without the Esc the test passes its mode assertion for the wrong reason — nothing happened at all.

+

#a_save_tells_the_client_the_document_is_clean

+

improvise-z754's own stated failure mode, shipped: "every client showing a [+] for a document that has been saved". Host::save applied effect::Save and returned only a possible Rejected, never projecting, so the dirty marker cleared outside any projection and no client was ever told.

+

The z754 test that was supposed to cover this called sessions.project_all by hand — which is exactly the call the production path omits. It is not a test that compares a thing with itself; it is a test that exercises a call the deployment never makes, which is the subtler shape. This one goes through Host::handle(ClientFrame::Save).

+

Two assertions guard the fixture: the edit must first report dirty: true, and app.model_state.dirty must really be false afterwards — otherwise a host that saved nothing would pass by projecting nothing. Fault injected: returning an empty vec from the success arm. No Document update is emitted.

+

#the_automatic_persist_tells_the_client_the_document_is_clean

+

The worker's half of the same defect, and a different cause: on_frame built out via frames_for and then called persist(), so the projections were built while dirty was still true. The browser is the deployment where this bites hardest, because nothing there ever sends ClientFrame::Save — the worker persists on its own behalf (gotcha 45), so this was the only path.

+

Fault injected: dropping the re-projection after persist(). The client's DocumentCache stays dirty. MainThread grew a document mirror and a projection dispatch for it, rather than a bespoke reach into the worker — the point is what a client folding the real stream ends up holding.

+

A first version asserted only !document.dirty, and DocumentCache::default() is already clean — so it could not tell "told it cleared" from "never told anything", and passed under an injection that stopped projecting the widget entirely. Found in re-review. It now records every marker the client is told and asserts [true, false] for the edit, with the bootstrap's own clean marker cleared first so the sequence is about the edit rather than the handshake. The stronger form fails under both injections.

+

#a_refused_or_duplicate_add_category_moves_no_counter

+

improvise-75ld. App::apply_effects bumped layout_version and raised model_state.dirty off the EffectKind before the abort_effects check, so the refusing effect counted itself. Two arms, and the second is the reachable one: a 13th category is refused (MAX_CATEGORIES) and aborts, while a duplicate add-cat Region returns Ok and does not abort at all — a pure no-op that still refused every other client's next Command::Model with Rejection::LayoutMoved and made the browser worker rewrite the whole OPFS document (gotcha 41).

+

Injected fault: put the two bumps back on the declared kind, inside the loop (if matches!(kind, EffectKind::Model(ModelChange::LayoutGeometry))). Both no-op arms fail; so do neither_shape_counter_moves_for_an_accepted_no_op, both_shape_counters_move_for_every_reshaping_command and a_configuration_change_that_draws_nothing_is_still_a_document_change, with 450 other tests still passing — the build ran.

+

The positive arm is not decoration: without it, "never bump anything" passes the two no-op arms. It also fails under a second injection — dropping the document_config comparison — because AddCategory is LayoutGeometry and the declared half of dirty no longer covers it.

+

#both_shape_counters_move_for_every_reshaping_command

+

improvise-pdvm. AddItem, AddItemInGroup and RemoveItem declare ModelChange::CellData and reshape the grid. Since improvise-9mo9 they call App::reset_cursor, so cursor_epoch moved while layout_version did not: a client was told its cursor was invalid while its layout basis tier still read current, and a batch composed against the old shape was still accepted.

+

It drives reshaping_commands() — the same table every_reshaping_model_command_invalidates_every_cursor drives, extracted into a shared function precisely so the two cannot answer for different cases — and asserts the grid shape really moved before asserting either counter, so a case that reshapes nothing fails loudly rather than proving nothing.

+

Injected fault: as above. Fails naming SetAxis, then AddItem once the earlier cases are removed.

+

#a_configuration_change_that_draws_nothing_is_still_a_document_change

+

The third case the shape/config split creates, and the one that forced two fingerprints rather than one. hide_item records a name whether or not the model has one, and SetNumberFormat writes the file while moving no row or column: both are dirty and neither is layout_version. A single fingerprint covering the persisted configuration made a ghost hide bump the basis; one covering only the drawn shape let a format change go unsaved.

+

Injected fault: remove the document_config comparison in observe_reshape. Fails naming HideItem { category: "Region", item: "NoSuchItem" }.

+

#a_reshape_of_one_view_leaves_a_client_on_another_view_alone

+

improvise-nynk. improvise-jytm made "which view a client is on" per-client; Sessions::adopt_cursor_epoch was not updated and still reset every mirror and returned one Broadcast::ToAll(Command::ResetCursor). A reshape of view Default therefore threw away the cursor of a client sitting on Second, which nothing reshaped.

+

The fixture is two views of one document with different axes, and the bystander switches to Second and scrolls there before the asker reshapes Default. The positive arm is kept in the same test — the asker is on the reshaped view, is reset, and is told — because without it "never invalidate anyone" passes.

+

Injected fault: drop the cursor_invalidated filter (.filter(|_| true)). Fails naming the bystander's thrown-away cursor, with 453 other tests passing.

+

#an_unknown_reshape_set_invalidates_everybody

+

The fail-safe arm of the same narrowing, tested separately because it is the half gotcha 24 calls strictly worse than doing nothing. An empty reshaped set means the batch moved a cursor with no view observed to change shape; the answer must be "everybody", not "nobody", or the diff believes each client already holds origin-positioned content and sends nothing, leaving them stale permanently.

+

Injected fault: delete the reshaped.is_empty() || disjunct. Fails by name. That injection is why the predicate is a named function rather than a closure — a fail-safe nothing can point at is a fail-safe that gets tidied away.

+

#a_batch_that_changes_the_layout_and_then_saves_ends_clean

+

A regression introduced by the first version of the observed-dirty change and caught in review of PR #70, not by the suite — the workspace was green at 1400 with this broken. observe_reshape ran after the effect loop and re-raised model_state.dirty from the document_config comparison, while Save had cleared it inside the loop. So gotcha 45's stated ordering property — "Save clears it in loop order, so a batch that edits and then saves ends clean" — survived for CellData/WholeWorkbook and was broken for LayoutGeometry.

+

[SetNumberFormat, Save] reported a successful save and left the document dirty: the TUI draws "Saved to …" while Quit's check refuses a later :q, and the browser worker — whose persist trigger is the flag — pays a whole-document OPFS rewrite. Reachable from :format ,.2f . w and from any Binding::Sequence pairing a geometry effect with a save.

+

The fix is position, not logic: the comparison moved inside the loop, taken after each LayoutGeometry effect against the value held after the previous one. Both arms are in the test — [SetCell, Save] never left the loop, so the geometry arm alone could not say whether the rule is general — and each asserts the save really happened (last_batch_aborted, and a status line starting "Saved") before asserting the flag, or a failed save would pass the dirty assertion for the wrong reason.

+

Injected fault: move the comparison back after the loop. Fails naming the layout arm, with 454 others passing.

+

The lesson worth carrying: an observation that is correct about what changed can still be wrong about when it is read. Everything above this entry argues that observing beats declaring; this is the cost of that move, and it lands precisely on the one effect that unsets what the observation sets.

+

#the_published_basis_names_the_records_the_grid_draws_and_no_others

+

improvise-23qd. The set of records a Projection::Basis carries was derived twice — ClientSession::basis_update, the producer, and server_truth, a #[cfg(test)] helper that is the yardstick every fold test measures against. So a change to which records get published moved the thing and its own measure together, and every test stayed green whatever the change was. Not a helper that fabricates an input; a helper that fabricates the expected output.

+

Both now call published_basis. Note what that alone does not buy: the ticket's stated proof ("alter the filter and watch the basis tests fail") is impossible once one function feeds both sides — they still move together. A shared derivation removes the drift and removes the yardstick with it.

+

So the yardstick is spelled out rather than derived: this test names North, South and East by hand for app_with_a_reshapeable_grid's three records over six cells, and asserts that a grid where every cell is an Aggregate publishes none — because an aggregate names no record, so there is nothing to claim a version for.

+

Fault injected: .take(2) on the one surviving filter — the ticket's exact "change which records get published" — fails this test and nothing else in the crate. A second injection, record_version_of(..) + 1, is caught instead by the two end-to-end basis_check tests, which is the split worth knowing: the counters have other guards, the membership had none.

+

The fixture carries its own premise check. The first version assumed a new category lands on Axis::None; it lands on Column, and the assertion said so instead of the test quietly proving nothing.

+

#an_aborted_batch_bumps_layout_version_for_what_it_did_run

+

improvise-2avl, with its negative twin an_aborted_batch_bumps_nothing_for_effects_it_never_ran. PR #70 moved the layout_version bump from "once per LayoutGeometry effect, inside the loop" to "once per batch, after the loop". Gotcha 40 records the in-loop position as load-bearing — an aborted batch must not claim effects it never ran — and the author's argument for why end-of-batch is safe for layout_version specifically (nothing ever clears it, so an end comparison sees only what ran, whereas dirty must stay in the loop because Save unsets it mid-batch) is sound and was written into context/code/command.md. An argument is not a test, and neither arm had one.

+

Both arms are proved independently, which is the part that earns them: bumping unconditionally fails only the second test, and skipping observe_reshape on abort fails only the first. A single injection failing both would not have shown that the two tests see different things.

+

Each fixture asserts app.last_batch_aborted() before asserting anything about the counter, because a batch that did not actually abort would pass both arms while testing neither.

+

#sorting_the_store_in_records_mode_invalidates_the_cursor

+

improvise-ni6k names SortData as its sharpest case: it changes neither the row count nor any label, and every position still names a different record. A guard written as "did the shape change" misses it; pending_edits are keyed by row index (gotcha 1), so row 0 being West and then East is gotcha 18's misdirected-write disease arriving through a different door.

+

view_frame therefore hashes each record's coordinates alongside the axis entries, because records-mode row_items are positional — 0,1,2… whatever order the store is in. Fault injected: deleting those three lines fails this test and nothing else in the workspace.

+

The fixture trap, which is the durable part. PR #78 first shipped without the coords hash, on the stated ground that no fixture could be built where SortData reorders anything. That was wrong, and the reason is worth more than the fix: ToggleRecordsMode pushes effect::SortData as its first effect, so a records-mode fixture built by pressing R has an already-sorted store by construction and cannot demonstrate anything order-sensitive. The store itself was fine all along. So this fixture sets _Index/_Dim directly rather than going through the key path.

+

The test asserts the count and the labels are unchanged before asserting the cursor moved, so it cannot pass by accidentally becoming an ordinary shape change.

+

#paging_with_nothing_pruned_invalidates_no_cursor

+

The negative twin of paging_across_a_prune_that_drops_a_row_invalidates_the_cursor, and the reason view_frame hashes no page selection.

+

improvise-7kvw is real only under prune_empty: a page flip changes which records are visible, not which categories frame them, so with pruning off a grid position keeps naming the same cell. PR #76 ruled that homing the cursor on every [/] regardless — [/] being navigation a user does constantly — is worse than the hole. Hashing the page selection does exactly that.

+

The seductive argument for hashing it anyway is that two pages with identical row labels really are different data. True, and irrelevant: that is record_versions' question, and view_frame answers the cursor's. A basis argument applied to a cursor decision is how this regression got written in the first place.

+

Fault injected: restoring the three page_selections lines fails this test and nothing else. The positive arm keeps passing either way, because with_frozen_records applies prune_empty and so already sees a dropped row in the axis entries — which is why the negative arm is the one that had to exist.

+

#a_write_that_prunes_a_row_out_of_another_clients_view_invalidates_it

+

improvise-mcq4's sting, found by the adversarial review of PR #78. The mcq4 fix narrowed the frame fingerprint from every view to the active one, on the strength of a measurement showing the narrowing cost nothing. It bought its speed partly with correctness: a CellData write that prunes a row out of another client's view became invisible. view_shapes is config-only — it hashes the prune_empty flag, not which rows survive — so last_reshaped_views is empty, the one-view frame covers only the host's, and a client on the other view is never sent ResetCursor while its grid loses a row underneath it. That is gotcha 24's "strictly worse than doing nothing": the diff then believes the bystander is current, so it stays stale permanently.

+

The repair keeps the narrow one-view frame on the LayoutGeometry arm — the held-key path improvise-748f exists for, where other views cannot reframe because view_mut() writes only the active one — and restores the all-views frame on the CellData arm. Measured: 0% on the held-key path, +5.7% on an 18k-cell two-view SetCell, scaling linearly in view count.

+

Writing this test found a second bug, which was masking the first. The precondition failed at first: grid_in built a non-active view with GridLayout::new, which does not apply prune_empty, so a client on another view was projected three rows where its own config gives two. Their grid never pruned, so it never lost a row, so the staleness was unobservable — a bug hiding a bug. It is the third instance of the new-versus-with_frozen_records trap (view_frame was the second), and fixing it broke no other test.

+

Fault injected: removing the all-views arm fails this test, plus sorting_the_store_in_records_mode_invalidates_the_cursor (SortData is CellData) and both cursor-audience tests — four in all.

+

#a_switch_then_a_geometry_effect_compares_within_one_view

+

Found by the same review. frame_basis was taken once at batch open against view_state.active_view; SwitchView is ViewOnly, so it did not re-base, and the next model effect hashed the new view and compared it against the old view's hash. A near-certain mismatch, so a spurious reset_cursor() and a wrong name in last_reframed_views.

+

Today's instances are benign — DrillIntoCell and EnterRecordsMode both emit [CreateView, SwitchView, SetAxis, …] in one batch, but against a view they just created. The harmful case needs a switch to an existing view with a saved cursor plus a model effect in the same batch, which a .-chained : line reaches. Low severity, unsound mechanism.

+

The basis is now keyed by view name and re-bases rather than comparing when the name moves. Fault injected: dropping the was == self.view_name() guard fails this test, with cursor_epoch moving for a batch that reframed nothing.

+

improvise-9c9o moved where the re-base happens, and this test was unchanged by it because it names the property, not the mechanism — the re-base now fires on the switch rather than on the next geometry effect, and a batch that reframes nothing still invalidates nothing.

+

Its fixture carried a line that did nothing at all: views["Second"].clone_into(&mut views["Default"].clone()) clones into a temporary that is immediately dropped. It read as the thing making the two views differ, so had create_view ever started copying the active view's config this test would have gone vacuous with that line standing there looking like it prevented exactly that. It is deleted and replaced by the assertion it was pretending to be: the two views' view_frame hashes really differ.

+

#a_geometry_effect_after_a_mid_batch_switch_still_invalidates_the_cursor

+

The other side of the guard above, and introduced by it. PR #78 round 1 compared frame hashes across a view change, which threw a cursor away spuriously; round 2 added was == self.view_name(), which fixed that and also exempted the first geometry effect after a mid-batch SwitchView from the check altogether — there is no basis for the new view yet, so the code re-based and skipped. Passes on pre-#78 main and on round-1 head; fails at the merged head.

+

Direction is why this is worth more than the bug it replaced. Round 1 kept a cursor it should have thrown away; round 2 threw away nothing it should have kept — which is gotcha 35's expensive side, since a stale cursor has nothing in the stream to correct it where a spurious invalidation costs one projection.

+

The fix is the ticket's: the basis follows active_view, re-based in the loop as soon as the name moves, rather than at the next geometry effect. It did not need SwitchView to stop being ViewOnly and it is not a protocol change — the basis is App's own bookkeeping, and nothing about it is on the wire. The was == self.view_name() guard is gone from the comparison because it is now unreachable: by the time the geometry arm runs, the name and the basis agree by construction.

+

What it changes elsewhere, and why that is fine: DrillIntoCell and EnterRecordsMode both emit [CreateView, SwitchView, SetAxis, SetAxis, …], so they now reset the cursor on the first SetAxis where they used to skip it and reset on the second. Same outcome, one effect earlier. DeleteView is the one LayoutGeometry effect that moves active_view itself; the re-base fires for it too, so the geometry arm compares the new view against itself and resets nothing — which is what the old skip did.

+

The fixture measures the row count the transpose must move rather than spelling it: a first version asserted row_count() == 1 and the real answer is 2, which is the shape of a precondition that would have gone quietly wrong later. It fails on the unfixed tree at the epoch assertion, left: 0 right: 0.

+

#a_frozen_drill_does_not_take_its_opening_frame_from_the_held_layout

+

improvise-mcq4's fix reads the batch's opening frame off App::layout rather than building a second pruned layout — that is where the 4.1×→1.9× came from. It is guarded on drill_state.is_none(), because a drill freezes the record list and the held layout is then not what view_frame would build for that view.

+

The guard was correct, load-bearing, and tested by nothing: reverting it to the unconditional form left all 1439 tests passing. Textbook doctrine(a_careful_comment_is_weak_evidence_that_anything_checks_it), and it was the most carefully argued comment in the PR.

+

The fixture freezes one record where the live query finds three — asserted, so a freeze that did not narrow could not pass — then runs a SetNumberFormat batch, which reframes nothing. Without the guard the opening basis is the frozen layout and the in-loop hash is the live one, so they differ and the cursor is reset for a batch that changed no row. Fault injected: replacing self.drill_state.is_none() with true fails this test and nothing else in the workspace.

+

#a_clients_grid_does_not_depend_on_which_view_its_host_is_on

+

improvise-t6db. The one-line grid_in fix landed in PR #78 with no test of its own; this is the general property the reviewer specced — host-view independence: grid_in(app_on_A, "B", vp) == grid_in(app_on_B, "B", vp) for every ordered pair of views. It needs no second derivation of what a view's config means, which is why it is a property and not a second copy of GridLayout.

+

The trap is the fixture, not the assertion. The reviewer's first version made the two views differ by turning prune_empty off in the second — plenty of differing config, and vacuous: with nothing to prune, GridLayout::new and GridLayout::with_frozen_records agree and the property passed on the buggy tree. So app_with_three_views_and_an_empty_row gives Service no cell at all and leaves all three views pruning, and the test asserts every view really is two rows before comparing anything — a fixture that stopped pruning would fail there rather than passing quietly.

+

The property is stated with drill_state.is_none() asserted, because a frozen drill snapshot is the one thing grid_in cannot reproduce for a view the host is not on (with_frozen_records(.., None) on the elsewhere path).

+

Fault injected 2026-09-21: restoring GridLayout::new in grid_in — the pre-#78 line — fails this test at the assert_eq!, not at the precondition. It also fails a_write_that_prunes_a_row_out_of_another_clients_view_invalidates_it and a_hosts_none_cats_leak_into_another_views_formula_columns: three tests, not the one an earlier draft of this entry claimed. The #78 fix has more coverage than it was credited with.

+

The none_cats asymmetry is REAL, and the first draft of this entry reported it as unobserved from a fixture that lacked the condition. That correction is worth more than the test. The claim was that the fixture put Product on None in Default and on Column in Second; it did not — add_category defaults a new category to Column and nothing moved it, so all three views had none_cats {_Index, _Dim}, the stated input to the hazard was absent, and Default and Second were indistinguishable to grid_in apart from a number_format that is itself a no-op. An unprobed path reported as a negative result, which is doctrine(an_injection_that_does_not_fail_names_an_untested_path) applied to a setup, and the expensive kind: it stops the next person looking.

+

Setting Product to Axis::None explicitly made the property fail on the first run, and not on the value of a cell — Second lost both Total columns entirely when the host stood on Default. Filed as improvise-ac2z, pinned by the characterization test below, and the property now skips pairs whose none_cats differ. That skip is a bug being worked around, not a carve-out; it asserts compared > 0 so a future fixture cannot skip everything and pass.

+

#a_window_narrows_the_cells_and_neither_the_totals_nor_the_widths

+

improvise-jmjz. project::grid_of evaluated the whole grid roughly three times per frame — compute_col_widths walked every cell for its text and then every cell again for the column totals it must size, and grid_of then evaluated the window a third time. ui/viewport.rs::grid_body now evaluates each cell once and derives the widths, the totals and the window from that value.

+

That fusion creates a hazard the three separate passes did not have: one walk now serves three consumers with different scopes. Two faults follow, not one — a window that leaks into the totals makes a column total the sum of the visible rows, and a window that leaks into the widths sizes a column by what happens to be on screen. Neither is visible from the output: the numbers simply get smaller, consistently.

+

The fixture took three attempts, and what each one was blind to is the entry.

+

First: rows of 1, 20, 300000 under a measure named Revenue, asserting col_widths[0] > "300021".len(). It could not see a windowed width at all — Revenue is 7 characters against a 6-character value, so the assertion passed on the header alone. Worse, its other half was structurally incapable: comparing grid_body's widths against compute_col_widths cannot test windowing, because compute_col_widths is grid_body with an empty window. Windowing degrades both sides identically, both collapsing to the MIN_COL_WIDTH clamp. The same function twice is not a check, and it reads exactly like one. Measured by the PR #92 review: if drawn { widen(..) } left 1476 passing, 0 failing.

+

Second, the reviewer's: 1, 888888888, -888888888 with a one-character measure name, so the total cancels to 1.0 and neither it nor the header can supply the width. This fixed the width half and broke the other: the windowed row is 1.0 and the cancelled total is also 1.0, so the totals assertion went blind to the row-window fault, which the widths assertion then happened to catch. Two assertions, one no longer discriminating, and nothing saying which.

+

Third, and current: the same values with the large ones first, so the window holds 888888888 while the total is 1.0. Each injection now fails on the assertion that names it:

+ +

10 rather than 5 is a smaller gap than the reviewer's fixture gave and a better witness: the missing character is the minus sign, which only -888888888 carries and the window does not contain. The test also asserts the window really is narrower than the grid, so a fixture that stopped windowing would fail at the precondition.

+

What this test does not do is prove the speedup, and no test can — the outputs are identical by construction, which is why the whole suite passed unchanged across the change. The evidence is the benchmark, before and after, in context/code/command.md: about 3x, the number of passes removed.

+

#a_formulas_columns_survive_a_host_that_aggregates_differently

+

improvise-ac2z, fixed. This replaces the characterization test a_hosts_none_cats_leak_into_another_views_formula_columns, whose message said that failing meant the bug was fixed; it did, and it is gone.

+

App::rebuild_layout calls recompute_formulas(&none_cats) for the host's view, populating Model::formula_cache. project::grid_in on the elsewhere path built the other view's GridLayout against that same cache, which is keyed by the coordinates the recompute used: with Product on None in the host's view the Total cells are keyed without a Product coordinate, a view with Product on Column asks for (Region, Product, _Measure=Total), finds nothing, and — prune_empty being on — drops both Total columns. Not a wrong number: absent columns.

+

host on Second, projecting Second 6 columns, Total present host on Default, projecting Second 4 columns, no Total at all

+

The fix is grid_in::rederived_for: where the target view aggregates over different categories than the host's, it clones the model and recomputes the formulas for that view's none_cats. Three conditions gate the clone — another view, different none_cats, and at least one formula — so the common case pays nothing and a formula-free document pays nothing ever.

+

This is an interim, and the cost is the reason. A clone of the whole model plus a fixed-point recompute, per projection, on exactly the path that is broken today. The ticket's first candidate — keying formula_cache by none_cats so one Model can hold more than one view's derived values — is the durable answer and is filed separately; it is an improvise-core change and did not belong in this wave's ground. What is bought meanwhile is correctness on a path that was silently wrong, doctrine(the_elsewhere_path_is_only_as_correct_as_the_inputs_it_can_rebuild).

+

Fault injected: rederived_for's condition forced to false. Two tests fail — this one, and a_clients_grid_does_not_depend_on_which_view_its_host_is_on, whose none_cats carve-out was removed at the same time and which is the general property this one is an instance of. The test asserts its two preconditions first — that Second on its own really draws the formula's columns, and that the two views really do aggregate over different categories — so a fixture that lost either fails rather than passing on two empties.

+

#a_view_named_command_lands_on_the_view_it_names

+

improvise-lltt, the oldest open P1 here, opened by improvise-jytm making ViewState::active_view authoritative per client.

+

Nine ModelCommand variants configure a view — SetAxis, CycleAxis, TransposeAxes, SetPageSelection, HideItem, ShowItem, ToggleGroup, TogglePruneEmpty, SetNumberFormat — and each named its view implicitly: model_effects built an effect that wrote App::view_mut(), which is the host's slice. A client that had switched to Second therefore moved Default: its own grid did not change, and every other client's did.

+

The fix is the protocol change the ticket asks for. Each of the nine carries a view: String, model_effects maps it to ViewTarget::Named, and the nine effects carry a ViewTarget rather than reading an ambient one. There is no operandless form left: App::view_mut() is deleted, replaced by view_mut_of(&ViewTarget), so a future geometry effect has to say which view it writes. The host's own key path passes ViewTarget::Active, which is right because the TUI is the host and has no session to ask.

+

Fault injected: model_effects' SetAxis arm rebuilt with ViewTarget::Active (the pre-fix behaviour, with view bound and discarded so it still compiles). The test fails on the asker's own view never moved — Second's Channel stayed on Row while Default's moved to Page.

+

The fixture must seat the asker somewhere the host is not, and the two views must disagree about the category being moved: Channel is Axis::None in Default and Axis::Row in Second, and the command asks for Axis::Page, so both assertions separate the fixed code from the broken code in both directions.

+

#a_reframe_of_a_view_the_host_is_not_on_invalidates_its_clients

+

improvise-lltt's second half, and the part the ticket did not predict. Naming the view made a reframe of a view the host is not on reachable for the first time, and Sessions::adopt_cursor_epoch could not see one: it returned early unless App::cursor_epoch had moved, and the epoch is bumped only by reset_cursor, which apply_effects calls only for the host's own view. So the clients on the reframed view were reset by nobody and told nothing — gotcha 24's strictly-worse-than-nothing state, permanently stale rather than stale until the next scroll.

+

App::last_reframed_elsewhere is the second trigger. Its opening frames are taken lazily inside App::view_mut_of, which is the one door such a write goes through, so the held-key path still hashes exactly one view: the wide-observation cost improvise-mcq4 declined is not paid.

+

Fault injected: the gate restored to if !host_moved { return Vec::new(); }. The test fails on a client on the reframed view was never told.

+

Two negative arms, and they are not decoration. A third client left on Default must keep its offset, or "invalidate everyone" passes; and paging_with_nothing_pruned_invalidates_no_cursor is what forbids relaying against last_reframed_views, which is a union with the reshaped set and so fires for a page selection that pruned nothing. That test failed during development and is why the elsewhere set is precise.

+

#a_command_naming_a_view_the_document_has_not_got_writes_nothing

+

App::resolve_view falls back to the document's default for a name the document has not got, which is right for reading — a slice naming a deleted view must not panic in rebuild_layout — and wrong for writing, where it would move a view nobody asked about. view_mut_of therefore looks the name up directly and returns None.

+

Fault injected: view_mut_of's Named arm routed through self.resolve_view(name).0. The test fails on a name the document has not got fell back onto a view that has one.

+

The window this closes is narrow — Sessions::adopt_deleted_views already moves a mirror off a deleted view and tells it — but it is a race, not an impossibility, and the silent alternative is a write to the wrong document state. Not closed here: the client hears nothing. A Rejection variant for it is filed rather than smuggled in, since Rejection is wire vocabulary.

+

#the_diff_tells_a_client_the_grid_became_a_records_grid

+

The producer half of GridCache::records (improvise-e5pf). grid_of setting the field is exercised by leaving_a_panel_in_records_mode_lands_where_the_host_lands; the diff is a separate arm, and a client that connects before records mode is entered learns only through it.

+

Both preconditions are asserted — the fixture starts on an ordinary grid, and R really reaches a records one — so the comparison is between two answers that had something to disagree about.

+

Fault injected: the records arm deleted from grid_updates. The test fails on the client is never told. The negative arm (an unchanged flag is not resent) is what stops "always push it" from passing.

+

#a_batch_that_leaves_records_settles_out_of_records_normal

+

improvise-4dpj, gap 1. settle_normal_mode matches Normal | RecordsNormal, and replacing that with a one-way AppMode::Normal left the entire workspace suite passing. The existing crossing test, a_batch_that_leaves_records_and_then_sets_normal_lands_in_normal, drives the layout both ways, but both of its branches end in enter-mode normal, so the mode handed to the settle is AppMode::Normal in both directions. It discriminates "settles at all" and "settles against the ending layout", and not "settles from either mode".

+

This drives the production repro: from a records grid, apply_effects([SetAxis { "_Index", Axis::None }]) leaves layout.is_records_mode() false with the mode still RecordsNormal, so the RecordsNormal arm is live and not defensive. Both the records-ness of the grid and the mode before the batch are asserted as preconditions, so a fixture that stopped entering records mode fails loudly rather than proving nothing.

+

Injected fault: if matches!(self.view_state.mode, AppMode::Normal). Fails by name, left: RecordsNormal, right: Normal. It is the only test in the workspace that fails.

+

#switching_views_across_records_ness_settles_against_the_view_switched_to

+

improvise-4dpj, gap 2 — raised by ChatGPT Codex against PR #91 and verified correct on main before being tested. SwitchViewAtCursor emits [SwitchView(target), ChangeMode(Normal)], and this is a different path from the one gap 1's siblings drive: that test toggles records mode explicitly, where a view switch crosses records-ness only because it is auto-detected from the axes (gotcha 1). Nothing in the batch says "records".

+

Driven in both directions, with the crossing asserted before and after so a fixture that stopped crossing fails rather than passes. Injected fault: move settle_normal_mode() back above rebuild_layout() at the end of apply_effects — the exact regression PR #91's round 2 fixed. This test fails by name; so does gap 1's, along with six pre-existing records-mode tests (measured in review; an earlier draft of this entry said five).

+

Scope note, stated because an overclaim is worse than a gap: this drives the effects SwitchViewAtCursor emits, not the view-panel key path that composes them. Reaching the panel needs a view_panel_cursor on a CmdContext built in command/cmd/panel.rs, which is another wave's ground this round. The command's own coverage lives there; what is unexercised is the two joined.

+

#the_layout_geometry_cases_are_every_wire_effect_of_that_kind

+

improvise-26dm. layout_geometry_effects_actually_change_document_config was guarded by assert_eq!(cases.len(), 13), which catches a case being deleted from the list and never one being absent from it — and the two are written identically as an assert_eq over a number. The adversarial review of PR #78 added a fourteenth pub struct declaring LayoutGeometry, pointed at by nothing, and the whole suite stayed green.

+

improvise-5ubs's mechanism cannot reach it: an effect is a struct implementing a trait, so serde has no derived enumeration to diff against. model_effects maps each ModelCommand variant back to the effect it came from (gotcha 38), so the list is diffed against the LayoutGeometry effects reachable through ModelCommand. The two sets are equal today, which is the evidence the substitution lost nothing.

+

The right-hand side is bounded by the derive, and that took a second round. Review of PR #96 injected the deletion of ModelCommand::TogglePruneEmpty from every_model_command(): this was the only test in the workspace that failed, and every_model_command's doc comment claiming the list was compiler-forced was wrong — variant_name's exhaustive match forces an arm there, not an entry in the list. ChatGPT Codex raised the same thing on the PR. the_enumeration_names_every_variant_the_derive_declares closes it, so the chain is derive → enumeration → model_effects → case list with every link checked. The assertion here still names both directions, because a short right-hand side reads as a spurious case on the left and invites deleting it — degrading both sides together.

+

Injected fault: drop TogglePruneEmpty from layout_geometry_cases(). The old count said 12 != 13; this says right: [… "TogglePruneEmpty" …] and names the effect.

+

Residual, stated rather than closed: this is total over the wire and not over the effects. An effect declaring LayoutGeometry that no ModelCommand names is still invisible — which is precisely PR #78's probe. Closing that needs an enumeration of effects the compiler maintains, and none exists without a proc macro or inventory-style registration.

+

#the_enumeration_names_every_variant_the_derive_declares

+

The bound the entry above rests on. Injection is Codex's own, run end to end: a ModelCommand::ProbeGeometry whose effect declares LayoutGeometry, with the model_effects and variant_name arms the compiler forces, omitted from both lists. Before this test, the whole workspace stayed green. After it, the failure is staged and each stage names the probe — first every_model_command() never constructs ["ProbeGeometry"], then, once the enumeration is repaired and only the case list is short, the_layout_geometry_cases_are_every_wire_effect_of_that_kind with ProbeGeometry on the right. That staging is the point: the two tests answer different questions and the injection separates them.

+

The name reader is shared, not copied: reduce.rs's effect_type_name and its own the_effect_name_reader_reads_all_three_debug_shapes are what this leans on, because a second reader here would be a derivation done twice and could drift while one copy was repaired. It reads the leading identifier off the derived Debug, Effect being a trait object that type_name cannot reach.

+
+
+ + diff --git a/site/docs/test-evidence/core.html b/site/docs/test-evidence/core.html new file mode 100644 index 00000000..6d219136 --- /dev/null +++ b/site/docs/test-evidence/core.html @@ -0,0 +1,181 @@ + + + + + +improvise-core — improvise docs + + + +
+ +
+

Source: context/test-evidence/core.md

On this page

Test evidence — improvise-core

+

What broke, what fault was injected, and why each test can fail. Keyed by exact test name (design-principles §7). A renamed test orphans its entry; scripts/check-comment-runs.py --check-evidence catches a heading naming a test that does not exist.

+

Only tests whose narrative is longer than the code appear here. Most tests need none.

+
+

#model/types.rs

+

#a_hidden_dimension_list_reduces_to_one_of_two_states

+

improvise-5jm5. evaluate_aggregated took none_cats: &[String] and consulted it once, for emptiness — so two lists with different contents and the same emptiness gave the same answer, while the name promised a filter. Nothing misbehaved; the signature described something the body never did.

+

This test is the smaller half of the guard and says so. The real guard is the signature: the defect the ticket describes is no longer constructible, exactly as Checked's constructor is the guard for the basis check rather than a test that can fail. What this test does add is that Aggregation::of is the only place the list is read, and that a list of two hidden dimensions is not somehow a third state — the reduction is emptiness, and the enum is not a count.

+

#apply_agg_over_nothing_is_absent_for_every_function

+

An empty cell must read as absent, not as zero — otherwise every blank intersection in a pivot renders 0 and a sparse model looks fully populated. Fails if apply_agg ever gains a Sum => 0.0 identity arm for the empty case. eval_agg deliberately has the opposite rule; see the next entry.

+

#eval_agg_over_no_rows_is_zero_only_where_an_identity_exists

+

A formula aggregate over no rows is 0 for SUM/COUNT, which have an identity, and an error for AVG/MIN/MAX, which do not. This is deliberately not apply_agg's rule, and the two are separate functions precisely so the difference is visible. improvise-kdp named both after they had been duplicated inline in two evaluators; the test pins the contract directly rather than only through formula eval.

+

#find_item_category_falls_back_to_formula_targets

+

add_formula never inserts its target into _Measure (the item is implied by the formula definition). So find_item_category must fall back to the formula list, or Expr::Ref("Profit") fails to resolve and every cell of a chained formula errors ref:Profit (improvise-nrs). The test asserts the setup — that the target really is not a stored item — before asserting the lookup, so it cannot pass by the item having been added after all.

+

#stored_virtual_categories_match_their_names

+

CategoryKind::of_name is used all over the codebase in place of looking the Category up. That substitution is only sound because (a) a fresh Model really does store each reserved name under its virtual kind and (b) add_category hands back the existing virtual category rather than shadowing a reserved name with a Regular one. The second half is the arm that would break silently: a shadowed _Measure would still answer of_name correctly while the stored category had the wrong kind (improvise-2lh).

+

#aggregating_one_text_value_yields_that_text

+

The head of the improvise-l4qx family. aggregate_cell_values used to be filter_map(as_f64), which dropped text. _Index/_Dim sit on Axis::None in every default view, so none_cats is non-empty almost always and the aggregating path was almost always taken — making every text cell in every document render blank. .improv carries pipe-quoted text as a first-class value, so a user could type a text value, save, load, and never see it again.

+

#aggregating_differing_text_values_is_an_explicit_error

+

Two different words have no reduction. The user is told mixed rather than shown one of them arbitrarily, or a blank.

+

#aggregating_text_among_numbers_is_an_explicit_error

+

The case that produced a plausible number before: the text was dropped and the rest summed, so the cell read as authoritative while under-reporting. A blank would have been noticed; a wrong total is not.

+

#a_lone_number_still_counts_as_one_rather_than_passing_through

+

The regression guard for the hot path, and the test that pins the branch order in aggregate_cell_values. The all-numeric branch is checked first, which is the entire difference between this rule and "a single value passes through". Count is the only function that can tell them apart — every other function is the identity on a one-element set — so this test sets measure_agg["Status"] = Count and requires 1, not 7. Reordering the two branches passes every other test in the family.

+

#aggregating_nothing_is_still_absent

+

The text rule must not turn apply_agg's emptiness rule into an error or a zero. Fails if aggregate_cell_values answers on an empty slice at all.

+

#where_filter_absent_category_does_not_apply_formula

+

Profit = 42 WHERE Region = "East" evaluated against a key with no Region coordinate must not apply the formula: the condition cannot be satisfied, so the raw cell value (absent) is the answer. The bug was an if let Some(item_val) that failed to bind when the category was absent and fell through, applying the formula unconditionally — so a grand-total cell showed the filtered formula's value.

+

#sum_inner_expression_constrains_which_cells_are_summed

+

SUM(Revenue) ignored its inner expression and summed every numeric cell matching the partial key. With Revenue=100 and Cost=50 both stored for Region=East, Total = SUM(Revenue) returned 150. The fixture needs two measures on one coordinate or the bug is invisible.

+

#add_formula_same_target_name_different_category_both_coexist

+

add_formula deduplicated by target name alone, ignoring target_category, so Profit in _Measure and Profit in KPI could not coexist — the second silently replaced the first.

+

#add_formula_same_target_name_different_category_evaluates_independently

+

The consequence of the same bug seen through evaluation rather than through formulas.len(): the dropped formula's cell evaluated to absent. Worth keeping beside the count assertion because a fix that deduplicates correctly but dispatches on name alone would pass the first and fail this one.

+

#remove_formula_only_removes_specified_target_category

+

The mirror image: remove_formula matched by target name alone, so removing Profit from _Measure also destroyed the Profit formula in KPI.

+

#near_zero_divisor_consistent_with_equality_epsilon

+

improvise-0bf. =/!= compare with FLOAT_EQ_EPSILON = 1e-10, but the division-by-zero guard checked rv == 0.0 exactly. With X = 1e-11 the comparison X = 0 was true and 100 / X happily returned 1e13 — incoherent semantics for IF(X = 0, a, y / X), whose else branch divides by a value its own condition called zero. Any value the equality operator treats as zero must also trigger div/0. This exercises the direct eval path (eval_expr inside eval_formula_depth).

+

#near_zero_divisor_consistent_in_cached_eval_path

+

The same bug in the fixed-point/cached path (eval_expr_cached inside eval_formula_with_cache), whose division guard also used exact == 0.0. Two tests because there are two evaluators and a rule stated in one and not the other is a live bug — which is exactly how this one shipped.

+

#recompute_formulas_multi_stem_values_stable

+

improvise-6os replaced an O(n²) Vec::contains dedupe of coordinate stems with an IndexSet — a pure performance change. The test pins the observable contract instead of the mechanism: a multi-stem model (chained formulas, several measures per stem) produces the stated values, and a second recompute_formulas call reproduces the exact same cache, so a dedupe that lost or duplicated stems would show up as a changed value or a non-idempotent cache.

+

#formula_chain_preserves_full_precision

+

Display rounds; eval must not. Tax = Price * 0.075 on Price = 10 is exactly 0.75, which format_f64(_, true, 0) displays as "1". The test asserts both: that the display really does round (so the fixture is discriminating) and that Total = Price + Tax is 10.75 and not 11. Without the first assertion the test would pass on a fixture where rounding changes nothing.

+

#model/cell.rs

+

#matching_cells_yields_the_primary_insertion_order

+

improvise-shyt. matching_cells walked the secondary index's HashSet, so its order was hash order rather than entry order. DataStore::cells is an IndexMap precisely so records mode can list rows in entry order. The check is against iter_cells — the primary map's own order — and not against a literal, which would only pin whatever the hasher happened to do on this machine today.

+

#the_same_cells_entered_in_opposite_orders_list_in_opposite_orders

+

The half a single-process test cannot reach by calling twice: two stores holding the same logical cells, built by opposite insertion histories. Each must list its own history, so one result is the reverse of the other. Under the index walk the two orders were unrelated to both histories and to each other, because RandomState keys each fresh HashMap differently. doctrine(an_order_test_needs_two_histories).

+

#matching_values_yields_the_primary_insertion_order

+

The same guarantee for the value-only hot path, which aggregation folds f64 over. f64 addition is not associative, so an unstable order made a SUM reproducible only within one process — drift in the last bits, which is the kind of wrongness nobody reports and nobody can reproduce. Min/Max/Count and the text-unanimity rule are order-free and were never affected.

+

#an_empty_partial_matches_every_cell_from_either_door

+

improvise-hkt6. matching_positions indexed sets[0] and panicked on an empty partial. Both public callers guard, so it was unreachable — but a guard one level away from the indexing is how this class of bug returns, so the input the guards prevent is fed in directly here.

+

It asserts the answer, not merely the absence of a panic: an empty partial constrains nothing, so every cell matches, and that must be the same answer the callers' fast paths give. Returning vec![] would also have been total and would have made deleting a fast path silently flip "all" to "none".

+

#live_records_come_back_in_the_stores_own_insertion_order

+

improvise-2yc2. live_records walked the records: HashMap, so it yielded a different order in every process and in every instance — the improvise-shyt defect, caught here before a caller could inherit it. The order decided on is the store's own: a record sits where its first cell sits in the primary IndexMap. The fixture interleaves the measures deliberately — East's second measure arrives after West, and must not move East behind it.

+

#the_same_records_entered_in_opposite_orders_list_in_opposite_orders

+

doctrine(an_order_test_needs_two_histories) applied to records: one history cannot tell "ordered" from "hashed that way today, on this machine".

+

#a_recreated_record_takes_its_new_place_not_its_old_one

+

The claim live_records makes about why it derives the order from cells rather than memoising a position on RecordEntry — and which nothing else could see (Copilot, PR 20). A memoised position passes all three order tests above and still puts a re-created record back at its original place while its cells sit at the end of the store. The entry survives the deletion on purpose (it is a tombstone, below), which is exactly what makes the memoised position available to be wrong.

+

#deleting_and_recreating_a_record_does_not_reset_its_version

+

The tombstone earns its place here. If RecordEntry were dropped when the last cell went, a delete-and-recreate would reset the counter to zero — and a client holding version 0 against the old record would be told its basis is current and its write applied. That is precisely the silent accept ADR 001's basis check exists to prevent.

+

#a_surrogate_for_a_removed_record_fails_to_resolve

+

The first arm of staleness: a surrogate for a record whose cells are all gone must fail to resolve rather than resolve to some other record. The fixture holds two records, so a bug returning "any record" is visible; with one, the assertion would pass by naming the only record there is.

+

#a_second_measure_on_a_record_resolves_to_the_same_record

+

The records-mode unit seen from the index side: two measures on one coordinate tuple are one record, so they share one entry. A cell-level unit would be violated by one legal edit (renaming a category coordinate moves every measure cell in the row).

+

#a_live_records_surrogate_is_the_one_that_resolves_it

+

The pair live_records yields is not two facts that could drift apart: the surrogate is the record's own, so it is the one that resolves it back.

+

#a_deserialized_store_can_resolve_its_records

+

The record index is derived state and is #[serde(skip)]-equivalent — it must come back with the data, rebuilt through set, rather than being written alongside it.

+

#model/record.rs

+

#a_records_surrogate_is_pinned_to_a_literal

+

The surrogate must mean the same thing in the process that minted it and in the one that restarts under it, so it is pinned to a literal rather than to "whatever this build computes". A failure here is not a bug in the test: it means the hash, the key or the byte serialization moved, and every surrogate any client holds is now a name for something else. That is a wire-format break.

+

#the_two_kinds_do_not_share_a_hash

+

A record and an aggregate may be built from the same coordinates — an aggregate over a single record is the degenerate case. A client holding an aggregate id must not be able to spend it as a record id by accident, which is what the domain tag byte is for.

+

#coordinates_that_concatenate_alike_do_not_hash_alike

+

The length prefixes in hash_coords earn their place: without them ("Ab","c") and ("A","bc") serialize to the same bytes, and a collision here is a silent write to the wrong record.

+

#renaming_either_half_of_a_coordinate_changes_the_hash

+

Two categories, not one: a one-coordinate fixture cannot tell "the item moved" from "the category moved" — the same blind spot as the one-category fixture that could not distinguish TransposeAxes from CycleAxis.

+

#a_surrogate_round_trips_through_json_as_a_string

+

serde_json writes a u128 as a bare number and a JavaScript host parses every JSON number as f64, so anything above 2^53 comes back changed — resolving to a different record or to none. The test asserts the wire shape (a quoted 32-hex string, 34 bytes with quotes), not merely that a round trip succeeds, because a numeric encoding round-trips fine inside Rust.

+

#construction_order_does_not_reach_the_hash

+

CellKey is canonically sorted and without preserves that order, so "the normalized record" is a property the type guarantees rather than a convention to maintain.

+

#view/types.rs

+

#a_pivot_view_still_seats_categories_row_then_column_then_page

+

improvise-2zq, the negative arm, and the load-bearing one. The fix makes a records view seat a new category on Axis::None; without this test, "always seat on None" would pass every assertion about records mode while quietly deleting the auto-assignment on_category_added exists for.

+

Probing confirmed the asymmetry: seating every category on None fails 12 tests, while removing the virtual bump failed none at all — which is how the bump came to be wrong in two directions with nothing noticing.

+

#a_records_view_seats_a_new_category_on_none_and_stays_a_records_view

+

on_category_added used to bump a virtual off Row or Column to make room. With _Index on Row and _Dim on Column that is records mode, so naming a category with N while in records mode silently ejected the user from the grid they were standing in and returned them to a pivot.

+

#a_measure_on_column_is_not_bumped_off_it

+

The second thing the bump broke, and the reason the fix is about virtuals in general rather than about records mode: _Measure on Column is an ordinary configuration — it is what web/demo.improv ships — and adding a second regular category kicked it off the axis, making every measure vanish from the grid. The fixture is deliberately not a records layout, so it exercises the pivot path rather than the early return.

+

#cycle_axis_unknown_category_is_noop

+

improvise-t8s: cycle_axis panicked through axis_of's expect when given a category not registered with the view, while set_axis silently no-ops on the same input. It now goes through try_axis_of.

+

#tileable_cats_omits_filter_but_keeps_every_other_axis

+

improvise-dz8: a coordinate the drill fixed must not appear in tile select — the user cannot move it, so offering it offers an action that cannot be taken. The test flips the category back to Page afterwards, so it distinguishes "omits Filter" from "omits this particular category".

+

#filtering_cats_covers_page_and_filter_but_not_the_rest

+

Filter constrains the data exactly as Page does; only Page is navigable. The fixture puts a category on each of Page, Filter, Row and None so the result separates the two questions rather than merely counting.

+

#view/layout.rs

+

#prune_axis_items_indexes_data_positions_not_visual_ones

+

prune_axis_items asks keep about data indices — the positions a row/col argument means — so a group header between two data items must not shift the numbering the predicate sees. The fixture places the header at visual index 1 and prunes data index 1: if the predicate saw visual indices, "A" would be the one dropped. improvise-n3wt also removed the &[bool] + debug_assert pair, whose length could disagree with the axis in release builds only.

+

#a_text_cell_renders_in_a_default_view

+

The user-visible face of improvise-l4qx, asserted at the layout rather than at the aggregation helper. The test asserts its own premise first — !layout.none_cats.is_empty() — because if a default view ever stopped putting _Index/_Dim on Axis::None the test would pass without exercising the bug at all.

+

#records_mode_gives_each_measure_its_own_value_column

+

Pins the whole column vocabulary in order: [Region, +cat, Revenue, Cost, +] — the regular categories, the new-category column (improvise-2zq, sitting where the column it creates will appear), one column per raw measure, then the new-measure column (improvise-c5v). It checks both the labels and the RecordCol variants, so a column drawn with the right label but resolved to the wrong kind fails.

+

#records_mode_initial_layout_is_sorted

+

On entry into records mode the store is sorted by CellKey, so rows appear in a deterministic order regardless of insertion order. The fixture inserts East after North, the opposite of alphabetical, so "did nothing" fails.

+

#records_mode_new_record_appends_at_bottom

+

The complement: once the layout exists, a new record appends rather than re-sorting into the middle. The new record's coordinate (AAA) sorts before every existing one, so a re-sort would move it to the top and move the row the user was looking at.

+

#records_display_out_of_bounds_col_returns_none

+

improvise-byz: records_display for an out-of-bounds column called col_label(col), which returns ""; "" was then looked up as a category name in the record, never found, and the function returned Some("") — masking view/data mismatches as blank cells. It must return None, symmetric with cell_key.

+

#view/axis.rs

+

#the_label_vocabularies_are_unchanged

+

improvise-rml, characterization: these strings were scattered across four render/command sites before being pulled onto Axis. Pinned verbatim so the move could not quietly restyle the UI.

+

#filter_is_outside_the_rotation

+

improvise-dz8: Filter names a coordinate the drill fixed, so it is outside the Row→Column→Page→None rotation entirely. Both directions are asserted — cycling a Filter must not hand the user a navigable axis, and no navigable axis may rotate into Filter, or tile select could hand one out.

+

#next_is_a_four_cycle_over_the_navigable_axes

+

Applying next four times is the identity from every starting point and no navigable axis is a fixed point, so no variant is stranded or skipped. Together with the entry above this pins the rotation without enumerating it twice.

+

#every_axis_gets_its_own_color

+

Distinct axes must be distinguishable at a glance; color() is the only reason AxisColor exists, so a collision defeats it.

+

#a_routing_formula_over_a_group_sums_the_members_answers

+

improvise-4zl7, and the hand-computed half of its evidence. Every number in route_then_aggregate_tests is written out rather than obtained from a second call into the evaluator, because the bug being guarded is the evaluator answering consistently and wrongly.

+

The fixture is four records — Cost of 10 and 100 in North, 20 and 200 in South — and the formula is Pred = SUM(IF(Cost < 50, 1, 10)). Cost straddles the threshold inside each region, which is the whole design of the fixture:

+
cellroute-then-aggregateaggregate-then-route
one record (Cost 10)11
North (Tag away)1 + 10 = 11110 < 50 → 10
everything1+10+1+10 = 22330 < 50 → 10
+

A fixture whose records fall on the same side of the split would give the same answer either way at the record level and differ only by a factor of the record count, which is the kind of agreement that reads as correctness.

+

Two faults were injected and both were watched to fail. Restoring the old eval_agg — sum the matching raw values and ignore the inner expression — fails 7 of the 8 tests with 330 for 22, 110 for 11 and 10 for 1; the one survivor is is_is_one_when_the_cells_own_coordinate_matches_and_zero_otherwise, which has no SUM and so never reaches the fold. Making eval_is return 0.0 unconditionally fails exactly the 3 coordinate tests, with 400 for 214 and 200 for 14. The two faults failing disjoint-but-overlapping sets is itself the evidence that the tests separate the fold from the predicate.

+

#a_coordinate_split_survives_aggregating_over_that_very_category

+

The case the ruling exists for, and the one the ticket's own text called the product question. With Region aggregated away the cell has no Region coordinate of its own, so a formula that read the cell's coordinate would see nothing and take Expr::Is's unwrap_or(false) branch — answering 100 once. The fold hands the expression a record key, so each of the four records answers for its own coordinate: 7 + 7 + 100 + 100 = 214.

+

#a_member_whose_input_is_missing_errors_rather_than_being_dropped

+

The asymmetry between the Expr::Ref fast path and the fold, asserted rather than left to be discovered. SUM(Revenue) skips a record with no Revenue cell; SUM(<expr>) cannot, because the expression is what is being asked and an expression that will not evaluate is an error. The fixture clears one record's Cost and writes a different measure on the same record, so the record still exists and still has a member cell to be found by — otherwise the record would simply vanish from matching_cells and the test would prove nothing.

+

#feature_roles_line_up_with_the_names_preprocess_emitted

+

Lives in gp-tree, recorded here because the rule it guards is improvise-4zl7's. feature_roles is the one statement of preprocess's expansion order and encode_row drives off it, so a drift is silent in both directions: the emitter would spell the wrong feature and a re-encoded row would land in the wrong column. The assertion rebuilds each feature's name from its role and compares it with the name preprocess emitted, which is the only thing that ties the two walks together.

+

The last four assertions exist because the loop is vacuous over a fixture that produces only one kind of feature. The fixture has a numeric column with a missing cell, a categorical column and a date column, so all four roles appear. Fault injected: swapping the Numeric and Missing pushes in feature_roles fails this test and encode_row_replays_the_training_expansion, which is the pair that would have disagreed.

+

#a_formula_targeting_another_category_still_folds_over_its_records

+

Found by self-review, not by the ticket. The fold evaluates the inner expression at each member record's key, and the first version built that key by folding the record's coordinates onto context — which still carries the formula's own target coordinate. For the ordinary _Measure target that is harmless, because Expr::Ref overwrites _Measure on the way in. For a formula targeting any other category it is not: the record key keeps Levy=Tax, so evaluate_depth matches the formula again at that key and the whole cell converges to Error("circular") — a formula quietly self-referencing through its own aggregate.

+

The key is now folded onto partial (context.without(target_category), plus any inline WHERE), which drops the target coordinate by construction rather than by every caller remembering to. Fault injected by restoring the context fold: this test fails with Error("circular") against 11.0 and the other eight pass, which is why it is a separate test and not an extra assertion on one of them.

+

#the_two_evaluators_agree_when_the_fold_reads_a_derived_measure

+

Adversarial review of PR #54, and the sharpest finding of the round: the first version of the fold reintroduced exactly the divergence the ruling exists to remove, through the cached path.

+

eval_expr_cached's Expr::Ref looks the value up in formula_cache at context.with(cat, name). Inside the fold, context is a member-record key carrying the none_cats coordinates, while formula_cache is keyed by stems with none_cats stripped. The lookup could never hit, fell through to aggregate_raw, found no raw cell behind a formula target, and answered Err("ref:…"). So a routing formula reading a derived measure gave Number(11.0) uncached and Error("ref:Margin") cached — and the cached path is the one a pivot grid runs whenever any category sits on Axis::None.

+

The fixture is the four-record one plus Margin = Cost * 2, so the fold's input is a formula rather than raw data. The test asserts the uncached value and that the two agree; asserting only agreement would pass if both broke.

+

The fix makes them agree by construction rather than by this test: eval_expr is hoisted to module level and both evaluators hand it to eval_agg as eval_at. Note the repair that was not taken — teaching the fold to hit the cache by stripping none_cats off the member key returns the group's value to every member, which is aggregate-then-route through a cache lookup, silent. Fault injected by restoring eval_expr_cached as the fold's evaluator: this test fails with Error("ref:Margin") and the other eleven pass.

+

#the_cached_evaluator_has_its_own_working_coordinate_predicate

+

The same review's second finding, and a lesson about where to inject a fault. Expr::Is has an arm in each evaluator. The original evidence injected the fault into the shared eval_is, which both arms call — proving the tests see the predicate, not that they see the cached copy. Replacing only the cached arm with Ok(0.0) left the suite at 1330 passing.

+

The first attempt to close it — a coordinate split run through recompute_formulas + evaluate_aggregated — also stayed green, and that is the interesting part: once the fold routes through eval_expr, an Is inside a SUM never reaches the cached arm at all. The cached arm is now reachable only by a bare Is outside an aggregate, which is what this test uses. Fault injected into that arm alone: it fails, Number(0.0) against Number(1.0).

+

The general shape: an injected fault proves coverage only of the code path it is injected into. Two arms of the same rule need two injections, and if one of them turns out to be unreachable, that is a fact about the design worth writing down.

+

#a_user_category_whose_name_leads_with_an_underscore_is_seated_as_regular

+

improvise-bidq. View::on_category_added's early return read cat_name.starts_with('_') || self.is_records_layout() — the same expression mixing two derivations one operator apart, the left a name prefix and the right CategoryKind. Four lines up, is_records_layout asks the enum.

+

Model::add_category does not validate names, so a user category literally named _Foo was CategoryKind::Regular to records_columns, cell_id and is_empty_model, and "virtual" to these two lines — seated on Axis::None, so it never appeared in a pivot while every other consumer treated it as ordinary. The second site (holds_a_regular) had the same split, which is why the test asserts a second category's seat too: fixing only the first leaves _Foo on Row while Region also takes Row.

+

Decided explicitly, per the ticket: add_category is not made to refuse a leading underscore. The three reserved names are the ones CategoryKind::of_name knows, _Foo is a legal document, and a user-visible restriction is a larger change than making one answer out of two. The fix is that both sites now ask CategoryKind::of_name(..).is_virtual().

+

Fault injected by restoring either starts_with('_'): the test fails, None against Row on the first site and Row against Column on the second.

+

#an_ungrouped_item_after_a_grouped_run_belongs_to_no_group

+

#a_grouped_run_does_not_leak_into_the_next_prefix

+

improvise-gd83, the two arms of one defect. expand_category emitted a GroupHeader on entering a group and no boundary on leaving one, so row_group_for/col_group_for's nearest-preceding walk answered with a group that had already ended.

+

The arms are separate because the two escapes are different and a fix can close one and miss the other. The first is within one expansion — Jan[Q1], Feb[Q1], Sep — closed by emitting AxisEntry::GroupEnd on a Some -> None transition. The second is across expansions: cross_product expands the inner category once per prefix, each call starting with last_group = None, so Food/Bob[Family] is immediately followed by Clothing/Carol, which is ungrouped and inherited Family with no transition anywhere to hook onto. That one needs a GroupEnd at the end of an expansion whose last item was grouped.

+

GroupEnd is deliberately NOT emitted on Some(A) -> Some(B): B's own header already ends A's run, and emitting one there moved data_row_to_visual_skips_headers and data_col_to_visual_skips_headers for no gain. Those two are untouched, which is the evidence the boundary is as narrow as it claims.

+

Fault injected by deleting either emission: the corresponding test fails with Some(("Month", "Q1")) / Some(("Recipient", "Family")) against None, and the_two_front_ends_draw_the_same_column_header fails with it.

+

#a_trailing_conversion_letter_does_not_erase_the_precision

+

improvise-711n. parse_number_format read the precision as fmt[i+1..].parse::<u8>().ok().unwrap_or(0) over everything after the last ., so any trailing conversion letter made the whole suffix unparseable and the format fell back to zero decimals, silently. format: ,.2f — the spelling context/repo-map.md's own .improv example carries, and the printf/Python convention a user reaches for first — displayed integers everywhere.

+

Fails before the fix with left: (true, 0), right: (true, 2).

+

The test also pins the reading against its nearest wrong neighbour. Two implementations both fix ,.2f: take the leading digit run, or trim the trailing non-digits off the tail. They differ only on a tail with digits on both sides of a letter, so ".2f3" is in the assertion. Injecting s.trim_end_matches(|c: char| !c.is_ascii_digit()) in place of leading_digits fails the test at left: (false, 0), right: (false, 2) — so the test sees the rule, not merely the bug.

+

#a_zero_padded_picture_string_is_read_as_printf_not_as_excel

+

The companion to the above, and the reason the fix is not "be generous". A forgiving reader would have to answer .00, where printf says precision zero and an Excel picture says two places. This pins the printf reading — "0.00" is (false, 0) — so that "accept the trailing f" cannot quietly grow into "accept anything and guess". It passes before the fix as well as after: it is there to stop the next change, not to catch this one.

+

#a_precision_too_large_to_name_reads_as_none

+

".300" does not fit u8, so it reads as no precision rather than as 300 decimal places. Unchanged by improvise-711n and pinned so the fallback is a stated rule rather than an artefact of the integer type.

+

#a_format_naming_no_precision_at_all_reads_as_zero

+

",f" has no ., and ".f" has a . with no digits after it. Both are zero decimals. Pinned because leading_digits returns "" for the second, and an empty string reaching parse::<u8>() is the one path in the function with no digits to carry the answer.

+
+
+ + diff --git a/site/docs/test-evidence/formula-and-io.html b/site/docs/test-evidence/formula-and-io.html new file mode 100644 index 00000000..bfb4b4d0 --- /dev/null +++ b/site/docs/test-evidence/formula-and-io.html @@ -0,0 +1,227 @@ + + + + + +improvise-formula and improvise-io — improvise docs + + + +
+ +
+

Source: context/test-evidence/formula-and-io.md

On this page

Test evidence — improvise-formula and improvise-io

+

What each of these tests is evidence of: the bug it reproduced, the fault that was injected, and why it can fail. Keyed by exact test name — a rename orphans the entry, and scripts/check-comment-runs.py --check-evidence catches that. The mechanism arguments are in context/code/formula-and-io.md.

+

### is reserved for a name that really is an annotated test. A fixture, strategy or shared harness whose narrative is worth keeping gets a #### subordinate heading that says so. The checker as written accepts any fn, so a helper under ### would pass today and fail the moment that is tightened.

+
+

#improvise-formula/src/parser.rs

+

#parse_sum_with_inline_where_filter

+

Regression: a WHERE inside the aggregate's parens must parse as the aggregate's inline filter, not as a top-level WHERE clause on the formula. The two land in different places on the AST — Expr::Agg(_, _, Some(filter)) against Formula::filter — and the difference is silent at parse time.

+

#hyphen_in_bare_identifier_reference

+

Regression. The hand-rolled tokenizer that preceded the pest grammar did not allow - inside bare identifiers, so a reference to a persistence-legal name like east-coast was silently parsed as east - coast: a subtraction of two unrelated Expr::Refs, which evaluates rather than erroring. The pest grammar mirrors improv.pest's bare_name, which does allow -.

+

#hyphen_in_bare_identifier_inside_aggregate

+

The same fix through the aggregate path, because SUM(east-coast) reaches the identifier rule by a different route.

+

#pipe_quoted_target_is_unquoted_like_a_reference

+

Regression, and a data bug rather than a cosmetic one. The tree walker took the target's raw span, so a pipe-quoted target came back with its delimiters still attached while an Expr::Ref for the very same name came back unquoted. Downstream, Model::add_formula then created a _Measure item literally named |My Target|, which never matched the user's item My Target: the formula wrote into a ghost column and nothing referring to it by name resolved.

+

The test asserts both halves — the stripped target and that the reference spelled identically names the same item — because asserting only the target would pass under a walker that unquoted both wrongly in the same way.

+

Why no existing test caught it: the writer emits Formula::raw, not target, so the file text is identical either way and every round-trip test was blind to it. See context/code/formula-and-io.md § "A formula target is unquoted".

+

#pipe_quoted_target_applies_escapes

+

The escape table applies to the target too, not only to Expr::Ref — the companion that stops the fix above from being "strip the outer pipes".

+

#bare_target_is_unchanged

+

The negative arm: a bare target has nothing to strip. Without it, "always strip the first and last character" would pass the two tests above.

+
+

#improvise-formula/src/quote.rs

+

#unescaping_inverts_quoting

+

The reading half inverts the writing half over every escape both grammars define, plus the \X passthrough neither of them assigns a meaning to. The "42" case is there because pipe_quote is unconditional: a .improv text cell holding 42 must not read back as a number.

+

#a_trailing_lone_backslash_is_kept

+

Before improvise-i2m5 the unescape table existed twice — here (as parser.rs::unquote_pipe) and as improvise-io/src/persistence/quoting.rs::unescape_pipe. The two agreed on every input either grammar's pipe_inner can produce, and differed only on a trailing lone backslash, which pipe_inner cannot emit: io returned it, parser.rs panicked. This pins the surviving answer for that unreachable input, so the surviving function is total.

+

#a_quoted_name_round_trips_through_the_parser

+

The property the whole module exists for: a name quote_name spells parses, and the parser hands back the name that went in — as a target and as a reference, which after the target-unquoting fix are the same rule. The fixture list is the interesting part: keywords (WHERE, SUM), a leading digit (9lives), the escapes, and the empty name.

+

#a_keyword_is_quoted_even_though_it_looks_bare

+

Pins the keyword guard, which is deliberately stricter than the grammar requires and is user-confirmed — do not revert it as over-caution. The argument (PEG rule ordering, and ## Formulas carrying formula text verbatim) is in context/code/formula-and-io.md.

+
+

#improvise-io/src/storage.rs

+

#a_shared_backend_is_a_backend_and_a_clone_is_the_same_one

+

improvise-hmws. A worker must hold the concrete backend — for the four async methods the trait does not carry — and hand the App a Storage. Before the blanket impl Storage for Rc<T>, Rc<OpfsFs> was not a Storage and could not be made one outside this crate (orphan rule), so improvise-worker-server wrote a forwarding newtype, and so would every worker after it.

+

The regression guard is the signature: this does not compile without the impl. What it asserts beyond compiling is the part a newtype could still get wrong — that the clone the App holds and the handle the message loop keeps are one backend and not two.

+

#memory_fs_satisfies_the_storage_contract

+

#physical_fs_satisfies_the_storage_contract

+

Both run contract::storage_contract at a non-empty dir, which is the whole point of dir being a parameter: at Path::new("") a backend that answers list in the wrong path shape passes, because joining is the identity there. OpfsFs did exactly that and the disagreement was invisible until improvise-vhcz (see context/code/formula-and-io.md).

+
+

#improvise-io/src/persistence/mod.rs

+

#The shared harness: load_corrupt_gzip

+

Not a test, and so deliberately not a ### heading — the harness the two gzip-corruption tests below share. It asserts two preconditions before handing the damaged bytes to the loader: that the corruption actually changed the bytes, and that the result still looks like a gzip document by its magic bytes. Without the second, a corruption that also broke the routing would make the loader fail for a different reason and the test would pass while proving nothing.

+

#a_truncated_gzip_document_fails_as_an_unexpected_eof

+

A .gz document cut short — the disk filled, or the process died mid-save. The kind is asserted (io::ErrorKind::UnexpectedEof), not merely that an error occurred, because "an error occurred" would also pass if the loader had stopped decompressing altogether and choked on the compressed bytes as text.

+

#a_gzip_document_with_a_flipped_byte_fails_its_checksum

+

The other way a .gz arrives damaged, and it is a different kind (InvalidInput), which is the reason to test both: one test against one corruption cannot tell "the decoder refuses damaged input" from "the decoder refuses short input". The first is incomplete, the second is wrong — "you lost the tail" versus "you cannot trust any of it".

+

The byte flipped is in the CRC32 field, the first four of the eight-byte gzip trailer, and not in the DEFLATE payload (Copilot, PR #22). A payload byte also yields InvalidInput, but it can do so for malformed compressed data rather than for the mismatch the test is named after, and the name would then be a guess about which. Corrupting the trailer leaves the stream decodable, so the only thing left to fail is the checksum — hence the assertion on the source error's message, since anyhow's to_string shows only the outermost context.

+

#a_valid_gzip_of_invalid_utf8_is_reported_as_not_utf8

+

The third failure, and the one that made the other two worth keeping apart in the code as well as in the tests (Copilot, PR #22). A gzip stream can be perfectly well-formed and still hold bytes that are not UTF-8; read_to_string decodes as it decompresses, so a single Context around it called that document unreadable gzip — a wrong diagnosis of a document that decompressed without complaint.

+

The precondition asserts the payload really is invalid UTF-8; without it the test would also pass on a document that merely failed to decompress, which is the case the other two already own. It also asserts the message does not mention gzip: the gzip stream was fine, and blaming it is the bug.

+

#The fixture: storage_fixture

+

Not a test. A workbook with a category, items, a formula and data — enough that a round-trip exercises every section of the format. Every gzip test above and every storage round-trip below reuses it, so that "the document survived" is a claim about a real document rather than about an empty one.

+

#round_trips_through_memory_storage

+

improvise-6mq: persistence goes through Storage, so a workbook can be saved and loaded with no filesystem at all — no tempfile, no disk. This is the shape the browser backend plugs into.

+

#gzip_round_trips_through_memory_storage

+

The .gz suffix selects gzip regardless of backend: a property of the document name, not of the filesystem.

+

#physical_and_memory_backends_write_identical_bytes

+

Swapping the backend must not change the bytes written. This is what stops a backend from quietly normalising or re-encoding what it is handed.

+
+

#improvise-io/src/persistence/parse.rs

+

#two_sections_naming_one_category_round_trip_to_a_stable_group_order

+

improvise-349m: format_md(parse_md(text)) was not idempotent when a file carried two ## Category: sections with the same name — the two outputs differed only in the order of the > group lines. It looked like a flake because only grammar_prop_tests::generated_file_roundtrips saw it, and only when the generator happened to emit a duplicate header; taking proptest's minimal input and running it deterministically is what turned it into a bug.

+

The fixture is the smallest shape that shows it: a first section defining a group by a > line only, and a second whose item names a different group that also gets a > line. Both groups must have parents, or format_md writes only one > line and the differing internal order is invisible in the text — which is the trap when shrinking this fixture further. Fault injected: reverting ParsedFile::push_category to a plain Vec::push fails it, and the ticket's own minimal input was checked against the fix as well.

+

Stability is only half of what the merge owes, and the first version of this test could see only that half. Adversarial review injected the other fault — Some(_existing) => {}, which drops the duplicate section outright — and the whole improvise-io suite stayed green at 2000 proptest cases while the fixture's - X[G2] and > G2[P2] vanished from the document without a word. Main's behaviour was unstable-but-lossless; a test that compares two printed forms cannot tell that from stable-and-lossy, because both sides of the equality lose the same thing. The three contains assertions run before the equality for exactly that reason, and they fail under the drop.

+

#parse_md_formula_category_suffix_without_space

+

Bug improvise-zgb (b). The formulas walker located the [Category] suffix with raw.rfind(" ["), so a hand-written line with no space before the bracket — - Total = Food + Gas[Type] — was not recognised and the formula was silently retargeted to _Measure. The suffix is now part of the grammar (formula_line) and walked structurally.

+

#parse_md_formula_ending_in_bracketed_pipe_name_keeps_raw

+

Companion to improvise-zgb: a formula ending in a pipe-quoted identifier that contains " [" must not have the quoted name mistaken for a category suffix. It stays a _Measure formula with its raw text intact. Without this arm, "look for the last [" would pass the test above.

+

#unparseable_number_token_yields_error_not_zero

+

Bug improvise-4yc. The Rule::number arm used as_str().parse().unwrap_or(0.0), so any numeric token the grammar admitted but f64::from_str rejected silently became Number(0.0) — a fabricated value in a spreadsheet.

+

No token the current grammar admits actually fails f64 parsing (overflow saturates to inf), so the test exercises parse_number_cell directly, with the forms the grammar could plausibly drift toward admitting. That is the point: the guard exists for the drift, not for today.

+

#malformed_numbers_in_data_never_silently_zero

+

The same bug through the grammar, covering all three routings: --5 fails the number rule entirely and becomes bare_value → Text; 3.14.15 matches the 3.14 prefix, commits, and the rest fails → a whole-file parse error naming the text; 1e999 is admitted and saturates to +inf. None of the three is 0.0.

+

#parse_error_names_line_number_and_contents

+

Bug improvise-6kj: parse failures were wrapped as Parse error: {e}, leaning on pest's raw rendering instead of stating the location in plain words. A user fixing a hand-edited file needs the line number and the offending line.

+

#roundtrip_preserves_an_awkward_category_name

+

improvise-pczv as a property rather than a list. quote_header decides which names the header line cannot carry, and that decision is a small set of clauses — so the property is what earns its place: for any name, save-then-load must give the name back. It runs against populated_model rather than a fresh one because a workbook with no data cannot show a coordinate losing its category, which is the silent half of the bug.

+

#Why populated_model and not a fresh workbook

+

Not a test — the fixture the property above runs against. It already holds categories, items and cells, and the reason is the evidence: a default-constructed workbook makes the round-trip test vacuous for the data-coordinate half, because there is no coordinate to lose its category.

+

#The strategy: awkward_name

+

Not a test — the input strategy the property above draws from. The generated shapes are the whole-line contract's stress points: edge whitespace, embedded line breaks, pipes, and text that looks like another section marker (\n## Data).

+
+

#improvise-io/src/persistence/format.rs

+

#virtual_views_are_not_written

+

improvise-60z. _Records and _Drill are scratch views the app builds on demand. They put _Index/_Dim on axes, and those categories are never written to the file — so persisting the views produces axis lines referring to categories the loader will not find. The test also asserts a real view is written, so "write no views at all" cannot pass it.

+
+

#improvise-io/src/persistence/roundtrip_tests.rs

+

#initial_view_skips_a_virtual_active_view

+

improvise-60z's writing half: saving while in records mode must not leave Initial View: _Records pointing at a view the file no longer contains. It asserts the loaded workbook's active view exists in the loaded workbook, not merely that the string changed.

+

#virtual_views_in_an_existing_file_are_dropped_on_load

+

The reading half: a file written by an older build may already carry the scratch views, and loading one must not reinstate them.

+

#filter_axis_round_trips_with_its_selection

+

improvise-dz8. Axis::Filter arises only inside the _Drill view, which is never written, so this is defensive — but the alternative to supporting it in the format is silently writing a Filter axis as something else.

+

#parse_md_round_trips_formula_category_with_special_chars

+

Bug improvise-zgb (a). rfind(" [") mis-split a suffix whose category name itself contains " [": the writer emitted [Margin [%]] unquoted and the parser then split inside the name, producing garbage raw text and a category called %]. Category names in the suffix are now pipe-quoted on write and parsed via the grammar's name rule.

+

#a_keyword_named_category_is_pipe_quoted

+

improvise-i2m5. .improv now spells names through improvise_formula::quote_name, which pipe-quotes a formula keyword. Before the switch, io's own copy of the rule had no keyword guard and wrote ## Category: SUM, - WHERE and SUM=WHERE = 1 bare — that was the only output difference the whole persistence suite showed.

+

The names round-trip either way (improv.pest accepts pipe_quoted wherever a name goes), so this pins the spelling, not the survival. It also pins the header exception: category_section takes the whole line and never goes through quote_name on either side, so only references are quoted.

+

#a_keyword_named_category_round_trips_everywhere_it_appears

+

The round trip the quoting exists for, at every position a keyword-named category or item reaches the file: the axis line, the page selection, hidden:, the category section and the data coordinates. A guard that covered only some of them would pass a single-position test.

+

#a_formula_targeting_a_keyword_named_measure_round_trips

+

Why the keyword guard is worth its cosmetic cost even though bare keywords parse: a ## Formulas line carries formula text verbatim, so a name written by format_md is later read by parse_formula. The two grammars share one namespace, and quote_name is the one rule spelling names for both.

+

#a_category_name_the_header_line_cannot_carry_round_trips

+

improvise-pczv. A section header was the one name position that never went through a quoting rule, so a name the header line could not carry was written raw into it. The table NAMES_THE_LINE_CANNOT_CARRY is the whole rule, and each entry is a different failure:

+ +

This is reachable and not theoretical — see a_csv_header_with_an_embedded_newline_survives_a_save.

+

#a_model_and_view_name_the_header_line_cannot_carry_round_trip

+

The same rule at the other three header positions: the model title, the view title, and the Initial View: reference that must still resolve to the view.

+

#an_ordinary_spaced_header_stays_bare

+

The negative arm, and the one that stops quote_header from quietly becoming quote_name. The header carries its name as the whole line — that is the contract — and quoting a name the line carries fine would rewrite the header of every existing document for nothing. It also asserts that a reference to the same name is still quoted, so the two rules stay distinguishable.

+

#pipes_that_do_not_close_the_header_line_are_part_of_the_name

+

The one genuinely ambiguous line, decided by the grammar's &NEWLINE: ## Category: |a| and |b| is a category called |a| and |b|. Without the lookahead the pipe-quoted arm would match a and silently drop the rest of the line.

+

#a_header_written_by_an_older_build_still_loads

+

A header written before improvise-pczv is a bare line and must still load as one — spaces, pipes in the middle and all. The compatibility direction of the same grammar change.

+

#a_csv_header_with_an_embedded_newline_survives_a_save

+

The bug as a user meets it, and the reason improvise-pczv is a corruption bug rather than tidiness. An RFC 4180 CSV may quote a header field containing a newline; nothing on the import path forbids it, so the category carried one — and the .improv file the user then saved could not be reopened. The test asserts the precondition (the import still admits the header verbatim) so it cannot pass by the import having quietly started rejecting it.

+

#an_item_name_the_item_line_cannot_carry_round_trips

+

improvise-odbc. bank-info.improv in the repo root failed to load at - JORDAN MARTINEZ (line 293) with expected pipe_quoted. The ticket named two readings — a stale file, or a writer/reader gap in which improvise emits an item line the grammar rejects — and said to decide by asking whether improvise can write that line today.

+

It cannot. item_list's name is pipe_quoted | bare_name, and bare_name is [A-Za-z_][A-Za-z0-9_-]* with no space; quote_name mirrors it character for character, so every one of those item names is emitted pipe-quoted and read straight back. The file was hand-edited in c48a5cd — the v2025-04-09 migration quoted the Date items and left the Entity ones bare — and never round-tripped. So the reading is "stale file", and the standing finding was that nothing exercised it. This test is what that finding turned into: the item names from the file that motivated the ticket, driven through the writer and back. It asserts both directions — the spaced name is quoted, the bare-legal name stays bare.

+

#a_bare_item_name_with_a_space_is_rejected

+

The other half of the same finding: the exact line the file carries is not something the reader accepts, so the file really was broken rather than the error message being misleading. Asserting the failure keeps the test above honest — without it, "the writer quotes it" would be consistent with the reader also accepting the unquoted form. The trailing precondition (the same line quoted does parse) stops it passing because of some unrelated syntax error.

+
+

#improvise-io/src/persistence/export.rs

+

#export_csv_quotes_commas_per_rfc4180

+

Bug improvise-1cz. export_csv joined labels and values with bare commas — no RFC 4180 quoting — and ignored the view's number format. Under a comma-grouping format a value of 1200 renders as 1,200 and a label like Smith, Jr. splits into two fields, corrupting every row. The test reads the file back with a real CSV reader and asserts the field count, which is the thing a string contains check cannot see.

+

Strengthened by improvise-711n. The assertion on the value was records[1][1].starts_with("1,200"), and the fixture's format is ,.2f — so the check could not tell 1,200 from 1,200.00 and sat green while parse_number_format threw the .2 away. It is now an exact equality. Reverting the format.rs fix fails it with left: "1,200", right: "1,200.00", which is the cross-crate demonstration that the format bug reached a user's exported file and not only the screen.

+
+

#improvise-io/src/import/analyzer.rs

+

#detect_short_year_format

+

Two-digit years are ambiguous with the four-digit format, so %m/%d/%Y matches first. Expected, not a defect: the wizard shows the detected format and the user can override it. The same argument as the ambiguity below.

+

#an_ambiguous_date_is_read_american_first

+

01/02/2025 and 02/03/2025 parse under %m/%d/%Y and %d/%m/%Y, and nothing in the data can tell them apart. The heuristic is first-match over DATE_FORMATS, so the American reading wins — a European CSV imports with its days and months swapped, silently, and the derived _Month/_Quarter categories inherit the error. improvise-l8rh ruled this a known limit rather than a defect, so this test is what makes changing the answer a decision somebody makes rather than a side effect of reordering the table. The argument is in context/code/formula-and-io.md.

+

The loop at the end is the precondition that makes this a statement about ambiguity rather than about one format merely fitting: the other reading has to fit too, or there is nothing to resolve.

+

#a_day_first_only_value_settles_the_ambiguity

+

Where the two readings genuinely can be told apart, the unambiguous value decides — first match still, but %m/%d/%Y now fails. Without this, "always American" would pass the test above.

+

#the_distinct_values_are_in_first_seen_order

+

improvise-7yx, found while pinning the date heuristic. analyze_records built its distinct values by collecting into a HashSet and iterating it, so the order was different in every process. This is gotcha 18's bug in a second place, and it reaches further than a tidy-up: build_model adds items in exactly this order, IndexMap preserves insertion order, and persistence relies on it — so the same CSV imported twice produced two different .improv files and two different row orders on screen.

+

The answer is first-seen order: deterministic, and the order the records themselves are in, which is the one a user can predict. Five runs is enough to see it, because each HashSet gets its own RandomState; the pre-fix code produced five different orders here.

+

#the_detected_date_format_does_not_depend_on_the_hash_seed

+

The consequence of the same bug that a user would have found hardest to explain. detect_date_format tests only the first ten samples, so when the sample order was a hash order, which ten it saw varied — and with it the detected format, whenever the eleventh-and-later values are the ones that rule a format out.

+

Twelve values are ambiguous and one (25/12/2025) is day-first only. In hash order it landed inside the first ten about half the time, so the column imported as %d/%m/%Y on some runs and %m/%d/%Y on others — every date in it off by the difference. In first-seen order it is last, deliberately: the answer is %m/%d/%Y stably, which is the property asserted. The ten-sample truncation itself is unchanged and is still a real limit.

+
+

#improvise-io/src/import/csv_parser.rs

+

#short_rows_pad_missing_trailing_columns_with_null

+

Bug improvise-k8i. The reader is configured .flexible(true), so rows shorter than the header parsed without error — but the missing trailing columns were simply absent from the record map. No error, no warning, and field-kind analysis saw skewed value counts. Short rows must keep every header column, missing ones as Null.

+

#short_rows_produce_warning_naming_column_and_count

+

Companion to improvise-k8i: the warning must name the affected column and say how many rows were short. A warning that said only "some rows were short" would pass a weaker test and be useless to the user.

+

#full_rows_produce_no_warnings

+

The negative arm: padding must not fire on a complete file.

+

#a_leading_utf8_bom_is_not_part_of_the_first_header

+

A header name becomes a category name and survives into the .improv file, so a byte-order mark riding along invisibly is silent corruption rather than cosmetics. Excel writes one; the csv crate strips a leading one, so the first header is Region and not \u{feff}Region.

+

The control at the end is what makes this an assertion rather than a tautology: a BOM in the second header is not leading, is not stripped, and shows up in the key — so the check demonstrably can see a BOM where there is one.

+

#crlf_line_endings_leave_no_carriage_return_in_the_data

+

The \r belongs to the terminator, not to the last field of the row: a Revenue of 1\r would be a string rather than a number, and every row would import as text.

+

The control: a lone \r is itself a record terminator to this reader, so Ea\rst splits one row into two. That is what a stray carriage return does when it is not part of a CRLF pair, and it is why "two records, clean values" is a real observation rather than a coincidence.

+

#an_empty_file_is_no_records_and_no_warnings

+

An empty file is no records and no complaint. The wizard's own preconditions (wizard.rs::validation_error) are what refuse to build a model from nothing; the parser's job is to say truthfully that there was nothing there.

+

#a_header_with_no_rows_is_no_records

+

The same answer, and the case a user actually hits — an export filtered down to nothing.

+

#a_single_column_file_yields_one_field_per_record

+

The degenerate shape for everything downstream that splits a row into coordinates and a measure: one field, present, and not mistaken for a short row.

+
+

#improvise-io/src/import/wizard.rs

+

#wizard_previous_reverses_each_advance

+

Bug improvise-907 (from improvise-o1c): no step bound a back key, so reaching a later step and wanting to change a field kind meant Esc — discarding every proposal toggle, formula and name. previous walks back through the path actually taken, including the steps a conditional transition skipped on the way forward, which is why it is a history stack and not a reverse table.

+

#wizard_build_parks_the_workbook_on_done

+

Bug improvise-907 (from improvise-c6j): advance mapped NameModel → Done but nothing ever called it there, so Done was reachable only from tests. build now parks the workbook on Done for confirmation.

+

#wizard_build_failure_stays_on_name_model

+

A build that fails despite the earlier gates leaves the user on NameModel with the error, rather than moving to a Done step with nothing to confirm.

+

#wizard_blocks_advance_when_over_category_limit

+

Bug improvise-mzv: with more than MAX_CATEGORIES (12) accepted Category proposals, the wizard let the user advance through every remaining step and build_model() only failed at the final confirm. Advancing past ReviewProposals must be blocked, with a message telling the user to mark fields as Measure or toggle them off — the actions still available on that step.

+

#wizard_blocks_advance_when_no_category_accepted

+

Bug improvise-907 (from improvise-gfq): build_model raised "At least one category must be accepted" only on Enter at NameModel, the final step. An import whose fields all classify as Measure walked the user through DefineFormulas and NameModel before refusing. The gate belongs at ReviewProposals, where c can still change a field's kind — the same treatment improvise-mzv gave the too-many-categories case.

+

#build_model_null_category_value_is_missing_not_the_item_null

+

improvise-ych. A category value of JSON null used to be stringified by the v.to_string() fallback, inventing an item literally named "null" and placing the record under it. A CSV empty cell parses to Value::Null too, so every blank column value in a CSV import grew a phantom "null" item. Null means absent: the record must be skipped and counted, exactly as if the key were missing.

+

#build_model_literal_string_null_is_a_real_category_item

+

The other side of improvise-ych: the string "null" is an ordinary value — a region really called "null" — and must survive as its own item. Only Value::Null means absent. Without this arm, "drop anything that stringifies to null" would pass the test above.

+

#build_model_empty_string_category_value_is_missing

+

improvise-ych, third arm. An empty string is the JSON spelling of a blank cell — the CSV parser already turns a blank field into Value::Null, so the two inputs must agree. An item with an empty name is unnameable in the grid, so it is treated as missing rather than created.

+

#build_model_warns_about_records_skipped_for_missing_category

+

improvise-5j9: skipping a record for a missing category value used to be silent, so a 1000-row import could drop rows with no user-visible sign. The warning must name the count, the total, and the offending field(s) — a bare "some records were skipped" is not actionable.

+

#is_does_not_swallow_the_start_of_a_longer_word

+

Adversarial review of PR #54, low severity and fixed anyway. is_expr was identifier ~ ^"IS" ~ filter_value, so Region ISLAND parsed as Is(Region, "LAND") and Region isotope as Is(Region, "otope") — inputs that were parse errors before the production existed, now silently-wrong formulas. Gotcha 34's whole theme is that a silently wrong parse costs more than a refusal.

+

The repair went through two wrong shapes before the right one, and both are worth keeping. A negative lookahead ^"IS" ~ !(ASCII_ALPHANUMERIC | …) in an ordinary rule cannot work: pest auto-inserts WHITESPACE between sequence elements, so it tested the character after the space — rejecting Region IS East while still accepting Region ISLAND, i.e. exactly backwards. Making is_expr compound-atomic ($) stops the auto-insertion and forces the separator to be written out as sp+.

+

At which point the lookahead was redundant and was removed, which only re-injecting the fault revealed: deleting !bare_char left the suite green, because sp+ after IS already refuses ISLAND — IS matches, then sp+ demands whitespace and finds L. A guard nobody has seen fail is indistinguishable from one that always passes (doctrine(check_the_checker)), so it went. sp+ is now the mechanism, and relaxing it to sp* — the obvious "let people write Region IS|East|" change — reopens the hole; that substitution is what this test was finally proved against. The negative arm is paired with a positive one, or "reject everything" would pass.

+

^"WHERE" had the identical hole. It is closed now — see below.

+

#where_does_not_swallow_the_start_of_a_longer_word

+

improvise-r08u, the WHERE half of the same defect. where_clause and inline_where were ordinary rules, so pest auto-inserted WHITESPACE after ^"WHERE" and SUM(Rev WHEREAS x = y) parsed as a WHERE on a category AS: a silently wrong filter where it should have been a syntax error.

+

The test carries two negative shapes because the repair has two independent halves, and this was found by injecting each separately. Making the rules compound-atomic alone already refuses WHEREAS x = y — once implicit whitespace is gone, AS x cannot be one identifier — so a test built only on the ticket's reproducer passes over a rule still spelled sp*, and WHEREAS=y is what that rule still reads as a filter. Faults injected: the original non-atomic rule (fails on the first assertion) and sp+ relaxed to sp* (fails on the second). The positive arms are there so "refuse every WHERE" cannot pass.

+

#where_requires_a_space_after_the_keyword

+

The one spelling sp+ costs, stated rather than discovered later: WHERE|Region Name|=|East Coast| used to parse and now does not. IS has required its space since improvise-4zl7, so this is the two keywords agreeing. The paired positive arm — the same text with the space — is what stops it degrading into "any WHERE with a quoted category fails".

+

#the_grammar_is_parsed_once_however_many_files_the_strategy_yields

+

improvise-8shu. improvise-io's lib binary was the run's outlier — 1.42s here against a 0.01s median, measured at 1.40–1.45s, 1.50s and 3.70–5.24s on three machines. The ticket's diagnosis was that the grammar walk had no size bound, so a few deeply nested documents dominated. That was wrong, and measuring before fixing is the only reason it did not ship as the fix: instrumenting the two halves separately gave load_grammar() ×1000 = 2.50s and the walk itself ×1000 = 15.7ms. The walk was 0.6% of the cost. improv_file() called load_grammar() inside its prop_map, so pest_meta re-parsed improv.pest once for every generated case — 1000 times across the two properties, plus shrink attempts.

+

The strategy's input space was never the problem and is unchanged: the same entropy range, the same 0..N repetitions, the same properties over groups, formulas, views, hidden items and pipe quoting. The grammar is now read once through a OnceLock.

+

The guard counts parses rather than calls, which is what makes it order-independent: other tests in the binary reach grammar() too, and the count stays 1 however many of them run first. It drives improv_file() itself through a TestRunner rather than re-implementing what the strategy does, so a future edit that reverts the call site is what it catches. Fault injected: Gen::new(grammar(), …) restored to let rules = load_grammar(); — the test fails with left: 64, right: 1.

+

Case counts went 500 → 256 in the same file, which is the default context/design-principles.md §7 prescribes and never below it; that is worth a further 0.39s → 0.21s but it is not the fix, and reporting it as one would be the mistake this entry exists to record. Net: 1.42s → 0.21s, and improvise-io is no longer the run's longest binary.

+

#export_csv_omits_the_rows_the_view_prunes

+

improvise-iot6. render_csv built its layout with GridLayout::new, which does not apply prune_empty — only GridLayout::with_frozen_records does. So :export in the TUI and the CLI export path both wrote rows the grid does not draw. User-visible, in production, and shipping for as long as the two constructors have differed.

+

This is the fourth instance of the same trap; the third was project::grid_in, fixed in PR #78, and this fix is that one line for line. No fault needed injecting: on the unfixed tree the test fails with

+

CSV wrote a row the grid does not draw: left ["Food", "Gas", "Rent"], right ["Food", "Gas"]

+

Why the fixture is discriminating, which is the whole difficulty here. The sibling property written for improvise-t6db passed on the buggy tree in its first version, because it made its two views differ by turning pruning off — plenty of differing config, and vacuous, since with nothing to prune both code paths agree. So this fixture does two things that one cannot fake:

+ +

None of export.rs's three pre-existing tests mentions pruning at all, which is exactly why this shipped.

+

Both axes, because prune_empty prunes both. A first version of this fixture had a single Month, so it demonstrated the row half and left the column half asserted by nothing -- the same defect one axis over. Feb is now the empty column, the precondition is a pair on each layout ((2, 1) drawn against (3, 2) unpruned), and the CSV header is asserted to carry Jan alone.

+
+
+ + diff --git a/site/docs/test-evidence/gp.html b/site/docs/test-evidence/gp.html new file mode 100644 index 00000000..5890b0c9 --- /dev/null +++ b/site/docs/test-evidence/gp.html @@ -0,0 +1,232 @@ + + + + + +gp-tree and improvise-gp — improvise docs + + + +
+ +
+

Source: context/test-evidence/gp.md

On this page

Test evidence: gp-tree and improvise-gp

+

What each test's bug was, what fault was injected to prove the test can fail, and why the obvious weaker version of it proved nothing. Keyed by exact test name — a rename orphans the entry, which scripts/check-comment-runs.py --check-evidence catches.

+

The architectural arguments these tests defend are in context/code/gp.md.

+
+

#gp-tree/src/lib.rs

+

#splitmix64_matches_the_reference_vectors

+

The vendored RNG's constants had nothing pinning them. Changing the gamma from 0x9E3779B97F4A7C15 to …7C17 left every other test in the workspace green, because they only ever assert that two runs at one seed agree.

+

The crate's promise is stronger: a fit's seed is printed to the user and saved in data::Model, so a later run at an old seed must reproduce the old fit. These are therefore the published SplitMix64 vectors for seed 0, not a golden capture of this code — which is what makes them evidence rather than a tautology.

+

#splitmix64_matches_the_reference_vectors_from_a_nonzero_seed

+

Seed 0 alone cannot catch a change to how the seed enters the state (state = seed versus state = 0), because both give the same stream at 0. Also asserts seed() still answers the seed after the generator has advanced.

+
+

#gp-tree/src/fit.rs

+

#mse_averages_over_exactly_the_rows_it_was_given

+

mse used to take parallel x/y slices, sum over zip(x, y) — which stops at the shorter — and divide by x.len(). A caller that got the two out of step was handed a mean that was quietly too small rather than an error: the improvise-n3wt hazard, a count supplied alongside the data rather than taken from it.

+

The remedy was not a length check but a type: one slice of Row, so the shorter slice cannot be built (improvise-dkqn). This test is what is left to say about it — numerator and denominator count the same rows, for any prefix.

+

#scoring_no_rows_is_zero_error_rather_than_nan

+

Every mean in the module reached 0.0 / 0.0 on an empty slice and handed back a NaN a caller could not tell from a score: it propagates through FitConfig::cost and sorts unpredictably against real costs. One answer, in mean_over, covers mse, misclass_rate and standardize, and the test asserts all three plus std == 1e-6 (a degenerate spread still never divides by zero).

+

#asking_for_no_folds_yields_one_rather_than_dividing_by_zero

+

k reaches folds from a CLI flag, so zero is a value a caller can supply rather than an impossible state. out[pos % k] divided by it and panicked — a public library function taking its host down with it.

+

#cross_validating_without_a_held_out_fold_pools_nothing

+

The sharper half of the same parameter (improvise-ry89). A cross-validated prediction is one made by a tree that never saw the row it scores, so k < 2 asks for something that cannot exist: the single fold holds every index, the training complement is empty, and the tree is fit and standardized on nothing before predicting every row.

+

This used to announce itself — standardize(&[]) gave a NaN mean and every pooled prediction came back NaN. Once mean_over answered the empty row set with 0.0, the same call handed back near-zero numbers indistinguishable from a model's output, and an R² computed off them is a number rather than a NaN. So fixing one bug hid another, and the count is the contract: no held-out row means no pair. The test also asserts the generator is untouched, because declining before the shuffle is what keeps a caller's next draw the one it would have made.

+

#the_training_split_is_what_the_contains_scan_gave

+

The O(n²) fix (improvise-t3ux) replaced test.contains(i) per row with a bool mask. It is only safe because the mask yields the same indices in the same order — the training rows are what every downstream draw from the generator depends on. Pinned directly, at every fold count, rather than inferred from a fit that happened to agree.

+

#a_fit_improves_on_the_tree_it_started_from

+

Without it, a fit that returned its seed tree unchanged would satisfy both the_same_seed_and_config_fit_the_same_tree and a_different_seed_fits_a_different_tree, because the initial random_tree already differs across seeds.

+

#fit_detailed_starts_from_the_best_of_generation_zero

+

What start has to mean for the question it exists to answer: the best of generation 0, not the seed ancestor it grew from (improvise-ut7p). The two differ whenever a mutated member of the initial population beats member 0, and only the former makes "the search improved on where it started" a claim about breeding. Before this, improvise-gp kept a second copy of the whole GA loop just to reach that starting point.

+

#a_tiny_lone_importance_is_still_the_whole_of_it

+

Copilot on PR #25. fit_csv divided by total.max(1e-9), which is right for the only input it ever had and wrong as a contract: one feature carrying an importance of 1e-10 carries all of what the model knows and was reported at 10%. Importances this small are ordinary — they are error differences over a standardized target. the_shares_sum_to_a_hundred_at_any_scale is the general form across four orders of magnitude.

+

#denormalize_rescales_the_leaves_and_leaves_the_splits_alone

+

The negative arm is the load-bearing one: standardize rescales the target, so a threshold is already in its feature's own units and re-scaling it with the leaves would be the bug. a_tree_fitted_on_standardized_rows_predicts_original_units_after_denormalize pins the arithmetic against a hand-built tree rather than a search, so it does not depend on the GA.

+
+

#gp-tree/src/data.rs

+

#an_entirely_missing_column_is_skipped_and_reported_as_data

+

preprocess used to say so with an eprintln!, which was correct while this code was examples/fit_csv.rs. A library does not own its caller's stderr: a TUI on an alternate screen, a wasm build, or a caller wanting the notice in its own status line could none of them suppress, redirect or translate it. The notice now comes back as data.

+

#seed_zero_is_distinguishable_from_no_seed

+

0 is a legal seed, so "unknown" and "fitted at 0" have to be two different values — anything reporting the seed of a pre-seed model would otherwise state a seed that never ran. That is why Model::seed is an Option and not a sentinel.

+

#a_saved_model_round_trips_with_its_whole_ga_config

+

The seed alone is necessary and not sufficient: the same seed under a different FitConfig searches differently and lands somewhere else, so a model carrying only a seed looks reproducible while being reproducible only by whoever still remembers the flags.

+

The fixture a_fit_config() differs from FitConfig::default() in every field, and the test asserts that explicitly. A round trip that silently substituted the default would otherwise pass against a default-constructed value — the failure mode this test exists for.

+

#a_model_saved_before_configs_still_loads

+

A backward-compatibility test aimed at the absent field, not at the attribute. #[serde(default)] on an Option is documentation rather than mechanism — serde defaults an Option anyway, so a test that merely deletes the attribute fails nothing and is not discriminating. This one removes config from the serialized JSON and asserts it loads as None (unknown), not as Some(FitConfig::default()). a_model_saved_before_seeds_still_loads is the same shape for seed.

+
+

#gp-tree/tests/scoring_known_answers.rs

+

Whole-file note: these are f32 bit patterns, not approximate values, because a change that merely moves an accumulation around silently re-points every seed already saved at a different tree. assert!((a - b).abs() < 1e-6) cannot see that, and neither can the CLI's output, rounded to three or four decimals. The numbers were captured from the parallel-x/y code before improvise-dkqn reshaped the signatures and improvise-t3ux replaced the contains scan, and are unchanged by both. If you are changing the search on purpose these numbers change with it — but then every seed recorded by an older --save names a different tree, and that is the decision this file forces you to make out loud.

+

#standardize_is_bit_for_bit_what_it_was

+

A witness that could not see the bit. The first and last rows plus mean/std were the whole witness, and four numbers cannot separate two ways of computing the same quantity: injecting r.target / std - mean / std for (r.target - mean) / std — algebraically identical, a different rounding — left this test and both cross-validation vectors green (improvise-ry89). All thirty-seven values do separate them: that reassociation moves 21 of the rows, and none of them were rows 0, 36 or 19.

+

Note what is not evidence here. Reversing the mean's accumulation order also left every vector green, but on this fixture the two orders sum to the same f32 — so that injection changed no behaviour to catch. A perturbation has to move a bit before a bit-level witness can be blamed for missing it.

+

#cv_regress_pools_the_same_predictions_and_leaves_the_rng_where_it_was

+

The rng_after witness is the part that pins the order of the generator's calls: the next u64 after a cross-validation depends on exactly how many draws that run made, so a reshaped loop that drew differently would move it even if every pooled prediction happened to agree. 37 rows is a prime, so no fold count divides it evenly and the last fold is short — where an off-by-one in the train/test split shows.

+

#cv_classify_pools_the_same_codes_and_leaves_the_rng_where_it_was

+

The classification half of the same guard. The mask replaced contains in both loops, and one arm passing says nothing about the other (doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only)).

+
+

#improvise-gp/src/lib.rs

+

#improvise_evaluation_matches_gp_tree_eval

+

The acceptance property: over random trees and random feature vectors, improvise's evaluation of the emitted formula matches gp_tree::eval exactly — widening f32 to f64 is lossless and the thresholds are exact, so there is no tolerance to allow.

+

#The improvise_eval helper, and the shape to remember

+

This test and a_value_sitting_on_the_threshold_falls_the_same_side both run through the improvise_eval helper, and its scope is what was once read as covering more than it does. It fabricates a _Measure item per feature name, so it asserts the emitter and gp_tree::eval agree against a model that could not exist. It says Ok to outlook=rainy because it just made an item called that. A helper that builds the world the code wants is not evidence that the world exists.

+

So the awkward names above are a quoting fixture and nothing more: outlook=rainy is there because it is awkward to spell, not because a document could hold it — it could not, and fitted_formula refuses it. The helper is still the right tool for the arithmetic-and-quoting claims; the property it could not state is asserted separately, against a real document, in evaluability_in_a_real_document.

+

#an_awkward_target_name_is_quoted_and_round_trips

+

This test used to assert parsed.target == "|Total Revenue|" and call the pipes correct. They are not: the target names a _Measure item, so a target carrying its delimiters writes the fit into an item literally named |Total Revenue|, which is not the item the user has. What the emitter owes its caller is a semantic round trip — the name goes in, the same name comes back out. an_awkward_target_names_the_users_measure_item is the consequence that makes it a data bug rather than a cosmetic one.

+

#an_infinite_threshold_is_refused_rather_than_emitted_as_inf

+

number_text(f32::INFINITY) emitted "inf", which "inf".parse::<f64>() accepts — so the round-trip loop returned it and a formula_text caller got text the grammar rejects, with nothing on the way saying so. fitted_formula would surface it as a parse error; a bare formula_text caller would write it into a file.

+

#a_nan_threshold_is_refused_by_name

+

NaN fell through the loop to unreachable!, so the panic blamed the decimal expansion instead of the caller's value.

+

#a_threshold_is_the_shortest_decimal_that_reads_back_the_same

+

The doc used to overclaim the emitted text as the f32's full decimal expansion. It is the shortest decimal that reads back as the same f64 — the weaker claim, and the one the routing argument actually needs: equal text means equal f64 means the same branch.

+

#a_value_sitting_on_the_threshold_falls_the_same_side

+

The boundary case the whole number_text design exists for. 0.7f32 is really 0.699999988…; the shortest text that reads back as that f32 is "0.7", which parses as an f64 strictly greater than it. A row whose feature value equals the threshold routes hi under eval and would route lo through a formula carrying the rounded threshold. The test asserts the trap is live — "0.7".parse::<f64>() > thresh as f64 — before relying on it.

+

#a_split_on_a_one_hot_feature_is_spelled_as_a_coordinate

+

improvise-8xhy's defect, and improvise-4zl7's answer to it. preprocess one-hots a categorical column into features named Region=North, an encoding artefact no document has an item for, so find_item_category returned None and every cell of the fitted measure came back as Error("ref:Region=North") — silently, with the emitter having said nothing. 8xhy shipped a refusal; 4zl7 replaced the refusal with a spelling, Region IS North.

+

The assertion is the emitted text, exactly, and not merely that the call is Ok: the whole failure mode here was a well-formed formula meaning the wrong thing, so "it parsed" is the assertion that missed it the first time. The fault to inject is Spelling::Coordinate rendering as quote_name(name) — byte-identical for every measure, and wrong for exactly this family.

+

#a_coordinate_naming_no_category_of_this_document_is_refused

+

The negative arm of the same rule, and it is not redundant with the artefact refusal: a Coordinate is spellable and may still be unresolvable. Without it, resolves_in's Coordinate arm could return true unconditionally and every test above would still pass.

+

#a_measure_that_merely_looks_like_a_one_hot_is_accepted

+

The negative arm, and why the check asks the document rather than refusing names that look like encoding artefacts: a measure literally named Region=North is a legal document. The two are indistinguishable as strings. the_missing_flag_and_the_date_sub_features_are_refused_too covers the other two artefact families with nothing enumerating them — the check asks the exact precondition Expr::Ref has, so a future naming scheme is covered without this crate learning about it.

+

#a_formula_the_emitter_accepted_evaluates_everywhere

+

The property the lying test could not state: if fitted_formula says Ok, no cell of the target is a ref: error. It is also what stops the refusal tests above from passing by refusing everything.

+

#a_tree_indexing_past_the_spec_says_so

+

improvise-hkt6. A tree whose feat runs past the spec violates the same precondition in two places, and the two answered differently: expr_text said what was wrong, mark_features said index out of bounds. Whichever ran first decided what the caller was told, and fitted_formula runs mark_features first — so the legible message was the one nobody ever saw. Driven through fitted_formula because both private functions are reached that way, and the message is asserted rather than merely the panic, since an out-of-bounds index panics too.

+

#a_record_added_after_the_fit_is_scored_by_the_formula_alone

+

The load-bearing half of improvise-7na's answer: scoring a new row needs no saved encoding plan. fit_csv --predict calls encode_row because its new row arrives as untyped CSV text; improvise's new row is a record in the same document, scored through the stored formula from its own measures. So nothing about the --predict path argues for putting a gp_tree::data::Model in the file.

+

#breeding_improves_on_generation_zero

+

What rules out a GA whose breed returned its input unchanged. a_different_seed_fits_a_different_formula is weak evidence on its own, since the initial random_tree already differs across seeds.

+
+

#improvise-gp/src/dataset.rs

+

#a_measure_on_the_page_axis_does_not_filter_away_the_training_rows

+

improvise-vx3. Workbook::new's default view puts _Measure on Axis::Page, so filtering_cats() named it, filter_coords pinned it to the first measure, and every "record" then carried exactly one measure cell — never the target. The whole table came back as "no visible record has a Revenue value to learn from" on a document plainly full of them, which reads as a data problem rather than as an axis one.

+

Every test in the module used View::new + on_category_added, under which a leading-underscore category lands on Axis::None — so the fixture was quietly building the one world the code worked in, and no test could see the default a real Workbook hands out.

+

The assertion compares the contents, not the shape. Pinning only the header and the row count passes on this fixture even with the bug present: _Measure's first item is Revenue, so every row still has a target and only the feature columns come back blank — the shape survives while the data does not.

+

#a_paged_measure_that_is_not_the_target_does_not_empty_the_table

+

The sharper half: filter_coords pinned to the first item, so on a document whose target is not first the table came back empty.

+

#the_feature_table_is_one_row_per_record_with_the_target_last

+

The literal row order is half of what this pins, and f32 accumulation over the rows is why: a reordered table makes a seeded fit irreproducible in a way nothing prints. Since improvise-shyt matching_cells yields insertion order, so the assertion now holds by construction; before it, a test in one process could not tell "sorted" from "this process happened to hash them in that order".

+

#a_formula_measure_is_not_a_feature

+

The fixture lists Profit as an explicit _Measure item as well as making it a formula target, which is legal and is the only case this test can see: a target that is only implied by its formula never reaches the explicit item list, so a fixture without the explicit item passes whether the rule is implemented or not.

+
+

#improvise-gp/src/fit.rs

+

Whole-module fixture note — document(). Every category is a Label, and that is the only thing that makes the success path deterministic. An earlier version left Region regular with a single item and claimed the constant one-hot had a degenerate (1, 1) range that simplify would prune. That claim is false: data.rs::build_categorical pushes (0.0, 1.0) for every one-hot whatever the column contains, so simplify cannot know it is constant and the champion was free to split on Region=North. These tests were passing on seed luck, and the comment asserting otherwise was the worst part (Copilot on PR #25). The targets are in the thousands so a fit that forgets to de-normalize is visibly wrong rather than subtly so. The view puts _Measure on Axis::Page because that is what Workbook::new does — fitting under the real default is not an extra case, it is the only honest one.

+

#the_fitted_formula_predicts_in_the_documents_own_units

+

The de-normalization regression. fit runs against a standardized target, so a champion written down untouched predicts standard deviations: a well-formed formula, parsing cleanly, evaluating to ±5 where the user's data is 1000–4000. Nothing downstream can see that — it is a number in a cell.

+

Fault injected: dropping the denormalize call in fit_measure fails this and only this test. The loop panic!s rather than continues on a non-numeric cell, because a loop that skips every record asserts nothing and "no cell evaluated" is itself the improvise-8xhy failure. The claim is then read a second way off the formula text, which does not depend on evaluation at all.

+

#a_category_and_a_measure_of_the_same_name_are_refused

+

Copilot on PR #25, and the sharpest finding of the round: asking the document whether it can resolve a feature name is not the same question as asking whether that name means the column the tree was fitted on. A numeric category called Cost encodes to a feature called Cost; with a _Measure item of the same name the emitted Expr::Ref("Cost") resolves cleanly — to the measure — and the stored rule silently reads a different column from the one it learned.

+

The first version of this test never built the collision. Adding a category Cost with items puts no Cost coordinate on the existing records, so feature_table emits an empty column, preprocess drops it, and there was no duplicate name to find; it passed on the positional role bug instead. The records here carry the coordinate, and the test asserts the fixture really produces two columns named Cost before asserting the refusal.

+

#an_empty_category_before_a_measure_does_not_break_the_role_check

+

Copilot on PR #25. preprocess continues past an entirely-missing column without pushing a ColumnSpec, so Dataset::columns is not index-aligned with the header once anything is dropped. A positional i >= category_columns role test then read the surviving numeric measure as a category and refused a perfectly good fit. The test asserts the fixture really drops a column first.

+

#a_coordinate_is_spelled_as_one_even_when_an_item_shares_the_one_hots_name

+

The collision a name cannot resolve, now closed by construction rather than by a refusal. The fixture adds a _Measure item literally named Region=North and asserts find_item_category resolves it — so a check that matched the one-hot's feature name against the document would have found the wrong column.

+

Under improvise-4zl7 the one-hot is never spelled by its feature name at all: Features::of reads the role off preprocess's plan and emits Region IS North, which cannot collide with any measure however the document is named. So the assertion is the absence of |Region=North| in the emitted text as much as the presence of Region IS. Injecting the fault means having spell fall through to Spelling::Measure for a OneHot role: the fit then succeeds, the formula reads |Region=North|, and it silently evaluates against the wrong column — which is the original bug wearing the new API.

+

#a_high_cardinality_numeric_measure_is_one_feature_not_many

+

The preflight approximates preprocess's type inference, so it can disagree in both directions. A numeric measure with more distinct values than MAX_FEATURES encodes to one feature and must not be refused for having many values.

+

#a_category_with_a_distinct_value_per_record_is_refused_not_searched

+

FitConfig bounds the search; nothing bounded the width. A regular category with one distinct value per record one-hots to one column per record, and permutation importance is repeats × rows × features evaluations on the draw thread.

+

The fixture had to be pushed past MAX_FEATURES, and the assertion names the width refusal specifically. At twelve records this document is refused too — but by fitted_formula noticing the champion split on Tag=r0, a different rule, which would have made the test pass with the guard removed entirely.

+ +

The width bound does not bound the work: every stage scales with rows. The assertion also requires the message to say what to do (Page or Filter).

+

#a_document_with_no_usable_feature_is_refused_rather_than_panicking

+

preprocess drops an entirely-missing column, so a document whose only candidate feature is empty encodes to zero features — and random_tree then draws rng.below(0), which SplitMix64 answers with 0, and sample_thresh indexes spec.ranges[0]. A panic, on a legal document, for any seed that does not happen to open on a leaf.

+

This needs the all-Label fixture rather than merely an empty measure: with a regular category present its one-hot survives — range (0.0, 1.0) whatever it holds — and the fit returns a constant leaf instead of crashing.

+

#a_non_finite_cell_is_refused_rather_than_panicking

+

A .improv cell holding inf parses as an f64, so this is reachable from a document rather than only from a bug. standardize turns an infinite target into NaN leaves, an inf..inf range makes sample_thresh produce a NaN threshold, and number_text then asserts — a panic out of a function whose signature promised a Result.

+

Verified by disabling the guard, and the two directions differ: an infinite target really does panic in number_text ("wants a finite value, got inf"), while an infinite feature poisons its range and quietly yields a constant leaf. A panic and a meaningless rule; both are refused.

+

NaN is deliberately absent from the cases: f64::NAN.to_string() is "NaN", which is_missing already spells as missing, so it is imputed away and never reaches the search.

+

#a_name_containing_a_newline_cannot_split_the_status_line

+

.improv pipe quoting makes a newline legal inside a name, so a document can hold a measure whose name would split the TUI status bar in two.

+

#a_fit_that_needs_a_category_now_spells_it_as_a_coordinate

+

improvise-8xhy reaching a real user, and what improvise-4zl7 bought: the same fixture, the same champion, and a formula instead of a refusal. The fixture clears both measures on every North record: clearing only Cost would leave North carrying both target values, so the one-hot would explain nothing and the champion would have no reason to split on it.

+

The second assertion is the one that earns its place. Checking only that the text says Region IS would pass for a rule that routed both groups to the same leaf — which is what a tree that could not see the coordinate would produce. The fixture's two groups are 1000 and 4000, so requiring one leaf under 2000 and one over 3000 separates "spelled it" from "spelled it and meant it".

+

The \n-and-double-space assertions moved with the refusal they guarded; the one-line rule itself is unchanged and is still asserted by a_name_containing_a_newline_cannot_split_the_status_line.

+

#the_fold_count_moves_the_champion_and_so_is_recorded

+

Copilot on PR #25, and it is right: the outcome used to carry seed plus config, which is the pair gotcha 37b names — and that pair is not sufficient here. Cross-validation draws from the same generator the fit then draws from, so changing cv_folds alone moves the champion. Recording two of four fields documented the reproducibility contract without meeting it. The test asserts the seed and config are equal across the two requests first, so the difference can only be the fold count.

+ +

The knob varied is pop, not generations. More generations over an exactly-learnable fixture can land on the same optimum and simplify normalises it, so that version of the test would fail one day for a reason that is not a regression. pop changes how many RNG draws each generation makes, so the two searches diverge from the first one.

+

#a_fit_named_after_a_category_item_is_refused

+

Copilot on PR #25. The check used to ask only about measures, and Expr::Ref does not: find_item_category finds a regular category's item before falling back into _Measure (gotcha 7). So naming the fit after some category's item installed cleanly and then made every later formula referring to it read the category coordinate. a_later_formula_referring_to_the_fit_reads_the_fit is the property that refusal protects, asserted against a real document.

+

#a_fitted_rule_at_an_aggregated_cell_routes_per_record_and_sums

+

The characterization PR #25 left here, now replaced by the rule it was waiting for. A tree is fitted on individual records; an improvise formula is evaluated at a CellKey, and evaluate_aggregated used to aggregate each Expr::Ref first and evaluate the IF once on the totals. Linear formulas are unaffected — SUM(Revenue) - SUM(Cost) is SUM(Revenue - Cost) — and a tree is not, so the stored rule meant one thing on a record cell and a different thing one row up. improvise-4zl7's ruling was to aggregate the rule over the members.

+

The old test asserted the negative and carried a message telling whoever broke it to replace it with the rule. It fired on the first run of the fold.

+

What this one asserts is the property the ticket names: the aggregated cell equals the sum of the per-record predictions. That compares two different code paths — a fully-specified key evaluates one member, an aggregating key folds twelve — rather than one path against itself, but it is still an end-to-end agreement rather than a hand-computed number. The hand-computed evidence is in improvise-core, under route_then_aggregate_tests, on a four-record fixture whose every expected value is written out in the test name or the assertion.

+

The third assertion is what stops this one being satisfiable by accident: routing once on the total lands on a single leaf, so the test refuses a fixture in which a leaf happens to equal the right answer.

+

#a_missing_flag_a_date_part_and_a_measures_value_are_refused_differently

+

improvise-wd6n. Spelling::Artefact was one variant standing for five distinct families — Missing, DatePart, a one-hot over a measure column, the (missing) pseudo-item, and a Numeric role over a category column — the last five reached through a _ => wildcard in spell. The single refusal they all fed said the tree "splits on a col.missing flag or a date part", so three of the five were declined in words naming something they are not. A user fitting over a text-valued measure column was told about missing-value flags.

+

The test drives Features::of over a real preprocess output rather than fabricating Spellings — which is the point, since what is under test is spell's classification, and the helper artefacts() beside it fabricates exactly what spell decides. That helper now takes the family as an argument so a fabricated spelling at least names one on purpose; Features::of is still what really decides, and this test is the one that asks it.

+

Each family's assertion has two clauses, and the second is what earns it: the error must contain that family's reason and not the other two's. The first clause alone passes against one message naming every family at once, which is precisely the state the ticket describes. Fault injected: the five arms of Artefact::reason collapsed to one shared string — the test fails on the first family, and with the injection removed all three arms pass.

+

The fixture reaches three families from one preprocess call (a text measure column, a measure with a hole in it, a column of ISO dates), which is why the module is named for what it reaches rather than for "each family". The remaining two are not reachable from it and are not asserted here. AbsentCoordinate is a permanent refusal; NumericCategory was not, and the first draft of this entry and of gp.md §11 both called it one — review found that fit_measure simply never tells preprocess which columns are categories. improvise-zv8t has since closed it (below). Worth recording as the failure mode: the branch that invented a classification also wrote the prose declaring one of its arms unfixable, and nothing but a reader could have caught that — the suite was green on either reading.

+

#fit::tests::a_category_whose_items_are_numbers_is_still_spelled_as_a_coordinate

+

#fit::tests::a_category_whose_items_are_dates_is_still_spelled_as_a_coordinate

+

improvise-zv8t. fit_measure handed preprocess an Overrides::default(), so it never said which header columns are document categories — although the very next line told Features::of. preprocess infers Numeric → Date → Categorical from the values, so a Year category with items 2024/2025 became a numeric column and a category of ISO dates became a date column, and both were refused as encoding artefacts with Year IS 2024 spellable all along. User-visible: a fit over such a document could not be stored at all.

+

No fault had to be injected, because the bug was live. On the unfixed tree both tests fail inside fit_measure with the real refusals, which are the two arms the ticket named and are worth quoting:

+ +

The second is why gp.md §11's date decline was narrower than it read.

+

Why each fixture is discriminating. year_document gives the model one regular category and no other feature column, and makes the target a function of that category alone — so a tree that does not split on Year is a poor fit and the GA does not settle for one. That matters because the refusal is conditioned on used_features: a fit whose champion happened to be a bare leaf would be accepted on the unfixed tree and prove nothing. Asserting formula.raw.contains("Year IS") rather than merely is_ok() closes that hole in both directions. Note what the unfixed tree actually does, because a first draft of this line got it wrong: it produces no text at all — fitted_formula refuses before emitting, which is what the messages quoted above are. The only unfixed path that emits anything is the bare-leaf champion named in the previous paragraph, and its text is SUM(<constant>), carrying no Year in any spelling. So the assertion closes the hole; "it would have said Expr::Ref(Year)" was a guess about a branch that does not run.

+

The numeric test then adds the property the refusal exists to protect: the formula goes in through add_formula and every record evaluates to a Number, not Error("ref:…") (improvise-8xhy).

+

#fit::tests::the_preflight_counts_a_numeric_category_at_its_cardinality

+

The other half of improvise-zv8t, and the one that is a cost rather than a fix. Forcing a category categorical widens it from one numeric feature to one per distinct value, so encoded_width — the cheap preflight that exists to refuse a too-wide table before preprocess materialises it (§14) — had to learn the same header[..category_columns] slice, or it would under-count exactly the column this ticket is about.

+

Fault injected: the col >= category_columns clause dropped from encoded_width, leaving the pre-ticket inference. The test fails by name, and the failure message is the evidence — it shows the exact check speaking where the preflight should:

+

the PREFLIGHT must refuse it, before anything is materialised: fitting "Score" encoded 70 features, over the 64 this host will search.

+

A test asserting only expect_err would have passed under that injection, since both guards refuse; the discriminator is which message, because the two guards differ in what they have already paid for. The fixture also asserts its own precondition — that Year really is the header's first column — so a feature_table change that reorders the header fails here rather than silently slicing the wrong names.

+

#fit::tests::a_wide_numeric_category_now_refuses_a_fit_that_used_to_ignore_it

+

The cost of improvise-zv8t, pinned rather than argued — and it exists because the first draft of gp.md §11 claimed there was none. That draft said "no document that fitted before stops fitting", reasoning that a tree splitting on a numeric category was refused as an artefact anyway. The refusal is conditioned on used_features, so a fit that never split on the column succeeded, and the wide columns are exactly the ones a fit has least reason to split on.

+

Measured both ways with a throwaway #[ignore] probe before this test was written, on a fixture of 100 records with a 100-item numeric Year beside a Cost that predicts Score on its own:

+ +

So no fault needed injecting; the change itself is what the test observes, and it fails on either side if the behaviour moves. The second half is what keeps it from being a bare characterization: it marks Year a Label and asserts the fit then lands on Cost, which is the repair the refusal's own message names. A test asserting only the refusal would leave that advice unchecked — and advice in an error message is a promise, so it is the half worth asserting.

+

The generalisable part is the review finding, not the test: prose claiming a change has no cost is a claim, and it is checkable. The suite was green with the false sentence in place, exactly as it was green with improvise-wd6n's "permanent refusal" — the same section, the same failure mode, one paragraph apart. See improvise-e8no for the case the trade genuinely loses.

+

#fit::tests::a_duplicate_column_name_is_refused_before_the_matrix_is_built

+

Found by adversarial review of improvise-zv8t, and introduced by it: as_categories forces by column name, while encoded_width counts by column index. A document may legally name a regular category and a raw measure the same thing -- a_category_and_a_measure_of_the_same_name_are_refused already ships that fixture -- and then the measure column is forced categorical by the name match while the preflight still counts it numeric by index.

+

Measured on the fixture here (40 records, a 40-item Dup category beside a Dup measure): the preflight answers 41 where forcing materialises 80. The refusal that catches the document, no_ambiguous_column_names, ran after preprocess, so the expansion the preflight exists to prevent was allocated and only then refused. At the default max_records = 20_000 that is a 20k-row one-hot matrix built before the error -- doctrine(guard_the_cost_not_the_result) defeated by an inconsistency the ticket itself introduced. The bounded drift that doctrine(a_budget_check_is_advisory_or_exact_and_must_say_which) licenses is approximation, not this.

+

The repair is a data dependency, not a moved line. as_categories now runs the check itself and returns Result<Overrides>, so the Overrides that preprocess requires cannot exist until the header has been accepted. That is doctrine(an_invariant_is_made_unrepresentable_not_validated) -- the answer prune_axis_items, Row<T> and Checked each got.

+

So the regression guard is the signature, and the injection is a compile error. Swapping the two statements back gives error[E0425]: cannot find value 'overrides' in this scope, verified. What that does not cover, stated rather than implied: the check could still be deleted from as_categories deliberately. The accidental reordering -- the way this actually happened -- is what is now unrepresentable.

+

The test therefore asserts the hazard rather than the ordering: that the fixture really names one column twice, that the preflight's 41 really disagrees with what forcing would build, and that as_categories itself refuses. A test asserting only the error message would pass on either ordering, since the message is identical -- the whole defect was in work done, not in the result.

+

#the_preflight_counts_the_missing_category_a_forced_column_materialises

+

improvise-x1ns, found by ChatGPT Codex reviewing PR #90 and live on main at ce895ff. encoded_width computed has_missing and used it in the numeric arm only:

+
true  => 1 + usize::from(has_missing),
+false => present.iter().collect::<HashSet<_>>().len().max(1),
+

build_categorical maps every missing cell to MISSING_CAT and includes it in distinct(&mapped), so a categorical column with N distinct present values and at least one absent cell materialises N+1 one-hots. The preflight promised N.

+

It under-reports by exactly one per such column, and it under-reports in the direction that costs: the preflight runs before preprocess, the exact check on feature_names.len() runs after, so the exact check bounds the error message and not the allocation — which is the whole reason the preflight exists. The previous wave's review recorded this shape as "read, not injected" and "bounded by the exact check"; the second half is false and is now corrected in context/code/gp.md §14.

+

Fails before the fix with

+
the preflight under-reported: it promised 5 features and `preprocess`
+materialised 6 — ["Year=(missing)", "Year=2020", "Year=2021", "Year=2022",
+"Year=2023", "Cost"]
+

The fixture is Codex's suggested case, and the reason it is the interesting one: a forced numeric category with a missing coordinate. Forcing is what sends a numeric column down the categorical branch, so only a forced column reaches the arm that was wrong, and only a missing cell makes the arms disagree.

+

The test does not assert a number. It asks preprocess what it actually built and requires encoded_width >= data.feature_names.len(), so it cannot pass by agreeing with a stale constant, and it keeps holding if the encoding widens for some other reason. Three preconditions are asserted first, because each would make it vacuous: that exactly one record lacks the coordinate, that the present cells really are numeric (otherwise the column would be categorical by inference and forcing would prove nothing), and that a Year=(missing) one-hot is really among the materialised features.

+

The .max(1) is deliberately outside the addition. A column whose cells are entirely missing has no present values, and preprocess skips it with a warning rather than encoding it; keeping the clamp on the sum leaves that case counted as 1 exactly as before, so the fix moves only the case it names.

+

#a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored

+

The evidence improvise-e8no is declined on, and the only coverage Artefact::NumericCategory has ever had. It was one of three families with no test, not the only one: MissingFlag and DatePart are covered by the_missing_flag_and_the_date_sub_features_are_refused_too, and ValueOfAMeasure and AbsentCoordinate still have none — filed, not fixed here.

+

improvise-e8no observes that forcing a document category to Forced::Categorical (improvise-zv8t) widens the encoding, so a high-cardinality numeric category now exceeds MAX_FEATURES where inference used to read it as one numeric column and the fit succeeded. The decline turns on what that success was worth: a tree that splits on a numeric column over a category column produces a formula fitted_formula refuses, because a bare reference to the category's name would resolve to a _Measure item of the same name (improvise-8xhy). So the pre-zv8t success was either a fit that never touched the column — for which marking it a Label gives a fit over the same remaining features, within budget — or one that could not be stored. Note what this does not establish: an ordinal encoding is spellable as a weighted indicator sum, so it is declined on semantics and formula size rather than on spellability (context/code/gp.md).

+

The test asks preprocess with Forced::Numeric, which is the pre-zv8t reading reproduced deliberately rather than described, and then asks Features::of and fitted_formula. It does not use the artefacts() helper: that helper fabricates a Spelling, which is the shape gotcha 37(d) warns about — it would assert that the emitter refuses what the test told it to refuse. Here the document and the encoder decide.

+

Injected fault: change spell()'s FeatureRole::Numeric { .. } arm from Spelling::Artefact(Artefact::NumericCategory) to Spelling::Measure(column.clone()) — the "fix" someone would reach for on reading improvise-e8no, and the one that silently reopens improvise-8xhy. Fails with

+
assertion `left == right` failed: a numeric column over a category column is unspellable
+  left: Measure("Year")
+ right: Artefact(NumericCategory)
+

#an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum

+

Not a bug fix: this pins a grammar fact a prose argument rests on, after that argument got it wrong. improvise-e8no's decline was first written claiming that one-hot is the only encoding of a category whose splits the formula language can spell, and therefore that ordinal encoding was ruled out by spellability. An adversarial review disproved it by running parse_formula instead of reading the rule that looked relevant: is_expr is a primary, so it is an Expr yielding 1/0, and cmp_op's operands are full exprs — so IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, 1, 2) parses.

+

The decline survives on other grounds (a category's items carry insertion order, not magnitude; and the emitted formula still spells one IS per item, so the width is not saved) — but the stated reason was false, and a decline is the one artefact nothing checks. This test is the check: if the grammar ever narrows so that an indicator sum stops parsing, the paragraph that says it does becomes discoverable rather than quietly wrong.

+

The negative arm is the incidental fact that explains the emitter: Year IS 2020 does not parse, because filter_value is string | pipe_quoted | bare_ident and a bare number is none of them. That is why formula_text pipe-quotes a numeric item name through quote_name, and asserting it here keeps the positive arm from passing for the trivial reason that everything parses.

+

#the_preflight_counts_the_parts_a_date_measure_expands_into

+

improvise-x1ns, second instance — found by ChatGPT Codex on PR #97, live at f128087, i.e. in the very change that fixed the first one.

+

preprocess infers Numeric → Date → Categorical (data.rs:406), and encoded_width had no date arm: numeric-or-categorical only. A date string does not parse as f64, so a measure column holding dates took the categorical arm and was counted by its distinct strings. build_date emits epoch_day/year/month/day/dow — seven parts once any cell has a time (data.rs:326) — so a column with one distinct timestamp promised 1 and materialised 5.

+

Fails before the fix, on the plain-date case:

+
a plain date: the preflight under-reported: it promised 4 features and
+`preprocess` materialised 8 — ["Region=East", "Region=North", "Region=South",
+"When.epoch_day", "When.year", "When.month", "When.day", "When.dow"]
+

The fixture carries a time case because of the rival fix, not because of the bug. Two implementations repair the plain-date failure: "a date column is 5 features", and "5, or 7 when any present cell carries a time". They differ only on a column containing a time, so the second sub-case is what separates them. Injected, date_parts hard-coded to 5:

+
a date with a time: the preflight under-reported: it promised 8 features and
+`preprocess` materialised 10 — [… "When.hour", "When.minute"]
+

Note the plain-date sub-case passes under that injection — so without the time fixture this test would have been satisfied by either reading, and would have pinned the bug without pinning the rule.

+

Both arms compare against data.feature_names.len() rather than against a width computed a second way, so the test cannot pass by agreeing with a stale constant or by reimplementing the thing under test. Two preconditions guard vacuity: every When cell really is the same string (so the categorical count really would be 1), and the fixture's timestamp really does or does not carry a time, asked of parse_datetime rather than of the literal's shape.

+

The column must be a measure — as_categories forces every document category to Forced::Categorical, so a category of dates never reaches the date branch at all. That is gotcha 37(d)'s "the date decline is narrower than it read": this is the genuine date measure.

+
+
+ + diff --git a/site/docs/test-evidence/protocol.html b/site/docs/test-evidence/protocol.html new file mode 100644 index 00000000..50f26a22 --- /dev/null +++ b/site/docs/test-evidence/protocol.html @@ -0,0 +1,190 @@ + + + + + +improvise-protocol — improvise docs + + + +
+ +
+

Source: context/test-evidence/protocol.md

On this page

Test evidence — improvise-protocol

+

What each test's bug was, what fault it can see, and why it can fail. Keyed by exact test name (design-principles §7). A renamed test orphans its entry; python3 scripts/check-comment-runs.py --check-evidence catches that.

+

Tests whose name already states the whole property are not listed here. An entry earns its place by carrying something the assertion cannot: the bug, the injected fault, or the reason a negative arm exists.

+
+

#src/render.rs — the fold

+

#cell_updates_outside_the_cache_are_dropped

+

A projection naming a cell outside the current viewport must be dropped, not panic: frames and cell updates race on the wire, and a client that crashes on a stale index is a client that crashes. This is the totality half of the WidgetCache contract, stated as an example; the ordering half is a_cell_update_that_overtakes_its_labels_is_lost.

+

#a_cell_update_that_overtakes_its_labels_is_lost

+

The fold is not commutative, and a client must not be written as though it were. The frames frame the cells, so a Cells update that overtakes the ColFrame/RowFrame naming its positions is dropped. Reordering the identical three updates gives a different cache, and the assert_ne! is the load-bearing line — without it, a fold where order did not matter would pass.

+

#a_compacted_seed_folds_to_the_same_cache_as_the_whole_history

+

Bootstrap is not a second code path, but a server does not replay history to a late client either — it sends the state it holds now. So the property is compaction: folding the seed lands exactly where folding the whole history landed, including the history's relabels and overwrites.

+

The history is built to be worth compacting — overwritten, relabelled, populated and grown, with the extent moving twice so only the last survives — and seed.len() < history.len() is asserted, because a "seed" that is not smaller than the history is not a compaction and the equality would be vacuous.

+

Note the general rule this is an instance of: asserting fold(prefix) + fold(rest) == fold(whole) is vacuous, being the definition of a left fold (doctrine(fold_tests_must_test_order_not_the_axiom)).

+

#a_snapshot_seeds_every_widget_from_its_compacted_state

+

The same property across the envelope, with two subscribers each folding only what is addressed to it — the dispatch a client does for real.

+

#a_totals_update_that_overtakes_its_columns_is_cut_to_fit

+

The totals strip is framed by the columns on the same terms the cells are: a strip that overtakes the ColFrame naming its positions is cut down, because a renderer zips it with the columns and a length mismatch indexes off the end.

+

#an_empty_totals_strip_is_not_padded_into_blanks

+

An empty strip says "this grid has no totals" — records mode, or an empty grid — and must not be padded into a row of blanks by the re-fit. Padding it would draw an empty total row under a records-mode grid that has none, which is the one thing the emptiness is for. This is the asymmetry that separates totals from cells and col_widths, which are always padded.

+

#a_recategorized_axis_blanks_the_totals_too

+

The totals are contents, not frame: a reframing blanks them for the same reason it blanks the cells. A stale sum under fresh cells is the one number on the screen a user would take at face value. The length assertion is the other half — the frame survives, the numbers do not.

+

Renamed by improvise-n89z, which deleted GridUpdate::Invalidated. The test used to reach the blanking through that arm; it now reaches it through a cats change, which is the thing that really produces it. Proved discriminating by pinning recategorized to false.

+

#a_relabelled_row_is_blanked_and_the_frame_survives

+

The other half of the same deletion. invalidation_blanks_contents_but_keeps_ the_frame asserted what the retired arm did; this asserts what invalidate_relabelled does, and it is strictly stronger, because the old arm blanked everything and so could not tell "blanks what it reframes" from "blanks the lot". Row 1's label does not change and row 1 keeps its value. Proved discriminating by weakening the stale filter from || to &&.

+

#the_page_bar_outlives_a_reshape

+

page_coords is framed by nothing: it names categories, not positions. So it survives a reshape that drops every row and column. Without this, "re-fit everything on a frame update" would look correct.

+

#the_first_rendered_entry_always_shows_its_level

+

Bug (improvise-jjx), moved here with the rule (improvise-anr9): header suppression compared against the entry at idx - 1 even when that entry was scrolled off-screen, so scrolling blanked the group label at the top of the viewport. The first rendered entry always shows its full tuple.

+

The fixture is two-level so the level-0 prefix of index 1 genuinely matches index 0's — a single-level frame could not express the case.

+

#a_repeated_prefix_below_the_first_rendered_entry_is_suppressed

+

The negative arm of the above, and it has to stay: without it, "always show every level" would pass the_first_rendered_entry_always_shows_its_level and the multi-level header would repeat its outer label on every row.

+

#a_data_index_skips_group_headers_and_still_names_its_group

+

Group headers do not consume a data index, and a data index resolves to the group whose header precedes it — which is what makes a column group span drawable by a client that holds no GridLayout. The collapse state travelling with the group is the third assertion, and it is why FrameGroup::collapsed is on the wire rather than re-derived.

+

#shared_ancestry_widens_as_the_level_rises

+

The underline rule: at the leaf level only the selected column, at a higher level every column under the same ancestor. Both levels are checked because a version comparing whole tuples would pass the leaf arm and light only one column at the outer level, and a version comparing only the outermost label would pass the outer arm and light a sibling at the leaf.

+

#body_from_starts_at_the_item_not_at_the_header_above_it

+

A window opening mid-group must be able to name its group without painting that group's header as a row inside the window. The second assertion — body_from(0) is everything, headers included — is what stops the fix being "drop all group headers".

+

#header_widths_are_per_level_and_measured_in_display_cells

+

The one width rule (improvise-1125): per level, the widest label plus a gap, clamped. Measured in display cells, so 日本 counts four and not two — a chars().count() would make level 1 two cells wide, clamp it to MIN_ROW_HEADER_W, and draw the header short. Both clamps are exercised ("ab" is under the floor, "x" * 100 over the ceiling), because a rule whose clamps are never reached is a rule nobody has tested.

+

#a_column_without_its_width_yet_is_measured_in_display_cells

+

Raised by Copilot on PR #23. The fallback width a column is drawn at between its ColFrame and its ColWidths must be in the same unit the authoritative widths are in. chars().count() would draw a wide label half as wide as it needs. The fixture is deliberately a two-scalar / four-cell string, which is the smallest input that separates the two counts.

+

#an_axis_with_no_categories_still_has_one_level

+

An axis carrying no category still draws one header line, and its item tuples are empty — reading a level off one must answer "" rather than panic. This is the shape a view with nothing on Row produces, so it is a live layout and not a degenerate fixture.

+

#a_relabelled_column_does_not_keep_the_old_columns_value

+

A column dropped from the middle used to slide the cells to its right one place left, because fit_to_labels truncates by position: [A,B,C] shrinking to [A,C] left the column now labelled C showing B's value. Nothing on the wire ever forced a producer to declare it first, so apply blanks what it reframes.

+

#a_label_change_blanks_only_the_positions_it_reframes

+

The same rule on the row axis, and the case a bare label shrink gets right by accident: only the relabelled positions are blanked, so an unchanged row keeps what the server already sent for it.

+

#renaming_a_row_category_invalidates_every_cell_under_it

+

Renaming the category re-identifies every cell on the axis, even where every label is byte-identical: a coordinate is cats zipped with the item tuple, so Region = [North] and Department = [North] are different cells wearing the same name.

+

Frame-only, and that word is the test. No Cells and no Totals follow the frame update, because the producer's follow-up is exactly what would hide the bug — apply has to be right with nothing behind it, the same shape as a_cell_update_that_overtakes_its_labels_is_lost.

+

The id assertion is the sharp half and is separate from the text on purpose. A stale CellId::Record under a re-identified coordinate is an editable cell whose write addresses the record the old coordinate named — silent, invisible in any rendering, and the same family of defect as gotcha 18's drill snapshot.

+

The precondition assertion (the items are identical) is load-bearing: without it the test is just the ordinary relabelling case, and cats proves nothing.

+

#renaming_a_column_category_invalidates_every_cell_under_it

+

The column half of the same rule. Here the per-position arm already blanks each total as its column goes stale, so this pins the cells and the ids; the row arm above is the one that needed the totals handled separately.

+

#an_unchanged_category_leaves_the_cells_that_still_fit

+

The negative arm, and it is load-bearing: without it, "blank everything on every frame update" would pass both renaming tests above, and the fold would resend the whole grid after a group collapse.

+

#a_relabelled_column_drops_its_total

+

improvise-hkt6. invalidate_relabelled blanked the cells under a renamed column and left its total standing, so a client that heard ColFrame without a Totals behind it drew the old column's sum under the new column's blanks.

+

Unreachable from grid_updates, which always sends the changed totals in the same batch — which is the point: the rule is that apply does not trust a producer to remember, and the only way to test it is to send the update the producer never sends alone.

+

Three columns and a rename in the middle, so "blanked the right one" is separable from "blanked all of them".

+

#a_relabelled_row_leaves_the_totals_alone

+

The row half is the negative arm and has to stay: a total spans every row, so no single row frames it. Without this, "blank the totals on any relabel" would pass a_relabelled_column_drops_its_total.

+

#col_widths_always_span_the_column_labels

+

col_widths is framed by the columns: a renderer zips the two, so a cache where they disagree in length is one that indexes off the end. ColWidths is its own update and may be a frame behind the frames.

+

#a_col_widths_update_is_fitted_to_the_labels_it_is_framed_by

+

The same invariant seen from the other arm. ColWidths used to be assigned verbatim, so a widths update that disagreed with the frame put the cache straight back into the state col_widths_always_span_the_column_labels exists to forbid — the frame arm was fitted and the widths arm was not.

+

A short update is the ordinary case: the server resized a column the client has not been told about yet. A long one means the widths overtook the ColFrame that frames them, and the fold drops what overtakes its frame — the same rule Cells is already held to. Both directions are asserted, because fitting only one of them is exactly the bug.

+

#item_updates_touch_only_the_named_category

+

Plus the dropped-update arm at the end: an Items update for a category the cache has never heard of is dropped, not inserted — the totality rule again, on a different widget.

+

#an_axes_update_moves_several_categories_at_once

+

A transpose moves two axes at once, which is why Axes takes a list rather than a single pair. The third assertion — membership is undisturbed — is what stops the fix being "resend the whole category list".

+

#every_projection_round_trips_through_json

+

One populated value per CellDisplay shape: numeric decides the alignment and error the styling, so a round trip that only ever carries them at their default proves nothing about either (doctrine(populate_before_roundtrip)).

+

#a_populated_snapshot_round_trips_through_json

+

The Snapshot is the only thing on the wire that is not an event, and the only point at which ViewState is serialized (improvise-q08) — so it is the one message whose round trip nothing else covers. Populated on both halves: a default ViewState beside an empty seed would round-trip through almost no code.

+
+

#src/basis.rs

+

#a_client_that_has_folded_nothing_cannot_claim_a_basis

+

The floor. A client that has been told nothing cannot compose a basis, so it cannot compose a write. Basis has no Default and no public constructor, so BasisCache::claim is the only door — which is what this asserts is shut before the server has published anything.

+

#an_unknown_record_is_omitted_rather_than_guessed

+

A record the client has never been told about is left out, not invented. Defaulting it to zero would be a claim the client cannot support, and the server would read it as "untouched" — the exact forged-basis case Rejection::RecordUnstated exists to catch.

+

#a_claim_carries_the_counter_for_each_record_it_names

+

The second assertion is the one that earns its place: a claim naming one record must not carry another's counter. A claim that returned the whole cache would pass the first assertion and fail this.

+

#the_published_record_list_is_stable_under_traversal_order

+

The published list is deduplicated and ordered, so a client is not sent a fresh basis every time a row moves on screen. Two maps built in opposite insertion order must compare equal — which is the property the BTreeMap parameter buys and a Vec would not.

+
+

#src/frame.rs

+

#the_retired_spellings_do_not_parse

+

The names the deleted envelopes used, which must not resurrect as serde aliases: an envelope accepting two spellings of one variant is two homes again, wearing one type.

+

Not a hypothetical rename — Bootstrap and Command are exactly what the worker pair (improvise-wasm-client::wire) spelled these two payloads before improvise-xks6. ClientFrame::Command joined the list with improvise-bpq2, and it is the sharper of the two: the singleton spelling was this crate's own until that commit, so an alias would be a revival rather than a resurrection, and the payload shapes differ only by a pair of brackets — {"Command":{...}} against {"Commands":[{...}]} — which is exactly the difference a permissive alias would paper over.

+

The injected-fault discipline is the part to keep (doctrine(check_the_checker)): the mutated input is asserted well-formed JSON first, and the failure is asserted to be about the name — serde's unknown variant — not merely that an error occurred. A rewrite that also breaks the JSON makes this pass against an enum that really does have the variant.

+
+

#src/reduce.rs

+

#a_model_command_leaves_the_view_slice_untouched

+

A model command is the document's business, not this client's. Folding one into the view slice would be a client writing state it does not own — the document's answer comes back as projections. The cursor is set to a non-default value first, so "the whole state is still default" cannot stand in for "the state did not move".

+

#folding_the_same_stream_twice_gives_the_same_state

+

Both ends fold the same commands, so the same stream must produce the same state — otherwise the server's mirror drifts from the client it is mirroring (improvise-m91). A Projection is included in the stream, to pin that it passes through reduce_view untouched rather than being a case it handles.

+

#every_command_round_trips_through_json

+

One populated value per payload family (doctrine(populate_before_roundtrip)), not per variant: the exhaustive walk over the 26 ModelCommand variants is every_model_command in improvise-command, where the effect each one names can also be checked.

+

The modelled() helper composes its basis through a seeded BasisCache rather than building a Basis literal — not because this test is about the basis, but because that is the only constructor there is, and a test that needed a second one would be evidence the door had been left open.

+

#cursor_commands_land_in_the_active_views_entry

+

The second assertion is the point: the view we switched away from never gained an entry. An implementation writing a single cursor and copying it per view would pass the first assertion.

+

#switching_views_restores_each_cursor

+

The point of keying the cursor map: switching back restores where you were, and the client can do that without a Workbook.

+

#help_pages_stop_at_both_ends_and_set_clamps

+

Both bounds and the clamp, because going back from page 0 is the one that underflows if written as help_page - 1, and Set out of range is the one that panics if written as an index.

+

It used to be named ..._wrap_in_both_directions_... and asserted wrapping, which was a second derivation of the effects it came from and disagreed with them: effect::HelpPageNext clamps at the last page and HelpPagePrev saturates at zero, while this arm wrapped modulo HELP_PAGE_COUNT. Nothing compared the two, because no client had ever sent Command::HelpPage — the help overlay's h/l were still upstream keys. improvise-e24v made them a client's, and walking_off_the_end_of_the_help_pages_agrees_with_the_host failed on the first press past the end (client 0, host 4). The wire was changed to match the effect rather than the other way round: the effect is what ships in the TUI, and changing that is a product decision nobody asked for. Whether wrapping is wanted on both ends is filed separately.

+
+

#src/lib.rs

+

#the_wire_types_need_nothing_but_core_and_serde

+

improvise-cqi AC 2: this crate crosses a process boundary, so it must not drag a terminal backend with it. A cargo tree check would be external; this is the part the compiler can hold — the types here resolve with only improvise-core and serde in scope, and adding a ratatui or crossterm dependency would have to be written into Cargo.toml deliberately rather than arriving through a use. The CI wasm job is the other half: a crate pulling in crossterm would stop building for wasm32.

+
+

#src/viewmodel.rs

+

#expanding_an_absent_category_is_harmless

+

An expanded name the cache does not carry must not shift the flattening: expanded_cats is per-client and survives a document swap, so it routinely names categories that are gone. A flattening that indexed by expanded-name would go off the end here.

+

#only_quit_advertises_nothing

+

improvise-2hi, carried over from only_quit_has_no_hint when the hints moved out of AppMode (improvise-k099). The old hint_text had a _ => "" arm, so the modes without hints were invisible; ExportPrompt was blank too (improvise-1c9), and the status bar went empty at the one moment the user has to be told what to type. Only Quit may say nothing — it is never rendered.

+

What it asserts now is one step stronger, because hint_line is derived: a mode goes quiet not by someone forgetting a string but by every command it advertises becoming unbound, which every_advertised_hint_names_a_binding_that_exists catches first.

+

#every_advertised_hint_names_a_binding_that_exists

+

The replacement for every_hint_key_resolves_in_its_mode (improvise-2hi), which is deleted, and the deletion is the point (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)).

+

That test existed because AppMode::hint_text was a hand-written string promising what the keys do, with nothing tying it to the keymaps. It walked every Key:verb token in every hint through a 60-line tokenizer that had to know the hint dialect — named keys, Ctrl-<c>, F/C/V alternatives, runs like hjkl, the em-dash prose tail, CommandMode's ex-commands — and panicked on a token it could not classify, because a token it skipped was exactly the one that would go unchecked.

+

None of that is needed now. A hint names a command, and its keys are looked up in Keymaps::default_keymaps(), so "the hint names a key that resolves" is true by construction and cannot be otherwise. What remains checkable, and is what this test checks, is the other direction: a mode advertising a command that nothing binds. That was never checkable before.

+

Note what neither version reaches, and gotcha 14 stated for the old one: that the caption is accurate. i:edit where i enters records mode still passes. The caption is English beside a command name, and only a reader can judge it.

+

#the_retired_invalidated_spelling_does_not_parse

+

improvise-n89z deleted GridUpdate::Invalidated, and the one way it can come back without anyone noticing is as a serde alias — a wire spelling that parses and that nothing produces, which is the state the deletion was for.

+

It keeps the discipline the_retired_spellings_do_not_parse earned in frame.rs: assert the mutated input is still well-formed first (a ColWidths projection of the same shape parses), and assert the error kind — serde's unknown variant — rather than merely that an error occurred, because a rewrite that also broke the JSON would make the check pass against an enum that really did have the variant.

+

Proved discriminating by re-adding the variant with an empty apply arm: the retired spelling parses again and the test fails on expect_err.

+

#boundary_tests::the_app_mode_cycle_reaches_every_variant_serde_declares

+

#basis::tests::the_rejection_cycle_reaches_every_variant_serde_declares

+

improvise-5ubs. Both all() functions walk a cyclic after(), and both doc comments claimed a new variant could not be omitted. Answering after() for a new variant is compiler-forced; pointing an existing arm at it is not, so a variant nothing names is never walked. The reviewer of PR #78 measured it: AppMode::all().len() came back 17 against 18 variants, suite green.

+

The independent enumeration is serde's — the Deserialize derive builds the variant-name list from the enum definition and publishes it in its unknown variant ... expected one of ... message. variants.rs reads it back and diffs it against the names the cycle reached, each name taken off the variant's own serialization rather than off Debug.

+

Fault injected 2026-09-21: a Rejection::Probe variant added to the enum, with an after() arm returning DocumentReplaced and nothing pointing at it — precisely the shape the reviewer probed. the_rejection_cycle_reaches_every_variant_serde_declares fails naming ["Probe"]; the other 120 tests in improvise-protocol's lib pass, including the_rejection_cycle half of every basis test. (The workspace does not compile with the probe, because improvise-wasm-client/src/socket.rs matches Rejection exhaustively — which is a different guard and does not catch the omission, since the omitted variant there is one it was forced to name.)

+

#variants::tests::a_type_that_is_not_an_externally_tagged_enum_is_refused

+

#variants::tests::a_message_with_no_expected_clause_is_refused

+

#variants::tests::a_message_that_lists_something_other_than_variants_is_refused

+

#variants::tests::a_list_that_collapses_to_one_name_is_refused

+

#variants::tests::a_well_formed_message_yields_its_names

+

The checker's own checker. A parse of serde's message that silently failed would return an empty name list, and an empty list is a subset of everything — every cycle would pass and the guard would be worse than absent.

+

The first version named one thing and exercised another, found by the PR #92 review. It pointed declared at Basis, a struct, so serde answers invalid type … expected struct Basis — which reaches only the split_once → None branch. The starts_with("unknown variant") and names.len() > 1 refusals were exercised by nothing, and all three panics shared the literal "serde no longer says", so expected = … could not discriminate which had fired. A should_panic whose message matches several panics is a check that cannot say what it caught.

+

The parse is now parse_expected(&str), a seam fed hand-written mutated messages, each refusal carrying its own wording:

+ +

plus a happy-path case, so the three refusals are not the only evidence the parser runs at all. a_type_that_is_not_an_externally_tagged_enum_is_refused keeps the end-to-end shape through a real Deserialize.

+

#render::tests::a_frame_that_lost_its_origin_does_not_claim_to_lead_the_axis

+

improvise-1wl0, found by the adversarial review of PR #77. AxisFrame::first carried #[serde(default)], so a frame whose first was dropped in transit deserialized as 0 — which is not an absence but the claim that the frame leads the axis, and exactly what body_from acts on: match self.first + idx == 0. A scrolled browser would then draw the carried group header above its first row, at every scroll position, silently: the mirror image of the improvise-0o6x bug the field was added to fix.

+

The test serializes a frame built for a scrolled window (first: 3), removes the key, and asserts the refusal. Three things make it discriminating rather than a check that something went wrong:

+ +

Run against the pre-fix tree 2026-09-21: it fails at the expect_err, printing AxisFrame { .., first: 0, .. } — the forged claim itself. The fix is the removal of one attribute; the whole workspace stays green (1464), which is also the evidence that nothing was relying on the permissive parse.

+

#render::tests::two_runs_under_two_headers_of_one_name_are_two_runs

+

#render::tests::the_two_headers_are_equal_as_values_so_comparing_cat_and_name_cannot_split_them

+

improvise-h0iv. group_runs coalesced adjacent positions by group name, so two runs under two different headers sharing a name read as one and both front ends drew one span across the boundary.

+

The first test is the demonstration, at the level the rule lives: a frame with [Group(Month/Q1), North/Jul, North/Aug, Group(Month/Q1), South/Jul, South/Aug] and nothing ungrouped between the runs. Before the fix it returns [(Q1, 4)]; after, [(Q1, 2), (Q1, 2)]. It asserts the frame really holds two FrameEntry::Group entries first, so a fixture that lost one would fail at the precondition rather than pass with a single run.

+

The second test exists because the ticket proposed the wrong fix and a reader will propose it again. h0iv says FrameGroup carries cat and "comparing the pair would fix it" — but its own reachability case is a single category nested under an outer one, so both headers are (Month, Q1), equal as values. The test asserts that equality directly. What actually separates them is that they are two distinct entries in self.entries, which std::ptr::eq on the borrowed header states exactly.

+

Fault injected 2026-09-21: restoring prev.map(|p| p.name.as_str()) == ... fails the first test with left: [(Some("Q1"), 4)], and fails two_abutting_runs_of_one_group_name_are_two_spans_at_both_ends end to end. The second test passes under both rules, deliberately — it is a statement about the data, not about the comparison.

+

#render::tests::a_client_never_holds_widths_of_a_different_length_from_its_labels

+

improvise-44ad's replacement test, landed while the move it was written for was declined. A renderer zips col_widths with the column labels, so a length mismatch indexes off the end (gotcha 33(c)). The existing guards assert the re-fit on individual arms; this one asserts the invariant across a fold, at every step, over a stream that widens the frame before its widths arrive, lands a cell, catches the widths up, narrows the frame, lands totals, and catches up again.

+

It asserts the first update really widens past the widths held, so a fixture that stopped exercising the gap would fail at the precondition rather than pass.

+

Fault injected 2026-09-21: deleting the width re-fit from fit_to_labels (self.col_widths.truncate(cols); if self.col_widths.len() < cols { replaced by if false {) fails this at step 0 — 3 widths for 5 columns.

+

What that injection also established, and it corrects improvise-44ad. Only four tests in the whole workspace fail: this one and three that already existed (a_col_widths_update_is_fitted_to_the_labels_it_is_framed_by, a_column_without_its_width_yet_is_measured_in_display_cells, col_widths_always_span_the_column_labels). The ticket's premise was that a_client_on_a_socket_is_seeded_and_gets_its_edit_back is gotcha 33(c)'s standing evidence and that rewriting it would destroy the evidence for a rule. It is not, and it does not fail: it asserts a reframe carried widths, which is a statement about the producer's diff. The reason for declining is therefore not the one the ticket gives — see context/code/protocol.md §3.

+

#a_chord_is_recognised_by_its_modifiers_and_not_by_its_label

+

help::compact dropped a chorded key from a status-bar hint by testing whether the formatted label started with "Ctrl-" or "Alt-" — parsing a string it had just formatted. Finding 3 of the adversarial review of PR #50, filed as improvise-thpe and deliberately not fixed there: the reviewer enumerated every format_key_label output and none begins with either, so there was no false positive. The defect was the shape.

+

The regression guard is the signature, as it is for Checked (gotcha 43): compact now takes &[(KeyCode, KeyModifiers)], so the string it used to parse is not in scope to parse. keys_for returns the pairs and Hint::keys carries them; help.rs formats last, through Hint::labels and the one spelling in keymap::key_label. Keymap::labels_for — the "throw the modifiers away into a string" step the ticket names — has no caller left and is deleted, because a consolidation that keeps its old path keeps the defect available (doctrine(a_consolidation_that_keeps_its_compatibility_test_is_unfinished)).

+

The rule also got wider, and that is the part a test can state: a chord is any modifier beyond shift, which is total over the bitset, where the string test enumerated two prefixes. Shift alone is not a chord — it is how a key is named rather than how it is held, which S-Tab is the case for.

+

This test cannot fail under the rule it names, and its doc comment says so rather than claiming otherwise. CONTROL|ALT|SHIFT on F(5) spells Ctrl-Alt-F5, which the old prefix test drops too. It pins the rule is_chord states; the discrimination is the injection below and the signature.

+

Injected fault, run rather than argued. A KeyModifiers::SUPER bit and a key_label arm spelling it Super-x were added temporarily. The old filter keeps it — ["T", "Super-x"], two survivors where one is wanted — while compact returns "T". That is the ticket's own reproduction, executed, and it is the only way to see the difference: with KeyModifiers as shipped, every chord's label does begin with Ctrl- or Alt-, so no permanent test can separate the two rules. Both the bit and the probe were removed afterwards.

+

One behaviour that had to be preserved deliberately: keys_for used to dedup labels, and deduping the pairs instead would let two patterns that spell the same key both reach the status bar. It now dedups by label explicitly.

+

The binding dump is byte-identical across this change (4406 lines, diff clean), which is what doctrine(binding_dump_diff) asks of anything touching the keymap: the dump's prefix rows go through Hint, so they had to be checked rather than assumed.

+
+
+ + diff --git a/site/docs/test-evidence/tooling.html b/site/docs/test-evidence/tooling.html new file mode 100644 index 00000000..2d59b878 --- /dev/null +++ b/site/docs/test-evidence/tooling.html @@ -0,0 +1,66 @@ + + + + + +The repository's own gates — improvise docs + + + +
+ +
+

Source: context/test-evidence/tooling.md

On this page

Test evidence — the repository's own gates

+

Evidence for the checks in scripts/ and for the tests that hold CI's shape. The arguments behind the checkers themselves are in context/code/tooling.md; this file records what fault each test was watched to catch.

+

Note what is deliberately absent. scripts/check-comment-runs.py --self-test, scripts/kb-test.sh and scripts/assemble-stamp-test.sh are check-the-checker suites whose cases are already injected faults, and none of their case names is a Rust fn or a JS test( the evidence checker could resolve — so their evidence is the case list itself plus context/code/tooling.md, and inventing headings for them here would be headings naming nothing.

+

#"no test in web/tests/ reaches for a built bundle"

+

The js CI job runs node --test web/tests/*.test.js and installs nothing; the browser job runs web/tests/bundle/*.test.js after three wasm-pack builds. The moment a flat web/tests/ file reads web/pkg/, the fast job inherits the slow one's dependency — and the likely repair is to add a build step to it, silently spending the ~4m30s the split exists to avoid. Until improvise-4ktt that constraint was a directory convention and a glob, checked by nothing, and it failed in the direction that looks like ordinary breakage: a confusing ENOENT in CI on a machine where the bundle happens not to be built.

+

Fault injected, 2026-09-21: appended const injected = readFileSync("../pkg/improvise_wasm_client.js"); to web/tests/grid_plan.test.js. The test failed naming that file and pointing at web/tests/bundle/. Reverted; green again.

+

Two things the test does that a simpler one would not. It asserts its own filename is among the files it scans, so it cannot become the one place a violation may hide. And it strips comments before matching, because this file's own header and the test's header both name the directory in prose — a lexical grep over raw text would have had to exempt them, and an exemption is where the next violation goes.

+

What it cannot see, stated because the check itself uses the evasion: a path assembled at runtime ("../" + "p" + "kg") is invisible to a lexical match, and the test's own FORBIDDEN constant is built exactly that way to stay out of its own scan. That is a deliberate, disclosed hole — closing it needs a parser, and the failure it would catch is one somebody has to write on purpose, where this gate exists for the one written by accident.

+

#"the subdirectory that MAY use a bundle is not the one being checked"

+

The companion, and the one that stops the first from passing vacuously. If web/tests/bundle/ were emptied or its files stopped touching the bundle, the first test would still be green while having nothing to be scoped against — "no flat test uses the bundle" is uninteresting if no test anywhere does. This asserts the other half of the split exists and really does what the flat half is forbidden to do.

+

Fault injected, 2026-09-21: moved web/tests/bundle/wasm_surface.test.js out of the tree. The test failed with "web/tests/bundle/ is empty — the split lost its other half" rather than passing. Restored; green again.

+

#"no test in web/tests/ imports a module that reaches the bundle"

+

The companion to the lexical check above, and the reason that one is not enough: import "../app.js" needs the bundle transitively and names nothing forbidden, so the js job breaks loudly but unattributed. app.js and worker.js are the two entry points that load the bundle; shell.js and serial.js do not, and serial.test.js imports one legitimately.

+

Fault injected, 2026-09-21: appended import "../app.js"; to web/tests/grid_plan.test.js. The lexical test stayed green; this one failed naming the file. Reverted.

+

#Note — the checker's own blockers, found by review of PR #83

+

Two gates in scripts/check-comment-runs.py produced a pass while not doing what their prose said. Both were in the mode the same PR rewrote, which is the part worth keeping: writing the doctrine down does not exempt the writer.

+

A leaf name is not an identity. Six leaf names collide in this workspace and two were cited in evidence. Renaming ui::project::tests::a_load_that_fails_is_not_a_rebase — the one its entry describes — left the checker printing all name a test that runs, exit 0, with the heading answered by the ui::host::tests namesake. Now an ambiguous leaf is refused with both candidates named, and the two entries are qualified. Re-verified both ways, 2026-09-21: the refusal fires on the unqualified headings and names exactly those two; with the heading qualified, the same rename is reported as an orphan rather than absorbed.

+

--rust-tests was strictly more permissive and said the same words. It called cargo_test_names with one argument, so the --ignored subtraction never ran, and the success line was byte-identical to the honest path's. Injecting ### eyeball_the_rendered_hints_and_pages orphaned on the default path and passed on the flag's. Now the flag requires --rust-tests-ignored (one alone exits 2) and the success line names its source.

+

A missing evidence directory exited 0. The early return preceded the citation scan, so deleting context/test-evidence/ reported nothing while 170 citations pointed at a directory that was not there. Verified 2026-09-21 by moving the directory aside: exit 1, 170 tests cite context/test-evidence and have no entry there.

+

#Note — the flag that selected the cap away (improvise-ijql)

+

scripts/check-comment-runs.py --check-evidence early-returned in main, so the flagged run never answered the comment-run cap and printed nothing about having skipped it. The reviewer's position when it was first raised was that CI runs all three spellings (ci.yml, the three check-comment-runs.py steps in the test job) so this was documentation only. The counter that carried is that the flagged form is the expensive one — it needs a compiled workspace — which makes it the one a human runs alone; three agents hit it independently in one campaign, and in one case a local gate went green while CI went red on a three-line doc comment.

+

Reproduced before the fix, 2026-09-21, in a temporary tree holding one .rs file with a four-line comment run:

+
$ check-comment-runs.py src --check-evidence --rust-tests … --rust-tests-ignored …
+test-evidence: 0 entries, all name a test that runs …
+EXIT=0
+$ check-comment-runs.py src
+1 comment runs longer than 2 lines:
+  src/bad.rs:1  4 lines
+EXIT=1
+

Two invocations over one tree, opposite verdicts, and the one that exits 0 is the one a human is more likely to run by itself.

+

The guard is _dispatch_self_test, which runs the script as a subprocess — the only level at which this is visible, since every unit in the file was correct and the defect was in main's dispatch. Both arms were demonstrated:

+
injected faultwhat failed
restore the --check-evidence early returnDISPATCH: --check-evidence: exited 0 over a four-line comment run
make the cap always report a violationDISPATCH: bare: refused a clean tree and the same for --check-evidence
+

The second arm is what stops an always-failing cap from satisfying the first, and it is why SELF_TEST_DISPATCH asserts a clean tree passes as well as a dirty one failing. Why the fix is additive rather than a printed disclaimer, and why --self-test/--list-doctests are excluded from the table rather than exempted from the rule: context/code/tooling.md §2, "A flag adds a question".

+

#Note — the stamp gate's discrimination, measured

+

scripts/assemble-stamp-test.sh separates three outcomes rather than two, which is what makes it evidence rather than a smoke test. Measured 2026-09-21 over its nine cases: the real script passes 9; deleting the final test -z "$missing" (the PR #69 injection, which the inline version survived at exit 0) passes 4 and fails 5; a script that does nothing passes 2; one that always fails passes 0.

+

The PR body first cited "4 of 9" for the middle row, from a run taken when the suite had eight cases. A figure is what the next reader checks, so it is recorded here where it can be re-measured rather than only in a message nothing can audit (design-principles §6).

+

#Note — the stamp gate matches literally

+

scripts/assemble-stamp.sh greps with -F. As a regex, the . in improvise_wasm_client.js?v=abc matches any character, so a stamp corrupted in exactly that position reads as present — a pass in the one direction a guard must not fail in. Inherited from the inline version this was extracted from, and found by reading the extraction rather than by anything failing.

+

Fault injected, 2026-09-21: dropping the -F makes improvise_wasm_clientXjs?v=abc satisfy the pattern, and a pattern is matched literally, not as a regex goes from pass to fail.

+

#Note — the doc site's derived checks

+

scripts/build-doc-site.mjs carries two gates that no test names, because the generator is the test: it runs on every assembly and exits non-zero.

+

Faults injected, 2026-09-21, each observed:

+
injectedreported
a new context/code/newarea.mdcontext/code/newarea.md is neither published nor a named omission
context/plan.md moved out of the treecontext/plan.md is listed as omitted but does not exist
a line appended below the generator in web/assemble.shweb/assemble.sh must invoke this generator LAST, naming the offending command
a trailing echo "see scripts/build-doc-site.mjs"the same, once the check asks for an INVOCATION rather than a mention — .includes had accepted it (review, PR #83)
+

scripts/check-ci-graph.py is the third, and it is a script rather than a stanza because its own refusals need checking: adding a zz-new-check: job to ci.yml reports pages-deploy does not need zz-new-check and exits 1, where before the rule was prose and nothing noticed. Its five self-test graphs include the one a naive parse gets wrong — another job's needs: read as the deploy's.

+

The second arm is the one that would not get written first, and it is the point: without it, deleting or renaming an omitted file leaves an entry asserting a reason for a file nobody can find — the same failure as a dead ticket id, which AGENTS.md already calls worse than no id.

+

The first arm was filed as latent and was live when it was written: context/code/kb-tool.md had been unpublished, unargued and unnoticed since it was added.

+

#Duplicate evidence headings — SELF_TEST_DUPLICATES

+

improvise-hsoe. --check-evidence verified that a heading names a test that runs, and a duplicate heading names one just as well, so both copies pass. PR #92 shipped two entries under a_layout_change_moves_the_layout_version_and_no_record, and two more pairs sat in tui.md — one of them asserting that THE_MEMBERSHIP_CORRECT_SPANS was "a live tripwire, do not remove it" some four hundred lines above the entry recording that improvise-gd83 had removed it. A reader meeting the older copy first takes it as current, which is worse than an orphan: an orphan announces itself as broken.

+

The fault is injected by the six cases in SELF_TEST_DUPLICATES, and three of them are the ones that would not get written first. Two different tests sharing a leaf must come back ambiguous and not duplicate, because the wrong diagnosis files the wrong repair. A prose heading must be free to repeat, because it claims no test. And the bare and module-qualified spellings of one test must collide — grouping by the string rather than by the resolved test would have missed exactly the case improvise-rm4s's qualification repair creates, which is the first way this would have come back.

+

The live tree was itself the first injection: before the three merges the check exited 1 naming all three pairs with both line numbers; after them it exits 0. The clean arms (two headings naming two tests, and prose repeated) are what stop an always-refuse from satisfying the rest.

+
+
+ + diff --git a/site/docs/test-evidence/tui.html b/site/docs/test-evidence/tui.html new file mode 100644 index 00000000..adfee263 --- /dev/null +++ b/site/docs/test-evidence/tui.html @@ -0,0 +1,149 @@ + + + + + +The TUI — improvise docs + + + +
+ +
+

Source: context/test-evidence/tui.md

On this page

Test evidence — the terminal front end (src/, tests/, examples/)

+

Keyed by exact test name, per design-principles §7. A renamed test orphans its entry; python3 scripts/check-comment-runs.py --check-evidence catches a heading that names no test.

+

A ### heading is a test, and --check-evidence only counts #[test] functions, so it cannot be satisfied by a helper of the same name. Fixtures and helpers that carry an argument get a subordinate #### heading under the tests they serve — unchecked, and deliberately so: there is nothing to orphan, because a renamed helper is found by the compiler.

+
+

#tests/grid_frame_agreement.rs — the two front ends draw the same frame

+

improvise-anr9/s1xw. The divergence this file exists to rule out was real and shipped: GridCache::row_labels was a Vec<String> built by GridLayout::row_label, which slash-joins a multi-level tuple — the exact rendering src/ui/grid.rs::two_row_categories_produce_cross_product_labels asserts must not appear. So the protocol carried a form of the labels the TUI itself considered wrong, and the browser drew Food/Alice where the TUI drew two header columns, with no groups and no collapse indicators.

+

The shape of the check is the point. Each front end is driven the way it is really driven — the widget into a ratatui buffer, the client through a Snapshot of the same cache's seed() — and the two are compared on what they draw, not on what they were handed. Comparing the caches would be vacuous: it is one cache.

+

#two_level_rows (helper)

+

The fixture. Both axes multi-level and grouped — the cross product the slash-join used to flatten, on rows and columns, so the frame carries header levels, repeat suppression, group spans and a collapse state in one fixture.

+

Two deliberate asymmetries:

+ +

#tui_row_headers (helper)

+

Slices the painted row header by AxisFrame::header_widths, because that is the same rule the widget lays the sub-columns out with (improvise-1125). Reading the widths a second way here would be the very duplication under test. The body ends at the blank space below the grid, or at the rule above the totals strip — which is not a row of the frame.

+

#tui_header_cells (helper)

+

Cuts one painted header line by the cache's own col_widths past the row header, for the same reason. Labels are right-aligned into width - 1, so a trim recovers them.

+

#the_two_front_ends_draw_the_same_row_header

+

Injected fault, watched fail before keeping: re-introducing the slash-join on either side — labels: vec![rows.item(r).join("/")] in the client's render_model, or rows.item(ri).join("/") in the widget's row header — makes this test fail, naming the row and the two spellings. Both were watched fail.

+

A group header is one label across the whole row header; a data row is one label per level, and the TUI's slicing puts a group label in the first sub-column, so the comparison joins the non-empty text. The group arm compares the full painted string, glyph and name: a branch that checked only the glyph would pass on any group name whatsoever. indicator is the TUI's glyph spelled out locally so the comparison is against what is really painted rather than against a prefix test.

+

The tui.len() > 4 precondition exists because a fixture with no group headers and few data rows compares almost nothing.

+

#the_two_front_ends_draw_the_same_column_header

+

A separate test because the row-header one drives one arm: a col_levels that stopped suppressing repeats, or a col_groups that emitted one span per column, would sit green behind it (doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only)).

+

Each header line is cut into columns and compared exactly. A contains would be one-directional: it cannot see a label the browser wrongly blanked (the filter would skip it) and it cannot see one painted under the wrong column.

+

improvise-gd83 is fixed and THE_MEMBERSHIP_CORRECT_SPANS is gone, along with the assert_ne! tripwire that named it. The value it predicted — [(Q1,2), (None,1), (Q1,2), (None,1)] — is now what the test asserts, element for element. Sep is ungrouped, so the run ends before it and resumes under the second measure's own Q1 header; a precondition asserts the fixture still contains an ungrouped column, because without one the test cannot see a leaked group at all.

+

The superseded expectation — Q1 covering all six columns including the two under the ungrouped Sep, because expand_category emitted a header on entering a group and no boundary on leaving one, so the nearest-preceding walk kept answering Q1 — is recorded here because it is what AxisFrame::group_of answered before the fix, and the frame it was derived from still explains the current value. The frame is not one Q1 run — _Measure is the outer level, so the column axis is:

+

col_cats = ["_Measure", "Month"] Group(Q1) [Revenue,Jan] [Revenue,Feb] [Revenue,Sep] Group(Q1) [Cost,Jan] [Cost,Feb] [Cost,Sep]

+

Two Q1 headers, one per measure, and the ungrouped Sep sits inside each run rather than after both. So membership-correct spans are four runs, not two — and they do not coalesce across the second header, because Sep breaks the run between them. Hence [(Some("Q1"), 2), (None, 1), (Some("Q1"), 2), (None, 1)].

+

improvise-8qlr: the TUI arm was group_line.contains("▼ Q1"), which is one-directional and passes while the TUI labels every column. The run rule now lives once, as AxisFrame::group_runs/group_run_heads, and both ends call it; the TUI's group line is compared cell for cell against spans derived from the browser's own runs.

+

Fault injected exactly as that ticket specifies — forcing the TUI to emit the group label on every column (head = col_groups.get(ci), dropping the run-head lookup). Before: cargo test -p improvise stayed green. After: this test fails with ["▼ Q1","▼ Q1","","▼ Q1","▼ Q1",""] against ["▼ Q1","","","▼ Q1","",""].

+

One thing the shared rule does not flatten: the TUI labels its first visible column even mid-run, because a scrolled grid would otherwise draw an unlabelled group line. That is a scrolling concession, kept explicitly, and it is why the comparison is run at col_offset == 0.

+

#neither_front_end_produces_a_slash_joined_label

+

A separate assertion because the agreement tests above would also hold if both ends started slash-joining. This one is the bug's own words.

+

#the_collapse_state_crosses_as_a_fact_not_a_glyph

+

The group's collapse state reaches the browser as a fact, not as a glyph. The TUI spells it ▼/▶ (presentation, gotcha 26); the client gets the boolean and spells it however the DOM wants. Both states are checked, because a collapsed hard-wired to false would pass the first half. The non-empty precondition states the rule it depends on: a collapsed group still owes the user a header to expand again.

+
+

#src/ui/grid.rs

+

#render (helper)

+

The widget takes a GridCache, so the fixture builds one the way the TUI does: through an App, over the whole grid. Going through App rather than hand-rolling a cache is deliberate — these are characterization tests, and a handcrafted cache would test the widget against a fixture rather than against what the projection actually produces.

+

#grouped_model (helper)

+

A grid with a row group, a column group, a page axis and a total row, written against the pre-migration path (improvise-n10) so the move to GridCache is provably behaviour-preserving (doctrine(characterization_survives_the_move)). The fixture and GROUPED_FRAME are byte-identical to the pre-improvise-anr9 versions, which is what makes the comparison mean anything.

+

#the_whole_frame_is_unchanged

+

Asserts the exact buffer text, not a contains, because the facts it does cover — group indicator placement, column alignment, adaptive row-header width, the page bar, the totals strip — are all positional and a contains cannot see any of them.

+

What it does not cover, stated because an overclaim here is worse than a gap: Region is alone on Row and Month alone on Column, so both axes are single-level and this exercises no multi-level header and no repeat suppression — which is the whole subject of improvise-anr9. Widening the fixture would destroy the only byte-exact before/after evidence in that change, so it stays as it is; the multi-level agreement is tests/grid_frame_agreement.rs, which drives both front ends over a two-level, grouped fixture.

+

Re-pinned for improvise-0o6x. GROUPED_FRAME moved by exactly one line: the leading ▼ Co… header now draws above East, and everything below shifts down one. Nothing else in the buffer changed. That the group line, the column headers, the separator, the data rows and the totals strip are byte-identical is the evidence the change is the boundary condition it claims to be and not a reshuffle of the draw loop. It also fails alongside a_group_header_first_on_its_axis_reaches_both_front_ends when body_from's match is forced to false.

+

#two_row_categories_produce_cross_product_labels

+

Names the rendering the protocol must not carry: multi-level row headers show category values separately, never joined with /. It is the assertion tests/grid_frame_agreement.rs's module argument cites as the thing GridLayout::row_label violated.

+
+

#src/ui/import_wizard_ui.rs

+

#a_cut_inside_a_multibyte_character_does_not_panic

+

The bug, found by the PR #12 Copilot review and pre-existing on main: truncate sliced by byte index, so a cut landing inside a multi-byte character panicked. Every string it cuts can carry user input — a CSV column name, a formula being typed, an error message naming a field — so a document with an accented header crashed the TUI rather than eliding a name.

+

"Régions" is 8 bytes for 7 characters: the é occupies bytes 1..3, so max = 3 puts max - 1 = 2 inside it.

+

#no_cut_point_of_a_multibyte_string_panics

+

Not one cut point, every cut point. The interesting indices are the ones inside a character, and which those are depends on the string — so the test walks them rather than picking one and hoping it was the bad one. Each result must also be a prefix of the input, which is what says the boundary search walked down rather than dropping the character it landed in.

+

#ascii_truncation_is_unchanged

+

ASCII is unaffected, which is what lets the characterization fixtures in this file keep asserting exact frames written before the boundary fix.

+

#render (helper)

+

Drives a real ImportWizard and draws it through the projection, which is the whole path the TUI takes. Building the cache from the live wizard rather than by hand is deliberate: a handcrafted fixture would test the widget against the fixture instead of against what wizard_of actually produces, and the mapping is the half of improvise-764 most able to be silently wrong.

+

#every_step_draws_what_it_drew_before

+

Every step the wizard can draw, byte for byte. Written against the pre-migration path (improvise-764), where the widget reached into ImportWizard, ImportPipeline, FieldKind and Workbook directly, so the move onto a projected cache is provably behaviour-preserving (doctrine(characterization_survives_the_move)).

+

Exact text rather than contains: the interesting facts are positional — the column stops in the proposals table, where the hint line sits relative to the panel bottom, how far the date components are indented — and a contains sees none of them.

+

The walk is bounded and stops when advance no longer moves: the wizard's transitions are conditional (improvise-907) and a step that declines to advance must end the walk rather than spin it. NameModel is terminal for advance (gotcha 22) — the Enter there builds, and building is what parks the workbook on Done.

+

#the_formula_step_draws_its_transient_state

+

The transient bits of DefineFormulas: a formula in the list, one being typed, and a message overriding the hint line. Its FORMULA_STEP fixture is the one of the eight pinned frames that improvise-q0vb moved: it used to read something went wrongcel, the cel being the tail of Esc: cancel left behind because the message was written over a longer hint without clearing the rest of the row. The frame changing there was the point of that commit, not a side effect.

+

#a_message_shorter_than_the_hint_leaves_none_of_it

+

improvise-q0vb. The wizard's bottom row has two writers — the step's key hint and the message that overrides it — and set_string writes only as many cells as its string has. A message shorter than the hint therefore left the hint's tail beside it, and the artifact was pinned as correct output in FORMULA_STEP.

+

Asserted on ReviewProposals, whose hint is the longest in the wizard, so the tail it would leave is unmistakable. The precondition asserts the hint really is longer than the message — without it the test would pass on a step where there was nothing to leave behind.

+

#set_bottom_line_clears_every_cell_it_covers_and_no_more

+

The padding is in terminal cells, not characters (Copilot, PR #22). Buffer lays a string out by display width, so a chars().count() pad is wrong in both directions: a wide character counts one and occupies two, so the helper runs past the panel and over its right border; a combining mark counts one and occupies none, so the helper stops short and leaves cells it was supposed to clear.

+

Asserted against a sentinel, which is the correction Copilot made to a first version of this test. Drawing through the widget cannot see the under-padding case at all: the hint that ran first had already filled the row with its own padding, so the cells the combining message failed to reach held spaces rather than anything visible, and a "does the hint survive" assertion passed either way. The first version also looked like it had been proved, because the loop panicked on the wide-character case before ever reaching the combining one — doctrine(an_injected_fault_must_be_the_fault_you_named), from the other end.

+

So the test fills the row with X first and asks the helper directly: every interior cell must be the text or a space, and the two cells standing in for the panel border must still be X. Both faults are then visible as a surviving X — one inside, one outside.

+

#the_array_path_step_draws_its_candidates

+

One of the two steps the flat-array fixture skips. SelectArrayPath is reached by a document with more than one candidate array, which is also the only step that draws pipeline.array_paths.

+

#the_preview_step_draws_an_empty_document

+

WizardStep::Preview is the one step every_step_draws_what_it_drew_before never reaches (improvise-g6s1). Both fixtures there start at ReviewProposals or SelectArrayPath, and nothing advances into Preview — it is only ever an entry step, chosen by ImportWizard::new when the document needs no path selection and parses to zero records.

+

That left it the single arm where a change to wizard_cache could alter what is drawn with nothing failing, which is the same hole the other steps were pinned to close. An empty array is the smallest document that lands there.

+

It also puts on the record a wart the step has today: the summary ends Sample keys: with nothing after the colon, because there are no records to take keys from. Pinned as it is rather than quietly fixed — the point of a characterization fixture is that the next change to it is deliberate.

+
+

#tests/shipped_documents_load.rs

+

#every_shipped_document_loads

+

improvise-odbc. The finding behind it: bank-info.improv sat in the repo root for five months unable to parse — hand-edited through the v2025-04-09 format change rather than round-tripped, so its item lines carried bare names the grammar cannot spell (- JORDAN MARTINEZ) and its data lines named a category Measure the file declares as _Measure. Nothing loaded it, so nothing said so.

+

The grammar itself is covered far better than a fixture can manage, by the generator-driven proptests in improvise-io. What those cannot see is a checked-in document drifting out of the format, because they generate their inputs. This test is aimed at exactly that gap.

+

It is deliberately a directory walk rather than a list of names: a list is a second place to remember, and the file that gets forgotten is the one nobody added to it. The scope is the two directories .gitignore un-ignores — examples/ and web/; everything else matching *.improv is somebody's working file.

+

It matches on the full name, not on Path::extension(): .improv.gz is the same document gzipped and extension() calls it gz, so matching on the extension alone would skip it and put back the very hole this test exists to close.

+
+

#tests/web_demo_model.rs

+

#every_web_demo_cell_is_settable_except_the_column_a_formula_owns

+

improvise-bck. The demo document the standalone page opens must be one a visitor can type into, and the bug is not hypothetical: examples/demo.improv leaves Customer and Date on the None axis, so every cell in it is an aggregate over records the grid does not name, and an aggregate is not settable — correct behaviour, and a demo where the first thing a visitor clicks does nothing.

+

The question is asked exactly the way the browser asks it. ADR 002 says a cell is settable iff its CellId is Record, project::cell_id is the one place that decides, and render.rs::editable in the wasm client is that variant and nothing else (gotcha 42). So the test projects the demo's whole grid through grid_of — the same function the server calls per frame — and asserts on the surrogate, not on the axes. Asserting the axes instead would re-derive the rule and could agree with itself while disagreeing with the client.

+

It was proved discriminating by pointing it at examples/demo.improv, where it names the aggregate at (0,0).

+

improvise-ir7l renamed it, because the old name was false. The demo's third column is Profit = Revenue - Cost, and a formula target is a stored coordinate, so it projected Record and the page offered an editor over it — the test passed while its name was wrong for a third of the landing-page grid, which is exactly the shape this repo keeps finding. The assertion is now two-sided: the two raw columns must be Record and the Profit column must be CellId::Derived, so it fails if either half moves. Proved discriminating in both directions by deleting the derived branch of project::cell_id (Profit reads back as a Record) and, separately, by pointing it at examples/demo.improv again (the aggregate at (0,0) still fails it).

+

#the_web_demo_shows_a_formula_beside_its_inputs

+

The payoff a visitor should see without pressing a key: a formula column beside the two columns that feed it, so an edit visibly moves it. Profit at (0,2) is 18,400 - 6,200, recomputed on load.

+
+

#src/draw.rs

+

#release_kind_converts_like_press

+

The pre-conversion event loop never inspected KeyEventKind, so all kinds (Press/Repeat/Release) were dispatched. The conversion keeps that behaviour: a Release event converts like any other.

+
+

#The panel-migration equivalence tests (improvise-35e)

+

Each panel widget carries a pair: one test saying the inputs are unchanged by the move onto a projected cache, one pinning the drawing of them. The pair is the shape — a cache equivalence alone would not notice a changed renderer, and a rendered fixture alone would not notice the cache carrying different data.

+

#ui::tile_bar::tests::the_cache_carries_what_the_widget_used_to_read_off_the_model

+

#ui::formula_panel::tests::the_cache_carries_what_the_widget_used_to_read_off_the_model

+

Two tests share this leaf name, which is why both headings are qualified: one entry describes the pair, and an unqualified heading would have been satisfied by whichever of them the checker happened to see. The tile bar used to read view.tileable_cats(&model.category_names()) and view.axis_of(cat) off the model directly; the formula panel used to read model.formulas()[i].raw. Each now reads its cache, and a divergence is a rendering change nobody asked for.

+

#the_cache_reproduces_the_summary_the_widget_used_to_build

+

src/ui/view_panel.rs. The axis summary used to be built from workbook.views plus View::categories_on; it is now built from ViewSummary::axes. Same names, same order, same virtual-category filtering.

+

#the_active_view_summary_is_not_empty

+

The fixture must actually exercise the summary, or the equivalence above is two empty strings agreeing.

+
+

#examples/gen-grammar.rs

+

Not a test, but the generator the improvise-io proptests drive. It adds constraints beyond what the grammar requires, to produce realistic, round-trippable output rather than degenerate minima: bare names are drawn from a word pool instead of random letters, pipe_inner is never empty, rest_of_line always produces at least one character, and repetitions (*) produce 1–4 items rather than 0. Each invocation is seeded from the current time plus the PID.

+

#a_status_message_reaches_the_bottom_row_while_a_minibuffer_is_open

+

#the_minibuffer_prompt_is_not_displaced_by_the_message

+

#an_empty_message_never_pushes_a_row_that_fits_past_its_width

+

improvise-mai9. draw_bottom_bar rendered the minibuffer instead of the status line, so a message written while a text-entry mode was open was never drawn — not truncated, not overwritten. A test asserting app.status_msg was set passed in every one of those cases, which is how this survived improvise-6t3's careful effect ordering.

+

Of the ticket's two candidate fixes, the row is shared rather than the callers being made structurally unable to report: giving the status a second row changes frame geometry (improvise-q0vb's cost), and "always leave the mode before reporting" is what CommitSave does by hand and would have to be repeated at every text-entry commit — the thing the ticket calls a workaround.

+

The first version of these tests could not fail for the bug, and that is the entry worth reading. They called the composing helper minibuffer_line directly. The defect was in the wiring — draw_bottom_bar building its own format! and never consulting status_msg — so reverting that one line restored the bug in full with all 1411 tests green. The helper did not even go dead: the test module's use kept it alive while it asserted nothing about the product. Found by review (PR #77), reproduced here before rewriting.

+

So the guard is now Buffer-backed: a real Terminal<TestBackend> renders draw_bottom_bar and the assertion reads the row the terminal received. That is the only assertion that can fail for mai9's actual defect. Fault injected by restoring format!("{}{}", mb.prompt, buf) plus the cursor glyph — both wiring tests fail, the first naming the missing message and printing the row it did compose.

+

The second test exists because a "fix" that dropped to draw_status would satisfy the first on the message alone; it pins that the prompt still leads.

+

The third test is the negative arm, and its first version was unfalsifiable for arithmetic reasons. fill_line computes pad = width - left.len() - right.len(), so with an empty status an unconditional format!(" {msg} ") produces a byte-identical row at any width where the line fits: left plus width - left.len() spaces, either way. The match status_msg.is_empty() is observable only in the band where the prompt fits but left.len() + 4 does not — widths 16..=19 for this fixture, where the unguarded arm returns a row wider than the terminal.

+

That test then failed a second time in a more interesting way: it derived its lower bound by calling minibuffer_line itself, so the injected fault moved the bound with it and the loop skipped the failing band entirely. The bound is now spelled locally. A bound read off the function under test is not a bound — it is the fault grading its own exam.

+

Known and not fixed here: fill_line pads on left.len(), which is bytes, so the right-hand message sits two cells further right than intended whenever the left contains the multi-byte cursor glyph. draw_status has always done this with its own multi-byte content; it is pre-existing and out of scope. It is also what makes the band above four columns wide rather than two.

+

#a_group_header_first_on_its_axis_reaches_both_front_ends

+

improvise-0o6x. AxisFrame::body_from(idx) started at the first item, so at the top of the axis a leading FrameEntry::Group was skipped on the way to the screen — in both front ends. A collapsed group first on its axis therefore had its rows hidden and its header nowhere, so there was no affordance to get them back. The frame carried the header correctly; body_from dropped it.

+

It is a boundary condition, as the ticket suspected, but the boundary is not idx == 0. The windowing rule carries the nearest header ABOVE a scrolled window for group_of and body_from discards it — correctly. At the top of the axis there is no "above", and a frame cannot tell the two apart from idx alone, because the browser's frame is windowed and always passes 0. So AxisFrame gained first, the absolute data index of its first item, and the rule is first + idx == 0. Without it the browser would have drawn an extra header row at every scrolled position that the TUI does not.

+

This test is separate from the_two_front_ends_draw_the_same_row_header because that fixture orders Carol before Family to dodge this case on purpose, and says so. Both collapse states are driven: the uncollapsed arm alone passes against a frame that leads with a header only when it has items under it, which is the shape the windowing bug below produced.

+

The windowing half was found by this test and not predicted by the ticket. A collapsed leading group has NO item to carry its header into the window, so it reaches the frame only through axis_frame's pending queue — and a first version of the fix had GroupEnd clear pending above the window, which deleted exactly that header. GroupEnd is now queued like a header and clears nothing.

+

Fault injected by forcing body_from's match to false: this test fails and the_whole_frame_is_unchanged fails with it.

+

#two_abutting_runs_of_one_group_name_are_two_spans_at_both_ends

+

improvise-h0iv, end to end. The unit demonstration is in context/test-evidence/protocol.md; this is the same rule driven through a real GridLayout, a real project::grid_of and both renderers, because the reachability of the shape was itself in question.

+

two_level_rows has exactly h0iv's shape — Q1 grouping two of three months, cross-producted under two measures — and is saved by the ungrouped Sep between the runs, which is also what makes improvise-gd83's spans come out as [(Q1,2), (None,1), (Q1,2), (None,1)]. abutting_column_groups removes Sep and nothing else, so the two Q1 runs abut. The test asserts that no run is unlabelled before comparing anything: a fixture that grew an ungrouped column back would fail there rather than passing on the easy case.

+

It then compares the TUI's painted group line against the browser's runs exactly, which is the part that makes it an agreement test rather than a second assertion about group_runs.

+

Fault injected 2026-09-21: restoring the name-only comparison in AxisFrame::group_runs fails this with left: [(Some("Q1"), 4)] while the five existing agreement tests all pass — the ungrouped Sep in their fixture is precisely why they cannot see it.

+
+
+ + diff --git a/site/docs/tickets.html b/site/docs/tickets.html new file mode 100644 index 00000000..49398a6a --- /dev/null +++ b/site/docs/tickets.html @@ -0,0 +1,20 @@ + + + + + +Ticket index — improvise docs + + + +
+ +
+

Ticket index

+

The prose on this site cites issue ids like improvise-9c8c. They are this project's own tracker ids, kept in-repo with beads; there is no public issue URL to link to, so this page is what they resolve to. Titles come from .beads/issues.jsonl, a passive export that lags the database — an id it does not carry is listed here without a title and is not linked from the prose.

+

254 ids are cited, of which 254 resolve.

+
IdStatusTitle
improvise-08wbclosed[bug] A refused ClientFrame::Commands keeps the commands before the refusal
improvise-0aiclosedgp-tree: expose a fit/evolve entry point in the library
improvise-0bfclosedInconsistent float-equality tolerance in formula eval
improvise-0hbdeferredMake Sequence bindings transactional at command level
improvise-0o6xclosedA group header first on its axis is never drawn, so a leading collapsed group has no way back
improvise-0x9uclosedA click cannot move the grid selection while an editor is open
improvise-0yq8closedMove ClientState and keys.rs::step into improvise-protocol
improvise-1125closedRow-header width is computed per frame in the widget
improvise-1c9closedExportPrompt mode shows no status-bar hint
improvise-1czclosedexport_csv emits unquoted CSV — commas in values corrupt output
improvise-1eyopenBrowser frontend MVP: end-to-end working demo
improvise-1l9oclosedfurnished_client hand-builds a basis instead of taking the server's
improvise-1wl0closedAxisFrame::first is serde(default), so a frame with no origin claims it leads the axis
improvise-21dclosedEpic: Integrate gp-tree — fit interpretable decision trees as computed measures
improvise-23qdclosedThe Projection::Basis record set is derived twice, and the second copy is the tests' yardstick
improvise-26dmclosedThe fourth length-not-completeness list: cases.len() == 13 in ui/effect.rs
improvise-2avlclosedlayout_version moved outside the effect loop and neither abort arm is tested
improvise-2hiclosedExtract AppMode dispatch methods (mode_label, mode_style, mode_key)
improvise-2lhclosedMigrate name-string comparisons to CategoryKind enum checks
improvise-2oxtclosedThe landing page's casts predate the records columns and show the recorder's shell prompt
improvise-2wmclosedWizardKey effect embeds the import-wizard state machine inside Effect::apply
improvise-2yc2closedDataStore::live_records walks a HashMap -- decide its order before writing the caller
improvise-2zqclosedRecords mode has no inline way to add a category, unlike the + column for measures
improvise-349mclosedformat_md/parse_md is not idempotent when a .improv file has two Category sections with the same name
improvise-35eclosedTemporary adapter: RenderCache::from_app bridge for incremental widget migration
improvise-368hclosed[bug] No client on any transport can ask its host to save
improvise-3gyclosed4.2 Enable GitHub Pages
improvise-3vrclosedvb4 step 1: scaffold ModelState and ViewState types
improvise-3x3uclosedResolve the two unaddressed nullary wire variants before concurrency lands
improvise-3zqclosedRecords-mode Enter/Tab at bottom-right: persist empty record, re-enter editing
improvise-44adclosedColWidths is redundant under the label fold, and the test that would change is gotcha 33(c)'s evidence
improvise-4bwaclosedThree bindings improvise-e24v listed as needing no enrichment do need it
improvise-4dpjclosedThe RecordsNormal-to-Normal arm of settle_normal_mode has zero coverage, and it is live
improvise-4juclosedMake EditOrDrill mode-agnostic; delete EnterEditMode
improvise-4kttclosedThe js job's bundle-freedom is a convention, not a gate
improvise-4pgclosedExtract panel mode keymap template
improvise-4s8closedExtract a gp-tree feature matrix from a View's records
improvise-4ycclosedMalformed numbers in .improv file silently become 0.0
improvise-4zl7closedA fitted tree must route per record and aggregate the results, not route on the aggregate
improvise-56ekclosedSetPanelOpen maps a Panel to a field twice, in two crates
improvise-5j9closedImportPipeline silently skips records missing category values
improvise-5jm5closedModel::evaluate_aggregated takes a list it only tests for emptiness
improvise-5jnaclosedThe browser page does not paint the title bar it is now told about
improvise-5rayclosedcheck-evidence ignores #### headings and every browser-test entry uses them
improvise-5ubsclosedA cyclic-successor all() can silently omit a variant while its doc says it cannot
improvise-60zclosedVirtual views _Records and _Drill should not be persisted to .improv files
improvise-61fclosedAdd invariant test for command registry name mapping
improvise-669bclosedThe harness spec and the page can still disagree about HANDSHAKE_TIMEOUT_MS through a stale assemble
improvise-6c6qclosed[bug] apply_switch relays a refused switch to everyone but the asker
improvise-6kjclosedPest parse errors lack file:line context for users
improvise-6mqclosedIntroduce Storage/VFS abstraction in persistence layer
improvise-6nsdopenA prune-by-write renumbers the grid without moving layout_version -- ADR 001 middle tier
improvise-6osclosedO(n^2) stem deduplication in recompute_formulas
improvise-6sqclosedMerge EditOrDrill and EnterEditAtCursorCmd into EnterEdit{mode}
improvise-6t3closedUnify Commit{Formula,CategoryAdd,ItemAdd} into CommitBuffer(kind)
improvise-711nclosedparse_number_format silently yields 0 decimals for the ,.2f spelling our own docs show
improvise-748fclosedA drill exited with no edits, and view-back, mark the document dirty
improvise-75ldclosedA refused or no-op AddCategory still bumps layout_version and raises dirty
improvise-764closedMigrate import wizard widget and state to consume RenderCache
improvise-78v6closedPR #61 deleted the help-overlay spec and nothing replaced it
improvise-7kvwclosedSetPageSelection renumbers the grid under prune_empty and does not reset the cursor
improvise-7naclosedDecide whether fitted gp-tree models are persisted alongside the formula
improvise-7v62closedChecked does not bind the App it was checked against
improvise-7ybzclosedThe browser category panel has no pointer affordance at all
improvise-7yxclosedClose test gaps flagged by deep review
improvise-864aopenConcurrent merges to main race on one Pages environment and the superseded deploys fail
improvise-8ec3closedimprovise-io's test binary breaches the 2s per-binary rule on some machines, and the pinned figure is not reproducible
improvise-8fyclosedIntroduce simple_effect! macro for single-field setter effects
improvise-8qlrclosedColumn-group run coalescing is derived twice, and the agreement test does not pin the TUI arm
improvise-8shuclosedConstrain the .improv grammar generator's strategy; improvise-io's lib binary is the run's outlier
improvise-8tkclosedZZ / :wq quits even when the save fails, silently discarding the model
improvise-8uj8closedevery_rejection_message_says_the_edit_did_not_land covers five of seven variants
improvise-8uzclosedCursor and scroll are per-client state stored in the shared View
improvise-8xhyclosed[bug] A fit over a categorical feature emits an unevaluable formula
improvise-907closedImport wizard step machine: validate early, allow going back, make the terminal step real
improvise-955nclosed[bug] Host::commands applies a batch partially before rejecting
improvise-9c8cclosedMove the reasoning out of code comments into context/ prose, cap comments at two lines
improvise-9c9oclosedA geometry effect right after a mid-batch SwitchView skips the cursor check entirely
improvise-9cnclosedIntroduce register_cmd! macro to prevent name drift
improvise-9ixclosedSQLite persistence format (alternative save/restore)
improvise-9iyvopenThe route-then-aggregate fold's cost, measured
improvise-9mo9closed[bug] Geometry effects that renumber the grid but never reset the cursor
improvise-9v0jopenview_frame and view_shape are separated by example, not by property
improvise-9x6closedWorker-hosted server bundle (wasm, MessageChannel transport)
improvise-9za9closedDemoting an evidence heading to prose is an unguarded escape hatch
improvise-a2xclosedDerive Serialize/Deserialize for the ViewState field types
improvise-aawclosedReplace CreateAndSwitchView with Binding::Sequence
improvise-ac2zopenA host's none_cats leak into another view's grid: formula columns vanish
improvise-ahdfclosedrepo-map gotchas 36 and 43 still name Sessions::apply as the production door
improvise-anr9closedProject the five grid-frame facts: unblocks H, z, gz and fixes Food/Alice at once
improvise-azhbopenDecide who sets a document's default view now that a switch does not
improvise-bckopenStandalone web MVP: end-to-end offline demo
improvise-bgtfclosedThe landing page is served by the harness and asserted by nothing
improvise-bidqclosedTwo answers to 'is this category virtual', four lines apart in view/types.rs
improvise-bpq2closedClient-side key resolution: what must the projections carry?
improvise-byzclosedrecords_display returns Some("") for out-of-bounds columns instead of None
improvise-c5vclosedImprove records view data entry by making each measure a normal value column (wide format instead of synthetic Value column)
improvise-c6jclosedWizardStep::Done is unreachable from the UI
improvise-cl7yclosedA browser reports "Formula added" for a formula only the host can parse
improvise-cqiclosedExtract improvise-protocol crate (Command, ViewState, reduce_view)
improvise-cqqclosedServer-side projection emission layer + per-client viewport tracking
improvise-cr3closedDOM renderer for browser grid (reads ViewState + render cache)
improvise-ct5closedi on an empty records grid reports 'cannot drill', leaking a pivot-mode concept
improvise-d44sclosedgp_tree::data::preprocess writes to stderr from library code
improvise-djmclosedStandalone main-thread bundle: wasm-client + worker transport
improvise-djnclosedAddCategory effect silently swallows failures
improvise-djooclosedPublish the context/ prose as a documentation site folded into GitHub Pages
improvise-dkqnclosedgp-tree::fit should take one slice of rows, not parallel x/y slices
improvise-dqnclosedCyclePanelFocus does not actually cycle between panels
improvise-drgclosedvb4 step 5: narrow Effect::apply signatures where one-slice
improvise-dz8closedAdd Axis::Filter for drill-fixed coordinates
improvise-e24vclosedSecond tranche of client-side key resolution: ~15 bindings needing no enrichment
improvise-e5pfopenCommand::ChangeMode(Normal) is on the wire and reduce_view folds it without the records resolution
improvise-e8noopenA wide numeric category now refuses a fit that used to ignore it
improvise-edpclosedEstablish ViewModel layer: derived data between RenderCache and widgets
improvise-ehzclosedEnterEditAtCursor effect calls rebuild_layout and builds a CmdContext inside apply
improvise-eteclosedAdd Model/Category getter methods to absorb .unwrap().items/groups chains
improvise-ew0closedvb4 step 3: move UI session fields into ViewState
improvise-fdt6closedA browser cannot add a formula, so FormulaEdit is a room that swallows typing
improvise-fkfgclosedADR-001 b: ModelCommand carries its basis; Applied gains a rejected case
improvise-fujclosedTwo effects read files directly instead of going through Storage
improvise-g0fclosedDecide whether two clients may view different views
improvise-g6s1closedWizardStep::Preview has no characterization fixture
improvise-gcvbclosedThe browser cannot commit a records-mode creating column, though it is now nameable
improvise-gd28openDrill over the wire needs the addressed command form
improvise-gd83closedAxisEntry carries no group-end boundary, so group_of gives an ungrouped item the preceding group
improvise-gdlclosedSetBuffer effect special-cases the "search" buffer name
improvise-gfqclosedImport wizard defers the 'no category accepted' failure to the very last keystroke
improvise-gitclosedReplace SaveAndQuit with Binding::Sequence in keymap
improvise-gv9rclosed[bug] A wire model command never marks the document dirty
improvise-gvnbclosed[bug] A geometry command resets the App cursor but not the session mirrors
improvise-gxiclosedDefine unified reduce_full(&mut App, &Command) -> Vec<Command>
improvise-h04zopenDecide whether UI glyphs move to the protocol, as short_label() did
improvise-h074in_progressTwo edges of App::resolve_view: a worse panic message, and a host-side DeleteView bypass
improvise-h0bclosedEvict wizard and drill_state from ViewState (bootstrap-snapshot hygiene)
improvise-h0ivclosedgroup_runs coalesces by name alone, so two runs under different headers merge
improvise-hkr8closedA click that cannot compose its activation keeps the prefix and drops the payload
improvise-hkt6closedThree small totality gaps found in review
improvise-hmuclosedTAB/Enter on bottom-right of records view should insert record below (TAB→first cell, Enter→same column); o inserts below cursor
improvise-hmwsclosedStorage cannot expose an async seam through Box<dyn Storage>
improvise-hsoeclosedcheck-evidence cannot see a duplicate heading, and three exist
improvise-hwpiclosedThe browser shell fails quietly, and never reads a key
improvise-i2m5closedimprovise-io still carries its own pipe-quoting copy, without the keyword guard
improvise-i34closedOPFS-backed Storage implementation for wasm target
improvise-i5tkclosedNarrow the TUI's GridCache to its viewport -- benchmark first
improvise-igxpclosedThe browser help overlay's rows are asserted as present, never as content
improvise-iioiclosedThe view and formula panels have no pointer affordance while the category panel does
improvise-ijqlclosedcheck-comment-runs.py --check-evidence silently skips the comment-run cap it is named for
improvise-iot6closedrender_csv builds GridLayout::new, so CSV export carries rows the view prunes
improvise-ir7lclosed[bug] A formula cell is offered as editable and silently reverts
improvise-izkclosedgp-tree step 1: emit a fitted Tree as an improvise formula string
improvise-j31xclosedServer hosts several documents: a registry above Sessions
improvise-j7v6closedThe typed-string classification lives in two crates, and its guard test asserts by hand
improvise-jc8closedEvict host-only state from ViewState: flip the transient-keymap Arc
improvise-jjxclosedScrolled grid suppresses first visible row-header label
improvise-jmjzclosedgrid_of evaluates the whole grid three times per frame; compute_col_widths is ~45% of it
improvise-jq8lclosedweb/app.js never calls scrollIntoView, so a selection can sit outside the window
improvise-jytmclosedSplit active_view: document default vs per-client authoritative (improvise-g0f)
improvise-k099closedAppMode::hint_text/label put terminal UI strings on a serde wire enum
improvise-k2i4closedThe two nullary model commands are unaddressed and act on one client's staged state
improvise-k69yclosed[bug] draw_status reads the document's active view, not the client's
improvise-k8hclosedTie MinibufferConfig.buffer_key to the keymap's buffer name
improvise-k8iclosedCSV import silently drops columns when row widths mismatch
improvise-kc7nclosedimprovise-ws-server never checks Origin, so any page a user visits can drive their session
improvise-kdpclosedDuplicated aggregation dispatch block
improvise-kh8closedPhase 4: Landing page (optional)
improvise-kmgsclosedDerive the model half of the Command vocabulary from the EffectKind::Model effects
improvise-kosclosed_Dim and _Index should work as pivot axes outside Records/Drill view; must default to None axis on non-records views including empty models
improvise-kpnropenimprovise-ws-server has no authentication, and the documented sharing path now exposes it
improvise-ksstclosedA gp-tree fit records its seed but not its GA config
improvise-ktv1closedMove the help content into the shared layer so a browser can show help
improvise-l4qxclosed[bug] A text cell is invisible whenever any category is on the None axis
improvise-l8rhclosedThe ambiguous-date heuristic reads American-first, so a European CSV imports with days and months swapped
improvise-laeclosedZZ / :wq with no file path should prompt for a path instead of only erroring
improvise-llttopen[bug] A view-scoped ModelCommand names no view, so it lands on the host's
improvise-lyycclosedcheck-evidence is one-directional: a test citing the evidence file can have no entry
improvise-m91deferred(Stretch) Command log + native undo/replay
improvise-maeclosedTag existing effects as model / view / projection-emitting
improvise-mai9closedA status message written while a minibuffer is open is never drawn
improvise-mcq4openThe frame fingerprint is a 4x regression on the held-key path improvise-748f just optimized
improvise-mzvclosedImport wizard does not pre-check 12-category limit
improvise-n10closedMigrate GridWidget to consume RenderCache + ViewState (not &App)
improvise-n3wtclosedprune_axis_items: keep-slice length invariant is only a debug_assert
improvise-n89zclosedThree Projection mutations have no producer
improvise-ni6kclosedInvalidate the cursor by observing the grid frame, not by a per-effect table
improvise-nrsclosedDuplicated find_item_category helper in eval paths
improvise-nynkclosedadopt_cursor_epoch resets every client's cursor, including clients on a view nothing reshaped
improvise-o1cclosedImport wizard has no way back; Esc discards all field decisions silently
improvise-o31uclosedThe landing page's casts render at a 4.8px font on a phone, and it is now the site root
improvise-oaqclosedRedundant file re-read in ImportJsonHeadless effect
improvise-odbcclosed[bug] bank-info.improv in the repo root does not parse
improvise-ondjclosed[bug] reduce_full/apply_command apply a Command::Model without the basis check
improvise-p1ymclosedProject the page-axis item lists so [ and ] work in a browser
improvise-pcaclosedMigrate panel widgets (category/formula/view/tile_bar) to consume RenderCache
improvise-pczvclosed.improv section headers are a second, unquoted naming rule
improvise-pdvmclosedAddItem/RemoveItem reshape the grid but declare CellData, so layout_version and cursor_epoch disagree
improvise-plcxclosedThe browser harness needs a grouped, multi-level fixture — the demo has neither
improvise-poiaclosedThe records-mode + column has no projected identity
improvise-pubtclosed[bug] display_value_in reads the host's drill edits against a foreign layout
improvise-q08closedimprovise-ws-server binary (tokio + tungstenite session wrapper)
improvise-q0vbclosedPad the import wizard's hint line to the panel width
improvise-qf57closedThe formula-row click test lost its no-editor-opened assertion in the #61 relocation
improvise-qsmpclosedApp::cmd_context's panel-open propagator is untested: C cannot open a closed panel and nothing fails
improvise-r08uclosedIS and WHERE differ on trailing word boundary in formula.pest
improvise-r4loclosedTwo answers to what a typed string means, and they are adjacent
improvise-res9closedDoc site: derive the published set, so an unpublished context/ doc fails the build
improvise-rkeqclosedDecide RenderCache's shape: grid-only wire type vs per-widget sub-caches
improvise-rm4sclosedThe evidence check accepts headings naming tests that never run
improvise-rmlclosedConsolidate Axis display into a method on Axis
improvise-rmvxclosedSix advertised keys do nothing in the browser, and the guard is scoped to one mode
improvise-rucfdeferredRead arbitrary SQLite databases as a data source
improvise-ry89closedcv_regress/cv_classify with k=1 now return plausible numbers instead of NaN
improvise-s13closedDelete mode-entry shim commands; use enter-mode with arg
improvise-s17oclosedreduce_full and apply_command's view-slice arm have no production caller
improvise-seikclosedClient::viewport is kept for a reason web/app.js falsified in the same wave
improvise-shytclosed[bug] Records-mode row order is not stable across processes
improvise-sihrclosedEvery minibuffer mode entered from records mode exits to plain Normal while the layout is still records
improvise-sxsfopencargo test -- --list cannot name the binary, so two crates could share a full test path
improvise-t3uxclosedcv_regress and cv_classify are O(n-squared)
improvise-t6dbclosedgrid_in ignored prune_empty for a non-active view, hiding a staleness bug
improvise-t8sclosedView::axis_of panics on missing category
improvise-taxxclosed[bug] The socket close listener overwrites a precise error with disconnected
improvise-th7mclosedDead F(1) binding: bound to bare 'enter-mode' then immediately rebound
improvise-thpeclosedhelp::compact discriminates a chord by parsing a string it formatted
improvise-tm6openEpic: Standalone static-web deployment via wasm + VFS storage
improvise-tocclosedEffect::changes_mode() is a static flag but ViewBack/ViewForward/StartImportWizard change the mode conditionally
improvise-tom1closedDrill pending edits are invisible to remote clients, and there are now three answers to 'what text is in this cell'
improvise-trd3openHelp pages clamp at both ends; decide whether they should wrap
improvise-tvtclosedRoute mutations through Workbook facade instead of app.workbook.model chains
improvise-ut7pclosedimprovise-gp fit_from_gen0 is a third copy of the GA loop
improvise-vhczclosed[bug] OpfsFs::list disagrees with the other backends on path shape
improvise-vjmsclosedDecide whether a category panel shows the virtual categories
improvise-vvl3closedA headless browser harness: run storage_contract and the shell assertions under Playwright
improvise-vx3closedfit-measure command: fit a tree and add it as a formula
improvise-w3qclosedExtract text-entry keymap template for minibuffer modes
improvise-wd6nclosedA fit may still not split on col.missing or a date sub-feature
improvise-wsawclosedThe browser ignores category_panel_open, so C toggles nothing visible
improvise-x1nsclosedencoded_width undercounts a forced category that has a missing coordinate
improvise-x2cclosedvb4 step 2: move workbook/file_path/dirty into ModelState
improvise-x7cqclosedA kb.pl inconsistency can still reach the published site
improvise-x7swclosedWhich-key hints have no protocol home (the open half of improvise-jb3)
improvise-xf1deferredVerify OpfsFs against Safari — Chromium and Firefox are covered
improvise-xks6closedOne home for the client/server frame envelopes
improvise-xps8closedThe handshake watchdog overwrites a genuine mid-session disconnect eight seconds later
improvise-xwi6closedConcurrent edits: basis check (ADR 001, layer 1 — the correctness floor)
improvise-y4xgclosedCmdContext carries a fourth Panel-to-field mapping, and it is completely unguarded
improvise-ychclosedNull category values become the literal item "null" on import
improvise-ywdclosedWasm compatibility audit and fixes across core/formula/command/io
improvise-z05rclosedCommitExport does not bail on an empty path, and ExportCsv does not abort the batch
improvise-z5egclosedGate improvise-gp in the CI wasm job
improvise-z754closedDocument name, dirty flag and file path have no home a client can read
improvise-zgbclosed.improv formula loader mis-splits category suffix via rfind(" [")
improvise-zkkwclosedThe browser shell is only ever run on Chromium, but improvise-bck's criterion says Firefox too
improvise-zv8tclosedfit_measure never tells preprocess which columns are categories, so a numeric- or date-valued category is unspellable
+
+
+ +