387 Commits

Author SHA1 Message Date
Ed Langley 6ad4ef9bcc Merge pull request #323 from fiddlerwoaroof/bd-updates
CI / Build all bundles (push) Has been cancelled
CI / Test and lint (host) (push) Has been cancelled
CI / Nix build (crate2nix) and the repo-stats pipeline (push) Has been cancelled
CI / wasm32 check (browser bundle crates) (push) Has been cancelled
CI / Browser JS unit tests (node --test) (push) Has been cancelled
CI / Knowledge base consistency (push) Has been cancelled
CI / Lint the harness crate (push) Has been cancelled
CI / Browser harness (OPFS contract + shell) (push) Has been cancelled
CI / Deploy to GitHub Pages (push) Has been cancelled
chore: update issues
2026-09-25 00:09:24 -07:00
Edward Langley 14141771e4 chore: update issues 2026-09-25 00:08:54 -07:00
Edward Langley cf33a5d78a chore: update issues 2026-09-25 00:07:05 -07:00
Ed Langley 9ddaffead1 Merge pull request #313 from fiddlerwoaroof/chore/1xfb-5-1-repo-stats
chore(tooling): scripts/repo-stats — churn, coupling and doc-growth report (improvise-1xfb.5.1)
2026-09-25 00:03:18 -07:00
Ed Langley e8ea316ace Merge pull request #317 from fiddlerwoaroof/feat/felo6-group-parents
feat: groups nest inside a category and start collapsed; views persist expanded(N) (improvise-felo.6)
2026-09-24 23:39:28 -07:00
Claude (agent-stats) d6adb9e0e3 docs(tooling): the green run could not tell sparse from padded; record the rivals that now can (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 23:07:25 -07:00
Claude (agent-stats) 4ef847ce30 test(tooling): the repo-stats suite tells a sparse model from a padded one, and --out refuses a foreign CSV (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 23:07:25 -07:00
Claude (agent-hier) 44718e8ab3 Merge commit 'c91cc055c8ad81afd8f055b71c74af6819020637' into feat/felo6-group-parents
# Conflicts:
#	context/test-evidence/formula-and-io.md
2026-09-24 22:42:49 -07:00
Claude (agent-hier) 8d3230e685 fix(core): an expanded fold's depth is non-zero by type, so every fold the API builds reads back (improvise-felo.6)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:29:02 -07:00
Claude (agent-stats) 3564410952 docs(tooling): repo-stats runs on folder import; the interim steps are gone (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:06:25 -07:00
Claude (agent-stats) 14a91c9cc2 feat(tooling): repo-stats imports its directory with improvise import <dir> (improvise-1xfb.5.1)
Folder import (improvise-lpzu, #314) keeps sparse records and makes a
view per file, so the category padding, the explicit file list and the
generated per-file views are gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:05:49 -07:00
Claude (agent-stats) 9639c577a4 Merge commit 'c91cc055c8ad81afd8f055b71c74af6819020637' into chore/1xfb-5-1-repo-stats 2026-09-24 22:02:45 -07:00
Ed Langley c91cc055c8 Merge pull request #314 from fiddlerwoaroof/feat/lpzu-import-folder
feat(import): import a folder of CSVs as one sparse model (improvise-lpzu)
2026-09-24 22:02:16 -07:00
Claude (agent-stats) e32a4de406 docs(tooling): cite each headless gap's ticket, and record the empty window and the tools package (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:01:08 -07:00
Claude (agent-stats) 2dcf001b87 ci: keep code-maat and its JRE out of the ci shell; nix-build takes them from .#repo-stats-tools (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:59:36 -07:00
Claude (agent-stats) 024543c72c fix(tooling): repo-stats survives a window of empty commits, and the fixture covers renames, the root crate and a refused script (improvise-1xfb.5.1)
churn.csv now has a fixed header. When no file changed in the window,
no model is built and the report says so. docs/*.md, README.md and
RELEASING.md count as prose, and .codex/ as agent docs. The suite no
longer builds improvise itself, because checkers.sh scrubs the linker's
environment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:56:54 -07:00
Claude (agent-hier) e09ec75226 docs: group hierarchies and fold lines, with the evidence for each rival (improvise-felo.6)
The harness fixtures open their groups explicitly now that groups start closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:54:23 -07:00
Claude (agent-lpzu) c43ee72402 fix(import): a header-only or measureless file keeps its identity and a view of its own (improvise-lpzu, improvise-8jk6.13, improvise-8jk6.12)
ImportPipeline::from_merge seeds the File items and every file's columns
from Merged::sources, so a header-only CSV keeps its File item and its
columns (unseen columns are proposed as Labels, as an all-null column
is). A file with no measure column gets a records view filtered to it
instead of paging onto another file's hidden measure. Four per-file-view
rules the review found untested now each have a test that fails when
the rule is broken.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:47:11 -07:00
Claude (agent-hier) e497aef110 feat: nested group runs reach the wire, and views persist expanded(N) and collapsed folds (improvise-felo.6)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:43:55 -07:00
Claude (agent-lpzu) 005b7a769a feat(import): a folder of one CSV keeps its File category and view (improvise-lpzu)
The user's ruling on PR #314: a folder always merges, so even a single
CSV in it records its file identity. A CSV named directly still imports
alone. The "(none)" question is answered as no position, per the user.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:36:00 -07:00
Claude (agent-hier) 8dad4b7476 feat(core): groups nest within a category and start collapsed (improvise-felo.6)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 21:22:16 -07:00
Claude (agent-lpzu) 4e68372c25 Merge commit 'b639266b0d8852b106709b19162a0dadf1d114f0' into feat/lpzu-import-folder 2026-09-24 20:59:10 -07:00
Claude (agent-lpzu) ee0ff83a58 docs(import): folder import, sparse records and per-file views (improvise-lpzu, improvise-8jk6.2)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:58:58 -07:00
Claude (agent-stats) 77b6f2dbba Merge commit 'b639266b0d8852b106709b19162a0dadf1d114f0' into chore/1xfb-5-1-repo-stats
# Conflicts:
#	context/test-evidence/tooling.md
2026-09-24 20:55:47 -07:00
Ed Langley b639266b0d Merge pull request #310 from fiddlerwoaroof/chore/1xfb3-agent-author
chore(agents): author agent commits as Claude (<actor>) (improvise-1xfb.3)
2026-09-24 20:54:46 -07:00
Claude (agent-stats) 6709a4e84a docs(tooling): the repo-stats pipeline, its failure split and the headless gaps it found (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:52:43 -07:00
Ed Langley 3ad7d9103b Merge pull request #312 from fiddlerwoaroof/chore/1xfb4-untrack-site
chore(site): stop tracking generated site/ (improvise-1xfb.4)
2026-09-24 20:52:27 -07:00
Claude (agent-lpzu) 736e9e5e18 test(import): pin where an absent coordinate draws and how a saved file view projects (improvise-8jk6.2, improvise-lpzu)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:51:40 -07:00
Claude (agent-stats) acd724079a ci: run the repo-stats suite and a report-only push step in nix-build (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:51:04 -07:00
Claude (agent-lpzu) bb505f8faf feat(import): give each merged file a view filtered to it (improvise-lpzu)
A CSV merge now records each file's columns, and the build adds one view
per file: File on Filter at that file's item, its own categories seated
as a fresh import seats them, the rest on None, and other files'
measures hidden. The merged Default view stays the initial view. The
headless and wizard paths both carry the sources.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:48:16 -07:00
Claude (agent-stats) 20335ee616 test(tooling): the repo-stats suite runs the whole pipeline over its fixture (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:46:21 -07:00
Ed Langley a4feee44b8 Merge pull request #311 from fiddlerwoaroof/docs/record-303-307
docs(context): record #303–#307 in the repo map and kb
2026-09-24 20:44:34 -07:00
Claude (agent-stats) 2cd246780b feat(tooling): repo-stats computes its churn tables in improvise (improvise-1xfb.5.1)
code-maat's CSVs and a per-file churn CSV go into one directory, one
sparse model is imported from it, and a checked-in improvise script
builds and exports the views the report prints.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:43:53 -07:00
Claude (agent-lpzu) e65d87fda9 feat(import): import a folder of CSVs as one sparse model (improvise-lpzu, improvise-8jk6.2)
A folder argument expands to its *.csv documents through Storage::list,
and a record with no value for a category keeps its cell without that
coordinate instead of being skipped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:43:31 -07:00
Claude (agent-1xfb3) 0539fb37ac docs(agents): a cherry-pick keeps its source's author, and the check is per own commit (improvise-1xfb.3)
The wrapper's contract drops cherry-pick, because git ignores
GIT_AUTHOR_NAME there, and rule 8's cherry-pick -x should keep who wrote
the commit. A new case asserts it. The pre-push check now asks that every
commit you made names your actor, since a stacked or cherry-picked commit
keeps its own author. Answers the Codex P2 and the review on PR #310.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:39:18 -07:00
Claude (agent-stats) 73a8556937 chore(tooling): scripts/repo-stats and its fixture suite (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:28:12 -07:00
Claude (agent-stats) 3ae7650d8a chore(nix): pin code-maat 1.0.4 and a JRE in the dev and ci shells (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:24:45 -07:00
Claude (agent-1xfb4) 676347674a chore(site): stop tracking generated site/ (improvise-1xfb.4)
site/ is web/assemble.sh's default output, and site/docs is
scripts/build-doc-site.mjs's. 27 generated pages under site/docs were
committed by accident and went stale. Nothing reads the tracked copy:
CI's build job assembles with `nix build .#site` into the store, the
harness assembles into harness/serve/, and build-web-release.sh
assembles into a temporary stage.

Remove site/ from the index, ignore /site/, and say so in
context/code/doc-site.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:22:23 -07:00
Claude (agent-1xfb3) 256057b7f8 docs(agents): agents author their commits as Claude (<BEADS_ACTOR>) (improvise-1xfb.3)
The rule sits beside the claim in agent-rules.md, and the brief checklist
in wave/SKILL.md ties the actor to it. The evidence entry records why the
wrapper sets GIT_AUTHOR_NAME and not user.name, and its two rivals.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:19:00 -07:00
Edward Langley c0368123fd docs(context): drop the grid_width location fact that kb-test counts against
scripts/kb-test.sh case 18 expects exactly 35 location/3 facts; the
location is kept in the repo map and in decision(scroll_measured_where_drawn).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:18:45 -07:00
Claude (agent-1xfb3) eff92f3e3c chore(tooling): add scripts/agent-git, which authors a commit as Claude ($BEADS_ACTOR) (improvise-1xfb.3)
Sets GIT_AUTHOR_NAME for one git command and refuses when BEADS_ACTOR
is unset or empty. Its suite runs in the kb job's checkers.sh row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:18:17 -07:00
Edward Langley d5fd1ec824 docs(context): record #303-#307 in the kb and answer improvise-at5k
Facts for gtn7.1, cgqt/9bit/4klp, felo.1 and the merge-only history
rule; the two doctrines that read as licensing a rewrite now forbid it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:16:36 -07:00
Edward Langley c673fa3b61 docs(context): record #303, #304 and #307 in the repo map
The cap is 24 with one home, the grid width is one derivation shared by
draw and keypress, and the import wizard asks about colliding records.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:14:48 -07:00
Ed Langley 1719f2cfad Merge pull request #307 from fiddlerwoaroof/feat/felo1-max-categories-24
Raise MAX_CATEGORIES from 12 to 24, with one home (improvise-felo.1)
2026-09-24 20:10:50 -07:00
Edward Langley 74e460be6a docs: quote the derived label-reload assertion; place the sk32 note (improvise-felo.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 20:04:39 -07:00
Ed Langley 57d385e155 Merge pull request #306 from fiddlerwoaroof/wave/1xfb-2-conventional-commits
docs(agents): default to a new conventional commit; amend only an unpushed HEAD (improvise-1xfb.2)
2026-09-24 19:45:01 -07:00
Edward Langley bc713a2590 docs(tui): re-read the wizard residual against a doubled cap (improvise-felo.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:43:46 -07:00
Edward Langley cc81e71a85 docs: describe the derived cap and cite the coupling beads (improvise-felo.1)
Review of #307 at 7b9e868b: the duplicate-step evidence and argument still
said 12, tui.md bounded a collision tuple by twelve, and the coupling
inventory named no beads. The page-bar entry now notes its unclipped
set_string can overwrite the right border.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:42:19 -07:00
Edward Langley 94260d58b8 docs: list every Conventional Commits type rule 8 allows (improvise-1xfb.2)
Answers the Codex P2 on #306: the type list read as exhaustive and left out
refactor, perf and style, all in use on main. Ad-hoc prefixes become a
standard type with a scope, and the user's quote is marked [sic].

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:35:56 -07:00
Edward Langley 7b9e868bd2 test+docs: derive #304's cap fixtures from MAX_CATEGORIES; record the coupling found (improvise-felo.1)
The duplicate step's cap check already reads the one constant after the
merge; its tests filled 12 categories and now fill to the cap. An
injected literal-12 check fails keeping_duplicates_separate_is_refused_at_the_category_cap.
core.md section 9 gains the inventory of what degrades as categories grow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:26:41 -07:00
Edward Langley bd957fd619 Merge commit '4f38a35e945f54e1a0bbde3155ba518d4f298b71' into feat/felo1-max-categories-24 2026-09-24 19:22:37 -07:00
Ed Langley 4f38a35e94 Merge pull request #304 from fiddlerwoaroof/feat/cgqt-import-collision-step
Import asks what to do with records that share every coordinate (improvise-cgqt)
2026-09-24 19:22:07 -07:00
Edward Langley d1b3258aeb docs: default to a new conventional commit; amend only an unpushed HEAD (improvise-1xfb.2)
Narrows what #305 allowed to the user's ruling: reordering, folding and
tidying WIPs are out, and commit messages follow Conventional Commits with
the ticket id at the end of the subject.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:21:15 -07:00
Edward Langley 434c1206df Merge remote-tracking branch 'gh/main' into feat/felo1-max-categories-24 2026-09-24 19:19:45 -07:00
Edward Langley 7d1f90dbf9 feat(core): raise MAX_CATEGORIES from 12 to 24, with one home (improvise-felo.1)
The wizard's private copy of the limit is gone; it reads the model's
now-public constant. Tests that fill to the cap derive it. Why 12 was
chosen and what 24 costs: context/code/core.md section 9.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:19:39 -07:00
Ed Langley 6427312da2 Merge pull request #305 from fiddlerwoaroof/docs/1xfb-1-merge-only
docs: reconcile diverging history only by merge (improvise-1xfb.1)
2026-09-24 19:15:22 -07:00
Edward Langley 22cbfa679e Wizard text is clipped to the panel and measured in cells; the list header says what is shown (review B1, B2, Codex P2 on #304)
Every wizard write goes through Panel, which clips to the panel and saturates
its height arithmetic, so no step panics on a small terminal. The bottom row
truncates in cells, so a wrapped message with a multibyte character is drawn
whole. A scrolling list's range is named on its header row, where no item can
cover it. Tests: a multibyte message at every width, wrap bytes vs cells, a
no-panic sweep of every step at 20-100 x 3-20, the first item filling the list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:11:42 -07:00
Edward Langley 9b9b1efaab docs: name what Codex caught in the #145 paragraph (improvise-1xfb.1)
Review nit on #305: after the new recipe paragraphs, "ChatGPT Codex caught
it" read as the cherry-pick. It now starts its own paragraph and names the
#145 base problem.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:08:48 -07:00
Edward Langley 1fb1bd00c4 docs: the line is the push (improvise-79h5)
The user settled the second half of improvise-79h5: "I don't mind
amending on a branch that hasn't been pushed, but don't do anything that
requires a force push". Rule 8, AGENTS.md's top line, "Commit early",
cloud-agents and the code-review summary now say so. Unpushed commits may
be tidied but never moved onto another base; nothing pushed is rewritten.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:07:34 -07:00
Edward Langley 46e8792aca docs: put the merge first in the wrong-base recipe (improvise-1xfb.1)
Review of #305 at ac2efdc8: "no merge can do it" is false when the
foreign commits belong to an open PR that will land as a merge commit.
Stack on it, or wait for it and merge a newly pinned gh/main. cherry-pick
-x is kept only for commits that will never reach main as they are.
Rule 8 now says to re-pin before merging a newer main (improvise-79h5,
first half), and context/cloud-agents.md states the new rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:00:30 -07:00
Edward Langley f00ed3e979 A collision is a cell two records write, and every summed cell is reported (improvise-9bit, improvise-4klp)
Records with disjoint measures on one tuple no longer count as colliding, so
the Refuse default does not refuse a sparse-measure CSV. Only the records
that share a cell are listed and numbered. The Sum confirmation lists every
summed cell rather than three and a count. Adds the date-derived name test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 19:00:29 -07:00
Edward Langley ac2efdc8a4 docs: reconcile diverging history only by merge (improvise-1xfb.1)
agent-rules gains rule 8, which forbids rebase, squash in any form, reset
that drops commits, amending a pushed commit and force-push. Rule 7 keeps
the SHA pinning for merge, merge-base and diff. The user's reason is quoted
in the rule: the merge structure of git history is a causal record.

AGENTS.md: the fix-after-merge recipe is now a merge. The wrong-base
rebuild stays the one sanctioned cherry-pick, with -x, and the text says
why no merge can do it. Merge PRs with a merge commit, never squash or
rebase. Also bounds test loops that walk a state machine until a step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 18:50:43 -07:00
Edward Langley 92233cff14 Merge remote-tracking branch 'gh/main' into feat/cgqt-import-collision-step
# Conflicts:
#	context/test-evidence/tui.md
2026-09-24 18:43:40 -07:00
Edward Langley 1a2952058d Import asks what to do with records that share every coordinate (improvise-cgqt)
When source records collide on every coordinate, the import wizard now asks:
sum them, keep them separate with a generated occurrence category, or go
back. Collisions are found before the build by the same placement pass the
build uses, and are carried to the TUI as structured data (WizardCache) and
listed in full, scrollable. At the 12-category cap, "keep separate" is shown
disabled with its reason and remedy.

Every wizard warning and message now wraps to the panel instead of being
truncated, and long lists scroll.

Headless paths no longer sum silently: `import --no-wizard` takes
`--on-duplicate refuse|sum|separate` (default refuse), and `:import-json`
refuses and points at `:import`.

Reasoning: context/code/formula-and-io.md, "Colliding records are a
question, not a default"; context/code/tui.md; context/code/cli.md §11.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 18:43:19 -07:00
Ed Langley bb01cd1957 Merge pull request #303 from fiddlerwoaroof/fix/gtn7-1-tui-hscroll
fix(tui): grid and tile bar scroll by the width they are drawn in (improvise-gtn7.1)
2026-09-24 18:41:34 -07:00
Edward Langley 241cc5f384 fix(tui): the grid and the tile bar scroll by the width they are drawn in (improvise-gtn7.1)
Grid: App::cmd_context counted visible_cols from term_width while
draw_content gave the grid term_width minus the 32-column side panel, so
with any panel open the cursor was scrolled past the last drawn column.
viewport::grid_width, reached through App::grid_width, is now the one
answer: draw_content splits on it and the keypress path counts on it.

Tile bar: the scroll measured from the end of the prefix while the draw
began one column later, two once scrolled (the left arrow), so the
selected tile could be dropped. tiles_left(start) is now where both the
scroll (first_tile) and the draw begin.

The grid defect dates from b5418f2e; it became the launch state when
50e4a372 opened the category panel by default, which is why it read as
a regression. Mixed-width columns still lose the cursor with no panel
open: improvise-gtn7.2. Reasoning: context/code/tui.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 18:32:47 -07:00
Edward Langley b945a7f881 test: the TUI cursor stays drawn beside a side panel and on the tile bar (improvise-gtn7.1)
Both fail on gh/main: the grid at column 4 with any side panel open, the
tile bar at width 42.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 18:25:40 -07:00
Ed Langley ae5cf40824 Merge pull request #302 from fiddlerwoaroof/docs/adr007-bookkeeping
docs: record ADR 007 in the repo map and kb
2026-09-24 17:56:38 -07:00
Edward Langley 95d4814bd1 docs: record ADR 007 in the repo map and kb
Add ADR 007 (stable identities, accepted 2026-09-25 on PR #210) and the
missing ADR 004 to the repo map's context-docs list, noting that no ADR
005/006 file exists. Add decision(improvise-sboz, ...) for clause 12:
a formula edit's display names are mapped to ids when the editor opens.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 17:49:20 -07:00
Ed Langley 2212d9b4c0 Merge pull request #210 from fiddlerwoaroof/docs/sboz-stable-identities
ADR 007 (proposed): stable identities for categories, items, formulas and views (improvise-sboz)
2026-09-24 17:28:35 -07:00
Edward Langley 69e8549124 ADR 007: accepted (improvise-sboz)
The user approved revision 8 on PR #210 (owner comment 5824468104,
2026-09-25 00:17:10Z, "looks good, approved"). Status-only change on top
of fcb8f539, which the delta review passed (issuecomment-5824516651).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 17:22:33 -07:00
Edward Langley fcb8f5397a ADR 007 revision 8: clause 12, capture-at-open for formula edits; fix stale facts (improvise-sboz, improvise-8vs2)
The adversarial review of 4830fa30 found clause 6 and the km71 section
contradicted each other on a concurrent rename (B1, also Codex's P1 on
29ad14f4). Clause 12, the ADR's own reading, fixes the display-name
mapping when the formula editor opens, as OpenedCell does for cells, and
requires a regression test. fdk1 (#221) and wpxr (#281) have landed, and
CellId::Coordinate exists. Status stays proposed pending the user's
confirmation of revision 8.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:51:52 -07:00
Edward Langley 4830fa3069 Merge commit 'ad217cc12a48ed7e3900aa02f94aae1a6fe4c857' into docs/sboz-stable-identities 2026-09-24 15:38:24 -07:00
Ed Langley ad217cc12a Merge pull request #301 from fiddlerwoaroof/fix/pr-reviews-argmax
pr-reviews: hand jq its JSON on stdin, not argv (improvise-5jxt)
2026-09-24 15:37:55 -07:00
Ed Langley d6cab6350f Merge pull request #300 from fiddlerwoaroof/docs/bookkeeping-2026-09-24
docs: bookkeeping for PRs #229-#299 (repo-map, AGENTS.md, wave rules, kb)
2026-09-24 15:33:43 -07:00
Edward Langley c6b53eaa77 pr-reviews: hand jq its JSON on stdin, not argv (improvise-5jxt)
On #210 the review and comment JSON outgrew ARG_MAX, and
`jq -n --argjson ...` failed to exec with exit 126 before printing
anything. A `combine` helper now pipes the three documents to `jq -s`
through the printf builtin, on both the fixture path and the live path.

The new suite cases put about 2 MB on each surface, so a fix that moves
only one document off argv still fails. Both rivals fail 5 of 30: the
pre-fix script, and a partial fix that keeps the issue comments on argv.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:22:00 -07:00
Edward Langley d25efa2a0f docs: #300 review fixes — fdk1 clause, stacked-PR rule, run: shell scope, §7 heuristic, fifteen not sixteen
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:19:59 -07:00
Edward Langley 56fb9abb88 ADR 007 revision 7: hand-written ids are canonical (improvise-sboz)
Answer 29: the gensym suffix only ever goes on a generated id; an id
read from a file is never renamed, and two identical hand-written ids
refuse the load (answers 11, 22, 29). No question open; still proposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:19:32 -07:00
Edward Langley 9d0b660d56 Merge commit 'c1772ca9b4113a6824768afb9aa30610dc9d1265' into docs/sboz-stable-identities 2026-09-24 15:06:23 -07:00
Edward Langley b89bb49b78 ADR 007 revision 6: fold in the fifth round; no question left open (improvise-sboz)
Answers 26-28: casefold is full Unicode case folding; the suffix is a
plain number; any colliding id, on import, in a session or across kinds,
gets a suffix from a gensym-style increment-only document counter.
Status stays proposed until the user's Approved.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:06:23 -07:00
Edward Langley 78e2676451 docs: bookkeeping for PRs #229-#299 (2026-09-24)
AGENTS.md and the wave skill: Copilot reviews again and is answered like
Codex; check baseRefName before merging a stacked PR; land a batch as one
integration PR; never resolve with --ours/--theirs, diff against both
parents; a clean merge of one test file can still fail together; CI takes
its tools from the .#ci dev shell; watch open ADR PRs; fmt-workspaces must
not be piped; Playwright serves the assembled harness/serve/.

repo-map: gotchas 1, 3, 18, 19, 38, 42, 44 and 47 brought up to gh/main
(view-named record resolution and Grids, nested group runs and remark,
the fold's spanned narrowing, uneditable/NonItem, the browser editor's
opened cell, NEEDS_THE_HOST, notify/settle_status); recounted the wire
vocabulary (28 model effects, 26 ModelCommand variants); File Inventory
line counts refreshed at c1772ca9; the median-binary ratio is degenerate.

kb.pl, through scripts/kb: 19 new facts; five stale ones replaced (the
checker's bash pin, the nix-shebang premise, the Copilot lapse, the
dev-shell environment, the fmt pipe).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 15:04:19 -07:00
Ed Langley c1772ca9b4 Merge pull request #281 from fiddlerwoaroof/fix/wpxr-empty-record-view
improvise-wpxr: an empty record resolves against the view that drew it
2026-09-24 14:19:48 -07:00
Edward Langley ffc7d0aecc Merge gh/main 08ad7b81 into fix/wpxr-empty-record-view
Every write the merged PRs added now names a view, chosen deliberately:

- The keyboard commit names OpenedCell::view, captured when the editor
  opened, as EditTarget does for the mouse (context/code/clients.md).
- RenameRecordCoordinate names no view. Its record must hold cells, so it
  resolves in the store alone: NamedRecord::Stored, as does the rename
  destination records_at_stake adds. named_records replaces cells_named as
  the exhaustive list records_named is read off.
- #252/#265's non_item_refusal moves into Grids::unwritable. The strict view
  lookup (a missing view is RecordGone) and the Grids memo are kept.
- Test fixtures name a view their document has: Default for host() and
  host_on_a_fold(), the records view for the rename pair, and the viewing
  client's own view in browser_round_trip.

Evidence files: both sides kept, every heading of both parents verified
present. Test names: the merge holds every test of both parents.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:53:03 -07:00
Edward Langley e044c9683a Merge commit 'df22ce08' into docs/sboz-stable-identities 2026-09-24 12:47:18 -07:00
Edward Langley 8cf1443d68 ADR 007 revision 5: fold in the fourth round; add clause 10, the file version (improvise-sboz)
Answers 21-25: spaces collapse to one _, a duplicate slug gets a numeric
suffix, a file has ids on everything or nothing; ids are globally unique;
virtual ids are _index/_dim/_measure and no other id starts with _; an
old file's undeclared item is declared at load; the version stays an
explicit first-line literal and a second literal marks the new format.
Q18(b) and a new Q22 remain open. Status stays proposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:47:00 -07:00
Ed Langley df22ce085a Merge branch 'main' into docs/sboz-stable-identities 2026-09-24 12:46:26 -07:00
Edward Langley 595111ff95 Merge commit '08ad7b8136baa8d1d26f532ab18a5f78bdfefb61' into docs/sboz-stable-identities 2026-09-24 12:43:42 -07:00
Ed Langley 08ad7b8136 Merge pull request #299 from fiddlerwoaroof/wave/integration-editor-2026-09-24
Integration: the browser-editor stack (#295, #221, #251, #274, #278)
2026-09-24 12:40:03 -07:00
Edward Langley 8dd39fd5ba Merge commit '37ff61f9de10b8df679b80ee4002061f117e2021' into wave/integration-editor-2026-09-24 2026-09-24 12:17:55 -07:00
Edward Langley 32c555767a Merge #274 (bce496b7, improvise-8xq0 editor commits the opened record) into the editor integration branch
Resolutions:
- resolve.rs: dropped #295's record_at_cursor in favour of 8xq0's
  captured-cell path (commit_value / opened_at_cursor); nothing references
  it any more. declined_at keeps #292's NonItem arm. #274's two new
  exhaustive matches (opened_at_cursor, uncaptured) gain a NonItem arm that
  answers None; refused_here already declines a commit over a NonItem cell
  when nothing is captured, so uncaptured never reaches it.
- client.rs: kept main's status_seen/status_owner beside #274's
  Opened/OpenedCell/OpenedTarget.
- lockstep.rs: #274 made the same sent -> sent_to_host rename #295 did;
  kept this branch's sent_or_upstream (main already owns sent_to_host).
- context/test-evidence/clients.md: kept both sides. Because #274 and #295
  both carry the 7mro commits, the three 7mro entries
  (keys_sent_before_a_scroll_is_answered_agree_with_the_tui,
  a_held_cell_is_read_at_its_position_and_not_its_cache_index,
  resolve::tests::a_cursor_on_an_empty_grid_waits_for_nothing) arrived
  twice. The first two copies were byte-identical; #274's copy of the third
  also carries its "Amending the entry ... for #274's capture" paragraph,
  so #274's copies are kept and #295's are removed. Heading union against
  both parents: nothing missing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:17:49 -07:00
Edward Langley a7ec482552 Merge #251 (3f5f90a5, improvise-20ek/milr rename races) into the editor integration branch
Resolutions:
- client.rs imports: EditorSession (main) and ModelCommand (#251).
- client.rs refused(): #251 keeps the popped frame to learn which record it
  renamed; main (#279/3kfc) calls settle_sent() after the pop. Both kept, in
  the order pop, settle_sent, replay, then #251's Opened::Retry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:14:44 -07:00
Edward Langley 5e8e838f23 Merge #221 (c7b5b825, improvise-fdk1 coordinate rename) into the editor integration branch
Resolutions:
- CellId: #221 gives Coordinate fields { record, category }; main (#292)
  adds NonItem { category, item }. Every exhaustive match now names both
  (render.rs, core.rs x2, project.rs, resolve.rs x3, wasm-client lib.rs and
  render.rs); project.rs's basis arm keeps #221's Coordinate-carries-a-record.
- resolve.rs edit_or_drill: #221 drops the "renaming a coordinate" decline
  (Coordinate opens an editor); main's NonItem decline arm is kept.
  coordinate_at_cursor's None arm gains NonItem. reopened_at keeps #295's
  declined_at shape; declined_at's editable arm names Coordinate { .. }.
- client.rs: kept main's status_seen/status_owner and #221's opened +
  OpenedCoordinate; every settle site calls both settle_opened and
  settle_status. Main's new rebase() also gets settle_opened, since it
  rebuilds the view exactly as apply() does.
- view/mod.rs: re-export both GridEval (main) and record_row (#221).
- context/code/command.md and context/test-evidence/clients.md: both sides
  appended independent sections; kept both.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:13:42 -07:00
Edward Langley 2fbb43b9ee Merge #295 (65d6a3e1, improvise-7mro projected reads) into the editor integration branch
Resolutions against gh/main 8d870c15:
- client.rs: kept main's settle_status (1x9h) and #295's
  settle_declined_editor (5m9z) side by side.
- resolve.rs reopened_at: took #295's declined_at shape and moved #292's
  CellId::NonItem arm into declined_at, which reopened_at consults before
  any seed choice, so a NonItem landing still re-opens nothing.
- lockstep.rs: both sides appended at EOF; kept main's tail then #295's.
  Both defined sent_to_host with different semantics (main's panics on a
  non-local key, #295's lets an upstream key reach a silent host), so
  #295's copy is renamed sent_or_upstream with its three callers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 12:10:53 -07:00
Ed Langley 8d870c15d0 Merge pull request #298 from fiddlerwoaroof/wave/integration-2026-09-24
Integration wave 2026-09-24: 19 approved PRs, merged once
2026-09-24 12:06:38 -07:00
Edward Langley 46896a7512 Merge PR #250 (3e417729) into the 2026-09-24 integration wave
Conflicts with #294 in render.rs and project.rs resolved as tested:
#250's side of every hunk (levelled GroupEnd { level }, groups()/group_of,
axis_frame's carried headers, the try_from level check), then #294's shape
(Item { tuple, .. }, FrameEntry::item(..), the DataItem { tuple, collapsed }
arm, and GroupEnd { .. } in folds). Evidence EOF appends keep both sides.
2026-09-24 02:49:32 -07:00
Edward Langley 586264d9c5 Merge PR #267 (eded3179) into the 2026-09-24 integration wave 2026-09-24 02:48:15 -07:00
Edward Langley 78e9d930b9 Merge PR #261 (b32a7690) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/command.md
2026-09-24 02:48:15 -07:00
Edward Langley 903d824e38 Merge PR #254 (3c513540) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/command.md
#	context/test-evidence/core.md
#	crates/improvise-command/src/ui/app.rs
2026-09-24 02:48:03 -07:00
Edward Langley f39a95709e Merge PR #279 (044e8d2d) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/clients.md
2026-09-24 02:47:40 -07:00
Edward Langley 783f92cdce Merge PR #266 (d0e36142) into the 2026-09-24 integration wave
# Conflicts:
#	crates/improvise-wasm-client/tests/lockstep.rs
2026-09-24 02:47:20 -07:00
Edward Langley 52946bd3e3 Merge PR #246 (37dbf02e) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
2026-09-24 02:46:58 -07:00
Edward Langley 88598c6f7e Merge PR #292 (3f98a17c) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/command.md
#	crates/improvise-command/src/ui/app.rs
#	tests/shipped_documents_load.rs
2026-09-24 02:46:37 -07:00
Edward Langley 1efb774f9f Merge PR #294 (088d469f) into the 2026-09-24 integration wave
# Conflicts:
#	crates/improvise-wasm-client/tests/lockstep.rs
2026-09-24 02:45:54 -07:00
Edward Langley d80392aa58 Merge PR #293 (ff7ea6d7) into the 2026-09-24 integration wave 2026-09-24 02:45:17 -07:00
Edward Langley 58b8e7902e Merge PR #289 (8f218644) into the 2026-09-24 integration wave 2026-09-24 02:44:56 -07:00
Edward Langley 90ea09360f Merge PR #222 (32b64f18) into the 2026-09-24 integration wave 2026-09-24 02:44:48 -07:00
Edward Langley 789b505547 Merge PR #238 (a6b5acc2) into the 2026-09-24 integration wave 2026-09-24 02:44:40 -07:00
Edward Langley 0185ed1e4f Merge PR #270 (fa080f48) into the 2026-09-24 integration wave 2026-09-24 02:44:29 -07:00
Edward Langley bc7e5f9bb3 Merge PR #290 (4fe8c0dc) into the 2026-09-24 integration wave 2026-09-24 02:44:20 -07:00
Edward Langley 5b73e67caf Merge PR #223 (e1453797) into the 2026-09-24 integration wave 2026-09-24 02:44:11 -07:00
Edward Langley 24778cf284 Merge PR #242 (1c6600c3) into the 2026-09-24 integration wave
# Conflicts:
#	context/test-evidence/command.md
2026-09-24 02:43:51 -07:00
Edward Langley 670b726402 Merge PR #245 (941eb0fd) into the 2026-09-24 integration wave 2026-09-24 02:43:18 -07:00
Edward Langley 224cd65014 Merge PR #228 (9a0d1ba7) into the 2026-09-24 integration wave 2026-09-24 02:43:18 -07:00
Ed Langley fb3358a0ff Merge pull request #291 from fiddlerwoaroof/fold-memo/metf-opbo
Land #258 268 to main (merged into stale bases)
2026-09-24 02:40:11 -07:00
Edward Langley 3f98a17cd5 Merge gh/main (220abc61) into fix/gk51-non-item-record-projection
resolve.rs, `reopened_at`: #227/#272 made the editable arm a
`match writes { true => awaiting_seed, false => opened_on(state, ..) }`.
Kept that new shape and re-applied this branch's declining
`NonItem => SetStatus(non_item_refusal(..))` arm ahead of it, so a
non-item cell still reopens no editor whether or not the frame writes.
`uneditable`'s NonItem arm merged cleanly and is unchanged. Both guard
tests pass; the rival (NonItem folded into the editable arm) fails
an_advance_in_the_browser_re_opens_no_editor_on_a_non_item_cell.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:24:15 -07:00
Edward Langley 4fe8c0dc7b Merge gh/main (220abc61) into fix/cota-assume-unchanged
context/test-evidence/tooling.md: both sides appended at the same point
(this branch's cota entry; gh/main's checkers-test and later entries).
Both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:21:29 -07:00
Edward Langley 37ff61f9de Merge #274's head (bce496b7, gh/main 220abc61) into fix/joed-waiting-commit-keeps-text
Conflicts, all in resolve.rs:
- commit_value and rename_coordinate: joed's `Step::Local(commands)`
  return, plus #227's success-path `cleared(&edit_mode)` and
  `advanced(.., writes)`. The waiting, `Gone` and 3h9w declines all
  return (as `Step::Declined`) before any clear. That is the joed
  ordering constraint.
- `STILL_WAITING` and `cleared` kept side by side.

Fault injected: a waiting decline that also clears the buffer fails
a_commit_that_waits_for_the_host_keeps_the_typed_text,
a_value_commit_before_the_host_answers_the_last_one_waits_out_loud and
a_rename_that_waits_for_the_host_keeps_the_typed_text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:21:24 -07:00
Edward Langley e14537973f Merge gh/main (220abc61) into wave/tej1-browser-delete-formula
context/test-evidence/command.md: both sides inserted entries at the
same point (this branch's remove_formula_of_a_missing_formula_is_a_refusal;
gh/main's fy9o entries from #227). Both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:17:41 -07:00
Edward Langley bce496b715 Merge #251's head (3f5f90a5, gh/main 220abc61) into fix/8xq0-editor-commits-opened-record
This branch carries #282 and #286 (5m9z/7mro). The other side now
carries #227 and #272 (fy9o/p6v6). This applies the #227 x #295 recipe
(resolve-b/fy9o-with-295, 3918b126) on top of 8xq0's captured-target
commit.

- client.rs: `settle_awaited_seed()` then `settle_declined_editor()`.
  One `settle_awaited_seed`, which needs both no unanswered write and a
  projected cell (`projected_seed`).
- resolve.rs:
  - `opened_on(state, ..)` and `seeded` from p6v6. 8xq0's
    `opened_at_cursor` is kept.
  - 8xq0's `written` over the captured `OpenedCell` and `still_waiting`
    are kept, and fy9o's `cleared` is added. The position-based
    `written` and `record_at_cursor` from the other side are dropped,
    because 8xq0 replaced them.
  - `commit_value` and `rename_coordinate`: `cleared` after
    `ChangeMode(Normal)`, after every decline's early return (joed).
    `advanced(.., writes)`.
  - `reopened_at(.., writes)`: `declined_at` first, then
    `awaiting_seed` after a write or on an unprojected landing, else
    `opened_on(state, ..)`. One `awaiting_seed`.
  - The capture still reads through `projected_at` (`cell_at_cursor`),
    and `refused_here`'s 3h9w arm is kept (`state.opened.is_some()`).
- lockstep: #286's `sent` renamed `sent_to_host` (E0428). The held-cell
  test delivers its held replies before it reads an awaiting editor.
- Ids for the three p6v6 fixture cells that had none.
- context/test-evidence/clients.md: both sides kept, plus the held-cell
  amendment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:17:16 -07:00
Edward Langley fa080f481f Merge gh/main (220abc61) into fix/a61f-aopa-test-doors
context/test-evidence/clients.md: both sides appended at the same point;
both kept.

resolve.rs merged cleanly. Since improvise-3uzg (#272), an editor
reopened after a write waits for the host's seed, and only the send door
records the frame as awaiting that answer. Two tests assumed the answer
without stating it:
- the end of enter_commits_the_cell_advances_and_re_enters_editing
  folded through `ClientState::fold` (a tenth host-door site). It now
  folds through `fold_outgoing` and then calls `accepted()`;
- the_editor_an_advance_reopens_starts_untouched already used
  `press_folded`, so it failed after the merge. It now calls `accepted()`
  after Enter.
Rival: `OpenedEditor::awaiting()` built with `touched: true` fails both.
The aopa evidence entry records it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:14:43 -07:00
Edward Langley eded3179bc Merge fix/8wy9-restore-rebuild (b32a7690) into fix/3til-restore-order 2026-09-24 02:12:01 -07:00
Edward Langley 3f5f90a5c9 Merge #221's head (c7b5b825, gh/main 220abc61) into fix/20ek-milr-rename-races
Conflicts:
- resolve.rs: milr's `claimed` and fy9o's `cleared` sit side by side.
  Both functions kept. `rename_coordinate` keeps milr's `Opened` arms. The
  waiting and `Gone` declines return before #221's success-path
  `cleared` (the joed constraint).
- context/test-evidence/clients.md: both sides kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:11:18 -07:00
Edward Langley 044e8d2d8d Merge fix/xd3q-reset-clobbers-pipelined-placement (d0e36142) into fix/1x9h-browser-refusal-stands 2026-09-24 02:07:46 -07:00
Edward Langley c7b5b825e8 Merge gh/main (220abc61, #227+#272, #297) into fix/fdk1-coordinate-rename
Conflicts:
- client.rs accepted(): both settles, in `fold`'s order:
  settle_editor (fy9o/p6v6 awaited seed), settle_opened (fdk1 capture),
  then the revision bump.
- resolve.rs edit_or_drill: a Coordinate opens like a Record, through
  p6v6's `opened_on(state, ..)`, which waits while a write is unanswered.
- resolve.rs rename_coordinate: fy9o unwelded `clear-buffer` from the
  commit keys, so the rename clears its own buffer on the success path,
  after ChangeMode(Normal). It passes `advanced(.., writes)`, where
  `writes` is whether the frame composed the rename. The waiting decline
  still returns before any clear (the joed constraint).
- resolve.rs reopened_at: the decline arm first (Coordinate { .. }), then
  `awaiting_seed` after a write, else `opened_on(state, ..)`.
- lockstep.rs: both sides' new tests kept.
- context/test-evidence/clients.md: both sides kept.
- browser_round_trip `a_second_rename_after_a_reorder_renames_the_row_the_host_shows`:
  the reopened editor is now seeded from the answered cell. The test
  asserts that seed, then clears it before typing. Evidence amended.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:07:26 -07:00
Edward Langley 1c6600c3e7 Merge gh/main (220abc61) into fix/oplp-followups
context/test-evidence/clients.md: both sides inserted entries at the
same point (this branch's two tile-select lockstep entries; gh/main's
fy9o/5wai entries from #227). Both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:05:49 -07:00
Edward Langley d0e36142ff Merge fix/hv0r-jpba-relays (37dbf02e) into fix/xd3q-reset-clobbers-pipelined-placement
# Conflicts:
#	context/test-evidence/clients.md
2026-09-24 02:04:55 -07:00
Edward Langley 941eb0fd8e Merge commit '9a0d1ba7' into rb-245 2026-09-24 02:03:28 -07:00
Edward Langley 9a0d1ba77a Merge gh/main (220abc61) into tooling/dak5-ci-suites
#297's rulings kept: no bash floor, no find_modern_bash probe in
checkers.sh (it runs suites under the dev shell's $BASH), and no
toolchain-policing step in the build job. This branch's CI table kept:
ci_suites, --ci, --would-run, and the --ci steps in ci.yml (build, js, kb),
which replace gh/main's hand-listed kb, --self-test and remaining-checkers
steps. The kb row already runs every suite those steps named.
check-ci-graph passes on the merged ci.yml.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 02:00:44 -07:00
Edward Langley 37dbf02e4e Merge gh/main (220abc61) into fix/hv0r-jpba-relays
# Conflicts:
#	context/test-evidence/clients.md
#	crates/improvise-protocol/src/client.rs
2026-09-24 02:00:09 -07:00
Edward Langley 65d6a3e163 Merge gh/main (220abc61, #227+#272, #297) into fix/7mro-projected-reads
Applies the #227 x #295 resolution recipe from resolve-b/fy9o-with-295
(3918b126). The tree equals `merge-tree 3918b126 220abc61`, which
merges cleanly, because main moved past #227 only in tooling files.

- client.rs: `settle_awaited_seed()` then `settle_declined_editor()`.
  One `settle_awaited_seed`, which needs both no unanswered write and a
  projected cell (`projected_seed`).
- resolve.rs `reopened_at(.., writes)`: the decline arm first, then
  `awaiting_seed` after a write, or when the landing is unprojected,
  else `opened_on(state, ..)`. One `awaiting_seed`.
- lockstep: #286's `sent` renamed `sent_to_host` (E0428).
- Ids for the three p6v6 fixture cells that had none.
- `a_held_cell_is_read_at_its_position_and_not_its_cache_index` delivers
  its held replies before it reads an awaiting editor. Evidence amended.
- `cleared` stays on commit's success path only (the joed constraint).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 01:58:59 -07:00
Ed Langley 3e41772921 Merge branch 'main' into fix/e265-levelled-groups 2026-09-24 01:57:38 -07:00
Ed Langley b32a769044 Merge branch 'main' into fix/8wy9-restore-rebuild 2026-09-24 01:57:34 -07:00
Ed Langley 3c5135402d Merge branch 'main' into test/4h8a-sfwy-guards 2026-09-24 01:57:32 -07:00
Ed Langley f0d68b9d99 Merge branch 'main' into fold-memo/metf-opbo 2026-09-24 01:57:25 -07:00
Ed Langley 088d469fbb Merge branch 'main' into fix/oyaa-host-list-complete 2026-09-24 01:57:22 -07:00
Ed Langley ff7ea6d706 Merge branch 'main' into fix/qnry-ybum-clamps 2026-09-24 01:57:19 -07:00
Ed Langley 8f218644fe Merge branch 'main' into fix/3j70-config-witness 2026-09-24 01:57:16 -07:00
Ed Langley 32b64f18e2 Merge branch 'main' into fix/qz9p-drill-refusal 2026-09-24 01:57:13 -07:00
Ed Langley a6b5acc24d Merge branch 'main' into fix/i3o5b-panel-dispatch-refuse 2026-09-24 01:57:10 -07:00
Ed Langley 220abc61dd Merge pull request #297 from fiddlerwoaroof/fix/nixos-bin-paths
ci: take bash and tools from the .#ci dev shell; no hard-coded paths or version policing (improvise-3uow)
2026-09-24 01:56:36 -07:00
Edward Langley c29e32f1bb pr-reviews-test: delete the fake clock's untested date pass-through
Copilot on #297: `pr-reviews` only calls `date +%s`, so the REAL_DATE
pass-through never ran. It is deleted rather than tested. The fake now refuses
any other arguments with exit 2, so a new `date` call shows up as a failure.

Rival: with pr-reviews calling `date -u +%s`, "a review arriving on the third
poll is seen" and "--wait keeps polling until its deadline" fail (23/25).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 01:38:37 -07:00
Ed Langley 0d1f0c9eb5 Merge pull request #227 from fiddlerwoaroof/fix/fy9o-reopened-editor-reseeded
A reopened cell editor shows the cell it landed on (improvise-fy9o), stacked on #205
2026-09-24 00:56:10 -07:00
Edward Langley cfa8bf1e0a Refuse a frame whose level names no category; lend the open-run stack
Two ChatGPT Codex findings on the #250 stack.

- #250, P2 (improvise-e265): a FrameGroup.level from the wire sized the
  open-run stack in AxisFrame::groups, so a huge level drove a huge
  allocation. AxisFrame now deserializes through UncheckedAxisFrame and
  refuses any group or GroupEnd level not below cats.len(), naming it.
  context/code/protocol.md §3b.
- #273, P3 (improvise-0r01): the lazy open_runs still cloned a stack per
  skipped row under .nth(idx). walk_open_runs lends one stack to a visitor
  that can stop the walk, so a lookup copies nothing for earlier rows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:41:48 -07:00
Edward Langley 754add31e2 ci: take bash and tools from the .#ci dev shell; stop policing versions
This follows the user's rulings: CI should not care which bash or which
standard utilities it runs, and jobs take what the ci dev shell provides.

- checkers.sh: remove find_modern_bash, the 5.1 floor (improvise-fd0j) and
  exit 3. Suites run under $BASH with PATH unchanged.
- checkers-test.sh: remove sections 2, 5 and 5b (the fake bashes and utilbin).
  They existed only to pin the floor.
- scripts/kb: drop the BSD `stat -f` fallback. kb needs the dev shell for tpl,
  and that shell's stat is GNU.
- ci.yml: remove every `shell: bash` override, so every step runs in the dev
  shell. Remove the web-toolchain version probe and the Playwright
  version/browser probe. The shell's PLAYWRIGHT_BROWSERS_PATH pins the
  browsers. npm ci stays because @playwright/test is not nix-provided.
- flake.nix: update the comments that promised those CI checks.

Reasoning: context/code/tooling.md section 5. Evidence:
context/test-evidence/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:32:54 -07:00
Edward Langley e44b1491ca Merge gh/main (cc95cfbf) into fix/e265-levelled-groups
# Conflicts:
#	context/test-evidence/command.md
#	context/test-evidence/core.md
2026-09-23 23:26:55 -07:00
Edward Langley 8308cb8b7e Merge fix/8wy9-restore-rebuild (3edb6302) into fix/3til-restore-order
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 23:24:28 -07:00
Edward Langley 68f32a8a55 pr-reviews-test: resolve sleep and date from PATH, not /bin (improvise-3uow)
On the NixOS runners /bin holds only sh. With no /bin/sleep, the watchdog in
bounded_reviews exits at once and wins the `wait -n`, so every --wait case
reports 124. Both tools are now resolved with `type -P` before any case stubs
PATH.

Red: with both paths pointed at /nonexistent/, the suite fails 25 of 25.
Rival: REAL_SLEEP=/nonexistent/sleep fails 25 of 25 again. The date
pass-through has no discriminating rival, because pr-reviews only calls
`date +%s`. Evidence: context/test-evidence/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:23:46 -07:00
Edward Langley 3edb63022c Merge gh/main (cc95cfbf) into fix/8wy9-restore-rebuild
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 23:22:07 -07:00
Edward Langley e64e8f14ee test: a non-item cell re-opens no editor, on either end
The merge with #277 gave `uneditable` and `reopened_at` a declining NonItem
arm. These tests fail if either arm is moved to the editable branch.
Rivals and results are in context/test-evidence/command.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:21:37 -07:00
Edward Langley d94dad4046 Merge #228 (0588d433) into fix/zs2r-harness-lock-gate
ci.yml: this branch's check-harness-lock --allow-network step is kept, ahead
of main's nix-build clippy and test steps that #228's merge brought in.
test-evidence/tooling.md: both EOF appends kept, no heading lost (42 = union).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:19:47 -07:00
Edward Langley 1629944eac Merge gh/main into fix/gk51-non-item-record-projection (#292)
Conflicts:
- cmd/core.rs committable_cell_key: took main's form, which asks
  `uneditable` (#277), and gave `uneditable` a declining NonItem arm.
  That keeps this branch's refusal of a commit on a non-item cell.
- resolve.rs reopened_at (auto-merged, non-exhaustive after the merge):
  a declining NonItem arm, the sentence `i` already says.
  The #277 review asked for both arms to decline, never to open.
- tests/shipped_documents_load.rs: kept both, this branch's
  every_record_a_shipped_document_projects_is_writable and main's git()
  helper.
- context/test-evidence/{command,core}.md: both sides appended at EOF; kept
  both, the branch's first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:19:46 -07:00
Edward Langley 86afdb0a36 Merge gh/main (cc95cfbf) into test/4h8a-sfwy-guards
# Conflicts:
#	context/test-evidence/command.md
#	context/test-evidence/core.md
2026-09-23 23:19:23 -07:00
Edward Langley 024aa2d301 Merge commit 'cc95cfbf' into docs/sboz-stable-identities 2026-09-23 23:18:39 -07:00
Ed Langley cc95cfbfdd Merge pull request #296 from fiddlerwoaroof/fix/checkers-bash-floor-ci
checkers-test: run section 5b on the hermetic utilbin PATH (unbreaks KB job)
2026-09-23 23:17:35 -07:00
Edward Langley 0588d433ca Merge gh/main (666de199) into tooling/dak5-ci-suites
ci.yml: main's NixOS-runner rewrite, with this branch's --ci build and --ci kb
steps. Main's separate `checkers.sh kb` step is dropped (the kb row runs it),
and so is the `nix shell nixpkgs#trealla` wrapper (the dev shell supplies tpl).
test-evidence/tooling.md: both EOF appends kept, no heading lost (41 = union).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:15:11 -07:00
Edward Langley 0b645a8f0e Merge fix/xd3q-reset-clobbers-pipelined-placement (75ce4230) into fix/1x9h-browser-refusal-stands 2026-09-23 23:15:08 -07:00
Edward Langley 4989bd55f7 checkers-test: run section 5b on the hermetic utilbin PATH
Section 5b (the bash 5.1 floor, #237) was written on a branch cut before the
NixOS-runner change (af44653f). That change moved section 5 off /usr/bin:/bin,
because on NixOS that path holds only env and sh. The two merged with no
conflict. On NixOS the runner then dies on `dirname` with exit 1, which turned
the KB job red on every PR at 666de199.

5b now uses utilbin too, and utilbin gains head and env for the 5.1 control.
Evidence and rivals: context/test-evidence/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:14:30 -07:00
Edward Langley 901f7fadc5 check-ci-graph: a defaults: run: shell: no longer disqualifies a step (the user's ruling)
Main's PR #287 added a workflow-level defaults shell, and the refusal made every
suite read as unrun. The two cases that pinned it are deleted; one shows a step
under workflow and job defaults is counted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:13:36 -07:00
Edward Langley ef30dcd941 Merge gh/main (666de199) into fix/7mro-projected-reads
Conflicts:
- context/test-evidence/clients.md: both sides' entries kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:12:35 -07:00
Edward Langley 75ce42309a Merge fix/hv0r-jpba-relays (361ba50e) into fix/xd3q-reset-clobbers-pipelined-placement
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
2026-09-23 23:11:00 -07:00
Edward Langley d6f74847f8 Merge gh/main into fold-memo/metf-opbo (#291)
One conflict, context/test-evidence/command.md: both sides appended at EOF.
Kept both entries, the branch's first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:10:49 -07:00
Edward Langley 271f34978b Merge gh/main (666de199) into fix/fy9o-reopened-editor-reseeded
This head already carries #272 (p6v6), merged at d9c65181.

Conflicts:
- resolve.rs advanced: main's `reopened_at` (decline an aggregate or a
  formula's result) now takes this branch's `writes` and answers the
  decline arm first, then `awaiting_seed` or `opened_on(state, ..)`, as
  the #282 review prescribed for this combination.
- context/test-evidence/{clients,command}.md: both sides' entries kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:07:51 -07:00
Edward Langley 361ba50e6c Merge gh/main (666de199) into fix/hv0r-jpba-relays
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
2026-09-23 23:07:10 -07:00
Edward Langley 393f885e44 Merge gh/main (666de199) into fix/qz9p-drill-refusal
context/test-evidence/command.md: both sides appended at EOF; both kept.
gh/main's block comes first because it opens with `###` entries that
belong under the preceding `##` section. This branch's self-contained
`## ui/effect.rs — a drill commit ... (improvise-qz9p)` section follows it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:07:06 -07:00
Edward Langley 37e28155c9 Merge gh/main (666de199) into fix/i3o5b-panel-dispatch-refuse
ui/app.rs: both sides added test helpers at the same point (this
branch's press_key and panel_cursor_at, gh/main's
with_a_document_to_load). All three kept.
context/test-evidence/command.md: both sides appended at EOF; both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:04:22 -07:00
Ed Langley 5008ec07f7 Merge pull request #259 from fiddlerwoaroof/fix/xoxh-lockstep-doors
lockstep: deliver folds host frames through fold (improvise-xoxh)
2026-09-23 23:03:01 -07:00
Ed Langley 4471d4e46e Merge pull request #263 from fiddlerwoaroof/fix/uv9e-toggle-group-cursor
Pin the column group toggle to the cursor; guard run identity in e265's enumeration (uv9e, 6u6s)
2026-09-23 23:02:48 -07:00
Ed Langley d9c65181fc Merge pull request #272 from fiddlerwoaroof/fix/p6v6-editor-opener-awaits
improvise-p6v6: every cell-editor opener awaits its seed while a write is unanswered (stacked on #227)
2026-09-23 23:02:14 -07:00
Edward Langley f892e8014f Merge #274's head (a468f2b2, gh/main 666de199) into fix/joed-waiting-commit-keeps-text
No conflicts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:02:02 -07:00
Ed Langley 7b71342b8d Merge pull request #273 from fiddlerwoaroof/fix/pb30-col-toggle-follow
Pin the column toggle's cursor-follow; a group lookup stops at its row (pb30, 0r01)
2026-09-23 23:01:58 -07:00
Edward Langley cc5e2635d5 Merge gh/main (666de199) into wave/tej1-browser-delete-formula
context/test-evidence/command.md: both sides inserted a block at the
same point, before the "improvise-i3o5: four declines" subsection. Both
kept: this branch's remove_formula_of_a_missing_formula_is_a_refusal,
then gh/main's totals-strip section (improvise-wm20).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 23:00:13 -07:00
Edward Langley a468f2b28f Merge #251's head (7eeba18e, gh/main 666de199) into fix/8xq0-editor-commits-opened-record
Conflicts:
- resolve.rs reopened_at: this branch's `declined_at` form kept. It
  already names `CellId::Coordinate { .. }` and reads the frame's own
  origin; main's form (read through `cell_at`) is what 8xq0 replaced.
- context/test-evidence/clients.md: both sides' entries kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:58:29 -07:00
Edward Langley 53f1118014 Merge gh/main (666de199) into fix/a61f-aopa-test-doors
context/test-evidence/clients.md: both sides appended at EOF; both kept.

resolve.rs merged cleanly, but gh/main added a ninth test-side fold of
what `resolve` composed (the armed-prefix loop in
the_command_names_that_still_need_the_host_are_these), still through
`ClientState::fold`. Moved to `fold_outgoing` like the other eight, and
the aopa evidence entry says so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:55:53 -07:00
Edward Langley 7eeba18eb9 Merge #221's head (516ad66a, gh/main 666de199) into fix/20ek-milr-rename-races
Conflicts:
- context/test-evidence/clients.md: both sides' entries kept, this
  branch's first (its amendment reads "above").

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:54:31 -07:00
Edward Langley c7f93515ba Merge gh/main (666de199) into fix/oplp-followups
context/test-evidence/command.md: both sides appended at EOF; both kept
(this branch's two drill tile-select entries, then gh/main's
Host::mutating_frame abort-arm section).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:49:52 -07:00
Edward Langley 29ad14f403 ADR 007 revision 4: fold in the third round; check against km71, tt22, wpxr, NonItem (improvise-sboz)
Answers 18-20 settle Q15-Q17: slugify is casefold, space -> _, strip; the
formula panel shows display names with the target's id; hand-written ids
are lower case. km71 landed an opaque interim FormulaId and ReplaceFormula
{ id, was, raw }; the ADR now revises that rather than inventing
EditFormula, and keeps was. tt22 has landed. New open questions Q18-Q21.
Status stays proposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:47:11 -07:00
Edward Langley 516ad66af2 Merge gh/main (666de199) into fix/fdk1-coordinate-rename
Conflicts:
- cmd/core.rs committable_cell_key: take main's `uneditable` door; its
  pattern names fdk1's struct `CellId::Coordinate { .. }`.
- resolve.rs reopened_at (main, clean text): same pattern fix, a
  compile-only semantic conflict.
- context/test-evidence/{clients,command}.md: both sides' entries kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:47:10 -07:00
Edward Langley 9fd048c8fa doc-site: publish ADR 007; core.md cites it by its number (improvise-sboz)
The build job failed on the PR head: build-doc-site.mjs refuses a context/
doc that is neither in the manifest nor a named omission. core.md §18,
landed with km71, still called this ADR 005.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 22:43:22 -07:00
Edward Langley 651bf1b7a9 Merge commit '881c5807c39e1ce566042726582c5cbf5e7472af' into docs/sboz-stable-identities 2026-09-23 22:42:50 -07:00
Ed Langley 666de19963 Merge pull request #247 from fiddlerwoaroof/collapse-fu/h6z1
improvise-h6z1: one fold memo per grid build (stacked on #239)
2026-09-23 22:42:07 -07:00
Ed Langley eb9ec7e98b Merge pull request #277 from fiddlerwoaroof/fix/enz6-advance-onto-fold
No editor re-opens on a cell i declines: a fold or a formula's result (improvise-enz6)
2026-09-23 22:41:16 -07:00
Ed Langley 6e013511b9 Merge pull request #284 from fiddlerwoaroof/perf/i5z6-formula-tests
Parse formula.pest once per test process (improvise-i5z6)
2026-09-23 22:41:01 -07:00
Ed Langley eade33c94e Merge pull request #264 from fiddlerwoaroof/fix/uln5-help-black-holes
Mark the help rows a browser cannot run, and guard it from one list (improvise-uln5)
2026-09-23 22:40:34 -07:00
Ed Langley 8d1df439d6 Merge pull request #253 from fiddlerwoaroof/fix/pr-watch-blocked-merged
improvise-9pfs/1qaa: pin a red under BLOCKED; a merged or closed PR ends pr-watch with exit 5
2026-09-23 22:40:19 -07:00
Ed Langley 852ffcda2e Merge pull request #257 from fiddlerwoaroof/fix/su4u-minibuffer-refusal
A minibuffer refusal stands while the user fixes it (improvise-su4u)
2026-09-23 22:40:04 -07:00
Ed Langley 881c5807c3 Merge pull request #256 from fiddlerwoaroof/fix/rq3f-resolver-gaps
Pin the browser's Tab-commit advance and the tile cursor's upper clamp (improvise-rq3f)
2026-09-23 22:39:26 -07:00
Ed Langley 4e8c5d4c9e Merge pull request #252 from fiddlerwoaroof/fix/pr8t-nacd-item-coordinates
Refuse a wire write unless every coordinate names an item (improvise-nacd, improvise-pr8t)
2026-09-23 22:39:11 -07:00
Ed Langley 2700afc027 Merge pull request #239 from fiddlerwoaroof/collapse-fu/wm20
improvise-wm20: a column total is its cell as if every drawn row were one fold
2026-09-23 22:38:33 -07:00
Ed Langley 86b22921b7 Merge pull request #244 from fiddlerwoaroof/tooling/lqs0-audit-exits
tooling: bd-audit-ids tells grep's no-match from a read error (lqs0)
2026-09-23 22:38:07 -07:00
Ed Langley 932ff408e6 Merge pull request #237 from fiddlerwoaroof/tooling/7fl8-exact-polls
tooling: pr-reviews timeout case asserts WAIT+1 polls, and a hang fails (7fl8)
2026-09-23 22:37:52 -07:00
Ed Langley 90ea08beaf Merge pull request #249 from fiddlerwoaroof/fix/m6op-measure-agg-fixture
improvise-m6op: the cross-instance fixture carries measure_agg entries
2026-09-23 22:37:13 -07:00
Ed Langley 87a6c874a6 Merge pull request #248 from fiddlerwoaroof/fix/53zz-scratch-repo-env
Isolate the shipped-documents scratch repos from the caller environment (improvise-53zz)
2026-09-23 22:36:57 -07:00
Ed Langley 5074d66d10 Merge pull request #229 from fiddlerwoaroof/fix/we1a-abort-refuses
Pin that every aborting model effect also refuses (improvise-we1a)
2026-09-23 22:36:27 -07:00
Ed Langley 283af3737e Merge pull request #288 from fiddlerwoaroof/bd-update
bd update
2026-09-23 22:31:28 -07:00
Edward Langley f2093dc0c1 bd update 2026-09-23 22:31:05 -07:00
Edward Langley fc4a41cb87 Merge #274's head (3dad3b09, improvise-3h9w) into fix/joed-waiting-commit-keeps-text
refused_here now skips the commit keys for a captured editor. The test
file was rebuilt as #278's tests plus #274's new block, because git's
hunks split a function.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 20:14:23 -07:00
Edward Langley 3dad3b0954 improvise-3h9w: a captured commit is not refused for the cursor's cell
refused_here (from #286) ran before commit() consulted the capture and,
for the commit keys, judged the cell under the cursor. After a reframe
left the cursor on an aggregate, a captured editor's Enter was declined
on every retry. The commit-key guard now applies only to an editor that
captured nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 20:10:36 -07:00
Edward Langley 0458b43519 improvise-3h9w: red test, a captured edit is refused for the cursor's aggregate
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 20:04:29 -07:00
Edward Langley 2cb88203d1 Merge #274's head (8c28360f, now carrying #286) into fix/joed-waiting-commit-keeps-text
step() keeps #286's refused_here (declined, so it keeps the text too)
ahead of joed's committed(); both functions are kept. Evidence appended
on both sides.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:47:30 -07:00
Edward Langley 8c28360f47 Merge #286 (fix/7mro-projected-reads, 6b604a9e) into fix/8xq0-editor-commits-opened-record
#274's capture read the cell through the cursor's offsets and stuck, so a
scrolling Enter captured the old window's cell even with every reply
delivered (improvise-7mro's horizontal wrap: Jan/Revenue for Feb/Revenue).
#286's cell_at_cursor reads through projected_at, so the capture waits
until the host projects the cell. Taken by merge, not by port: the matrix
test's held cases need #282/#277's waits too.

Resolutions: committable_cell_key takes #286's uneditable(); the two unit
CellId::Coordinate patterns become struct patterns; evidence is appended
on both sides.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:40:25 -07:00
Edward Langley 6b604a9e3b improvise-7mro: evidence and the clients.md account
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:29:55 -07:00
Edward Langley 51943d1bbc improvise-7mro: an empty grid has no cell to wait for
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:28:08 -07:00
Edward Langley fb246a8331 improvise-7mro: the seed settles on a projected cell, and the matrix holds replies three ways
An awaited seed waits for a cell the host projected (id present), never the
blank a relabel leaves between a frame and its cells. The advance's reopen
keeps #282's shape: on this base the welded clear-buffer empties it anyway.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:24:09 -07:00
Edward Langley ff418d33bd Restore the harness-lock network step the recompose dropped
improvise-dndu, from #245's review and Codex. Resolving the ci.yml
conflict with `git checkout --theirs` took #228's whole file, not the
conflicting hunk, so #245's `test`-job step running
`check-harness-lock.sh --allow-network` against the real harness lock
went with it. `git diff 1bec7c0f f41a946b -- ci.yml` was empty: CI kept
only the kb row's self-test, which never reads the real lock. The merge
message's "the --allow-network step is unchanged" was wrong; this
supersedes it.

The four lines are restored after the fmt step, where #245 had them.
The evidence note records that the self-test and the network step are
two different checks, and how to verify a conflict resolution against
each parent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:18:12 -07:00
Edward Langley 2007a21557 improvise-7mro: the matrix covers x and p, and a cell inside the held window
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:16:22 -07:00
Edward Langley c1e5a4aaae improvise-7mro: every cache read goes through the frame's own origin
cell_at_cursor and seed_at read through projected_at, and cell_at (the
cursor-offset read) is gone. A cell of the grid outside the window the
cache holds is unprojected: i and an advance open an editor awaiting its
seed, which the fold settles once the host projects the cell, and a
commit, x, yy or p on it waits out loud rather than composing anything.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:15:07 -07:00
Edward Langley f41a946b6f Merge #228 (tooling/dak5-ci-suites) into fix/zs2r-harness-lock-gate
Recomposes #245 onto #228's CI table. The kb job's checker step no
longer lists suites by hand: it runs `checkers.sh --ci kb`, and
check-ci-graph.py holds the ci_suites table to `checkers.sh --list`.
So check-harness-lock joins the table's kb row, with the one-line
reason that the old ci.yml comment carried. The conflict in ci.yml is
resolved to #228's step. The --allow-network step is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:12:51 -07:00
Edward Langley 32f6067423 improvise-7mro: red lockstep matrix, keys sent before a scroll is answered
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:11:18 -07:00
Edward Langley 5b7e0223aa Merge fold-memo/pntu-0jw5 (fa3350ac) into fold-memo/metf-opbo
Carries #268 onto #258's new head, which now sits on #247's re-merge
(12f206fb) and so on #239's spanned_values and &Expansion signatures.
Code auto-merged: GridLayout::evaluator / GridEval stays the only public
path to a FoldMemo, and prune_empty still shares one memo. The one
conflict was benches/grid_projection.rs's main(): both new sections kept,
the pruned block kept directly after the folded chain it calls "the same".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:08:07 -07:00
Edward Langley fa3350acfb Merge collapse-fu/h6z1 (12f206fb) into fold-memo/pntu-0jw5
Carries #258 onto #247's re-merged head, which now has #239's
spanned_values/spanned_positions and the &Expansion fold signatures.
Code auto-merged: FoldMemo::new(agg, covers) still owns both, and
fold_stored reads through spanned_values. The only conflicts were the
append-at-EOF evidence files (command.md, core.md); both sides kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:02:39 -07:00
Edward Langley 1bec7c0fb9 check-ci-graph: a line holding a YAML-only line break or a control character is refused (xpzh)
The gate split on \n alone while YAML also breaks at U+0085, U+2028 and
U+2029, so an if: after one was invisible and the gate exited 0. Every line,
comments and block-scalar bodies included, is now checked for C0 (but tab),
DEL, C1 and U+2028/2029. CR is folded by the one read path, read_workflow, and
file-based controls pin that. Argument: context/code/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 19:00:08 -07:00
Edward Langley 2cf3303437 improvise-wpxr: a cell write naming a view the document lacks resolves nothing
Answers the ChatGPT Codex P2 on ba3d2875. Grids looked the view up through
resolve_view, whose read-side fallback to the default accepted SetCell
{ view: "Nope" } wherever Default drew the record. The view is now looked
up strictly, and a stored record is gated too, so a missing view is
RecordGone at the basis check for every record.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:58:33 -07:00
Edward Langley 7025ec2471 Merge #266 (fix/xd3q) into #279: one notify, and a rebase settles the status (improvise-0dbk)
Both PRs added ClientState::notify, so the merge failed with E0592. #266's
is kept: it queues the sentence through fold_outgoing, which runs
settle_status, so the owner is still recorded. #279's direct-write notify
is deleted.

#266's rebase() rebuilt the view without settle_status, so a relay that
moved the view off the editor left the message owned, and an equal
editor reopened afterwards took it back. rebase() now settles it, pinned
by a_rebase_off_the_editor_ends_the_message_standing (red before).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:57:06 -07:00
Edward Langley 5c69381b69 Walk every mode a browser can reach; an empty status is not a decline
From PR #280's review.

improvise-s2gb: every_silent_binding_is_on_the_list walked Normal and
search mode only, so a black hole bound in FormulaPanel (the reviewer's
Z -> commit-export) passed the whole suite. The modes are now derived:
reachable_modes starts from both halves of Normal (the grid, not a key,
picks RecordsNormal, e5pf) and closes over the modes a pressed binding
lands in. Doors `declined` shuts (the three stranding prompts, the
import wizard, records editing, search now that `/` declines) fall out
with no list to maintain. A floor assertion names seven modes the walk
must reach, so it cannot pass by shrinking. furnished_to_type seeds the
mode's own minibuffer as well as the search query.

`heard` counts a SetStatus as a decline only when it says something: a
resolver answering SetStatus("") left the user seeing nothing and passed
this guard, caught only incidentally by the Ctrl-Q tests.

Merges #275's new head (improvise-4k1j) and fixes the fmt gate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:52:49 -07:00
Edward Langley 6942e53e5e improvise-ik6l: name the lockstep case tuple for clippy
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:51:53 -07:00
Edward Langley dffbf99ff7 improvise-y5eb/ik6l: evidence and the clients.md account
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:40:09 -07:00
Edward Langley d0a0784feb improvise-ik6l: a commit over a declined cell is refused out loud
Enter or Tab on a typed-into editor over a fold or a formula's cell went
Upstream, which nothing answers. refused_here declines the step, which
takes the whole binding with it, exactly as the TUI's failed
committable_cell_key does: status said, buffer and mode kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:39:40 -07:00
Edward Langley 7ae871cb1e check-ci-graph: every value it reads is one of a few one-line shapes, or refused (0wn3)
A multi-line flow value (- name: [a, "b ... run: ... c: x"]) is one string to
go-yaml, PyYAML and Psych, but the gate counted the inner run:. Rather than
refuse one more construct, values are now an allowlist: empty, a block header,
a closed quote, a one-line flow with no quotes or nesting, or a plain scalar
not starting with an indicator. The real ci.yml needs nothing else.
Argument: context/code/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:38:35 -07:00
Edward Langley 1a58fa37fc improvise-ik6l: red test, Enter over a declined cell goes silently upstream
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:37:25 -07:00
Edward Langley 51a61f4b17 improvise-kimw: derive a view's grid once per borrow, not once per question
App::grids() memoises the named view's layout for as long as it borrows the
App: once for a whole frame check, once in model_effects. A frame of n writes
to another view's empty records now derives 1 + 2n grids instead of about 5n.
Empty-record ClearCell on 18k cells with a formula, M4 Max: 1.44s -> 0.66s.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:36:41 -07:00
Edward Langley 100627fd57 improvise-y5eb: read the declined cell through the frame's own origin
declined_at indexed the cache with the cursor's offsets, which the
client's own outgoing frame moves before the host's projection arrives.
A wrapping Down read the old window's top row and closed an editor on
Revenue/Feb with Profit's refusal. projected_at reads against
AxisFrame::first, so a stale cache answers None instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:36:34 -07:00
Edward Langley bb2f889732 A retry whose cell is Gone keeps the typed text (improvise-joed on #251's decline)
renamed_away() is a Step::Declined, like still_waiting(), so the Enter
binding's welded clear-buffer no longer empties the editor the user must
Esc away from. Shared by the value and rename paths. Red first:
a_retry_whose_cell_is_gone_keeps_the_typed_text failed with Some("").

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:36:24 -07:00
Edward Langley 6609a743a4 Merge branch 'fix/fo0w-black-holes' into fix/oyaa-host-list-complete 2026-09-23 18:35:47 -07:00
Edward Langley 1e49bc0df7 H refuses on every fold the host refuses on; carry the fold flag
improvise-4k1j, from PR #275's review. hidden_row_item refused on a fold
only when the NEAREST group header was collapsed. Under two row
categories that header can be the inner one's: with Region's Coast
collapsed and Product's Kit expanded, the client composed
HideItem{Region, "Coast"} and a twin host wrote a group name into the
document, where the host refuses.

The frame cannot answer this on gh/main: FrameGroup names no level and
an inner GroupEnd reads as an outer one. So FrameEntry::Item now carries
the layout's own DataItem::collapsed flag (a fold, or under one, at any
level) and AxisFrame::folds reads it, the same bit
GridLayout::row_folds_a_group reads. FrameEntry::item builds an
unfolded one, as AxisEntry::item does.

hiding_agrees_with_the_host_on_every_row_of_nested_groups presses H on
all 25 rows of four nested grouped grids, in two windows each. It was
red on the nearest-header check at rows 4 and 5, as the review found.

Also fixes the fmt gate the review found red: fmt-workspaces.sh is a
check, not a formatter, and its diff output was misread as success.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:35:38 -07:00
Edward Langley 347b2154a5 improvise-y5eb: red test, the settle reads a stale cell after a wrapping advance
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:35:35 -07:00
Edward Langley 12f206fb0a Merge collapse-fu/wm20 (500d4609) into collapse-fu/h6z1
# Conflicts:
#	benches/grid_projection.rs
2026-09-23 18:33:53 -07:00
Edward Langley 12a92a8669 Give run_git's config nulls a witness no -c reaches (improvise-3j70)
GIT_CONFIG_GLOBAL=/dev/null and GIT_CONFIG_NOSYSTEM=1 in run_git were
witnessed by a config excludesFile, which vwjj's -c
core.excludesFile=/dev/null outranks. So dropping either line passed
every test from vwjj onwards.

The hostile child's config, read as both GIT_CONFIG_GLOBAL and
GIT_CONFIG_SYSTEM, now also sets core.sharedRepository = nonsense.
Measured on git 2.55: git init and git add die on it, git ls-files
does not read it (so the walk, which does not null the user's config,
is unaffected), and run_git passes no -c for it. init.defaultObjectFormat
and init.defaultRefFormat were tried first; they only warn.

Dropping either env line is now red. Every other rival in the combined
test is unchanged, and there is still one child per test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:32:41 -07:00
Edward Langley 500d46092b improvise-jq0m: pin both narrowings against the per-member definition; record the bench and the evidence
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:32:33 -07:00
Edward Langley 1572258e03 Merge #274's head (8d9de94e) into fix/joed-waiting-commit-keeps-text
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:31:51 -07:00
Edward Langley 8d9de94e08 A value retry writes only the cell it named; Enter follows the capture (improvise-8phw)
8phw: when A's Enter crossed B's HideItem, the LayoutMoved refusal replayed
the editor over South and settle_opened recaptured South/Cost, so the
retry wrote there. #251's Opened::Retry/Gone now covers values:
written_to() recognises SetCell/ClearCell as well as a rename, and Retry
and Gone carry the whole OpenedTarget, which the recapture must match.
A Gone value editor declines with RECORD_RENAMED_AWAY. seen stays
rename-only.

Codex P2 on #274: commit picked value/rename/create from the cell under
the cursor before it consulted the capture, so a reframe that left no
cell there sent a captured editor's Enter upstream. commit now
dispatches on opened.target(); only an uncaptured editor is dispatched by
the cursor. A records grid's measure cell is not captured as a value,
since the browser still declines it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:29:33 -07:00
Edward Langley b1715f49b4 improvise-i5z6: parse formula.pest once per test process
The two generator strategies re-parsed the grammar inside their
prop_map, once per generated case: the same defect improvise-8shu fixed
in improvise-io. A OnceLock takes the lib test binary from ~0.9s to
~0.06s on the same machine and load.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:23:45 -07:00
Edward Langley 5b8bdc19b1 One child per test, carrying every hostile git setting at once (improvise-eahe)
tests/shipped_documents_load.rs re-ran itself in 18 child processes, one
per hostile setting, and each child spawns several gits. So the binary
took 2.1-3.3s against design-principles 7's 2s-per-binary tripwire.

The six per-setting tests (53zz x2, vwjj, pr3i, a0i3 and 2iws/vav0) are
now one test, a_hostile_git_environment_changes_nothing. It has two
children: the gzip shape test and every_shipped_document_loads. Each
carries every setting at once:
- the template dir;
- the global and system config (with trace2 targets);
- XDG_CONFIG_HOME;
- GIT_DIR and GIT_INDEX_FILE;
- the five pathspec and attribute settings;
- the six trace variables.

Afterwards the test asserts the outer index is still empty.

Each setting breaks the walk on its own, so a rival that drops one
defence leaves that setting live, and the child fails. Every rival that
was red against the per-setting tests is red against the combined one.
The rivals that were green before are green now:
- ICASE;
- the trace2 names under the 0 override;
- the GIT_CONFIG_GLOBAL and NOSYSTEM nulls, whose excludes witness vwjj's
  -c override masks. That last pair is filed as improvise-3j70.

Measured on an Apple M4 Max, three workspace runs each: this binary went
from 2.10-3.31s (load 37-39) to 0.57-0.91s (load 30-33). It is now
below the next-largest binary in every run.

The evidence sections keep their reasoning, under prose headings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:23:40 -07:00
Edward Langley 3f9f74a7da improvise-i5z6: red test, formula.pest is re-parsed per generated case
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:21:52 -07:00
Edward Langley 9dc8a643b2 check-ci-graph: a multi-line quoted value is refused, and the quote tracker is deleted (6zf1)
The tracker closed a multi-line quote on an escaped \" or a doubled '', so a
later quote reopened it over a job's real if:. Both styles exited 0 on the real
ci.yml. The opening line is refused instead; _opens_quote's escape handling
is now load-bearing and has a case per style. Argument: context/code/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:20:36 -07:00
Edward Langley 57b713cf15 A refusal's sentence queues at the tail, so a rebase cannot erase it (improvise-n0pb)
`fold_frame` wrote the rejection sentence straight into `view.status_msg`,
so it was in neither `settled` nor `in_flight`, and the xd3q rebase erased
it whenever a relay queued behind a frame still in flight: an in-flight
frame's own `SetStatus` was replayed over it.

`ClientState::notify` queues the sentence as a view-only entry at the tail
of `in_flight`, where `fold_outgoing` puts a frame the host never answers.
Frames sent before it replay first, relays queue ahead of it, frames sent
after it may replace it, and it settles when everything ahead is answered.
Writing it into `settled` too was tried as a rival and fails the same way.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:20:12 -07:00
Edward Langley cc7e49ac67 Merge #251 (fix/20ek-milr-rename-races, ff8ea239) into fix/8xq0-editor-commits-opened-record
OpenedCoordinate becomes #274's OpenedCell, now carrying #251's seen
(filled for a rename only) inside Opened::Target. Retry/Gone keep #251's
rename-only semantics in this commit; the value-cell retry (improvise-8phw)
follows. #221's superseded "a retry after a refusal is a force" paragraph
is dropped in favour of #251's rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:17:20 -07:00
Edward Langley 08f84c25e2 improvise-jq0m: a small span narrows per covered item, not by the whole category; grouped bench cases
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:16:29 -07:00
Edward Langley fbfe37802f Pin the GIT_TRACE prefix with all six variables; override config trace2 targets (improvise-2iws review, improvise-vav0)
The review of #276 found that an exact list of the three names the test
set passed it. The child now sets all six variables that write to their
target on git 2.55: GIT_TRACE, _SETUP, _PERFORMANCE, 2, 2_EVENT and
2_PERF. An exact three-name list leaves GIT_TRACE_SETUP behind and
fails.

vav0: a trace2.*Target in user config is not an environment variable,
so removal cannot reach it, and the walk's git does not null the user's
config. untraced_git now also sets GIT_TRACE2, GIT_TRACE2_EVENT and
GIT_TRACE2_PERF to 0, which outranks the config. The child adds a
GIT_CONFIG_GLOBAL whose [trace2] section sends all three targets to
/dev/stdout.

Rivals, each of which fails:
- no strip;
- the exact three names;
- the prefix minus GIT_TRACE, minus _SETUP, or minus _PERFORMANCE;
- no 0 override.

The evidence entry moves from the web_demo_model section to the
shipped_documents_load one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:16:22 -07:00
Edward Langley 942f57e527 improvise-5m9z: name the lockstep case tuple for clippy
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:14:35 -07:00
Edward Langley 689c8c6eb5 improvise-5m9z: an editor opened past the window settles on the host's answer
reopened_at opens provisionally where the landing cell is not projected
yet; ClientState::settle_declined_editor closes an untouched editor once
the host projects that cell as one i declines. Both ask one
resolve::declined_at. A touched editor is kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:11:33 -07:00
Edward Langley d09a27c7e7 EditOrDrill refuses a NonItem cell at open, and the four declines are pinned (improvise-yc9r)
`i`/Enter opened an editor on a NonItem cell and the refusal came only at
commit, under a minibuffer that hides the status. `EditOrDrill` now asks
`cursor_cell_id` after the formula check, as the browser already does.

Three rivals each passed the whole workspace before this: NonItem
writable in `value_cell_key`, a Record in `committable_cell_key`, and
`None` from the browser's open-editor arm. Each now fails its own test.
The commit guard is reached through `:enter-mode editing`, which opens
an editor past `EditOrDrill`.

command.md also records improvise-4daa's argument beside gk51's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:11:26 -07:00
Edward Langley 519c8ddefd Model::is_item: membership without building the item list (improvise-4daa)
`first_non_item` asked `effective_item_names(cat).contains(name)` per
coordinate per projected cell, cloning every item name each time, and the
TUI projects the whole grid every frame. `is_item` asks `contains_key`,
plus the formula targets for a measure category, and is pinned equal to
the list it replaces. #252's write path already goes through
`first_non_item`, so both callers get it.

Measured on an Apple M4 Max, interleaved 5x, minimum per build: pivot
18k cells 193 ms -> 136 ms (base 131 ms); records with a 2000-item label
145 ms -> 2.8 ms (base 1.9 ms).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:11:18 -07:00
Edward Langley 88e53aed73 Hold NEEDS_THE_HOST complete from the keymaps, not the help rows
improvise-oyaa. every_silent_binding_is_on_the_list presses every
binding the Normal and search-mode keymaps reach (parents unshadowed,
prefixes expanded) on the furnished fixture, and requires each silent
one to be on NEEDS_THE_HOST. The two earlier guards could not see an
unlisted binding that no help row names: dropping force-quit (Ctrl-Q)
from the list failed nothing. Now it fails this test.

every_binding_reaches_parents_and_prefixes checks the walk itself
against a hand-built keymap, because no silent binding in today's maps
sits in a parent or behind a prefix, so a walk that skipped either
passed the first test unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:09:54 -07:00
Edward Langley ba3d2875c4 improvise-wpxr: fmt, and the SetCell doc comment within the two-line cap
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:08:36 -07:00
Edward Langley 9eb83538fe A message beside its open editor stands in the browser too (improvise-1x9h)
The toast hid every status after six seconds, so a refusal the user was
still fixing vanished with its prompt open: su4u's bug on a timer. The
browser now asks the TUI's question. EditorSession moves into
improvise-protocol, ClientState records which editor a status was
written in, and RenderModel::status_stands tells the toast to hold it
until that editor closes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:08:26 -07:00
Edward Langley 7817d74da1 improvise-5m9z: red lockstep test, an advance past the window opens the editor
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:06:41 -07:00
Edward Langley d79a8bd5b0 improvise-wpxr: argue the named view in context/, and record the evidence
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:03:42 -07:00
Edward Langley c704ea1263 A commit that waits for the host keeps the typed text (improvise-joed)
Enter in Editing/RecordsEditing is [commit-cell-edit, clear-buffer edit].
The "still waiting" answer came back from composed() as an ordinary
Local step, so the sequence ran the welded clear after it and emptied
the editor it had left open, on the value path and on #221's rename
path alike.

The two cell commits are now answered by committed(), ahead of
composed(), and still_waiting() is a Step::Declined: the existing
channel by which a step stops a sequence. Under #227 (no weld; the
commit clears at its head) the binding is one command and the waiting
return precedes that clear, so the fix fits both.

Stacked on #274.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:03:39 -07:00
Edward Langley 588094b7bb check-ci-graph: a ' #' on a read line is refused, one string per line, read_graph opens wide (j49r)
A job header with a trailing comment was accepted by the readability check
(comment-stripped line) and never opened (raw line), so its if: went to the
previous job and read_graph never listed it. The audit found the stripper also
cut a quoted "x # y" open, hiding a later job-level if:. Both exited 0 on the
real ci.yml. Refusing the comment makes stripped == raw on every accepted
line, so the stripper is deleted. Argument: context/code/tooling.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:02:08 -07:00
Edward Langley ddcdf93a6b The shipped-document walk strips every GIT_TRACE* variable (improvise-2iws)
GIT_TRACE, GIT_TRACE2_EVENT and GIT_TRACE_PERFORMANCE are process-wide,
so --local-env-vars lists none of them. Pointed at /dev/stdout, git
wrote trace lines into the ls-files -z output, and the walk read them
as paths. git() now builds on untraced_git(), which removes every
variable starting with GIT_TRACE: a prefix, because the family is
open-ended. The rev-parse --local-env-vars call goes through it too,
since its stdout is parsed as well.

the_walk_ignores_a_git_trace_sent_to_stdout reruns the gzip shape test
and every_shipped_document_loads in children with all three set to
/dev/stdout. It failed at #269's head. Rivals, each of which fails it:
- no strip;
- the exact name only;
- keep GIT_TRACE_PERFORMANCE;
- keep GIT_TRACE2_EVENT.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:01:54 -07:00
Edward Langley c3a0f100f2 improvise-wpxr: tests for the named view, the writability twin, and the client side
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:01:17 -07:00
Edward Langley cdefeeb9b8 Resolve Enter and H in the browser; decline / and n out loud
improvise-fo0w. Three black-hole keys closed over vocabulary that
already exists, each mirroring its Cmd line for line with a lockstep case:

- enter-advance (Normal Enter) resolves as cursor arithmetic over
  GridExtent; the host's bottom-right AbortChain is Ignored here, which
  loses nothing because Enter is never bound inside a Sequence.
- hide-selected-row-item (H) resolves to ModelCommand::HideItem over the
  outer row category and the selected tuple's first label, and refuses
  on a fold with the host's own FOLD_REFUSAL, now in improvise-protocol.
- search (/) and search-navigate (n) decline out loud as a pair: n needs
  the whole model, and / alone would open a box that finds nothing.

NEEDS_THE_HOST drops to exit-search-mode, force-quit and
toggle-group-under-cursor. Tests that used Enter or "Start search" as
their upstream example move to Ctrl-Q and "Exit search mode", including
shell.spec.js's not-sent case.

Also improvise-0629: the help render test now asserts every ex row is
drawn dim, not just that its description is marked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 18:01:08 -07:00
Edward Langley aab62a362a improvise-enz6: pin the commit door directly, and write up the evidence
The advance no longer reaches the commit door on a fold or a formula's
cell, so the two older advance tests stopped seeing it (measured: the
cell_key rival passes them). Two tests seat the editor there directly.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:56:41 -07:00
Edward Langley 5036b4483a check-ci-graph: a comment on a line the gate reads is refused, and the stripper is gone (j49r)
WIP; tooling.md and evidence follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:56:28 -07:00
Edward Langley 58cd5d3858 improvise-wpxr: SetCell/ClearCell name the view that drew them
An empty record exists only as a layout position, so resolve it against the
layout of the view the command names -- the one grid_in projects for that
session -- rather than the host's own layout.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:55:01 -07:00
Edward Langley f4c0feec2c Close the entry half of improvise-enz6: no editor re-opens on a cell i declines
EnterEditAtCursor asks core::uneditable, the commit door's own question,
now shared with committable_cell_key. The browser's advance asks the
projected id at the landing cell; lockstep pins both ends on a fold and
on a formula's cell.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:49:44 -07:00
Edward Langley 7bf8debd12 The keyboard editor commits to the cell it opened on (improvise-8xq0, improvise-xz2z)
resolve.rs::written read its operands at Enter: the record and measure
from the cell under the cursor, and the basis from claim(), i.e. the
version current at commit. So a HideItem from another client that
renumbered the rows under an open editor sent the write to whichever
cell the cursor then named (8xq0), and a write by another client to the
record while the editor was open was silently overwritten instead of
refused RecordMoved (xz2z, 0j0f's rule on the keyboard door).

The fix reuses #221's capture rather than adding a second one.
ClientState::opened becomes OpenedCell { record, target, version,
document_generation }, where OpenedTarget is Value { measure } or
Coordinate { category }. settle_opened captures a Record cell as a
Value target, and written builds the RecordCell and the claim_as_read
basis from it. With no capture while the host has yet to answer this
client's own last write, the commit declines with the rename's "still
waiting" status.

Red first: both round-trip probes failed on gh/main 202e7eac and on
#221's head 39821d5f. Unit-test fixtures that set Editing by assignment
now open the editor through the fold, which is where the capture is
taken.

Stacked on #221 (fix/fdk1-coordinate-rename).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:49:43 -07:00
Edward Langley 0c3ec183d3 Pin the column toggle's cursor-follow; a group lookup stops at its row (improvise-pb30, improvise-0r01)
pb30: gz puts the cursor on the collapsed group through landing and
place_cursor, and their column arms were untested. Three rivals passed
all 1948 tests: the arm swapped, landing returning None for columns, and
landing reading col_tuple(0). A new test uses two column levels
(Region over Month) with the cursor at (1, 7), South/May. It requires
the cursor to land on South's Q2 at (1, 5). Each rival fails it and
nothing else. The col_tuple(0) rival needs the two column levels, since
with one level the prefix is empty everywhere.

0r01 (Codex's P2 on #263): innermost_group built open_runs for the
whole axis, cloning a level stack per row, to answer for one row on
every z. open_runs is now a lazy iterator, still the one reader, and
the lookup takes .nth(idx). a_lookup_does_not_read_past_its_row gives
it an unreadable header after row 0 (level usize::MAX). The eager rival
panics with capacity overflow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:49:05 -07:00
Edward Langley f8c262e986 improvise-wpxr: red test — an empty record resolves through the host's layout
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:44:31 -07:00
Edward Langley 69764bd52b improvise-enz6: red tests, the re-opened editor lands on a fold or a formula's cell
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:41:36 -07:00
Edward Langley 0947001b70 Each crate's tests compile alone; resolve.rs folds what it sends through the send door (improvise-a61f, improvise-aopa)
a61f: `cargo test -p improvise-wasm-client --lib` failed with E0277,
because `FormulaEntry: From<&str>` exists only under
improvise-protocol's `test-support` feature, which that crate's
dev-dependencies never asked for. `--workspace` hid it: feature
unification pulled the feature in through improvise-command's dev-dep.
A sweep of every member with `cargo test -p <crate> --no-run` found
the same failure in the root `improvise` crate
(tests/bug_hunt_reproductions.rs). Both now request the feature. The
normal wasm tree still carries no `test-support`.

aopa: eight helpers in resolve.rs's tests folded client-composed
commands through `fold`, the host's door, so a composed Command::Model
was never awaited in `in_flight`. All eight use `fold_outgoing`, one
frame per key, as web/app.js does.
`a_folded_browser_commit_waits_for_the_host` pins press_folded; the
rival that folds through `fold` again fails it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:35:27 -07:00
Edward Langley be99391818 Merge collapse-fu/wm20 (f668ddc9) into collapse-fu/h6z1
# Conflicts:
#	context/code/command.md
#	context/test-evidence/core.md
#	crates/improvise-core/src/model/types.rs
2026-09-23 17:34:11 -07:00
Edward Langley 5738e8e0c0 check-ci-graph: a job's keys start at indent 4, or its lines are refused (rjaa)
w0j4 read any line deeper than 4 outside steps:/needs: as content, so a
needed job whose own keys sat at indent 5 or 6 (valid YAML) had its if:
and needs: missed and the gate exited 0. A deeper line is now content
only once its job has a 4-space key. tooling.md names the three valid
styles the gate refuses loudly.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:34:06 -07:00
Edward Langley 462471871b Pin Tab's rightmost-column clamp and bound the tile-bar walks
improvise-qnry: a lockstep case compares Tab at the rightmost column with
the host's real CommitAndAdvance, and the tile lockstep case now presses
past the last tile. improvise-ybum: browser_round_trip's tile and column
walks are bounded and fail on a stuck press instead of hanging.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:34:00 -07:00
Edward Langley 9872cc936f Every cell-editor opener awaits its seed while a write is unanswered (improvise-p6v6)
mdfc put the "any unanswered write" check inside `advanced` alone, so an
editor opened with i/a (`edit_or_drill`) still seeded from the cache the
write had made stale: `i 9 Tab Esc i 5 Enter`, typed faster than the
answer, stored 15 in the browser where the TUI stores 195.

The in-flight half now lives in `opened_on`, so no opener can forget it.
`advanced` keeps the "this frame writes" half, since the commit being
resolved is not in flight yet; dropping it fails the 3tt4/3uzg tests.

Stacked on #227 (fix/fy9o-reopened-editor-reseeded, de4a7d78).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:33:33 -07:00
Edward Langley 4cef40c951 Strip pathspec and attribute-source settings from the walk's git (improvise-a0i3)
GIT_LITERAL/GLOB/NOGLOB/ICASE_PATHSPECS and GIT_ATTR_SOURCE are
process-wide, so --local-env-vars names none of them. Remove them in
git() beside the local-vars loop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:30:59 -07:00
Edward Langley d7dceef549 a grid evaluates through its own evaluator, never another's memo (opbo)
GridLayout::evaluate_in and column_total took any &FoldMemo, so a memo
from a layout showing Feb, handed to one hiding it, gave 5.0 where 2.0
was right. GridLayout::evaluator(model) now returns a GridEval that owns
a memo its layout built, with cell() and column_total(); the memo-taking
methods and the public fold_memo are gone. grid_body and prune_empty use
it, and the earlier rivals still fail when restated for the new API.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:30:41 -07:00
Edward Langley 2c16f7f2ce Guard App::restore's rebuild-after-view_state order
improvise-3til: rebuilding before view_state is restored passed every test.
A rolled-back frame that deleted the host's view now asserts the host is
back on that view with that view's layout shape.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:29:50 -07:00
Edward Langley da0eb3d7cb improvise-a0i3: red test, the walk under the user's pathspec and attribute settings
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:28:43 -07:00
Edward Langley f668ddc94e rustfmt the yb4b/b5dp changes
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:24:40 -07:00
Edward Langley fd221c6c6c prune_empty shares one fold memo per build (metf)
prune_empty evaluated every cell with a fresh memo, h6z1's shape on the
same keypress path. Measured on an M4 Max: a pruned build of the folded
chain went 705-900 ms -> ~350 ms at 18k and 6.4-6.6 s -> 3.0-5.0 s at
90k; the dense grid is unchanged within noise. A new bench line times
the pruned build. The test counts 9 evaluations where a fresh memo per
cell counts 45, and checks the pruned rows are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:24:34 -07:00
Edward Langley cf5c6fc711 Mark the help rows a browser cannot run, and guard it from one list
improvise-uln5. The browser painted every help-overlay row plainly,
including six whose keys go upstream to a host that answers nothing:
/, n, H, z, Enter, and search mode's Enter/Esc. Every ex-command row was
painted plainly too, although `:` opens a command line the browser
cannot run. A dead key is worse than a gap: the keys typed after it land
on live bindings (improvise-koh6).

- resolve::NEEDS_THE_HOST is now the one list of names this client has
  no arm for. KeyRow keeps the command it was built for.
- The browser's RenderHelpRow draws a listed row, and every ex row while
  the command line is stranded, as `dim` with "(not in the browser yet)".
  It marks rather than omits: both front ends keep drawing the same rows,
  the feature stays findable, and web/ needs no change.
- the_command_names_that_still_need_the_host_are_these now drives that
  list rather than a table of its own.
- every_help_row_the_browser_cannot_run_is_one_it_marks holds it from
  the other side: silent help rows == listed rows, and no hint line
  names one.
- every_advertised_command_answers_in_its_own_mode no longer restates
  the black holes.

Red: a_help_row_the_browser_cannot_run_is_drawn_marked at 202e7eac.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:22:48 -07:00
Edward Langley 1a9d73259c Merge gh/main (202e7eac) into collapse-fu/wm20
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 17:19:55 -07:00
Edward Langley e4b5ce9046 A relay heard while frames are in flight rebases the view on them (improvise-xd3q)
A pruning commit is answered with ResetCursor and an echo of its own
placement. A frame sent before that answer had already been folded
optimistically, and fold applied the relays straight onto view on top of
it. The echo is per frame, so the later placement was never re-sent: the
client sat at (1,0) while its mirror sat at (2,0), and the next write
landed on West where the TUI writes East.

#246 already queues each relay behind the head frame, so the queue holds
the right order. fold now rebuilds view as settled plus everything still
in flight whenever a relay queues, the same replay a refusal uses.
Replaying from settled, rather than re-applying what queued behind the
relay, is pinned by a ViewBack test: re-applying stepped back twice.

Stacked on #246 (fix/hv0r-jpba-relays, 790a33ba).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:19:22 -07:00
Edward Langley adb6b81416 improvise-gk51: the pivot branch too, a shipped-document enumeration, and the argument
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:19:01 -07:00
Edward Langley 076ead7c7c improvise-yb4b/b5dp: pin the scan against the per-member fold; record the ruling, the bench and the evidence
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:18:55 -07:00
Edward Langley 09db8eb0cd The same mode is not the same editor: a message does not follow a commit (improvise-sx7h)
#257 restored a stale message whenever the mode after a key equalled the
mode before it. Editing and RecordsEditing commit with CommitAndAdvance,
which re-opens an equal AppMode on the next cell. So the message followed
the editor from cell to cell until Esc. The reviewer's repro was R, o, 5,
Tab: "Added new record row", or o's "That record row is already here"
refusal, survived the Tab.

press now compares an EditorSession, made of the things an opening sets:
- the whole AppMode, not its variant;
- the cursor cell, which is a cell editor's target;
- the editor's seed, which EnterEditAtCursor re-seeds from the cell, so a
  commit that re-opens on the same cell (Tab in the last column) is new.

A generation counter was rejected: editors are opened by more than one
effect, and a counter in each is a hand-kept list. The view was left out:
no minibuffer key can switch it and stay.

Tests:
- four press scenarios: the two records repros, Enter in a plain grid,
  and Tab in the last column;
- a direct check that ItemAdd on another category is another editor.

Rivals: mode-only (#257 as pushed), no cell, no seed and discriminant
each fail a distinct step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:18:44 -07:00
Edward Langley ff8ea23909 A rename retry after any refusal may rename only the record it named
improvise-9ww8: a crossing rename by another client bumps layout_version,
so the refusal is LayoutMoved, not RecordGone, and the retry recaptured
whatever record was under the cursor. refused() now pins Opened::Retry(r)
for any refused rename of r; the recapture becomes the target only if it
is r, else Gone(r). RecordMoved on the same record stays a force. The
explicit RecordGone pin is subsumed and removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:13:58 -07:00
Edward Langley ca34cdf811 improvise-29w2: a CLOSED PR the loop waits on (MERGEABLE/BLOCKED) pins the state check at the loop top
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:11:23 -07:00
Edward Langley 3d374d8b3e improvise-gk51 WIP: one predicate for the projection and the refusal; CellId::NonItem
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:09:25 -07:00
Edward Langley 2ea2407456 Pin the column group toggle to the cursor; guard run identity in e265's enumeration (improvise-uv9e, improvise-6u6s)
uv9e: ToggleGroupAtCursor { is_row: false } (gz) had no command-level
test, and a version reading col_group_for(0) passed the workspace. A new
test seats the cursor at (0, 2) with Month on the columns and requires
Q2. Reading column 0, or reading the row half of the cursor, both fail
it. The row half was already closed: the grid tests sit on row 2, and
injecting row_group_for(0) fails five of them. The browser's z is still
Upstream with no resolver arm (improvise-uln5), so it has no cursor
reading to test.

6u6s: the e265 enumeration could not see run identity. A pruner keying
runs by (level, category, group name) instead of header position
passed. The enumeration now:
- compares every level's run name and whether each survivor shares a
  run with the previous one;
- adds three fixtures: a non-contiguous group, a level with no groups,
  and two adjacent outer groups;
- builds each layout once per collapse subset and drives
  prune_axis_items with a keep predicate per row set: 4,992 cases.

The name-keyed rival now fails at fixture 2, mask 0b101. open_runs is
the one reader of the markers. improvise-core's lib went from
0.86-0.92s to 0.17-0.23s on an Apple M4 Max under load average 39-77.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:08:02 -07:00
Edward Langley de4a7d7862 protocol.md §3: name the type-ahead divergence that remains after mdfc (improvise-n4i9)
Measured in Chromium over the worker: with keys pressed back to back,
`i 9 Tab Tab 5 Enter` stores 5 or 5095 depending on whether `5` beats the
answer; with a pause, 5095 in 5 of 5 runs. Never the stale 505.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:07:53 -07:00
Edward Langley 97a9379f79 bd-audit-ids: deleted means a deletion git will record
improvise-cota, found in the delta review of #244. The tdfd fix skipped an
absent tracked path whenever `ls-files -d` listed it. `ls-files -d` also
lists an absent --assume-unchanged path. `status` hides that path, and
`commit -a` never records the deletion, so its citations stayed in HEAD
unread and the audit exited 0.

The deletion oracle is now `diff --name-only --diff-filter=D`. It honours
both index flags and still lists a plain unstaged delete.

The new case in bd-audit-ids-test.sh was red at 33d563a9. A rival that
treats every absent path as unread fails only the deliberate
deleted-file control.

Stacked on #244 (tooling/lqs0-audit-exits).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:06:23 -07:00
Edward Langley 59072d8105 improvise-gk51: red test, a projected Record the host then refuses
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:04:39 -07:00
Edward Langley 4ff79ff21e WIP red: a rename retry after a crossing LayoutMoved retargets (improvise-9ww8)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:04:25 -07:00
Edward Langley ec85401a44 improvise-yb4b: a stored fold scans the store once instead of looking up each member
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:03:05 -07:00
Edward Langley 86b3d72c70 Guard App::restore's layout rebuild with a host-level test
improvise-8wy9: deleting self.rebuild_layout() from App::restore kept the
workspace green. A refused frame that removes an empty row is rolled back,
and the next frame's write to that row must resolve against the restored
grid; without the rebuild it is refused RecordGone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:02:29 -07:00
Edward Langley 3eafd4dec7 check-ci-graph: refuse a key or needs: it cannot read plainly (w0j4)
Quoted, anchored and multi-line needs: values, and quoted, spaced or
explicit ? keys, all skipped kb on a PR while the gate reported every
suite run. Rather than parse more YAML, every structural line inside
jobs: must take one plain form or the gate names it and fails. Also a
1be2 case at the one-column boundary, which the owner+1 rival passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:02:13 -07:00
Edward Langley 322de37053 Init scratch repos with an empty template directory
improvise-pr3i, found reviewing #255. git init copies GIT_TEMPLATE_DIR
into the new .git. That variable is not in --local-env-vars, and it
bypasses config, so a template whose info/exclude holds *.gz still broke
a_gzipped_document_is_walked at git add. scratch_repo now runs
`git init -q --template=`, which copies no template at all, including a
distribution's compiled-in default. Stripping the variable would still
copy that default.

Also: rerun_with_env's callers print the child's stderr as well as its
stdout (told), because the child's panic goes to stderr.

Stacked on #255 (fix/vwjj-default-excludes).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 17:01:33 -07:00
Edward Langley 74920ade5d improvise-b5dp: pin the pooled-ratio ruling for a formula column's total; bench mid-size cases
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 16:59:40 -07:00
Edward Langley 3fcd407b08 A reopened editor awaits any unanswered write, settles only while editing, and states its awaiting flag (improvise-mdfc, improvise-4cb5, improvise-mgis)
improvise-mdfc: an untouched commit seeded the reopened editor from the
cache even while an earlier write from this client was unanswered, so
Tab Tab at the rightmost column stored 505 where the TUI stores 5095.
`advanced` now awaits when this frame writes or any in-flight frame is
awaited. The lockstep test sends three Tabs before answering any.

improvise-4cb5: `settle_awaited_seed` also requires an editing mode, so
an Esc before the answer no longer leaves the seed in Normal's buffer.

improvise-mgis: `OpenedEditor::awaiting` loses `serde(default)`; a frame
missing it fails to parse as the missing field (protocol.md §3, 1wl0).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 16:59:23 -07:00
Edward Langley 8cdea3a430 improvise-24lz: the head checks precede the ended-PR exit; CLOSED fixture takes its live shape
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 16:58:59 -07:00
Edward Langley c349d14dd6 lockstep: deliver folds host frames through fold, not fold_outgoing
improvise-xoxh, found in the delta review of #246. deliver hands a client
what the HOST sent, so a relay must go through fold, which queues it
directly behind the awaited head. fold_outgoing queues it at the back as
the client's own frame. That is the pre-jpba ordering, so a test built on
deliver could not see a jpba regression. The bug was latent, because
deliver's one caller sent only view-only frames.

deliver_queues_a_host_relay_behind_the_awaited_head was red at 790a33ba.
With deliver fixed, putting the pre-jpba ordering back in settle_relay
fails this test and nothing else in the lockstep binary.

The renaming test that came in through the gh/main merge now folds its
own commands through fold_outgoing. It also asserts that the commit is
the one awaited frame, so the two doors are told apart.

Stacked on #246 (fix/hv0r-jpba-relays).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 16:57:50 -07:00
Edward Langley 6b3577a09e fold memo owns its agg and covers; grid_body's sharing is guarded (pntu, 0jw5)
improvise-pntu: FoldMemo was keyed by (key, span) while the aggregation
and covers were call arguments, so one memo across two covers answered
the first (5.0 where 2.0 was right). FoldMemo::new(agg, covers) now owns
both and the _in functions read them off it; GridLayout::fold_memo builds
the grid's. The mismatched call no longer compiles.

improvise-0jw5: a test in improvise-command counts fold evaluations across
one grid_body call (18) against a fresh-memo control (90). A per-cell or
per-total fresh memo fails it. The counter is exposed through a new
test-support feature on improvise-core, enabled only as a dev-dependency.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 16:57:06 -07:00
Edward Langley 2b1c5bf7ad A minibuffer refusal stands while the user fixes it (improvise-su4u)
#233 clears the status message at the start of every TUI keypress. In a
text-entry mode the message sits beside the prompt, and a formula that
does not parse keeps the editor open with the error. So the error
vanished on the first correcting key.

press now remembers the mode and the message. It clears the message as
before and dispatches the key. It puts the stale message back only if
the key left the user in the same text-entry mode and wrote no message
of its own. Editing keys keep the refusal on screen, a key's own message
still wins, and leaving the minibuffer (a successful commit or Esc)
drops it.

The mode check is observed, not predicted: which keys commit is the
keymap's knowledge, and the TUI loop does not copy it.

Rivals, each injected into press and restored: never restoring (#233),
restoring over a key's own message, and restoring whatever the mode each
fail a distinct step of the test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:33:00 -07:00
Edward Langley eb146bccab Pin the browser's Tab-commit advance and the tile cursor's upper clamp
improvise-rq3f: two resolver faults passed the whole workspace. Two unit
tests in resolve.rs now fail under each, and only under its own fault.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:29:13 -07:00
Edward Langley 56ac72a291 Null the default excludes and attributes files for scratch repos
improvise-vwjj, found reviewing #248. GIT_CONFIG_GLOBAL=/dev/null empties
the config files. It does not stop git falling back to
$XDG_CONFIG_HOME/git/ignore (or ~/.config/git/ignore) when
core.excludesFile is unset, and git/attributes has the same kind of
default. So a *.gz line in the user's default ignore file still failed
a_gzipped_document_is_walked.

run_git now passes -c core.excludesFile=/dev/null and
-c core.attributesFile=/dev/null. A -c argument outranks every config
file and every default path.

The new test re-runs the gzip shape test under an XDG_CONFIG_HOME whose
git/ignore holds *.gz, and again with a git/attributes line that breaks
git add. Each half was red on its own.

Stacked on #248 (fix/53zz-scratch-repo-env).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:29:01 -07:00
Edward Langley 67438b8f66 test: guard every field App::restore puts back; assert the records coordinate by value
improvise-4h8a: restore_puts_back_every_field_the_snapshot_took writes all
thirteen AppSnapshot fields, asserts each moved, restores, and asserts each
is back. Dropping any one assignment from App::restore, or restoring
view_state as only active_view + status_msg, fails it naming the field.

improvise-sfwy: records_mode_records_display_returns_values asserted the
Region column was non-empty; it now asserts Some("North"). A GARBAGE or
category-name fault passed the old assertion and fails the new one.

Test-only; production code is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:23:28 -07:00
Edward Langley 47b166eed9 Refuse a wire write unless every coordinate names an item
improvise-nacd and improvise-pr8t.

The b114 rule refused a coordinate that is a group. The invariant it
protects (improvise-gw18) is positive: every coordinate names an item.
Two writes onto a stored record fell in the gap and were accepted: a
measure that is no item at all (Bogus), and a parent group that holds no
item directly (Totals), where items_in_group is empty.

App::group_refusal becomes App::non_item_refusal. It refuses any
coordinate that is not in Model::effective_item_names for its category.
Formula targets count as _Measure items, so a write under a formula is
still refused by the Derived rule, with the true reason.

Tests:
- pr8t pins the same-name case: an item named like a group of its
  category is writable. Removing the old exemption fails it.
- nacd is red at 202e7eac for both Bogus and Totals.
- A third test pins that a formula cell is refused for the formula.

Two reduce.rs fixtures wrote _Measure=Revenue without the item. They now
add it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:20:51 -07:00
Edward Langley d6e366dceb Group runs nest: markers name their level, pruning re-marks (improvise-e265)
On an axis with two grouped categories, runs nest: North in Upper holds
Jan in Q1, and after Q1 ends North/Mar is still in Upper. The markers
carried no level, and every reader was flat, so the nearest marker
decided. Pruning an inner header dropped its GroupEnd too, and a
surviving row changed group. The unpruned answer was lossy as well:
North/Mar was in no group while it was drawn under Upper.

The user ruled per-level membership, with z toggling the innermost group
(the one nearest the cursor). So:

- AxisEntry::GroupHeader and GroupEnd carry a level. expand_category
  reads it off its prefix, which holds one item per outer category.
  Membership is one open run per level, and a row's group is the
  innermost one (row_group_for/col_group_for, hence z).
- prune_axis_items re-marks the survivors instead of filtering the
  markers. Before each kept item it emits the markers that make the
  output's open runs equal the input's, closing deeper runs innermost-out
  first. So each survivor keeps exactly its runs, no header is kept over
  nothing, no trailing GroupEnd is left, and when nothing is pruned the
  output equals the input. This replaces the pending/close_runs_of/
  run_open machinery.
- The wire: FrameGroup gains level (an index into AxisFrame::cats), and
  FrameEntry::GroupEnd becomes GroupEnd { level }. groups()/group_of
  read the same per-level rule, and group_runs is unchanged (h0iv).
- axis_frame carries the open run at each level above a scrolled
  window, not the single nearest header, so a window opening inside
  North/Q1 still knows Upper.

Tests:
- an enumeration over every funded mask x collapse subset of a two- and
  a three-level fixture (about 4,000 cases), asserting that pruning
  moves no row's group, that no header is drawn over nothing and no axis
  ends on a marker, and that nothing-pruned is marker-for-marker equal;
- the ruling at the layout and through z;
- frame/layout agreement at every window offset;
- the nested window carry.

Rivals, each injected and restored, are in the evidence files: flat
membership, a pruner that owes no GroupEnd, ends passed through eagerly,
headers kept eagerly, nearest-only window carry, and a flat frame.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:20:03 -07:00
Edward Langley b13355c74b improvise-9pfs/1qaa: pin a red under BLOCKED; a merged or closed PR ends the watch with exit 5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:19:54 -07:00
Edward Langley 29d2adef6b Evidence: never_clear_opened also fails the swap test
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:15:15 -07:00
Edward Langley 33d563a902 Merge commit '202e7eac637a95c8ac0814cb65a5739bcd3afe33' into tooling/lqs0-audit-exits 2026-09-23 11:14:25 -07:00
Edward Langley acecc63dff bd-audit-ids: only ls-files -d says deleted; other absent paths are exit 4 (tdfd)
A skip-worktree or sparse path is absent without being deleted, and
the filter skipped it, so its ids were never read and the audit
passed. The xargs comment no longer claims 124, which is GNU only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:14:24 -07:00
Edward Langley c0442e68da improvise-m6op: the cross-instance fixture carries measure_agg entries, so its sort is pinned
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:12:40 -07:00
Edward Langley 94d93a9cca Merge fix/fdk1-coordinate-rename (39821d5f) into fix/20ek-milr-rename-races
The rename capture now carries both the records seen at open (milr) and the
document generation (vndi); the claim passes both.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:10:44 -07:00
Edward Langley abf4bb5f38 Merge commit '202e7eac637a95c8ac0814cb65a5739bcd3afe33' into tooling/7fl8-exact-polls
# Conflicts:
#	context/test-evidence/tooling.md
2026-09-23 11:10:40 -07:00
Edward Langley e45a9c4206 checkers: the bash floor is 5.1, where wait -n -p arrived (fd0j)
find_modern_bash accepted any 5.x, so a bash 5.0 was pinned and
pr-reviews-test.sh then failed on an invalid option. The probe now
reads major and minor; a 5.0 case and a 5.1 control guard it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:10:15 -07:00
Edward Langley f5cfc7edd3 Isolate shipped_documents_load's scratch repos from the caller's git env
improvise-53zz. The scratch-repo shape tests inherited the caller's git
environment. A hook in a linked worktree exports GIT_DIR (and
GIT_INDEX_FILE), which override -C, so `git -C <scratch> add` staged the
scratch files into the outer checkout's index. The user's global
excludesFile applied too, so a `*.gz` line there broke the gzip test.

One helper, git(root), now removes every name that
`git rev-parse --local-env-vars` prints. run_git also sets
GIT_CONFIG_GLOBAL=/dev/null and GIT_CONFIG_NOSYSTEM=1.

Two new tests re-run a shape test in a child process of this binary with
the variables set. Both were red before the fix. Five rivals are recorded
in context/test-evidence/tui.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:10:11 -07:00
Edward Langley 39821d5fb4 Compose the rename capture with vndi's generation and wdmp's placement
claim_as_read now takes the document generation read at open (#219,
improvise-vndi), so OpenedCoordinate captures it and the rename claims it.
A swap closes the rename editor, so no rival of that is observable;
a_document_swap_closes_an_open_rename_editor pins the precondition.

Since #230 (improvise-wdmp) the asker keeps its placement after a reframing
write, so the reorder test's re-entered editor can sit on Zed; its value
table now includes Zed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:09:37 -07:00
Edward Langley 2aafff3b4c Merge commit '202e7eac637a95c8ac0814cb65a5739bcd3afe33' into tooling/dak5-ci-suites 2026-09-23 11:06:54 -07:00
Edward Langley 1501def5c2 check-ci-graph: a continued run: or a skipped need does not count
A plain scalar folds a deeper-indented next line into run:, so
`|| true` there made the counted call fail-open (improvise-1be2).
A job that needs a job under if: is skipped with it, directly or
transitively (improvise-bbwa). The unrun message now points at the
ci_suites row. tooling.md names the checkout-with-ref hole.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:06:47 -07:00
Edward Langley 790a33ba8e Merge gh/main 202e7eac into fix/hv0r-jpba-relays
Evidence: append/append in clients.md, gh/main's entries first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:04:51 -07:00
Edward Langley e0dec9660e improvise-3kfc: a refusal settles what queued behind it; lockstep folds its own commands outgoing
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:03:53 -07:00
Edward Langley 006a156482 Merge gh/main (202e7eac) into fix/fdk1-coordinate-rename
ModelCommand is 26 with #214's ReplaceFormula: both sides had edited the
reduce.rs count to 25, which auto-merged unchanged; fixed by hand.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 11:02:40 -07:00
Edward Langley baabe1a9b1 Evidence and the argument for seeding a reopened editor from the host's answer (improvise-3uzg, improvise-xbgm)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 10:59:41 -07:00
Edward Langley 409ae6a8df Merge gh/main (202e7eac) into fix/oplp-followups
EOF conflict in context/test-evidence/clients.md: both sides appended
entries. Both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 10:55:18 -07:00
Edward Langley 1d2032ef82 Merge remote-tracking branch 'gh/main' into fix/fy9o-reopened-editor-reseeded
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
#	crates/improvise-wasm-client/tests/lockstep.rs
2026-09-23 10:54:59 -07:00
Edward Langley 9975347b38 In a drill, tile select's fallback is a Label or _Index, not _Measure (improvise-h3cv)
command.md said the fallback is always _Measure. In _Drill, drill_steps
puts _Measure on Filter and tileable_cats drops Filter, so it is not a
tile. The Label rank and the _Index fallback are live exactly there.

Two TUI tests drive a real drill (>) and T: with a Label on None the
cursor opens on it, without one on _Index. Rivals: #231's
first-regular-else-zero and a Label ranked below _Index both fail the
first. They are not lockstep tests: a browser declines >.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 10:54:36 -07:00
Edward Langley 96ecfd6018 After a write, the reopened editor takes its seed from the host's answer (improvise-3uzg)
landed_seed assumed the client's cache was stale only at the cell the frame
wrote. Under prune_empty a write can take its row away and renumber the rest,
so the advance landed on a position the cache still filled with another
record. The advance after a write now opens the editor awaiting its seed, and
the client settles it from the grid it holds once nothing it sent is
unanswered. A seed never overwrites typing that arrived first.

This also removes CellValue::edit_text from the client bundle, so the
nontrapping-float-to-int wasm-opt flag a093d126 added is no longer needed and
is reverted (improvise-xbgm).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 10:53:59 -07:00
Edward Langley ce7a57bb1f Merge gh/main (4502ed2b) into fix/oplp-followups
Conflict in context/code/formula-and-io.md: #214 added FormulaId to the
wire-type list, this branch added CategoryKind. Both kept.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 10:47:53 -07:00
Edward Langley 4f75fffcb9 improvise-h6z1: one fold memo per grid build, keyed by fold key and span
viewport::grid_body makes one FoldMemo and passes it to every cell and
total through GridLayout::evaluate_in / column_total, so a fold row's
formula chain is evaluated once across the row (9, not 45). The bench
gains a folded five-formula chain case; stored-only grids are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 09:09:34 -07:00
Edward Langley 23b8dbf56d Merge remote-tracking branch 'gh/main' into fix/zs2r-harness-lock-gate 2026-09-23 09:05:45 -07:00
Edward Langley e26498209b tooling: bd-audit-ids tells grep's no-match from a read error (lqs0)
improvise-lqs0, from the #226 review. xargs reports grep's no-match and
its read error alike, as 123. So a cited file the scan could not read
passed the audit, and an unreadable file holding the only dead id exited
0. Each grep batch now maps exit 1 to success and exit 2 to 255, so xargs
stops and the audit is exit 4. A tracked file deleted from the working
tree is left out on purpose, because its ids are leaving the tree.

Two exit-code mismatches with the stated contract are fixed too. Running
outside a repository is now exit 4, not 1. A bd list whose objects carry
no string id is now exit 3, not "every id dead".

Six new cases reproduced the bug as 10 passed, 4 failed, and now pass
14/14. Four rivals each fail their own case.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 09:05:43 -07:00
Edward Langley c4450a4d68 A fast, offline gate for a stale harness/opfs-contract lock (improvise-zs2r) 2026-09-23 09:04:47 -07:00
Edward Langley 6c09f35f56 Merge commit '4502ed2b1178ebff92d341ab88567ab801534fd8' into wave/tej1-browser-delete-formula
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
2026-09-23 09:01:20 -07:00
Edward Langley 600d0a01d3 Merge commit 'a634b3f356bdbc725bf2c42bb5a56e0b2f2ef0eb' into fix/we1a-abort-refuses
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 08:59:59 -07:00
Edward Langley 92515d6271 Keep rename_coordinate's doc comment on the function
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:49 -07:00
Edward Langley d5da9747ee rustfmt
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:49 -07:00
Edward Langley 542dd9ce2e improvise-jpba: a relay heard while a frame is in flight outlives its refusal; evidence and docs
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:43 -07:00
Edward Langley dc944ea24b Document the RecordGone retry and the rename destination's basis
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:42 -07:00
Edward Langley d6706774a2 tooling: CI runs checkers.sh --ci <job>; the gate reads a table, not shell
improvise-st8q, improvise-tvte, from the #228 re-review at c1417359. The
maax parser still failed open. A bare checkers.sh inside a heredoc or a
folded run: value counted as every suite, and step and job if: were
ignored. Every round of parsing met another shell or YAML form, so the
parsing is gone.

checkers.sh holds a job-to-suites table (ci_suites), and each CI job
makes one call, `checkers.sh --ci <job>`. check-ci-graph.py counts a step
only when all of these hold: it sits inside steps: at step-key
indentation, its whole run: value is that call, the call names its own
job, and no if:, continue-on-error:, shell: or defaults: covers it. Every
other mention of scripts/checkers.sh is reported. A key with an inline
value owns deeper lines, and an open quote owns lines until it closes, so
heredocs, folds and quoted values are content, never structure.

There are 19 suite self-test cases and 7 rivals, each failing a case only
it can. The reviewer's heredoc, fold and if: attacks on the real ci.yml
now exit 1. checkers-test.sh gains five --ci cases. context/code/tooling.md
corrects two overstated claims ("--would-run anywhere is a usage error",
"fails closed").

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:18 -07:00
Edward Langley efca099cae improvise-hv0r/jpba: relays settle after their frame; pin the echo's audience and kinds
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:17 -07:00
Edward Langley 3e435e88df Refuse a rename retry after RecordGone; check the rename's destination
improvise-20ek: a RecordGone on the record a rename named pins the editor's
target as Gone, so a retry declines instead of renaming whatever row the
cursor now sits on. RecordMoved still recaptures, which is the force.

improvise-milr: the server adds the rename's destination record, when it
exists and differs from the source, to the records a frame answers for
(records_at_stake). The client cannot compute that hash (ADR 002), so it
claims every record counter it held when the editor opened.

Also: ModelCommand has 25 variants, not 24.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:59:12 -07:00
Edward Langley 0d14f49e51 WIP red: a retry after RecordGone, and a rename's unchecked destination (improvise-20ek, improvise-milr) 2026-09-23 08:58:21 -07:00
Edward Langley de1524c093 CategoryKind is a wire type again: TileEntry carries it
improvise-tt22 merged into gh/main while this branch was open and dropped
serde from core's document types, CategoryKind included, keeping only the
wire types (Axis, PageAxis). TileEntry now carries a CategoryKind, so it
joins them. formula-and-io.md's list of what remains is updated to say so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:56:55 -07:00
Edward Langley 42aeeb6b03 rustfmt
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:53:43 -07:00
Edward Langley 1b9a285802 Merge gh/main (5932ad74) into fix/fdk1-coordinate-rename
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:52:57 -07:00
Edward Langley 1257789e33 Merge gh/main (ca2d1b19) into fix/fdk1-coordinate-rename
Composes fdk1's coordinate rename with #205's opened editor (improvise-ixxx):
a coordinate editor opens through opened_on and records SetOpenedEditor like
any value editor, and an untouched rename writes nothing, as the host's
CommitAndAdvance already did. The record/version capture for the basis claim
stays in ClientState::opened; context/code/clients.md argues why it cannot
live in OpenedEditor. Also: core.rs E0533 (CellId::Coordinate is a struct
variant since #197), and the ModelCommand count prose (25).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:51:44 -07:00
Edward Langley 9da9e08e57 Merge commit '5932ad749f0df01d4321cc961f57a488ab637dde' into fix/we1a-abort-refuses
# Conflicts:
#	context/test-evidence/command.md
#	crates/improvise-command/src/ui/app.rs
2026-09-23 08:51:42 -07:00
Edward Langley 9948aaf55a rustfmt the wm20 totals tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:51:01 -07:00
Edward Langley c9bbe32ec5 Merge remote-tracking branch 'gh/main' into fix/i3o5b-panel-dispatch-refuse
# Conflicts:
#	context/code/cli.md
#	context/test-evidence/command.md
#	crates/improvise-command/src/ui/app.rs
2026-09-23 08:49:08 -07:00
Edward Langley 66bd3fcbf2 Tile select ranks by CategoryKind, and the docs and tests catch up (oplp follow-ups)
improvise-bqm0: opening_tile fell back to tile 0, _Index, when a view had
no Regular category, so T Enter still emptied that grid. TileEntry now
carries the host's CategoryKind in place of `regular: bool`, and
opening_tile ranks Regular, then _Measure, then Label, then _Index/_Dim,
leftmost on a tie, through an exhaustive match. A bool could not express
the fallback without a client sniffing names. CategoryKind is re-exported
from the protocol so the wasm client names it through its one dependency.

improvise-52od: in every fixture the first Regular tile was also the first
tile on Row, so a rival `position(axis == Row)` passed the workspace. A
protocol unit test on a records-shaped bar and a lockstep case in records
mode now fail that rival; a no-regular unit test and lockstep case fail
#231's own `first regular, else 0`.

improvise-q69m: context/code/command.md said tile 0 is _Index and that
the fix would be the tile order, and clients.md's Rule 3 counterexample
named the leftmost tile. Both now describe opening_tile, and command.md
says why placing the cursor beat reordering: click targets are indices,
and the offered order is #192's pinned characterization.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:48:54 -07:00
Edward Langley 426cce3732 Merge gh/main (5932ad74) into collapse-fu/wm20
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:48:38 -07:00
Edward Langley ce56e9b248 Merge gh/main (b4779ed) into tooling/dak5-ci-suites
Resolved context/test-evidence/tooling.md: gh/main's version, with
#226's bd-audit-ids entry, plus this branch's dak5 and maax entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:48:29 -07:00
Edward Langley bdf2f2b76e Cite improvise-cwfj, the bead this remainder is tracked as 2026-09-23 08:48:12 -07:00
Edward Langley fb3da78741 improvise-wm20: a column total is its cell as if every drawn row were one fold
GridLayout::column_total asks Model::evaluate_spanning, whose Span adds to
#203's fold the categories a key omits. A formula is evaluated over the
pooled rows, a stored measure folds its records under its own aggregator,
text is unanimous or mixed, an error propagates, and _Measure on rows
leaves no strip (the user's rulings). Hidden rows stay out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:48:11 -07:00
Edward Langley 785202b8f0 tooling: pr-reviews timeout case asserts WAIT+1 polls; a hang fails (7fl8)
improvise-7fl8: on the one-tick-per-read clock --wait 5 polls exactly 6
times, but the test asserted at least 5. So a loop capped at 5 polls, or
a deadline of now + WAIT - 1, passed at 25/25. It now asserts
polls = WAIT + 1, and both rivals fail it.

With the deadline check deleted, the suite used to hang at the first
real-clock timeout case. Every pr-reviews call now runs beside a
/bin/sleep watchdog (wait -n -p), and loses to it as exit 124. The same
rival now finishes in 93s with four FAILs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:46:41 -07:00
Edward Langley a093d126bc Let the client bundle's wasm-opt accept saturating float conversion; rustfmt
landed_seed parses committed text with typed(), which puts
i64.trunc_sat_f64_s in the client bundle; the bundled wasm-opt rejects it
unless the feature is named, as the worker bundle already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:44:56 -07:00
Edward Langley 2fa34359e3 context: panel and dispatch refusals; a bound key that will not build is a refusal (improvise-i3o5) 2026-09-23 08:44:44 -07:00
Edward Langley 301b1f8fc1 WIP: panel.rs declines and dispatch's arg-error arm refuse (improvise-i3o5) 2026-09-23 08:42:58 -07:00
Edward Langley 3475b02956 improvise-9trl: pin the remove-formula refusal and its exit code
A script that removes a formula that is not there now refuses and exits 1,
where it used to exit 0. The None arm had no test, so reverting it to a silent
no-op left the suite green.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:39:57 -07:00
Edward Langley f3069032f5 Merge remote-tracking branch 'gh/main' into fix/fy9o-reopened-editor-reseeded 2026-09-23 08:38:28 -07:00
Edward Langley 85c483bd24 Tab at the rightmost column reopens on the value just written; pin the +cat clear (improvise-3tt4, improvise-5wai)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:38:25 -07:00
Edward Langley 9a86981876 Merge branch 'wave/km71-edit-formula' into wave/tej1-browser-delete-formula
# Conflicts:
#	context/test-evidence/clients.md
2026-09-23 08:36:56 -07:00
Edward Langley c141735977 tooling: check-ci-graph asks checkers.sh what a step runs (improvise-maax)
The #228 review and Codex's P1 found the suite check failing open. A bare
scripts/checkers.sh after an inline comment, in a step name or as echo's
argument read as "every suite runs". --list alone read as every suite, and
--self-test's discarded names, and --list's, counted as run.

checkers.sh gains --would-run, honoured only as the first argument. It
reports what the rest of the arguments would run, from the real argument
loop and name resolution. check-ci-graph.py only finds invocations: in run:
text, comments stripped by bash's rule, command position, every command on
the line, quotes removed. It then asks --would-run about each. An
unparseable line contributes nothing and fails closed.

16 suite self-test cases, 8 of which the old parser gets wrong. Six
rivals each fail a case only they can. checkers-test.sh gains four
--would-run cases.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:36:13 -07:00
Edward Langley 60179150b8 improvise-wm20: red — the totals strip sums rows whatever the column is
Ratio 1.40 (pooled 0.80), Avg 45 (records mean 20), text 0.00, an error
row dropped, a strip under measures-on-rows, and a Max total that moves on z.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:35:06 -07:00
Edward Langley 66b6a973dd Merge commit 'e406edc8e846f147dc0828f7087ab2a4008a41c9' into fix/we1a-abort-refuses
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 08:30:36 -07:00
Edward Langley 425a517426 Check in apply_effects that a model effect never aborts without refusing (improvise-lasd)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:29:16 -07:00
Edward Langley af61e1bf8d Merge gh/main (194d8f5) into tooling/dak5-ci-suites
Resolved context/test-evidence/tooling.md: gh/main's version, with
#218's two entries, plus this branch's improvise-dak5 entry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:27:55 -07:00
Edward Langley a1b23c0595 Merge remote-tracking branch 'gh/main' into fix/fy9o-reopened-editor-reseeded
# Conflicts:
#	context/test-evidence/clients.md
#	context/test-evidence/command.md
2026-09-23 08:26:51 -07:00
Edward Langley 18c82e1ccc ADR 007: fold in the user's second-round answers; status stays proposed (improvise-sboz)
Thirteen inline answers on 5e46ce03 settle Q5, Q6 and Q8 to Q13. Each is
quoted verbatim with the line it sits on. Q14 is settled in part. The
affected clauses are rewritten:
- slugify: fold to lower case, then strip to [a-z0-9_-], applied to all
  id generation;
- old files are read whole-line, selected by an exact version literal;
- the host assigns ids, and create-then-act becomes one command;
- a duplicate id refuses the load;
- ids may be reused across a reload.

Q15 to Q17 are left open: the order of slugify's steps, what the formula
panel shows, and whether hand-written ids may contain capitals.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:26:35 -07:00
Edward Langley cc070529a4 Merge gh/main (968f65af) into fix/fdk1-coordinate-rename
Resolves context/test-evidence/{clients,command}.md: both sides appended at EOF;
kept gh/main's entries, then this branch's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:17:24 -07:00
Edward Langley 2fea6e8c64 Merge commit '968f65afe02ce400facce953659a54b0058a49b8' into fix/qz9p-drill-refusal
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 08:17:08 -07:00
Edward Langley a79ed50878 Pin the renames half of the qz9p check; say every staged edit was discarded (improvise-6r2y, improvise-o9q7)
6r2y: a sparse drill row with no Type coordinate that stages a Type
rename after Type is removed reaches the unwritable check only through
its renames. Red with .chain(renames.keys()) removed: an orphan
Drink/Jan/Cost cell is written.

o9q7: when any row is unwritable, merged keeps only the unwritable
rows' reasons, as one refusal saying every staged edit was discarded,
so a later writable row's blank-coordinate refusal cannot overwrite it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:16:02 -07:00
Edward Langley a1d27cc1cf Merge commit '33cf49692781f1bd0f4b7654bce6ac9f7a30cc4f' into fix/we1a-rtud-refusal-tests 2026-09-23 08:06:53 -07:00
Edward Langley eb061391fe Pin that every aborting model effect also refuses (improvise-we1a)
Host::mutating_frame rejects on refused || aborted, and deleting the
aborted arm left the workspace green. The wire table cannot see an
abort-without-refuse because that arm is what rejects it, so the new
table applies each failing ModelCommand's effects to a bare App and
asserts it refused, pinning which cases abort so it is not vacuous.
The arm is kept as defence: AbortChain shows an abort need not be an
error, so making abort-without-refuse unconstructible is declined.

Also keeps rtud's three key-path table rows (load, import-json of an
absent JSON, a formula that does not parse) as added coverage; rtud's
prose fix lands separately.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:06:45 -07:00
Edward Langley cee61b94b9 tooling: check-ci-graph fails when a checkers.sh suite runs in no CI step
improvise-dak5: ci.yml names its checker suites by hand, so a new
scripts/X-test.sh ran locally and never in CI. check-ci-graph.py now
asks `checkers.sh --list` for the suites, so the naming rule keeps one
home, and reads every checkers.sh invocation in ci.yml. A bare run
covers every suite, --self-test covers `checkers`, and a YAML comment is
never a run.

A suite planted in the real tree fails the check, naming it, and adding
it to the step clears it. Seven new self-test cases. Rivals: counting
comments fails the comment case; answering "everything runs" fails five.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:06:15 -07:00
Edward Langley 1af56ecfec Merge branch 'wave/km71-edit-formula' into wave/tej1-browser-delete-formula 2026-09-23 08:02:33 -07:00
Edward Langley 8ee5fade27 fy9o: corner lockstep guard, evidence and protocol.md; file improvise-jmti
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:57:01 -07:00
Edward Langley 7d45b11d14 improvise-tej1: delete the formula under the cursor from the browser
The browser composes delete-formula-at-cursor now that FormulaCache carries
each formula's id. RemoveFormula names its formula by FormulaId on the wire and
in the effect, and a delete whose id no longer resolves is refused with the
batch stopped rather than silently doing nothing, so a frame naming a formula
another client removed is rolled back whole. The remove-formula script command
resolves its names to an id and refuses when nothing matches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:54:27 -07:00
Edward Langley 8490f34f98 A drill commit naming a removed category is refused and writes nothing (improvise-qz9p)
While a drill was open, another client's RemoveCategory could remove a
category the drill's rows name. On commit, AddItem refused, but the rewrite's
clears and writes still ran, leaving a cell keyed under a category the model no
longer has. A measure edit alone on such a row was worse: no AddItem, so no
refusal, and the orphan was written silently.

RecordRewrite::of now asks whether every category a row names, its
coordinates and its renames, still exists. If one is gone, the row is
unwritable: it refuses and carries no writes. merged drops every write when any
row is unwritable, because a drill commit is atomic (design-principles §4). The
effect refuses rather than stops (gotcha 21): ViewBack after it assumes nothing
about the writes, and the category cannot come back through the drill, so
stopping would trap the user.

Reasoning: context/code/command.md, RecordRewrite.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:53:48 -07:00
Edward Langley ca19b7686d A reopened cell editor shows the cell it landed on (improvise-fy9o)
The cell-editing commits own their buffer clear at their head instead of
the keymap welding one after them, so the re-seed survives and Enter,
Enter writes nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:53:18 -07:00
Edward Langley ee5cba8865 Merge remote-tracking branch 'gh/main' into fix/fdk1-coordinate-rename
# Conflicts:
#	context/test-evidence/command.md
2026-09-23 07:47:48 -07:00
Edward Langley 3a56daa509 WIP improvise-tej1: compose delete-formula-at-cursor in the browser
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:47:29 -07:00
Edward Langley 5e46ce0357 ADR 007: fold in the user's answers on #210; status stays proposed (improvise-sboz)
Storage and every reference key by id, with a display field for the human
name. Ids are interned symbols made by slugify. The formula id goes after the
target. The space before the id tag is optional. Data keys use ids. RecordHash is
defined over ids. Legacy JSON goes (improvise-tt22). Each answer is quoted
verbatim under "What the user decided". Still open: Q5, Q6, Q8 (restated), Q9,
Q10 and Q11 (restated), and new Q12 to Q14.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 07:46:28 -07:00
Edward Langley 6319a4be89 rustfmt (improvise-fdk1) 2026-09-23 07:43:27 -07:00
Edward Langley 31e0190b61 Evidence: one heading per test (improvise-fdk1) 2026-09-23 07:42:20 -07:00
Edward Langley 9bea7c57b8 Docs and evidence: the coordinate rename (improvise-fdk1) 2026-09-23 07:41:14 -07:00
Edward Langley 25b804c187 ADR 007 (was 005): fix the premises the review found before the user answers (improvise-sboz)
Renumbered: ADR 005 and 006 are claimed by improvise-l2b8 and improvise-ilvl.
Restates Q3 (improvise-80rm), completes Q1 and Q9 (improvise-65kf,
improvise-gkyo), and adds Q10, id reuse after reload (improvise-8tk5), and Q11,
who assigns ids. Fixes id_token and the formula round-trip vacuity
(improvise-rxz9), the wire clauses (improvise-gkyo) and the projections and
ViewState keys (improvise-i0n7). Status stays proposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-22 23:44:26 -07:00
Edward Langley bf0525524f WIP: reorder race and coordinate-only basis tests (improvise-fdk1) 2026-09-22 23:44:16 -07:00
Edward Langley de439944cc WIP: empty rename mirrors the host; the renamed-record chain on a records host (improvise-fdk1) 2026-09-22 23:42:01 -07:00
Edward Langley 5c81d8bc06 WIP: coordinate rename green; tile-route walk admits the coordinate (improvise-fdk1) 2026-09-22 23:38:52 -07:00
Edward Langley d46b7691d2 Merge remote-tracking branch 'gh/main' into fix/fdk1-coordinate-rename 2026-09-22 23:37:10 -07:00
Edward Langley 5d9dcceaa0 WIP: coordinate rename on the wire and in the browser (improvise-fdk1) 2026-09-22 23:25:12 -07:00
Edward Langley 9d97664c4b ADR 005: stable identities for categories, items, formulas and views (proposed, improvise-sboz)
Docs only. Status: proposed. The nine open questions are the user's to answer on
the PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-22 23:17:52 -07:00
Edward Langley dd51f4316c WIP red: a records coordinate rename, from the browser (improvise-fdk1) 2026-09-22 23:15:39 -07:00
149 changed files with 26358 additions and 7713 deletions
+1 -1
View File
@@ -27,7 +27,7 @@ them here would let the copy drift:
| `context/design-principles.md` | The "why" — functional-first, polymorphism over conditionals, correctness by construction, layering, §7 testing doctrine |
| `context/repo-map.md` | The "where", plus **numbered gotchas** — read the ones your diff touches |
| `prolog-sessions/kb.pl` | Executable knowledge: `invariant/2`, `doctrine/2`, `forbidden/2`, `decision/3`, and prior `work_done/3` |
| `AGENTS.md` | Working rules: bug-first tests, `Option`/`Result`/`Box` at the edges, merges over rebases |
| `AGENTS.md` | Working rules: bug-first tests, `Option`/`Result`/`Box` at the edges, history reconciled only by merge, a new conventional commit by default (amend only an unpushed HEAD, for a small fix), and nothing that needs a force push |
| `.agents/skills/wave/agent-rules.md`, "GitHub" | The reviewer's rule: what to read before your first comment, how to answer each existing review, and what every review you post must state. Read it before reviewing a PR |
`kb.pl` is the highest-value and least-obvious of these. Grep it before
+5 -2
View File
@@ -12,12 +12,15 @@ scripts/pr-watch <pr-number> [--expect <sha>] [--timeout <sec>] [--interval <sec
```
exit 0 every check COMPLETED, none failed, and MERGEABLE with a state of
CLEAN, UNSTABLE or HAS_HOOKS
exit 1 a check failed, or the PR is not mergeable (CONFLICTING, BLOCKED,
DIRTY, BEHIND, DRAFT, or a state it does not know)
exit 1 a check failed (under any mergeability, BLOCKED included), or the
PR is not mergeable (CONFLICTING, DIRTY, BEHIND, DRAFT, or a state
it does not know)
exit 2 the head moved during the wait, or is not --expect
exit 3 timed out with checks still running, mergeability still UNKNOWN,
or still BLOCKED with every registered check green
exit 4 usage or environment error
exit 5 the PR is MERGED or CLOSED, at the head the watch pinned; it ends on
the first poll that sees it closed, which may be mid-wait
```
**Exit 0 now means mergeable, not only green.** GitHub computes mergeability
+26 -8
View File
@@ -52,7 +52,10 @@ nobody corrects. The brief adds only what varies per wave:
suite, which it is not. The agent still measures its own.
3. **Its ground, and the ground that is not.** Name any orchestrator-only file
it is exceptionally allowed to edit.
4. **Its `BEADS_ACTOR` and its tickets.**
4. **Its `BEADS_ACTOR` and its tickets.** The actor also names the agent's
commits: it commits through `scripts/agent-git`, which authors each one as
`Claude (<BEADS_ACTOR>)` (`agent-rules.md`, "Discipline per ticket"). So
give each agent its own actor, and do not tell it to set `user.name`.
5. **The attribution line, commit trailers and PR footer**, spelled out.
They name the agent (Claude's line opens `_Posted by Claude (<model>)…_`)
and this session's URL, which is why the shared `agent-rules.md` cannot
@@ -75,17 +78,32 @@ defect is a test that passes.
- Wait on checks with `scripts/pr-watch <pr> --expect <sha>`. Never hand-roll a
poll; see the `pr-watch` skill for the four ways they report green while wrong.
- Review before merging. Copilot's quota may be exhausted and its request is a
**silent no-op**; an absent review and an unrequested one both read as
"pending". Dispatch an adversarial reviewer and give it the head SHA, the base,
and a test-merge, so it reviews the tree that will exist. Point it at
`agent-rules.md`, "GitHub", for what a reviewer reads and answers first.
- Review before merging. Dispatch an adversarial reviewer and give it the head
SHA, the base, and a test-merge, so it reviews the tree that will exist. Point
it at `agent-rules.md`, "GitHub", for what a reviewer reads and answers first.
Copilot reviews here again (2026-09-24), mostly in inline comments, and Codex
as quota allows. Read both with `scripts/pr-reviews <n>` and see each finding
answered before merging. Do not wait on either: an absent review and an
unrequested one both read as "pending".
- Only the user's explicit approval accepts an ADR, and lets its PR merge. The
review gates still apply, and none of them is approval:
`AGENTS.md`, "ADR pull requests".
`AGENTS.md`, "ADR pull requests". **While an ADR PR is open, keep a
background poll** for new non-agent comments and reviews, and resume its
steward on each one. The user answers inline, and an answer left unread for
hours is a failure.
- **Before merging, check `baseRefName` is `main`**
(`gh pr view <n> --json baseRefName`). A stacked PR whose base has already
merged still targets the stale base branch, and merging it lands nothing in
`main`. Retarget it with `gh pr edit <n> --base main` first. `AGENTS.md`,
"Shipping a wave", has the 2026-09-24 incident.
- **Land a batch of approved PRs as one integration PR.** Evidence files are
append-at-EOF, so merging one at a time conflicts every other open PR. Then
diff every conflict resolution against both parents and the merge base, and
run the gates on the merged tree: `AGENTS.md`, "Merging many PRs at once".
- A re-review is of the **delta**, at a named SHA. A review of a superseded
commit reads as a pass.
- Merge on GitHub, then `git fetch gh && git merge --ff-only gh/main`. Close
- Merge on GitHub with a merge commit, never a squash or rebase merge
(agent-rules, rule 8). Then `git fetch gh && git merge --ff-only gh/main`. Close
the bead only once the merge has landed. That is when the work is done:
`AGENTS.md`, "Repository policy over the Beads blocks".
- Anything posted to GitHub goes out under the owner's account: open every
+133 -11
View File
@@ -32,17 +32,75 @@ not in a second copy.
6. **The Beads "Session Completion" block near the end of `AGENTS.md` does not
set your git policy.** This repository does not opt in to the block's
"team-maintainer" profile. On a wave branch that profile's pull and bare
`git push` would be refused (rule 4), or worse. Rules 1–5 and 7 are your
`git push` would be refused (rule 4), or worse. Rules 1–5, 7 and 8 are your
git policy. The block is bd's and bd regenerates it whole, so it is not
edited by hand: `AGENTS.md`, "Repository policy over the Beads blocks" (improvise-x1uk).
7. **Pin `gh/main` to a literal SHA when the task starts.** Run
`git rev-parse gh/main` once. Use that SHA, not the ref name, for every
reset, merge-base and diff. Remote refs are shared by every worktree of the
repository, so `gh/main` moves whenever another session fetches. An agent
ran `git reset --soft gh/main` mid-task and staged a reverse diff of #196.
It noticed only because it read `git status` (improvise-diho). Re-resolve
merge, merge-base and diff. Remote refs are shared by every worktree of the
repository, so `gh/main` moves whenever another session fetches. Re-resolve
only on purpose, for example when you mean to merge a newer main. Rule 3's
check before a PR is such a time.
8. **Reconcile diverging history only by merge.** To take a newer main,
re-pin on purpose first (rule 7): `git fetch gh`, then `git rev-parse
gh/main`. Then run `git merge <new SHA>`. Merge commits and new
commits are what the rule wants.
**The default is a new commit.** The user: "As a general rule, you shuold
just add a new conventional commit. Small adjustments can ammend HEAD, but
when in doubt don't." [sic] (2026-09-25; improvise-1xfb.2). That narrows an
earlier ruling: "I don't mind amending on a branch that hasn't been pushed,
but don't do anything that requires a force push" (2026-09-25;
improvise-79h5).
- **Amend only HEAD, only for a small adjustment, and only while HEAD is
unpushed.** A typo, a file left out of the commit you just made, a
formatter's fix to it. When in doubt, add a new commit.
- **Never reorder, fold or squash commits, pushed or not**, and never move
them onto another base: no `git rebase` onto main and no `git reset
--soft gh/main`. Both incidents below were exactly that, on commits that
had not been pushed.
- **Once a commit is pushed**, nothing may rewrite it: no rebase, squash
(`git reset --soft`, `git merge --squash`, a `rebase -i` fixup), amend or
`git reset` that drops it. Anything that would need a force push is
forbidden, and so is the force push itself.
**Messages are Conventional Commits**: `type(scope): subject`, with the
scope optional. The types are Conventional Commits' own: `feat`, `fix`,
`refactor`, `perf`, `style`, `test`, `docs`, `ci` and `chore`. Do not
invent new ones. History also has ad-hoc prefixes (`tooling:`, `context:`,
`clippy:`, `checkers:`); write those as a standard type with a scope
instead, e.g. `chore(tooling):`. A ticket id goes at the end of the subject
in parentheses, as most recent conventional commits carry it: `fix(tui): the grid and the tile bar scroll
by the width they are drawn in (improvise-gtn7.1)`. A merge commit keeps
the message git writes for it. The message says what changed. The argument
for the change lives in `context/`, never only in the message
(`context/design-principles.md` §6).
The user's reason: "the merge structure of git history is a causal record"
(2026-09-25; improvise-1xfb.1). A merge keeps that record, and a rewrite
replaces it with a story. The rewrite is also where the damage comes from.
A squash or rebase computes a new commit against a ref another session can
move, and when that ref moves, the commit silently reverts whatever landed
in between. It has happened twice, and making the operation safer did not
stop the second:
- **#196:** an agent ran `git reset --soft gh/main` mid-task and staged a
reverse diff of #196. Only `git status` caught it (improvise-diho).
- **#303:** after #303 merged mid-session, the improvise-cgqt agent squashed
with `git reset --soft gh/main`. The staged commit reverted #303, and only
rule 3's numstat caught it before the push.
A cherry-pick copies a change and drops its causal link, so use a merge
wherever a merge does the job. That includes a branch that carries another
PR's unmerged commits, when that PR is open and will land as a merge
commit. Either stack on that PR, or wait for it to merge, re-pin, and merge
the new SHA. The foreign commits then leave your diff.
The one place a merge cannot help is commits that will never reach main as
they are. That means an abandoned or rebuilt PR, or commits that exist only
on a mirror or a local main (#148). Only then, cut a fresh branch from your
pinned SHA and `git cherry-pick -x` your own commits onto it (`-x` records
where each came from). Close the old branch rather than force-pushing it:
`AGENTS.md`, "Shipping a wave".
## Commit early: a worktree is not durable storage
@@ -50,12 +108,16 @@ Commit a WIP as soon as your first test run is green, and again after every
meaningful step. Push your branch (rule 4) once it is worth keeping. An agent
finished a whole ticket with every gate green and lost all of it, uncommitted,
when its worktree was deleted in a disk cleanup. A local commit survives a
deleted worktree. Tidying history before the PR is fine; losing it is not.
deleted worktree. Make each step its own conventional commit, and leave it
that way: steps are not reordered or folded together afterwards. A small fix
to the step you just committed may amend it while it is unpushed. Anything
more, or any doubt, is a new commit (rule 8).
## Gates
```
direnv exec . ./scripts/fmt-workspaces.sh # NOT cargo fmt --all; exit 2 = the gate did not run
direnv exec . ./scripts/check-harness-lock.sh # offline; a Cargo.toml change can stale harness/opfs-contract's lock
direnv exec . cargo clippy --workspace --all-targets -- -D warnings
direnv exec . cargo test --workspace --no-fail-fast
direnv exec . python3 scripts/check-comment-runs.py --self-test
@@ -94,6 +156,16 @@ out on purpose:
- **Never pipe a gate.** `$?` must be the gate's own. And
`check-comment-runs.py --check-evidence` prints two lines with the comment
cap **first**, so `| tail -1` silently drops half of it.
`fmt-workspaces.sh` is the one most often lost this way: an unformatted tree
exits **1** and prints a diff, and `| tail` shows the diff's last lines and
exits 0. Read its exit status, or run it with `--fix` and commit what it
rewrites.
- **CI takes every tool from `nix develop .#ci`, and so should your scripts**:
no `/bin/…` or `/usr/bin/…` paths, and no bash-version or tool-version
probes (`AGENTS.md`, "CI takes its tools from the dev shell"). A test that
needs a restricted `PATH` builds one on purpose. It does not assume the
runner's `/bin` holds anything, because on the NixOS runners it holds only
`env` and `sh`.
- **`--no-fail-fast`**, because plain `cargo test --workspace` stops at the
first failing binary. Use it for any claim about blast radius.
- If you touch a browser-bundle crate (core, formula, io, protocol, command,
@@ -108,6 +180,13 @@ out on purpose:
never runs them. They failed only in CI once. A fresh worktree has no
`harness/node_modules`, and Playwright then reports a missing
`@playwright/test`, which reads like a broken checkout.
- **Playwright serves `harness/serve/`, not `web/`.** `harness/build.sh`
assembles that tree and it is gitignored, so after any change to `web/*.js`
— yours, or one a merge brought in — re-run `./harness/build.sh` before the
specs. Otherwise they test the old page. That failure looks exactly like a
real regression (`doctrine(rebuild_the_assembled_site_after_merging_web)`).
CI runs Chromium and Firefox only (`--project=chromium --project=firefox`),
so run those two locally.
- **If you change the grid's row or column count, also grep
`harness/fixtures/*.improv` and `harness/tests/*.spec.js`.** On PR #176 a
Playwright spec pinned the old collapsed-group row count. `cargo test` cannot
@@ -137,6 +216,33 @@ out on purpose:
- **Claim first:** `export BEADS_ACTOR=<your actor>` in every shell, then
`bd update <id> --claim`. Exit 13 means another actor holds it. Pick another;
do not retry.
- **Author every commit as `Claude (<BEADS_ACTOR>)`.** The user: "agents
should override git author name, I think to include an agent identifier
(matching BEADS_ACTOR)" (2026-09-25; improvise-1xfb.3). Make every commit
through the wrapper, merge commits included:
`BEADS_ACTOR=<your actor> scripts/agent-git commit …`, and the same for
`merge`. The wrapper sets the author name for that one command, and it refuses (exit 2) when `BEADS_ACTOR` is unset or empty. It
reads `BEADS_ACTOR` from its own environment, so the prefix works in a shell
that keeps nothing between commands.
- **The author email, the committer and the `Co-Authored-By` trailer stay as
they are.** The email is the owner's configured one, so GitHub still links
the commit.
- **Never set `user.name` with `git config`.** Worktrees share
`.git/config`, so it renames every agent at once. Do not use
`git -c user.name=…` either: `user.name` names the committer as well as
the author. Why the wrapper uses `GIT_AUTHOR_NAME`:
`context/test-evidence/tooling.md`.
- A plain `git commit --amend` keeps the author it amends, so amend with
plain git or with the wrapper.
- **A cherry-pick keeps its source's author**, through the wrapper or not:
git ignores `GIT_AUTHOR_NAME` there. That is right, because rule 8 allows
only `cherry-pick -x` of commits that keep whoever wrote them.
- Before you push, check `git log --format='%an <%ae> | %cn' <pinned
SHA>..HEAD`. Every commit **you made** must name your actor. A
cherry-picked commit, or one from a PR you stack on, keeps its own author.
- **The format is a proposal.** `Claude (<BEADS_ACTOR>)` is the default
proposed on improvise-1xfb.3's PR. The user may change it there. The format
lives in `scripts/agent-git` and nowhere else.
- **`bd` needs `.envrc` allowed in your worktree.** Without it you get a
**version mismatch**, not a clean failure. Run `direnv allow` in the worktree,
or rely on the `[whitelist] prefix` in `~/.config/direnv/direnv.toml`.
@@ -150,6 +256,10 @@ out on purpose:
it; do not reason about it. Report *where* it failed, by case name. An
injection that does not compile reads as a clean run, so get a positive
signal that the build ran.
- **Bound any test loop that walks a state machine until a step.** An
unbounded `while step != target { advance() }` hangs the suite when the
transition it counts on is broken. Loop a fixed number of times and assert
the step was reached, so the failure is a named assertion, not a timeout.
- **Check your test against every vacuity shape seen here:**
- an assertion the default state already satisfies;
- `f(x) == f(x)`: calling a pure `Cmd` twice against one immutable context;
@@ -180,6 +290,12 @@ out on purpose:
(module-qualify the name when the leaf is not unique). It is append-at-EOF, so
**expect a conflict**. Keep both sides and verify line by line against both
parents: `--check-evidence` cannot see a dropped entry.
- **Never resolve a conflict with `git checkout --ours` or `--theirs`.** Either
one takes the whole file from one side and silently drops the other side's
*non*-conflicting changes to it. That is how a `ci.yml` step vanished while
the merge message said it was unchanged (improvise-dndu). After resolving,
diff the merge against each parent and against the merge base
(`AGENTS.md`, "Merging many PRs at once").
- **Comment runs are at most two lines.** Reasoning goes in `context/`, not in
comments (`context/design-principles.md` §6).
- **If the acceptance criteria look wrong, argue it** rather than following
@@ -203,9 +319,11 @@ text for any of these in your hand-back.
apart from an automated Codex review (`AGENTS.md`, "Shipping a wave").
- The attribution line, commit trailers and PR footer all come from your brief.
They name the agent and the session, so this shared file cannot hold them.
- Read reviews with `./scripts/pr-reviews <n>`, and do not pipe it. How to tell
a ChatGPT Codex review from ours, and why Copilot is not coming: `AGENTS.md`,
"Shipping a wave".
- Read reviews with `./scripts/pr-reviews <n>`, and do not pipe it. It also
shows inline comments, which is where Copilot puts most of its findings.
Copilot reviews here again (2026-09-24), so answer its findings as you answer
Codex's. How to tell a Codex review from ours: `AGENTS.md`, "Shipping a
wave".
- **As a PR's author, reply on the PR to every review finding you address.**
Name the commit that fixes it, or say why you declined it. Putting it only in
your hand-back to the orchestrator is not enough: the PR thread is where the
@@ -218,8 +336,12 @@ text for any of these in your hand-back.
and the thread is still where the next reader looks.
- **Do not merge, and do not close beads.** Your part ends at an open PR plus
your hand-back. What "done" means is in `AGENTS.md`, "Repository policy over
the Beads blocks". Avoid stacked PRs. If you must stack, never let the base
branch be deleted.
the Beads blocks". Avoid stacked PRs. If you must stack, say in the PR body which PR it is
stacked on. Once that base PR merges, check the stacked PR's `baseRefName`
(`gh pr view <n> --json baseRefName`) and, unless it is `main`, retarget it
with `gh pr edit <n> --base main` before anyone merges it: merging it
otherwise lands it in the stale base branch, not in `main`. Fifteen PRs did
that on 2026-09-24 (`AGENTS.md`, "Shipping a wave").
## Hand-back, per ticket
+511 -111
View File
File diff suppressed because one or more lines are too long
+42 -68
View File
@@ -52,30 +52,7 @@ jobs:
# assembly still exited 0 (improvise-w8ih). It guards the assemble step
# below, so it fails under this job's name rather than another's.
- name: The stamp gate refuses what it claims to
run: ./scripts/checkers.sh assemble-stamp
# scripts/web-toolchain-versions.env stays the one home for the pins, so
# the release capture (scripts/build-web-release.sh, improvise-oujb)
# provisions the same versions. CI no longer installs them — the dev
# shell supplies wasm-pack and node — so this checks the shell against
# the pins instead: a nixpkgs bump that moves either fails here, by name.
# wasm-pack runs its own bundled `wasm-opt`, which is why
# improvise-wasm-client's Cargo.toml names `--enable-bulk-memory`.
- name: The dev shell supplies the pinned web toolchain
run: |
. scripts/web-toolchain-versions.env
test "$(wasm-pack --version)" = "wasm-pack ${WASM_PACK_VERSION#v}"
test "$(node -p 'process.versions.node.split(".")[0]')" = "$NODE_VERSION"
wasm-pack --version
node --version
# wasm-pack uses the shell's wasm-bindgen only at Cargo.lock's exact
# version; otherwise it cargo-installs one (71s) — so a lock bump
# must move flake.nix's wasmBindgenCli too, and this says so.
lock="$(awk '/^name = "wasm-bindgen"$/ {getline; gsub(/[^0-9.]/, ""); print}' Cargo.lock)"
test "$(wasm-bindgen --version)" = "wasm-bindgen $lock" || {
echo "flake.nix wasmBindgenCli is $(wasm-bindgen --version); Cargo.lock wants $lock" >&2
exit 1
}
run: ./scripts/checkers.sh --ci build
# Built by Nix (packages.web and packages.site in nix/ci.nix), which runs
# the same harness/build.sh and web/assemble.sh a developer runs, so there
# is still one answer to "how is the bundle built". `web` is the three
@@ -84,14 +61,12 @@ jobs:
# the harness tree is stamped `harness`. Dependencies are their own
# derivations, substituted from titan's cache when Cargo.lock is unchanged.
- name: Build the bundles, the harness tree and the site
shell: bash
run: |
nix build --print-build-logs --out-link result-web .#web
nix build --print-build-logs --out-link result-site .#site
# The upload steps below read these paths. Copied rather than linked:
# an artifact of a symlink into /nix/store is a dangling link elsewhere.
- name: Lay the outputs out where the uploads read them
shell: bash
run: |
rm -rf harness/serve web/pkg web/worker-pkg site
cp -rL result-web/harness-serve harness/serve
@@ -140,15 +115,17 @@ jobs:
# roots. Exit 2 here is a usage error, never a verdict (improvise-b4ks).
- name: Every workspace root is formatted
run: ./scripts/fmt-workspaces.sh
# A manifest-only change to a crate the harness reaches by path stales
# its own lock, which only the slow `build` job saw (improvise-zs2r).
- name: The harness lock matches the manifests it resolves against
run: ./scripts/check-harness-lock.sh --allow-network
# checks.clippy and checks.test in nix/ci.nix: the same cargo commands,
# on the dev/test profiles, with dependencies as a cached derivation.
- name: cargo clippy --workspace --all-targets -- -D warnings
shell: bash
run: nix build --print-build-logs .#checks.x86_64-linux.clippy
# Also runs the test-evidence gate (check-comment-runs.py
# --check-evidence), which lists the tests this derivation compiled.
- name: cargo test --workspace, and test-evidence headings name real tests
shell: bash
run: nix build --print-build-logs .#checks.x86_64-linux.test
# The comment cap and the test-evidence keys (improvise-9c8c); without
# this step the rule is advisory. See context/design-principles.md §6.
@@ -169,15 +146,39 @@ jobs:
# dependency crate each run. The NixOS runners substitute from titan's
# harmonia first, and titan builds gh/main's `#improvise` hourly, so a PR that
# leaves Cargo.lock alone rebuilds only the workspace crates it changed.
#
# The repo-stats pipeline runs here because it needs this job's binary:
# `.#repo-stats-tools` is that build plus code-maat and its JRE, which stay
# out of the ci shell every other job enters.
# Its suite gates; the report on a push to main never fails the build, and
# its step fails only when improvise refuses the pipeline's input
# (context/code/tooling.md section 7). Full history is for the report.
nix-build:
name: Nix build (crate2nix)
name: Nix build (crate2nix) and the repo-stats pipeline
runs-on: [self-hosted, nixos, microvm]
steps:
- uses: actions/checkout@v7
# Plain bash, not the dev shell: this step needs nix and nothing else.
with:
fetch-depth: 0
- name: nix build .#improvise
shell: bash
run: nix build --print-build-logs .#improvise
- name: The repo-stats pipeline computes what it claims to
run: nix shell .#repo-stats-tools --command ./scripts/checkers.sh --ci nix-build
- name: Churn, coupling and doc-growth report (report only)
if: github.event_name == 'push'
continue-on-error: true
run: |
rc=0
nix shell .#repo-stats-tools --command ./scripts/repo-stats --out repo-stats > repo-stats.txt || rc=$?
{ echo '```'; cat repo-stats.txt; echo '```'; } >> "$GITHUB_STEP_SUMMARY"
exit "$rc"
- name: Upload the repo-stats model and its CSVs
if: github.event_name == 'push'
uses: actions/upload-artifact@v7
with:
name: repo-stats
path: repo-stats
retention-days: 30
wasm:
name: wasm32 check (browser bundle crates)
@@ -190,7 +191,6 @@ jobs:
# depends on crossterm, which does not build for wasm — that is correct,
# not a gap (improvise-ywd). The log names each crate as it is checked.
- name: cargo check --target wasm32-unknown-unknown, per bundle crate
shell: bash
run: nix build --print-build-logs .#checks.x86_64-linux.wasm-check
# ---------------------------------------------------------------------------
@@ -214,7 +214,7 @@ jobs:
# The guard above is only worth its line if its refusals are themselves
# checked -- the step it replaces was green while asserting nothing.
- name: The node test runner refuses what it claims to
run: ./scripts/checkers.sh node-test
run: ./scripts/checkers.sh --ci js
kb:
name: Knowledge base consistency
@@ -229,27 +229,12 @@ jobs:
run: |
tpl prolog-sessions/kb.pl \
-g "(kb_consistent -> halt(0) ; (kb_report, halt(1)))"
# scripts/kb is the only thing that writes kb.pl, so its refusals are as
# load-bearing as kb_consistent itself. Without this the replacement
# writer could regress green -- the review that found the directive
# bypass also found that nothing ran this suite.
- name: The kb tool refuses what it claims to
run: ./scripts/checkers.sh kb
# Every suite goes through scripts/checkers.sh, which pins a bash >= 5
# and scrubs the environment, so the shell and the variables a suite
# passes under here are the ones it passes under locally (improvise-f9qf).
- name: The checker runner refuses what it claims to
run: ./scripts/checkers.sh --self-test
# pr-watch is driven entirely from fixtures. The release preflight is the
# one guard whose absence is invisible until a tag is public and
# immutable. pr-reviews stands between a merge and an unread review, and
# was blind to inline comments for as long as it existed (improvise-xgra).
# fmt-workspaces is here because its own failure mode is the bug it
# exists for: a usage error wearing a formatting failure's exit code.
# bd-audit-ids is the dead-ID audit, whose recipe once read a bd that
# never ran as a clean audit (improvise-52eq).
- name: The remaining checkers refuse what they claim to
run: ./scripts/checkers.sh pr-watch release-preflight pr-reviews fmt-workspaces bd-audit-ids
# Every suite goes through scripts/checkers.sh, which scrubs the
# environment (improvise-f9qf); its bash is the dev shell's. Which suites
# this job runs is the `kb` row of the CI table in checkers.sh, deliberately
# not listed here. check-ci-graph.py holds that table to `checkers.sh --list`.
- name: Every checker in scripts/ refuses what it claims to
run: ./scripts/checkers.sh --ci kb
# ---------------------------------------------------------------------------
# Lints the one crate the `test` job cannot reach.
@@ -273,7 +258,6 @@ jobs:
# checks.harness-lint in nix/ci.nix, run from inside harness/opfs-contract:
# `cargo fmt -p opfs-contract -- --check` and clippy for wasm32.
- name: cargo fmt -p opfs-contract and clippy --target wasm32
shell: bash
run: nix build --print-build-logs .#checks.x86_64-linux.harness-lint
# ---------------------------------------------------------------------------
@@ -301,19 +285,9 @@ jobs:
- name: Install the harness
working-directory: harness
run: npm ci
# No browser install step: the dev shell points PLAYWRIGHT_BROWSERS_PATH
# at nixpkgs' Chromium, Firefox and WebKit, since Playwright's own
# downloads cannot run on NixOS. All three, because improvise-xf1's
# acceptance criterion names Chrome, Firefox and Safari and
# `createSyncAccessHandle` is implemented separately by each engine.
# WebKit's OPFS is absent today; the spec skips on a runtime probe.
# @playwright/test must equal nixpkgs' playwright-driver version, or it
# looks for browser builds that are not there; this says which.
- name: Playwright matches the dev shell's browsers
working-directory: harness
run: |
npx playwright --version
ls "$PLAYWRIGHT_BROWSERS_PATH"
# No browser install step and no version probe: the dev shell sets
# PLAYWRIGHT_BROWSERS_PATH to nixpkgs' browsers, which pin them. `npm ci`
# stays because @playwright/test itself is not nix-provided.
# The tree `build` assembled, not one this job rebuilds. Same bytes as
# the Pages artifact, modulo the cache-bust token `web/assemble.sh`
# stamps — the harness tree is stamped `harness` and the site the commit
+4
View File
@@ -61,6 +61,10 @@ GSYMS
__pycache__/
*.pyc
# The assembled site, written by web/assemble.sh (and the doc generator's
# default OUT_DIR). Build output: CI assembles it from source (improvise-1xfb.4).
/site/
# The release web capture, written to the repo root by
# scripts/build-web-release.sh (improvise-oujb).
/improvise-web.tar.gz
+149 -19
View File
@@ -6,7 +6,12 @@
from logic
- @context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated.
- @context/design-principles.md is also important for keeping the repository consistent.
- prefer merges to rebasing.
- reconcile diverging history only by merge: the merge structure of git history is a causal record (the
user, 2026-09-25). "As a general rule, you shuold just add a new conventional commit. Small adjustments
can ammend HEAD, but when in doubt don't." (the user). So the default is a new Conventional Commit. Amend
only an unpushed HEAD, and only for a small fix. Never reorder, fold, squash or rebase commits, and never
force-push. The rule, the message convention and the incidents are
`.agents/skills/wave/agent-rules.md`, rule 8.
- always start responses with bananaS!
- there is a prolog MCP available: load a session from `kb` at the root of the repository at the beginning of
every session, and query it as you work. **Reading only** — the MCP must never write `kb.pl`.
@@ -147,8 +152,21 @@ once prescribed exactly that command (see "Repository policy over the Beads bloc
It cost a review round on PR #145,
where four unmerged commits — including another PR's whole diff — were pulled
into a branch based on `gh/main` and had to be rebuilt by cherry-picking.
ChatGPT Codex caught it; nothing in CI could, because the result compiles and
passes. The tell is a PR diff naming files the ticket has no reason to touch,
Today the repair **starts with a merge**. If the foreign commits belong to
another open PR that will land as a merge commit, either stack on that PR (see
the stacked-PR rule below) or wait for it to merge. Then re-pin `gh/main` and
merge the new SHA, and the foreign commits drop out of your diff. Never revert
them in your branch: merging the branch would then land an undo of that PR.
`git cherry-pick -x` is sanctioned **only** when those commits will never reach
`main` as they are: an abandoned or rebuilt PR, or commits that exist only on a
mirror or a local `main`, as in #148 below. Only then, cut a fresh branch from
the pinned `gh/main` SHA and cherry-pick only your own commits. `-x` keeps the
link to each original. Open a new PR, and close the old one without
force-pushing it.
ChatGPT Codex caught the #145 base problem; nothing in CI could, because the
result compiles and passes. The tell is a PR diff naming files the ticket has no reason to touch,
so **read your own PR's file list before asking for review** — and diff against `gh/main...<branch>`, never the local equivalent. **The second vector is the worktree base**: a branch cut from the orchestrator's *local* `main` inherits whatever that checkout has merged but not yet pushed, and nothing in the branch says so. That hit PR #148 with another PR's whole diff — including `.beads/`, `kb.pl` and `context/repo-map.md`, the three files agents are forbidden to touch, so the review read as a rule violation rather than a base problem. Note the window
is transient — the two remotes are equal immediately after the orchestrator
syncs — which makes it worse, not better: the hazard is absent exactly when
@@ -172,9 +190,12 @@ on merged PRs and were not read until afterwards, one of them a live P2
(improvise-x1ns). Both were correct.
```bash
gh pr view <n> --json reviews -q '.reviews[]|"[\(.state)] \(.submittedAt) \(.commit.oid[0:8])\n\(.body)"'
./scripts/pr-reviews <n> # bodies, inline comments and issue comments, in time order; do not pipe it
```
`gh pr view <n> --json reviews` is not a substitute: it shows review bodies
only, and most of Copilot's findings are inline comments.
Two things make a Codex review easy to miss. It arrives as a **review**, not a
comment, so a glance at the comment thread does not show it. And it posts under
the repository owner's account — the same account our own agents post under —
@@ -188,15 +209,26 @@ Reviews arrive **as the owner's quota allows**, so absence is not evidence.
Wait where a review is plausible. How a review is answered once found, by the
author and by a reviewer, is `.agents/skills/wave/agent-rules.md`, "GitHub".
**Copilot is not reviewing this repository, and the request is a silent no-op.**
Measured 2026-09-20: PRs #12, #21 and #27 carry 2, 8 and 4 Copilot reviews; #61,
#66, #68 and #70 carry none. Both request paths report success and produce zero
`review_requested` events — the GraphQL mutation returns `clientMutationId: null`
and the REST call returns 200 with `requested_reviewers: []`. So **do not wait on
it**: an absent review and an unrequested one look identical from outside, which
is `doctrine(review_is_requested_not_inherited)` seen from the requester's side.
Review is an adversarial agent the orchestrator dispatches, given the head SHA.
The mechanics below are kept for the day it comes back.
**Copilot is reviewing this repository again, so read its findings and answer
them exactly as you answer Codex's.** On 2026-09-24 it posted inline findings on
#221, #251, #274, #295 and #298, after a lapse that began between #27 and #61
(measured 2026-09-20: #12, #21 and #27 carry 2, 8 and 4 Copilot reviews, and
#61, #66, #68 and #70 carry none, while both request paths reported success and
registered nothing). Two things carry over from the lapse:
- **Its findings arrive mostly as inline review comments**, not as a review
body. `scripts/pr-reviews <n>` reads bodies, inline comments and issue
comments together; `gh pr view --json reviews` shows only the bodies, so it
misses them. Answer each on its own thread, as agent-rules, "GitHub" says.
- **Do not wait on it.** It reviews as its quota allows, and an absent review
and an unrequested one still look identical from outside, which is
`doctrine(review_is_requested_not_inherited)` seen from the requester's side.
The review that gates a merge is still an adversarial agent the orchestrator
dispatches, given the head SHA. Copilot's findings are read *in addition*,
before the merge, like Codex's.
The request mechanics below are what worked before the lapse. Whether they work
now is not re-measured.
Copilot review must be **re-requested after every fix push**; it does not
re-review on its own. The mutation takes a **`botIds`** field — `userIds` fails
@@ -248,15 +280,106 @@ will never see, CI runs on it, and the commit is nowhere. The tell is the push
printing `* [new branch]` where you expected an `old..new` range. It happened on
#51: the third review round completed four minutes after the merge, and the
correction it prompted went to a re-created `wave/b-projection-gaps`. The repair
is to branch from `gh/main`, cherry-pick, open a follow-up PR, and delete the
stray re-creation. Check `gh pr view <n> --json state` *before* pushing a fix to
is a merge, not a cherry-pick. The PR merged with a merge commit, so the stray
branch's base is already an ancestor of `main`. Branch from the pinned
`gh/main` SHA, `git merge` the stray branch's SHA, open a follow-up PR, and
delete the stray re-creation. The merge brings exactly the fix commits, under
their own SHAs, with the link to the PR they answered. A cherry-pick would copy
them and drop that link. This recipe works only because PRs here never
squash-merge. Check `gh pr view <n> --json state` *before* pushing a fix to
a PR you have not looked at in a few minutes, rather than after.
**A stacked PR whose base PR has already merged merges into that stale base
branch, not into `main`.** GitHub merges a PR into whatever `baseRefName` says,
and it did not retarget these. On 2026-09-24 fifteen approved stacked PRs
(#255–#286) were merged into a base whose own PR had **already** merged, and
the merges read as done. Recovery took seven landing PRs (#289–#295), each
opened from a stack's tip against `main`, and their titles name exactly those
fifteen. Four more in that range (#259, #263, #272, #273) also merged into a
non-`main` base, but one whose PR was still open; they reached `main` when their
base did, which is stacking working as intended. The repository reports `delete_branch_on_merge: true` as measured on
2026-09-24, and the stale bases (`collapse-fu/h6z1`, `fix/enz6-advance-onto-fold`
and others) were still on GitHub afterwards, so that setting is not the guard.
The guard is a check before every merge:
```bash
gh pr view <n> --json baseRefName -q .baseRefName # must print main
gh pr edit <n> --base main # if not, once its base PR has landed
```
Then read the PR's diff again: a retargeted PR's diff now includes its base's
commits if the base did not land.
Merge only once a review of the PR's current head has passed — an adversarial
agent's while Copilot is absent. What a review must state is agent-rules,
"GitHub". Prefer a **merge commit** —
`AGENTS.md` says merges over rebases, and a PR's commits are individually
meaningful.
agent's — and every Codex and Copilot finding on the PR has an answer. What a
review must state is agent-rules, "GitHub". Merge with a **merge commit**. Never
use GitHub's squash or rebase merge: history is reconciled only by merge (the
rule at the top of this file). A PR's commits are individually meaningful, and
the fix-after-merge recipe above needs the PR's head to be an ancestor of
`main`.
### Merging many PRs at once
Three things went wrong when a day's approved PRs landed together, and each is
invisible to CI on either side alone.
**Land a batch as one integration PR, not one PR at a time.** Every PR appends
its bug evidence at the end of a `context/test-evidence/*.md` file. So each
merge makes every other open PR conflict at EOF, and merging one at a time
means re-resolving the same files once per PR — and a PR that conflicts gets no
CI run at all (agent-rules, "Gates"). On 2026-09-24 the approved PRs landed as
two integration PRs instead: #298 (nineteen PRs) and #299 (the browser-editor
stack). This is what "one PR per wave" above already intends.
**After resolving a conflict, diff the result against each parent and against
the merge base.** `git checkout --ours <file>` or `--theirs` takes the whole
file from one side, so it silently drops every *non*-conflicting change the
other side made to that file. #245 resolved a `ci.yml` conflict with #228 by
`git checkout --theirs`, took #228's whole file, and lost its own
`check-harness-lock --allow-network` step, while the merge message said the step
was unchanged (improvise-dndu). Nothing refused it: `check-ci-graph.py` counts
suites, not that step. Resolve by hand, then check the merge commit:
```bash
git diff <merge-base> HEAD -- <file> # both sides' changes together, and nothing else
git diff HEAD^1 HEAD -- <file> # what the second side brought
git diff HEAD^2 HEAD -- <file> # what the first side brought
```
**Two changes to one test file can merge cleanly and still fail together.** #287
(the NixOS runners) found that `/usr/bin:/bin` there holds only `env` and `sh`,
and gave section 5 of `checkers-test.sh` a hermetic `utilbin` PATH. #237 added
section 5b on a branch cut before that, still using `PATH=…:/usr/bin:/bin`. Each
passed CI alone, the merge was textually clean, and 5b failed on every PR after
it (#296). So a fix to an idiom — a hermetic PATH, a scrubbed environment, a
helper every case must call — has to be applied to **every case in the file**,
including cases that exist only on another open branch. When you merge, grep
the merged file for the old idiom. Then run the gates on the merged tree, not
on each side.
### CI takes its tools from the dev shell
These are the user's rulings of 2026-09-24:
- **CI runs on self-hosted NixOS microVM runners** (`runs-on: [self-hosted,
nixos, microvm]`). Every `run:` step runs in `nix develop .#ci`, through the
workflow's `defaults.run.shell`. A `uses:` step (an action such as
`actions/checkout`) is not a shell step and does not go through it. A missing tool goes into the `.#ci` shell in
`flake.nix`, not into a per-step `nix shell` wrapper.
- **No hard-coded `/bin` or `/usr/bin` paths.** On these runners those
directories hold only `env` and `sh`. Take tools from `PATH`.
- **No version floors and no probes.** Scripts and tests must not care which
bash or standard utilities are present. `checkers.sh` no longer pins a bash
(it did, at 5.1, until #297). A script that needs a newer feature fails where
it uses it. The argument is `context/code/tooling.md` §5.
- **Playwright's browsers come from nix.** The dev shell sets
`PLAYWRIGHT_BROWSERS_PATH` to nixpkgs' browsers, so there is no install step
and no version check. `npm ci` stays, because `@playwright/test` is not
nix-provided.
A test that fakes an environment must build it on purpose, for example a
hermetic `utilbin` of linked tools. It must not assume the runner's `/bin`
holds anything.
### ADR pull requests
@@ -277,6 +400,13 @@ modelled on PR #122 (ADR 004):
applies after the approval. On #122 the user's own review landed as the PR
merged.
**An open ADR PR is watched, not visited.** The user answers the numbered
questions inline, and reviews arrive as they arrive. An answer left unread for
hours is a failure (the user, 2026-09-24). So while an ADR PR is open the
orchestrator keeps a background poll for new non-agent comments and reviews
since its last pass, and resumes the steward agent on each new item. The
steward folds each answer in and replies on its thread.
### Never hand-edit `.beads/issues.jsonl`
bd manipulates that file and auto-exports to it. Editing it — or even
+3 -1
View File
@@ -60,7 +60,9 @@ enum_dispatch = "0.3.13"
# binary does not grow a wasm client; `improvise-wasm-client` builds
# natively, which is how its own tests already run.
improvise-wasm-client = { path = "crates/improvise-wasm-client" }
improvise-protocol = { path = "crates/improvise-protocol" }
# `test-support` gives `FormulaEntry: From<&str>`, which the tests use; asked for
# here, or `-p improvise` alone does not compile (improvise-a61f).
improvise-protocol = { path = "crates/improvise-protocol", features = ["test-support"] }
# `tests/headless_cli.rs` detaches the binary from any terminal with `setsid`.
libc = "0.2"
pest = "2.9.2"
+9 -2
View File
@@ -102,7 +102,7 @@ rollback target — it is not what the published site is running.
Every cell lives at the intersection of named categories and items —
`(Region=East, Measure=Revenue)` — not at a grid address like `B3`. A model
can have up to 12 categories, each with an ordered list of items that can be
can have up to 24 categories, each with an ordered list of items that can be
organized into collapsible groups (e.g. months grouped into quarters).
Formulas reference dimension names, not cell addresses:
@@ -178,7 +178,14 @@ opening one fails with an error that says so. JSON *import* is unaffected.
The import wizard analyzes CSV columns and proposes each as a category
(string-valued), measure (numeric), time dimension (date-like, with optional
year/month/quarter extraction), or skip. Multiple CSVs merge automatically
with a synthetic "File" category.
with a synthetic "File" category, even when their headers differ. A record
simply has no value for a column its file lacks. A folder argument imports
every `*.csv` in it, in name order, even when it holds a single CSV. Each file
also gets its own view, filtered to that file and showing only its columns:
```sh
improvise import reports/ --no-wizard -o reports.improv
```
All model mutations go through a typed command registry, so the same
operations that back the TUI work headless:
+128 -3
View File
@@ -5,7 +5,7 @@ use std::time::{Duration, Instant};
use improvise::format::FormatPlan;
use improvise::model::cell::{CellKey, CellValue};
use improvise::view::Axis;
use improvise::view::{Axis, GridLayout};
use improvise::workbook::Workbook;
use improvise_command::ui::app::App;
use improvise_command::ui::project::{self, Viewport};
@@ -54,6 +54,96 @@ fn dense_app(regions: usize, products: usize, measures: usize) -> App {
app
}
/// Collapsed groups on the row axis and a formula chain on the column axis, so
/// every fold cell and every total re-derives the chain (improvise-h6z1).
fn chain_app(regions: usize, products: usize) -> App {
let mut wb = Workbook::new("Chain");
wb.add_category("Region").unwrap();
wb.add_category("Product").unwrap();
for r in 0..regions {
wb.add_item_in_group("Region", format!("Region{r}"), format!("G{}", r / 10))
.unwrap();
}
for p in 0..products {
wb.add_item("Product", format!("Product{p}")).unwrap();
}
wb.add_item("_Measure", "Revenue").unwrap();
let chain = [
"F1 = Revenue * 2",
"F2 = F1 + Revenue",
"F3 = F2 + F1",
"F4 = F3 + F2",
"F5 = F4 / F1",
];
for text in chain {
let formula = improvise::formula::parse_formula(text, "_Measure").unwrap();
wb.model.add_formula(formula);
}
{
let v = wb.active_view_mut();
v.set_axis("Region", Axis::Row);
v.set_axis("Product", Axis::Row);
v.set_axis("_Measure", Axis::Column);
}
let mut n = 0.0f64;
for r in 0..regions {
for p in 0..products {
n += 1.0;
let key = CellKey::new(vec![
("Region".to_string(), format!("Region{r}")),
("Product".to_string(), format!("Product{p}")),
("_Measure".to_string(), "Revenue".to_string()),
]);
wb.model.set_cell(key, CellValue::Number(n));
}
}
let mut app = App::new(wb, None);
app.rebuild_layout();
app
}
/// Days in collapsed groups of `per_group` on Row, with Product: every row is a
/// fold over a few days, never the whole category (improvise-jq0m).
fn grouped_app(days: usize, per_group: usize, products: usize, measures: usize) -> App {
let mut wb = Workbook::new("Grouped");
wb.add_category("Day").unwrap();
wb.add_category("Product").unwrap();
for d in 0..days {
let group = format!("G{}", d / per_group);
wb.add_item_in_group("Day", format!("Day{d}"), group)
.unwrap();
}
for p in 0..products {
wb.add_item("Product", format!("Product{p}")).unwrap();
}
for m in 0..measures {
wb.add_item("_Measure", format!("Measure{m}")).unwrap();
}
{
let v = wb.active_view_mut();
v.set_axis("Day", Axis::Row);
v.set_axis("Product", Axis::Row);
v.set_axis("_Measure", Axis::Column);
}
let mut n = 0.0f64;
for d in 0..days {
for p in 0..products {
for m in 0..measures {
n += 1.0;
let key = CellKey::new(vec![
("Day".to_string(), format!("Day{d}")),
("Product".to_string(), format!("Product{p}")),
("_Measure".to_string(), format!("Measure{m}")),
]);
wb.model.set_cell(key, CellValue::Number(n));
}
}
}
let mut app = App::new(wb, None);
app.rebuild_layout();
app
}
fn whole_grid(app: &App) -> Viewport {
Viewport {
row_offset: 0,
@@ -105,7 +195,10 @@ fn time_widths(iters: u32, app: &App) -> Duration {
}
fn case(label: &str, regions: usize, products: usize, measures: usize, iters: u32) {
let app = dense_app(regions, products, measures);
report(label, dense_app(regions, products, measures), iters);
}
fn report(label: &str, app: App, iters: u32) {
let rows = app.layout.row_count();
let cols = app.layout.col_count();
let cells = rows * cols;
@@ -120,17 +213,49 @@ fn case(label: &str, regions: usize, products: usize, measures: usize, iters: u3
true => "UNSTABLE".to_string(),
false => format!("{share:.0}%"),
};
// The TUI pays both on every keypress: widths in `cmd_context`, the grid in draw.
let keypress = full + widths;
println!(
"{label:<22} {rows:>5} x {cols:>3} = {cells:>7} cells whole-grid {full:>11.3?} \
windowed {windowed:>11.3?} ratio {ratio:>4.1}x col_widths {widths:>11.3?} ({share} of the frame)"
windowed {windowed:>11.3?} ratio {ratio:>4.1}x col_widths {widths:>11.3?} ({share} of the frame) \
keypress {keypress:>11.3?}"
);
}
/// The layout build with `prune_empty` on: every cell evaluated to find the empty
/// rows and columns, on the same keypress path as `grid_of` (improvise-metf).
fn report_prune(label: &str, mut app: App, iters: u32) {
app.model_state.workbook.active_view_mut().prune_empty = true;
let model = &app.model_state.workbook.model;
let view = app.model_state.workbook.active_view();
let mut best = Duration::MAX;
for _ in 0..iters.max(3) {
let start = Instant::now();
std::hint::black_box(GridLayout::with_frozen_records(model, view, None));
best = best.min(start.elapsed());
}
println!("{label:<22} layout build with prune_empty {best:>11.3?}");
}
fn main() {
println!("project::grid_of, one call = one TUI frame's grid cache");
println!("a 60fps frame budget is 16.7ms; the TUI redraws on each keypress\n");
case("typical", 20, 5, 4, 200);
case("1.2k cells", 60, 5, 4, 100);
case("2.4k cells", 60, 10, 4, 100);
case("4.8k cells", 60, 20, 4, 50);
case("large", 60, 20, 8, 50);
case("18k cells (gotcha 41)", 150, 20, 6, 20);
case("pathological", 300, 30, 10, 5);
println!("\nfolded rows and a five-formula chain (improvise-h6z1)\n");
report("folded chain ~18k", chain_app(1000, 30), 5);
report("folded chain ~90k", chain_app(5000, 30), 3);
println!("\nthe same, pruned (improvise-metf)\n");
report_prune("dense 18k", dense_app(150, 20, 6), 5);
report_prune("folded chain ~18k", chain_app(1000, 30), 5);
report_prune("folded chain ~90k", chain_app(5000, 30), 3);
println!("\ncollapsed small groups on the row axis (improvise-jq0m)\n");
report("364 days in weeks", grouped_app(364, 7, 20, 6), 5);
report("364 days in pairs", grouped_app(364, 2, 20, 6), 5);
report("1000 days in pairs", grouped_app(1000, 2, 10, 4), 3);
}
+916
View File
@@ -0,0 +1,916 @@
# ADR 007 — Stable identities for categories, items, formulas and views
**Status:** accepted (2026-09-25). The user approved revision 7. Revision 8 then
followed the adversarial review of that head: it adds clause 12, the ADR's own reading
of when a formula edit's names become ids, and corrects stale facts about main. The
user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17:10Z:
"looks good, approved"). ·
**Decided by:** the user; see "What the user decided" ·
**Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 ·
**Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and
improvise-fdk1's `RenameRecordCoordinate` (landed in PR #221) ·
**Builds on:** ADR 001, ADR 002 · **Depended on:** improvise-tt22 (remove legacy JSON),
which landed in PR #224
**Numbering.** The ADR was first filed as 005, but improvise-l2b8 already claims ADR 005
(exact decimal arithmetic) and improvise-ilvl claims 006. The review tickets filed
against the first draft still say "ADR 005" in their titles; they mean this document.
**Direction from** the user, 2026-09-23, in three messages:
> "there would be a lot of benefit to categories/formulas having stable identities, even if
> they were only transient: it would help pin editing operations and distinguish them from
> delete/remove"
> "It's probably worth a file format change (in a opt-in way, that gets rectified on save if
> the input file was missing them)"
> "Maybe something like `View: Default(default)` and the standard
> `View: |Default (default)|(default)` for quoting."
## What the user decided
The user answered the **first draft** (`context/adr/005-stable-identities.md` at
`9d97664c`) with inline comments on PR #210, and then posted one comment on the thread.
Each answer is quoted verbatim below, with where it was attached and what it overturned.
**Five of the first draft's positions were reversed.** The rest of this document is
rewritten to match.
**1. Storage keys by id; the human name becomes a display field.** On first-draft
line 92, the clause "`CellKey`, `DataStore`, views and formula text **stay keyed by
name**":
> "Shouldn't it be easy to replace this to key by ID? There's no type change so, at worst,
> it would be renaming field access. We could also add the name as something like
> `display` and reserve `name` or whatever it is currently for the id."
**2. The point of ids is that keys use them.** On first-draft line 97, "ids do not make a
rename cheap. They make it *expressible*":
> "This is a lot lower value if CellKeys, etc. don't refer to the new id"
**3. Every reference uses the id.** On first-draft line 275, the heading of clause 8,
"ADR 002: `RecordHash` stays defined over names":
> "No, anything referring to something with an id should refer to the id as the stable
> identity to avoid glitches."
This answers Q7 by reversing it, and reverses clause 5's "references carry names" too.
**4. Q1, the id's internal type.** On first-draft line 469, the recommendation to wrap "a
small string":
> "We probably can reuse symbols as the internal type with interning?"
Ids are strings, interned through the existing `SymbolTable`.
**5. Q2, formula id placement.** On first-draft line 473, recommendation F1
(`- Profit(3) = Revenue - Cost`):
> "Yes, before the equal to the right of the name"
**6. Q3, header syntax.** On first-draft line 480, the claim that "a header with a space
before its parenthesis (`Sales (copy)`) is unaffected":
> "Space before the open paren is optional: document this edge case that might break
> existing docs."
**7. Q4, data keys.** On first-draft line 485, the recommendation "no — names only":
> "In Data, we should prefer ids that are guaranteed unique to names that might not be."
**8. Import generates ids; legacy JSON goes.** Thread comment, 2026-09-23 06:51:
> "For CSV or JSON import, the dimensions are to be treated as the human friendly name and
> never parsed for IDs: during import, ids should be generated by a slugify algorithm that
> strips non-alphanumeric characters and replaces spaces with hyphens(collapsing runs).
> The only supported native file format is .improv, cleanup any legacy JSON format."
Legacy JSON removal is its own implementation ticket, **improvise-tt22**. This ADR
depended on it and did not implement it; it landed in PR #224. It also dissolves
improvise-65kf: with no legacy JSON to load, there are no numeric JSON ids to accept.
**The answers change the ADR's centre of gravity.** The first draft put an id *beside*
the name as an index. The user's answers make the **id the key** and the name a
**display** string. So a rename becomes cheap: it changes one display field and moves no
key, no surrogate and no basis tier. The first draft had called that impossible.
### Second round: answers on revision 2
The user answered revision 2 (`context/adr/007-stable-identities.md` at `5e46ce03`)
with thirteen inline comments on PR #210, posted 2026-09-23 15:03–15:11Z. Each answer
is mapped to the question **on the line it is attached to**, and quoted verbatim.
**9. Q5, group ids.** On line 468, inside Q5 ("Recommendation: not now … the one
exception to 'every reference is an id'"):
> "This seems right"
Groups get no id in this ADR.
**10. Q6, per-entity versions.** On line 469, Q6 ("Recommendation: not in this ADR"):
> "Agreed"
**11. Q8, a duplicate or unknown id.** On line 473, the recommendation to refuse the
load:
> "Yeah, don't load the file because you can't safely resave it"
**12. Q9, the reading of "opt-in".** On line 476, the recommendation that an id-less
file is read forever and every save writes ids:
> "Yes"
**13. Q10, id reuse after a reload.** On line 485, inside option **(b)**, "Accept
reuse across a reload (recommended)":
> "Yes, b"
This is option (b), and (b) was the recommendation. Within a session, retirement still
prevents reuse. Across a save and reload, a removed name that comes back gets its old
slug back.
**14. Q11, who assigns ids.** On line 492, inside option (a):
> "The host assigns"
**15. Q12, old-version files and the new header rule.** On line 502, inside option (a),
"No (recommended)":
> "Yes, a as long as version is unambiguously marked"
Old files are read as they always were, on the condition that the version is marked
unambiguously. Clause 9 says how the loader meets that condition.
**16. Q13, slugify.** Five answers, one per item:
- **(a) where it applies.** On line 510:
> "All id generation"
- **(b) case.** On line 512, against the recommendation to keep case:
> "lower case only"
- **(c) `_` and `-`.** On line 515:
> "Yeah, keep both"
- **(d) non-ASCII letters.** On line 518:
> "Strip non lower alphanumeric or -_"
- **(e) the empty slug.** On line 520, the recommendation `<kind>-<n>`:
> "Yes"
So an id is lower case, and its alphabet is `a`–`z`, `0`–`9`, `-` and `_`. Clause 4
states the rule. Q15 asked the user to confirm the order of its steps (answer 18).
**17. Q14, formula text.** On line 529, the recommendation "ids in the file":
> "Display in the formula panel but not as column/row/page headers."
This does not contradict ids in the file, which answer 3 already requires. It answers
**where formula text is displayed**, and it could be read two ways. That was Q16,
which answer 19 settles.
### Third round: answers on revision 3
The user answered revision 3 (`context/adr/007-stable-identities.md` at `18c82e1c`)
with three inline comments on PR #210, posted 2026-09-24 00:34–00:36Z. Each is mapped
to the question on the line it is attached to, and quoted verbatim.
**18. Q15, the order of slugify's steps.** On line 571, Q15's recommendation "fold, then
strip":
> "casefold, replace space with `_`, strip"
The fold comes first, as recommended. The answer also **changes answer 8's middle
step**: a space becomes `_`, not `-`. Two details of that sentence are Q18.
**19. Q16, what the formula panel shows.** On line 584, the paragraph that declined to
recommend a reading:
> "It should be at least `Profit (profit) = Revenue - Cost` but if `Revenue` is
> ambiguous, you'll have to display the slugs. Working with slugs internally and making
> the friendly names view-layer is almost always the right move here."
So the panel shows formula text over **display names**, with the target's id beside it
in parentheses. Where a display name in the text is ambiguous, the panel shows the id
instead. Ids are what the file and the model hold, and display names belong to the view
layer. This also settles answer 17: the thing displayed "in the formula panel but not as
column/row/page headers" is the target's **id**. Whether an id is enough to resolve an
ambiguous name is Q19.
**20. Q17, capitals in a hand-written id.** On line 589, the question "May a
hand-written id contain capitals?":
> "no."
This is option (a). `id_token` is lower case only, and a header ending in a
parenthesised run that contains a capital loads as a whole-line display with a load
warning. The virtual categories are spelled with capitals today. That is Q20.
(Answer 21 later made a new-version file all-or-nothing, so that header is now refused
rather than warned about. See clause 5.)
### Fourth round: answers on revision 4
The user answered revision 4 (`context/adr/007-stable-identities.md` at `024aa2d3`)
with five inline comments on PR #210, posted 2026-09-24 06:05–06:08Z. Each is mapped
to the question on the line it is attached to, and quoted verbatim.
**21. Q18(a), runs of spaces, and the collision suffix.** On line 658, Q18(a) ("Does a
run of spaces collapse to one `_`?"):
> "runs of spaces are collapsed to one `_` and then `-###` numeric suffix is added if
> duplicate (only when importing a file without IDs: a file must either have no IDs at
> all or always have IDs)"
This decides three things:
- a run of spaces collapses to one `_`, as recommended;
- a duplicate slug gets a numeric suffix after a `-`;
- **a file has ids on everything or on nothing.** A new-version file with a declaration
missing its id is refused, like any file that cannot be resaved safely (answer 11).
So the earlier rule that a hand-written id wins over a generated one in the same file
is gone.
The parenthesis also limits the suffix to generating ids for a file that has none. Two
details of that are Q22. Q18(b), what "casefold" means, was not answered in this round
(answer 26 settles it).
**22. Q19, whether an id disambiguates formula text.** On line 678, option (a) ("Make
item ids unique in the document"):
> "Ids have to be globally unique."
This is stronger than option (a). **No two ids in a document are equal, whatever their
kind**, so a category, an item, a formula and a view never share one. An id in formula
text or on the wire therefore always names exactly one thing.
**23. Q20, the virtual categories.** On line 691, option (a) ("Lower-case ids,
capitalised displays"):
> "Virtual categories have IDs: we need to add a "slug" version such as `_index`, but
> no non-virtual id can start with `_`"
The ids are `_index`, `_dim` and `_measure`, and the displays stay `_Index`, `_Dim` and
`_Measure`. A leading `_` is reserved for them. Clause 4 says how slugify keeps a
generated id out of that space, and clause 5 says what happens to a hand-written one.
**24. Q21, an old file naming an undeclared item.** On line 708, option (a) ("Declare
the item at load"):
> "yes, this is right."
**25. The file version.** On line 731, the Consequences line "A new file version.":
> "File version should be added explciitly if it's not already part of the format."
It is already part of the format. The first line of every `.improv` file is a
mandatory, exact version literal (`version_line = { "v2025-04-09" ~ NEWLINE ~ … }` in
`improv.pest`), and the writer emits it. So the new format is marked by a **new
literal**, and nothing is inferred. Clause 10, new in this revision, states the rule in
one place.
### Fifth round: answers on revision 5
The user answered revision 5 (`context/adr/007-stable-identities.md` at `e044c968`)
with three inline comments on PR #210, posted 2026-09-24 19:50–19:51Z. Each is mapped
to the question on the line it is attached to, and quoted verbatim.
**26. Q18(b), what "casefold" means.** On line 766, the recommendation to take
"casefold" literally, as full case folding:
> "full unicode"
Slugify's first step is **Unicode full case folding**, so `Straße` → `strasse`.
**27. Q22(a), how the suffix is spelled.** On line 771, the recommendation "a plain
number":
> "Just a plane number."
The suffix is `-` followed by a plain decimal number, with no padding.
**28. Q22(b), collisions outside import.** On line 782, option (i), "Suffix it there
too":
> "Any ambiguous id gets a suffix, if if the id is between different types of things
> identified. Don't bother implementing a strict order here: `east` `east-1`
> `east-2`, etc. just do what gensym does in CL: global increment-only counter appended
> when dedupe needed."
This decides three things:
- **every** colliding id gets a suffix: on import, in a session, and across kinds (a
category and an item that slug alike);
- the number comes from **one counter for the document that only ever increments**, as
Common Lisp's `gensym` does, so it is not "the lowest free number";
- the suffix is appended **only when a dedupe is needed**, so an id that does not
collide stays bare.
Clause 4 is rewritten to match. The rules this replaces (lowest free number, and
suffixing only on import) were revision 5's.
### Sixth round: answer on revision 6
The user replied at 2026-09-24 22:09Z to their own answer 28, on line 782 of
`e044c968`:
**29. Hand-written ids are canonical.**
> "You have to treat the file's hand-written ids as canonical, doing otherwise will
> corrupt the data section."
This constrains answer 28. The `gensym` suffix applies only to an id the program is
**generating**, on import, when an id-less file loads, or when the host creates
something in a session. **An id written in a file is never renamed**, because every
data key and reference in that file spells it. Clause 4 says what follows:
- a generated id that would collide with a file's id takes the suffix, and the file's
id keeps its spelling;
- two identical hand-written ids in one file are refused at load, not repaired.
## Context
**Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name.
`CellKey` and `DataStore`'s interned symbols are names. Views, axis maps, page
selections, hidden and collapsed sets, and formula text all refer by name. The user's
first message names what that costs:
- **An edit cannot be told apart from a delete followed by an add.** No rename operation
exists for a category or an item. A rename is remove plus add, which loses every cell
(improvise-fz77). A formula is effectively identified by `(target, target_category)`,
and `Model::add_formula` *upserts* on that pair. improvise-km71 (PR #214) made formula
editing its own operation, with an interim id built from that pair (below).
- **An edit whose target was concurrently replaced cannot be told apart from an edit of
the replacement.**
- **A records-mode coordinate rename names its category and item by name.**
improvise-fdk1 (PR #221) landed `RenameRecordCoordinate { record, category, item }`
with `String` operands (below).
**What already exists.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize`
aliases, assigned from monotone counters. **No production caller uses either value.**
`Model::add_category` and `Category::add_item` return them on the idempotent path, and
nothing reads the returned value; the only reader is one test. Neither has ever been written to `.improv`. `View` has no
id.
**`FormulaId` exists, as an interim.** km71 landed an opaque `FormulaId` whose private
fields are `(target, target_category)`, built only by `Formula::id()`
(`context/code/core.md` §18). It is on the wire in
`ModelCommand::ReplaceFormula { id, was, raw }`, in `FormulaSlot::Existing` and in
`FormulaEntry`. It was made opaque so that this ADR changes one struct rather than every
signature. It is still a name pair, so a formula whose target is renamed gets a
different `FormulaId`. `Model::replace_formula` therefore refuses to rename a target
outside `_Measure` until this ADR lands (improvise-kb1d).
**Legacy JSON is gone.** improvise-tt22 landed in PR #224, so `.improv` is the only
native format.
## Decision
**Categories, items, formulas and views each have an id, and the id is their key.** A
category or item's current `name` field *becomes* the id (the user's suggestion), and the
human name moves to a new `display` field.
- **Every reference is an id.** That covers `CellKey`s, axis maps, page selections,
hidden and collapsed sets, formula references, data keys, and wire operands that name
an existing thing.
- **Display names are for people.** They appear on screen, are typed into minibuffers,
and are resolved to ids at the input edge.
Twelve clauses.
### 1. What carries an identity
| kind | id | unique within | wire address |
|---|---|---|---|
| category | yes | the document, across every kind (answer 22) | `CategoryId` |
| item | yes | the document, across every kind (answer 22) | `ItemId` |
| formula | yes | the document, across every kind (answer 22) | `FormulaId` |
| view | yes (the user's example) | the document, across every kind (answer 22) | `ViewId` |
| record | `RecordHash`, now over ids (clause 8) | the document | unchanged shape |
| group | **no**, not in this ADR (answer 9) | | |
**Ids are unique in the whole document** (answer 22). One namespace covers categories,
items, formulas and views, so an item id names one item without its category. A
`CellKey` still pairs each item with its category, because the key needs the
categories. An operation that creates an item still names the parent category.
**Virtual categories have reserved ids** (answer 23): `_index`, `_dim` and `_measure`,
with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.**
A `_Measure` formula target has no item of its own. Its handle is its
formula's `FormulaId`, which is why the category panel's `_Measure` rows need
`ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6).
### 2. Storage keys by id; `display` is the only place a human name lives
Following answers 1 to 3:
- **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id,
and each `Category`, `Item` and `View` gains a `display: String`.
- **Cells and derived surrogates:** a `CellKey` is a sorted list of
`(CategoryId, ItemId)` pairs. It stays sorted, now **by category id**, so gotcha 5's
canonical form is kept with a different sort key. `DataStore` interns ids exactly as it
interns names today.
- **View configuration:** the axis maps, page selections, and hidden and collapsed sets
are keyed by ids.
- **Formulas:** a formula's references are ids (clause 5). The formula panel renders
them as display names (answer 19; clause 6).
**What a rename now costs:** one `display` write. It moves no `CellKey`, no `RecordHash`
and no `layout_version`, so an edit a client composed before the rename still lands after
it. That is the "avoid glitches" the user's rule is for. The grid's labels are display
strings, so the rename's effect must still dirty the grid and the panels
(`Dirty::of` gotcha 35), but no basis tier moves.
**What the migration costs, stated honestly.** Every API that takes a name today
(`Model::category(name)`, `Workbook::add_item(cat, item)`, `CellKey::new` callers, the
formula evaluator's `Expr::Ref` lookup, `View` methods) takes an id instead. The
display→id resolution moves to the places a human types, such as the minibuffer commits,
the formula parser's reference resolution, and import. The change is mechanical but wide,
and it is the bulk of the implementation. The user judged it cheap ("at worst, it would
be renaming field access"), and for the field that is right. The call sites are the
cost.
**Display names stay unique within their scope**, as names are today. So a human-typed
name resolves to exactly one id, and a rename to a display name already in use is
refused. An item's scope is its category, so two categories may each have an item
displayed `Revenue`. Formula text crosses categories, which is where answer 19's
"ambiguous" arises. Their ids differ (answer 22), so the panel resolves the ambiguity
by showing the ids.
### 3. Ids are interned symbols
Following answer 4, an id is a **string**, and in memory it is a `Symbol` from the
existing `SymbolTable`: copyable, with integer comparison. Two consequences.
- **A `Symbol` never crosses the wire or reaches the file.** It is an index into one
process's table. The wire and `.improv` carry the id's **text**, and each end interns
it on arrival.
- **`Rejection` stops being `Copy`**, because `Rejection::Gone` (clause 7) carries an id's
text. `basis.rs`'s `SaveFailed` carries no message *because* `Rejection` was `Copy`, so
that reason lapses. Whether `SaveFailed` now gains its message is the implementer's
call.
### 4. How an id is made: slugify, one rule everywhere
Following answers 8, 16 and 18, **every** id the app generates comes from the display
name by **slugify**. That covers import, interactive creation, and assigning ids to an
id-less `.improv` file at load (answer 16(a), "All id generation"). So a thing's id
never depends on how it was made. The steps, in order (answer 18, "casefold, replace
space with `_`, strip"):
1. **Casefold, with Unicode full case folding** (answers 16(b), 18 and 26). Rust's
`str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped), so this
needs a case-folding table or a small crate in `improvise-core`. It must pass the
`wasm` gate, because that crate is in the browser bundle.
2. **Replace each run of spaces with one `_`** (answers 18 and 21; answer 18 changed
answer 8's `-`).
3. **Strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`** (answers 16(c)
and 16(d)). This strips non-ASCII letters, so an id never needs quoting in an
`.improv` reference (clause 5 notes where formula text differs).
4. **Remove any leading `_`** (answer 23: no non-virtual id may begin with `_`). The
answer states the constraint. Removing the underscores is this ADR's way of meeting
it, so `_Draft` → `draft`.
5. **If nothing is left, use `<kind>-<n>`** (answer 16(e)), such as `category-1` or
`item-4`. `n` comes from the same counter as every other suffix (answer 28), not
from the lowest free number.
Examples: `East` → `east`, `Time Period` → `time_period`, `Coastal_East` →
`coastal_east`, `Q1-2024` → `q1-2024`, `Region(code)` → `regioncode`, `Café` → `caf`,
`Ü` → `category-<n>`, `Straße` → `strasse` (answer 26). `Time Period` and
`Time_Period` slug alike, so whichever comes second gets `time_period-<n>`. `East` in
`Region` and `East` in `Direction` slug alike too, and since ids are global (answer
22), the second is `east-<n>`. Here `<n>` is the counter's next value, for example
`east-1`.
The remaining rules:
- **A file has ids on everything or on nothing** (answer 21). An id-less file (old
version, or a CSV or JSON import) gets every id from slugify. A new-version file must
carry an id on every declaration, and one that does not is refused (clause 5). So no
file mixes hand-written and generated ids.
- **Any colliding id gets a suffix, `gensym`-style** (answers 21, 27 and 28).
- **When:** whenever a generated slug is already taken, whatever kind of thing holds
it. That applies on import, when an id-less file loads, and when the host creates
something in a session. An id that does not collide stays bare.
- **What:** `-` and the next value of **one increment-only counter per document**,
as a plain number (`east-1`, not `east-001`). The counter never goes down and is
never reused, so no order is enforced among suffixes: `east-7` may follow `east`
directly.
- **Retirement:** within a session a removed id is never handed out again. The
counter has already moved past any suffix it issued, and a bare slug that was
removed is still reserved in memory.
- **Determinism:** the counter advances in file order or creation order, so the same
input gives the same ids, never hash order (gotcha 18).
- **Hand-written ids are canonical** (answer 29). The suffix only ever goes on the
id being generated. An id read from a new-version file keeps its spelling, even
when it looks like a slug or a suffixed one (`east-3`), because the file's data
keys and references spell it. So when a generated candidate `slug-<n>` is already
taken by a file's id, the counter advances again until the candidate is free.
- **Two identical hand-written ids in one file are refused at load** (answers 11,
22 and 29). By answer 22 they are a duplicate even when they declare different
kinds of thing, such as a category and an item. Renaming either one is exactly
what answer 29 forbids, and answer 11 already refuses a duplicate because such a
file "can't safely resave". The error names both lines (clause 5). This follows
from the answers, so it is not asked as a new question.
- **The counter** lives in memory and starts at 1 when a document loads. It is not
written to the file, since retired ids are not persisted (answer 13). That is
this ADR's reading.
- **Across a save and reload, reuse is accepted** (answer 13, Q10(b)). Retired ids are
not persisted. So remove `East`, save, reload and add `East`, and the new item gets
`east` again. An in-flight command from before the reload is refused by the
generation tier. The one remaining risk is an outside reference that outlives the
file, and none exists today.
### 5. Persistence: ids on declarations *and* references
Following answers 3 and 7, the file now carries ids everywhere:
- **Declarations** (section headers, item lines, formula lines) carry the display name
and the id.
- **Every reference** carries the id: axis lines, `hidden:` and `collapsed:`, page
selections, `## Aggregation`, `Initial View:`, `## Data` keys, and names inside formula
text (answer 3). The formula panel shows display names instead (answer 19).
**Id alphabet.** Every id, generated or hand-written, is spelled only in `a`–`z`,
`0`–`9`, `_` and `-` (clause 4; answer 20). The grammar spells them **with ranges**,
because the generator drops `ASCII_ALPHANUMERIC` (improvise-rxz9). So an id never needs
pipe quoting in an `.improv` declaration or reference. A header ending in a
parenthesised run that contains a capital, such as `Sales(Copy)`, does not match
`id_tag`, and neither does an empty or trailing-space tag. In a new-version file that
declaration has no id, so the load is refused (answer 21), and the error names the
line. Revision 4 generated an id and warned. A hand-written id beginning with `_`,
other than the three virtual ids, is refused too (answer 23).
**Inside formula text an id may still need quoting.** The formula grammar's
`bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So
the writer spells an id in formula text through `quote::quote_name`, as it spells a
name today. An id beginning with a digit (`2024`) or equal to a keyword (`sum`, `if`,
`where`, `is`) is written `|2024|` or `|sum|`. No new rule is needed, but "never needs
quoting" does not extend to formula text.
```pest
id_token = @{ ('a'..'z' | '0'..'9' | "_" | "-")+ }
// The space is optional (answer 6).
id_tag = ${ " "* ~ "(" ~ id_token ~ ")" }
// Declarations: display name, then its id. The id is optional in the grammar so the
// loader, not the parser, refuses a missing one and can name the line (answer 21).
header_decl = { pipe_quoted ~ id_tag ~ &NEWLINE | header_display ~ id_tag ~ &NEWLINE | line_name }
header_display = @{ (!(id_tag ~ NEWLINE) ~ !NEWLINE ~ ANY)+ }
item_decl = { name ~ id_tag? }
grouped_item = { "- " ~ name ~ id_tag? ~ "[" ~ name ~ "]" ~ NEWLINE ~ blank_lines }
// References: an id, never a display name.
id_ref = { id_token }
```
`model_name` has no id, so it keeps `header_name`. `Initial View:` becomes a reference
and takes an `id_ref`. A group reference (`[Coastal]`) stays a name while groups have no
id (answer 9).
**`header_display` is guarded only by a lookahead**, which the generator skips (gotcha
34). So the generator may emit a display ending in something that later parses as an id.
Round-trip property 3 still holds, because the first parse settles the split. But
property 1 must be stated over the parsed result, not over the generated text.
**Which regime a file is read under is decided by its version line alone** (clause
10). In an old-version file, a header is the whole line, a reference resolves against
display names, and ids are assigned by slugify. In a new-version file, declarations
carry ids and a reference is an id.
**An old file naming an item it never declares** declares it at load (answer 24). This
happens today when a `## Data` line names an undeclared item (improvise-98pd, second
source). The item gets a slug like any other, and the first save writes it, so a file
that loads today keeps loading (the lesson of improvise-80rm). The NonItem work
(improvise-gk51, PR #265) projects such a cell as `CellId::NonItem`. From a loaded file
that case no longer arises. Ids do not settle 98pd's other two sources, `set-cell` and
a stale page selection: a page selection holds an item id, and removing the item leaves
it dangling exactly as a removed name does today.
**Slugify folds case (answer 16(b)), so old data keys change spelling on first save.**
An old file's `Region=East` resolves at load through the display `East`. The first
save writes the id: `Region=East` becomes `region=east`. The data is the same, but a
`git diff` across that first save touches every data line.
**Round-trip properties:**
1. an id-less (old-version) file loads, gets ids, and is written in the new version on
its first save;
2. a new-version file loads with every id kept verbatim;
3. `format(parse(format(parse(x)))) == format(parse(x))`.
A **formula-line generator strategy** is required. The grammar walker emits no `=` and
the walker drops such lines, so without its own strategy every formula property is
vacuous (improvise-rxz9).
**A duplicate, unknown or missing id in a new-version file refuses the load** (answers
11, 21, 22 and 29; a duplicate across two kinds of thing counts, and a hand-written id
is never renamed to resolve one; answer 11:
"don't load the file because you can't safely resave it"). The error names the id,
and for a duplicate it names both declarations. Repairing silently is ruled out,
because references carry ids (answer 3): reassigning a duplicate would silently change
what its references point at.
**Legacy JSON is removed** (improvise-tt22), so it needs no migration here.
**Import** (CSV, JSON) never parses ids out of dimension names (answer 8). A header
`Region(code)` is the display name `Region(code)`, and its id is its slug. The writer
pipe-quotes any display name that would otherwise read as display plus id:
`## Category: |Region(code)|(regioncode)`.
### 6. The wire: address by id, create by display name, learn ids from projections
- **The host assigns every id** (answer 14). A client never proposes one.
- **Every operand naming an existing thing is an id.** The one operand naming the thing
being created is a display name, and the host slugifies it:
- `AddCategory(display)` and `CreateView(display)`;
- `AddItem { category: CategoryId, display }`,
`AddItemInGroup { category: CategoryId, display, group }` and
`AddFormula { target_category: CategoryId, raw }`.
- **A frame cannot address a thing it creates**, because it does not know the id the
host will assign. So a create-then-act compound becomes **one command** carrying
both halves. `create-and-switch-view` sends `[CreateView, SwitchView]` today and
becomes `CreateView { display, switch: true }`. Any other compound of this shape
gets the same treatment.
- **New operations are born with ids:**
- `RenameCategory { category, to: display }`;
- `RenameItem { category, item, to: display }`;
- `ReplaceFormula { id, was, raw }`, which km71 already landed with an interim id
(below);
- fdk1's `RenameRecordCoordinate`, which landed with name operands (below);
- the removes (`RemoveCategory`, `RemoveItem`, `RemoveFormula`, `DeleteView`).
- **Older variants** (`SetAxis`, `SetPageSelection`, `HideItem`, …) **must also move to
ids**. Their operands are references, and a reference is now an id. They may move one
whole variant at a time. That includes `SetCell { view, … }` and
`ClearCell { view, … }`, which improvise-wpxr (PR #281, merged) gave a view named by
its name, like every other view-scoped variant (improvise-lltt).
- **Projections carry id and display together:**
- `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }`;
- `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows
(answer 19): the target's display name with its id in parentheses, then the
expression over display names, with an id wherever a display name is ambiguous
(ids are unique, answer 22). `stored` is the id-form text the file carries, and it is what
`ReplaceFormula`'s `was` is compared against. So a rename of `Revenue` elsewhere
changes what the panel shows but does not refuse an edit of `Profit`, which is
the "avoid glitches" of answer 3. What makes that hold for an edit already open
is clause 12;
- `ViewSummary { id, display, … }`.
`AxisFrame` labels stay display strings. They gain ids when a consumer needs them.
- **`ViewState` keys by id**: the cursors, `expanded_cats`, and the view history
(`view_back_stack` / `view_forward_stack`). Answer 1 makes this natural rather than a
migration of its own. A view or category rename then orphans nothing.
### 7. ADR 001: an id that no longer resolves is gone
A command naming an id that does not resolve is rejected before anything is applied, as
`Rejection::Gone(Named)`:
- `Named` is one of `Category`, `Item`, `Formula` or `View`, and carries the id's text.
- It is **structural**: force never creates structure.
The check has two halves, like the record tier:
- **Before the frame:** an exhaustive `ids_named` feeds `Checked::against`.
- **Within the frame:** an **id arm in `broken_chain`** refuses
`[RemoveFormula(f), ReplaceFormula(f)]` (improvise-gkyo).
Per-entity versions are not added (answer 10).
### 8. ADR 002: `RecordHash` is now defined over ids
Following answer 3, `RecordHash` hashes the explicit serialization of the sorted
`(CategoryId, ItemId)` text pairs, the same function over different strings.
- **ADR 002's decision is unchanged in substance.** It still hashes an explicit
serialization of a normalized record, with a fixed key and a width argued for.
- **Its prose will be out of date once this lands.** ADR 002 says `CellKey::new`
"sorts coordinates by category name", and a record is spelled in names. Both become
ids. ADR 002 is **not edited here**. Updating that wording belongs to the
implementation PR.
**What changes in behaviour:** a display rename no longer changes any `RecordHash` or
bumps `layout_version`, because `view_shape` hashes ids too. So a client's surrogate
survives a rename, and its in-flight edit still lands. A **coordinate** change, as in
fdk1 or a records-mode move, still produces a different record and a different hash, as
it should: it *is* a different record.
### 9. Headers that change meaning: the edge case answer 6 asks to document
Under answer 6, `header_decl` reads a trailing `(token)`, with or without a space before
it, as the id. **It applies only to new-version files** (answer 15, Q12(a)). An
old-version file is read as it always was, so its headers never split. In a new-version
file the writer never produces an ambiguous header, because it pipe-quotes any display
that would split. So the edge cases below arise **only when a new-version file is
written by hand**. The table shows what the header rule would do to lines that exist
in old files today. That is why old files do not get the rule.
| header line | whole-line reading (today) | under `header_decl` |
|---|---|---|
| `## View: Default(default)` | `Default(default)` | display `Default`, id `default` |
| `## View: \|Default (default)\|(default)` | pipes literal | display `Default (default)`, id `default` |
| `## View: Sales (copy)` | `Sales (copy)` | **display `Sales`, id `copy`** |
| `## Category: Region(code)` | `Region(code)` | **display `Region`, id `code`** |
| `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** |
| `## Category: Time Period` | `Time Period` | unchanged; id `time_period` generated |
**The documents that change meaning** are those with a view or category header ending in
a parenthesised run of letters, digits, `_` or `-`. **The app writes such headers
itself**: `improvise import` on a CSV with headers like `Region(code)` or `Size(cm)`
saves `## Category: Region(code)` today (improvise-80rm). Any view someone named
`Sales (copy)` does the same.
**What it would cost if old files got the new rule** (the case answer 15 rules out):
- The category is renamed on load: display `Region`, id `code`.
- Every reference in that old file spells `Region(code)`, which resolves to neither the
display nor the id.
- So the axis lines and every data key for that category do not resolve.
- Today's loader would silently drop the axis lines and orphan the data, so nothing is
drawn.
**Decided (answer 15):** old-version files keep the whole-line reading, so none of
this happens to a file that exists today. On the first save the writer pipe-quotes
such a display, for example `## Category: |Region(code)|(regioncode)`, so the new file
cannot split it either.
### 10. The file version is explicit, and it alone picks the regime
Following answers 15 and 25:
- **The version is already explicit.** The first line of every `.improv` file is a
mandatory version line, matched as an **exact literal**
(`version_line = { "v2025-04-09" ~ NEWLINE ~ blank_lines }` in `improv.pest`), and
`format_md` writes it. Nothing is added to make the version explicit. What is added is
a second literal.
- **The new format gets a new literal.** The grammar's `file` rule gets one arm per
literal, and each arm uses its own header and reference rules. The new literal's
exact spelling is left to the implementation.
- **The regime is chosen only by the first line**, never inferred from what the body
looks like. So the old literal means no ids anywhere, and the new literal means ids
on every declaration (answer 21).
- **A file whose first line is neither literal is refused**, as it is today.
- **Every save writes the new literal** (answer 12). So an old file changes version on
its first save.
### 11. Migration
1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224.
2. **Types:** the id newtypes over `Symbol`, the `display` fields, and `FormulaId` and
`ViewId`.
3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with
display→id resolution moved to every input edge.
4. **Persistence:** the new version literal (clause 10), the grammar in clause 5, the
loader and its refusals, slugify, the writer, and the round-trip properties with a
formula-line strategy.
5. **Projections and `ViewState`** (clause 6).
6. **Vocabulary:** the id-addressed variants, `Rejection::Gone`, `ids_named` and
`broken_chain`'s id arm.
### 12. A formula edit's names are fixed when the editor opens
**This clause is the ADR's own reading, not a user answer.** It closes a contradiction
the adversarial review of `4830fa30` found between clause 6 and the km71 section, which
is also ChatGPT Codex's P1 on `29ad14f4`.
**The problem.** The formula panel shows display names (answer 19), and the user edits
that text. If the host maps those names to ids at **commit**, a concurrent rename breaks
the edit one of two ways:
- the name the user typed no longer resolves, so the edit is refused, which contradicts
clause 6's "does not refuse an edit of `Profit`";
- or the name now belongs to something else (`Revenue` renamed away, another item
renamed to `Revenue`), so the edit silently binds to a different thing. That is the
glitch answer 3 exists to prevent.
**The rule.** The display-name-to-id mapping is **captured when the formula editor
opens**, and the commit resolves the edited text through that captured mapping. This
is the capture-at-open rule the cell editor already follows: `OpenedCell` in
`improvise-protocol/src/client.rs` records the record, its version, the document
generation and the view at open, and the commit answers for those rather than for
whatever the document holds by then.
- The mapping covers every display name the panel showed in that formula's text,
including the target's.
- A name the user types that was **not** in the captured mapping is new to the edit. It
is resolved against the document at commit, as any typed name is. If it does not
resolve, the edit is refused and says which name.
- A captured id that no longer exists at commit is `Rejection::Gone` (clause 7).
- `was` is still compared against the stored id-form text (clause 6), so a concurrent
*replace* of the formula is still seen.
**Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. From
another client, rename the item `Revenue` to `Sales` and rename a different item to
`Revenue`. Commit the unedited text. The formula must still reference the original
item's id, and the commit must not be refused. The test must fail under
commit-time resolution, which binds to the new `Revenue`, and it must fail under a
refusal on rename. Both are the rivals this clause rules out.
## What this means for you
### improvise-km71 — editing a formula (landed as an interim)
km71 landed in PR #214, before this ADR, and was built to be revised by it
(`context/code/core.md` §18):
- **What landed.** `ModelCommand::ReplaceFormula { id: FormulaId, was, raw }`.
`FormulaId` is opaque, with private fields `(target, target_category)`.
`Model::replace_formula` writes in place, keeping the position. It refuses a new
target that another formula owns, and it refuses any target change outside
`_Measure` (improvise-kb1d). `was` is the text the editor opened on. It makes a
concurrent *replace* visible, which even a stable id cannot do, so **`was` stays**.
- **What this ADR changes.**
- `FormulaId`'s fields become one id. The signatures that carry `FormulaId` do not
change, which is what making it opaque was for.
- A target change becomes a **display** rename of the target. The id does not move,
and nothing that references the target is rewritten. So kb1d's refusal outside
`_Measure` can be lifted, and so can the dangling-reference report core.md §18
describes.
- The panel shows `Profit (profit) = Revenue - Cost` (answer 19). The user edits that
text. Its display names are resolved to ids through the mapping captured when the
editor opened (clause 12), not through the names the document holds at commit, and
`was` is compared against the stored id-form text (clause 6). The parenthesised id is not the user's to
change, since the host assigns ids (answer 14). An edit that changes it is refused.
This is the ADR's reading, not a user answer.
- **Rejections.** Another client's deletion gives `Rejection::Gone(Formula(id))`. A
deletion earlier in the same frame gives the broken-chain refusal.
### improvise-fz77 — renaming a category or item
- **The operations.** `RenameCategory { category, to }` and
`RenameItem { category, item, to }`. The panel rows carry the ids.
- **It is now cheap** (answers 1 and 2): one `display` write. No `CellKey`, axis map,
page selection, hidden or collapsed set, or formula text is rewritten, because all of
them hold ids. No `RecordHash` or `layout_version` moves, so in-flight edits survive.
- **What it still owes:**
- **Uniqueness.** Refuse a display name already used in the same scope.
- **Dirtiness.** Mark the effect so the grid labels and panels are re-projected
(gotcha 35). The labels are display strings.
- **Tests.** Assert that the *labels* changed and the *surrogates did not*. That is
the property this design exists for.
- **It depends on step 3 of the migration**, the re-keying. Until then there is no cheap
rename to build on.
### improvise-fdk1 — renaming a records coordinate
fdk1 landed in PR #221, before this ADR. What exists on main:
- `ModelCommand::RenameRecordCoordinate { record: RecordHash, category: String, item:
String }`, which moves a whole record to a new coordinate and creates the item if need
be;
- `CellId::Coordinate { record: RecordHash, category: String }`, the surrogate a
records-mode coordinate cell carries.
**What this ADR changes**, as it does for km71's `FormulaId`:
- `category` becomes a `CategoryId`, in the command and in `CellId::Coordinate`.
- `item` becomes an `ItemRef`: `Existing(ItemId)` or `New(display)`. That is the
create-by-display half of clause 6, since the host assigns the new item's id.
- Unlike a display rename, this **changes a coordinate**, so the record's `CellKey`s and
`RecordHash` change. It is a different record, and that does not change.
## Questions as asked, and where each now stands
The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14,
revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All
are answered.**
| # | question | my recommendation | status |
|---|---|---|---|
| 1 | Integer or word ids? | first draft: words; revision 1: integers only | **answered**: interned symbols (answer 4), generated by slugify (answers 8, 16). Words. |
| 2 | Formula id placement | F1, `- Profit(3) = …` | **answered: F1** (answer 5) |
| 3 | Headers in old files | first draft: accept the rare break; revision 1: gate on version | **answered**: the space is optional (answer 6), and old files keep the whole-line reading (answer 15, Q12) |
| 4 | Do `## Data` keys carry ids? | no | **answered: yes, ids** (answer 7), reversing the recommendation |
| 5 | Group ids? | not now | **answered: not now** (answer 9) |
| 6 | Per-entity versions? | not in this ADR | **answered: not in this ADR** (answer 10) |
| 7 | `RecordHash` over ids? | no, keep names | **answered: over ids** (answer 3), reversing the recommendation |
| 8 | Duplicate id: repair or refuse? | revision 2: refuse | **answered: refuse the load** (answer 11) |
| 9 | Reading of "opt-in" | read optional, write always | **answered: yes** (answer 12) |
| 10 | Id reuse after a reload | revision 2: (b) accept reuse | **answered: (b)** (answer 13) |
| 11 | Who assigns ids? | the host | **answered: the host** (answer 14) |
| 12 | Old files and the new header rule | (a) no | **answered: (a), with the version marked unambiguously** (answer 15) |
| 13 | Slugify details | all generation; keep case; keep `_` `-`; strip non-ASCII; `<kind>-<n>` | **answered** (answer 16): all generation, **lower case only** (reversing the recommendation), keep both, strip, `<kind>-<n>` |
| 14 | Formula text: ids or display names? | ids in the file | **answered**: ids in the file (answer 3); the panel is Q16 |
| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-` |
| 16 | What the formula panel shows | none | **answered: display names, target shown as `Profit (profit)`, ids where a name is ambiguous** (answer 19) |
| 17 | Capitals in a hand-written id | (a) no | **answered: no** (answer 20) |
| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse**, plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) |
| 18(b) | What "casefold" means | full case folding | **answered: full Unicode case folding** (answer 26) |
| 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answer 22), stronger than (a) |
| 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23) |
| 21 | An old file naming an undeclared item | (a) declare it at load | **answered: (a)** (answer 24) |
| — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal |
| 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) |
## Still open
Nothing is open among Q1–Q22, and answer 29 settles the one case it raises: two
identical hand-written ids are refused at load. **The user approved revision 7.
Revision 8 adds clause 12, which is the ADR's own reading, and the user approved it
("looks good, approved", 2026-09-25 00:17:10Z).** Three follow-ups from the same
review stay as tickets, not ADR text: improvise-le4p, improvise-m56i and
improvise-p003.
## Consequences
**Good.**
- Edit, rename and replace are distinct, addressed operations.
- A rename is one display write: it moves no key, surrogate or basis tier, and in-flight
edits survive it.
- `## Data` keys are unique by construction.
- km71's `FormulaId` becomes a real id by changing one struct, and kb1d's refusal can be
lifted.
**Bad.**
- Re-keying storage is wide: every name-taking API moves to ids, and display→id
resolution moves to the input edges.
- A second version literal (clause 10), and every old file changes version on its first
save.
- `Rejection` loses `Copy`.
- `Formula::raw` is no longer verbatim: the file carries formula text over ids.
- The formula panel is a second rendering of formula text, over display names, so the
host must resolve display names typed there back to ids (answer 19).
- Ids are lower case, so an old file's data keys change spelling on its first save.
- ADR 002's prose needs a wording update when this lands.
**Not addressed.** Per-entity versions (Q6), group ids (Q5), and ids in `AxisFrame` and
`CellId::Derived`.
+5 -2
View File
@@ -136,7 +136,10 @@ environment, where egress leaves through your own network boundary.
been rejected in review.
- A merged PR cannot carry follow-up work: restart from the latest default
branch and open a new PR.
- Prefer merges to rebasing (`AGENTS.md`), including when bringing the
default branch into a working branch.
- Reconcile diverging history only by merge, including when bringing the
default branch into a working branch. Never rebase onto main, and never do
anything that needs a force push. Add a new Conventional Commit by default.
Only an unpushed HEAD may be amended, and only for a small fix. Never
reorder or fold commits (`.agents/skills/wave/agent-rules.md`, rule 8).
- Keep `context/repo-map.md` and `context/design-principles.md` current —
they are the two docs agents are expected to maintain by hand.
+77 -10
View File
@@ -312,16 +312,83 @@ carries it or has to agree with it. The TUI colouring the ticket hoped for is
still possible later, and would want the severity *on the message*; that is a
separate change with a separate staleness problem.
**The classification is per writer, and it is not total.** Every refusal in
`ui/effect.rs` and in `cmd/{cell,commit,fit,grid,mode,search,text_buffer}.rs`
goes through `refuse`. `EditOrDrill`'s collapsed-group arm and the three
spelled-paste declines in `cell.rs` (aggregate, coordinate column, inside a
drill) joined them with improvise-i3o5. Those still written with `set_status`
exit 0 as before, and are listed on that bead: three in `cmd/panel.rs`, and
`command/keymap.rs::dispatch`'s arg-error arm. The sweeps that pin refusals
revert `refuse(` calls, so they **cannot see a refusal written as
`set_status`**. Finding one takes a read of every `set_status` on a decline
path, and that is how these were found. `q` on a dirty document is deliberately left a
**The classification is per writer.** Every refusal in `ui/effect.rs`, in
`cmd/{cell,commit,fit,grid,mode,panel,search,text_buffer}.rs` and in
`command/keymap.rs::dispatch` goes through `refuse`. The last of them were
converted in two steps:
- **improvise-i3o5:** `EditOrDrill`'s collapsed-group arm, and the three
spelled-paste declines in `cell.rs` (aggregate, coordinate column, inside a
drill).
- **improvise-cwfj:** the three `cmd/panel.rs` declines, and `dispatch`'s
arg-error arm.
The sweeps that pin refusals revert `refuse(` calls, so they **cannot see a
refusal written as `set_status`**. Finding one takes a read of every
`set_status` on a decline path, and that is how both batches were found.
**A bound key whose command will not build is a refusal** (the orchestrator's
ruling, improvise-cwfj). The key asked for something that did not happen,
which is this section's definition. So `dispatch` answers a constructor's
`Err` with `refuse(e)` rather than `set_status(e)`. That arm is where every
key-path decline lands, including `p`, `x` and `i` onto an aggregate, a fold,
a coordinate column or inside a drill.
Before the conversion, the key path was checked for any reader of the flag:
- `last_batch_refused` is read in production only by `Host::mutating_frame`,
which handles frames and not keys, and by `main.rs`'s headless loop, which
handles `:` lines and not keys.
- `Refuse` is `ViewOnly`, like `SetStatus`, so the `Dirty` a key returns and
the projections built from it are unchanged.
- The browser lockstep suite compares `ViewState`, including `status_msg`,
never the flag, and `Refuse` writes the same message.
The whole workspace stayed green with the change. `q` on a dirty document is deliberately left a
report: headless, its text is false — the CLI saves anyway (improvise-ep5o).
The two rivals udt2 named both stay green: a successful `:w` and a successful
`fit-measure` report through `set_status` and exit 0.
**`remove-formula` of a formula that is not there now exits 1** (improvise-tej1,
improvise-9trl). It used to exit 0: the effect was keyed by names and removing
nothing was a silent no-op. Since tej1, `RemoveFormula` names its formula by
`FormulaId`, and `remove-formula <category> <target>` resolves the two names
to an id in `execute`. When nothing matches it `refuse`s with `No formula for
<target> in <category>`, so a script that removes a formula it never added now
fails where it used to pass. That is the §10 rule applied: nothing changed, so
the request was not met. `remove_formula_of_a_missing_formula_is_a_refusal`
pins it, with a present formula as the control.
## 11 — a headless import refuses colliding records unless told otherwise (improvise-cgqt)
`import --no-wizard` takes `--on-duplicate refuse|sum|separate`, and the
default is `refuse`. The argument is in `context/code/formula-and-io.md`,
"Colliding records are a question, not a default". In short, a default of sum
is the silent outcome the user called critical, and a default of separate
reshapes a scripted document. The refusal prints every colliding tuple through
`WizardCollision::describe`, the wizard's own wording, then the flag. It exits
non-zero and writes nothing. It is checked before the build rather than read
off the build's error, so the listing is structured data and not a parsed
sentence. `separate` uses `ImportPipeline::suggested_duplicate_category`, and
there is deliberately no flag to name the category yet.
## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
`improvise import` takes any mix of files and folders. A folder stands for its
`*.csv` files, in name order. A folder always merges, even when it holds a
single CSV, so that file keeps its `File` item and view. A single file named
directly still imports alone (the user's ruling on the PR). The rules
and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
one sparse model, with a view per file". The CLI's part is the one question
`Storage` cannot answer: whether a path is a folder. `get_import_data` asks
`Path::is_dir` for each argument, hands the resulting `ImportPath`s to
`expand_import_paths` over `PhysicalFs`, and from there takes the path a list
of files always took. A folder with no CSV in it is refused before anything is
read. The message names the folder, and the run exits non-zero and writes
nothing.
A merge now reaches the build as an `ImportPipeline` carrying the merged files'
`sources`, rather than as a bare JSON value. Both `--no-wizard` and the TUI
wizard (`run_tui` → `App::start_import`) therefore build the per-file views.
The headless path matters most, because the code-maat pipeline
(improvise-1xfb.5.1) runs there. `a_headless_import_of_a_folder_merges_its_csv_files_and_repeats_byte_for_byte`
pins the views, the sparse Data lines and the byte-identical second run, with
the first run's output sitting in the folder it reads.
+606 -21
View File
@@ -447,9 +447,10 @@ the status after.
- `toggle-prune-empty` words its status off `View::prune_empty`, which is in
the `Workbook`, in no cache, and not in `ViewState`.
- `delete-formula-at-cursor` needs `(target, target_category)`; `FormulaCache`
carries the formula's raw *text*, and recovering the two from it means
reimplementing `parse_formula` on the client.
- `delete-formula-at-cursor` needed `(target, target_category)` while
`FormulaCache` carried only the formula's raw *text*. **Closed by
improvise-tej1**: since km71 the cache carries a `FormulaId` per formula, and
`ModelCommand::RemoveFormula` now takes that id.
- `view-back` was listed here too, and **improvise-cfly removed it**: once
PR #115 put `Command::ViewBack` in the view half there was nothing left to
enrich. See "The three improvise-e24v listed as free" below.
@@ -516,6 +517,113 @@ seeds the `edit` buffer from the cell it landed on. The trailing `clear-buffer`
the keymap welds to every commit then empties that buffer again. That ordering
looks wrong and is exactly what the TUI does; mirroring it is the point.
The re-entry is `reopened_at`, and it declines where the host's
`EnterEditAtCursor` declines (improvise-enz6). The host asks
`core::uneditable` of its layout. The browser asks the landing cell's projected
`CellId`, which is the same answer, because `cell_id` is what fills it. Asking
the cache has two consequences:
- **The wording differs on a fold.** The host can tell a fold from any other
aggregate and names the fold. The browser sees only `CellId::Aggregate`, so it
gives the host's generic aggregate text. `lockstep.rs` compares the mode, the
cursor and the buffers here, never the status text.
- **A landing past the window is decided when the host answers**
(improvise-5m9z). The cache holds only the announced window, so a `Down` from
the window's last row lands on a cell with no projected id yet. `reopened_at`
opens the editor there provisionally, and the fold settles it:
`ClientState::settle_declined_editor` runs after every fold. It closes an
**untouched** editor whose cursor cell the host has now projected as one `i`
declines, and says why. The client then ends where the TUI does. Both ends
ask the same `resolve::declined_at`, so the immediate decline and the late
one cannot disagree.
The decline waits for the host's answer, as #227's awaiting seed does. It does
not choose a seed, which stays `seed_at`'s and #227/#272's.
**A touched editor is kept.** If the user typed before the answer came, the
editor stays open with their text. Closing it would throw input away. Its
commit is then refused out loud (improvise-ik6l): `refused_here` declines the
`commit-cell-edit` or `commit-and-advance-right` **step**, which takes the
whole binding with it. That includes the welded `clear-buffer`, as the TUI's
failed `committable_cell_key` does, so the status is said and the mode and
the typed text stay. Before this fix the commit composed nothing, and Enter
went `Upstream`, which no host answers: a silent dead key.
**`declined_at` reads the cache through the frame's own origin**
(`AxisFrame::first`), never the cursor's offsets (improvise-y5eb). The settle
also runs when the client folds its **own** outgoing frame. By then the
offsets have moved but the cache still holds the old window. Indexing it with
the new offsets read the wrong cell. A `Down` that wraps from the bottom row
to the next column's top read the old window's top row, and closed an editor
on Revenue/Feb with Profit's formula refusal. Read against the frame's
origin, a stale cache answers `None` ("not projected yet"), and the host's
own reply decides. The earlier version of this section called that aliasing
unreachable. It is reachable, and the wrap is the case that reaches it.
The settle is not specific to advancing. An untouched editor also closes
when the host re-projects its cell as declined for any other reason, such as
another client adding a formula over it.
**Every cache read goes through the frame's origin, not only
`declined_at`** (improvise-7mro). `cell_at_cursor` and `seed_at` read through
`projected_at` too, and the cursor-offset read `cell_at` is gone. Before
this, every other read had the y5eb aliasing. A client folds its own scroll
at once, but its cache holds the old window until the host answers, so a key
resolved in that gap read another cell. Pipelined `$ i 7 Enter` in a
one-column window wrote **Jan/Revenue** where the TUI wrote Feb/Revenue: the
seed and the write's target both came from the cell at the old window's
index. `G i` and `gg i` refused an editable cell with a formula's refusal.
The aliasing had a second form, inside the held window: after `k k j` the
cursor is on a cell the cache holds, but the offsets no longer match the
frame's origin, so index arithmetic through the offsets named the next row.
Reading at the frame's origin answers both forms with one rule.
**A cell of the grid outside the held window is `unprojected`**: unknown,
never "the cell at the same index". Inside the grid's extent but outside
either frame's range. An empty grid has nothing to wait for, so it is not
unprojected. What each key does on such a cell:
- **`i` opens an editor awaiting its seed** (`OpenedEditor::awaiting()`).
The fold settles it once the host projects that cell
(`settle_awaited_seed`), and never over typing. The settle waits for a cell
with an **id**. Between a frame and its cells, a relabel leaves a blank
`CellDisplay` at the position, and settling on that seeded `""` before the
cell arrived. A declined cell closes through `settle_declined_editor`,
which runs first.
- **A commit, `x`, `yy` and `p` wait out loud.** `refused_here` declines the
step with "waiting for the host to show this cell", which keeps the mode
and the typed text, as ik6l's refusal does. Nothing is composed: the write
target is unknown, and guessing it is the bug. Pressing the key again after
the host answers does the write. This is the Codex finding on #282.
- **An advance's reopen keeps #282's shape.** It reads `seed_at`, which is
`""` for an unprojected landing. On this base the keymap's welded
`clear-buffer` empties the reopened buffer anyway, in the TUI too, so an
awaited seed there would be written into a buffer the TUI has cleared.
**What a held reply still costs.** With every reply held, the browser cannot
know a seed the TUI shows, so `G i 7` types `7` into an empty editor, where
the TUI types `47`. The write lands on the TUI's record once the host has
answered, never on another. Matching the TUI exactly would need the client
to defer keys until the projection arrives. That means `on_frame` handing
queued frames back to `web/app.js`, which is a change to the page's surface
and is not made here.
**Merging with #227/#272 and #274.** #272 adds the same `OpenedEditor::awaiting`
field and an `awaiting_seed` for the editor an advance reopens after a write,
settled once no write is unanswered. Merged, the settle must require both:
no write unanswered, and the cell projected (`projected_seed`). With #227's
commit-owned clear, the reopen then awaits for an unprojected landing too.
#274 captures the committed cell in `settle_opened` through
`opened_at_cursor`, which reads `cell_at_cursor`. Without this fix that
capture aliases as well: a horizontal wrap with replies **delivered** writes
Jan/Revenue on #274's head. With `cell_at_cursor` reading at the origin, the
capture waits for the projected cell, because `settle_opened` retries on
every fold.
**When `CellId::NonItem` lands (#265), both `declined_at` and
`core::uneditable` need an arm for it.** Both matches are exhaustive, so the
compiler will ask. The trap is answering in the `None` arm: a non-item cell
would then open an editor whose commit writes a ghost record.
`GridMove` is a hand-spelled mirror of `navigation.rs`'s `MoveKind`, because the
command crate is not on a browser's dependency list and never will be. The
*keys* that reach it are no longer spelled twice. `scrolled_to` mirrors
@@ -578,11 +686,76 @@ immutable `CmdContext`; the commands are then folded in order.
### What still needs the host
`the_command_names_that_still_need_the_host_are_these` names them, and
`Host::handle` answers a `Key` frame with an empty vec, so each one is a binding
that **does nothing at all** in a browser
`resolve::NEEDS_THE_HOST` names them, and
`the_command_names_that_still_need_the_host_are_these` drives each through the
key that reaches it. `Host::handle` answers a `Key` frame with an empty vec, so
each one is a binding that **does nothing at all** in a browser
(`doctrine(an_upstream_key_is_a_black_hole)`).
**improvise-fo0w took four names off it and left three** (`exit-search-mode`,
`force-quit`, `toggle-group-under-cursor`). Two of the four now resolve, and two
now decline.
- **`enter-advance` (Normal `Enter`) resolves.** It is cursor arithmetic over
`GridExtent`: down, then to the top of the next column. The host's
bottom-right `AbortChain` becomes `Ignored` on the client. Nothing is lost,
because `Enter` is bound alone and never inside a `Sequence`, so there is no
later step for the abort to cut.
- **`hide-selected-row-item` (`H`) resolves** to the one `ModelCommand::HideItem`
the host's effect is, over `row_frame.cats[0]` and the selected tuple's `[0]`.
That is the outer category and its item, exactly as the Cmd reads
`row_cats.first()` and `items[0]`. On a fold the client refuses in the host's
own words. It knows a fold from `AxisFrame::folds`, the layout's own
per-item flag carried on the frame, never from the nearest group header: under
two row categories that header can be the inner one's (improvise-4k1j,
`context/code/protocol.md`). That is why `FOLD_REFUSAL` moved from the command layer into
`improvise-protocol`: one string, not two copies.
- **`search` (`/`) and `search-navigate` (`n`) decline out loud.** `n` scans the
whole model, and a client holds one window. `/` alone would open a box that
can find nothing. Shipping `/` without `n` is the trap the ticket names, so
both decline until search can find.
Declining is not silence, so the two search rows are no longer on the list, and
the help page stops marking them. They are in the same class as `ZZ`: pressed,
they say why.
**The help overlay marks them, and it marks rather than omits** (improvise-uln5).
The overlay is shared content, and before this the browser painted every row
plainly, including six that do nothing there. That is worse than a gap:
improvise-koh6 showed that the keys typed after a dead one land on live Normal
bindings, and `i` opens an editor. The browser's `RenderHelpRow` now draws a row
whose command is on the list as `dim`, and appends "(not in the browser yet)"
to its description. It does the same for every ex-command row while `:` opens a
command line this client cannot run (`resolve::runs_the_command_line`, which is
`stranding`'s answer, not a second one).
Marking was chosen over omitting for three reasons:
- **Both front ends draw the same rows.** A page with rows missing is a second
layout of the same content.
- **The feature stays findable.** The row still says it exists in the terminal.
- **It needs no change in `web/`.** `dim` is a kind the page already paints.
The TUI is unaffected, because the marking lives in the browser's renderer.
**One list, held from three sides.** `NEEDS_THE_HOST` is production data because
the renderer needs it. Three tests hold it to what `resolve` actually does:
- `the_command_names_that_still_need_the_host_are_these` fails if a name on it
resolves locally.
- `every_help_row_the_browser_cannot_run_is_one_it_marks` fails if a help row
is silent in the browser without being on the list, or on the list without
being silent. It also fails if a *hint line* ever names a listed command,
because the status bar is never marked.
- `every_silent_binding_is_on_the_list` fails if any binding in any mode a
browser can reach is silent and unlisted (improvise-oyaa, s2gb). It walks the
keymaps, parents and prefixes included, because the help rows are only a
subset: no help row names Ctrl-Q, so before it `force-quit` could leave the
list with every test still passing. Which modes are reachable is derived by
pressing, starting from both halves of Normal. A door that `declined` shuts
therefore drops out with no list to keep.
`every_advertised_command_answers_in_its_own_mode` therefore no longer restates
the black holes. It skips a silent row whose command is on the list, because
the other test owns it.
**An upstream key that the app also ADVERTISES is a different severity, and it
had its own class of defect** (improvise-rmvx). A key nobody offers doing
nothing is a gap; a key named in the mode's own hint line doing nothing reads as
@@ -639,11 +812,13 @@ The classes that remain:
longer blocked** and are simply not built here yet (improvise-e24v): since
improvise-anr9 `GridCache` carries an `AxisFrame` per axis, so `cats[0]` +
`label(idx, 0)` is exactly what `HideSelectedRowItem` reads off `GridLayout`,
and `group_of(idx)` is exactly `ToggleGroup`'s `(category, group)`. The one
and `group_of(idx)` gives `ToggleGroup`'s `(category, group)` as
`(cats[level], name)` (improvise-e265). The one
thing to get right is that the frame is **windowed** and the cursor is
absolute, so a client must reconcile the scroll offset first, as `render.rs`
does for the selection. `delete-formula-at-cursor` stays: it names a position
in the formula panel, not in the grid.
does for the selection. `delete-formula-at-cursor` left this list with
improvise-tej1: the position it names is in the formula panel, and the id
the panel's cache now carries is enough to resolve it.
- **Anything touching storage or an arbitrary command line.** The host's by
definition.
- **The records-mode commit** — *except the two creating columns*. A wide-records
@@ -656,7 +831,7 @@ The classes that remain:
column renumbers every index to its right and the advance was computed from
the pre-commit layout (gotcha 23), so `CommitAndAdvance` returns straight
after it and a client that advanced anyway would land one column off,
silently. `AddCategory` can also be refused by the 12-category limit and sets
silently. `AddCategory` can also be refused by the category limit and sets
`abort_effects`; the mode change therefore goes **first**, where an abort
cannot reach it (gotcha 21). **Since improvise-bd9o that exception is
unreachable**, and this bullet describes `commit` alone: the records decline
@@ -698,7 +873,9 @@ fixture standing on one of them cannot see the other:
even open an editor. Nothing happened and nothing was said. The ticket did not
predict this one, and it is not a second bug: it is improvise-fdk1 seen from
the key path, since a coordinate rename has no wire form precisely because the
`RecordHash` surrogate names records that already exist.
`RecordHash` surrogate names records that already exist. **fdk1 has since
closed it**: the coordinate is now the one records cell the browser edits. See
"A coordinate rename" below.
So the guard goes at the **entrance** — the first statement of `edit_or_drill`,
before the cell is even looked at — and not on the `CellId::Record` arm where
@@ -1019,8 +1196,14 @@ recoverable without reimplementing `parse_formula` on the client — the gap
`decision(improvise-e24v)` recorded for `delete-formula-at-cursor`. km71 closed
it the gotcha 47 way, by **enriching the projection**: `FormulaCache::formulas`
is `Vec<FormulaEntry { id: FormulaId, raw }>`. The id is opaque on purpose
(`context/code/core.md` §14). `delete-formula-at-cursor` is still declined in
the browser; it now needs only its resolver.
(`context/code/core.md` §18). improvise-tej1 then composed
`delete-formula-at-cursor` in the browser on the same id. `RemoveFormula`
names its formula by `FormulaId` on the wire and in the effect, and a delete
whose id no longer resolves is **refused with the batch stopped**, where it was
a silent no-op: a frame that named a formula someone else removed is rolled
back whole (`Host::mutating_frame`) and answered `Rejected(MutationRefused)`.
The script command `remove-formula <category> <target>` resolves its names to
an id in `execute` and refuses out loud when nothing matches.
One divergence is pinned rather than closed: a rename. The host's
`ReplaceFormula` rewrites the status to say what was renamed and which formulas
@@ -1075,9 +1258,11 @@ implemented rule is "clicking the already-selected tile cycles it; clicking any
other selects it", where *already selected* is `mode == TileSelect` **and**
`tile_cat_idx == index`. The tempting summaries — "a click never mutates on its
own", "a **second** click cycles" — both have the same counterexample, found on
review: `T` enters `TileSelect` leaving `tile_cat_idx` at **0**, so a click on
the **leftmost** tile is a click on the already-selected one and cycles on the
*first* gesture. Two adjacent tiles therefore answer the identical first click
review: `T` enters `TileSelect` with `tile_cat_idx` at the tile
`opening_tile` picks (the first Regular category, improvise-oplp; see
`context/code/command.md`), so a click on **that** tile is a click on the
already-selected one and cycles on the *first* gesture. Before improvise-oplp
this was the leftmost tile, `_Index`. Two adjacent tiles therefore answer the identical first click
differently, and that is correct rather than a hole — a click does what `Enter`
does there, and `Enter` in `TileSelect` cycles the tile under the cursor. The
resolver cannot see *how* the state arose and must not try to.
@@ -1360,6 +1545,96 @@ the first discards the second, which the host *did* apply. A frame the host will
not answer settles as soon as nothing older is waiting — only from the front,
because settling it out of turn would lose the order.
**What the host *relays* is settled too, and it settles after the frame it
arrived during (improvise-jpba).** `settled` used to take only this client's
own frames. Relays never reached it: a `ResetCursor`, the `SwitchView` sent when
the view a client is on is deleted, and the placement echoes after a reframe
(improvise-wdmp). The host has already applied each of them to this client's
mirror, so the next refusal rebuilt `view` from a settled slice that had never
heard them. A client whose reframe was accepted, and whose next frame was then
refused, went back to its pre-reset cursor while its mirror stayed at the
origin. That is the same client/mirror split wdmp closed, reopened by the
rollback. `fold` is the door for what the host sends (`fold_outgoing` is the
door for what this client sends), so `fold` now records every non-projection
command. It goes straight into `settled` when nothing is in flight. Otherwise
it is queued as an unawaited `Sent { relayed: true }` **directly behind the
head frame**, after any relays already queued there. `accepted` and
`settle_sent` then settle it in the right order, and `replay` replays it.
**Every answer settles, the refusal included** (improvise-3kfc). `refused`
pops the head and then calls `settle_sent` before it replays. Without that call,
an unawaited entry behind a refused head became the front while still unsettled,
and the next answer popped it instead of its own frame. That entry can be a
relay or this client's own view-only frame, and the view-only case predated
relays.
**A relay that queues also rebases `view` (improvise-xd3q).** Recording a relay
in `settled` fixed only the refusal. On the accepted path, `fold` still applied
the relay straight onto `view`, on top of the frames this client had already
sent behind the one being answered. A pruning commit is answered with
`ResetCursor` and an echo of its own placement (improvise-wdmp). A `j` sent
before that answer arrived had already been folded, and the reset clobbered it.
The echo re-sent only the pruning frame's placement, because the host's
`asker_reset` is per frame. So the client sat at (1,0) while its mirror sat at
(2,0), and the next write landed on West where the TUI writes East. The queue
already held the right order, `[head, reset, echo, j]`, so `fold` now rebuilds
`view` with the same `replay` a refusal uses whenever a relay is queued. That is
the settled slice plus everything still in flight, in order. With nothing in
flight, the relay is applied directly, as before.
It is a **replay from `settled`**, not "apply the relay, then re-apply what
queued behind it". Re-applying reruns every frame behind the relay a second
time. That looks the same for idempotent setters like `SetSelected`, but
`ViewBack` pops a stack, and applying it twice stepped a client back two views
while its mirror stepped back one. The host could not have fixed this. It
cannot know whether a frame was composed before or after the client heard the
reset, and only the client knows what it sent.
**What the client says to itself queues too (improvise-n0pb).** A refusal's
sentence ("…your edit was not applied") is written by the client, not sent by
the host. It used to go straight into `view.status_msg`, after `refused` had
rolled back. That put it in neither `settled` nor `in_flight`, so the xd3q
rebase erased it the next time a relay queued behind a frame still in flight.
A frame in flight that carried its own `SetStatus` was replayed over it. The
sentence is a local event that happened at a moment, so it takes its place in
the queue at that moment. `ClientState::notify` queues it as a view-only entry
at the **tail**, which is what `fold_outgoing` does with a frame the host never
answers. This gives it the right order against everything else:
- Every frame sent before it replays first, so none of them overwrites it.
- A relay queues behind the head, so it lands before the sentence and cannot
erase it either.
- A frame sent after it lands after it, and may replace it. That is the same
order in which the user saw them.
- It settles once everything ahead of it is answered.
Two simpler fixes were rejected:
- **Writing the sentence into `settled` as well:** the next rebase replays the
in-flight frames over it, which puts the sentence *before* frames it
followed.
- **A status channel outside `ViewState`:** this would be a second answer to
"what does the status line say", which every renderer would then have to
merge.
Two simpler shapes were rejected, and each fails the Host-level test:
- **Advancing `settled` directly** gets the order wrong. The head frame's own
commands settle only when it is accepted, which is *after* the relays that
arrived during it. So for `[SetRowOffset(1), <reframe>, SetSelected]`,
`settled` ends at `row_offset 1` while the mirror is at 0, because the reset
was settled before the offset it cancelled.
- **Appending the relay to the head frame's own commands** loses it whenever
that frame is refused. The frame is dropped, and the relay goes with it.
Why "behind the head" is right: the host answers frames in order, so a relay
that arrives before the head's answer happened on the host before or during
the head frame. A relay from the head frame itself (its own reset and echoes)
is only ever sent when that frame is accepted, since `undo_frame` sends no
relays, and it belongs after the frame's commands. A relay from **another**
client's frame that the host processed *before* the head also lands behind the
head, which is one position late. That is a pre-existing pipelining residue
and is not addressed here. A reframe by someone else before the head frame
usually makes the head stale anyway, which the host refuses, and on refusal
the relative order of the two no longer matters.
What this does **not** do is force. A rejected batch now keeps its values and
leaves the user where they were; re-composing the same intent against a current
basis at one keystroke is still `improvise-3al1`, and it is unblocked rather
@@ -1496,14 +1771,17 @@ generation is captured at open and the swap refuses the commit. So the result is
Clearing the cell under an open editor is `RecordMoved`, **not** `RecordGone`
(improvise-4img). The store no longer holds the record, but
`App::resolve_record` falls back to the layout, and the grid still draws the
row. So the record resolves, and its counter has moved. `RecordGone` needs a
record that neither the store nor the layout resolves.
`App::resolve_record` falls back to the layout of the view the editor opened
on, and that grid still draws the row. So the record resolves, and its counter
has moved. `RecordGone` needs a record that neither the store nor that layout
resolves.
**`EditTarget` is the one object an editor captures at open, and it is a
struct so it can grow.** Two fields have joined `cell` and `version` since:
struct so it can grow.** Three fields have joined `cell` and `version` since:
the seed as `opened` (improvise-ixxx: a commit whose buffer equals the seed
writes nothing) and the generation (improvise-vndi). The page did not change
writes nothing), the generation (improvise-vndi), and the `view` whose grid
drew the cell (improvise-wpxr: an empty record resolves only against that
grid, so the commit names it, captured at open like the version). The page did not change
for either, because the target is opaque to it. `typed_target_frame` asks the
two in that order: an untouched editor writes nothing, so it is never refused.
@@ -2267,6 +2545,72 @@ reads that text. An emptied node cannot tell "nothing was said" from "something
else was said", and the assertion would pass vacuously the moment the timer
fired.
#### A message beside its open editor stands until that editor closes (improvise-1x9h)
The timer is wrong for one kind of message: a refusal the user is still
fixing. Six seconds after it appears, the toast hid it while the minibuffer it
was about stayed open. That is the TUI's su4u bug with a timer instead of a
keypress, and the rule is the TUI's, so both front ends ask one question.
**The rule: a message stands while the editor it was written in is still the
open one.** "The editor" is `improvise_protocol::EditorSession`, the value the
TUI's `press` compares before and after a key (`context/code/tui.md`, "What
'the same editor' means"). It moved from `src/draw.rs` into the protocol crate
so that the browser asks the same question rather than a second derivation of
it (design principles §5).
**The decision is Rust's, and the page only obeys it.** `ClientState` observes
the status wherever the view changes (`settle_status`, beside `settle_mode` and
`settle_editor`). That means every applied command, and also every `rebase()`,
`refused()` and `bootstrap()`, the three paths that rebuild the view wholesale:
- a status that has changed since it was last observed is owned by the editor
open at that moment, or by nothing outside a text-entry mode;
- an unchanged status loses its owner, for good, as soon as a different editor
(or none) is open.
`RenderModel::status_stands` is `ClientState::status_stands()`. The page passes
it to `createToasts.offer(text, stands)`. A standing message arms no timer. When
it stops standing it hides at once, not six seconds later, because the TUI drops
a refusal on the key that leaves its editor.
**"For good" is the part that needs memory.** A refusal in `FormulaEdit`, then
`Esc`, then `n` again opens an editor *equal* to the first: the same mode, the
same cursor, no seed. Comparing the current editor with nothing would make the
old message stand again beside the new opening. The owner is therefore
recorded when the message is written and cleared when that editor closes, not
recomputed from what is open now.
That clearing has to happen on every path that moves the view, and a relay's
rebase is one of them (improvise-0dbk). A relay heard while a frame is in
flight rebuilds the view by `rebase()`. If `rebase()` does not settle the
status, a rebase that moves the view off the editor leaves the owner in place.
When the user returns to an equal editor, the old message stands again.
`a_rebase_off_the_editor_ends_the_message_standing` is that case.
**A status written outside a fold must be observed where it is written.**
`socket.rs` writes a rejection's sentence *after* `refused()`, so the sentence
survives the rollback. It writes it through `ClientState::notify`, which queues
the sentence at the tail of `in_flight` through `fold_outgoing`
(improvise-n0pb). That path applies it at once and runs `settle_status`, so the
editor open at that moment owns the sentence. Assigning `status_msg` directly
leaves it unobserved until the next fold. By then the editor may have changed,
and the message would be owned by the wrong one. A relay's rebase would also
erase a direct write, which is why improvise-n0pb queues it.
**What it inherits from the identity rule above.** An identical message written
again in a *new* editor is not a new write, so it neither stands nor re-shows.
That is the "two identical messages show once" cost, and it has the same cause:
nothing beside `status_msg` says that it was written again.
**Where the rule is live.** A formula refusal still never reaches a browser.
The client folds `CommitBuffer`'s optimistic "Formula added", and the host's
error is a write to the host's own slice
(`a_formula_the_client_cannot_parse_diverges_on_the_status_line`). What
reaches a browser today is a host rejection that leaves the editor open, such
as a refused save. The rule becomes fully live when a host-addressed status
reaches its composer.
### What the styling is, and what no test here can see
`web/style.css` stopped being "legibility only" and is now a small design
@@ -2778,3 +3122,244 @@ files and refuses one that names the bundle directory, and its companion
asserts `web/tests/bundle/` still holds a test that *does* — a scoping rule is
uninteresting if nothing falls outside it. Evidence in
`context/test-evidence/tooling.md`.
## A coordinate rename (improvise-fdk1)
The coordinate is the one records-mode cell the browser edits. `edit_or_drill`
still declines a records grid, except on a `CellId::Coordinate`. There it opens
the editor on the coordinate's `edit_text`, just as the host's `EditOrDrill`
does. `commit` sends a records grid to `rename_coordinate`, which composes
`ModelCommand::RenameRecordCoordinate` plus the mode change and the advance
that `CommitAndAdvance` makes. Every other records cell is still declined out
loud, and `the_tile_bar_route_into_records_mode_does_not_strand_the_user` walks
the whole row to check that both answers hold.
**The target is captured when the editor opens, not when it commits.** The
record, the category and the record's write counter are captured into
`ClientState::opened` (session-local, never serialized). The commit renames that
record, and its basis claims that counter through `BasisCache::claim_as_read`,
the same door the inline editor's `EditTarget` uses (improvise-0j0f). So if
another client writes the record while the editor is open, the rename is refused
with `RecordMoved`, and the other client's value survives.
**The capture waits for the host's answer, and that wait is the part easy to
get wrong.** A commit re-enters editing one row down in the same batch, and the
client folds that batch optimistically. At that moment its grid cache still
shows the rows as they were **before** its own rename. Records rows are sorted
by coordinate, so a rename moves its row, and the row now under the cursor
belongs to a different record on the host. So `settle_opened` captures only when
no awaited frame is in flight. It runs after every fold and after `accepted()`.
The host sends `Accepted` last, after the projections that describe the answer,
so the capture reads the refreshed cache. A commit made before any capture
exists (typing faster than the host answers) is declined out loud with "still
waiting".
**After a rename, the browser's cursor goes to the origin, not one row down.**
The rename reframes the grid, so the host broadcasts `ResetCursor` to every
client, the asker included (gotcha 24). The TUI's key path stays one row down,
because `apply_effects` observes the reframe and `CommitAndAdvance` restores the
cursor afterwards. `tests/lockstep.rs` cannot see the difference: its twin host
has no `Sessions`, so nothing broadcasts. The difference predates this ticket
and applies to any browser write that reframes the grid.
**An empty coordinate writes nothing, on both ends.** The host's key path
refuses it (`RecordRewrite` sets `refused`, and `App::refuse` does not abort),
says `RECORD_COORDS_CANNOT_BE_EMPTY`, and still advances. Composing the rename
would instead have the host undo the whole frame (`MutationRefused`) and show
the generic sentence. So the client composes the host's answer itself: the same
status, taken from the protocol's one copy of the message, plus the advance and
no write.
**A coordinate editor opens the way a value editor does, and an untouched one
writes nothing.** `edit_or_drill` opens both through `opened_on`, so both record
`SetOpenedEditor` with the seed, as the host's `EditOrDrill` already did for
every cell it opens. `rename_coordinate` then applies improvise-ixxx's
`untouched` rule before anything else: `i` then `Enter` composes the mode change
and the advance and no rename, which is what the host's `CommitAndAdvance` does.
The untouched check comes before the "still waiting" decline and the `Gone`
decline below. An untouched `Enter` writes nothing either way, so it needs no
basis capture and cannot retarget.
**The capture carries the document generation it was read in (improvise-vndi),**
and the rename claims it through `claim_as_read`, as the inline `EditTarget`
does. It cannot be observed: a document swap's rebase `Snapshot` closes the
rename editor and `settle_opened` drops the capture, so no rename crosses a
generation. `a_document_swap_closes_an_open_rename_editor` pins that
precondition.
**Two captures, one editor.** When an editor opens, two things are recorded, and
they answer different questions:
- `ViewState::opened_editor` (`OpenedEditor { seed, touched }`) answers **what
text it opened on, and whether the user has changed it**. It is portable
state. It is composed optimistically at the keypress and folded the same way
on the host, where `CommitAndAdvance` reads it.
- `ClientState::opened` (`Opened::Target(OpenedCell { record, target, version,
seen, document_generation, view })`, or `Opened::Retry`/`Opened::Gone` after a
refused commit, below) answers **which cell the commit writes, and which
record versions and document its basis claims**. `target` is
`Coordinate { category }` for a rename and `Value { measure }` for a stored
value; `seen` is filled for a rename only. It is session-local and never
serialized.
Neither answers "is an editor open": the mode does, for both. `opened_editor` is
not cleared when the editor closes, only replaced when the next one opens.
The record capture cannot move into `OpenedEditor`, and the reason is the
capture rule above. `settle_opened` waits for the host's answer, because a
capture read from the optimistic cache names the row as it stood before this
client's own rename. `SetOpenedEditor` is composed at exactly that optimistic
moment, so an `OpenedEditor` carrying a `RecordHash` would reintroduce the bug
the `capture_unanswered` rival demonstrates. It would also put a client-only
basis read into state the host folds and `tests/lockstep.rs` compares, where
the host's value would have no record in it. Gating the capture on
`opened_editor.is_some()` was considered and rejected: an editor entered by
`enter-mode` records no opening, yet the host still commits a rename from it,
so the gate would strand that path on "still waiting".
**A retry after a refusal may write only the cell it named (improvise-20ek,
improvise-9ww8, improvise-8phw).** A refusal replays the view slice, so the
editor reopens with the typed text over whatever cell the cursor now shows. The
rule is about that cell, and it holds for a rename and for a value alike:
- `refused()` pins `ClientState::opened` to `Opened::Retry(record, target)`,
naming the cell the refused frame wrote: a rename's record and category, or a
`SetCell`/`ClearCell`'s record and measure (`written_to`). The refusal's kind
plays no part.
- Once the host has answered, `settle_opened` captures what is under the cursor.
If that is the same record **and** the same `OpenedTarget`, it becomes the
target at its **current** version, so a second `Enter` goes through. That is
the one-keystroke force `RecordMoved` promises (gotcha 43), and the user has
just been told someone else changed the row.
- If it is anything else, or nothing, `opened` becomes `Opened::Gone`, and
`Enter` answers with a status and no write. Esc leaves the editor, and opening
it again captures afresh.
The whole target is compared, not only the record, because a record is a row
and a value editor names a cell in it: a crossing `HideItem` on a measure leaves
the **same** record under the cursor in another column, and a record-only
comparison forces the retry onto a measure the user never typed for. `seen`
stays rename-only, because only a rename has a destination to claim.
**Why the rule names the record rather than a rejection.** The first version
(20ek) pinned only on `RecordGone` of the renamed record. But `stale_basis`
checks the layout tier before the record tier, and another client's rename
bumps `layout_version`. So when A's `Enter` crosses B's rename in flight, which
takes nothing more than ordinary latency, the refusal is `LayoutMoved`, not
`RecordGone`. The retry then renamed whatever record had moved under the
cursor. That happened both when B had renamed A's own record, and when B's
rename merely re-sorted the rows so that a different, still-living record sat
under the cursor. So "is the source gone" is not the discriminator either. The
only question that separates a force from a retarget is whether the capture is
the record the user typed for. With that rule the explicit `RecordGone` pin is
subsumed: after `RecordGone(r)` the capture can never be `r`. It was removed
once a rival that dropped it failed no test. The reviewer's rival on PR #221,
which kept the opened target through every refusal, still breaks the reorder
case: after a *successful* rename, the next editor must capture the row it now
shows.
**The destination is a record too (improvise-milr).** A rename onto an existing
coordinate overwrites that record (improvise-kndq), so it is a write to the
destination as well, and the destination has to answer to the basis. The
server decides that (`records_at_stake`, `context/code/command.md`). The client
cannot name the destination: it would need the hash of coordinates it has never
been sent, which ADR 002 forbids. So it claims **every record counter it held
when the editor opened**, captured into `OpenedCell::seen` at the same
moment as the target. Two consequences follow:
- If another client writes the destination while the editor is open, the rename
is refused with `RecordMoved`. This holds even when the write has already
reached this client's cache, because the claim is the version at open time.
- If the destination is a record this client was never shown (outside its
window, say), the rename is refused with `RecordUnstated`. Overwriting a
record you have not seen is exactly the lost update the check exists to stop.
**A value editor commits to the cell it opened on, too (improvise-8xq0).** The
keyboard commit over a plain grid cell used to read the cursor at `Enter`: the
record from `record_at_cursor`, the measure from the `CellId` under it, and the
basis from `claim`, which reads the version as it is *now*. Another client's
`HideItem` renumbers the rows under an open editor, the cursor then names a
different cell, and the write landed there with the text typed for the first.
The inline `EditTarget` fixed this for the mouse editor (improvise-2wu3), but it
lives in the page, so the keyboard path never saw it. The fix reuses the rename
capture rather than adding a second one: `settle_opened` captures a `Record`
cell as `OpenedTarget::Value`, and `written` builds the `RecordCell` and the
`claim_as_read` basis from it. Nothing about the write is read at `Enter` except
the text. **The capture reads the cell through the frame's own origin (improvise-7mro).**
`captured()` asks `opened_at_cursor`, which asks `cell_at_cursor`, and since
#286 that reads through `projected_at` (`AxisFrame::first`), never through the
cursor's offsets. That matters more here than for any other reader, because the
capture *sticks*: `settle_opened` takes it as soon as no awaited write is in
flight, which includes the fold of the client's own outgoing frame. An untouched
`Enter` that scrolls moves the offsets before the host has sent the new window,
so reading through the offsets captured the old window's cell at the same cache
index (Jan/Revenue where the TUI writes Feb/Revenue, with every reply delivered),
and every later `Enter` wrote it. Read through the origin, a cell outside the
held window is `None`, `settle_opened` simply tries again on the next fold, and
the capture lands once the host projects the cell. #274 took #286 by merge
rather than porting the reader; the PR body says why.
**The capture includes the view that drew the cell (improvise-wpxr).** A
`SetCell`/`ClearCell` names the view the host resolves it in, because a record
with no stored cell exists only as a position in some view's grid. The keyboard
commit therefore names `OpenedCell::view`, taken by `captured()` from
`ViewState::active_view` at the moment the record, the version and the
generation are taken. It is not the view current at `Enter`. The reason is the
one this whole section rests on: everything a commit writes is fixed when the
editor opens, and `Enter` contributes only the text. A view read at `Enter` and
a record read at open could name two different grids, and the host would
resolve the record in one that never drew it. It is also the rule the mouse
editor's `EditTarget::view` already follows, so the two paths answer this the
same way (`every_composed_cell_write_names_the_view_the_client_is_on` and
`socket::tests::a_mouse_commit_names_the_view_its_editor_opened_on`). A rename
names no view: its record holds cells, so the host finds it in the store
(`context/code/protocol.md`).
**A captured commit is not judged by the cursor's cell either (improvise-3h9w).**
#286's `refused_here` declines a commit key whose cursor cell `i` would decline,
and waits on one that is unprojected. It runs in `step()` before `commit()`,
so it must skip an editor that captured its target: that commit writes, or
basis-refuses, the captured cell, whatever now sits under the cursor. The guard
is kept for an editor that captured nothing, which is the case it was written
for.
**What the commit is follows the capture too.** Whether `Enter` writes a value
or renames a coordinate is read off `opened.target()`, not off the cell under
the cursor (ChatGPT Codex on PR #274): a relayed reframe can leave the cursor on
no cell, on a creating column, or on a records grid, and dispatching on that
sent a captured value editor's `Enter` upstream or down the wrong path. Only an
editor that captured nothing (`uncaptured`) is dispatched by the cursor, and
that is the creating columns plus the waiting cases. A records grid's measure
cell is deliberately not captured as a value, since the browser still declines
editing it (gotcha 1); `every_advertised_command_answers_in_its_own_mode`
fails if it is.
Two things come with it. **The version is the one read at open**, so a write by
another client while the editor is open is refused `RecordMoved` on this path
as well: improvise-0j0f's rule, which the keyboard path had not had
(improvise-xz2z). And **a
value commit can now wait**: the capture waits for the host's answer to this
client's own last write, so `Enter` pressed again inside one round trip has no
target yet. It declines with the rename's "still waiting" status and writes
nothing. With no write in flight and no capture, the cursor is on no cell an
editor writes, and the commit answers `None` as before.
**A waiting commit is a decline, so it keeps the typed text (improvise-joed).**
`Enter` in `Editing` is the sequence `[commit-cell-edit, clear-buffer edit]`,
and a sequence runs every step unless one is `Step::Declined`. The waiting
answer used to come back from `composed` as an ordinary `Local` step, so the
welded clear ran after it and emptied the editor it had just left open. The two
cell commits are therefore answered by `committed`, ahead of `composed`, and
`still_waiting` is a `Step::Declined`. That is the same channel `declined` uses
for commands the browser cannot compose, and the same rule gotcha 21 gives an
abort: what follows a refusal in one binding does not run. #251's other
decline, a retry whose cell is `Opened::Gone`, keeps its text the same way:
`renamed_away()` is a `Step::Declined` too, and it serves both the value and the
rename path.
This also fits improvise-fy9o (PR #227), which drops the weld and has the
commit clear its own buffer at its head. There the binding is one command, so
`Declined` and `Local` answer the same, and the text survives only if the
waiting return comes **before** that clear. A merge of the two must keep that
order. The joed tests fail if it does not.
+612 -27
View File
@@ -368,6 +368,36 @@ Only `CellData` variants that carry a `RecordCell` name a record. The item and
formula commands are `CellData` too but name no *record*: they move the axes a
record's coordinates are drawn from, which is what the layout tier is for.
### `records_at_stake` — `records_named` plus a rename's destination
`ui/reduce.rs::records_at_stake(app, cmd)` is what `stale_basis`,
`stale_records_in_a_frame` and `caught_up` ask. It is `records_named(cmd)`, plus
one extra record for `RenameRecordCoordinate`: the **destination**, meaning the
source's coordinates with the category replaced by the new item. That record is
added only when it exists in the document and differs from the source
(improvise-milr). A rename onto an existing record overwrites it
(improvise-kndq), so without this a write another client made to the
destination was lost without any check.
**Why not in `records_named` itself,** which is what the ticket asked for.
`records_named` is pure and lives in `improvise-protocol`: `&ModelCommand ->
Vec<RecordHash>`, with no document. The destination is not in the command.
Deriving it needs the source's coordinates, which only the document holds,
and "when it exists" is also a question about the document. Moving
`records_named` behind an `&App` would take away what makes it useful to the
client. `BasisCache::claim` and `caught_up` rely on it being pure, so a client
can claim exactly what the server will check. So the protocol keeps the
command's own records, and the server adds what only the document can say.
The price is on the client side. It cannot compute the destination's hash (ADR
002: a client never computes a surrogate), so it claims every record counter it
held when the editor opened (`context/code/clients.md`, "A coordinate rename").
An existing destination the client was never shown is refused with
`RecordUnstated`. That is a refusal, not an overwrite, and it is the right
direction to err. `broken_chain` still asks `records_named`: whether a later
command's record was renamed away by an earlier one in the frame is a question
about the source only.
### `stale_basis`
`None` means apply it. The three tiers are checked in order of how much they
@@ -609,6 +639,47 @@ the client was told nothing at all.
`AddItem` case the same way and found none, so `AddItem` does not stop either.
An abort is added where a later effect depends on the failed one, never for
symmetry.
- **The `|| last_batch_aborted()` arm is kept, and the invariant it guards is
pinned (improvise-we1a).** Every model effect that aborts today also refuses,
so deleting the arm left the workspace green: nothing tested it. There were
two ways to close that. One was to make an abort without a refusal
impossible to write. The other, taken here, was to test the invariant with
the arm left in place. `every_failing_model_effect_that_aborts_also_refuses`
applies each failing `ModelCommand`'s own effects to an `App`, bypassing
`Host`, and asserts it refused. It also pins which cases abort, so it cannot
pass on a table that aborts nothing. The wire table cannot hold this
invariant, because the arm itself rejects a frame that aborted without
refusing. With AddCategory's arm reverted to status-only, the wire table
stays green. The option not taken is right for today's rules and wrong for
the future:
- `AbortChain` aborts without refusing, correctly. It is `ViewOnly`, and it
stops a chain whose premise no longer holds, which is not an error. A
type-level rule of "every abort refuses" would first have to exempt it.
- Making it refuse would change which `:` lines the headless CLI reports as
failures.
- Keeping the raw `abort_effects` setter away from `Model` effects alone would
be a new visibility split, and today no effect needs it.
So the arm stays, as defence for a future `Model` effect that aborts on
something other than an error.
**The table alone did not hold the invariant (improvise-lasd).** It lists
failing commands, so an abort at a site it does not list got past it. With
`on_view`'s missing-view branch changed to abort without refusing, the
workspace stayed green, and only the arm caught it. So `App::apply_effects`
now carries the rule itself. When the abort flag is set after a `Model`
effect, a `debug_assert!` requires that the batch has refused. `ViewOnly`
effects are exempt, so `AbortChain` is untouched. That answers the objection
above: the rule is on `EffectKind`, not a new visibility split.
- **Coverage:** it covers every model effect that any test drives to an
abort, listed or not. Under the `on_view` rival it fails two existing
tests.
- **Where it stops:** it is a debug assertion, so release builds rely on the
arm. It reads the batch-wide `batch_refused`, so a model effect that aborts
without refusing after an earlier effect in the same batch has refused
still gets past it.
- **Why the table stays:** it drives every failing command, and so it
exercises the assertion.
In a client's frame, any refusal becomes `Rejected(MutationRefused)`. The frame
is undone whole, and the client is told, which is the channel the narrated
@@ -637,12 +708,13 @@ Consequences:
suffices. So each `refuse_and_stop` site needs a key-path assertion for its
refusal as well as one for its abort. The abort is
`a_failed_load_or_import_does_not_run_the_quit_after_it` (improvise-514o).
Three sites are not in the key-path table (improvise-rtud). The quit test's
"setup:" assertion holds the refusal of `load` and `import-json absent.json`.
Every `refuse_and_stop` site now has a row in
`every_refusal_site_refuses_its_failure_and_only_its_failure`. The last three
were added by improvise-we1a: `load`, `import-json absent.json`, and
`AddFormula`'s parse arm. Before that, only indirect assertions held them.
The quit test's "setup:" assertion held `load` and `import-json absent.json`.
`a_formula_that_does_not_parse_keeps_the_user_in_the_editor_with_their_text`
holds `AddFormula`'s parse arm, which has no case in the quit test. The other
five import arms are in
`every_refusal_site_refuses_its_failure_and_only_its_failure`.
held the parse arm (improvise-rtud). Those assertions remain.
Each case in that table has a succeeding twin, so a site that refused
unconditionally would fail too. That matters because these are exactly the
lines a merge conflict could silently drop.
@@ -678,6 +750,15 @@ The effect itself never inspects `app.view_state.mode`: the mode is decided
statically by whoever invoked it. An earlier version branched on
`app.view_state.mode.is_records()` and got it wrong mid-batch.
It opens nothing on a cell `i` declines, and says why (improvise-enz6). The
question is `cmd::core::uneditable`, the one `committable_cell_key` asks. A
fold or another aggregate, or a formula's result, gets its refusal text in
the status line. The mode stays where the batch put it. `CommitAndAdvance`
already dropped to normal at its head (gotcha 21), so the user lands on the
fold in normal mode. It is a status line and not `App::refuse`, because the
commit before it succeeded. A `commit-cell-edit` script line whose advance lands on a
fold must not report its write as refused.
### Which effects reset the cursor, and why it is decided one at a time
Swapping the axes renumbers every row and column, so the old indices name
@@ -1014,6 +1095,80 @@ single edit the output is the effects the plain path used to emit, reordered fro
One behaviour moved on the drill side: it used to discard `add_item`'s error, and
now goes through `effect::AddItem`, which refuses when the category has gone.
**A row naming a category the model no longer has is unwritable, and one
unwritable row makes the whole drill commit write nothing** (improvise-qz9p).
Another client's `RemoveCategory` can land while a drill is open. That removes the
category and every cell under it, and the drill's staged edits then address
coordinates nothing owns. The fix has two parts.
- **The check is structural, not a reaction to `AddItem` refusing.** The ticket
framed it as "should a failed `AddItem` drop the row's writes", but that misses
the plainer case. A **measure edit alone** on such a row stages no `AddItem`, so
nothing refuses, and it still writes an orphan cell keyed under the removed
category. `RecordRewrite::of` therefore asks the model whether every category
the row names still exists, both its coordinates and its renames (a sparse
row with no coordinate for a category it renames reaches the check only
through the renames; improvise-6r2y). If any is
gone, the row is `unwritable`: it refuses with `Unknown category '…'` and
carries no writes. This is ADR 001's "force never creates structure", applied
inside the drill.
- **Why the whole commit and not just that row.** The drill's contract is atomic
(`design-principles.md` §4: "`ApplyAndClearDrill` writes them all atomically …
No partial writes reach the model"). A per-row skip would commit the other
rows' edits and drop this one's, a partial write of exactly the kind that
contract rules out. So when any row is unwritable, `merged` writes nothing.
- **What the user is told (improvise-o9q7).** `merged` first kept every row's
refusals. Refusals are applied in row order and each one overwrites the
status line, so a later writable row's own refusal, such as a blank
coordinate, replaced the one that mattered. The user read "Record coordinates
cannot be empty" when in fact every staged edit had been thrown away. So when
any row is unwritable, `merged` keeps only the unwritable rows' reasons and
combines them into one refusal that says so:
`Unknown category 'Type': every staged edit was discarded`. Sibling refusals
are dropped because nothing those rows staged is written, so their reasons
describe edits that are no longer pending.
- **`refuse`, not `refuse_and_stop`.** This follows #181's rule (gotcha 21):
abort only where a later effect in the batch assumes the failed one happened.
The effect after `ApplyAndClearDrill` in `view-back`'s batch is `ViewBack`,
which assumes nothing about the writes. Stopping would also keep the user in a
drill they cannot fix, because the category cannot come back through it, so
every exit would fail again. The staged edits are discarded, as on cancel, and
the refusal says why.
A blank coordinate edit is a different kind of refusal. It is refused **alone**,
and the row's other edits still land, as before: that edit is invalid input, not
a missing structure.
**The wire half: `effect::RenameRecordCoordinate` (improvise-fdk1).** One effect
serves both doors. `model_effects` maps `ModelCommand::RenameRecordCoordinate` to
it, after resolving the surrogate. The plain records key path emits it for a
category column. It reads the record's row off the store at apply time
(`view::record_row`, which groups the same way the records layout does) and
applies `RecordRewrite::of` with the one edit. The key path used to call
`RecordRewrite::of` itself, with the layout's row. In plain records mode that
row is the store's, so behaviour is unchanged. What changes is that the key path
and the wire now take one door, so a host and a browser renaming the same record
cannot drift. The name matches the variant, as
`every_model_command_names_the_effect_of_the_same_name` requires. The collision
rule is kept (improvise-kndq): a rename onto an existing record still
overwrites it, on both paths.
**What `[RenameRecordCoordinate(r), SetCell(r)]` means depends on the view the
write names, and that is filed, not fixed (improvise-8id2).** A surrogate names
coordinates, not an object. After the rename, `r` names the vacated coordinates.
This paragraph first said "depends on the host's view". Since improvise-wpxr the
write names its own view and is resolved there, so the host's view no longer
enters into it. A write naming a **records** view finds no fallback, because a
records layout names positions. `broken_chain` answers `ChainBroken(r)`, and
clause 4 undoes the rename too. That case is pinned by
`a_write_to_a_record_its_own_frame_renamed_is_a_broken_chain`, whose write names
the records view the rename was composed in. A write naming a **pivot** view
that still draws the vacated cell resolves `r` there and re-creates a record at
the old coordinates: measured, not pinned. The rename itself resolves in the
store only (`NamedRecord::Stored`), so no view affects its source. The shipped
browser composes one rename per frame, so only a hand-built client sends this
pair. It is the same fallback as improvise-ciw8.
### `Save` / `SaveAs` abort the batch (improvise-8tk / improvise-lae)
A failed save aborts the rest of the batch, so the `Quit` that `wq`/`ZZ` chains
@@ -1254,6 +1409,10 @@ this order, and the order is the point.
exactly the records that have no data to index. It is bounded by the grid rather
than the viewport, so an off-screen empty cell resolves too, and a layout change
correctly makes a surrogate composed against the old shape unresolvable.
**Which** layout is the view the command names, not the host's
(improvise-wpxr): see "An empty record resolves against the view that drew
it" below. So a `RecordHash` is document-scoped only while its record holds
a cell. An empty one is layout-scoped, like an aggregate.
`resolve_aggregate` goes against the layout and *only* the layout — not because
that is cheaper but because an aggregate has no existence outside the layout that
@@ -1753,6 +1912,140 @@ frames at 60fps, and the remaining cost is one genuine evaluation per cell. The
next lever is caching *across* frames, which needs an invalidation story the
`record_versions`/`layout_version` tiers could plausibly supply.
**The totals left the walk (improvise-wm20).** A column's total used to be the
walk's running sum of its cells, which is wrong for everything except a stored
measure under Sum or Count. It gave a ratio column the sum of its rows' ratios,
an Avg measure the sum of its rows' means, and a text column `0.00`. The user's
rule is that **a column's total is its cell as if every drawn row were one
fold**. `GridLayout::column_total` asks `Model::evaluate_spanning` for the
column key with the row categories spanned, so a formula is evaluated over the
pooled rows (Margin = total Profit / total Revenue) and a stored measure folds
its records under its own aggregator. The user ruled on three cases:
- **Text** follows the unanimity rule (gotcha 7): the text if every row agrees,
otherwise the `mixed` error.
- **An error** in any row makes the total an error, which is what that rule
answers for an error among numbers. That holds for a **stored** error. A
formula column is evaluated over the pooled inputs, so a row's `div/0` is not
an input and does not propagate. Only the fold's own error does. Rulings (1)
and (2) disagree here: the column shows an error in one row and a clean
number under it. The user chose the **pooled ratio** (improvise-b5dp). Margin
over rows 25/30, 27/30 and -4/0 totals 48/60 = 0.80, not `div/0`.
- **With `_Measure` on the row axis** the strip is empty, because a column adds
different measures together. An empty strip already means "no totals"
(gotcha 33).
The coverage is the **drawn** rows, as a fold's is (improvise-f2oo): a hidden
row stays out, and so does a record that lacks a row coordinate. That
deliberately differs from `Axis::None`, which counts both. The browser and the
TUI agree by construction, since both draw the projected `GridCache.totals`.
The cost is one more evaluation per column, and it is not free. The same
`cargo bench --bench grid_projection` on the orchestrator's Mac (Apple M4 Max),
before and after:
| case | whole grid before | after | `col_widths` before | after |
|---|---|---|---|---|
| typical (400) | 0.86 ms | 1.24 ms | 0.53 ms | 0.92 ms |
| 18k | 114 ms | 177 ms | 75 ms | 142 ms |
| pathological (90,000) | 1.54 s | 3.49 s | 1.07 s | 1.96 s |
`col_widths` is the keypress path, and it pays too, because a total's text sizes
its column. The event loop draws once per key, so every key pays the fold twice:
once in `App::cmd_context` for `visible_cols`, and once in the draw's `grid_of`.
At 4.8k cells that took a held arrow key from about 31 ms to about 52 ms, past
the ~33 ms key-repeat interval (improvise-yb4b). The walk no longer carries a
sum, so the window-into-totals fault below cannot happen any more. The test
still guards the widths half.
**The fold scans instead of enumerating (improvise-yb4b).** The cost was not the
fold's arithmetic. It was *how the fold found its records*: `fold_stored` built
one `CellKey` per member, the cross product of every spanned category's covered
items (3,000 keys for an 18k column), and looked each one up. Now
`group_expansion` returns the expansion as data: a `base` key and the covered
items of each expanded category. `fold_stored` hands that to
`DataStore::spanned_values`, which narrows through the index and filters each
candidate by membership. In `Exact` mode a candidate must also carry no
coordinate beyond those, which is what `get(member)` answered. Only a formula
still enumerates members (`contexts_of`), since its references are evaluated
per context.
**How it narrows is a choice, and the first version got it wrong
(improvise-jq0m).** Narrowing by `base` alone is right for a column total, which
spans *every* item of each row category. It is wrong for a collapsed group,
which spans two or seven items of a category with hundreds: `base` drops the
grouped category entirely, so every fold cell scanned every record of the
column, and a grid of small groups went quadratic. The review measured 1000
days in pairs at 11–15.7 s a keypress, against 0.44–0.51 s before #239. So
`spanned_positions` now prices both narrowings off the index's own set sizes
and takes the cheaper: once by `base`, or once per covered item of the span
with the fewest items, each intersected with `base`. The per-item positions are
disjoint (a record has one item per category) and are sorted afterwards, so
both paths return the store's order. A `base` naming a name the store never
interned matches nothing on both paths.
The bench now carries the grouped cases, which it could not see before: Day in
collapsed groups on Row, with Product, and `_Measure` on Column. The machine is
an Apple M4 Max (16 cores, 64 GB) at load average 15–63. The figures are
keypress (`whole-grid + col_widths`), min-of-N, from two alternating rounds,
the second reversed. Each cell gives both rounds:
| case | before #239 (`5932ad74`) | `f668ddc9` | jq0m | jq0m ÷ before |
|---|---|---|---|---|
| 4.8k dense | 33.9 / 39.1 ms | 40.7 / 40.2 ms | 36.3 / 33.3 ms | 1.07 / 0.85 |
| 18k dense | 237 / 239 ms | 238 / 232 ms | 196 / 211 ms | 0.83 / 0.88 |
| 90k dense | 3.04 / 3.46 s | 3.24 / 2.70 s | 2.68 / 2.84 s | 0.88 / 0.82 |
| 364 days in weeks | 483 / 415 ms | 1.42 / 1.18 s | 356 / 399 ms | 0.74 / 0.96 |
| 364 days in pairs | 715 / 539 ms | 5.02 / 4.62 s | 471 / 518 ms | 0.66 / 0.96 |
| 1000 days in pairs | 406 / 399 ms | 10.8 / 20.2 s | 352 / 364 ms | 0.87 / 0.91 |
The grouped cases now come in **below** the pre-#239 figures. The old
per-member path looked up each member through a full `CellKey`, and the
per-item narrowing intersects index sets directly.
Two levers were **not** taken. Combining the walk's row values for decomposable
aggregators was rejected under §5 as a second derivation of the total. It is
also wrong wherever a formula owns a row-axis item: the row reads the formula,
while the spanning fold reads the store. Sharing one total between
`cmd_context` and the draw was also rejected: a key's effects run between the
two, so sharing needs an invalidation story. Dropping the totals from the widths
would make the two width computations disagree about `visible_cols`.
**A grid-wide fold memo (improvise-h6z1)** is the other half, and it answers a
different cost. It removes the repeated **formula** chains that a fold row and a
total re-derive, about 2x on a folded five-formula chain. It does **not** move
the table below. Those grids hold only stored measures, a stored fold is never
memoised, and each total reads its column's records once. That one read per
column is what the scan above made cheap.
**What it changes besides speed:** a stored fold's values now arrive in the
store's insertion order, not member order. That is the order gotcha 18 names as
the store's answer. Min, Max, Count and text unanimity are order-free. A Sum or
Avg over non-integral values may differ in the last bits.
Measured by alternating three builds of `cargo bench --bench grid_projection` on
an Apple M4 Max (16 cores, 64 GB) under heavy shared load (load average 30–90).
This is min-of-N, three rounds, with the order reversed in the third. The
keypress figure is `whole-grid + col_widths`. The millisecond columns are
ranges over the rounds. The ratio columns are each round's own pair, in round
order:
| cells | before #239 (`5932ad74`) | #239 (`9948aaf5`) | yb4b | #239 ÷ before | yb4b ÷ before |
|---|---|---|---|---|---|
| 400 | 1.36–1.77 ms | 2.36–2.76 ms | 1.69–1.82 ms | 2.03, 1.34, 1.78 | 1.24, 0.98, 1.25 |
| 1.2k | 4.1–5.2 ms | 7.2–8.2 ms | 5.0–5.3 ms | 2.00, 1.39, 1.85 | 1.23, 0.96, 1.25 |
| 2.4k | 10.8–13.6 ms | 18.4–21.5 ms | 12.0–13.9 ms | 1.98, 1.36, 1.91 | 1.18, 0.88, 1.29 |
| 4.8k | 32.8–41.8 ms | 55.4–64.1 ms | 34.5–42.0 ms | 1.77, 1.32, 1.92 | 1.15, 0.82, 1.26 |
| 9.6k | 102.8–121.2 ms | 190.7–209.6 ms | 107.2–140.0 ms | 2.04, 1.57, 1.92 | 1.36, 0.88, 1.20 |
| 18k | 192.9–208.3 ms | 338.6–448.0 ms | 230.4–273.3 ms | 2.32, 1.63, 1.72 | 1.42, 1.11, 1.23 |
| 90k | 2.68–3.26 s | 4.92–6.49 s | 3.18–4.00 s | 1.99, 1.84, 1.79 | 1.23, 1.47, 1.14 |
Read the ratio columns, not the milliseconds. Round 2's `before` ran at load
86 against about 31 for its `yb4b`, which is why its ratios fall below 1. The
two clean rounds put #239 at about 1.8–2.3× the pre-#239 keypress, and put
yb4b at about 1.15–1.4×. At 4.8k a held key is back under the ~33 ms repeat
interval in the quietest round (34.5 ms against 32.8 ms before).
The guard is `a_window_narrows_the_cells_and_neither_the_totals_nor_the_widths`:
with one walk serving three consumers, a window that leaked into the other two
would silently make a column total the sum of the visible rows, or size a column
@@ -1873,11 +2166,14 @@ Three edge rules, and every one of them is about a group header, because a heade
occupies a row without consuming a data index (gotcha 1's cousin: the data index
counts items, the visual position counts entries).
- **One header above the window comes with it.** A column window opening inside
`Q1` must still be able to say the column is in `Q1` (`AxisFrame::group_of`), so
the nearest header above the first in-window item is carried — and only that one,
or the payload grows with the groups a deep scroll passed. `AxisFrame::body_from`
then declines to *paint* it, which is the cut `GridLayout::data_row_to_visual`
- **The open run at each level above the window comes with it.** A column window
opening inside `Q1` must still be able to say the column is in `Q1`
(`AxisFrame::group_of`), so the header of every run still open above the first
in-window item is carried: one per level, never the closed ones, or the payload
grows with the groups a deep scroll passed. It used to be the single nearest
header, which lost `Upper` for a window opening inside `North/Q1`
(improvise-e265, `a_window_opening_inside_a_nested_run_carries_every_level`).
`AxisFrame::body_from` then declines to *paint* them, which is the cut `GridLayout::data_row_to_visual`
always made. In the loop, `entries.is_empty()` is true exactly until the first
in-window item lands, which is what makes the compaction correct.
- **Consecutive headers *inside* the window all come.**
@@ -1886,7 +2182,7 @@ counts items, the visual position counts entries).
one would lose the only row saying those items exist. (Seen, not yet acted on, in
a browser: `z` is `Upstream` and `web/app.js` puts no listener on a group
header.) **Above** the window it is the other way round: nothing there is
painted, so only the nearest is kept.
painted, so only the open run at each level is kept.
- **A trailing header comes only when the window reaches the last item.** A
collapsed group at the bottom of the grid has no items to hang off and is still a
row the user must see; but a window scrolled to the middle must not be handed the
@@ -2306,6 +2602,22 @@ reframes and then *switches views* before placing would reset the view the
client ends on rather than the one it reframed. No resolver composes that frame
today.
The echo has two limits, and each is asserted because each has a plausible
wrong neighbour (improvise-hv0r). **It goes to the asker alone.** A bystander on
the reframed view is owed the reset, but never the asker's placement, since
that would be another client moving where I am looking. **It covers every
placing kind**, not only `SetSelected`. A records `+` commit carries an offset
after its mutation, so an echo narrowed to the selection would leave the
client's offset reset and its mirror's set.
`a_reframing_frame_echoes_only_its_later_placements_and_only_to_the_asker`
asserts the exact relay list for each session. The wdmp test above cannot see
either rival.
The echo also needs the client to *keep* what it hears. A relay is applied to
the mirror at once, so the client's own rollback must not undo it: see
`context/code/clients.md`, "What the host *relays* is settled too"
(improvise-jpba).
What is *not* broadcast is a position: another client's transpose must not move
where I am looking — that is UI and it is mine (improvise-g0f). It may only declare
that where I was looking has stopped meaning anything, which is a fact about the
@@ -3002,11 +3314,51 @@ for three reasons, each checked rather than argued:
`renders_each_tile_with_its_axis_label` fail, as well as the two characterization
tests here.
What the default cursor does cost, stated: one cycle key on tile 0 puts `_Index`
on Row alone, and in a pivot view that empties the grid (0 rows on
`web/demo.improv`). The status line says `_Index → Row`, and three more presses
undo it. If that turns out to matter, the fix is the order the tile bar lists
categories in, not taking entries out of it.
**Where the tile cursor opens (improvise-oplp, improvise-bqm0).** It used to
open on tile 0, which is `_Index`, so one cycle key (`T Enter`, read as
"confirm", or `T r`) put `_Index` alone on Row and emptied a pivot grid (0 rows
on `web/demo.improv`). The browser worker saves every accepted model command, so
there the empty grid became the stored document at once.
The fix places the cursor and keeps the order. Both ends call
`improvise_protocol::opening_tile`, which opens on the best-ranked tile (the
leftmost on a tie): a Regular category first, then `_Measure`, then a Label, and
`_Index`/`_Dim` last. **Placing the cursor beat reordering the tiles** because
the order is itself a decided, tested output. The browser's click targets are
tile *indices*, so reordering would move every click position. And
`the_tile_bar_offers_every_category_by_name_measure_and_records_axes_included`
pins the offered list, which is the characterization #192 made. Moving the cursor
changes only where one keypress lands.
**The ranking, and why it is an enum and not a bool.** `TileEntry` carries the
host's `CategoryKind`, because a client cannot tell `_Index` from a user category
without sniffing names (design-principles §2). #231 first carried
`regular: bool`, which answers only one question. It could not say what to open
on when a view has *no* Regular category, and so fell back to tile 0, `_Index`,
which is the original bug (improvise-bqm0, confirmed on a document holding only
`_Measure` data). The rank is an exhaustive `match`, so a new kind forces a
decision. `_Measure` is the fallback, not the primary: it is on an axis in every
shipped document, and `Enter` reads as "confirm" on the user's own dimension. It
ranks above a Label because a Label is a high-cardinality per-row field, and
cycling it onto Row would explode the grid rather than empty it. The fallback is
**not** always `_Measure` (improvise-h3cv). In the `_Drill` view `drill_steps`
puts every coordinate of the drilled key on `Axis::Filter`, `_Measure` included,
and `tileable_cats` drops Filter, so `_Measure` is not a tile there. A drill's
tiles are `_Index`, `_Dim` and whatever sits on None, so the Label rank and the
`_Index` fallback are live exactly in a drill: a Label opens if there is one,
and otherwise `_Index`, because nothing better is on offer.
`in_a_drill_tile_select_opens_on_a_label_not_on_index` and
`in_a_drill_with_nothing_better_tile_select_opens_on_index` drive both through
the TUI's own keys. They are not lockstep tests, because a browser declines `>`.
**The tests had to separate the rule from its neighbours** (improvise-52od). In
every early fixture the first Regular tile was also the first tile on Row, so a
rival `position(axis == Row)` passed the whole workspace. Records mode is where
they part: `_Index` holds Row there, and the rival cycles it, dropping the user
out of records mode. `tile_select_opens_on_a_regular_category_even_when_a_virtual_holds_row`
and `in_records_mode_tile_select_opens_on_a_regular_category_on_both_ends` pin
that; the evidence is in `context/test-evidence/protocol.md` and
`clients.md`.
**`_Measure` must stay tileable.** It is on an axis in every shipped document, so
`regular_category_names()`, the advice for *prompts*, is the wrong filter here.
@@ -3417,12 +3769,17 @@ the one writer that must see the synthetic key: renaming a coordinate column,
creating a column and staging a drill edit all resolve the records position
inside `commit_cell_value`. Its door refuses the two arms `i` refuses, an
aggregate (a fold included) and a formula's result. It needed one because
`Enter` and `Tab` advance the editor, and `EnterEditAtCursor` re-opens it at the
`Enter` and `Tab` advance the editor, and `EnterEditAtCursor` re-opened it at the
new cursor with no guard. `i` declines to edit `Profit`, yet `Tab` from `Cost`
landed the editor on it, and the next commit stored a value under the formula.
That is a fourth door nobody had listed. **The entry half is still open**
(improvise-enz6): the editor still opens on a fold or a formula's cell. Only the
write is refused, out loud, at commit. Closing the entry belongs in `ui/effect.rs`.
That is a fourth door nobody had listed. **The entry half is closed too**
(improvise-enz6). The door's two arms are now one function,
`core::uneditable`, and `EnterEditAtCursor` asks it before opening. So the
re-opened editor and the commit that would follow it cannot disagree about which
cells they decline. The commit door stays: a `:commit-cell-edit` line runs it at
the cursor with no editor opened there. Since the advance no longer reaches it,
the door has tests of its own that seat the editor directly
(`an_editor_already_open_on_a_fold_commits_nothing_there` and its formula twin).
**Drill keeps the narrowest door** because drilling into an aggregate is its
purpose. Only a fold is refused, since no record carries a group's name. The
@@ -3591,7 +3948,7 @@ and any formula that still mentions the old one (`context/code/core.md` §18).
## A wire write reaches only a cell the host projects as a Record (improvise-ciw8)
`App::resolve_record` answers "which record does this surrogate name". Its
fallback, `record_in_layout`, accepts any key the current layout names, and that
fallback, `record_in`, accepts any key the named view's layout draws, and that
includes a collapsed group's fold and a key the None axis aggregates over. It
has to stay that permissive: it is also the basis check's question, and a record
nobody has stored yet (the empty cell a client is about to fill) is found only
@@ -3607,10 +3964,32 @@ the refusal marks the batch, `Host::mutating_frame` undoes the frame, the asker
gets `Rejected(MutationRefused)`, and the reason is on the host's status line.
The question it asks is the projection's own, never a second derivation of it:
- A coordinate that names a **group** rather than an item is refused, in any
view (improvise-b114). A group has no cell of its own, so writing one stores a
non-item coordinate. This rule asks the document, not the layout, and runs
first, because a fold in the measure half hides from both rules below. Take
- A coordinate that names **no item of its category** is refused, in any view
(improvise-b114, then improvise-nacd). This rule asks the document, not the
layout, and runs first, because a fold in the measure half hides from both
rules below.
**It is stated positively, and that is the point of the nacd change.** The
b114 rule was negative: refuse a name that *is a group*. The invariant it
protects (improvise-gw18) is positive: every coordinate names an item. A
negative rule covers only the ways of not being an item that someone listed,
and review found two more within a day. `Bogus` is no item and no group. A
parent group such as `Totals`, above `Money`, holds no item directly, so
`items_in_group("Totals")` is empty and the group test missed it. Both wrote
`{Month=Feb, _Measure=…}` onto a stored record and were accepted. The
positive rule covers both, and any shape not yet thought of, because it asks
whether the name *is* an item rather than what else it might be.
"Item" means `Model::effective_item_names`: the category's items, plus the
formula targets for `_Measure`. The formula targets are included so that a
write under a formula passes this rule and is refused by the next one with
the reason that is actually true ("computed by a formula"), not as a non-item.
The same-name case falls out: an item called `Money` beside a group called
`Money` is an item, and is writable (improvise-pr8t). The b114 rule needed an
explicit exemption for that case, and nothing tested the exemption.
A group has no cell of its own, so writing one stores a non-item
coordinate. Take
`{Month=Feb, _Measure=Money}`, where `Money` groups Revenue and Cost. Its
record `{Month=Feb}` is stored, so the second rule below accepts it and never
asks the projection. And with `Money` expanded the host draws no such cell at
@@ -3632,8 +4011,64 @@ The question it asks is the projection's own, never a second derivation of it:
so every column of a row carries the same record, and with a collapsed
`_Measure` group the first of them can be the fold. That refused the empty
`Units` cell beside a `Money` fold, which the host itself projects as a
`Record`. A write whose whole key the host's grid does not draw is not
refused here, as it was not before this rule existed.
`Record`. A write whose whole key that grid does not draw is not refused
here, as it was not before this rule existed. "That grid" is the layout of
the view the command names, the same one resolution used (improvise-wpxr).
**The projection asks the first rule too (improvise-gk51).** The non-item rule
asked the document, while `project::cell_id` did not ask at all. In records mode
every measure cell of a stored record was a `Record`, including a record whose
`Region=Mars` names no item. `set-cell`, an undeclared `## Data` item and a drill
commit after an item was removed all make such records. So the host offered a
write it then refused. The two ends now share one predicate,
`Model::first_non_item`: `App::non_item_refusal` refuses with it, and
`cell_id` declines with it, in both branches. The pivot branch needs it as well.
A page selection naming no item (`set_page_selection` does not validate) puts
the stale item into every cell's key, and that too was a `Record` the host
refused.
Such a cell projects as a new kind, **`CellId::NonItem { category, item }`**,
and not as one of the existing ones. Each existing kind would lie about it:
- `Aggregate` invites a drill, and its refusal says the cell "combines several
records". This is one stored cell.
- `Coordinate` tells the user that renaming is not built yet.
- `Derived` names a formula that does not exist.
- `None` means "no cell" (`protocol.md` §3), and this cell holds a value.
The variant carries the offending coordinate, so every decline says the same
sentence: `improvise_protocol::non_item_refusal`, which the host's refusal uses
too. The two front ends agree because the TUI's `value_cell_key` and
`committable_cell_key` also read `cursor_cell_id`: `p`, `x` and a commit on
such a cell are refused with that sentence in the terminal as well. A
`NonItem` publishes no record version, since no basis can claim a write it
could never make.
**`i` refuses at open, as the browser does (improvise-yc9r).** `EditOrDrill`
first asked only about folds, aggregates and formulas. So `i` opened an editor
on a `NonItem` cell, and the refusal came at commit. There, the minibuffer
draws in place of the status line (gotcha 21), so the user saw nothing happen.
`EditOrDrill` now asks `cursor_cell_id` after the formula check, which is the
order `cell_id` asks in. The commit guard stays, and it is still reachable:
`:enter-mode editing` opens an editor without passing through `EditOrDrill`.
Each of the four declines has its own test. Before these, three rivals passed
the whole workspace: writable in `value_cell_key`, a `Record` in
`committable_cell_key`, and `None` in the browser's open-editor arm.
**Membership is a lookup, not a list (improvise-4daa).** `first_non_item` runs
for every coordinate of every projected cell, and the TUI projects the whole
grid every frame (gotcha 44). It used to build `effective_item_names` (a fresh
`Vec<String>` of every item) for each coordinate. That made `grid_of` quadratic
in a label category: 145 ms against 1.9 ms for 2000 records. `Model::is_item`
answers the same question with `contains_key`, plus the formula targets for a
measure category, which is where `effective_item_names` differs from the
category's own items. `is_item_agrees_with_effective_item_names_everywhere`
holds the two answers equal.
Option (b) on the bead, closing the sources, is not done here. It is
improvise-98pd. With (a) in place, a source that still writes a non-item
produces a cell that is declined everywhere, not one that is offered and then
refused.
**Why the `vec![]` arm stays.** A reviewer asked whether a `SetCell` whose cell
does not resolve at all, which `model_effects` turns into `vec![]` and so
@@ -3649,3 +4084,153 @@ the arm is unreachable from the wire. Making it a refusal would add a second
answer to "does this resolve" beside `broken_chain`, and that answer could only
ever fire after the first had already refused. It is left as the mutation
path's own behaviour for callers that do not go through the session door.
## An empty record resolves against the view that drew it (improvise-wpxr)
`SetCell` and `ClearCell` carry `view: String`, the view whose grid drew the
cell. `App::resolve_record(view, hash)` still asks the store first. It asks the
layout only for a record that holds no cell, and that layout is now the named
view's, not the host's.
**The bug.** `resolve_record`'s layout fallback read `self.layout`, the host's
active view. Since improvise-jytm that need not be the client's view. A client
on `Default`, where South's empty cell is drawn, had its write refused
`RecordGone` whenever the host sat on a view that hides South. With the host on
`Default` the same write was accepted. So gotcha 44's "a `RecordHash` is
document-scoped" held only for a record with a stored cell. An empty record
exists only as a position in some layout. That makes it layout-scoped, like an
aggregate, and it is the record-shaped twin of improvise-obtu.
**Two candidate fixes, and why the view goes on the command.**
1. *The surrogate carries its coordinates*, so it resolves with no layout.
Rejected. It is the alternative improvise-obtu rules out for aggregates, and
for the same reason: ADR 002 exists so that a client holds an opaque token
and never the document's coordinate space. The ticket's variant, "resolve an
empty record from the document's categories", needs no client change but
cannot work either. A hash is one-way, so it would have to enumerate
candidate keys. Which categories a key carries depends on the view (a
None-axis category is absent from the key), so the candidates are every
subset of categories crossed with their items.
2. *Resolve against a layout derived for the client's view*, as `grid_in`
already does to project it. Taken. The remaining question is how the host
learns the view, and the answer is improvise-obtu's: **the command names
it**, as every view-scoped `ModelCommand` already does (improvise-lltt). It
is not read off the asking session's mirror, for three reasons. `Checked` is
answered with an `&App` and no session, and obtu's argument is that the
mutation is unaddressed and only its operand is view-scoped. A frame is
checked whole before any of it is applied, so a `SwitchView` earlier in the
same frame would leave the mirror on the wrong view for a later write. And
the view the client *composed against* is the fact that matters, which is
what the command states.
A third option was considered and not taken: resolve against *any* view in the
document. It needs no wire change, but it answers a question nobody asked, and
it leaves writability ambiguous. `unwritable` asks the projection at the cell's
position, and a record can be an aggregate in one view and absent from another.
It would also derive every view's layout on every empty-cell write.
**What the named view does and does not decide.**
- A record that holds a cell resolves in whatever view is named, as long as the
document has that view. The index is document-scoped, so the named view's
grid is consulted only on a store miss.
- An empty record resolves only if the named view's grid draws it. A write
naming a view that hides the record is `RecordGone` even when the host's own
grid draws it (`a_write_naming_a_view_that_hides_an_empty_record_finds_no_record`).
- **Writability is asked of the same layout** (`aggregate_refusal`). Otherwise
a half-fix resolves through the client's view and asks the host's grid,
where the cell has no position. The aggregate rule then has nothing to ask,
and the write lands on a cell the client was shown as an aggregate
(`an_empty_record_its_view_aggregates_is_refused_though_the_host_hides_it`).
- **A view the document does not have resolves nothing**, stored record or
empty, and the write is refused `RecordGone`. This was first written the other
way, falling back to the default through `App::resolve_view`. The argument
then was that a stale name cannot mis-write, because a record hash names the
same coordinates in every view that draws it. That is true about coordinates
and misses the point, as ChatGPT Codex found on PR #281. It gave the wire two
answers to one question: every other view-scoped `ModelCommand` refuses a view
the document lacks (`SetAxis` answers `MutationRefused`, "No view named").
And a client naming the wrong view, such as a hard-coded `"Default"`, kept
working whenever the default happened to draw the record, so the `view` field
checked nothing. `resolve_view`'s fallback exists so that *rendering* never
panics on a slice naming a deleted view (gotcha 29). That is a read-side rule,
and a write can simply refuse. So `Grids` looks the view up strictly, in two
places:
- `resolve_record` answers only for a view the document has. Gating the
store path matters: a strict lookup inside `with_layout_of` alone still
accepted a *stored* record under a missing name
(`a_write_to_a_stored_record_naming_a_view_the_document_lacks_is_refused`).
- `with_layout_of` returns `None` for a missing view instead of the
default's grid (`a_write_naming_a_view_the_document_lacks_finds_no_record`).
**Why `RecordGone` rather than the siblings' `MutationRefused`.** The
siblings refuse inside their effect, because an effect is the only place they
ask about the view. A cell write asks earlier: the view is part of *what the
surrogate names*, so a missing view is a failure to resolve. It is answered
by the basis check before anything is applied, by the same code, and with no
second rule beside resolution. It is also the right structural answer
(ADR 002): nothing resolves, so there is nothing to force, and force never
creates a view. A view deleted *earlier in the same frame* is still
`ChainBroken`, from `broken_chain`, because that is where a frame's own
structural change is caught.
- This is not a security boundary, any more than the basis is (gotcha 43). A
hand-rolled client may name any view. It can only reach a record some view of
the document draws, and the write lands on exactly that record's coordinates.
**One derivation, two callers.** `Grids::with_layout_of` builds the layout the same
way `grid_in` does: the host's own where the names agree, otherwise
`project::layout_elsewhere` over `project::rederived_for`'s model. Both are
shared with `grid_in`, so the grid a client was drawn and the grid its write is
resolved against cannot disagree. The cost is a layout derivation, and only for
an empty record on a view the host is not on. That is the cost `grid_in` already
pays for that client on every projection.
**Derived once per borrow, not once per question (improvise-kimw).** A write
asks about its record several times: the basis check, `broken_chain`, and in
`model_effects` both `unwritable` (resolution plus the aggregate rule) and
`resolve_cell`. As first written, each question derived the grid again. When the
document has a formula and the two views aggregate over different categories,
`rederived_for` clones the model and recomputes every formula, so each
derivation is expensive. `App::grids()` returns a `Grids`, which borrows the
`App` and memoises one layout per view name. The borrow is the invalidation
rule: while a `Grids` is alive nothing can mutate the `App`, so a memo cannot
go stale. It is therefore held for exactly one immutable stretch, never across
a mutation:
- `CheckedBatch::against` holds one for the whole frame check. Nothing is
applied until the check is over, so every command in the frame shares it.
- `model_effects` holds one for its command. `unwritable` and `resolve_cell`
share it.
- `broken_chain` holds its own. It runs just before the command is applied, at
the same state as `model_effects`, but it sits in `Host::mutating_frame`, and
`model_effects` sits below `Sessions::apply_checked`. Sharing across that gap
would mean passing owned layouts through the session door. That was judged
not worth it for one derivation.
So a frame of *n* writes to empty records on another view derives `1 + 2n`
grids, where it used to derive about `5n`.
`a_frame_of_writes_to_another_views_empty_record_derives_its_grid_sparingly`
counts them. `App::resolve_record`, `resolve_cell` and `unwritable` remain as
one-question conveniences, and each makes its own `Grids`.
One further cut is possible and was **not** taken. `GridLayout` reads cell
values only in `prune_empty`, so for a view without `prune_empty` the
rederived model makes no difference to the layout and the clone could be
skipped. That would be a second rule for "which model builds this view's grid",
beside `grid_in`'s. It would also stay correct only while `GridLayout::new`
reads no values, and no test would notice if that changed.
**The client side.** `resolve.rs` names `state.view.active_view` on each
composed write (`clear-cell`, `paste`, the Enter commit). The mouse path's
`EditTarget` captures the view **at open**, like its version and generation, so
a commit names the grid the editor was opened on (`context/code/clients.md`).
Every pre-existing fixture sits on `Default`, which is also `ViewState`'s
default. So a client that named a hard-coded `"Default"` passed all of them.
`every_composed_cell_write_names_the_view_the_client_is_on` puts the client on
`Other`.
**For improvise-obtu.** An aggregate resolves the same way:
`resolve_aggregate` can take the command's view and go through
`with_layout_of`. It is still host-only today, because no wire command carries
an `AggregateHash` yet.
+349 -12
View File
@@ -254,6 +254,69 @@ A group header survives only when a data item that follows it survives. In
column pass still reads the original grid. Records mode is exempt entirely — the
user drilled in to see all the raw data.
### Group runs nest, and pruning re-marks rather than filters (improvise-e265)
On an axis with two grouped categories the runs **nest**: `North` in `Upper`
holds `Jan` in `Q1`, and after `Q1` ends `North/Mar` is still in `Upper`. So a
marker names its level. `GroupHeader { level, depth, .. }` opens a run at
`(level, depth)` and closes every deeper one; `GroupEnd { level, depth }` closes
that run and every deeper one. "Deeper" is the ordering of the pair: the level
first, then the depth inside one category's hierarchy (§19, improvise-felo.6).
`expand_category` reads the level off its prefix, which holds one item per
outer category, so it is derived rather than passed. A row's group, for
`row_group_for`/`col_group_for` and so for `z`, is the **innermost** run open
at it. That is the user's ruling on the ticket: membership is per level, and
`z` toggles the group nearest the cursor.
The marker used to carry no level, and every reader was flat: the nearest
marker decided. Under a flat reading, pruning and membership could not both be
right. Before #200 a pruned axis could draw a header over nothing; after it, a
surviving row changed group when an inner header was pruned, because the
inner `GroupEnd` went with it (45,806 of 300,580 enumerated cases, every one
that shape). The unpruned answer was also lossy: `North/Mar` was in no group
while it was drawn under the `Upper` header row.
**Pruning therefore re-marks the survivors instead of filtering the markers.**
`prune_axis_items` tracks which run the input has open at each level (keyed by
the header's position, so two runs under two equal headers stay two, as
improvise-h0iv requires) and which run the kept entries have open. Before each
surviving item, `remark` emits a header wherever a different run should be open
and a `GroupEnd` wherever none should. Three properties follow by construction:
- each survivor keeps exactly the runs it was in;
- no header is emitted without a member, because headers are emitted only
before an item;
- no trailing `GroupEnd`, because a `GroupEnd` is also emitted only before an
item.
This replaced the `pending`/`close_runs_of`/`run_open` machinery, which is
where improvise-gd83, u6gc, lhi4 and b5zh each landed. When nothing is pruned
the output is the input, marker for marker, so the exact-sequence tests from
those tickets still pass unchanged. That equality needs one rule: `remark`
closes the deeper runs innermost-out before it touches a level, as
`expand_category`'s trailing `GroupEnd`s do. Without it the output is
equivalent but not equal (`End0` where the input has `End1, End0`), and the
enumeration test asserts equality whenever every row survives.
**How it is tested** (improvise-6u6s). An enumeration compares each
surviving row's whole open-run stack, and whether it shares a run with the
previous survivor, pruned against unpruned. It runs over five fixtures,
including a non-contiguous group, a level with no groups, and two adjacent
outer groups. Without the non-contiguous fixture, a pruner that keyed runs by
name instead of header position passed every test. `walk_open_runs` is the
one reading of the markers: `innermost_group` and the tests both use it. It
keeps one stack and lends it to a visitor at each data item, and the visitor
may stop the walk. So a single lookup (`z`, which asks about one row) walks
the axis only up to that row and copies nothing for the rows before it
(improvise-0r01). An iterator could not do this. Its first version yielded an
owned stack per row, and ChatGPT Codex found on #273 that `.nth(idx)` still
cloned one for every row it skipped. The tests collect every row's stack
through the same visitor.
A header's level also names its category (`cats[level]`) on the wire, where
`FrameGroup` carries `level` and not the category (improvise-7xyx removed that
field): see `context/code/protocol.md` §3a.
## 7. Axes: `Page` and `Filter` are two different questions
`Axis` is `Row | Column | Page | Filter | None`. `Filter` names a coordinate the
@@ -457,12 +520,129 @@ explicitly would be redundant in the file format and confusing in the UI.
`Label` is a high-cardinality per-row field (description, id, note): stored
alongside the data so it shows up in record/drill views, but defaulting to
`Axis::None` and excluded from the `MAX_CATEGORIES = 12` pivot limit and from
`Axis::None` and excluded from the `MAX_CATEGORIES` pivot limit and from
auto Row/Column assignment.
`Category::add_item` deduplicates by name and assigns IDs from a private
counter, so external code cannot fabricate an `ItemId`.
### The limit is 24, and nothing is exponential in it (improvise-felo.1)
`MAX_CATEGORIES` counts **regular** categories only. It is `pub` in
`model/types.rs` and re-exported as `improvise_core::model::MAX_CATEGORIES`,
and it has one home: the import wizard's precondition reads it rather than
keeping a copy. It had a copy until felo.1, a private `const` of its own, and
raising the model's number alone would have left the wizard refusing a 13th
category the model accepts. Tests that fill to the cap derive it from the
constant; `add_category_max_limit` is the one place the number 24 is spelled.
**Why 12.** Nothing recorded says. The limit arrived in the initial commit
with no comment, ADR or ticket behind it, and nothing in `context/` or `kb.pl`
argues for it. The likeliest source is Lotus Improv, which this tool follows
by name (`context/plan.md`) and whose worksheets are remembered as capped at
12 categories. That is not verified here. It was never a cost bound. The user raised it on 2026-09-25: "12 categories is
feeling small, we should increase to 24". An E*Trade import hit exactly 12,
leaving no room for improvise-cgqt's disambiguating category.
**What scales with the number of categories**, read off the code:
- `CellKey` is a sorted `Vec` of pairs: building one is O(k log k), and
comparing or hashing one is O(k).
- The `DataStore` secondary index holds one `(category, item)` entry per
coordinate per cell. That is O(k) per insert and O(cells × k) in memory.
- Records mode has one column per regular category, so it is linear in k.
- `cross_product` is the product of the item counts of the categories **on
an axis**. That is exponential in what the user places on the axis, and the
cap never bounded it: 12 categories of 4 items on rows is already 16M rows.
`prune_empty` is what keeps it drawable.
- `improvise-gp` bounds its encoded width separately (`MAX_FEATURES = 64`),
so a wide fit is refused by name rather than slowed.
**Measured**, `cargo run --release -p improvise-core --example
category_width_bench`, on an Apple M4 Max. Each category has 4 items and each
record carries 2 measures. The pivot is one category on rows, `_Measure` on
columns and the rest aggregated away. Figures are the best of 5 (the store
build is the best of 3), and they held across two runs.
| categories | records | build store | pivot, every cell | records build | records draw |
|---|---|---|---|---|---|
| 12 | 2,500 | 26–30 ms | 1.6 ms | 79 ms | 1.6 ms |
| 24 | 2,500 | 80–83 ms | 3.0–3.7 ms | 82 ms | 3.7 ms |
| 12 | 5,000 | 54–57 ms | 3.2–3.9 ms | 302 ms | 3.1 ms |
| 24 | 5,000 | 163–172 ms | 6.3–8.0 ms | 321 ms | 7.0 ms |
Doubling the width doubles the per-cell work and triples loading the store,
which also builds and hashes CellKeys twice as long. The records-mode build is
the outlier, and width is not its cause: it grows 4× when the **records**
double, at either width. `RecordRow::group` finds each cell's row with a linear
scan (improvise-sk32).
#### Coupling found
Raising the constant was used to look for code that assumes few categories.
This inventory was taken at felo.1's merge of `gh/main` `4f38a35e`. Line
numbers are dated, so re-derive them before citing.
**Reads the constant (fixed in felo.1).** The wizard's precondition, and the
duplicate step's cap check that improvise-cgqt added (`separate_blocked`). A
fixture that fills to the cap either derives it or fills until
`add_category` refuses.
**Separate limits, left alone.** None of these is a category count:
- `MAX_ROW_HEADER_W = 24` (`improvise-protocol/src/render.rs`) is the widest
one header level is drawn, in cells. It equals the new cap by coincidence.
- `MAX_FEATURES = 64` (`improvise-gp/src/fit.rs`) is the encoded-width budget
of one fit.
- `CATEGORY_THRESHOLD = 20` (`import/analyzer.rs`) is the distinct-value count
at which a text field becomes a `Label`.
**Degrades as categories grow.** These are not fixed here; each is a place
where 24 is worse than 12. Most were already reachable at 12.
- **The TUI row header is the sum of its levels** (improvise-gtn7.5). In `src/ui/grid.rs`, around
`row_header_width`/`data_area_width` (and
`ui::viewport::compute_row_header_width`), each level takes 4 to 24 cells.
Categories on Row therefore cost at least 4 cells each, and the data area
saturates to zero width with no data drawn.
- **Column headers take one line per level, but the keypress path does not
count them** (improvise-gtn7.6). `App`'s `visible_rows` is `term_height − 8`, a fixed allowance
that ignores the column levels, the group row and the page bar. So with
several categories on Column, the cursor can scroll below the drawn grid.
- **The page bar is one unclipped line** (improvise-gtn7.7). It is
`GridWidget::render`'s `page_coords` join, drawn with a bare `set_string`
that is not clipped to the inner rectangle, so a long bar can overwrite the
block's right border. Default seating (`View::on_category_added`) sends every
regular category after the second to Page. A 24-category import therefore
puts 22 selectors on one line, clips most of them without a mark, and opens
on a single slice.
- **Side panels do not scroll** (improvise-gtn7.8). In `src/ui/panel.rs`, an item past
`item_height` is not drawn, and the cursor can move onto it unseen. The
category panel lists every category plus its expanded items.
- **The wizard's field list does not scroll** (improvise-8jk6.1). Its `ReviewProposals` arm in
`src/ui/import_wizard_ui.rs` breaks at the popup's bottom (the popup is at
most 30 rows high), unlike the steps that use `scrolling`. About 22 fields
fit, so the over-the-cap case cannot show every field it asks the user to
fix.
- **Loading the store is O(k log k) per cell and O(cells × k) in memory**
(improvise-felo.2). It
measured 3× from 12 to 24 (the table above).
- **A fit declines more columns.** `unaffordable_categories` drops the widest
category columns once the encoding passes `MAX_FEATURES`. A wide document
is fit on fewer of its categories, and the fit says so.
- **`cross_product` is exponential in the categories placed on one axis**, and
`prune_empty`'s `has_value` matrix is rows × columns. The cap never bounded
either. Virtualizing the grid is improvise-felo.3.
- **Header repeat suppression costs O(levels) per position and level**
(`render.rs::shows`). That is O(rows × levels²) per frame, which is small at
24.
The records-mode build's quadratic (improvise-sk32) is not category coupling;
it is named in the measurement paragraph above because this measurement found it.
The browser adapts where the TUI does not. `#tiles` and `#categories` scroll,
and `web/shell.js` charges each column level to the extent.
### Seating takes the kind, because a name cannot carry it (improvise-cj4u)
`View::on_category_added` takes a `CategoryKind` beside the name. It used to
@@ -929,16 +1109,47 @@ every member. Five decisions:
without the memo, about 60 ms with it, and about 10 ms on gh/main. gh/main
was cheap because it read the members out of the formula cache, which is
what gave the wrong ratios. The remaining ~6× is each column's cell
re-deriving the chain below it (45 evaluations per row where 9 would do). A
memo scoped to the whole grid would remove it; it is not built.
re-deriving the chain below it (45 evaluations per row where 9 would do).
**The memo is now shared across one grid build** (improvise-h6z1).
`viewport::grid_body` takes one `GridLayout::evaluator` and asks it for
every cell and total, so a fold row's chain is evaluated 9 times, not 45.
`prune_empty` does the same (improvise-metf). A memo is valid only for the
model state it read. That is why it is made per build and never stored on
`GridLayout`, which outlives cell writes.
The memo is keyed by the fold key **and** the span's categories. Within one
grid build the span half cannot yet differ for one key, because a cell's key
always names the row categories and a total's never does. It is kept because
`evaluate_spanning_in` is public, and a caller sharing one memo across two
spans would otherwise read the wrong fold. No test can see it today.
**The aggregation and the covers are the memo's, not the call's**
(improvise-pntu). They are the same argument as the span, but they were
left as parameters. So one memo passed with covers=all and then with Feb
hidden returned the first fold's answer (5.0 where 2.0 was right). Widening
the key instead would have needed a closure's identity, and a closure has
none worth hashing. So `FoldMemo::new(agg, covers)` owns both, and
`evaluate_covering_in`/`evaluate_spanning_in` read them off the memo. A
mismatched pair can no longer be passed.
**A grid's evaluations cannot be handed another grid's memo either**
(improvise-opbo). `GridLayout::evaluate_in` and `column_total` used to take
any `&FoldMemo`. A memo built from a layout showing Feb, passed to a layout
hiding it, gave 5.0 where 2.0 was right: a consistent pair, only for a
different grid. `GridLayout::evaluator(model)` now returns a `GridEval` that
owns a memo its own layout built, with `cell` and `column_total` on it, and
the memo-taking methods are gone. `Model::evaluate_covering_in` and
`evaluate_spanning_in` still take a `&FoldMemo`, but a memo there carries
its own agg and covers, so the only mismatch left is a memo over a
different *model state*. That is the per-build rule above, unchanged.
- **A fold covers the members the grid draws** (improvise-f2oo). Hidden members
are not in it. So a fold is exactly the rows it replaces, the total of a
**stored** measure under Sum or Count does not move on `z`, and a group whose
every member is hidden has no row. Under Avg, Min or Max a stored measure's
total does move, and so does a **formula** column's (PR #203's review measured
Margin 1.90 expanded, 1.30 collapsed). The strip sums the rows' values rather
than folding the column the way a fold does, so it sums a Max row's maximum
and a formula row's result. That is improvise-wm20, and it is not this rule. That last
column's total does not move on `z`, and a group whose every member is hidden
has no row. That holds for every aggregator and for a formula column only
since improvise-wm20, because the strip used to **sum** the drawn rows. That
moved a Max measure's total on `z`, and a ratio's (PR #203's review measured
Margin 1.90 expanded and 1.30 collapsed). A total is now the column's cell as
if every drawn row were one fold, so collapsing a group changes nothing it
reads. The fold's machinery is shared through `Model::evaluate_spanning`: its
`Span` adds, to the groups a key names, the categories a key **omits**, which
it expands to their covered items. That last
point was already true, and it contradicted the old rule, under which a hidden
value appeared only while a sibling was visible. This deliberately **differs
from `Axis::None`**, where hidden items still count, because the two replace
@@ -1300,10 +1511,11 @@ identified, in practice and by `add_formula`'s upsert, by `(target,
target_category)`. **`FormulaId` is that pair behind one opaque type**: its
fields are private, `Formula::id()` is its only constructor, and nothing
outside `improvise-formula` can read a name out of it. That is deliberate. The
stable-identity work (improvise-sboz, ADR 005) will make a formula's identity
stable-identity work (improvise-sboz, ADR 007) will make a formula's identity
something other than its names, and every signature that already says
`FormulaId` — `Model::formula`, `Model::replace_formula`,
`ModelCommand::ReplaceFormula`, `FormulaSlot::Existing`, `FormulaEntry` — then
`ModelCommand::ReplaceFormula`, `ModelCommand::RemoveFormula` (since improvise-tej1),
`Model::delete_formula`, `FormulaSlot::Existing`, `FormulaEntry` — then
changes by changing this one struct. A name pair spread across those signatures
would have made the swap a sweep.
@@ -1326,7 +1538,7 @@ so the position (and with it `_Measure`'s item order, since
**A target change is a rename of the same formula.** The id travels with the
edit, so `Profit = …` edited to `Gross = …` is the formula that was `Profit`,
in the same place. That is the least surprising reading of "edit this row", and
it is what ADR 005 proposes. What a rename cannot carry under name identity is
it is what ADR 007 proposes. What a rename cannot carry under name identity is
everything else that names the old target: other formulas' references, a
view's hidden list, a `measure_agg` entry. Those cannot follow without rewriting
`raw` text the parser does not print back, so the edit says so out loud:
@@ -1346,3 +1558,128 @@ editor stays open over the user's text (gotcha 21). `was` is not redundant
with a stable id: a concurrent *replace* keeps the id and changes the text,
which only `was` can see.
## 19. Group hierarchies: parents nest runs, and every group starts closed (improvise-felo.6)
`Group::parent` was parsed (`> Q1[2024]`), written, round-tripped and hashed,
and nothing read it: `Q1` was never drawn inside `2024`. It is now the
hierarchy.
### The user's rulings
- "I think hierarchies should start closed unless we somehow opt into a
specific one opened to a certain depth (view config maybe?)" — and the
reason: "in most cases, the closed by default start is preferable because it
makes the grid more dense."
- "Everything opens closed, including old files." There is no version gate: a
file written before this change opens with its groups closed, and its
`collapsed:` lines simply agree with the default.
- "I think we'll need a depth for expanded, something like expanded(2): ...".
- "expanded+collapsed in the same group is an error".
### The model
- **A group's chain** is the group and its ancestors, root first
(`Category::group_chain`). An item's chain is its group's
(`Category::item_chain`); an ungrouped item has none. A group's depth is its
number of ancestors.
- **A group covers every leaf beneath it.** `items_in_group` reads each item's
chain, so `2024` covers the months of `Q1-2024` and `Q2-2024`. That is the
whole change to evaluation: `evaluate_covering` already expanded a group
coordinate through `items_in_group` (§14), so a year's fold is the fold over
its months with no second path.
- **An item may sit directly in a parent group, beside its child groups.**
Chosen because nothing is simpler: the chain handles it (`Jun[H1]` after a
run of `Q2` inside `H1` ends the `Q2` run and stays in `H1`), and refusing it
would be a rule with nothing to protect. A date import never produces one.
- **A cycle is refused on load, naming its members** (`Refusal::HierarchyCycle`,
`persistence/parse.rs`). A cycle built in code cannot hang anything either:
`group_chain` stops where it would repeat.
- **There is no such thing as a dangling parent in this format.** A root group
is declared *only* by being named as a parent: the writer emits `>` lines
only for groups that have a parent, and nothing else can name `2024`. So
"a parent that names no group" and "a root" are the same line, and refusing
one refuses the other.
### Which groups are open
State lives on the `View`, in two places that cannot disagree about one group:
- `expand_depth: cat → N` — `expanded(N): Cat`, the category's top N levels
open.
- `group_folds: cat → group → GroupFold` — ONE map, so a group has at most one
fold. `GroupFold::Collapsed` is `collapsed: Cat/Group`;
`GroupFold::Expanded(N)` is `expanded(N): Cat/Group`, the group open and
N-1 levels beneath it. Two sets would have made "both" representable, which
is the state the user ruled an error
(`doctrine(an_invariant_is_made_unrepresentable_not_validated)`).
`View::is_group_collapsed(cat, chain)` answers, in order:
1. the group's own fold, if it has one — so a `collapsed:` group inside an
`expanded(N)` region stays collapsed, and an `expanded:` group below the
category's depth opens;
2. open, if an ancestor's `Expanded(n)` reaches it (the ancestor `k` levels up
reaches it when `k < n`), or the category's depth does (`depth < N`);
3. closed.
**`z` writes one fold, or none.** `toggle_group_collapse` flips the answer and
records the flip as `Collapsed` or `Expanded(1)` — unless the flip lands on
what the group would inherit from rule 2, in which case it removes the group's
fold. So `z z` leaves the view as it was, and no sequence of toggles writes a
file that names one group twice
(`no_sequence_of_toggles_writes_a_conflicting_file`). Closing a group that
carried `Expanded(3)` drops the 3: reopening it with `z` opens one level.
### The layout
`expand_category` walks each drawn item's chain. The **outermost** collapsed
group in the chain folds everything beneath it: the row's tuple is that group's
name, it folds once however non-contiguous its members are (§14), and the
headers above it are drawn. The run markers move from item to item exactly as
`remark` moves them after pruning — past the shared prefix of the two chains,
end the deeper runs innermost-out, then open what differs — so an unpruned axis
survives `prune_axis_items` marker for marker, and the enumeration in
`group_membership` (now with a `NESTED` fixture: a non-contiguous quarter, an
item directly in a half-year, an outer grouped category) asserts it.
A header carries its own `collapsed`, computed once in `expand_category`, so
the frame projection and the frame hash read it rather than asking the view
again with a chain they would have to rebuild.
### Persistence
`expanded(N): Cat`, `expanded(N): Cat/Group`, bare `expanded:` for N = 1, and
the existing `collapsed: Cat/Group` (`improv.pest`, `expanded_line`). N is at
least 1 by the grammar (`'1'..'9' ~ ASCII_DIGIT*`), so `expanded(0)` is a parse
error rather than a value to explain. Two further decisions:
- **The writer keeps what the view holds.** An old file's `collapsed:` lines
load as `Collapsed` folds and are written back verbatim, though they agree
with the default. They are the user's explicit closes, and they survive a
later `expanded(N): Cat` that would otherwise open those groups; dropping
them on save would change what that later line means. A `z` round trip
removes such a line, because `z` normalises (above).
- **A category depth is never written bare.** `axis_line` is tried before
`expanded_line`, so a category literally named `expanded` keeps its
`expanded: row`. The price is that `expanded: row` meaning "category `row`,
depth 1" would read as that axis line, so the writer always spells the
category form `expanded(N): Cat`. The group form has a `/` and is never
ambiguous.
A view naming one group with two different folds, or one category with two
depths, is refused on load, naming the view and the group
(`Refusal::FoldConflict`). The same line twice is not a conflict.
The grammar-walking generator emits cycles and conflicts with ordinary
probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a
`Refusal` — a typed error, so it cannot accept a parse failure by accident.
### Known limits
- The TUI draws nested headers exactly as it draws any other: one line per
header, not indented by depth, and the column band shows each column's
innermost group only. Both are pre-existing presentation choices; the frame
carries `depth` for the day a front end wants to indent.
- Two `>` lines giving one group two different parents: the last wins, as it
did before this change. That makes the hierarchy a function of line order.
+9
View File
@@ -181,6 +181,15 @@ The general form: when a check misses a defect, add the check that could have
seen it rather than only fixing the defect, and ask what *question* the
existing check was answering.
**`site/` is build output and is not tracked** (improvise-1xfb.4). It is
`web/assemble.sh`'s default `OUT_DIR`, and `site/docs` is this script's.
Every reader builds its own copy from source: CI's `build` job runs
`nix build .#site`, which assembles into the store; the harness assembles
into `harness/serve/`; `scripts/build-web-release.sh` assembles into a
temporary stage. For a while 27 generated pages under `site/docs` were
committed by accident. They went stale, still saying 12 categories after the
cap was raised, and nothing read them.
## 5. Why a node script and not a generator
The landing page is hand-written HTML with no build step and one pinned CDN
+300 -8
View File
@@ -165,7 +165,7 @@ share, which is why the existing suite was silent — the same shape as the
`Category::kind` rather than the emitted text.
The consequence was a lost document, not a lost flag. `Label` sits outside
`MAX_CATEGORIES = 12` on purpose, and the import wizard's own precondition
`MAX_CATEGORIES` on purpose (12 then, 24 since improvise-felo.1), and the import wizard's own precondition
scores a label as zero, so it will happily build twelve categories plus a
label. On load every label came back `Regular`, the thirteenth was refused, and
`parse_md` failed: **any document where regular + label exceeded twelve was
@@ -218,6 +218,32 @@ times. The probe was removed rather than kept: its rate depends on proptest's
internal RNG, so as a permanent assertion it would fail on a dependency bump
for a reason unrelated to the format.
### Folds are view lines: `collapsed:` and `expanded(N):` (improvise-felo.6)
Groups start collapsed, so a view persists what it opens. `expanded(N): Cat`
opens a category's top N levels, `expanded(N): Cat/Group` opens that group and
N-1 levels beneath it, a bare `expanded:` is N = 1, and `collapsed: Cat/Group`
keeps its old meaning. The semantics, the precedence and the refusals are
`context/code/core.md` §19; what is particular to the format:
- **No version bump.** The line is additive, and the user ruled that old files
open closed like new ones, so `v2025-04-09` still reads every file it did.
- **N >= 1 is grammar, not validation**: `depth_number = @{ '1'..'9' ~
ASCII_DIGIT* }`. Spelled with a `Range` and `ASCII_DIGIT` rather than
`ASCII_NONZERO_DIGIT`, because the generator knows the first two and would
emit nothing for the third — `expanded(): ` — a file its own grammar refuses.
- **`axis_line` comes before `expanded_line`** in `view_entry`, so a category
named `expanded` keeps its axis line, and the writer never spells a category
depth bare (`expanded: row` would be that axis line).
- **Two refusals the grammar cannot express** — a cycle among `>` lines and one
group given two folds — are a typed `Refusal`, which `generated_file_parses`
accepts and nothing else does. The generator produces both at ordinary rates.
- **The generator reaches the new rule**, asserted by
`the_generator_writes_expanded_lines_with_and_without_a_depth` over 512
deterministic files. Unlike the kind-line probe above, it asserts that each
spelling occurs at all, not a rate, so a proptest bump fails it only if the
walk stopped reaching the rule.
### The tree walkers are infallible by construction
Every `build_*` in `parser.rs` operates on a `Pair` pest has already
@@ -493,7 +519,9 @@ compiler named every other consumer. Two remained, and both are the wire:
A third arrived with improvise-km71 by the same rule: `FormulaId`, which crosses
the wire in `ModelCommand::ReplaceFormula`, `FormulaSlot` and `FormulaEntry`.
So `improvise-formula` depends on serde again, for that one type. No derive went
back on the formula AST.
back on the formula AST. A fourth, `CategoryKind`, went back on when `TileEntry`
began carrying it (improvise-bqm0; `context/code/command.md`). It returned as a
wire type, not as a document one: nothing loads a `Category` through serde.
`CellValue` was never a candidate, since it is carried on the wire too. JSON
*import* (`StartImportWizard`, `ImportJsonHeadless`) is a separate feature and is
untouched.
@@ -630,6 +658,144 @@ one proves nothing.
backend rather than only a file (improvise-fuj); the path-taking wrappers remain
for the native CLI.
### A folder of CSVs is one sparse model, with a view per file (improvise-lpzu)
`improvise import <folder>` exists for improvise-1xfb.5.1. code-maat writes one
CSV per analysis into a directory, and the schemas differ: coupling has
`entity,coupled,degree,average-revs`, age has `entity,age-months`, and
abs-churn has `date,added,deleted,commits`. The user ruled (2026-09-25):
"Records are already sparse: the model can just leave null/empty values for
columns not in a given CSV file." So files with different headers merge into
**one** model. It is not refused, and it is not split into one model per schema.
**A folder argument means exactly its `*.csv` files.** `expand_import_paths`
takes the arguments in order and replaces each folder with its `*.csv`
entries. The extension check is case-insensitive, the listing is not recursive,
and the order is `Storage::list`'s, which is sorted by contract. Then the
existing path runs, with one difference: **a folder always merges**. Two or more
files merge through `merge_csvs_from` with a `File` category, and so does a
folder holding a single CSV. That gives the one file its `File` item and its
per-file view.
The single-CSV rule is **the user's ruling (2026-09-25)**, which replaced this
PR's first answer, "`import dir/` is exactly `import dir/*.csv`". Their words:
"yes, we might want to have some kind of update import eventually (don't plan
this yet, but we'd want file identity for this)". A folder names a collection
that may grow, so a document built from it should record which file each
record came from from the start.
**A single CSV named directly, not through a folder, still imports alone,
without `File`.** That is deliberate. Naming one file asks for that file, not
for a collection, and the shipped recipes (`examples/demo.improv` from
`examples/demo.csv`) and every existing single-file script would otherwise gain
a `File` category and a second view. File identity for update-import can be
asked for by naming a folder. The test is
`a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not`.
Apart from that, a folder still behaves as the glob would, and four cases follow
from that rather than being chosen one by one:
- **Non-CSV entries are skipped**, not refused, because a glob skips them. This
includes the output `.improv`, which a second run finds in the folder.
- **A folder with no CSV at all is refused**, naming the folder. A glob that
matches nothing is an error too, and importing nothing would write an empty
document.
- **Files and folders mix freely**, in argument order.
- **A file named twice is read twice.** For example, `dir/a.csv dir/` reads
`a.csv` twice, and its records collide with themselves. The headless path
refuses that like any other collision. Nothing deduplicates, because the
shell glob would not either.
The listing goes through `Storage::list` so the browser host can reuse it
(gotcha 2). `Storage` knows documents, not folders, so **whether an argument
is a folder is the caller's question**. The CLI answers it at the edge with
`Path::is_dir` and passes an `ImportPath::{File, Folder}`. An OPFS host would
answer it from its own UI.
**Sparse merge.** A record from a file without some column has no key for it,
so `Plan::place` gives it no coordinate there (improvise-8jk6.2 above), and no
measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already
classified each column over the values present (improvise-l1f5), so a column
only one file has is judged by that file's values alone. The category cap
counts the **union** of every file's categories, header-only files included
(see the next paragraph). Rows from different files cannot collide, because
`File` differs. One caveat on "judged by its values": a JSON `""` is not yet
dropped by the analyzer the way `null` is, so a JSON numeric column holding
`""` is proposed as a Label (improvise-8jk6.11). CSV is unaffected, because
its parser turns a blank field into `null`.
**A header-only file is still a file** (the Codex P2 on PR #314). code-maat
writes a header and no rows whenever nothing passes its threshold, so
`coupling.csv` is often exactly that. The pipeline is analyzed from *records*,
and such a file has none, so it had no `File` item, its columns never reached
the model, and its view filtered to an item that did not exist.
`ImportPipeline::from_merge` therefore **seeds from `Merged::sources`**:
- the `File` proposal's items are the sources' items, in file order, so every
file has its item whether it has rows or not;
- a source column that no record has a value in is proposed as a **Label**.
That is the analyzer's own verdict for an all-null column (improvise-l1f5),
and a column with no values is exactly that: nothing can be classified, so it
takes no pivot slot and does not count toward the category cap until the user
reclassifies it in the wizard or with `--category`/`--measure`.
This is also the file identity the user wants kept for a future update-import.
**A file with no measure column gets a records view** (improvise-8jk6.13). A
record with no measure makes no cell, so such a file contributes items and no
data. Its view had nothing to page `_Measure` to, and paged onto another file's
measure, which the same view hides; the cell there was a `Record`, editable,
under this file's `File`. Now `source_view` gives such a file the records
layout (`view::records_steps`, the one home of what a records view is) with
`File` still on Filter at the file's item. It draws zero rows, because there
are no cells. A header-only file whose columns are
all Labels is the same case. In records mode the columns are the whole model's
categories and measures, not just this file's: records mode has no per-view
column filter. With no rows that is cosmetic.
**Each file gets a view** (the user, 2026-09-25: "dir import should do this:
one view filtered by the file column and projected to the columns in that
file"). `merge_csvs_from` returns a `Merged` whose `sources` list each `File`
item with the columns its header named. Two files with one stem share an item,
so their columns are unioned. The pipeline carries `sources`, and the build
adds one view per source after the merged `Default` view, which stays the
initial view. Each per-file view:
- **Is named after the file's stem.** A name already taken, including the
virtual `_Records` and `_Drill`, gets ` 2`, ` 3` and so on. A view named
`_Drill` would never be written and would be dropped on load.
- **Puts `File` on `Axis::Filter`, fixed at that file's item.** Filter and
not Page, because the view *is* that file's projection. Paging to another
file would show that file's records through this file's columns and hidden
measures, which is wrong on every page but one. Filter constrains without
being navigable (gotcha 27). Until now it appeared only in the never-saved
`_Drill` view, but the `.improv` grammar has always round-tripped it.
`per_file_views_survive_a_round_trip_and_repeat_byte_for_byte` proves that on
a saved per-file view, and `a_saved_per_file_view_projects_only_its_file`
proves the shared `grid_in` projection, which both the TUI and the browser
draw from, carries it as a fixed coordinate with nothing to cycle.
- **Seats the file's own categories** (its columns, plus date parts derived
from them) through `View::on_category_added`, in model order: the rule a
single-file import uses. Every other category goes on `None`. The file's
records lack those, so nothing splits.
- **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files'
measures hidden** and the page selection on the file's first measure. A
formula target comes from no file, so it stays visible. A file with no measure
gets a records view instead (above).
- **Is never named like another view from the same import.** `Default.csv` and
`Default 2.csv` become `Default 2` and `Default 2 2`; without the same-batch
check both would be named `Default 2` and one view would silently replace the
other.
A per-file view is made only while `File` is an accepted category. A user who
skipped or reclassified it has no filter to build views from.
Two things this does not do. Page cycling still steps onto a hidden item and
draws an empty grid: `[`/`]` in a per-file view can reach another file's
measure (improvise-yd2t; the Default view already behaved this way). And a
CSV with its own `File` column has it overwritten by the merge's, which is the
existing behaviour of `merge_csvs`.
### The wizard's back path is a history stack
`advance` pushes the step it leaves; `previous` pops it. Keeping the path
@@ -641,8 +807,9 @@ Preconditions live in `validation_error()` and are checked **on the step where
the user can still fix them** — never deferred to `build_model` after the final
confirm. Two gates earn that: too many categories (improvise-mzv) and no
category at all (improvise-gfq); both are fixable with `c` at `ReviewProposals`,
and both used to refuse only at the last keystroke. `MAX_CATEGORIES` here mirrors
the model's private limit and counts regular categories only.
and both used to refuse only at the last keystroke. `MAX_CATEGORIES` here is the
model's own constant, which counts regular categories only. Until improvise-felo.1
it was a private copy (core.md §9).
`NameModel` is terminal for `advance`: leaving it is a **build**, not a step
change. `build()` parks the workbook in `built` on `Done`; a build failure leaves
@@ -655,10 +822,135 @@ and a blank string are **absences, not names**; stringifying them used to invent
an item literally called "null" (improvise-ych). Every other value keeps its
printed form.
A record missing a value for an accepted Category field has no place in the grid
and is dropped; `build_model_with_warnings` counts it against the *first*
category it is missing and reports the total, because without a warning that
loss is invisible in a thousand-row import (improvise-5j9).
**A record with no value for a category keeps its cell, without that
coordinate** (improvise-8jk6.2). Records are sparse: a `CellKey` names only the
coordinates it has. So a missing field, a `null` and a blank string all mean
"no coordinate here". None of them makes an item: not `"null"`, not `""`, and
no synthetic placeholder. The same rule covers labels, which used to become an
item named `""`. The user's words, 2026-09-25: "this limit in import should be
fixed, it dates to when this wasn't sparse." That limit was improvise-5j9's
rule: such a record "has no place in the grid" and was dropped, counted and
reported as `Skipped N of M records missing category values`. 5j9's warning
made the loss visible, and the loss is now gone, so the warning went with it.
The only record still skipped is one that is not a JSON object.
Where such a record shows up is decided by the axes, not by the import. The
test is `a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_the_grid`:
- **The category on Row or Column:** no position holds the record. Its other
coordinates may still draw a row, but the cell under the category's items is
empty, and the column total leaves it out.
- **The category on Page:** no selection shows it.
- **The category on None:** it is aggregated like any other record.
- **Records mode:** its row shows a blank cell in that category's column.
- **`.improv`:** its Data line omits the category, and it round-trips.
- **Drill:** a drill that does not constrain the category reaches it.
**No "(none)" position, by the user's decision (2026-09-25).** They were asked
whether Row, Column or Page should give such records a place of their own. Their
answer: "I think leaving it out for now is fine, maybe a UI element somewhere
that shows the number of records underneath the current view?" That indicator
is improvise-gtn7.10, and it is not an import concern.
### Colliding records are a question, not a default (improvise-cgqt)
Two source records that land on the same coordinate tuple are one cell, so
something has to happen to the second one. improvise-2cw9 made that "sum and
warn". The user ruled on 2026-09-25, after an Etrade import summed 31 cells and
reported it in one truncated line, that the **wizard asks**. It offers three
choices: (a) sum them, (b) keep them separate with a generated category, or
(c) go back. So `ImportPipeline::on_duplicate` is a `DuplicatePolicy`
(`Sum | Separate(name) | Refuse`), and its default is **`Refuse`**. Any caller
that never asked therefore gets an error and never a silent sum.
`ImportJsonHeadless` builds its pipeline as a struct literal, so the compiler
made it choose too. It chose `Refuse` and points at `:import`.
**Detection comes before the build and shares the build's placement.** Where a
record lands is `Plan::place`: its categories, then its date parts, then its
labels, then its measure values. Only a record that is not an object is
dropped, since a missing value is no coordinate rather than a reason to drop
(improvise-8jk6.2). `collisions()` and
`build_model_with_warnings` both call it, so the question the wizard asks and
the thing the build does cannot disagree. Neither has a second walk of the
record. **A collision is a cell that more than one record writes**, meaning a
tuple plus a measure (improvise-9bit). A `Collision` names the tuple and only
the records that share one of its cells. A record with no measure makes no
cell, so it collides with nothing. The tuple includes label values, as
`CellKey` always has.
This PR's first version keyed on the tuple alone. It was **superseded in review
(improvise-9bit)**, and the argument it gave is withdrawn. Under that rule, two
records with **disjoint** measures on one tuple (one has `Revenue`, the other
`Cost`) counted as colliding although no cell sums. The wizard then offered
"Sum them into one cell" where nothing would sum, and with the `Refuse` default
the CLI refused a sparse-measure CSV that had always imported cleanly. The
question the user is asked is "these values would be added together", so the
rule has to be the one that decides whether anything is added.
The data is structured. A `Collision` holds `coords` plus 1-based `records`
positions. `WizardCache::collisions` carries it, and
`WizardCollision::describe` is the one wording, used by the TUI and by the CLI.
A browser wizard must draw the same rows (gotcha 32), not a second sentence.
**Where the step sits.** It comes after `ReviewProposals` and after
`ConfigureDates` when that step is visited, and before `DefineFormulas`. It
appears only when `collisions()` is non-empty, and the list is recomputed each
time the step is entered, because it depends on the field choices. The two
field steps are exactly the steps that can change it. Date parts are functions
of a value that is already a coordinate, so they can never split or merge a
collision, but they count toward the category cap, and that is why the question
waits until they are known. The history stack (gotcha 22) handles the new
paths without change: `ResolveDuplicates → NameDuplicateCategory →
DefineFormulas` retraces backwards. The chooser acts **on the cursor**. Enter
on this step is `ChooseDuplicate`, never a bare `Advance`: the policy starts
at `Refuse`, which `validation_error` blocks, so a bare advance could never
leave the step.
**Separate is an occurrence index, not a row number.** This is the user's
preferred shape. The records in a collision get `1, 2, 3…` in source order,
and every other record gets `1`. That includes a record on the same tuple that
shares no cell with the others: it lands on occurrence `1` beside the first
colliding record, and since the two write different measures, nothing sums. The category holds only as many items as the
deepest collision, rather than one item per record, and every existing
coordinate keeps its meaning. Items are added `1..=n` in numeric order
(gotcha 18). The category goes on **`Axis::None`**. Left to
`on_category_added` it would land on `Page`, and the default grid would then
show only occurrence `1`, which silently hides the other records. On `None`
the grid aggregates across occurrences, so the first screen shows the same
totals Sum would. The records stay distinct, so `Avg` and records mode see
each one. A policy is **moot when nothing collides**: no category is added.
The name is **suggested and editable**: `Occurrence`, or the first
`Occurrence_N` that no header (accepted or not) and no category the build will
create already uses. `NameDuplicateCategory` refuses an empty name or one that
is already a category. `suggested_duplicate_category` lives on the pipeline, so
the CLI's `separate` uses the same rule.
**At the category cap, Separate is shown and disabled.** The reported import
had 12 regular categories, which was `MAX_CATEGORIES` when this was written.
Option (b) adds a
**regular** category, and the user declined a hidden virtual one. So at the cap
the option is drawn "(unavailable)" with its reason and remedy: go back and
Skip a column or make it a Measure, or choose Sum. The reason is decided in
`ImportWizard::duplicate_options`, so every front end words it the same way.
This was the conservative reading, and it left an open question for the user.
**The user answered it on 2026-09-25 by raising the cap to 24**
(improvise-felo.1), so "keep separate" is available for the reported import
now. The option is still disabled for any import that fills the new cap.
**Headless paths refuse by default.** `improvise import --no-wizard` has
nobody to ask, so `--on-duplicate refuse|sum|separate` defaults to `refuse`.
It prints every colliding tuple and the flag, exits non-zero, and writes
nothing. Summing by default would repeat exactly the silent outcome the user
called critical. Separating by default would reshape a scripted import's
document and fail outright at the cap. A refusal is the one default that
cannot be wrong without saying so. This changes behaviour: a script that relied
on 2cw9's implicit sum now needs `--on-duplicate sum`. `:import-json` has no
flag and refuses, naming `:import`.
The 2cw9 argument that summing is the right default is **superseded** here, as
its own evidence entry anticipated. Its warning survives as the Sum
confirmation.
---
+266 -36
View File
@@ -178,12 +178,14 @@ attempt in a fourth place is the disease.
- The **model half** is `ModelCommand`: every `EffectKind::Model(..)` effect,
one variant apiece, carrying that effect's own fields (improvise-kmgs).
The count is 27 model effects, minus two: `ApplyAndClearDrill` and
The count is 28 model effects, minus two: `ApplyAndClearDrill` and
`AcceptWizardModel`, which are **not addressable** (improvise-3x3u). That
leaves **25**. `SwitchView`, `ViewBack` and `ViewForward` are no longer model
effects at all (improvise-jytm), so they are not subtracted. This paragraph
said "29 minus three, 26" until improvise-km71 added `ReplaceFormula`, and
that figure was already stale: the enum had 24. A variant with
leaves **26**: 24, plus `ReplaceFormula` (improvise-km71) and
`RenameRecordCoordinate` (improvise-fdk1). `SwitchView`, `ViewBack` and
`ViewForward` are no longer model effects at all (improvise-jytm), so they are
not subtracted. This paragraph said "29 minus three, 26" while the enum had 24,
and each of those two PRs then corrected it to 25 without seeing the other.
Count by brace-matching the `impl Effect` blocks, as below. A variant with
no data mirrors a nullary effect whose operand is document state the
authoritative `App` already holds.
@@ -443,7 +445,7 @@ message** and did nothing to close ignoring **on arrival**, which is the half
a struct variant is usually reached for. That hole needed a type —
`Checked`/`CheckedBatch` in `improvise-command` (improvise-ondj).
### `ModelCommand::SetCell` takes a `RecordCell` and nothing else
### `ModelCommand::SetCell` takes a `RecordCell` and the view that drew it
A `CellKey` here would require the client to know the coordinate space —
every row and column category plus the page and filter selections (gotcha
@@ -452,6 +454,25 @@ which is precisely the knowledge the server exists to own. The surrogate is a
token to echo back; and because it is a *record* identity, a client holding
one for an aggregate cell cannot build this variant at all (ADR 002).
The `view` beside it is a view *name*, which the client already holds
(`ViewState::active_view`). It is not a coordinate. A record with no stored cell
exists only as a position in some grid, so the host resolves it against the grid
of the view the client composed in, not its own (improvise-wpxr,
`context/code/command.md`). This is the `view: String` convention every
view-scoped variant follows (improvise-lltt). `ClearCell` is a struct variant for
the same reason.
`named_records` is the exhaustive list of records a command writes, and
`records_named` is read off it. Each entry says where the host looks the record
up. A cell write is `NamedRecord::Drawn { view, record }`, resolved in the view
that drew it. `RenameRecordCoordinate` is `NamedRecord::Stored(record)`, and it
carries no view. Only a record that holds cells can be renamed: a coordinate
cell exists only in records mode, whose rows are the store's own records, and
a record with no cells has nothing for the rename to move. So the store is the
whole answer, and a view name would add a way to be wrong with nothing to
resolve. The destination the server adds for a rename (`records_at_stake`) is
`Stored` for the same reason.
`LoadModel` and `ImportJsonHeadless` name paths in the **host's** `Storage`,
never the client's filesystem (gotcha 2).
@@ -544,7 +565,7 @@ it. A browser's answer is its OPFS entry name; neither end has to know which.
### `CellId`: the kind of identity is the permission
`SetCell`/`ClearCell` take a `RecordCell` and nothing else, `Drill` takes an
`SetCell`/`ClearCell` take a `RecordCell` (and a view name), `Drill` takes an
`AggregateHash` and nothing else. A client cannot compose a write against an
aggregate **or** a drill against a stored cell — not because either is
checked, but because neither carries an id of the other's type
@@ -620,8 +641,8 @@ it, and the address it needs is the name.
What was missing was therefore never an address but a **recognition**. `Create`
carries the kind and nothing else, because the kind plus the typed name is the
whole operand. It is not a `SetCell` target, so `render.rs::editable` stays
`Record`-only and `record_at_cursor` still answers `None` — both by matching,
which the compiler now forces.
`Record`-only and `opened_at_cursor` captures no editor target for it — both
by matching, which the compiler now forces.
**No client composes it yet**, and that is one gap rather than this one:
`keys.rs::commit` returns early for every records-mode commit, because a
@@ -641,29 +662,49 @@ belongs to the host. So absence was encoding a permission, the same conflation
`a_coordinate_column_is_projected_with_no_identity`, asserted the bug and is
gone.
`Coordinate` carries **nothing**. Editing a coordinate renames it across every
measure cell in the row (gotcha 1), and no client composes that operation: it
needs `record_col`, and the browser declines every records-mode edit before
reaching a cell. Giving the variant a `RecordHash` would hand a client a
surrogate with nothing to spend it on. The operand belongs with the command
that consumes it, whenever that command exists. Nothing a client can do has
changed. `render.rs::editable` is now an exhaustive `match` rather than a
`matches!`, so the next variant has to be classified. `typed_cell_command`,
`record_at_cursor` and `record_cell_at_cursor` still answer `None` for it.
So paste onto a coordinate is still a silent `Upstream` in the browser. The
TUI has declined it out loud since PR #184. That divergence is
improvise-kkcy, and a named kind is what lets the browser decline deliberately
now.
**Since improvise-fdk1 it carries `{ record, category }`**, the operands of
`ModelCommand::RenameRecordCoordinate`. Editing a coordinate renames it across
every measure cell in the row (gotcha 1). When exdg landed, no client composed
that operation, so the variant was left empty on the grounds that the operand
belongs with the command that consumes it. That command now exists, and both
operands come from the **projection**:
- **`record` is the row's own record.** A client could borrow the hash from a
sibling `Record` cell in the same row. That would be a second derivation of
which cells share a record, which is what ADR 002 forbids. It would also have
nothing to borrow in a document with no measures, where the coordinate is
the only cell a row has. For the same reason `published_basis` counts a
`Coordinate`'s record, or a rename of such a row would be `RecordUnstated`.
- **`category` is the column's identity.** Its label is not: a category and a
measure may share a name (gotcha 1), so reading the category off the column
label would be a guess.
`RenameRecordCoordinate` is position-free (improvise-hkbc): it names the record
by surrogate and the new coordinate by value. `records_named` names the record,
because a rename moves it (gotcha 43). The destination, when it exists, is
added by the server's `records_at_stake` rather than here, because only the
document can derive it (improvise-milr, `context/code/command.md`). Creating the item is internal to the
effect. `render.rs::editable` still answers `false` for a coordinate: the page's
inline editor writes stored cells only, and the rename's producer is the
keyboard path (`context/code/clients.md`). `typed_cell_command` and
`record_cell_at_cursor` still answer `None` for it, so paste onto a coordinate
is still improvise-kkcy.
### `CellDisplay::id` — `None` is not "aggregate"
That is `Some(CellId::Aggregate(..))`, and the distinction is the whole point
of the enum. `None` means the slot holds **no projected cell**: a `GridCache`
position padded by `fit_to_labels`, or one blanked by a reframing. Every
records-mode column is some kind of cell (`Record`, `Derived`, `Create` or
`Coordinate`). Conflating the two either hides a cell or offers a
records-mode column is some kind of cell (`Record`, `Derived`, `Create`,
`Coordinate` or `NonItem`). Conflating the two either hides a cell or offers a
write the type system exists to make unconstructible.
`NonItem { category, item }` is a stored value whose `category` coordinate
names no item of it (improvise-gk51). It is drawn, and it takes no write. The
host would refuse one, so it is not offered as a `Record`. Why it is a kind of
its own rather than one of the others: `context/code/command.md`, "The
projection asks the first rule too".
### `CellDisplay::edit_text` — the difference corrupts data
What an editor opened on this cell should start with: the value as the user
@@ -923,8 +964,8 @@ type and there is nothing else to ask. Note what this cure cost that
shared derivation could not be added beside the old one.
`cats` is the categories outermost-first; `entries` interleaves
`Group(FrameGroup)` headers with `Item(Vec<String>)` label tuples exactly as
`GridLayout::row_items` interleaves them. On the row axis the TUI paints a
`Group(FrameGroup)` headers with `Item { tuple, collapsed }` label tuples exactly
as `GridLayout::row_items` interleaves them. On the row axis the TUI paints a
`Group` as its own row; on the column axis it paints the runs as spans above
the headers. **That difference is painting, and it is the only thing that
differs.**
@@ -935,7 +976,20 @@ one**. `AxisFrame::flat` leaves `cats` empty on purpose, so it is not a way to
smuggle a multi-level axis in flattened.
`FrameGroup::collapsed` is on the wire rather than left to the client because
it is a fact about the *view*, which only the server holds. There is no
it is a fact about the *view*, which only the server holds.
**`Item::collapsed` is on the wire too, and for a sharper reason** (improvise-4k1j).
It is the layout's own `AxisEntry::DataItem::collapsed`, which `GridLayout`
sets on a fold and on every item expanded beneath one, at any level. A client
cannot derive it from the headers. `FrameGroup` names no level, and an inner
`GroupEnd` looks the same as an outer one, so under two row categories the
nearest header can be the inner category's. That is how `H` came to hide a
group name as an item. Tuples cannot answer it either, because a group and an
item may share a name. `AxisFrame::folds(idx)` reads the flag, and it answers
exactly what `GridLayout::row_folds_a_group` answers, because it is the same
bit. improvise-e265 puts a level on headers and ends, which makes per-level
membership derivable. The flag still stays: a client reading the host's fact is
stronger than one re-deriving it, however correct the derivation. There is no
`FrameGroup::cat` any more (improvise-7xyx). Its doc comment promised that a
client echoes it back in `ToggleGroup`, but no client ever did. `resolve.rs`
has no `toggle-group-under-cursor` arm (`z` is on the still-needs-the-host
@@ -959,8 +1013,8 @@ future renderer and a measured regression.
every call. It is deliberately not made cheaper, because a fast
per-position form is an invitation to the loop `header_labels` exists to
replace.
- `group_of` walks `entries` twice over, once to place `idx` and once
backwards.
- `group_of` is `groups()` at one position, so it walks the whole frame
(improvise-e265).
The one-pass forms a draw loop wants:
@@ -984,9 +1038,10 @@ drawing the *wrong* header, not a shorter one.
always shows its full tuple, or a scrolled viewport loses its group context at
the top edge (improvise-jjx).
**`body_from(idx)`** drops the group header that owns the first drawn item: a
frame opening mid-group carries that header so `group_of` can answer for it,
but it is *above* the first drawn row and must not be painted as one. This is
**`body_from(idx)`** drops the group headers that own the first drawn item: a
frame opening mid-group carries the open run at every level so `group_of` can
answer for it (improvise-e265), but those headers are *above* the first drawn
row and must not be painted as rows. This is
the same cut `GridLayout::data_row_to_visual` has always made.
Two things about its argument, and the first of them **used to be the whole
@@ -1390,8 +1445,8 @@ on *entering* a group and nothing on leaving one, so an ungrouped item after a
grouped run inherited it (`improvise-gd83`). The §3 paragraph that recorded the
old rule — and forbade this change — is corrected there, not merely superseded
here. `AxisEntry::GroupEnd` is
the boundary, carried onto the wire as `FrameEntry::GroupEnd`; `groups()` clears
`current` on it and `group_of` stops walking. It is emitted only on
the boundary, carried onto the wire as `FrameEntry::GroupEnd`; `groups()` closes
its level on it, and every deeper one. It is emitted only on
`Some -> None`, never between two groups — the next header already ends the
previous run — and at the end of an expansion whose last item was grouped, which
is the across-prefix arm `cross_product` creates.
@@ -1399,8 +1454,48 @@ is the across-prefix arm `cross_product` creates.
Nothing draws a `GroupEnd`. Both front ends skip it and it consumes no data
index, so it is invisible except through `group_of`/`groups`.
**Runs nest, so both markers name a level (improvise-e265).** `FrameGroup`
carries `level`, an index into `AxisFrame::cats`, and `FrameEntry::GroupEnd` is
`GroupEnd { level }`. `groups()` keeps one open run per level: a group at L
opens L and closes deeper runs, and a `GroupEnd` at L closes L and deeper. Each
position's answer is its **innermost** open run. So with `Region` over `Month`,
`North/Mar` after `Q1` ends is in `Upper`, and it was in nothing while a flat
reading let `Q1`'s end close `Upper` too. The user ruled per-level membership,
and that `z` toggles the innermost group. The layout reads its markers by the
same rule (`context/code/core.md` §6), and
`the_frame_agrees_with_the_layout_on_every_rows_innermost_group` holds the two
together at every window offset.
The level goes on the `GroupEnd` as well as on the header because the markers
are cut by three producers (expansion, pruning and windowing), and a marker
that names its own level can be read correctly inside any fragment of the
sequence. The header's level also gives a client the group's category,
`cats[level]`, which a browser `z` needs to compose `ToggleGroup`. That is not
the dead field improvise-7xyx removed: `level` is read by `groups()`, and the
category name is not carried twice.
**Within one category runs nest by `depth` (improvise-felo.6).** A group's
parent (`> Q1[2024]`) puts the quarter's run inside the year's on the SAME
level, so both markers also carry `depth`, the group's number of ancestors.
A run's key is `(level, depth)`, compared in that order: a header at a key
closes every run with a greater key, and so does a `GroupEnd`. With `Date`
alone on an axis, `2024-03` after `Q1-2024` ends sits in `2024`, and after
`2024` ends in nothing. Depth rides beside `level` rather than being folded
into one flat index, because a flat index would need every category's tree
height to be known before the first marker, and `level` would stop naming
`cats[level]`. Both front ends draw a nested header exactly as they draw any
other: one inline row per `Group` entry, the innermost group in the column
band. `the_two_front_ends_draw_the_same_nested_hierarchy_on_rows` and
`..._on_columns` in `tests/grid_frame_agreement.rs` hold them together, and
`the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy` does the same for the
windowed frame against the layout.
`group_runs` is unchanged. It coalesces whatever `groups()` answers, by header
identity (improvise-h0iv), so the band still paints one group per position, now
the innermost. One band per level would be a separate feature.
**`AxisFrame::first` is the absolute data index of the frame's first item.** The
windowing rule carries the nearest header *above* a scrolled window so
windowing rule carries the open header at each level *above* a scrolled window so
`group_of` can answer, and `body_from` drops it on the way to the screen. At the
top of an axis there is no "above", so that same leading header is content — a
collapsed first group has no other affordance (`improvise-0o6x`). `idx` alone
@@ -1473,6 +1568,41 @@ boundary is a matter of taste; a wrong collapse glyph is a wrong fact. So: two
runs. Reversing it is one comparison, and the test is renamed
`column_groups_coalesce_per_header_as_the_tui_does` rather than deleted.
## §3b. A frame's levels are checked as it arrives
A `FrameGroup.level` and a `GroupEnd`'s `level` name a category: they index
`cats`. `AxisFrame::groups` used to keep one open run per level and size that
stack with `open.resize(g.level, None)`, so a level arriving over the wire
decided how much it allocated. A frame whose level is `4294967295` would ask for a
multi-gigabyte vector, and the client would be taken down instead of the frame
being declined. ChatGPT Codex found this on #250, the PR that gave levels to the
wire (improvise-e265).
Since improvise-felo.6 `groups()` keeps only the headers it has received,
ordered by `(level, depth)`, so no number on the wire sizes anything and the
new `depth` field needs no bound. The level check stays for its other reason:
`cats[level]` names the group's category, which a browser `z` reads.
The bound is enforced at deserialization, by `#[serde(try_from =
"UncheckedAxisFrame")]`. That is the one place a frame enters from outside. A
level that is not below `cats.len()` fails to parse, with an error that names the
level. A malformed frame therefore fails loudly: `Client::on_frame` returns the
error to the page, the same way it returns any other frame that does not parse.
It is not clamped, because a clamped level would be a guess about which group
the server meant, and a wrong group is a wrong fact.
Three alternatives were weighed and not taken:
- **Bounding inside `groups()`.** The same allocation would still happen in any
other reader, and a frame that is wrong would pass through silently.
- **Checking only the first marker.** A frame can be well-formed up to its last
group, so the check walks every marker. The test puts the bad level on the
second marker for that reason.
- **A frame built in Rust is still unchecked.** The fields are public, and the
producers build levels from `AxisEntry`, whose levels are category indices by
construction. The hazard this closes is the wire, where the input is not
ours.
## 4. `frame.rs` — one envelope pair, two deployments
### The framing adds as little as it can get away with
@@ -1997,7 +2127,107 @@ edit are the same string **by construction** rather than by agreement
(improvise-k8h) — the keymap never spells a buffer name.
`commit(key, cmd, args)` welds the command to its `clear-buffer` so no caller
can bind a commit without its clear (improvise-w3q). `rebind` exists for
can bind a commit without its clear (improvise-w3q).
**The two cell-editing commit keys are the exception, and the clear moved rather
than disappeared** (improvise-fy9o). `commit-cell-edit` and
`commit-and-advance-right` reopen an editor on the cell they advance to, and the
welded clear ran **after** them, so it blanked the re-seeded buffer. The
reopened editor looked empty, and a second Enter committed `""` and cleared the
landed cell. The user's ruling is that the reopened editor shows the landed
cell's current value, so with improvise-ixxx's untouched rule an unchanged Enter
writes nothing. `commit_reopening(key, cmd, args)` therefore binds the command
alone, and the command clears its own buffer **at its head**:
- in the TUI, `CommitAndAdvance` does it right after the mode change;
- in the browser, `resolve::cleared` does it, reached from both `commit` and
`create_column`.
Being at the head is also gotcha 21's rule: a head clear survives an abort
where the trailing weld did not. The corner commit and the `+cat` insertion
reopen nothing, and they still leave no text behind.
**After a write, the browser's reopened editor takes its seed from the host's
answer, not from its cache** (improvise-3uzg). The first fix (improvise-3tt4)
seeded from the cache everywhere except the cell the frame had just written.
That assumed the write changed the meaning of only that one cell, which is
false. Under `prune_empty`, emptying the last value in a row takes the row
away. The advance then lands on a renumbered position, and the cache still
holds the old record there. The browser opened on South's `2` while the TUI
opened on West's `3`, and typing `5` wrote 25 into West. Any write can change
what another position means, and the client cannot tell which. So the rule is
stated once, over the write, never over a list of cases:
- **A commit that writes** reopens its editor as `OpenedEditor::awaiting()`, an
empty buffer whose seed is still to come. `create_column` is included,
because `AddItem` adds a column.
- **`ClientState::settle_awaited_seed`** fills the seed from the grid the
client holds, but only once no frame it sent is still unanswered. The answer
folds the host's projection before `Accepted`, so the grid is the host's by
then. `accepted()` therefore settles.
- **A commit that writes nothing** awaits as well while any earlier frame's
write is unanswered. Otherwise it seeds from the cache at once. "Nothing
changed, so the cache is current" was the rule here, and it was false
(improvise-mdfc). *This* commit changed nothing, but an earlier one might
have, and the cache predates it until the answer arrives. Tab Tab pressed
quickly at the rightmost column opened the second editor on the value from
before the first Tab's write: 505 where the TUI stores 5095. So the test is
"does this frame write, or is a write still in flight", which is
`writes || in_flight.iter().any(|sent| sent.awaited)`. It checks *any*
in-flight write, not the newest frame: a frame that writes nothing stays in
the queue behind an unanswered write, so the newest frame can be one of
those.
- **Every opener asks the in-flight half, because it lives in `opened_on`**
(improvise-p6v6). mdfc first put the whole test inside `advanced`, which left
`i`/`a` (`edit_or_drill`) seeding from the cache with no in-flight check.
`i 9 Tab Esc i 5 Enter`, typed faster than the answer, stored 15 in the
browser where the TUI stores 195. The two halves now live in different
places, for a reason. "A write is still in flight" is true of the *client*,
so `opened_on` asks it, and no opener can forget. "This frame writes" is
true only of the commit being resolved, which is not in `in_flight` yet, so
`advanced` still asks it itself. Folding both into `opened_on` would lose the
second: the 3tt4 and 3uzg tests fail when `advanced` defers entirely.
The click-edit path (`socket.rs::edit_target_at`, improvise-ugj4) also seeds
from the cache, but it claims the record at the version it *read*. Behind an
unanswered write the host therefore refuses its commit as `RecordMoved`: a
loud failure, not a silent overwrite.
- **The seed settles only into an open editor** (improvise-4cb5). An Esc typed
before the answer arrives leaves the awaiting `opened_editor` in place. The
late seed would then fill the edit buffer in `Normal` mode, where the TUI's is
empty.
- **`awaiting` has no `serde(default)`** (improvise-mgis), for the reason
`AxisFrame::first` has none (§3a, improvise-1wl0). `awaiting: false` is a
claim, "the seed is settled". A peer that dropped the field would hand over an
editor that stays empty forever, and whose Enter writes nothing. A missing
field is therefore a parse failure.
Asking the host to send the seed was the alternative. It would have needed a
new message, and the host already sends it: the landed cell's `edit_text` in
the projection it owes this client anyway. Two things are accepted costs:
- Until the answer arrives, the editor is empty. Over a worker that takes
well under a frame.
- A key typed before the answer arrives marks the editor touched, and the late
seed then does not overwrite it. What the user sees is then what gets
written. That is never another record's number only because of the mdfc
rule above: before it, an untouched commit behind an unanswered write opened
on the stale cache, and typing then extended the wrong number. What is left
is a divergence from the TUI rather than a wrong record. A key that beats the
answer replaces the value the TUI would have extended: back to back,
`i 9 Tab Tab 5 Enter` stores 5 or 5095 in Chromium over the worker
(improvise-n4i9).
The TUI is unchanged: its `RebuildLayout` in front of `EnterEditAtCursor`
already reads the renumbered grid (gotcha 23). This also took
`CellValue::edit_text` out of the client bundle. That call formats an `f64`,
which is what compiled to `i64.trunc_sat_f64_s`, so `a093d126`'s
`nontrapping-float-to-int` flag on the wasm-client is reverted. A `?ws=` page
loads only the client bundle, so it works without the feature.
Reordering the weld to `[clear-buffer, cmd]` was tried first and rejected. Four
readers take a `Sequence`'s **first** step as its identity: `keymap::runs`,
`help.rs`, and two resolver audits. So every hint on those keys started
reporting `clear-buffer`. `text_entry_modes_clear_the_buffer_they_type_into`
keeps its guarantee by asking a plain-`Cmd` commit key's command what it emits.
`rebind` exists for
exactly one case: CommandMode's Backspace, which exits the mode on an empty
`:` line instead of popping a character that isn't there.
+549 -14
View File
@@ -259,6 +259,245 @@ the deploy's and pass. `README.md` therefore states the rule and deliberately
**does not list the jobs**: a second copy with nothing holding it to the first
is what went stale, twice.
It holds a second rule of the same kind: every suite `scripts/checkers.sh
--list` names must run in some CI step (improvise-dak5). `ci.yml` names its
suites by hand, because they need different jobs and environments (`kb` runs
under a nix trealla, `node-test` in the `js` job), so a single bare
`checkers.sh` was not an option. The check does not re-derive what a suite is.
It asks `checkers.sh --list`, so the naming rule keeps its one home.
**What CI runs is a table in `checkers.sh`, not something read out of shell**
(improvise-st8q, improvise-tvte). Two rounds of review each found the gate
passing while a suite never ran, and every finding was the gate
*interpreting* `ci.yml`. It read `--list` as "run everything". It counted the
names after `--self-test`. Then it read a bare `scripts/checkers.sh` inside a
heredoc, a folded `>` value or a multi-line string as a command, and it ignored
`if:`. Each fix added parsing, and shell and YAML always had another form. So
the parsing went. `checkers.sh` holds a job→suites table, `ci_suites`, and each
CI job has exactly one step whose whole `run:` value is
`./scripts/checkers.sh --ci <job>`. A `nix shell <pkg> --command` prefix is
allowed. `--ci` runs that row and nothing else, and `--ci` beside a suite name
or `--self-test` is a usage error. `checkers.sh --would-run --ci <job>` reports
the row without running it, so the table has one home.
The gate reads no shell. It counts a step only when all of these hold:
- the step sits in a job's `steps:` sequence, at step-key indentation;
- its `run:` value is exactly that call, on one line. A plain scalar folds a
deeper-indented next line into the value, so `|| true` there would become part
of the command. A `run:` that continues onto another line is not counted
(improvise-1be2);
- the call names its own job;
- neither the step nor the job carries `if:`, `continue-on-error:` or `shell:`;
- no job it `needs:`, directly or through other jobs, carries a job-level
`if:`. GitHub skips a job whose need was skipped, so a false `if:` two jobs
away skips the call as surely as one on its own job (improvise-bbwa).
The only YAML it models is what makes a line structure rather than content. A
key with an inline value owns every following line indented deeper. So a
heredoc, a folded or plain continuation, or a whole fake job inside a
top-level block scalar is skipped as content. Every other line's value must be
one of a few shapes that end on their own line, or the line is refused
(improvise-0wn3, below). Any line outside a YAML comment that mentions
`scripts/checkers.sh` and is not a counted step is **reported**, not ignored.
The union of the counted jobs' rows must cover `checkers.sh --list`.
**A key the gate cannot read plainly is refused, not parsed** (improvise-w0j4).
YAML has many spellings of one key or one list. `"if":`, `'continue-on-error':`,
`if :`, and `? if` then `: false` all mean the same key. `needs: ["gate"]`,
`needs: 'gate'`, `- "gate"`, `needs: &d [gate]`, and a flow list split over
two lines all name `gate`. The gate read none of them. Each one skipped `kb`
on a PR while the gate reported every suite run.
Teaching it each spelling is the treadmill st8q stepped off. So every
structural line inside `jobs:` must take one plain form, or the gate names the
line and fails:
- a top-level key;
- a job name;
- a job key;
- a step key;
- a `needs:` value that is empty (a block list follows), one bare name, or a
one-line flow list of bare names;
- a block-list item that is a bare name.
Content under a job key other than `steps:` and `needs:`, such as `env:` or
`strategy:`, is not structure the gate reads, so it is not checked.
**But a line is content only once its job has a 4-space key** (improvise-rjaa).
The first version accepted any line deeper than 4 outside `steps:` and
`needs:`. So a *needed* job whose own keys sat at indent 5 or 6, which is valid
YAML, had its `if:` and `needs:` read as content. The gate exited 0 with `kb`
skipped. Now a job's first key must be at indent 4. After that, a deeper line
can only be the content of an empty-valued key, because a key with an inline
value already owns every deeper line.
Three valid styles are refused loudly rather than read:
- an indentless block list (`needs:` then `- gate` at indent 4);
- a block list at indent 8;
- a quoted top-level key such as `"on":`.
Each exits 1 and names its line. Spell it the plain way.
**A ` #` on any line the gate reads is refused, and every matcher reads the
same string** (improvise-j49r). The gate used to strip a trailing comment for
the readability check and match the raw line everywhere else. So
` gate: # skipped on purpose` was accepted as a plain job header by one and
never opened as a job by the other. Its `if: false` was credited to the job
before it, and `read_graph` did not list `gate` at all. Both rules passed with
`kb` skipped and the deploy ungated by `gate`. An audit of every match found
the stripper wrong in a second way: it cut `A: "x # y"` to `A: "x`, the quote
tracker then held the next lines open, and a job-level `if:` after it vanished.
That also exited 0 on the real `ci.yml`.
There were two ways to make the matchers agree:
- **strip everywhere.** A correct stripper has to know where a quoted value
starts and ends, because `"x # y"` has no comment and `"x" # y` has one. That
is parsing again, the thing st8q and w0j4 gave up.
- **Refuse the comment.** This was chosen. Once every line containing ` #` is
refused, the stripped and raw forms of every accepted line are the same
string, so no two matchers can read different versions of it. So the
stripper is deleted, not kept consistent.
This refuses some valid styles, on any line that is not block-scalar content:
- a trailing comment on a step, such as `- uses: actions/checkout@v7 # pin`;
- a one-line `run: echo "a # b"`;
- a quoted `with:` value containing ` #`.
This costs future readability, not correctness. The real `ci.yml` has none of
them. A full-line comment is still skipped, and a `#` with no space before it,
as in `nixpkgs#trealla`, is not a comment. After a refused line only the exit
is guaranteed: what the gate reads next may be garbled, and the exit is 1
either way.
**A quoted value that does not close on its own line is refused, and the quote
tracker is deleted** (improvise-6zf1). The tracker held a multi-line quoted
value open until the next line containing its quote character. YAML does not
close on an escaped `\"` inside double quotes, or on a doubled `''` inside
single quotes. So the tracker closed early, a later content line such as
`b: "` opened a phantom string, and that string swallowed the job's real
`if:`. Both quote styles exited 0 on the real `ci.yml`. This was the fourth
review round whose fail-open lived in quote or comment handling. Closing the
tracker correctly would have meant modelling YAML's quote rules one more time,
so the gate stops reading multi-line quotes at all. The real `ci.yml` has
none.
The opening line is still judged by `_opens_quote`, and that check is now
load-bearing: a value it wrongly calls closed is not refused, and its
continuation lines are read as structure. A `name: "a\"` followed by
`needs: ok` would then overwrite a job's real `needs: gate`. So `_opens_quote`
drops `\\.` before looking for the closing `"`, and `''` before looking for
the closing `'`, and a case covers each.
**Every value the gate reads must be one of a few shapes, each ending on its
own line** (improvise-0wn3). The 6zf1 fix refused a multi-line *quote* and
left a multi-line *flow* value (`[`, `{`) alone. The argument for leaving it
was that a flow continuation line is read as a key, so it fails closed. That
argument was wrong. It covered the closing line it imagined, not one written
to look like a step key:
```
- name: [a, "b
run: ./scripts/checkers.sh --ci kb
c: x"]
```
go-yaml, PyYAML and Psych all read this step as one `name` string with no
`run:`. The gate read the middle line as the step's `run:` and counted it. So
did the single-quoted form, a flow map, a flow opened on a line of its own, and
an anchored or tagged flow (`&a [x, "b`, `!!seq [x, "b`). All exited 0.
This was the fifth review round in a row, and each fail-open was a construct
the gate did not *recognise* but also did not *refuse*. So the check is now an
allowlist over values, not a list of refused constructs. `_line_value` takes the
value a line carries: a key's, a list item's, or a whole line of its own. That
value must be one of:
- empty (a block follows);
- a block-scalar header (`|`, `>`, with chomping and indentation indicators),
whose body the owner rule skips;
- a quoted scalar that closes on the same line (`_opens_quote`);
- a one-line flow with no quotes, no nesting and no `#` inside it, such as
`needs: [test, wasm]`;
- a plain scalar that does not start with a YAML indicator (`[ ] { } & * ! | >
% @ \` ? ,`).
Anything else is refused, whatever it is. The real `ci.yml` needs nothing
more: 153 plain values, 41 empty, 2 one-line flows and 5 block headers, with 0
refused.
The obvious narrower fix, "a flow value must end in its closer", is not enough.
`[a, "b]` ends in `]` while its quote is still open, and a case pins that.
This is an allowlist over *values*, and it applies on every line the gate
reads, at any depth. The *keys* at indent 8 or less inside `jobs:` were already
allowlisted by `_readable` (w0j4, rjaa). A deeper key is not shape-checked,
because its value is. So a quoted key deep in a job, such as `"FOO": bar` under
`env:`, is now refused, because the gate cannot tell it from an unclosed quote.
That is a valid style refused loudly, beside the others in this section.
**A line holding a line break YAML knows and the gate does not, or a control
character, is refused** (improvise-xpzh). The gate splits the text on `\n`
only. YAML also breaks lines at U+0085 (NEL), U+2028 and U+2029, so
`runs-on: x<U+2028> if: false` was one line to the gate and two to go-yaml.
The job's `if:` was invisible, and the gate exited 0. The same break inside a
full-line comment, or inside a `run: |` body, ends that line for YAML too. A
body line `echo a<U+2028> if: false` makes the `if:` a real job key.
So the check runs on **every** line, before the comment and owner skips, and
refuses:
- C0 controls other than tab (CR included);
- DEL;
- the whole C1 block, which contains U+0085;
- U+2028 and U+2029.
CR is the one break the read folds. `read_workflow` opens with Python's
universal newlines, which turn CR and CRLF into `\n`, and a lone CR is a line
break to YAML too, so the two agree. A CR that survives to the split can only
come from a read that did not fold it, and it is refused. Two self-test
controls write real files, one CRLF and one lone CR, and read them back
through `read_workflow`. The code path is the point: a string handed straight
to `suite_verdict` never meets the fold.
**Why not refuse all non-ASCII,** which would be the simpler allowlist. The
real `ci.yml` has 27 non-ASCII characters (em dash, `§`, arrows) in its
comments, so it would have to be rewritten to pass. The set of characters that
changes line structure is not open-ended, though. It is YAML's own `b-char`:
LF and CR, plus NEL, LS and PS in YAML 1.1. So refusing that set, and the
controls around it, is complete by the spec's definition, not a list that
grows with each review.
`read_graph` refuses nothing, so it cannot rely on refusal. It opens a job on
any ` name:` line followed by whitespace or the end of the line, which is
wider than the exact header `ci_steps` accepts. Reading too wide only adds jobs
`pages-deploy` must need, which fails closed. Reading too narrow is what lost
`gate`.
**A `defaults: run: shell:` does not disqualify a step** (the user's ruling,
PR #228). An earlier version refused to count any step under a workflow- or
job-level `defaults:`, because a default shell can change what the call runs
under. The user called that "a silly check". It also broke on the day
`ci.yml` gained a real one: PR #287 set a workflow-level
`shell: nix develop .#ci --command bash …`, and every suite then read as run by
no CI step. A default shell is how the workflow chooses its environment, not a
way to skip a call. A shell that failed to run the call would fail the step, red
in CI. A step's own `shell:` still disqualifies it. That rule was not part of
the ruling.
**What this does not close, stated rather than guarded.** Once a counted step
is reached, the gate trusts that it runs the script it names. An earlier step
in the same job that rewrote `scripts/checkers.sh` or its table would defeat
that. So would an earlier `actions/checkout` step with a `ref:`, which
replaces the tree, so the counted step runs another revision's `checkers.sh`
and its table. Shell control flow *around* the call cannot hide in the counted
step, because its value is exactly the call. A needed job that *fails* also
skips the job, and the gate does not model that. The failure is itself red in
CI, and it is why `pages-deploy` must still `needs:` every job.
**Two claims earlier versions of this section made were wrong.** "`--would-run`
anywhere but first is a usage error" was false: `checkers.sh --list
--would-run` exits 0 at `--list` before it reaches the second flag. And "a line
`shlex` cannot parse fails closed" was true of that one line, but read as a
property of the gate, which at that point still passed on heredocs and folded
values.
### pr-watch and mergeability (improvise-edpl, improvise-zlj2)
`scripts/pr-watch` is what the orchestrator's merge procedure stands on, and
@@ -278,7 +517,29 @@ The rule now:
exit 1.
- **A known answer ends the wait early**: a failed check (exit 1 whatever
mergeability says, since waiting would turn a known red into a timeout), or
`mergeable == CONFLICTING`.
`mergeable == CONFLICTING`. "Whatever mergeability says" includes
`MERGEABLE/BLOCKED`, which is how a failed *required* check reports here, so
it is the commonest red this script sees (improvise-9pfs).
- **A PR that is no longer open ends the wait on the first poll that sees it
closed, with exit 5** (improvise-1qaa). That poll is the first when the PR was
already merged, and can come mid-wait when someone merges it during the watch.
A merged PR reports `UNKNOWN/UNKNOWN` permanently (#189, read live), so the
UNKNOWN wait slept the whole timeout there and then blamed mergeability. A
closed PR's mergeability is whatever it was when it closed, so it can land on
either side of the wait. #74 reads `CONFLICTING/DIRTY`, which ends the loop at
once, but as a misleading exit 1. #46 has green checks and reads
`MERGEABLE/BLOCKED`, which the loop waits on, so it slept the full timeout
too (improvise-29w2). The suite carries all three shapes, and only the
waited-on ones can tell whether the state is checked at the top of the loop.
The script now fetches `state`. Exit 5 is
its own code, not 1 or 0, because under merge-as-soon-as-green "someone else
already merged it" is a normal answer, and the caller should hear it as that.
- **The head checks come first, and the ended check after them**
(improvise-24lz). A PR that merged at a head the watcher never saw is the
PR #67 hazard in a new form. It exits 2, whether the head moved during the
watch or `--expect` names another. Only then is a closed PR exit 5. On the
first poll the pin block only records the head, so an already-merged PR
still ends on its first read.
- **The pass condition is an allowlist**: `mergeable == MERGEABLE` *and*
`mergeStateStatus` in `CLEAN`, `UNSTABLE` or `HAS_HOOKS`. Anything else exits
1. That includes `null` and any state GitHub adds later, so the check fails
@@ -638,30 +899,47 @@ sixteen scripts, and the review cost is the whole diff rather than the shebang
line. The runner also does not ship zsh, so every job would need an `apt-get`.
Declined.
What was adopted instead pins the *interpreter and the environment together*, in
What was adopted instead fixes the *environment* a suite runs in, in
one place, with no per-invocation nix cost:
**`scripts/checkers.sh` is the one entry point for every check_the_checker.** CI
calls it and the local gate calls it, so there is no second invocation to drift.
It does three things a bare `./scripts/X-test.sh` does not:
It runs every suite under the bash that runs it: in CI and under direnv, that
is the dev shell's bash. It does two things a bare `./scripts/X-test.sh` does
not:
1. **Pins a bash ≥ 5.** The ambient one is tried first, so a runner that already
has bash 5.2 pays nothing; nix supplies one only when the ambient bash is
older, which on macOS outside the dev shell it is. No bash ≥ 5 and no nix is
**exit 3** — a refusal, never a quiet run under 3.2.
2. **Scrubs the environment to an allowlist** (`env -i` plus `PATH`, `HOME`,
1. **Scrubs the environment to an allowlist** (`env -i` plus `PATH`, `HOME`,
`TMPDIR`, the `NIX_*` and `GITHUB_*` names, extensible through
`CHECKERS_KEEP`). This is the part that closes the direnv hole above, and it
is a *positive* list on purpose: the failure mode of a missing name is a loud
"command not found", where the failure mode of an extra name is the silence
this section is about.
3. **Enforces the shebang rule**, so the `sh`/`bash` split cannot come back.
2. **Enforces the shebang rule**, so the `sh`/`bash` split cannot come back.
The pinned bash's directory goes on the front of `PATH`, so a script the suite
invokes through its *own* `#!/usr/bin/env bash` — or through `sh`, which
nixpkgs' bash also provides — gets the pinned one too. Pinning only the outer
shell would have left every inner one on the system bash, which is where the
fault actually lived.
**The runner no longer polices the bash version or where tools live.** It used
to pin a bash of at least 5.1 (`find_modern_bash`: try the ambient bash, fall
back to `nix shell nixpkgs#bash`, and exit 3 if neither was available). The
floor was 5.1 because `pr-reviews-test.sh` uses `wait -n -p` (improvise-fd0j).
All of that was removed on the user's ruling, when CI moved to the NixOS
runners: "we shouldn't care which version of bash and other standard utilities
are running in CI", and "most jobs should just take what comes in the ci dev
shell".
So every CI step runs inside `nix develop .#ci`, which provides the bash and the
GNU coreutils. The runner calls suites with `$BASH` and an unchanged `PATH`. A
script that needs a newer bash feature fails where it uses it, not at a version
gate.
Two things went with the floor:
- **Its self-tests.** `checkers-test.sh` sections 2, 5 and 5b faked old
bashes, or stripped `PATH` down to hand-picked utilities, to prove the floor
held. Those were the tests that broke on the NixOS runners, where `/usr/bin`
and `/bin` hold only `env` and `sh` (#296, improvise-3uow).
- **The hard-coded tool paths** in the suites (`/bin/sleep` and `/bin/date` in
`pr-reviews-test.sh`) and `scripts/kb`'s BSD `stat -f` fallback. `scripts/kb`
already needs the dev shell for `tpl`, so GNU `stat` is always present when it
runs.
**A suite is `X-test.sh` where `X` or `X.sh` is present beside it.** That rule
exists because `scripts/node-test.sh` matches `*-test.sh` and is a test
@@ -706,3 +984,260 @@ configuration.** A script may declare `#!/usr/bin/env bash`, pass the gate, and
still be handed to dash by whatever invokes it. The only defence there is
reading, which is how this one was found — by grepping for the *call* after
fixing the *declaration*, rather than stopping at the gate going green.
## 6. The harness lock gate (improvise-zs2r)
`harness/opfs-contract` declares its own `[workspace]`, so it has its own
`Cargo.lock`. It reaches `improvise-io`, and through it `improvise-core` and
`improvise-formula`, by path. So a change to one of those crates' manifests,
such as removing a dependency or a feature, can stale that lock while the root
lock is updated and every host gate stays green. Before this gate, the only
thing that noticed was `harness/build.sh`'s `cargo metadata --locked`, in the
slow `build` job, and #214 and #224 both found out that way.
`scripts/check-harness-lock.sh` runs that same `cargo metadata --locked`
against the harness manifest. It runs in the `test` job (with
`--allow-network`) and locally, where it is **offline** by default.
**Why offline is sound, measured rather than assumed.** `--locked --offline`
resolves from the local registry cache and fails on a real mismatch with its
own message: "cannot update the lock file … because --locked was passed". A
dependency missing from the cache fails differently, with a resolution error
naming the index ("location searched: crates.io index"). Both were measured:
a manifest-only change to `improvise-formula` produced the first, and an empty
`CARGO_HOME` produced the second. So the script can tell the two apart, and it
does:
- exit 0: the lock is current;
- exit 1: stale, and the fix is named;
- exit 2: the gate did not run.
That follows `fmt-workspaces.sh`'s rule that "did not run" is never collapsed
into a verdict. Offline, then, it never needs the network and never lies:
- a developer whose cache is warm, which is anyone who has built the root
workspace, gets a real answer;
- one whose cache is cold is told the gate did not run, not that the lock is
stale.
**The one residual, stated.** A manifest change that adds a crate never
downloaded is, offline, a resolution failure. So it reads as exit 2, not
exit 1. The developer is still stopped, but told the gate did not run rather
than that the lock is stale. CI runs with `--allow-network` and answers it
properly.
`scripts/check-harness-lock-test.sh` pins the three answers against path-only
fixture workspaces, which need no registry at all. The fixtures have the same
shape as the harness: a root that reaches a crate by path, whose manifest then
changes. Two rivals were run:
- reading every failure as stale fails the offline case (exit 1, wanted 2);
- dropping `--locked` fails the stale case (exit 0, wanted 1). Without
`--locked`, cargo quietly rewrites the lock and reports success.
## 7. The repo-stats pipeline (improvise-1xfb.5.1)
`scripts/repo-stats` reports churn, change coupling and doc growth over a window
of history: by default the last 7 days of `gh/main`, falling back to `HEAD`,
without merges. It reports and never gates. The user asked for code-maat to do
the analyses it has, and for improvise to be the engine for the rest. So the
pipeline is also an end-to-end run of improvise's headless product: `import`,
`script` and `export`.
### The shape
1. `git log --numstat --no-renames --no-merges` in code-maat's `git2` format.
2. code-maat runs `coupling` (with `--min-revs`/`--min-shared-revs`), `abs-churn`,
`age` and `soc`. Each writes one CSV into the output directory, named after
the analysis. code-maat is pinned in `flake.nix` as a `fetchurl` of the
1.0.4 standalone jar, run by a JRE. It is in the dev shell and in the
`.#repo-stats-tools` package, but not in the `ci` shell: the JRE is about
0.6 GB on Linux, and every CI job enters that shell.
3. The script adds `churn.csv`, one row per (commit, file) with the date, the ISO
week, the path (`entity`, code-maat's own name for it), the area, the kind
(prose, code or other), and the lines added and deleted. It also adds two
flags to code-maat's `coupling.csv`: `crosses-crate` and `agent-rules`.
4. `improvise import <dir>` (improvise-lpzu) merges the directory's five CSVs
into **one** sparse model, `repo-stats.improv`. The merge's `File` category
names each source, and the import makes one view per file, named after its
stem and filtered to it. That is the user's ruling: "Records are already
sparse: the model can just leave null/empty values for columns not in a
given CSV file." A record keeps no coordinate in a category its file lacks.
5. `improvise script` runs the checked-in `scripts/repo-stats.improvise`. It
adds the prose-to-code formula, builds four views on top of the per-file
ones and `export`s each to `report/*.csv`. The four names (`By area`,
`By kind`, `Area by week`, `By day`) must never be a file's stem:
`create-view` of an existing name replaces that view silently (gap 17). The
suite's view count, 10, fails if one does. None of the report reads the
merged `Default` view, which can be empty for a typical code-maat pair
(improvise-8jk6.15).
6. The text report is those exports laid out as tables. The code-maat rankings,
doc sizes and the coupling flags are added to it. `--json` gives the same
data as JSON.
The `.improv` file is the artifact. It opens in the TUI, and CI uploads it with
the CSVs.
**A window in which no file changed builds no model.** Its commits are all
empty (`--allow-empty`), so there is no churn to import and nothing for
code-maat to analyse. The script writes `churn.csv` with its header alone, skips
code-maat and improvise, and the report says so. It exits 0. Running improvise
over an empty CSV was the other option. It was not taken, because it would test
only how improvise treats a CSV with no records, which is not what this pipeline
exists to exercise.
**`--out` must hold no CSV the script did not write.** `improvise import .`
merges every CSV in the directory, so a stale or foreign one would silently
join the model. The script refuses one with exit 2 and names it. It refuses
rather than deleting, because deleting files in a directory the caller named is
the wrong default. Its own five CSVs are overwritten on every run, so a rerun
into the same `--out` works.
**No collisions, by construction.** A `churn.csv` record is keyed by commit and
file, which `--no-renames` makes unique. code-maat's records are keyed by
entity, by entity pair or by date. `File` separates the sources. So the import
keeps its default, `--on-duplicate refuse`. A collision would be a bug in this
script, and the import refuses it.
**What improvise computes and what it does not.** Everything that is a sum or a
ratio of sums is a view:
- churn by area, with `DeletionRatio = deleted / added` as a `_Measure` formula;
- prose, code and other, with `ProseToCode = prose / code` as a formula on the
`kind` category;
- area by ISO week;
- lines added per day from `churn.csv` next to code-maat's `abs-churn`. The two
columns must agree, and the suite checks that they do.
Two kinds of output stay in Python. The ranked code-maat tables (coupling, soc
and age) are read from code-maat's own CSVs, because improvise cannot sort rows
(gap 11 below). Doc sizes are a property of files at one ref, not of the window.
### Definitions
- **Areas**: first match wins, in the order of `AREAS` in the script.
`generated` comes first, so `harness/opfs-contract/Cargo.lock` counts as
generated and not as harness. `.beads/` and `site/` are generated too.
`docs/*.md`, `README.md` and `RELEASING.md` are "other docs", and `.codex/`
is agent docs and skills. A Rust test is a `.rs` file under a `tests/` or
`benches/` directory, or one named `tests.rs` or `*_test(s).rs`. A
`#[cfg(test)] mod tests` inside a source file counts as Rust src, because no
path can tell the two apart.
- **Prose** is `context/code`, `context/test-evidence`, other `context/`, other
docs, `repo-map`, and the agent docs and skills. **Code** is Rust src, Rust tests,
scripts/CI, and web and harness. `kb.pl`, generated files and other files are
neither.
- **The deletion ratio** is deleted lines over added lines. It is 0 when nothing
was deleted, and blank when nothing was added.
- **A crate crossing**: both paths are in a crate (`crates/<name>/`, the root
crate's `src/`, `tests/`, `benches/`, `examples/` or `Cargo.toml`, or
`harness/opfs-contract/`), and the two crates differ. A pair with one side
outside every crate is not flagged.
- **Agent rules**: both paths are in the agent docs and skills area. Such a pair
means one rule stated in several places.
- **Gotchas** are the numbered items between `## Gotchas` and the next `---` or
`##`. The longest gotcha is measured in characters. Most gotchas are one long
line, so a line count would say little.
- **Test-evidence entries** are the `###` and `####` headings.
### What fails, and what never does
- **The numbers never fail anything.** No budget is checked. Budgets are a
separate decision under the compaction epic, improvise-1xfb.5.
- **improvise refusing the pipeline's input exits 3.** That covers a non-zero
exit from `import` or `script`, and any `warning:` on their stderr. The
warning rule is there because a skipped record is data loss that exits 0
(gaps 3 and 4). It is a product regression, not a statistic.
- **In CI**, the suite runs on every PR in `nix-build` and gates like any other
suite. The report runs only on a push to `main`. Its step has
`continue-on-error`, so an improvise failure there marks the step failed and
the job and the deploy go on. Whether that failure should block the deploy
is a question for the user on the PR. The report appends to
`$GITHUB_STEP_SUMMARY`, and the directory is uploaded as the `repo-stats`
artifact.
- The report runs in `nix-build` because that job already builds
`.#improvise`. Its steps run under `nix shell .#repo-stats-tools`, which joins
that same build with code-maat, so nothing is built twice. It is one package
rather than two installables because `check-ci-graph.py` reads a counted
step as `nix shell <one> --command`. The job's checkout fetches full history
(`fetch-depth: 0`) for the report.
### Gaps in the headless product, and the workarounds used
Each of these was found by running the pipeline, and each is a finding against
improvise's command and model layer. Where a workaround exists, the script
names it.
1. **A forced `--category` value is still parsed as a number.** The commit
`00565796` imported as item `565796`, and `220944e2` imported as
`22094400.0`. Workaround: `churn.csv` writes commits as `g<sha>`.
improvise-8jk6.4.
2. **`import --formula` drops a formula that fails to parse, and exits 0.**
`--formula 'X = ('` saves a model with no formula and says nothing.
`add-formula` in a script refuses it and exits 1, so the pipeline's own
category formula is added there. improvise-8jk6.5.
3. **A record that lacks any category's value is skipped.** A merge of CSVs with
different headers, which is the user's sparse model, imports **no records**.
Closed by improvise-lpzu, whose sparse import this pipeline now uses. Until
it landed, every CSV was padded with `-` for the categories it lacked. A
related hole is still open: a record with no measure value stores nothing
(improvise-8jk6.14).
4. **An import that skipped records exits 0**, even when it skipped all of
them. It saves an empty model and prints only a warning. improvise-8jk6.6.
5. **`set-axis` and `set-page` accept names that do not exist, and exit 0.** An
unknown category, an unknown axis name (`rwo`) and an unknown item are all
silent no-ops. improvise-31oe.2.
6. **The command language does not read `|pipe quoting|`.**
`create-view |By area|` made a view named `|By`, and the second word was
ignored. Commands that take a fixed number of arguments ignore extra ones
without saying so. Workaround: `"double quotes"`. improvise-31oe.3.
7. **`create-view` puts every category it is not told about on Page**, with its
first item selected. That silently filters the view. Every view has to set
every category's axis, one `set-axis` line each. improvise-31oe.4.
8. **A view cannot hide a category's column.** A per-file records view shows
every category and measure in the model. improvise-31oe.5.
9. **`export` of a records view writes the editor's `+cat` and `+` columns.**
improvise-8jk6.7.
10. **`export` writes display text.** It uses the view's number format, so the
import's default `,.2` gives `"17,512.00"`. Workaround: `set-format .0` on
every exported view, and `FORMAT .3` on the ratio formulas.
improvise-8jk6.8.
11. **Nothing sorts.** Items stay in first-seen order, so weeks come out
newest first. A top-N table cannot be produced, so code-maat's rankings are
read from its CSVs. improvise-31oe.6.
12. **`--extract` has no ISO week.** It has year, month and quarter. Workaround:
`churn.csv` carries a `week` column. improvise-8jk6.9.
13. **`import --formula` targets only `_Measure`.** A formula on another
category needs `add-formula <category> …` in a script. improvise-8jk6.10.
14. **The saved model opens on the import's `Default` view**, which puts `File`
on rows and `area` on columns, and not on the view the script left active.
That is improvise-azhb.
15. **No folder import.** Closed by improvise-lpzu (PR #314), which this
pipeline now uses. Until it landed, the script named the five files and
generated its own per-file views.
16. **There is no headless output but a file.** `export` writes a CSV, and the
tables are laid out in Python. improvise-31oe.7.
17. **`create-view` of an existing name replaces that view, and exits 0.**
`Workbook::create_view` inserts into an `IndexMap`, so it overwrites without
a word. Folder import makes a view per file stem, so a report view named
like a stem would replace that file's view.
improvise-31oe.8.
### The suite
`scripts/repo-stats-test.sh` builds a fixture repository whose every number is
known, runs the whole pipeline, code-maat and improvise included, and asserts
improvise's exported tables. It runs in `nix-build`, which puts improvise and
code-maat on `PATH`. Locally it uses improvise from `PATH`, or else
`target/debug/improvise`, and it fails with a message when there is neither.
The fallback does not check how fresh that binary is, so locally the suite can
test a `target/debug/improvise` older than the tree. Rebuild after changing
improvise.
It does not build improvise itself. `checkers.sh` scrubs the environment to an
allowlist, and on macOS that removes what the linker needs: a `cargo build`
inside the suite failed with `library not found for -liconv`. That is a bug in
`checkers.sh`: its allowlist drops the Nix linker variables, and the
orchestrator has filed it separately. Build first with
`cargo build --bin improvise`; `cargo test --workspace` also leaves the binary
there. The suite is the slowest in `checkers.sh`, at about 25 seconds, most of
it JVM start-up for code-maat's four analyses per run.
The evidence and the rivals are in `context/test-evidence/tooling.md`.
+176
View File
@@ -220,6 +220,45 @@ belong in `advertised()` keyed by step, which would make them derived like
everything else and let the status bar carry them correctly instead of being
switched off.
### The grid's width is one answer, and a scroll is measured where it is drawn
Two parts of the TUI scroll horizontally, and both lost their cursor for the
same reason: **the code that chose the scroll offset measured a different width
from the code that drew it** (improvise-gtn7.1).
- **The grid.** `App::cmd_context` counts `visible_cols` for the keypress path,
and `GridWidget` counts again from the `Rect` it is handed. The keypress path
counted from `term_width`, while `draw_content` gave the grid `term_width`
minus the 32-column side panel. So with any of the three panels open the
cursor was scrolled to a column the widget never drew: it went "behind the
panel". Now `viewport::grid_width(term_width, side_panel_open)` is the one
answer, reached through `App::grid_width`. `draw_content` splits the content
row on it, and `cmd_context` counts on it. `App::side_panel_open` is the one
answer to "is a panel open", so a panel that did not count toward the width
would not be drawn either, and the two sides still agree.
- **The tile bar.** `TileBar` chose its first tile against the width after
the prefix, but drew tiles one column later, and two columns later once
scrolled, to leave room for `◀`. When the selected tile fitted within those
one or two columns, the scroll stopped too early and the draw loop dropped the
tile. `tiles_left(start)` is now where both the scroll (`first_tile`) and the
draw begin.
**Why it looked like a regression.** The grid defect dates from `b5418f2e`
(2026-04-06), when `compute_visible_cols` replaced a rough estimate and took
`term_width`. It was always wrong with a panel open. But until `50e4a372`
(2026-09-20) no panel was open at launch. That commit opened the category panel
by default, in the shared `ViewState::default()`, and from then on every TUI
session started in the broken configuration. The tile bar's offset has been
wrong since `4686f470` (2026-04-09), when it first scrolled.
**What is still wrong: improvise-gtn7.2.** `visible_cols` is a *count*, taken at
the *old* offset, and `viewport_effects` applies it at the *new* one. Across
columns of mixed width no single count is right, so the cursor can still leave
the grid with no panel open. The fix is to measure the new offset against the
widths, as `first_tile` does. It is not in this change because
`resolve.rs::scrolled_to` mirrors `viewport_effects` over a window counted in
columns, and `lockstep.rs` holds the two in agreement.
### The caches are built here, per frame
`project::formulas_of`, `category_tree_of`, `view_list_of`, `tile_bar_of`,
@@ -424,3 +463,140 @@ is always drawn. So a browser user sees a message briefly and never loses the
hint. The TUI now does the same, on a keypress rather than a timer, because a
terminal status row has one line to share and a keypress is the event the TUI
already has.
### Except while a minibuffer stays open under it (improvise-su4u)
In a text-entry mode `minibuffer_line` draws the message to the right of the
prompt. Since improvise-v4hg, a formula that does not parse keeps the user in
`FormulaEdit` with the error. Under the plain rule, the error vanished on the
first correcting key: one Backspace and the user was fixing text they could no
longer see the complaint about. The same happened to any refusal that leaves
the user in a minibuffer.
**The rule: a key that leaves the user in the same editor, and says nothing of
its own, puts the stale message back.** `press` remembers the editor session and
the message, clears the message as before, dispatches the key, and then restores
the message only if all three of these hold:
- a text-entry mode was open before the key;
- the same editor session is open after it (below);
- the key wrote no message of its own.
So:
- editing keys keep the refusal on screen until the user acts on it;
- a key's own message always wins, so a second refused `Enter` shows the new
error, not the old one;
- leaving the minibuffer, by a successful commit or by `Esc`, drops it, exactly
as the plain rule does.
**Rejected alternatives.**
- *Expire only on the commit key.* That needs to know which keys commit, which
is the keymap's knowledge (`TextEntry::commit`), and a second copy of it in
the TUI loop. "The mode did not change" is observed rather than predicted,
like gotcha 31's `ReturnToNormalUnlessMoved`.
- *Never clear in a text-entry mode.* The message would then outlive `Esc` and
stand in the mode the user returns to, which is the cq3d bug again.
- *Compare the message after the key with the stale one.* A key that wrote the
same text would be indistinguishable from a key that wrote none. Clearing
first and restoring only an empty slot has no such ambiguity.
### What "the same editor" means (improvise-sx7h)
The first version of this rule compared modes: an equal `AppMode` before and
after the key meant the same editor. That is false for the two cell editors.
`Editing` and `RecordsEditing` commit with `CommitAndAdvance`, which drops to
Normal, writes the cell, moves the cursor and re-opens the editor through
`EnterEditAtCursor`, which sets an **equal** `AppMode`. So after an `Enter` or
`Tab` that committed and moved on, the mode looked unchanged, and the stale
message was restored beside a different cell's editor. It came back again on
every later commit until `Esc`. The reviewer's repro was `R`, `o`, `5`, `Tab`,
`7`: "Added new record row" survived the `Tab`, and so did `o`'s refusal
"That record row is already here…" after the `Tab` that filled the row in.
`FormulaEdit` could not show it, because its successful commit leaves the mode
and writes a message of its own.
An editor session is one *opening* of a minibuffer. The TUI loop cannot see
openings, so `EditorSession` compares what an opening sets:
- **the whole `AppMode`**, not only its variant: an `ItemAdd` for `Month` is
not the `ItemAdd` for `Region` it replaced;
- **the cursor cell**, which is a cell editor's target, so a commit that
advances is a new editor. The prompt modes do not move the cursor while the
user types, so for them this is inert;
- **the editor's seed** (`opened_editor.seed`). `EnterEditAtCursor` re-seeds
from the cell. So a commit that re-opens on the **same** cell, such as `Tab`
in the last column, is a new editor too, because the seed is now the
committed text.
What it cannot see: a commit that re-opens on the same cell with the same seed.
That means the user committed the text the cell already held, which writes
nothing. Restoring a message across a commit that changed nothing is harmless.
**Rejected: an editor generation counter.** It would be exact, but an editor is
opened by more than one effect (`EnterEditAtCursor`, and `ChangeMode` from
`enter-mode`). A counter bumped in each is a hand-kept list of the places a mode
is written, which is the shape gotcha 24 retired. The three fields are set by
the opening itself, so they need no list.
**Not included: the active view.** The cursor is per view, so in principle the
view belongs with the cell. But no key a minibuffer binds can switch the view
and stay in the minibuffer: a `:` line that switches view settles back to
Normal. So nothing could test the field, and an untestable field is a claim no
test can check.
**The browser asks the same question (improvise-1x9h).** `EditorSession` now
lives in `improvise-protocol` and is built from a `ViewState`, so the TUI's
`press` and the browser's `ClientState::status_stands` compare the same value.
The browser's mechanism differs: its message is a toast on a timer, and there
is no per-key clear. So it keeps the message's *owner* and suspends the timer
while that owner is open. See `context/code/clients.md`, "A message beside its
open editor stands until that editor closes".
## The import wizard wraps and scrolls; it never truncates a message (improvise-cgqt)
The user's ruling of 2026-09-25: "a message like this should wrap not truncate
so I can read the whole thing." Every warning and message the wizard draws goes
through `ui::text::wrap`. That covers the Confirm step's warnings, the bottom-line message,
the duplicates step's intro, options, disabled reasons and colliding records,
and the name step's explanation. `wrap` measures in cells, like `truncate`
(gotcha 50). It breaks at spaces, cuts a word only when the word alone is wider
than the line, and emits a lone double-width character rather than dropping it
when the line is narrower than that character. A leading indent is kept on every
line, so an indented row hangs under itself.
**The bottom-line message wraps upward** from the row the key hint uses. A
one-line message therefore draws exactly as before, which is why the pinned
frames (`every_step_draws_what_it_drew_before`,
`the_formula_step_draws_its_transient_state`) did not move.
**Lists scroll by item, and the header says what is shown.** `Panel::scrolling`
draws the items from `WizardCache::scroll` on, each one wrapped whole. The
list's own header row reads `Colliding records 3–9 of 14 (PgUp/PgDn):` whenever
some items are off screen. A first version put "more above" and "more below"
markers inside the list's rows. When the first item filled the list exactly,
the marker landed on the hint row and was drawn over (Codex, PR #304). The
header row belongs to no item, so no item can cover it, and no item loses a line
to make room for a marker. `ImportWizard::scroll` clamps to `len - 1`, so the last item can reach the top
and the list can never scroll away. It is by item, not by line, because the
wizard state cannot know the panel width that line-level clamping needs. The
residual is honest. A **single** item taller than the whole list area, roughly
1,500 characters at 80 columns, is still clipped at the panel's bottom. A
collision tuple has at most `MAX_CATEGORIES` coordinates (24 since
improvise-felo.1, up from 12) plus labels. With short names that is a few
hundred characters. Neither the names nor a label value is bounded, though, so
reaching the residual got easier when the cap doubled.
**Every write is clipped to the panel.** The widget draws through `Panel`,
whose `put` writes only inside the panel's inner rectangle, and its height
arithmetic saturates. Before this change, the duplicates step wrote at an
unbounded row and ratatui panicked at 80×10, and four older steps panicked on
`u16` underflow at heights of 3 to 5 (review B2 on PR #304). The bottom row's
`set_bottom_line` now truncates in cells through `ui::text::truncate`. It had
used the widget's byte-measured `truncate`, so a wrapped line holding a `←`
was cut one character early, with `…`, although it fit (review B1). The local
byte `truncate` still draws field names and labels. It is byte-identical on
ASCII, which keeps every pinned frame, and those rows are labels rather than
messages.
Other TUI surfaces still truncate text a user needs. They are out of this
ticket's scope and reported in its hand-back rather than fixed here.
+11 -9
View File
@@ -69,7 +69,7 @@ same effect without the early return.
three shapes; dispatch pattern-matches exhaustively.
- `CategoryKind` is `Regular | VirtualIndex | VirtualDim | VirtualMeasure | Label`.
Business rules (e.g., the 12-category limit counts only `Regular`) are
Business rules (e.g., the 24-category limit counts only `Regular`) are
enforced by matching on the enum, not by checking name prefixes. Virtual
categories (`_Index`, `_Dim`, `_Measure`) always exist: `_Index` and `_Dim`
support drill-down/records mode; `_Measure` holds numeric data fields and
@@ -99,7 +99,8 @@ sort. Property tests verify this invariant.
- `CellKey::new()` is the only way to build a key — it always sorts.
- `Category::add_item()` deduplicates by name and auto-assigns IDs via a private
counter. External code cannot fabricate an `ItemId`.
- `Model::add_category()` checks the 12-category limit before insertion.
- `Model::add_category()` checks the category limit (`MAX_CATEGORIES`, 24)
before insertion.
- `Formula::new()` takes all required fields; there is no default/empty formula
to accidentally leave half-initialized.
@@ -343,13 +344,14 @@ worse than no claim at all. Three consequences:
`finished in` per binary. Read your own run.
- **If you cite a figure, name the machine you took it on.** A number with no
machine is a property of the suite, which it is not.
- **The portable signal is the ratio, not the seconds.** One binary an order of
magnitude above the run's **median** is the thing to look at — not above
second place, which it need not be: on one machine `improvise-io` is 2.2×
the runner-up and ~15× the median. It is the same binary on all three
machines it has been measured on: `improvise-io`'s lib, which
was proptest-dominated by the grammar-walking `.improv` generator and the
round-trip properties over it.
- **The portable signal is a ratio, not the seconds — and not the ratio to
the median.** Most binaries print `0.00s` at two decimals, so the median is
zero and a ratio to it is undefined (improvise-i5z6, 39 binaries). Compare
the slowest binary with the next few slowest, across several runs: an
outlier is one that stays well clear of them *in every run*. Both outliers so
far were found that way and had one cause, a generator re-parsing its pest
grammar per case: `improvise-io`'s lib (improvise-8shu) and
`improvise-formula`'s lib (longest of 39 in all three i5z6 runs).
**improvise-8shu is closed, and its diagnosis was wrong** — which is the more
useful half. The ticket blamed an unbounded grammar walk and asked for a
+1 -1
View File
@@ -257,7 +257,7 @@ The second weekend (or the second half of the first weekend) is reserved for wha
The following are real issues that have been identified but are explicitly not launch-blocking. Do not start any of them as part of this plan. They can be addressed after launch based on what real users actually trip over.
- **Do not** raise or restructure the `MAX_CATEGORIES = 12` limit, even though it's tight for some real use cases.
- ~~**Do not** raise or restructure the `MAX_CATEGORIES = 12` limit, even though it's tight for some real use cases.~~ **Superseded** by the user's decision of 2026-09-25 ("12 categories is feeling small, we should increase to 24"). The limit is 24 (improvise-felo.1).
- **Do not** refactor the `ExecuteCommand` vim command match ladder, even though it duplicates registry logic.
- **Do not** fix the hardcoded `"Measure"` category name in `evaluate_aggregated`, even though it's the one place in the code that assumes a specific user-facing name.
- **Do not** change `Model::evaluate` from a linear formula walk to a HashMap lookup, even though it's O(N·M·F) in the render hot path.
+184 -132
View File
File diff suppressed because one or more lines are too long
+62
View File
@@ -396,3 +396,65 @@ It asserts the status mentions the seed first, so a fit that said nothing
cannot pass. Rival: the success arm emitting `refuse` instead of `set_status` —
fails here. The refusal arm is asserted in
`a_refusal_names_what_went_wrong_and_stores_nothing`.
### a_headless_import_with_duplicates_refuses_by_default_and_names_them
improvise-cgqt, CLI parity: `import --no-wizard` summed colliding records
silently apart from a warning, as 2cw9 left it. Red before the flag existed,
because the import succeeded. It asserts a non-zero exit, the colliding records
named in the wizard's own wording (`2 records (#1, #3): Region=East`), the flag
named, and no output file. It has two siblings. `..._sums_duplicates_when_told_to`
asserts `= 30` and no `Occurrence`. `..._keeps_duplicates_separate_when_told_to`
asserts the category, `= 10` and `= 20`, and no `= 30`. So the two flags cannot
both be summing.
### a_headless_import_of_a_folder_merges_its_csv_files_and_repeats_byte_for_byte
improvise-lpzu, at the process boundary. The folder holds code-maat's three
schemas (coupling, age and abs-churn) and a `README.txt`. At `e65d87fd`'s
parent the run failed with `Cannot read '…/maat': Is a directory (os error
21)`, because a single path went straight to `read_to_string`. The test
asserts:
- sparse Data lines for all three schemas, with no README;
- the four views, `Default` first and initial, then one per file;
- `File: filter, coupling`;
- a second run, with the first run's `maat.improv` now inside the folder,
writing the same bytes.
Rivals R2, R5, R6 and R7 each fail it.
### a_headless_import_takes_files_and_folders_together_and_refuses_an_empty_folder
improvise-lpzu. A file argument and a folder argument together are one merge,
and the extra file's records carry its own `File` item. Before the fix this
failed with `Multi-file import only supports CSV files`, because the folder was
not a `.csv` path. An empty folder exits non-zero, writes nothing, and names
itself: `no CSV files in folder …/empty`. Rivals R2, R4 and R5 each fail it.
### a_saved_per_file_view_projects_only_its_file
improvise-lpzu (`tests/folder_import_views.rs`). A folder import is saved,
reloaded and projected through `project::grid_in`, the one derivation the TUI
grid and the browser both draw from (gotcha 44). This is why no separate TUI
or browser test was written. For each file's view:
- the page bar carries `File` fixed at that file with nothing to cycle
(`items: None`, gotcha 27);
- the drawn cells are that file's alone: `2.00, 7.00` for age and
`5.00, 3.00` for coupling.
Rivals R2, R7 and R9 each fail it. Since the review of `4e68372c` the fixture
also holds a header-only `soc.csv`. Its view must carry `File` at `soc`, be a
records view, and draw zero rows. Rivals F7 and F8 fail that half.
### a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not
improvise-lpzu, after the user's ruling on PR #314 ("we'd want file identity").
At `4e68372c`, a folder holding one CSV took the single-file path, because
`import dir/` was exactly `dir/*.csv`. That document had no `File` category and
no per-file view, and the test failed at the `## Category: File` assertion. The
other half pins the direction the ruling did *not* change: the same CSV named
directly imports without any `File`. The rival that merges every all-CSV
import, lone files included, fails that half ("a lone file keeps today's
shape"). So the test separates "a folder merges" from "everything merges".
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+407
View File
@@ -1511,3 +1511,410 @@ are each compared with the same document with Month on `Axis::None`. At
`Flag` at `Some(0.0)`, false rather than absent, so a fix that dropped the cell
cannot pass. Rival: `read_coordinate` ignoring the reach (`if false && …`).
This test fails.
### `is_item_agrees_with_effective_item_names_everywhere`
**improvise-4daa**, found by the #265 review at `adb6b814`. `Model::first_non_item`
used to ask `effective_item_names(cat).contains(name)` for every coordinate. That
built and cloned the whole item list per coordinate per projected cell.
`Model::is_item` asks the same question with no allocation. This test holds the
two answers equal across every pairing of three categories (`Region`, `_Measure`,
and one that does not exist) and six names:
- an item;
- a measure item;
- a measure formula target;
- a formula target on a regular category, which `add_formula` makes a real item;
- a name that is nothing.
A setup assertion checks that `Profit` is not in `_Measure`'s explicit items, so
only the formula branch can answer it.
**Rival:** `is_item` without the formula-target branch fails this test, and
`ui::host::tests::a_write_under_a_formula_is_refused_for_the_formula_not_as_a_non_item`
in `improvise-command`.
**Bench**, not committed, on an Apple M4 Max, 16 cores. It is the reviewer's
`bench265`, and each build ran interleaved 5 times; figures are the minimum per
build.
| build | pivot 150×120 (18k cells) | records 2000×5 (2000-item label) |
|---|---|---|
| `47b166ee` (#252, before) | 131 ms | 1.93 ms |
| `adb6b814` (#265 head) | 193 ms | 145 ms |
| with `is_item` | 136 ms | 2.82 ms |
Single, non-interleaved runs on this machine varied by more than 2×. That is why
each build was run 5 times, interleaved, and the minimum taken.
### records_mode_records_display_returns_values
**What was vacuous (improvise-sfwy).** The category-column half asserted only
`!display.is_empty()`, where the fixture's value is `North`. The ticket named
the test under its old name, `records_mode_resolve_display_returns_values`.
improvise-umi4 renamed it when `resolve_display` was removed, and kept the
weak assertion.
**Fault first, then the fix** (the improvise-jkxr procedure). Two faults were
injected into `records_value`'s `RecordCol::Category` arm. The first returns
`"GARBAGE"`. The second returns the category's *name*, `Region`, a plausible
wrong answer. Under the old assertion the test **passed** with each fault.
Now it asserts `Some("North")` exactly. With the faults it fails:
`left: Some("GARBAGE")` and `left: Some("Region")`. Without them it passes.
The production code is unchanged.
### pruning_keeps_every_surviving_rows_runs_at_every_level
improvise-e265, strengthened by improvise-6u6s. On a two-level grouped axis,
`prune_empty` used to change a surviving row's group. Pruning the inner header
dropped its `GroupEnd` too, and under the flat reading the ungrouped inner item
then joined the outer group.
**What it asserts.** For every surviving row and at every level:
- the run the row is in names the same `(category, group)` pruned as unpruned
(the whole open-run stack, from `open_runs`, not only the innermost);
- whether it is in the **same run** as the previous survivor is the same
answer pruned as unpruned, where "the same run" means the same header entry.
When every row survives, the pruned entries must equal the unpruned ones,
marker for marker.
**The fixtures** are five, each under every collapse subset of its groups:
- `Region` over `Month`;
- three levels, `Region` over `Month` over `Kind`;
- `Month = [Jan in Q1, Mar, Feb in Q1]`, a non-contiguous group, so one parent
holds two `Q1` runs (improvise-h0iv);
- a middle level with no groups at all (`Kind` between `Region` and `Month`);
- two outer groups side by side (`North` in `Upper`, `South` in `Lower`).
**How it is built** (improvise-6u6s). Each fixture's layout is built once per
collapse subset, with no cells, so every row is present. `prune_axis_items` is
then called directly with a keep predicate for every non-empty set of rows.
That makes 4,992 cases. The first version built two `Workbook`s per funded-cell
mask, about 4,000 cases, and put `improvise-core`'s lib at 0.86–0.92s on an
Apple M4 Max (16 cores) under load average 39–77, against about 0.16s without
it. The reviewer measured 0.94–2.46s under heavier load, once past the 2s
tripwire. Now the lib runs in 0.17–0.23s on the same machine, against 0.16–0.18s
with the module skipped. `prune_empty` is still covered, because it calls the
same pruner, and `prune_empty_keeps_a_row_past_an_inner_run_in_the_outer_group`
pins that wiring.
**Red and rivals.** The e265 form failed at base 202e7eac, on `mask 0b100` with
only `North/Mar` funded: pruned said `Upper`, unpruned said `None`. Rivals, each
injected and restored:
- a flat reader (every marker read at level 0) fails it;
- a pruner that owes no `GroupEnd` fails it (u6gc's shape: a run whose closing
header was pruned is left open);
- a pruner that keeps headers as they arrive fails it;
- **runs keyed by `(level, category, group name)` instead of header position**
fails it, at `fixture 2, mask 0b101, collapsed []: run identity at level 1,
row 1`. Pruning `Mar` must leave North's two `Q1` runs as two, and the rival
merges them.
That last rival passed every test in #250 (improvise-6u6s). Those fixtures had
no non-contiguous group and compared only the innermost group, so they could not
see a merged run.
The equality half was added after the first version of `remark` emitted `End0`
where the input had `End1, End0`. That output was equivalent but not equal, and
the assertion caught it.
### a_pruned_axis_draws_no_header_over_nothing_and_ends_on_no_marker
improvise-e265's other property, over the same 4,992 cases. Every kept header
must be followed, past any deeper headers, by a data item, and the axis must
not end on a `GroupEnd`. At base 202e7eac, `mask 0b1` left a trailing
`GroupEnd`. The ticket counted the header-over-nothing shape on main before #200.
Rivals:
- a pruner that keeps headers as they arrive fails the first half;
- a pruner that passes `GroupEnd`s through as they arrive (the pre-e265 shape)
fails the second.
In the shipped pruner both properties hold by construction, because markers are
emitted only before an item. So the obvious rival, deleting a trailing-marker
strip, has nothing to delete. An earlier draft had that strip, and it was dead
code.
### prune_empty_keeps_a_row_past_an_inner_run_in_the_outer_group
The enumeration drives `prune_axis_items` directly, so this test pins that
`prune_empty` reaches it. With only `North/Mar` funded, the pruned grid has one
row, and it is in `Upper`. It is a wiring test, not a discriminating one: #200's
pruner also answered `Upper` here. What it catches is a `prune_empty` that
stops pruning (the row count) or a reader that loses the outer run.
### a_row_after_an_inner_groups_end_is_still_in_the_outer_group
The user's ruling on improvise-e265, at the layout: membership is per level,
and `row_group_for` answers the innermost open group. With nothing pruned or
folded:
- `North/Jan` is in `Month`/`Q1`, the innermost;
- `North/Mar` is in `Region`/`Upper`, because `Q1` ended and `Upper` did not;
- `South/Mar` is in nothing.
Failed at base 202e7eac on the second assertion: the flat reading said `None`.
The flat-membership rival fails it again.
### a_lookup_does_not_read_past_its_row
improvise-0r01, which is ChatGPT Codex's P2 on #263. `innermost_group` built
`open_runs` for the whole axis, cloning a level stack per row, to answer for one
row on every `z` keypress. `open_runs` was made lazy and the lookup took
`.nth(idx)`. That is superseded: the walk is now `walk_open_runs`, which lends
one stack to a visitor rather than yielding a copy per row (see
`a_late_lookup_answers_for_its_own_row`).
The walk's extent cannot be seen through a well-formed axis, so the test builds
an axis whose second header is unreadable on purpose: `level: usize::MAX`, which
`Vec::resize` cannot allocate. A lookup for row 0 must answer `Upper` without
reaching it.
Rival: collecting `open_runs` before `.nth(idx)`, which is the eager shape. It
fails with `capacity overflow`, and the other 335 core tests pass. That rival
was run against the iterator form. Under `walk_open_runs` the equivalent is a
visitor that never breaks, and the witness holds the same way.
### a_fold_row_evaluates_each_formula_once_across_the_row
improvise-h6z1. `F0 = Revenue`, `F{i} = F{i-1} + F{i-1}` up to F8, as columns,
with Q1 collapsed so the Q1 row is a fold. The test evaluates every cell of that
row twice and compares the two runs: once with a fresh memo per cell (`evaluate`),
where each column re-derives the chain below it (1 + 2 + … + 9 = **45**), and
once with one memo shared across the row (`evaluate_in`), which gives **9**. It
also asserts the two runs produce identical values, so a memo that returned a
wrong entry cannot pass. The 45 is asserted as a setup, so a fixture whose chain
had collapsed into one column could not pass the 9 by accident. Rival:
`evaluate_covering_in` ignoring the memo it is given and making a fresh one.
Injected into a full workspace run, it fails this test (45 against 9) and
nothing else.
### a_spanned_scan_finds_what_a_lookup_per_member_finds
improvise-yb4b. `fold_stored` used to enumerate one `CellKey` per member of a
fold and look each one up. Since #239 a column total spans every row
category, which is 3,000 lookups for one 18k column, paid twice per keypress.
`DataStore::spanned_values` now answers with one index-narrowed scan. This test
compares the scan with the per-member definition it replaces, in both modes, on
a store built to separate them. It has a record with an **extra coordinate**:
`get` misses it and `matching_values` finds it. It has an **uncovered item**, a
record **lacking** the spanned category, **another measure**, and another
value of an unspanned coordinate. It also has a covered item that was never
interned. The values are distinct powers of two, so a sum identifies the
admitted set. The order differs on purpose: the store's order, not member
order, as gotcha 18 names it.
Three faults were injected, each run against the full `improvise-core` and
`improvise-command` suites:
- Exact mode loses its length check (`==` → `>=`). **Only this test fails.**
Nothing else in either suite reaches an `Exact` fold over a record with an
extra coordinate.
- The span ignores coverage (`|| true`). This test fails, and so do five others
(collapsed-group, hidden-row and CSV-agreement tests).
- A category the store never interned is skipped instead of matching nothing.
**Only this test fails.**
### a_span_finds_what_its_members_find_in_store_order
improvise-jq0m. `spanned_values` now chooses between two narrowings, and a
fixed example can only ever take one of them. This property builds random
stores over `R`, `P` and `M`. Some records lack `R`, some carry an extra `X`,
and `r6` is a covered item the store never interned. It spans `R` and sometimes
`P`, in both modes, and compares the result with the per-member definition,
**in store order**: the values the members' own lookups find, read back in
`iter_cells` order. Store and span sizes vary across cases, so both narrowings
are taken.
Three faults were injected, each run against the full `improvise-core` and
`improvise-command` suites:
- The per-item narrowing drops the base, so it keeps each item's pair alone.
This test and `a_spanned_scan_finds_what_a_lookup_per_member_finds` fail.
**Nothing in either crate's grid or command tests does.** Those fixtures never
hold a second value of the base's coordinate within a small span.
- The base narrowing drops the base. This test fails, along with eight
fold and totals tests.
- The per-item positions are not sorted after they are merged. **Only this
test fails.** Values stay correct and only their order moves, which changes
a Sum or Avg in the last bits.
The regression itself is a cost, which no test here can see. The evidence for
it is the `grid_projection` bench's grouped cases, added for this ticket.
### a_memo_answers_for_the_covers_it_was_built_with
**What broke (improvise-pntu).** Found reviewing PR #247, and it was also
ChatGPT Codex's P2 there. `FoldMemo` was keyed by (fold key, span categories),
while `evaluate_covering_in` took the aggregation and the covers as separate
arguments. With `margin_book(["R = Revenue / Cost"])`, key
`{Month=Q1, _Measure=R}` and `Exact`, one memo was passed first with covers=all
and then with Feb hidden. The second call returned **5.0** (40 / 8, the first
fold's answer) where a fresh memo gives **2.0** (10 / 5, Jan alone).
**Red first.** The test was first written against the old API, exactly as the
reviewer gave the repro, and it failed `left: Some(Number(5.0)), right:
Some(Number(2.0))`. The fix moves agg and covers into the memo
(`FoldMemo::new(agg, covers)`), so that shape **no longer compiles**. The
signature is the guard. The test now builds one memo per covers and asserts
three things. Each memo gives its own fold's value. The Feb-hidden memo agrees
with a fresh `evaluate_covering`. The covers=all memo still says 5.0 afterwards.
**Rival:** `FoldMemo::new` storing `|_, _| true` in place of the covers it was
given. This test fails, because Jan alone reads 5.0.
### a_memo_answers_for_the_aggregation_it_was_built_with
pntu's other half. `{_Measure=Revenue}` names no month. So under an `Exact`
memo it is not a stored cell (`None`), and under an `OverHidden` memo it sums
Jan and Feb (`40.0`).
**Rivals:**
- `evaluate_covering_in` passing `Aggregation::Exact` instead of the memo's
agg: this test fails.
- The same fault in `evaluate_spanning_in`: caught by two existing tests,
`a_max_measures_total_does_not_move_when_a_row_group_collapses` and
`an_average_measures_total_is_the_mean_over_its_records`. So it needed no
new test.
### pruning_evaluates_each_fold_formula_once_and_keeps_the_same_rows
**What was slow (improvise-metf).** `GridLayout::prune_empty` evaluated every
cell with its own fresh fold memo. That is h6z1's shape, and it sits on the same
keypress path, since `with_frozen_records` prunes whenever the view asks. Each
column of a folded chain re-derived the chain below it.
**Measured first**, on an Apple M4 Max (64 GB) running other agents' builds.
The bench is `cargo bench --bench grid_projection`, whose new "pruned" lines time
a `with_frozen_records` build with `prune_empty` on. There were two back-to-back
pairs, and the second put the old line back for the "before" half:
| Case | Before (fresh memo per cell) | After (one memo per build) |
|---|---|---|
| dense 18k | 90 ms, 88 ms | 82 ms, 100 ms |
| folded chain ~18k | 900 ms, 705 ms | 355 ms, 351 ms |
| folded chain ~90k | 6.40 s, 6.62 s | 3.00 s, 4.96 s |
It pays on the folded chain: about 2.0× to 2.5× at 18k, and 1.3× to 2.1× at 90k.
The dense grid has no folds and moves only within noise. So `prune_empty` now
shares one memo per build, taken from `GridLayout::fold_memo` and dropped before
the axes are pruned.
**The test.** It uses `margin_book`'s F0..F8 chain with Q1 collapsed. It counts
fold-formula evaluations across one pruned `with_frozen_records` build and
asserts **9**. It also asserts that the pruned rows are exactly the rows a fresh
`evaluate` per cell would keep, so sharing the memo cannot change what is
pruned. And it asserts that the Q1 fold row survives, so the count is not
measured over an empty grid.
**Rival:** the old line, `self.evaluate(model, &k)` per cell. This test fails:
`left: 45, right: 9`.
### a_grid_evaluates_under_its_own_covers_not_another_grids
**What was open (improvise-opbo, P3).** It confirms ChatGPT Codex's P2 on #258.
`GridLayout::evaluate_in` and `column_total` took any `&FoldMemo`. With
`margin_book(["R = Revenue / Cost"])` and Q1 collapsed, there are two layouts
over one model: one draws Jan and Feb, the other hides Feb. The first layout's
memo, handed to the second layout's `evaluate_in`, gave **5.0** for
`{Month=Q1, _Measure=R}` where the second grid reads **2.0**. Nothing in
production reached it, because each build made its own memo.
**Red first.** The test was first written against `evaluate_in` with a foreign
memo. It failed `left: Some(Number(5.0)), right: Some(Number(2.0))`.
**Fix: the review's proposal.** `GridLayout::evaluator(model)` returns a
`GridEval` that borrows the layout and owns a memo the layout built itself. It
has `cell(key)` and `column_total(col)`. `evaluate_in`, the memo-taking
`column_total`, and `fold_memo` are gone. So the shape the test first used no
longer compiles, and the signature is the guard. The test now asserts two
things. Each layout's evaluator answers for its own covers: 5.0 and 2.0. And
asking one does not move the other.
**Rival:** the evaluator's covers ignoring the layout's hidden items. This test
fails `left: Some(Number(5.0)), right: Some(Number(2.0))`.
**The earlier rivals, restated for the new API, still fail:**
- `pruning_evaluates_each_fold_formula_once_and_keeps_the_same_rows` with a
fresh evaluator per cell: `left: 45, right: 9`.
- `one_grid_build_evaluates_each_fold_formula_once_for_cells_and_totals` with
a fresh evaluator per cell: `left: 54, right: 18`.
- The same test with a fresh evaluator per total: `left: 54, right: 18`.
The entries above for h6z1, 0jw5 and metf name `evaluate_in` and `fold_memo`
because those were the API when they were written. This entry supersedes the
names, not the arguments.
### a_late_lookup_answers_for_its_own_row
improvise-0r01, the second round: ChatGPT Codex's P3 on #273. The lazy
`open_runs` iterator still yielded `Some(open.clone())` for every data item, so
`innermost_group(..).nth(idx)` allocated and dropped a stack for every row
before the target. A `z` near the end of a large axis paid nearly the whole
original cost. `a_lookup_does_not_read_past_its_row` could not see this: it asks
for row 0, and it proves only that the suffix is not read. The walk is now
`walk_open_runs`, which lends one stack to a visitor and copies nothing.
This test asks a late index. It covers rows 0 to 5 of a
`Q1{Jan, Feb}, Mar, Q2{Apr, May}` axis, and index 5 is past the end. So it pins
the row counting that the visitor form moved into `innermost_group`. Both rivals
were run against the whole `improvise-core` suite:
- A counter that never advances, so every lookup answers row 0: this test fails,
along with four others.
- A walker that ignores `GroupEnd`: this test fails, along with seven others.
**What it cannot see is the allocation itself.** A visitor that clones the
stack at every row returns the same answers, so no assertion here can tell the
two apart without a counting allocator. The copy-free property is held by the
shape: the visitor receives a borrowed slice of open runs (`&[OpenRun]` since
improvise-felo.6), and no owned stack
leaves the walk. It is not held by a test.
### a_parent_group_covers_every_leaf_beneath_its_child_groups
improvise-felo.6. `items_in_group` read only the items whose own `group` named
the group, so a year covered nothing: its months sit in quarters. Rival run
against `improvise-core`: the direct-only filter restored. Four tests fail —
this one, `a_hierarchy_opens_closed_as_one_fold_per_root` (the 2024 fold reads
empty), `collapsing_a_parent_folds_its_children_whatever_their_state` and
`hidden_items_leave_every_fold_above_them`. The fixture adds `2024-03` last, so
the year and its first quarter are non-contiguous in insertion order.
### a_year_quarter_month_hierarchy_nests_its_runs
improvise-felo.6: the whole marker sequence of a Year > Quarter > Month axis
opened to depth 2, compared exactly. Rival: every header inside a category
emitted at depth 0 (a flat hierarchy). This test fails, and so do
`pruning_keeps_every_surviving_rows_runs_at_every_level` (the `NESTED` fixture),
`the_nested_fixture_draws_runs_inside_one_category` and
`pruning_drops_an_empty_inner_group_and_keeps_its_parent`.
### a_hierarchy_opens_closed_as_one_fold_per_root
The user's ruling on improvise-felo.6: groups start closed. Two rivals fail it.
The old open default (`inherits_collapsed` always false) fails twelve tests
across `improvise-core` and `improvise-io`, this one among them. The INNERMOST
collapsed group folding instead of the outermost (`.rev()` on the fold search)
fails this one, `hidden_items_leave_every_fold_above_them` and
`the_group_under_the_cursor_is_the_innermost_at_every_depth`: under it a closed
year shows its quarters as rows.
### a_groups_depth_opens_it_and_n_minus_one_levels_beneath
`expanded(2): Time/2024` opens 2024 and its quarters, not its months. Rival: an
ancestor reaching `k <= n` levels instead of `k < n`. Only this test fails, which
is why it exists: the boundary is the whole semantics of N.
### a_toggle_overrides_what_a_group_inherits_either_way_and_toggling_back_clears_it
`z` writes `Collapsed` or `Expanded(1)`, or removes the fold where the flip
lands on what the group inherits. Rival: never remove. This test and
`command::cmd::grid::tests::z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again`
fail; every layout test passes, since a redundant fold draws the same grid. What
the rival breaks is the file (a line per toggle, forever) and `z z` being the
identity on the view.
### pruning_drops_an_empty_inner_group_and_keeps_its_parent
`prune_empty` at depth: an empty quarter goes, header and all, and its year's
header stays for the sibling quarter that has data. Rival: `prune_axis_items`
keying open runs by level alone. This test and the `NESTED` enumeration fail.
+414 -14
View File
@@ -1002,7 +1002,7 @@ the error, rather than moving to a `Done` step with nothing to confirm.
### wizard_blocks_advance_when_over_category_limit
Bug improvise-mzv: with more than `MAX_CATEGORIES` (12) accepted Category
Bug improvise-mzv: with more than `MAX_CATEGORIES` (12 then, 24 since improvise-felo.1) accepted Category
proposals, the wizard let the user advance through every remaining step and
`build_model()` only failed at the final confirm. Advancing past
`ReviewProposals` must be blocked, with a message telling the user to mark
@@ -1026,6 +1026,14 @@ every blank column value in a CSV import grew a phantom "null" item. Null means
*absent*: the record must be skipped and counted, exactly as if the key were
missing.
**The skip is superseded by improvise-8jk6.2.** Null still means absent, and
still invents no item. But an absent coordinate no longer drops the record: it
keeps its cell without that coordinate. So the test now asserts both cells
(`region=East` and the bare `_Measure=revenue`) and no warning. The rival that
restores the skip (a present-but-blank category returns the record as
unplaceable) fails here, beside
`a_record_with_no_value_for_a_category_is_kept_without_that_coordinate`.
### build_model_literal_string_null_is_a_real_category_item
The other side of improvise-ych: the *string* `"null"` is an ordinary value — a
@@ -1038,7 +1046,9 @@ null" would pass the test above.
improvise-ych, third arm. An empty string is the JSON spelling of a blank cell —
the CSV parser already turns a blank field into `Value::Null`, so the two inputs
must agree. An item with an empty name is unnameable in the grid, so it is
treated as missing rather than created.
treated as missing rather than created. Since improvise-8jk6.2 "missing"
means the record keeps its cell without a `region` coordinate. It is no longer
skipped, and the same skip-restoring rival fails here.
### build_model_sums_records_that_share_every_coordinate
@@ -1097,19 +1107,29 @@ loop was therefore re-ordered to `[backward, forward]` under the sort fault and
observed failing on the backward history specifically, with
`left: ["alpha", …]` (sorted) against `right: ["charlie", …]` (backward).
### a_long_collision_list_is_truncated_with_a_count
### every_summed_cell_is_reported_in_full
The warning is rendered into a one-line status bar. A file with ten thousand
duplicate coordinates would otherwise compose a warning naming every one. Three
cells are named and the rest counted; the empty case returns `None` rather than
a warning that says nothing happened.
This replaces `a_long_collision_list_is_truncated_with_a_count`, which pinned
2cw9's three-then-"and N more" warning. After the Sum choice, the Confirm step
still showed "…; and 28 more". The user's rule is that a message they need is
never truncated, so each summed cell is now its own warning, and the step wraps
and scrolls them. It was red at compile time against the old `Option<String>`
signature, then red against a `Vec` that still held one elided sentence. The
rival **elided after three** fails it.
### build_model_warns_about_records_skipped_for_missing_category
### improvise-5j9's skip warning, retired by improvise-8jk6.2
improvise-5j9: skipping a record for a missing category value used to be silent,
so a 1000-row import could drop rows with no user-visible sign. The warning must
name the count, the total, and the offending field(s) — a bare "some records were
skipped" is not actionable.
so a 1000-row import could drop rows with no user-visible sign. The warning had
to name the count, the total, and the offending field(s); a bare "some records
were skipped" is not actionable. Its test,
`build_model_warns_about_records_skipped_for_missing_category`, was **deleted**
by improvise-8jk6.2. A record missing a category value is now kept without that
coordinate, so there is no skip left to report. Its successor is
`a_record_with_no_value_for_a_category_is_kept_without_that_coordinate`, which
asserts no warning at all. `build_model_record_with_missing_category_value_skipped`
became `build_model_record_with_missing_category_field_is_kept` and now asserts
the kept cell rather than only the East one.
### is_does_not_swallow_the_start_of_a_longer_word
@@ -1197,6 +1217,41 @@ a further 0.39s → 0.21s but it is not the fix, and reporting it as one would
be the mistake this entry exists to record. Net: 1.42s → 0.21s, and
`improvise-io` is no longer the run's longest binary.
### parser::generator::the_grammar_is_parsed_once_however_many_formulas_the_strategies_yield
improvise-i5z6 is improvise-8shu's twin, in `improvise-formula`. Both
generator strategies, `formula_string()` and `expr_string()`, called
`load_grammar()` inside their `prop_map`. So `pest_meta` re-parsed
`formula.pest` for every generated case: three 256-case properties plus
shrinks. The `improvise-io` fix never reached this crate, which has its own
copy of the generator.
All figures were measured on an Apple M4 Max (16 cores) under heavy parallel
load (load average 15–33), because an unloaded window was not available. The
ratios are more trustworthy than the seconds.
- **Before:** the lib binary was the **longest binary in all three workspace
runs**, at 0.91s, 1.02s and 1.95s. The runner-up was 0.50–0.92s, and a fourth
run gave 3.13s. The median binary reports `0.00s` at two decimals, so the
ratio to the median is not defined. By any finite reading it is two orders
of magnitude above the median. Alone, the binary took 0.84–0.98s over five
runs. The three generator properties alone took 0.94–1.03s, and the other 82
tests together took 0.01s.
- **After:** the binary alone takes 0.06–0.07s over five runs, and
0.06–0.09s inside the workspace run. `headless_cli`, which spawns processes,
is now the longest binary at 0.49–0.75s.
The guard copies 8shu's shape. It counts **parses**, not calls, so it does not
depend on test order. It drives both strategies through a `TestRunner` rather
than re-implementing them. Red before the fix with `left: 64, right: 1`.
Rivals, one per strategy: restoring `let rules = load_grammar();` in either
`formula_string` or `expr_string` alone fails the test with `left: 33`. So the
two strategies are guarded independently.
The case count stays 256. It is already the default that
`context/design-principles.md` §7 prescribes, and it was never the cost.
### export_csv_omits_the_rows_the_view_prunes
improvise-iot6. `render_csv` built its layout with `GridLayout::new`, which does
@@ -1317,7 +1372,7 @@ Faults injected, both against the fixed tree:
### a_document_at_the_category_limit_plus_a_label_reloads
The P1 half, and the reason improvise-dy3j was not merely a lost flag. `Label`
is outside `MAX_CATEGORIES = 12` by design — `Model::add_label_category` has no
is outside `MAX_CATEGORIES` (12 then) by design — `Model::add_label_category` has no
limit check, and the import wizard's `proposed_category_count` scores a label
as zero — so a document with twelve categories and one label is legally
constructible and the wizard will build one from any CSV carrying a free-text
@@ -1326,7 +1381,7 @@ refused, and `parse_md` returned
`Err("Maximum of 12 categories reached")`: **the document was permanently
unloadable.** Reproduced at `d46ec59` with exactly that error.
The second assertion (`regular_category_names().len() == 12`) is what keeps the
The second assertion (`regular_category_names().len() == crate::model::MAX_CATEGORIES`) is what keeps the
first honest. A "fix" that simply raised or removed the limit would let the
document load while putting a high-cardinality label into the pivot count, and
this assertion fails on it.
@@ -1398,7 +1453,16 @@ The fix reports and does not relocate: the row still lands with the coordinates
it does have, exactly as before. Changing where such a row is placed is a
behaviour change the ticket did not ask for.
### import::wizard::tests::a_row_dropped_for_a_missing_category_is_not_charged_to_the_date_tally
### import::wizard::tests::a_row_kept_without_a_category_is_charged_to_the_date_tally
**Renamed from `a_row_dropped_for_a_missing_category_is_not_charged_to_the_date_tally`
by improvise-8jk6.2.** Its premise went with the skip. A row missing `Region`
now lands, so its unparseable `Date` *is* a derived column's loss, and it is
charged once to each of `Date_Month` and `Date_Year`. The test asserts the
landed cell (`Date=2025-02-05, _Measure=Amount = 200`, with no `Region` and no
derived part) and exactly those two warnings. The history below is kept
because the rule it established (charge only landed rows) still holds. Every
object row now lands.
improvise-yss5, found by ChatGPT Codex reviewing PR #150 at `af7fae0e` and
correct. The first fix incremented `unparsed_dates` while walking a record's
@@ -1466,3 +1530,339 @@ which the since-deleted bit-for-bit test above demonstrated.
- **Refusing only when the path is not gzipped.** The plain case passes and the
test fails on `legacy.improv.gz` with the same grammar error. So the gzip
case is what makes the check prove it runs on decompressed text.
## Colliding records (improvise-cgqt)
The user's decision on improvise-cgqt: when records collide on every
coordinate, the wizard asks whether to sum them, keep them separate, or go
back. The fixture `colliding_records` has **two** colliding tuples of
different sizes (three records and two), interleaved with a record that
collides with nothing. A one-collision fixture cannot tell a per-tuple
occurrence counter from a global row number, and a fixture with only one group
cannot tell a filter from no filter. Every rival below was injected into
`wizard.rs`, the io suite run, and the file restored.
### collisions_are_known_before_the_model_is_built
Red before the feature: `collisions()` was a stub returning nothing. It asserts
the structured result exactly: two groups, first-seen order, the coordinates in
build order, and the 1-based record numbers `[1, 3, 5]` and `[2, 6]`. The rival
**every tuple reported, colliding or not** (the `retain` removed) fails it and
thirty-odd other tests.
### summing_duplicates_adds_their_values_into_one_cell
A characterization of 2cw9's sum, now chosen explicitly. It was green before the
feature, and its role is to be the **other half** of the Sum/Separate
discrimination: it asserts three cells (70, 6, 7) and no extra category, where
Separate must give six cells. It was first red for a test bug: `CellKey`
displays with braces, which the expected strings lacked.
### keeping_duplicates_separate_keeps_every_record_distinct
Red before the feature. It asserts all six cells with their occurrence numbers,
the items `1, 2, 3` in order, the category on `Axis::None`, and the
confirmation warning. The rivals each fail it:
- **occurrence always 1** (no per-tuple counter). The records re-collide and sum.
- **a global row number** (the counter keyed by an empty tuple). The items run
`1..6`.
- **Separate treated as Sum.**
- **the category left on its default axis.** It lands on `Page`, which would
show only occurrence 1 in the default grid.
### an_unchosen_policy_refuses_to_build_colliding_records
The rival **Refuse treated as Sum** fails it. This is what keeps a caller that
never asked, like `ImportJsonHeadless` or a future path, from summing silently.
### a_policy_is_moot_when_nothing_collides
Separate and Refuse must add nothing and refuse nothing when nothing collides.
It was green against the stub, because the stub ignored the policy entirely.
It guards against a Separate that always adds its category.
### the_duplicates_step_is_absent_when_no_records_collide
Green against the stub, which never showed the step, so its evidence is the
rival. **step always shown** fails it along with eight existing wizard walks.
Its first version looped `while step != NameModel` and **hung** under that
rival, because advancing from the step with `Refuse` is blocked. It is now
bounded to four advances, so the rival fails it rather than stalling the suite.
### keeping_duplicates_separate_is_refused_at_the_category_cap
A full set of categories plus a collision, which is the shape of the reported
import. The fixture is `category_records(MAX_CATEGORIES)`, so it tracks the cap
(12 when the test was written, 24 since improvise-felo.1). It asserts option (b)
is disabled with a reason naming the cap (`reason.contains(&cap)`, where `cap`
is `MAX_CATEGORIES` spelled out) and Skip, that choosing it leaves
the step and the policy (`Refuse`) untouched, and that the message is the
reason. The rival **cap checked with `>`** fails it.
### choosing_separate_asks_for_a_name_then_moves_on
Also `choosing_go_back_returns_to_the_step_before`, `back_retraces_the_duplicates_steps`,
`a_name_that_is_already_a_category_is_refused` and
`the_suggested_name_avoids_a_header_that_already_uses_it`. The cursor sits at
**1 and 2**, never only 0, so "the option under the cursor" and "the first
option" are different. The rival **chooser ignores the cursor** fails six of
them. **suggestion ignores headers** and **name check accepts a taken name**
each fail their own test.
### records_with_disjoint_measures_on_one_tuple_do_not_collide
improvise-9bit, found by the adversarial review of PR #304 at `92233cff`.
`collisions_among` keyed on the tuple alone, so `{East, revenue}` and
`{East, cost}` were reported as colliding although no cell sums. Under the
`Refuse` default, a sparse-measure CSV that imported cleanly before was refused.
It was red before the fix: `collisions()` returned the East group. The rival
**tuple-only rule** fails it, and fails
`only_the_records_that_share_a_cell_are_listed_and_numbered` too.
### a_record_with_no_measure_collides_with_nothing
The rule the reviewer's injection `if false && measures.is_empty()` showed was
unguarded. It asserts records `[1, 3]` with the measureless `#2` excluded, and
that the two measureless West records form no collision at all. It was green
before the fix, since the rule already held, so its evidence is the rival:
**a measureless record counts as writing** fails it.
### only_the_records_that_share_a_cell_are_listed_and_numbered
A tuple with one `cost` record and two `revenue` records. Only `#2` and `#3`
are listed. Under Separate, the `cost` record keeps occurrence `1`, beside the
first `revenue` record, and the second `revenue` record gets `2`. The rivals
**tuple-only rule** and **a non-colliding record numbered past 1** each fail it.
### the_name_check_covers_categories_derived_from_dates
improvise-4klp, filed by the same review. The name step's check against
date-derived categories was untested. The test extracts `when_Year` and types
that name, which must be refused. It was green on arrival, so its evidence is
the reviewer's rival, **date-derived names dropped from the taken names**,
which fails it. That fault would have let `add_category` reuse the existing
category, and every cell key would then name `when_Year` twice.
### a_group_both_expanded_and_collapsed_is_refused_naming_view_and_group
The user's ruling on improvise-felo.6. Three spellings of one group given two
folds, each refused as `Refusal::FoldConflict` naming the view and the group,
plus a category given two depths. Rival: a second fold for a group silently
wins. Only this test fails. Its control is
`a_collapsed_group_inside_an_expanded_region_loads`: a DIFFERENT group
collapsed inside an expanded region, and the same line twice, both load.
### a_cycle_among_group_parents_is_refused_naming_it
`> Q1-2025[|2025|]` and `> |2025|[Q1-2025]`. Rival: the cycle check disabled;
only this test fails. The fixture's first version spelled `2024-01` bare, which
the grammar refuses (a bare name starts with a letter), so every test on it
passed or failed for the parse error, `a_malformed_depth_is_a_parse_error`
vacuously. It now carries a control that parses.
### an_axis_line_for_a_category_named_expanded_is_still_an_axis_line
`axis_line` is tried before `expanded_line`, so the writer must never spell a
category depth bare. Rival: the writer writing `expanded: Cat` for depth 1. This
test fails: the category `row` at depth 1 reads back as an axis line for a
category named `expanded`.
### an_old_file_with_only_collapsed_lines_loads_closed_and_keeps_its_lines
"Everything opens closed, including old files." Every group of the file opens
closed, the one its `collapsed:` line names and the three it does not, and the
line is written back. Rival: the old open default; this test fails with the
other eleven in that rival's list.
### every_fold_the_api_can_build_reads_back_from_the_file_it_writes
ChatGPT Codex, P2 on #317 at `e09ec752`: `GroupFold::Expanded(usize)` and
`View::set_group_fold` are public, so `Expanded(0)` could be built, written as
`expanded(0): …` and then refused by the grammar that wrote it. Confirmed by
reading: the writer formatted any `usize`, and `depth_number` starts at `1`.
The fix makes the state unrepresentable (`Expanded(NonZeroUsize)`), so the
regression guard is the type, and the old failing input no longer compiles.
This test covers every fold the API CAN build (`Collapsed` and depths 1 to 3,
beside a category depth), each saved and read back.
### a_record_with_no_value_for_a_category_is_kept_without_that_coordinate
improvise-8jk6.2, reported by the user from the wizard: "Skipped 6 of 342
records missing category values: Settlement Date (6)". `Plan::place` returned
`Placement::Missing` for a category with no value, so the build dropped the
whole record, measures and all. The fixture has one complete record, one with
`"Settlement Date": null` and one with `""`, for two different payees. At
`e65d87fd`'s parent the build stored only the first cell. Now the stored cells
are exactly the three records, the two blanks with no date coordinate, and
there is no warning. `Settlement Date` holds only its one real item, so neither
`"null"` nor `""` became one. Rival R1, a record with a present-but-blank
category treated as unplaceable, fails this test and the two improvise-ych
tests.
### files_with_different_headers_import_as_one_sparse_model
improvise-lpzu. code-maat's `age.csv` (`entity,age-months`) and `coupling.csv`
(`entity,coupled,degree,average-revs`) merged through `merge_csvs_from`. Before
the fix, the age records had no `coupled` key, so they took the same `Missing`
path and **every age record was dropped**. The build stored the four coupling
cells and nothing else. The test pins the proposals (each column's kind,
judged over its present values), all six stored cells sorted, no empty-named
item in any category, and no warning. Rival R2, a record whose file lacks a
category column skipped, fails it along with the two CLI folder tests, the
projection test and the per-file view test.
### a_label_column_a_record_lacks_is_no_coordinate_not_an_empty_item
improvise-lpzu, the label half. Labels were attached with
`scalar_name(..).unwrap_or_default()`, so an age record, whose file has no
`coupled`, stored `coupled=` as a coordinate, and `coupled` grew an item named
`""`. Before the fix the first sorted cell was
`File=age, _Measure=age-months, coupled=, entity=src/a.rs = 2`. Rival R3
(`unwrap_or_default` restored) fails it.
### a_folder_expands_to_its_csv_files_in_name_order_and_skips_the_rest
improvise-lpzu. The folder holds three CSVs written out of name order, one
with an upper-case `.CSV`, plus a `.txt`, an `.improv` (the output a second run
finds) and a CSV in *another* folder. Only the three are returned, sorted.
`MemoryFs` stores in a `HashMap`, so the order comes from `Storage::list`'s
contract and not from insertion. Rival R5 (the extension filter dropped) and
rival R6 (the listing passed through a `HashSet`) each fail it.
### a_folder_with_no_csv_files_is_refused_by_name
improvise-lpzu. Both an empty folder and a folder holding only non-CSV files
must refuse with `no CSV files in folder maat`. Importing nothing would write
an empty document. Rival R4 (the `ensure!` removed) fails it, and so does R5,
because the text-only folder then "has" files.
### files_and_folders_expand_in_argument_order
improvise-lpzu. `[z.csv, b/, a.csv]` must expand to
`[z.csv, b/one.csv, b/two.csv, a.csv]`. Argument order is kept, and a folder
is sorted within itself. A fixture listing only folders, or files in name
order, could not tell "argument order" from "sort everything". R6 fails it.
### two_files_with_one_stem_union_their_columns
improvise-8jk6.12 (3). `a/x.csv` (`entity,one`) and `b/x.csv` (`entity,two`) are
one `File` item, `x`, whose columns are `entity, one, two`. Rival F3 (the
reviewer's fault: clear the stem's columns before each file) leaves
`entity, two`, and the `x` view would then hide the measure `one` that half its
records carry.
### files_with_different_headers_merge_without_inventing_the_missing_columns
improvise-lpzu, at the parser. Each merged record carries exactly its own
file's keys plus `File`: no key, `null` or `0` for a column another file has.
Each `SourceFile` lists its file's columns in header order. The per-file views
are built from those lists.
### a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_the_grid
improvise-8jk6.2's downstream question, asked of the real layout rather than
argued. Bolt has no `Date`:
- With `Date` on Column or on Page, the grid draws only Acme's `10`.
- With `Date` on None, it draws both `10` and `20`.
- In records mode, Bolt's row reads `["", "Bolt", "", "20", ""]`, a blank
`Date` cell.
- The Data line is `Payee=Bolt, _Measure=Amount = 20`, and it round-trips.
- A drill on `Payee=Bolt` reaches the record.
This is a characterization of the model's existing sparse semantics. It found
no bug. Its purpose is to make the Row/Column answer ("invisible") a stated
fact the user can overrule, rather than an accident. R2 fails it.
### a_merge_gives_each_file_a_view_projected_to_its_columns
improvise-lpzu, the user's "one view filtered by the file column and
projected to the columns in that file". The test asserts:
- the view order (`Default`, then the files in read order), with `Default`
still active;
- `File` on Filter at the file's item;
- the file's own categories seated, and the other file's on None;
- `_Measure` on Page at the file's first measure, with the other file's
measures hidden;
- the drawn values.
The age view draws `[2, 7]`. The coupling view draws `[5, 3]`, because its
first measure in model order is `average-revs`: serde_json's map orders keys
alphabetically. Failed at `bb505f8f`'s parent, where no view existed. Rivals
R7 (File on None), R9 (every category seated) and R10 (nothing hidden) each
fail it.
### a_files_view_holds_only_that_files_records
improvise-lpzu. The code-maat fixture cannot show that the *filter* works,
because its files share no measure, so the measure page alone would isolate
them. Two quarterly files with the same `Region,Revenue` schema can: without
the filter, `q1` would draw `30` and `3`. Rival R7 fails it. R9 and R10 do
not, and are not meant to.
### a_files_view_name_skips_names_already_held
improvise-lpzu. Two stems collide with names already held: `Default`, the
merged view, and `_Drill`, a virtual view that is never written and is dropped
on load. The first becomes `Default 2` and the second `_Drill 2`, and each
still filters to its own item. Rival R8 (the virtual names not checked) fails
it: the file's view would have been saved as `_Drill` and lost.
### per_file_views_survive_a_round_trip_and_repeat_byte_for_byte
improvise-lpzu. Two builds from two independently written `MemoryFs` give
identical `.improv` text, and `format(parse(text)) == text`. The reloaded
document still has the three views in order, `File: filter` at the file's item,
and the same drawn values. This is the first persisted, non-virtual use of
`Axis::Filter` (gotcha 27). R7 and R9 fail it.
### a_header_only_file_keeps_its_file_item_its_columns_and_its_view
The ChatGPT Codex P2 on PR #314 at `4e68372c`, confirmed by the adversarial
review. `age.csv` (one row), a header-only `coupling.csv` and `rev.csv` (one
row). At `005b7a76` the test failed at the first assertion: `File` held only
`age` and `rev`, so the `coupling` view filtered to an item that did not exist,
and `coupled`, `degree` and `average-revs` were not in the model. Now `File`
reads `age, coupling, rev` in file order, the three columns exist (as Labels),
and the `coupling` view is a zero-row records view filtered to `coupling`, while
the `age` view still draws `2`. Rival F5 (sources not seeded) and F6 (unseen
columns not proposed) each fail it, and so do F7 and F8 below.
### a_file_with_no_measure_gets_a_records_view_not_another_files_measure
improvise-8jk6.13, from the review of PR #314. `a.csv` (`entity,x`) and
`tags.csv` (`entity,tag`). At `005b7a76` the `tags` view had `_Measure` on Page
with no selection, so the layout paged onto `x`, a measure the same view hides.
The test failed there, with `row_count() == 1`: `a`'s record, drawn under
`File=tags`'s view. Now the view has no `_Measure` page, is a records layout,
and draws zero rows, while `a`'s view is still a pivot. Rival F7 (the old
paging) fails it, and F8 (the File selection lost when `records_steps` moves
`File` off Filter) fails it with the same `1`. F8 is the mistake the first
version of the fix made.
### a_files_view_seats_the_date_parts_of_its_own_date_column
improvise-8jk6.12 (1). The reviewer replaced the date-parts clause of
`source_view`'s `own` with `false` and every test passed, because no folder
fixture extracted a date part. Here `abs-churn.csv` has a `date` column,
extracted to `date_Month`, and `age.csv` does not. `date_Month` is seated in the
`abs-churn` view and on None in the `age` view. Rival F1 (the reviewer's fault)
puts it on None in both and fails the first assertion.
### two_files_whose_names_collide_in_one_batch_keep_a_view_each
improvise-8jk6.12 (2). `Default.csv` and `Default 2.csv`. The first takes
`Default 2`, so the second must take `Default 2 2`. Without the same-batch
check both are named `Default 2`, and `IndexMap::insert` silently replaces the
first, leaving two views where three are owed. The test asserts all three names
with their filters. Rival F2 (the reviewer's fault) fails it.
### a_whitespace_only_category_value_is_no_coordinate
improvise-8jk6.12 (4). A `region` of `" "` is kept as a record with no
`region` coordinate, and `region` holds only `East`. The trim in `scalar_name`
predates this PR (improvise-ych), but the sparse-record rule now rests on it:
without it the value becomes an item named `" "`. Rival F4 (the reviewer's
fault) fails it.
+76
View File
@@ -1191,3 +1191,79 @@ compares **content in order**: the drawn keys rows must equal
`help_bindings()` looked up through the same `keys`, row for row. The same
injection now fails. What still passes is a bypass that draws exactly what the
builders asked for, in the order they asked — which is harmless by definition.
### tile_select_opens_on_a_regular_category_even_when_a_virtual_holds_row
improvise-52od. `opening_tile`'s rule is "the best-ranked tile", and in every
earlier fixture its top rank, the first Regular tile, was also the first tile on
Row. So the rival `position(|t| t.axis == Axis::Row).unwrap_or(0)`, which needs
no wire field at all, passed the whole workspace. The fixture is a records-shaped
bar: `_Index` on Row, `_Dim` on Column, `_Measure` on None, `Region` on Page and
`Product` on Row. The rule answers 3 (`Region`). The Row rival answers 0
(`_Index`), and a "first Regular on Row" rival answers 4 (`Product`), so all
three are separated. Injected: the Row rival fails this test.
### with_no_regular_category_tile_select_opens_on_measure_not_index
improvise-bqm0. #231's rule, `position(regular).unwrap_or(0)`, fell back to tile
0, which is `_Index`, when a view has no Regular category, which is exactly the
bug #231 fixed. The fixture is `_Index`, `_Dim`, a Label and `_Measure`. The rule
answers 3 (`_Measure`), then 2 (the Label) with `_Measure` removed, and 0 for an
empty bar. Injected: #231's rule answers 0 and fails. The Row rival fails too.
### a_frame_whose_group_level_names_no_category_does_not_deserialize
improvise-e265, ChatGPT Codex's P2 on #250. A `FrameGroup.level` arrives over
the wire, and `AxisFrame::groups` sizes its open-run stack with
`open.resize(g.level, None)`. A frame carrying `level: 4294967295` therefore
drove a multi-gigabyte allocation in the client instead of being refused. At
`e44b1491` the frame deserialized without complaint. The fix checks every level
against `cats.len()` as the frame arrives: `context/code/protocol.md` §3b.
The test serializes a well-formed two-category frame and checks that it reads
back equal. That is the control. It then injects two faults into the JSON, one
at a time: an oversized group level on the **second** marker, and a `GroupEnd`
one level past `cats`. For each fault it asserts two things. The mutated text is
still well-formed JSON. And the refusal names the level ("names no category"),
so the frame is refused for its level and not for its shape. The collapsed,
item-free frame keeps the fixture independent of `FrameEntry::Item`'s shape,
which #275 changes.
Rivals, each run:
- The `try_from` attribute removed: `unwrap_err` panics, because the frame
parses.
- Only the first marker checked (`.next().filter(..)` instead of `.find(..)`):
it fails the same way, because the first marker is in range.
### every_composed_cell_write_names_the_view_the_client_is_on
improvise-wpxr: a `SetCell`/`ClearCell` now names the view whose grid drew the
cell, because the host resolves an empty record against that view's grid. Every
existing `resolve.rs` fixture sits on `Default`, which is also
`ViewState::default().active_view`. So a composer that named a hard-coded
`"Default"`, or the host's default, passed all of them. This test puts the
client on `Other` and checks all three compositions: `clear-cell`, `paste` and
the Enter commit.
- **Rival** (`typed_cell_command` ignoring its `view` and naming `"Default"`),
run across the whole workspace: fails this test at the Enter commit and
`socket::tests::a_mouse_commit_names_the_view_its_editor_opened_on`, and
nothing else.
- **Rival** (`clear-cell` naming `"Default"`): fails **only** this test, at the
`clear-cell` assertion. `clear-cell` and `paste` build their command in
`resolve` itself rather than through `typed_cell_command`.
- **After the merge with gh/main `08ad7b81`,** the Enter commit writes the cell
its editor OPENED on (improvise-8xq0), so the test now opens the editor on
`Other`, moves the client to `Later`, and only then commits. The view comes
from `OpenedCell::view`, captured at open.
- **Rival** (the commit naming `state.view.active_view`, the view at
`Enter`), run across the whole workspace: fails **only** this test, with
`["Later"]` against `["Other"]`.
### a_quarters_end_leaves_its_year_open_on_the_same_level
improvise-felo.6: runs inside one category nest by `(level, depth)`. `Direct`
sits in `2024` after `Q1`'s `GroupEnd { level: 0, depth: 1 }`, and `Undated`
in nothing after `2024` ends. Rival: `groups()` closing by level alone, as it did
before depth existed. Only this test fails in `improvise-protocol`: every other
frame fixture puts one group per level.
File diff suppressed because it is too large Load Diff
+498
View File
@@ -490,6 +490,264 @@ in a scratch repository and asserts the walk lists it. All four were red
against the `.gitignore` walk. The account, including the rivals, is under
`every_shipped_document_loads` above.
### a_hostile_git_environment_changes_nothing
improvise-eahe. Each section below it once had a test of its own, and each of those tests re-ran this binary in child processes, one child per setting (a0i3 alone ran ten). By #276 that was 18 children, and each one spawns several `git` processes, so this binary took 2.1–3.3s against the 2s-per-binary tripwire (`context/design-principles.md` §7).
**What changed.** The six tests are now one test, and its two children carry every hostile setting at once:
- **Children:** one re-runs `a_gzipped_document_is_walked` and one re-runs `every_shipped_document_loads`.
- **Settings:** each child gets the template dir, the global and system config, `XDG_CONFIG_HOME`, `GIT_DIR`/`GIT_INDEX_FILE`, the five pathspec and attribute settings, the six trace variables, and the config's trace2 targets.
- **Outer index:** the test then asserts the outer repository's index is still empty. That is the check that makes the inherited `GIT_DIR` visible, as in its section below.
**Why one child still separates the settings.** The sections below show that each setting breaks the walk on its own. So a rival that disables the defence against one setting leaves that setting live in the child, and the child fails. Every rival was re-run against the combined test. The ones that were red against the per-setting tests are still red:
- dropping `--template=`;
- the `--local-env-vars` loop;
- the `core.excludesFile` or `core.attributesFile` null;
- each of `GIT_LITERAL_PATHSPECS`, `GIT_GLOB_PATHSPECS`, `GIT_NOGLOB_PATHSPECS` and `GIT_ATTR_SOURCE`;
- the trace prefix: no strip, the exact name only, the exact three names, and the prefix minus `GIT_TRACE`, `_SETUP` or `_PERFORMANCE`;
- the trace2 `0` override.
**The ones green before are green now.**
- `GIT_ICASE_PATHSPECS` is green, as its section explains.
- The prefix minus `GIT_TRACE2_EVENT` is green, because the `0` override covers it. It was red against the per-setting test at ddcdf93a, before #276 added the override.
- **Dropping `GIT_CONFIG_GLOBAL=/dev/null` or `GIT_CONFIG_NOSYSTEM=1` was green, both at #276's head and in eahe's first version.** Their witness was an excludes line, and vwjj's `-c core.excludesFile=/dev/null` outranks every config file, so the witness could not see them. Consolidation did not cause this. improvise-3j70 has since given them a witness that no `-c` reaches (see “The user's global and system excludes” below), and both rivals are red.
**Measured** on an Apple M4 Max (16 cores), three `cargo test --workspace` runs each, before and after. The median counts only binaries that ran a test:
| | this binary | load average | run median | ratio | next largest |
|---|---|---|---|---|---|
| before (#276, 18 children) | 2.10, 3.04, 3.31s | 37–39 (1 min) | 0.00–0.01s | ≈300× | 0.90s, one 2.66s spike |
| after (2 children) | 0.57, 0.59, 0.91s | 30–33 (1 min) | 0.00–0.02s | ≈30–90× | 1.02s, one 2.49s spike |
The median is at the 0.01s resolution cargo prints, so the ratio is coarse. Before, this binary was the outlier by more than two orders of magnitude and over the tripwire in every run. After, it is below the next-largest binary in every run and well under 2s. The occasional 2.5s spike in "next largest" is some other binary, in both runs; that is load, not this file.
### An inherited `GIT_DIR` (improvise-53zz)
improvise-53zz. The scratch repositories inherited the caller's git
environment, and two parts of it change what `git -C <scratch>` does.
**The repo-local variables beat `-C`.** A hook running in a linked worktree
exports `GIT_DIR` (and, during a commit, `GIT_INDEX_FILE`). Under those,
`git -C <scratch> init` re-initialised the *outer* repository and `git add`
staged the scratch files into the *outer* index, so the test suite wrote into
the checkout that ran it. Three of the four shape tests still passed that way,
because `ls-files` read the same wrong index that `add` had just written. So a
green run was no evidence the walk had looked at the scratch repository.
The fix is one helper, `git(root)`, which removes every name that
`git rev-parse --local-env-vars` prints before adding `-C`. It asks git rather
than hard-coding a list, so the list follows the installed git. Both
`shipped_documents` and the scratch helpers go through it, so the production
walk ignores an inherited `GIT_DIR` too.
The test must set the variable, and a test thread cannot set it without
setting it for every other thread. So the test re-runs this same binary as a
child, restricted to `a_single_file_un_ignore_is_walked`, with `GIT_DIR` and
`GIT_INDEX_FILE` naming a throwaway outer repository. It asserts two things:
the child passed, and the outer index is still empty. Before the fix the second
assertion failed with `the outer index gained: .gitignore`.
Rivals, each run against the fixed file:
- Remove only `GIT_DIR`: red, because `GIT_INDEX_FILE` still sends the add to
the outer index.
- Remove only `GIT_INDEX_FILE`: red, because `GIT_DIR` does.
- Rename the child test so the child runs nothing: red on
`… did not run`. `rerun_with_env` asserts `running 1 test`, because
libtest exits 0 when a filter matches nothing, and both of these tests would
then pass without running anything.
### The user's global and system excludes (improvise-53zz)
improvise-53zz. The scratch repositories also read the user's own git config. A
`*.gz` line in a global `core.excludesFile` made `git add` refuse
`docs/model.improv.gz`, so `a_gzipped_document_is_walked` failed on that
machine and passed on others. `run_git` now sets `GIT_CONFIG_GLOBAL=/dev/null`
and `GIT_CONFIG_NOSYSTEM=1`. `shipped_documents` does not need them: `ls-files`
reads the index, and excludes never apply to a tracked file.
The test writes a config whose `excludesFile` holds `*.gz`, and re-runs the
gzip shape test in a child once under `GIT_CONFIG_GLOBAL` and once under
`GIT_CONFIG_SYSTEM`. Before the fix the global run failed at `git add`.
Rivals:
- Drop `GIT_CONFIG_GLOBAL=/dev/null`: red, in the global run.
- Drop `GIT_CONFIG_NOSYSTEM=1`: red, in the system run. The system scope is
there so that this half of the fix has a witness at all.
**Both rivals later went green, and nothing noticed (improvise-3j70).**
improvise-vwjj added `-c core.excludesFile=/dev/null` to `run_git`, and a `-c`
argument outranks every config file. So a global or system `excludesFile` could
no longer reach `git add` whether or not the config was nulled, and the two
rivals above passed from vwjj onwards. The config lines were still worth having,
since they null every *other* key, but nothing pinned them.
The witness is now a second key in the same config, `core.sharedRepository =
nonsense`. Measured on git 2.55:
- `git init` and `git add` die on it (`fatal: bad boolean config value
'nonsense' for 'core.sharedrepository'`);
- `git ls-files` does not read it, so the walk itself, which does not null the
user's config, is unaffected;
- `run_git` passes no `-c` for it.
Two keys were tried first and rejected: `init.defaultObjectFormat` and
`init.defaultRefFormat` only **warn** on an unknown value, so `init` carries on.
The key sits in `a_hostile_git_environment_changes_nothing`'s config, which the
child reads as both `GIT_CONFIG_GLOBAL` and `GIT_CONFIG_SYSTEM`, so there is
still one child. Dropping either env line is red. Every other rival in the
combined test is unchanged.
### The user's default ignore and attributes files (improvise-vwjj)
improvise-vwjj, found reviewing PR #248 at `f5cfc7ed`. The test above sets
`core.excludesFile` *inside* a config file, so emptying the config files looks
like a complete fix. It is not. When `core.excludesFile` is unset, git falls
back to `$XDG_CONFIG_HOME/git/ignore`, or `~/.config/git/ignore` when
`XDG_CONFIG_HOME` is unset. No config file is involved, so
`GIT_CONFIG_GLOBAL=/dev/null` leaves the fallback in force. It is also the
commonest place to keep a global ignore, because it needs no config line.
`core.attributesFile` has the same kind of default, `git/attributes`.
The test points `XDG_CONFIG_HOME` at a temp directory and re-runs the gzip
shape test in a child twice:
- once with `git/ignore` holding `*.gz`;
- once with `git/attributes` holding `*.gz working-tree-encoding=UTF-16`.
The attributes line is chosen because it breaks `git add`: the one-byte file
cannot be decoded as UTF-16. An attribute that only changes line endings would
leave the add succeeding, and the case would witness nothing.
Before the fix (at `f5cfc7ed`) the ignore run failed at `git add`. With only
`-c core.excludesFile=/dev/null` added, the attributes run failed. So each half
of the fix is witnessed separately.
The fix is two `-c` arguments in `run_git`: `core.excludesFile=/dev/null` and
`core.attributesFile=/dev/null`. A `-c` argument outranks every config file and
every default path, so the `~/.config` fallback is closed by the same two lines.
This test sets only `XDG_CONFIG_HOME`, so the `~/.config` path is covered by
that argument, not exercised.
Rivals, each run against the fixed file:
- Drop `core.excludesFile=/dev/null`: red, `fails under a default git/ignore`.
- Drop `core.attributesFile=/dev/null`: red, `fails under a default
git/attributes`.
### The user's template directory (improvise-pr3i)
improvise-pr3i, found reviewing PR #255 at `56ac72a2`. This is a third path to the
53zz symptom. `git init` copies a template directory into the new `.git`, and
`GIT_TEMPLATE_DIR` names one. The variable is not in
`git rev-parse --local-env-vars`, so `git(root)` does not remove it. It also
bypasses config, so `GIT_CONFIG_GLOBAL=/dev/null` (which nulls
`init.templateDir`) does not reach it either. A template whose `info/exclude`
holds `*.gz` therefore became the scratch repository's own exclude file, and
`git add` refused the gzipped document.
The test points `GIT_TEMPLATE_DIR` at such a template and re-runs
`a_gzipped_document_is_walked` in a child. Red at `56ac72a2`, failing at `git add`
with `Use -f if you really want to add them`. That line appears in the failure
message only because this PR also makes `rerun_with_env`'s callers print the
child's **stderr** (`told`). The child's panic goes to stderr, and before this
change the message printed an empty stdout.
The fix is `git init -q --template=`. An empty template directory copies nothing,
whatever the environment, config or compiled-in default says. The obvious
alternative, `env_remove("GIT_TEMPLATE_DIR")`, passes this test equally well, so
the choice between them is made by argument, not by the test. Removing the
variable still lets git copy its compiled-in default template, which a
distribution may have edited. `--template=` closes that too, for the same reason
the `-c` arguments in `run_git` beat relying on the absence of a config file.
Rival: drop `--template=`. Red, as above.
### Pathspec and attribute settings (improvise-a0i3)
improvise-a0i3. `git(root)` strips what `git rev-parse --local-env-vars` names,
and that list holds only the variables that point at a repository (`GIT_DIR`,
`GIT_INDEX_FILE` and so on). Pathspec magic and the attribute source are
process-wide settings, so the list names none of them. They still change what
`git ls-files -- '*.improv' '*.improv.gz'` answers:
- `GIT_LITERAL_PATHSPECS=1` reads `*.improv` as a file literally named
`*.improv`, so the walk finds nothing.
- `GIT_GLOB_PATHSPECS=1` stops `*` at `/`, so only a document at the root
matches. `GIT_NOGLOB_PATHSPECS=1` reads `*` literally, like the literal
setting. Every document these tests walk sits below the root, so both
settings made both tests fail.
- `GIT_ATTR_SOURCE=HEAD` makes git read attributes from `HEAD`. The scratch
repository in the gzip shape test has no commit, so `git add` fails with
`bad --attr-source`.
The test reruns `a_gzipped_document_is_walked` and `every_shipped_document_loads`
in a child under each of five settings, which is ten runs, and collects every
failure before it asserts. Red at #260's head `322de370`: 7 of 10 runs failed.
`LITERAL`, `GLOB` and `NOGLOB` each broke both tests. `ATTR_SOURCE` broke only
the gzip shape test, because the real checkout has a `HEAD`. The ticket reported
`ATTR_SOURCE` breaking both tests. Here it broke one, so the difference is in
the checkout the ticket measured, not in the fix.
The fix is to remove the five variables in `git()`, next to the
`--local-env-vars` loop. A `--literal-pathspecs`-style command-line flag would
have to be chosen per variable, and `GIT_ATTR_SOURCE` has no negating flag.
Rivals, each dropping one name from `PATHSPEC_AND_ATTRIBUTE_SETTINGS`:
- Without `GIT_LITERAL_PATHSPECS`, `GIT_GLOB_PATHSPECS` or
`GIT_NOGLOB_PATHSPECS`: red, both tests under that setting.
- Without `GIT_ATTR_SOURCE`: red, the gzip shape test.
- Without `GIT_ICASE_PATHSPECS`: **green.** A case-insensitive match of
`*.improv` still finds every document, so this row of the test cannot fail.
The name stays in the list because it is the fourth of git's pathspec
variables, and a future spelling of the walk that depends on case would be
exposed to it. The test covers it, but that coverage is vacuous today.
### A git trace sent to stdout (improvise-2iws, improvise-vav0)
improvise-2iws, with improvise-vav0 folded in. The `GIT_TRACE*` family is
process-wide, like improvise-a0i3's settings, so `--local-env-vars` lists none
of them either. Pointed at `/dev/stdout`, git writes trace lines into the
`ls-files -z` output the walk parses, and they are read as paths.
`every_shipped_document_loads` panicked on "Cannot open <root>/17:39:40… git.c:502
trace: built-in: git ls-files …".
**The fix has two parts.**
- `untraced_git()` removes every variable whose name starts with `GIT_TRACE`. A
prefix match, because the family is open-ended. `git()` builds on it, and so
does the `rev-parse --local-env-vars` call, whose stdout is parsed as well.
- It then sets `GIT_TRACE2`, `GIT_TRACE2_EVENT` and `GIT_TRACE2_PERF` to `0`.
A user config's `trace2.normalTarget`, `eventTarget` or `perfTarget` is a
**config** setting, which no environment removal reaches, and the walk's git
does not null the user's config (only `run_git` does). An environment value of
`0` outranks it (improvise-vav0, verified by the #276 reviewer on git 2.55).
**The test** re-runs `a_gzipped_document_is_walked` and
`every_shipped_document_loads` in child processes. Each child gets all six
family variables that write to their target on git 2.55, set to `/dev/stdout`:
`GIT_TRACE`, `GIT_TRACE_SETUP`, `GIT_TRACE_PERFORMANCE`, `GIT_TRACE2`,
`GIT_TRACE2_EVENT` and `GIT_TRACE2_PERF`. It also gets a `GIT_CONFIG_GLOBAL`
whose `[trace2]` section sends all three targets to `/dev/stdout`. There is one
child per test, not one per setting, to keep this binary's child count down
(improvise-eahe). The test failed at #269's head 4cef40c9 for both tests.
The first version set only three variables, so an exact list of those three
names passed it: the review of #276 found that. With all six set, an exact
three-name list leaves `GIT_TRACE_SETUP` behind and fails.
Rivals, each injected into `untraced_git` and restored. Every one fails:
- **no strip**;
- **the exact names `GIT_TRACE`, `GIT_TRACE2_EVENT`, `GIT_TRACE_PERFORMANCE`**:
this is the rival the review found passing;
- **the prefix minus `GIT_TRACE`**, **minus `GIT_TRACE_SETUP`**, and **minus
`GIT_TRACE_PERFORMANCE`**: each of the three variables that the `0` override
does not cover breaks the walk on its own;
- **no `0` override** (removal only): the config's targets break the walk.
The `GIT_TRACE2*` variables have no removal rival of their own. The `0` override
covers them whether or not they are removed.
---
## `tests/web_demo_model.rs`
@@ -946,3 +1204,243 @@ than `App::handle_key`, because the expiry is the front end's rule.
at step 1: `" TILES Saved to x.improv …"`.
- With the expiry moved *after* dispatch, it fails at step 2: "the axis cycle
must say what it did", since every message is wiped before it can be drawn.
### a_minibuffer_refusal_stands_while_the_user_fixes_it
improvise-su4u. It drives `draw.rs::press`, the function `run_tui` calls,
because the expiry is the TUI loop's rule. In the formula panel it presses `n`,
types `Total = (` and `Enter`. The formula does not parse, so the editor stays
open with `Formula error` on the bottom row. Then:
1. `Backspace`: still editing, and the error must still be drawn. It failed here
at base 202e7eac: `"formula: Total = ▌"`, with the error gone.
2. `Enter` on the still-bad `Total = `: a new refusal, which must differ from
the first, and whose text must be on the row. A key's own message beats the
stale one.
3. `1` and `Enter`: the fixed formula commits and leaves the editor, and the
error must be gone.
4. A second run: `Esc` straight after the refusal must drop the error too.
Rivals, each injected into `press` and restored:
- **Never restore** (#233's behaviour) fails step 1.
- **Restore even when the key wrote its own message** fails step 2.
- **Restore whatever the mode after the key** fails step 4, because the error
outlives `Esc`. It does not fail step 3, because a successful commit writes
its own message. That is why step 4 exists.
### a_message_does_not_follow_the_editor_onto_the_next_cell
improvise-sx7h, a regression #257 introduced against #233. Equal modes stood for
"the same editor", but `Editing`/`RecordsEditing` commit by re-opening an equal
mode on the next cell. The test drives `press` through four scenarios:
1. **Records.** `R`, then `o` ("Added new record row", in `RecordsEditing`),
then `5`. The message stands while typing in the same editor. Then `Tab`: the
editor re-opens on the next cell, and the message must be gone.
2. **Records.** `R`, `o`, `Esc`, then `o` again, which is refused with "That
record row is already here…". Then `5`, with the refusal still drawn. Then
`Tab`, which fills the row in, and the refusal must be gone.
3. **A plain grid.** `i` on `(0, 0)`, with a note set while editing. Then `7`,
with the note kept. Then `Enter`: the editor re-opens a row down, and the
note must be gone.
4. **The last column.** `i`, a note, `7`, `Tab`. The cursor cannot advance, so
the editor re-opens on the **same** cell, and the note must be gone.
Scenarios 1 and 2 are the reviewer's repro.
Rivals, each injected into `EditorSession::of` and restored:
- **mode only**, which is #257's rule as pushed at 2b1c5bf7, fails at
scenario 1;
- **no cursor cell** fails at scenario 1, because the new record's cells both
seed `""`, so the seed alone cannot tell them apart;
- **no seed** fails at scenario 4, the only place the cell does not change.
### an_editor_on_another_category_is_another_editor
improvise-sx7h's pinning test, asked for by the review. The mode is compared
whole, not by variant. Two `ItemAdd` sessions for different categories must be
different editors, and the same category must be the same. No key reaches that
transition, so it is asserted on `EditorSession::of` directly. The rival that
compares `std::mem::discriminant` fails it, and passes every other test.
### moving_right_keeps_the_selected_cell_drawn_beside_any_side_panel
improvise-gtn7.1, the grid half. The user's report was that "the cursor can go
behind the category screen". The keypress path counted `visible_cols` from
`term_width`, while `draw_content` gave the grid `term_width - 32` whenever a
side panel was open. So `viewport_effects` scrolled only after the cursor had
passed a column the widget never drew. The reasoning, and why it read as a
regression, is in `context/code/tui.md`.
**The test is a boundary test, on purpose.** The defect was a disagreement
between two measurements, one in `improvise-command` and one in the frame
layout. A unit test of either would pass against the other. So the test renders
the real `draw` into an 80×24 `TestBackend`, presses `Right` eleven times across
twelve columns through `press`, and after each press requires the selected
cell's text to be on screen. Each column holds a distinct value (`101 + i`), and
the selection is read off the cells drawn in the selection style (black on
cyan). So "the cursor is somewhere" and "this column is drawn" cannot be
confused.
It runs four times: no panel, and each of the three panels alone. The no-panel
run is the control, and it passes on the unfixed code.
**The fixture was nearly vacuous.** The first version's "no panel" run failed
too, because `ViewState::default()` opens the category panel at launch
(`50e4a372`). So the control was a fourth copy of the category case. All three
panels are now closed before one is opened. That launch default is also why a
defect from April read as a regression in September.
Red at `ae5cf408`: `formula panel: column 4 is selected (offset 0) but not
drawn`. Seven 10-cell columns fit beside a 4-cell row header in 78 cells, but
only four fit in 46.
Rivals, each injected and restored:
- **`cmd_context` counts on `self.term_width` again:** red, `formula panel:
column 4`.
- **`draw_content` splits 8 columns narrower than `grid_width`**, a second
derivation that disagrees with the first: red, `no panel: column 6`.
- **`side_panel_open` counts only the category panel:** green, and correctly.
`draw_content` asks the same function whether to draw a panel, so it draws no
formula panel, the grid gets the full width, and the two sides still agree.
This is the benefit of one derivation: a wrong answer there is wrong
everywhere at once, not a disagreement.
Not covered: columns of mixed width with no panel open still lose the cursor,
for a different reason (improvise-gtn7.2).
### the_selected_tile_is_drawn_at_every_width_it_fits_in
improvise-gtn7.1, the tile half: "the cursor goes off screen completely when
navigating the tiles". `TileBar::render` chose its first tile against
`area.width - prefix`, but drew from `prefix + 1`, and from `prefix + 2` once
scrolled (for the `◀`). When the selected tile fitted within the one or two
columns the scroll had over-counted, the scroll stopped early and the draw loop
`break`s before it, so the tile vanished. Red at `ae5cf408`: `width 42: tile 3
(Category_0) is selected but not drawn: " Tiles: ◀ [_Measure Pag] ▶"`.
The test enumerates rather than picks. It uses eight regular categories, which
with the three virtual ones gives eleven tiles. It checks every width from 40 to
200 and every tile as the selection, and requires the selected tile's label on
a cyan cell. The failing band is one or two columns wide at each scroll
position, so a hand-picked width would very likely have missed it.
Rivals, injected into `first_tile` and restored:
- **Measure from the end of the prefix**, as the old loop did: red at width 42.
- **Forget only the `◀` column** (`prefix + 1`): red at width 43. So the test
sees an off-by-one, not only the two-column case.
### wrapping_keeps_every_word_within_the_width
improvise-cgqt: the Confirm step drew each warning with `truncate`, so the
user's 31-cell collision warning read as one line ending in `…`. The test wraps
a warning-shaped string, with a double-width run and an accented word, at six
widths from 1 to 200. It asserts every word survives in order and no line is
wider than asked. The exemption is a lone character wider than the line, which
is emitted rather than dropped; the test's first version asserted otherwise,
which is impossible at width 1. Rivals, each injected and restored:
**wrap truncates an over-long word** fails it, and **wrap drops the indent**
fails `wrapping_keeps_a_leading_indent_on_every_line`.
### a_warning_longer_than_the_panel_is_drawn_whole
The property the user asked for, through the real path: a colliding import is
summed and built, and the Confirm step is rendered from
`project::wizard_cache`. It asserts every word of the (over 80 character)
warning is on screen in order. It was red against a stub `wrap`. The rival
**warnings truncated to one line each**, which is the old draw, fails it. Also
`a_message_longer_than_the_panel_is_drawn_whole`, for the bottom-line message.
Its rival **message drawn on one line** fails it.
### scrolling_reaches_the_last_collision
Fourteen colliding tuples, more than fit. The first frame must show the first
tuple and a header naming the range shown, and not show the last one. After scrolling,
the last tuple must be drawn. Since the Codex fix, the header row names the
range shown (`Colliding records 14–14 of 14`) in place of the in-list markers.
The rivals **list ignores the scroll offset** and **the header claims every item
shown** each fail it.
`keeping_separate_at_the_cap_is_drawn_unavailable_with_its_reason` fails under
**disabled reason not drawn**.
### a_multibyte_message_is_drawn_whole_at_every_width
Review B1 on PR #304. Messages were wrapped in cells and then drawn through
`set_bottom_line`, which truncated with the widget's **byte**-measured
`truncate`. A wrapped line holding a `←` (three bytes) that fit in cells was
still cut, with `…`. At 80 columns the category-cap reason lost `(←)`, and
words were lost at 22 panel widths from 30 to 100. The earlier
`a_message_longer_than_the_panel_is_drawn_whole` was ASCII-only and could not
see it. This test drives the real cap reason and a message of accents, CJK
and emoji across every width from 30 to 100. It asserts no `…` and every word
in order. It was red before the fix. The rival **bottom row measured in bytes
again** fails it.
Its first run exposed a bug in the test's own helper. `buf_text` read the
padding cell after a double-width character as a space, so `日本語` read as
`日 本 語`. The helper now skips that cell. This changes nothing for the ASCII
pinned frames.
### wrapping_measures_cells_not_bytes
Review B1's second half. A byte count is never smaller than a cell count, so
byte-wrapping only ever wraps early, and "no line wider than asked" plus "no
word lost" cannot tell the two apart. The reviewer swapped `.width()` for
`.len()` in `wrap_words` and the lib stayed green. This test takes strings that
fit in cells but not in bytes (`←←← éé` is 6 cells and 14 bytes; `日本 ab` is 7
cells and 10 bytes) and asserts each stays on **one** line. That rival now
fails it. It was green on arrival, since `wrap` was already right.
### no_step_panics_on_a_small_terminal
Review B2 on PR #304. The duplicates step wrote at an unbounded row, and
ratatui panicked with index-outside-buffer at 80×≤10, 60×≤12 and 50×≤14. It
sweeps every step the wizard can reach at every size from 20 to 100 wide and
3 to 20 high:
- Preview, SelectArrayPath, ReviewProposals and ConfigureDates;
- ResolveDuplicates, both plain and at the cap with its message;
- NameDuplicateCategory;
- DefineFormulas, empty and with formulas and an open editor;
- NameModel, and Done with and without warnings.
It was red before the fix with 3,513 panics, and not only in the new steps.
NameDuplicateCategory panicked up to 13 high, and ResolveDuplicates at every
height to 20. Preview, SelectArrayPath, DefineFormulas and NameModel panicked
at heights 3 to 5 on `u16` underflow in `inner.height - N`, which predates
this ticket.
The fix is `Panel`, one clipped writer, plus saturating arithmetic. Its rivals
are **writes not clipped to the panel** and **height arithmetic not
saturating**. The second one **survived** at first, because the sweep's
DefineFormulas step had no formulas, so the loop holding `inner.bottom() - 5`
never ran. A DefineFormulas step with formulas was added, and the rival now
fails.
The steps run in parallel threads, each over its own prebuilt `WizardCache`.
Run serially the sweep took 3.8s, over the 2-second binary limit, and 0.55s
for the whole lib in parallel on an Apple M4 Max.
### a_first_item_that_fills_the_list_still_says_more_follow
The Codex P2 on PR #304 at `92233cff`. When the first visible item's wrapped
height exactly filled the list, the "more below" marker was written on the next
row, the hint row, and the hint drew over it. The fix moves the scroll status
to the list's header row, which no item can cover. This test fills three rows
with a three-line item and a second item after it. It asserts the first item
is drawn whole and the header reads `1–1 of 2`. The rival **the header claims
every item shown** fails it.
### the_two_front_ends_draw_the_same_nested_hierarchy_on_rows
improvise-felo.6: Year > Quarter > Month in one category, under a grouped outer
category, with a year open two levels, a quarter closed and a year folded. The
precondition pins the header sequence, so the agreement cannot pass on a frame
that drew no nested header. What makes the two ends agree is that neither
derives membership: both walk the same `FrameEntry` list. Rival: the browser's
`render.rs` skipping a `Group` entry whose `depth > 0`. This test fails on the
row count (the TUI draws the quarter line, the browser does not); the column
test does not, since the band shows only each column's innermost group, which
the rival leaves alone. Group names are short because the TUI
truncates a band label to its column's width, which the browser does not: the
first version named its years `Y2024` and the column test failed on `▶ Y2…`
against `▶ Y2025`, a truncation, not a disagreement.
+1
View File
@@ -23,5 +23,6 @@ unicode-width = "^0.2"
web-time = "1"
[dev-dependencies]
improvise-core = { path = "../improvise-core", features = ["test-support"] }
improvise-protocol = { path = "../improvise-protocol", features = ["test-support"] }
tempfile = "3"
@@ -525,6 +525,54 @@ mod tests {
);
}
/// The commit door alone on a formula's cell, with no advance to reach it by.
/// See context/test-evidence/command.md.
#[test]
fn an_editor_already_open_on_a_formula_cell_commits_nothing_under_it() {
let mut app = formula_app();
app.cursor_mut().selected = (1, 2);
app.view_state.mode = crate::ui::app::AppMode::editing();
app.view_state.buffers.insert("edit".into(), "7".into());
app.handle_key(KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE))
.unwrap();
let (text, back) = saved_and_reloaded(&app.model_state.workbook);
assert!(
!text.contains("_Measure=Profit"),
"saved under the formula:\n{text}"
);
assert_eq!(back.model.get_cell(&record("West", "Profit")), None);
}
/// improvise-enz6's formula twin: Tab does not re-open the editor on a cell `i`
/// would decline. See context/test-evidence/command.md.
#[test]
fn tab_advancing_onto_a_formula_cell_does_not_reopen_the_editor_there() {
let tab = KeyEvent::new(KeyCode::Tab, KeyModifiers::NONE);
let mut app = formula_app();
app.cursor_mut().selected = (1, 0);
press(&mut app, 'i');
app.handle_key(tab).unwrap();
assert_eq!(app.cursor().selected, (1, 1), "setup: Tab moved to Cost");
assert!(
app.view_state.mode.is_editing(),
"the control: Cost is stored, so the editor re-opens there"
);
app.handle_key(tab).unwrap();
assert_eq!(app.cursor().selected, (1, 2), "Tab still moves onto Profit");
assert!(
!app.view_state.mode.is_editing(),
"an editor was opened on the formula: {:?}",
app.view_state.mode
);
assert!(
app.view_state
.status_msg
.contains("Profit is computed by a formula"),
"the decline is out loud, got {:?}",
app.view_state.status_msg
);
}
/// The second door onto a formula cell: Tab advances the editor onto it, where
/// `i` would have declined. See context/test-evidence/command.md.
#[test]
@@ -7,7 +7,7 @@ use crate::ui::effect::{self, Effect};
use crate::view::RecordCol;
use improvise_protocol::{OpenedEditor, untouched};
use super::core::{Cmd, CmdContext, read_buffer};
use super::core::{Cmd, CmdContext, read_buffer, write_buffer};
use super::grid::AddRecordRow;
use super::navigation::{CursorState, EnterAdvance, viewport_effects};
@@ -830,6 +830,72 @@ mod tests {
"the reopened editor did not capture the cell it landed on"
);
}
/// Every cell of the 2×2 fixture, by its key.
fn every_cell(app: &crate::ui::app::App) -> Vec<Option<CellValue>> {
use crate::model::cell::CellKey;
let mut cells = Vec::new();
for region in ["North", "West"] {
for measure in ["Cost", "Revenue"] {
let key = CellKey::new(vec![
("Region".into(), region.into()),
("_Measure".into(), measure.into()),
]);
cells.push(app.model_state.workbook.model.get_cell(&key).cloned());
}
}
cells
}
/// BUG (improvise-fy9o) — see `context/test-evidence/command.md`.
#[test]
fn enter_enter_leaves_every_cell_unchanged() {
use improvise_protocol::key::KeyCode;
for (records, start) in [(false, (1, 0)), (true, (1, 2))] {
let mut app = grid_holding(CellValue::Number(4.0), records);
if records {
app.view_state.mode = AppMode::RecordsNormal;
}
app.cursor_mut().selected = start;
let before = every_cell(&app);
press(&mut app, KeyCode::Char('i'));
press(&mut app, KeyCode::Enter);
let reopened = app.view_state.buffers.get("edit").cloned();
press(&mut app, KeyCode::Enter);
assert_eq!(
every_cell(&app),
before,
"records={records}: i, Enter, Enter changed a cell (the reopened editor held {reopened:?})"
);
}
}
/// The reopened editor shows the cell it landed on, so typing extends it.
/// BUG (improvise-fy9o) — see `context/test-evidence/command.md`.
#[test]
fn typing_after_the_reopen_still_writes() {
use crate::model::cell::CellKey;
use improvise_protocol::key::KeyCode;
let mut app = grid_holding(CellValue::Number(4.0), false);
app.cursor_mut().selected = (1, 0);
press(&mut app, KeyCode::Char('i'));
press(&mut app, KeyCode::Enter);
assert_eq!(
app.view_state.buffers.get("edit").map(String::as_str),
Some("2"),
"the editor reopened on (North, Revenue) is not seeded with its value"
);
press(&mut app, KeyCode::Char('5'));
press(&mut app, KeyCode::Enter);
let landed = CellKey::new(vec![
("Region".into(), "North".into()),
("_Measure".into(), "Revenue".into()),
]);
assert_eq!(
app.model_state.workbook.model.get_cell(&landed),
Some(&CellValue::Number(25.0))
);
}
}
// ── Commit commands (mode-specific buffer consumers) ────────────────────────
@@ -867,7 +933,13 @@ fn commit_plain_records_edit(
return;
};
match col {
RecordCol::Category(_) | RecordCol::Measure(_) => {
// The one door the wire's `RenameRecordCoordinate` also takes (improvise-fdk1).
RecordCol::Category(category) => effects.push(Box::new(effect::RenameRecordCoordinate {
record: row.coords.clone(),
category: category.clone(),
item: value.to_string(),
})),
RecordCol::Measure(_) => {
effects.extend(effect::RecordRewrite::of(ctx.model, row, [(col, value)]).effects());
}
RecordCol::NewMeasure => add_measure(value, effects),
@@ -976,6 +1048,11 @@ impl Cmd for CommitAndAdvance {
// Drop to normal **before** the commit, which can abort: tidy-up goes where
// an abort cannot reach it (improvise-2zq, gotcha 21).
effects.push(effect::change_mode(exit_mode_for(&self.edit_mode)));
// This command owns its buffer's clear, at the head, so the re-entry
// below re-seeds after it rather than being blanked (improvise-fy9o).
if let Some(mb) = self.edit_mode.minibuffer() {
effects.push(write_buffer(&mb.buffer_key, String::new()));
}
// An untouched editor writes nothing; everything around the write still runs.
let writes = !untouched(self.opened.as_ref(), &self.value);
if writes {
@@ -11,10 +11,10 @@ use crate::ui::app::AppMode;
use crate::ui::effect::{Effect, Panel};
use crate::view::{Axis, GridLayout, View};
use crate::workbook::Workbook;
use improvise_protocol::CellId;
use improvise_protocol::{CellId, non_item_refusal};
/// What a command says when the cursor sits on a collapsed group's row (§14).
pub const FOLD_REFUSAL: &str = "Collapsed group — press z to expand it first";
pub use improvise_protocol::FOLD_REFUSAL;
const NO_CELL: &str = "no cell at cursor";
pub const AGGREGATE_REFUSAL: &str =
"Cannot write to an aggregate — it combines several records, so there is no one cell to write";
@@ -27,6 +27,23 @@ pub fn derived_refusal(target: &str) -> String {
format!("{target} is computed by a formula — edit the formula, not the cell")
}
/// Why no editor may open on or commit to `(row, col)`: an aggregate (a fold, §14),
/// a formula's result, or a coordinate naming no item. `i` declines all three.
pub fn uneditable(model: &Model, layout: &GridLayout, row: usize, col: usize) -> Option<String> {
match crate::ui::project::cell_id(model, layout, row, col) {
Some(CellId::Aggregate(_)) => Some(
match layout.folds_a_group(row, col) {
true => FOLD_REFUSAL,
false => AGGREGATE_REFUSAL,
}
.to_string(),
),
Some(CellId::Derived { target }) => Some(derived_refusal(&target)),
Some(CellId::NonItem { category, item }) => Some(non_item_refusal(&category, &item)),
Some(CellId::Record(_) | CellId::Create(_) | CellId::Coordinate { .. }) | None => None,
}
}
/// Read-only context available to commands for decision-making: a `&Model`, the
/// active `&View`, and the `&Workbook` for the rare command that enumerates views.
pub struct CmdContext<'a> {
@@ -99,12 +116,10 @@ impl<'a> CmdContext<'a> {
/// The cursor's key for a commit, which resolves a records position itself
/// (gotcha 19). An aggregate or a formula's result is refused, as `i` refuses it.
pub fn committable_cell_key(&self) -> Result<CellKey, String> {
match self.cursor_cell_id() {
Some(CellId::Aggregate(_)) => Err(self.aggregate_refusal()),
Some(CellId::Derived { target }) => Err(derived_refusal(&target)),
Some(CellId::Record(_) | CellId::Create(_) | CellId::Coordinate) | None => {
self.cell_key().ok_or_else(|| NO_CELL.to_string())
}
let (row, col) = self.selected;
match uneditable(self.model, self.layout, row, col) {
Some(refusal) => Err(refusal),
None => self.cell_key().ok_or_else(|| NO_CELL.to_string()),
}
}
@@ -115,7 +130,10 @@ impl<'a> CmdContext<'a> {
Some(CellId::Record(_)) => self.stored_cell_key(),
Some(CellId::Aggregate(_)) => Err(self.aggregate_refusal()),
Some(CellId::Derived { target }) => Err(derived_refusal(&target)),
Some(CellId::Create(_) | CellId::Coordinate) => Err(NO_VALUE_REFUSAL.to_string()),
Some(CellId::Create(_) | CellId::Coordinate { .. }) => {
Err(NO_VALUE_REFUSAL.to_string())
}
Some(CellId::NonItem { category, item }) => Err(non_item_refusal(&category, &item)),
None => Err(match self.cell_key() {
Some(_) => NO_VALUE_REFUSAL,
None => NO_CELL,
@@ -125,7 +143,7 @@ impl<'a> CmdContext<'a> {
}
/// What the cursor's cell is, asked exactly as the projection asks it.
fn cursor_cell_id(&self) -> Option<CellId> {
pub(crate) fn cursor_cell_id(&self) -> Option<CellId> {
crate::ui::project::cell_id(self.model, self.layout, self.selected.0, self.selected.1)
}
@@ -67,6 +67,34 @@ mod tests {
assert_eq!(view.categories_on(Axis::Column), vec!["Type", "Month"]);
}
/// improvise-9trl: a script naming a formula that is not there is refused,
/// so `main.rs` exits 1. It was a silent no-op that exited 0.
#[test]
fn remove_formula_of_a_missing_formula_is_a_refusal() {
let mut m = two_cat_model();
let tax = crate::formula::parse_formula("Tax = Food * 0.1", "Type").unwrap();
m.model.add_formula(tax);
let layout = make_layout(&m);
let reg = make_registry();
let ctx = make_ctx(&m, &layout, &reg);
let run = |args: [&str; 2]| {
let args = args.iter().map(|s| s.to_string()).collect();
let mut app = crate::ui::app::App::new(m.clone(), None);
app.apply_effects(RemoveFormulaCmd(args).execute(&ctx));
app
};
let missing = run(["Type", "Profit"]);
assert!(
missing.last_batch_refused(),
"{}",
missing.view_state.status_msg
);
assert_eq!(missing.model_state.workbook.model.formulas().len(), 1);
let present = run(["Type", "Tax"]);
assert!(!present.last_batch_refused(), "the control must not refuse");
assert!(present.model_state.workbook.model.formulas().is_empty());
}
#[test]
fn write_cmd_without_args_saves() {
let m = two_cat_model();
@@ -251,11 +279,19 @@ effect_cmd!(
RemoveFormulaCmd,
"remove-formula",
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::RemoveFormula {
target_category: args[0].clone(),
target: args[1].clone(),
})]
|args: &Vec<String>, ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
let (category, target) = (&args[0], &args[1]);
let named = ctx
.model
.formulas()
.iter()
.find(|f| &f.target_category == category && &f.target == target);
match named {
Some(f) => vec![Box::new(effect::RemoveFormula { id: f.id() })],
None => vec![effect::refuse(format!(
"No formula for {target} in {category}"
))],
}
}
);
@@ -66,6 +66,63 @@ mod tests {
assert_eq!(non_item_coordinates(&back), Vec::<(String, String)>::new());
}
/// improvise-enz6, the entry half: the editor that Enter re-opens is not opened
/// on a fold. See context/test-evidence/command.md.
#[test]
fn enter_advancing_onto_a_fold_does_not_reopen_the_editor_there() {
use crate::command::key::{KeyCode, KeyEvent, KeyModifiers};
let enter = KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE);
let mut app = app_on_a_fold();
app.cursor_mut().selected = (0, 1);
press(&mut app, 'i');
app.handle_key(enter).unwrap();
assert_eq!(app.cursor().selected, (1, 1), "setup: Enter moved to Feb");
assert!(
app.view_state.mode.is_editing(),
"the control: Feb is an item, so the editor re-opens there"
);
app.handle_key(enter).unwrap();
assert_eq!(
app.cursor().selected,
(2, 1),
"Enter still moves onto the fold"
);
assert!(
!app.view_state.mode.is_editing(),
"an editor was opened on the fold: {:?}",
app.view_state.mode
);
assert!(
app.view_state.status_msg.contains("ollapsed group"),
"the decline is out loud, got: {:?}",
app.view_state.status_msg
);
}
/// The commit door alone, now the advance no longer opens an editor on a fold.
/// See context/test-evidence/command.md.
#[test]
fn an_editor_already_open_on_a_fold_commits_nothing_there() {
use crate::command::key::{KeyCode, KeyEvent, KeyModifiers};
let mut app = app_on_a_fold();
app.view_state.mode = AppMode::editing();
app.view_state.buffers.insert("edit".into(), "7".into());
app.handle_key(KeyEvent::new(KeyCode::Enter, KeyModifiers::NONE))
.unwrap();
let (text, back) = saved_and_reloaded(&app.model_state.workbook);
let data = text.split("## Data").nth(1).expect("a data section");
assert!(
!data.contains("Q2"),
"a group coordinate was saved:\n{text}"
);
assert_eq!(non_item_coordinates(&back), Vec::<(String, String)>::new());
assert!(
app.view_state.status_msg.contains("ollapsed group"),
"the refusal is out loud, got: {:?}",
app.view_state.status_msg
);
}
/// The control for the refusal above: the same key one row up drills.
#[test]
fn drilling_into_the_row_above_a_fold_still_opens_a_drill() {
@@ -120,6 +177,7 @@ mod tests {
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.add_item_in_group("Month", "Apr", "Q2").unwrap();
wb.add_item_in_group("Month", "May", "Q2").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb
}
@@ -156,7 +214,7 @@ mod tests {
!app.model_state
.workbook
.active_view()
.is_group_collapsed("Month", "Q2"),
.is_group_collapsed("Month", &["Q2"]),
"the fixture starts expanded, or the assertion below proves nothing"
);
@@ -168,15 +226,170 @@ mod tests {
);
let view = app.model_state.workbook.active_view();
assert!(
view.is_group_collapsed("Month", "Q2"),
view.is_group_collapsed("Month", &["Q2"]),
"the effect was returned but the view did not move"
);
assert!(
!view.is_group_collapsed("Month", "Q1"),
!view.is_group_collapsed("Month", &["Q1"]),
"the neighbour was left alone"
);
}
/// The column twin had no command-level test: an `is_row: false` toggle that
/// read column 0 passed the workspace (improvise-uv9e). Evidence file.
#[test]
fn a_column_toggle_names_the_group_under_the_cursors_column() {
use crate::command::key::{KeyCode, KeyModifiers};
use crate::ui::app::App;
use crate::view::Axis;
let mut wb = two_groups_on_the_row_axis();
wb.active_view_mut().set_axis("Month", Axis::Column);
wb.active_view_mut().set_axis("Type", Axis::Row);
let mut app = App::new(wb, None);
app.rebuild_layout();
// Row 0 and column 2: off the origin on the axis the toggle reads, and on
// the origin of the other, so reading the wrong half is seen too.
app.cursor_mut().selected = (0, 2);
assert_eq!(app.layout.col_label(2), "Apr", "the cursor sits on Apr");
let effects = {
let ctx = app.cmd_context(KeyCode::Char('z'), KeyModifiers::NONE);
ToggleGroupAtCursor { is_row: false }.execute(&ctx)
};
let dbg = effects_debug(&effects);
assert!(
dbg.contains(r#"category: "Month", group: "Q2""#),
"Apr is in Q2, got: {dbg}"
);
app.apply_effects(effects);
let view = app.model_state.workbook.active_view();
assert!(view.is_group_collapsed("Month", &["Q2"]));
assert!(
!view.is_group_collapsed("Month", &["Q1"]),
"the neighbour stays"
);
}
/// The column twin of the row landing tests: two column levels, the cursor off
/// both origins (improvise-pb30). Evidence: `context/test-evidence/command.md`.
#[test]
fn a_column_collapse_lands_on_the_cursors_own_run_and_keeps_its_row() {
use crate::command::key::{KeyCode, KeyModifiers};
use crate::ui::app::App;
use crate::view::Axis;
let mut wb = crate::workbook::Workbook::new("T");
wb.add_category("Region").unwrap();
wb.add_category("Month").unwrap();
wb.add_category("Type").unwrap();
wb.add_item("Region", "North").unwrap();
wb.add_item("Region", "South").unwrap();
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.add_item("Type", "Tools").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Column);
wb.active_view_mut().set_axis("Month", Axis::Column);
wb.active_view_mut().set_axis("Type", Axis::Row);
let mut app = App::new(wb, None);
app.rebuild_layout();
assert_eq!(app.layout.col_cats, ["Region", "Month"], "Month is inner");
app.cursor_mut().selected = (1, 7);
assert_eq!(
app.layout.col_tuple(7),
Some(&["South".into(), "May".into()][..]),
"the cursor sits on South/May, the tail of South's Q2 run"
);
let effects = {
let ctx = app.cmd_context(KeyCode::Char('z'), KeyModifiers::NONE);
ToggleGroupAtCursor { is_row: false }.execute(&ctx)
};
app.apply_effects(effects);
let cols: Vec<String> = (0..app.layout.col_count())
.map(|c| app.layout.col_label(c))
.collect();
assert_eq!(
cols,
[
"North/Jan",
"North/Feb",
"North/Q2",
"South/Jan",
"South/Feb",
"South/Q2"
]
);
assert_eq!(
app.cursor().selected,
(1, 5),
"South's Q2 column, on the row the cursor was on"
);
}
/// improvise-felo.6: `z` walks DOWN a closed hierarchy — year, then quarter —
/// and back up, the cursor on the group it acted on each time.
#[test]
fn z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again() {
use crate::ui::app::App;
let mut wb = crate::workbook::Workbook::new("T");
wb.add_category("Date").unwrap();
wb.add_category("Type").unwrap();
for (month, quarter) in [
("Jan", "Q1"),
("Feb", "Q1"),
("Apr", "Q2"),
("Jan25", "Q1-25"),
] {
wb.add_item_in_group("Date", month, quarter).unwrap();
}
let date = wb.model.category_mut("Date").unwrap();
for (quarter, year) in [("Q1", "Y2024"), ("Q2", "Y2024"), ("Q1-25", "Y2025")] {
date.add_group(crate::model::category::Group::new(quarter).with_parent(year));
}
wb.add_item("Type", "Food").unwrap();
let mut app = App::new(wb, None);
app.rebuild_layout();
assert_eq!(row_labels(&app), ["Y2024", "Y2025"], "closed by default");
app.cursor_mut().selected = (1, 0);
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Q1-25"],
"Y2025 opens to its quarter"
);
assert_eq!(app.cursor().selected, (1, 0));
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Jan25"],
"then the quarter to its month"
);
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Q1-25"],
"z on the month closes its quarter"
);
app.cursor_mut().selected = (0, 0);
toggle_group_at_cursor(&mut app);
assert_eq!(row_labels(&app), ["Q1", "Q2", "Q1-25"]);
assert_eq!(
app.model_state
.workbook
.active_view()
.group_fold("Date", "Q1-25"),
None,
"closing back to the default leaves no fold behind"
);
}
/// Evidence: `context/test-evidence/command.md`.
#[test]
fn toggling_a_group_collapses_that_group_and_the_cursor_follows_it() {
@@ -242,6 +455,7 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("Month", Axis::Row);
@@ -553,7 +767,9 @@ impl ToggleGroupAtCursor {
};
let prefix = tuple.get(..cats.iter().position(|c| c == cat)?)?;
let mut view = ctx.view.clone();
view.toggle_group_collapse(cat, group);
let chain = ctx.model.group_chain(cat, group);
let chain: Vec<&str> = chain.iter().map(String::as_str).collect();
view.toggle_group_collapse(cat, &chain);
let after = GridLayout::with_frozen_records(ctx.model, &view, None);
match self.is_row {
true => after.row_run_start(cat, group, prefix),
@@ -155,6 +155,7 @@ pub(super) mod test_helpers {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("_Measure", "Revenue").unwrap();
wb.add_item("_Measure", "Cost").unwrap();
for (i, month) in ["Jan", "Feb", "Apr", "May"].into_iter().enumerate() {
@@ -168,7 +169,7 @@ pub(super) mod test_helpers {
}
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let mut app = crate::ui::app::App::new(wb, None);
app.rebuild_layout();
app.cursor_mut().selected = (2, 1);
@@ -197,8 +197,9 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let layout = make_layout(&wb);
let reg = make_registry();
let mut ctx = make_ctx(&wb, &layout, &reg);
@@ -388,6 +389,13 @@ impl Cmd for EditOrDrill {
&formula.target,
))];
}
// Refused at open, as the browser refuses it: a commit would only be
// refused later, under a minibuffer that hides why (improvise-yc9r).
if let Some(improvise_protocol::CellId::NonItem { category, item }) = ctx.cursor_cell_id() {
return vec![effect::refuse(improvise_protocol::non_item_refusal(
&category, &item,
))];
}
vec![
Box::new(effect::SetBuffer {
name: "edit".to_string(),
@@ -402,7 +402,7 @@ mod tests {
let effects = DeleteFormulaAtCursor.execute(&ctx);
let dbg = effects_debug(&effects);
assert!(
dbg.contains("RemoveFormula { target: \"Tax\", target_category: \"Type\" }"),
dbg.contains("target: \"Tax\", target_category: \"Type\""),
"must remove the formula AT THE CURSOR (Tax, index 1), got: {dbg}"
);
assert!(
@@ -668,10 +668,8 @@ impl Cmd for DeleteFormulaAtCursor {
let cursor = ctx.formula_cursor.min(formulas.len().saturating_sub(1));
if cursor < formulas.len() {
let f = &formulas[cursor];
let mut effects: Vec<Box<dyn Effect>> = vec![Box::new(effect::RemoveFormula {
target: f.target.clone(),
target_category: f.target_category.clone(),
})];
let mut effects: Vec<Box<dyn Effect>> =
vec![Box::new(effect::RemoveFormula { id: f.id() })];
if cursor > 0 {
effects.push(Box::new(effect::SetPanelCursor {
panel: Panel::Formula,
@@ -724,7 +722,7 @@ impl Cmd for CycleAxisAtCursor {
.extend(cycled.map(|axis| effect::set_status(axis_change_status(&cat_name, axis))));
effects
} else {
vec![effect::set_status(
vec![effect::refuse(
"Move cursor to a category header to change axis".to_string(),
)]
}
@@ -742,7 +740,7 @@ impl Cmd for OpenItemAddAtCursor {
if let Some(cat_name) = ctx.cat_at_cursor() {
vec![effect::change_mode(AppMode::item_add(cat_name))]
} else {
vec![effect::set_status(
vec![effect::refuse(
"No category selected. Press n to add a category first.",
)]
}
@@ -834,7 +832,7 @@ impl Cmd for DeleteCategoryAtCursor {
effects.extend(retreat_from_deleted_item(ctx));
effects
}
None => vec![effect::set_status("No category to delete")],
None => vec![effect::refuse("No category to delete")],
}
}
}
@@ -27,6 +27,8 @@ fn step_effects(wizard: &ImportWizard, key: KeyCode) -> Vec<Box<dyn Effect>> {
WizardStep::SelectArrayPath => select_path_keys(key),
WizardStep::ReviewProposals => review_proposal_keys(key),
WizardStep::ConfigureDates => configure_date_keys(key),
WizardStep::ResolveDuplicates => resolve_duplicate_keys(key),
WizardStep::NameDuplicateCategory => name_duplicate_category_keys(key),
WizardStep::DefineFormulas if wizard.formula_editing => formula_edit_keys(key),
WizardStep::DefineFormulas => formula_list_keys(key),
WizardStep::NameModel => name_model_keys(key),
@@ -68,6 +70,24 @@ fn configure_date_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
}
}
/// Up/Down choose an option; PageUp/PageDown scroll the colliding records.
fn resolve_duplicate_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
match key {
KeyCode::Enter => op(WizardOp::ChooseDuplicate),
_ => scroll_keys(key).unwrap_or_else(|| cursor_keys(key)),
}
}
fn name_duplicate_category_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
match key {
KeyCode::Enter => op(WizardOp::Advance),
KeyCode::Backspace => op(WizardOp::PopDuplicateNameChar),
KeyCode::Char(c) => op(WizardOp::PushDuplicateNameChar(c)),
KeyCode::Esc => close(),
_ => back_key(key),
}
}
fn formula_list_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
match key {
KeyCode::Char('n') => op(WizardOp::StartFormulaEdit),
@@ -103,7 +123,18 @@ fn done_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
match key {
KeyCode::Enter => vec![Box::new(effect::AcceptWizardModel)],
KeyCode::Esc => op(WizardOp::Back),
_ => back_key(key),
KeyCode::Down | KeyCode::Char('j') => op(WizardOp::Scroll(1)),
KeyCode::Up | KeyCode::Char('k') => op(WizardOp::Scroll(-1)),
_ => scroll_keys(key).unwrap_or_else(|| back_key(key)),
}
}
/// PageUp/PageDown move a step's scrolling list by one item.
fn scroll_keys(key: KeyCode) -> Option<Vec<Box<dyn Effect>>> {
match key {
KeyCode::PageDown => Some(op(WizardOp::Scroll(1))),
KeyCode::PageUp => Some(op(WizardOp::Scroll(-1))),
_ => None,
}
}
@@ -214,6 +245,8 @@ mod tests {
WizardStep::SelectArrayPath,
WizardStep::ReviewProposals,
WizardStep::ConfigureDates,
WizardStep::ResolveDuplicates,
WizardStep::NameDuplicateCategory,
WizardStep::DefineFormulas,
WizardStep::NameModel,
WizardStep::Done,
@@ -236,6 +269,35 @@ mod tests {
assert_eq!(step_effects(&w, KeyCode::Char('z')).len(), 0);
}
/// improvise-cgqt: Enter takes the option under the cursor, never a bare
/// advance, which would skip the question; the page keys scroll the list.
#[test]
fn the_duplicates_step_chooses_on_enter_and_scrolls_on_the_page_keys() {
let w = wizard_at(WizardStep::ResolveDuplicates);
assert_eq!(ops(&w, KeyCode::Enter), "[WizardApply(ChooseDuplicate)]");
assert!(ops(&w, KeyCode::Down).contains("MoveCursor(1)"));
assert!(ops(&w, KeyCode::PageDown).contains("Scroll(1)"));
assert!(ops(&w, KeyCode::PageUp).contains("Scroll(-1)"));
assert!(ops(&w, KeyCode::Esc).contains("CloseWizard"));
}
/// The name step is text entry: `j` is a letter here, not a cursor key.
#[test]
fn the_category_name_step_types_into_the_name() {
let w = wizard_at(WizardStep::NameDuplicateCategory);
assert!(ops(&w, KeyCode::Char('j')).contains("PushDuplicateNameChar('j')"));
assert!(ops(&w, KeyCode::Backspace).contains("PopDuplicateNameChar"));
assert!(ops(&w, KeyCode::Enter).contains("Advance"));
}
/// The confirm step's warnings scroll, so a long list is never cut off.
#[test]
fn done_scrolls_its_warnings() {
let w = wizard_at(WizardStep::Done);
assert!(ops(&w, KeyCode::Down).contains("Scroll(1)"));
assert!(ops(&w, KeyCode::PageUp).contains("Scroll(-1)"));
}
#[test]
fn unbound_keys_emit_nothing() {
let w = wizard_at(WizardStep::ReviewProposals);
+37 -10
View File
@@ -61,7 +61,7 @@ fn effects_of(
Ok(acc)
}),
};
Some(built.unwrap_or_else(|e| vec![crate::ui::effect::set_status(e)]))
Some(built.unwrap_or_else(|e| vec![crate::ui::effect::refuse(e)]))
}
/// The shared keymap table plus the registry that gives its names meaning.
@@ -753,6 +753,22 @@ mod tests {
}
/// The buffer `append-char` is bound to in `mode`'s keymap.
/// The effects `name` builds in `mode` with `buffer` holding text, as Debug.
fn clears_itself(mode: &AppMode, name: &str, args: &[String], buffer: &str) -> String {
use crate::command::cmd::test_helpers::*;
let wb = two_cat_model();
let layout = make_layout(&wb);
let reg = make_registry();
let bufs: HashMap<String, String> = [(buffer.to_string(), "typed".to_string())].into();
let mut ctx = make_ctx(&wb, &layout, &reg);
ctx.mode = mode;
ctx.buffers = &bufs;
let cmd = reg
.interactive(name, args, &ctx)
.unwrap_or_else(|e| panic!("{name} {args:?} does not build: {e}"));
effects_debug(&cmd.execute(&ctx))
}
fn append_target(ks: &KeymapSet, mode: &AppMode) -> String {
let key = ModeKey::from_app_mode(mode, false).expect("a text-entry mode key");
let km = ks.maps().get(&key).expect("every mode has a keymap");
@@ -827,15 +843,26 @@ mod tests {
let km = ks.maps().get(&mode).unwrap();
for key in exits {
let Some(Binding::Sequence(steps)) = km.lookup(*key, none) else {
panic!("{mode:?}: {key:?} should be a commit/exit sequence");
};
assert!(
steps
.iter()
.any(|(n, a)| *n == "clear-buffer" && a.as_slice() == [buffer.to_string()]),
"{mode:?}: {key:?} must clear \"{buffer}\", got {steps:?}"
);
match km.lookup(*key, none) {
Some(Binding::Sequence(steps)) => {
assert!(
steps.iter().any(|(n, a)| *n == "clear-buffer"
&& a.as_slice() == [buffer.to_string()]),
"{mode:?}: {key:?} must clear \"{buffer}\", got {steps:?}"
)
}
// A reopening commit owns its clear (improvise-fy9o): ask the command.
Some(Binding::Cmd { name, args }) => {
let dbg = clears_itself(&app_mode, name, args, &buffer);
assert!(
dbg.contains(&format!(
"SetBuffer {{ name: \"{buffer}\", value: \"\" }}"
)),
"{mode:?}: {key:?} runs {name} alone, which must clear \"{buffer}\": {dbg}"
);
}
other => panic!("{mode:?}: {key:?} should commit or exit, got {other:?}"),
}
}
}
}
+564 -111
View File
@@ -23,7 +23,9 @@ use crate::model::cell::{CellKey, CellValue, DataStore};
use crate::persistence;
use crate::ui::effect::{EffectKind, ModelChange, ViewTarget};
use crate::ui::project::Dirty;
use crate::ui::viewport::{compute_col_widths, compute_row_header_width, compute_visible_cols};
use crate::ui::viewport::{
compute_col_widths, compute_row_header_width, compute_visible_cols, grid_width,
};
use crate::view::{GridLayout, View};
use crate::workbook::Workbook;
use improvise_core::model::record::{AggregateHash, RecordHash};
@@ -48,17 +50,17 @@ fn view_shape(model: &crate::model::types::Model, view: &View) -> u64 {
.filter(|item| !hidden.is_some_and(|h| h.contains(item)))
.collect();
drawn.hash(&mut h);
// Off the drawn items, not `Category::groups`: `add_item_in_group` does not
// populate that, and a group with nothing drawn under it is not shape.
// Off the drawn items' chains, not `Category::groups`: a group with nothing
// drawn under it is not shape, and a fold at any depth is.
let mut collapsed: Vec<&str> = model
.category(cat)
.into_iter()
.flat_map(|c| drawn.iter().filter_map(|item| c.items.get(item)))
.filter_map(|item| item.group.as_deref())
.filter(|group| {
view.collapsed_groups
.get(cat)
.is_some_and(|c| c.contains(*group))
.flat_map(|c| drawn.iter().map(move |item| c.item_chain(item)))
.flat_map(|chain| {
(0..chain.len())
.filter(|&d| view.is_group_collapsed(cat, &chain[..=d]))
.map(|d| chain[d])
.collect::<Vec<_>>()
})
.collect();
collapsed.sort_unstable();
@@ -75,15 +77,14 @@ fn view_shape(model: &crate::model::types::Model, view: &View) -> u64 {
fn view_frame(model: &crate::model::types::Model, view: &View) -> u64 {
// `with_frozen_records`, not `new`: `new` does NOT apply `prune_empty`, and a
// pruned row is exactly the kind of reframing this observation exists for.
frame_of(
&crate::view::layout::GridLayout::with_frozen_records(model, view, None),
view,
)
frame_of(&crate::view::layout::GridLayout::with_frozen_records(
model, view, None,
))
}
/// The same hash off a layout already built. The batch's opening basis is free
/// this way — `context/code/command.md` (improvise-mcq4).
fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
fn frame_of(layout: &crate::view::layout::GridLayout) -> u64 {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
layout.row_cats.hash(&mut h);
@@ -93,13 +94,17 @@ fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
for entry in entries {
match entry {
crate::view::layout::AxisEntry::GroupHeader {
depth,
cat_name,
group_name,
collapsed,
..
} => {
0u8.hash(&mut h);
depth.hash(&mut h);
cat_name.hash(&mut h);
group_name.hash(&mut h);
view.is_group_collapsed(cat_name, group_name).hash(&mut h);
collapsed.hash(&mut h);
}
crate::view::layout::AxisEntry::DataItem { tuple, collapsed } => {
1u8.hash(&mut h);
@@ -108,7 +113,11 @@ fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
}
// Drawn as nothing, but it ends a group's run (improvise-gd83),
// so where it sits is what a row's group attribution depends on.
crate::view::layout::AxisEntry::GroupEnd => 2u8.hash(&mut h),
crate::view::layout::AxisEntry::GroupEnd { level, depth } => {
2u8.hash(&mut h);
level.hash(&mut h);
depth.hash(&mut h);
}
}
}
}
@@ -198,7 +207,16 @@ pub(crate) fn document_config(workbook: &Workbook) -> u64 {
axis.hash(&mut h);
}
sorted_pairs(&view.hidden_items).hash(&mut h);
sorted_pairs(&view.collapsed_groups).hash(&mut h);
let mut depths: Vec<_> = view.expand_depth.iter().collect();
depths.sort_unstable();
depths.hash(&mut h);
let mut folds: Vec<_> = view
.group_folds
.iter()
.flat_map(|(cat, groups)| groups.iter().map(move |(g, fold)| (cat, g, fold)))
.collect();
folds.sort_unstable();
folds.hash(&mut h);
let mut pages: Vec<(&String, &String)> = view.page_selections.iter().collect();
pages.sort_unstable();
pages.hash(&mut h);
@@ -257,6 +275,135 @@ fn record_fingerprints(data: &DataStore) -> std::collections::HashMap<RecordHash
out
}
/// Resolution against the grid each view is drawn, each derived at most once per
/// borrow of the `App` (improvise-kimw) — `context/code/command.md`.
pub struct Grids<'a> {
app: &'a App,
derived: std::cell::RefCell<std::collections::HashMap<String, Option<GridLayout>>>,
}
#[cfg(test)]
thread_local! {
/// How many grids a `Grids` has derived on this thread: the kimw guard's witness.
pub(crate) static GRIDS_DERIVED: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
impl Grids<'_> {
/// See [`App::resolve_record`].
pub fn resolve_record(&self, view: &str, hash: RecordHash) -> Option<CellKey> {
let workbook = &self.app.model_state.workbook;
// A view the document lacks drew nothing, stored or not (Codex P2 on #281).
let stored = workbook
.views
.contains_key(view)
.then(|| workbook.model.data.resolve_record(hash));
stored
.flatten()
.cloned()
.or_else(|| self.with_layout_of(view, |layout| record_in(layout, hash))?)
}
/// A record a command names, looked up where it says: the view that drew it, or
/// the store alone for a rename (improvise-wpxr, `context/code/command.md`).
pub fn resolve_named(&self, named: improvise_protocol::NamedRecord) -> Option<CellKey> {
match named {
improvise_protocol::NamedRecord::Drawn { view, record } => {
self.resolve_record(view, record)
}
improvise_protocol::NamedRecord::Stored(record) => {
let data = &self.app.model_state.workbook.model.data;
data.resolve_record(record).cloned()
}
}
}
/// See [`App::resolve_cell`].
pub fn resolve_cell(&self, view: &str, cell: &RecordCell) -> Option<CellKey> {
let record = self.resolve_record(view, cell.record)?;
Some(match &cell.measure {
Some(measure) => record.with(crate::model::category::MEASURE_CATEGORY, measure),
None => record,
})
}
/// See [`App::unwritable`].
pub fn unwritable(&self, view: &str, cell: &RecordCell) -> Option<String> {
let key = self.resolve_cell(view, cell)?;
let model = &self.app.model_state.workbook.model;
let derived = model.formula_for(&key).map(|f| {
format!(
"{} is computed by a formula — edit the formula, not the cell",
f.target
)
});
let refusal = self.app.non_item_refusal(&key).or(derived);
refusal.or_else(|| self.aggregate_refusal(view, cell.record, &key))
}
/// A record the store does not hold is known only by its layout position, and
/// is a record only where the projection at ITS CELL says so (improvise-sjdb).
fn aggregate_refusal(&self, view: &str, hash: RecordHash, key: &CellKey) -> Option<String> {
let model = &self.app.model_state.workbook.model;
model.data.resolve_record(hash).map_or_else(
|| self.with_layout_of(view, |layout| aggregate_at(model, layout, key))?,
|_| None,
)
}
/// Run `f` over the grid `view` is drawn — the host's own where the names agree.
/// `None` for a view the document lacks: no `resolve_view` fallback on a write.
fn with_layout_of<T>(&self, view: &str, f: impl FnOnce(&GridLayout) -> T) -> Option<T> {
let app = self.app;
let (name, view) = app.model_state.workbook.views.get_key_value(view)?;
let name = name.as_str();
let mut derived = self.derived.borrow_mut();
let elsewhere = derived.entry(name.to_string()).or_insert_with(|| {
let model = crate::ui::project::rederived_for(app, name, view);
let model = model.as_ref().unwrap_or(&app.model_state.workbook.model);
let built = crate::ui::project::layout_elsewhere(app, name, view, model);
#[cfg(test)]
GRIDS_DERIVED.with(|n| n.set(n.get() + usize::from(built.is_some())));
built
});
Some(f(elsewhere.as_ref().unwrap_or(&app.layout)))
}
}
/// Every cell key `layout` names, with its position. Records mode is excluded:
/// its `cell_key` carries *positions*, the trap ADR 002 rejected `_Index` for.
fn cells_of(layout: &GridLayout) -> impl Iterator<Item = (usize, usize, CellKey)> + '_ {
let rows = match layout.is_records_mode() {
true => 0,
false => layout.row_count(),
};
(0..rows).flat_map(move |r| {
(0..layout.col_count()).filter_map(move |c| layout.cell_key(r, c).map(|k| (r, c, k)))
})
}
/// The record of some cell `layout` names, if any names it.
fn record_in(layout: &GridLayout, hash: RecordHash) -> Option<CellKey> {
cells_of(layout)
.map(|(_, _, key)| key)
.find(|key| RecordHash::of(key) == hash)
.map(|key| record_of(&key))
}
/// Why `key` is no one cell to write in `layout`: its position there projects as
/// an aggregate. By the whole key, since a row's columns share one `RecordHash`.
fn aggregate_at(
model: &crate::model::types::Model,
layout: &GridLayout,
key: &CellKey,
) -> Option<String> {
let (row, col) = cells_of(layout)
.find(|(_, _, at)| at == key)
.map(|(row, col, _)| (row, col))?;
let projected = crate::ui::project::cell_id(model, layout, row, col);
matches!(projected, Some(CellId::Aggregate(_)))
.then(|| "an aggregate combines several records, so there is no one cell to write".into())
}
/// Drill-down state: frozen record snapshot + pending edits that have not
/// yet been applied to the model.
#[derive(Debug, Clone, Default)]
@@ -423,6 +570,17 @@ impl App {
app
}
/// Whether any of the three side panels has taken its column.
pub fn side_panel_open(&self) -> bool {
let vs = &self.view_state;
vs.formula_panel_open || vs.category_panel_open || vs.view_panel_open
}
/// The terminal columns the grid gets (improvise-gtn7.1).
pub fn grid_width(&self) -> u16 {
grid_width(self.term_width, self.side_panel_open())
}
/// This client's cursor in the active view. Absent means the origin, so
/// a view visited for the first time opens at (0, 0).
pub fn cursor(&self) -> ViewCursor {
@@ -540,105 +698,47 @@ impl App {
}
/// Resolve a client's record surrogate to the cell it names (ADR 002). `None`
/// means reject. Index first, layout second — see `context/code/command.md`.
pub fn resolve_record(&self, hash: RecordHash) -> Option<CellKey> {
let stored = self
.model_state
.workbook
.model
.data
.resolve_record(hash)
.cloned();
stored.or_else(|| self.record_in_layout(hash))
/// means reject. Index first, then `view`'s layout — `context/code/command.md`.
pub fn resolve_record(&self, view: &str, hash: RecordHash) -> Option<CellKey> {
self.grids().resolve_record(view, hash)
}
/// The full cell key a [`RecordCell`] names: its record, plus the
/// measure coordinate that picks one of the record's cells.
pub fn resolve_cell(&self, cell: &RecordCell) -> Option<CellKey> {
let record = self.resolve_record(cell.record)?;
Some(match &cell.measure {
Some(measure) => record.with(crate::model::category::MEASURE_CATEGORY, measure),
None => record,
})
/// The full cell key a [`RecordCell`] drawn in `view` names: its record, plus
/// the measure coordinate that picks one of the record's cells.
pub fn resolve_cell(&self, view: &str, cell: &RecordCell) -> Option<CellKey> {
self.grids().resolve_cell(view, cell)
}
/// Why a wire write to `cell`, drawn in `view`, is refused: that view projects
/// it as no `CellId::Record` (improvise-ciw8, ADR 002). `None` for a writable cell.
pub fn unwritable(&self, view: &str, cell: &RecordCell) -> Option<String> {
self.grids().unwritable(view, cell)
}
/// The questions above, sharing each view's derived grid for as long as the
/// borrow lasts (improvise-kimw). Hold one across a check, not across a mutation.
pub fn grids(&self) -> Grids<'_> {
Grids {
app: self,
derived: std::cell::RefCell::new(std::collections::HashMap::new()),
}
}
/// Resolve an aggregate surrogate (ADR 002) against the layout and only the
/// layout: an aggregate has no existence outside the layout deriving it.
pub fn resolve_aggregate(&self, hash: AggregateHash) -> Option<CellKey> {
self.layout_cell_keys()
cells_of(&self.layout)
.map(|(_, _, key)| key)
.find(|key| AggregateHash::of(key) == hash)
}
/// The record of some cell the current layout names, if any names it.
fn record_in_layout(&self, hash: RecordHash) -> Option<CellKey> {
self.layout_cell_keys()
.find(|key| RecordHash::of(key) == hash)
.map(|key| record_of(&key))
}
/// Why a wire write to `cell` is refused: the host itself projects it as no
/// `CellId::Record` (improvise-ciw8, ADR 002). `None` for a writable cell.
pub fn unwritable(&self, cell: &RecordCell) -> Option<String> {
let key = self.resolve_cell(cell)?;
/// Every coordinate must name an item of its category, asked of the document
/// (improvise-b114, improvise-nacd). See `context/code/command.md`.
fn non_item_refusal(&self, key: &CellKey) -> Option<String> {
let model = &self.model_state.workbook.model;
let derived = model.formula_for(&key).map(|f| {
format!(
"{} is computed by a formula — edit the formula, not the cell",
f.target
)
});
let refusal = self.group_refusal(&key).or(derived);
refusal.or_else(|| self.aggregate_refusal(cell.record, &key))
}
/// A coordinate naming a group rather than an item is no cell in ANY view, so
/// this asks the document, never the layout (improvise-b114).
fn group_refusal(&self, key: &CellKey) -> Option<String> {
let model = &self.model_state.workbook.model;
key.0.iter().find_map(|(cat, name)| {
let category = model.category(cat)?;
let group = !category.items.contains_key(name.as_str())
&& !category.items_in_group(name).is_empty();
group.then(|| format!("{name} is a group of {cat}, so there is no one cell to write"))
})
}
/// A record the store does not hold is known only by its layout position, and
/// is a record only where the projection at ITS CELL says so (improvise-sjdb).
fn aggregate_refusal(&self, hash: RecordHash, key: &CellKey) -> Option<String> {
let stored = self.model_state.workbook.model.data.resolve_record(hash);
let (row, col) = stored.map_or_else(|| self.cell_position(key), |_| None)?;
let projected =
crate::ui::project::cell_id(&self.model_state.workbook.model, &self.layout, row, col);
matches!(projected, Some(CellId::Aggregate(_))).then(|| {
"an aggregate combines several records, so there is no one cell to write".into()
})
}
/// Where in the current layout the cell `key` sits, if anywhere. By the whole
/// key: a `RecordHash` drops `_Measure`, so a row's columns all share one.
fn cell_position(&self, key: &CellKey) -> Option<(usize, usize)> {
self.layout_cells()
.find(|(_, _, at)| at == key)
.map(|(row, col, _)| (row, col))
}
/// Every cell key the current grid names. Records mode is excluded: its
/// `cell_key` carries *positions*, the trap ADR 002 rejected `_Index` for.
fn layout_cell_keys(&self) -> impl Iterator<Item = CellKey> + '_ {
self.layout_cells().map(|(_, _, key)| key)
}
/// [`Self::layout_cell_keys`] with each key's position.
fn layout_cells(&self) -> impl Iterator<Item = (usize, usize, CellKey)> + '_ {
let layout = &self.layout;
let rows = match layout.is_records_mode() {
true => 0,
false => layout.row_count(),
};
(0..rows).flat_map(move |r| {
(0..layout.col_count()).filter_map(move |c| layout.cell_key(r, c).map(|k| (r, c, k)))
})
model
.first_non_item(key)
.map(|(cat, name)| improvise_protocol::non_item_refusal(cat, name))
}
/// Point **this client's** slice at `name`, if the document has such a view,
@@ -774,7 +874,7 @@ impl App {
fn opening_view_frame(&self) -> u64 {
let (_, view) = self.resolve_view(self.view_state.active_view.as_str());
match self.drill_state.is_none() {
true => frame_of(&self.layout, view),
true => frame_of(&self.layout),
// A drill froze the record list, so the held layout is not what
// `view_frame` would build and the two must not be compared.
false => view_frame(&self.model_state.workbook.model, view),
@@ -886,7 +986,7 @@ impl App {
compute_visible_cols(
&col_widths,
row_header_width,
self.term_width,
self.grid_width(),
cursor.col_offset,
)
},
@@ -998,6 +1098,10 @@ impl App {
if self.abort_effects {
// An abort-setting effect asked for the rest to be skipped. Reset
// the flag, and record the outcome where a caller can read it.
debug_assert!(
matches!(kind, EffectKind::ViewOnly) || self.batch_refused,
"a model effect aborted without refusing: {effect:?} (improvise-lasd)"
);
self.abort_effects = false;
self.batch_aborted = true;
break;
@@ -1132,7 +1236,12 @@ impl App {
}
pub fn start_import_wizard(&mut self, json: serde_json::Value) {
self.wizard = Some(ImportWizard::new(json));
self.start_import(crate::import::wizard::ImportPipeline::new(json));
}
/// Open the wizard over a pipeline the host built, sources and all.
pub fn start_import(&mut self, pipeline: crate::import::wizard::ImportPipeline) {
self.wizard = Some(ImportWizard::from_pipeline(pipeline));
self.view_state.mode = AppMode::ImportWizard;
}
@@ -3222,7 +3331,7 @@ mod tests {
/// Eleven regular categories: one short of the limit.
fn one_category_short_of_the_limit() -> App {
let mut wb = Workbook::new("T");
for n in 0..11 {
for n in 0..crate::model::MAX_CATEGORIES - 1 {
wb.add_category(format!("C{n}")).unwrap();
}
App::new(wb, None)
@@ -3298,6 +3407,33 @@ mod tests {
app
}
/// One key through the keymap, as the TUI delivers it; true when it refused.
fn press_key(app: &mut App, code: KeyCode) -> bool {
app.handle_key(KeyEvent::new(code, KeyModifiers::NONE))
.unwrap();
app.last_batch_refused()
}
fn panel_cursor_at(mut app: App, at: usize) -> App {
app.view_state.cat_panel_cursor = at;
app
}
/// Storage holding one loadable document, `doc.improv`, and nothing else.
fn with_a_document_to_load() -> App {
let storage = crate::storage::MemoryFs::new();
let text = crate::persistence::format_md(&Workbook::new("Doc"));
crate::storage::Storage::write(
&storage,
std::path::Path::new("doc.improv"),
text.as_bytes(),
)
.expect("fixture writes");
let mut app = two_col_model();
app.storage = Box::new(storage);
app
}
/// Months grouped into quarters with Q2 collapsed: row 2 is the fold.
fn folded_quarter() -> App {
let mut wb = Workbook::new("T");
@@ -3306,8 +3442,9 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
app
@@ -3345,12 +3482,13 @@ mod tests {
fn every_refusal_site_refuses_its_failure_and_only_its_failure() {
type Setup = fn() -> App;
type Drive = fn(&mut App) -> bool;
let limit = format!("Maximum of {} categories", crate::model::MAX_CATEGORIES);
let cases: Vec<(&str, Setup, Drive, &str, Setup, Drive)> = vec![
(
"add-category past the limit",
one_category_short_of_the_limit,
|app| run_line(app, "add-category A . add-category B"),
"Maximum of 12 categories",
&limit,
one_category_short_of_the_limit,
|app| run_line(app, "add-category A"),
),
@@ -3521,6 +3659,30 @@ mod tests {
run_line(app, "search-navigate forward")
},
),
(
"load of an absent file",
with_a_document_to_load,
|app| run_line(app, "load absent.improv"),
"Load error",
with_a_document_to_load,
|app| run_line(app, "load doc.improv"),
),
(
"import-json of an absent JSON",
|| with_import_fixtures(two_col_model()),
|app| run_line(app, "import-json absent.json"),
"Cannot read 'absent.json'",
|| with_import_fixtures(two_col_model()),
|app| run_line(app, "import-json good.json"),
),
(
"a formula that does not parse",
two_col_model,
|app| commit_in(app, AppMode::formula_edit(), "formula", "Total = ("),
"Formula error",
two_col_model,
|app| commit_in(app, AppMode::formula_edit(), "formula", "Total = 1"),
),
(
"a : line that does not parse",
two_col_model,
@@ -3568,6 +3730,43 @@ mod tests {
|| yanked(records_model_with_two_rows()),
|app| run_line(app, "paste _Index/1 _Dim/2"),
),
// improvise-cwfj: the category panel's three "nothing at the cursor"
// declines, and a bound key whose command will not build.
(
"cycle-axis-at-cursor past the panel's last row",
|| panel_cursor_at(two_col_model(), 99),
|app| run_line(app, "cycle-axis-at-cursor"),
"Move cursor to a category header",
|| panel_cursor_at(two_col_model(), 1),
|app| run_line(app, "cycle-axis-at-cursor"),
),
(
"open-item-add-at-cursor past the panel's last row",
|| panel_cursor_at(two_col_model(), 99),
|app| run_line(app, "open-item-add-at-cursor"),
"No category selected",
|| panel_cursor_at(two_col_model(), 1),
|app| run_line(app, "open-item-add-at-cursor"),
),
(
"a bound key whose command will not build (p onto an aggregate)",
|| yanked(aggregated_grid(true)),
|app| press_key(app, KeyCode::Char('p')),
crate::command::cmd::core::AGGREGATE_REFUSAL,
|| yanked(two_col_model()),
|app| {
app.cursor_mut().selected = (1, 1);
press_key(app, KeyCode::Char('p'))
},
),
(
"delete-category-at-cursor past the panel's last row",
|| panel_cursor_at(two_col_model(), 99),
|app| run_line(app, "delete-category-at-cursor"),
"No category to delete",
|| panel_cursor_at(two_col_model(), 1),
|app| run_line(app, "delete-category-at-cursor"),
),
];
for (what, fail_setup, fails, says, ok_setup, succeeds) in cases {
let mut app = fail_setup();
@@ -5135,3 +5334,257 @@ mod formula_edit_tests {
assert_eq!(buffer(&app), Some("Sum = East + West"));
}
}
/// Where `T` opens inside a drill, where `_Measure` is on Filter and so is no tile
/// (improvise-h3cv). `context/test-evidence/command.md`.
#[cfg(test)]
mod drill_tile_select {
use super::*;
use crate::command::key::{KeyCode, KeyEvent, KeyModifiers};
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
use crate::workbook::Workbook;
fn drilled(with_label: bool) -> App {
let mut wb = Workbook::new("Drill");
wb.add_category("Region").unwrap();
wb.add_item("Region", "North").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
if with_label {
wb.add_label_category("Note").unwrap();
}
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.model.set_cell(
CellKey::new(vec![
("Region".into(), "North".into()),
("_Measure".into(), "Revenue".into()),
]),
CellValue::Number(5.0),
);
let mut app = App::new(wb, None);
let key = |c| KeyEvent::new(KeyCode::Char(c), KeyModifiers::NONE);
app.handle_key(key('>')).expect("drill");
assert_eq!(app.view().axis_of("_Measure"), Axis::Filter, "a real drill");
app.handle_key(key('T')).expect("tile select");
app
}
fn opened_on(app: &App) -> String {
let names = app.model_state.workbook.model.category_names();
let tiles = app.view().tileable_cats(&names);
tiles[app.view_state.tile_cat_idx].to_string()
}
#[test]
fn in_a_drill_tile_select_opens_on_a_label_not_on_index() {
assert_eq!(opened_on(&drilled(true)), "Note");
}
#[test]
fn in_a_drill_with_nothing_better_tile_select_opens_on_index() {
assert_eq!(opened_on(&drilled(false)), "_Index");
}
}
/// A `NonItem` cell reached through the TUI's own keys (improvise-yc9r).
#[cfg(test)]
mod non_item_key_tests {
use super::*;
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
/// Region on page at `page`, Month (Jan) on rows, one measure: East/Jan = 1.
fn paged_app(page: &str) -> App {
let mut wb = Workbook::new("Paged");
wb.add_category("Region").unwrap();
wb.add_category("Month").unwrap();
wb.add_item("Region", "East").unwrap();
wb.add_item("Month", "Jan").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Page);
view.set_axis("Month", Axis::Row);
view.set_axis("_Measure", Axis::Column);
view.set_page_selection("Region", page);
wb.model.set_cell(
CellKey::new(vec![
("Month".into(), "Jan".into()),
("Region".into(), "East".into()),
("_Measure".into(), "Revenue".into()),
]),
CellValue::Number(1.0),
);
App::new(wb, None)
}
fn press(app: &mut App, keys: &str) {
for c in keys.chars() {
let code = match c {
'\n' => KeyCode::Enter,
c => KeyCode::Char(c),
};
app.handle_key(KeyEvent::new(code, KeyModifiers::NONE))
.unwrap();
}
}
/// Every stored key naming `Region=region`, however it got there.
fn stored_under(app: &App, region: &str) -> usize {
let model = &app.model_state.workbook.model;
model
.data
.iter_cells()
.filter(|(key, _)| key.get("Region") == Some(region))
.count()
}
fn refusal(item: &str) -> String {
improvise_protocol::non_item_refusal("Region", item)
}
/// BUG (improvise-yc9r) — see `context/test-evidence/command.md`.
#[test]
fn i_on_a_non_item_cell_refuses_at_open_and_says_why() {
let mut app = paged_app("Mars");
press(&mut app, "i");
assert_eq!(app.view_state.mode, AppMode::Normal, "no editor was opened");
assert_eq!(app.view_state.status_msg, refusal("Mars"));
}
/// improvise-yc9r: `p` on a non-item cell writes no orphan. Evidence file.
#[test]
fn p_on_a_non_item_cell_writes_nothing() {
let mut app = paged_app("Mars");
app.view_state.yanked = Some(CellValue::Number(7.0));
press(&mut app, "p");
assert_eq!(
stored_under(&app, "Mars"),
0,
"an orphan Mars cell was written"
);
let status = &app.view_state.status_msg;
assert!(status.ends_with(&refusal("Mars")), "{status}");
}
/// improvise-yc9r: an editor opened past `i` (`:enter-mode editing`) still
/// refuses its commit rather than writing an orphan. Evidence file.
#[test]
fn a_commit_on_a_non_item_cell_writes_no_orphan() {
let mut app = paged_app("Mars");
press(&mut app, ":enter-mode editing\n");
assert!(
app.view_state.mode.is_editing(),
"setup: the editor is open"
);
press(&mut app, "5\n");
assert_eq!(
stored_under(&app, "Mars"),
0,
"the commit wrote an orphan Mars cell"
);
let status = &app.view_state.status_msg;
assert!(status.ends_with(&refusal("Mars")), "{status}");
}
/// `uneditable` declines a non-item cell, so an advance re-opens no editor
/// on one (#265 meets #277). Evidence: `context/test-evidence/command.md`.
#[test]
fn an_advance_re_opens_no_editor_on_a_non_item_cell() {
use crate::ui::effect::Effect;
let reopen = crate::ui::effect::EnterEditAtCursor {
target_mode: AppMode::editing(),
};
let mut control = paged_app("East");
reopen.apply(&mut control);
assert!(
control.view_state.mode.is_editing(),
"control: a live page re-opens"
);
let mut app = paged_app("Mars");
reopen.apply(&mut app);
assert_eq!(app.view_state.mode, AppMode::Normal, "no editor re-opened");
assert_eq!(app.view_state.status_msg, refusal("Mars"));
}
}
#[cfg(test)]
mod snapshot_restore {
use super::*;
fn every_field(app: &App) -> Vec<(&'static str, String)> {
vec![
("workbook", format!("{:?}", app.model_state.workbook)),
("dirty", format!("{:?}", app.model_state.dirty)),
(
"document_generation",
format!("{:?}", app.document_generation),
),
("layout_version", format!("{:?}", app.layout_version)),
("cursor_epoch", format!("{:?}", app.cursor_epoch)),
("view_state", format!("{:?}", app.view_state)),
("abort_effects", format!("{:?}", app.abort_effects)),
("batch_aborted", format!("{:?}", app.batch_aborted)),
("batch_refused", format!("{:?}", app.batch_refused)),
(
"last_reshaped_views",
format!("{:?}", app.last_reshaped_views),
),
(
"last_reframed_views",
format!("{:?}", app.last_reframed_views),
),
(
"written_view_frames",
format!("{:?}", app.written_view_frames),
),
(
"last_reframed_elsewhere",
format!("{:?}", app.last_reframed_elsewhere),
),
]
}
fn write_every_field(app: &mut App) {
app.model_state.workbook.add_category("Region").unwrap();
app.model_state.dirty = !app.model_state.dirty;
app.document_generation += 1;
app.layout_version += 1;
app.cursor_epoch += 1;
app.view_state.mode = AppMode::Help;
app.cursor_mut().selected = (3, 2);
app.abort_effects = !app.abort_effects;
app.batch_aborted = !app.batch_aborted;
app.batch_refused = !app.batch_refused;
app.last_reshaped_views.insert("Moved".into());
app.last_reframed_views.insert("Moved".into());
app.written_view_frames.insert("Moved".into(), 7);
app.last_reframed_elsewhere.insert("Moved".into());
}
#[test]
fn restore_puts_back_every_field_the_snapshot_took() {
let mut app = App::new(Workbook::new("T"), None);
let before = every_field(&app);
let snapshot = app.snapshot();
write_every_field(&mut app);
for ((name, was), (_, now)) in before.iter().zip(every_field(&app)) {
assert_ne!(*was, now, "the fixture did not move `{name}`");
}
app.restore(snapshot);
for ((name, was), (_, now)) in before.iter().zip(every_field(&app)) {
assert_eq!(*was, now, "restore left `{name}` as the frame wrote it");
}
}
#[test]
fn restore_puts_back_the_mode_and_cursor_not_only_the_view_name_and_status() {
let mut app = App::new(Workbook::new("T"), None);
let snapshot = app.snapshot();
app.view_state.mode = AppMode::Help;
app.cursor_mut().selected = (3, 2);
app.restore(snapshot);
assert_eq!(app.view_state.mode, AppMode::Normal);
assert_eq!(app.cursor().selected, (0, 0));
}
}
+288 -40
View File
@@ -13,7 +13,7 @@ use super::app::{App, AppMode, ViewFrame};
/// inverses) and re-exported here, where most callers already look for it.
pub use super::app::Panel;
pub(crate) const RECORD_COORDS_CANNOT_BE_EMPTY: &str = "Record coordinates cannot be empty";
pub(crate) use improvise_protocol::RECORD_COORDS_CANNOT_BE_EMPTY;
/// What an effect changes, so a server can decide whether connected clients
/// have to be told about it (improvise-mae).
@@ -231,17 +231,17 @@ impl Effect for AddFormula {
}
}
/// Remove the formula `id` names. A formula already gone is refused and stops
/// the batch, so a frame that named it is rolled back (improvise-tej1).
#[derive(Debug)]
pub struct RemoveFormula {
pub target: String,
pub target_category: String,
pub id: FormulaId,
}
impl Effect for RemoveFormula {
fn apply(&self, app: &mut App) {
app.model_state
.workbook
.model
.remove_formula(&self.target, &self.target_category);
if let Err(e) = app.model_state.workbook.model.delete_formula(&self.id) {
refuse_and_stop(app, e.to_string());
}
}
fn kind(&self) -> EffectKind {
EffectKind::Model(ModelChange::CellData)
@@ -324,6 +324,20 @@ pub struct EnterEditAtCursor {
}
impl Effect for EnterEditAtCursor {
fn apply(&self, app: &mut App) {
let (row, col) = app.cursor().selected;
let model = &app.model_state.workbook.model;
match crate::command::cmd::core::uneditable(model, &app.layout, row, col) {
Some(refusal) => app.view_state.status_msg = refusal,
None => self.open(app),
}
}
fn kind(&self) -> EffectKind {
EffectKind::ViewOnly
}
}
impl EnterEditAtCursor {
/// Open the editor on the cursor's cell, seeded with its unformatted value.
fn open(&self, app: &mut App) {
let value = app.display_value_at_cursor();
app.view_state
.buffers
@@ -331,9 +345,6 @@ impl Effect for EnterEditAtCursor {
app.view_state.opened_editor = Some(improvise_protocol::OpenedEditor::on(value));
app.view_state.mode = self.target_mode.clone();
}
fn kind(&self) -> EffectKind {
EffectKind::ViewOnly
}
}
#[derive(Debug)]
@@ -539,8 +550,14 @@ pub struct ToggleGroup {
}
impl Effect for ToggleGroup {
fn apply(&self, app: &mut App) {
let chain = app
.model_state
.workbook
.model
.group_chain(&self.category, &self.group);
let chain: Vec<&str> = chain.iter().map(String::as_str).collect();
on_view(app, &self.view, |v| {
v.toggle_group_collapse(&self.category, &self.group);
v.toggle_group_collapse(&self.category, &chain);
});
}
fn kind(&self) -> EffectKind {
@@ -748,11 +765,36 @@ impl Effect for ApplyAndClearDrill {
}
}
/// Move the record at `record` to `category = item`: the key path's and the wire's
/// one door, through the same [`RecordRewrite`] a drill uses (improvise-fdk1).
#[derive(Debug)]
pub struct RenameRecordCoordinate {
pub record: CellKey,
pub category: String,
pub item: String,
}
impl Effect for RenameRecordCoordinate {
fn apply(&self, app: &mut App) {
let model = &app.model_state.workbook.model;
let row = crate::view::record_row(model, &self.record);
let col = RecordCol::Category(self.category.clone());
let rewrite = RecordRewrite::of(model, &row, [(&col, self.item.as_str())]);
for effect in rewrite.effects() {
effect.apply(app);
}
}
fn kind(&self) -> EffectKind {
EffectKind::Model(ModelChange::CellData)
}
}
/// A records row's edits composed into the one record they mean, read off the model
/// before anything is written. Why composed: `context/code/command.md`, `RecordRewrite`.
#[derive(Debug, Default, PartialEq)]
pub(crate) struct RecordRewrite {
refused: bool,
refusals: Vec<String>,
/// A category the row names is gone: nothing of this rewrite may be written.
unwritable: bool,
items: Vec<(String, String)>,
clears: Vec<CellKey>,
sets: Vec<(CellKey, CellValue)>,
@@ -766,12 +808,14 @@ impl RecordRewrite {
row: &RecordRow,
edits: impl IntoIterator<Item = (&'a RecordCol, &'a str)>,
) -> Self {
let mut refused = false;
let mut refusals = Vec::new();
let mut renames: BTreeMap<&str, &str> = BTreeMap::new();
let mut measures: BTreeMap<&str, Option<CellValue>> = BTreeMap::new();
for (col, value) in edits {
match col {
RecordCol::Category(_) if value.is_empty() => refused = true,
RecordCol::Category(_) if value.is_empty() => {
refusals.push(RECORD_COORDS_CANNOT_BE_EMPTY.to_string())
}
RecordCol::Category(category) => {
renames.insert(category.as_str(), value);
}
@@ -783,6 +827,40 @@ impl RecordRewrite {
RecordCol::NewMeasure | RecordCol::NewCategory => {}
}
}
let gone: std::collections::BTreeSet<&str> = row
.coords
.0
.iter()
.map(|(category, _)| category.as_str())
.chain(renames.keys().copied())
.filter(|category| model.category(category).is_none())
.collect();
match gone.is_empty() {
true => Self::compose(model, row, refusals, renames, measures),
false => Self::unwritable(gone),
}
}
/// A row naming a category the model no longer has: refused, and nothing
/// written, because a write would create a cell no category owns (improvise-qz9p).
fn unwritable(gone: std::collections::BTreeSet<&str>) -> Self {
Self {
refusals: gone
.into_iter()
.map(|category| format!("Unknown category '{category}'"))
.collect(),
unwritable: true,
..Self::default()
}
}
fn compose(
model: &crate::model::Model,
row: &RecordRow,
refusals: Vec<String>,
renames: BTreeMap<&str, &str>,
mut measures: BTreeMap<&str, Option<CellValue>>,
) -> Self {
let renamed = |key: &CellKey| {
renames.iter().fold(key.clone(), |key, (category, item)| {
key.with(*category, *item)
@@ -799,7 +877,7 @@ impl RecordRewrite {
.collect(),
};
let mut rewrite = RecordRewrite {
refused,
refusals,
items: renames
.iter()
.map(|(category, item)| (category.to_string(), item.to_string()))
@@ -824,20 +902,40 @@ impl RecordRewrite {
rewrite
}
/// Several rows' rewrites as one: every clear lands before any write, so a
/// swap is a swap, and where two rows land on one cell the later row wins.
/// Several rows' rewrites as one: every clear before any write, the later row winning
/// a shared cell, nothing written if any row is unwritable. `context/code/command.md`.
pub(crate) fn merged(rewrites: impl IntoIterator<Item = Self>) -> Self {
rewrites.into_iter().fold(Self::default(), |mut all, one| {
all.refused |= one.refused;
all.items.extend(one.items);
all.clears.extend(one.clears);
all.sets.extend(one.sets);
all
})
let rewrites: Vec<Self> = rewrites.into_iter().collect();
match rewrites.iter().any(|one| one.unwritable) {
true => Self::discarded(rewrites.into_iter().filter(|one| one.unwritable)),
false => rewrites.into_iter().fold(Self::default(), |mut all, one| {
all.refusals.extend(one.refusals);
all.items.extend(one.items);
all.clears.extend(one.clears);
all.sets.extend(one.sets);
all
}),
}
}
/// One refusal naming every gone category and saying the whole commit was dropped,
/// so no writable row's refusal can overwrite it (improvise-o9q7).
fn discarded(unwritable: impl Iterator<Item = Self>) -> Self {
let why: std::collections::BTreeSet<String> =
unwritable.flat_map(|one| one.refusals).collect();
let why: Vec<String> = why.into_iter().collect();
Self {
refusals: vec![format!(
"{}: every staged edit was discarded",
why.join("; ")
)],
unwritable: true,
..Self::default()
}
}
pub(crate) fn effects(self) -> Vec<Box<dyn Effect>> {
let refusal = self.refused.then(|| refuse(RECORD_COORDS_CANNOT_BE_EMPTY));
let refusal = self.refusals.into_iter().map(refuse);
let items = self
.items
.into_iter()
@@ -963,6 +1061,10 @@ pub enum WizardOp {
PopFormulaChar,
PushNameChar(char),
PopNameChar,
ChooseDuplicate,
PushDuplicateNameChar(char),
PopDuplicateNameChar,
Scroll(i32),
}
impl WizardOp {
@@ -984,6 +1086,10 @@ impl WizardOp {
Self::PopFormulaChar => wizard.pop_formula_char(),
Self::PushNameChar(c) => wizard.push_name_char(*c),
Self::PopNameChar => wizard.pop_name_char(),
Self::ChooseDuplicate => wizard.choose_duplicate(),
Self::PushDuplicateNameChar(c) => wizard.push_duplicate_name_char(*c),
Self::PopDuplicateNameChar => wizard.pop_duplicate_name_char(),
Self::Scroll(delta) => wizard.scroll(*delta),
}
}
}
@@ -1140,7 +1246,7 @@ pub struct ImportJsonHeadless {
impl Effect for ImportJsonHeadless {
fn apply(&self, app: &mut App) {
use crate::import::analyzer::{FieldKind, analyze_records};
use crate::import::wizard::ImportPipeline;
use crate::import::wizard::{DuplicatePolicy, ImportPipeline};
let is_csv = self
.path
@@ -1213,6 +1319,9 @@ impl Effect for ImportJsonHeadless {
.unwrap_or("Imported Model")
.to_string(),
formulas: vec![],
// Headless: nobody to ask, so a collision refuses (improvise-cgqt).
on_duplicate: DuplicatePolicy::Refuse,
sources: vec![],
};
match pipeline.build_model_with_warnings() {
@@ -1226,7 +1335,11 @@ impl Effect for ImportJsonHeadless {
};
}
Err(e) => {
refuse_and_stop(app, format!("Import error: {e}"));
let hint = match pipeline.collisions().is_empty() {
true => "",
false => " (use :import to choose)",
};
refuse_and_stop(app, format!("Import error: {e}{hint}"));
}
}
}
@@ -1820,8 +1933,8 @@ mod tests {
#[test]
fn add_category_past_the_limit_reports_the_failure() {
let mut app = test_app();
// test_app() holds 2 regular categories; fill up to the limit of 12.
for i in 0..10 {
// test_app() holds 2 regular categories; fill up to the limit.
for i in 0..crate::model::MAX_CATEGORIES - 2 {
AddCategory(format!("Cat{i}")).apply(&mut app);
}
app.view_state.status_msg = "Added category \"Overflow\"".to_string();
@@ -1832,7 +1945,7 @@ mod tests {
.model
.category("Overflow")
.is_none(),
"the 13th category must not be added"
"the category past the limit must not be added"
);
assert!(
app.view_state.status_msg.contains("Maximum"),
@@ -1971,8 +2084,9 @@ mod tests {
.apply(&mut app);
assert!(!app.model_state.workbook.model.formulas().is_empty());
RemoveFormula {
target: "Clothing".to_string(),
target_category: "Type".to_string(),
id: crate::formula::parse_formula("Clothing = Food * 2", "Type")
.unwrap()
.id(),
}
.apply(&mut app);
assert!(app.model_state.workbook.model.formulas().is_empty());
@@ -2947,8 +3061,9 @@ mod tests {
}),
("RemoveFormula", || {
Box::new(RemoveFormula {
target: "Profit".into(),
target_category: "_Measure".into(),
id: crate::formula::parse_formula("Profit = Revenue - Cost", "_Measure")
.unwrap()
.id(),
})
}),
];
@@ -2971,8 +3086,9 @@ mod tests {
}
if label == "AddFormula" {
app.apply_effects(vec![Box::new(RemoveFormula {
target: "Profit".into(),
target_category: "_Measure".into(),
id: crate::formula::parse_formula("Profit = Revenue - Cost", "_Measure")
.unwrap()
.id(),
})]);
}
app.model_state.dirty = false;
@@ -3547,6 +3663,107 @@ mod tests {
);
}
// ── A category removed while the drill is open (improvise-qz9p) ─────
/// Leave the drill after staging `edits` into a drill whose `Type` was removed
/// underneath it; the store left behind, and whether the commit was refused.
fn commit_after_removing_type(
mut app: App,
edits: &[(usize, RecordCol, &str)],
) -> (std::collections::BTreeSet<String>, bool) {
app.model_state.workbook.remove_category("Type");
for (row, col, value) in edits {
stage(&mut app, *row, col.clone(), value);
}
app.apply_effects(vec![Box::new(ApplyAndClearDrill)]);
(drill_store(&app), app.last_batch_refused())
}
#[test]
fn a_drill_rename_into_a_removed_category_is_refused_and_writes_nothing() {
let (store, refused) = commit_after_removing_type(
two_record_drill(),
&[(1, category("Type"), "Drink"), (1, measure("Cost"), "999")],
);
assert!(refused, "the commit must say it could not be applied");
assert_eq!(store, std::collections::BTreeSet::new(), "no orphan cell");
}
#[test]
fn a_drill_measure_edit_on_a_record_whose_category_was_removed_is_refused() {
let (store, refused) =
commit_after_removing_type(two_record_drill(), &[(1, measure("Cost"), "999")]);
assert!(refused, "the commit must say it could not be applied");
assert_eq!(store, std::collections::BTreeSet::new(), "no orphan cell");
}
/// A drill over `Food/Jan/Cost = 2` (row 0) and a sparse `Jan/Cost = 17` naming no
/// Type (row 1).
fn typed_and_untyped_drill() -> App {
let mut app = test_app();
let typed = drill_cell("Food", "Jan", "Cost");
let untyped = CellKey::new(vec![
("Month".into(), "Jan".into()),
("_Measure".into(), "Cost".into()),
]);
let cells = [
(typed, CellValue::Number(2.0)),
(untyped, CellValue::Number(17.0)),
];
for (key, value) in cells.clone() {
app.model_state.workbook.model.set_cell(key, value);
}
let records = RecordRow::group(cells);
assert_eq!(records.len(), 2);
assert_eq!(records[1].coords.get("Type"), None, "row 1 names no Type");
StartDrill(records).apply(&mut app);
app
}
#[test]
fn a_sparse_drill_row_renamed_into_a_removed_category_is_refused_and_writes_nothing() {
let (store, refused) = commit_after_removing_type(
typed_and_untyped_drill(),
&[(1, category("Type"), "Drink")],
);
assert!(refused, "the commit must say it could not be applied");
assert_eq!(
store,
["-/Jan/Cost = 17".to_string()].into_iter().collect(),
"row 1 names Type only through its rename; no orphan Drink cell"
);
}
#[test]
fn an_unwritable_drill_commit_reports_the_discard_not_a_later_rows_refusal() {
let mut app = typed_and_untyped_drill();
app.model_state.workbook.remove_category("Type");
stage(&mut app, 0, measure("Cost"), "5");
stage(&mut app, 1, category("Month"), "");
app.apply_effects(vec![Box::new(ApplyAndClearDrill)]);
let status = &app.view_state.status_msg;
assert!(status.contains("Unknown category 'Type'"), "{status}");
assert!(
status.contains("every staged edit was discarded"),
"{status}"
);
assert!(!status.contains(RECORD_COORDS_CANNOT_BE_EMPTY), "{status}");
}
#[test]
fn a_drill_commit_with_one_unwritable_row_writes_none_of_its_rows() {
let (store, refused) = commit_after_removing_type(
typed_and_untyped_drill(),
&[(0, measure("Cost"), "5"), (1, measure("Cost"), "999")],
);
assert!(refused, "the commit must say it could not be applied");
assert_eq!(
store,
["-/Jan/Cost = 17".to_string()].into_iter().collect(),
"row 1 is writable, but a drill commit is atomic"
);
}
// ── Toggle effects ──────────────────────────────────────────────────
#[test]
@@ -3665,10 +3882,11 @@ mod tests {
}
.apply(&mut app);
assert!(
app.model_state
!app.model_state
.workbook
.active_view()
.is_group_collapsed("Type", "MyGroup")
.is_group_collapsed("Type", &["MyGroup"]),
"a group starts collapsed, so the first toggle opens it"
);
ToggleGroup {
view: ViewTarget::Active,
@@ -3677,10 +3895,10 @@ mod tests {
}
.apply(&mut app);
assert!(
!app.model_state
app.model_state
.workbook
.active_view()
.is_group_collapsed("Type", "MyGroup")
.is_group_collapsed("Type", &["MyGroup"])
);
}
@@ -3742,6 +3960,36 @@ mod tests {
);
}
/// improvise-cgqt: `:import-json` cannot ask, so colliding records refuse
/// rather than sum, and the refusal points at the wizard that can.
#[test]
fn import_headless_refuses_records_that_collide() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("data.csv");
std::fs::write(&path, "Region,Revenue\nEast,10\nEast,20\n").unwrap();
let mut app = test_app();
let before = app.model_state.workbook.model.data.iter_cells().count();
ImportJsonHeadless {
path,
model_name: None,
array_path: None,
}
.apply(&mut app);
let status = &app.view_state.status_msg;
assert!(status.contains("shared by 2 records"), "{status}");
assert!(
status.contains(":import"),
"names the way to choose: {status}"
);
assert_eq!(
app.model_state.workbook.model.data.iter_cells().count(),
before,
"the document is untouched"
);
}
#[test]
fn import_headless_csv_full_rows_report_plain_success() {
let dir = tempfile::tempdir().unwrap();
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+133 -46
View File
@@ -2,11 +2,11 @@
//! state (improvise-gxi/s17o). See `context/code/command.md`.
use improvise_core::model::record::RecordHash;
pub use improvise_protocol::records_named;
use improvise_protocol::{Basis, Command, ModelCommand, Rejection};
use improvise_protocol::{Basis, Command, ModelCommand, NamedRecord, Rejection};
pub use improvise_protocol::{named_records, records_named};
use crate::command::key::KeyEvent;
use crate::ui::app::App;
use crate::ui::app::{App, Grids};
use crate::ui::effect::{self, Effect, ViewTarget};
use crate::ui::project::{Dirty, document_projections};
@@ -28,15 +28,16 @@ impl<'a> CheckedBatch<'a> {
// The frame's basis is the one its first mutation stated; a later
// command's own `Basis` field is deliberately not read (clause 1).
let mut frame_basis: Option<&Basis> = None;
let grids = app.grids();
let mut checked = Vec::with_capacity(cmds.len());
for cmd in cmds {
let one = match frame_basis {
Some(basis) => Checked::within_a_frame(app, basis, cmd)?,
Some(basis) => Checked::within_a_frame(app, &grids, basis, cmd)?,
None => {
if let Command::Model { basis, .. } = cmd {
frame_basis = Some(basis);
}
Checked::against(app, cmd)?
Checked::against_in(app, &grids, cmd)?
}
};
checked.push(one);
@@ -53,10 +54,11 @@ pub fn broken_chain(app: &App, cmd: &Command) -> Option<Rejection> {
};
// No create-exemption here, unlike the pre-frame check: a record that does
// not resolve NOW is one `resolve_cell` will drop the write for, silently.
records_named(model)
let grids = app.grids();
named_records(model)
.into_iter()
.find(|hash| app.resolve_record(*hash).is_none())
.map(Rejection::ChainBroken)
.find(|named| grids.resolve_named(*named).is_none())
.map(|named| Rejection::ChainBroken(named.record()))
}
impl<'a> IntoIterator for CheckedBatch<'a> {
@@ -72,8 +74,13 @@ impl<'a> Checked<'a> {
/// Answer for a command, against the state it would be applied to. Total
/// over the vocabulary; only [`Command::Model`] can be refused.
pub fn against(app: &App, cmd: &'a Command) -> Result<Self, Rejection> {
Self::against_in(app, &app.grids(), cmd)
}
/// [`Self::against`], resolving through grids a whole frame shares (improvise-kimw).
fn against_in(app: &App, grids: &Grids, cmd: &'a Command) -> Result<Self, Rejection> {
let refused = match cmd {
Command::Model { basis, cmd: model } => stale_basis(app, basis, model),
Command::Model { basis, cmd: model } => stale_basis(app, grids, basis, model),
_ => None,
};
match refused {
@@ -84,9 +91,14 @@ impl<'a> Checked<'a> {
/// A command in a frame whose document tiers are already answered
/// (clause 2), answering only for its own records. Private, not a bypass.
fn within_a_frame(app: &App, basis: &Basis, cmd: &'a Command) -> Result<Self, Rejection> {
fn within_a_frame(
app: &App,
grids: &Grids,
basis: &Basis,
cmd: &'a Command,
) -> Result<Self, Rejection> {
let refused = match cmd {
Command::Model { cmd: model, .. } => stale_records_in_a_frame(app, basis, model),
Command::Model { cmd: model, .. } => stale_records_in_a_frame(app, grids, basis, model),
_ => None,
};
match refused {
@@ -134,21 +146,38 @@ pub fn apply_command(app: &mut App, cmd: Checked<'_>) -> Dirty {
/// The other half of the derivation: a [`ModelCommand`] back to the effect it
/// was derived from (improvise-kmgs). Exhaustive on purpose.
pub fn model_effects(app: &App, cmd: &ModelCommand) -> Vec<Box<dyn Effect>> {
let grids = app.grids();
let one: Box<dyn Effect> = match cmd {
// A cell the host projects as no `Record` is refused, so the frame is
// A cell its view projects as no `Record` is refused, so the frame is
// Rejected rather than written (improvise-ciw8, ADR 002).
ModelCommand::SetCell { cell, value } => {
match (app.unwritable(cell), app.resolve_cell(cell)) {
ModelCommand::SetCell { view, cell, value } => {
match (grids.unwritable(view, cell), grids.resolve_cell(view, cell)) {
(Some(why), _) => effect::refuse(why),
(None, None) => return vec![],
(None, Some(key)) => Box::new(effect::SetCell(key, value.clone())),
}
}
ModelCommand::ClearCell(cell) => match (app.unwritable(cell), app.resolve_cell(cell)) {
(Some(why), _) => effect::refuse(why),
(None, None) => return vec![],
(None, Some(key)) => Box::new(effect::ClearCell(key)),
},
ModelCommand::ClearCell { view, cell } => {
match (grids.unwritable(view, cell), grids.resolve_cell(view, cell)) {
(Some(why), _) => effect::refuse(why),
(None, None) => return vec![],
(None, Some(key)) => Box::new(effect::ClearCell(key)),
}
}
ModelCommand::RenameRecordCoordinate {
record,
category,
item,
} => {
let Some(record) = grids.resolve_named(NamedRecord::Stored(*record)) else {
return vec![];
};
Box::new(effect::RenameRecordCoordinate {
record,
category: category.clone(),
item: item.clone(),
})
}
ModelCommand::AddItem { category, item } => Box::new(effect::AddItem {
category: category.clone(),
item: item.clone(),
@@ -173,13 +202,7 @@ pub fn model_effects(app: &App, cmd: &ModelCommand) -> Vec<Box<dyn Effect>> {
raw: raw.clone(),
target_category: target_category.clone(),
}),
ModelCommand::RemoveFormula {
target,
target_category,
} => Box::new(effect::RemoveFormula {
target: target.clone(),
target_category: target_category.clone(),
}),
ModelCommand::RemoveFormula { id } => Box::new(effect::RemoveFormula { id: id.clone() }),
ModelCommand::ReplaceFormula { id, was, raw } => Box::new(effect::ReplaceFormula {
id: id.clone(),
was: was.clone(),
@@ -268,35 +291,72 @@ pub fn model_effects(app: &App, cmd: &ModelCommand) -> Vec<Box<dyn Effect>> {
/// Is this batch's basis still current? `None` means apply it. The three tiers
/// are checked largest-first, so the answer names the largest thing that moved.
pub fn stale_basis(app: &App, basis: &Basis, cmd: &ModelCommand) -> Option<Rejection> {
fn stale_basis(app: &App, grids: &Grids, basis: &Basis, cmd: &ModelCommand) -> Option<Rejection> {
let document_moved = (basis.document_generation != app.document_generation())
.then_some(Rejection::DocumentReplaced);
let layout_moved =
(basis.layout_version != app.layout_version()).then_some(Rejection::LayoutMoved);
document_moved.or(layout_moved).or_else(|| {
records_named(cmd)
records_at_stake(app, cmd)
.into_iter()
.find_map(|h| stale_record(app, basis, h))
.find_map(|named| stale_record(app, grids, basis, named))
})
}
/// [`records_named`], plus what only the document can name: a rename's
/// destination, when it is a stored record (improvise-milr, `command.md`).
pub fn records_at_stake<'a>(app: &App, cmd: &'a ModelCommand) -> Vec<NamedRecord<'a>> {
let mut records = named_records(cmd);
if let ModelCommand::RenameRecordCoordinate {
record,
category,
item,
} = cmd
{
let destination = app
.model_state
.workbook
.model
.data
.resolve_record(*record)
.map(|source| RecordHash::of(&source.clone().with(category, item)))
.filter(|dest| dest != record)
.filter(|dest| {
app.model_state
.workbook
.model
.data
.resolve_record(*dest)
.is_some()
});
records.extend(destination.map(NamedRecord::Stored));
}
records
}
/// The record tier for a command inside a frame whose document tiers are
/// already answered, skipping the records the frame is about to create.
fn stale_records_in_a_frame(app: &App, basis: &Basis, cmd: &ModelCommand) -> Option<Rejection> {
records_named(cmd)
fn stale_records_in_a_frame(
app: &App,
grids: &Grids,
basis: &Basis,
cmd: &ModelCommand,
) -> Option<Rejection> {
records_at_stake(app, cmd)
.into_iter()
.filter(|hash| !will_be_created_by_the_frame(app, basis, *hash))
.find_map(|hash| stale_record(app, basis, hash))
.filter(|named| !will_be_created_by_the_frame(grids, basis, *named))
.find_map(|named| stale_record(app, grids, basis, named))
}
/// Is this a record the frame is about to create? **A property, not a list of
/// commands** — see `context/code/command.md`. Only the pair exempts.
fn will_be_created_by_the_frame(app: &App, basis: &Basis, hash: RecordHash) -> bool {
app.resolve_record(hash).is_none() && basis.record(hash).is_none()
fn will_be_created_by_the_frame(grids: &Grids, basis: &Basis, named: NamedRecord) -> bool {
grids.resolve_named(named).is_none() && basis.record(named.record()).is_none()
}
fn stale_record(app: &App, basis: &Basis, hash: RecordHash) -> Option<Rejection> {
match (app.resolve_record(hash), basis.record(hash)) {
fn stale_record(app: &App, grids: &Grids, basis: &Basis, named: NamedRecord) -> Option<Rejection> {
let hash = named.record();
match (grids.resolve_named(named), basis.record(hash)) {
(None, _) => Some(Rejection::RecordGone(hash)),
(Some(_), None) => Some(Rejection::RecordUnstated(hash)),
(Some(_), Some(claimed)) => {
@@ -312,9 +372,9 @@ pub fn caught_up(app: &App, cmd: ModelCommand) -> Command {
basis: Basis {
document_generation: app.document_generation(),
layout_version: app.layout_version(),
records: records_named(&cmd)
records: records_at_stake(app, &cmd)
.into_iter()
.map(|h| (h, app.record_version_of(h)))
.map(|named| (named.record(), app.record_version_of(named.record())))
.collect(),
},
cmd,
@@ -654,6 +714,8 @@ pub(crate) mod tests {
/// resolves — otherwise every walk below fails for a reason not its own.
fn model_app() -> App {
let mut app = app();
let workbook = &mut app.model_state.workbook;
workbook.add_item(MEASURE_CATEGORY, "Revenue").unwrap();
app.model_state
.workbook
.model
@@ -667,10 +729,19 @@ pub(crate) mod tests {
pub(crate) fn every_model_command() -> Vec<ModelCommand> {
vec![
ModelCommand::SetCell {
view: "Default".into(),
cell: record_cell(&a_stored_cell()),
value: CellValue::Number(1.0),
},
ModelCommand::ClearCell(record_cell(&a_stored_cell())),
ModelCommand::ClearCell {
view: "Default".into(),
cell: record_cell(&a_stored_cell()),
},
ModelCommand::RenameRecordCoordinate {
record: record_cell(&a_stored_cell()).record,
category: "Region".into(),
item: "West".into(),
},
ModelCommand::AddItem {
category: "Region".into(),
item: "South".into(),
@@ -689,8 +760,9 @@ pub(crate) mod tests {
target_category: "_Measure".into(),
},
ModelCommand::RemoveFormula {
target: "Profit".into(),
target_category: "_Measure".into(),
id: crate::formula::parse_formula("Profit = Revenue", "_Measure")
.unwrap()
.id(),
},
ModelCommand::ReplaceFormula {
id: crate::formula::parse_formula("Profit = Revenue", "_Measure")
@@ -757,7 +829,8 @@ pub(crate) mod tests {
fn variant_name(cmd: &ModelCommand) -> &'static str {
match cmd {
ModelCommand::SetCell { .. } => "SetCell",
ModelCommand::ClearCell(_) => "ClearCell",
ModelCommand::ClearCell { .. } => "ClearCell",
ModelCommand::RenameRecordCoordinate { .. } => "RenameRecordCoordinate",
ModelCommand::AddItem { .. } => "AddItem",
ModelCommand::AddItemInGroup { .. } => "AddItemInGroup",
ModelCommand::RemoveItem { .. } => "RemoveItem",
@@ -959,6 +1032,16 @@ pub(crate) mod tests {
]);
wb.model.set_cell(key, CellValue::Number(n as f64 + 1.0));
}
// Several entries, so two instances' HashMaps can disagree on order (improvise-m6op).
use crate::formula::AggFunc;
for (measure, agg) in [
("Revenue", AggFunc::Avg),
("Cost", AggFunc::Min),
("Units", AggFunc::Max),
("Orders", AggFunc::Count),
] {
wb.model.measure_agg.insert(measure.into(), agg);
}
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Row);
view.set_axis("Month", Axis::Column);
@@ -969,9 +1052,9 @@ pub(crate) mod tests {
view.hide_item("Region", "West");
view.hide_item("Region", "South");
view.hide_item("Month", "Mar");
view.toggle_group_collapse("Region", "Coastal");
view.toggle_group_collapse("Region", "Inland");
view.toggle_group_collapse("Month", "Q1");
view.toggle_group_collapse("Region", &["Coastal"]);
view.toggle_group_collapse("Region", &["Inland"]);
view.toggle_group_collapse("Month", &["Q1"]);
app.rebuild_layout();
app
}
@@ -1086,6 +1169,7 @@ pub(crate) mod tests {
fn a_wire_model_command_marks_the_document_dirty() {
for cmd in [
ModelCommand::SetCell {
view: "Default".into(),
cell: RecordCell {
record: RecordHash::of(&crate::model::cell::CellKey::new(vec![(
"Region".to_string(),
@@ -1101,6 +1185,9 @@ pub(crate) mod tests {
},
] {
let mut app = app();
let workbook = &mut app.model_state.workbook;
workbook.add_item(MEASURE_CATEGORY, "Revenue").unwrap();
app.rebuild_layout();
assert!(!app.model_state.dirty, "the fixture starts clean");
let wire = caught_up(&app, cmd.clone());
apply_checked(&mut app, &wire);
+110 -18
View File
@@ -7,8 +7,9 @@ use improvise_protocol::header_level_widths;
use crate::format::{FormatPlan, NumberFormat};
use crate::model::Model;
use crate::model::category::MEASURE_CATEGORY;
use crate::model::cell::{CellKey, CellValue};
use crate::view::{AxisEntry, GridLayout};
use crate::view::{AxisEntry, GridEval, GridLayout};
/// Minimum column width — enough for short numbers/labels + 1 char gap.
pub const MIN_COL_WIDTH: u16 = 5;
@@ -26,9 +27,9 @@ pub struct GridBody {
/// The format each column's TOTAL is drawn with, resolved from the
/// coordinates that survive the row aggregation — `context/code/core.md`.
pub col_formats: Vec<NumberFormat>,
/// Column sums over every row. **Empty** in records mode or an empty grid,
/// which is "this grid has no totals" rather than a strip of zeroes.
pub totals: Vec<f64>,
/// Each column's total over every drawn row (improvise-wm20). **Empty** in
/// records mode, an empty grid, or with `_Measure` on the row axis.
pub totals: Vec<Option<CellValue>>,
/// The requested window, row-major. `None` is a cell the layout has not got.
pub window: Vec<Vec<Option<CellValue>>>,
}
@@ -45,9 +46,14 @@ pub fn grid_body(
let n = layout.col_count();
let records = layout.is_records_mode();
let mut widths = header_level_seed(layout, n);
let totalled = !records && layout.row_count() > 0 && n > 0;
let mut totals = match totalled {
true => vec![0.0f64; n],
// One evaluator for the whole build: a fold every cell and total reaches is
// evaluated once (improvise-h6z1). The model does not change during it.
let eval = layout.evaluator(model);
// A column mixing measures has no total to show: `context/code/command.md`.
let measures_on_rows = layout.row_cats.iter().any(|c| c == MEASURE_CATEGORY);
let totalled = !records && !measures_on_rows && layout.row_count() > 0 && n > 0;
let totals: Vec<Option<CellValue>> = match totalled {
true => (0..n).map(|ci| eval.column_total(ci)).collect(),
false => Vec::new(),
};
let mut window: Vec<Vec<Option<CellValue>>> = Vec::with_capacity(rows.len());
@@ -64,16 +70,13 @@ pub fn grid_body(
false => Vec::new(),
};
for ci in 0..n {
let (key, value) = cell_value(model, layout, records, ri, ci);
let (key, value) = cell_value(layout, &eval, records, ri, ci);
let fmt = plan.at(key.as_ref());
let text = match records {
true => value.as_ref().map(|v| v.to_string()).unwrap_or_default(),
false => fmt.value(value.as_ref()),
};
widen(&mut widths, ci, text.width());
if let Some(sum) = totals.get_mut(ci) {
*sum += value.as_ref().and_then(|v| v.as_f64()).unwrap_or(0.0);
}
if drawn && cols.contains(&ci) {
row_out.push(value);
}
@@ -84,7 +87,7 @@ pub fn grid_body(
}
for (ci, total) in totals.iter().enumerate() {
let fmt = col_formats.get(ci).copied().unwrap_or(plan.view_format());
widen(&mut widths, ci, fmt.number(*total).width());
widen(&mut widths, ci, fmt.value(total.as_ref()).width());
}
GridBody {
@@ -100,8 +103,8 @@ pub fn grid_body(
/// The one evaluation. Records mode reads its wide row; pivot mode aggregates.
fn cell_value(
model: &Model,
layout: &GridLayout,
eval: &GridEval,
records: bool,
row: usize,
col: usize,
@@ -110,7 +113,7 @@ fn cell_value(
true => (None, layout.records_value(row, col)),
false => {
let key = layout.cell_key(row, col);
let value = key.as_ref().and_then(|k| layout.evaluate(model, k));
let value = key.as_ref().and_then(|k| eval.cell(k));
(key, value)
}
}
@@ -122,7 +125,7 @@ fn header_level_seed(layout: &GridLayout, n: usize) -> Vec<u16> {
let mut widths = vec![0u16; n];
let items = layout.col_items.iter().filter_map(|e| match e {
AxisEntry::DataItem { tuple, .. } => Some(tuple),
AxisEntry::GroupHeader { .. } | AxisEntry::GroupEnd => None,
AxisEntry::GroupHeader { .. } | AxisEntry::GroupEnd { .. } => None,
});
for (ci, levels) in items.enumerate().take(n) {
for level in levels {
@@ -151,22 +154,37 @@ pub fn compute_col_widths(model: &Model, layout: &GridLayout, plan: &FormatPlan)
pub fn compute_row_header_width(layout: &GridLayout) -> u16 {
let items = layout.row_items.iter().filter_map(|e| match e {
AxisEntry::DataItem { tuple, .. } => Some(tuple),
AxisEntry::GroupHeader { .. } | AxisEntry::GroupEnd => None,
AxisEntry::GroupHeader { .. } | AxisEntry::GroupEnd { .. } => None,
});
header_level_widths(items, layout.row_cats.len().max(1))
.iter()
.sum()
}
/// The TUI's side-panel column, and the least the grid keeps beside it.
pub const SIDE_PANEL_WIDTH: u16 = 32;
pub const MIN_GRID_WIDTH: u16 = 40;
/// The grid's width once any side panel has its column: the one answer the
/// keypress path and the frame layout share (`context/code/tui.md`).
pub fn grid_width(term_width: u16, side_panel_open: bool) -> u16 {
match side_panel_open {
true => term_width
.saturating_sub(SIDE_PANEL_WIDTH)
.max(MIN_GRID_WIDTH.min(term_width)),
false => term_width,
}
}
/// Count how many columns fit starting from `col_offset` given the available width.
pub fn compute_visible_cols(
col_widths: &[u16],
row_header_width: u16,
term_width: u16,
grid_width: u16,
col_offset: usize,
) -> usize {
// Account for grid border (2 chars)
let data_area_width = term_width
let data_area_width = grid_width
.saturating_sub(2)
.saturating_sub(row_header_width);
let mut acc = 0u16;
@@ -181,3 +199,77 @@ pub fn compute_visible_cols(
}
count.max(1)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::model::types::FOLD_FORMULA_EVALUATIONS;
use crate::view::Axis;
use crate::workbook::Workbook;
/// Q1 = {Jan, Feb} collapsed, Q2 = {Apr}; F0 = Revenue and F(i) = F(i-1) twice.
fn folded_chain() -> Workbook {
let mut wb = Workbook::new("T");
wb.add_category("Month").unwrap();
wb.add_item_in_group("Month", "Apr", "Q2").unwrap();
wb.add_item_in_group("Month", "Jan", "Q1").unwrap();
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("_Measure", "Revenue").unwrap();
for (month, revenue) in [("Jan", 10.0), ("Feb", 30.0)] {
let key = CellKey::new(vec![
("Month".into(), month.into()),
("_Measure".into(), "Revenue".into()),
]);
wb.model.set_cell(key, CellValue::Number(revenue));
}
wb.model
.add_formula(crate::formula::parse_formula("F0 = Revenue", "_Measure").unwrap());
for i in 1..=8 {
let text = format!("F{i} = F{p} + F{p}", p = i - 1);
wb.model
.add_formula(crate::formula::parse_formula(&text, "_Measure").unwrap());
}
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().toggle_group_collapse("Month", &["Q1"]);
wb.model.recompute_formulas(&wb.active_view().none_cats());
wb
}
fn evaluations(run: impl FnOnce()) -> usize {
FOLD_FORMULA_EVALUATIONS.with(|n| n.set(0));
run();
FOLD_FORMULA_EVALUATIONS.with(|n| n.get())
}
/// improvise-0jw5: context/test-evidence/command.md.
#[test]
fn one_grid_build_evaluates_each_fold_formula_once_for_cells_and_totals() {
let wb = folded_chain();
let view = wb.active_view();
let layout = GridLayout::new(&wb.model, view);
let plan = FormatPlan::of(&wb.model, view);
let (rows, cols) = (0..layout.row_count(), 0..layout.col_count());
let apart = evaluations(|| {
for (r, c) in rows.clone().flat_map(|r| cols.clone().map(move |c| (r, c))) {
let key = layout.cell_key(r, c).unwrap();
layout.evaluate(&wb.model, &key);
}
for c in cols.clone() {
layout.evaluator(&wb.model).column_total(c);
}
});
let together = evaluations(|| {
grid_body(&wb.model, &layout, &plan, rows.clone(), cols.clone());
});
assert_eq!(
apart, 90,
"setup: 45 for the Q1 row's cells, 45 for the totals"
);
assert_eq!(
together, 18,
"F0..F8 once for the Q1 row, once for the totals"
);
}
}
+5
View File
@@ -17,6 +17,11 @@ serde = { version = "1", features = ["derive"] }
# wasm-clean.
siphasher = "1"
[features]
# Exposes the fold-evaluation counter to other crates' tests (improvise-0jw5).
# Off in every shipped build; enabled only by improvise-command's dev-dependency.
test-support = []
[dev-dependencies]
proptest = "1"
serde_json = "1"
@@ -0,0 +1,128 @@
//! What a wider document costs: the same records at 12 and at 24 regular
//! categories. improvise-felo.1; the argument is `context/code/core.md` §9.
use std::time::Instant;
use improvise_core::model::cell::{CellKey, CellValue};
use improvise_core::view::{Axis, GridLayout, ViewStep, records_steps};
use improvise_core::workbook::Workbook;
const MEASURE: &str = "_Measure";
const ITEMS: u64 = 4;
/// `records` records over `width` categories of `ITEMS` items, two measures
/// each; coordinates from a fixed LCG so both widths see comparable data.
fn document(width: usize, records: usize) -> Workbook {
let mut wb = Workbook::new("bench");
for c in 0..width {
wb.add_category(format!("C{c}")).expect("under the limit");
}
let mut state: u64 = 0x9E37_79B9_7F4A_7C15;
let mut next = move || {
state = state
.wrapping_mul(6_364_136_223_846_793_005)
.wrapping_add(1_442_695_040_888_963_407);
(state >> 33) % ITEMS
};
for _ in 0..records {
let coords: Vec<(String, String)> = (0..width)
.map(|c| (format!("C{c}"), format!("i{}", next())))
.collect();
for (name, value) in [("Cost", 3.0), ("Rev", 5.0)] {
let mut key = coords.clone();
key.push((MEASURE.to_string(), name.to_string()));
wb.model
.set_cell(CellKey::new(key), CellValue::Number(value));
}
}
for c in 0..width {
let cat = wb.model.category_mut(&format!("C{c}")).expect("added");
for i in 0..ITEMS {
cat.add_item(format!("i{i}"));
}
}
for name in ["Cost", "Rev"] {
wb.model
.category_mut(MEASURE)
.expect("virtual")
.add_item(name);
}
wb
}
/// Best of `runs`, in milliseconds.
fn best(runs: usize, mut f: impl FnMut()) -> f64 {
(0..runs)
.map(|_| {
let t = Instant::now();
f();
t.elapsed().as_secs_f64() * 1000.0
})
.fold(f64::INFINITY, f64::min)
}
fn pivot(wb: &mut Workbook, width: usize) -> f64 {
let view = wb.active_view_mut();
for c in 0..width {
view.set_axis(
&format!("C{c}"),
if c == 0 { Axis::Row } else { Axis::None },
);
}
view.set_axis(MEASURE, Axis::Column);
let (model, view) = (&wb.model, wb.active_view());
best(5, || {
let layout = GridLayout::with_frozen_records(model, view, None);
let eval = layout.evaluator(model);
for r in 0..layout.row_count() {
for c in 0..layout.col_count() {
let key = layout.cell_key(r, c).expect("in bounds");
std::hint::black_box(eval.cell(&key));
}
}
})
}
/// (build, draw): the records layout, then every cell's display text.
fn records(wb: &mut Workbook) -> (f64, f64) {
let steps = records_steps(&wb.model);
let view = wb.active_view_mut();
for step in steps {
if let ViewStep::Axis { category, axis } = step {
view.set_axis(&category, axis);
}
}
let (model, view) = (&wb.model, wb.active_view());
let build = best(5, || {
std::hint::black_box(GridLayout::with_frozen_records(model, view, None));
});
let layout = GridLayout::with_frozen_records(model, view, None);
let draw = best(5, || {
for r in 0..layout.row_count() {
for c in 0..layout.col_count() {
std::hint::black_box(layout.records_display(r, c));
}
}
});
(build, draw)
}
fn main() {
println!(
"{:>6} {:>8} {:>7} {:>9} {:>9} {:>14} {:>13}",
"width", "records", "cells", "store ms", "pivot ms", "rec build ms", "rec draw ms"
);
for records_n in [2_500, 5_000] {
for width in [12, 24] {
let mut built = None;
let store = best(3, || built = Some(document(width, records_n)));
let mut wb = built.expect("ran");
let cells = wb.model.data.iter_cells().count();
let p = pivot(&mut wb, width);
let (rb, rd) = records(&mut wb);
println!(
"{width:>6} {records_n:>8} {cells:>7} {store:>9.1} {p:>9.1} {rb:>14.1} {rd:>13.1}"
);
}
}
}
+111 -4
View File
@@ -1,4 +1,5 @@
use indexmap::IndexMap;
use serde::{Deserialize, Serialize};
pub type CategoryId = usize;
pub type ItemId = usize;
@@ -49,7 +50,7 @@ impl Group {
/// Regular categories store their items explicitly; virtual ones are
/// synthesized at query time by the layout layer. context/code/core.md §9.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, Default)]
pub enum CategoryKind {
#[default]
Regular,
@@ -198,15 +199,61 @@ impl Category {
self.items.keys().map(|s| s.as_str()).collect()
}
/// The items whose own `group` names `group`, in insertion order. Read off the
/// items, never `groups`: `add_item_in_group` registers no `Group` (core.md §14).
/// The items `group` covers — its own and every child group's, through any
/// depth — in insertion order. Read off the items, not `groups` (core.md §14).
pub fn items_in_group(&self, group: &str) -> Vec<&str> {
self.items
.values()
.filter(|i| i.group.as_deref() == Some(group))
.filter(|i| self.item_chain(&i.name).contains(&group))
.map(|i| i.name.as_str())
.collect()
}
/// The parent a `>` line gave `group`; `None` for a root.
pub fn group_parent(&self, group: &str) -> Option<&str> {
self.groups
.iter()
.find(|g| g.name == group)
.and_then(|g| g.parent.as_deref())
}
/// `group` and its ancestors, ROOT FIRST. A cycle built in code stops the walk
/// where it would repeat; a file cannot carry one (`hierarchy_cycle`).
pub fn group_chain<'a>(&'a self, group: &'a str) -> Vec<&'a str> {
let mut chain = vec![group];
while let Some(parent) = chain.last().and_then(|g| self.group_parent(g)) {
if chain.contains(&parent) {
break;
}
chain.push(parent);
}
chain.reverse();
chain
}
/// The groups an item sits in, root first; empty for an ungrouped item.
pub fn item_chain(&self, item: &str) -> Vec<&str> {
self.items
.get(item)
.and_then(|i| i.group.as_deref())
.map(|g| self.group_chain(g))
.unwrap_or_default()
}
/// A cycle among the declared parents, its members in walk order: the one
/// shape a hierarchy can take that is not a forest (core.md §19).
pub fn hierarchy_cycle(&self) -> Option<Vec<&str>> {
self.groups.iter().find_map(|start| {
let mut path: Vec<&str> = vec![start.name.as_str()];
while let Some(parent) = path.last().and_then(|g| self.group_parent(g)) {
if let Some(at) = path.iter().position(|g| *g == parent) {
return Some(path[at..].to_vec());
}
path.push(parent);
}
None
})
}
}
#[cfg(test)]
@@ -296,6 +343,66 @@ mod tests {
assert_eq!(c.groups.len(), 1);
}
/// Year > Quarter > Month, with `2024-03` added LAST so Q1-2024 and 2024 are
/// non-contiguous in insertion order.
fn dates() -> Category {
let mut c = Category::new(0, "Date");
c.add_item_in_group("2024-01", "Q1-2024");
c.add_item_in_group("2024-04", "Q2-2024");
c.add_item_in_group("2025-01", "Q1-2025");
c.add_item("Undated");
c.add_item_in_group("2024-03", "Q1-2024");
c.add_group(Group::new("Q1-2024").with_parent("2024"));
c.add_group(Group::new("Q2-2024").with_parent("2024"));
c.add_group(Group::new("Q1-2025").with_parent("2025"));
c
}
#[test]
fn a_parent_group_covers_every_leaf_beneath_its_child_groups() {
let c = dates();
assert_eq!(
c.items_in_group("2024"),
vec!["2024-01", "2024-04", "2024-03"]
);
assert_eq!(c.items_in_group("Q1-2024"), vec!["2024-01", "2024-03"]);
assert_eq!(c.items_in_group("2025"), vec!["2025-01"]);
assert!(c.items_in_group("Undated").is_empty());
}
#[test]
fn an_items_chain_runs_root_first() {
let c = dates();
assert_eq!(c.item_chain("2024-03"), vec!["2024", "Q1-2024"]);
assert!(c.item_chain("Undated").is_empty());
assert_eq!(c.group_chain("2024"), vec!["2024"]);
assert_eq!(c.group_chain("Q2-2024"), vec!["2024", "Q2-2024"]);
}
#[test]
fn a_cycle_among_parents_is_found_and_named() {
let mut c = dates();
assert_eq!(c.hierarchy_cycle(), None);
c.add_group(Group::new("2024").with_parent("Q2-2024"));
let cycle = c.hierarchy_cycle().expect("2024 -> Q2-2024 -> 2024");
assert!(
cycle.contains(&"2024") && cycle.contains(&"Q2-2024"),
"{cycle:?}"
);
assert!(!cycle.contains(&"Q1-2025"), "{cycle:?}");
assert!(
c.group_chain("Q1-2024").len() <= 3,
"a walk through a cycle stops"
);
}
#[test]
fn a_self_parent_is_a_cycle_of_one() {
let mut c = cat();
c.add_group(Group::new("G").with_parent("G"));
assert_eq!(c.hierarchy_cycle(), Some(vec!["G"]));
}
#[test]
fn item_index_reflects_insertion_order() {
let mut c = cat();
+175 -7
View File
@@ -378,16 +378,30 @@ impl DataStore {
/// Positions in `cells` of every cell matching `partial`, **in insertion
/// order**. The index narrows; the primary map orders (§8).
fn matching_positions(&self, partial: &[(String, String)]) -> Vec<usize> {
// Intern the partial key (lookup only, no new symbols)
let interned_partial: Vec<(Symbol, Symbol)> = partial
self.intern_partial(partial)
.map(|pairs| self.positions_of(&pairs))
.unwrap_or_default()
}
/// `partial` as symbols, or `None` when a name was never interned, which
/// no stored cell can then match.
fn intern_partial(&self, partial: &[(String, String)]) -> Option<Vec<(Symbol, Symbol)>> {
partial
.iter()
.filter_map(|(c, i)| Some((self.symbols.get(c)?, self.symbols.get(i)?)))
.collect();
.map(|(c, i)| Some((self.symbols.get(c)?, self.symbols.get(i)?)))
.collect()
}
if interned_partial.len() < partial.len() {
return vec![];
}
/// How many candidates narrowing by `pairs` would visit: its smallest index set.
fn narrowing_cost(&self, pairs: &[(Symbol, Symbol)]) -> usize {
pairs
.iter()
.map(|pair| self.index.get(pair).map_or(0, HashSet::len))
.min()
.unwrap_or(self.cells.len())
}
fn positions_of(&self, interned_partial: &[(Symbol, Symbol)]) -> Vec<usize> {
let mut sets: Vec<&HashSet<InternedKey>> = interned_partial
.iter()
.filter_map(|pair| self.index.get(pair))
@@ -426,6 +440,71 @@ impl DataStore {
.collect()
}
/// The union of `matching_values` (or, `exact`, of `get`) over every key `partial`
/// extends by one listed item per spanned category — scanned once (improvise-yb4b).
pub fn spanned_values(
&self,
partial: &[(String, String)],
spans: &[(&str, Vec<&str>)],
exact: bool,
) -> Vec<&CellValue> {
let wanted: Option<Vec<(Symbol, HashSet<Symbol>)>> = spans
.iter()
.map(|(cat, items)| {
let items = items.iter().filter_map(|i| self.symbols.get(i)).collect();
Some((self.symbols.get(cat)?, items))
})
.collect();
let Some(wanted) = wanted else {
return Vec::new();
};
let admits = |ikey: &InternedKey| {
(!exact || ikey.0.len() == partial.len() + wanted.len())
&& wanted
.iter()
.all(|(cat, items)| ikey.0.iter().any(|(c, i)| c == cat && items.contains(i)))
};
self.spanned_positions(partial, &wanted)
.into_iter()
.filter_map(|n| self.cells.get_index(n))
.filter(|(ikey, _)| admits(ikey))
.map(|(_, value)| value)
.collect()
}
/// Candidates for a span: narrowed once by `partial`, or once per covered item
/// of its smallest span, whichever visits fewer. Sorted (improvise-jq0m).
fn spanned_positions(
&self,
partial: &[(String, String)],
wanted: &[(Symbol, HashSet<Symbol>)],
) -> Vec<usize> {
self.intern_partial(partial)
.map(|base| self.narrowed(&base, wanted))
.unwrap_or_default()
}
fn narrowed(
&self,
base: &[(Symbol, Symbol)],
wanted: &[(Symbol, HashSet<Symbol>)],
) -> Vec<usize> {
let with = |cat: Symbol, item: Symbol| [base, &[(cat, item)]].concat();
let per_item: Vec<Vec<(Symbol, Symbol)>> = wanted
.iter()
.min_by_key(|(_, items)| items.len())
.map(|(cat, items)| items.iter().map(|i| with(*cat, *i)).collect())
.unwrap_or_default();
let item_cost: usize = per_item.iter().map(|p| self.narrowing_cost(p)).sum();
let narrow_per_item = !per_item.is_empty() && item_cost < self.narrowing_cost(base);
let mut positions: Vec<usize> = match narrow_per_item {
true => per_item.iter().flat_map(|p| self.positions_of(p)).collect(),
false => self.positions_of(base),
};
positions.sort_unstable();
positions
}
/// Every cell matching `partial`, in insertion order. Allocates a
/// `CellKey` per match — prefer `matching_values` when values suffice.
#[allow(dead_code)]
@@ -769,6 +848,41 @@ mod data_store {
"the fast path and the general path must agree"
);
}
/// improvise-yb4b: one scan answers what a lookup per member answered, in both
/// modes. context/test-evidence/core.md.
#[test]
fn a_spanned_scan_finds_what_a_lookup_per_member_finds() {
let mut store = DataStore::new();
let cells: [(&[(&str, &str)], f64); 7] = [
(&[("M", "Rev"), ("R", "North"), ("P", "A")], 1.0),
(&[("M", "Rev"), ("R", "South"), ("P", "A")], 2.0),
(&[("M", "Rev"), ("R", "East"), ("P", "A")], 4.0),
(&[("M", "Rev"), ("R", "North"), ("P", "A"), ("X", "x")], 8.0),
(&[("M", "Rev"), ("P", "A")], 16.0),
(&[("M", "Cost"), ("R", "North"), ("P", "A")], 32.0),
(&[("M", "Rev"), ("R", "North"), ("P", "B")], 64.0),
];
for (coords, v) in cells {
store.set(key(coords), CellValue::Number(v));
}
let base = key(&[("M", "Rev"), ("P", "A")]);
let covered = vec!["North", "South", "West"];
let members: Vec<CellKey> = covered.iter().map(|r| base.clone().with("R", *r)).collect();
let sum = |values: Vec<&CellValue>| values.iter().filter_map(|v| v.as_f64()).sum::<f64>();
let per_member_exact = members.iter().filter_map(|k| store.get(k)).collect();
let per_member_over = members
.iter()
.flat_map(|k| store.matching_values(&k.0))
.collect();
let spans = [("R", covered.clone())];
let scan_exact = store.spanned_values(&base.0, &spans, true);
let scan_over = store.spanned_values(&base.0, &spans, false);
assert_eq!((sum(scan_exact), sum(per_member_exact)), (3.0, 3.0));
assert_eq!((sum(scan_over), sum(per_member_over)), (11.0, 11.0));
let unknown = [("Nowhere", covered)];
assert!(store.spanned_values(&base.0, &unknown, false).is_empty());
}
}
#[cfg(test)]
@@ -941,6 +1055,60 @@ mod prop_tests {
"returned key {result_key} does not match partial {partial:?}");
}
}
/// improvise-jq0m: whichever narrowing it picks, a span returns what a lookup
/// per member returns, in store order. context/test-evidence/core.md.
#[test]
fn a_span_finds_what_its_members_find_in_store_order(
cells in prop::collection::vec((0u8..6, 0u8..3, 0u8..2, any::<bool>()), 0..40),
covered_r in prop::collection::vec(any::<bool>(), 7),
covered_p in prop::option::of(prop::collection::vec(any::<bool>(), 3)),
base_m in 0u8..2,
exact in any::<bool>(),
) {
let mut store = DataStore::new();
for (n, (r, p, m, extra)) in cells.iter().enumerate() {
let mut key = CellKey::new(vec![
("P".into(), format!("p{p}")),
("M".into(), format!("m{m}")),
]);
// r = 5 leaves the record without the spanned category at all.
key = match r { 5 => key, r => key.with("R", format!("r{r}")) };
key = match extra { true => key.with("X", "x"), false => key };
store.set(key, CellValue::Number(n as f64));
}
// r6 is never stored, so it is a covered item the store never interned.
let pick = |prefix: &str, flags: &[bool]| -> Vec<String> {
flags.iter().enumerate().filter(|(_, f)| **f)
.map(|(i, _)| format!("{prefix}{i}")).collect()
};
let rs = pick("r", &covered_r);
let ps = covered_p.as_deref().map(|f| pick("p", f));
let base = CellKey::new(vec![("M".into(), format!("m{base_m}"))]);
let mut spans: Vec<(&str, Vec<&str>)> = vec![("R", rs.iter().map(String::as_str).collect())];
if let Some(ps) = &ps {
spans.push(("P", ps.iter().map(String::as_str).collect()));
}
let mut members = vec![base.clone()];
for (cat, items) in &spans {
members = members.iter()
.flat_map(|k| items.iter().map(|i| k.clone().with(*cat, *i)))
.collect();
}
let matched: std::collections::HashSet<CellKey> = members.iter()
.flat_map(|k| match exact {
true => store.get(k).map(|_| k.clone()).into_iter().collect::<Vec<_>>(),
false => store.matching_cells(&k.0).into_iter().map(|(k, _)| k).collect(),
})
.collect();
let expected: Vec<CellValue> = store.iter_cells()
.filter(|(k, _)| matched.contains(k))
.map(|(_, v)| v.clone())
.collect();
let got: Vec<CellValue> = store.spanned_values(&base.0, &spans, exact)
.into_iter().cloned().collect();
prop_assert_eq!(got, expected);
}
}
}
+1 -1
View File
@@ -4,4 +4,4 @@ pub mod record;
pub mod symbol;
pub mod types;
pub use types::{Aggregation, Model};
pub use types::{Aggregation, MAX_CATEGORIES, Model};
+273 -71
View File
@@ -7,7 +7,9 @@ use super::category::{Category, CategoryId, CategoryKind, MEASURE_CATEGORY};
use super::cell::{CellKey, CellValue, DataStore};
use crate::formula::{AggFunc, Expr, Filter, Formula, FormulaId};
const MAX_CATEGORIES: usize = 12;
/// Regular categories a model may hold; labels and virtuals do not count.
/// Why 24 and what it costs: context/code/core.md §9 (improvise-felo.1).
pub const MAX_CATEGORIES: usize = 24;
/// Shared by `=`/`!=` **and** the division-by-zero guard, in both evaluators.
/// They must agree; why, and why 1e-10: context/code/core.md §4.
@@ -138,19 +140,78 @@ fn agg_values(
}
// How many formulas a fold has evaluated: what improvise-o1y0's regression counts.
#[cfg(test)]
#[cfg(any(test, feature = "test-support"))]
thread_local! {
static FOLD_FORMULA_EVALUATIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
pub static FOLD_FORMULA_EVALUATIONS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// One fold evaluation's memory: each formula key is evaluated once, and a key met
/// while still in progress (`None`) is a cycle (improvise-o1y0).
type FoldMemo = std::cell::RefCell<HashMap<CellKey, Option<Option<CellValue>>>>;
/// Each (fold key, span) evaluated once per grid build, under the agg and covers it
/// owns (improvise-o1y0, h6z1, pntu). One model state only: `context/code/core.md` §14.
pub struct FoldMemo<'a> {
agg: Aggregation,
covers: OwnedCovers<'a>,
seen: std::cell::RefCell<HashMap<(CellKey, Vec<String>), MemoEntry>>,
}
/// The members a memo's folds cover, owned so a caller cannot pass another set.
type OwnedCovers<'a> = Box<dyn Fn(&str, &str) -> bool + 'a>;
/// `None` while in progress, which is how a cycle is seen (improvise-o1y0).
type MemoEntry = Option<Option<CellValue>>;
impl<'a> FoldMemo<'a> {
pub fn new(agg: Aggregation, covers: impl Fn(&str, &str) -> bool + 'a) -> Self {
Self {
agg,
covers: Box::new(covers),
seen: Default::default(),
}
}
fn get(&self, key: &CellKey, span: Span) -> Option<MemoEntry> {
let at = (key.clone(), span.over.to_vec());
self.seen.borrow().get(&at).cloned()
}
fn put(&self, key: &CellKey, span: Span, entry: MemoEntry) {
let at = (key.clone(), span.over.to_vec());
self.seen.borrow_mut().insert(at, entry);
}
}
/// Which members of a collapsed group its fold covers. The model knows no
/// view, so the grid passes the members it draws (core.md §14).
pub type Covers<'a> = &'a dyn Fn(&str, &str) -> bool;
/// What a fold reaches: the covered members of each group its key names, and
/// every covered item of each category in `over` its key omits (improvise-wm20).
#[derive(Clone, Copy)]
struct Span<'a> {
covers: Covers<'a>,
over: &'a [String],
}
/// A fold's reach as data: `base` is the key with each expanded category removed,
/// and `spans` the covered items of each, in expansion order (improvise-yb4b).
struct Expansion<'a> {
base: CellKey,
spans: Vec<(&'a str, Vec<&'a str>)>,
}
impl Expansion<'_> {
/// One key per member, the cross product in expansion order.
fn members(&self) -> Vec<CellKey> {
let mut expanded = vec![self.base.clone()];
for (cat_name, covered) in &self.spans {
expanded = expanded
.iter()
.flat_map(|k| covered.iter().map(|m| k.clone().with(*cat_name, *m)))
.collect();
}
expanded
}
}
/// Where a formula's references are read: at one record, or across the members
/// a collapsed group's fold covers, as a None axis reads them (core.md §14).
#[derive(Clone, Copy)]
@@ -158,8 +219,8 @@ enum Reach<'a> {
Record,
Fold {
agg: Aggregation,
covers: Covers<'a>,
memo: &'a FoldMemo,
span: Span<'a>,
memo: &'a FoldMemo<'a>,
},
}
@@ -184,9 +245,7 @@ fn eval_expr(
let new_key = context.clone().with(cat, name);
let value = match reach {
Reach::Record => model.evaluate_depth(&new_key, depth),
Reach::Fold { agg, covers, memo } => {
model.evaluate_fold(&new_key, agg, covers, memo)
}
Reach::Fold { agg, span, memo } => model.evaluate_fold(&new_key, agg, span, memo),
};
match value {
Some(CellValue::Number(n)) => Ok(n),
@@ -466,6 +525,15 @@ impl Model {
self.categories.get(name)
}
/// `group` and its ancestors in `cat`, root first and owned: the chain a
/// `View` folds by (core.md §19). Just `[group]` where `cat` is unknown.
pub fn group_chain(&self, cat: &str, group: &str) -> Vec<String> {
match self.category(cat) {
Some(c) => c.group_chain(group).into_iter().map(String::from).collect(),
None => vec![group.to_string()],
}
}
pub fn set_cell(&mut self, key: CellKey, value: CellValue) {
self.data.set(key, value);
}
@@ -505,6 +573,18 @@ impl Model {
&self.formulas
}
/// Remove the formula `id` names, refusing if none does — a delete that
/// finds nothing is a stale claim, not a no-op (improvise-tej1).
pub fn delete_formula(&mut self, id: &FormulaId) -> Result<()> {
let pos = self
.formulas
.iter()
.position(|f| f.id() == *id)
.ok_or_else(|| anyhow!("That formula no longer exists"))?;
self.formulas.remove(pos);
Ok(())
}
pub fn formula(&self, id: &FormulaId) -> Option<&Formula> {
self.formulas.iter().find(|f| f.id() == *id)
}
@@ -589,6 +669,33 @@ impl Model {
}
}
/// The first coordinate of `key` naming no item of its category: the one
/// question a write's refusal and a cell's projection share (improvise-gk51).
pub fn first_non_item<'k>(&self, key: &'k CellKey) -> Option<(&'k str, &'k str)> {
key.0
.iter()
.find(|(cat, name)| !self.is_item(cat, name))
.map(|(cat, name)| (cat.as_str(), name.as_str()))
}
/// `effective_item_names(cat).contains(name)`, asked without building the list:
/// it runs per coordinate per projected cell (improvise-4daa).
pub fn is_item(&self, cat_name: &str, name: &str) -> bool {
let listed = |cat: &str| {
self.category(cat)
.is_some_and(|c| c.items.contains_key(name))
};
match CategoryKind::of_name(cat_name).is_measure() {
true => {
listed(MEASURE_CATEGORY)
|| self.formulas.iter().any(|f| {
f.target == name && CategoryKind::of_name(&f.target_category).is_measure()
})
}
false => listed(cat_name),
}
}
/// Category names excluding virtual categories (_Index, _Dim).
pub fn regular_category_names(&self) -> Vec<&str> {
self.categories
@@ -654,7 +761,43 @@ impl Model {
agg: Aggregation,
covers: Covers,
) -> Option<CellValue> {
self.evaluate_fold(key, agg, covers, &FoldMemo::default())
self.evaluate_covering_in(key, &FoldMemo::new(agg, covers))
}
/// [`Self::evaluate_covering`] under `memo`'s aggregation and covers, sharing it
/// with one grid build; it must not outlive the model state (improvise-h6z1, pntu).
pub fn evaluate_covering_in(&self, key: &CellKey, memo: &FoldMemo) -> Option<CellValue> {
let span = Span {
covers: &*memo.covers,
over: &[],
};
self.evaluate_fold(key, memo.agg, span, memo)
}
/// The value `key` has as ONE fold over every covered item of the `over`
/// categories it omits — a column's total (improvise-wm20, core.md §14).
pub fn evaluate_spanning(
&self,
key: &CellKey,
over: &[String],
agg: Aggregation,
covers: Covers,
) -> Option<CellValue> {
self.evaluate_spanning_in(key, over, &FoldMemo::new(agg, covers))
}
/// [`Self::evaluate_spanning`] under one grid build's `memo` (improvise-h6z1, pntu).
pub fn evaluate_spanning_in(
&self,
key: &CellKey,
over: &[String],
memo: &FoldMemo,
) -> Option<CellValue> {
let span = Span {
covers: &*memo.covers,
over,
};
self.evaluate_fold(key, memo.agg, span, memo)
}
/// A key naming a group folds; any other key reads as before. A formula is
@@ -663,16 +806,16 @@ impl Model {
&self,
key: &CellKey,
agg: Aggregation,
covers: Covers,
span: Span,
memo: &FoldMemo,
) -> Option<CellValue> {
let members = self.group_expansion(key, covers);
match (members.is_empty(), self.formula_for(key)) {
let expansion = self.group_expansion(key, span);
match (expansion.spans.is_empty(), self.formula_for(key)) {
(true, _) => self.evaluate_unfolded(key, agg),
(false, Some(formula)) => {
self.fold_formula_once(formula, key, &members, (agg, covers, memo))
self.fold_formula_once(formula, key, &expansion, (agg, span, memo))
}
(false, None) => self.fold_stored(key, &members, agg),
(false, None) => self.fold_stored(key, &expansion, agg),
}
}
@@ -682,20 +825,20 @@ impl Model {
&self,
formula: &Formula,
key: &CellKey,
members: &[CellKey],
(agg, covers, memo): (Aggregation, Covers, &FoldMemo),
expansion: &Expansion,
(agg, span, memo): (Aggregation, Span, &FoldMemo),
) -> Option<CellValue> {
let seen = memo.borrow().get(key).cloned();
let seen = memo.get(key, span);
match seen {
Some(Some(value)) => value,
Some(None) => Some(CellValue::Error("circular".into())),
None => {
memo.borrow_mut().insert(key.clone(), None);
#[cfg(test)]
memo.put(key, span, None);
#[cfg(any(test, feature = "test-support"))]
FOLD_FORMULA_EVALUATIONS.with(|n| n.set(n.get() + 1));
let reach = Reach::Fold { agg, covers, memo };
let value = self.eval_formula_folded(formula, key, members, agg, reach);
memo.borrow_mut().insert(key.clone(), Some(value.clone()));
let reach = Reach::Fold { agg, span, memo };
let value = self.eval_formula_folded(formula, key, expansion, agg, reach);
memo.put(key, span, Some(value.clone()));
value
}
}
@@ -712,31 +855,40 @@ impl Model {
aggregate_cell_values(&self.data.matching_values(&key.0), self.agg_for(key))
}
/// One key per covered member where `key` names a collapsed GROUP rather than
/// an item — the cross product over every such coordinate. Empty when none does.
fn group_expansion(&self, key: &CellKey, covers: Covers) -> Vec<CellKey> {
let mut expanded = vec![key.clone()];
for (cat_name, name) in &key.0 {
/// Each category `key` folds, with its covered members: a collapsed GROUP it
/// names, or a spanned category it omits. No spans when neither.
fn group_expansion<'a>(&'a self, key: &'a CellKey, span: Span<'a>) -> Expansion<'a> {
let grouped = key.0.iter().map(|(cat_name, name)| {
let members: Vec<&str> = self
.category(cat_name)
.filter(|cat| !cat.items.contains_key(name.as_str()))
.map(|cat| cat.items_in_group(name))
.unwrap_or_default()
.into_iter()
.filter(|m| covers(cat_name, m))
.collect();
if members.is_empty() {
continue;
}
expanded = expanded
.iter()
.flat_map(|k| members.iter().map(|m| k.clone().with(cat_name, *m)))
.collect();
}
match expanded.as_slice() {
[only] if only == key => Vec::new(),
_ => expanded,
}
.unwrap_or_default();
(cat_name.as_str(), members)
});
let spanned = span
.over
.iter()
.filter(|cat_name| key.get(cat_name).is_none())
.map(|cat_name| {
let items = self.category(cat_name).map(|cat| cat.ordered_item_names());
(cat_name.as_str(), items.unwrap_or_default())
});
let spans: Vec<(&str, Vec<&str>)> = grouped
.chain(spanned)
.map(|(cat_name, members)| {
let covered: Vec<&str> = members
.into_iter()
.filter(|m| (span.covers)(cat_name, m))
.collect();
(cat_name, covered)
})
.filter(|(_, covered)| !covered.is_empty())
.collect();
let base = spans
.iter()
.fold(key.clone(), |k, (cat_name, _)| k.without(cat_name));
Expansion { base, spans }
}
/// A coordinate as a formula reads it. At a fold a group's name is no item, so
@@ -762,30 +914,28 @@ impl Model {
/// The contexts an aggregate inside a formula pools over at `context`.
fn contexts_of(&self, context: &CellKey, reach: Reach) -> Vec<CellKey> {
let members = match reach {
Reach::Record => Vec::new(),
Reach::Fold { covers, .. } => self.group_expansion(context, covers),
let expansion = match reach {
Reach::Record => None,
Reach::Fold { span, .. } => Some(self.group_expansion(context, span)),
};
match members.is_empty() {
true => vec![context.clone()],
false => members,
match expansion.filter(|e| !e.spans.is_empty()) {
None => vec![context.clone()],
Some(e) => e.members(),
}
}
/// A group's stored values fold ONCE over every record the group covers.
/// A group's stored values fold ONCE over every record the group covers, in
/// the store's own order: one scan, not a lookup per member (improvise-yb4b).
fn fold_stored(
&self,
key: &CellKey,
members: &[CellKey],
expansion: &Expansion,
agg: Aggregation,
) -> Option<CellValue> {
let values: Vec<&CellValue> = members
.iter()
.flat_map(|k| match agg {
Aggregation::Exact => self.data.get(k).into_iter().collect(),
Aggregation::OverHidden => self.data.matching_values(&k.0),
})
.collect();
let exact = agg == Aggregation::Exact;
let values = self
.data
.spanned_values(&expansion.base.0, &expansion.spans, exact);
aggregate_cell_values(&values, self.agg_for(key))
}
@@ -795,7 +945,7 @@ impl Model {
&self,
formula: &Formula,
key: &CellKey,
members: &[CellKey],
expansion: &Expansion,
agg: Aggregation,
reach: Reach,
) -> Option<CellValue> {
@@ -804,7 +954,7 @@ impl Model {
.as_ref()
.is_none_or(|f| self.read_coordinate(key, &f.category, reach) == Some(f.item.as_str()));
if !applies {
return self.fold_stored(key, members, agg);
return self.fold_stored(key, expansion, agg);
}
let depth = Self::MAX_EVAL_DEPTH;
match eval_expr(
@@ -1178,6 +1328,40 @@ mod model_tests {
use super::{Aggregation, Model};
use crate::model::cell::{CellKey, CellValue};
/// improvise-4daa: `is_item` answers exactly what the list it replaces answers.
/// Evidence: `context/test-evidence/core.md`.
#[test]
fn is_item_agrees_with_effective_item_names_everywhere() {
let mut m = Model::new("Test");
m.add_category("Region").unwrap();
for item in ["East", "North"] {
m.category_mut("Region").unwrap().add_item(item);
}
m.category_mut("_Measure").unwrap().add_item("Revenue");
for (raw, category) in [
("Profit = Revenue * 2", "_Measure"),
("Total = 1", "Region"),
] {
m.add_formula(crate::formula::parse_formula(raw, category).unwrap());
}
let names = ["East", "North", "Revenue", "Profit", "Total", "Mars"];
for cat in ["Region", "_Measure", "Missing"] {
for name in names {
assert_eq!(
m.is_item(cat, name),
m.effective_item_names(cat).iter().any(|n| n == name),
"{cat}/{name}"
);
}
}
let listed = m.category("_Measure").unwrap().items.contains_key("Profit");
assert!(!listed, "setup: only the formula branch can answer Profit");
assert!(
m.is_item("_Measure", "Profit"),
"a measure formula target is an item"
);
}
fn coord(pairs: &[(&str, &str)]) -> CellKey {
CellKey::new(
pairs
@@ -1213,7 +1397,8 @@ mod model_tests {
assert_eq!(id, m.categories["C0"].id);
assert_eq!(m.categories.len(), before, "nothing was created");
// The limit still refuses a category that WOULD be created.
assert!(m.add_category("C12").is_err(), "the limit still holds");
let fresh = format!("C{}", super::MAX_CATEGORIES);
assert!(m.add_category(fresh).is_err(), "the limit still holds");
}
/// improvise-s08t — see context/test-evidence/core.md.
@@ -1377,13 +1562,20 @@ mod model_tests {
assert_eq!(m.category_names().len(), 4);
}
/// The user's number (improvise-felo.1), spelled once here; every other
/// fill-to-the-cap test derives it from `MAX_CATEGORIES`.
#[test]
fn add_category_max_limit() {
let mut m = Model::new("Test");
for i in 0..12 {
m.add_category(format!("Cat{i}")).unwrap();
for i in 0..24 {
m.add_category(format!("Cat{i}"))
.unwrap_or_else(|e| panic!("category {} of 24 refused: {e}", i + 1));
}
assert!(m.add_category("TooMany").is_err());
let err = m.add_category("TooMany").expect_err("the 25th is refused");
assert!(
err.to_string().contains("24"),
"the refusal names the real limit: {err}"
);
}
#[test]
@@ -2252,6 +2444,16 @@ mod formula_tests {
assert_eq!(targets(&m), vec!["Profit", "Total", "Tax", "Margin"]);
}
#[test]
fn delete_formula_removes_the_one_its_id_names_and_refuses_a_stale_id() {
let mut m = three_formulas();
let tax = m.formulas[1].id();
m.delete_formula(&tax).unwrap();
assert_eq!(targets(&m), vec!["Profit", "Margin"]);
assert!(m.delete_formula(&tax).is_err(), "a second delete is stale");
assert_eq!(targets(&m), vec!["Profit", "Margin"]);
}
#[test]
fn a_formula_mentions_what_its_expression_or_its_where_names() {
let margin = parse_formula("Margin = Profit / Revenue", "_Measure").unwrap();
@@ -2969,11 +3171,11 @@ mod five_category {
// 4 regular + 3 virtual.
assert_eq!(m.category_names().len(), 7);
let mut m2 = build_model();
for i in 0..8 {
for i in 0..super::MAX_CATEGORIES - 4 {
m2.add_category(format!("Extra{i}")).unwrap();
}
// 12 regular + 3 virtuals = 15.
assert_eq!(m2.category_names().len(), 15);
// The limit's worth of regular categories, plus 3 virtuals.
assert_eq!(m2.category_names().len(), super::MAX_CATEGORIES + 3);
assert!(m2.add_category("OneMore").is_err());
}
}
@@ -3042,7 +3244,7 @@ mod prop_tests {
if let Some(c) = m.category_mut("_Measure") { c.add_item("Revenue"); }
let key = CellKey::new(vec![("_Measure".into(), "Revenue".into())]);
m.set_cell(key.clone(), CellValue::Number(val));
let _ = m.add_category(&new_cat); // may hit the 12-cat limit
let _ = m.add_category(&new_cat); // may be refused; either way the cell survives
prop_assert_eq!(m.evaluate(&key), Some(CellValue::Number(val)));
}
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -4,10 +4,10 @@ pub mod types;
pub use axis::{Axis, AxisColor};
pub use layout::{
AxisEntry, GridLayout, NEW_CATEGORY_COL, NEW_MEASURE_COL, RecordCol, RecordRow, ViewStep,
drill_constraint, drill_snapshot, drill_steps, records_spine, records_steps,
synthetic_record_info,
AxisEntry, GridEval, GridLayout, NEW_CATEGORY_COL, NEW_MEASURE_COL, RecordCol, RecordRow,
ViewStep, drill_constraint, drill_snapshot, drill_steps, record_row, records_spine,
records_steps, synthetic_record_info,
};
pub use types::{
DEFAULT_VIEW, DRILL_VIEW, RECORDS_VIEW, VIRTUAL_VIEW_NAMES, View, unused_view_name,
DEFAULT_VIEW, DRILL_VIEW, GroupFold, RECORDS_VIEW, VIRTUAL_VIEW_NAMES, View, unused_view_name,
};
+177 -38
View File
@@ -1,5 +1,6 @@
use indexmap::IndexMap;
use std::collections::{HashMap, HashSet};
use std::num::NonZeroUsize;
use super::axis::Axis;
use crate::model::category::CategoryKind;
@@ -27,6 +28,16 @@ pub fn unused_view_name<'a>(existing: impl IntoIterator<Item = &'a str>) -> Stri
.expect("a name is free below usize::MAX, there being fewer views")
}
/// One group's own fold, overriding what it inherits (core.md §19).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum GroupFold {
/// `collapsed: Cat/Group` — closed, whatever reaches it from above.
Collapsed,
/// `expanded(N): Cat/Group` — open, with N-1 levels beneath it. Non-zero by
/// type, since `expanded(0)` is no spelling the grammar reads back.
Expanded(NonZeroUsize),
}
#[derive(Clone)]
pub struct View {
pub name: String,
@@ -36,8 +47,11 @@ pub struct View {
pub page_selections: HashMap<String, String>,
/// Hidden items per category
pub hidden_items: HashMap<String, HashSet<String>>,
/// Collapsed groups per category
pub collapsed_groups: HashMap<String, HashSet<String>>,
/// How deep each category's hierarchy opens: groups shallower than this start
/// open, the rest closed. Absent is 0 — everything closed (core.md §19).
pub expand_depth: HashMap<String, usize>,
/// Per-group answers that override the depth default, in either direction.
pub group_folds: HashMap<String, HashMap<String, GroupFold>>,
/// Number format string (e.g. ",.0f" for comma-separated integer)
pub number_format: String,
/// When true, empty rows/columns are pruned from the display.
@@ -54,7 +68,8 @@ impl std::fmt::Debug for View {
category_axes,
page_selections,
hidden_items,
collapsed_groups,
expand_depth,
group_folds,
number_format,
prune_empty,
} = self;
@@ -64,12 +79,18 @@ impl std::fmt::Debug for View {
.collect()
};
let pages: BTreeMap<_, _> = page_selections.iter().collect();
let depths: BTreeMap<_, _> = expand_depth.iter().collect();
let folds: BTreeMap<_, BTreeMap<_, _>> = group_folds
.iter()
.map(|(k, v)| (k, v.iter().collect()))
.collect();
f.debug_struct("View")
.field("name", name)
.field("category_axes", category_axes)
.field("page_selections", &pages)
.field("hidden_items", &sorted(hidden_items))
.field("collapsed_groups", &sorted(collapsed_groups))
.field("expand_depth", &depths)
.field("group_folds", &folds)
.field("number_format", number_format)
.field("prune_empty", prune_empty)
.finish()
@@ -88,7 +109,8 @@ impl View {
category_axes: IndexMap::new(),
page_selections: HashMap::new(),
hidden_items: HashMap::new(),
collapsed_groups: HashMap::new(),
expand_depth: HashMap::new(),
group_folds: HashMap::new(),
// Decimals by default: a ratio is the case a format cannot be
// guessed for, and `.0` draws 2/3 as `1` (improvise-e0ex).
number_format: ",.2".to_string(),
@@ -135,7 +157,8 @@ impl View {
self.category_axes.shift_remove(cat_name);
self.page_selections.remove(cat_name);
self.hidden_items.remove(cat_name);
self.collapsed_groups.remove(cat_name);
self.expand_depth.remove(cat_name);
self.group_folds.remove(cat_name);
}
/// Forget an item this view names. A hidden or paged-to item that outlives
@@ -222,23 +245,78 @@ impl View {
self.page_selections.get(cat_name).map(|s| s.as_str())
}
pub fn toggle_group_collapse(&mut self, cat_name: &str, group_name: &str) {
let set = self
.collapsed_groups
.entry(cat_name.to_string())
.or_default();
if set.contains(group_name) {
set.remove(group_name);
} else {
set.insert(group_name.to_string());
/// How many of `cat_name`'s top levels open (`expanded(N): Cat`); 0 — all
/// closed — if unset.
pub fn expand_depth(&self, cat_name: &str) -> usize {
self.expand_depth.get(cat_name).copied().unwrap_or(0)
}
/// Open `cat_name`'s top `depth` levels; 0 forgets the setting.
pub fn set_expand_depth(&mut self, cat_name: &str, depth: usize) {
match depth {
0 => self.expand_depth.remove(cat_name),
_ => self.expand_depth.insert(cat_name.to_string(), depth),
};
}
/// Whether the group ending `chain` (root first, from `Category::group_chain`)
/// is collapsed: its own fold wins, else what it inherits (core.md §19).
pub fn is_group_collapsed(&self, cat_name: &str, chain: &[&str]) -> bool {
match chain.last().and_then(|g| self.group_fold(cat_name, g)) {
Some(GroupFold::Collapsed) => true,
Some(GroupFold::Expanded(_)) => false,
None => self.inherits_collapsed(cat_name, chain),
}
}
pub fn is_group_collapsed(&self, cat_name: &str, group_name: &str) -> bool {
self.collapsed_groups
.get(cat_name)
.map(|s| s.contains(group_name))
.unwrap_or(false)
/// The answer without the group's own fold: open when the category's depth or
/// an ancestor's `expanded(N)` reaches it, closed otherwise.
fn inherits_collapsed(&self, cat_name: &str, chain: &[&str]) -> bool {
let depth = chain.len().saturating_sub(1);
let by_ancestor = chain[..depth].iter().enumerate().any(|(at, ancestor)| {
matches!(self.group_fold(cat_name, ancestor),
Some(GroupFold::Expanded(n)) if depth - at < n.get())
});
!by_ancestor && depth >= self.expand_depth(cat_name)
}
/// The fold this view holds for one group, if any.
pub fn group_fold(&self, cat_name: &str, group_name: &str) -> Option<GroupFold> {
self.group_folds.get(cat_name)?.get(group_name).copied()
}
/// Record a fold as given — what a file's `collapsed:`/`expanded:` says.
pub fn set_group_fold(&mut self, cat_name: &str, group_name: &str, fold: GroupFold) {
self.group_folds
.entry(cat_name.to_string())
.or_default()
.insert(group_name.to_string(), fold);
}
/// Flip the group ending `chain`: `z`. It writes `Collapsed` or `Expanded(1)`
/// — or nothing, where the flip lands on what the group inherits (§19).
pub fn toggle_group_collapse(&mut self, cat_name: &str, chain: &[&str]) {
let Some(group) = chain.last() else {
return;
};
let collapse = !self.is_group_collapsed(cat_name, chain);
match collapse == self.inherits_collapsed(cat_name, chain) {
true => {
if let Some(folds) = self.group_folds.get_mut(cat_name) {
folds.remove(*group);
if folds.is_empty() {
self.group_folds.remove(cat_name);
}
}
}
false => {
let fold = match collapse {
true => GroupFold::Collapsed,
false => GroupFold::Expanded(NonZeroUsize::MIN),
};
self.set_group_fold(cat_name, group, fold);
}
}
}
pub fn hide_item(&mut self, cat_name: &str, item_name: &str) {
@@ -292,9 +370,10 @@ impl View {
#[cfg(test)]
mod tests {
use super::{View, unused_view_name};
use super::{GroupFold, View, unused_view_name};
use crate::model::category::CategoryKind;
use crate::view::Axis;
use std::num::NonZeroUsize;
/// improvise-vw47: the count is the starting guess, never the answer.
/// What a taken name costs: `context/test-evidence/core.md`.
@@ -563,28 +642,87 @@ mod tests {
assert_eq!(v.page_selection("Region"), None);
}
/// The user's ruling on improvise-felo.6: every group starts closed.
#[test]
fn toggle_group_collapse_toggles_twice() {
fn a_group_starts_collapsed_at_every_depth() {
let v = View::new("Test");
assert!(v.is_group_collapsed("Time", &["2024"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1-2024"]));
}
/// `expanded(1): Time` opens the roots and nothing below them.
#[test]
fn a_category_depth_opens_its_top_levels() {
let mut v = View::new("Test");
assert!(!v.is_group_collapsed("Time", "Q1"));
v.toggle_group_collapse("Time", "Q1");
assert!(v.is_group_collapsed("Time", "Q1"));
v.toggle_group_collapse("Time", "Q1");
assert!(!v.is_group_collapsed("Time", "Q1"));
v.set_expand_depth("Time", 1);
assert!(!v.is_group_collapsed("Time", &["2024"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1-2024"]));
assert!(v.is_group_collapsed("Other", &["G"]), "per category");
v.set_expand_depth("Time", 0);
assert!(v.expand_depth.is_empty(), "0 is the absence, not a setting");
}
/// `expanded(2): Time/2024` opens 2024 and one level beneath it, no deeper,
/// and nothing of the sibling year.
#[test]
fn a_groups_depth_opens_it_and_n_minus_one_levels_beneath() {
let mut v = View::new("Test");
v.set_group_fold(
"Time",
"2024",
GroupFold::Expanded(NonZeroUsize::new(2).unwrap()),
);
assert!(!v.is_group_collapsed("Time", &["2024"]));
assert!(!v.is_group_collapsed("Time", &["2024", "Q1"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1", "Jan"]));
assert!(v.is_group_collapsed("Time", &["2025", "Q1"]));
}
/// A group's own `collapsed:` wins over an `expanded(N)` that reaches it.
#[test]
fn a_collapsed_group_inside_an_expanded_region_stays_collapsed() {
let mut v = View::new("Test");
v.set_expand_depth("Time", 3);
v.set_group_fold("Time", "Q1", GroupFold::Collapsed);
assert!(v.is_group_collapsed("Time", &["2024", "Q1"]));
assert!(!v.is_group_collapsed("Time", &["2024", "Q2"]));
}
/// `z` writes `Expanded(1)` or `Collapsed`, in BOTH directions, and
/// toggling back leaves no fold behind.
#[test]
fn a_toggle_overrides_what_a_group_inherits_either_way_and_toggling_back_clears_it() {
let mut v = View::new("Test");
v.set_expand_depth("Time", 1);
v.toggle_group_collapse("Time", &["2024", "Q1-2024"]);
assert_eq!(
v.group_fold("Time", "Q1-2024"),
Some(GroupFold::Expanded(NonZeroUsize::MIN))
);
v.toggle_group_collapse("Time", &["2024"]);
assert_eq!(v.group_fold("Time", "2024"), Some(GroupFold::Collapsed));
assert!(
!v.is_group_collapsed("Time", &["2025"]),
"a sibling is untouched"
);
v.toggle_group_collapse("Time", &["2024", "Q1-2024"]);
v.toggle_group_collapse("Time", &["2024"]);
assert!(v.group_folds.is_empty(), "{:?}", v.group_folds);
}
#[test]
fn is_group_collapsed_isolated_across_categories() {
let mut v = View::new("Test");
v.toggle_group_collapse("Cat1", "G1");
assert!(!v.is_group_collapsed("Cat2", "G1"));
v.toggle_group_collapse("Cat1", &["G1"]);
assert!(!v.is_group_collapsed("Cat1", &["G1"]));
assert!(v.is_group_collapsed("Cat2", &["G1"]));
}
#[test]
fn is_group_collapsed_isolated_across_groups() {
let mut v = View::new("Test");
v.toggle_group_collapse("Cat1", "G1");
assert!(!v.is_group_collapsed("Cat1", "G2"));
v.toggle_group_collapse("Cat1", &["G1"]);
assert!(v.is_group_collapsed("Cat1", &["G2"]));
}
/// Arm 2 of improvise-qx1z: the other three lines of `on_category_removed`
@@ -593,12 +731,13 @@ mod tests {
fn on_category_removed_forgets_a_collapsed_group() {
let mut v = View::new("V");
v.on_category_added("Month", CategoryKind::of_name("Month"));
v.toggle_group_collapse("Month", "Q1");
v.set_expand_depth("Month", 1);
v.toggle_group_collapse("Month", &["Q1"]);
v.on_category_removed("Month");
v.on_category_added("Month", CategoryKind::of_name("Month"));
assert!(
!v.is_group_collapsed("Month", "Q1"),
"a re-added category came back with its group collapsed"
v.group_folds.is_empty() && v.expand_depth.is_empty(),
"a re-added category came back with its old fold state"
);
}
@@ -808,10 +947,10 @@ mod prop_tests {
group in "[A-Za-z][a-z]{1,7}",
) {
let mut v = View::new("T");
let initial = v.is_group_collapsed(&cat, &group);
v.toggle_group_collapse(&cat, &group);
v.toggle_group_collapse(&cat, &group);
prop_assert_eq!(v.is_group_collapsed(&cat, &group), initial);
let initial = v.is_group_collapsed(&cat, &[group.as_str()]);
v.toggle_group_collapse(&cat, &[group.as_str()]);
v.toggle_group_collapse(&cat, &[group.as_str()]);
prop_assert_eq!(v.is_group_collapsed(&cat, &[group.as_str()]), initial);
}
}
}
+37 -8
View File
@@ -1167,9 +1167,23 @@ mod generator {
use pest_meta::parser;
use proptest::prelude::*;
use std::collections::HashMap;
use std::sync::OnceLock;
use std::sync::atomic::{AtomicUsize, Ordering};
/// Counts how often formula.pest is actually parsed; the guard below keeps it
/// from becoming a per-case parse again.
static GRAMMAR_PARSES: AtomicUsize = AtomicUsize::new(0);
/// The parsed grammar, read once for the process. Evidence:
/// `context/test-evidence/formula-and-io.md`.
fn grammar() -> &'static HashMap<String, (RuleType, Expr)> {
static RULES: OnceLock<HashMap<String, (RuleType, Expr)>> = OnceLock::new();
RULES.get_or_init(load_grammar)
}
/// Parse formula.pest and return rules keyed by name.
fn load_grammar() -> HashMap<String, (RuleType, Expr)> {
GRAMMAR_PARSES.fetch_add(1, Ordering::Relaxed);
let grammar = include_str!("formula.pest");
let pairs = parser::parse(parser::Rule::grammar_rules, grammar)
.unwrap_or_else(|e| panic!("Bad grammar: {e}"));
@@ -1301,18 +1315,33 @@ mod generator {
/// Proptest strategy: generate a random valid formula by walking the
/// grammar's `formula` rule.
pub fn formula_string() -> impl Strategy<Value = String> {
prop::collection::vec(any::<u8>(), 32..=128).prop_map(|choices| {
let rules = load_grammar();
Gen::new(&rules, choices).generate("formula")
})
prop::collection::vec(any::<u8>(), 32..=128)
.prop_map(|choices| Gen::new(grammar(), choices).generate("formula"))
}
/// Proptest strategy: generate a random valid standalone expression.
pub fn expr_string() -> impl Strategy<Value = String> {
prop::collection::vec(any::<u8>(), 32..=128).prop_map(|choices| {
let rules = load_grammar();
Gen::new(&rules, choices).generate("expr_eoi")
})
prop::collection::vec(any::<u8>(), 32..=128)
.prop_map(|choices| Gen::new(grammar(), choices).generate("expr_eoi"))
}
/// improvise-i5z6. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn the_grammar_is_parsed_once_however_many_formulas_the_strategies_yield() {
use proptest::strategy::ValueTree;
use proptest::test_runner::TestRunner;
let mut runner = TestRunner::deterministic();
let (formulas, exprs) = (formula_string(), expr_string());
for _ in 0..32 {
let _ = formulas.new_tree(&mut runner).unwrap().current();
let _ = exprs.new_tree(&mut runner).unwrap().current();
}
assert_eq!(
GRAMMAR_PARSES.load(Ordering::Relaxed),
1,
"formula.pest was re-parsed per generated case"
);
}
}
+222 -8
View File
@@ -1,10 +1,12 @@
use std::collections::HashSet;
use std::path::Path;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use anyhow::{Context, Result, ensure};
use csv::ReaderBuilder;
use serde_json::Value;
use crate::storage::{PhysicalFs, Storage};
pub fn csv_path_p(path: &Path) -> bool {
path.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("csv"))
@@ -31,6 +33,12 @@ pub fn parse_csv_with_warnings(path: &Path) -> Result<(Vec<Value>, Vec<String>)>
/// As [`parse_csv_with_warnings`] but over bytes, so import can come from any
/// [`Storage`](crate::storage::Storage) backend, not only a file (improvise-fuj).
pub fn parse_csv_bytes_with_warnings(bytes: &[u8]) -> Result<(Vec<Value>, Vec<String>)> {
let (_, records, warnings) = parse_csv_bytes(bytes)?;
Ok((records, warnings))
}
/// The header (after renaming), the records and the warnings.
fn parse_csv_bytes(bytes: &[u8]) -> Result<(Vec<String>, Vec<Value>, Vec<String>)> {
let mut reader = ReaderBuilder::new()
.has_headers(true)
.flexible(true)
@@ -110,7 +118,7 @@ pub fn parse_csv_bytes_with_warnings(bytes: &[u8]) -> Result<(Vec<Value>, Vec<St
}),
);
Ok((records, warnings))
Ok((headers, records, warnings))
}
/// A header name keys the record object, so two equal names were one column and
@@ -145,7 +153,41 @@ fn disambiguate_headers(raw: Vec<String>) -> (Vec<String>, Vec<String>) {
/// Parse multiple CSV files and merge into a single JSON array.
/// Each record gets a "File" field set to the filename stem (e.g., "sales" from "sales.csv").
pub fn merge_csvs(paths: &[impl AsRef<Path>]) -> Result<Vec<Value>> {
let mut all_records = Vec::new();
let merged = merge_csvs_from(&PhysicalFs, paths)?;
for warning in &merged.warnings {
eprintln!("warning: {warning}");
}
Ok(merged.records)
}
/// The category every merged record names its source file by.
pub const FILE_CATEGORY: &str = "File";
/// One merged file: its `File` item and the columns its header named.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SourceFile {
pub item: String,
pub columns: Vec<String>,
}
/// Several CSV files as one record set, with each file's own columns kept.
#[derive(Debug, Clone, PartialEq)]
pub struct Merged {
pub records: Vec<Value>,
/// One per distinct `File` item, in the order the files were read.
pub sources: Vec<SourceFile>,
/// Each file's parse warnings, prefixed with its path.
pub warnings: Vec<String>,
}
/// As [`merge_csvs`] over any [`Storage`]. Files may have different headers,
/// and two files with one stem share its item and its columns (improvise-lpzu).
pub fn merge_csvs_from(fs: &dyn Storage, paths: &[impl AsRef<Path>]) -> Result<Merged> {
let mut merged = Merged {
records: Vec::new(),
sources: Vec::new(),
warnings: Vec::new(),
};
for path in paths {
let path = path.as_ref();
let stem = path
@@ -153,15 +195,75 @@ pub fn merge_csvs(paths: &[impl AsRef<Path>]) -> Result<Vec<Value>> {
.and_then(|s| s.to_str())
.unwrap_or("unknown")
.to_string();
let records = parse_csv(path)?;
let bytes = fs.read(path)?;
let (headers, records, warnings) = parse_csv_bytes(&bytes)
.with_context(|| format!("Failed to parse CSV file: {}", path.display()))?;
let at = merged.source_for(&stem);
let columns = &mut merged.sources[at].columns;
for header in headers {
if header != FILE_CATEGORY && !columns.contains(&header) {
columns.push(header);
}
}
merged
.warnings
.extend(warnings.iter().map(|w| format!("{}: {w}", path.display())));
for mut record in records {
if let Value::Object(ref mut map) = record {
map.insert("File".to_string(), Value::String(stem.clone()));
map.insert(FILE_CATEGORY.to_string(), Value::String(stem.clone()));
}
all_records.push(record);
merged.records.push(record);
}
}
Ok(all_records)
Ok(merged)
}
impl Merged {
fn source_for(&mut self, item: &str) -> usize {
match self.sources.iter().position(|s| s.item == item) {
Some(at) => at,
None => {
self.sources.push(SourceFile {
item: item.to_string(),
columns: vec![],
});
self.sources.len() - 1
}
}
}
}
/// One `improvise import` argument. Whether a path names a folder is the
/// host's question, so the caller classifies and [`expand_import_paths`] lists.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ImportPath {
File(PathBuf),
Folder(PathBuf),
}
/// The files an import reads: each argument in turn, a folder standing for its
/// `*.csv` documents in name order. See `context/code/formula-and-io.md`.
pub fn expand_import_paths(fs: &dyn Storage, args: &[ImportPath]) -> Result<Vec<PathBuf>> {
let mut paths = Vec::new();
for arg in args {
match arg {
ImportPath::File(path) => paths.push(path.clone()),
ImportPath::Folder(dir) => paths.extend(csv_documents_in(fs, dir)?),
}
}
Ok(paths)
}
/// A folder's `*.csv` documents, sorted by `Storage::list`'s contract. A folder
/// with none is refused by name rather than imported as nothing.
fn csv_documents_in(fs: &dyn Storage, dir: &Path) -> Result<Vec<PathBuf>> {
let csvs: Vec<PathBuf> = fs
.list(dir)?
.into_iter()
.filter(|p| csv_path_p(p))
.collect();
ensure!(!csvs.is_empty(), "no CSV files in folder {}", dir.display());
Ok(csvs)
}
fn parse_csv_field(field: &str) -> Value {
@@ -302,6 +404,118 @@ mod tests {
assert_eq!(records[0]["Name"], Value::String("A".to_string()));
}
// ── Folder arguments (improvise-lpzu) ─────────────────────────────
use crate::storage::MemoryFs;
fn folder(files: &[(&str, &str)]) -> MemoryFs {
let fs = MemoryFs::new();
for (path, body) in files {
fs.write(Path::new(path), body.as_bytes()).unwrap();
}
fs
}
fn names(paths: &[PathBuf]) -> Vec<String> {
paths.iter().map(|p| p.display().to_string()).collect()
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_folder_expands_to_its_csv_files_in_name_order_and_skips_the_rest() {
let fs = folder(&[
("maat/coupling.csv", "entity\n"),
("maat/notes.txt", "not data"),
("maat/age.CSV", "entity\n"),
("maat/out.improv", "v2025-04-09\n"),
("maat/abs-churn.csv", "date\n"),
("elsewhere/other.csv", "x\n"),
]);
let paths = expand_import_paths(&fs, &[ImportPath::Folder("maat".into())]).unwrap();
assert_eq!(
names(&paths),
["maat/abs-churn.csv", "maat/age.CSV", "maat/coupling.csv"]
);
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_folder_with_no_csv_files_is_refused_by_name() {
let only_text = folder(&[("maat/notes.txt", "x"), ("maat/out.improv", "y")]);
let empty = MemoryFs::new();
for fs in [&only_text, &empty] {
let err = expand_import_paths(fs, &[ImportPath::Folder("maat".into())])
.expect_err("a folder with nothing to import is refused");
assert_eq!(err.to_string(), "no CSV files in folder maat");
}
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn files_and_folders_expand_in_argument_order() {
let fs = folder(&[
("b/two.csv", "x\n"),
("b/one.csv", "x\n"),
("a.csv", "x\n"),
("z.csv", "x\n"),
]);
let args = [
ImportPath::File("z.csv".into()),
ImportPath::Folder("b".into()),
ImportPath::File("a.csv".into()),
];
let paths = expand_import_paths(&fs, &args).unwrap();
assert_eq!(names(&paths), ["z.csv", "b/one.csv", "b/two.csv", "a.csv"]);
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn files_with_different_headers_merge_without_inventing_the_missing_columns() {
let fs = folder(&[
(
"m/coupling.csv",
"entity,coupled,degree\nsrc/a.rs,src/b.rs,80\n",
),
("m/age.csv", "entity,age-months\nsrc/a.rs,2\n"),
]);
let merged = merge_csvs_from(&fs, &["m/coupling.csv", "m/age.csv"]).unwrap();
assert!(merged.warnings.is_empty(), "{:?}", merged.warnings);
let source = |item: &str, columns: &[&str]| SourceFile {
item: item.to_string(),
columns: columns.iter().map(|c| c.to_string()).collect(),
};
assert_eq!(
merged.sources,
[
source("coupling", &["entity", "coupled", "degree"]),
source("age", &["entity", "age-months"]),
]
);
assert_eq!(
merged.records,
[
serde_json::json!({"entity": "src/a.rs", "coupled": "src/b.rs", "degree": 80, "File": "coupling"}),
serde_json::json!({"entity": "src/a.rs", "age-months": 2, "File": "age"}),
]
);
}
/// improvise-8jk6.12 (3): two files with one stem are one item, and its
/// columns are both files'. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn two_files_with_one_stem_union_their_columns() {
let fs = folder(&[
("a/x.csv", "entity,one\ne,1\n"),
("b/x.csv", "entity,two\ne,2\n"),
]);
let merged = merge_csvs_from(&fs, &["a/x.csv", "b/x.csv"]).unwrap();
let expected = SourceFile {
item: "x".to_string(),
columns: ["entity", "one", "two"].map(String::from).to_vec(),
};
assert_eq!(merged.sources, [expected]);
}
// ── RFC 4180 edge cases ───────────────────────────────────────────
#[test]
File diff suppressed because it is too large Load Diff
@@ -214,6 +214,7 @@ mod tests {
wb.add_category("Type").unwrap();
wb.add_item_in_group("Month", "Jan", "Q1").unwrap();
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
for (month, value) in [("Jan", 3.0), ("Feb", 4.0)] {
wb.model.set_cell(
@@ -226,7 +227,7 @@ mod tests {
v.set_axis("Month", Axis::Row);
v.set_axis("Type", Axis::Column);
v.prune_empty = true;
v.toggle_group_collapse("Month", "Q1");
v.toggle_group_collapse("Month", &["Q1"]);
}
let csv = String::from_utf8(super::render_csv(&wb, "Default").unwrap()).unwrap();
let rows: Vec<&str> = csv.lines().skip(1).collect();
+34 -15
View File
@@ -2,7 +2,7 @@
use crate::model::category::CategoryKind;
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
use crate::view::{Axis, GroupFold};
use crate::workbook::Workbook;
// The one home of the pipe-quoting rule (improvise-i2m5); a name written here
@@ -119,19 +119,37 @@ pub fn format_md(workbook: &Workbook) -> String {
if view.prune_empty {
w!(out, "prune: true");
}
for (prefix, map) in [
("hidden", &view.hidden_items),
("collapsed", &view.collapsed_groups),
] {
let mut pairs: Vec<_> = map
.iter()
.flat_map(|(cat, items)| {
items.iter().map(move |item| (cat.as_str(), item.as_str()))
})
.collect();
pairs.sort();
for (cat, item) in pairs {
w!(out, "{prefix}: {}/{}", quote_name(cat), quote_name(item));
let mut hidden: Vec<_> = view
.hidden_items
.iter()
.flat_map(|(cat, items)| items.iter().map(move |item| (cat.as_str(), item.as_str())))
.collect();
hidden.sort();
for (cat, item) in hidden {
w!(out, "hidden: {}/{}", quote_name(cat), quote_name(item));
}
let mut depths: Vec<_> = view.expand_depth.iter().collect();
depths.sort();
// Never bare: `expanded: row` would read back as an axis line.
for (cat, depth) in depths {
w!(out, "expanded({depth}): {}", quote_name(cat));
}
let mut folds: Vec<_> = view
.group_folds
.iter()
.flat_map(|(cat, groups)| {
groups
.iter()
.map(move |(g, fold)| (cat.as_str(), g.as_str(), *fold))
})
.collect();
folds.sort();
for (cat, group, fold) in folds {
let target = format!("{}/{}", quote_name(cat), quote_name(group));
match fold {
GroupFold::Collapsed => w!(out, "collapsed: {target}"),
GroupFold::Expanded(n) if n.get() == 1 => w!(out, "expanded: {target}"),
GroupFold::Expanded(n) => w!(out, "expanded({n}): {target}"),
}
}
}
@@ -370,7 +388,8 @@ mod tests {
#[test]
fn format_md_collapsed_group() {
let mut m = two_cat_model();
m.active_view_mut().toggle_group_collapse("Type", "MyGroup");
m.active_view_mut()
.set_group_fold("Type", "MyGroup", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
assert!(text.contains("collapsed: Type/MyGroup"));
}
@@ -120,7 +120,9 @@ view_section = {
view_entry*
}
view_entry = _{ format_line | prune_line | hidden_line | collapsed_line | axis_line }
// `axis_line` is tried before `expanded_line`, so a category named `expanded`
// keeps its `expanded: row`; the writer never spells a category depth bare.
view_entry = _{ format_line | prune_line | hidden_line | collapsed_line | axis_line | expanded_line }
axis_line = {
name ~ ": " ~ axis_kind ~ (", " ~ name)? ~ NEWLINE ~ blank_lines
@@ -135,6 +137,12 @@ prune_line = { "prune: " ~ bool_kw ~ NEWLINE ~ blank_lines }
bool_kw = @{ "true" | "false" }
hidden_line = { "hidden: " ~ name ~ "/" ~ name ~ NEWLINE ~ blank_lines }
collapsed_line = { "collapsed: " ~ name ~ "/" ~ name ~ NEWLINE ~ blank_lines }
// Groups start collapsed. `expanded(N): Cat` opens the category's top N levels;
// `expanded(N): Cat/Group` opens the group and N-1 levels beneath it. A bare
// `expanded:` is N = 1, and N is at least 1. context/code/formula-and-io.md.
expanded_line = { "expanded" ~ fold_depth? ~ ": " ~ name ~ ("/" ~ name)? ~ NEWLINE ~ blank_lines }
fold_depth = { "(" ~ depth_number ~ ")" }
depth_number = @{ '1'..'9' ~ ASCII_DIGIT* }
// ── Names ────────────────────────────────────────────────────────────────────
//
+161 -13
View File
@@ -9,10 +9,10 @@ use pest_derive::Parser;
use crate::formula::{AggFunc, parse_formula};
use crate::model::category::{CategoryKind, Group};
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
use crate::view::{Axis, GroupFold};
use crate::workbook::Workbook;
use crate::formula::quote::unescape_pipe_body;
use crate::formula::quote::{quote_name, unescape_pipe_body};
#[derive(Parser)]
#[grammar = "persistence/improv.pest"]
@@ -40,9 +40,74 @@ struct PView {
format: String,
prune_empty: bool,
hidden: Vec<(String, String)>,
collapsed: Vec<(String, String)>,
folds: Vec<PFold>,
}
/// One `collapsed:` or `expanded(N):` line. `group` is `None` for the category
/// form, which only `expanded` has.
struct PFold {
category: String,
group: Option<String>,
fold: GroupFold,
}
/// A file the grammar accepts and a model cannot hold (improvise-felo.6).
/// Typed, so the generator's property can tell it from a parse failure.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Refusal {
/// `> A[B]` and `> B[A]`: a hierarchy must be a forest.
HierarchyCycle {
category: String,
groups: Vec<String>,
},
/// One view gives one group (or one category's depth) two different folds.
FoldConflict {
view: String,
category: String,
group: Option<String>,
},
}
impl std::fmt::Display for Refusal {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Refusal::HierarchyCycle { category, groups } => write!(
f,
"category {}: the group parents form a cycle through {}",
quote_name(category),
groups
.iter()
.map(|g| quote_name(g))
.collect::<Vec<_>>()
.join(" > ")
),
Refusal::FoldConflict {
view,
category,
group: Some(group),
} => write!(
f,
"view {}: group {}/{} is both expanded and collapsed, or expanded to two depths",
quote_name(view),
quote_name(category),
quote_name(group)
),
Refusal::FoldConflict {
view,
category,
group: None,
} => write!(
f,
"view {}: category {} is expanded to two depths",
quote_name(view),
quote_name(category)
),
}
}
}
impl std::error::Error for Refusal {}
/// Everything pass 1 collects, before any of it is resolved against the model.
#[derive(Default)]
struct ParsedFile {
@@ -332,7 +397,31 @@ fn walk_axis_line(entry: Pair<'_, Rule>, pv: &mut PView) -> Result<()> {
Ok(())
}
/// `## View: Name` plus its axis, format, hidden and collapsed lines.
/// `expanded(N): Cat` or `expanded(N): Cat/Group`; a bare `expanded:` is N = 1.
fn walk_expanded_line(entry: Pair<'_, Rule>) -> Result<PFold> {
let mut depth = std::num::NonZeroUsize::MIN;
let mut names = Vec::new();
for part in entry.into_inner() {
match part.as_rule() {
Rule::fold_depth => {
let digits = first_str(part)?;
depth = digits
.parse()
.with_context(|| format!("expanded({digits}): depth out of range"))?;
}
_ => names.push(extract_name(part)?),
}
}
let mut names = names.into_iter();
let category = names.next().context("expanded_line: no category")?;
Ok(PFold {
category,
group: names.next(),
fold: GroupFold::Expanded(depth),
})
}
/// `## View: Name` plus its axis, format, hidden, collapsed and expanded lines.
fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
let mut inner = pair.into_inner();
let mut pv = PView {
@@ -342,7 +431,7 @@ fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
format: String::new(),
prune_empty: false,
hidden: Vec::new(),
collapsed: Vec::new(),
folds: Vec::new(),
};
for entry in inner {
match entry.as_rule() {
@@ -350,7 +439,15 @@ fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
Rule::format_line => pv.format = first_str(entry)?,
Rule::prune_line => pv.prune_empty = first_str(entry)? == "true",
Rule::hidden_line => pv.hidden.push(extract_name_pair(entry)?),
Rule::collapsed_line => pv.collapsed.push(extract_name_pair(entry)?),
Rule::collapsed_line => {
let (category, group) = extract_name_pair(entry)?;
pv.folds.push(PFold {
category,
group: Some(group),
fold: GroupFold::Collapsed,
});
}
Rule::expanded_line => pv.folds.push(walk_expanded_line(entry)?),
other => bail!("unhandled rule in view_section: {other:?}"),
}
}
@@ -411,6 +508,13 @@ fn build_category(wb: &mut Workbook, pc: &PCategory) -> Result<()> {
None => cat.add_group(Group::new(group_name).with_parent(parent)),
}
}
if let Some(cycle) = cat.hierarchy_cycle() {
return Err(Refusal::HierarchyCycle {
category: pc.name.clone(),
groups: cycle.into_iter().map(String::from).collect(),
}
.into());
}
Ok(())
}
@@ -436,8 +540,29 @@ fn build_view(wb: &mut Workbook, pv: &PView) -> Result<()> {
for (cat, item) in &pv.hidden {
view.hide_item(cat, item);
}
for (cat, grp) in &pv.collapsed {
view.toggle_group_collapse(cat, grp);
for PFold {
category,
group,
fold,
} in &pv.folds
{
let conflict = || Refusal::FoldConflict {
view: pv.name.clone(),
category: category.clone(),
group: group.clone(),
};
match (group, fold) {
(Some(g), _) => match view.group_fold(category, g) {
Some(held) if held != *fold => return Err(conflict().into()),
_ => view.set_group_fold(category, g, *fold),
},
(None, GroupFold::Expanded(n)) => match view.expand_depth(category) {
0 => view.set_expand_depth(category, n.get()),
held if held != n.get() => return Err(conflict().into()),
_ => {}
},
(None, GroupFold::Collapsed) => bail!("a category-wide collapse has no spelling"),
}
}
Ok(())
}
@@ -534,6 +659,7 @@ mod grammar_coverage {
Rule::axis_line | Rule::axis_kind => "walk_axis_line",
Rule::format_line | Rule::prune_line | Rule::bool_kw => "walk_view_section",
Rule::hidden_line | Rule::collapsed_line => "walk_view_section",
Rule::expanded_line | Rule::fold_depth | Rule::depth_number => "walk_expanded_line",
Rule::view_entry => "silent: hoisted into view_section",
Rule::name => "silent: hoisted into its parent",
Rule::pipe_quoted | Rule::pipe_inner | Rule::bare_name => "extract_name",
@@ -1059,12 +1185,14 @@ mod parser_edge_cases {
fn collapsed_group_with_slash_in_name() {
let mut m = Workbook::new("Test");
m.add_category("Type").unwrap();
m.active_view_mut().toggle_group_collapse("Type", "Q1/Q2");
m.active_view_mut()
.set_group_fold("Type", "Q1/Q2", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
let loaded = parse_md(&text).unwrap();
assert!(
loaded.active_view().is_group_collapsed("Type", "Q1/Q2"),
loaded.active_view().group_fold("Type", "Q1/Q2")
== Some(crate::view::GroupFold::Collapsed),
"Collapsed group with '/' in name was corrupted.\n{text}"
);
}
@@ -1604,6 +1732,24 @@ mod generator {
})
}
/// improvise-felo.6: the round-trip properties see `expanded(N):` lines only
/// if the walk emits them, depths included.
#[test]
fn the_generator_writes_expanded_lines_with_and_without_a_depth() {
use proptest::strategy::ValueTree;
use proptest::test_runner::TestRunner;
let mut runner = TestRunner::deterministic();
let strategy = improv_file();
let files: Vec<String> = (0..512)
.map(|_| strategy.new_tree(&mut runner).unwrap().current())
.collect();
let lines = || files.iter().flat_map(|f| f.lines());
assert!(lines().any(|l| l.starts_with("expanded(")), "a depth");
assert!(lines().any(|l| l.starts_with("expanded: ")), "a bare line");
assert!(lines().any(|l| l.starts_with("collapsed: ")), "a collapse");
}
/// improvise-8shu. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn the_grammar_is_parsed_once_however_many_files_the_strategy_yields() {
@@ -1625,7 +1771,7 @@ mod generator {
#[cfg(test)]
mod grammar_prop_tests {
use super::{generator, parse_md};
use super::{Refusal, generator, parse_md};
use crate::persistence::format_md;
use proptest::prelude::*;
@@ -1633,11 +1779,13 @@ mod grammar_prop_tests {
proptest! {
#![proptest_config(ProptestConfig::with_cases(256))]
/// parse(generate()) — every generated file parses without error.
/// parse(generate()) — every generated file parses, or is refused for a
/// named reason the grammar cannot express (a `Refusal`, improvise-felo.6).
#[test]
fn generated_file_parses(file in generator::improv_file()) {
let result = parse_md(&file);
prop_assert!(result.is_ok(),
let refused = result.as_ref().err().is_some_and(|e| e.downcast_ref::<Refusal>().is_some());
prop_assert!(result.is_ok() || refused,
"Generated file failed to parse:\n{}\nError: {}",
file, result.unwrap_err());
}
@@ -267,10 +267,198 @@ fn parse_md_round_trips_text_cell_values() {
#[test]
fn parse_md_round_trips_collapsed_group() {
let mut m = two_cat_model();
m.active_view_mut().toggle_group_collapse("Type", "MyGroup");
m.active_view_mut()
.set_group_fold("Type", "MyGroup", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
let loaded = parse_md(&text).unwrap();
assert!(loaded.active_view().is_group_collapsed("Type", "MyGroup"));
assert!(
loaded.active_view().group_fold("Type", "MyGroup")
== Some(crate::view::GroupFold::Collapsed)
);
}
// ── Group hierarchies and folds (improvise-felo.6) ─────────────────────
/// Year > Quarter > Month as a file spells it: `> Quarter[Year]` lines.
const DATES: &str = "v2025-04-09\n# T\nInitial View: Default\n\n## View: Default\nDate: row\n\
_Measure: column\n@FOLDS@\n## Category: Date\n\
- |2024-01|[Q1-2024]\n- |2024-04|[Q2-2024]\n- |2025-01|[Q1-2025]\n\
> Q1-2024[|2024|]\n> Q2-2024[|2024|]\n> Q1-2025[|2025|]\n\n\
## Category: _Measure\n- Revenue\n";
fn dates_with(folds: &str) -> String {
DATES.replace("@FOLDS@", folds)
}
/// Every spelling of a fold, together in one view, survives save and load —
/// and the writer's own text is stable.
#[test]
fn expanded_and_collapsed_lines_round_trip_together() {
use crate::view::GroupFold;
let text = dates_with(
"expanded(2): Date\ncollapsed: Date/Q1-2024\nexpanded: Date/Q2-2024\n\
expanded(3): Date/|2025|",
);
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
assert_eq!(v.expand_depth("Date"), 2);
assert_eq!(v.group_fold("Date", "Q1-2024"), Some(GroupFold::Collapsed));
assert_eq!(
v.group_fold("Date", "Q2-2024"),
Some(GroupFold::Expanded(std::num::NonZeroUsize::MIN))
);
assert_eq!(
v.group_fold("Date", "2025"),
Some(GroupFold::Expanded(std::num::NonZeroUsize::new(3).unwrap()))
);
let written = format_md(&loaded);
for line in [
"expanded(2): Date",
"collapsed: Date/Q1-2024",
"expanded: Date/Q2-2024",
"expanded(3): Date/|2025|",
] {
assert!(written.lines().any(|l| l == line), "{line} in:\n{written}");
}
assert_eq!(format_md(&parse_md(&written).unwrap()), written);
}
/// A file from before improvise-felo.6 carries only `collapsed:` lines. Every
/// group opens closed, the ones it names AND the ones it does not.
#[test]
fn an_old_file_with_only_collapsed_lines_loads_closed_and_keeps_its_lines() {
let text = dates_with("collapsed: Date/Q1-2024");
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
let cat = loaded.model.category("Date").unwrap();
for group in ["2024", "Q1-2024", "Q2-2024", "2025"] {
assert!(
v.is_group_collapsed("Date", &cat.group_chain(group)),
"{group} opens closed"
);
}
assert!(
format_md(&loaded)
.lines()
.any(|l| l == "collapsed: Date/Q1-2024"),
"the writer keeps the line the file had, though it agrees with the default"
);
}
/// The user's ruling: expanded and collapsed on ONE group is an error, named.
#[test]
fn a_group_both_expanded_and_collapsed_is_refused_naming_view_and_group() {
use super::parse::Refusal;
for folds in [
"collapsed: Date/Q1-2024\nexpanded: Date/Q1-2024",
"expanded(2): Date/Q1-2024\ncollapsed: Date/Q1-2024",
"expanded: Date/Q1-2024\nexpanded(2): Date/Q1-2024",
] {
let err = parse_md(&dates_with(folds)).unwrap_err();
assert_eq!(
err.downcast_ref::<Refusal>(),
Some(&Refusal::FoldConflict {
view: "Default".into(),
category: "Date".into(),
group: Some("Q1-2024".into()),
}),
"{folds}: {err}"
);
assert!(err.to_string().contains("Default"), "{err}");
assert!(err.to_string().contains("Date/Q1-2024"), "{err}");
}
let err = parse_md(&dates_with("expanded(2): Date\nexpanded(3): Date")).unwrap_err();
assert!(err.downcast_ref::<Refusal>().is_some(), "{err}");
}
/// Not the error: a DIFFERENT group collapsed inside an expanded region, and
/// the same line twice.
#[test]
fn a_collapsed_group_inside_an_expanded_region_loads() {
let text = dates_with(
"expanded(2): Date\nexpanded(2): Date/|2024|\ncollapsed: Date/Q1-2024\n\
collapsed: Date/Q1-2024",
);
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
assert!(v.is_group_collapsed("Date", &["2024", "Q1-2024"]));
assert!(!v.is_group_collapsed("Date", &["2024", "Q2-2024"]));
}
/// A cycle among `>` lines is refused on load, naming its members.
#[test]
fn a_cycle_among_group_parents_is_refused_naming_it() {
use super::parse::Refusal;
let text = DATES
.replace("@FOLDS@", "")
.replace("> Q1-2025[|2025|]", "> Q1-2025[|2025|]\n> |2025|[Q1-2025]");
let err = parse_md(&text).unwrap_err();
match err.downcast_ref::<Refusal>() {
Some(Refusal::HierarchyCycle { category, groups }) => {
assert_eq!(category, "Date");
let mut groups = groups.clone();
groups.sort();
assert_eq!(groups, vec!["2025", "Q1-2025"]);
}
other => panic!("not a cycle refusal: {other:?} ({err})"),
}
}
/// `expanded(0)` opens nothing and is not a spelling; `(x)` is not a depth.
#[test]
fn a_malformed_depth_is_a_parse_error() {
assert!(
parse_md(&dates_with("expanded(10): Date")).is_ok(),
"the control"
);
for bad in ["expanded(0): Date", "expanded(x): Date", "expanded(): Date"] {
assert!(parse_md(&dates_with(bad)).is_err(), "{bad}");
}
}
/// A category named `expanded` keeps its axis line, and a category whose NAME is
/// an axis keyword keeps its depth: the writer never spells a depth bare.
#[test]
fn an_axis_line_for_a_category_named_expanded_is_still_an_axis_line() {
let mut wb = Workbook::new("T");
wb.add_category("expanded").unwrap();
wb.add_category("row").unwrap();
wb.active_view_mut().set_axis("expanded", Axis::Column);
wb.active_view_mut().set_expand_depth("row", 1);
let loaded = parse_md(&format_md(&wb)).unwrap();
assert_eq!(loaded.active_view().axis_of("expanded"), Axis::Column);
assert_eq!(loaded.active_view().expand_depth("row"), 1);
}
/// `z` writes through ONE map, so no sequence of toggles can save a file that
/// names a group both ways. Every toggle order over three levels, twice over.
#[test]
fn no_sequence_of_toggles_writes_a_conflicting_file() {
let base = parse_md(&dates_with("expanded(1): Date")).unwrap();
let chains: [&[&str]; 4] = [
&["2024"],
&["2024", "Q1-2024"],
&["2024", "Q2-2024"],
&["2025", "Q1-2025"],
];
for order in 0..(4usize.pow(4)) {
let mut wb = base.clone();
let mut code = order;
for _ in 0..4 {
let chain = chains[code % 4];
code /= 4;
wb.active_view_mut().toggle_group_collapse("Date", chain);
let text = format_md(&wb);
let back = parse_md(&text).unwrap_or_else(|e| panic!("{e}\n{text}"));
for chain in chains {
assert_eq!(
back.active_view().is_group_collapsed("Date", chain),
wb.active_view().is_group_collapsed("Date", chain),
"{chain:?} after order {order}:\n{text}"
);
}
}
}
}
// ── parse_md: number format roundtrip ───────────────────────────────
@@ -336,7 +524,8 @@ fn full_roundtrip_preserves_all_features() {
m.active_view_mut().set_axis("Month", Axis::Page);
m.active_view_mut().set_page_selection("Month", "Jan");
m.active_view_mut().hide_item("Type", "Gas");
m.active_view_mut().toggle_group_collapse("Type", "G1");
m.active_view_mut()
.set_group_fold("Type", "G1", crate::view::GroupFold::Collapsed);
m.active_view_mut().number_format = ",.0".to_string();
let text = format_md(&m);
@@ -361,7 +550,7 @@ fn full_roundtrip_preserves_all_features() {
assert_eq!(v.axis_of("Month"), Axis::Page);
assert_eq!(v.page_selection("Month"), Some("Jan"));
assert!(v.is_hidden("Type", "Gas"));
assert!(v.is_group_collapsed("Type", "G1"));
assert!(v.group_fold("Type", "G1") == Some(crate::view::GroupFold::Collapsed));
assert_eq!(v.number_format, ",.0");
}
@@ -1187,7 +1376,7 @@ fn a_label_category_survives_the_round_trip_as_a_label() {
#[test]
fn a_document_at_the_category_limit_plus_a_label_reloads() {
let mut wb = Workbook::new("T");
for i in 0..12 {
for i in 0..crate::model::MAX_CATEGORIES {
wb.add_category(format!("C{i}")).unwrap();
}
wb.add_label_category("Note").unwrap();
@@ -1197,7 +1386,10 @@ fn a_document_at_the_category_limit_plus_a_label_reloads() {
loaded.model.category("Note").unwrap().kind,
CategoryKind::Label
);
assert_eq!(loaded.model.regular_category_names().len(), 12);
assert_eq!(
loaded.model.regular_category_names().len(),
crate::model::MAX_CATEGORIES
);
}
/// A file written before the `kind:` line existed reads its categories as
@@ -1234,3 +1426,28 @@ fn a_loaded_label_category_is_not_on_a_pivot_axis() {
);
}
}
/// ChatGPT Codex on #317: a fold built through the public API must survive its
/// own save. `Expanded(0)` is now unconstructible; every depth that is survives.
#[test]
fn every_fold_the_api_can_build_reads_back_from_the_file_it_writes() {
use crate::view::GroupFold;
use std::num::NonZeroUsize;
let base = parse_md(&dates_with("")).unwrap();
for n in 1..=3 {
let depth = NonZeroUsize::new(n).unwrap();
for fold in [GroupFold::Collapsed, GroupFold::Expanded(depth)] {
let mut wb = base.clone();
wb.active_view_mut().set_group_fold("Date", "2024", fold);
wb.active_view_mut().set_expand_depth("Date", n);
let text = format_md(&wb);
let back = parse_md(&text).unwrap_or_else(|e| panic!("{e}\n{text}"));
assert_eq!(
back.active_view().group_fold("Date", "2024"),
Some(fold),
"{text}"
);
assert_eq!(back.active_view().expand_depth("Date"), n, "{text}");
}
}
}
+505 -8
View File
@@ -2,10 +2,11 @@
//! The two properties this dispatch owes: `context/code/clients.md`.
use crate::{
BasisCache, CategoryTreeCache, Command, DocumentCache, FormulaCache, GridCache, Projection,
Rejection, Snapshot, TileBarCache, ViewListCache, ViewState, WidgetCache, reduce_view,
settled_mode,
BasisCache, CategoryTreeCache, Command, DocumentCache, EditorSession, FormulaCache, GridCache,
ModelCommand, Projection, Rejection, Snapshot, TileBarCache, ViewListCache, ViewState,
WidgetCache, reduce_view, settled_mode,
};
use improvise_core::model::record::RecordHash;
use serde::Serialize;
/// One client's whole state: the portable session slice plus one cache per
@@ -43,6 +44,58 @@ pub struct ClientState {
/// replay the rest. Why a queue: `context/code/clients.md`.
#[serde(skip)]
pub in_flight: std::collections::VecDeque<Sent>,
/// The status text last observed, and the editor it was written in while
/// that editor is still open (improvise-1x9h, `context/code/clients.md`).
#[serde(skip)]
pub status_seen: String,
#[serde(skip)]
pub status_owner: Option<EditorSession>,
/// The cell an open editor commits to, captured as the user saw it.
/// When it is taken, and when not yet: `context/code/clients.md`.
#[serde(skip)]
pub opened: Option<Opened>,
}
/// What an open cell editor commits to. After a refusal the retry may name only
/// the cell it named (`Retry`) or none (`Gone`): improvise-20ek, -9ww8, -8phw.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Opened {
Target(OpenedCell),
Retry(RecordHash, OpenedTarget),
Gone(RecordHash, OpenedTarget),
}
impl Opened {
/// Which kind of cell the editor commits to, whether or not it is captured.
pub fn target(&self) -> &OpenedTarget {
match self {
Opened::Target(cell) => &cell.target,
Opened::Retry(_, target) | Opened::Gone(_, target) => target,
}
}
}
/// A cell editor's target: the record, which cell of it, the record's write
/// counter at open, and (for a rename) every counter shown: improvise-8xq0, -milr.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct OpenedCell {
pub record: RecordHash,
pub target: OpenedTarget,
pub version: u64,
pub seen: Vec<(RecordHash, u64)>,
/// The document the editor opened on, so a swap refuses it (improvise-vndi).
pub document_generation: u64,
/// The view that drew the cell: an empty record resolves only there (improvise-wpxr).
pub view: String,
}
/// Which cell of the record an editor opened on, as the projection named it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum OpenedTarget {
/// A stored value: `SetCell`/`ClearCell` names it (improvise-8xq0).
Value { measure: Option<String> },
/// A records-mode coordinate: `RenameRecordCoordinate` names it.
Coordinate { category: String },
}
/// One frame this client has sent. `awaited` is true exactly when the host
@@ -51,6 +104,9 @@ pub struct ClientState {
pub struct Sent {
pub cmds: Vec<Command>,
pub awaited: bool,
/// Heard from the host rather than sent: it settles after the frame it
/// arrived during, never with it (improvise-jpba, `clients.md`).
pub relayed: bool,
}
impl ClientState {
@@ -75,18 +131,65 @@ impl ClientState {
}
self.settle_mode();
self.settle_editor();
self.settle_opened();
self.settle_status();
self.revision += 1;
}
/// Fold one command. Total and deterministic: a command naming something
/// the client does not have is dropped, never a panic.
/// Fold one command **the host sent**, and keep it for the settled slice.
/// Total: a command naming something absent is dropped, never a panic.
pub fn fold(&mut self, cmd: &Command) {
self.settle_relay(cmd);
match (cmd, self.in_flight.is_empty()) {
(Command::Project(_), _) | (_, true) => self.apply(cmd),
// Queued behind the frame it answers, so what was sent after it
// replays on top of it (improvise-xd3q, `clients.md`).
(_, false) => self.rebase(),
}
}
/// The view rebuilt as the settled slice plus everything still in flight.
fn rebase(&mut self) {
self.replay();
self.settle_mode();
self.settle_editor();
self.settle_opened();
self.settle_status();
self.revision += 1;
}
/// A relay the host has already applied to this client's mirror, so a
/// later refusal must not roll it back. Where it queues: `clients.md`.
fn settle_relay(&mut self, cmd: &Command) {
let behind = 1 + self
.in_flight
.iter()
.skip(1)
.take_while(|s| s.relayed)
.count();
match (cmd, self.in_flight.is_empty()) {
(Command::Project(_), _) => {}
(_, true) => reduce_view(&mut self.settled, cmd),
(_, false) => self.in_flight.insert(
behind,
Sent {
cmds: vec![cmd.clone()],
awaited: false,
relayed: true,
},
),
}
}
fn apply(&mut self, cmd: &Command) {
match cmd {
Command::Project(projection) => self.apply_projection(projection),
other => reduce_view(&mut self.view, other),
}
self.settle_mode();
self.settle_editor();
self.settle_opened();
self.settle_status();
self.revision += 1;
}
@@ -97,13 +200,20 @@ impl ClientState {
self.in_flight.push_back(Sent {
cmds: cmds.to_vec(),
awaited,
relayed: false,
});
self.settle_sent();
for cmd in cmds {
self.fold(cmd);
self.apply(cmd);
}
}
/// A status this client says to itself now. It queues after every frame
/// already sent, so no rebase replays one over it (improvise-n0pb, `clients.md`).
pub fn notify(&mut self, message: String) {
self.fold_outgoing(&[Command::SetStatus(message)]);
}
/// The host applied the oldest awaited frame whole, so it is the settled
/// slice's now.
pub fn accepted(&mut self) {
@@ -113,6 +223,9 @@ impl ClientState {
}
}
self.settle_sent();
self.settle_editor();
self.settle_opened();
self.revision += 1;
}
/// The host applied **none** of the oldest awaited frame (ADR 001 clause
@@ -121,15 +234,27 @@ impl ClientState {
// A refused *save* answers no `Commands` frame, so it must not consume
// one's entry — `Rejection::refused_an_edit` is that question.
if rejection.refused_an_edit() {
self.in_flight.pop_front();
let dropped = self.in_flight.pop_front();
self.settle_sent();
self.replay();
self.opened = written_to(dropped.as_ref()).map(|(r, t)| Opened::Retry(r, t));
}
self.last_rejection = Some(rejection);
self.settle_mode();
self.settle_editor();
self.settle_opened();
self.settle_status();
self.revision += 1;
}
/// Whether the status belongs to the editor still open, so a renderer must
/// not expire it (improvise-1x9h). The TUI asks the same `EditorSession`.
pub fn status_stands(&self) -> bool {
!self.view.status_msg.is_empty()
&& self.status_owner.is_some()
&& self.status_owner == EditorSession::of(&self.view)
}
/// The settled slice plus every frame still in flight, in order. What a
/// dropped frame leaves behind is the *rest* of them, not nothing.
fn replay(&mut self) {
@@ -161,6 +286,51 @@ impl ClientState {
self.view.mode = settled_mode(&self.view.mode, self.grid.records);
}
/// Whether a frame the host will answer is still in flight, so this
/// client's cache does not yet show the host's answer to its own writes.
pub(crate) fn awaiting_answer(&self) -> bool {
self.in_flight.iter().any(|sent| sent.awaited)
}
/// Capture the cell an open editor commits to, but only from a cache that
/// shows the host's answer to this client's own writes (improvise-fdk1).
fn settle_opened(&mut self) {
let answered = !self.awaiting_answer();
self.opened = match (self.view.mode.is_editing(), self.opened.take()) {
(false, _) => None,
(true, Some(Opened::Retry(record, target))) if answered => {
Some(match self.captured() {
Some(cell) if cell.record == record && cell.target == target => {
Opened::Target(cell)
}
_ => Opened::Gone(record, target),
})
}
(true, Some(opened)) => Some(opened),
(true, None) if answered => self.captured().map(Opened::Target),
(true, None) => None,
};
}
/// The cell under the cursor, with its counter; a rename also keeps every
/// counter this client holds, so its destination is claimed (improvise-milr).
fn captured(&self) -> Option<OpenedCell> {
let (record, target) = crate::resolve::opened_at_cursor(self)?;
let basis = self.basis.current()?;
let seen = match target {
OpenedTarget::Coordinate { .. } => basis.records.clone(),
OpenedTarget::Value { .. } => Vec::new(),
};
Some(OpenedCell {
record,
target,
version: basis.record(record)?,
seen,
document_generation: basis.document_generation,
view: self.view.active_view.clone(),
})
}
/// An editor that arrived without its buffer is seeded here, so what is
/// drawn and what Enter commits read one buffer (improvise-ktlk).
fn settle_editor(&mut self) {
@@ -175,6 +345,53 @@ impl ClientState {
self.view.buffers.insert(key, seed.clone());
self.view.opened_editor = Some(crate::OpenedEditor::on(seed));
}
self.settle_awaited_seed();
self.settle_declined_editor();
}
/// An editor opened after a write, or on a cell not projected yet, takes its
/// seed once both are answered, and never over typing (improvise-3uzg/7mro).
fn settle_awaited_seed(&mut self) {
let waiting = self.view.mode.is_editing()
&& self
.view
.opened_editor
.as_ref()
.is_some_and(|opened| opened.awaiting && !opened.touched);
let answered = !self.in_flight.iter().any(|sent| sent.awaited);
if let (true, true, Some(seed)) = (waiting, answered, crate::resolve::projected_seed(self))
{
self.view.buffers.insert("edit".to_string(), seed.clone());
self.view.opened_editor = Some(crate::OpenedEditor::on(seed));
}
}
/// A new message is owned by the editor open as it arrives; an old one loses
/// its owner for good once that editor is not the open one.
fn settle_status(&mut self) {
let now = EditorSession::of(&self.view);
if self.view.status_msg != self.status_seen {
self.status_seen = self.view.status_msg.clone();
self.status_owner = now;
} else if self.status_owner != now {
self.status_owner = None;
}
}
/// An untouched editor on a cell `i` declines closes once the host projects
/// that cell: an advance past the window opens before it can know (5m9z).
fn settle_declined_editor(&mut self) {
let (row, col) = self.view.cursor().selected;
let untouched = self.view.mode.is_editing()
&& self
.view
.opened_editor
.as_ref()
.is_some_and(|opened| !opened.touched);
if let (true, Some(refusal)) = (untouched, crate::resolve::declined_at(self, row, col)) {
self.view.mode = settled_mode(&crate::AppMode::Normal, self.grid.records);
self.view.status_msg = refusal;
}
}
/// The widget dispatch: a projection names the widget it is addressed to.
@@ -192,6 +409,31 @@ impl ClientState {
}
}
/// The cell a refused frame wrote, if it wrote one: a rename's coordinate or a
/// value's record and measure (improvise-9ww8, improvise-8phw).
fn written_to(sent: Option<&Sent>) -> Option<(RecordHash, OpenedTarget)> {
sent?.cmds.iter().find_map(|cmd| match cmd {
Command::Model { cmd, .. } => match cmd {
ModelCommand::RenameRecordCoordinate {
record, category, ..
} => Some((
*record,
OpenedTarget::Coordinate {
category: category.clone(),
},
)),
ModelCommand::SetCell { cell, .. } | ModelCommand::ClearCell { cell, .. } => Some((
cell.record,
OpenedTarget::Value {
measure: cell.measure.clone(),
},
)),
_ => None,
},
_ => None,
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -281,7 +523,7 @@ mod tests {
])));
live.fold(&Command::Project(Projection::TileBar(vec![TileEntry {
category: "Region".into(),
regular: true,
kind: crate::CategoryKind::Regular,
axis: Axis::Row,
}])));
live.fold(&Command::Project(Projection::ViewList(vec![ViewSummary {
@@ -398,6 +640,166 @@ mod tests {
);
}
/// An awaited frame that also places the cursor at `row`.
fn placing_commit(row: usize) -> Vec<Command> {
let mut frame = commit_like("placed");
frame.push(Command::SetSelected { row, col: 0 });
frame
}
/// improvise-3kfc: a relay queued behind a refused head must settle, or the
/// next answer pops it instead of its frame. Evidence: clients.md.
#[test]
fn a_relay_behind_a_refused_frame_settles_before_the_next_answer() {
let mut state = ClientState::default();
state.fold_outgoing(&placing_commit(2));
state.fold_outgoing(&placing_commit(3));
state.fold(&Command::ResetCursor);
state.refused(Rejection::LayoutMoved);
state.refused(Rejection::LayoutMoved);
assert_eq!(
state.view.cursor().selected,
(0, 0),
"both placements were refused"
);
assert_eq!(
state.view.cursor(),
state.settled.cursor(),
"view and settled split"
);
assert!(state.in_flight.is_empty(), "both frames were answered");
}
/// improvise-3kfc, the root on gh/main: this client's own view-only frame
/// behind a refused head. Evidence: clients.md.
#[test]
fn a_view_only_frame_behind_a_refused_frame_settles_before_the_next_answer() {
let mut state = ClientState::default();
state.fold_outgoing(&placing_commit(3));
state.fold_outgoing(&[Command::SetSelected { row: 2, col: 0 }]);
state.fold_outgoing(&placing_commit(4));
state.refused(Rejection::LayoutMoved);
state.refused(Rejection::LayoutMoved);
assert_eq!(
state.view.cursor().selected,
(2, 0),
"the unanswered move stands"
);
assert_eq!(
state.view.cursor(),
state.settled.cursor(),
"view and settled split"
);
assert!(
state.in_flight.is_empty(),
"both awaited frames were answered"
);
}
/// A write, then the editor its advance reopens with the seed still to come.
fn write_then_awaiting_editor() -> Vec<Command> {
let mut frame = commit_like("written");
frame.extend([
Command::SetBuffer {
name: "edit".into(),
value: String::new(),
},
Command::SetOpenedEditor(Some(crate::OpenedEditor::awaiting())),
Command::ChangeMode(crate::AppMode::editing()),
]);
frame
}
/// One row, one column, the cell's stored text `edit_text`.
fn answered_with(state: &mut ClientState, edit_text: &str) {
let mut cell = CellDisplay::number(edit_text);
cell.edit_text = Some(edit_text.into());
cell.id = Some(crate::CellId::Record(crate::RecordCell {
record: serde_json::from_str("\"fe4c8e5e9cbd578e0d388f4b2b5a7879\"").unwrap(),
measure: Some("Revenue".into()),
}));
for cmd in [
GridUpdate::RowFrame(AxisFrame::flat(["North"])),
GridUpdate::ColFrame(AxisFrame::flat(["Revenue"])),
GridUpdate::Cells(vec![((0, 0), cell)]),
] {
state.fold(&Command::Project(Projection::Grid(cmd)));
}
}
/// BUG (improvise-3uzg) — see `context/test-evidence/clients.md`.
#[test]
fn an_awaited_seed_waits_for_the_answer_and_never_overwrites_typing() {
let mut state = ClientState::default();
answered_with(&mut state, "2");
state.fold_outgoing(&write_then_awaiting_editor());
assert_eq!(state.view.buffers.get("edit").map(String::as_str), Some(""));
answered_with(&mut state, "3");
assert_eq!(
state.view.buffers.get("edit").map(String::as_str),
Some(""),
"seeded while the write was still unanswered"
);
state.accepted();
assert_eq!(
state.view.buffers.get("edit").map(String::as_str),
Some("3"),
"the answered editor must open on the host's grid"
);
assert_eq!(
state.view.opened_editor,
Some(crate::OpenedEditor::on("3".into()))
);
let mut typed_ahead = ClientState::default();
answered_with(&mut typed_ahead, "2");
typed_ahead.fold_outgoing(&write_then_awaiting_editor());
typed_ahead.fold_outgoing(&[
Command::SetBuffer {
name: "edit".into(),
value: "5".into(),
},
Command::SetOpenedEditor(crate::resolve::edited(
typed_ahead.view.opened_editor.as_ref(),
"edit",
)),
]);
answered_with(&mut typed_ahead, "3");
typed_ahead.accepted();
assert_eq!(
typed_ahead.view.buffers.get("edit").map(String::as_str),
Some("5"),
"a seed arriving after the user typed must not overwrite the typing"
);
}
/// BUG (improvise-4cb5) — see `context/test-evidence/clients.md`.
#[test]
fn an_awaited_seed_arriving_after_escape_leaves_the_buffer_empty() {
let mut state = ClientState::default();
answered_with(&mut state, "2");
state.fold_outgoing(&write_then_awaiting_editor());
state.fold_outgoing(&[
Command::SetBuffer {
name: "edit".into(),
value: String::new(),
},
Command::ChangeMode(crate::AppMode::Normal),
]);
answered_with(&mut state, "3");
state.accepted();
assert_eq!(state.view.mode, crate::AppMode::Normal);
assert_eq!(
state.view.buffers.get("edit").map(String::as_str),
Some(""),
"the late seed was written into the buffer of an editor Esc had closed"
);
}
/// A frame the host does not answer must not consume an answer, or the
/// queue addresses the wrong frame. Evidence file.
#[test]
@@ -435,6 +837,101 @@ mod tests {
);
}
/// improvise-1x9h: a message written while an editor is open stands until
/// that editor closes, and never again after. Evidence file.
#[test]
fn a_message_stands_while_the_editor_it_was_written_in_is_open() {
let mut state = ClientState::default();
state.fold(&Command::ChangeMode(crate::AppMode::formula_edit()));
state.fold(&Command::SetStatus("Formula error".into()));
assert!(
state.status_stands(),
"a refusal beside its editor must stand"
);
state.fold(&Command::SetBuffer {
name: "formula".into(),
value: "Total = ".into(),
});
assert!(state.status_stands(), "typing is the same editor");
state.fold(&Command::ChangeMode(crate::AppMode::FormulaPanel));
assert!(!state.status_stands(), "the refusal outlived its editor");
state.fold(&Command::ChangeMode(crate::AppMode::formula_edit()));
assert_eq!(state.view.status_msg, "Formula error", "nothing cleared it");
assert!(
!state.status_stands(),
"an equal editor opened later is another opening, and the message is not its"
);
}
/// A relay that rebases the view off the editor ends its message's ownership,
/// as a fold would (improvise-0dbk). Evidence file.
#[test]
fn a_rebase_off_the_editor_ends_the_message_standing() {
let mut state = ClientState::default();
state.fold(&Command::SetSelected { row: 1, col: 1 });
state.fold(&Command::ChangeMode(crate::AppMode::editing()));
state.fold(&Command::SetStatus("a note about this cell".into()));
assert!(state.status_stands(), "precondition: the note stands");
state.fold_outgoing(&commit_like("unused")[..1]);
assert!(
!state.in_flight.is_empty(),
"the relay must rebase, not apply"
);
state.fold(&Command::ResetCursor);
assert_eq!(
state.view.cursor().selected,
(0, 0),
"the rebase moved the cursor"
);
state.fold_outgoing(&[Command::SetSelected { row: 1, col: 1 }]);
assert!(
!state.status_stands(),
"an equal editor after a rebase took the old message back"
);
}
/// The editor is the one it was WRITTEN in: not any editor, and not the next
/// cell's after a commit advances (improvise-1x9h, sx7h). Evidence file.
#[test]
fn a_message_does_not_stand_beside_an_editor_it_was_not_written_in() {
let mut state = ClientState::default();
state.fold(&Command::SetStatus("saved".into()));
state.fold(&Command::ChangeMode(crate::AppMode::editing()));
assert!(
!state.status_stands(),
"written in Normal, it is the timer's"
);
let mut state = ClientState::default();
state.fold(&Command::SetSelected { row: 1, col: 1 });
state.fold(&Command::ChangeMode(crate::AppMode::editing()));
state.fold(&Command::SetStatus("a note about this cell".into()));
assert!(state.status_stands());
state.fold(&Command::SetSelected { row: 2, col: 1 });
assert!(
!state.status_stands(),
"the note followed the editor down a row"
);
let mut state = ClientState::default();
state.fold(&Command::SetSelected { row: 1, col: 1 });
state.fold(&Command::ChangeMode(crate::AppMode::editing()));
state.fold(&Command::SetOpenedEditor(Some(crate::OpenedEditor::on(
"4".into(),
))));
state.fold(&Command::SetStatus("a note about this cell".into()));
assert!(state.status_stands());
state.fold(&Command::SetOpenedEditor(Some(crate::OpenedEditor::on(
"47".into(),
))));
assert!(
!state.status_stands(),
"a re-opening on the same cell is another editor"
);
}
#[test]
fn every_fold_bumps_the_revision_so_a_renderer_can_poll_it() {
let mut state = ClientState::default();
+13 -2
View File
@@ -335,6 +335,7 @@ mod key_row {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct KeyRow {
keys: Vec<String>,
command: &'static str,
description: &'static str,
}
@@ -343,6 +344,12 @@ mod key_row {
&self.keys
}
/// The command the keys run, kept so a front end can ask whether it
/// can run it (improvise-uln5).
pub fn command(&self) -> &'static str {
self.command
}
pub fn description(&self) -> &'static str {
self.description
}
@@ -351,7 +358,7 @@ mod key_row {
/// advertises a key that does not exist.
fn looked_up(
mode: &ModeKey,
command: &str,
command: &'static str,
args: Option<&[String]>,
description: &'static str,
) -> Option<Self> {
@@ -360,7 +367,11 @@ mod key_row {
.map(|(c, m)| key_label(c, *m))
.collect();
keys.extend(prefixed_labels(mode, command, args));
(!keys.is_empty()).then_some(Self { keys, description })
(!keys.is_empty()).then_some(Self {
keys,
command,
description,
})
}
}
+11 -4
View File
@@ -347,6 +347,13 @@ impl TextEntry {
self
}
/// Bind `key` to a commit that clears the buffer **itself**, first, because
/// it reopens an editor a trailing clear would blank (improvise-fy9o).
pub fn commit_reopening(mut self, key: KeyCode, cmd: &'static str, args: Vec<String>) -> Self {
self.km.bind_args(key, KeyModifiers::NONE, cmd, args);
self
}
/// Replace a spine binding — used only by CommandMode, whose Backspace
/// exits the mode on an empty buffer instead of popping a character.
pub fn rebind(mut self, key: KeyCode, cmd: &'static str) -> Self {
@@ -853,8 +860,8 @@ impl Keymaps {
// edit-mode as an argument, so the command stays mode-agnostic. ──
let ed = Arc::new(
TextEntry::new(&AppMode::editing(), "normal", global.clone())
.commit(KeyCode::Enter, "commit-cell-edit", vec!["editing".into()])
.commit(
.commit_reopening(KeyCode::Enter, "commit-cell-edit", vec!["editing".into()])
.commit_reopening(
KeyCode::Tab,
"commit-and-advance-right",
vec!["editing".into()],
@@ -869,12 +876,12 @@ impl Keymaps {
ModeKey::RecordsEditing,
Arc::new(
TextEntry::new(&AppMode::records_editing(), "records-normal", ed)
.commit(
.commit_reopening(
KeyCode::Enter,
"commit-cell-edit",
vec!["records-editing".into()],
)
.commit(
.commit_reopening(
KeyCode::Tab,
"commit-and-advance-right",
vec!["records-editing".into()],
+34 -3
View File
@@ -27,13 +27,17 @@ pub use frame::{ClientFrame, ServerFrame};
pub use help::{HelpLine, Hint, hint_line, hints, mode_label};
pub use key::{KeyCode, KeyEvent, KeyModifiers};
pub use keymap::{Binding, KeyPattern, Keymap, Keymaps, ModeKey};
pub use reduce::{Command, HelpPage, ModelCommand, records_named, reduce_view};
pub use reduce::{
Command, HelpPage, ModelCommand, NamedRecord, RECORD_COORDS_CANNOT_BE_EMPTY, named_records,
records_named, reduce_view,
};
pub use render::{
AxisFrame, CategoryRow, CategoryTreeCache, CategoryTreeUpdate, CellDisplay, CellId, CreateKind,
DocumentCache, FormulaCache, FormulaEntry, FrameEntry, FrameGroup, GridCache, GridExtent,
GridUpdate, MAX_ROW_HEADER_W, MIN_ROW_HEADER_W, Projection, RecordCell, Snapshot, TileBarCache,
TileEntry, ViewListCache, ViewSummary, WidgetCache, WizardCache, WizardDateField,
WizardFieldKind, WizardProposal, WizardStepKind, header_level_widths, opening_tile,
TileEntry, ViewListCache, ViewSummary, WidgetCache, WizardCache, WizardChoice, WizardCollision,
WizardDateField, WizardFieldKind, WizardProposal, WizardStepKind, header_level_widths,
non_item_refusal, opening_tile,
};
pub use resolve::{
Click, KeyOutcome, OpenedEditor, Request, click, edited, read_buffer, resolve, typed,
@@ -43,6 +47,8 @@ pub use viewmodel::{CatTreeEntry, flatten_cat_tree, view_axis_summary};
// `improvise-core`'s, because the index that resolves one lives in
// `DataStore`. Re-exported so a client depends on the protocol alone.
pub use improvise_core::model::record::{AggregateHash, RecordHash};
// A tile carries its kind so a client can rank it without sniffing names.
pub use improvise_core::model::category::CategoryKind;
pub use improvise_core::view::layout::PageAxis;
// On the wire in `ModelCommand::SetCell`, so a client needs to name it
@@ -56,6 +62,10 @@ pub use improvise_core::formula::FormulaId;
/// it bounds, so help-page effects stay terminal-independent.
pub const HELP_PAGE_COUNT: usize = 5;
/// What a command says when the cursor sits on a collapsed group's row. Here,
/// not in the command layer, because a browser refuses in the same words.
pub const FOLD_REFUSAL: &str = "Collapsed group — press z to expand it first";
/// Where one client is looking within one view. Per-client (improvise-8uz);
/// `View` no longer carries it — `context/code/protocol.md` §1.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
@@ -554,6 +564,27 @@ impl ViewState {
}
}
/// What one opening of a minibuffer sets. Equal at two moments means the same
/// editor is still open; both front ends ask this. `context/code/tui.md`.
#[derive(Debug, Clone, PartialEq)]
pub struct EditorSession {
mode: AppMode,
cell: (usize, usize),
seed: Option<String>,
}
impl EditorSession {
/// The editor open in `view`, or `None` outside a text-entry mode.
pub fn of(view: &ViewState) -> Option<Self> {
view.mode.minibuffer()?;
Some(Self {
mode: view.mode.clone(),
cell: view.cursor().selected,
seed: view.opened_editor.as_ref().map(|o| o.seed.clone()),
})
}
}
#[cfg(test)]
mod boundary_tests {
/// improvise-cqi AC 2: the part of the boundary the compiler can hold.
+68 -13
View File
@@ -79,17 +79,28 @@ pub enum Command {
Project(Projection),
}
/// A mutation of the shared document: 25 variants, derived not invented (§2). A
/// A mutation of the shared document: 26 variants, derived not invented (§2). A
/// view-configuration variant **names its view** (improvise-lltt).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum ModelCommand {
// ── ModelChange::CellData ───────────────────────────────────────
/// Write one stored cell: a [`RecordCell`] and **nothing else** (ADR 002).
/// Write one cell: a [`RecordCell`] and the view that drew it (ADR 002, wpxr).
SetCell {
view: String,
cell: RecordCell,
value: CellValue,
},
ClearCell(RecordCell),
ClearCell {
view: String,
cell: RecordCell,
},
/// Move a whole record to a new coordinate, creating the item if need be.
/// Position-free: it names the record, never a row (improvise-fdk1).
RenameRecordCoordinate {
record: RecordHash,
category: String,
item: String,
},
AddItem {
category: String,
item: String,
@@ -107,9 +118,9 @@ pub enum ModelCommand {
raw: String,
target_category: String,
},
/// Remove the formula `id` names; refused if it is already gone.
RemoveFormula {
target: String,
target_category: String,
id: FormulaId,
},
/// Edit the formula `id` names in place, if it still reads `was` — an edit
/// of that formula, not a remove-then-add (improvise-km71).
@@ -176,11 +187,48 @@ pub enum ModelCommand {
},
}
/// Why a rename to an empty coordinate is refused. One home, read by the host's
/// refusal and by a client composing the same answer (improvise-fdk1).
pub const RECORD_COORDS_CANNOT_BE_EMPTY: &str = "Record coordinates cannot be empty";
/// Which records a [`ModelCommand`] writes. The **one** home: the server
/// checks against it and a client claims by it — `context/code/protocol.md` §2.
pub fn records_named(cmd: &ModelCommand) -> Vec<RecordHash> {
named_records(cmd)
.into_iter()
.map(|named| named.record())
.collect()
}
/// A record a [`ModelCommand`] writes, and where the host looks for it. Why a
/// rename is `Stored`: `context/code/protocol.md` (improvise-wpxr).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum NamedRecord<'a> {
/// A cell write, resolved in the view that drew it: an empty record too.
Drawn { view: &'a str, record: RecordHash },
/// A coordinate rename, which only a record holding cells can undergo.
Stored(RecordHash),
}
impl NamedRecord<'_> {
pub fn record(&self) -> RecordHash {
match self {
NamedRecord::Drawn { record, .. } | NamedRecord::Stored(record) => *record,
}
}
}
/// The exhaustive list [`records_named`] is read off, each record with where it
/// is looked for (improvise-wpxr).
pub fn named_records(cmd: &ModelCommand) -> Vec<NamedRecord<'_>> {
match cmd {
ModelCommand::SetCell { cell, .. } | ModelCommand::ClearCell(cell) => vec![cell.record],
ModelCommand::SetCell { view, cell, .. } | ModelCommand::ClearCell { view, cell } => {
vec![NamedRecord::Drawn {
view: view.as_str(),
record: cell.record,
}]
}
ModelCommand::RenameRecordCoordinate { record, .. } => vec![NamedRecord::Stored(*record)],
ModelCommand::AddItem { .. }
| ModelCommand::AddItemInGroup { .. }
@@ -512,12 +560,13 @@ mod tests {
}])),
Command::Project(Projection::TileBar(vec![TileEntry {
category: "Region".into(),
regular: true,
kind: crate::CategoryKind::Regular,
axis: Axis::Page,
}])),
// One populated value per payload family: `RecordCell`, `Axis`,
// `PathBuf`. The exhaustive 26-variant walk is in the host crate.
modelled(ModelCommand::SetCell {
view: "Default".into(),
cell: RecordCell {
record: RecordHash::of(&CellKey::new(vec![("Region".into(), "East".into())])),
measure: Some("Revenue".into()),
@@ -621,8 +670,17 @@ mod tests {
}
// No `..` anywhere: a field added to a variant must be classified here.
match cmd {
ModelCommand::SetCell { cell, value: v } => record(cell) + value(v),
ModelCommand::ClearCell(cell) => record(cell),
ModelCommand::SetCell {
view: v_name,
cell,
value: v,
} => name(v_name) + record(cell) + value(v),
ModelCommand::ClearCell { view, cell } => name(view) + record(cell),
ModelCommand::RenameRecordCoordinate {
record: r,
category,
item,
} => surrogate(r) + name(category) + name(item),
ModelCommand::AddItem { category, item }
| ModelCommand::RemoveItem { category, item } => name(category) + name(item),
ModelCommand::AddItemInGroup {
@@ -634,10 +692,7 @@ mod tests {
raw,
target_category,
} => name(raw) + name(target_category),
ModelCommand::RemoveFormula {
target,
target_category,
} => name(target) + name(target_category),
ModelCommand::RemoveFormula { id } => formula(id),
ModelCommand::ReplaceFormula { id, was, raw } => formula(id) + name(was) + name(raw),
ModelCommand::CleanEmptyRecords | ModelCommand::SortData => 0,
ModelCommand::AddCategory(category)
+330 -55
View File
@@ -4,6 +4,7 @@
use serde::{Deserialize, Serialize};
use unicode_width::UnicodeWidthStr;
use improvise_core::model::category::CategoryKind;
use improvise_core::model::cell::CellValue;
use improvise_core::model::record::{AggregateHash, RecordHash};
use improvise_core::view::Axis;
@@ -56,9 +57,21 @@ pub enum CellId {
/// A records-mode creating column: typing a **name** into it adds that
/// measure or category and writes no cell at all (gotcha 1, improvise-poia).
Create(CreateKind),
/// A records-mode coordinate. Renaming it moves every measure cell in the
/// row (gotcha 1), which no client composes: `protocol.md` §3 (improvise-exdg).
Coordinate,
/// A records-mode coordinate: the row's record and the category this column
/// names. Renaming it moves the whole record — `protocol.md` §3 (improvise-fdk1).
Coordinate {
record: RecordHash,
category: String,
},
/// A stored cell whose `category` coordinate names no item of it, so no write
/// can address it: `context/code/command.md` (improvise-gk51).
NonItem { category: String, item: String },
}
/// Why a cell whose coordinate names no item takes no write. One wording for the
/// projection's decline and the host's refusal (improvise-gk51).
pub fn non_item_refusal(category: &str, item: &str) -> String {
format!("{item} is no item of {category}, so there is no one cell to write")
}
/// What a creating column creates. The name typed into it is the whole
@@ -198,6 +211,12 @@ pub const MAX_ROW_HEADER_W: u16 = 24;
/// because it is a fact about the *view*, which only the server holds.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct FrameGroup {
/// Which header level opened it — an index into [`AxisFrame::cats`]. Runs
/// nest, so membership is per level (improvise-e265).
pub level: usize,
/// The group's depth in its category's hierarchy, 0 for a root. Runs nest
/// by `(level, depth)`: `context/code/protocol.md` §3a (improvise-felo.6).
pub depth: usize,
pub name: String,
pub collapsed: bool,
}
@@ -207,17 +226,35 @@ pub struct FrameGroup {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum FrameEntry {
Group(FrameGroup),
/// The run opened by the last `Group` ends here — `AxisEntry::GroupEnd`
/// carried onto the wire (improvise-gd83). Nothing is drawn for it.
GroupEnd,
/// The run at `(level, depth)`, and every run inside it, ends here —
/// `AxisEntry::GroupEnd` on the wire (improvise-gd83, e265). Nothing is drawn.
GroupEnd {
level: usize,
depth: usize,
},
/// One label per header level, outermost first. Data indices — the
/// `row`/`col` of every other field on [`GridCache`] — count only these.
Item(Vec<String>),
Item {
tuple: Vec<String>,
/// The layout's own fold flag: `context/code/protocol.md` (improvise-4k1j).
collapsed: bool,
},
}
impl FrameEntry {
/// An ordinary, unfolded item, as `AxisEntry::item` is on the layout side.
pub fn item(tuple: Vec<String>) -> Self {
FrameEntry::Item {
tuple,
collapsed: false,
}
}
}
/// One axis of the grid's *frame*: header levels, label tuples and groups
/// (improvise-anr9). **One** type, on purpose — `context/code/protocol.md` §3.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "UncheckedAxisFrame")]
pub struct AxisFrame {
/// The categories on this axis, outermost first. Empty is a real layout
/// (one nameless row or column), not a gap — hence [`AxisFrame::levels`].
@@ -228,6 +265,42 @@ pub struct AxisFrame {
pub entries: Vec<FrameEntry>,
}
/// An [`AxisFrame`] as the wire delivers it, before its levels are checked
/// against its categories: `context/code/protocol.md` §3b.
#[derive(Deserialize)]
struct UncheckedAxisFrame {
cats: Vec<String>,
first: usize,
entries: Vec<FrameEntry>,
}
impl TryFrom<UncheckedAxisFrame> for AxisFrame {
type Error = String;
fn try_from(raw: UncheckedAxisFrame) -> Result<Self, String> {
let beyond = raw
.entries
.iter()
.filter_map(|e| match e {
FrameEntry::Group(g) => Some(g.level),
FrameEntry::GroupEnd { level, .. } => Some(*level),
FrameEntry::Item { .. } => None,
})
.find(|level| *level >= raw.cats.len());
match beyond {
Some(level) => Err(format!(
"group level {level} names no category: this axis has {}",
raw.cats.len()
)),
None => Ok(Self {
cats: raw.cats,
first: raw.first,
entries: raw.entries,
}),
}
}
}
impl AxisFrame {
/// A single-level axis with no groups. `cats` is left empty, so this is
/// not a way to smuggle a multi-level axis in flattened.
@@ -237,7 +310,7 @@ impl AxisFrame {
first: 0,
entries: labels
.into_iter()
.map(|l| FrameEntry::Item(vec![l.into()]))
.map(|l| FrameEntry::item(vec![l.into()]))
.collect(),
}
}
@@ -246,8 +319,8 @@ impl AxisFrame {
/// index counts items only, exactly as `GridLayout` does.
pub fn items(&self) -> impl Iterator<Item = &Vec<String>> {
self.entries.iter().filter_map(|e| match e {
FrameEntry::Item(v) => Some(v),
FrameEntry::Group(_) | FrameEntry::GroupEnd => None,
FrameEntry::Item { tuple: v, .. } => Some(v),
FrameEntry::Group(_) | FrameEntry::GroupEnd { .. } => None,
})
}
@@ -275,6 +348,19 @@ impl AxisFrame {
self.items().nth(idx)
}
/// Whether the item at `idx` is a fold or lies under one, at any level: the
/// layout's own flag, as `GridLayout::row_folds_a_group` reads it (4k1j).
pub fn folds(&self, idx: usize) -> bool {
self.entries
.iter()
.filter_map(|e| match e {
FrameEntry::Item { collapsed, .. } => Some(*collapsed),
FrameEntry::Group(_) | FrameEntry::GroupEnd { .. } => None,
})
.nth(idx)
.unwrap_or(false)
}
/// How many header levels this axis draws. **At least one**: an axis
/// with no categories still draws a single, empty header line.
pub fn levels(&self) -> usize {
@@ -336,34 +422,31 @@ impl AxisFrame {
.collect()
}
/// The group each data item sits under, in order — [`AxisFrame::group_of`]
/// for every position, in one pass rather than a scan apiece.
/// The innermost group each data item sits in, in order — one pass. A group
/// at `(level, depth)` closes deeper runs; so does a `GroupEnd` there (§3a).
pub fn groups(&self) -> Vec<Option<&FrameGroup>> {
let mut current: Option<&FrameGroup> = None;
let mut open: Vec<&FrameGroup> = Vec::new();
let close = |open: &mut Vec<&FrameGroup>, key: (usize, usize)| {
open.truncate(open.partition_point(|g| (g.level, g.depth) < key))
};
let mut out = Vec::new();
for entry in &self.entries {
match entry {
FrameEntry::Group(g) => current = Some(g),
FrameEntry::GroupEnd => current = None,
FrameEntry::Item(_) => out.push(current),
FrameEntry::Group(g) => {
close(&mut open, (g.level, g.depth));
open.push(g);
}
FrameEntry::GroupEnd { level, depth } => close(&mut open, (*level, *depth)),
FrameEntry::Item { .. } => out.push(open.last().copied()),
}
}
out
}
/// The item's group **membership** — the nearest preceding header, unless
/// a `GroupEnd` closed it first (improvise-gd83). Not for a draw loop; §3.
/// The item's innermost group — [`AxisFrame::groups`] at one position. It
/// walks the whole frame, so it is not for a draw loop; §3.
pub fn group_of(&self, idx: usize) -> Option<&FrameGroup> {
let visual = self.visual_of(idx)?;
self.entries[..visual]
.iter()
.rev()
.find_map(|e| match e {
FrameEntry::Group(g) => Some(Some(g)),
FrameEntry::GroupEnd => Some(None),
FrameEntry::Item(_) => None,
})
.flatten()
self.groups().get(idx).copied().flatten()
}
/// The runs of adjacent positions sharing a group — one home, both front
@@ -396,7 +479,7 @@ impl AxisFrame {
self.entries
.iter()
.enumerate()
.filter(|(_, e)| matches!(e, FrameEntry::Item(_)))
.filter(|(_, e)| matches!(e, FrameEntry::Item { .. }))
.map(|(vi, _)| vi)
.nth(idx)
}
@@ -793,19 +876,29 @@ impl WidgetCache for ViewListCache {
}
}
/// One tile in the tile bar. `regular` is the host's `CategoryKind` answer: a
/// client cannot tell `_Index` from a user category without sniffing names.
/// One tile in the tile bar. `kind` is the host's answer: a client cannot tell
/// `_Index` from a user category without sniffing names (improvise-bqm0).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TileEntry {
pub category: String,
pub axis: Axis,
pub regular: bool,
pub kind: CategoryKind,
}
/// Where tile select opens: the first regular category, so `T Enter` cannot
/// put `_Index` alone on Row (improvise-oplp). Virtuals stay reachable.
/// Where tile select opens: the best-ranked tile, the leftmost on a tie, so
/// `T Enter` never puts `_Index` alone on Row. `context/code/command.md`.
pub fn opening_tile(tiles: &[TileEntry]) -> usize {
tiles.iter().position(|tile| tile.regular).unwrap_or(0)
let rank = |kind: CategoryKind| match kind {
CategoryKind::Regular => 0,
CategoryKind::VirtualMeasure => 1,
CategoryKind::Label => 2,
CategoryKind::VirtualIndex | CategoryKind::VirtualDim => 3,
};
tiles
.iter()
.enumerate()
.min_by_key(|(i, tile)| (rank(tile.kind), *i))
.map_or(0, |(i, _)| i)
}
/// The tile bar's tileable categories and axes (improvise-pca). Its overlap
@@ -831,6 +924,8 @@ pub enum WizardStepKind {
SelectArrayPath,
ReviewProposals,
ConfigureDates,
ResolveDuplicates,
NameDuplicateCategory,
DefineFormulas,
NameModel,
Done,
@@ -868,6 +963,41 @@ pub struct WizardDateField {
pub components: Vec<(String, bool)>,
}
/// Several source records sharing every coordinate (improvise-cgqt). Data,
/// so every front end lists the same rows; [`Self::describe`] words it.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WizardCollision {
pub coords: Vec<(String, String)>,
/// 1-based positions of the colliding records in the source.
pub records: Vec<usize>,
}
impl WizardCollision {
/// One line naming the records and the tuple they share, never
/// abbreviated: a front end wraps it rather than cutting it.
pub fn describe(&self) -> String {
let records: Vec<String> = self.records.iter().map(|r| format!("#{r}")).collect();
let coords: Vec<String> = self
.coords
.iter()
.map(|(c, i)| format!("{c}={i}"))
.collect();
format!(
"{} records ({}): {}",
self.records.len(),
records.join(", "),
coords.join(", ")
)
}
}
/// One option on the duplicates step; `disabled` says why it cannot be taken.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct WizardChoice {
pub label: String,
pub disabled: Option<String>,
}
/// The import wizard as the widget draws it (improvise-764). **Deliberately
/// not a [`Projection`]** — that absence is the argument; see §3.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -887,6 +1017,14 @@ pub struct WizardCache {
pub proposals: Vec<WizardProposal>,
/// `ConfigureDates`: the accepted time fields and their components.
pub date_fields: Vec<WizardDateField>,
/// `ResolveDuplicates`: every colliding tuple, in first-seen order.
pub collisions: Vec<WizardCollision>,
/// `ResolveDuplicates`: sum, keep separate, go back — in cursor order.
pub duplicate_choices: Vec<WizardChoice>,
/// `NameDuplicateCategory`: the name being typed.
pub duplicate_category: String,
/// The first item drawn of the step's scrolling list.
pub scroll: usize,
/// `DefineFormulas`: the formulas defined so far.
pub formulas: Vec<String>,
/// `DefineFormulas`: suggestions derived from the accepted measures.
@@ -931,6 +1069,51 @@ pub struct Snapshot {
mod tests {
use super::*;
fn tile(category: &str, axis: Axis, kind: CategoryKind) -> TileEntry {
TileEntry {
category: category.to_string(),
axis,
kind,
}
}
/// Records mode puts `_Index` on Row and the first regular elsewhere, so
/// "first regular" and "first on Row" part (improvise-52od, protocol.md).
#[test]
fn tile_select_opens_on_a_regular_category_even_when_a_virtual_holds_row() {
let tiles = [
tile("_Index", Axis::Row, CategoryKind::VirtualIndex),
tile("_Dim", Axis::Column, CategoryKind::VirtualDim),
tile("_Measure", Axis::None, CategoryKind::VirtualMeasure),
tile("Region", Axis::Page, CategoryKind::Regular),
tile("Product", Axis::Row, CategoryKind::Regular),
];
assert_eq!(opening_tile(&tiles), 3, "Region, the first regular tile");
}
/// No regular category: open on `_Measure`, never on `_Index` or `_Dim`
/// (improvise-bqm0). Why this ranking: `context/code/command.md`.
#[test]
fn with_no_regular_category_tile_select_opens_on_measure_not_index() {
let tiles = [
tile("_Index", Axis::None, CategoryKind::VirtualIndex),
tile("_Dim", Axis::None, CategoryKind::VirtualDim),
tile("Note", Axis::None, CategoryKind::Label),
tile("_Measure", Axis::Column, CategoryKind::VirtualMeasure),
];
assert_eq!(
opening_tile(&tiles),
3,
"_Measure outranks a label and the records axes"
);
assert_eq!(
opening_tile(&tiles[..3]),
2,
"a label outranks _Index and _Dim"
);
assert_eq!(opening_tile(&[]), 0, "an empty bar opens at 0");
}
fn valued(value: CellValue) -> CellDisplay {
CellDisplay {
value: Some(value),
@@ -1168,13 +1351,15 @@ mod tests {
first: 0,
cats: vec!["Type".into(), "Who".into()],
entries: vec![
FrameEntry::Item(vec!["A".into(), "x".into()]),
FrameEntry::Item(vec!["A".into(), "y".into()]),
FrameEntry::item(vec!["A".into(), "x".into()]),
FrameEntry::item(vec!["A".into(), "y".into()]),
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Late".into(),
collapsed: true,
}),
FrameEntry::Item(vec!["B".into(), "z".into()]),
FrameEntry::item(vec!["B".into(), "z".into()]),
],
}
}
@@ -1183,6 +1368,8 @@ mod tests {
/// ungrouped between them — improvise-h0iv's reachable shape.
fn repeated_group_frame() -> AxisFrame {
let q1 = || FrameGroup {
level: 1,
depth: 0,
name: "Q1".into(),
collapsed: false,
};
@@ -1191,11 +1378,11 @@ mod tests {
cats: vec!["Region".into(), "Month".into()],
entries: vec![
FrameEntry::Group(q1()),
FrameEntry::Item(vec!["North".into(), "Jul".into()]),
FrameEntry::Item(vec!["North".into(), "Aug".into()]),
FrameEntry::item(vec!["North".into(), "Jul".into()]),
FrameEntry::item(vec!["North".into(), "Aug".into()]),
FrameEntry::Group(q1()),
FrameEntry::Item(vec!["South".into(), "Jul".into()]),
FrameEntry::Item(vec!["South".into(), "Aug".into()]),
FrameEntry::item(vec!["South".into(), "Jul".into()]),
FrameEntry::item(vec!["South".into(), "Aug".into()]),
],
}
}
@@ -1208,17 +1395,103 @@ mod tests {
cats: vec!["Month".into()],
entries: vec![
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Q1".into(),
collapsed: false,
}),
FrameEntry::Item(vec!["Jan".into()]),
FrameEntry::Item(vec!["Feb".into()]),
FrameEntry::GroupEnd,
FrameEntry::Item(vec!["Sep".into()]),
FrameEntry::item(vec!["Jan".into()]),
FrameEntry::item(vec!["Feb".into()]),
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Sep".into()]),
],
}
}
/// A level past the frame's categories is refused as it arrives, before
/// anything sizes a vector by it. Evidence file.
#[test]
fn a_frame_whose_group_level_names_no_category_does_not_deserialize() {
let group = |level: usize, name: &str| {
FrameEntry::Group(FrameGroup {
level,
depth: 0,
name: name.into(),
collapsed: true,
})
};
let frame = AxisFrame {
cats: vec!["Region".into(), "Month".into()],
first: 0,
entries: vec![
group(0, "Upper"),
group(1, "Q1"),
FrameEntry::GroupEnd { level: 1, depth: 0 },
],
};
let json = serde_json::to_string(&frame).unwrap();
let back: AxisFrame = serde_json::from_str(&json).expect("the control parses");
assert_eq!(back, frame);
for (from, to) in [
(
r#""level":1,"depth":0,"name":"Q1""#,
r#""level":4294967295,"depth":0,"name":"Q1""#,
),
(
r#"{"GroupEnd":{"level":1,"depth":0}}"#,
r#"{"GroupEnd":{"level":2,"depth":0}}"#,
),
] {
let bad = json.replacen(from, to, 1);
assert_ne!(bad, json, "the fault {to} was injected");
serde_json::from_str::<serde_json::Value>(&bad).expect("still well-formed JSON");
let err = serde_json::from_str::<AxisFrame>(&bad).unwrap_err();
assert!(
err.to_string().contains("names no category"),
"refused for its level, not its shape: {err}"
);
}
}
/// improvise-felo.6: runs nest by `(level, depth)` inside one category, so
/// the end of a quarter leaves its year open. See the evidence file.
#[test]
fn a_quarters_end_leaves_its_year_open_on_the_same_level() {
let group = |depth: usize, name: &str| {
FrameEntry::Group(FrameGroup {
level: 0,
depth,
name: name.into(),
collapsed: false,
})
};
let frame = AxisFrame {
cats: vec!["Date".into()],
first: 0,
entries: vec![
group(0, "2024"),
group(1, "Q1"),
FrameEntry::item(vec!["Jan".into()]),
FrameEntry::GroupEnd { level: 0, depth: 1 },
FrameEntry::item(vec!["Direct".into()]),
group(1, "Q2"),
FrameEntry::item(vec!["Apr".into()]),
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Undated".into()]),
],
};
let names: Vec<Option<&str>> = frame
.groups()
.into_iter()
.map(|g| g.map(|g| g.name.as_str()))
.collect();
assert_eq!(
names,
vec![Some("Q1"), Some("2024"), Some("Q2"), None],
"Direct sits in 2024 after Q1 ends; Undated in nothing"
);
}
/// improvise-gd83's property, asserted in the crate that owns the type.
/// See `context/test-evidence/protocol.md`.
#[test]
@@ -1245,11 +1518,13 @@ mod tests {
cats: vec!["Month".into()],
entries: vec![
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Q1".into(),
collapsed: true,
}),
FrameEntry::GroupEnd,
FrameEntry::Item(vec!["Sep".into()]),
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Sep".into()]),
],
};
assert_eq!(frame.len(), 1, "the collapsed group contributes no item");
@@ -1399,7 +1674,7 @@ mod tests {
fn body_from_starts_at_the_item_not_at_the_header_above_it() {
let frame = two_level_frame();
assert!(
matches!(frame.body_from(2), [FrameEntry::Item(_)]),
matches!(frame.body_from(2), [FrameEntry::Item { .. }]),
"the trailing group header is above item 2, not inside the body"
);
assert_eq!(frame.body_from(0).len(), 4, "everything, headers included");
@@ -1412,7 +1687,7 @@ mod tests {
let frame = AxisFrame {
first: 0,
cats: vec!["A".into(), "B".into()],
entries: vec![FrameEntry::Item(vec!["Clothing".into(), "日本".into()])],
entries: vec![FrameEntry::item(vec!["Clothing".into(), "日本".into()])],
};
assert_eq!(
frame.header_widths(),
@@ -1450,7 +1725,7 @@ mod tests {
let frame = AxisFrame {
first: 0,
cats: Vec::new(),
entries: vec![FrameEntry::Item(Vec::new())],
entries: vec![FrameEntry::item(Vec::new())],
};
assert_eq!(frame.levels(), 1);
assert_eq!(frame.label(0, 0), "");
@@ -1533,7 +1808,7 @@ mod tests {
let tiles = TileBarCache::folded(&[vec![TileEntry {
category: "Region".into(),
axis: Axis::Page,
regular: true,
kind: crate::CategoryKind::Regular,
}]]);
assert_eq!(tiles.tiles[0].axis, Axis::Page);
}
@@ -1583,7 +1858,7 @@ mod tests {
cats: vec![cat.into()],
entries: labels
.iter()
.map(|l| FrameEntry::Item(vec![(*l).into()]))
.map(|l| FrameEntry::item(vec![(*l).into()]))
.collect(),
};
GridCache::folded(&[
@@ -1931,7 +2206,7 @@ mod tests {
Projection::TileBar(vec![TileEntry {
category: "Region".into(),
axis: Axis::Page,
regular: true,
kind: crate::CategoryKind::Regular,
}]),
];
for projection in &projections {
@@ -1968,7 +2243,7 @@ mod tests {
Projection::TileBar(vec![TileEntry {
category: "Region".into(),
axis: Axis::Page,
regular: true,
kind: crate::CategoryKind::Regular,
}]),
],
};
File diff suppressed because it is too large Load Diff
+3
View File
@@ -40,6 +40,9 @@ wasm-bindgen = "0.2"
[dev-dependencies]
improvise-command = { path = "../improvise-command" }
improvise-core = { path = "../improvise-core" }
# The tests build `FormulaEntry` from `&str`, which only `test-support` provides.
# Asked for here, or `-p improvise-wasm-client` alone does not compile (a61f).
improvise-protocol = { path = "../improvise-protocol", features = ["test-support"] }
improvise-ws-server = { path = "../improvise-ws-server" }
# The `wasm-opt` wasm-pack downloads predates bulk-memory being enabled by
+7 -6
View File
@@ -201,7 +201,8 @@ impl Client {
CellId::Aggregate(_)
| CellId::Derived { .. }
| CellId::Create(_)
| CellId::Coordinate => None,
| CellId::Coordinate { .. }
| CellId::NonItem { .. } => None,
}
}
@@ -377,26 +378,26 @@ mod tests {
let rendered: serde_json::Value =
serde_json::from_str(&client.state_json().unwrap()).unwrap();
assert_eq!(
rendered["grid"]["row_frame"]["entries"][0]["Item"][0],
rendered["grid"]["row_frame"]["entries"][0]["Item"]["tuple"][0],
"North"
);
assert_eq!(rendered["view"]["buffers"]["category"], "R");
assert_eq!(rendered["revision"], client.revision());
}
/// `Enter` rather than an arrow: since improvise-bpq2 the arrows resolve
/// here, and `enter-advance` is one of the movements left to the host.
/// Ctrl-Q, `force-quit`: the arrows (bpq2) and `Enter` (fo0w) resolve here
/// now, and what Ctrl-Q means in a browser tab is still a ruling.
#[test]
fn an_unresolvable_key_is_forwarded_and_changes_nothing_locally() {
let mut client = init();
let before = client.revision();
let outbound = client
.on_key_event("Enter", false, false, false, false)
.on_key_event("q", true, false, false, false)
.unwrap()
.unwrap();
assert!(
outbound.contains("Key"),
"a Normal-mode Enter is the host's to resolve: {outbound}"
"a Normal-mode Ctrl-Q is the host's to resolve: {outbound}"
);
assert_eq!(
client.revision(),
+92 -22
View File
@@ -2,7 +2,7 @@
//! already made. The four silent ones are in `context/code/clients.md`.
use improvise_protocol::viewmodel::{CatTreeEntry, flatten_cat_tree};
use improvise_protocol::{AxisFrame, CellDisplay, CellId, FrameEntry};
use improvise_protocol::{AxisFrame, CellDisplay, CellId, FrameEntry, resolve};
use serde::Serialize;
use crate::{Client, ClientState};
@@ -16,6 +16,9 @@ pub struct RenderModel {
/// The mode's own label, off `AppMode` — the client does not spell it.
pub mode: String,
pub status: String,
/// Whether `status` belongs to the editor still open, so the page must not
/// expire it on a timer (improvise-1x9h).
pub status_stands: bool,
/// The document's active view, as this client knows it (gotcha 29).
pub view_name: String,
/// The title bar: the document's name, whether it has unsaved changes, and
@@ -147,7 +150,8 @@ impl RenderCell {
CellId::Aggregate(_)
| CellId::Derived { .. }
| CellId::Create(_)
| CellId::Coordinate,
| CellId::Coordinate { .. }
| CellId::NonItem { .. },
)
| None => false,
},
@@ -238,6 +242,7 @@ pub fn render_model(state: &ClientState) -> RenderModel {
revision: state.revision,
mode: improvise_protocol::mode_label(&state.view.mode).to_string(),
status: state.view.status_msg.clone(),
status_stands: state.status_stands(),
view_name: state.view.active_view.clone(),
doc_name: state.document.name.clone(),
doc_dirty: state.document.dirty,
@@ -252,12 +257,12 @@ pub fn render_model(state: &ClientState) -> RenderModel {
// `GroupEnd` is a boundary, not a row: it draws nothing and
// consumes no data index (improvise-gd83).
let row = match entry {
FrameEntry::GroupEnd => return Some(None),
FrameEntry::GroupEnd { .. } => return Some(None),
FrameEntry::Group(group) => RenderRow::Group {
label: group.name.clone(),
collapsed: group.collapsed,
},
FrameEntry::Item(_) => {
FrameEntry::Item { .. } => {
let r = *data_idx;
*data_idx += 1;
RenderRow::Data {
@@ -423,22 +428,37 @@ impl RenderHelpRow {
L::Heading(t) => prose("heading", t),
L::Text(t) => prose("text", t),
L::Accent(t) => prose("accent", t),
L::Keys(row) => Self {
keys: row.keys().join(" "),
description: row.description().to_string(),
kind: "keys",
},
L::Keys(row) => Self::row(
row.keys().join(" "),
row.description(),
!resolve::needs_the_host(row.command()),
false,
),
L::Literal {
keys,
description,
dim,
} => Self {
keys: (*keys).to_string(),
description: description.to_string(),
kind: match dim {
true => "dim",
false => "keys",
},
} => Self::row(
(*keys).to_string(),
description,
resolve::runs_the_command_line(),
*dim,
),
}
}
/// A keys row. One this client cannot run is drawn dim and says so, since a
/// dead key followed by live ones edits the grid (improvise-uln5, koh6).
fn row(keys: String, description: &str, runs: bool, dim: bool) -> Self {
Self {
keys,
description: match runs {
true => description.to_string(),
false => resolve::not_here(description),
},
kind: match dim || !runs {
true => "dim",
false => "keys",
},
}
}
@@ -570,13 +590,60 @@ mod tests {
/// one fixture answers every editability question.
fn group(name: &str, collapsed: bool) -> FrameEntry {
FrameEntry::Group(improvise_protocol::FrameGroup {
level: 0,
depth: 0,
name: name.into(),
collapsed,
})
}
fn item(label: &str) -> FrameEntry {
FrameEntry::Item(vec![label.into()])
FrameEntry::item(vec![label.into()])
}
fn help_page(page: usize) -> Vec<RenderHelpRow> {
let view = ViewState {
mode: improvise_protocol::AppMode::Help,
help_page: page,
..ViewState::default()
};
help_rows(&view)
}
/// improvise-uln5: a row whose key this client sends to a host that answers
/// nothing is drawn marked. Evidence: `context/test-evidence/clients.md`.
#[test]
fn a_help_row_the_browser_cannot_run_is_drawn_marked() {
let rows = help_page(1);
let row = |prefix: &str| {
let found = rows.iter().find(|r| r.description.starts_with(prefix));
found.unwrap_or_else(|| panic!("no Navigation row reads {prefix:?}"))
};
let search = row("Exit search mode");
assert_eq!(search.kind, "dim", "{search:?}");
assert!(
search.description.contains("not in the browser"),
"{search:?}"
);
let control = row("Jump to last row");
assert_eq!(
control.kind, "keys",
"a row the browser runs is drawn plainly"
);
assert_eq!(control.description, "Jump to last row");
let commands = help_page(4);
let ex: Vec<_> = commands
.iter()
.filter(|r| r.keys.trim_start().starts_with(':'))
.collect();
assert!(
!ex.is_empty(),
"precondition: the Commands page lists ex-commands"
);
for row in ex {
assert_eq!(row.kind, "dim", "{row:?}");
assert!(row.description.contains("not in the browser"), "{row:?}");
}
}
/// A run is opened by a HEADER, not by a name — improvise-h0iv, and both
@@ -634,7 +701,10 @@ mod tests {
((1, 0), CellDisplay::text("pending")),
(
(1, 2),
CellDisplay::text("North").with_id(CellId::Coordinate),
CellDisplay::text("North").with_id(CellId::Coordinate {
record: surrogate(),
category: "Region".into(),
}),
),
])),
],
@@ -798,12 +868,12 @@ mod tests {
TileEntry {
category: "Region".into(),
axis: Axis::Row,
regular: true,
kind: improvise_protocol::CategoryKind::Regular,
},
TileEntry {
category: "Product".into(),
axis: Axis::None,
regular: true,
kind: improvise_protocol::CategoryKind::Regular,
},
])));
client
@@ -1123,12 +1193,12 @@ mod tests {
TileEntry {
category: "Quarter".into(),
axis: Axis::Page,
regular: true,
kind: improvise_protocol::CategoryKind::Regular,
},
TileEntry {
category: "Region".into(),
axis: Axis::Column,
regular: true,
kind: improvise_protocol::CategoryKind::Regular,
},
])));
let tiles = client.model().tiles;
+103 -3
View File
@@ -14,6 +14,8 @@ use serde::{Deserialize, Serialize};
pub struct EditTarget {
/// The document the editor opened on; a swap under it refuses the commit (improvise-vndi).
pub document_generation: u64,
/// The view that drew the cell: an empty record resolves only there (improvise-wpxr).
pub view: String,
pub cell: RecordCell,
/// The record's write counter as this client had it at open.
pub version: u64,
@@ -95,7 +97,7 @@ impl Client {
// The sentence goes on *after* the rollback: it is this
// client's own notification, and the slice it restores has none.
state.refused(rejection);
state.view.status_msg = message;
state.notify(message);
}
ServerFrame::Accepted => self.state_mut().accepted(),
}
@@ -125,6 +127,7 @@ impl Client {
let seed = self.state().grid.cell(row, col)?.edit_text.clone();
Some(EditTarget {
document_generation,
view: self.state().view.active_view.clone(),
cell,
version,
opened: OpenedEditor::on(seed.unwrap_or_default()),
@@ -143,7 +146,11 @@ impl Client {
)?;
let cmds = vec![Command::Model {
basis,
cmd: typed_cell_command(&improvise_protocol::CellId::Record(target.cell), text)?,
cmd: typed_cell_command(
&target.view,
&improvise_protocol::CellId::Record(target.cell),
text,
)?,
}];
// Folds nothing — a `Command::Model` is not the view slice's business —
// but it **registers**, or the host's answers address the wrong frame.
@@ -195,6 +202,26 @@ mod tests {
);
}
/// A rejection's sentence is written outside a fold, so it must still be owned
/// by the editor it arrived in (improvise-1x9h). Evidence file.
#[test]
fn a_rejection_that_arrives_in_an_open_editor_stands_beside_it() {
let mut client = Client::default();
client
.state_mut()
.fold(&Command::ChangeMode(improvise_protocol::AppMode::editing()));
client.fold_frame(ServerFrame::Rejected(Rejection::NoDocumentPath));
assert!(
client.state().view.mode.is_editing(),
"a refused save closes nothing"
);
assert!(!client.state().view.status_msg.is_empty());
assert!(
client.state().status_stands(),
"the refusal was not observed where it arrived, so it would expire"
);
}
/// A rejection that refused an **edit** must say nothing was applied; the
/// two `Save` answers refused none. Evidence: clients.md (improvise-8uj8).
#[test]
@@ -219,6 +246,50 @@ mod tests {
}
}
/// An awaited frame that also says `status`, as a commit's frame does.
fn awaited_saying(status: &str) -> Vec<Command> {
let mut cache = improvise_protocol::BasisCache::default();
let basis = improvise_protocol::basis::published(0, 0, Default::default());
improvise_protocol::WidgetCache::apply(&mut cache, &basis);
vec![
Command::Model {
basis: cache.claim([]).expect("seeded"),
cmd: improvise_protocol::ModelCommand::CleanEmptyRecords,
},
Command::SetStatus(status.into()),
]
}
/// BUG (improvise-n0pb) — see `context/test-evidence/clients.md`.
#[test]
fn a_relay_heard_after_a_refusal_keeps_the_refusals_sentence() {
let mut client = Client::default();
for status in ["first", "second"] {
client.state_mut().fold_outgoing(&awaited_saying(status));
}
client.fold_frame(ServerFrame::Rejected(Rejection::LayoutMoved));
let sentence = rejection_message(&Rejection::LayoutMoved);
assert_eq!(
client.state().view.status_msg,
sentence,
"setup: the refusal says so"
);
client.fold_frame(ServerFrame::Commands(vec![Command::ResetCursor]));
assert_eq!(
client.state().view.status_msg,
sentence,
"a relay behind the second frame erased the refusal's sentence"
);
client.fold_frame(ServerFrame::Accepted);
assert_eq!(
client.state().view.status_msg,
sentence,
"and settling keeps it"
);
assert_eq!(client.state().settled.status_msg, sentence);
}
/// The rejection belongs to whoever composed the refused batch, so it does
/// not survive a document swap: it names a record that is gone.
#[test]
@@ -367,6 +438,30 @@ mod tests {
);
}
/// improvise-wpxr: a mouse commit names the view the editor OPENED on, as it
/// claims the version read then. Evidence: test-evidence/clients.md.
#[test]
fn a_mouse_commit_names_the_view_its_editor_opened_on() {
let mut client = client_showing(Some(CellId::Record(record("Revenue"))));
client.state_mut().view.active_view = "Opened".into();
let target = client.edit_target_at(0, 0).expect("a record cell");
client.state_mut().view.active_view = "Later".into();
let Some(ClientFrame::Commands(cmds)) = client.typed_target_frame(target, "7") else {
panic!("a changed value composes a frame");
};
let [
Command::Model {
cmd: ModelCommand::SetCell { view, .. },
..
},
] = cmds.as_slice()
else {
panic!("one SetCell: {cmds:?}");
};
assert_eq!(view, "Opened");
}
fn client_holding(mut cell: CellDisplay) -> Client {
cell.id = Some(CellId::Record(record("Revenue")));
let mut client = init();
@@ -402,6 +497,7 @@ mod tests {
Some(ClientFrame::command(Command::Model {
basis: published_basis(),
cmd: ModelCommand::SetCell {
view: "Default".into(),
cell: record("Revenue"),
value: CellValue::Number(42.5),
},
@@ -412,6 +508,7 @@ mod tests {
Some(ClientFrame::command(Command::Model {
basis: published_basis(),
cmd: ModelCommand::SetCell {
view: "Default".into(),
cell: record("Revenue"),
value: CellValue::Text("pending".into()),
},
@@ -421,7 +518,10 @@ mod tests {
client.typed_cell_frame(0, 0, ""),
Some(ClientFrame::command(Command::Model {
basis: published_basis(),
cmd: ModelCommand::ClearCell(record("Revenue")),
cmd: ModelCommand::ClearCell {
view: "Default".into(),
cell: record("Revenue"),
},
})),
"an empty string clears the cell; it is not a Text write"
);
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -333,6 +333,7 @@ mod tests {
let cmd = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell: addressed(&grid, 0, 0),
value: CellValue::Number(4242.0),
},
@@ -399,6 +400,7 @@ mod tests {
let cmd = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(42.0),
},
@@ -436,6 +438,7 @@ mod tests {
let cmd = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(42.0),
},
@@ -466,6 +469,7 @@ mod tests {
let cmd = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(42.0),
},
@@ -489,6 +493,7 @@ mod tests {
let wire = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(99.0),
},
@@ -535,6 +540,7 @@ mod tests {
let stale = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell: cell.clone(),
value: CellValue::Number(1.0),
},
@@ -543,6 +549,7 @@ mod tests {
let other = reduce::caught_up(
worker.app(),
ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(2.0),
},
@@ -167,6 +167,7 @@ fn the_worker_loads_projects_edits_and_persists_over_post_message() {
let cell = main.cell(2, 0);
main.post(&main.caught_up(ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(31.0),
}));
@@ -205,6 +206,7 @@ fn a_reload_over_the_same_origin_reopens_what_was_saved() {
main.post(&ClientFrame::Viewport { rows: 10, cols: 4 });
let cell = main.cell(0, 0);
main.post(&main.caught_up(ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(77.0),
}));
@@ -242,10 +244,12 @@ fn a_batch_composed_against_a_moved_document_is_refused() {
let cell = main.cell(0, 0);
let stale = main.caught_up(ModelCommand::SetCell {
view: "Default".into(),
cell: cell.clone(),
value: CellValue::Number(1.0),
});
let winner = main.caught_up(ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(2.0),
});
@@ -359,6 +363,7 @@ fn the_shipped_demo_seeds_a_worker_whose_raw_cells_are_settable() {
// formula beside it, through the projection stream.
let revenue = main.cell(0, 0);
main.post(&main.caught_up(ModelCommand::SetCell {
view: "Plan".into(),
cell: revenue,
value: CellValue::Number(20000.0),
}));
@@ -379,6 +384,7 @@ fn the_automatic_persist_tells_the_client_the_document_is_clean() {
main.dirty_marks.clear();
let cell = main.cell(0, 0);
main.post(&main.caught_up(ModelCommand::SetCell {
view: "Default".into(),
cell,
value: CellValue::Number(31.0),
}));
@@ -293,6 +293,7 @@ mod tests {
fn set_cell(row: &str, value: f64) -> ModelCommand {
ModelCommand::SetCell {
view: "Default".into(),
cell: addressed(row),
value: CellValue::Number(value),
}
@@ -111,6 +111,7 @@ fn a_client_on_a_socket_is_seeded_and_gets_its_edit_back() {
.claim([target.record])
.expect("the bootstrap published a basis"),
cmd: ModelCommand::SetCell {
view: "Default".into(),
cell: target,
value: CellValue::Number(12.0),
},
+23 -5
View File
@@ -43,7 +43,7 @@
# behind the runners' TLS-auditing proxy those downloads fail (binaryen
# with no fallback). On PATH, it downloads nothing. wasm-pack only takes
# a PATH wasm-bindgen whose version equals Cargo.lock's `wasm-bindgen`,
# so this pin moves with the lock; CI checks that it does.
# so this pin moves with the lock: on a mismatch the sandboxed build fails.
wasmBindgenCli = pkgs.buildWasmBindgenCli rec {
src = pkgs.fetchCrate {
pname = "wasm-bindgen-cli";
@@ -73,19 +73,29 @@
# `web/build.sh` — and so the Pages job's local dry run — needs this.
# The `wasm` CI job only runs `cargo check`, which does not, so the
# shell was missing the one tool that turns a passing check into a
# loadable page (improvise-d31). Must match WASM_PACK_VERSION in
# scripts/web-toolchain-versions.env; CI checks that it does.
# loadable page (improvise-d31). CI takes whatever version this is.
pkgs.wasm-pack
wasmBindgenCli
pkgs.binaryen
# NODE_VERSION's major, same check. The js and browser jobs ran 22
# while `build` ran the pinned 24; one shell means one node.
# The js and browser jobs once ran 22 while `build` ran 24; one shell
# means one node.
pkgs.nodejs_24
pkgs.python3
pkgs.jq
];
# scripts/repo-stats runs code-maat (improvise-1xfb.5.1). The JRE is about
# 0.6 GB, so it is kept out of ciInputs; only `nix-build` takes it.
codeMaatJar = pkgs.fetchurl {
url = "https://github.com/adamtornhill/code-maat/releases/download/v1.0.4/code-maat-1.0.4-standalone.jar";
hash = "sha256-QoeuIDSQGERFD3aVR7xEl6DaGm0cf6b63IWHBeZ0O18=";
};
codeMaat = pkgs.writeShellScriptBin "code-maat" ''
exec ${pkgs.jre_headless}/bin/java -jar ${codeMaatJar} "$@"
'';
# Playwright's downloaded browsers are generic-Linux builds that cannot
# start on NixOS, so nixpkgs' build is used. harness/package.json pins
# @playwright/test to this playwright-driver's version: they must match.
@@ -115,6 +125,7 @@
pkgs.global
pkgs.universal-ctags
beads.packages.${system}.default
codeMaat
# Demo recording and release tooling
pkgs.asciinema
@@ -130,6 +141,13 @@
// {
improvise = cargoNix.workspaceMembers.improvise.build;
default = self.packages.${system}.improvise;
code-maat = codeMaat;
# What the repo-stats pipeline runs besides the ci shell: one
# installable, because check-ci-graph reads `nix shell <one> --command`.
repo-stats-tools = pkgs.symlinkJoin {
name = "repo-stats-tools";
paths = [self.packages.${system}.improvise codeMaat];
};
};
# `nix flake check` builds every CI build: crane's checks plus the
+2
View File
@@ -11,6 +11,8 @@ _Measure: page, Revenue
_Index: none
_Dim: none
collapsed: Region/Upland
expanded(1): Month
expanded(1): Region
format: ,.0
## Category: Region
+2
View File
@@ -8,6 +8,7 @@ _Dim: none
_Measure: page
Region: row
format: ,.0
expanded(1): Region
## View: Tall
_Index: none
@@ -15,6 +16,7 @@ _Dim: none
_Measure: column
Region: row
format: ,.0
expanded(1): Region
## Category: _Measure
- Revenue, Cost
+34
View File
@@ -182,4 +182,38 @@ test.describe("the app bar is reserved, not fitted", () => {
const toast = await page.locator("#toast").boundingBox();
expect(toast.y + toast.height).toBeLessThanOrEqual(alone.y);
});
// improvise-1x9h. Whether a message stands is Rust's answer; this is the
// wire from the render model to the toast's timer, which node cannot see.
test("a message written beside an open minibuffer outlives the timer", async ({ page }) => {
await page.addInitScript(() => {
const Real = window.Worker;
window.Worker = class extends Real {
constructor(...args) {
super(...args);
window.__worker = this;
}
};
});
await page.clock.install();
await theDemo(page);
await page.keyboard.press("F");
await page.keyboard.press("n");
await expect(page.locator("#mode")).toHaveText("FORMULA");
// A host message arriving during text entry has no keyboard route in the
// demo, so the frame is delivered as the worker would deliver it.
await page.evaluate(() => {
const frame = JSON.stringify({ Commands: [{ SetStatus: "Formula error" }] });
window.__worker.dispatchEvent(new MessageEvent("message", { data: frame }));
});
await expect(page.locator("#toast")).toContainText("Formula error");
await page.clock.runFor(10_000);
await expect(page.locator("#toast")).toBeVisible();
await page.keyboard.press("Escape");
await expect(page.locator("#mode")).toHaveText("FORMULAS");
await expect(page.locator("#toast")).toBeHidden();
});
});
+3 -3
View File
@@ -132,9 +132,9 @@ test.describe("the shell reports what its peer is doing", { tag: "@mocked-peer"
await expect(status(page)).toContainText("connected", { timeout: 15_000 });
await expect(status(page)).toContainText("disconnected", { timeout: 15_000 });
// Enter is forwarded UPSTREAM, so it composes a frame even on a client
// that never got a snapshot — a cursor key here composes nothing at all.
await page.keyboard.press("Enter");
// Ctrl-Q is forwarded UPSTREAM, so it composes a frame with no snapshot;
// Enter resolves locally since improvise-fo0w, and composes nothing here.
await page.keyboard.press("Control+q");
await expect(status(page)).toContainText("not sent", { timeout: 15_000 });
const log = await page.evaluate(() => window.__statusLog);
expect(
+42 -7
View File
@@ -138,6 +138,12 @@ invariant(a_reported_share_cannot_overflow_its_scale,'gp_tree::fit::importance_r
invariant(viewport_extent_is_external_on_both_front_ends,'The viewport extent is an EXTERNAL INPUT that reaches the document, and it is not a browser problem. run_headless_commands reads crossterm::terminal::size() into app.term_{width,height} and CmdContext::{visible_rows,visible_cols} derive from it: 80x24 gives visible_rows 16, 200x60 gives 52. Measured on a 40-row grid, the SAME command sequence -- one PageDown then clear-cell at the cursor -- deleted R12 at 80x24 and R39 at 200x60. So Client::set_viewport is not a browser-introduced input; it is the browser SPELLING of one the TUI has always had, and gotcha 47 \"not derivable\" is true of both front ends. Any ADR treating extent as a browser problem produces a log that replays in a browser and not in a terminal. Cheapest of the five external inputs to close -- one pair of integers with an existing wire spelling, ClientFrame::Viewport -- but it must land in the SAME CHANGE as improvise-6uva, which currently masks it. A test must drive TWO sizes; one cannot see this.').
invariant(aggregate_hash_is_layout_scoped,'A RecordHash falls back to DataStore maintained index and so resolves against the DOCUMENT, which is why a connected client SetCell works from any view. An AggregateHash has no such index AND CANNOT HAVE ONE -- an aggregate has no existence outside the layout that derives it. So App::resolve_aggregate answers only for the HOST active view, which since improvise-jytm is not the client. Guarded by an_aggregate_a_session_was_shown_does_not_resolve_against_the_hosts_view, whose CONTROL is the load-bearing half: the same surrogate resolves once the host is moved onto that view, so a blanket None -- a typo in the hash, an empty layout -- cannot pass.').
invariant(fmt_gate_exit_two_means_the_gate_did_not_run,'scripts/fmt-workspaces.sh: exit 1 NAMES A FILE and blames the tree; exit 2 names none and blames the invocation. Every exit-2 cause is checked ABOVE the formatter loop, so the two cannot be confused. The contract is deliberately \"did the gate run\" rather than \"what kind of failure was it\": telling a rustfmt diff from a parse error means PARSING RUSTFMT PROSE, which is the check-adjacent-to-its-subject shape this repo keeps losing to. Measured while narrowing it: an unparseable source exits 1, identical to a diff, and an unknown rustfmt.toml option is only a warning at 0. Caught by ChatGPT Codex, which found the script committing its own ticket defect via a blanket \`|| rc=1\`.').
invariant(a_frame_group_level_names_a_category,'An AxisFrame deserialises through UncheckedAxisFrame and REFUSES a FrameGroup or GroupEnd whose level is >= cats.len(). A level naming no category is a malformed frame from the wire, never something to clamp, so the check is at the one door every frame arrives through. context/code/protocol.md section 3b.').
invariant(a_grid_evaluates_only_through_its_own_evaluator,'GridLayout::evaluator builds a GridEval whose fold memo that layout built from its own none_cats and hidden items, and a grid evaluates only through it, so no other grid memo can reach its cells or totals (improvise-opbo, where a memo built by another layout silently lent its aggregation and hidden set). prune_empty shares one GridEval across its whole emptiness test rather than a fresh memo per cell (improvise-metf).').
invariant(an_unprojected_cell_is_unknown,'Every client read of the grid cache goes through projected_at, which subtracts the frame origin (AxisFrame::first) before indexing. A position outside the window the cache holds is UNKNOWN, never the value at the same cache index, and a cell op on it waits. Reading the cache through offsets the client had scrolled ahead to made a pipelined edit write a different cell than the TUI would (improvise-7mro).').
invariant(default_duplicate_policy_is_refuse,'ImportPipeline::on_duplicate defaults to DuplicatePolicy::Refuse, so any path that never asked the question, the headless CLI included (--on-duplicate defaults to refuse), errors naming the collisions rather than summing. On ResolveDuplicates, Refuse blocks advance with a message; the build under Refuse returns an error when collisions exist (improvise-cgqt).').
invariant(a_collision_is_a_cell_written_twice,'A collision is a cell -- a coordinate tuple AND a measure -- written by two or more records, never merely a shared tuple: records with disjoint measures on one tuple write different cells and collide with nothing, and only the records that share a cell are listed and numbered (improvise-9bit). Plan::place is the one placement pass, used by both collisions() and build_model_with_warnings, so the question asked and the model built cannot disagree.').
invariant(wizard_draws_are_clipped_to_the_panel,'Every import-wizard write goes through Panel in src/ui/import_wizard_ui.rs, which clips to the panel rows and width, so no step can draw past the buffer; text is measured in cells by ui::text::truncate and wrap, never in bytes (review B1/B2 and a Codex P2 on PR #304).').
doctrine(bug_first,'write a failing test demonstrating the bug before fixing it').
doctrine(unit_over_e2e,'prefer small unit tests over integration/e2e').
doctrine(suite_speed,'Two numbers, because the ambiguity between them is what let this drift: NO SINGLE TEST BINARY may exceed 2 seconds, and the WORKSPACE WALL CLOCK is the sum of them. Measured at 22d43b2: 1309 tests over 33 binaries, ~2.2s wall, longest single binary 1.42s -- so the workspace is already past 2s and the rule that bites is the per-binary one. Re-MEASURE rather than re-assert: cargo test --workspace prints a \"finished in\" per binary. Excludes compilation, and excludes harness/ (Playwright, minutes, its own CI job).').
@@ -219,7 +225,6 @@ doctrine(a_destructive_default_warns_after_the_write,'bd update --notes REPLACES
doctrine(parallel_actors_need_distinct_names,'bd\'s --actor defaults to $BEADS_ACTOR, then git user.name, then $USER, so a wave of agents sharing a machine shares an actor and the claim guards cannot tell them apart -- two agents claim one bead and neither is told. Exit 13 means another actor won the race; that is the mechanism working, not a failure to retry around. User instruction, 2026-09-20.').
doctrine(a_stale_gotcha_asserts_the_negation,'repo-map gotcha 17 said float equality was \"currently mixed\" and named the div-by-zero guard as the exact-comparison half, long after both had been unified on FLOAT_EQ_EPSILON. A doc that goes stale does not merely omit -- it tells the reader not to assume the thing that is now true. This is why the orchestrator doc pass VERIFIES each claim against the tree rather than applying a queued edit.').
doctrine(an_advisory_gate_is_not_a_gate,'scripts/check-comment-runs.py was written, fixed seven times, and quoted by seven agents as their definition of done while being invoked by NO workflow. A rule enforced only by the agents who agreed to it is a convention. The gate landed last, deliberately, because a gate that goes green before the work is done is the same class of lie the checker itself carried.').
doctrine(a_review_names_the_sha_it_reviewed,'A review of a superseded commit is worse than no review: it reads as a pass. So the reviewer names the exact SHA, the author does not rewrite history while a review is in flight and announces any push, and a re-review is of the delta. This matters more now that adversarial agents are the ONLY gate -- Copilot\'s quota is exhausted for the rest of 2026-09 and its request is a SILENT no-op (the mutation returns success and registers no reviewer), which is itself an instance of doctrine(an_advisory_gate_is_not_a_gate).').
doctrine(review_is_requested_not_inherited,'No PR is reviewed because it exists. Requesting review is a step in the author\'s own definition of done, ranked beside \"tests pass\", not a courtesy notification afterwards. The failure mode it prevents is the quiet one: an agent finishes, opens a PR, reports success and waits on a review that was never going to come -- indistinguishable, from the outside, from a PR waiting its turn. Note the shape repeats: an ABSENT gate and a SILENTLY no-op request both read as \"in progress\", which is an_advisory_gate_is_not_a_gate seen from the requester\'s side. The user had turned Copilot AUTO-review off, which is the actual explanation for PRs #37/#41/#42 merging unreviewed; the orchestrator had attributed that to quota, and the two are independent facts.').
doctrine(construction_is_not_loading,'A lazily-loading embedded player proves nothing by existing. asciinema-player with autoPlay:false does not FETCH its cast until the viewer clicks, so a page whose every cast path 404s constructs the full set of players and throws no page error. The agent\'s own check passed on a page broken exactly that way (./casts/ -> ./cats/); pressing play returned 0 rendered lines and four 404s. Generalises: any check on a lazily-initialised embed must trigger the load it is about.').
doctrine(a_deployed_path_is_a_grep_across_the_tests,'Moving where something is served is not a build-script change, it is a grep across the TEST surface. The landing-page inversion (user ruling on improvise-kh8) cost one line in web/assemble.sh and six navigations in harness/tests/shell.spec.js, a CI-gating job the agent did not own. Nothing about \"serve the landing page at the root\" says \"a browser test job goes red\". In this repo the harness is the surface LEAST likely to be in the mental model, and for a structural reason: it is not a workspace member and cargo test never touches it, so every habit the repo has built is blind to it. Cross-reference: the harness serves the DEPLOYED arrangement on purpose (improvise-vvl3), which is what makes it trustworthy and also what makes a layout change a test change.').
@@ -250,7 +255,6 @@ doctrine(a_message_assertion_is_a_uniqueness_claim_about_the_whole_vocabulary,'A
doctrine(run_the_gate_ci_runs,'A local gate differing from CI in its flags or its output filter can only fail in directions the defect is not. On PR #71 the author ran clippy WITHOUT -D warnings and grepped for a lint name in brackets, which cannot match the bare \"warning:\" unused_imports emits, so the check printed zero and read clean while CI went red. Worse, the repo own code-review skill specified the same flagless invocation, so it was the INSTRUCTION rather than one person shell mistake -- a reviewer following it would report clean and be wrong. Invoke the gate exactly as the workflow does, flags and all.').
doctrine(an_observation_correct_about_what_changed_can_be_wrong_about_when_it_is_read,'improvise-pdvm/75ld replaced a DECLARED classification with an OBSERVED one, which is right. The cost landed on WHEN the observation is read: comparing document_config once AROUND the batch is correct about what changed and wrong about ordering, because Save clears dirty INSIDE the loop, so [SetNumberFormat, Save] saved and then re-raised the flag. The fix is position, not suppression -- the comparison moved into the loop, using |= so it only ever sets. The asymmetry that decides it: nothing ever CLEARS layout_version, so an end-of-batch comparison is right there, while dirty has an effect that unsets it mid-batch. The discriminating probes rule out BOTH wrong fixes: [SetNumberFormat] alone must still be dirty, or LayoutGeometry has been demoted and the document never saves; and [Save, SetNumberFormat] must be dirty, or the fix is a blanket never-dirty-after-save.').
doctrine(a_siblings_unmerged_pr_is_invisible_to_bd_search,'AGENTS.md requires a bd search before filing. In a parallel wave that is not sufficient: a fix living in a sibling agent UNMERGED PR is in no tracker state at all, so the search correctly returns nothing and the duplicate is filed anyway. Observed 2026-09-20 -- the tooling agent filed improvise-v5pc to restore a deleted help assertion while the harness agent was landing exactly that in PR #72. Neither agent erred. Check OPEN PRS as well as the tracker, and have the orchestrator reconcile filings against merged work at wave end rather than trusting each agent to have seen the others.').
doctrine(a_hard_reset_discards_knowledge_written_to_the_working_branch,'The orchestrator wrote doctrines to kb.pl on local main, pushed each as a branch, then ran git reset --hard gh/main to resync -- twice -- and silently discarded six clauses that existed only in the unmerged local commits. Nothing failed: kb_consistent passed on the reduced file, the counts simply read lower, and the loss was found only by grepping for clause names remembered from the session. Two rules follow. Write a fact ONCE, on the branch that will carry it to main, not on a local main that is going to be resynced. And after any reset or force-move of a branch holding knowledge, re-count the predicates and diff clause NAMES against what you believe you wrote -- a total that merely looks plausible is the same non-evidence a passing test count is.').
doctrine(a_checker_that_reads_headings_cannot_see_a_contradicting_body,'check-evidence validates that a heading names a test that exists. It says nothing about the PROSE under the heading, which is what a human actually reads. Demonstrated on this repo 2026-09-20: renaming a retired entry to the test that had been restored made the checker green while the body three paragraphs down still said \"The test no longer exists\" and \"nothing replaced the assertion\" -- one entry making mutually exclusive claims, and the gate could not see it because the gate was never about that. Found by review, not by the checker. The general form is that a mechanical check licenses exactly the claim it makes and no neighbouring one, so a rename is a read of the whole entry rather than an edit of its first line -- a_fact_stated_twice_is_changed_by_grep applied WITHIN a document rather than across them.').
doctrine(a_cost_note_names_the_half_the_author_was_thinking_about,'Twice now a reviewer has been asked to check a claim of the form \"X is now O(n) where it was O(1)\", and both times the interesting answer was not whether the claim was true but WHICH SIDE OF IT A READER WOULD ACT ON. On PR #76 the author wrote that the cost is \"O(cells) per CellData OR LayoutGeometry effect, where it was per LayoutGeometry effect\" -- which reads as though CellData is the newly burdened path. Measured at 18,000 cells it is the reverse: CellData was ALREADY O(cells) through record_fingerprints, so widening it is a constant factor (+27%), while LayoutGeometry went from O(config) to O(cells + config) and got 33x slower, 78us to 2.63ms, on the held-key-repeat operations. Not dishonest -- the author was thinking about WHICH EFFECTS NEWLY QUALIFY rather than WHICH PATH NEWLY CHANGED COMPLEXITY CLASS, and those are different questions with different answers. A cost note is worth a measurement precisely when the author can say confidently which half is worse, because that confidence is what stops anyone measuring.').
doctrine(name_the_test_that_fails_when_only_this_edit_is_reverted,'A repair is usually N edits, and the failure mode is that N-1 are guarded while the last is explained in a COMMENT instead. Three independent reviews in one wave found the same shape: improvise-mai9 (the helper was tested, the caller that actually rendered was not), improvise-0x9u (preventDefault and the close/dismiss split were both tested, the drawn-vs-seed restore was not, and reverting it shows 18400 where 18,400 belongs on the shipped demo), improvise-o31u (the phone arm was measured, the desktop arm was justified by a mechanism measurement contradicts). The unguarded edit is reliably the one the author understood BEST -- which is why it carries the most careful comment, and why it felt covered: the comment is doing the work the test should, and it reads as evidence because it is the most thoughtful thing on the screen. The check is per-edit, not per-fix: for each distinct edit, name the test that fails when ONLY that edit is reverted. A line with a comment and no answer is the line to inject.').
@@ -314,7 +318,6 @@ doctrine(cfg_test_and_pub_crate_are_different_walls,'A visibility plan written f
doctrine(cargo_tree_shows_dev_edges_by_default,'Asking whether a feature leaked a dependency into a shipped build, a plain cargo tree -p improvise-protocol --target wasm32-unknown-unknown reports serde_json PRESENT -- because the default output includes dev edges. --edges normal is what answers the question, and it reports 0. Verified both ways by the orchestrator. Same class as grep_over_git_diff_is_silently_empty_under_an_external_differ: a command that looks like it answers the question, answering a neighbouring one, in the direction that reads as a finding rather than as a pass. A feature that adds a dependency to a wasm bundle crate must be checked with --edges normal, and per-crate, because the CI wasm job checks each crate separately and feature unification cannot reach across that.').
doctrine(an_unreachability_claim_is_only_as_wide_as_the_routes_enumerated,'PR #98 predicted the composition-time mode drift itself and declared it unreachable BECAUSE the toggle route (R) is a stranding decline in the browser. True of that route. A VIEW SWITCH reached the same drift, because records-ness is auto-detected from the axes, so a switch changes it with nothing in the batch saying \"records\" -- found by ChatGPT Codex. The ENUMERATION of routes is the part nothing checks, so an unreachability argument is evidence about the routes you thought of and about nothing else. Same shape as a_mitigation_that_names_the_wrong_file_is_not_a_mitigation. The remedy is to close the CLASS by observation rather than to patch the route.').
doctrine(a_precondition_derived_from_its_own_expectation_guards_nothing,'A loop test whose precondition is computed from the same tuple element as the assertion cannot be relaxed one iteration at a time: relaxing the assertion relaxes the guard too, and the test then fails for the wrong reason -- which reads exactly like the bug. Assert the two as SEPARATE facts. Corollary for check_the_checker over a loop: an injection that stops the loop at iteration 1 leaves every later iteration untested, so each direction must be reached deliberately. Found by the PR #98 author in its own bidirectional test while proving the second direction.').
doctrine(a_pipe_discards_the_exit_code_you_meant_to_read,'FOURTH occurrence in one campaign, and the last one was in a tool written TO BE the reliable answer. scripts/pr-watch once read tr exit status through a pipe. An orchestrator probe reported EXIT=0 from a checker that had exited 1, because the pipeline ended in tail. Another read PIPESTATUS in zsh, where the array is lowercase pipestatus. And scripts/pr-reviews --wait correctly exited 1 on a timeout while the invocation \`... | tail -20; echo\` reported 0 -- so a review window that had EXPIRED read as a review that had ARRIVED, which is the dangerous direction. In bash and zsh the exit status of a pipeline is the LAST command, so any \`cmd | head\`, \`| tail\`, \`| grep\` discards it. Redirect to a file and read the file, or check PIPESTATUS[0] in bash / pipestatus[1] in zsh -- and note the two shells differ, which is its own trap. A gate whose status you sampled through a pipe has told you about the pager.').
doctrine(editing_a_shell_script_corrupts_the_instances_already_running_it,'bash reads a script INCREMENTALLY rather than loading it whole, so editing a file while an instance is executing it shifts the byte offsets underneath that instance. Two background pr-reviews watchers, started ten minutes earlier and sleeping in their poll loops, both died with \"syntax error near unexpected token do\" the moment the orchestrator appended a paragraph to the usage text -- and the script itself was syntactically fine, which bash -n confirmed. The failure is in the RUNNING instance, not the file, so it is invisible to every check you would think to run afterwards. Before editing a script, ask what is still executing it; a long-poll watcher is exactly the shape that will be. Worse in this case than a crash: the watcher exited having printed only the EARLIER review, so the wait it was supposed to perform silently did not happen.').
doctrine(a_worktree_agent_merges_the_orchestrator_uncommitted_local_main,'Agents in git worktrees share the repository local refs, so \`git merge main\` in an agent worktree takes the ORCHESTRATOR LOCAL main -- including commits never pushed to any remote. That is how an unlisted context/code/pr-review.md, committed straight to local main by the orchestrator, reached PR #98 branch and turned its build job red on the doc-site manifest gate (improvise-res9 working exactly as designed). The agent diagnosed it as \"main has been red since PR #96\", and BOTH halves were wrong: CI on #96 merge was green, and the doc was not in main when #96 agent merged at ea206bf -- it entered between, so only the LATER branch saw it. The orchestrator local main is an unreviewed publishing channel into every running agent, and a commit there is subject to the same gates as a PR without getting them. Either run the gates before committing to local main, or do not commit there at all.').
doctrine(a_manifest_gate_fires_at_merge_time_not_at_authoring_time,'scripts/build-doc-site.mjs refuses a context/**/*.md that is neither published nor a named omission -- a good gate (improvise-res9). But it runs in the \`build\` CI job, so a doc added in one branch makes every LATER branch red while the author who added it never sees a failure. The cost lands on whoever merges next, who then diagnoses somebody else change. Worth moving to the \`test\` job, which is fast and needs no wasm build. Generalises: a gate over a REPOSITORY-WIDE invariant should run wherever the violation is authored, not only where the tree is assembled.').
@@ -364,7 +367,6 @@ doctrine(a_hard_floor_across_widths_names_the_container_not_the_child,'improvise
doctrine(never_hide_the_overflow_you_are_asserting_the_absence_of,'improvise-tg27 rejected body{overflow-x:hidden}, which would have made scrollWidth == clientWidth UNCONDITIONALLY and rendered its whole spec vacuous -- every assertion still passing, the bar still not fitting. The check would then answer \"is overflow hidden\" where it means \"does the content fit\". A fix that makes the measurement unable to fail is not a fix, and it is especially tempting in CSS, where suppressing a symptom and correcting a layout look equally like one line.').
doctrine(a_deferral_cost_must_be_measured_not_named,'improvise-k844 sat open because context/code/gp.md section 7 deferred it on the claim that reordering the arithmetic \"moves the bit patterns misclass_rate_and_importance_are_bit_for_bit_what_they_were pins\". That test pins permutation_importance RAW SCORES; importance_ranking consumes them AFTERWARDS, so no path it witnesses runs through the share -- and scoring_known_answers.rs is byte-identical across the actual fix, verified by the orchestrator as absent from the diff. A COST PREDICTED FROM A TEST NAME RATHER THAN FROM WHAT IT CALLS is how a one-line fix stays filed, and it reads as diligence, which is why it survives review. Measure the claimed cost or state none. Pairs with an_invariant_is_made_unrepresentable_not_validated: the clamp improvise-pv3g added instead was a call-site check that left inf% in the data behind it.').
doctrine(a_reported_number_can_overflow_while_every_input_is_finite,'The gp crates learned this twice. standardize sums (t-mean)^2 in f32 (improvise-pv3g) and importance_ranking scaled a share by 100.0* in f32 (improvise-k844), and the second was REACHABLE PAST THE FIRST. Both are fixed by making the arithmetic incapable of it -- a Scaling that refuses a non-finite part, an f64 accumulator -- never by a call-site check, and pv3g own f32 clamp WAS that call-site check: it stopped the abort and left inf% in the data behind it. The silent twin is the half to look for: pv3g had cv_regress pooling a fold it could not standardize, k844 had an f32 total going inf with no panic to announce it. Ask what ELSE consumes the derived quantity.').
doctrine(a_dev_shell_is_a_third_environment,'.envrc puts nix bash 5.3 on PATH -- the RIGHT INTERPRETER, CI semantics -- and pkgs.mkShell ALSO exports about thirty generically-named lowercase variables the runner does not have: name, out, outputs, system, shell, builder, stdenv, phases, buildInputs, doCheck and more. A script reading one by accident under set -u is SILENT IN THE DEV SHELL AND FATAL IN CI. improvise-f9qf: pr-reviews-test.sh passed 19/19 under macOS bash 3.2 AND 19/19 under the dev shell bash 5.3, while failing 2/19 in CI -- two different reasons for green, one of them the correct interpreter, and unsetting \`name\` alone reproduced CI 2/17 exactly. Note nix shell does NOT export name and nix develop DOES (verified with env -u name as the control), so the dev shell is the one nix entry point that masks this class. The orchestrator own verification of this was itself confounded by inheriting name from a direnv-loaded shell -- it answered \"does name end up set\" where the question is \"does nix shell set it\".').
doctrine(an_observation_is_a_function_of_the_trajectory_not_the_state,'improvise-ilvl claimed \"an observation is a pure function of state\". It is not -- layout_version, cursor_epoch and model_state.dirty all differ between two routes to an IDENTICAL document (0/0/false against 2/2/true). The weaker true claim, that they are reproducible FROM THE LOG, is all replay needs. But the stronger false one is exactly what would justify SNAPSHOT-PLUS-SUFFIX LOG COMPACTION, and that is wrong silently in the direction that looks like it works: Basis puts document_generation and layout_version ON THE WIRE and clients compare them for EQUALITY, so a compacted replay is not basis-compatible with any client holding a claim, and dirty is the worker OPFS persist trigger so the same document reached by a different route persists a different number of times.').
doctrine(a_determinism_check_can_pass_because_the_divergent_path_is_unreachable,'Three identical replays of a 26-line script fixture proved the script path deterministic and proved NOTHING, because the fixture cursor moves were silent no-ops (improvise-6uva) -- the divergence had no way to occur. The defect surfaced only when clear-cell FAILED TO PARSE for want of coordinates. Sharper form of \"a green suite is not evidence\": a passing determinism or idempotence check must be shown to be capable of FAILING, by driving the input the check exists to discriminate. It is also how one defect MASKS another -- improvise-up8g viewport hazard is invisible in improvise script precisely because 6uva discards the cursor, so fixing 6uva makes up8g live and they must be closed together.').
doctrine(an_injected_fault_is_not_reverted_until_the_full_suite_is_rerun,'An agent injected two rivals and reverted one onto the WRONG OCCURRENCE: replace(\".is_measure()\",\".is_index_or_dim()\",1) hits the FIRST match in the file, which was drill_constraint pre-existing predicate rather than the injected one. The commit shipped with TWO faults where it had demonstrated one. Every gate re-run afterwards PASSED over the broken logic -- clippy, fmt, check-evidence, both node suites, the eight-crate wasm gate, the full Playwright run -- because the one gate that could see it, cargo test --workspace, had been measured BEFORE the injections and never again. The PR body therefore carried a figure that was checkable and false. Caught by ChatGPT Codex review. THE RULE: the targeted test going green says nothing about what the revert did on its way past, so the full suite must be re-run after every revert.').
@@ -383,6 +385,13 @@ doctrine(every_agent_appends_to_the_same_evidence_file_at_eof,'Four evidence-fil
doctrine(a_family_of_bugs_means_the_question_is_wrong,'improvise-lhi4, improvise-0o6x and improvise-6qet are three beads about the same missing way back out of a collapsed group. The first two both fixed the header VISIBILITY -- and AxisEntry doc comment promise that a GroupHeader is always visible \"so its label and collapse state show\" became true in both halves without the header becoming ADDRESSABLE. Three people re-finding one gap is evidence that each fix answered a NEIGHBOURING question, and the cure was to ask what the user actually expected (the cursor follows the content) rather than to fix a third symptom. So: when a third ticket arrives in one family, stop fixing and re-derive the question -- and expect the answer to be a shape none of the accumulated options list, because the options were all generated under the wrong framing.').
doctrine(a_stub_must_fail_in_the_shape_of_the_real_fault,'A test double that breaks MORE than the real fault breaks satisfies the test through a DIFFERENT arm and reads exactly like coverage. Found three times on improvise-b4ks, each time only by injecting a rival: a cargo stub that failed for EVERY argument also broke cargo metadata, so the run exited on the metadata arm and the rustfmt probe could be DELETED with the suite green. The stub has to fail where the real thing fails and succeed where it succeeds; otherwise the assertion is answered by a path that will never occur in production.').
doctrine(a_checker_fixture_must_carry_the_structure_the_hazard_needs,'A simplified fixture tests a repository that does not exist, and a green suite over one is the same non-evidence as a vacuous assertion. On improvise-b4ks the fixture was vacuous three times: with no PATH DEPENDENCY, cargo fmt --all had nothing to leak through; with a VIRTUAL outer manifest, the bare --manifest-path form happened to reach every member. Each time the structural feature the hazard depends on had been simplified away, and only an injected rival showed it. So enumerate what the hazard NEEDS -- here two workspace roots, a path dependency crossing them, and an outer root that is itself a package -- and build the fixture to carry all of it.').
doctrine(a_pipe_discards_the_exit_code_you_meant_to_read,'FOURTH occurrence in one campaign, and the last one was in a tool written TO BE the reliable answer. scripts/pr-watch once read tr exit status through a pipe. An orchestrator probe reported EXIT=0 from a checker that had exited 1, because the pipeline ended in tail. Another read PIPESTATUS in zsh, where the array is lowercase pipestatus. And scripts/pr-reviews --wait correctly exited 1 on a timeout while the invocation \`... | tail -20; echo\` reported 0 -- so a review window that had EXPIRED read as a review that had ARRIVED, which is the dangerous direction. In bash and zsh the exit status of a pipeline is the LAST command, so any \`cmd | head\`, \`| tail\`, \`| grep\` discards it. Redirect to a file and read the file, or check PIPESTATUS[0] in bash / pipestatus[1] in zsh -- and note the two shells differ, which is its own trap. A gate whose status you sampled through a pipe has told you about the pager. The gate most often lost this way since is scripts/fmt-workspaces.sh: an unformatted tree exits 1 AND PRINTS A DIFF, so \`| tail\` shows the tail of a diff and exits 0, which reads as a clean run with some output. Read its status, or run it with --fix.').
doctrine(a_dev_shell_is_a_third_environment,'.envrc puts nix bash 5.3 on PATH -- the RIGHT INTERPRETER, CI semantics -- and pkgs.mkShell ALSO exports about thirty generically-named lowercase variables the runner does not have: name, out, outputs, system, shell, builder, stdenv, phases, buildInputs, doCheck and more. A script reading one by accident under set -u is SILENT IN THE DEV SHELL AND FATAL IN CI. improvise-f9qf: pr-reviews-test.sh passed 19/19 under macOS bash 3.2 AND 19/19 under the dev shell bash 5.3, while failing 2/19 in CI -- two different reasons for green, one of them the correct interpreter, and unsetting \`name\` alone reproduced CI 2/17 exactly. Note nix shell does NOT export name and nix develop DOES (verified with env -u name as the control), so the dev shell is the one nix entry point that masks this class. The orchestrator own verification of this was itself confounded by inheriting name from a direnv-loaded shell -- it answered \"does name end up set\" where the question is \"does nix shell set it\". CHANGED 2026-09-24 (#287, #297): every CI step now runs inside nix develop .#ci, itself a pkgs.mkShell, so CI exports those variables too and the dev shell no longer masks a CI failure of this shape. The hazard moved rather than vanished: the environment that differs now is a BARE local shell with no direnv, and checkers.sh still scrubs to an allowlist so a suite cannot come to depend on one of them either way.').
doctrine(checkout_ours_or_theirs_drops_the_other_sides_clean_changes,'git checkout --ours or --theirs on a conflicted file takes the WHOLE file from one side, so it silently drops every non-conflicting change the other side made to that file. #245 resolved a ci.yml conflict with #228 by --theirs and lost its own check-harness-lock --allow-network step while its merge message said the step was unchanged (improvise-dndu). No gate saw it: check-ci-graph.py counts suites, not that step, and a message cannot be checked at all. Resolve by hand, then diff the merge against the merge base and against EACH parent -- the question is not whether git combined the texts but whether the result holds both sides.').
doctrine(a_clean_merge_of_one_test_file_can_still_fail_together,'Two branches can each pass CI, merge with no conflict, and fail together. #287 found that on the NixOS runners /usr/bin:/bin holds only env and sh, and gave section 5 of checkers-test.sh a hermetic utilbin PATH; #237, cut earlier, added section 5b still using PATH=...:/usr/bin:/bin. The merge was textually clean and 5b broke every PR after it (#296). A fix to an IDIOM -- a hermetic PATH, a scrubbed environment, a helper every case must call -- is owed by every case in the file, including cases that exist only on another open branch. After a merge, grep the merged file for the old idiom, and run the gates on the merged tree rather than on each side.').
doctrine(history_is_reconciled_only_by_merge,'The user, 2026-09-25: the merge structure of git history is a causal record (improvise-1xfb.1). A merge keeps that record, and a rewrite replaces it with a story. So: (1) the default is a NEW Conventional Commit -- As a general rule, you shuold just add a new conventional commit. Small adjustments can ammend HEAD, but when in doubt don\'t [sic] (improvise-1xfb.2); (2) amend only HEAD, only for a small adjustment, and only while it is unpushed; (3) never reorder, fold or squash commits, pushed or not, and never move them onto another base -- no git rebase and no git reset --soft gh/main; (4) nothing that would need a force push, and no force push; (5) a newer main is taken by re-pinning gh/main and running git merge on that SHA; (6) git cherry-pick -x only for commits that will never reach main as they are -- an abandoned or rebuilt PR, or commits only on a mirror or a local main -- since a cherry-pick drops the causal link. The rewrite is also where the damage came from: twice a git reset --soft gh/main computed a squash against a ref another session had moved and staged a silent revert of a merged PR (#196, improvise-diho; #303, caught by the numstat check). The rule text is .agents/skills/wave/agent-rules.md rule 8.').
doctrine(a_review_names_the_sha_it_reviewed,'A review of a superseded commit is worse than no review: it reads as a pass. So the reviewer names the exact SHA, the author announces every push, and a re-review is of the delta. The author never rewrites history, in review or out of it: new commits and merges only, an amend only to an unpushed HEAD, and nothing that needs a force push (doctrine(history_is_reconciled_only_by_merge); agent-rules rule 8; improvise-at5k). Each fix is therefore a new commit on top of the reviewed SHA, and the delta is exactly what the re-review reads. This matters more now that adversarial agents are the gate a merge waits on -- Copilot\'s request was a SILENT no-op for much of 2026-09 (the mutation returned success and registered no reviewer), which is itself an instance of doctrine(an_advisory_gate_is_not_a_gate).').
doctrine(a_hard_reset_discards_knowledge_written_to_the_working_branch,'The orchestrator wrote doctrines to kb.pl on local main, pushed each as a branch, then ran git reset --hard gh/main to resync -- twice -- and silently discarded six clauses that existed only in the unmerged local commits. Nothing failed: kb_consistent passed on the reduced file, the counts simply read lower, and the loss was found only by grepping for clause names remembered from the session. Three rules follow. Write a fact ONCE, on the branch that will carry it to main, not on a local main. Resync local main by git merge --ff-only gh/main, never by a reset: a reset or force-move that drops commits is forbidden under doctrine(history_is_reconciled_only_by_merge) (improvise-at5k), and the ff-only merge refuses exactly where the reset would have discarded something. And if a branch holding knowledge has lost commits anyway, re-count the predicates and diff clause NAMES against what you believe you wrote -- a total that merely looks plausible is the same non-evidence a passing test count is.').
work_done('improvise-8tk',bug_fix,'ZZ/:wq quit even when the save failed, silently discarding the model. Save now aborts the effect batch on failure; save_to_current_path/1 returns Result<String,String>; three tests in ui/effect.rs.').
work_done('improvise-907',bug_fix,'Import wizard step machine redesign: validation_error() gate at ReviewProposals (no-category and over-limit), history-stack previous() bound to Left on every step, and Done made a real confirmation step holding the built workbook. Superseded improvise-gfq, improvise-o1c, improvise-c6j.').
work_done('improvise-6sq',wont_do,'Merging EditOrDrill and EnterEditAtCursorCmd into one EnterEdit{mode} is not safe: they differ in WHEN the cell value is read, not in a mode parameter. Added ui/app.rs::tests::i_and_o_diverge_on_an_empty_records_grid as executable evidence.').
@@ -434,6 +443,11 @@ work_done('improvise-nxah',bugfix,'PR #136, merge eefd992. The client rolls back
work_done('improvise-tg27',bugfix,'PR #138, merge 327a76b. header flex-wrap over a breakpoint; main min-width:0; panels capped at 40vw; harness/tests/narrow.spec.js asserts documentElement.scrollWidth <= clientWidth at 360/520/768, so the page own overflow leaves gotcha 48 list of what the harness cannot see. Fault injected by restoring style.css to 22ab58f: 6 of 9 assertions failed, and the two that passed are the two that should. Merge with improvise-nxah verified STRUCTURALLY -- 198 insertions and ZERO deletions across context/, with evidence entries rising 812 to 818 as the independent witness. Found not predicted: the keyboard already reached every clipped column, so what was broken was a FALSE scroll affordance beside a working one.').
work_done('improvise-k844',bugfix,'PR #139, merge 1ae2d2e. importance_ranking sums and divides in f64, narrowing only the bounded share. The injected rival is THE TICKET OWN TITLE -- reassociating to 100.0*(imp/total) in f32 passes the lone-importance and sums-to-100 tests and fails the twin and the guard, 58 passed 2 failed, converting NaN% into a serene 0% for every feature. Scoring bit patterns byte-identical, which is the evidence the fix is confined. context/code/gp.md section 7 corrected in place: it had deferred this bug on a cost that does not exist.').
work_done('improvise-f9qf',tooling,'PR #137, merge a9cad15. scripts/checkers.sh plus scripts/checkers-test.sh (12 cases, each INJECTING rather than asserting). The proof: restoring the multi-assignment local gives 22 passed under the old local gate, 22 passed under direnv, and 2 passed / 20 failed / exit 1 under checkers.sh. The second undetected split is closed -- the four #!/bin/sh and one #!/usr/bin/env sh scripts are now bash, a strict superset needing no porting. Unrun and stated: scripts/build-web-release.sh executes only under dist in build-global-artifacts, which no branch push exercises.').
work_done('improvise-gtn7.1',bugfix,'PR #303, merge bb01cd19. The TUI grid width is one derivation: viewport::grid_width(term_width, side_panel_open), reached through App::grid_width, is what draw.rs::draw_content splits the content row on and what App::cmd_context counts visible_cols on. The keypress path had counted from term_width while the draw gave the grid term_width minus the 32-column side panel, so with any panel open the cursor scrolled behind it. The tile bar had the same shape and now scrolls (first_tile) and draws from one tiles_left(start). Guarded by draw.rs content_tests::moving_right_keeps_the_selected_cell_drawn_beside_any_side_panel and tile_bar the_selected_tile_is_drawn_at_every_width_it_fits_in. Mixed-width columns are still wrong: improvise-gtn7.2.').
work_done('improvise-cgqt',feature,'PR #304, merge 4f38a35e. The import wizard asks what to do with records that write one cell (ResolveDuplicates, NameDuplicateCategory), DuplicatePolicy {Sum, Separate(name), Refuse} with Refuse the default, Collision as data so every front end draws the same rows, and --on-duplicate on the headless CLI.').
work_done('improvise-9bit',bugfix,'PR #304 (commit f00ed3e9), merge 4f38a35e. The collision check counted records with disjoint measures on one tuple as colliding; collisions_among now keeps only records that write a measure another record on the tuple also writes. Rival: the tuple-only rule, which fails only_the_records_that_share_a_cell_are_listed_and_numbered.').
work_done('improvise-4klp',test,'PR #304 (commit f00ed3e9), merge 4f38a35e. the_name_check_covers_categories_derived_from_dates: a date-derived category name such as when_Year is refused as the occurrence category name. Green on arrival; its evidence is the reviewer rival that drops date-derived names from the taken names.').
work_done('improvise-felo.1',feature,'PR #307, merge 1719f2cf. MAX_CATEGORIES raised from 12 to 24 in improvise-core model/types.rs, pub and re-exported as improvise_core::model::MAX_CATEGORIES; the import wizard private copy is deleted and it reads the model constant. Fill-to-the-cap tests derive the number from the constant. crates/improvise-core/examples/category_width_bench.rs measures 12 against 24; the coupling inventory is context/code/core.md.').
work_followup('improvise-8tk','improvise-lae','no-file-path UX: prompt for a path rather than only erroring. ExportPrompt is CSV-only, so a new SavePrompt TextEntry mode committing to SaveAs is the likely shape.').
work_followup('improvise-6sq','improvise-ct5','i on an empty records grid reports \'cannot drill\', leaking a pivot-mode concept into records mode').
work_followup('improvise-h0b','improvise-a2x','ViewState field types (AppMode, MinibufferConfig, ViewFrame, KeyCode, KeyModifiers) still derive no Serialize, and improvise-command depends only on serde_json. Deferred: nothing serializes ViewState until improvise-cqi builds a Snapshot, so the derives would have no consumer.').
@@ -530,8 +544,6 @@ decision('improvise-xgra',an_inline_comment_at_the_head_satisfies_the_wait,'A pu
decision('improvise-0ygd',split_on_the_buffer_not_the_column,'In records mode +measure is ALWAYS the rightmost column, so cursor.col >= col_max names that one cell and nothing else -- which means the entire \"Tab at the bottom-right starts the next record\" bulk-entry contract is reachable ONLY from that cell with an EMPTY buffer. A guard keyed on the column alone fixes improvise-0ygd and SILENTLY DELETES BULK ENTRY. The injected rival proves it: dropping the value-awareness passes the new test and fails tab_on_an_empty_new_measure_column_still_starts_a_record_row. When one cell is the only member of a class, a guard on the class is a guard on the cell.').
decision('improvise-nxah',a_refused_frame_is_rolled_back_by_the_fold_not_a_cache,'What rolls back on ServerFrame::Rejected is ViewState -- the thing the fold produces -- and never a cache, because a Command::Model is A NO-OP IN reduce_view: the optimistic fold never touches a cache at all, and page_coords moves only under Projection::Grid. That single fact refutes the bug report own evidence (which blamed page_coords) and dissolves the wave recorded contradiction between nxah and improvise-3al1 -- the staged typing 3al1 needs rides in SetBuffer frames, which carry no model command, are never refused, and have already settled, so rolling back the COMMIT restores the buffer TO THE TYPING. Only a frame carrying a Command::Model is rejectable, stated exhaustively by Checked::against.').
decision('improvise-tg27',a_reserve_is_a_floor_and_must_reflow_not_widen,'header{flex-wrap:wrap} over a @media breakpoint: a flex item wraps exactly when the line cannot hold it, so the desktop bar is untouched BY CONSTRUCTION rather than by a chosen number -- no magic width to drift. Plus main{min-width:0}, 40vw caps on the side panels, and max-width caps so a reserve cannot exceed its own line. The reflow threshold is deliberately asserted NOWHERE, because #status composes the DOCUMENT NAME: the same 1024px window is one line under a short name and two under a long one. Only 1280 is pinned, being where improvise-n6t8 stability spec runs.').
decision('improvise-f9qf',one_checker_entry_point,'scripts/checkers.sh is the ONE entry point for every check_the_checker, called verbatim by CI and by the local gate so there is no second invocation to drift. It pins a bash >= 5 (ambient first, nix only when the ambient one is older, exit 3 when neither), SCRUBS THE ENVIRONMENT to an allowlist (env -i plus PATH/HOME/TMPDIR/NIX_*/GITHUB_*, extensible via CHECKERS_KEEP), and refuses any shell shebang in scripts/ that is not #!/usr/bin/env bash. The pinned bash goes on the FRONT of PATH so scripts the suite invokes get it too -- which is where the fault actually lived. CI logs \"checkers: bash 5.2 at /usr/bin/bash\": ambient, no nix, zero startup.').
decision('improvise-f9qf',nix_shebang_declined_on_availability_not_speed,'Measured warm, 20 invocations: ambient bash -c true 5.1ms; nix shell nixpkgs#bash --command 61.8ms; nix-shell -p bash --run, THE MANPAGE FORM THE USER POINTED AT, 675.2ms -- 132x, and that inside git merge machinery for beads-merge-driver.sh. But the DECIDER is availability: nix is installed on ONLY the kb job. build, test, wasm, js, harness-lint, browser and pages-deploy have none, so a nix shebang needs install-nix-action added to three more jobs, paid every run, to reach a bash those runners already ship. zsh declined too -- not a shebang change (word splitting, 1-based arrays, globbing), sixteen scripts to read, and no zsh on the runner either.').
decision('improvise-6uva',script_replay_lost_its_context_at_35946af,'improvise script and improvise cmd resolve every command through CmdRegistry::parse -- the PURE arm, which builds CursorState::default() (0,0,0,0) -- never the interactive arm that builds CursorState::from_ctx. So every Move command and enter-advance snapshots a ZEROED cursor AT PARSE TIME, clamps to (0,0) and emits SetSelected(0,0): worse than a no-op, it RESETS the cursor, and no cursor-reading command is reachable from a script. Broken by 35946af (2026-04-04), a performance refactor introducing the register/register_pure/register_nullary split, FIVE MONTHS BEFORE the first wasm-client commit -- so the web refactor is innocent. Both directions demonstrated: at the parent 649d80c a move-then-clear cleared row 2 and a bare clear cleared row 0; at HEAD both clear row 0. THE SHAPE: a registry with two construction arms silently demotes every caller of the weaker arm, and nothing type-checks which arm a caller deserves.').
decision('improvise-gd28',browser_drill_blocked_by_surrogate_scope_not_layer_2,'The ticket recorded drill as blocked on ADR 001 layer 2 because ApplyAndClearDrill is session-local and unaddressable. That reasoning is about the COMMIT path. Entering a drill needs no commit, and CreateView/SetAxis/SetPageSelection are already unaddressed ModelCommands that NAME THEIR VIEW. What a browser cannot supply is the OPERAND -- CellId::Aggregate(AggregateHash), an opaque u128, which is ADR 002 working as designed. The real gap is one level down from where it was looked for: THE MUTATION AND ITS OPERAND HAVE DIFFERENT SCOPES. The mutation configures the shared _Drill and is addressed to nobody; the operand exists only inside one session layout. So the fix is not a new mechanism but the view:String convention every other view-scoped variant already follows, resolved against a layout derived for that name -- no SessionId needed. fdk1 is also not needed: edit_or_drill declines records-mode editing OUT LOUD before reaching a cell, so a browser cannot edit inside a drill grid at all and no divergent staging scheme is reachable.').
decision('improvise-rr8n',keep_the_table_derived_from_what_it_describes,'The ticket asked to rewrite the tile-bar hint literal and SUPPRESS the derived status line in TileSelect. The implementing agent did the opposite and the orchestrator ACCEPTED it: the derived status line is a STRICT SUPERSET of the literal (Space, h/l, Tab appear only there) and cannot go stale, so suppressing it would delete information and keep the table that had just rotted. Verified before accepting -- the literal read \"Ctrl+arrows to move tiles\" while those keys are bound to enter-tile-select, so it was already false. Both tile-bar literals deleted; Normal derived line already says T:tiles. The import wizard keeps the ticket ORIGINAL answer, and the asymmetry is the point: ITS status line is derived from the MODE keymap while the real keys are per-step inside HandleWizardKey, so there the derived table is the one that lies. RULE: keep whichever table is derived from the thing it describes. COROLLARY: every_hint_key_resolves_in_its_mode CANNOT SEE A STRING LITERAL IN A WIDGET, the same blind spot improvise-wk18 names for help.rs, so a hand-written key literal is invisible to every hint guard in the repo.').
@@ -540,6 +552,27 @@ decision('improvise-6uva',a_registry_with_two_arms_demotes_the_weaker_caller,'Cm
decision('improvise-55v2',a_borrowed_str_in_a_wire_adaptor_narrows_the_format,'Repr::Nonfinite(&\'a str) in an untagged serde adaptor can only be produced by a deserializer that BORROWS FROM ITS INPUT, so the spelling round-tripped through from_str and from_slice and through nothing else. Measured before fixing: {\"Number\":\"inf\"} parsed while {\"Number\":\"in\\u0066\"} -- THE SAME JSON DOCUMENT, since escaping is not a distinction JSON makes -- was refused, and to_value/from_value did not round-trip the adaptor own output. So the wire accepted a SUBSET of the encodings of the spelling it claimed to share with the .improv file. Cow<\'a, str> fixes all four paths and leaves the serialized form unchanged. THE TESTS COULD NOT SEE IT because both used from_str exclusively, which is precisely the one path that borrows -- a check adjacent to the question it names, arriving inside the change that DEFINES the question. Found by adversarial review, confirmed by injection.').
decision('improvise-6qet',a_collapsed_group_is_one_aggregate_row,'USER RULING 2026-09-23: \"When collapsing a group, I would generally expect the cursor to move to whichever place the group collapsed into.\" That reframes the bug -- it is not \"the GroupHeader is not addressable\", it is \"THE CURSOR DID NOT FOLLOW THE CONTENT IT WAS ON\", and the ticket four options were all answers to the wrong question. CHOSEN SHAPE, a fifth none of them listed: expand_category stops SKIPPING a collapsed group items (the bare \`continue\` at layout.rs:755, which is the whole mechanism of the bug) and emits ONE DataItem for the group, whose cell is an Aggregate. It DISSOLVES the problem rather than working around it -- the cursor lands on the collapsed group BY ARITHMETIC with no special case, z becomes an involution again without anything being taught about involution, ViewCursor and row_group_for and every cursor-addressed command are untouched, and settability is already answered by ADR 002 since grid_of returns CellId::Aggregate. It is also what a pivot table MEANS by a collapsed group: the totals in one row, where today collapsing shows nothing in their place. Rejected: a cursor that names a visual entry (literally delivers the ruling but changes row_group_for signature and forces every cursor-addressed command to define a header row), and resolving against the visual stream (narrow, but only ACTS as if the cursor had moved).').
decision('improvise-b4ks',fmt_per_root_by_member,'This repo has TWO workspace roots -- harness/opfs-contract declares its own [workspace] -- and BOTH obvious spellings fail in the direction that looks like success, each measured. \`cargo fmt --all\` (or --manifest-path <root> --all) CROSSES THE BOUNDARY, because --all formats local PATH DEPENDENCIES and harness/opfs-contract depends on improvise-io by path, so the inner root reported src/draw.rs; this is wider than the ticket note, which blamed only cd-then---all. And bare --manifest-path is BLIND TO MEMBERS, since the repo root is a package as well as a workspace -- an injected fault in crates/improvise-core exited 0. The only sound spelling is \`-p <each member>\` scoped by that root own manifest. Note \`cargo fmt -p opfs-contract\` FROM THE REPO ROOT exits 1 with \"not a member of the workspace\", indistinguishable from a real verdict, which is the ticket itself; from INSIDE that root it is correct, which is why CI harness-lint job was always sound while the root-level gate every brief prescribed was checking less than it claimed.').
decision('improvise-f9qf',one_checker_entry_point,'scripts/checkers.sh is the ONE entry point for every check_the_checker, called verbatim by CI and by the local gate so there is no second invocation to drift. It SCRUBS THE ENVIRONMENT to an allowlist (env -i plus PATH/HOME/TMPDIR/NIX_*/GITHUB_*, extensible via CHECKERS_KEEP) and refuses any shell shebang in scripts/ that is not #!/usr/bin/env bash. It used to pin a bash >= 5.1 as well (ambient first, nix only when the ambient one was older, exit 3 when neither), and that half is GONE since #297 (improvise-3uow, 2026-09-24): CI moved to NixOS runners where every step runs in nix develop .#ci, and the user ruled that scripts must not care which bash or standard utilities are present. So the runner calls each suite with $BASH and an unchanged PATH, and a script needing a newer feature fails where it uses it. Which suites each CI job runs is the ci_suites table in checkers.sh (checkers.sh --ci <job>), held to checkers.sh --list by check-ci-graph.py. context/code/tooling.md section 5.').
decision('improvise-f9qf',nix_shebang_declined_on_availability_not_speed,'Measured warm, 20 invocations: ambient bash -c true 5.1ms; nix shell nixpkgs#bash --command 61.8ms; nix-shell -p bash --run, THE MANPAGE FORM THE USER POINTED AT, 675.2ms -- 132x, and that inside git merge machinery for beads-merge-driver.sh. But the DECIDER was availability: when this was decided nix was installed on ONLY the kb job, so a nix shebang needed install-nix-action added to three more jobs, paid every run, to reach a bash those runners already shipped. zsh declined too -- not a shebang change (word splitting, 1-based arrays, globbing), sixteen scripts to read, and no zsh on the runner either. SUPERSEDED IN ITS PREMISE on 2026-09-24 (#287, #297, improvise-3uow): every CI job now runs on a self-hosted NixOS runner inside nix develop .#ci, so bash comes from the dev shell everywhere and no shebang needs to reach for nix. The decline stands, for a different reason: the dev shell already supplies the bash, so a nix shebang would add startup cost and buy nothing.').
decision('pr-landing-2026-09-24',land_a_batch_as_one_integration_pr,'Approved PRs that land together land as ONE integration PR, not one at a time. Every PR appends its evidence at the end of a context/test-evidence file, so each single merge makes every other open PR conflict at EOF, and a conflicting PR gets no CI run at all. On 2026-09-24 the day went through #298 (nineteen PRs) and #299 (the browser-editor stack). This is what one PR per wave already meant; the EOF-append convention is what makes it a necessity rather than a preference.').
decision('adr-process-2026-09-24',an_open_adr_pr_is_watched,'User 2026-09-24: an open ADR PR must be actively watched, so the steward agent acts on the user inline answers and on reviews as they arrive. An answer left unread for hours is a failure. The orchestrator keeps a background poll for non-agent comments and reviews since its last pass, and resumes the steward on each new item. Only the user explicit approval still merges an ADR; the watch is about latency, not authority. bd memory adr-pr-needs-a-watcher.').
decision('improvise-wpxr',a_record_resolves_in_the_view_the_command_names,'A SetCell or ClearCell carries the view that drew it, and its RecordHash resolves only when the document has that view: the store index when the record holds cells, else that view layout, where an EMPTY record still exists as a drawn row. A view the document lacks drew nothing, stored or not, so the answer is RecordGone rather than a resolve_view fallback (Codex P2 on #281). RenameRecordCoordinate resolves in the store alone, since only a record holding cells can be renamed. protocol::named_records says which, per command, as NamedRecord::Drawn or NamedRecord::Stored, one exhaustive match.').
decision('improvise-kimw',grids_memoise_one_layout_per_view_per_borrow,'App::grids() returns a Grids borrow that derives each named view GridLayout at most once for as long as it lives, so a command asking resolve_record, resolve_cell and unwritable of one view no longer re-derives the layout about five times (and, with formulas, clones and recomputes the model each time). The memo is only sound because it is a BORROW: hold one across a check, never across a mutation, since its layouts describe the model it borrowed.').
decision('improvise-e265',group_runs_nest_and_pruning_remarks,'FrameGroup and GroupEnd carry a level, an index into the axis cats, and runs NEST: a header at level L replaces L and closes every deeper run, a GroupEnd at L closes L and deeper, and an item belongs to the innermost open run. So prune_axis_items does not filter markers, which cannot tell an emptied header or an orphaned GroupEnd; it re-marks the survivors, emitting before each kept item exactly the markers that make its open runs equal the input ones at every level (remark, context/code/core.md section 6).').
decision('improvise-4k1j',the_frame_carries_the_layouts_fold_flag,'FrameEntry::Item is { tuple, collapsed }, and collapsed is the layout own fold flag, read by AxisFrame::folds. A client asks the frame whether a position folds; re-deriving it from the nearest group was the bug, which hid a group name under nested row categories.').
decision('improvise-jq0m',spanned_narrowing_chooses_by_index_size_and_keeps_store_order,'Model::fold_stored folds a collapsed group once over every record it covers, through DataStore::spanned_values and spanned_positions (improvise-yb4b). spanned_positions narrows either once by the base key or once per covered item of the smallest span, whichever visits fewer by index-set size; scanning every item of the category was quadratic on a grid of small collapsed groups. Both arms sort their positions, so both return STORE ORDER: a fold in any other order would drift SUM in the last bits, and would make the choice between the arms visible in a total.').
decision('improvise-8xq0',a_commit_writes_the_cell_its_editor_opened_on,'The browser keyboard commit writes ClientState::opened, not the cursor. OpenedCell captures at open the record, the OpenedTarget (a value by measure, or a records coordinate by category), the record version, the document generation and the view. So the commit lands on the cell it opened on, claims the basis it was opened against rather than one read at commit (improvise-xz2z), and a retry after a crossing LayoutMoved must match the WHOLE OpenedTarget, not the record alone (improvise-8phw, improvise-9ww8). refused_here exempts a captured editor, so the cursor current cell cannot refuse a commit aimed elsewhere (improvise-3h9w).').
decision('improvise-uln5',needs_the_host_is_one_list_held_three_ways,'resolve::NEEDS_THE_HOST is the one list of commands a browser cannot run and answers by sending the key upstream. Three tests hold it from three directions: the_command_names_that_still_need_the_host_are_these (no false entry), a_help_row_the_browser_cannot_run_is_drawn_marked (the help overlay marks them), and every_silent_binding_is_on_the_list, which walks every binding of every mode a browser can reach (improvise-oyaa, improvise-s2gb) because a check read off the help rows could not see a binding no row names. A decline that must stop the rest of a binding is Step::Declined (improvise-joed, improvise-ik6l).').
decision('improvise-n0pb',a_client_notice_queues_behind_in_flight,'ClientState::notify folds a client-side status as an outgoing frame at the tail of in_flight, so a rebase that rebuilds the view from settled and replays in_flight replays the notice too instead of erasing it. settle_status runs on every path that rebuilds the view, including rebase (improvise-0dbk), and gives a new message to the editor open as it arrives; EditorSession, that owner key, lives in improvise-protocol beside ViewState (improvise-su4u, improvise-1x9h).').
decision('improvise-gk51',uneditable_is_one_decline_for_open_and_commit,'cmd::core::uneditable is the one decline for opening an editor (EnterEditAtCursor) and for committing one (CmdContext::committable_cell_key): a fold or other aggregate, a formula result, or CellId::NonItem, a stored value whose coordinate names no item of its category. Asking only at open let Enter-to-advance commit onto a collapsed group row (improvise-enz6), and projecting a NonItem cell as Record made the host refuse a write the projection had offered. It reads the CellId the projection uses, and the browser declined_at matches on that CellId, so both front ends refuse the same cells.').
decision('improvise-i5z6',compare_the_slowest_binary_to_the_next_slowest,'The ratio to the median test binary is degenerate: most binaries print 0.00s at two decimals, so the median is zero and every ratio to it undefined. An outlier is instead a binary that stays well clear of the next few slowest in every run. improvise-formula lib was read that way (longest of 39 in all three runs) and had improvise-8shu own cause: a generator re-parsing its pest grammar per case, fixed by reading it once through a OnceLock (0.84 to 0.98s down to 0.06 to 0.07s).').
decision('pr-landing-2026-09-24',check_base_ref_before_merging,'Before merging ANY pull request, its baseRefName must be main. A stacked PR whose base PR has already merged still targets the stale base branch, and merging it lands the change there, not in main, while the merge reads as done. On 2026-09-24 fifteen approved stacked PRs in #255 to #286 merged into a base whose own PR had already merged; recovery took seven landing PRs (#289 to #295), each from a stack tip against main, whose titles name exactly those fifteen. Four more in that range (#259, #263, #272, #273) merged into a non-main base whose PR was still open and reached main with it, which is stacking working as intended. The repository reported delete_branch_on_merge true when measured that day, and the stale bases were still on GitHub afterwards, so that setting is not the guard: the check is. Retarget with gh pr edit <n> --base main once the base PR has landed, then read the diff again. bd memory stacked-pr-merges-into-stale-base (which says sixteen).').
decision('improvise-3uow',ci_takes_its_tools_from_the_ci_dev_shell,'User rulings 2026-09-24. CI runs on self-hosted NixOS microVM runners, and every run: step runs inside nix develop .#ci through the workflow defaults.run.shell; a uses: step (an action) does not go through a shell at all. A missing tool goes into the .#ci shell in flake.nix, never into a per-step nix shell wrapper. No hard-coded /bin or /usr/bin paths, since those hold only env and sh there. No bash-version floors, no tool-version probes: scripts and tests must not care which bash or standard utilities are present, so checkers.sh lost its bash 5.1 pin and the dev-shell toolchain-pin step left ci.yml (#297). Playwright browsers come from nix via PLAYWRIGHT_BROWSERS_PATH, so there is no browser install step; npm ci stays for @playwright/test. A test that fakes an environment builds it on purpose (a utilbin of linked tools) rather than assuming the runner holds anything. bd memory ci-is-version-agnostic.').
decision('improvise-sboz',a_formula_edit_maps_display_names_to_ids_when_the_editor_opens,'ADR 007 clause 12 (the ADR own reading, accepted with revision 8 on PR #210). The formula panel shows display names (answer 19); the display-name-to-id mapping for every name the panel showed in that formula, the target included, is captured when the formula editor OPENS, and the commit resolves the edited text through it -- the capture-at-open rule OpenedCell already follows for cells (decision improvise-8xq0). Mapping at commit fails both ways under a concurrent rename: a name that no longer resolves refuses the edit (contradicting clause 6), and a name now owned by something else silently rebinds it (the glitch answer 3 exists to prevent). A name typed that was not captured is resolved at commit and refused, naming it, if it does not resolve; a captured id gone at commit is Rejection::Gone (clause 7); was is still compared against the stored id-form text, so a concurrent replace is still seen. Required regression test: open on Profit = Revenue - Cost, rename Revenue to Sales and another item to Revenue from a second client, commit unedited; it must still name the original id and not be refused -- failing under commit-time resolution and under refuse-on-rename.').
decision('improvise-gtn7.1',scroll_measured_where_drawn,'A scroll offset is chosen against the same width the widget is drawn in, from one function both sides call, never from two derivations that agree by coincidence. For the grid that is viewport::grid_width; App::side_panel_open is the one answer to whether a panel took its column, so a panel that did not count toward the width would not be drawn either. For the tile bar it is tiles_left(start), where first_tile measures from and where the draw begins. context/code/tui.md, The grid width is one answer.').
decision('improvise-cgqt',import_collisions_are_asked_not_defaulted,'PR #304, merge 4f38a35e. When records would write the same cell, the import wizard asks: ResolveDuplicates offers sum, keep separate under a new category, or go back, and appears only when ImportPipeline::collisions() is non-empty, after the field and date steps and before DefineFormulas. Silently summing was improvise-2cw9 fix for last-row-wins, and it too was a default the user never chose. The headless --no-wizard import takes the answer as --on-duplicate refuse|sum|separate. context/code/formula-and-io.md, Colliding records are a question, not a default.').
decision('improvise-cgqt',occurrence_category_goes_on_axis_none,'Under DuplicatePolicy::Separate the build adds the occurrence category with items 1..=deepest collision and sets it to Axis::None in the active view, so the default grid still shows each tuple total and nothing new is pivoted. A record that collides with nothing is occurrence 1 (improvise-cgqt, improvise-9bit).').
decision('improvise-felo.1',category_cap_has_one_home,'The regular-category cap is improvise_core::model::MAX_CATEGORIES and nothing else. Until felo.1 the import wizard kept a private const of its own, so raising the model number alone would have left the wizard refusing a 13th category the model accepts -- a derivation stated twice that agreed only by coincidence. The only place the literal 24 is spelled is the test add_category_max_limit; every other test that fills to the cap derives it. No recorded reason for 12 was found; the user raised it on 2026-09-25 (12 categories is feeling small, we should increase to 24).').
wire_carries(command,both_directions,'user intent up, projection commands down').
wire_carries(snapshot,on_connect_only,'improvise-q08: full ViewState + render cache filled from current viewport').
wire_carries(command,both_directions,'user intent client->server, projections server->client').
@@ -681,7 +714,6 @@ kb_issue_found(static_str_blocks_deserialize,'a borrowed &\'static str field mak
kb_issue_found(ticket_premise_false_8uz,'improvise-8uz asserted selected/row_offset/col_offset were \"PERSISTED in .improv\". They were not: improv.pest has no cursor production and format_md emitted none. The claim shaped the ticket\'s whole option set, so it was checked with a test before any of the options was weighed.').
kb_issue_found(ticket_premise_false_rml,'improvise-rml called category_panel.rs an \"exact copy\" of tile_bar.rs. It was not: the colors were identical but the labels differed (\"Row ↕\" vs \"Row\", \"None ∅\" vs \"·\"). Only the color table was true duplication; the four label sets are four deliberate vocabularies and were preserved as four methods.').
kb_issue_found(duplicated_active_view,'adding ViewState::active_view created a second copy of \"which view is active\". The improvise-8uz characterization test caught the desync immediately because it called workbook.switch_view directly. Held in lockstep by two writers until improvise-jytm, which made the divergence deliberate: the guard test the_two_active_view_names_never_diverge was REPLACED, not deleted, by the_documents_default_view_outlives_this_clients_switches and a_fresh_session_opens_the_documents_default_view -- a deliberate split with no guard is how one silently regresses into the other.').
kb_issue_found(copilot_stopped_reviewing_this_repo,'Measured 2026-09-20: PRs #12, #21 and #27 carry 2, 8 and 4 copilot-pull-request-reviewer[bot] reviews; #61, #66, #68 and #70 carry none. Both request paths report success and register nothing -- the GraphQL requestReviews mutation with botIds BOT_kgDOCnlnWA returns clientMutationId null, and POST /pulls/<n>/requested_reviewers with copilot-pull-request-reviewer[bot] returns 200 with requested_reviewers empty -- and no review_requested timeline event appears. Cause unknown; something changed between #27 and #61. Consequence for the process: AGENTS.md said \"merge only once Copilot passes\", which is an instruction to wait forever on a review that was never requested, since an ABSENT review and an UNREQUESTED one look identical from outside. That is doctrine(review_is_requested_not_inherited) seen from the requesters side, and doctrine(an_advisory_gate_is_not_a_gate) with the gate removed rather than never added. AGENTS.md now says review is an adversarial agent the orchestrator dispatches, naming the head SHA, and keeps the Copilot mechanics for the day it returns.').
kb_issue_found(github_build_setup_injects_into_the_wrong_job,'dist github-build-setup is documented as running extra steps before dist own build step, and is the obvious way to give a release job a wasm toolchain. Measured on dist 0.30.4 by injecting a named probe step and regenerating: it lands in build-local-artifacts ONLY -- the four-platform matrix job -- and never in build-global-artifacts, which is where a GLOBAL [[dist.extra-artifacts]] is actually built. So it cannot provision anything for a global extra artifact, and the provisioning must live inside the build command itself. The hook reports success and simply lands elsewhere: nothing distinguishes \"injected somewhere useless\" from \"not injected\", which is doctrine(an_empty_actions_log_is_retention_not_history) in its configuration form. NOTE ON CITING: the proposing agent named that doctrine from memory and invented a plausible key for a fact that exists under a different one -- the kb form of the dead-ticket-ID hazard. scripts/kb refused the delta over it, which is the gate doing its job; grep a key before citing it.').
kb_issue_found(minibufcolor_roles_are_not_what_they_look_like,'MinibufColor Green/Cyan/Yellow was read by the orchestrator as editing/destructive/search. BOTH HALVES ARE WRONG at the eight MinibufferConfig construction sites. Cyan is the FORMULA prompt; there is no search minibuffer at all, SearchMode being Normal plus a flag (gotcha 14) with the search MATCH drawn Black-on-Yellow in the grid. And green/yellow does not split on editing-vs-prompting: item_add is GREEN while category_add is YELLOW, the same act -- typing a name into the document -- on opposite sides. Consequence for improvise-nn4s: its own cheapest proposed step, renaming the three variants to roles, CANNOT BE DONE HONESTLY, because no three-way split fits eight sites; the honest move is a face per text-entry mode, eight faces with three resolving alike today. The general point: a colour-named enum does not merely fail to be themeable, it actively MISLEADS the next reader about what the roles are -- the orchestrator read roles off the colour names and was wrong twice in one sentence.').
kb_issue_found(stale_count_in_tui_md_default_chrome_arm,'context/code/tui.md line 125 says mode_style spells out \"the fourteen variants that take the default chrome\". The arm lists THIRTEEN -- verified by the orchestrator against src/draw.rs: Normal, RecordsNormal, FormulaEdit, FormulaPanel, CategoryPanel, CategoryAdd, ItemAdd, ViewPanel, ImportWizard, ExportPrompt, SavePrompt, Help, Quit. 13 + Editing + RecordsEditing + CommandMode + TileSelect is the 17 AppMode variants, so the arithmetic confirms thirteen. Nothing checks a count written in prose, which is why it survived; the exhaustive match itself is sound and only the sentence about it is wrong.').
@@ -698,6 +730,9 @@ kb_issue_found(a_layout_measured_before_the_status_settles_is_not_the_layout,'im
kb_issue_found(check_evidence_cannot_see_a_templated_test_name,'scripts/check-comment-runs.py --check-evidence scrapes LITERAL test(\"...\") strings out of *.spec.js, so a Playwright test named by a template literal -- test(\`at ${width}px ...\`) -- is invisible to it and its evidence heading reports as an orphan. improvise-tg27 hit this and restructured to five literally-named tests, each looping widths internally with the width in the assertion message. Parameterise INSIDE one literally-named test rather than looping test().').
kb_issue_found(k844_description_was_verbatim_e8no_body,'improvise-k844 stored description was a verbatim copy of improvise-e8no body -- the wide-numeric-category budget -- contradicting its own title. Confirmed by diffing the two DESCRIPTION blocks: identical. The agent worked the TITLE, which was right, and the defect had one good consequence: reading e8no text while working k844 is what made it check whether e8no was done, and e8no turned out to be STALE-OPEN with both its guard tests passing. A ticket carrying the wrong body is worse than an empty one, because an agent that trusts it works the wrong problem while believing it has a specification.').
kb_issue_found(the_old_checker_gate_overrode_every_shebang,'The gate in the orchestrator brief -- \`for t in scripts/*-test.sh; do sh \"$t\"; done\` -- OVERRIDES EVERY SHEBANG with /bin/sh, so on macOS it ran every suite under bash 3.2 in sh-mode regardless of what the file declared. The gate was the disease it was meant to catch. Seven further sites ran a repo script through an explicit \`sh\`, and ONE OF THEM IS dist-workspace.toml, which a first-line shebang gate STRUCTURALLY CANNOT SEE: the shebang rule closes DECLARATIONS and never INVOCATIONS, and that one was found by grepping for the CALL after fixing the declaration, not by the gate going green. Also: scripts/node-test.sh matches *-test.sh and is a test RUNNER, so a naive glob ran it with no arguments and it printed its usage INSIDE A GREEN GATE -- the rule is now that a suite is X-test.sh where X or X.sh exists beside it.').
kb_issue_found(copilot_stopped_reviewing_this_repo,'Measured 2026-09-20: PRs #12, #21 and #27 carry 2, 8 and 4 copilot-pull-request-reviewer[bot] reviews; #61, #66, #68 and #70 carry none. Both request paths reported success and registered nothing -- the GraphQL requestReviews mutation with botIds BOT_kgDOCnlnWA returned clientMutationId null, and POST /pulls/<n>/requested_reviewers with copilot-pull-request-reviewer[bot] returned 200 with requested_reviewers empty -- and no review_requested timeline event appeared. Consequence for the process at the time: AGENTS.md said \"merge only once Copilot passes\", which is an instruction to wait forever on a review that was never requested, since an ABSENT review and an UNREQUESTED one look identical from outside. That is doctrine(review_is_requested_not_inherited) seen from the requesters side. THE LAPSE ENDED: on 2026-09-24 Copilot posted inline findings on #221, #251, #274, #295 and #298, so this issue is history, not the present. AGENTS.md now says its findings are read and answered like Codex, through scripts/pr-reviews because most arrive as inline comments that gh pr view --json reviews does not show, and still that nobody waits on it: the review that gates a merge is an adversarial agent given the head SHA.').
kb_issue_found(tui_grid_width_bug_latent_since_b5418f2e,'improvise-gtn7.1 looked like a regression and was not. compute_visible_cols took term_width from b5418f2e (2026-04-06), so the keypress path was always wrong with a side panel open; nothing opened one at launch until 50e4a372 (2026-09-20) put category_panel_open in the shared ViewState::default(), and from then every TUI session started in the broken configuration. A default changed in a shared type is a new configuration for every consumer, and a latent bug in that configuration surfaces as if the default change caused it. The tile bar offset had been wrong since 4686f470, when it first scrolled.').
kb_issue_found(record_row_group_is_quadratic_in_records,'RecordRow::group (improvise-core view/layout.rs) finds each cell row with rows.iter().position, a linear scan, so the records-mode build is O(cells x rows). Found by the felo.1 width benchmark: the records build grows about 4x when the records double, at 12 and at 24 categories alike, so it is not category coupling. Filed as improvise-sk32.').
functional(invariant/2).
functional(location/3).
functional(doctrine/2).
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Run git with the commit AUTHOR named after the agent: `Claude ($BEADS_ACTOR)`.
# Why GIT_AUTHOR_NAME and not `-c user.name`: .agents/skills/wave/agent-rules.md.
set -euo pipefail
if [ -z "${BEADS_ACTOR:-}" ]; then
echo 'agent-git: BEADS_ACTOR is unset; refusing to author as the owner' >&2
echo 'usage: BEADS_ACTOR=<actor> scripts/agent-git commit|merge ...' >&2
exit 2
fi
GIT_AUTHOR_NAME="Claude ($BEADS_ACTOR)" exec git "$@"
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env bash
# check_the_checker for scripts/agent-git (improvise-1xfb.3). A throwaway repo
# whose only identity is its own config; no global or system git config.
set -uo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RUN="$here/agent-git"
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1
unset GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL
pass=0; fail=0
ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); }
bad() { printf ' FAIL %s (%s)\n' "$1" "$2"; fail=$((fail+1)); }
# $1 name, $2 wanted, $3 got
same() { if [ "$2" = "$3" ]; then ok "$1"; else bad "$1" "got '$3', wanted '$2'"; fi; }
repo="$work/repo"
git init -q "$repo"
git -C "$repo" config user.name Owner
git -C "$repo" config user.email owner@example.com
git -C "$repo" commit -q --allow-empty -m base
cd "$repo" || exit 1
base="$(git rev-parse HEAD)"
# The refusals: nothing may be committed, since the commit would be the owner's.
(unset BEADS_ACTOR; "$RUN" commit -q --allow-empty -m unset) > "$work/out" 2>&1
same "an unset BEADS_ACTOR is refused" 2 "$?"
grep -q 'BEADS_ACTOR is unset' "$work/out" && ok "the refusal names BEADS_ACTOR" \
|| bad "the refusal names BEADS_ACTOR" "$(head -c 200 "$work/out")"
BEADS_ACTOR='' "$RUN" commit -q --allow-empty -m empty > /dev/null 2>&1
same "an empty BEADS_ACTOR is refused" 2 "$?"
same "a refusal commits nothing" "$base" "$(git rev-parse HEAD)"
# A commit: the author is the agent, and nothing else about the identity moves.
BEADS_ACTOR=agent-a1 "$RUN" commit -q --allow-empty -m one
same "the author is Claude (<actor>)" 'Claude (agent-a1)' "$(git log -1 --format=%an)"
same "the author email is the configured one" owner@example.com "$(git log -1 --format=%ae)"
same "the committer is untouched" 'Owner <owner@example.com>' "$(git log -1 --format='%cn <%ce>')"
same "shared config is untouched" Owner "$(git config user.name)"
# A merge commit is authored too; a plain amend keeps the agent as author.
git checkout -q -b side "$base"
BEADS_ACTOR=agent-b2 "$RUN" commit -q --allow-empty -m side
git checkout -q -
BEADS_ACTOR=agent-a1 "$RUN" merge -q --no-edit side
same "a merge commit is authored by the agent" 'Claude (agent-a1)' "$(git log -1 --format=%an)"
same "the merge has two parents" 2 "$(git log -1 --format=%P | wc -w | tr -d ' ')"
# A cherry-pick keeps its source's author, whatever the wrapper sets.
git checkout -q -b pick "$base"
BEADS_ACTOR=agent-a1 "$RUN" cherry-pick -x --allow-empty "$(git rev-parse side)" > /dev/null 2>&1
same "a cherry-pick keeps its source's author" 'Claude (agent-b2)' "$(git log -1 --format=%an)"
git checkout -q -
# git's own failure is passed through, not swallowed.
BEADS_ACTOR=agent-a1 "$RUN" commit -q -m nothing-staged > /dev/null 2>&1
rc=$?
[ "$rc" != 0 ] && ok "git's own failure is passed through" \
|| bad "git's own failure is passed through" "exit 0"
printf '\nagent-git-test: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" = 0 ]
+38 -8
View File
@@ -12,7 +12,9 @@ Run from anywhere inside the repository. Takes no arguments.
exit 0 every cited id is a ticket bd lists
exit 1 at least one cited id is not; each is printed
exit 3 bd could not answer: missing, failing, or listing nothing
exit 4 usage error, or the file scan itself failed
exit 4 usage error, not inside a git repository, or a cited file
could not be read or is absent (only a file `git diff
--diff-filter=D` reports as deleted is skipped)
The comparison is POSITIVE: a cited id passes only by appearing in what
`bd list --all` returns. The recipe this replaced looked for bd's "not found"
@@ -36,15 +38,41 @@ EXCLUDE='^improvise-(core|formula|io|protocol|command|gp|ws|worker|wasm|harness|
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT
cd "$(git rev-parse --show-toplevel)"
git ls-files -z --cached --others --exclude-standard -- "${ROOTS[@]}" > "$work/files"
top=$(git rev-parse --show-toplevel 2>/dev/null) || {
echo "bd-audit-ids: not inside a git repository, so nothing was checked." >&2
exit 4
}
cd "$top"
# A deleted tracked file is skipped ON PURPOSE: its ids are leaving the tree. Only
# `diff --diff-filter=D` says deleted; any other absent path is exit 4 (tdfd, cota).
: > "$work/absent"
git ls-files -z --cached --others --exclude-standard -- "${ROOTS[@]}" |
while IFS= read -r -d '' f; do
if [ -e "$f" ] || [ -L "$f" ]; then printf '%s\0' "$f"
elif [ -z "$(git --literal-pathspecs diff --name-only --diff-filter=D -- "$f")" ]; then
printf '%s\n' "$f" >> "$work/absent"
fi
done > "$work/files"
if [ -s "$work/absent" ]; then
echo "bd-audit-ids: tracked but absent, and not deleted, so never read:" >&2
sed 's/^/ /' "$work/absent" >&2
exit 4
fi
# xargs exits 123 when one grep batch matched nothing, which is not an error.
# xargs reports grep's no-match (1) and its read error (2) alike, as 123, so
# each batch maps 1 to success and 2 to 255: xargs stops, nonzero (GNU 124, BSD 1).
: > "$work/raw"
set +e
xargs -0 grep -d skip -ohE 'improvise-[a-z0-9]{3,8}\b' < "$work/files" > "$work/raw"
if [ -s "$work/files" ]; then
xargs -0 bash -c 'grep -d skip -ohE "$0" "$@"; [ $? -le 1 ] || exit 255' \
'improvise-[a-z0-9]{3,8}\b' < "$work/files" > "$work/raw"
fi
rc=$?
set -e
[ "$rc" -eq 0 ] || [ "$rc" -eq 123 ] || { echo "bd-audit-ids: grep failed ($rc)" >&2; exit 4; }
[ "$rc" -eq 0 ] || {
echo "bd-audit-ids: a cited file could not be read (xargs $rc), so the audit is incomplete." >&2
exit 4
}
sort -u "$work/raw" | { grep -vE "$EXCLUDE" || true; } > "$work/cited"
if ! command -v "$BD" >/dev/null 2>&1; then
@@ -56,10 +84,12 @@ if ! "$BD" list --all --limit 0 --json > "$work/known.json" 2> "$work/bd.err"; t
cat "$work/bd.err" >&2
exit 3
fi
if ! jq -r '.[].id' "$work/known.json" 2>/dev/null | sort -u > "$work/known"; then
echo "bd-audit-ids: '$BD list' did not print a JSON list of issues." >&2
if ! jq -e 'type == "array" and all(.[]; (.id | type) == "string")' \
"$work/known.json" > /dev/null 2>&1; then
echo "bd-audit-ids: '$BD list' did not print a JSON list of issues with ids." >&2
exit 3
fi
jq -r '.[].id' "$work/known.json" | sort -u > "$work/known"
if [ ! -s "$work/known" ]; then
echo "bd-audit-ids: '$BD list' returned no issues; refusing to call that a pass." >&2
exit 3
+78
View File
@@ -77,6 +77,84 @@ got=$?
[ "$got" = 3 ] && ok "the real tree does not pass against a silent bd" \
|| bad "the real tree does not pass against a silent bd" "exit $got"
# improvise-lqs0: a file the scan cannot read, and the two exit-code mismatches.
# A second repo, so the file list is exactly these cases' files.
r2="$work/repo2"
mkdir -p "$r2/context"
git -C "$r2" init -q
printf 'improvise-ab12\n' > "$r2/context/a.md"
printf 'improvise-dead9\n' > "$r2/context/b.md"
git -C "$r2" add -A
git -C "$r2" -c user.name=t -c user.email=t@t commit -qm fixture
export FAKE_BD_OUT="$work/known.json" FAKE_BD_RC=0
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
[ "$got" = 1 ] && grep -q 'improvise-dead9' "$work/out" \
&& ok "control: both files readable, the dead id is found" \
|| bad "control: both files readable, the dead id is found" "exit $got"
# A symlink whose target is gone reads as an error for any user, root included.
ln -s missing-target "$r2/context/c.md"
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
[ "$got" = 4 ] && ok "a cited file grep cannot read is exit 4, not a pass" \
|| bad "a cited file grep cannot read is exit 4, not a pass" "exit $got: $(head -c 200 "$work/out")"
rm -f "$r2/context/c.md"
if [ "$(id -u)" != 0 ]; then
chmod 000 "$r2/context/b.md"
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
chmod 644 "$r2/context/b.md"
[ "$got" = 4 ] && ok "an unreadable cited file is exit 4, not a pass" \
|| bad "an unreadable cited file is exit 4, not a pass" "exit $got: $(head -c 200 "$work/out")"
fi
# Decided, not accidental: a tracked file deleted from the working tree is not
# scanned. Its ids are leaving the tree, so its dead id is not reported.
mv "$r2/context/b.md" "$work/b.md.away"
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
mv "$work/b.md.away" "$r2/context/b.md"
[ "$got" = 0 ] && ok "a tracked file deleted from the working tree is not scanned" \
|| bad "a tracked file deleted from the working tree is not scanned" "exit $got: $(head -c 200 "$work/out")"
# Absent is not deleted: a skip-worktree path is missing too, but ls-files -d
# does not list it, so its ids were never read (improvise-tdfd).
git -C "$r2" update-index --skip-worktree context/b.md
mv "$r2/context/b.md" "$work/b.md.away"
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
mv "$work/b.md.away" "$r2/context/b.md"
git -C "$r2" update-index --no-skip-worktree context/b.md
[ "$got" = 4 ] && grep -q 'context/b.md' "$work/out" \
&& ok "a skip-worktree file missing from the tree is exit 4, and named" \
|| bad "a skip-worktree file missing from the tree is exit 4, and named" "exit $got: $(head -c 200 "$work/out")"
# ls-files -d DOES list an absent assume-unchanged path, yet status hides it and
# commit -a never records the deletion (improvise-cota).
git -C "$r2" update-index --assume-unchanged context/b.md
mv "$r2/context/b.md" "$work/b.md.away"
(cd "$r2" && "$AUDIT") > "$work/out" 2>&1
got=$?
mv "$work/b.md.away" "$r2/context/b.md"
git -C "$r2" update-index --no-assume-unchanged context/b.md
[ "$got" = 4 ] && grep -q 'context/b.md' "$work/out" \
&& ok "an assume-unchanged file missing from the tree is exit 4, and named" \
|| bad "an assume-unchanged file missing from the tree is exit 4, and named" "exit $got: $(head -c 200 "$work/out")"
mkdir -p "$work/not-a-repo"
(cd "$work/not-a-repo" && GIT_CEILING_DIRECTORIES="$work" "$AUDIT") > "$work/out" 2>&1
got=$?
[ "$got" = 4 ] && ok "run outside a git repository is exit 4" \
|| bad "run outside a git repository is exit 4" "exit $got"
printf '[{"title":"no id here"}]\n' > "$work/noid.json"
FAKE_BD_OUT="$work/noid.json" \
run_case "a bd listing issues with no id is exit 3, not every id dead" 3 'did not print'
unset FAKE_BD_OUT FAKE_BD_RC
"$AUDIT" --help > "$work/out" 2>&1
got=$?
[ "$got" = 4 ] && grep -q 'POSITIVE' "$work/out" \
+1
View File
@@ -45,6 +45,7 @@ const SECTIONS = [
["context/adr/002-naming-a-cell.md", "adr/002-naming-a-cell", "ADR 002 — Naming a cell"],
["context/adr/003-live-backed-models.md", "adr/003-live-backed-models", "ADR 003 — Live-backed models"],
["context/adr/004-faces-and-themes.md", "adr/004-faces-and-themes", "ADR 004 — Faces"],
["context/adr/007-stable-identities.md", "adr/007-stable-identities", "ADR 007 — Stable identities"],
],
},
{
+648 -6
View File
@@ -6,16 +6,26 @@ README name three of seven while the rest read as ungated (improvise-x7cq).
Stating it was not enough -- a new job was added during review of PR #83 and
nothing noticed, so the rule is a gate. See context/code/tooling.md.
Second rule: every suite `scripts/checkers.sh --list` knows must run in some
CI step. ci.yml names its suites by hand, so a new `*-test.sh` ran locally and
never in CI until someone remembered to add it (improvise-dak5). Each job runs
`checkers.sh --ci <job>`, whose table lives in checkers.sh; no shell is parsed
here, and what a row runs is asked of `--would-run` (improvise-maax, st8q).
`--self-test` checks the checker: each case is a graph this must refuse.
"""
import os
import re
import subprocess
import sys
import tempfile
WORKFLOW = ".github/workflows/ci.yml"
DEPLOY = "pages-deploy"
_JOB = re.compile(r"^ ([A-Za-z0-9_-]+):$")
_JOB_OPEN = re.compile(r"^ ([A-Za-z0-9_-]+):(\s|$)")
_NEEDS = re.compile(r"^ needs:\s*\[(.*)\]\s*$")
@@ -24,7 +34,9 @@ def read_graph(text):
Both come off indentation, which is what a workflow's own shape is. A job
key is two spaces; `needs:` is four INSIDE the job it belongs to, so the
current job is tracked rather than the first `needs:` taken.
current job is tracked rather than the first `needs:` taken. A job opens on
`_JOB_OPEN`, wider than `_JOB`: this reader refuses nothing, so it reads wide
(improvise-j49r).
"""
jobs, needs, current = [], None, None
in_jobs = False
@@ -34,7 +46,7 @@ def read_graph(text):
continue
if not in_jobs:
continue
m = _JOB.match(raw)
m = _JOB_OPEN.match(raw)
if m:
current = m.group(1)
jobs.append(current)
@@ -60,6 +72,611 @@ def verdict(jobs, needs):
return out
_JOB_KEY = re.compile(r"^ ([A-Za-z_][\w-]*):(.*)$")
_STEP_START = re.compile(r"^ - ")
_STEP_KEY = re.compile(r"^ (?:- | )([A-Za-z_][\w-]*):(.*)$")
_ANY_KEY = re.compile(r"^(\s*)(?:- )?([A-Za-z_][\w-]*):(.*)$")
_CI_CALL = re.compile(r"^(?:nix shell \S+ --command )?\./scripts/checkers\.sh --ci ([\w-]+)$")
_DISQUALIFY = ("if", "continue-on-error", "shell")
_NAME = r"[A-Za-z_][\w-]*"
_TOP_KEY = re.compile(rf"^{_NAME}:(\s|$)")
_NEEDS_VALUE = re.compile(rf"^(|{_NAME}|\[\s*{_NAME}(\s*,\s*{_NAME})*\s*\])$")
_NEEDS_ITEM = re.compile(rf"^ - {_NAME}$")
_COMMENT = re.compile(r"\s#")
_UNSAFE_CHAR = re.compile("[\x00-\x08\x0b-\x1f\x7f-\x9f\u2028\u2029]")
def _readable(line, indent, in_steps, in_needs, job_keyed):
"""Whether a structural line inside `jobs:` is one this gate can read. Anything
else is refused, not parsed: context/code/tooling.md (improvise-w0j4, rjaa)."""
if indent == 2:
return bool(_JOB.match(line))
if indent == 4:
m = _JOB_KEY.match(line)
return bool(m) and (m.group(1) != "needs"
or bool(_NEEDS_VALUE.match(_inline_value(m.group(2)))))
if in_needs:
return bool(_NEEDS_ITEM.match(line))
if in_steps:
return indent in (6, 8) and bool(_STEP_KEY.match(line))
return indent > 4 and job_keyed
def _inline_value(rest):
"""A key's inline value. No comment is stripped: a line carrying one is
refused, so every matcher reads the same string (improvise-j49r)."""
return rest.strip()
_FLOW_LINE = re.compile(r"""^(\[[^\[\]{}"'#]*\]|\{[^\[\]{}"'#]*\})$""")
_BLOCK_HEADER = re.compile(r"^[|>][-+0-9]*$")
def _plain_value(value):
"""Whether a value is one of the few shapes the gate reads, all ending on their own
line; anything else is refused: context/code/tooling.md (improvise-0wn3)."""
if value == "" or _BLOCK_HEADER.match(value) or _FLOW_LINE.match(value):
return True
if value[0] in "\"'":
return not _opens_quote(value)
return value[0] not in "[]{}&*!|>%@`?,"
def _line_value(raw):
"""The value a line carries: a key's, a list item's, or the whole line's."""
k = _ANY_KEY.match(raw)
if k:
return _inline_value(k.group(3))
body = raw.strip()
return body[2:].strip() if body.startswith("- ") else body
def _opens_quote(value):
"""True when a quoted inline value is not closed on its own line."""
if value[:1] not in ("'", '"') or len(value) < 2:
return value[:1] in ("'", '"')
q, body = value[0], value[1:]
if q == "'":
return not body.replace("''", "").endswith("'")
return not re.sub(r'\\.', "", body).endswith('"')
def ci_steps(text):
"""(job, value, disqualifiers) for every step of every job, and every line
mentioning scripts/checkers.sh that is not a step's whole `run:` value.
Nothing is read as shell. A key with an inline value owns every following
line indented deeper, which is how a block scalar, a folded value, a plain
continuation or a heredoc inside one is skipped whole (improvise-st8q)."""
steps, stray, unread = [], [], []
job, in_steps, job_flags, step = None, False, set(), None
in_jobs, job_keyed = False, False
owner, owner_step = None, None
needs, if_jobs, in_needs = {}, set(), False
def close_step():
if step is not None:
steps.append(step)
for n, raw in enumerate(text.split("\n"), 1):
if _UNSAFE_CHAR.search(raw):
unread.append((n, "char"))
continue
indent = len(raw) - len(raw.lstrip())
if owner is not None and (not raw.strip() or indent > owner):
if "scripts/checkers.sh" in raw:
stray.append(n)
if owner_step is not None and raw.strip():
owner_step["multiline"] = True
continue
owner, owner_step = None, None
if not raw.strip() or raw.lstrip().startswith("#"):
continue
if indent == 0:
in_jobs = raw.startswith("jobs:")
readable = bool(_TOP_KEY.match(raw))
else:
readable = not in_jobs or indent > 8 or _readable(raw, indent, in_steps,
in_needs, job_keyed)
k = _ANY_KEY.match(raw)
shaped = _plain_value(_line_value(raw))
if _COMMENT.search(raw) or not shaped or not readable:
unread.append((n, "comment" if _COMMENT.search(raw) else
"plain" if not readable else "value"))
m = _JOB.match(raw)
if m:
close_step()
step = None
job, in_steps, job_flags, in_needs = m.group(1), False, set(), False
job_keyed = False
continue
if in_needs and re.match(r"^ - ", raw):
needs[job].add(_inline_value(raw.strip()[2:]))
continue
m = _JOB_KEY.match(raw)
if m and job is not None:
job_keyed = True
close_step()
step = None
key, value = m.group(1), _inline_value(m.group(2))
in_steps = key == "steps" and value == ""
in_needs = key == "needs" and value == ""
if key == "needs":
needs[job] = {x.strip() for x in value.strip("[]").split(",") if x.strip()}
if key == "if":
if_jobs.add(job)
if key in _DISQUALIFY:
job_flags.add(key)
for s in steps:
if s["job"] == job:
s["flags"].add("job " + key)
if in_steps and _STEP_START.match(raw):
close_step()
step = {"job": job, "run": None, "flags": {"job " + f for f in job_flags},
"line": n}
m = _STEP_KEY.match(raw) if in_steps and step is not None else None
if m:
key, value = m.group(1), _inline_value(m.group(2))
if key == "run":
step["run"] = value
if "scripts/checkers.sh" in raw and not _CI_CALL.match(value):
stray.append(n)
elif key in _DISQUALIFY:
step["flags"].add(key)
elif "scripts/checkers.sh" in raw:
stray.append(n)
elif "scripts/checkers.sh" in raw:
stray.append(n)
if k and _inline_value(k.group(3)):
owner = len(k.group(1)) + (2 if raw.lstrip().startswith("- ") else 0)
owner_step = step if m and m.group(1) == "run" else None
close_step()
for s in steps:
s["flags"] |= {f"needs {j}, which has if" for j in _needed(s["job"], needs) & if_jobs}
return steps, stray, unread
def _needed(job, needs):
"""Every job `job` needs, transitively: a skipped one skips it too (improvise-bbwa)."""
seen, todo = set(), list(needs.get(job, ()))
while todo:
j = todo.pop()
if j not in seen:
seen.add(j)
todo += needs.get(j, ())
return seen
def would_run(args):
"""What checkers.sh itself says these arguments run. Its argument rules and
its CI table are not restated here (improvise-maax)."""
out = subprocess.run(["scripts/checkers.sh", "--would-run", *args],
capture_output=True, text=True).stdout
ran, missing = set(), set()
for line in out.split("\n"):
if line.startswith("would-run: "):
ran.add(line[len("would-run: "):])
elif line.startswith("missing: "):
missing.add(line[len("missing: "):])
return ran, missing
def _unrun(s):
return (f"`{s}` is a checkers.sh suite that no CI step runs; add it to a row of "
"ci_suites in scripts/checkers.sh")
def _unread(n, kind="plain"):
if kind == "char":
return (f"ci.yml line {n} holds a control character or a line break YAML sees and "
"this gate does not (U+0085, U+2028, U+2029); remove it")
if kind == "value":
return (f"ci.yml line {n} has a value that is not one plain line (a multi-line "
"quote or flow, a nested or quoted flow, an anchor, alias or tag); spell "
"it plainly")
if kind == "comment":
return (f"ci.yml line {n} has a ` #`, which YAML may read as a comment; this gate "
"reads no comments, so put it on a line of its own")
return (f"ci.yml line {n} is a key or `needs:` value this gate cannot read as a "
"plain name, so it cannot trust the graph; spell it plainly")
def suite_verdict(suites, text):
"""The complaints: suites no counted step runs, and every step or line that
does not count, each named so the fix is obvious rather than silent."""
steps, stray, unread = ci_steps(text)
out, ran, missing = [_unread(n, c) for n, c in unread], set(), set()
for s in steps:
m = _CI_CALL.match(s["run"] or "")
if not m:
continue
if m.group(1) != s["job"]:
out.append(f"job `{s['job']}` runs `checkers.sh --ci {m.group(1)}`; "
"a job runs only its own row of the CI table")
elif s.get("multiline"):
out.append(f"job `{s['job']}`'s `checkers.sh --ci` step's run: continues "
"onto another line, so it does not count")
elif s["flags"]:
out.append(f"job `{s['job']}`'s `checkers.sh --ci` step is conditional "
f"({', '.join(sorted(s['flags']))}), so it does not count")
else:
r, mi = would_run(["--ci", s["job"]])
ran |= r
missing |= mi
out += [f"ci.yml line {n} mentions scripts/checkers.sh outside a step whose "
"whole `run:` is `./scripts/checkers.sh --ci <job>`" for n in stray]
out += [_unrun(s) for s in sorted(set(suites) - ran)]
out += [f"checkers.sh's CI table names `{s}`, which is not a suite"
for s in sorted(missing)]
return out
def read_workflow(path):
"""The workflow's text. Universal newlines fold CR and CRLF into the one break the
gate splits on; every other break YAML knows is refused (improvise-xpzh)."""
return open(path, encoding="utf8").read()
def list_suites():
"""The suites, from checkers.sh itself, so its naming rule has one home."""
out = subprocess.run(["scripts/checkers.sh", "--list"], capture_output=True,
text=True, check=True).stdout
return [s for s in out.split("\n") if s.strip()]
def _wf(*jobs):
return "jobs:\n" + "".join(jobs)
def _job(name, *steps, head=""):
return f" {name}:\n runs-on: x\n{head} steps:\n" + "".join(steps)
_BUILD = _job("build", " - run: ./scripts/checkers.sh --ci build\n")
_REQ = ["assemble-stamp", "pr-watch"]
_NO_PW = [_unrun("pr-watch")]
def _stray(n):
return (f"ci.yml line {n} mentions scripts/checkers.sh outside a step whose "
"whole `run:` is `./scripts/checkers.sh --ci <job>`")
_GATE = " gate:\n if: false\n steps:\n - run: echo\n"
_KB_RUN = " - run: ./scripts/checkers.sh --ci kb\n"
_YAML_ONLY_BREAKS = ("\x85", "\u2028", "\u2029")
_INNER_RUN = " run: ./scripts/checkers.sh --ci kb\n"
_NEEDS_GATE = ["job `kb`'s `checkers.sh --ci` step is conditional (needs gate, which has if), "
"so it does not count"] + _NO_PW
def _unreadable(name, text, *markers, rest=(), kind="plain"):
"""A case whose workflow spells a key or a `needs:` value the gate must refuse."""
lines = text.split("\n")
at = [next(i for i, l in enumerate(lines, 1) if m in l) for m in markers]
return (name, _REQ, text, [_unread(n, kind) for n in at] + list(rest))
# (name, suites to require, workflow text, the complaints it must draw). Every
# case asks the REAL checkers.sh table, where `kb`'s row holds pr-watch.
SUITE_SELF_TEST = [
("two jobs each running their own row are accepted", _REQ,
_wf(_BUILD, _job("kb", " - run: nix shell nixpkgs#trealla --command "
"./scripts/checkers.sh --ci kb\n")), []),
_unreadable("a trailing comment on the counted run: line is refused (j49r)",
_wf(_BUILD, _job("kb", _KB_RUN.rstrip("\n") + " # the kb row\n")),
"# the kb row", rest=[_stray(9)] + _NO_PW, kind="comment"),
("a job that never calls its row leaves its suites unrun", _REQ,
_wf(_BUILD, _job("kb", " - run: echo nothing\n")), _NO_PW),
("a call inside a heredoc in a block run: does not count (st8q)", _REQ,
_wf(_BUILD, _job("kb", " - run: |\n cat >> x <<'EOF'\n"
" ./scripts/checkers.sh --ci kb\n EOF\n")),
[_stray(11)] + _NO_PW),
("a folded run: does not count (st8q)", _REQ,
_wf(_BUILD, _job("kb", " - run: >\n ./scripts/checkers.sh --ci kb\n")),
[_stray(10)] + _NO_PW),
("a plain continuation line does not count", _REQ,
_wf(_BUILD, _job("kb", " - run: echo\n ./scripts/checkers.sh --ci kb\n")),
[_stray(10)] + _NO_PW),
_unreadable("a run: line inside a multi-line quoted value: the quote is refused (6zf1)",
_wf(_BUILD, _job("kb", " - name: \"a\n"
" run: ./scripts/checkers.sh --ci kb\n b\"\n")),
'- name: "a', kind="value", rest=[_unread(11)]),
("a whole job inside a top-level block scalar does not count", _REQ,
_wf(_BUILD) + "notes: |\n kb:\n steps:\n"
" - run: ./scripts/checkers.sh --ci kb\n",
[_stray(9)] + _NO_PW),
("a step-shaped line inside a job-level block scalar does not count", _REQ,
_wf(_BUILD, _job("kb", " - run: echo\n",
head=" notes: |\n - run: ./scripts/checkers.sh --ci kb\n")),
[_stray(9)] + _NO_PW),
("a step-shaped item under a job key other than steps: does not count", _REQ,
_wf(_BUILD, " kb:\n services:\n - run: ./scripts/checkers.sh --ci kb\n"
" steps:\n - run: echo\n"),
[_stray(8)] + _NO_PW),
("the call with a continuation line does not count (1be2)", _REQ,
_wf(_BUILD, _job("kb", " - run: ./scripts/checkers.sh --ci kb\n"
" || true\n")),
["job `kb`'s `checkers.sh --ci` step's run: continues onto another line, "
"so it does not count"] + _NO_PW),
("a job that needs a job under if: does not count (bbwa)", _REQ,
_wf(_BUILD, " gate:\n if: false\n steps:\n - run: echo\n",
_job("kb", " - run: ./scripts/checkers.sh --ci kb\n",
head=" needs: [build, gate]\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (needs gate, which has if), "
"so it does not count"] + _NO_PW),
("the same, with needs: written as a list", _REQ,
_wf(_BUILD, " gate:\n if: false\n steps:\n - run: echo\n",
_job("kb", " - run: ./scripts/checkers.sh --ci kb\n",
head=" needs:\n - gate\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (needs gate, which has if), "
"so it does not count"] + _NO_PW),
("the 1be2 continuation exactly one column past the key does not count (w0j4)", _REQ,
_wf(_BUILD, _job("kb", _KB_RUN + " || true\n")),
["job `kb`'s `checkers.sh --ci` step's run: continues onto another line, "
"so it does not count"] + _NO_PW),
("control: a padded flow list is still read (w0j4)", _REQ,
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: [ gate ]\n")),
_NEEDS_GATE),
("control: a plain scalar needs: is still read (w0j4)", _REQ,
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: gate\n")),
_NEEDS_GATE),
_unreadable("a quoted name in a flow needs: is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=' needs: ["gate"]\n')),
'needs: ["gate"]'),
_unreadable("a quoted scalar needs: is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: 'gate'\n")),
"needs: 'gate'"),
_unreadable("a quoted item in a block needs: is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=' needs:\n - "gate"\n')),
'- "gate"'),
_unreadable("an anchored flow needs: is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: &d [gate]\n")),
"needs: &d"),
_unreadable("an anchored block needs: is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: &d\n - gate\n")),
"needs: &d"),
_unreadable("a flow needs: over two lines is refused (w0j4)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: [build,\n gate]\n")),
"needs: [build,"),
_unreadable("a quoted job key is refused (w0j4)",
_wf(_BUILD, _job("kb", _KB_RUN, head=' "if": false\n')),
'"if"'),
_unreadable("a quoted step key is refused (w0j4)",
_wf(_BUILD, _job("kb", " - \"if\": false\n"
" run: ./scripts/checkers.sh --ci kb\n")),
'"if"'),
_unreadable("a single-quoted continue-on-error is refused (w0j4)",
_wf(_BUILD, _job("kb", _KB_RUN, head=" 'continue-on-error': true\n")),
"'continue-on-error'"),
_unreadable("a key with a space before its colon is refused (w0j4)",
_wf(_BUILD, _job("kb", _KB_RUN, head=" if : false\n")),
"if : false"),
_unreadable("a quoted job name is refused (w0j4)",
_wf(_BUILD, ' "gate":\n if: false\n steps:\n - run: echo\n',
_job("kb", _KB_RUN)),
'"gate"',
rest=["job `build`'s `checkers.sh --ci` step is conditional (job if), "
"so it does not count", _unrun("assemble-stamp")]),
_unreadable("a quoted top-level key is refused (w0j4)",
'"defaults":\n run:\n shell: bash\n' + _wf(_BUILD, _job("kb", _KB_RUN)),
'"defaults"'),
("control: a job key's block content at indent 6 is content (rjaa)", _REQ,
_wf(_BUILD, _job("kb", _KB_RUN, head=" env:\n FOO: bar\n")), []),
_unreadable("a needed job whose keys sit at indent 6 is refused (rjaa)",
_wf(_BUILD, " gate:\n if: false\n steps:\n - run: echo\n",
_job("kb", _KB_RUN, head=" needs: gate\n")),
" if: false", " steps:", " - run: echo"),
_unreadable("a needed job whose keys sit at indent 5 is refused (rjaa)",
_wf(_BUILD, " gate:\n if: false\n steps:\n - run: echo\n",
_job("kb", _KB_RUN, head=" needs: [gate]\n")),
" if: false", " steps:", " - run: echo"),
_unreadable("a job between, with its needs: at indent 6, is refused (rjaa)",
_wf(_BUILD, _GATE,
" mid:\n needs: gate\n steps:\n - run: echo\n",
_job("kb", _KB_RUN, head=" needs: mid\n")),
" needs: gate", " steps:", " - run: echo"),
_unreadable("an explicit ? key is refused (w0j4)",
_wf(_BUILD, _job("kb", _KB_RUN, head=" ? if\n : false\n")),
"? if", ": false"),
("the same, two needs: away", _REQ,
_wf(_BUILD, " gate:\n if: false\n steps:\n - run: echo\n",
" mid:\n needs: gate\n steps:\n - run: echo\n",
_job("kb", " - run: ./scripts/checkers.sh --ci kb\n",
head=" needs: [mid]\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (needs gate, which has if), "
"so it does not count"] + _NO_PW),
("a step under if: does not count (tvte)", _REQ,
_wf(_BUILD, _job("kb", " - if: false\n run: ./scripts/checkers.sh --ci kb\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (if), so it does not count"]
+ _NO_PW),
("a job under if: does not count (tvte)", _REQ,
_wf(_BUILD, _job("kb", " - run: ./scripts/checkers.sh --ci kb\n",
head=" if: false\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (job if), so it does not count"]
+ _NO_PW),
("a job-level if: after its steps still disqualifies them", _REQ,
_wf(_BUILD, " kb:\n steps:\n - run: ./scripts/checkers.sh --ci kb\n"
" if: false\n"),
["job `kb`'s `checkers.sh --ci` step is conditional (job if), so it does not count"]
+ _NO_PW),
("a step under a defaults: run: shell: is counted (the user's ruling)", _REQ,
"defaults:\n run:\n shell: nix develop .#ci --command bash -e {0}\n"
+ _wf(_BUILD, _job("kb", _KB_RUN, head=" defaults:\n run:\n shell: bash\n")),
[]),
("continue-on-error does not count", _REQ,
_wf(_BUILD, _job("kb", " - run: ./scripts/checkers.sh --ci kb\n"
" continue-on-error: true\n")),
["job `kb`'s `checkers.sh --ci` step is conditional (continue-on-error), "
"so it does not count"] + _NO_PW),
("shell control flow around the call does not count", _REQ,
_wf(_BUILD, _job("kb", " - run: true || ./scripts/checkers.sh --ci kb\n")),
[_stray(9)] + _NO_PW),
("a job running another job's row is refused", _REQ,
_wf(_BUILD, _job("kb", " - run: ./scripts/checkers.sh --ci build\n")),
["job `kb` runs `checkers.sh --ci build`; a job runs only its own row of the CI table"]
+ _NO_PW),
("a bare mention in a step name is refused", _REQ,
_wf(_BUILD, _job("kb", " - name: runs scripts/checkers.sh\n"
" run: ./scripts/checkers.sh --ci kb\n")),
[_stray(9)]),
_unreadable("a job header with a trailing comment is refused (j49r)",
_wf(_BUILD, " test:\n runs-on: x\n",
" gate: # skipped on purpose\n if: false\n steps:\n - run: echo\n",
_job("kb", _KB_RUN, head=" needs: gate\n")),
"gate: # skipped", kind="comment"),
_unreadable("a closed quote followed by a comment is refused (j49r)",
_wf(_BUILD, _job("kb", _KB_RUN, head=" env:\n A: \"x\" # note\n")
+ " if: false\n", " after:\n steps:\n - run: echo \"z\"\n"),
'"x" # note', kind="comment",
rest=["job `kb`'s `checkers.sh --ci` step is conditional (job if), "
"so it does not count"] + _NO_PW),
_unreadable("a # inside a quoted value, below indent 8, is refused (j49r)",
_wf(_BUILD, _job("kb", _KB_RUN + " - uses: x\n with:\n"
" foo: \"x # y\"\n")
+ " if: false\n", " after:\n steps:\n - run: echo \"z\"\n"),
'"x # y"', kind="comment",
rest=["job `kb`'s `checkers.sh --ci` step is conditional (job if), "
"so it does not count"] + _NO_PW),
_unreadable("a trailing comment on a step is refused (j49r)",
_wf(_BUILD, _job("kb", " - uses: actions/checkout@v7 # pin\n" + _KB_RUN)),
"# pin", kind="comment"),
_unreadable("a one-line run: with a ` #` inside quotes is refused (j49r)",
_wf(_BUILD, _job("kb", " - run: echo \"a # b\"\n" + _KB_RUN)),
'"a # b"', kind="comment"),
_unreadable("a multi-line double-quoted value is refused, escaped quote and all (6zf1)",
_wf(_BUILD, " gate:\n runs-on: x\n name: \"a\n it\\\" is\n b: \"\n"
" if: false\n steps:\n - run: echo \"z\"\n",
_job("kb", _KB_RUN, head=" needs: gate\n")),
'name: "a', kind="value", rest=_NEEDS_GATE),
_unreadable("a multi-line single-quoted value is refused, doubled quote and all (6zf1)",
_wf(_BUILD, " gate:\n runs-on: x\n name: 'a\n it''s\n b: '\n"
" if: false\n steps:\n - run: echo 'z'\n",
_job("kb", _KB_RUN, head=" needs: gate\n")),
"name: 'a", kind="value", rest=_NEEDS_GATE),
_unreadable("an opening line ending in an escaped \\\" still opens a quote (6zf1)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: gate\n name: \"a\\\"\n"
" needs: ok\n x: y\"\n")),
'name: "a', kind="value"),
_unreadable("an opening line ending in a doubled '' still opens a quote (6zf1)",
_wf(_BUILD, _GATE, _job("kb", _KB_RUN, head=" needs: gate\n name: 'it''\n"
" needs: ok\n x: y'\n")),
"name: 'it", kind="value"),
_unreadable("a flow list whose quote runs onto the next lines is refused (0wn3)",
_wf(_BUILD, _job("kb", " - name: [a, \"b\n" + _INNER_RUN + " c: x\"]\n")),
'name: [a, "b', kind="value"),
_unreadable("the same with a single-quoted string (0wn3)",
_wf(_BUILD, _job("kb", " - name: [a, 'b\n" + _INNER_RUN + " c: x']\n")),
"name: [a, 'b", kind="value"),
_unreadable("the same as a flow map (0wn3)",
_wf(_BUILD, _job("kb", " - name: {a: \"b\n" + _INNER_RUN + " c: x\"}\n")),
'name: {a: "b', kind="value"),
_unreadable("a flow list that ends in its closer with a quote still open is refused (0wn3)",
_wf(_BUILD, _job("kb", " - name: [a, \"b]\n" + _INNER_RUN + " c: x\"]\n")),
'name: [a, "b]', kind="value"),
_unreadable("a flow value opened on a line of its own is refused too (0wn3)",
_wf(_BUILD, _job("kb", " - name:\n [a, \"b\n" + _INNER_RUN
+ " c: x\"]\n")),
'[a, "b', kind="value"),
_unreadable("an anchored multi-line flow is refused (0wn3)",
_wf(_BUILD, _job("kb", " - name: &a [x, \"b\n" + _INNER_RUN + " c: x\"]\n")),
'name: &a', kind="value"),
_unreadable("a tagged multi-line flow is refused (0wn3)",
_wf(_BUILD, _job("kb", " - name: !!seq [x, \"b\n" + _INNER_RUN
+ " c: x\"]\n")),
'name: !!seq', kind="value"),
*[_unreadable(f"a U+{ord(sep):04X} inside a line is refused (xpzh)",
_wf(_BUILD, " kb:\n runs-on: x" + sep + " if: false\n steps:\n" + _KB_RUN),
sep, kind="char")
for sep in _YAML_ONLY_BREAKS],
_unreadable("a line break inside a block scalar's body is refused too (xpzh)",
_wf(_BUILD, _job("kb", _KB_RUN + " - run: |\n echo a\u2028 if: false\n")),
"echo a", kind="char"),
_unreadable("a line break inside a full-line comment is refused too (xpzh)",
_wf(_BUILD, " kb:\n runs-on: x\n # note\u2028 if: false\n steps:\n"
+ _KB_RUN), "# note", kind="char"),
("a YAML comment mentioning checkers.sh is fine", _REQ,
_wf(_BUILD, _job("kb", " # every suite goes through scripts/checkers.sh\n"
" - run: ./scripts/checkers.sh --ci kb\n")), []),
]
# (name, file bytes as text, the complaints it must draw), read back through
# read_workflow, because what a CR means depends on that read (improvise-xpzh).
FILE_SELF_TEST = [
("control: CRLF line ends are folded by the read and accepted (xpzh)",
_wf(_BUILD, _job("kb", _KB_RUN)).replace("\n", "\r\n"), []),
("control: a lone CR is a line break, as YAML reads it (xpzh)",
_wf(_BUILD, " kb:\n runs-on: x\r if: false\n steps:\n" + _KB_RUN),
["job `kb`'s `checkers.sh --ci` step is conditional (job if), so it does not count"]
+ _NO_PW),
]
# (name, workflow text, the complaints it must draw).
SELF_TEST = [
("a job nobody gates is named",
@@ -79,6 +696,11 @@ SELF_TEST = [
"jobs:\n test:\n x: 1\n pages-deploy:\n x: 1\n",
[f"{DEPLOY} has no `needs:` list, so nothing gates the deploy"]),
("a job header with a trailing comment is still a job (j49r)",
"jobs:\n test:\n x: 1\n gate: # skipped on purpose\n x: 1\n"
" pages-deploy:\n needs: [test]\n",
[f"{DEPLOY} does not need `gate`, so a failing gate cannot stop a deploy"]),
# `needs:` belongs to the job it is indented under. Taking the first one
# seen would read ANOTHER job's list as the deploy's and pass.
("another job's needs is not mistaken for the deploy's",
@@ -95,28 +717,48 @@ def self_test():
got = verdict(jobs, needs)
if got != want:
failures.append(f" {name}: expected {want}, saw {got}")
for name, suites, text, want in SUITE_SELF_TEST:
got = suite_verdict(suites, text)
if got != want:
failures.append(f" {name}: expected {want}, saw {got}")
for name, content, want in FILE_SELF_TEST:
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "ci.yml")
with open(path, "w", encoding="utf8", newline="") as f:
f.write(content)
got = suite_verdict(_REQ, read_workflow(path))
if got != want:
failures.append(f" {name}: expected {want}, saw {got}")
if failures:
print("ci-graph self-test FAILED:", file=sys.stderr)
print("\n".join(failures), file=sys.stderr)
return 1
print(f"ci-graph self-test: {len(SELF_TEST)} graphs judged")
print(f"ci-graph self-test: {len(SELF_TEST)} graphs, "
f"{len(SUITE_SELF_TEST)} suite lists and {len(FILE_SELF_TEST)} files judged")
return 0
def main():
if "--self-test" in sys.argv[1:]:
return self_test()
jobs, needs = read_graph(open(WORKFLOW, encoding="utf8").read())
text = read_workflow(WORKFLOW)
jobs, needs = read_graph(text)
if len(jobs) < 2:
print(f"error: {WORKFLOW}: found {len(jobs)} jobs; the parse is wrong",
file=sys.stderr)
return 2
complaints = verdict(jobs, needs)
suites = list_suites()
if not suites:
print("error: checkers.sh --list named no suites; the listing is wrong",
file=sys.stderr)
return 2
complaints = verdict(jobs, needs) + suite_verdict(suites, text)
if complaints:
for c in complaints:
print(f"ci-graph: {c}", file=sys.stderr)
return 1
print(f"ci-graph: {DEPLOY} needs all {len(jobs) - 1} other jobs")
print(f"ci-graph: {DEPLOY} needs all {len(jobs) - 1} other jobs; "
f"all {len(suites)} checkers.sh suites run in CI")
return 0
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bash
# check_the_checker for scripts/check-harness-lock.sh (improvise-zs2r). The
# fixtures are path-only crates, so every case runs offline with no cache.
set -uo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RUN="$here/check-harness-lock.sh"
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
pass=0; fail=0
ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); }
bad() { printf ' FAIL %s (exit %s, wanted %s)\n' "$1" "$2" "$3"; fail=$((fail+1)); }
# $1 name, $2 expected exit, $3 expected stderr substring ("" for none), rest: args
run_case() {
local name="$1" want="$2" says="$3"; shift 3
local err
err="$("$RUN" "$@" 2>&1 >/dev/null)"
local got=$?
if [ "$got" != "$want" ]; then
bad "$name" "$got" "$want"
printf '%s\n' "$err" | sed 's/^/ /'
elif [ -n "$says" ] && [[ "$err" != *"$says"* ]]; then
bad "$name: stderr did not say $says" "$got" "$want"
else
ok "$name"
fi
}
# $1 dir, $2 crate name, $3 extra [dependencies] lines, $4 "workspace" or ""
crate() {
mkdir -p "$1/src"
: > "$1/src/lib.rs"
{
printf '[package]\nname = "%s"\nversion = "0.0.0"\nedition = "2021"\n' "$2"
[ "${4:-}" = workspace ] && printf '\n[workspace]\n'
printf '\n[dependencies]\n%s' "$3"
} > "$1/Cargo.toml"
}
# A two-crate workspace shaped like the harness: `app` is its own root and
# reaches `lib` by path, so `lib`'s manifest is one the lock resolves against.
fixture() {
local root="$1"
crate "$root/lib" lib ""
crate "$root/extra" extra ""
crate "$root/app" app 'lib = { path = "../lib" }
' workspace
cargo generate-lockfile --offline --manifest-path "$root/app/Cargo.toml" 2>/dev/null
}
fixture "$work/current"
run_case "a current lock passes" 0 "" "$work/current/app/Cargo.toml"
# The shape improvise-zs2r names: only a PATH DEPENDENCY's manifest changed.
fixture "$work/stale"
crate "$work/stale/lib" lib 'extra = { path = "../extra" }
'
run_case "a manifest-only change to a path dependency is stale" 1 "STALE" \
"$work/stale/app/Cargo.toml"
# Offline and unresolvable is NOT a stale lock: the gate did not run.
fixture "$work/cold"
crate "$work/cold/lib" lib 'zs2r-no-such-crate-anywhere = "1"
'
run_case "an offline resolution failure is not a verdict" 2 "did not run" \
"$work/cold/app/Cargo.toml"
run_case "a missing manifest is a usage error" 2 "no manifest" \
"$work/nowhere/Cargo.toml"
run_case "two manifests is a usage error" 2 "" \
"$work/current/app/Cargo.toml" "$work/stale/app/Cargo.toml"
run_case "an unknown option is a usage error" 2 "unknown option" --bogus
printf '\ncheck-harness-lock: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" = 0 ]
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Fails in seconds when harness/opfs-contract/Cargo.lock no longer matches the
# manifests it resolves against. Reasoning: context/code/tooling.md, "The
# harness lock gate" (improvise-zs2r).
set -uo pipefail
usage() {
cat >&2 <<'EOF'
usage: check-harness-lock.sh [--allow-network] [manifest]
manifest the Cargo.toml whose lock is checked; defaults to
harness/opfs-contract/Cargo.toml beside this script
--allow-network resolve without --offline (CI); the default never
touches the network
exit 0 the lock is current
exit 1 the lock is stale: a manifest it resolves against changed
exit 2 the gate did not run: usage error, no cargo, or cargo could not
resolve at all (offline, with a dependency missing from the local
cache). NO verdict was produced. Never collapse 2 into 1.
EOF
}
main() {
local network=0 manifest=""
while [ $# -gt 0 ]; do
case "$1" in
--allow-network) network=1 ;;
-h|--help) usage; return 2 ;;
-*) printf 'check-harness-lock: unknown option %s\n' "$1" >&2; usage; return 2 ;;
*)
if [ -n "$manifest" ]; then usage; return 2; fi
manifest="$1" ;;
esac
shift
done
if [ -z "$manifest" ]; then
manifest="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/harness/opfs-contract/Cargo.toml"
fi
if [ ! -f "$manifest" ]; then
printf 'check-harness-lock: no manifest at %s\n' "$manifest" >&2
return 2
fi
if ! command -v cargo >/dev/null 2>&1; then
echo 'check-harness-lock: no cargo on PATH' >&2
return 2
fi
local -a args=(metadata --locked --format-version 1 --manifest-path "$manifest")
if [ "$network" = 0 ]; then args+=(--offline); fi
local err rc
err="$(cargo "${args[@]}" 2>&1 >/dev/null)"
rc=$?
if [ "$rc" = 0 ]; then
printf 'check-harness-lock: %s is current\n' "${manifest%Cargo.toml}Cargo.lock"
return 0
fi
# The one message that means "resolution succeeded and disagrees with the lock".
if printf '%s' "$err" | grep -q 'cannot update the lock file .* because --locked was passed'; then
printf 'check-harness-lock: STALE: %s\n' "${manifest%Cargo.toml}Cargo.lock" >&2
printf ' Run: cargo update -w --manifest-path %s, and commit the lock.\n' "$manifest" >&2
return 1
fi
printf 'check-harness-lock: the gate did not run; cargo could not resolve:\n%s\n' "$err" >&2
if [ "$network" = 0 ]; then
echo ' Offline, this is usually a dependency missing from the local cache.' >&2
echo ' Retry with --allow-network, or fetch once and rerun.' >&2
fi
return 2
}
main "$@"
+46 -49
View File
@@ -67,21 +67,7 @@ direct=$?
unset name
# ---------------------------------------------------------------------------
# 2. The bash-5 `local` semantics. A multi-assignment `local` does not read
# left to right on bash 5, so this suite passes on macOS bash 3.2 and dies
# under `set -u` on bash 5 -- PR #133's 19/19 local against 2/19 in CI.
d="$(sandbox bash5)"
suite "$d" vers <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
f() { local a="$1" b="$a/x"; echo "$b"; }
f one
EOF
expect "a bash-3.2-only construct fails under the runner's pinned bash" \
1 "$d/checkers.sh" vers
# ---------------------------------------------------------------------------
# 3. The shebang rule. `#!/bin/sh` is dash in CI and bash-in-sh-mode on macOS.
# 2. The shebang rule. `#!/bin/sh` is dash in CI and bash-in-sh-mode on macOS.
d="$(sandbox shebang)"
suite "$d" fine <<'EOF'
#!/usr/bin/env bash
@@ -101,7 +87,7 @@ rm -f "$d/legacy.sh"
expect "and the same sandbox passes once it is removed" 0 "$d/checkers.sh" fine
# ---------------------------------------------------------------------------
# 4. A failing suite is reported, and a passing one is not.
# 3. A failing suite is reported, and a passing one is not.
d="$(sandbox rc)"
suite "$d" red <<'EOF'
#!/usr/bin/env bash
@@ -117,38 +103,7 @@ EOF
expect "a passing suite passes the runner" 0 "$d/checkers.sh" blue
# ---------------------------------------------------------------------------
# 5. No bash >= 5 and no nix is exit 3, not a silent run under whatever is
# there. This is the arm that would have refused the macOS system bash.
d="$(sandbox oldbash)"
suite "$d" any <<'EOF'
#!/usr/bin/env bash
exit 0
EOF
mkdir -p "$work/fakebin"
cat > "$work/fakebin/bash" <<'EOF'
#!/bin/sh
case "$*" in *BASH_VERSINFO*) echo 3; exit 0 ;; esac
exit 0
EOF
chmod +x "$work/fakebin/bash"
# The runner is invoked through a REAL bash, so the fake one below is only
# what its own `command -v bash` probe finds -- not its interpreter.
# utilbin carries only the utilities the runner itself needs, and no nix, so
# the fake bash is the only candidate and the nix fallback is absent. Linked
# one by one rather than taking /usr/bin:/bin, which on NixOS hold only `env`
# and `sh` -- the runner died on `dirname` there, exit 1 instead of 3.
mkdir -p "$work/utilbin"
for tool in dirname; do
ln -sf "$(command -v "$tool")" "$work/utilbin/$tool"
done
env PATH="$work/fakebin:$work/utilbin" HOME="$HOME" \
"${BASH:-bash}" "$d/checkers.sh" any > "$work/out" 2>&1
got=$?
[ "$got" = 3 ] && ok "an old bash with no nix is exit 3" \
|| bad "an old bash with no nix is exit 3" "exit $got, wanted 3"
# ---------------------------------------------------------------------------
# 6. A named suite that does not exist is a usage error, not a silent no-op.
# 4. A named suite that does not exist is a usage error, not a silent no-op.
d="$(sandbox missing)"
suite "$d" present <<'EOF'
#!/usr/bin/env bash
@@ -157,7 +112,7 @@ EOF
expect "an unknown suite name is exit 4" 4 "$d/checkers.sh" absent
# ---------------------------------------------------------------------------
# 7. --list names every suite in the directory and nothing else.
# 5. --list names every suite in the directory and nothing else.
d="$(sandbox listing)"
suite "$d" alpha <<'EOF'
#!/usr/bin/env bash
@@ -182,5 +137,47 @@ listed="$(tr '\n' ' ' < "$work/out")"
&& ok "an X-test.sh with no X beside it is not a suite" \
|| bad "an X-test.sh with no X beside it is not a suite" "got [$listed]"
# ---------------------------------------------------------------------------
# 8. --would-run reports what the rest of the arguments run, and runs nothing.
# check-ci-graph.py asks it, so each argument rule below is answered HERE.
d="$(sandbox wouldrun)"
for n in alpha beta; do
suite "$d" "$n" <<EOF
#!/usr/bin/env bash
touch "$work/ran-$n"
EOF
done
rm -f "$work"/ran-*
"$d/checkers.sh" --would-run > "$work/out" 2>&1
got="$(tr '\n' ' ' < "$work/out")"
[ "$got" = "would-run: alpha would-run: beta " ] && [ ! -e "$work/ran-alpha" ] \
&& ok "--would-run with no names reports every suite and runs none" \
|| bad "--would-run with no names reports every suite and runs none" "got [$got]"
"$d/checkers.sh" --would-run --self-test alpha > "$work/out" 2>&1
got="$(tr '\n' ' ' < "$work/out")"
[ "$got" = "missing: checkers " ] \
&& ok "--would-run --self-test reports checkers and discards the names" \
|| bad "--would-run --self-test reports checkers and discards the names" "got [$got]"
"$d/checkers.sh" --would-run --list alpha > "$work/out" 2>&1
grep -q '^would-run:' "$work/out" \
&& bad "--would-run --list reports nothing as run" "$(tr '\n' ' ' < "$work/out")" \
|| ok "--would-run --list reports nothing as run"
expect "--would-run anywhere but first is a usage error" 4 "$d/checkers.sh" alpha --would-run
[ ! -e "$work/ran-alpha" ] && ok "and it runs nothing" \
|| bad "and it runs nothing" "alpha ran"
# ---------------------------------------------------------------------------
# 9. --ci <job> runs that job's row of the CI table and nothing else. The
# sandbox has none of the real suites, so a row reads back as `missing:`.
"$d/checkers.sh" --would-run --ci js > "$work/out" 2>&1
got="$(tr '\n' ' ' < "$work/out")"
[ "$got" = "missing: node-test " ] && ok "--ci reads that job's row of the table" \
|| bad "--ci reads that job's row of the table" "got [$got]"
expect "--ci with a job the table does not have is exit 4" 4 "$d/checkers.sh" --would-run --ci nope
expect "--ci with nothing after it is exit 4" 4 "$d/checkers.sh" --would-run --ci
expect "--ci beside a suite name is exit 4, not a union" 4 \
"$d/checkers.sh" --would-run --ci js alpha
expect "--ci beside --self-test is exit 4" 4 "$d/checkers.sh" --would-run --self-test --ci js
printf '\ncheckers-test: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+56 -42
View File
@@ -3,15 +3,14 @@
# and so does the local gate, so a suite cannot be green under one shell and
# red under the other. Reasoning: context/code/tooling.md section 3.
#
# It does three things a bare `./scripts/X-test.sh` does not:
# It does two things a bare `./scripts/X-test.sh` does not. It runs every
# suite under the bash running it (the dev shell's), and it:
#
# 1. pins a bash >= 5, taken from nix when the ambient one is older, so the
# macOS system bash 3.2 can never be the shell a suite passes under;
# 2. scrubs the environment down to an allowlist, because `nix develop`
# 1. scrubs the environment down to an allowlist, because `nix develop`
# exports `name`, `out`, `system`, `shell`, `builder` and two dozen more
# generically-named variables that CI does not have -- a `set -u` fault
# reading one of them is SILENT inside the dev shell and fatal in CI;
# 3. refuses a shell script in scripts/ whose shebang is not
# 2. refuses a shell script in scripts/ whose shebang is not
# `#!/usr/bin/env bash`, so the sh/bash split cannot come back.
set -euo pipefail
@@ -19,7 +18,7 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
usage() {
cat >&2 <<'EOF'
usage: checkers.sh [--list] [--self-test] [name ...]
usage: checkers.sh [--list] [--self-test] [--ci <job>] [name ...]
name a suite to run, spelled without the -test.sh suffix
(e.g. `pr-reviews` for scripts/pr-reviews-test.sh).
@@ -27,10 +26,16 @@ usage: checkers.sh [--list] [--self-test] [name ...]
--list print the suite names and exit
--self-test run scripts/checkers-test.sh, which injects the faults this
runner exists to catch
--ci <job> run the suites the CI table gives that job of
.github/workflows/ci.yml, and nothing else. Each CI job runs
exactly this, so the table is the one list of what CI runs
--would-run as the FIRST argument only: run nothing, and print
`would-run: <suite>` (or `missing: <name>`) for each suite
the rest of the arguments would run. check-ci-graph.py
asks this, so the argument rules have one home
exit 0 every suite passed
exit 1 a suite failed, or a shebang is not `#!/usr/bin/env bash`
exit 3 no bash >= 5 could be found and nix could not supply one
exit 4 usage error
THIS SCRIPT'S `$?` MUST BE THE ONE YOU READ. Nothing may follow it in the same
@@ -46,19 +51,6 @@ ALLOW="HOME TMPDIR TMP TEMP TERM LANG LC_ALL USER LOGNAME
NIX_PATH NIX_REMOTE NIX_SSL_CERT_FILE SSL_CERT_FILE
XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME"
# Prints the absolute path of a bash >= 5, or nothing. The ambient one is tried
# first so a machine that already has a modern bash pays no nix startup at all.
find_modern_bash() {
local candidate major
candidate="$(command -v bash || true)"
if [ -n "$candidate" ]; then
major="$("$candidate" -c 'echo "${BASH_VERSINFO[0]}"' 2>/dev/null || echo 0)"
if [ "$major" -ge 5 ] 2>/dev/null; then echo "$candidate"; return 0; fi
fi
command -v nix >/dev/null 2>&1 || return 0
nix shell nixpkgs#bash --command bash -c 'command -v bash' 2>/dev/null || true
}
# Every shell script in scripts/ says `#!/usr/bin/env bash` and nothing else.
# `#!/bin/sh` is dash in CI and bash 3.2 in sh-mode on macOS: a second platform
# split, undetected for the same reason the first one was.
@@ -91,28 +83,42 @@ suites() {
done
}
# $1 the pinned bash, $2 the suite file. The suite runs with the pinned bash
# FIRST on PATH, so a script it invokes through its own `#!/usr/bin/env bash`
# -- or through `sh` -- gets the pinned one too, not the system 3.2.
# $1 the suite file, run by the bash running this script, with PATH unchanged.
run_suite() {
local pinned="$1" file="$2"
local bindir; bindir="$(dirname "$pinned")"
local file="$1"
local -a env_args=(-i)
local v
for v in $ALLOW ${CHECKERS_KEEP:-}; do
if [ -n "${!v+set}" ]; then env_args+=("$v=${!v}"); fi
done
env_args+=("PATH=$bindir:${PATH:-/usr/bin:/bin}")
env "${env_args[@]}" "$pinned" "$file"
env_args+=("PATH=$PATH")
env "${env_args[@]}" "$BASH" "$file"
}
# The CI table: the suites each job of .github/workflows/ci.yml runs, by that
# job's `checkers.sh --ci <job>` step. check-ci-graph.py holds it to --list.
ci_suites() {
case "$1" in
build) echo assemble-stamp ;;
js) echo node-test ;;
# Needs improvise and code-maat, which only this job has (tooling.md section 7).
nix-build) echo repo-stats ;;
# check-harness-lock must never read "could not resolve" as "stale" (improvise-zs2r).
kb) echo kb checkers pr-watch release-preflight pr-reviews fmt-workspaces bd-audit-ids check-harness-lock agent-git ;;
*) return 1 ;;
esac
}
main() {
local self_test=0
local self_test=0 would_run=0 ci_job=""
local -a names=()
# Only as the FIRST argument, so a CI step spelling it anywhere is an error.
if [ "${1:-}" = --would-run ]; then would_run=1; shift; fi
while [ $# -gt 0 ]; do
case "$1" in
--list) suites; return 0 ;;
--self-test) self_test=1 ;;
--ci) ci_job="${2:-}"; [ -n "$ci_job" ] || { usage; return 4; }; shift ;;
-h|--help) usage; return 4 ;;
-*) printf 'checkers: unknown option %s\n' "$1" >&2; usage; return 4 ;;
*) names+=("$1") ;;
@@ -120,25 +126,33 @@ main() {
shift
done
local pinned; pinned="$(find_modern_bash)"
if [ -z "$pinned" ]; then
echo 'checkers: no bash >= 5 on PATH and nix cannot supply one.' >&2
echo ' Enter the dev shell (direnv allow / nix develop) and retry.' >&2
return 3
fi
printf 'checkers: bash %s at %s\n' \
"$("$pinned" -c 'echo "${BASH_VERSINFO[0]}.${BASH_VERSINFO[1]}"')" "$pinned"
local rc=0
check_shebangs || rc=1
if [ "$self_test" = 1 ]; then
if [ -n "$ci_job" ]; then
if [ "$self_test" = 1 ] || [ ${#names[@]} -gt 0 ]; then
echo 'checkers: --ci takes a job and nothing else' >&2; return 4
fi
local table
table="$(ci_suites "$ci_job")" || {
printf 'checkers: no CI job %s in the table\n' "$ci_job" >&2; return 4; }
read -r -a names <<< "$table"
elif [ "$self_test" = 1 ]; then
names=(checkers)
elif [ ${#names[@]} -eq 0 ]; then
while IFS= read -r line; do names+=("$line"); done < <(suites)
fi
local n file
if [ "$would_run" = 1 ]; then
for n in ${names[@]+"${names[@]}"}; do
if [ -f "$HERE/$n-test.sh" ]; then printf 'would-run: %s\n' "$n"
else printf 'missing: %s\n' "$n"; fi
done
return 0
fi
printf 'checkers: bash %s at %s\n' "$BASH_VERSION" "$BASH"
local rc=0
check_shebangs || rc=1
for n in ${names[@]+"${names[@]}"}; do
file="$HERE/$n-test.sh"
if [ ! -f "$file" ]; then
@@ -146,7 +160,7 @@ main() {
rc=4; continue
fi
printf '\n=== %s ===\n' "$n"
run_suite "$pinned" "$file" || rc=1
run_suite "$file" || rc=1
done
return $rc
}

Some files were not shown because too many files have changed in this diff Show More