fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge main (d7bfcbb, PR #96) into wave/operandless-commands

Browse Source 
Two textual conflicts, and one the merge resolved cleanly and wrongly.

effect.rs: #96 extracted the LayoutGeometry list into
layout_geometry_cases() shared by two tests, one of which compares it
against the wire vocabulary; my side had added ViewTarget arguments and
carried the base's assert_eq!(cases.len(), 13). Kept #96's structure with
my arguments and dropped the count -- the list-versus-vocabulary check is
strictly stronger and a count beside it would be a second answer to the
same question. Verified the count came from the merge base, not from my
side, before dropping it.

test-evidence/command.md: both sides appended entries at the same point.
Kept both.

app.rs auto-merged CLEANLY and did not compile: improvise-4dpj's new
a_batch_that_leaves_records_settles_out_of_records_normal constructs
effect::SetAxis, which now carries a view operand. Fixed with
ViewTarget::Active -- the host's own key path, which is what that test is.

Checked rather than assumed, per the brief:

- Test names diffed BOTH ways against the union of the two parents, not
  the count. Union 1510, merged 1509, exactly one name absent and it is
  the intended one: a_hosts_none_cats_leak_into_another_views_formula_columns,
  the ac2z characterization test my own branch replaced with
  a_formulas_columns_survive_a_host_that_aggregates_differently. Nothing
  else lost, nothing unexpected gained.
- #96's the_enumeration_names_every_variant_the_derive_declares is still
  discriminating against the reshaped ModelCommand variants: dropping
  TransposeAxes from every_model_command() fails it by name.
- The test-support feature stays off in shipped builds: cargo tree
  --edges normal --target wasm32-unknown-unknown finds no serde_json.

And the finding the brief asked for, which is the opposite of a
disagreement: #96's switching_views_across_records_ness_settles_against_
the_view_switched_to is the HOST-side twin of the client fix in this
branch. Same batch, same crossing in both directions, and independently
the same phrase in its message. Neither is redundant -- nothing in the
projection stream carries the host's ViewState, so a host-side guard
cannot see a client's mode or the reverse. Cross-referenced in
context/test-evidence/clients.md so neither is deleted as a duplicate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-21 11:08:17 -07:00
parent 81fe546594 26456c698e
commit 3d0850c132
27 changed files with 1458 additions and 128 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.beads/issues.jsonl
+5 -5
View File
@@ -40,9 +40,9 @@
{"_type":"issue","id":"improvise-km8","title":"1.2 Fix Cargo.toml metadata","description":"Add missing package fields: description, repository, homepage, documentation, readme, keywords, categories, license. Description: 'Terminal pivot-table modeling in the spirit of Lotus Improv'. Keywords: tui, pivot, spreadsheet, data, improv. Categories: command-line-utilities, visualization.","status":"closed","priority":1,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:06:48Z","created_by":"Edward Langley","updated_at":"2026-07-30T23:06:31Z","closed_at":"2026-04-09T07:20:17Z","close_reason":"Metadata already present: description, license, repository, homepage, readme, keywords, categories all populated.","dependency_count":0,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"improvise-902","title":"1.1 Audit and remove or update context/SPEC.md","description":"Compare context/SPEC.md against actual code. If spec contradicts code significantly, delete it. Optionally move salvageable conceptual content to docs/design-notes.md with a staleness disclaimer. Do not try to bring spec into sync.","status":"closed","priority":1,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:06:46Z","created_by":"Edward Langley","updated_at":"2026-07-30T23:06:31Z","closed_at":"2026-04-09T07:10:05Z","close_reason":"Audited SPEC.md against code. Found minor inaccuracies (storage internals, wizard step count, search mode representation) but no major architectural contradictions. Deleted SPEC.md since it was largely redundant with repo-map.md and design-principles.md. Salvaged product vision and non-goals into docs/design-notes.md with staleness disclaimer.","dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"improvise-tik","title":"Phase 1: Repository hygiene","description":"Raise the quality floor of what a stranger sees when they land on the repo. Subtasks cover spec audit, Cargo.toml metadata, publish-readiness, CSV audit, and example files.","status":"closed","priority":1,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-09T04:05:45Z","created_by":"Edward Langley","updated_at":"2026-04-09T07:22:34Z","closed_at":"2026-04-09T07:22:34Z","close_reason":"All 5 subtasks complete: SPEC.md audited, Cargo.toml metadata fixed, publish dry-run clean, CSV audit done, example files created.","dependencies":[{"issue_id":"improvise-tik","depends_on_id":"improvise-2fr","type":"blocks","created_at":"2026-04-08T23:37:31Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-902","type":"blocks","created_at":"2026-04-08T23:37:30Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-bv1","type":"blocks","created_at":"2026-04-08T23:37:32Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-km8","type":"blocks","created_at":"2026-04-08T23:37:30Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-tik","depends_on_id":"improvise-n1h","type":"blocks","created_at":"2026-04-08T23:37:32Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-x1ns","title":"encoded_width undercounts a forced category that has a missing coordinate","description":"Found by ChatGPT Codex reviewing PR #90 at 99cfb7f. THE REVIEW WAS NOT READ BEFORE THE PR MERGED — orchestrator process failure, see improvise-\u003cthis\u003e's sibling note. Re-verified on main at ce895ff: STILL LIVE.\n\nencoded_width (crates/improvise-gp/src/fit.rs:18) computes has_missing at line 32 and uses it ONLY in the numeric branch:\n\n true =\u003e 1 + usize::from(has_missing), // numeric: feature + .missing flag\n false =\u003e present.iter().collect::\u003cHashSet\u003c_\u003e\u003e().len().max(1), // categorical: present only\n\nbuild_categorical (crates/gp-tree/src/data.rs:359) maps every missing cell to MISSING_CAT and includes it in distinct(\u0026mapped), emitting an ADDITIONAL one-hot.\n\nSo a forced categorical column with N distinct present values plus one record lacking that coordinate passes the preflight as N and materializes N+1 columns. It is caught only by the LATER EXACT CHECK — which is precisely the allocation the preflight exists to prevent. At default max_records that is a large one-hot built before the error.\n\nThis is doctrine(guard_the_cost_not_the_result) defeated, and it is the SECOND time on this code path: improvise-zv8t's own adversarial round found forcing-by-name-while-budgeting-by-index, repaired as a data dependency. This one is the same class one level in — the two branches of one match disagree about what a column costs.\n\nNote the Wave K agent's own review had spotted a related shape (recorded as F5, \"build_categorical counts MISSING_CAT, the preflight does not — under-counts by one per category with an absent coordinate\") and recorded it as READ, NOT INJECTED and bounded by the exact check. Codex is right that \"bounded by the exact check\" is not a defence: the exact check runs after the allocation.\n\n## Acceptance Criteria\n- Count the missing level in the categorical branch.\n- A boundary regression with a FORCED NUMERIC CATEGORY containing a missing coordinate — the interesting case, because forcing is what makes a numeric column take the categorical branch.\n- Prove it by injection: without the fix, the preflight must under-report by exactly one and the test must fail naming that.\n- While there: gp.md §14 documents where the preflight and the exact check \"still differ\". Re-read it — it is the prose that made this look bounded.\n","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T17:13:05Z","created_by":"Edward Langley","updated_at":"2026-09-21T17:13:05Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-x1ns","title":"encoded_width undercounts a forced category that has a missing coordinate","description":"Found by ChatGPT Codex reviewing PR #90 at 99cfb7f. THE REVIEW WAS NOT READ BEFORE THE PR MERGED — orchestrator process failure, see improvise-\u003cthis\u003e's sibling note. Re-verified on main at ce895ff: STILL LIVE.\n\nencoded_width (crates/improvise-gp/src/fit.rs:18) computes has_missing at line 32 and uses it ONLY in the numeric branch:\n\n true =\u003e 1 + usize::from(has_missing), // numeric: feature + .missing flag\n false =\u003e present.iter().collect::\u003cHashSet\u003c_\u003e\u003e().len().max(1), // categorical: present only\n\nbuild_categorical (crates/gp-tree/src/data.rs:359) maps every missing cell to MISSING_CAT and includes it in distinct(\u0026mapped), emitting an ADDITIONAL one-hot.\n\nSo a forced categorical column with N distinct present values plus one record lacking that coordinate passes the preflight as N and materializes N+1 columns. It is caught only by the LATER EXACT CHECK — which is precisely the allocation the preflight exists to prevent. At default max_records that is a large one-hot built before the error.\n\nThis is doctrine(guard_the_cost_not_the_result) defeated, and it is the SECOND time on this code path: improvise-zv8t's own adversarial round found forcing-by-name-while-budgeting-by-index, repaired as a data dependency. This one is the same class one level in — the two branches of one match disagree about what a column costs.\n\nNote the Wave K agent's own review had spotted a related shape (recorded as F5, \"build_categorical counts MISSING_CAT, the preflight does not — under-counts by one per category with an absent coordinate\") and recorded it as READ, NOT INJECTED and bounded by the exact check. Codex is right that \"bounded by the exact check\" is not a defence: the exact check runs after the allocation.\n\n## Acceptance Criteria\n- Count the missing level in the categorical branch.\n- A boundary regression with a FORCED NUMERIC CATEGORY containing a missing coordinate — the interesting case, because forcing is what makes a numeric column take the categorical branch.\n- Prove it by injection: without the fix, the preflight must under-report by exactly one and the test must fail naming that.\n- While there: gp.md §14 documents where the preflight and the exact check \"still differ\". Re-read it — it is the prose that made this look bounded.\n","status":"closed","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T17:13:05Z","created_by":"Edward Langley","updated_at":"2026-09-21T17:56:36Z","closed_at":"2026-09-21T17:56:36Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-ac2z","title":"A host's none_cats leak into another view's grid: formula columns vanish","description":"A client on a view whose none_cats differ from the HOST's is projected a grid\nmissing that view's formula columns entirely. Not a wrong number -- absent\ncolumns.\n\nFOUND BY CORRECTING A FIXTURE, which is the part worth keeping. The wave-I\nhand-back for improvise-t6db reported this hazard as \"argued and NOT observed\",\non the strength of a fixture that was said to put Product on the None axis in\none view and on Column in another. It did not: add_category defaults a new\ncategory to Column and nothing moved it, so all three views had none_cats\n{_Index, _Dim} and the stated input to the hazard was absent. An unprobed path\nreported as a negative result. Setting Product to Axis::None explicitly made the\nproperty test fail on the first run.\n\nMECHANISM. App::rebuild_layout calls recompute_formulas(\u0026none_cats) for the\nHOST's view, populating Model::formula_cache. project::grid_in on the elsewhere\npath builds the other view's GridLayout against that same, stale cache. The\ncache is keyed by the coordinates the recompute used, so with Product on None in\nthe host's view the Total cells are keyed WITHOUT a Product coordinate; the other\nview, which has Product on Column, asks for (Region, Product, _Measure=Total),\nfinds nothing, and -- because prune_empty is on -- drops both Total columns.\n\nMEASURED, two-view fixture, formula Total = SUM(Revenue):\n\n host on Second, projecting Second 6 columns incl. Total x Widget/Gadget\n host on Default, projecting Second 4 columns NO Total columns at all\n\nTHE FIX IS NOT LOCAL. grid_in takes \u0026App and recompute_formulas needs \u0026mut Model,\nso grid_in cannot recompute for the view it is projecting. Candidates, none\nchosen:\n\n - Key formula_cache by view (or by none_cats), so one Model can hold more than\n one view's derived values.\n - Recompute on the session's behalf at a \u0026mut App boundary (Sessions::project\n is one) and cache per session.\n - Declare it out of scope: formulas are evaluated for the host's view and a\n client on another view sees the host's derivation. That is a decision, not\n the current state, which is an accident.\n\nNote this is the SAME SHAPE as improvise-t6db: the elsewhere path of grid_in is\nonly as correct as the inputs it can rebuild, and prune_empty was the first such\ninput. none_cats is the second, and it is worse, because prune_empty was one\nline and this one cannot be fixed where it is observed.\n\nGUARDED MEANWHILE by a_hosts_none_cats_leak_into_another_views_formula_columns\nin ui/project.rs, a characterization test that asserts the bug. Its message says\nso: if it fails, the bug is fixed, and the none_cats guard should come out of\na_clients_grid_does_not_depend_on_which_view_its_host_is_on at the same time.\n\n## Steps to Reproduce\n\n1. Two views over one document, both with prune_empty on, with a formula.\n2. Put a category on Axis::None in view A and on Axis::Column in view B.\n3. Host on B: grid_in(app, \"B\") draws the formula's columns.\n4. Host on A: grid_in(app, \"B\") does not.\n","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:31:45Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:31:45Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-711n","title":"parse_number_format silently yields 0 decimals for the ,.2f spelling our own docs show","description":"Found by the adversarial review of PR #92, as a side finding while printing a test fixture's rendered cells and noticing they said \"1\" where the fixture had set format \",.2f\".\n\nparse_number_format (crates/improvise-core/src/format.rs:14) reads the decimal count as:\n\n fmt.rfind('.').and_then(|i| fmt[i + 1..].parse::\u003cu8\u003e().ok()).unwrap_or(0)\n\nFor \",.2f\" that takes everything after the dot — \"2f\" — and parses it as u8. It FAILS, so unwrap_or(0) yields ZERO DECIMALS. Silently.\n\n## Why this is not hypothetical\n- context/repo-map.md's own .improv format example documents `format: ,.2f`.\n- persistence/format.rs:109 writes `format: {}` with view.number_format verbatim, so whatever a user types round-trips into the file unchanged and is re-parsed the same wrong way on every load.\n- The trailing-f spelling is the Python/printf convention a user would most naturally reach for.\n\nSo a document whose author wrote the spelling our own documentation shows renders every number with zero decimals, and nothing reports it.\n\n## Steps to Reproduce\nSet a view's number_format to \",.2f\" and render any non-integer value. Expect 1.00; observe 1.\n\n## Acceptance Criteria\n- Decide which spellings are legal and say so in ONE place. The grammar is the source of truth for .improv, so if \",.2f\" is legal the parser must honour it; if it is not, loading must refuse it rather than silently reinterpreting it.\n- A test with the \",.2f\" spelling asserting the decimals actually applied. It must fail on today's code.\n- Whichever way it goes, the repo-map example and the format doc must agree with the parser — this bug is partly a documentation bug, and a fact stated twice is changed by grep.\n- Consider the neighbouring spellings while you are there: \".2\", \",.2\", \"0.00\", \"%.2f\".\n\nNOT verified: whether any shipped document or fixture uses the trailing-f spelling, and whether the import wizard can produce it.\n","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:29:53Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:29:53Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-711n","title":"parse_number_format silently yields 0 decimals for the ,.2f spelling our own docs show","description":"Found by the adversarial review of PR #92, as a side finding while printing a test fixture's rendered cells and noticing they said \"1\" where the fixture had set format \",.2f\".\n\nparse_number_format (crates/improvise-core/src/format.rs:14) reads the decimal count as:\n\n fmt.rfind('.').and_then(|i| fmt[i + 1..].parse::\u003cu8\u003e().ok()).unwrap_or(0)\n\nFor \",.2f\" that takes everything after the dot — \"2f\" — and parses it as u8. It FAILS, so unwrap_or(0) yields ZERO DECIMALS. Silently.\n\n## Why this is not hypothetical\n- context/repo-map.md's own .improv format example documents `format: ,.2f`.\n- persistence/format.rs:109 writes `format: {}` with view.number_format verbatim, so whatever a user types round-trips into the file unchanged and is re-parsed the same wrong way on every load.\n- The trailing-f spelling is the Python/printf convention a user would most naturally reach for.\n\nSo a document whose author wrote the spelling our own documentation shows renders every number with zero decimals, and nothing reports it.\n\n## Steps to Reproduce\nSet a view's number_format to \",.2f\" and render any non-integer value. Expect 1.00; observe 1.\n\n## Acceptance Criteria\n- Decide which spellings are legal and say so in ONE place. The grammar is the source of truth for .improv, so if \",.2f\" is legal the parser must honour it; if it is not, loading must refuse it rather than silently reinterpreting it.\n- A test with the \",.2f\" spelling asserting the decimals actually applied. It must fail on today's code.\n- Whichever way it goes, the repo-map example and the format doc must agree with the parser — this bug is partly a documentation bug, and a fact stated twice is changed by grep.\n- Consider the neighbouring spellings while you are there: \".2\", \",.2\", \"0.00\", \"%.2f\".\n\nNOT verified: whether any shipped document or fixture uses the trailing-f spelling, and whether the import wizard can produce it.\n","status":"closed","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:29:53Z","created_by":"Edward Langley","updated_at":"2026-09-21T17:56:34Z","closed_at":"2026-09-21T17:56:34Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-e5pf","title":"Command::ChangeMode(Normal) is on the wire and reduce_view folds it without the records resolution","description":"WIDENED 2026-09-21 after a sweep of resolve.rs. The fault is ONE SUBSTITUTION AT THREE DECISION POINTS, not one command on the wire, and the first of the three composes a malformed write rather than declining.\n\n wherever the host asks layout.is_records_mode(),\n the client asks state.view.mode.is_records()\n\nCommand::ChangeMode(Normal) is HOW the client reaches the bad state; the substitution is WHY the bad state does damage. The two want fixing together, and the second decides whether enriching the projection is enough.\n\n## How the client reaches the bad state\nimprovise-sihr made the TUI settle AppMode::Normal against the layout the batch ends on (App::settle_normal_mode). The wire carries Command::ChangeMode(AppMode::Normal); improvise-protocol/src/reduce.rs:195 folds it as `view.mode = mode.clone()` with no resolution. improvise_protocol::resolve — re-exported as improvise_wasm_client::resolve, and the LIVE browser resolver; keys.rs holds only BrowserKey — emits it at resolve.rs lines 369, 692, 972, 1461, 1613 and 2045.\n\nDemonstrated with a lockstep-shaped probe (resolve + ClientState::fold against reduce_key; enter records with R, open a panel, Esc):\n left: Normal (client)\n right: RecordsNormal (host)\n\n## What the bad state then breaks\nresolve.rs:685 commit() early return — host counterpart CmdContext::is_records_mode() via CommitAndAdvance. The client is meant to DECLINE a records commit it cannot compose. With the mode wrong it does not decline: it composes typed_cell_command(id, \u0026text) and a Basis claim against a records grid, whose columns are [coords.., +cat, measures.., +] (gotcha 1). That is A WRONG WRITE, strictly worse than the black hole it was written to produce.\n\nresolve.rs:708 create_column() exit mode — picks Normal where it should pick RecordsNormal, re-entering the same wrong state after every column creation. Self-sustaining.\n\nresolve.rs:744 advanced() — host counterpart commit.rs:547, `at_bottom_right \u0026\u0026 ctx.is_records_mode()` -\u003e AddRecordRow. The client returns None to decline what it cannot name; with the mode wrong it clamps and re-enters, so TAB AT THE BOTTOM-RIGHT OF A RECORDS GRID SILENTLY STOPS APPENDING ROWS. That is the records-mode Tab contract failing silently.\n\n## Why the suite cannot see any of it\nlockstep.rs is blind to the class BY CONSTRUCTION. Its records coverage (typing_at, records_pair) sets host.view_state.mode = AppMode::records_editing() by ASSIGNMENT and seeds the client with client.view = app.view_state.clone() — it starts from a state where mode and layout ALREADY AGREE, and every divergence above requires them to disagree. A guard seeded on agreement cannot detect drift: the same vacuity shape as gotcha 36's bystander already sitting on the relayed value.\n\n## Acceptance Criteria\n- lockstep DRIVES THE KEYS into records mode and back out of a minibuffer rather than hand-placing the mode. Without this, any fix is unguarded.\n- The projection carries what the client needs to settle the mode (gotcha 47: where a client cannot compose the answer, ENRICH THE PROJECTION rather than moving resolution back).\n- All three decision points answer the same question the host answers.\n- Or: a written argument that ChangeMode(Normal) should not be on the wire, plus its removal — which still leaves the substitution at the three sites.\n\nDo NOT close this by having the client re-derive records-ness from the axes: that is a second derivation, improvise-edp's disease.\n","status":"open","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:04:35Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:29:10Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-9c9o","title":"A geometry effect right after a mid-batch SwitchView skips the cursor check entirely","description":"Introduced by PR #78 round 2. PASSES on pre-PR main (0947fd0) and on round-1 head (bcccfde); FAILS at d9cb16c.\n\nRound 1 compared frame hashes across a view change, which threw a cursor away spuriously. Round 2 added a 'was == self.view_name()' guard, which fixes that — but it also exempts the FIRST geometry effect after a mid-batch SwitchView from the check altogether, because there is no basis for the new view yet, so the code re-bases and skips.\n\n## Steps to Reproduce\n app.apply_effects(vec![SwitchView(\"Second\"), TransposeAxes]);\nBoth preconditions hold: the switch lands, and row_count() really changes. Yet cursor_epoch does not move and the cursor stays at (2,0) in a transposed grid.\n\n## Acceptance Criteria\n- The probe above lands as a guard.\n- The basis re-bases to the new view AT THE SWITCH rather than at the next geometry effect — i.e. the basis follows active_view changes, which SwitchView being ViewOnly currently prevents. Not a one-liner.\n\nWHY THE DIRECTION MATTERS: round 1 threw away a cursor that should have been kept; round 2 keeps one that should have been thrown away. Gotcha 35's asymmetry says under-reporting is the expensive side — a stale cursor has nothing in the stream to correct it, where an extra invalidation costs one redundant projection.\n\nReachability is the same narrow class as the bug it replaced: a . -chained : line, or any future Binding::Sequence composing the two. DrillIntoCell and EnterRecordsMode hit the pattern constantly but their target view is freshly created, so it is benign there. Merged on that basis rather than blocked.","status":"closed","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T05:15:07Z","created_by":"Edward Langley","updated_at":"2026-09-21T17:09:50Z","closed_at":"2026-09-21T17:09:50Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-iot6","title":"render_csv builds GridLayout::new, so CSV export carries rows the view prunes","description":"FOURTH instance of the GridLayout::new versus with_frozen_records trap, and the first one in production that a user can see.\n\ncrates/improvise-io/src/persistence/export.rs:33 builds GridLayout::new(\u0026workbook.model, view), which does NOT apply prune_empty. So :export in the TUI and the CLI export path both write rows the grid does not draw.\n\n## Steps to Reproduce\nFixture: Region {North, Service, South}, Revenue set for North and South only, prune_empty = true. The view draws 2 rows (assert this as the precondition).\n\nrender_csv output:\n Region,Revenue\n North,1\n Service, \u003c- pruned on screen, exported anyway\n South,1\n\n## Acceptance Criteria\n- render_csv honours the view's prune_empty, via the same path grid_in now uses.\n- A test whose precondition asserts the view really prunes (a row that is actually empty, pruning left ON in the view). A fixture that turns pruning off, or has nothing to prune, is vacuous — that exact vacuity was demonstrated while writing the improvise-t6db property.\n- None of export.rs's three existing tests mentions pruning; that is why this shipped.\n\nPre-existing, not a regression from PR #78. Found by the adversarial review of #78 while enumerating every GridLayout::new call site: three are outside test modules, two of those are test helpers, and this is the third.","status":"closed","priority":2,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T05:14:59Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:09:27Z","closed_at":"2026-09-21T11:09:27Z","dependency_count":0,"dependent_count":0,"comment_count":0}
@@ -198,10 +198,10 @@
{"_type":"issue","id":"improvise-pby","title":"2.2 Write the README","description":"Replace existing README.md with new one under 250 lines. Sections in order: Title, one-sentence pitch, inline demo GIF, why this exists (Lotus Improv reference), quick start, key bindings, installation (Nix/crates.io/prebuilt - no Homebrew), codebase overview, expectations disclaimer, license. No badges, TOC, or contributing guide.","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:07:32Z","created_by":"Edward Langley","updated_at":"2026-07-30T23:06:31Z","closed_at":"2026-04-09T07:02:52Z","close_reason":"README written with all 10 required sections, committed. License updated to Apache-2.0 per user request.","dependency_count":0,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"improvise-ihv","title":"2.1 Add Nix tooling for asciinema, VHS, and cargo-dist","description":"Modify flake.nix to add pkgs.asciinema and pkgs.vhs to dev shell nativeBuildInputs. Add cargo-dist if packaged in nixpkgs. Add runtime deps for VHS (ttyd, ffmpeg) if needed. Verify tools work via nix develop --command. Add nix run app or shell script for demo recording workflow.","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-09T04:07:30Z","created_by":"Edward Langley","updated_at":"2026-07-30T23:06:31Z","closed_at":"2026-04-09T07:30:19Z","close_reason":"asciinema, vhs, cargo-dist added to flake.nix dev shell and verified. cargo-dist wrapper enables 'cargo dist' subcommand. scripts/record-demo.sh added.","dependency_count":0,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"improvise-yk4","title":"Phase 2: README and demo artifacts","description":"The main launch work. README is 80% of the launch, demo artifacts are 20%. Covers Nix tooling for asciinema/VHS, writing the README, creating demo GIF and asciinema casts, and verifying all artifacts.","status":"closed","priority":2,"issue_type":"feature","assignee":"spot","owner":"el-github@elangley.org","created_at":"2026-04-09T04:05:49Z","created_by":"Edward Langley","updated_at":"2026-04-14T07:47:43Z","closed_at":"2026-04-14T07:47:43Z","close_reason":"All 5 subtasks closed; artifacts verified present and tracked in git (README.md, docs/demo.gif, docs/demo.tape, docs/casts/*.cast).","dependencies":[{"issue_id":"improvise-yk4","depends_on_id":"improvise-abz","type":"blocks","created_at":"2026-04-08T23:37:37Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-d4w","type":"blocks","created_at":"2026-04-08T23:37:36Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-ihv","type":"blocks","created_at":"2026-04-08T23:37:34Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-odx","type":"blocks","created_at":"2026-04-08T23:37:35Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-yk4","depends_on_id":"improvise-pby","type":"blocks","created_at":"2026-04-08T23:37:36Z","created_by":"Edward Langley","metadata":"{}"}],"dependency_count":5,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-hsoe","title":"check-evidence cannot see a duplicate heading, and three exist","description":"Found by the Wave I agent while merging PR #92, which had accumulated TWO entries headed `### a_window_narrows_the_cells_and_neither_the_totals_nor_the_widths` — the original, describing a fixture that no longer existed and repeating a claim the adversarial review had measured false, plus the superseding one. A reader hitting the first would have taken it as current.\n\n--check-evidence CANNOT SEE THIS. It verifies that a heading names a test that runs, and a duplicate heading names one just as well. Both copies pass.\n\n## Three already on main, verified 2026-09-21\n ### a_layout_change_moves_the_layout_version_and_no_record (test-evidence/command.md)\n ### the_two_front_ends_draw_the_same_column_header (test-evidence/tui.md)\n ### the_whole_frame_is_unchanged (test-evidence/tui.md)\n\nReproduce:\n grep -rhoE '^### [A-Za-z_:][A-Za-z_0-9:]*$' context/test-evidence/*.md | sort | uniq -d\n\n## Why it matters beyond tidiness\ncontext/test-evidence records WHY a test can fail. Two entries under one heading means one of them is stale by construction — they cannot both describe the current test — and nothing indicates which. That is the same class as an evidence entry whose mechanism measurement contradicts (doctrine an_evidence_entry_that_measurement_contradicts_is_worse_than_none): a reader gets a confident wrong answer rather than no answer.\n\n## Acceptance Criteria\n- --check-evidence refuses a duplicate heading, naming the file and both line numbers. It should refuse rather than warn, per the precedent set for ambiguous leaf names in improvise-rm4s.\n- Prove it by injection: add a second copy of an existing heading and watch the checker fail by name.\n- Resolve the three above. Each needs a read to decide which copy is current, or whether they describe genuinely different aspects and want distinct names.\n- Consider whether the same check should apply ACROSS files, not only within one: a heading is keyed to a test name, and test names are unique workspace-wide once module-qualified (improvise-rm4s), so two files claiming the same test is also a defect.\n","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T12:39:28Z","created_by":"Edward Langley","updated_at":"2026-09-21T12:39:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-hsoe","title":"check-evidence cannot see a duplicate heading, and three exist","description":"Found by the Wave I agent while merging PR #92, which had accumulated TWO entries headed `### a_window_narrows_the_cells_and_neither_the_totals_nor_the_widths` — the original, describing a fixture that no longer existed and repeating a claim the adversarial review had measured false, plus the superseding one. A reader hitting the first would have taken it as current.\n\n--check-evidence CANNOT SEE THIS. It verifies that a heading names a test that runs, and a duplicate heading names one just as well. Both copies pass.\n\n## Three already on main, verified 2026-09-21\n ### a_layout_change_moves_the_layout_version_and_no_record (test-evidence/command.md)\n ### the_two_front_ends_draw_the_same_column_header (test-evidence/tui.md)\n ### the_whole_frame_is_unchanged (test-evidence/tui.md)\n\nReproduce:\n grep -rhoE '^### [A-Za-z_:][A-Za-z_0-9:]*$' context/test-evidence/*.md | sort | uniq -d\n\n## Why it matters beyond tidiness\ncontext/test-evidence records WHY a test can fail. Two entries under one heading means one of them is stale by construction — they cannot both describe the current test — and nothing indicates which. That is the same class as an evidence entry whose mechanism measurement contradicts (doctrine an_evidence_entry_that_measurement_contradicts_is_worse_than_none): a reader gets a confident wrong answer rather than no answer.\n\n## Acceptance Criteria\n- --check-evidence refuses a duplicate heading, naming the file and both line numbers. It should refuse rather than warn, per the precedent set for ambiguous leaf names in improvise-rm4s.\n- Prove it by injection: add a second copy of an existing heading and watch the checker fail by name.\n- Resolve the three above. Each needs a read to decide which copy is current, or whether they describe genuinely different aspects and want distinct names.\n- Consider whether the same check should apply ACROSS files, not only within one: a heading is keyed to a test name, and test names are unique workspace-wide once module-qualified (improvise-rm4s), so two files claiming the same test is also a defect.\n","status":"closed","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T12:39:28Z","created_by":"Edward Langley","updated_at":"2026-09-21T18:04:26Z","closed_at":"2026-09-21T18:04:26Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-swre","title":"grid_of recomputes an unchanged grid every frame; memoise on the tiers that already exist","description":"Split out of improvise-jmjz on the adversarial review of PR #92, which closed jmjz on its own acceptance criterion (fuse the passes) and argued this is the separate, larger win.\n\ngrid_of recomputes the whole grid from scratch EVERY FRAME, even when nothing moved. After jmjz that is exactly one pass rather than three — 95ms at 18,000 cells, 1.32s at 90,000 — but the common case during scrolling, panel toggling and cursor movement is that the grid has not changed at all.\n\n## Why this is cheap to do here\nThe invalidation signal ALREADY EXISTS and is maintained for other reasons:\n- document_generation, layout_version, record_versions (gotcha 40, ADR 001's three tiers)\n- improvise-mcq4's view_shapes / view_frame observation\n\nA GridBody memo keyed on those takes an unchanged grid to roughly zero.\n\n## Why the remaining pass cannot be narrowed instead\nRuled out twice over, and the review measured it: totals sum EVERY row including ones a viewport scrolled past (gotcha 44), and widths are content-adaptive over every row. The benchmark's windowing ratio stayed 1.2-1.4x precisely because the surviving pass is the one a viewport cannot narrow. improvise-i5tk's decline stands.\n\n## Acceptance Criteria\n- A memo on the projected grid body keyed on the existing invalidation tiers; no NEW tier invented, because a second description of \"has the document changed\" is the disease (see the reasoning behind gotcha 39 and 45, where the trigger is the counter moving rather than a table of command variants).\n- Benchmarked: an unchanged grid across frames should approach zero, and the first frame must not regress.\n- The negative arm matters as much as the positive: a test that a grid which DID change is recomputed. A memo that never invalidates is fast and wrong, and it is the failure mode nothing will notice.\n\n## Second-order, much bigger, not this ticket\n~5us per cell at 18k is dominated by evaluate_aggregated doing a matching_values set intersection per cell (gotcha 18). An incremental aggregate index is the deeper fix. 90,000 cells at 1.32s remains unusable; that case is labelled pathological and memoisation helps it most.\n","status":"open","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-09-21T11:30:15Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:30:15Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-4dpj","title":"The RecordsNormal-to-Normal arm of settle_normal_mode has zero coverage, and it is live","description":"Two gaps in settle_normal_mode's coverage, both live code with zero tests. The second was raised by ChatGPT Codex on PR #91 and is recorded here because the behaviour turned out CORRECT and only the guard is missing.\n\n## Gap 1 — the RecordsNormal-to-Normal arm (found by injection)\nReplacing settle_normal_mode's guard with a one-way version — matching only AppMode::Normal, dropping the RecordsNormal arm — leaves THE ENTIRE WORKSPACE SUITE PASSING.\n\nThe existing crossing test drives the LAYOUT both ways, but both branches end with `enter-mode normal`, so the mode handed to the settle is AppMode::Normal in BOTH directions. It discriminates \"settles at all\" and \"settles against the ending layout\", but not \"settles from either mode\".\n\nThe arm is live, proved reachable in production:\n\n records mode -\u003e apply_effects([SetAxis { \"_Index\", Axis::None }])\n layout.is_records_mode() == false, mode was RecordsNormal\n -\u003e settle yields AppMode::Normal\n\nOne extra case in the existing loop — seed the mode as RecordsNormal, cross OUT — closes it, and the one-way injection must then fail by name.\n\n## Gap 2 — the SwitchView path, raised by Codex and VERIFIED CORRECT\nCodex reviewed PR #91 at c186909 (round 1, the per-effect resolution) and pointed out that SwitchViewAtCursor emits [SwitchView(target), ChangeMode(Normal)]: open the view panel over a records view, select a pivot view, and round 1 would leave RecordsNormal over the pivot grid, with the inverse leaving Normal over records.\n\nRound 2 (a10e95e) moved the settle to ONE call after rebuild_layout at the end of apply_effects, so it resolves against the POST-batch layout. Verified on main at ce895ff with a throwaway probe driving a real `:` line that crosses records-ness within one batch:\n\n start_records=true -\u003e ended_records=false mode=Normal\n start_records=false -\u003e ended_records=true mode=RecordsNormal\n\nCorrect in both directions. So the behaviour Codex named is fixed.\n\nTHE GUARD IS STILL MISSING. Codex asked specifically for \"a regression that switches between records and pivot views through the view panel\", and no such test exists. The path it names is different from the one the crossing test drives (toggle-records-mode, not SwitchView), and records-ness is auto-detected from the axes (gotcha 1), so a view switch can change it.\n\n## Acceptance Criteria\n- The gap-1 case added to the existing loop; the one-way-guard injection fails by name.\n- A regression driving SwitchViewAtCursor over the view panel, records view to pivot view AND back, asserting the mode matches the grid the batch ends on.\n- Both must fail if the settle is moved back before rebuild_layout.\n","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:29:18Z","created_by":"Edward Langley","updated_at":"2026-09-21T17:13:17Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-26dm","title":"The fourth length-not-completeness list: cases.len() == 13 in ui/effect.rs","description":"improvise-5ubs is closed for the two CYCLIC all() functions, by reading serde's\nderive-generated variant list out of its \"unknown variant ... expected one of\"\nmessage and diffing it against the names the cycle reached. That closes the\nAppMode and Rejection halves and the mechanism generalises to any serde enum.\n\nIt does NOT reach the FOURTH instance 5ubs records in its notes:\nlayout_geometry_effects_actually_change_document_config in\ncrates/improvise-command/src/ui/effect.rs, whose assert_eq!(cases.len(), 13) is\nover a hand-built list of EFFECTS, not of enum variants. serde has nothing to\nsay about it: an effect is a struct implementing a trait, so there is no derived\nenumeration to diff against.\n\nThe general statement stands and is the reason this is worth its own ticket:\ndoctrine(a_list_checked_for_length_reads_like_a_list_checked_for_completeness).\nA count catches a case being DELETED from the list and never one being ABSENT\nfrom it, and the two are written identically as an assert_eq over a number. The\nadversarial review of PR #78 added a fourteenth pub struct declaring\nEffectKind::Model(ModelChange::LayoutGeometry), pointed at by nothing, and the\nwhole suite stayed green.\n\nWhat would close it: an enumeration of the effects the compiler maintains. The\ncandidates are a proc-macro over the impl blocks, or inventory-style\nregistration, or reading the Debug names off a derived list the way\nevery_model_command_names_the_effect_of_the_same_name already does for the\nModelCommand mapping -- that last one is the closest existing precedent in this\nrepo and is worth looking at first.\n\nNot fixed in wave I because ui/effect.rs was another wave's ground that round,\nand because a fourth hand-rolled attempt in a fourth place is the disease 5ubs\nnames.\n","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:09:56Z","created_by":"Edward Langley","updated_at":"2026-09-21T11:09:56Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-4dpj","title":"The RecordsNormal-to-Normal arm of settle_normal_mode has zero coverage, and it is live","description":"Two gaps in settle_normal_mode's coverage, both live code with zero tests. The second was raised by ChatGPT Codex on PR #91 and is recorded here because the behaviour turned out CORRECT and only the guard is missing.\n\n## Gap 1 — the RecordsNormal-to-Normal arm (found by injection)\nReplacing settle_normal_mode's guard with a one-way version — matching only AppMode::Normal, dropping the RecordsNormal arm — leaves THE ENTIRE WORKSPACE SUITE PASSING.\n\nThe existing crossing test drives the LAYOUT both ways, but both branches end with `enter-mode normal`, so the mode handed to the settle is AppMode::Normal in BOTH directions. It discriminates \"settles at all\" and \"settles against the ending layout\", but not \"settles from either mode\".\n\nThe arm is live, proved reachable in production:\n\n records mode -\u003e apply_effects([SetAxis { \"_Index\", Axis::None }])\n layout.is_records_mode() == false, mode was RecordsNormal\n -\u003e settle yields AppMode::Normal\n\nOne extra case in the existing loop — seed the mode as RecordsNormal, cross OUT — closes it, and the one-way injection must then fail by name.\n\n## Gap 2 — the SwitchView path, raised by Codex and VERIFIED CORRECT\nCodex reviewed PR #91 at c186909 (round 1, the per-effect resolution) and pointed out that SwitchViewAtCursor emits [SwitchView(target), ChangeMode(Normal)]: open the view panel over a records view, select a pivot view, and round 1 would leave RecordsNormal over the pivot grid, with the inverse leaving Normal over records.\n\nRound 2 (a10e95e) moved the settle to ONE call after rebuild_layout at the end of apply_effects, so it resolves against the POST-batch layout. Verified on main at ce895ff with a throwaway probe driving a real `:` line that crosses records-ness within one batch:\n\n start_records=true -\u003e ended_records=false mode=Normal\n start_records=false -\u003e ended_records=true mode=RecordsNormal\n\nCorrect in both directions. So the behaviour Codex named is fixed.\n\nTHE GUARD IS STILL MISSING. Codex asked specifically for \"a regression that switches between records and pivot views through the view panel\", and no such test exists. The path it names is different from the one the crossing test drives (toggle-records-mode, not SwitchView), and records-ness is auto-detected from the axes (gotcha 1), so a view switch can change it.\n\n## Acceptance Criteria\n- The gap-1 case added to the existing loop; the one-way-guard injection fails by name.\n- A regression driving SwitchViewAtCursor over the view panel, records view to pivot view AND back, asserting the mode matches the grid the batch ends on.\n- Both must fail if the settle is moved back before rebuild_layout.\n","status":"closed","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:29:18Z","created_by":"Edward Langley","updated_at":"2026-09-21T18:04:27Z","closed_at":"2026-09-21T18:04:27Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-26dm","title":"The fourth length-not-completeness list: cases.len() == 13 in ui/effect.rs","description":"improvise-5ubs is closed for the two CYCLIC all() functions, by reading serde's\nderive-generated variant list out of its \"unknown variant ... expected one of\"\nmessage and diffing it against the names the cycle reached. That closes the\nAppMode and Rejection halves and the mechanism generalises to any serde enum.\n\nIt does NOT reach the FOURTH instance 5ubs records in its notes:\nlayout_geometry_effects_actually_change_document_config in\ncrates/improvise-command/src/ui/effect.rs, whose assert_eq!(cases.len(), 13) is\nover a hand-built list of EFFECTS, not of enum variants. serde has nothing to\nsay about it: an effect is a struct implementing a trait, so there is no derived\nenumeration to diff against.\n\nThe general statement stands and is the reason this is worth its own ticket:\ndoctrine(a_list_checked_for_length_reads_like_a_list_checked_for_completeness).\nA count catches a case being DELETED from the list and never one being ABSENT\nfrom it, and the two are written identically as an assert_eq over a number. The\nadversarial review of PR #78 added a fourteenth pub struct declaring\nEffectKind::Model(ModelChange::LayoutGeometry), pointed at by nothing, and the\nwhole suite stayed green.\n\nWhat would close it: an enumeration of the effects the compiler maintains. The\ncandidates are a proc-macro over the impl blocks, or inventory-style\nregistration, or reading the Debug names off a derived list the way\nevery_model_command_names_the_effect_of_the_same_name already does for the\nModelCommand mapping -- that last one is the closest existing precedent in this\nrepo and is worth looking at first.\n\nNot fixed in wave I because ui/effect.rs was another wave's ground that round,\nand because a fourth hand-rolled attempt in a fourth place is the disease 5ubs\nnames.\n","status":"closed","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T11:09:56Z","created_by":"Edward Langley","updated_at":"2026-09-21T18:04:28Z","closed_at":"2026-09-21T18:04:28Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-e8no","title":"A wide numeric category now refuses a fit that used to ignore it","description":"improvise-zv8t forces every document category to `Forced::Categorical`, which is\nright (a `Year` category with items 2024/2025 is a coordinate, not a number).\nThe cost: a WIDE numeric category one-hots past `MAX_FEATURES` and refuses the\nwhole fit, where before it encoded as one numeric feature and a fit that never\nsplit on it succeeded.\n\nMeasured on 100 records with a 100-item numeric `Year` beside a `Cost` that\npredicts `Score` on its own:\n- pre-zv8t: fit lands on `IF(Cost \u003c 48.015...)`; `Year` never appears.\n- post-zv8t: preflight refuses at 101 features.\n\nPinned by `a_wide_numeric_category_now_refuses_a_fit_that_used_to_ignore_it`\nin crates/improvise-gp/src/fit.rs, and argued in context/code/gp.md section 11.\n\nThe workaround the refusal names works: mark the category a Label and the\npre-zv8t feature set is restored exactly. What is lost is the case where the\nuser wants neither -- a wide numeric category left alone while the fit uses the\nmeasures.\n\n## Steps to Reproduce\n\nBuild a model with a Regular category `Year` holding 100 numeric items\n(1900..1999), a Label category for record identity, and two measures `Cost` and\n`Score` where Score = 4000 if Cost \u003e 50 else 1000. Put `Year` on Axis::Row and\n_Measure on Axis::Page. Run `fit_measure(.., \"Score\", ..)`.\n\nPre-zv8t it lands on `IF(Cost \u003c 48.015..)`. Post-zv8t it is refused with\n\"would encode 101 features from 100 records\". The fixture is\n`wide_category_beside_a_good_measure()` in crates/improvise-gp/src/fit.rs.\n\n## Acceptance Criteria\n\nThe honest fix is a budget that can decline A COLUMN rather than the whole\ntable: `encoded_width` (and the exact check beside it) would drop the widest\ncategory and report it as a `Dataset::warnings` notice -- gotcha 37(a), it comes\nback as DATA and nothing prints. That keeps the zv8t decision intact (a category\nis still never inferred numeric) and changes only what the budget does when it\ncannot afford one.\n\nDo NOT close this by making the forcing conditional on cardinality: that is\ninference by another name, and makes the ENCODING depend on a budget rather than\non what the document says.\n","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T10:55:32Z","created_by":"Edward Langley","updated_at":"2026-09-21T10:55:39Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-sxsf","title":"cargo test -- --list cannot name the binary, so two crates could share a full test path","description":"MEASURED 2026-09-21 (PR #88). Re-verified latent: 1444 names, zero duplicate full paths, zero duplicate (crate, path) pairs.\n\nTHE PROPOSED FIX IS DECLINED ON TWO INDEPENDENT GROUNDS, and the second is the interesting one:\n\n1. COST: 45.2s against 2.4s for the one workspace call (11 members, warm) — 18x. And --check-evidence needs the --ignored twin, so the honest figure is 22 cargo invocations.\n\n2. IT DOES NOT CLOSE THE CLASS IT NAMES. A package is not a binary. The improvise crate alone has FIVE integration test binaries beside its lib, and an integration test's top-level fn lists with NO MODULE PATH AT ALL — 60 of 1444 names are bare. So '-p improvise' prefixes all six identically and tests/a.rs::foo still cannot be told from tests/b.rs::foo.\n\nAny real fix is per-TARGET (--lib, --test \u003cname\u003e), not per-package. Whoever takes this should start there.\n\nWorth noting how this was found: the agent set out to time the per-package fix and discovered it does not work while reading the target list. The cost measurement was the excuse for reading it; the correctness finding was the payoff — an argument for measuring declines rather than asserting them.","status":"open","priority":3,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-21T04:49:16Z","created_by":"Edward Langley","updated_at":"2026-09-21T05:23:48Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"improvise-8shu","title":"Constrain the .improv grammar generator's strategy; improvise-io's lib binary is the run's outlier","description":"Split out of improvise-8ec3, whose second half (stop pinning one machine's number) is done. This is the first half, left open.\n\nimprovise-io's lib binary is an order of magnitude above every other binary in the run on every machine it has been measured on: 1.40-1.45s over three runs here (vs a ~0.06s median), 3.70-5.24s on the machine that filed 8ec3. The ratio is the portable signal; the seconds are not.\n\nIts slow half is the grammar-walking generator in persistence/parse.rs plus the .improv roundtrip proptests at the default 256 cases. A grammar walk with no size bound generates deeply nested documents whose cost is not uniform across cases, so a few cases dominate.\n\ndoctrine(proptest_cases) rules out the cheap fix: constrain the strategy, do NOT lower the count below 256.\n\nAcceptance: improvise-io's lib binary is no longer an order of magnitude above the run's median, with the case count still 256 and the properties still covering groups, formulas, views, hidden items and pipe-quoting edges. Report the before/after ratio, not seconds.","status":"closed","priority":3,"issue_type":"task","owner":"el-github@elangley.org","created_at":"2026-09-21T04:14:01Z","created_by":"Edward Langley","updated_at":"2026-09-21T05:15:42Z","closed_at":"2026-09-21T05:15:42Z","dependency_count":0,"dependent_count":0,"comment_count":0}
context/code/command.md
+66
View File
@@ -732,6 +732,24 @@ layout (gotcha 23), so a batch that **crosses** — `:toggle-records-mode . ente
grid. One edit, after the rebuild, has no crossing case to get wrong, and it
also removes the per-edit coverage question three separate writers created.
**The guard is two-armed and the crossing test could only see one arm**
(improvise-4dpj). `settle_normal_mode` matches `Normal | RecordsNormal`, and
replacing that with a one-way `Normal` left the whole workspace green: the `:`
crossing drives the layout both ways, but both of its branches end in
`enter-mode normal`, so the mode handed to the settle is `AppMode::Normal`
either way. It discriminates "settles at all" and "settles against the ending
layout", and not "settles from **either** mode" — three questions that look
like one until an injection separates them. The `RecordsNormal` arm is live:
`SetAxis { "_Index", Axis::None }` applied from a records grid leaves
`is_records_mode()` false with the mode still `RecordsNormal`.
**And records-ness is auto-detected from the axes (gotcha 1), so a view switch
crosses it too.** `SwitchViewAtCursor` emits `[SwitchView(target),
ChangeMode(Normal)]`, which is a genuinely different path from the toggle the
`:` test drives — nothing in it says "records", the crossing falls out of the
target view's axis map. Moving the settle back before `rebuild_layout` is the
injection for both, and it fails each by name.
**This fixes the TUI and not the browser, and that is LIVE (improvise-e5pf).**
`Command::ChangeMode(AppMode::Normal)` is on the wire and `reduce.rs` folds it
as `view.mode = mode.clone()` with no resolution, so the browser front end still
@@ -1337,6 +1355,54 @@ cells will bump their records when it shares a batch with a `CellData` one — a
over-bump, costing a rejection that need not have happened, which is the same
asymmetry `Dirty::of` is deliberately sloppy in.
**The case list is checked for completeness, not for length** (improvise-26dm).
`layout_geometry_effects_actually_change_document_config` walked a hand-built
list under an `assert_eq!(cases.len(), 13)`, and a count catches a case being
*deleted* and never one being *absent* — the two are written identically. The
adversarial review of PR #78 added a fourteenth `pub struct` declaring
`LayoutGeometry` and the suite stayed green.
`doctrine(a_list_checked_for_length_reads_like_a_list_checked_for_completeness)`.
improvise-5ubs's answer — read serde's derive-generated variant list out of its
`unknown variant … expected one of` message — cannot reach this one: an effect
is a struct implementing a trait, so there is no derived enumeration of effects
at all. But gotcha 38 says `ModelCommand` is derived from the model effects,
with `model_effects` mapping each variant back to the effect it came from, so
the list is now diffed against the `LayoutGeometry` effects *reachable through
`ModelCommand`*, and the failure names the missing effect rather than a number.
The two lists agree exactly today, which is itself the evidence the
substitution lost nothing.
**The right-hand side had to be bounded before it could be believed** (PR #96,
found independently by this branch's review and by ChatGPT Codex).
`every_model_command()`'s doc comment claimed the enumeration was
compiler-forced — `variant_name` matches exhaustively, so a new variant fails
the build. That inference is invalid: the build fails in *`variant_name`*, and
nothing obliged the value list to gain an entry. Deleting one from it failed
this check and nothing else in the workspace.
So the list is now itself checked, by `improvise-5ubs`'s mechanism:
`the_enumeration_names_every_variant_the_derive_declares` diffs it against
`variants::declared::<ModelCommand>()`, serde's derive-generated variant list.
That makes the chain **derive → `every_model_command()` → `model_effects` →
case list**, every link checked, and the whole thing is what the ticket asked
for. `variants` was `#[cfg(test)] pub(crate)`, so reaching it costs a
`test-support` feature on `improvise-protocol` gating an optional `serde_json`
— **off in every shipped build**, which is why the wasm gate is unaffected.
The assertion message still names **both** directions, because a short
right-hand side reads as a spurious case and invites deleting the left,
degrading both sides together — the failure the ticket exists to close.
**Its bound is stated rather than assumed**: this closes the effects on the
wire, and an effect declaring `LayoutGeometry` that no `ModelCommand` names is
still invisible — which is precisely the probe variant PR #78 injected. That is
an honest residual and not a fix, and it is the reason the ticket asked for a
compiler-maintained enumeration of *effects*: none exists without a proc macro
or inventory-style registration, and a fourth hand-rolled list in a fourth
place is the disease. A second `[…; 13]` array survives one function away in
`every_layout_geometry_effect_is_visible_to_the_narrow_fingerprint` and is
filed, not smuggled in.
On abort, the loop resets `abort_effects` so the next dispatch starts clean and
records `batch_aborted` where a caller can still read it.
context/code/core.md
+53
View File
@@ -641,3 +641,56 @@ targets, which is what the column list shows anyway.
And the list is **not** filtered to visible items. Hidden items are still cycled,
because that is what the TUI has always done, and making the browser differ would
be the divergence rather than the fix. Changing it is a change to both, together.
## The number format — `format.rs` (improvise-711n)
A view's `number_format` is a **printf-style picture, and only that**. A `,`
anywhere turns on thousands grouping; a `.` introduces the precision, which is
the **leading digit run after the last `.`**; anything following those digits —
a conversion letter — is ignored. So `,.0`, `,.2`, `.2`, `,.2f` and `%.2f` all
name what they look like they name.
**Why the trailing `f` is accepted rather than refused.** It round-trips into
documents today and always has. `improv.pest` spells the line
`format_line = { "format: " ~ rest_of_line ~ NEWLINE ~ blank_lines }` — the
grammar takes the format string as an **opaque line**, so `format: ,.2f` is a
document the grammar accepts; `:set-number-format` stores whatever the user
typed verbatim (`cmd/effect_cmds.rs`), and `format_md` writes `view.number_format`
back out unchanged. Refusing the spelling at load would therefore reject
documents this repo's own grammar admits and its own `.improv` example spells,
to no benefit: by gotcha 8 the format is display-only, so misreading it costs a
wrong pixel and can never cost a wrong number. A refusal here buys nothing and
costs a document.
**Why printf and not Excel.** The two conventions disagree on exactly the
spelling that distinguishes them: `.00` is *precision zero* to printf and *two
places* to Excel, so a parser that tried to honour both would have to guess. One
is picked, and it is the one every spelling in this repository already uses.
Consequences worth stating rather than discovering: `0.00` reads as
`(false, 0)`, and an Excel picture such as `#,##0.00` reads as
`(true, 0)` — grouped, no decimals.
**Nothing is validated, deliberately.** A spelling the rule cannot read names no
precision and formats with none; there is no error channel and no `Result`,
because the reader is display-only and four call sites would have to thread one.
**What the bug was, and why six tests sat on top of it.** The precision was read
as `fmt[i+1..].parse::<u8>()` over the *whole* tail, so `"2f"` failed to parse
and fell through to `unwrap_or(0)` — `,.2f` displayed **zero** decimals, in the
TUI, in the browser projection and in CSV export. `",.2f"` and its `,.3f`/`,.4f`
siblings are what this workspace's own fixtures reach for, in about ten tests,
and none could see it: they assert the stored *string* round-trips or use it as
an opaque value to dirty a widget, or — in `export_csv_quotes_commas_per_rfc4180`
— they assert the rendered value `starts_with("1,200")`, which `1,200` and
`1,200.00` both satisfy. A prefix check over a value whose *suffix* is the thing
in question answers a neighbouring question, which is why that one is now an
exact equality.
**Only two of the three consumers are guarded.** Reverting the fix fails
`format.rs`'s own unit tests and `export_csv_quotes_commas_per_rfc4180`, and
nothing else in the workspace: `ui/project.rs::grid_of` (the browser projection)
and `ui/app.rs` both call `parse_number_format` and neither has a test that
renders a cell and asserts its digits. That gap is named rather than closed —
those files are `improvise-command`, which was another wave's ground when this
landed — and it is the more useful half of this entry, because it says where the
next regression would go unseen.
context/code/gp.md
+167 -7
View File
@@ -561,14 +561,79 @@ the fix (`Label`), because that is what the user actually wants.
side of the cost: a category with a distinct value per record allocates a
rows × rows one-hot *during* preprocessing, so the refusal arrives after the
memory and the O(rows²) work have been spent. It approximates `preprocess`'s
type inference rather than sharing it — a column whose present cells all parse
as a number is one feature (two with a missing indicator), anything else is
counted at its distinct-value count. Copying the predicate is the lesser evil;
type inference rather than sharing it. Copying the predicate is the lesser evil;
the alternative is exporting the inference and calling this a second
implementation of it. **The cost of the copy drifting is bounded in both
directions by the exact check** on `Dataset::feature_names.len()`:
over-counting costs a refusal the exact check would not have made,
under-counting only lets the exact check do its job.
implementation of it.
**The debt that copy carries is that the preflight owes EVERY branch
`preprocess` can take, and it paid two of three.** `preprocess` infers
`Numeric → Date → Categorical`, in that order, and the preflight had no date arm
at all: a column whose cells parse as neither numbers nor a forced category fell
through to the categorical count. So a **measure column holding dates** — one
distinct timestamp in every row — promised **1** and materialised **5**, because
`build_date` emits `epoch_day/year/month/day/dow`, and **7** once any cell
carries a time (improvise-x1ns, second instance, found by ChatGPT Codex on #97).
That is an under-count of four or six on a single column, against a budget of 64.
The two instances are the same defect one inference-branch apart, which is why
the shape is now structural rather than remembered. `inferred_kind` reproduces
`preprocess`'s match arm for arm and returns a `Forced`, and `encoded_width`
matches that **with no wildcard** — so a fourth `Forced` variant is a compile
error here instead of silently taking the categorical arm, which is exactly how
`Date` was missed. The `.missing` flag is added once outside the match, because
every branch owes it on the same terms.
The count each arm owes:
| `preprocess` builds | features |
|---|---|
| `build_numeric` | 1 |
| `build_date` | 5, or 7 if any present cell has a time |
| `build_categorical` | distinct present values, **+1** for the reserved `(missing)` |
plus `col.missing` wherever any cell is missing, and clamped to at least 1.
**The two directions of a drift do not cost the same, and this paragraph said
they did** (improvise-x1ns). It read: *the cost of the copy drifting is bounded
in both directions by the exact check* — over-counting costs a needless refusal,
under-counting "only lets the exact check do its job". That is wrong, and it is
wrong against the argument **two paragraphs above it**: the exact check runs on
`Dataset::feature_names.len()`, which exists only *after* `preprocess` has
materialised the matrix. So it bounds the **error message**, never the **cost** —
which is precisely the thing the preflight was introduced to bound. Over-counting
is cheap; under-counting spends the memory and the O(rows²) work the preflight
exists to prevent, and then reports it politely.
The live instance was the `(missing)` column: the categorical arm counted
`present` values only, so a forced category with N distinct present values and
one record lacking that coordinate promised N and materialised N+1. `has_missing`
was already computed and used in the numeric arm alone — **two arms of one match
disagreeing about what a column costs**, which is improvise-zv8t's forcing-by-name
/ budgeting-by-index defect one level in. That one was repaired by making the
ordering a data dependency and so a compile error; this one has no such shape
available, so the guard is a test that asks `preprocess` what it actually built
rather than asserting a number
(`the_preflight_counts_the_missing_category_a_forced_column_materialises`).
The previous wave's review saw a version of this and reasoned it away as *"read,
not injected"* and *"bounded by the exact check"*. Both halves of that are the
failure mode this file exists to catch: the first declined to run the experiment,
and the second repeated the sentence above instead of checking it. **An argument
that a cost is bounded is checkable, and this one was checkable the day it was
written**: `encoded_width` landed 2026-09-19 and the sentence 2026-09-20, so the
code it described was one commit old and one file away.
`encoded_width` remains **advisory** in the direction that is safe: it may
over-count, and a column whose cells are *entirely* missing is counted as one
where `preprocess` skips it altogether. It must never under-count.
**That sentence was written as a proposed `kb.pl` invariant while the tree held a
counter-example**, in the same change that fixed the categorical one — the date
branch was missing the whole time and nothing in the hand-back noticed. An
invariant proposed from the defect you just fixed generalises the *fix*, not the
code; the thing that would have caught it is the question "what else does
`preprocess` do that I have not mirrored", which is a question about the other
function and not about the bug.
`MAX_RECORDS = 20_000` bounds the *work*, which the width bound does not: a
one-feature table with a million records still runs cross-validation, the final
@@ -586,6 +651,101 @@ sequence, five-fold CV included. There is deliberately **no timing test**: a
wall-clock assertion is flaky on shared CI and would fail for a reason that is
not this code's. Re-measure after changing the numbers.
### Forcing a numeric category is a *representability* choice, not a budget one
improvise-zv8t made `fit_measure` force every document category to
`Forced::Categorical`, and improvise-e8no records the cost: a hundred-item
numeric category one-hots to a hundred columns, past `MAX_FEATURES`, and is
refused where inference used to read it as one numeric column and the fit
succeeded. **improvise-e8no is declined, and the refusal is correct.**
The ticket rules out making the forcing conditional on cardinality, and rightly:
that is inference by another name, and it makes the *encoding* depend on a budget
rather than on what the document says. But the stronger reason to decline is that
the alternatives are not merely distasteful — they are **unstorable**, and the
code already says so.
A coordinate becomes an `Expr` **only by passing through `IS`**: `is_expr` is
`identifier ~ sp+ ~ ^"IS" ~ sp+ ~ filter_value`, a single-value equality, and
there is no `IN` and no ordering over a category's items. (`where_clause` and
`inline_where` are coordinate predicates too, and also single-value equalities —
but they *gate* a formula or an aggregate rather than being an `Expr`, which is
gotcha 37(d)'s distinction and is why `IS` is the one that matters here.) So
whatever a tree says about a category, the emitted formula names its items **one
at a time**.
**A first draft of this paragraph said that made one-hot the only spellable
encoding. That is false**, and an adversarial review disproved it by running
`parse_formula` rather than reading the grammar: `is_expr` is a `primary`, so it
is an `Expr` yielding 1/0, and `cmp_op`'s operands are full `expr`s — so an
ordinal split *is* spellable, as a weighted indicator sum
(`IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, …)`). The claim is
recorded here rather than quietly deleted because it is the shape this file keeps
catching: a grammar argument made by reading the rule that looked relevant
instead of running the parser.
So **ordinal is not ruled out by spellability, and is ruled out anyway**, for two
reasons that survive. It invents a **metric the document does not assert** — a
category's items carry insertion order, not magnitude, so `Year < 3.5` means "the
first four items as they happen to be ordered" and stops meaning that when
someone reorders them. And the width it saves does not stay saved: the *encoding*
would be one feature, but the *formula* still spells one `IS` per item, so it
trades a `MAX_FEATURES` problem for a formula-size problem while adding a false
order. What the current emitter does with it is narrower and is not the argument:
`spell()` maps `FeatureRole::Numeric` over a *category* column to
`Artefact::NumericCategory`, because a bare reference to the category's name
would resolve to a `_Measure` item of the same name (improvise-8xhy) — a fact
about the bare-ref spelling, not about ordinals in general. **Hashing** is ruled
out outright: a bucket has no name in the document, so it cannot even be refused
informatively.
And this is what makes the decline clean rather than resigned. The "fit that
previously succeeded" was one of two things, never a third. Either the tree
**never split** on the numeric category — in which case the column contributed
nothing to the rule, and marking it a `Label` gives a fit over the same remaining
features, inside the budget, which is exactly what the refusal already advises.
Or the tree **did** split on it, and `fitted_formula` refused the result as
`Artefact::NumericCategory` at storage time. So **no fit is lost that was both
storable and used the column**.
Two things that claim less than a first draft of this paragraph did, both worth
keeping. Marking the category a `Label` does **not** yield *the same fit*: the
feature count changes, so the RNG draw sequence in `random_tree`/`sample_thresh`
changes, so the champion for a given seed changes — gotcha 37(b)'s own rule,
applied to this paragraph. "The same fit" is a checkable cost claim and it is
false; what is true is a fit over the same remaining features. And the dichotomy
above is **argued**, from `spell()`'s exhaustive match over `FeatureRole`, not
asserted: what
`a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored` asserts is one
instance of its second branch, over one three-row fixture. That is the most a
test can do here, and saying so is the difference between evidence and a
flourish.
The test still had no predecessor. `NumericCategory` had no coverage before it —
**and it was one of three, not the only one**: grepping each `Artefact` variant
and each variant's `reason()` text, `MissingFlag` and `DatePart` are covered by
`the_missing_flag_and_the_date_sub_features_are_refused_too`, and
**`ValueOfAMeasure` and `AbsentCoordinate` have no test at all**, before this
change or after it. Two families remain uncovered; that is filed, not fixed
here.
What the decline does cost, stated rather than waved at: a user whose `Year`
category really is a quantity they want fitted **as a number** has no way to say
so through `fit_measure`. The document says `Year` is a category of items, and
that is what is honoured. The remedy is a document change — `Year` belongs in
`_Measure` if it is a measurement — and `crate::dataset` is `pub` and takes the
caller's `Overrides`, so a host that wants to offer the choice can (gotcha 37d).
Offering it *inside* `fit_measure` would be a new user-facing control, and
nobody has asked for one.
The refusal's wording was narrower than the rule and is now aligned in both
places that carry it: it said *a category with a distinct value per record
becomes one column per record*, which diagnoses the runaway shape and not
improvise-e8no's — a category of modest cardinality that is simply wider than
the budget. Both now say a category becomes one column per distinct item,
**whether its items read as numbers or not**, since that clause is the whole of
what changed under zv8t.
### `Quality` is an enum, not an `Option`
`cv_regress` pools no pairs below two folds, and scoring an empty pool gives an
context/code/pr-review.md
+39
View File
@@ -0,0 +1,39 @@
# Reading a PR's reviews
Twenty PRs merged in one campaign with their reviews unread. CI was green and
an adversarial agent had been dispatched, which felt like the whole gate. Two
ChatGPT Codex reviews were found afterwards on merged PRs; both were correct
and one was a live P2. Three more were found on the next three PRs — one per
PR, every one a P1.
`scripts/pr-reviews <n>` prints them. `--wait <sec>` blocks until one lands
against the current head.
## Why a review is easy to miss here
It arrives as a **review**, not a comment, so reading the comment thread does
not show it.
It posts under the repository **owner's** account — the same account this
repo's own agents post under — so **authorship does not distinguish it**. The
tell is placement: a Codex review *ends* with
`_Automated review by ChatGPT Codex (…)_`, where an agent's comment *opens*
with `_Posted by Claude (…)_`.
It may target an **earlier head**. `pr-reviews` says which, because that
changes what you do: a finding against an old commit may already be fixed or
may still be live, and only checking tells you which. Both cases have occurred
— one fixed by a later round, one live at the head after three more commits.
## Absence is not approval
Reviews arrive as the owner's quota allows. Observed latency is ten to fifteen
minutes after a push. `--wait` exists so that a merge four minutes after a
push does not race the review that was about to arrive; when it times out,
that is a decision to merge unreviewed rather than evidence of a clean bill.
## Answer them on the thread
Including after the PR has merged. That is where the next reader looks, and a
finding that was declined or already-fixed is worth as much as one that was
taken — the reasoning is the artefact, not the verdict.
context/code/tooling.md
+31
View File
@@ -208,6 +208,37 @@ also has no `context/test-evidence/` and no JS roots, so the evidence half is
vacuously clean there and the cap is the only thing that can decide the exit
code — the isolation is what makes the assertion name one question.
### Two headings, one test
An orphan is a heading whose test nobody can find; the mirror image is a test
that **two** headings answer for, and it is invisible to every rule above —
each copy names a test that runs, so each copy passes. PR #92 shipped two
entries under one heading where the older described a fixture the test no
longer had and repeated a claim an adversarial review had measured false. A
reader meeting the older one first has no way to know a successor exists two
thousand lines down, so the stale copy reads as current. That is worse than an
orphan, which at least announces itself as broken.
It is a **refusal and not a warning**, on the precedent `improvise-rm4s` set for
the ambiguous leaf: the two are the same defect seen from opposite ends — one
name answered by two tests, and one test answered by two names — so they are
answered the same way. The refusal names the file and every line, in file order.
The key is the **resolved test, not the spelling**. `### f` beside
`### m::tests::f` is two headings for one test, and grouping by the string would
miss it precisely because qualification is the repair `rm4s` asks for; a heading
qualified in one file and bare in another would have been the first way this
came back. Resolution reuses the same leaf table the ambiguity rule uses, so a
leaf that is genuinely ambiguous canonicalises to itself and is reported as
ambiguous rather than as a duplicate — two *different* tests sharing a leaf are
not two entries for one test, and conflating them would file the wrong repair.
The check is over the whole directory rather than per file, because a heading
keys a test name and a test name is unique workspace-wide once qualified.
What it still cannot see, and neither can any of the others: an entry that
exists, names a live test, is the only one that does — and describes it
backwards.
## 3. The others
`scripts/kb` is the only writer of `prolog-sessions/kb.pl`; its own reasoning is
context/repo-map.md
+2
View File
@@ -571,6 +571,8 @@ job gates **eight** crates, not seven: `improvise-gp` was added (improvise-z5eg)
explicitly *not* as a bundle member but as a candidate, because staying
wasm-clean is free now and expensive to retrofit — so the bundle is seven and
the gate is eight, and neither number is the other one's typo.
**A feature that adds a dependency to a bundle crate must be checked with `cargo tree --edges normal`** — the default output includes **dev** edges, so a plain `cargo tree -p improvise-protocol --target wasm32-unknown-unknown` reports a dev-only `serde_json` as *present* and reads exactly like a leak. Check per-crate too, because the `wasm` CI job invokes each crate separately and feature unification cannot reach across that (improvise-26dm added a `test-support` feature on `improvise-protocol` on those terms; it is off in every shipped build).
`improvise-ws-server` is the host and is native only — tokio, tungstenite and
`Box<dyn Storage>` — so `improvise-wasm-client` cannot depend on it in anger. It
no longer needs to: both ends `pub use` `improvise_protocol::frame` and neither
context/test-evidence/clients.md
+11
View File
@@ -2556,6 +2556,17 @@ derived from the other — a first version computed the records precondition fro
the mode expectation, so relaxing the mode silently relaxed the precondition too
and the test failed for the wrong reason.
**It has a host-side twin, and the two are a pair rather than a duplicate.**
improvise-4dpj landed `switching_views_across_records_ness_settles_against_the_view_switched_to`
in `ui/app.rs` while this was being written: the same defect, the same batch
(`[SwitchView(to), ChangeMode(Normal)]`), the same crossing in both directions,
and independently the same phrase in its message — "settled against the view it
LEFT". One is the host settling after the batch against the layout it ends on;
this one is a client settling after the fold against the grid it holds. Neither
is redundant: nothing in the projection stream carries the host's `ViewState`
(gotcha 47), so a host-side guard cannot see a client's mode and the reverse.
Delete either and that side goes unwatched.
### the_mode_a_switch_lands_in_is_the_one_a_host_on_that_view_holds
The lockstep half of the entry above: it asserts the host itself answers
context/test-evidence/command.md
+110 -10
View File
@@ -1017,6 +1017,15 @@ Tier 2 moves, tier 3 does not. A shape change invalidates every in-flight
`CellData` batch through `layout_version` alone; it must not also masquerade as a
write to any record.
Fixture repaired under improvise-75ld: it asked for
`SetAxis { category: "Month", axis: Axis::Column }` against a fixture where
`on_category_added` had already seated `Month` on Column — so the "layout change"
was a no-op, and the test passed only because the tier was bumped off the
declaration. It now asserts the axis before and after, and asks for `Axis::Row`.
This is the failure that turned "reclassify three effects" into "stop reading the
tier off the classification". Injected fault: the same as
`both_shape_counters_move_for_every_reshaping_command` — fails naming `SetAxis`.
### a_loaded_document_is_dirty_because_its_path_still_names_the_old_one
**A loaded document is dirty, and that is a behaviour change with a reason**
@@ -3076,16 +3085,6 @@ that reshapes nothing fails loudly rather than proving nothing.
Injected fault: as above. Fails naming `SetAxis`, then `AddItem` once the earlier
cases are removed.
### a_layout_change_moves_the_layout_version_and_no_record
Pre-existing test, fixture repaired under improvise-75ld. It asked for
`SetAxis { category: "Month", axis: Axis::Column }` against a fixture where
`on_category_added` had already seated `Month` on Column — so the "layout change"
was a no-op, and the test passed only because the tier was bumped off the
declaration. It now asserts the axis before and after, and asks for `Axis::Row`.
This is the failure that turned "reclassify three effects" into "stop reading the
tier off the classification".
### a_configuration_change_that_draws_nothing_is_still_a_document_change
The third case the shape/config split creates, and the one that forced two
@@ -3600,3 +3599,104 @@ that had something to disagree about.
**Fault injected**: the `records` arm deleted from `grid_updates`. The test
fails on `the client is never told`. The negative arm (an unchanged flag is not
resent) is what stops "always push it" from passing.
### a_batch_that_leaves_records_settles_out_of_records_normal
improvise-4dpj, gap 1. `settle_normal_mode` matches `Normal | RecordsNormal`,
and replacing that with a one-way `AppMode::Normal` left the **entire workspace
suite passing**. The existing crossing test,
`a_batch_that_leaves_records_and_then_sets_normal_lands_in_normal`, drives the
layout both ways, but both of its branches end in `enter-mode normal`, so the
mode handed to the settle is `AppMode::Normal` in both directions. It
discriminates "settles at all" and "settles against the ending layout", and not
"settles from **either** mode".
This drives the production repro: from a records grid,
`apply_effects([SetAxis { "_Index", Axis::None }])` leaves
`layout.is_records_mode()` false with the mode still `RecordsNormal`, so the
`RecordsNormal` arm is live and not defensive. Both the records-ness of the
grid and the mode before the batch are asserted as preconditions, so a fixture
that stopped entering records mode fails loudly rather than proving nothing.
Injected fault: `if matches!(self.view_state.mode, AppMode::Normal)`. Fails by
name, `left: RecordsNormal, right: Normal`. It is the only test in the
workspace that fails.
### switching_views_across_records_ness_settles_against_the_view_switched_to
improvise-4dpj, gap 2 — raised by ChatGPT Codex against PR #91 and verified
correct on `main` before being tested. `SwitchViewAtCursor` emits
`[SwitchView(target), ChangeMode(Normal)]`, and this is a **different path**
from the one gap 1's siblings drive: that test toggles records mode explicitly,
where a view switch crosses records-ness only because it is auto-detected from
the axes (gotcha 1). Nothing in the batch says "records".
Driven in both directions, with the crossing asserted before and after so a
fixture that stopped crossing fails rather than passes. Injected fault: move
`settle_normal_mode()` back above `rebuild_layout()` at the end of
`apply_effects` — the exact regression PR #91's round 2 fixed. This test fails
by name; so does gap 1's, along with six pre-existing records-mode tests
(measured in review; an earlier draft of this entry said five).
**Scope note, stated because an overclaim is worse than a gap**: this drives
the *effects* `SwitchViewAtCursor` emits, not the view-panel key path that
composes them. Reaching the panel needs a `view_panel_cursor` on a `CmdContext`
built in `command/cmd/panel.rs`, which is another wave's ground this round. The
command's own coverage lives there; what is unexercised is the two joined.
### the_layout_geometry_cases_are_every_wire_effect_of_that_kind
improvise-26dm. `layout_geometry_effects_actually_change_document_config` was
guarded by `assert_eq!(cases.len(), 13)`, which catches a case being *deleted*
from the list and never one being *absent* from it — and the two are written
identically as an `assert_eq` over a number. The adversarial review of PR #78
added a fourteenth `pub struct` declaring `LayoutGeometry`, pointed at by
nothing, and the whole suite stayed green.
improvise-5ubs's mechanism cannot reach it: an effect is a struct implementing
a trait, so serde has no derived enumeration to diff against. `model_effects`
maps each `ModelCommand` variant back to the effect it came from (gotcha 38),
so the list is diffed against the `LayoutGeometry` effects reachable through
`ModelCommand`. The two sets are equal today, which is the evidence the
substitution lost nothing.
**The right-hand side is bounded by the derive, and that took a second round.**
Review of PR #96 injected the deletion of `ModelCommand::TogglePruneEmpty` from
`every_model_command()`: this was the **only** test in the workspace that
failed, and `every_model_command`'s doc comment claiming the list was
compiler-forced was wrong — `variant_name`'s exhaustive match forces an arm
*there*, not an entry in the list. ChatGPT Codex raised the same thing on the
PR. `the_enumeration_names_every_variant_the_derive_declares` closes it, so the
chain is derive → enumeration → `model_effects` → case list with every link
checked. The assertion here still names **both** directions, because a short
right-hand side reads as a spurious case on the left and invites deleting it —
degrading both sides together.
Injected fault: drop `TogglePruneEmpty` from `layout_geometry_cases()`. The old
count said `12 != 13`; this says
`right: [… "TogglePruneEmpty" …]` and **names the effect**.
**Residual, stated rather than closed**: this is total over the wire and not
over the effects. An effect declaring `LayoutGeometry` that no `ModelCommand`
names is still invisible — which is precisely PR #78's probe. Closing that
needs an enumeration of effects the compiler maintains, and none exists without
a proc macro or inventory-style registration.
### the_enumeration_names_every_variant_the_derive_declares
The bound the entry above rests on. Injection is Codex's own, run end to end: a
`ModelCommand::ProbeGeometry` whose effect declares `LayoutGeometry`, with the
`model_effects` and `variant_name` arms the compiler forces, omitted from
**both** lists. Before this test, the whole workspace stayed green. After it,
the failure is staged and each stage names the probe — first
`every_model_command() never constructs ["ProbeGeometry"]`, then, once the
enumeration is repaired and only the case list is short,
`the_layout_geometry_cases_are_every_wire_effect_of_that_kind` with
`ProbeGeometry` on the right. That staging is the point: the two tests answer
different questions and the injection separates them.
The name reader is shared, not copied: `reduce.rs`'s `effect_type_name` and its
own `the_effect_name_reader_reads_all_three_debug_shapes` are what this leans
on, because a second reader here would be a derivation done twice and could
drift while one copy was repaired. It reads the leading identifier off the
derived `Debug`, `Effect` being a trait object that `type_name` cannot reach.
context/test-evidence/core.md
+41
View File
@@ -611,3 +611,44 @@ narrow as it claims.
Fault injected by deleting either emission: the corresponding test fails with
`Some(("Month", "Q1"))` / `Some(("Recipient", "Family"))` against `None`, and
`the_two_front_ends_draw_the_same_column_header` fails with it.
### a_trailing_conversion_letter_does_not_erase_the_precision
improvise-711n. `parse_number_format` read the precision as
`fmt[i+1..].parse::<u8>().ok().unwrap_or(0)` over everything after the last `.`,
so any trailing conversion letter made the whole suffix unparseable and the
format fell back to **zero decimals, silently**. `format: ,.2f` — the spelling
`context/repo-map.md`'s own `.improv` example carries, and the printf/Python
convention a user reaches for first — displayed integers everywhere.
Fails before the fix with `left: (true, 0)`, `right: (true, 2)`.
The test also pins the reading against its nearest *wrong* neighbour. Two
implementations both fix `,.2f`: take the leading digit run, or trim the
trailing non-digits off the tail. They differ only on a tail with digits on both
sides of a letter, so `".2f3"` is in the assertion. Injecting
`s.trim_end_matches(|c: char| !c.is_ascii_digit())` in place of `leading_digits`
fails the test at `left: (false, 0)`, `right: (false, 2)` — so the test sees the
rule, not merely the bug.
### a_zero_padded_picture_string_is_read_as_printf_not_as_excel
The companion to the above, and the reason the fix is not "be generous". A
forgiving reader would have to answer `.00`, where printf says precision zero
and an Excel picture says two places. This pins the printf reading — `"0.00"` is
`(false, 0)` — so that "accept the trailing `f`" cannot quietly grow into
"accept anything and guess". It passes before the fix as well as after: it is
there to stop the *next* change, not to catch this one.
### a_precision_too_large_to_name_reads_as_none
`".300"` does not fit `u8`, so it reads as no precision rather than as 300
decimal places. Unchanged by improvise-711n and pinned so the fallback is a
stated rule rather than an artefact of the integer type.
### a_format_naming_no_precision_at_all_reads_as_zero
`",f"` has no `.`, and `".f"` has a `.` with no digits after it. Both are zero
decimals. Pinned because `leading_digits` returns `""` for the second, and an
empty string reaching `parse::<u8>()` is the one path in the function with no
digits to carry the answer.
context/test-evidence/formula-and-io.md
+9
View File
@@ -455,6 +455,15 @@ into two fields, **corrupting every row**. The test reads the file back with a
real CSV reader and asserts the field *count*, which is the thing a string
`contains` check cannot see.
**Strengthened by improvise-711n.** The assertion on the value was
`records[1][1].starts_with("1,200")`, and the fixture's format is `,.2f` — so
the check could not tell `1,200` from `1,200.00` and sat green while
`parse_number_format` threw the `.2` away. It is now an exact equality. Reverting
the `format.rs` fix fails it with `left: "1,200"`, `right: "1,200.00"`, which is
the cross-crate demonstration that the format bug reached a user's exported file
and not only the screen.
---
## `improvise-io/src/import/analyzer.rs`
context/test-evidence/gp.md
+162
View File
@@ -749,3 +749,165 @@ fixture really names one column twice, that the preflight's 41 really disagrees
with what forcing would build, and that `as_categories` itself refuses. A test
asserting only the error message would pass on either ordering, since the
message is identical -- the whole defect was in work done, not in the result.
### the_preflight_counts_the_missing_category_a_forced_column_materialises
improvise-x1ns, found by ChatGPT Codex reviewing PR #90 and live on `main` at
`ce895ff`. `encoded_width` computed `has_missing` and used it in the **numeric**
arm only:
```
true => 1 + usize::from(has_missing),
false => present.iter().collect::<HashSet<_>>().len().max(1),
```
`build_categorical` maps every missing cell to `MISSING_CAT` and includes it in
`distinct(&mapped)`, so a categorical column with N distinct *present* values and
at least one absent cell materialises **N+1** one-hots. The preflight promised N.
It under-reports by exactly one per such column, and it under-reports in the
direction that costs: the preflight runs *before* `preprocess`, the exact check
on `feature_names.len()` runs *after*, so the exact check bounds the error
message and not the allocation — which is the whole reason the preflight exists.
The previous wave's review recorded this shape as "read, not injected" and
"bounded by the exact check"; the second half is false and is now corrected in
`context/code/gp.md` §14.
**Fails before the fix** with
```
the preflight under-reported: it promised 5 features and `preprocess`
materialised 6 — ["Year=(missing)", "Year=2020", "Year=2021", "Year=2022",
"Year=2023", "Cost"]
```
The fixture is Codex's suggested case, and the reason it is the interesting one:
a **forced numeric category with a missing coordinate**. Forcing is what sends a
numeric column down the categorical branch, so only a *forced* column reaches the
arm that was wrong, and only a missing cell makes the arms disagree.
The test does not assert a number. It asks `preprocess` what it actually built
and requires `encoded_width >= data.feature_names.len()`, so it cannot pass by
agreeing with a stale constant, and it keeps holding if the encoding widens for
some other reason. Three preconditions are asserted first, because each would
make it vacuous: that exactly one record lacks the coordinate, that the present
cells really are numeric (otherwise the column would be categorical by inference
and forcing would prove nothing), and that a `Year=(missing)` one-hot is really
among the materialised features.
The `.max(1)` is deliberately outside the addition. A column whose cells are
*entirely* missing has no present values, and `preprocess` skips it with a
warning rather than encoding it; keeping the clamp on the sum leaves that case
counted as 1 exactly as before, so the fix moves only the case it names.
### a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored
The evidence improvise-e8no is declined on, and the only coverage
`Artefact::NumericCategory` has ever had. It was **one of three** families with
no test, not the only one: `MissingFlag` and `DatePart` are covered by
`the_missing_flag_and_the_date_sub_features_are_refused_too`, and
`ValueOfAMeasure` and `AbsentCoordinate` still have none — filed, not fixed
here.
improvise-e8no observes that forcing a document category to `Forced::Categorical`
(improvise-zv8t) widens the encoding, so a high-cardinality numeric category now
exceeds `MAX_FEATURES` where inference used to read it as one numeric column and
the fit succeeded. The decline turns on what that success was worth: a tree that
splits on a numeric column over a *category* column produces a formula
`fitted_formula` **refuses**, because a bare reference to the category's name
would resolve to a `_Measure` item of the same name (improvise-8xhy). So the
pre-zv8t success was either a fit that never touched the column — for which
marking it a `Label` gives a fit over the same remaining features, within budget
— or one that could not be stored. Note what this does *not* establish: an
ordinal encoding is spellable as a weighted indicator sum, so it is declined on
semantics and formula size rather than on spellability (`context/code/gp.md`).
The test asks `preprocess` with `Forced::Numeric`, which is the pre-zv8t reading
reproduced deliberately rather than described, and then asks `Features::of` and
`fitted_formula`. It does **not** use the `artefacts()` helper: that helper
fabricates a `Spelling`, which is the shape gotcha 37(d) warns about — it would
assert that the emitter refuses what the test told it to refuse. Here the
document and the encoder decide.
Injected fault: change `spell()`'s `FeatureRole::Numeric { .. }` arm from
`Spelling::Artefact(Artefact::NumericCategory)` to `Spelling::Measure(column.clone())`
— the "fix" someone would reach for on reading improvise-e8no, and the one that
silently reopens improvise-8xhy. Fails with
```
assertion `left == right` failed: a numeric column over a category column is unspellable
left: Measure("Year")
right: Artefact(NumericCategory)
```
### an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum
Not a bug fix: this pins a **grammar fact a prose argument rests on**, after that
argument got it wrong. improvise-e8no's decline was first written claiming that
one-hot is the only encoding of a category whose splits the formula language can
spell, and therefore that ordinal encoding was ruled out by spellability. An
adversarial review disproved it by running `parse_formula` instead of reading the
rule that looked relevant: `is_expr` is a `primary`, so it is an `Expr` yielding
1/0, and `cmp_op`'s operands are full `expr`s — so
`IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, 1, 2)` parses.
The decline survives on other grounds (a category's items carry insertion order,
not magnitude; and the emitted formula still spells one `IS` per item, so the
width is not saved) — but the *stated reason* was false, and a decline is the one
artefact nothing checks. This test is the check: if the grammar ever narrows so
that an indicator sum stops parsing, the paragraph that says it does becomes
discoverable rather than quietly wrong.
The negative arm is the incidental fact that explains the emitter: `Year IS 2020`
does **not** parse, because `filter_value` is `string | pipe_quoted | bare_ident`
and a bare number is none of them. That is why `formula_text` pipe-quotes a
numeric item name through `quote_name`, and asserting it here keeps the positive
arm from passing for the trivial reason that everything parses.
### the_preflight_counts_the_parts_a_date_measure_expands_into
improvise-x1ns, **second instance** — found by ChatGPT Codex on PR #97, live at
`f128087`, i.e. in the very change that fixed the first one.
`preprocess` infers `Numeric → Date → Categorical` (`data.rs:406`), and
`encoded_width` had **no date arm**: numeric-or-categorical only. A date string
does not parse as `f64`, so a measure column holding dates took the categorical
arm and was counted by its distinct strings. `build_date` emits
`epoch_day/year/month/day/dow` — seven parts once any cell has a time
(`data.rs:326`) — so a column with **one** distinct timestamp promised 1 and
materialised 5.
Fails before the fix, on the plain-date case:
```
a plain date: the preflight under-reported: it promised 4 features and
`preprocess` materialised 8 — ["Region=East", "Region=North", "Region=South",
"When.epoch_day", "When.year", "When.month", "When.day", "When.dow"]
```
**The fixture carries a time case because of the rival fix, not because of the
bug.** Two implementations repair the plain-date failure: "a date column is 5
features", and "5, or 7 when any present cell carries a time". They differ only
on a column containing a time, so the second sub-case is what separates them.
Injected, `date_parts` hard-coded to 5:
```
a date with a time: the preflight under-reported: it promised 8 features and
`preprocess` materialised 10 — [… "When.hour", "When.minute"]
```
Note the plain-date sub-case **passes** under that injection — so without the
time fixture this test would have been satisfied by either reading, and would
have pinned the bug without pinning the rule.
Both arms compare against `data.feature_names.len()` rather than against a width
computed a second way, so the test cannot pass by agreeing with a stale constant
or by reimplementing the thing under test. Two preconditions guard vacuity:
every `When` cell really is the same string (so the categorical count really
would be 1), and the fixture's timestamp really does or does not carry a time,
asked of `parse_datetime` rather than of the literal's shape.
The column must be a **measure** — `as_categories` forces every document
category to `Forced::Categorical`, so a category of dates never reaches the date
branch at all. That is gotcha 37(d)'s "the date decline is narrower than it
read": this is the genuine date *measure*.
context/test-evidence/tooling.md
+27
View File
@@ -192,3 +192,30 @@ reason for a file nobody can find — the same failure as a dead ticket id, whic
The first arm was filed as latent and was live when it was written:
`context/code/kb-tool.md` had been unpublished, unargued and unnoticed since it
was added.
### Duplicate evidence headings — `SELF_TEST_DUPLICATES`
improvise-hsoe. `--check-evidence` verified that a heading names a test that
**runs**, and a duplicate heading names one just as well, so both copies pass.
PR #92 shipped two entries under
`a_layout_change_moves_the_layout_version_and_no_record`, and two more pairs
sat in `tui.md` — one of them asserting that `THE_MEMBERSHIP_CORRECT_SPANS` was
"a live tripwire, do not remove it" some four hundred lines above the entry
recording that improvise-gd83 had removed it. A reader meeting the older copy
first takes it as current, which is worse than an orphan: an orphan announces
itself as broken.
The fault is injected by the six cases in `SELF_TEST_DUPLICATES`, and three of
them are the ones that would not get written first. Two *different* tests
sharing a leaf must come back **ambiguous and not duplicate**, because the
wrong diagnosis files the wrong repair. A prose heading must be free to repeat,
because it claims no test. And the bare and module-qualified spellings of one
test must collide — grouping by the string rather than by the resolved test
would have missed exactly the case improvise-rm4s's qualification repair
creates, which is the first way this would have come back.
The live tree was itself the first injection: before the three merges the check
exited 1 naming all three pairs with both line numbers; after them it exits 0.
The clean arms (two headings naming two tests, and prose repeated) are what
stop an always-refuse from satisfying the rest.
context/test-evidence/tui.md
+40 -53
View File
@@ -87,22 +87,21 @@ Each header line is cut into columns and compared *exactly*. A `contains` would
be one-directional: it cannot see a label the browser wrongly blanked (the
filter would skip it) and it cannot see one painted under the wrong column.
**`THE_MEMBERSHIP_CORRECT_SPANS` is a deliberately-wrong assertion and a live
tripwire. Do not remove it.** The `assert_ne!` against it starts failing the day
improvise-gd83 is fixed, at which point the `assert_eq!` above it should become
that constant.
**improvise-gd83 is fixed and `THE_MEMBERSHIP_CORRECT_SPANS` is gone**, along
with the `assert_ne!` tripwire that named it. The value it predicted —
`[(Q1,2), (None,1), (Q1,2), (None,1)]` — is now what the test asserts, element
for element. `Sep` is ungrouped, so the run ends before it and resumes under
the second measure's own `Q1` header; a precondition asserts the fixture still
contains an ungrouped column, because without one the test cannot see a leaked
group at all.
The span assertion above it pins `[(Some("Q1"), 6)]` — Q1 covering all six
columns *including* the two under the ungrouped `Sep` — because
`expand_category` emits a header on entering a group and no boundary on leaving
one, so the nearest-preceding walk keeps answering Q1 (`AxisFrame::group_of`).
That is `GridLayout`'s rule and the TUI paints the same span; it is reproduced
rather than corrected, because correcting it changes what the TUI draws.
The membership-correct answer is spelled out rather than guessed, because a
wrong future expectation is worse than none: whoever picks up improvise-gd83
will read it and believe it. The frame here is **not** one Q1 run — `_Measure`
is the outer level, so the column axis is:
The superseded expectation — Q1 covering all six columns *including* the two
under the ungrouped `Sep`, because `expand_category` emitted a header on
entering a group and no boundary on leaving one, so the nearest-preceding walk
kept answering Q1 — is recorded here because it is what `AxisFrame::group_of`
answered before the fix, and the frame it was derived from still explains the
current value. The frame is **not** one Q1 run — `_Measure` is the outer level,
so the column axis is:
col_cats = ["_Measure", "Month"]
Group(Q1) [Revenue,Jan] [Revenue,Feb] [Revenue,Sep]
@@ -114,6 +113,23 @@ rather than after both. So membership-correct spans are **four** runs, not two
run between them. Hence
`[(Some("Q1"), 2), (None, 1), (Some("Q1"), 2), (None, 1)]`.
improvise-8qlr: the TUI arm was `group_line.contains("▼ Q1")`, which is
one-directional and passes while the TUI labels **every** column. The run rule
now lives once, as `AxisFrame::group_runs`/`group_run_heads`, and both ends
call it; the TUI's group line is compared cell for cell against spans derived
from the browser's own runs.
Fault injected exactly as that ticket specifies — forcing the TUI to emit the
group label on every column (`head = col_groups.get(ci)`, dropping the run-head
lookup). Before: `cargo test -p improvise` stayed green. After: this test fails
with `["▼ Q1","▼ Q1","","▼ Q1","▼ Q1",""]` against
`["▼ Q1","","","▼ Q1","",""]`.
One thing the shared rule does **not** flatten: the TUI labels its first
*visible* column even mid-run, because a scrolled grid would otherwise draw an
unlabelled group line. That is a scrolling concession, kept explicitly, and it
is why the comparison is run at `col_offset == 0`.
### neither_front_end_produces_a_slash_joined_label
A separate assertion because the agreement tests above would also hold if
@@ -165,6 +181,15 @@ so it stays as it is; the multi-level agreement is
`tests/grid_frame_agreement.rs`, which drives both front ends over a two-level,
grouped fixture.
Re-pinned for improvise-0o6x. `GROUPED_FRAME` moved by exactly one line: the
leading `▼ Co…` header now draws above `East`, and everything below shifts
down one. Nothing else in the buffer changed. That the group line, the column
headers, the separator, the data rows and the totals strip are byte-identical
is the evidence the change is the boundary condition it claims to be and not a
reshuffle of the draw loop. It also fails alongside
`a_group_header_first_on_its_axis_reaches_both_front_ends` when `body_from`'s
match is forced to `false`.
### two_row_categories_produce_cross_product_labels
Names the rendering the protocol must not carry: multi-level row headers show
@@ -509,44 +534,6 @@ nothing.
Fault injected by forcing `body_from`'s match to `false`: this test fails and
`the_whole_frame_is_unchanged` fails with it.
### the_whole_frame_is_unchanged
Re-pinned for improvise-0o6x. `GROUPED_FRAME` moved by exactly one line: the
leading `▼ Co…` header now draws above `East`, and everything below shifts
down one. Nothing else in the buffer changed. That the group line, the column
headers, the separator, the data rows and the totals strip are byte-identical
is the evidence the change is the boundary condition it claims to be and not a
reshuffle of the draw loop.
### the_two_front_ends_draw_the_same_column_header
Two changes, from two tickets.
improvise-gd83: `THE_MEMBERSHIP_CORRECT_SPANS` and its `assert_ne!` tripwire are
gone, and the value they predicted — `[(Q1,2), (None,1), (Q1,2), (None,1)]` —
is now what the test asserts, element for element. `Sep` is ungrouped, so the
run ends before it and resumes under the second measure's own `Q1` header. A
precondition asserts the fixture still contains an ungrouped column, because
without one the test cannot see a leaked group at all.
improvise-8qlr: the TUI arm was `group_line.contains("▼ Q1")`, which is
one-directional and passes while the TUI labels **every** column. The run rule
now lives once, as `AxisFrame::group_runs`/`group_run_heads`, and both ends
call it; the TUI's group line is compared cell for cell against spans derived
from the browser's own runs.
Fault injected exactly as the ticket specifies — forcing the TUI to emit the
group label on every column (`head = col_groups.get(ci)`, dropping the run-head
lookup). Before: `cargo test -p improvise` stayed green. After: this test fails
with `["▼ Q1","▼ Q1","","▼ Q1","▼ Q1",""]` against
`["▼ Q1","","","▼ Q1","",""]`.
One thing the shared rule does **not** flatten: the TUI labels its first
*visible* column even mid-run, because a scrolled grid would otherwise draw an
unlabelled group line. That is a scrolling concession, kept explicitly, and it
is why the comparison is run at `col_offset == 0`.
### two_abutting_runs_of_one_group_name_are_two_spans_at_both_ends
improvise-h0iv, end to end. The unit demonstration is in
crates/improvise-command/Cargo.toml
+1
View File
@@ -21,4 +21,5 @@ unicode-width = "^0.2"
web-time = "1"
[dev-dependencies]
improvise-protocol = { path = "../improvise-protocol", features = ["test-support"] }
tempfile = "3"
crates/improvise-command/src/ui/app.rs
+84
View File
@@ -2652,6 +2652,90 @@ mod tests {
}
}
/// The settle's `RecordsNormal` arm, driven from the mode it names. The
/// `:` crossing above enters the settle in `Normal` both ways — the evidence.
#[test]
fn a_batch_that_leaves_records_settles_out_of_records_normal() {
use crate::view::Axis;
let mut wb = Workbook::new("T");
wb.add_category("Region").unwrap();
wb.add_item("Region", "East").unwrap();
let mut app = App::new(wb, None);
app.handle_key(KeyEvent::new(KeyCode::Char('R'), KeyModifiers::NONE))
.unwrap();
assert!(app.layout.is_records_mode(), "setup: the grid is records");
assert_eq!(
app.view_state.mode,
AppMode::RecordsNormal,
"setup: the mode handed to the settle must be RecordsNormal"
);
app.apply_effects(vec![Box::new(super::super::effect::SetAxis {
view: ViewTarget::Active,
category: "_Index".into(),
axis: Axis::None,
})]);
assert!(
!app.layout.is_records_mode(),
"setup: the batch has to leave records, or nothing crossed"
);
assert_eq!(
app.view_state.mode,
AppMode::Normal,
"a batch that left records stayed in RecordsNormal (improvise-4dpj)"
);
}
/// The batch `SwitchViewAtCursor` emits, both ways across records-ness:
/// records-ness is auto-detected from the axes, so a view switch moves it.
#[test]
fn switching_views_across_records_ness_settles_against_the_view_switched_to() {
for to_records in [false, true] {
let mut wb = Workbook::new("T");
wb.add_category("Region").unwrap();
wb.add_item("Region", "East").unwrap();
let mut app = App::new(wb, None);
// `R` creates and enters `_Records`; the pivot view it left is the
// other end of every crossing below.
app.handle_key(KeyEvent::new(KeyCode::Char('R'), KeyModifiers::NONE))
.unwrap();
let records_view = app.view_state.active_view.clone();
let (from, to) = match to_records {
true => ("Default", records_view.as_str()),
false => (records_view.as_str(), "Default"),
};
app.apply_effects(vec![Box::new(super::super::effect::SwitchView(
from.into(),
))]);
assert_eq!(
app.layout.is_records_mode(),
!to_records,
"setup: the switch has to CROSS, so it must start on the other side"
);
app.apply_effects(vec![
Box::new(super::super::effect::SwitchView(to.into())),
super::super::effect::change_mode(AppMode::Normal),
]);
assert_eq!(
app.layout.is_records_mode(),
to_records,
"setup: the switch has to land, or nothing crossed"
);
let expected = match to_records {
true => AppMode::RecordsNormal,
false => AppMode::Normal,
};
assert_eq!(
app.view_state.mode, expected,
"the batch settled against the view it LEFT, not the one it \
switched to (improvise-4dpj)"
);
}
}
/// A `:` line that really runs settles through `ReturnToNormalUnlessMoved`,
/// which an empty line never reaches. See the evidence.
#[test]
crates/improvise-command/src/ui/effect.rs
+39 -8
View File
@@ -1444,11 +1444,38 @@ mod tests {
}
}
/// Effects declaring `LayoutGeometry` must move `document_config` — the
/// production hash `model_state.dirty` is decided by (improvise-1xf4).
/// The case list below is COMPLETE over the wire vocabulary, which the
/// compiler maintains — improvise-26dm, `context/code/command.md`.
#[test]
fn layout_geometry_effects_actually_change_document_config() {
let cases: Vec<Box<dyn Effect>> = vec![
fn the_layout_geometry_cases_are_every_wire_effect_of_that_kind() {
let app = grouped_app();
let mut reachable: Vec<String> = crate::ui::reduce::tests::every_model_command()
.iter()
.flat_map(|cmd| crate::ui::reduce::model_effects(&app, cmd))
.filter(|e| e.kind() == EffectKind::Model(ModelChange::LayoutGeometry))
.map(|e| crate::ui::reduce::tests::effect_type_name(e.as_ref()))
.collect();
let mut listed: Vec<String> = layout_geometry_cases()
.iter()
.map(|e| crate::ui::reduce::tests::effect_type_name(e.as_ref()))
.collect();
reachable.sort_unstable();
reachable.dedup();
listed.sort_unstable();
listed.dedup();
assert_eq!(
listed, reachable,
"the LayoutGeometry case list and the wire vocabulary disagree. A name \
only on the LEFT means every_model_command() has a hole, not that the \
case is spurious; a name only on the RIGHT is an effect nothing here \
exercises. Read `context/code/command.md` before deleting either side"
);
}
/// The `LayoutGeometry` effects, named once so two tests can ask the same
/// list the same questions — improvise-26dm, `context/code/command.md`.
fn layout_geometry_cases() -> Vec<Box<dyn Effect>> {
vec![
Box::new(AddCategory("Region".into())),
Box::new(RemoveCategory("Month".into())),
Box::new(CreateView("Second".into())),
@@ -1482,10 +1509,14 @@ mod tests {
Box::new(CycleAxis(ViewTarget::Active, "Type".into())),
Box::new(SetNumberFormat(ViewTarget::Active, ",.3f".into())),
Box::new(TogglePruneEmpty(ViewTarget::Active)),
];
// A COUNT, not a totality check: nothing forces a fourteenth here, and a
// probe variant proved it — improvise-5ubs, `context/code/command.md`.
assert_eq!(cases.len(), 13, "the LayoutGeometry effects were thirteen");
]
}
/// Effects declaring `LayoutGeometry` must move `document_config` — the
/// production hash `model_state.dirty` is decided by (improvise-1xf4).
#[test]
fn layout_geometry_effects_actually_change_document_config() {
let cases = layout_geometry_cases();
for effect in cases {
assert_eq!(
effect.kind(),
crates/improvise-command/src/ui/reduce.rs
+24 -5
View File
@@ -308,7 +308,7 @@ pub fn reduce_key(app: &mut App, key: KeyEvent) -> Vec<Command> {
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;
use crate::model::cell::CellValue;
use crate::ui::app::AppMode;
@@ -527,9 +527,9 @@ mod tests {
app
}
/// One value per `ModelCommand` variant. [`variant_name`] matches
/// exhaustively, so a new variant fails the build until it is listed here.
fn every_model_command() -> Vec<ModelCommand> {
/// One value per `ModelCommand` variant — by hand, and NOT compiler-forced:
/// [`variant_name`]'s exhaustive match only forces an arm there (PR #96).
pub(crate) fn every_model_command() -> Vec<ModelCommand> {
vec![
ModelCommand::SetCell {
cell: record_cell(&a_stored_cell()),
@@ -708,7 +708,7 @@ mod tests {
/// An effect's type name, off the leading identifier of its derived `Debug`.
/// `Effect` is a trait object, so `std::any::type_name` cannot reach it.
fn effect_type_name(effect: &dyn Effect) -> String {
pub(crate) fn effect_type_name(effect: &dyn Effect) -> String {
format!("{effect:?}")
.split(|c: char| !c.is_alphanumeric() && c != '_')
.next()
@@ -765,6 +765,25 @@ mod tests {
assert_eq!(names.len(), total, "a variant is listed twice: {names:?}");
}
/// ...and covers it WHOLLY, against serde's derive — the one enumeration the
/// compiler maintains (improvise-5ubs). `context/code/command.md`.
#[test]
fn the_enumeration_names_every_variant_the_derive_declares() {
let listed: Vec<String> = every_model_command()
.iter()
.map(|cmd| variant_name(cmd).to_string())
.collect();
let missed: Vec<String> = improvise_protocol::variants::declared::<ModelCommand>()
.into_iter()
.filter(|name| !listed.contains(name))
.collect();
assert!(
missed.is_empty(),
"every_model_command() never constructs {missed:?} — adding an arm to \
variant_name is compiler-forced, adding a VALUE to the list is not"
);
}
/// A model `Command` applied authoritatively lands where the keypress bound
/// to the same mutation lands, `Dirty` included.
#[test]
crates/improvise-core/src/format.rs
+52 -6
View File
@@ -10,14 +10,19 @@ pub fn format_value(v: Option<&CellValue>, comma: bool, decimals: u8) -> String
}
}
/// Parse a number format string like ",.0" into (use_commas, decimal_places).
/// The digits a precision is named by: everything up to the first non-digit.
fn leading_digits(s: &str) -> &str {
&s[..s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())]
}
/// Parse a printf-style number format like ",.2f" into (use_commas, decimals).
/// Which spellings are legal: `context/code/core.md`, "The number format".
pub fn parse_number_format(fmt: &str) -> (bool, u8) {
let comma = fmt.contains(',');
let decimals = fmt
.rfind('.')
.and_then(|i| fmt[i + 1..].parse::<u8>().ok())
.unwrap_or(0);
(comma, decimals)
let precision = fmt
.rsplit_once('.')
.map_or("", |(_, tail)| leading_digits(tail));
(comma, precision.parse::<u8>().unwrap_or(0))
}
/// Round half away from zero (the "normal" rounding people expect).
@@ -100,6 +105,47 @@ mod tests {
assert_eq!(parse_number_format(",.1.3"), (true, 3));
}
/// Bug improvise-711n: a trailing conversion letter made the whole suffix
/// unparseable. Evidence: `context/test-evidence/core.md`.
#[test]
fn a_trailing_conversion_letter_does_not_erase_the_precision() {
assert_eq!(parse_number_format(",.2f"), (true, 2));
assert_eq!(parse_number_format(".2f"), (false, 2));
assert_eq!(parse_number_format("%.2f"), (false, 2));
// The precision is the LEADING digit run, not the tail with its
// non-digits trimmed off — these two readings differ here.
assert_eq!(parse_number_format(".2f3"), (false, 2));
}
/// One convention, not two: `.00` is printf's "precision zero", never
/// Excel's "two places". Evidence: `context/test-evidence/core.md`.
#[test]
fn a_zero_padded_picture_string_is_read_as_printf_not_as_excel() {
assert_eq!(parse_number_format("0.00"), (false, 0));
assert_eq!(parse_number_format(",.0f"), (true, 0));
}
#[test]
fn a_precision_too_large_to_name_reads_as_none() {
assert_eq!(parse_number_format(".300"), (false, 0));
assert_eq!(parse_number_format(",.999f"), (true, 0));
}
/// `format:` is `rest_of_line` in the grammar, so the format string can hold
/// any text at all — including a multibyte char right after the dot.
#[test]
fn a_multibyte_character_after_the_dot_does_not_split_a_char_boundary() {
assert_eq!(parse_number_format(",.2\u{20ac}"), (true, 2));
assert_eq!(parse_number_format(".\u{20ac}"), (false, 0));
assert_eq!(parse_number_format(".\u{0662}"), (false, 0));
}
#[test]
fn a_format_naming_no_precision_at_all_reads_as_zero() {
assert_eq!(parse_number_format(",f"), (true, 0));
assert_eq!(parse_number_format(".f"), (false, 0));
}
// ── format_f64 basic ───────────────────────────────────────────────
#[test]
crates/improvise-gp/src/fit.rs
+223 -18
View File
@@ -1,7 +1,7 @@
//! One fit, end to end: a [`View`]'s records in, an improvise [`Formula`] and the
//! evidence for believing it out. It never prints — `context/code/gp.md` §14.
use gp_tree::data::{Forced, Overrides, Target, preprocess};
use gp_tree::data::{Forced, Overrides, Target, parse_datetime, preprocess};
use gp_tree::fit::{
FitConfig, RegressionQuality, cv_regress, denormalize, fit, importance_ranking, mse,
permutation_importance, regression_quality, standardize,
@@ -13,6 +13,31 @@ use improvise_formula::Formula;
use crate::dataset::{FeatureTable, feature_table};
/// Which branch `preprocess` would take for this column. It mirrors that function's
/// own `match`, arm for arm — the preflight owes every branch, `context/code/gp.md` §14.
fn inferred_kind(present: &[&str], is_document_category: bool) -> Forced {
match present.is_empty() {
// `preprocess` SKIPS an entirely-missing column. Counting it as one
// categorical over-counts by one, which is the safe direction.
true => Forced::Categorical,
false if is_document_category => Forced::Categorical,
false if present.iter().all(|c| c.parse::<f64>().is_ok()) => Forced::Numeric,
false if present.iter().all(|c| parse_datetime(c).is_some()) => Forced::Date,
false => Forced::Categorical,
}
}
/// `build_date` emits five parts, or seven once any cell carries a time.
fn date_parts(present: &[&str]) -> usize {
match present
.iter()
.any(|c| parse_datetime(c).is_some_and(|d| d.has_time))
{
true => 7,
false => 5,
}
}
/// A cheap upper bound on how wide `preprocess` would expand this table, computed
/// without expanding it. **Advisory, not authoritative** — `context/code/gp.md` §14.
fn encoded_width(table: &FeatureTable, category_columns: usize) -> usize {
@@ -29,20 +54,21 @@ fn encoded_width(table: &FeatureTable, category_columns: usize) -> usize {
.copied()
.filter(|c| !gp_tree::data::is_missing(c))
.collect();
let has_missing = present.len() < cells.len();
// A document category is forced categorical below, so the preflight
// counts it by cardinality too. Where the two still differ: gp.md §14.
let numeric = col >= category_columns
&& !present.is_empty()
&& present.iter().all(|c| c.parse::<f64>().is_ok());
match numeric {
// A numeric column is one feature, plus the `.missing` flag.
true => 1 + usize::from(has_missing),
// Everything else: one column per distinct value.
false => present
// Every branch adds the `.missing` flag on the same terms, so it is
// added once here rather than in each arm.
let missing = usize::from(present.len() < cells.len());
// No wildcard: a new `Forced` variant must be counted here rather
// than silently falling into the categorical arm.
match inferred_kind(&present, col < category_columns) {
Forced::Numeric => 1 + missing,
Forced::Date => date_parts(&present) + missing,
// One column per distinct value, plus the one `build_categorical`
// reserves for `(missing)` — gp.md §14.
Forced::Categorical => (present
.iter()
.collect::<std::collections::HashSet<_>>()
.len()
+ missing)
.max(1),
}
})
@@ -239,8 +265,9 @@ pub fn fit_measure(
anyhow::ensure!(
width <= MAX_FEATURES,
"fitting {target:?} would encode {width} features from {} records, which is too wide \
to search on the UI thread. A category with a distinct value per record becomes one \
column per record — mark such a category as a Label so it is not a feature",
to search on the UI thread. A category becomes one column per distinct item, whether \
its items read as numbers or not — mark such a category as a Label so it is not a \
feature",
table.rows.len(),
);
@@ -264,13 +291,13 @@ pub fn fit_measure(
cannot spell and a search cannot minimise. Correct or clear those cells first"
);
// The exact width, now that it is known. The preflight above is advisory, and
// this is the check that enforces the bound in both directions of a drift.
// The exact width, now that it is known. It bounds the error MESSAGE, not the
// cost: `preprocess` has already run — the preflight is the cost bound (§14).
anyhow::ensure!(
data.feature_names.len() <= MAX_FEATURES,
"fitting {target:?} encoded {} features, over the {MAX_FEATURES} this host will \
search. A category with many distinct values becomes one column per value — \
mark such a category as a Label so it is not a feature",
search. A category becomes one column per distinct item, whether its items read as \
numbers or not — mark such a category as a Label so it is not a feature",
data.feature_names.len(),
);
// Empty is a **panic**, not a poor fit: zero features makes `random_tree` draw
@@ -714,6 +741,184 @@ mod tests {
(model, view)
}
/// Bug improvise-x1ns (2nd): `preprocess` infers Date BEFORE Categorical and
/// `build_date` emits 5 or 7 parts. Evidence: `context/test-evidence/gp.md`.
#[test]
fn the_preflight_counts_the_parts_a_date_measure_expands_into() {
for (label, stamp, parts) in [
("a plain date", "2024-01-15", 5),
("a date with a time", "2024-01-15T09:30:00", 7),
] {
let (model, view) = a_date_valued_measure(stamp);
let table = feature_table(&model, &view, "Score").expect("the table builds");
let category_columns = model.regular_category_names().len();
// The fixture must really be the shape the bug needs: ONE distinct
// date, which the categorical arm would count as a single column.
let when: Vec<&str> = table
.rows
.iter()
.map(|r| r[category_columns].as_str())
.collect();
assert!(
when.iter().all(|c| *c == stamp),
"{label}: every When cell must be the same date: {when:?}"
);
assert!(
gp_tree::data::parse_datetime(stamp).is_some_and(|d| d.has_time == (parts == 7)),
"{label}: the fixture must really carry a time iff {parts} parts are expected"
);
let width = super::encoded_width(&table, category_columns);
let overrides =
super::as_categories(&table.header, category_columns).expect("the header is clean");
let data = preprocess(&table.header, &table.rows, &overrides);
assert_eq!(
data.feature_names
.iter()
.filter(|n| n.starts_with("When."))
.count(),
parts,
"{label}: the fixture must really expand into {parts} date parts — {:?}",
data.feature_names
);
assert!(
width >= data.feature_names.len(),
"{label}: the preflight under-reported: it promised {width} features \
and `preprocess` materialised {} — {:?}",
data.feature_names.len(),
data.feature_names,
);
}
}
/// One regular category and a `When` MEASURE whose every cell is the same
/// timestamp — the low-cardinality date column a distinct-count misses.
fn a_date_valued_measure(stamp: &str) -> (Model, View) {
let mut model = Model::new("sales");
model.add_category(REGION).expect("under the limit");
let measure = model.category_mut(MEASURE).expect("_Measure always exists");
for item in ["When", "Score"] {
measure.add_item(item);
}
for (i, region) in ["North", "South", "East"].iter().enumerate() {
model
.category_mut(REGION)
.expect("just added")
.add_item(*region);
model.set_cell(
CellKey::new(vec![
(REGION.to_string(), (*region).to_string()),
(MEASURE.to_string(), "When".to_string()),
]),
CellValue::Text(stamp.to_string()),
);
model.set_cell(
CellKey::new(vec![
(REGION.to_string(), (*region).to_string()),
(MEASURE.to_string(), "Score".to_string()),
]),
CellValue::Number(i as f64),
);
}
let mut view = View::new("Default");
for cat in model.category_names() {
view.on_category_added(cat);
}
view.set_axis(REGION, Axis::Row);
view.set_axis(MEASURE, Axis::Page);
(model, view)
}
/// Bug improvise-x1ns: the preflight counted a forced categorical by its
/// PRESENT values only. Evidence: `context/test-evidence/gp.md`.
#[test]
fn the_preflight_counts_the_missing_category_a_forced_column_materialises() {
let (model, view) = a_numeric_category_with_a_missing_coordinate();
let table = feature_table(&model, &view, "Score").expect("the table builds");
let category_columns = model.regular_category_names().len();
// The fixture must really carry the shape the bug needs: a forced
// category whose cells are numeric, one of them absent.
let year: Vec<&str> = table.rows.iter().map(|r| r[0].as_str()).collect();
assert_eq!(
year.iter().filter(|c| gp_tree::data::is_missing(c)).count(),
1,
"exactly one record must lack the Year coordinate: {year:?}"
);
assert!(
year.iter()
.filter(|c| !gp_tree::data::is_missing(c))
.all(|c| c.parse::<f64>().is_ok()),
"the present Year cells must all be numeric: {year:?}"
);
let width = super::encoded_width(&table, category_columns);
let overrides =
super::as_categories(&table.header, category_columns).expect("the header is clean");
let data = preprocess(&table.header, &table.rows, &overrides);
// `build_categorical` maps the absent cell to the reserved `(missing)`
// category, so it is one more one-hot column than the present values.
assert!(
data.feature_names
.iter()
.any(|n| n == &format!("Year={}", gp_tree::data::MISSING_CAT)),
"the forced column must really emit a (missing) one-hot: {:?}",
data.feature_names
);
assert!(
width >= data.feature_names.len(),
"the preflight under-reported: it promised {width} features and \
`preprocess` materialised {} — {:?}",
data.feature_names.len(),
data.feature_names,
);
}
/// A numeric `Year` category with one record that has no `Year` coordinate, so
/// forcing sends a numeric column down the categorical branch with a gap in it.
fn a_numeric_category_with_a_missing_coordinate() -> (Model, View) {
let mut model = Model::new("sales");
model.add_category("Year").expect("under the limit");
let measure = model.category_mut(MEASURE).expect("_Measure always exists");
for item in ["Cost", "Score"] {
measure.add_item(item);
}
for i in 0..4u32 {
let year = format!("{}", 2020 + i);
model
.category_mut("Year")
.expect("just added")
.add_item(year.clone());
for (m, v) in [("Cost", f64::from(i) * 10.0), ("Score", f64::from(i))] {
model.set_cell(
CellKey::new(vec![
("Year".to_string(), year.clone()),
(MEASURE.to_string(), m.to_string()),
]),
CellValue::Number(v),
);
}
}
// The record with no `Year` coordinate at all — an empty cell, which
// `is_missing` reads as absent rather than as a value.
for (m, v) in [("Cost", 99.0), ("Score", 9.0)] {
model.set_cell(
CellKey::new(vec![(MEASURE.to_string(), m.to_string())]),
CellValue::Number(v),
);
}
let mut view = View::new("Default");
for cat in model.category_names() {
view.on_category_added(cat);
}
view.set_axis("Year", Axis::Row);
view.set_axis(MEASURE, Axis::Page);
(model, view)
}
/// The review finding on improvise-zv8t: forcing is by NAME and the preflight
/// counts by INDEX. Evidence: `context/test-evidence/gp.md`.
#[test]
crates/improvise-gp/src/lib.rs
+64
View File
@@ -812,6 +812,70 @@ mod tests {
}
}
/// improvise-e8no declines ordinal encoding on semantics, NOT on
/// spellability — a first draft said otherwise. `context/code/gp.md`.
#[test]
fn an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum() {
// `is_expr` is a `primary`, so it is an `Expr` yielding 1/0, and
// `cmp_op`'s operands are full `expr`s.
for text in [
"Score = (Year IS |2020|) + (Year IS |2021|)",
"Score = IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, 1, 2)",
] {
assert!(
parse_formula(text, MEASURE).is_ok(),
"the grammar the decline reasons about must really accept {text:?}"
);
}
// And the reason a numeric item name must be pipe-quoted: a bare
// number is not a `filter_value`.
assert!(
parse_formula("Score = (Year IS 2020)", MEASURE).is_err(),
"a bare number as a filter_value must stay a parse error"
);
}
/// improvise-e8no's decline rests on this: the pre-zv8t encoding of a
/// numeric category yields fits no document can store. Evidence: `gp.md`.
#[test]
fn a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored() {
use gp_tree::data::{Forced, Overrides, preprocess};
// The pre-improvise-zv8t reading: a category of numbers inferred as a
// numeric column. Asked of `preprocess`, never a fabricated spelling.
let header = vec!["Year".to_string(), TARGET.to_string()];
let rows: Vec<Vec<String>> = [("2020", "1"), ("2021", "2"), ("2022", "3")]
.iter()
.map(|(y, t)| vec![(*y).to_string(), (*t).to_string()])
.collect();
let overrides = Overrides {
force: [("Year".to_string(), Forced::Numeric)]
.into_iter()
.collect(),
..Overrides::default()
};
let data = preprocess(&header, &rows, &overrides);
let features = crate::Features::of(&data, &["Year".to_string()]);
assert_eq!(
data.feature_names,
vec!["Year".to_string()],
"the fixture must really encode Year as ONE numeric column"
);
assert_eq!(
features.0[0].spelling,
crate::Spelling::Artefact(Artefact::NumericCategory),
"a numeric column over a category column is unspellable"
);
let tree = split(0, 2020.5, leaf(1.0), leaf(2.0));
let err = fitted_formula(&tree, &features, TARGET, MEASURE, &document())
.expect_err("a split on a numeric category is not storable")
.to_string();
assert!(err.contains("Year"), "{err}");
assert!(err.contains("cannot spell"), "{err}");
}
/// The property the *lying* test could not state: if `fitted_formula` says
/// `Ok`, no cell of the target is a `ref:` error. See the evidence file.
#[test]
crates/improvise-io/src/persistence/export.rs
+3 -5
View File
@@ -139,11 +139,9 @@ mod tests {
assert_eq!(r.len(), 2, "comma split a field: {r:?}");
}
assert_eq!(&records[1][0], "Smith, Jr.");
assert!(
records[1][1].starts_with("1,200"),
"expected comma-grouped value, got: {:?}",
&records[1][1]
);
// Exact, not `starts_with`: the fixture asks for `.2f`, and a prefix
// check cannot tell two decimals from none (improvise-711n).
assert_eq!(&records[1][1], "1,200.00");
}
/// Bug improvise-iot6: `render_csv` built its layout with `GridLayout::new`,
crates/improvise-protocol/Cargo.toml
+7
View File
@@ -13,6 +13,13 @@ serde = { version = "1", features = ["derive"] }
# (`AxisFrame::header_widths`). A terminal column is a display cell, not a
# `char`, and the TUI has always measured it this way.
unicode-width = "^0.2"
# Only for `variants`, which is off unless a test build asks for it.
serde_json = { version = "1", optional = true }
[features]
# `variants` reaches OUT of this crate: improvise-command bounds its own
# ModelCommand enumeration with it. Off in every shipped build (improvise-26dm).
test-support = ["dep:serde_json"]
[dev-dependencies]
serde_json = "1"
crates/improvise-protocol/src/lib.rs
+2 -2
View File
@@ -13,8 +13,8 @@ pub mod keymap;
mod reduce;
pub mod render;
pub mod resolve;
#[cfg(test)]
pub(crate) mod variants;
#[cfg(any(test, feature = "test-support"))]
pub mod variants;
pub mod viewmodel;
pub use basis::{Basis, BasisCache, Rejection};
prolog-sessions/kb.pl
+11
View File
@@ -122,6 +122,7 @@ invariant(where_and_is_share_a_word_boundary,'where_clause, inline_where and is_
invariant(evaluate_aggregated_takes_a_named_state,'Model::evaluate_aggregated takes Aggregation::{Exact, OverHidden}, not none_cats: &[String] (improvise-5jm5). The list was consulted once, for emptiness, in every revision since Axis::None was introduced -- the aggregation runs over key.0 via matching_values. Aggregation::of reduces at the boundary and the regression guard is the SIGNATURE. recompute_formulas and aggregate_raw keep &[String] because they walk the elements.').
invariant(normal_mode_is_resolved_not_carried,'AppMode::Normal is resolved through App::normal_mode() when the mode is SET -- ChangeMode and ReturnToNormalUnlessMoved are the only two places a batch sets it, and CloseWizard reads the same function -- so no keymap binding and no commit command carries an exit mode, and none can forget one (improvise-sihr). It reads the batch OPENING layout (gotcha 23), which is the property rather than a defect: that is the grid the user was looking at, and toggle-records-mode works in both directions because of the staleness rather than despite it. Carrying the entry mode on MinibufferConfig was the ticket proposal and was declined -- carried generically it also returns CategoryAdd to CategoryPanel. Known gap, filed as improvise-e5pf: Command::ChangeMode(Normal) is on the wire and reduce_view folds it without the resolution.').
invariant(the_frame_basis_follows_active_view,'The cursor frame basis re-bases AT THE SWITCH, in the loop, as soon as active_view moves -- not at the next geometry effect. Guarding the COMPARISON on the two names agreeing (PR #78 round 2) exempted the first geometry effect after a mid-batch SwitchView from the check altogether, which is under-reporting and gotcha 35 expensive side. SwitchView stays ViewOnly: the basis is App own bookkeeping and none of it crosses the wire, so this needed no protocol change. Costs one extra view_frame per batch that both switches and moves geometry, DrillIntoCell paying most.').
invariant(encoded_width_mirrors_every_preprocess_branch,'The preflight runs BEFORE preprocess and the exact check AFTER it, so the exact check bounds the error message and not the allocation. encoded_width must mirror EVERY arm of preprocess inference -- Numeric, Date (5 parts, 7 with a time) and Categorical (distinct plus the reserved (missing)) -- plus the .missing flag. It may over-count; it must never under-count. improvise-x1ns was TWO under-counts one inference-branch apart: the categorical (missing) one-hot, and the entire Date arm. The guard is that encoded_width matches Forced WITH NO WILDCARD, so a fourth variant is a compile error rather than a silent categorical fallback -- which is exactly how Date was missed.').
doctrine(bug_first,'write a failing test demonstrating the bug before fixing it').
doctrine(unit_over_e2e,'prefer small unit tests over integration/e2e').
doctrine(suite_speed,'Two numbers, because the ambiguity between them is what let this drift: NO SINGLE TEST BINARY may exceed 2 seconds, and the WORKSPACE WALL CLOCK is the sum of them. Measured at 22d43b2: 1309 tests over 33 binaries, ~2.2s wall, longest single binary 1.42s -- so the workspace is already past 2s and the rule that bites is the per-binary one. Re-MEASURE rather than re-assert: cargo test --workspace prints a \"finished in\" per binary. Excludes compilation, and excludes harness/ (Playwright, minutes, its own CI job).').
@@ -291,6 +292,15 @@ doctrine(read_the_reviews_before_merging_and_authorship_will_not_identify_them,'
doctrine(bounded_by_a_later_check_does_not_bound_the_cost,'improvise-x1ns: encoded_width is a PREFLIGHT whose whole purpose is to refuse an expansion BEFORE allocating it, and its categorical branch under-counted by the missing level that build_categorical actually emits. The wave own review had seen the shape and reasoned it away as \"bounded by the exact check\". Codex rebuttal is the keeper: THE EXACT CHECK RUNS AFTER THE ALLOCATION, so it bounds the error MESSAGE, not the cost. When a guard exists to prevent work rather than to report a wrong answer, a downstream correctness check is not a backstop for it -- the thing you were protecting has already happened. Second instance on this path: improvise-zv8t own round found forcing by NAME while budgeting by INDEX, repaired as a data dependency so the ordering became a compile error. Both are the same shape -- two halves of one decision disagreeing about what a column costs.').
doctrine(inject_the_rival_fix_not_only_the_bug,'check_the_checker says inject the fault and watch the check fail -- which proves the test SEES THE BUG. It does not prove the test encodes the RULE. On improvise-711n two implementations both repair \",.2f\": take the leading digit run after the last dot, or trim trailing non-digits. They differ on exactly one shape, \".2f3\", and without that case the assertion is satisfied by either. So the discriminating injection is not the original bug but the OTHER PLAUSIBLE FIX. Ask, for every repair: what else would make this test pass, and is that thing also correct? Where a rival exists, the fixture must contain the input that separates them or the test pins a behaviour rather than a rule.').
doctrine(a_preflight_must_mirror_every_branch_the_real_path_can_take,'encoded_width is a cost preflight over preprocess, which dispatches Numeric -> Date -> Categorical. It mirrored two of the three. improvise-x1ns was the categorical arm under-counting the (missing) one-hot; ChatGPT Codex then found the DATE arm missing entirely -- a date column takes the categorical fallback because \"2024-01-15\" does not parse as f64, so a column holding one distinct date promises 1 where build_date materialises 5, or 7 with a time. Two findings, one shape, one inference branch apart. A preflight is a SECOND IMPLEMENTATION of the thing it guards, so it inherits every branch of it, and the branch nobody mirrored is the one nobody was thinking about. Note the second finding contradicted an invariant proposed in the same hand-back that fixed the first: the counter-example was already in the tree.').
doctrine(an_invariant_generalised_from_a_fix_generalises_the_fix,'Having repaired the categorical under-count, the Wave M agent proposed a never-under-counts invariant -- WHILE THE TREE ALREADY HELD A COUNTER-EXAMPLE, the missing Date arm, which ChatGPT Codex then found. An invariant written immediately after a fix tends to state what the fix established rather than what the code guarantees, because the evidence in front of you is the defect you just closed. The question that would have caught it is about the OTHER function, not about the bug: \"what else does the thing I approximate do that I have not mirrored?\" The agent retracted it unprompted, before it was ever recorded, and replaced it with the per-arm form now held as encoded_width_mirrors_every_preprocess_branch, which is the right outcome and the reason to record this.').
doctrine(an_advisory_bound_over_another_function_is_a_duplicated_derivation,'encoded_width exists to approximate what preprocess will allocate, which makes it a SECOND IMPLEMENTATION in the improvise-edp sense -- and both of its bugs were invisible to a green suite for the same reason: NOTHING COMPARED THE TWO AT ALL. A width asserted against a constant passes at every stage while the approximation drifts. The only honest test of an advisory bound is to run the thing it approximates and compare against what that actually produced (here data.feature_names.len()), never against a number computed a second way -- which would be comparing_a_function_against_itself_tests_nothing wearing a different hat.').
doctrine(when_review_finds_a_check_weaker_than_claimed_strengthen_the_check,'improvise-26dm round 1 found its own defect -- the right-hand side of a completeness diff was a second HAND-WRITTEN list, not the derived enumeration the prose claimed -- and shipped an honest correction of the prose. ChatGPT Codex asked for the FIX instead, and was right: the disclosure was accurate and still the weaker artefact, because the fix was four lines of plumbing. Correcting prose is the FALLBACK, not the deliverable. The repaired chain is derive -> every_model_command() -> model_effects -> case list, every link checked, bounded by variants::declared::<ModelCommand>(). Note the staged failure is better evidence than a single one: Codex own injection now fails the two checks ONE AFTER THE OTHER, and that ordering demonstrates the chain has two links rather than one test doing two jobs badly.').
doctrine(cfg_test_and_pub_crate_are_different_walls,'A visibility plan written from a grep of the item line will miss the gate on the line ABOVE it. improvise_protocol::variants reads pub(crate) mod variants, so the follow-up -- filed by the agent that wrote it, and repeated by the orchestrator authorising \"one token\" -- said pub(crate) to pub. The attribute above is #[cfg(test)], and declared() needs serde_json, which that crate carries only as a DEV-dependency, so the one-token change could not compile. The shape that works is an optional serde_json behind a test-support feature, #[cfg(any(test, feature = \"test-support\"))] pub mod variants, and a dev-dependency enabling it from the consumer.').
doctrine(cargo_tree_shows_dev_edges_by_default,'Asking whether a feature leaked a dependency into a shipped build, a plain cargo tree -p improvise-protocol --target wasm32-unknown-unknown reports serde_json PRESENT -- because the default output includes dev edges. --edges normal is what answers the question, and it reports 0. Verified both ways by the orchestrator. Same class as grep_over_git_diff_is_silently_empty_under_an_external_differ: a command that looks like it answers the question, answering a neighbouring one, in the direction that reads as a finding rather than as a pass. A feature that adds a dependency to a wasm bundle crate must be checked with --edges normal, and per-crate, because the CI wasm job checks each crate separately and feature unification cannot reach across that.').
doctrine(an_unreachability_claim_is_only_as_wide_as_the_routes_enumerated,'PR #98 predicted the composition-time mode drift itself and declared it unreachable BECAUSE the toggle route (R) is a stranding decline in the browser. True of that route. A VIEW SWITCH reached the same drift, because records-ness is auto-detected from the axes, so a switch changes it with nothing in the batch saying \"records\" -- found by ChatGPT Codex. The ENUMERATION of routes is the part nothing checks, so an unreachability argument is evidence about the routes you thought of and about nothing else. Same shape as a_mitigation_that_names_the_wrong_file_is_not_a_mitigation. The remedy is to close the CLASS by observation rather than to patch the route.').
doctrine(a_precondition_derived_from_its_own_expectation_guards_nothing,'A loop test whose precondition is computed from the same tuple element as the assertion cannot be relaxed one iteration at a time: relaxing the assertion relaxes the guard too, and the test then fails for the wrong reason -- which reads exactly like the bug. Assert the two as SEPARATE facts. Corollary for check_the_checker over a loop: an injection that stops the loop at iteration 1 leaves every later iteration untested, so each direction must be reached deliberately. Found by the PR #98 author in its own bidirectional test while proving the second direction.').
doctrine(a_pipe_discards_the_exit_code_you_meant_to_read,'FOURTH occurrence in one campaign, and the last one was in a tool written TO BE the reliable answer. scripts/pr-watch once read tr exit status through a pipe. An orchestrator probe reported EXIT=0 from a checker that had exited 1, because the pipeline ended in tail. Another read PIPESTATUS in zsh, where the array is lowercase pipestatus. And scripts/pr-reviews --wait correctly exited 1 on a timeout while the invocation \`... | tail -20; echo\` reported 0 -- so a review window that had EXPIRED read as a review that had ARRIVED, which is the dangerous direction. In bash and zsh the exit status of a pipeline is the LAST command, so any \`cmd | head\`, \`| tail\`, \`| grep\` discards it. Redirect to a file and read the file, or check PIPESTATUS[0] in bash / pipestatus[1] in zsh -- and note the two shells differ, which is its own trap. A gate whose status you sampled through a pipe has told you about the pager.').
doctrine(editing_a_shell_script_corrupts_the_instances_already_running_it,'bash reads a script INCREMENTALLY rather than loading it whole, so editing a file while an instance is executing it shifts the byte offsets underneath that instance. Two background pr-reviews watchers, started ten minutes earlier and sleeping in their poll loops, both died with \"syntax error near unexpected token do\" the moment the orchestrator appended a paragraph to the usage text -- and the script itself was syntactically fine, which bash -n confirmed. The failure is in the RUNNING instance, not the file, so it is invisible to every check you would think to run afterwards. Before editing a script, ask what is still executing it; a long-poll watcher is exactly the shape that will be. Worse in this case than a crash: the watcher exited having printed only the EARLIER review, so the wait it was supposed to perform silently did not happen.').
work_done('improvise-8tk',bug_fix,'ZZ/:wq quit even when the save failed, silently discarding the model. Save now aborts the effect batch on failure; save_to_current_path/1 returns Result<String,String>; three tests in ui/effect.rs.').
work_done('improvise-907',bug_fix,'Import wizard step machine redesign: validation_error() gate at ReviewProposals (no-category and over-limit), history-stack previous() bound to Left on every step, and Done made a real confirmation step holding the built workbook. Superseded improvise-gfq, improvise-o1c, improvise-c6j.').
work_done('improvise-6sq',wont_do,'Merging EditOrDrill and EnterEditAtCursorCmd into one EnterEdit{mode} is not safe: they differ in WHEN the cell value is read, not in a mode parameter. Added ui/app.rs::tests::i_and_o_diverge_on_an_empty_records_grid as executable evidence.').
@@ -400,6 +410,7 @@ decision('improvise-mcq4',cursor_frame_is_one_view_on_geometry_and_every_view_on
decision('improvise-ni6k',cursor_invalidation_is_observed_not_declared,'No effect calls reset_cursor. apply_effects diffs the grid frame after every model effect, INSIDE the loop, because a batch that reframes and then places the cursor loses the placement otherwise (four records-mode tests pin that position; two after the arms were split). Total over EffectKind::Model(_) by construction rather than by a maintained list -- the thirteen per-effect calls it replaced are gone, and the \"a geometry effect that forgets to reset is invisible in both directions\" hazard is closed for model effects. Totality remains conditional on CLASSIFICATION, which nothing makes total: a ViewOnly effect that renumbers the grid is invisible to all of it.').
decision('improvise-864a',pages_artifact_reaped_by_a_foreign_successful_deploy,'Measured over 100 push-to-main runs 2026-09-19..2026-09-21: 12 of 12 artifact-not-found deploy failures had ANOTHER run SUCCESSFUL Pages deployment complete inside their own upload-to-deploy window, against 2 of 56 successes. Direction proved by the oldest run of the 17:47Z batch failing BEFORE the next deploy started and before any deployment existed, since deploy-pages fetches artifact metadata before creating one. NOT queue depth, NOT supersession, NOT retention, NOT the action deleting it. PR #46 tip-of-main guard aimed at the right quantity for the WRONG REASON and still must not ship: the association is not a mechanism, and 5f871f7 survived a foreign deploy inside its own window. Reproduce deliberately before fixing.').
decision('improvise-zv8t',a_document_category_is_never_inferred,'fit_measure now builds its Overrides from the same header[..category_columns] slice it already handed Features::of, forcing each to Forced::Categorical, so the one place the document own answer was known is the only place either caller reads it from. Closes Artefact::NumericCategory and DatePart-over-a-category from the only production entry point. The Artefact arms STAY: crate::dataset is pub and takes the caller Overrides, so a caller that builds one without forcing still produces a Numeric role over a category column -- deleting the arm would derive the classification from one caller habits rather than from the roles. Cost accepted and filed as improvise-e8no: a wide numeric category now one-hots past MAX_FEATURES and refuses a fit that used to ignore it. Do NOT close e8no by making the forcing conditional on cardinality -- that is inference by another name, and makes the ENCODING depend on a budget rather than on what the document says.').
decision('improvise-e5pf',the_mode_settles_where_the_grid_does,'resolve::changed_mode is DELETED. Every composition site emits the plain AppMode::Normal, and ClientState::fold settles against the grid it HOLDS via improvise_protocol::settled_mode -- one rule, also called by ClientSession::fold and App::settle_normal_mode, with App::normal_mode deleted as its host-side duplicate. Composition-time settling was PREDICTION: switch_view_at composes [SwitchView(target), ChangeMode(Normal)] and read the OUTGOING grid. Read-time derivation was rejected for having the same window and for moving the enumeration from composition sites to READERS, where a missed reader is silent. settled_mode settles the Normal pair ONLY -- Editing/RecordsEditing pass through -- which is why a records decision must ask the grid and never the mode. Same shape the repo already prefers: cursor_epoch over matching on CycleAxis, document_generation over matching on LoadModel, the dirty flag over a table of command variants.').
wire_carries(command,both_directions,'user intent up, projection commands down').
wire_carries(snapshot,on_connect_only,'improvise-q08: full ViewState + render cache filled from current viewport').
wire_carries(command,both_directions,'user intent client->server, projections server->client').
scripts/check-comment-runs.py
+94 -9
View File
@@ -413,8 +413,33 @@ def js_files(roots):
yield os.path.join(dirpath, f)
def _duplicate_claims(claims, by_leaf, defined):
"""Two headings naming ONE test, keyed by the test they resolve to.
Resolved and not spelled, so `### f` beside `### m::tests::f` is caught
too: the heading is the key a reader looks a test up by, and two entries
under one key means the first one found is taken as current. See
context/code/tooling.md.
"""
def canonical(kind, name):
if kind == "js" or name in defined["rust"]:
return name
matches = by_leaf.get(name, set())
return next(iter(matches)) if len(matches) == 1 else name
grouped = {}
for (kind, name), where in claims.items():
grouped.setdefault((kind, canonical(kind, name)), []).extend(where)
def in_file_order(loc):
path, _, n = loc.rpartition(":")
return (path, int(n) if n.isdigit() else 0)
return {key: sorted(where, key=in_file_order)
for key, where in grouped.items() if len(where) > 1}
def evidence_verdict(claims, defined, citing):
"""(orphans, unstated, ambiguous), keyed by ("rust"|"js", name).
"""(orphans, unstated, ambiguous, duplicates), keyed by ("rust"|"js", name).
`orphans` is evidence naming a test nobody can find. `unstated` is the
reverse obligation (improvise-lyyc): a test that POINTS AT the evidence
@@ -422,8 +447,11 @@ def evidence_verdict(claims, defined, citing):
`ambiguous` is a bare leaf that two tests answer to -- a refusal and not a
warning, because a heading satisfied by a test in another module answering
another question is a pass while the gate does nothing.
`duplicates` is the mirror image: two headings answered by one test, where
every copy passes and the reader takes whichever comes first as current
(improvise-hsoe). Also a refusal, on the improvise-rm4s precedent.
Only that narrower reverse obligation, because most tests have no evidence
and should not be forced to. What none of the three can see: an entry can
and should not be forced to. What none of the four can see: an entry can
exist and still describe its subject backwards.
"""
by_leaf = {}
@@ -452,7 +480,7 @@ def evidence_verdict(claims, defined, citing):
for name in names:
if (kind, name) not in claimed:
unstated.setdefault((kind, name), []).extend(sorted(names[name]))
return orphans, unstated, ambiguous
return orphans, unstated, ambiguous, _duplicate_claims(claims, by_leaf, defined)
def check_evidence(roots, rust_listing=None, rust_ignored=None):
@@ -493,10 +521,10 @@ def check_evidence(roots, rust_listing=None, rust_ignored=None):
for name in c:
citing["js"].setdefault(name, []).append(path)
orphans, unstated, ambiguous = evidence_verdict(claims, defined, citing)
if not orphans and not unstated and not ambiguous:
orphans, unstated, ambiguous, duplicates = evidence_verdict(claims, defined, citing)
if not orphans and not unstated and not ambiguous and not duplicates:
print(f"test-evidence: {len(claims)} entries, all name a test that runs "
f"(names from {source}); "
f"and no test is named twice (names from {source}); "
f"{sum(len(v) for v in citing.values())} tests cite the file and all have one; "
f"{prose} prose headings claim nothing")
return 0
@@ -514,6 +542,11 @@ def check_evidence(roots, rust_listing=None, rust_ignored=None):
print(f" [{kind}] {name} ({', '.join(where)})", file=sys.stderr)
for path in sorted(by_leaf[name]):
print(f" -> {path}", file=sys.stderr)
if duplicates:
print(f"\n{len(duplicates)} tests are named by more than one test-evidence "
"heading; merge them or give each its own test:\n", file=sys.stderr)
for (kind, name), where in sorted(duplicates.items()):
print(f" [{kind}] {name} ({', '.join(where)})", file=sys.stderr)
if unstated:
print(f"\n{len(unstated)} tests cite {EVIDENCE_DIR} and have no entry there:\n",
file=sys.stderr)
@@ -910,15 +943,64 @@ SELF_TEST_AMBIGUITY = [
]
def _ambiguity_self_test():
# (name, evidence markdown, cargo listing, js source, the tests named twice).
#
# A heading that names a test that RUNS passes -- and a second heading naming
# the same one passes just as well, so an entry superseded by its successor
# reads as current to whoever meets it first (improvise-hsoe, found on PR #92).
# Every case here is an injection; the clean arms stop an always-fail passing.
SELF_TEST_DUPLICATES = [
("one heading twice in one file is refused",
"### a_real_test\n### a_real_test\n", "m::tests::a_real_test: test\n", "",
["m::tests::a_real_test"]),
("the qualified and bare spellings of one test are the same heading",
"### a_real_test\n### m::tests::a_real_test\n", "m::tests::a_real_test: test\n", "",
["m::tests::a_real_test"]),
# A BARE heading, because two qualified ones resolve by the exact-match arm
# and never reach the ambiguity guard this case is named for (review, PR #96).
("a bare heading over an ambiguous leaf is not a duplicate of a qualified one",
"### shared\n### m::tests::shared\n### n::tests::shared\n",
"m::tests::shared: test\nn::tests::shared: test\n", "", []),
("a JS heading twice is refused too",
'#### "a spec"\n#### "a spec"\n', "", 'test("a spec", () => {});\n', ["a spec"]),
("two headings naming two tests are not duplicates",
"### one\n### two\n", "m::one: test\nm::two: test\n", "", []),
("prose headings repeat freely -- they claim no test",
"### Helper - x\n### Helper - x\n", "", "", []),
]
def _duplicate_self_test():
failures = []
for name, md, listing, js, want in SELF_TEST_DUPLICATES:
claims = {}
for kind, cname, n in evidence_claims(md):
if kind != "prose":
claims.setdefault((kind, cname), []).append(f"fixture:{n}")
jd, _ = js_tests(js)
defined = {"rust": cargo_test_names(listing), "js": jd}
*_, duplicates = evidence_verdict(claims, defined, {"rust": {}, "js": {}})
got = sorted(k[1] for k in duplicates)
if got != sorted(want):
failures.append(f" DUPLICATES: {name}: expected {sorted(want)}, saw {got}")
return failures
def _ambiguity_self_test():
failures = _duplicate_self_test()
for name, md, listing, want_o, want_a in SELF_TEST_AMBIGUITY:
claims = {}
for kind, cname, n in evidence_claims(md):
if kind != "prose":
claims.setdefault((kind, cname), []).append(f"fixture:{n}")
defined = {"rust": cargo_test_names(listing), "js": set()}
orphans, _, ambiguous = evidence_verdict(claims, defined, {"rust": {}, "js": {}})
orphans, _, ambiguous, _dupes = evidence_verdict(
claims, defined, {"rust": {}, "js": {}})
got_o = sorted(k[1] for k in orphans)
got_a = sorted(k[1] for k in ambiguous)
if got_o != sorted(want_o):
@@ -953,7 +1035,9 @@ def _evidence_self_test():
defined = {"rust": cargo_test_names(listing), "js": jd}
citing = {"rust": {k: ["fixture.rs"] for k in rc},
"js": {k: ["fixture.js"] for k in jc}}
orphans, unstated, ambiguous = evidence_verdict(claims, defined, citing)
orphans, unstated, ambiguous, duplicates = evidence_verdict(claims, defined, citing)
if duplicates:
failures.append(f" DUPLICATES: {name}: unexpected {sorted(k[1] for k in duplicates)}")
got_o = sorted(k[1] for k in orphans)
got_u = sorted(k[1] for k in unstated)
if got_o != sorted(want_orphans):
@@ -1037,6 +1121,7 @@ def self_test():
f"{len(SELF_TEST_EVIDENCE)} evidence verdicts, "
f"{len(SELF_TEST_LISTING) + len(SELF_TEST_HEADINGS)} listing readings, "
f"{len(SELF_TEST_AMBIGUITY)} name resolutions, "
f"{len(SELF_TEST_DUPLICATES)} duplicate-heading verdicts, "
f"{len(SELF_TEST_DISPATCH)} invocations answer the cap")
return 0
scripts/pr-reviews
Executable
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# Prints a PR's reviews, and optionally waits for one against the current head.
# Exists because twenty PRs merged with their reviews unread; see
# context/code/pr-review.md.
set -euo pipefail
usage() {
cat >&2 <<'EOF'
usage: pr-reviews <pr-number> [--wait <sec>] [--interval <sec>]
--wait <sec> wait up to this long for a review whose commit is the
current head (default 0 = print what exists and exit)
--interval <sec> seconds between polls (default 30)
exit 0 a review against the current head exists (or --wait was 0)
exit 1 --wait elapsed with no review against the current head
exit 4 usage or environment error
DO NOT PIPE THIS INTO head/tail. `$?` is then the pager's status, not this
script's, and a --wait that timed out reads as success. That trap has cost
this repo four separate diagnoses. Redirect to a file and read it instead.
A review may target an EARLIER head, which is why the head is compared rather
than the count: a finding against an old commit may be fixed or still live, and
only reading it tells you which.
EOF
exit 4
}
PR=""; WAIT=0; INTERVAL=30
while [ $# -gt 0 ]; do
case "$1" in
--wait) WAIT="${2:-}"; shift 2 ;;
--interval) INTERVAL="${2:-}"; shift 2 ;;
-h|--help) usage ;;
*) [ -z "$PR" ] || usage; PR="$1"; shift ;;
esac
done
[ -n "$PR" ] || usage
# `gh` is shell-aliased to a different tool in this repo's shells, so the name
# is never trusted: a real gh is located explicitly.
gh_() {
if command -v gh >/dev/null 2>&1 && gh --version 2>/dev/null | grep -q '^gh version'; then
gh "$@"
else
nix shell nixpkgs#gh --command gh "$@"
fi
}
fetch() { gh_ pr view "$PR" --json headRefOid,reviews; }
report() {
local json="$1" head
head=$(printf '%s' "$json" | jq -r '.headRefOid')
printf 'pr-reviews: #%s head=%s\n' "$PR" "${head:0:8}"
printf '%s' "$json" | jq -r --arg head "$head" '
if (.reviews | length) == 0 then " (no reviews)"
else .reviews[] |
" [\(.state)] \(.submittedAt) commit=\(.commit.oid[0:8])" +
(if .commit.oid == $head then " <- CURRENT HEAD" else " <- an EARLIER head; verify whether it still applies" end) +
"\n" + (.body | split("\n") | map(" " + .) | join("\n"))
end'
}
has_head_review() {
printf '%s' "$1" | jq -e --arg h "$(printf '%s' "$1" | jq -r .headRefOid)" \
'any(.reviews[]?; .commit.oid == $h)' >/dev/null 2>&1
}
json=$(fetch)
report "$json"
[ "$WAIT" -gt 0 ] || exit 0
if has_head_review "$json"; then exit 0; fi
deadline=$(( $(date +%s) + WAIT ))
while [ "$(date +%s)" -lt "$deadline" ]; do
sleep "$INTERVAL"
json=$(fetch)
if has_head_review "$json"; then
printf '\npr-reviews: a review against the current head arrived.\n\n'
report "$json"
exit 0
fi
done
printf '\npr-reviews: waited %ss; no review against the current head.\n' "$WAIT"
printf 'Reviews arrive as the owner quota allows, so absence is not approval.\n'
exit 1