Copilot on #297: `pr-reviews` only calls `date +%s`, so the REAL_DATE
pass-through never ran. It is deleted rather than tested. The fake now refuses
any other arguments with exit 2, so a new `date` call shows up as a failure.
Rival: with pr-reviews calling `date -u +%s`, "a review arriving on the third
poll is seen" and "--wait keeps polling until its deadline" fail (23/25).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
This follows the user's rulings: CI should not care which bash or which
standard utilities it runs, and jobs take what the ci dev shell provides.
- checkers.sh: remove find_modern_bash, the 5.1 floor (improvise-fd0j) and
exit 3. Suites run under $BASH with PATH unchanged.
- checkers-test.sh: remove sections 2, 5 and 5b (the fake bashes and utilbin).
They existed only to pin the floor.
- scripts/kb: drop the BSD `stat -f` fallback. kb needs the dev shell for tpl,
and that shell's stat is GNU.
- ci.yml: remove every `shell: bash` override, so every step runs in the dev
shell. Remove the web-toolchain version probe and the Playwright
version/browser probe. The shell's PLAYWRIGHT_BROWSERS_PATH pins the
browsers. npm ci stays because @playwright/test is not nix-provided.
- flake.nix: update the comments that promised those CI checks.
Reasoning: context/code/tooling.md section 5. Evidence:
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
On the NixOS runners /bin holds only sh. With no /bin/sleep, the watchdog in
bounded_reviews exits at once and wins the `wait -n`, so every --wait case
reports 124. Both tools are now resolved with `type -P` before any case stubs
PATH.
Red: with both paths pointed at /nonexistent/, the suite fails 25 of 25.
Rival: REAL_SLEEP=/nonexistent/sleep fails 25 of 25 again. The date
pass-through has no discriminating rival, because pr-reviews only calls
`date +%s`. Evidence: context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
Section 5b (the bash 5.1 floor, #237) was written on a branch cut before the
NixOS-runner change (af44653f). That change moved section 5 off /usr/bin:/bin,
because on NixOS that path holds only env and sh. The two merged with no
conflict. On NixOS the runner then dies on `dirname` with exit 1, which turned
the KB job red on every PR at 666de199.
5b now uses utilbin too, and utilbin gains head and env for the 5.1 control.
Evidence and rivals: context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
A skip-worktree or sparse path is absent without being deleted, and
the filter skipped it, so its ids were never read and the audit
passed. The xargs comment no longer claims 124, which is GNU only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
find_modern_bash accepted any 5.x, so a bash 5.0 was pinned and
pr-reviews-test.sh then failed on an invalid option. The probe now
reads major and minor; a 5.0 case and a 5.1 control guard it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-lqs0, from the #226 review. xargs reports grep's no-match and
its read error alike, as 123. So a cited file the scan could not read
passed the audit, and an unreadable file holding the only dead id exited
0. Each grep batch now maps exit 1 to success and exit 2 to 255, so xargs
stops and the audit is exit 4. A tracked file deleted from the working
tree is left out on purpose, because its ids are leaving the tree.
Two exit-code mismatches with the stated contract are fixed too. Running
outside a repository is now exit 4, not 1. A bd list whose objects carry
no string id is now exit 3, not "every id dead".
Six new cases reproduced the bug as 10 passed, 4 failed, and now pass
14/14. Four rivals each fail their own case.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-7fl8: on the one-tick-per-read clock --wait 5 polls exactly 6
times, but the test asserted at least 5. So a loop capped at 5 polls, or
a deadline of now + WAIT - 1, passed at 25/25. It now asserts
polls = WAIT + 1, and both rivals fail it.
With the deadline check deleted, the suite used to hang at the first
real-clock timeout case. Every pr-reviews call now runs beside a
/bin/sleep watchdog (wait -n -p), and loses to it as exit 124. The same
rival now finishes in 93s with four FAILs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-52eq: the AGENTS.md recipe searched `bd show` output for "not
found" and printed "all resolve" when there was none, so a bd that was
missing, failing or never run passed the audit. scripts/bd-audit-ids
keeps the file scan and exclusions, then `comm -23`s the cited ids
against `bd list --all --limit 0 --json`. A dead id is exit 1 and is
named. A bd that does not answer is exit 3, never a pass.
scripts/bd-audit-ids-test.sh has eight cases on a throwaway repo with a
fake bd. The old recipe as a rival fails 7 of the 8. Disabling the
comparison fails the dead-id case, and an empty exclusion list fails the
pass case. CI runs it with the other named suites. The "One environment
note" now reports which forms the guard refused and allowed, as observed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-7epn: two cases on the one-tick-per-read clock. One resolves on
poll 3, and one times out after polling up to the deadline. Replacing the
post-poll deadline check with `break` now fails exactly those two
(23 passed, 2 failed), where before it left the suite 23/23.
improvise-lnsh: when `cargo metadata --locked` refuses the opfs-contract
lock, build.sh now says not to remove --locked and names
`cargo update -w --manifest-path harness/opfs-contract/Cargo.toml`.
Checked by hand against a lock with pest pinned to 2.9.1. Evidence is in
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-qnpi: `pr-reviews --wait` checked its deadline before the first
re-poll, so a second boundary between two reads of the clock skipped the
only poll (exit 1 with polls=1, reproduced under CPU load). The deadline
is now checked after each poll. A new case drives it with a PATH shim
whose `date` ticks one second per read, so it fails deterministically on
the old loop rather than under load.
improvise-mkbp: harness/opfs-contract/Cargo.lock pinned pest 2.9.1 while
its path deps require 2.9.2, so every build rewrote it. The resolved lock
is committed and build.sh starts with `cargo metadata --locked`; the
wasm-pack `-- --locked` route was tried and does not work (wasm-pack
re-resolves first). opfs.spec.js now names the fix when harness/serve/opfs
is missing, which is what running web/assemble.sh alone leaves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-6yvg reproduced. A JSON import in improvise-command and a legacy
JSON .improv load in improvise-io stored 0.24815714285714285 as
0.24815714285714283 in each crate's own test build. The root binary's
`improvise import` used the same lossy parser. Now float_roundtrip is on every
normal serde_json dependency in the workspace: root, improvise-command,
improvise-io, and the optional test-support dependency of improvise-protocol.
improvise-dz8c: the hand-listed cargo-tree check over three frame-parsing
crates is replaced by tests/json_numbers.rs. It reads
`cargo metadata --no-deps` and names every package, including target-specific
entries, whose normal serde_json dependency lacks the feature. The rule and why
it beats "parses a frame", the build graph, and enabling the feature once in
the protocol crate are in context/code/protocol.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The required Browser harness check comes from a job that needs: build, so for
a moment per push every registered check is green and GitHub says
MERGEABLE/BLOCKED. #186 at 128a2a0 refused that and exited 1 on healthy PRs.
BLOCKED with no failed check is now waited on (exit 3 if it never clears).
Adds the gap sequence from PR #189's real rollup, and a head moving during
the UNKNOWN wait. Corrects tooling.md's claim that nothing else produces
BLOCKED, the evidence entry's rival-2 claims, and the fixture wording.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
GitHub reports UNKNOWN for a while after the base moves, and PRs #172 and
#179 both exited 0 there. UNKNOWN is now waited on (exit 3 on timeout), a
failed check or CONFLICTING ends the wait, and the pass condition is
MERGEABLE with CLEAN, UNSTABLE or HAS_HOOKS. BLOCKED, DIRTY, BEHIND, DRAFT
and any unshipped state fail closed. The suite enumerates the whole table.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The kb skill (both copies) now describes the HEAD comparison, the ledger,
exit code 3 and the recovery. kb-tool.md records the ih3k/yea8 arguments and
corrects the residue paragraph: .gitignore hides the leftovers, and there are
six, not one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The fixture loop covers every package the fixture has; this covers every
package the tree has. The review's skip-the-root rival now fails the real
repo's own gate (exit 2 naming Cargo.toml), not only the test suite.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
scripts/kb add wrote onto a working-tree kb.pl that had lost 276 of its 801
clauses and reported success, because kb_consistent checks consistency, not
completeness (improvise-usus). Every write now runs kb-merge.pl `complete`
on the candidate before kb_consistent. It refuses unless every clause HEAD
has is still present by key, or its removal was recorded by `kb rm` in a
per-worktree ledger in the git dir, which prunes itself on every write.
The candidate is now built beside kb.pl, so publishing it is a rename(2)
rather than a possible cross-device copy that a kill could truncate.
Exit codes: 0 done, 1 refused, 2 usage, 3 HEAD's kb.pl unreadable. kb-test.sh
now builds its fixtures from HEAD and adds cases 18-22. Argument in
context/code/kb-tool.md, evidence and rivals in context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
ChatGPT Codex on PR #159: both formatter invocations end `|| rc=1`, so any
failure after the rustfmt probe was reported as "misformatted" although no
verdict was produced. Correct, and the narrower form of the bug this gate
exists for.
Measured: an unparseable source exits 1, identical to a diff; an unknown
rustfmt.toml option is only a warning (exit 0).
Narrowed rather than re-classified, because telling a diff from a parse error
means parsing rustfmt's prose, which is the check-adjacent-to-its-subject this
repo keeps getting wrong. The line the split needs is "did the gate run":
exit 1 names a file and blames the tree; exit 2 names none and blames the
invocation, and every exit-2 cause is checked above the loop.
Two more rivals injected: deleting the `cargo fmt --version` probe, and
turning `|| rc=1` into `|| return 2`. The first exposed a vacuous test -- a
cargo stub that failed for every argument also breaks `cargo metadata`, so the
run exited 2 on the metadata arm and the probe could be deleted with the suite
green. The stub now delegates every subcommand but `fmt`, which is the shape a
missing rustfmt component actually has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-b4ks. From the repo root `cargo fmt -p opfs-contract -- --check`
exits 1 with "package `opfs-contract` is not a member of the workspace" and
never runs --check, so a caller reading $? cannot tell "misformatted" from
"nothing was checked". Reproduced at 7cef9f8.
scripts/fmt-workspaces.sh discovers every [workspace] root and checks each
with `-p <member>` scoped by that root's own --manifest-path. Exit 2 is a
usage error or an unrunnable cargo; exit 1 is a formatting verdict.
Both obvious spellings were measured and both fail in the direction that
looks like success: `--all` crosses the boundary, because it formats local
path dependencies and harness/opfs-contract depends on improvise-io by path
(an injected fault in src/draw.rs was reported by the harness root); and the
bare --manifest-path form is blind to workspace members, because the repo
root is a package as well as a workspace (an injected fault in
crates/improvise-core exited 0).
scripts/fmt-workspaces-test.sh is the check_the_checker. Four rivals were
injected and each failed it. Its fixture had to be corrected twice -- a
missing path dependency and a virtual outer manifest -- and both vacuities
were found only by injecting the rival, never by reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`scripts/pr-reviews-test.sh` passed 19/19 locally and 2/19 in CI on the same
commit, because a multi-assignment `local` is not left-to-right on bash 5
while macOS's bash 3.2 tolerates it. The obvious half of the fix is pinning
the shell. The half that decides the design is not.
.envrc DOES already put a nix bash 5.3.15 on PATH, which is CI's semantics --
and it is not enough. `pkgs.mkShell` exports `name`, `out`, `outputs`,
`system`, `shell`, `builder`, `stdenv`, `phases` and about twenty more
generically-named variables that CI does not have, so a script reading one by
accident under `set -u` is SILENT in the dev shell and fatal in CI. That is
exactly what happened: the broken clause read `$name`, and in the dev shell
`$name` was `nix-shell-env`. Measured at 051f541^ on one machine:
macOS /bin/bash 3.2 19 passed, 0 failed
direnv exec . ./scripts/pr-reviews-test.sh 19 passed, 0 failed
the same, with `name` unset 2 passed, 17 failed
Two reasons for green, one of them the correct interpreter. A dev shell is a
third environment, not a reproduction of CI.
A nix shebang was measured and declined, on availability rather than speed:
warm, `nix shell nixpkgs#bash --command true` is 61.8ms against an ambient
5.1ms, and `nix-shell -p bash --run true` -- the manpage form -- is 675.2ms,
132x, inside git's merge machinery for beads-merge-driver.sh. But only the
`kb` job installs nix; build, test, wasm, js, harness-lint and browser have
none, so a nix shebang needs an installer step on three more jobs to reach a
bash those runners already ship. zsh is not a shebang change and the runner
does not have it either. Both declined in context/code/tooling.md section 5.
What lands instead pins interpreter and environment together, in one place:
scripts/checkers.sh is the one entry point for every check_the_checker, used
verbatim by CI and by the local gate. It pins a bash >= 5 (ambient first, nix
only when the ambient one is older, exit 3 when neither), scrubs the
environment to an allowlist, and refuses any shell shebang in scripts/ that
is not `#!/usr/bin/env bash`. The pinned bash goes on the front of PATH so
inner invocations get it too -- which is where the fault actually lived.
The second undetected split is closed with it: the four `#!/bin/sh` and one
`#!/usr/bin/env sh` scripts are now bash, and the six places that ran a
script through an explicit `sh` -- overriding the callee's own shebang, which
is the same disease -- now invoke it directly.
Verification, injected against the real script: restoring pr-reviews'
multi-assignment `local` leaves `sh scripts/pr-reviews-test.sh` at 22/22 and
`direnv exec . ./scripts/pr-reviews-test.sh` at 22/22, while
`./scripts/checkers.sh pr-reviews` reports 2 passed, 20 failed, exit 1 --
matching the 2/19 CI actually reported. scripts/checkers-test.sh is twelve
cases, each injecting rather than asserting; the pair that carries it runs
one fixture both ways, so the `set -u` case is a test of the runner and not
of `set -u`.
The old local gate was `for t in scripts/*-test.sh; do sh "$t"; done`, which
overrides every shebang with /bin/sh -- so it ran every suite under bash 3.2
in sh-mode regardless of what the file said. It is replaced by
`./scripts/checkers.sh`.
Workspace suite unchanged at 1689 passing over 38 binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Both found by ChatGPT Codex on PR #133, both invisible on macOS, both
arriving as exit 2 with no output -- jq's answer to an invalid --argjson,
so a shell fault at the top of the script wears jq's exit code.
`local n="$1" name="$2" f="$DIR/$n.$name.json"` creates every name before
running the assignments on bash 5, so $n is unbound in the third clause
and set -u kills the shell. The dead command substitution then hands
--argjson an empty string. Works on macOS bash 3.2: the suite passed
19/19 locally and 2/19 in CI on the same commit. Each local now gets its
own statement.
`gh api --paginate` emits one JSON array PER PAGE, so a comment thread
past the page size is several concatenated documents that --argjson
rejects the same way. Both fetch paths now flatten through one
pages_to_array (`jq -s 'add // []'`), which is a no-op on a single page.
The fixture path shares that helper, which is the only reason a fixture
can test it: a fixture holding a single array would exercise a shape the
live fetch never produces. Three cases added, and removing the helper
from the fixture path fails exactly those three, the wait case with the
live exit-2 signature. Suite is 22 cases.
No PR in this repo has yet had enough inline comments to reach a second
page, so nothing but the test will see that one before a user does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
It printed review BODIES only and was blind to inline review comments --
the surface a human reviewer is most likely to use. On PR #122 the user
reviewed with the body "Generally good, added comments inline." and five
substantive rulings underneath it; pr-reviews printed the one-liner, and
an agent reported the PR as awaiting review while the answers sat on it.
pulls/{n}/comments and issues/{n}/comments join pulls/{n}/reviews, all
three interleaved in time order, each inline comment carrying its path
and line (falling back to original_line, which is what an outdated
comment has left).
The exit contract is unchanged: 0 means something exists against the
current head, 1 means the wait elapsed, 4 is usage. An inline comment at
the head now satisfies --wait -- it carries a commit_id and is exactly
the case that failed. An issue-level comment does not: it names no
commit and so makes no claim about which tree was read.
The usage's exit-code warning is widened from "do not pipe into
head/tail" to the real rule: nothing may follow the script in the same
`;` list either, because a trailing `echo "exit=$?"` makes the status
the echo's.
scripts/pr-reviews-test.sh is the check_the_checker, fixture-driven and
needing no network, wired into CI's kb job beside pr-watch-test.sh and
release-preflight-test.sh. Five faults were injected and watched to
fail; context/test-evidence/tooling.md records which case caught each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
All three found by the adversarial review of 019cbc1, each by injection.
1. every_model_command() is NOT compiler-maintained, and both this PR's prose
and reduce.rs's own doc comment said it was. variant_name's exhaustive match
forces an arm in variant_name, not an entry in the value list -- deleting
ModelCommand::TogglePruneEmpty from it failed the new completeness check and
NOTHING ELSE in the workspace. So the diff is two independent hand-written
lists: strictly better than one list against its own length, and not the
derived enumeration improvise-5ubs got for the two enums. Doc comment
corrected; the prose in context/code/command.md and the evidence entry now
say what it is. Closing it properly wants improvise_protocol::variants made
pub so declared::<ModelCommand>() can bound the list -- another wave's crate,
reported rather than taken.
The assertion message was also ONE-DIRECTIONAL ("a name only on the right
..."), which is the shape that injection produces and which invites the wrong
repair: delete the case on the left, degrading both sides together. It now
names both directions and says which is which.
2. SELF_TEST_DUPLICATES case 3 was named for the ambiguity guard and never
reached it: both its headings were module-qualified, so canonical() returned
on the exact-match arm before touching by_leaf. Dropping the guard
(`if matches` for `if len(matches) == 1`) left --self-test at exit 0. The
fixture is now one BARE heading over an ambiguous leaf plus BOTH qualified
spellings, so the injection collides with one of them whichever way
next(iter(set)) falls -- deterministic across five runs, verified.
3. The evidence entry said five pre-existing records-mode tests fail under the
settle-before-rebuild injection; measured, it is six. And "three hundred
lines above" is four hundred.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
--check-evidence verified that a heading names a test that RUNS, and a
duplicate names one just as well, so both copies pass. PR #92 shipped two
entries under one heading; the older described a fixture the test no longer
had.
Refuse rather than warn, on the improvise-rm4s precedent, naming the file and
every line in file order. Keyed by the RESOLVED test, so `### f` beside
`### m::tests::f` collides too -- the case rm4s's qualification repair creates.
Six SELF_TEST_DUPLICATES cases, including the three clean arms: two different
tests sharing a leaf are ambiguous and not duplicate, prose repeats freely, and
two headings naming two tests are fine.
Resolves the two tui.md duplicates: the older column-header copy called
THE_MEMBERSHIP_CORRECT_SPANS a live tripwire three hundred lines above the
entry recording its removal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
scripts/check-comment-runs.py early-returned on args.check_evidence, so the
flagged run answered the evidence question and said nothing at all about the
comment-run cap. The flagged form is the expensive one -- it needs a compiled
workspace -- which makes it the one a human runs alone; three agents hit this
independently in one campaign, and in one case a local gate went green while
CI went red on a three-line doc comment (improvise-ijql).
The cap moves into check_cap(), and --check-evidence now runs it first and
returns the worse of the two verdicts. The ticket's cheaper alternative -- keep
the early return and print a line saying the cap did not run -- is declined and
the reason is recorded: a provenance line qualifies an answer the run gave,
where a disclaimer qualifies one it did not, while still exiting 0. Exit 0 is
what CI branches on and what a human reads as clean, so the rule is stated over
the exit code: a tree with a long comment run must not pass any invocation that
can exit 0.
The guard is _dispatch_self_test, which runs the script as a subprocess against
a temporary tree -- the only level at which this is visible, since every unit in
the file was correct and the defect was in main()'s dispatch. Both arms were
demonstrated by injection: restoring the early return fails
"DISPATCH: --check-evidence: exited 0 over a four-line comment run", and forcing
the cap to always report a violation fails the clean arm for both invocations,
which is what stops an always-fail from satisfying the first.
Measured: the cap adds 0.29s to a 6.5s --check-evidence run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Both blockers from the adversarial review of #83, both in the mode this PR
rewrote, both producing a pass while the gate did not do what its prose said.
1. cargo_test_names reduced every listed path to its last segment. Six leaf
names collide in this workspace and two are cited in evidence, so a heading was
satisfiable by a test in another module answering another question -- renaming
ui::project::tests::a_load_that_fails_is_not_a_rebase, the one its entry
describes, left the checker at exit 0 with the ui::host namesake answering for
it. Paths are kept whole; a bare leaf resolves only when unique, and an
ambiguous one is REFUSED with both candidates named. _RUST_NAME accepts `::`, so
a qualified heading is a claim rather than prose, and a citation is matched on
the leaf so qualifying a heading cannot orphan its own test. The two colliding
entries are qualified. Neither half suffices alone: rename one of a colliding
pair and the leaf becomes unique again, which is why qualification is the fix
and the refusal is what forces it.
2. --rust-tests called cargo_test_names with one argument, so the --ignored
subtraction never ran, and it printed a byte-identical success line. It now
requires --rust-tests-ignored (one alone exits 2) and the line names its source.
Also, from the same review: a missing context/test-evidence/ exited 0 while 170
citations pointed at nothing (early return, now gone); the doc-site "runs last"
check accepted a trailing `echo` that merely NAMED the generator, so it asks for
an invocation; bundle_freedom gains an arm for a transitive `import "../app.js"`,
which reaches the bundle while naming nothing forbidden; and two prose overclaims
are corrected -- the prose-heading count is a convenience rather than a guard
(git diff is the mechanism), and the ratio rule says "above the run MEDIAN",
which is defensible where "above the others" is false against second place.
scripts/check-ci-graph.py makes claim 6's rule a gate: pages-deploy must `needs`
every other job in ci.yml. The rule was stated in the last commit and a
zz-new-check: job added during review was still ungated with nothing noticing.
README now states the rule and lists no jobs, the second copy being what went
stale twice.
Every fault re-injected and observed; figures in context/test-evidence/tooling.md
rather than only here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
1. A self-test case that was a duplicate wearing different clothes. "a heading
cargo's listing does not carry is an orphan however the source reads" passed on
an empty listing -- which is the first case in the table, already. Once defined
comes from cargo, the Rust beside a fixture is not read at all, so four cases
for the four source shapes would be four copies of one assertion dressed as
coverage. Deleted; the evidence for those shapes is the real-tree injection, and
context/code/tooling.md now says which half a fixture can carry and which it
cannot. This is exactly the shape improvise-84wp's prose, applied two commits
ago, tells a reviewer to look for.
2. assemble-stamp.sh grepped its patterns as REGEXES, inherited from the inline
version it was extracted from. The `.` in improvise_wasm_client.js?v=abc matches
any character, so a stamp corrupted in that position read as present -- a pass in
the one direction a guard must not fail in. grep -F, plus a case proved by
injection: dropping the -F flips it.
3. A missing cargo raised a traceback out of --check-evidence. It now names the
flag that takes a saved listing instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The manifest only ever grows, so a doc added without a line in it published
nothing and said nothing: build green, link check green, doc simply absent.
The generator now walks context/**/*.md plus docs/design-notes.md, subtracts the
manifest, and fails unless the remainder is exactly the OMITTED map -- checked
in BOTH directions, because an entry naming a file nobody can find asserts a
reason for nothing.
Filed as latent; it was live. context/code/kb-tool.md had been unpublished,
unargued and unnoticed since it was added, while doc-site.md's own table says
context/code/*.md is published. It is published now, with context/code/tooling.md
and context/test-evidence/tooling.md: 23 pages to 26.
Second gate, same file: the generator reads web/assemble.sh and refuses unless
its own invocation is that script's last command. Its outward-link check resolves
../index.html and ../app/ against the filesystem, so a line appended below it
would make that check pass against a half-built tree.
Both run at module scope, before any page is generated, so a manifest error is
answered before the 48 outward-link failures a bare run reports -- a missing doc
is actionable wherever you ran the generator, and the two are unrelated
diagnoses.
Three faults injected and observed; evidence in context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The stamp gate is scripts/assemble-stamp.sh, taking FILE PATTERN pairs, and
scripts/assemble-stamp-test.sh is its check_the_checker. Deleting the final
`test -z "$missing"` makes four of its eight cases fail; before the extraction
that same deletion was invisible, both misses printing while assemble.sh exited
0. It is a separate script because exercising it inside assemble.sh would need a
fabricated web/pkg, which is a second answer to "what does the site assemble
from" (improvise-vvl3). Wired into the `build` job, first, because it needs
nothing and guards a step in that job.
web/tests/bundle_freedom.test.js refuses a flat web/tests/ file that names the
bundle directory -- the constraint keeping the `js` job at seconds rather than
~4m30s, stated in prose and checked by nothing. It scans its own filename too,
so it cannot become the place a violation hides, and its companion asserts
web/tests/bundle/ still holds a test that DOES use the bundle: a scoping rule is
uninteresting if nothing falls outside it.
Both faults were injected and watched to fail; evidence in
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Edward Langley