improvise-52eq: the AGENTS.md recipe searched `bd show` output for "not
found" and printed "all resolve" when there was none, so a bd that was
missing, failing or never run passed the audit. scripts/bd-audit-ids
keeps the file scan and exclusions, then `comm -23`s the cited ids
against `bd list --all --limit 0 --json`. A dead id is exit 1 and is
named. A bd that does not answer is exit 3, never a pass.
scripts/bd-audit-ids-test.sh has eight cases on a throwaway repo with a
fake bd. The old recipe as a rival fails 7 of the 8. Disabling the
comparison fails the dead-id case, and an empty exclusion list fails the
pass case. CI runs it with the other named suites. The "One environment
note" now reports which forms the guard refused and allowed, as observed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-7epn: two cases on the one-tick-per-read clock. One resolves on
poll 3, and one times out after polling up to the deadline. Replacing the
post-poll deadline check with `break` now fails exactly those two
(23 passed, 2 failed), where before it left the suite 23/23.
improvise-lnsh: when `cargo metadata --locked` refuses the opfs-contract
lock, build.sh now says not to remove --locked and names
`cargo update -w --manifest-path harness/opfs-contract/Cargo.toml`.
Checked by hand against a lock with pest pinned to 2.9.1. Evidence is in
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-qnpi: `pr-reviews --wait` checked its deadline before the first
re-poll, so a second boundary between two reads of the clock skipped the
only poll (exit 1 with polls=1, reproduced under CPU load). The deadline
is now checked after each poll. A new case drives it with a PATH shim
whose `date` ticks one second per read, so it fails deterministically on
the old loop rather than under load.
improvise-mkbp: harness/opfs-contract/Cargo.lock pinned pest 2.9.1 while
its path deps require 2.9.2, so every build rewrote it. The resolved lock
is committed and build.sh starts with `cargo metadata --locked`; the
wasm-pack `-- --locked` route was tried and does not work (wasm-pack
re-resolves first). opfs.spec.js now names the fix when harness/serve/opfs
is missing, which is what running web/assemble.sh alone leaves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The required Browser harness check comes from a job that needs: build, so for
a moment per push every registered check is green and GitHub says
MERGEABLE/BLOCKED. #186 at 128a2a0 refused that and exited 1 on healthy PRs.
BLOCKED with no failed check is now waited on (exit 3 if it never clears).
Adds the gap sequence from PR #189's real rollup, and a head moving during
the UNKNOWN wait. Corrects tooling.md's claim that nothing else produces
BLOCKED, the evidence entry's rival-2 claims, and the fixture wording.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
GitHub reports UNKNOWN for a while after the base moves, and PRs #172 and
#179 both exited 0 there. UNKNOWN is now waited on (exit 3 on timeout), a
failed check or CONFLICTING ends the wait, and the pass condition is
MERGEABLE with CLEAN, UNSTABLE or HAS_HOOKS. BLOCKED, DIRTY, BEHIND, DRAFT
and any unshipped state fail closed. The suite enumerates the whole table.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The kb skill (both copies) now describes the HEAD comparison, the ledger,
exit code 3 and the recovery. kb-tool.md records the ih3k/yea8 arguments and
corrects the residue paragraph: .gitignore hides the leftovers, and there are
six, not one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The fixture loop covers every package the fixture has; this covers every
package the tree has. The review's skip-the-root rival now fails the real
repo's own gate (exit 2 naming Cargo.toml), not only the test suite.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
scripts/kb add wrote onto a working-tree kb.pl that had lost 276 of its 801
clauses and reported success, because kb_consistent checks consistency, not
completeness (improvise-usus). Every write now runs kb-merge.pl `complete`
on the candidate before kb_consistent. It refuses unless every clause HEAD
has is still present by key, or its removal was recorded by `kb rm` in a
per-worktree ledger in the git dir, which prunes itself on every write.
The candidate is now built beside kb.pl, so publishing it is a rename(2)
rather than a possible cross-device copy that a kill could truncate.
Exit codes: 0 done, 1 refused, 2 usage, 3 HEAD's kb.pl unreadable. kb-test.sh
now builds its fixtures from HEAD and adds cases 18-22. Argument in
context/code/kb-tool.md, evidence and rivals in context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
ChatGPT Codex on PR #159: both formatter invocations end `|| rc=1`, so any
failure after the rustfmt probe was reported as "misformatted" although no
verdict was produced. Correct, and the narrower form of the bug this gate
exists for.
Measured: an unparseable source exits 1, identical to a diff; an unknown
rustfmt.toml option is only a warning (exit 0).
Narrowed rather than re-classified, because telling a diff from a parse error
means parsing rustfmt's prose, which is the check-adjacent-to-its-subject this
repo keeps getting wrong. The line the split needs is "did the gate run":
exit 1 names a file and blames the tree; exit 2 names none and blames the
invocation, and every exit-2 cause is checked above the loop.
Two more rivals injected: deleting the `cargo fmt --version` probe, and
turning `|| rc=1` into `|| return 2`. The first exposed a vacuous test -- a
cargo stub that failed for every argument also breaks `cargo metadata`, so the
run exited 2 on the metadata arm and the probe could be deleted with the suite
green. The stub now delegates every subcommand but `fmt`, which is the shape a
missing rustfmt component actually has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-b4ks. From the repo root `cargo fmt -p opfs-contract -- --check`
exits 1 with "package `opfs-contract` is not a member of the workspace" and
never runs --check, so a caller reading $? cannot tell "misformatted" from
"nothing was checked". Reproduced at 7cef9f8.
scripts/fmt-workspaces.sh discovers every [workspace] root and checks each
with `-p <member>` scoped by that root's own --manifest-path. Exit 2 is a
usage error or an unrunnable cargo; exit 1 is a formatting verdict.
Both obvious spellings were measured and both fail in the direction that
looks like success: `--all` crosses the boundary, because it formats local
path dependencies and harness/opfs-contract depends on improvise-io by path
(an injected fault in src/draw.rs was reported by the harness root); and the
bare --manifest-path form is blind to workspace members, because the repo
root is a package as well as a workspace (an injected fault in
crates/improvise-core exited 0).
scripts/fmt-workspaces-test.sh is the check_the_checker. Four rivals were
injected and each failed it. Its fixture had to be corrected twice -- a
missing path dependency and a virtual outer manifest -- and both vacuities
were found only by injecting the rival, never by reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`scripts/pr-reviews-test.sh` passed 19/19 locally and 2/19 in CI on the same
commit, because a multi-assignment `local` is not left-to-right on bash 5
while macOS's bash 3.2 tolerates it. The obvious half of the fix is pinning
the shell. The half that decides the design is not.
.envrc DOES already put a nix bash 5.3.15 on PATH, which is CI's semantics --
and it is not enough. `pkgs.mkShell` exports `name`, `out`, `outputs`,
`system`, `shell`, `builder`, `stdenv`, `phases` and about twenty more
generically-named variables that CI does not have, so a script reading one by
accident under `set -u` is SILENT in the dev shell and fatal in CI. That is
exactly what happened: the broken clause read `$name`, and in the dev shell
`$name` was `nix-shell-env`. Measured at 051f541^ on one machine:
macOS /bin/bash 3.2 19 passed, 0 failed
direnv exec . ./scripts/pr-reviews-test.sh 19 passed, 0 failed
the same, with `name` unset 2 passed, 17 failed
Two reasons for green, one of them the correct interpreter. A dev shell is a
third environment, not a reproduction of CI.
A nix shebang was measured and declined, on availability rather than speed:
warm, `nix shell nixpkgs#bash --command true` is 61.8ms against an ambient
5.1ms, and `nix-shell -p bash --run true` -- the manpage form -- is 675.2ms,
132x, inside git's merge machinery for beads-merge-driver.sh. But only the
`kb` job installs nix; build, test, wasm, js, harness-lint and browser have
none, so a nix shebang needs an installer step on three more jobs to reach a
bash those runners already ship. zsh is not a shebang change and the runner
does not have it either. Both declined in context/code/tooling.md section 5.
What lands instead pins interpreter and environment together, in one place:
scripts/checkers.sh is the one entry point for every check_the_checker, used
verbatim by CI and by the local gate. It pins a bash >= 5 (ambient first, nix
only when the ambient one is older, exit 3 when neither), scrubs the
environment to an allowlist, and refuses any shell shebang in scripts/ that
is not `#!/usr/bin/env bash`. The pinned bash goes on the front of PATH so
inner invocations get it too -- which is where the fault actually lived.
The second undetected split is closed with it: the four `#!/bin/sh` and one
`#!/usr/bin/env sh` scripts are now bash, and the six places that ran a
script through an explicit `sh` -- overriding the callee's own shebang, which
is the same disease -- now invoke it directly.
Verification, injected against the real script: restoring pr-reviews'
multi-assignment `local` leaves `sh scripts/pr-reviews-test.sh` at 22/22 and
`direnv exec . ./scripts/pr-reviews-test.sh` at 22/22, while
`./scripts/checkers.sh pr-reviews` reports 2 passed, 20 failed, exit 1 --
matching the 2/19 CI actually reported. scripts/checkers-test.sh is twelve
cases, each injecting rather than asserting; the pair that carries it runs
one fixture both ways, so the `set -u` case is a test of the runner and not
of `set -u`.
The old local gate was `for t in scripts/*-test.sh; do sh "$t"; done`, which
overrides every shebang with /bin/sh -- so it ran every suite under bash 3.2
in sh-mode regardless of what the file said. It is replaced by
`./scripts/checkers.sh`.
Workspace suite unchanged at 1689 passing over 38 binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Both found by ChatGPT Codex on PR #133, both invisible on macOS, both
arriving as exit 2 with no output -- jq's answer to an invalid --argjson,
so a shell fault at the top of the script wears jq's exit code.
`local n="$1" name="$2" f="$DIR/$n.$name.json"` creates every name before
running the assignments on bash 5, so $n is unbound in the third clause
and set -u kills the shell. The dead command substitution then hands
--argjson an empty string. Works on macOS bash 3.2: the suite passed
19/19 locally and 2/19 in CI on the same commit. Each local now gets its
own statement.
`gh api --paginate` emits one JSON array PER PAGE, so a comment thread
past the page size is several concatenated documents that --argjson
rejects the same way. Both fetch paths now flatten through one
pages_to_array (`jq -s 'add // []'`), which is a no-op on a single page.
The fixture path shares that helper, which is the only reason a fixture
can test it: a fixture holding a single array would exercise a shape the
live fetch never produces. Three cases added, and removing the helper
from the fixture path fails exactly those three, the wait case with the
live exit-2 signature. Suite is 22 cases.
No PR in this repo has yet had enough inline comments to reach a second
page, so nothing but the test will see that one before a user does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
It printed review BODIES only and was blind to inline review comments --
the surface a human reviewer is most likely to use. On PR #122 the user
reviewed with the body "Generally good, added comments inline." and five
substantive rulings underneath it; pr-reviews printed the one-liner, and
an agent reported the PR as awaiting review while the answers sat on it.
pulls/{n}/comments and issues/{n}/comments join pulls/{n}/reviews, all
three interleaved in time order, each inline comment carrying its path
and line (falling back to original_line, which is what an outdated
comment has left).
The exit contract is unchanged: 0 means something exists against the
current head, 1 means the wait elapsed, 4 is usage. An inline comment at
the head now satisfies --wait -- it carries a commit_id and is exactly
the case that failed. An issue-level comment does not: it names no
commit and so makes no claim about which tree was read.
The usage's exit-code warning is widened from "do not pipe into
head/tail" to the real rule: nothing may follow the script in the same
`;` list either, because a trailing `echo "exit=$?"` makes the status
the echo's.
scripts/pr-reviews-test.sh is the check_the_checker, fixture-driven and
needing no network, wired into CI's kb job beside pr-watch-test.sh and
release-preflight-test.sh. Five faults were injected and watched to
fail; context/test-evidence/tooling.md records which case caught each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The `build` job failed, correctly: `scripts/build-doc-site.mjs` requires every
`context/**/*.md` to be either in the SECTIONS manifest or a named omission
(improvise-res9's cure), and the two files this branch added were neither.
Both are now published, beside the TUI in each section.
While there: `context/code/doc-site.md` asserted 'the seven
`context/code/*.md` documents' when there were already twelve, and '501
entries' when the checker reports 801. Both were stale before this branch and
this branch makes them staler, so they are de-numbered rather than re-asserted
— a count in prose that nothing checks is a fact with no owner, and the shape
that stops drifting is not having one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Three findings, all correct, all applied.
1. A contextual selection face cannot be resolved by `resolve(Face)`. Clause 3
fixed that signature while the vocabulary described one `selection` face
"parameterised by the surface's accent" — the same Face cannot yield the
grid's Cyan and the formula panel's Yellow. Resolved the way clause 2
already works: `selection` is a BASE face and each surface has a derived
one. A context argument was rejected explicitly, because it is a second
input three front ends would each have to supply correctly, which is the
three-derivations hazard one level down.
2. `Indexed` could not be "a detail". The ADR defined Colour as exactly
Default | Named | Rgb, promised the two existing Indexed greys would still
render exactly, and said the shipped theme used only the first two. Folding
an index into Rgb changes its meaning AND breaks that rule. `Indexed(u8)` is
now a fourth arm, the browser resolves 16-255 through the xterm cube and
ramp, and the shipped-theme rule is restated over the line that actually
matters: palette-relative versus absolute. The shipped theme contains no
`Rgb`. The rejected-alternatives entry now also says that dropping `Rgb`
entirely is a coherent version of this ADR, since that would make the
regression unrepresentable rather than defaulted-against.
3. "Build all bundles" was failing: the ADR was neither published nor a named
omission. Added to SECTIONS in scripts/build-doc-site.mjs. The checked-in
site/docs output is deliberately NOT regenerated here — assemble.sh builds
into its own output directory, and site/ is a periodic snapshot rather than
a per-commit artifact.
Also corrects the hex count: nine stray values in style.css, six light and
three dark, not six.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
#120 restyled the page and rewrote #backend-notice as a <details>; <header>'s
markup is untouched, so #build-version lands where it was written. What the
merge actually has to answer is the CSS, and the stamp gate after it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
v0.1.0-rc3 no longer exists anywhere -- not local, not gh, not origin, not
gitea. Verified here independently before changing anything; the releases are
untouched (rc1, rc2, and there never was an rc3 release).
THREE MEASUREMENTS OF ONE FACT IN ONE SESSION, and only the first was careless:
1. present on all three remotes and locally (from aa4e901's commit message)
2. on gh alone (git ls-remote, this branch)
3. nowhere at all (git ls-remote, now)
Reading 1 was asserted rather than asked, and correcting it was right. Readings
2 and 3 were each taken correctly, by two different people, and they disagree --
because the tag was deleted in between. So the lesson this section reached for
was too weak. It is not "ask rather than assert"; it is that A REMOTE-SHAPED
FACT HAS A SHELF LIFE EVEN AFTER YOU ASK. A verified answer decays, and prose
cannot distinguish a stale line from a live one, so a reader following
RELEASING.md at 2am has no way to tell which they are holding.
WHAT CHANGES. The present-tense inventory goes, everywhere I put it:
RELEASING.md's "It is on gh only; origin and gitea carry rc1 and rc2", the same
clause's "the mirrors did not agree", tooling.md's "rc3 is on gh alone", and
"pushed to gh" in both tooling.md and the script header -- that last one named a
specific remote too, and we do not actually know which remotes the push reached.
rc3 survives as a plainly historical example: it was pushed while every crate
still said 0.1.0-rc2, dist refused, no release was built. Nothing about where it
is now.
WHAT STAYS. The instruction to ask -- `git tag -l`, `git ls-remote --tags
<remote>` -- because that is the part that ages well: it tells the reader to
look rather than to believe. And the record that aa4e901's claim was wrong when
checked, which is durable precisely because its subject is a fixed text. Note
which half of the earlier correction survived and which did not: "the commit
message was wrong" is permanent; "the tag is on gh alone" was true and is now
false. The keeper is the one whose subject cannot change.
Both files now say explicitly that they record no inventory, so the next editor
knows the absence is deliberate rather than an omission to helpfully fill in.
The principle never decayed and is untouched: a wrong tag is left dead and the
next version cut; a published tag is not moved.
No version bumped, no tag cut, no tag deleted by me.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-oujb. A tagged release carried four native tarballs and nothing for
the web; the site shipped only by ci.yml's pages-deploy, on a push to main.
Pages keeps tracking main (user, 2026-09-22: "I like Pages showing the latest
main for now"), so this is a CAPTURE, not a second deployment: the tarball is a
self-hostable copy of the site as of the tag and a rollback target, not what a
visitor to the published site is running. pages-deploy is untouched.
Mechanism is dist's own [[dist.extra-artifacts]], so it survives `dist init`:
`dist plan` under the pinned 0.30.4 lists improvise-web.tar.gz beside the four
platform tarballs and `dist generate` leaves release.yml byte-identical. The
build command is scripts/build-web-release.sh, which calls web/build.sh and
web/assemble.sh -- the same two scripts ci.yml and a developer run, so there is
no second way to build the bundle. Its toolchain pins come from
scripts/web-toolchain-versions.env, which ci.yml's build job now reads too.
`github-build-setup` was measured and does NOT help: it injects into
build-local-artifacts only, which is the wrong job for a global artifact.
The page now names the build it was assembled from in #build-version, stamped
by web/assemble.sh with the same token as the bundles and verified by the same
scripts/assemble-stamp.sh gate. An unassembled tree honestly reads "unstamped".
Verified by running `dist build --artifacts=global` locally: the script ran,
the site assembled, and dist placed improvise-web.tar.gz in target/distrib.
Not verified: a real tag push, which would require cutting a tag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
REVIEW OF THE PREVIOUS COMMIT, NOT NEW MACHINERY. The diagnosis it states was
re-verified in both directions before anything here was written: `dist plan
--tag=v0.1.0-rc2` gets past the tag/version check (stopping at the pinned
cargo-dist-version, as expected) and `--tag=v0.1.0-rc3` reproduces the CI error
verbatim, naming `--tag=v0.1.0-rc2` as the fix in its own help text. The
workflow was never broken. Both claimed fault injections also reproduce exactly:
the comparison replaced by `if false;`, and `expected` derived from the tag, each
produce four failures and exit 1 -- exit code read unpiped, since a pager's
status would mask it.
WHAT THE TEST COULD NOT SEE. Disabling the dirty-tree guard outright left all
six checks green and the script exiting 0. That guard is not decoration: it is
what stops a tag naming a commit that was never tested, and its failure is
irreversible in the same way the version mismatch is.
The cause is the previous repair. Moving the negative cases to a clean throwaway
checkout is what made the tag comparison reachable at all -- and it thereby
removed the only circumstance in which the test could observe the tree guard.
The guard whose PRECEDENCE caused the original vacuity became the one thing the
suite could not see. The vacuity and its repair are not two events; the repair
relocated the blind spot rather than closing it.
So the new check is aimed the other way: a tag that MATCHES the version, run
against a dirtied checkout, so the comparison passes and a refusal can only be
the tree. Verified discriminating -- with the guard disabled it fails with exit
1, and the two faults are independent in both directions (comparison disabled
leaves the tree check passing, guard disabled leaves the three tag checks
passing), so the seventh check is not a restatement of the other six.
Also recorded, rather than guarded: the `dist plan` second opinion can be
disabled with the suite green, because every input the test feeds it has already
been refused by the textual check. That follows from the arm being optional by
design -- pinning it would mean installing a dist matching cargo-dist-version,
the very mismatch the script tolerates on purpose. Stated so the silence is not
mistaken for coverage.
Two corrections of fact. The CI step lives in the `kb` job beside
pr-watch-test.sh, where this repo keeps its check_the_checkers -- not in `test`;
tooling.md now says so. And the awk version parser was probed rather than
hardened: `version.workspace = true` yields empty, which the root path refuses
loudly and the per-crate loop skips; a non-first [package] block and build
metadata both parse correctly; a trailing comment on the version line mis-parses
but fails CLOSED and prints the mis-parsed value in the refusal. No manifest in
this workspace has one, so nothing was changed for it.
No version bumped, no tag cut.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
THE WORKFLOW WAS NEVER BROKEN. v0.1.0-rc3 was pushed to all three remotes
while every crate still said 0.1.0-rc2; dist matches a tag against the root
package version, refused with "This workspace doesn't have anything for dist
to Release!", and the four jobs behind `plan` skipped. Proved in both
directions locally: `dist plan --tag=v0.1.0-rc2` gets past the check and
`--tag=v0.1.0-rc3` reproduces the CI error exactly.
So this changes no trigger and no job graph. What it adds is a check that runs
BEFORE the irreversible step. dist named the right answer in its own error
text; the problem is when that answer arrives -- a tag is public and immutable
by the time CI reads it, so the repair is a new version rather than an edit.
scripts/release-preflight.sh refuses a dirty tree, compares the intended tag
against the root package version, runs `dist plan` as a second opinion when
dist is on PATH (and says so when it is not, since an absent dist must not
read as a pass), and prints the tag-and-push commands only once it passes.
THE TEST'S FIRST VERSION WAS VACUOUS AND THE INJECTION IS WHAT SHOWED IT.
release-preflight-test.sh originally asserted only "did it refuse". Every
negative case still passed with the tag comparison replaced by `if false;`,
because the script refuses a DIRTY WORKING TREE first and this repo's tree is
almost always dirty -- bd auto-exports .beads/issues.jsonl on nearly every
command. Four of five checks were answering "is the tree dirty" while claiming
to answer "does it catch a bad tag".
It now runs the negative cases against a clean throwaway checkout AND asserts
the refusal names the version mismatch, so a refusal for the wrong reason
fails. Two rival faults were injected against the repaired version -- the
comparison disabled, and `expected` derived from the tag itself -- and each
produces four failures and exit 1.
That is design-principles section 7's "a check that looks like the answer,
answering a neighbouring question", found inside a checker written to prevent
that class, within an hour of writing it. Recorded in context/code/tooling.md.
No version is bumped and no tag is cut here, per the user's instruction to
make the workflow work without releasing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
issues.jsonl is a passive export whose rows are ordered by insertion, so two
sides that file issues collide in the same region even when the issues are
disjoint. Git line-merges it and produces conflict markers INSIDE the JSONL,
which breaks bd's auto-export silently -- observed today, where a stash-pop
left `<<<<<<< Updated upstream` in the file and auto-export was skipping with
"invalid character '<'" until the file was regenerated.
The driver does no merge logic of its own, because bd already has the
semantics: `bd import` skips any row whose updated_at is not strictly newer
than the local one, creates ids it has not seen, and keeps the local row on a
tie. So importing BOTH sides cannot move the database backwards, and the file
is then regenerated from it. The database is the truth; the file is derived.
Deliberately not reimplemented here: field-level reconciliation. `bd conflicts
show/resolve` already does that against the Dolt working set, which is the real
sync path -- a second copy against the text export would have to track bd's
schema by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The `build` job fails on main and on every branch merged from it:
doc-site: context/code/pr-review.md is neither published nor a named omission
NOT a merge loss -- scripts/build-doc-site.mjs in this branch is
byte-identical to main's, and neither lists the file. PR #96 added
context/code/pr-review.md without a manifest line, which is exactly the
gate improvise-res9 added: a doc added without a line publishes nothing
and says nothing, so the generator refuses.
Omitted rather than published, and the line is cloud-agents.md's rather
than tooling.md's. tooling.md explains checkers a reader of the code will
meet; pr-review.md is about THIS REPOSITORY'S PROCESS -- where a Codex
review appears, how to tell it from our own agents' comments, why absence
is not approval. Nobody visiting a website is about to read a PR thread.
Both homes updated, because an omission is restated beside the manifest
AND in doc-site.md section 2, and the prose there counted them ("Neither
is referenced...") so it went stale at three.
This strays into scripts/, which is Wave N's ground. Flagged in the
hand-back; it is a one-line data entry either way if the orchestrator
would rather publish it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Fourth occurrence in one campaign, and this one was in a tool I wrote TO BE
the reliable answer. scripts/pr-reviews --wait correctly exited 1 on a
timeout, and my invocation ended in '| tail -20; echo', so the harness
reported 0 — a review window that had EXPIRED read as a review that had
ARRIVED, which is the dangerous direction.
The earlier three: pr-watch reading tr's status through a pipe; a probe
reporting EXIT=0 from a checker that had exited 1; and PIPESTATUS in zsh,
where the array is lowercase pipestatus — the two shells differ, which is its
own trap.
The script's usage now says not to pipe it.
Closes improvise-711n (,.2f yielded zero decimals — the spelling our own docs
show) and improvise-x1ns (the preflight undercount ChatGPT Codex found on a
PR I had already merged unread).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Prints a PR's reviews and says whether each targets the CURRENT head or an
earlier one, because that changes what you do: a finding against an old commit
may be fixed or still live, and both have happened this week.
--wait blocks until a review lands against the head. Observed latency is ten to
fifteen minutes after a push, so a merge four minutes later races the review
that was about to arrive. When it times out that is a decision to merge
unreviewed, not a clean bill.
The detection rule is placement, not authorship: Codex posts under the owner's
account, the same one our agents use, and ENDS with its attribution where ours
OPEN with theirs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
All three found by the adversarial review of 019cbc1, each by injection.
1. every_model_command() is NOT compiler-maintained, and both this PR's prose
and reduce.rs's own doc comment said it was. variant_name's exhaustive match
forces an arm in variant_name, not an entry in the value list -- deleting
ModelCommand::TogglePruneEmpty from it failed the new completeness check and
NOTHING ELSE in the workspace. So the diff is two independent hand-written
lists: strictly better than one list against its own length, and not the
derived enumeration improvise-5ubs got for the two enums. Doc comment
corrected; the prose in context/code/command.md and the evidence entry now
say what it is. Closing it properly wants improvise_protocol::variants made
pub so declared::<ModelCommand>() can bound the list -- another wave's crate,
reported rather than taken.
The assertion message was also ONE-DIRECTIONAL ("a name only on the right
..."), which is the shape that injection produces and which invites the wrong
repair: delete the case on the left, degrading both sides together. It now
names both directions and says which is which.
2. SELF_TEST_DUPLICATES case 3 was named for the ambiguity guard and never
reached it: both its headings were module-qualified, so canonical() returned
on the exact-match arm before touching by_leaf. Dropping the guard
(`if matches` for `if len(matches) == 1`) left --self-test at exit 0. The
fixture is now one BARE heading over an ambiguous leaf plus BOTH qualified
spellings, so the injection collides with one of them whichever way
next(iter(set)) falls -- deterministic across five runs, verified.
3. The evidence entry said five pre-existing records-mode tests fail under the
settle-before-rebuild injection; measured, it is six. And "three hundred
lines above" is four hundred.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
--check-evidence verified that a heading names a test that RUNS, and a
duplicate names one just as well, so both copies pass. PR #92 shipped two
entries under one heading; the older described a fixture the test no longer
had.
Refuse rather than warn, on the improvise-rm4s precedent, naming the file and
every line in file order. Keyed by the RESOLVED test, so `### f` beside
`### m::tests::f` collides too -- the case rm4s's qualification repair creates.
Six SELF_TEST_DUPLICATES cases, including the three clean arms: two different
tests sharing a leaf are ambiguous and not duplicate, prose repeats freely, and
two headings naming two tests are fine.
Resolves the two tui.md duplicates: the older column-header copy called
THE_MEMBERSHIP_CORRECT_SPANS a live tripwire three hundred lines above the
entry recording its removal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two findings from running /code-review on this branch.
The dispatch self-test's dirty arm asserted the run failed and named `f.rs`,
which is adjacent to the question -- a failure for any other reason naming that
file would have satisfied it. It now requires the cap's own message as well,
so the assertion names the question it means.
And .claude/skills/code-review/SKILL.md still told reviewers the flag "selects
rather than adds, so --check-evidence never runs the comment cap", which this
branch makes false. Grepped for every copy of the claim; that was the only one
outside context/. A doc that describes the script it gates is code, and leaving
it is the gotcha-48 failure exactly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
scripts/check-comment-runs.py early-returned on args.check_evidence, so the
flagged run answered the evidence question and said nothing at all about the
comment-run cap. The flagged form is the expensive one -- it needs a compiled
workspace -- which makes it the one a human runs alone; three agents hit this
independently in one campaign, and in one case a local gate went green while
CI went red on a three-line doc comment (improvise-ijql).
The cap moves into check_cap(), and --check-evidence now runs it first and
returns the worse of the two verdicts. The ticket's cheaper alternative -- keep
the early return and print a line saying the cap did not run -- is declined and
the reason is recorded: a provenance line qualifies an answer the run gave,
where a disclaimer qualifies one it did not, while still exiting 0. Exit 0 is
what CI branches on and what a human reads as clean, so the rule is stated over
the exit code: a tree with a long comment run must not pass any invocation that
can exit 0.
The guard is _dispatch_self_test, which runs the script as a subprocess against
a temporary tree -- the only level at which this is visible, since every unit in
the file was correct and the defect was in main()'s dispatch. Both arms were
demonstrated by injection: restoring the early return fails
"DISPATCH: --check-evidence: exited 0 over a four-line comment run", and forcing
the cap to always report a violation fails the clean arm for both invocations,
which is what stops an always-fail from satisfying the first.
Measured: the cap adds 0.29s to a 6.5s --check-evidence run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Both blockers from the adversarial review of #83, both in the mode this PR
rewrote, both producing a pass while the gate did not do what its prose said.
1. cargo_test_names reduced every listed path to its last segment. Six leaf
names collide in this workspace and two are cited in evidence, so a heading was
satisfiable by a test in another module answering another question -- renaming
ui::project::tests::a_load_that_fails_is_not_a_rebase, the one its entry
describes, left the checker at exit 0 with the ui::host namesake answering for
it. Paths are kept whole; a bare leaf resolves only when unique, and an
ambiguous one is REFUSED with both candidates named. _RUST_NAME accepts `::`, so
a qualified heading is a claim rather than prose, and a citation is matched on
the leaf so qualifying a heading cannot orphan its own test. The two colliding
entries are qualified. Neither half suffices alone: rename one of a colliding
pair and the leaf becomes unique again, which is why qualification is the fix
and the refusal is what forces it.
2. --rust-tests called cargo_test_names with one argument, so the --ignored
subtraction never ran, and it printed a byte-identical success line. It now
requires --rust-tests-ignored (one alone exits 2) and the line names its source.
Also, from the same review: a missing context/test-evidence/ exited 0 while 170
citations pointed at nothing (early return, now gone); the doc-site "runs last"
check accepted a trailing `echo` that merely NAMED the generator, so it asks for
an invocation; bundle_freedom gains an arm for a transitive `import "../app.js"`,
which reaches the bundle while naming nothing forbidden; and two prose overclaims
are corrected -- the prose-heading count is a convenience rather than a guard
(git diff is the mechanism), and the ratio rule says "above the run MEDIAN",
which is defensible where "above the others" is false against second place.
scripts/check-ci-graph.py makes claim 6's rule a gate: pages-deploy must `needs`
every other job in ci.yml. The rule was stated in the last commit and a
zz-new-check: job added during review was still ungated with nothing noticing.
README now states the rule and lists no jobs, the second copy being what went
stale twice.
Every fault re-injected and observed; figures in context/test-evidence/tooling.md
rather than only here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
1. A self-test case that was a duplicate wearing different clothes. "a heading
cargo's listing does not carry is an orphan however the source reads" passed on
an empty listing -- which is the first case in the table, already. Once defined
comes from cargo, the Rust beside a fixture is not read at all, so four cases
for the four source shapes would be four copies of one assertion dressed as
coverage. Deleted; the evidence for those shapes is the real-tree injection, and
context/code/tooling.md now says which half a fixture can carry and which it
cannot. This is exactly the shape improvise-84wp's prose, applied two commits
ago, tells a reviewer to look for.
2. assemble-stamp.sh grepped its patterns as REGEXES, inherited from the inline
version it was extracted from. The `.` in improvise_wasm_client.js?v=abc matches
any character, so a stamp corrupted in that position read as present -- a pass in
the one direction a guard must not fail in. grep -F, plus a case proved by
injection: dropping the -F flips it.
3. A missing cargo raised a traceback out of --check-evidence. It now names the
flag that takes a saved listing instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The manifest only ever grows, so a doc added without a line in it published
nothing and said nothing: build green, link check green, doc simply absent.
The generator now walks context/**/*.md plus docs/design-notes.md, subtracts the
manifest, and fails unless the remainder is exactly the OMITTED map -- checked
in BOTH directions, because an entry naming a file nobody can find asserts a
reason for nothing.
Filed as latent; it was live. context/code/kb-tool.md had been unpublished,
unargued and unnoticed since it was added, while doc-site.md's own table says
context/code/*.md is published. It is published now, with context/code/tooling.md
and context/test-evidence/tooling.md: 23 pages to 26.
Second gate, same file: the generator reads web/assemble.sh and refuses unless
its own invocation is that script's last command. Its outward-link check resolves
../index.html and ../app/ against the filesystem, so a line appended below it
would make that check pass against a half-built tree.
Both run at module scope, before any page is generated, so a manifest error is
answered before the 48 outward-link failures a bare run reports -- a missing doc
is actionable wherever you ran the generator, and the two are unrelated
diagnoses.
Three faults injected and observed; evidence in context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The stamp gate is scripts/assemble-stamp.sh, taking FILE PATTERN pairs, and
scripts/assemble-stamp-test.sh is its check_the_checker. Deleting the final
`test -z "$missing"` makes four of its eight cases fail; before the extraction
that same deletion was invisible, both misses printing while assemble.sh exited
0. It is a separate script because exercising it inside assemble.sh would need a
fabricated web/pkg, which is a second answer to "what does the site assemble
from" (improvise-vvl3). Wired into the `build` job, first, because it needs
nothing and guards a step in that job.
web/tests/bundle_freedom.test.js refuses a flat web/tests/ file that names the
bundle directory -- the constraint keeping the `js` job at seconds rather than
~4m30s, stated in prose and checked by nothing. It scans its own filename too,
so it cannot become the place a violation hides, and its companion asserts
web/tests/bundle/ still holds a test that DOES use the bundle: a scoping rule is
uninteresting if nothing falls outside it.
Both faults were injected and watched to fail; evidence in
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
kb-test case 10 said it was where same_key is discriminated. Reverting same_key
to unification leaves case 10 PASSING and fails the wildcard rm and case 17
instead -- a rule keys on its whole clause, so the `:-` term cannot unify with
probe_fact(a) and conflicts never fires there. Re-demonstrated by injection
before rewriting the comment. The site is guarded; the explanation was wrong.
replace's refusal said it would add the rules "beside the ones already there",
which is false when the predicate has none. The behaviour was always right and
fail-safe; only that clause of the sentence was wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Four shapes let a heading name something that never runs: a test inside a raw
string literal, one behind an unenabled cfg feature, one behind
cfg(target_arch = "wasm32"), and one behind #[ignore]. All four cleared the
scrape; injecting them together made the previous checker print "543 entries,
all name a test that exists", exit 0.
--check-evidence now takes the Rust names from `cargo test --workspace -- --list`
minus its `--ignored` twin. All 539 live headings resolve against a real listing,
so the stricter source costs nothing today. rust_tests is rust_citations: the
scrape survives only for the reverse direction, where a false positive is loud.
Prose headings become a third kind, counted and printed, so retiring a claim by
rewriting it as prose moves a number instead of passing in silence
(improvise-9za9). No gate can read prose for truth; the reviewer is the
mechanism, and the diff is what to read.
context/code/tooling.md is new and is the file four docstrings in
check-comment-runs.py already pointed at. design-principles §7 stopped saying
the check runs in one direction, which improvise-lyyc had already made false.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Five findings from the second review round, and the first is this PR's own
standard applied to its own P1.
**The two fixes improvise-dj1l names had no discriminating test.** Reverting
`has_key` to unification, or dropping `canon_key` from the duplicate gate, left
kb-test at 27 passed. Case 11 was carried by `check_total` refusing with a
DIFFERENT message, and case 10 by the other half of the fix. Both now assert
which refusal fired (`said()`), plus two cases aimed at each half: a rm key with
a variable in a non-final position must be refused as ABSENT rather than
matching everything, and a keyed FACT whose key holds a variable must ADD
without excluding the siblings it would unify with. Each proved by reverting
exactly one fix and watching exactly that case fail.
**Keying a rule on its whole clause made `replace` on an edited rule silently
become `add`** -- both arms live, the stale one still firing, and `check_total`
cannot see it because `NO + NF` is correct arithmetic for an add. Reachable on
the 15 `kb_error/1` arms and the `kb_text/2` rules, so editing the check
improvise-2f97 adds is the first thing to hit it. `check_rule_replace` refuses
and names the route (rm, then add); the negative arm -- replace of a rule
present verbatim -- is tested, or the refusal would just be "replace never
works on rules".
**The `decision(` arm was dead code.** `cited_name` took the first field, and
the repo spells `decision(<ticket>, <name>)`, whose ticket id fails
`name_shaped/1` on the hyphen -- so that marker could never fire for a real
citation. A second `cited_name` clause takes the second field. All three
markers are now exercised.
Minor, from the same round: `_JS_TEST` no longer accepts `test.only`, which
disables every other test in its file, with a case asserting a heading cannot
be satisfied by one; and `node-test-test.sh`'s last case no longer duplicates
an earlier one.
Filed rather than fixed: improvise-w8ih, extracting `web/assemble.sh`'s gate so
its own absence is visible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two findings from the PR #69 review, one class: a lexical `test(` scan is
satisfied by things that execute no assertion -- a commented-out call, the
words in a string, `test.skip`/`test.todo`, and an empty `test.describe`.
scripts/node-test.sh drops the scan and reads node's report instead. `# pass`
is 0 for a skip-only file, a todo-only file and an empty suite; and a file that
registered no test is reported as one passing test WHOSE NAME IS THE FILE PATH,
which is both how a barren file is detected and why `# tests` alone reads as
the file count. That naming convention is an assumption about node, so the
self-test pins it: a node that stops doing it fails the case rather than
letting the guard go quiet. Six cases added, including a barren file hiding
among passing ones.
--check-evidence had the parallel hole: _JS_TEST accepted `.skip`, `.todo` and
`.describe`, so a quoted heading could be satisfied by disabled coverage or by
an empty suite. Now runnable leaves only (`.only` included), with three cases
asserting that a skipped, todo or suite-only name leaves the heading orphaned
while a test nested in a suite still satisfies one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
dj1l (P1, found by review of PR #68 after it merged). kb-merge.pl matched
clause keys by UNIFICATION, which both lost and corrupted clauses.
check_self_consistent compared keys with ==, but findall hands back fresh
variables, so two clauses with variant heads were never == and passed the
duplicate gate; foldl_merge then matched by unification and the second removed
the first. Reproduced: 536 before, 2 in delta, 537 after. Worse, adding
`probe_fact(X) :- doctrine(X,_).` over a kb holding probe_fact(a) and
probe_fact(b) deleted probe_fact(a) AND wrote `probe_fact(a):-doctrine(a,A)` --
the match bound X, so the clause written was not the clause supplied. The
round-trip verify cannot see that: it compares the file against an intended
list already corrupted by the same unification.
same_key/2 (copy_term + numbervars) in has_key, conflicts and absent;
canonicalize before duplicate detection; a RULE keys on the whole clause, since
one head shape with different bodies is ordinary Prolog; and check_total/2
asserts the arithmetic the tool already PRINTED beside the delta size -- "2 in
delta, 537 after" was visible in its own output twice and nothing compared them.
2f97. kb_error(dangling_citation/3): a name cited inside another clause's text
must resolve as a doctrine/2, invariant/2 or decision/3, so retiring a clause
can no longer dangle every clause naming it. The token rule is narrow on
purpose -- a name-shaped token directly inside doctrine(, invariant( or
decision( -- so prose cannot false-positive. kb_error/1 is a gate predicate,
so this was written with KB_ALLOW_GATE=1 deliberately: the refusal exists so a
write cannot disarm its own gate, and an intentional change to the gate is
supposed to be a greppable act rather than an impossible one.
Seven cases added to kb-test.sh, each proved by injection: two variant-headed
rules both land; a non-ground rule deletes nothing and is written as supplied;
a duplicate key is still refused; an injected dropping fold is caught by the
total; a cited clause cannot be retired; and the citation arm fires on a name
that resolves to nothing without firing on one that resolves. scripts/kb gains
a KB_MERGE seam so a broken merge can be injected. One pre-existing case moved
off doctrine(bug_first), which the new arm correctly refuses to retire.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two blind spots with one consequence: an evidence file could promise a guard
that was not there, and a test could point at prose that did not exist.
5ray: headings were collected with `startswith("### ")` and resolved against
Rust fn names only. context/test-evidence/clients.md uses `#### "name"` for
every browser test precisely BECAUSE a `###` naming a Playwright test would
have been reported as an orphan -- so the convention that kept the checker
quiet was the same one that made it blind. Now `###` and `####` alike, decided
by SHAPE: a bare identifier is a Rust test, a quoted string is a JS test
(harness/tests and web/tests), anything else is prose naming nothing.
lyyc: the check was one-directional. A test whose doc comment names
context/test-evidence must now appear there. Deliberately only that narrower
obligation -- most tests have no evidence and should not be forced to. Neither
direction can see a STALE entry; that stays discipline.
Each half found a real instance on its first run. A `#### "? opens help..."`
heading outlived the test 36ade35 deleted, and is demoted to prose with the
coverage filed as improvise-v5pc. `an_entry_that_holds_no_document_is_seeded_over`
cited the file and had no entry; one is written from the test.
The checker's --self-test covered the comment scanner and nothing else, which
is how both survived. It grows ten evidence cases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Edward Langley