`scripts/pr-reviews-test.sh` passed 19/19 locally and 2/19 in CI on the same
commit, because a multi-assignment `local` is not left-to-right on bash 5
while macOS's bash 3.2 tolerates it. The obvious half of the fix is pinning
the shell. The half that decides the design is not.
.envrc DOES already put a nix bash 5.3.15 on PATH, which is CI's semantics --
and it is not enough. `pkgs.mkShell` exports `name`, `out`, `outputs`,
`system`, `shell`, `builder`, `stdenv`, `phases` and about twenty more
generically-named variables that CI does not have, so a script reading one by
accident under `set -u` is SILENT in the dev shell and fatal in CI. That is
exactly what happened: the broken clause read `$name`, and in the dev shell
`$name` was `nix-shell-env`. Measured at 051f541^ on one machine:
macOS /bin/bash 3.2 19 passed, 0 failed
direnv exec . ./scripts/pr-reviews-test.sh 19 passed, 0 failed
the same, with `name` unset 2 passed, 17 failed
Two reasons for green, one of them the correct interpreter. A dev shell is a
third environment, not a reproduction of CI.
A nix shebang was measured and declined, on availability rather than speed:
warm, `nix shell nixpkgs#bash --command true` is 61.8ms against an ambient
5.1ms, and `nix-shell -p bash --run true` -- the manpage form -- is 675.2ms,
132x, inside git's merge machinery for beads-merge-driver.sh. But only the
`kb` job installs nix; build, test, wasm, js, harness-lint and browser have
none, so a nix shebang needs an installer step on three more jobs to reach a
bash those runners already ship. zsh is not a shebang change and the runner
does not have it either. Both declined in context/code/tooling.md section 5.
What lands instead pins interpreter and environment together, in one place:
scripts/checkers.sh is the one entry point for every check_the_checker, used
verbatim by CI and by the local gate. It pins a bash >= 5 (ambient first, nix
only when the ambient one is older, exit 3 when neither), scrubs the
environment to an allowlist, and refuses any shell shebang in scripts/ that
is not `#!/usr/bin/env bash`. The pinned bash goes on the front of PATH so
inner invocations get it too -- which is where the fault actually lived.
The second undetected split is closed with it: the four `#!/bin/sh` and one
`#!/usr/bin/env sh` scripts are now bash, and the six places that ran a
script through an explicit `sh` -- overriding the callee's own shebang, which
is the same disease -- now invoke it directly.
Verification, injected against the real script: restoring pr-reviews'
multi-assignment `local` leaves `sh scripts/pr-reviews-test.sh` at 22/22 and
`direnv exec . ./scripts/pr-reviews-test.sh` at 22/22, while
`./scripts/checkers.sh pr-reviews` reports 2 passed, 20 failed, exit 1 --
matching the 2/19 CI actually reported. scripts/checkers-test.sh is twelve
cases, each injecting rather than asserting; the pair that carries it runs
one fixture both ways, so the `set -u` case is a test of the runner and not
of `set -u`.
The old local gate was `for t in scripts/*-test.sh; do sh "$t"; done`, which
overrides every shebang with /bin/sh -- so it ran every suite under bash 3.2
in sh-mode regardless of what the file said. It is replaced by
`./scripts/checkers.sh`.
Workspace suite unchanged at 1689 passing over 38 binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
1. A self-test case that was a duplicate wearing different clothes. "a heading
cargo's listing does not carry is an orphan however the source reads" passed on
an empty listing -- which is the first case in the table, already. Once defined
comes from cargo, the Rust beside a fixture is not read at all, so four cases
for the four source shapes would be four copies of one assertion dressed as
coverage. Deleted; the evidence for those shapes is the real-tree injection, and
context/code/tooling.md now says which half a fixture can carry and which it
cannot. This is exactly the shape improvise-84wp's prose, applied two commits
ago, tells a reviewer to look for.
2. assemble-stamp.sh grepped its patterns as REGEXES, inherited from the inline
version it was extracted from. The `.` in improvise_wasm_client.js?v=abc matches
any character, so a stamp corrupted in that position read as present -- a pass in
the one direction a guard must not fail in. grep -F, plus a case proved by
injection: dropping the -F flips it.
3. A missing cargo raised a traceback out of --check-evidence. It now names the
flag that takes a saved listing instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The stamp gate is scripts/assemble-stamp.sh, taking FILE PATTERN pairs, and
scripts/assemble-stamp-test.sh is its check_the_checker. Deleting the final
`test -z "$missing"` makes four of its eight cases fail; before the extraction
that same deletion was invisible, both misses printing while assemble.sh exited
0. It is a separate script because exercising it inside assemble.sh would need a
fabricated web/pkg, which is a second answer to "what does the site assemble
from" (improvise-vvl3). Wired into the `build` job, first, because it needs
nothing and guards a step in that job.
web/tests/bundle_freedom.test.js refuses a flat web/tests/ file that names the
bundle directory -- the constraint keeping the `js` job at seconds rather than
~4m30s, stated in prose and checked by nothing. It scans its own filename too,
so it cannot become the place a violation hides, and its companion asserts
web/tests/bundle/ still holds a test that DOES use the bundle: a scoping rule is
uninteresting if nothing falls outside it.
Both faults were injected and watched to fail; evidence in
context/test-evidence/tooling.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Edward Langley