Edward Langley
cb6866b2c6
`scripts/pr-reviews-test.sh` passed 19/19 locally and 2/19 in CI on the same commit, because a multi-assignment `local` is not left-to-right on bash 5 while macOS's bash 3.2 tolerates it. The obvious half of the fix is pinning the shell. The half that decides the design is not. .envrc DOES already put a nix bash 5.3.15 on PATH, which is CI's semantics -- and it is not enough. `pkgs.mkShell` exports `name`, `out`, `outputs`, `system`, `shell`, `builder`, `stdenv`, `phases` and about twenty more generically-named variables that CI does not have, so a script reading one by accident under `set -u` is SILENT in the dev shell and fatal in CI. That is exactly what happened: the broken clause read `$name`, and in the dev shell `$name` was `nix-shell-env`. Measured at 051f541^ on one machine: macOS /bin/bash 3.2 19 passed, 0 failed direnv exec . ./scripts/pr-reviews-test.sh 19 passed, 0 failed the same, with `name` unset 2 passed, 17 failed Two reasons for green, one of them the correct interpreter. A dev shell is a third environment, not a reproduction of CI. A nix shebang was measured and declined, on availability rather than speed: warm, `nix shell nixpkgs#bash --command true` is 61.8ms against an ambient 5.1ms, and `nix-shell -p bash --run true` -- the manpage form -- is 675.2ms, 132x, inside git's merge machinery for beads-merge-driver.sh. But only the `kb` job installs nix; build, test, wasm, js, harness-lint and browser have none, so a nix shebang needs an installer step on three more jobs to reach a bash those runners already ship. zsh is not a shebang change and the runner does not have it either. Both declined in context/code/tooling.md section 5. What lands instead pins interpreter and environment together, in one place: scripts/checkers.sh is the one entry point for every check_the_checker, used verbatim by CI and by the local gate. It pins a bash >= 5 (ambient first, nix only when the ambient one is older, exit 3 when neither), scrubs the environment to an allowlist, and refuses any shell shebang in scripts/ that is not `#!/usr/bin/env bash`. The pinned bash goes on the front of PATH so inner invocations get it too -- which is where the fault actually lived. The second undetected split is closed with it: the four `#!/bin/sh` and one `#!/usr/bin/env sh` scripts are now bash, and the six places that ran a script through an explicit `sh` -- overriding the callee's own shebang, which is the same disease -- now invoke it directly. Verification, injected against the real script: restoring pr-reviews' multi-assignment `local` leaves `sh scripts/pr-reviews-test.sh` at 22/22 and `direnv exec . ./scripts/pr-reviews-test.sh` at 22/22, while `./scripts/checkers.sh pr-reviews` reports 2 passed, 20 failed, exit 1 -- matching the 2/19 CI actually reported. scripts/checkers-test.sh is twelve cases, each injecting rather than asserting; the pair that carries it runs one fixture both ways, so the `set -u` case is a test of the runner and not of `set -u`. The old local gate was `for t in scripts/*-test.sh; do sh "$t"; done`, which overrides every shebang with /bin/sh -- so it ran every suite under bash 3.2 in sh-mode regardless of what the file said. It is replaced by `./scripts/checkers.sh`. Workspace suite unchanged at 1689 passing over 38 binaries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
92 lines
4.1 KiB
Bash
Executable File
92 lines
4.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# check_the_checker for scripts/release-preflight.sh.
|
|
#
|
|
# It exists for the reason scripts/assemble-stamp-test.sh does: the guard is the
|
|
# whole value, and a guard that silently stopped guarding is invisible from the
|
|
# outside -- a preflight that accepted everything would print the same reassuring
|
|
# lines it prints today.
|
|
#
|
|
# IT ASSERTS THE REASON, NOT THE EXIT CODE, and that is the point. A first version
|
|
# checked only "did it refuse", and every negative case passed while the tag
|
|
# comparison was disabled -- because the script refuses a DIRTY WORKING TREE first,
|
|
# and this repo's tree is usually dirty (bd auto-exports .beads/issues.jsonl). Four
|
|
# of five checks were answering "is the tree dirty" while claiming to answer "does
|
|
# it catch a bad tag". Injecting `if false;` into the comparison is what exposed it.
|
|
# See context/design-principles.md section 7 -- a check adjacent to the thing it names.
|
|
set -eu
|
|
|
|
root=$(cd "$(dirname "$0")/.." && pwd)
|
|
script="$root/scripts/release-preflight.sh"
|
|
fails=0
|
|
|
|
# Refuses, AND the refusal names the tag/version mismatch rather than the tree.
|
|
refuses_for_the_tag() {
|
|
what=$1; tag=$2
|
|
out=$("$script" "$tag" 2>&1) && { echo "FAIL: $what -- accepted" >&2; fails=$((fails+1)); return; }
|
|
case "$out" in
|
|
*"Cargo.toml has"*) echo " ok: $what (refused, naming the version)" ;;
|
|
*"uncommitted changes"*) echo "FAIL: $what -- refused for the DIRTY TREE, so this check proved nothing" >&2; fails=$((fails+1)) ;;
|
|
*) echo "FAIL: $what -- refused for an unrecognised reason:" >&2; echo "$out" | head -3 >&2; fails=$((fails+1)) ;;
|
|
esac
|
|
}
|
|
|
|
version=$(awk '/^\[package\]/{p=1;next} /^\[/{p=0} p&&/^version *=/{gsub(/[" ]/,"");sub(/^version=/,"");print;exit}' \
|
|
"$root/Cargo.toml")
|
|
test -n "$version" || { echo "FAIL: could not read the root version" >&2; exit 1; }
|
|
echo "release-preflight-test: root version is $version"
|
|
|
|
# The dirty-tree guard runs first, so the negative cases are exercised against a
|
|
# CLEAN checkout of this tree rather than the working copy. Without this the test
|
|
# cannot reach the comparison it exists to check.
|
|
tmp=$(mktemp -d)
|
|
trap 'rm -rf "$tmp"' EXIT
|
|
git -C "$root" archive HEAD | (cd "$tmp" && tar x)
|
|
cp -f "$script" "$tmp/scripts/release-preflight.sh"
|
|
cp -f "$root/Cargo.toml" "$tmp/Cargo.toml"
|
|
(cd "$tmp" && git init -q . && git add -A && git -c user.email=t@t -c user.name=t commit -qm t)
|
|
script="$tmp/scripts/release-preflight.sh"
|
|
|
|
if "$script" --self-test >/dev/null 2>&1; then
|
|
echo " ok: --self-test (accepted)"
|
|
else
|
|
echo "FAIL: --self-test refused" >&2; fails=$((fails+1))
|
|
fi
|
|
|
|
if "$script" >/dev/null 2>&1; then
|
|
echo "FAIL: no argument should be a usage error" >&2; fails=$((fails+1))
|
|
else
|
|
echo " ok: no argument is a usage error (refused)"
|
|
fi
|
|
|
|
refuses_for_the_tag "a tag one character off" "v${version}x"
|
|
refuses_for_the_tag "the shape that actually broke CI" "v0.1.0-rc99"
|
|
refuses_for_the_tag "a bare version with no v" "$version"
|
|
|
|
# The positive direction, which only a clean tree can exercise.
|
|
if "$script" "v$version" >/dev/null 2>&1; then
|
|
echo " ok: the matching tag v$version (accepted)"
|
|
else
|
|
echo "FAIL: the matching tag v$version was refused" >&2; fails=$((fails+1))
|
|
fi
|
|
|
|
# The dirty-tree guard, which every check above is deliberately blind to: they
|
|
# run on a CLEAN checkout precisely so the tag comparison is reachable at all.
|
|
# The tag here MATCHES the version, so the comparison passes and a refusal can
|
|
# only be the tree -- which is what separates this check from the ones above
|
|
# instead of duplicating them. Disabling the guard leaves all six of those green.
|
|
echo "dirtied by release-preflight-test" >> "$tmp/README.md"
|
|
if out=$("$script" "v$version" 2>&1); then
|
|
echo "FAIL: a dirty tree -- accepted" >&2; fails=$((fails+1))
|
|
else
|
|
case "$out" in
|
|
*"uncommitted changes"*) echo " ok: a dirty tree (refused, naming the tree)" ;;
|
|
*) echo "FAIL: a dirty tree -- refused, but not for the tree:" >&2; echo "$out" | head -3 >&2; fails=$((fails+1)) ;;
|
|
esac
|
|
fi
|
|
|
|
if [ "$fails" -ne 0 ]; then
|
|
echo "release-preflight-test: $fails failure(s)" >&2
|
|
exit 1
|
|
fi
|
|
echo "release-preflight-test: all checks passed"
|