f2bb8a62929c9bc4a594c8683327705e947f7d13
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Edward Langley
|
cb6866b2c6 |
fix(scripts): one shell and one environment for every checker (improvise-f9qf)
`scripts/pr-reviews-test.sh` passed 19/19 locally and 2/19 in CI on the same commit, because a multi-assignment `local` is not left-to-right on bash 5 while macOS's bash 3.2 tolerates it. The obvious half of the fix is pinning the shell. The half that decides the design is not. .envrc DOES already put a nix bash 5.3.15 on PATH, which is CI's semantics -- and it is not enough. `pkgs.mkShell` exports `name`, `out`, `outputs`, `system`, `shell`, `builder`, `stdenv`, `phases` and about twenty more generically-named variables that CI does not have, so a script reading one by accident under `set -u` is SILENT in the dev shell and fatal in CI. That is exactly what happened: the broken clause read `$name`, and in the dev shell `$name` was `nix-shell-env`. Measured at 051f541^ on one machine: macOS /bin/bash 3.2 19 passed, 0 failed direnv exec . ./scripts/pr-reviews-test.sh 19 passed, 0 failed the same, with `name` unset 2 passed, 17 failed Two reasons for green, one of them the correct interpreter. A dev shell is a third environment, not a reproduction of CI. A nix shebang was measured and declined, on availability rather than speed: warm, `nix shell nixpkgs#bash --command true` is 61.8ms against an ambient 5.1ms, and `nix-shell -p bash --run true` -- the manpage form -- is 675.2ms, 132x, inside git's merge machinery for beads-merge-driver.sh. But only the `kb` job installs nix; build, test, wasm, js, harness-lint and browser have none, so a nix shebang needs an installer step on three more jobs to reach a bash those runners already ship. zsh is not a shebang change and the runner does not have it either. Both declined in context/code/tooling.md section 5. What lands instead pins interpreter and environment together, in one place: scripts/checkers.sh is the one entry point for every check_the_checker, used verbatim by CI and by the local gate. It pins a bash >= 5 (ambient first, nix only when the ambient one is older, exit 3 when neither), scrubs the environment to an allowlist, and refuses any shell shebang in scripts/ that is not `#!/usr/bin/env bash`. The pinned bash goes on the front of PATH so inner invocations get it too -- which is where the fault actually lived. The second undetected split is closed with it: the four `#!/bin/sh` and one `#!/usr/bin/env sh` scripts are now bash, and the six places that ran a script through an explicit `sh` -- overriding the callee's own shebang, which is the same disease -- now invoke it directly. Verification, injected against the real script: restoring pr-reviews' multi-assignment `local` leaves `sh scripts/pr-reviews-test.sh` at 22/22 and `direnv exec . ./scripts/pr-reviews-test.sh` at 22/22, while `./scripts/checkers.sh pr-reviews` reports 2 passed, 20 failed, exit 1 -- matching the 2/19 CI actually reported. scripts/checkers-test.sh is twelve cases, each injecting rather than asserting; the pair that carries it runs one fixture both ways, so the `set -u` case is a test of the runner and not of `set -u`. The old local gate was `for t in scripts/*-test.sh; do sh "$t"; done`, which overrides every shebang with /bin/sh -- so it ran every suite under bash 3.2 in sh-mode regardless of what the file said. It is replaced by `./scripts/checkers.sh`. Workspace suite unchanged at 1689 passing over 38 binaries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH |
||
|
Edward Langley
|
708d335fa7 |
ci: the preflight repair relocated its blind spot; close it
REVIEW OF THE PREVIOUS COMMIT, NOT NEW MACHINERY. The diagnosis it states was re-verified in both directions before anything here was written: `dist plan --tag=v0.1.0-rc2` gets past the tag/version check (stopping at the pinned cargo-dist-version, as expected) and `--tag=v0.1.0-rc3` reproduces the CI error verbatim, naming `--tag=v0.1.0-rc2` as the fix in its own help text. The workflow was never broken. Both claimed fault injections also reproduce exactly: the comparison replaced by `if false;`, and `expected` derived from the tag, each produce four failures and exit 1 -- exit code read unpiped, since a pager's status would mask it. WHAT THE TEST COULD NOT SEE. Disabling the dirty-tree guard outright left all six checks green and the script exiting 0. That guard is not decoration: it is what stops a tag naming a commit that was never tested, and its failure is irreversible in the same way the version mismatch is. The cause is the previous repair. Moving the negative cases to a clean throwaway checkout is what made the tag comparison reachable at all -- and it thereby removed the only circumstance in which the test could observe the tree guard. The guard whose PRECEDENCE caused the original vacuity became the one thing the suite could not see. The vacuity and its repair are not two events; the repair relocated the blind spot rather than closing it. So the new check is aimed the other way: a tag that MATCHES the version, run against a dirtied checkout, so the comparison passes and a refusal can only be the tree. Verified discriminating -- with the guard disabled it fails with exit 1, and the two faults are independent in both directions (comparison disabled leaves the tree check passing, guard disabled leaves the three tag checks passing), so the seventh check is not a restatement of the other six. Also recorded, rather than guarded: the `dist plan` second opinion can be disabled with the suite green, because every input the test feeds it has already been refused by the textual check. That follows from the arm being optional by design -- pinning it would mean installing a dist matching cargo-dist-version, the very mismatch the script tolerates on purpose. Stated so the silence is not mistaken for coverage. Two corrections of fact. The CI step lives in the `kb` job beside pr-watch-test.sh, where this repo keeps its check_the_checkers -- not in `test`; tooling.md now says so. And the awk version parser was probed rather than hardened: `version.workspace = true` yields empty, which the root path refuses loudly and the per-crate loop skips; a non-first [package] block and build metadata both parse correctly; a trailing comment on the version line mis-parses but fails CLOSED and prints the mis-parsed value in the refusal. No manifest in this workspace has one, so nothing was changed for it. No version bumped, no tag cut. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH |
||
|
Edward Langley
|
aa4e901bd2 |
ci: a release preflight, because the tag is already public when CI refuses
THE WORKFLOW WAS NEVER BROKEN. v0.1.0-rc3 was pushed to all three remotes while every crate still said 0.1.0-rc2; dist matches a tag against the root package version, refused with "This workspace doesn't have anything for dist to Release!", and the four jobs behind `plan` skipped. Proved in both directions locally: `dist plan --tag=v0.1.0-rc2` gets past the check and `--tag=v0.1.0-rc3` reproduces the CI error exactly. So this changes no trigger and no job graph. What it adds is a check that runs BEFORE the irreversible step. dist named the right answer in its own error text; the problem is when that answer arrives -- a tag is public and immutable by the time CI reads it, so the repair is a new version rather than an edit. scripts/release-preflight.sh refuses a dirty tree, compares the intended tag against the root package version, runs `dist plan` as a second opinion when dist is on PATH (and says so when it is not, since an absent dist must not read as a pass), and prints the tag-and-push commands only once it passes. THE TEST'S FIRST VERSION WAS VACUOUS AND THE INJECTION IS WHAT SHOWED IT. release-preflight-test.sh originally asserted only "did it refuse". Every negative case still passed with the tag comparison replaced by `if false;`, because the script refuses a DIRTY WORKING TREE first and this repo's tree is almost always dirty -- bd auto-exports .beads/issues.jsonl on nearly every command. Four of five checks were answering "is the tree dirty" while claiming to answer "does it catch a bad tag". It now runs the negative cases against a clean throwaway checkout AND asserts the refusal names the version mismatch, so a refusal for the wrong reason fails. Two rival faults were injected against the repaired version -- the comparison disabled, and `expected` derived from the tag itself -- and each produces four failures and exit 1. That is design-principles section 7's "a check that looks like the answer, answering a neighbouring question", found inside a checker written to prevent that class, within an hour of writing it. Recorded in context/code/tooling.md. No version is bumped and no tag is cut here, per the user's instruction to make the workflow work without releasing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH |