Files
improvise/scripts/pr-watch
T
Edward Langley 661ca76e6d kb: close the directive bypass, and package both tools as skills
Review on PR #68 found two real holes, both reproduced by injection before
fixing and both now regression-tested:

- A DIRECTIVE is callable, so the callable check let one through -- and the
  candidate is validated by CONSULTING it, so a delta of `:- halt(0).` exited
  the check with success before kb_consistent ran. The gate passed by never
  running, and the directive landed in kb.pl. Directives and queries are now
  refused by shape.
- Keys were checked only against the EXISTING file, so two clauses sharing a
  previously-unused key clashed with nothing and the merge dropped the first --
  add discarding part of its own payload. Visible in the tool's own output ("2
  in delta" against one clause added) and nothing compared those numbers.

Also from the review: nothing ran kb-test.sh. A suite that gates the only writer
of kb.pl but is invoked by no workflow is doctrine(an_advisory_gate_is_not_a_gate)
in the PR that adds it. Both suites now run in the kb job.

AGENTS.md now leads with the rule instead of describing the retired procedure
first -- the review read against the earlier head, but it was half right at the
later one too: a skimming reader followed the batching paragraph.

Both tools are packaged as skills, because the failure is not that the scripts
are missing, it is that agents do not know they exist and write their own. Three
hand-rolled PR watchers in one session all reported green while wrong, which is
what scripts/pr-watch and its eight fixture cases exist to stop. One of those
cases -- a head that moves mid-wait -- was untestable until a subshell bug in the
fixture seam was fixed: fetch ran inside "$(...)", so its counter never advanced
and every poll re-read the first fixture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
2026-09-20 16:02:57 -07:00

115 lines
4.1 KiB
Bash
Executable File

#!/usr/bin/env bash
# Waits for a PR's checks to finish and says what happened. Every rule in here
# is one this repo got wrong first; see context/code/pr-watch.md.
set -euo pipefail
usage() {
cat >&2 <<'EOF'
usage: pr-watch <pr-number> [--expect <sha>] [--timeout <sec>] [--interval <sec>]
--expect <sha> fail if the PR head is not this (prefix ok) at any poll
--timeout <sec> give up after this long (default 1800)
--interval <sec> seconds between polls (default 20)
exit 0 every check COMPLETED and none failed
exit 1 a check failed, or the PR is not mergeable
exit 2 the head moved during the wait, or is not --expect
exit 3 timed out with checks still running
exit 4 usage or environment error
EOF
exit 4
}
PR=""; EXPECT=""; TIMEOUT=1800; INTERVAL=20
while [ $# -gt 0 ]; do
case "$1" in
--expect) EXPECT="${2:-}"; shift 2 ;;
--timeout) TIMEOUT="${2:-}"; shift 2 ;;
--interval) INTERVAL="${2:-}"; shift 2 ;;
-h|--help) usage ;;
*) [ -z "$PR" ] || usage; PR="$1"; shift ;;
esac
done
[ -n "$PR" ] || usage
# `gh` is shell-aliased to a different tool in this repo's shells, so the name
# is never trusted: a real gh is located explicitly.
fetch() {
if [ -n "${PR_WATCH_FIXTURE_DIR:-}" ]; then
# The counter lives on disk because fetch is called as "$(fetch)", and a
# command substitution is a SUBSHELL -- a variable incremented in it is
# lost, so every poll re-read the first fixture and no test could ever
# reach the second one.
local c="$PR_WATCH_FIXTURE_DIR/.poll"
local n=$(( $(cat "$c" 2>/dev/null || echo 0) + 1 ))
echo "$n" > "$c"
local f="$PR_WATCH_FIXTURE_DIR/$n.json"
[ -f "$f" ] || f="$PR_WATCH_FIXTURE_DIR/last.json"
cat "$f"
return
fi
if command -v gh >/dev/null 2>&1 && gh --version 2>/dev/null | grep -q '^gh version'; then
gh pr view "$PR" --json headRefOid,mergeable,mergeStateStatus,statusCheckRollup
else
nix shell nixpkgs#gh --command gh pr view "$PR" \
--json headRefOid,mergeable,mergeStateStatus,statusCheckRollup
fi
}
jqf() { jq -r "$1"; }
started=$(date +%s)
pinned=""
while :; do
json="$(fetch)" || { echo "pr-watch: cannot read PR $PR" >&2; exit 4; }
head="$(printf '%s' "$json" | jqf '.headRefOid')"
# A verdict about a tree that no longer exists reads as a pass. This is not
# hypothetical here: PR #67 was merged at a head that predated its fix.
if [ -n "$EXPECT" ] && [ "${head#"$EXPECT"}" = "$head" ]; then
echo "pr-watch: head is $head, expected $EXPECT*" >&2
exit 2
fi
if [ -z "$pinned" ]; then
pinned="$head"
elif [ "$head" != "$pinned" ]; then
echo "pr-watch: head moved $pinned -> $head during the wait" >&2
exit 2
fi
# total>0 AND nothing still running. "0 pending" alone reads identically for
# "all finished" and "none registered yet", which is how this exited instantly
# on a fresh push. And `conclusion` is a STRING, empty while running, never
# null -- so a null test matches nothing and every check looks finished.
total="$(printf '%s' "$json" | jqf '[.statusCheckRollup[]?] | length')"
running="$(printf '%s' "$json" | jqf '[.statusCheckRollup[]? | select(.status != "COMPLETED")] | length')"
if [ "$total" -gt 0 ] && [ "$running" -eq 0 ]; then
break
fi
now=$(date +%s)
if [ $((now - started)) -ge "$TIMEOUT" ]; then
echo "pr-watch: timed out after ${TIMEOUT}s with $running of $total still running" >&2
exit 3
fi
sleep "$INTERVAL"
done
printf '%s' "$json" | jqf '.statusCheckRollup[]? | select(.conclusion != "SKIPPED") | " \(.conclusion) \(.name)"'
failed="$(printf '%s' "$json" | jqf \
'[.statusCheckRollup[]? | select(.conclusion != "SUCCESS" and .conclusion != "SKIPPED" and .conclusion != "NEUTRAL")] | length')"
mergeable="$(printf '%s' "$json" | jqf '.mergeable')"
state="$(printf '%s' "$json" | jqf '.mergeStateStatus')"
echo "pr-watch: head=$head $mergeable/$state $total checks, $failed failed"
[ "$failed" -eq 0 ] || exit 1
# UNSTABLE means a non-required check is unhappy; it is not a merge blocker on
# its own, so only an outright non-mergeable state fails here.
[ "$mergeable" != "CONFLICTING" ] || exit 1
exit 0