Make --check-evidence additive, so no clean exit skips the comment cap
scripts/check-comment-runs.py early-returned on args.check_evidence, so the flagged run answered the evidence question and said nothing at all about the comment-run cap. The flagged form is the expensive one -- it needs a compiled workspace -- which makes it the one a human runs alone; three agents hit this independently in one campaign, and in one case a local gate went green while CI went red on a three-line doc comment (improvise-ijql). The cap moves into check_cap(), and --check-evidence now runs it first and returns the worse of the two verdicts. The ticket's cheaper alternative -- keep the early return and print a line saying the cap did not run -- is declined and the reason is recorded: a provenance line qualifies an answer the run gave, where a disclaimer qualifies one it did not, while still exiting 0. Exit 0 is what CI branches on and what a human reads as clean, so the rule is stated over the exit code: a tree with a long comment run must not pass any invocation that can exit 0. The guard is _dispatch_self_test, which runs the script as a subprocess against a temporary tree -- the only level at which this is visible, since every unit in the file was correct and the defect was in main()'s dispatch. Both arms were demonstrated by injection: restoring the early return fails "DISPATCH: --check-evidence: exited 0 over a four-line comment run", and forcing the cap to always report a violation fails the clean arm for both invocations, which is what stops an always-fail from satisfying the first. Measured: the cap adds 0.29s to a 6.5s --check-evidence run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
@@ -154,6 +154,42 @@ number no one is watching. Saying otherwise would claim a detectability it
|
||||
cannot support, which is the class of defect §7's adjacent-check section is
|
||||
about.
|
||||
|
||||
### A flag adds a question; it never selects one away
|
||||
|
||||
`--check-evidence` used to **early-return**, so the flagged run answered the
|
||||
evidence question and said nothing at all about the comment-run cap. Three
|
||||
agents hit that independently in one campaign, and in one case a local gate went
|
||||
green while CI went red on a three-line doc comment (improvise-ijql). It is now
|
||||
additive: `main` runs `check_cap` first, then `check_evidence`, and returns the
|
||||
worse of the two.
|
||||
|
||||
The ticket offered a cheaper shape — leave the early return and print one line
|
||||
saying the cap did not run — on the precedent of PR #83 making `--rust-tests`
|
||||
name its source. That precedent does not reach this case, and the distinction is
|
||||
worth keeping. **A provenance line and a disclaimer are not the same object.**
|
||||
Naming which listing the names came from qualifies an answer the run actually
|
||||
gave; saying "the cap did not run" qualifies an answer it did not give, while
|
||||
still **exiting 0**. Exit 0 is what CI branches on and what a human reads as
|
||||
clean, so a disclaimer leaves the gate exactly as skippable and merely documents
|
||||
the skip. The rule the fix is written to is therefore stated over the exit code:
|
||||
**a tree with a long comment run must not pass any invocation that can exit 0.**
|
||||
|
||||
Which invocations those are is a decision, not a sweep. `--self-test` and
|
||||
`--list-doctests` are excluded and neither is an exception to the rule: the
|
||||
first scans no tree, and the second is a *listing* that renders no verdict. Only
|
||||
the two that answer "is this tree clean" are in `SELF_TEST_DISPATCH`.
|
||||
|
||||
That table is checked by running the script as a **subprocess** against a
|
||||
temporary tree, which is the only level at which this bug is visible — every
|
||||
unit in this file was correct, and the defect lived in `main`'s dispatch. Both
|
||||
arms are needed and both were demonstrated: restoring the early return fails
|
||||
`DISPATCH: --check-evidence: exited 0 over a four-line comment run`, and forcing
|
||||
the cap to always report a violation fails the clean arm for both invocations,
|
||||
which is what stops an always-fail from satisfying the first. The temporary tree
|
||||
also has no `context/test-evidence/` and no JS roots, so the evidence half is
|
||||
vacuously clean there and the cap is the only thing that can decide the exit
|
||||
code — the isolation is what makes the assertion name one question.
|
||||
|
||||
## 3. The others
|
||||
|
||||
`scripts/kb` is the only writer of `prolog-sessions/kb.pl`; its own reasoning is
|
||||
|
||||
Reference in New Issue
Block a user