All three found by the adversarial review of 019cbc1, each by injection.
1. every_model_command() is NOT compiler-maintained, and both this PR's prose
and reduce.rs's own doc comment said it was. variant_name's exhaustive match
forces an arm in variant_name, not an entry in the value list -- deleting
ModelCommand::TogglePruneEmpty from it failed the new completeness check and
NOTHING ELSE in the workspace. So the diff is two independent hand-written
lists: strictly better than one list against its own length, and not the
derived enumeration improvise-5ubs got for the two enums. Doc comment
corrected; the prose in context/code/command.md and the evidence entry now
say what it is. Closing it properly wants improvise_protocol::variants made
pub so declared::<ModelCommand>() can bound the list -- another wave's crate,
reported rather than taken.
The assertion message was also ONE-DIRECTIONAL ("a name only on the right
..."), which is the shape that injection produces and which invites the wrong
repair: delete the case on the left, degrading both sides together. It now
names both directions and says which is which.
2. SELF_TEST_DUPLICATES case 3 was named for the ambiguity guard and never
reached it: both its headings were module-qualified, so canonical() returned
on the exact-match arm before touching by_leaf. Dropping the guard
(`if matches` for `if len(matches) == 1`) left --self-test at exit 0. The
fixture is now one BARE heading over an ambiguous leaf plus BOTH qualified
spellings, so the injection collides with one of them whichever way
next(iter(set)) falls -- deterministic across five runs, verified.
3. The evidence entry said five pre-existing records-mode tests fail under the
settle-before-rebuild injection; measured, it is six. And "three hundred
lines above" is four hundred.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
--check-evidence verified that a heading names a test that RUNS, and a
duplicate names one just as well, so both copies pass. PR #92 shipped two
entries under one heading; the older described a fixture the test no longer
had.
Refuse rather than warn, on the improvise-rm4s precedent, naming the file and
every line in file order. Keyed by the RESOLVED test, so `### f` beside
`### m::tests::f` collides too -- the case rm4s's qualification repair creates.
Six SELF_TEST_DUPLICATES cases, including the three clean arms: two different
tests sharing a leaf are ambiguous and not duplicate, prose repeats freely, and
two headings naming two tests are fine.
Resolves the two tui.md duplicates: the older column-header copy called
THE_MEMBERSHIP_CORRECT_SPANS a live tripwire three hundred lines above the
entry recording its removal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two findings from running /code-review on this branch.
The dispatch self-test's dirty arm asserted the run failed and named `f.rs`,
which is adjacent to the question -- a failure for any other reason naming that
file would have satisfied it. It now requires the cap's own message as well,
so the assertion names the question it means.
And .claude/skills/code-review/SKILL.md still told reviewers the flag "selects
rather than adds, so --check-evidence never runs the comment cap", which this
branch makes false. Grepped for every copy of the claim; that was the only one
outside context/. A doc that describes the script it gates is code, and leaving
it is the gotcha-48 failure exactly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
scripts/check-comment-runs.py early-returned on args.check_evidence, so the
flagged run answered the evidence question and said nothing at all about the
comment-run cap. The flagged form is the expensive one -- it needs a compiled
workspace -- which makes it the one a human runs alone; three agents hit this
independently in one campaign, and in one case a local gate went green while
CI went red on a three-line doc comment (improvise-ijql).
The cap moves into check_cap(), and --check-evidence now runs it first and
returns the worse of the two verdicts. The ticket's cheaper alternative -- keep
the early return and print a line saying the cap did not run -- is declined and
the reason is recorded: a provenance line qualifies an answer the run gave,
where a disclaimer qualifies one it did not, while still exiting 0. Exit 0 is
what CI branches on and what a human reads as clean, so the rule is stated over
the exit code: a tree with a long comment run must not pass any invocation that
can exit 0.
The guard is _dispatch_self_test, which runs the script as a subprocess against
a temporary tree -- the only level at which this is visible, since every unit in
the file was correct and the defect was in main()'s dispatch. Both arms were
demonstrated by injection: restoring the early return fails
"DISPATCH: --check-evidence: exited 0 over a four-line comment run", and forcing
the cap to always report a violation fails the clean arm for both invocations,
which is what stops an always-fail from satisfying the first.
Measured: the cap adds 0.29s to a 6.5s --check-evidence run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Both blockers from the adversarial review of #83, both in the mode this PR
rewrote, both producing a pass while the gate did not do what its prose said.
1. cargo_test_names reduced every listed path to its last segment. Six leaf
names collide in this workspace and two are cited in evidence, so a heading was
satisfiable by a test in another module answering another question -- renaming
ui::project::tests::a_load_that_fails_is_not_a_rebase, the one its entry
describes, left the checker at exit 0 with the ui::host namesake answering for
it. Paths are kept whole; a bare leaf resolves only when unique, and an
ambiguous one is REFUSED with both candidates named. _RUST_NAME accepts `::`, so
a qualified heading is a claim rather than prose, and a citation is matched on
the leaf so qualifying a heading cannot orphan its own test. The two colliding
entries are qualified. Neither half suffices alone: rename one of a colliding
pair and the leaf becomes unique again, which is why qualification is the fix
and the refusal is what forces it.
2. --rust-tests called cargo_test_names with one argument, so the --ignored
subtraction never ran, and it printed a byte-identical success line. It now
requires --rust-tests-ignored (one alone exits 2) and the line names its source.
Also, from the same review: a missing context/test-evidence/ exited 0 while 170
citations pointed at nothing (early return, now gone); the doc-site "runs last"
check accepted a trailing `echo` that merely NAMED the generator, so it asks for
an invocation; bundle_freedom gains an arm for a transitive `import "../app.js"`,
which reaches the bundle while naming nothing forbidden; and two prose overclaims
are corrected -- the prose-heading count is a convenience rather than a guard
(git diff is the mechanism), and the ratio rule says "above the run MEDIAN",
which is defensible where "above the others" is false against second place.
scripts/check-ci-graph.py makes claim 6's rule a gate: pages-deploy must `needs`
every other job in ci.yml. The rule was stated in the last commit and a
zz-new-check: job added during review was still ungated with nothing noticing.
README now states the rule and lists no jobs, the second copy being what went
stale twice.
Every fault re-injected and observed; figures in context/test-evidence/tooling.md
rather than only here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
1. A self-test case that was a duplicate wearing different clothes. "a heading
cargo's listing does not carry is an orphan however the source reads" passed on
an empty listing -- which is the first case in the table, already. Once defined
comes from cargo, the Rust beside a fixture is not read at all, so four cases
for the four source shapes would be four copies of one assertion dressed as
coverage. Deleted; the evidence for those shapes is the real-tree injection, and
context/code/tooling.md now says which half a fixture can carry and which it
cannot. This is exactly the shape improvise-84wp's prose, applied two commits
ago, tells a reviewer to look for.
2. assemble-stamp.sh grepped its patterns as REGEXES, inherited from the inline
version it was extracted from. The `.` in improvise_wasm_client.js?v=abc matches
any character, so a stamp corrupted in that position read as present -- a pass in
the one direction a guard must not fail in. grep -F, plus a case proved by
injection: dropping the -F flips it.
3. A missing cargo raised a traceback out of --check-evidence. It now names the
flag that takes a saved listing instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Four shapes let a heading name something that never runs: a test inside a raw
string literal, one behind an unenabled cfg feature, one behind
cfg(target_arch = "wasm32"), and one behind #[ignore]. All four cleared the
scrape; injecting them together made the previous checker print "543 entries,
all name a test that exists", exit 0.
--check-evidence now takes the Rust names from `cargo test --workspace -- --list`
minus its `--ignored` twin. All 539 live headings resolve against a real listing,
so the stricter source costs nothing today. rust_tests is rust_citations: the
scrape survives only for the reverse direction, where a false positive is loud.
Prose headings become a third kind, counted and printed, so retiring a claim by
rewriting it as prose moves a number instead of passing in silence
(improvise-9za9). No gate can read prose for truth; the reviewer is the
mechanism, and the diff is what to read.
context/code/tooling.md is new and is the file four docstrings in
check-comment-runs.py already pointed at. design-principles §7 stopped saying
the check runs in one direction, which improvise-lyyc had already made false.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Five findings from the second review round, and the first is this PR's own
standard applied to its own P1.
**The two fixes improvise-dj1l names had no discriminating test.** Reverting
`has_key` to unification, or dropping `canon_key` from the duplicate gate, left
kb-test at 27 passed. Case 11 was carried by `check_total` refusing with a
DIFFERENT message, and case 10 by the other half of the fix. Both now assert
which refusal fired (`said()`), plus two cases aimed at each half: a rm key with
a variable in a non-final position must be refused as ABSENT rather than
matching everything, and a keyed FACT whose key holds a variable must ADD
without excluding the siblings it would unify with. Each proved by reverting
exactly one fix and watching exactly that case fail.
**Keying a rule on its whole clause made `replace` on an edited rule silently
become `add`** -- both arms live, the stale one still firing, and `check_total`
cannot see it because `NO + NF` is correct arithmetic for an add. Reachable on
the 15 `kb_error/1` arms and the `kb_text/2` rules, so editing the check
improvise-2f97 adds is the first thing to hit it. `check_rule_replace` refuses
and names the route (rm, then add); the negative arm -- replace of a rule
present verbatim -- is tested, or the refusal would just be "replace never
works on rules".
**The `decision(` arm was dead code.** `cited_name` took the first field, and
the repo spells `decision(<ticket>, <name>)`, whose ticket id fails
`name_shaped/1` on the hyphen -- so that marker could never fire for a real
citation. A second `cited_name` clause takes the second field. All three
markers are now exercised.
Minor, from the same round: `_JS_TEST` no longer accepts `test.only`, which
disables every other test in its file, with a case asserting a heading cannot
be satisfied by one; and `node-test-test.sh`'s last case no longer duplicates
an earlier one.
Filed rather than fixed: improvise-w8ih, extracting `web/assemble.sh`'s gate so
its own absence is visible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two findings from the PR #69 review, one class: a lexical `test(` scan is
satisfied by things that execute no assertion -- a commented-out call, the
words in a string, `test.skip`/`test.todo`, and an empty `test.describe`.
scripts/node-test.sh drops the scan and reads node's report instead. `# pass`
is 0 for a skip-only file, a todo-only file and an empty suite; and a file that
registered no test is reported as one passing test WHOSE NAME IS THE FILE PATH,
which is both how a barren file is detected and why `# tests` alone reads as
the file count. That naming convention is an assumption about node, so the
self-test pins it: a node that stops doing it fails the case rather than
letting the guard go quiet. Six cases added, including a barren file hiding
among passing ones.
--check-evidence had the parallel hole: _JS_TEST accepted `.skip`, `.todo` and
`.describe`, so a quoted heading could be satisfied by disabled coverage or by
an empty suite. Now runnable leaves only (`.only` included), with three cases
asserting that a skipped, todo or suite-only name leaves the heading orphaned
while a test nested in a suite still satisfies one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Two blind spots with one consequence: an evidence file could promise a guard
that was not there, and a test could point at prose that did not exist.
5ray: headings were collected with `startswith("### ")` and resolved against
Rust fn names only. context/test-evidence/clients.md uses `#### "name"` for
every browser test precisely BECAUSE a `###` naming a Playwright test would
have been reported as an orphan -- so the convention that kept the checker
quiet was the same one that made it blind. Now `###` and `####` alike, decided
by SHAPE: a bare identifier is a Rust test, a quoted string is a JS test
(harness/tests and web/tests), anything else is prose naming nothing.
lyyc: the check was one-directional. A test whose doc comment names
context/test-evidence must now appear there. Deliberately only that narrower
obligation -- most tests have no evidence and should not be forced to. Neither
direction can see a STALE entry; that stays discipline.
Each half found a real instance on its first run. A `#### "? opens help..."`
heading outlived the test 36ade35 deleted, and is demoted to prose with the
coverage filed as improvise-v5pc. `an_entry_that_holds_no_document_is_seeded_over`
cited the file and had no entry; one is written from the test.
The checker's --self-test covered the comment scanner and nothing else, which
is how both survived. It grows ten evidence cases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Adversarial review of the previous commit: scan() survived 16 literal-shape
attacks, and both HIGH findings were in the code that did NOT get rewritten.
The attribute handling still asked the RAW LINE the questions this commit's
whole argument says a raw line cannot answer.
- Bracket depth was counted over the raw line, so `#[cfg(feature = "[")]`
set attr_depth=1 with no reset path and exempted EVERY REMAINING LINE IN THE
FILE. The raw-string spelling of it is the exact defect class this rewrite
was for: scan() knew those bytes were string content and line 150 asked the
line instead. The tree has 14 `#[arg(...)]` and 3 `#[command(...)]` clap
attributes whose help strings are free text.
- `s.startswith("#[doc")` is an UNANCHORED PREFIX, so `#[doc(hidden)]` counted
as a comment and flagged correct code. Fourth instance of that class in this
campaign, eleven lines above the comment naming the previous three.
Also from the same review: `#![doc = "..."]` is the inner spelling of a doc
comment and was never counted (unlimited free crate doc); an inner attribute
between two `//!` blocks split one crate doc in half, where the rule exists to
separate a crate doc from an ITEM doc — so it now ends a run only where the doc
KIND changes; and a block comment closing on a line with code got one free line.
Attributes are now asked of the scanner: `scan()` returns the code-only text,
and depth is counted over that. `_is_attribute` is deleted — dead since the
logic moved inline, and an orphaned explanation is what --check-evidence exists
to catch one directory over.
The coverage finding was the important one. Two of scan()'s four literal
mechanisms had NO self-test case: reverting char-literal or plain-string
handling left the suite green. Both are load-bearing in opposite directions —
without char literals, `'"'` opens a string that eats the file (false negative);
without plain strings, a multi-line string holding `//` lines is a false
positive, the CI-breaking direction. Both now have cases, and so does every
shape above. The attribute-depth mechanism needed a fixture with code AFTER the
attribute, because comments inside one are counted either way: the first
fixture I wrote passed under its own injection.
Self-test: 17 violations caught, 9 legitimate shapes allowed. Every mechanism
proved discriminating by reverting it alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial review demonstrated that the previous fix closed one class and
left three, then reintroduced the bug this file's own history is about.
- A ``` fence in a PLAIN `//` comment earned the doctest exemption. rustdoc
runs no test there ("running 0 tests"), so two lines exempted anything after
them. The cheapest bypass yet, and `--list-doctests` in the same file already
had it right: one of the two was wrong and it was this one.
- A block comment opened after code on the same line was invisible, since only
a line STARTING with /* was seen. That is the most natural way to write one.
- Nested block comments closed at the first */ and the rest read as code.
- And the false positive: `in_block` had no reset path, so a /* inside a raw
string swallowed the rest of the file. Latent — no .rs file starts a line
with /* today, so the block fix had zero effect on this repo — but 8 files
hold raw strings and this repo's habit is Rust fixtures inside them. The
first such fixture breaks CI with a diagnosis pointing at a string.
A line-based scanner cannot decide any of these, so `scan()` now walks
characters, tracking nested block comments, strings, raw strings (with hashes)
and char literals. `#![...]` ends a run where `#[...]` stays transparent,
because a crate doc and an item doc separated by an inner attribute are two
blocks a reader does not meet as one. `#[doc = "..."]` counts: it is the one
attribute that is a comment.
The self-test carries all of it: 12 bypasses, 6 shapes that must stay legal.
Each new fix was proved discriminating by injecting it alone.
One of those injections caught a VACUOUS test of my own: with raw-string
handling disabled, a plain-string fallback reached the same verdict, so the
fixture proved nothing. It now contains r#"x"y, which a non-raw parse closes
early — so the fault is visible. That is the defect this repo is named for,
found in the test written to prevent it.
Closes improvise-mbmf, which PR #45 claimed to close and did not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adversarial review of PR #44 constructed all three, each compiling clean with
no warnings, each passing the checker.
Block comments were invisible: the scanner only ever tested startswith("//"),
so a six-line /* */ passed — and /** */ is a DOC comment that rustdoc renders
exactly as /// does, making `/**` the cheapest way to write the long argument
the rule exists to forbid. There are zero block comments in the tree today,
which is the only reason this was unexploited; #44 is the change that gives
anyone a reason to look.
Fence state was per-FILE, so one unclosed ``` exempted every later comment in
the file. A dangling doctest would fail the build in a lib target, but doctests
do not run for bins, tests/, examples/ or harness/ — all default scan roots.
Fence state now belongs to the comment block: any non-comment line clears it.
Attributes terminated a run, so `#[inline]` between two pairs of doc lines
split one argument of four into two legal runs of two. The module docstring
had always promised attributes are "ignored"; the code disagreed with it, and
disagreed in the direction that passes.
Closing the last of those found two real violations the old checker could not
see. crates/improvise-command/src/command/cmd/text_buffer.rs carried TWO doc
comments straddling #[derive(Debug)] — both attach to the struct, so rustdoc
rendered both; a leftover from the comment wave. transport.rs had a three-line
run split by #[allow].
A checker with ten silent successes and no test of its own is the thing it
warns about, so --self-test runs every bypass this file has been fixed for
plus three shapes that must stay legal. Proved discriminating by reverting the
attribute fix: it fails and names that case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seventh silent success, found by CC-command on PR #41, and the THIRD instance
in this campaign of one class: matching a substring where an exact token was
meant.
'test' in s also matched '#[cfg(test)]', so every cfg-gated fixture was accepted
as a test by --check-evidence — a gate written to reject exactly those. Two
plain fixtures cleared it: Host::model and Sessions::apply_caught_up.
The first instance was the bd dead-ID audit's unanchored exclusion list, where
a real ticket improvise-gp12 would have been silently excluded because it
starts with a crate name. The second was --check-evidence accepting any fn.
This is the third.
Now matched by exact attribute path (test, tokio::test, async_std::test,
wasm_bindgen_test). Fault-injected: a #[cfg(test)] fn under a ### heading is
reported as an orphan, a #[test] fn is not. Re-ran over the five merged units:
215 entries still pass, so none of them relied on the hole.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sixth silent success, and I introduced it fixing the fourth. Found by review.
Deciding 'are we inside a fence' and 'is this fence exempt' from one flag meant
a ```text block's BARE CLOSING ``` looked runnable and opened an exemption --
so prose after a non-runnable fence evaded the cap entirely. Reproduced: four
prose lines after a closed ```text block reported clean.
They are now tracked separately, and the delimiters belong to the block they
bound, so a runnable fence's closing ``` is exempt too -- otherwise it lands in
the prose run and a genuine two-line doc reads as three.
Fault-injected both directions: prose after a closed ```text block is a
violation; a runnable block with one prose line either side is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fifth silent success in this file, found by review on PR #40.
in_proptest was set on the first 'proptest!' line and NEVER CLEARED, so every
later fn in that file -- helpers included -- was accepted as a test by
--check-evidence. A file with one property test made the whole file's functions
eligible, which is exactly the hole the any-fn fix was meant to close.
Now the block's braces are tracked and the flag leaves when they balance.
Fault-injected: a helper defined after a proptest! block is reported as an
orphan, while the #[test] inside the block still passes.
Five silent successes in one small checker in one day, every one failing in the
direction that looks like passing, and none found by the person who wrote it.
The gate and the thing it gates were written by the same hand and nobody
injected a fault -- which is the wave's own finding turned on its tooling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourth silent success in the same file, and the largest.
rustdoc does not execute ```text, ```ignore or ```no_run. The exemption
treated every fence alike, so prose wearing a fence was exempt from the cap --
and 74 lines of CLI documentation in gp-tree were sitting in exactly that hole.
Measured before narrowing it: `cargo test --doc` across the whole workspace was
0 passed / 2 ignored. NOT ONE of the six fences ran. So the exemption written to
protect doctests was protecting nothing, while shielding the longest comment
blocks in the tree from the rule it was written to enforce.
Both ignore blocks had also drifted out of agreement with the code, which is
precisely what ignore cannot catch: one called fit::fit with the parallel x/y
signature improvise-dkqn removed, against a Dataset with no rows field; the
other called a rank() that has never existed. CC-gp promoted both to running
doctests, so the workspace now has 2 that actually run.
Fault-injected: a ```text block with two prose lines either side is now
reported as a run of four, while a bare ``` block is still exempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two surprises reported by agents in the same hour, neither a behaviour change.
A fence SUSPENDS the count rather than ending the run, so two prose lines above
a fence and two below is a run of four. That is deliberate -- a reader meets it
as one uninterrupted block -- but an agent capped a file to what looked like
2 and 2 and the checker correctly called it 5.
And rustdoc does not execute ```text, ```ignore or ```no_run. Two of the six
fences in this tree were ```text, so they were claiming the doctest exemption
for prose. Only a runnable fence earns it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by two wave agents inside an hour, and all three are the same class as
the bd-audit exclusion list this repo already has a doctrine about: a filter
that over-matches reports success.
1. A FILE path walked nothing and printed 'none longer than 2 lines'. roots was
filtered by os.path.isdir, so 'check-comment-runs.py src/foo.rs' silently
scanned an empty set. One agent nearly shipped a file believing it done, and
every 'my unit reports none' claim taken with a file argument was worthless.
2. A NONEXISTENT path did the same thing, so a typo'd directory read as a pass.
Both now exit 2 and name the path. Defaults are unaffected.
3. --check-evidence accepted ANY fn, so a '### heading' naming a fixture or a
helper was reported as satisfied. One unit's '68 entries, all name a test'
was passing on that accident; re-checked strictly it was 64. A heading must
now name a fn carrying #[test] (or one inside a proptest! block), which is
what the convention always claimed.
Each fix fault-injected before keeping: a file path and a missing path now
error, and a heading naming a non-test fn is reported as an orphan while a real
test passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both findings from the review on #33, and the second is the sharper one.
The checker skipped harness/opfs-contract/src/lib.rs — a shipped Rust source
tree with 14 runs over the limit, including a 22-line module doc — so it could
have reported success while the convention was violated. harness/ is now in the
default roots.
And design-principles claimed a renamed test was made detectable because 'the
checker looks for both halves'. It did not: it counted comment lines and listed
doctest fences, and nothing read context/test-evidence/ at all. So the cost the
convention says is controlled was uncontrolled, in the same commit that
introduced the convention — a reviewed paragraph is not evidence, and this one
was mine.
--check-evidence now requires every '### name' heading to name a test that
exists. One direction only: a test may have no evidence (most do not), but
evidence naming a test nobody can find is a dangling explanation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The user's instruction (2026-09-20): reasoning moves out of code comments into
context/ prose, and a comment run is at most two consecutive lines — `///` and
`//!` included, and test doc-comments too.
scripts/check-comment-runs.py makes the rule checkable rather than remembered,
which is the difference between a convention and a guard. It reports 1321 runs
over the limit today; the move itself is improvise-9c8c.
A ``` fence in a doc comment is exempt: a doctest is a running test rather than
prose, and capping around one would delete it. `--list-doctests` finds the six
blocks that need individual decisions.
design-principles §7 said to record a bug in the test's doc-comment, which the
cap contradicts — that evidence now goes to context/test-evidence/ keyed by test
name, with the orphaning cost stated rather than discovered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Edward Langley