fix(tooling): three findings from reviewing this branch against itself
1. A self-test case that was a duplicate wearing different clothes. "a heading cargo's listing does not carry is an orphan however the source reads" passed on an empty listing -- which is the first case in the table, already. Once defined comes from cargo, the Rust beside a fixture is not read at all, so four cases for the four source shapes would be four copies of one assertion dressed as coverage. Deleted; the evidence for those shapes is the real-tree injection, and context/code/tooling.md now says which half a fixture can carry and which it cannot. This is exactly the shape improvise-84wp's prose, applied two commits ago, tells a reviewer to look for. 2. assemble-stamp.sh grepped its patterns as REGEXES, inherited from the inline version it was extracted from. The `.` in improvise_wasm_client.js?v=abc matches any character, so a stamp corrupted in that position read as present -- a pass in the one direction a guard must not fail in. grep -F, plus a case proved by injection: dropping the -F flips it. 3. A missing cargo raised a traceback out of --check-evidence. It now names the flag that takes a saved listing instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
+11
-2
@@ -82,8 +82,17 @@ entries, all name a test that exists*, exit 0. The current one names all four.
|
||||
|
||||
What this costs: `--check-evidence` now needs a toolchain and a compiled
|
||||
workspace, about three seconds warm. `--rust-tests FILE` takes a saved listing
|
||||
for a caller that has one. The comment-cap check is unaffected and needs
|
||||
neither.
|
||||
for a caller that has one, and a missing `cargo` says so rather than raising.
|
||||
The comment-cap check is unaffected and needs neither.
|
||||
|
||||
Note what the self-test can and cannot carry here. The listing *reader* is
|
||||
testable from fixtures — a leaf name taken off a module path, an `--ignored`
|
||||
name subtracted, cargo's own prose not mistaken for a test — and those cases
|
||||
are in `SELF_TEST_LISTING`. The four *source shapes* are not: once `defined`
|
||||
comes from cargo, a fixture asserting "a heading with an empty listing is an
|
||||
orphan" is the same case however the Rust beside it is spelled, and writing
|
||||
four of them would be four copies of one assertion dressed as coverage. The
|
||||
evidence for those is the injection above, against the real tree.
|
||||
|
||||
What it does **not** close: `citing` is still scraped, so a citation inside a
|
||||
string literal would be reported as a test owing an entry. That is a false
|
||||
|
||||
@@ -34,6 +34,13 @@ own header and the test's header both name the directory in prose — a lexical
|
||||
grep over raw text would have had to exempt them, and an exemption is where the
|
||||
next violation goes.
|
||||
|
||||
**What it cannot see**, stated because the check itself uses the evasion: a path
|
||||
assembled at runtime (`"../" + "p" + "kg"`) is invisible to a lexical match, and
|
||||
the test's own `FORBIDDEN` constant is built exactly that way to stay out of its
|
||||
own scan. That is a deliberate, disclosed hole — closing it needs a parser, and
|
||||
the failure it would catch is one somebody has to write on purpose, where this
|
||||
gate exists for the one written by accident.
|
||||
|
||||
### "the subdirectory that MAY use a bundle is not the one being checked"
|
||||
|
||||
The companion, and the one that stops the first from passing vacuously. If
|
||||
@@ -47,6 +54,18 @@ Fault injected, 2026-09-21: moved `web/tests/bundle/wasm_surface.test.js` out of
|
||||
the tree. The test failed with "web/tests/bundle/ is empty — the split lost its
|
||||
other half" rather than passing. Restored; green again.
|
||||
|
||||
### Note — the stamp gate matches literally
|
||||
|
||||
`scripts/assemble-stamp.sh` greps with `-F`. As a regex, the `.` in
|
||||
`improvise_wasm_client.js?v=abc` matches any character, so a stamp corrupted in
|
||||
exactly that position reads as present — a pass in the one direction a guard
|
||||
must not fail in. Inherited from the inline version this was extracted from,
|
||||
and found by reading the extraction rather than by anything failing.
|
||||
|
||||
Fault injected, 2026-09-21: dropping the `-F` makes
|
||||
`improvise_wasm_clientXjs?v=abc` satisfy the pattern, and
|
||||
`a pattern is matched literally, not as a regex` goes from pass to fail.
|
||||
|
||||
### Note — the doc site's derived checks
|
||||
|
||||
`scripts/build-doc-site.mjs` carries two gates that no test names, because the
|
||||
|
||||
@@ -64,6 +64,13 @@ run_case "a file that does not exist is a miss, not an abort" 1 "vanished.js" \
|
||||
|
||||
# A caller that drops one argument would otherwise have the last FILE read as
|
||||
# a PATTERN over the file before it -- a check that passes by accident.
|
||||
# The pattern is a literal. As a regex, `.` matches any character, so a stamp
|
||||
# corrupted in exactly that position would read as present -- the one direction
|
||||
# a guard must not fail in.
|
||||
printf 'import "./pkg/improvise_wasm_clientXjs?v=abc";\n' > "$work/corrupt.js"
|
||||
run_case "a pattern is matched literally, not as a regex" 1 "corrupt.js" \
|
||||
"$work/corrupt.js" "pkg/improvise_wasm_client.js?v=abc"
|
||||
|
||||
run_case "an odd number of arguments is a usage error" 2 "pairs" \
|
||||
"$work/stamped.js" "pkg/improvise_wasm_client.js?v=abc" "$work/bare.js"
|
||||
|
||||
|
||||
@@ -24,8 +24,10 @@ while [ $# -ge 2 ]; do
|
||||
file=$1
|
||||
pattern=$2
|
||||
shift 2
|
||||
if [ -f "$file" ] && grep -q "$pattern" "$file"; then
|
||||
grep -n "$pattern" "$file"
|
||||
# -F, because every pattern is a literal and `.` as "any character" would
|
||||
# let a corrupted stamp match -- a pass in the direction that looks like one.
|
||||
if [ -f "$file" ] && grep -qF "$pattern" "$file"; then
|
||||
grep -nF "$pattern" "$file"
|
||||
else
|
||||
echo "assemble: $file is missing the cache-bust stamp: $pattern" >&2
|
||||
missing="${missing} $file"
|
||||
|
||||
@@ -313,7 +313,12 @@ def cargo_test_names(listing, ignored_listing=""):
|
||||
|
||||
def _cargo_listing(extra):
|
||||
cmd = ["cargo", "test", "--workspace", "--", "--list", *extra]
|
||||
run = subprocess.run(cmd, capture_output=True, text=True)
|
||||
try:
|
||||
run = subprocess.run(cmd, capture_output=True, text=True)
|
||||
except OSError as e:
|
||||
raise SystemExit(f"error: cannot run cargo ({e}).\n"
|
||||
"--check-evidence needs a toolchain, or --rust-tests FILE "
|
||||
"holding a saved `cargo test --workspace -- --list`.")
|
||||
if run.returncode != 0:
|
||||
raise SystemExit(f"error: {' '.join(cmd)} failed:\n{run.stderr}")
|
||||
return run.stdout
|
||||
@@ -754,11 +759,6 @@ SELF_TEST_EVIDENCE = [
|
||||
("a ### naming a helper rather than a test is still an orphan",
|
||||
"### a_helper\n", "fn a_helper() {}\n", "", "", ["a_helper"], []),
|
||||
|
||||
("a heading cargo's listing does not carry is an orphan however the source reads",
|
||||
"### a_test_that_is_only_text\n",
|
||||
'pub const S: &str = r#"\n#[test]\nfn a_test_that_is_only_text() {}\n"#;\n',
|
||||
"", "", ["a_test_that_is_only_text"], []),
|
||||
|
||||
("a test that cites the evidence file and has no entry is caught",
|
||||
"", "/// See `context/test-evidence/clients.md`.\n#[test]\nfn a_cited_test() {}\n",
|
||||
"", "m::a_cited_test: test\n", [], ["a_cited_test"]),
|
||||
|
||||
Reference in New Issue
Block a user