fix(tooling): a barren file is refused by node's own report, not by grepping for test(

Two findings from the PR #69 review, one class: a lexical `test(` scan is
satisfied by things that execute no assertion -- a commented-out call, the
words in a string, `test.skip`/`test.todo`, and an empty `test.describe`.

scripts/node-test.sh drops the scan and reads node's report instead. `# pass`
is 0 for a skip-only file, a todo-only file and an empty suite; and a file that
registered no test is reported as one passing test WHOSE NAME IS THE FILE PATH,
which is both how a barren file is detected and why `# tests` alone reads as
the file count. That naming convention is an assumption about node, so the
self-test pins it: a node that stops doing it fails the case rather than
letting the guard go quiet. Six cases added, including a barren file hiding
among passing ones.

--check-evidence had the parallel hole: _JS_TEST accepted `.skip`, `.todo` and
`.describe`, so a quoted heading could be satisfied by disabled coverage or by
an empty suite. Now runnable leaves only (`.only` included), with three cases
asserting that a skipped, todo or suite-only name leaves the heading orphaned
while a test nested in a suite still satisfies one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-20 16:41:09 -07:00
parent 50de0a8fd2
commit 0b7a2351b5
3 changed files with 113 additions and 26 deletions
+48 -15
View File
@@ -6,17 +6,23 @@
# invisible from a developer's shell and visible only under CI's.
#
# So the glob is the SCRIPT's, not the shell's -- quote it at the call site.
# Two things are refused, and both are "the gate asserted nothing" wearing an
# exit 0: a pattern that matches no file, and a matched file that declares no
# test.
# Three things are refused, and all three are "the gate asserted nothing"
# wearing an exit 0: a pattern that matches no file, a run in which nothing
# passed, and a matched file that registered no test.
#
# The second is checked LEXICALLY, and that is not laziness. node's own count
# cannot see it: a file registering no test is reported as one passing test
# named after the file, so `# tests N` reads as the FILE count and a
# The last two are read off node's OWN report rather than scanned for lexically.
# A `grep` for `test(` is satisfied by a commented-out call, by a string
# mentioning one, by `test.skip`/`test.todo` and by an empty `test.describe` --
# every one of which executes no assertion (found by review on PR #69).
#
# Two measurements make the report usable, both on node 24.20. `# pass` is 0
# for a skip-only file, a todo-only file and an empty suite. And a file that
# registered NO test is reported as one passing test WHOSE NAME IS THE FILE
# PATH, which is why `# tests` alone reads as the file count and a
# minimum-count assertion over it is satisfied by the very file it should
# reject. Measured on node 24.20.
#
# scripts/node-test-test.sh proves each refusal by injecting its fault.
# reject. That naming convention is an assumption about node, so
# scripts/node-test-test.sh pins it: if a future node stops doing it, the
# self-test fails rather than the guard silently going quiet.
set -uo pipefail
usage() {
@@ -40,17 +46,44 @@ for pattern in "$@"; do
files+=("${matched[@]}")
done
tap="$(mktemp)"
trap 'rm -f "$tap"' EXIT
# Two reporters: the readable one to the terminal, TAP to a file, so the run
# can be read back without downgrading what a human sees.
node --test \
--test-reporter=spec --test-reporter-destination=stdout \
--test-reporter=tap --test-reporter-destination="$tap" \
"${files[@]}"
status=$?
[ "$status" -eq 0 ] || exit "$status"
passed="$(sed -n 's/^# pass \([0-9][0-9]*\)$/\1/p' "$tap" | tail -1)"
if [ -z "$passed" ]; then
echo "node-test: could not read a pass count out of node's TAP output" >&2
exit 1
fi
if [ "$passed" -eq 0 ]; then
echo "node-test: the run exited 0 with nothing passing (skipped, todo or an empty suite)" >&2
exit 1
fi
barren=()
for f in "${files[@]}"; do
grep -Eq '(^|[^A-Za-z0-9_.])(test|it)(\.[A-Za-z]+)?[[:space:]]*\(' "$f" || barren+=("$f")
awk -v want="$f" '
/^(ok|not ok) [0-9]+ - / {
desc = $0
sub(/^(ok|not ok) [0-9]+ - /, "", desc)
sub(/ # (SKIP|TODO).*$/, "", desc)
if (desc == want) { found = 1 }
}
END { exit found ? 0 : 1 }
' "$tap" && barren+=("$f")
done
if [ "${#barren[@]}" -gt 0 ]; then
echo "node-test: these files declare no test, so running them asserts nothing:" >&2
echo "node-test: these files registered no test, so running them asserted nothing:" >&2
printf ' %s\n' "${barren[@]}" >&2
exit 1
fi
node --test "${files[@]}"
status=$?
[ "$status" -eq 0 ] && echo "node-test: ${#files[@]} file(s) ran"
exit "$status"
echo "node-test: $passed test(s) passed over ${#files[@]} file(s)"