fix(tooling): a barren file is refused by node's own report, not by grepping for test(
Two findings from the PR #69 review, one class: a lexical `test(` scan is satisfied by things that execute no assertion -- a commented-out call, the words in a string, `test.skip`/`test.todo`, and an empty `test.describe`. scripts/node-test.sh drops the scan and reads node's report instead. `# pass` is 0 for a skip-only file, a todo-only file and an empty suite; and a file that registered no test is reported as one passing test WHOSE NAME IS THE FILE PATH, which is both how a barren file is detected and why `# tests` alone reads as the file count. That naming convention is an assumption about node, so the self-test pins it: a node that stops doing it fails the case rather than letting the guard go quiet. Six cases added, including a barren file hiding among passing ones. --check-evidence had the parallel hole: _JS_TEST accepted `.skip`, `.todo` and `.describe`, so a quoted heading could be satisfied by disabled coverage or by an empty suite. Now runnable leaves only (`.only` included), with three cases asserting that a skipped, todo or suite-only name leaves the heading orphaned while a test nested in a suite still satisfies one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
+48
-15
@@ -6,17 +6,23 @@
|
||||
# invisible from a developer's shell and visible only under CI's.
|
||||
#
|
||||
# So the glob is the SCRIPT's, not the shell's -- quote it at the call site.
|
||||
# Two things are refused, and both are "the gate asserted nothing" wearing an
|
||||
# exit 0: a pattern that matches no file, and a matched file that declares no
|
||||
# test.
|
||||
# Three things are refused, and all three are "the gate asserted nothing"
|
||||
# wearing an exit 0: a pattern that matches no file, a run in which nothing
|
||||
# passed, and a matched file that registered no test.
|
||||
#
|
||||
# The second is checked LEXICALLY, and that is not laziness. node's own count
|
||||
# cannot see it: a file registering no test is reported as one passing test
|
||||
# named after the file, so `# tests N` reads as the FILE count and a
|
||||
# The last two are read off node's OWN report rather than scanned for lexically.
|
||||
# A `grep` for `test(` is satisfied by a commented-out call, by a string
|
||||
# mentioning one, by `test.skip`/`test.todo` and by an empty `test.describe` --
|
||||
# every one of which executes no assertion (found by review on PR #69).
|
||||
#
|
||||
# Two measurements make the report usable, both on node 24.20. `# pass` is 0
|
||||
# for a skip-only file, a todo-only file and an empty suite. And a file that
|
||||
# registered NO test is reported as one passing test WHOSE NAME IS THE FILE
|
||||
# PATH, which is why `# tests` alone reads as the file count and a
|
||||
# minimum-count assertion over it is satisfied by the very file it should
|
||||
# reject. Measured on node 24.20.
|
||||
#
|
||||
# scripts/node-test-test.sh proves each refusal by injecting its fault.
|
||||
# reject. That naming convention is an assumption about node, so
|
||||
# scripts/node-test-test.sh pins it: if a future node stops doing it, the
|
||||
# self-test fails rather than the guard silently going quiet.
|
||||
set -uo pipefail
|
||||
|
||||
usage() {
|
||||
@@ -40,17 +46,44 @@ for pattern in "$@"; do
|
||||
files+=("${matched[@]}")
|
||||
done
|
||||
|
||||
tap="$(mktemp)"
|
||||
trap 'rm -f "$tap"' EXIT
|
||||
|
||||
# Two reporters: the readable one to the terminal, TAP to a file, so the run
|
||||
# can be read back without downgrading what a human sees.
|
||||
node --test \
|
||||
--test-reporter=spec --test-reporter-destination=stdout \
|
||||
--test-reporter=tap --test-reporter-destination="$tap" \
|
||||
"${files[@]}"
|
||||
status=$?
|
||||
[ "$status" -eq 0 ] || exit "$status"
|
||||
|
||||
passed="$(sed -n 's/^# pass \([0-9][0-9]*\)$/\1/p' "$tap" | tail -1)"
|
||||
if [ -z "$passed" ]; then
|
||||
echo "node-test: could not read a pass count out of node's TAP output" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$passed" -eq 0 ]; then
|
||||
echo "node-test: the run exited 0 with nothing passing (skipped, todo or an empty suite)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
barren=()
|
||||
for f in "${files[@]}"; do
|
||||
grep -Eq '(^|[^A-Za-z0-9_.])(test|it)(\.[A-Za-z]+)?[[:space:]]*\(' "$f" || barren+=("$f")
|
||||
awk -v want="$f" '
|
||||
/^(ok|not ok) [0-9]+ - / {
|
||||
desc = $0
|
||||
sub(/^(ok|not ok) [0-9]+ - /, "", desc)
|
||||
sub(/ # (SKIP|TODO).*$/, "", desc)
|
||||
if (desc == want) { found = 1 }
|
||||
}
|
||||
END { exit found ? 0 : 1 }
|
||||
' "$tap" && barren+=("$f")
|
||||
done
|
||||
if [ "${#barren[@]}" -gt 0 ]; then
|
||||
echo "node-test: these files declare no test, so running them asserts nothing:" >&2
|
||||
echo "node-test: these files registered no test, so running them asserted nothing:" >&2
|
||||
printf ' %s\n' "${barren[@]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
node --test "${files[@]}"
|
||||
status=$?
|
||||
[ "$status" -eq 0 ] && echo "node-test: ${#files[@]} file(s) ran"
|
||||
exit "$status"
|
||||
echo "node-test: $passed test(s) passed over ${#files[@]} file(s)"
|
||||
|
||||
Reference in New Issue
Block a user