docs: what the harness now reaches, and the evidence for each assertion
The engine-coverage and measurement passages in context/code/clients.md, the boundary table (four boundaries become six), and README's caveat -- which named the grouped and multi-level DOM as never executed, and it now is. A fact stated in more than one place is a grep, not an edit (gotcha 48). context/test-evidence/clients.md records, per test, what fault was injected to prove it can fail: the blanked help keys, the five grid-frame injections, the 404'd casts and the dead front-door link, and the mis-wired item-row click -- plus the two-command test-name diff that found that last one. Playwright: 26 passed + 3 skipped, 34.4s / 33.7s (was 17 + 3, 15.1s); the landing page's cast playback is most of the difference. node --test web/tests 26, cargo test --workspace 1394, both unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
@@ -429,10 +429,13 @@ binary whose job is a pivot table". If sharing a TUI-hosted document to a
|
||||
remote browser ever becomes a common case rather than an occasional one, this
|
||||
is worth revisiting on that evidence.
|
||||
|
||||
Caveat, narrowed: `harness/` runs `web/app.js` under Chromium and `OpfsFs` under
|
||||
Chromium and Firefox, in CI. What is still never executed is `bindings.rs`, the
|
||||
grouped and multi-level DOM, the CSS and the `ResizeObserver` arithmetic — and
|
||||
Safari, whose Playwright build provides no OPFS at all.
|
||||
Caveat, narrowed twice: `harness/` runs `web/app.js` under Chromium — the
|
||||
standalone deployment under Firefox too — and `OpfsFs` under Chromium and
|
||||
Firefox, in CI. The grouped and multi-level DOM is executed since
|
||||
improvise-plcx, against a fixture document the harness serves beside the demo.
|
||||
What is still never executed is `bindings.rs`, the CSS and the
|
||||
`ResizeObserver` arithmetic — and Safari, whose Playwright build provides no
|
||||
OPFS at all.
|
||||
|
||||
## What's interesting about the architecture
|
||||
|
||||
|
||||
+30
-8
@@ -1253,9 +1253,21 @@ OPFS it fails rather than skipping. An unexpected skip on any other engine is a
|
||||
failure — a silently skipped check is exactly what this harness exists to
|
||||
prevent.
|
||||
|
||||
The shell spec is **Chromium-only on purpose**: it is about `web/app.js`'s own
|
||||
logic, which is engine-independent, and `routeWebSocket` plus the worker
|
||||
handshake are slower and flakier to no benefit across three.
|
||||
The shell spec's **mocked-peer** block is Chromium-only on purpose: it is about
|
||||
`web/app.js`'s own logic, which is engine-independent, and `routeWebSocket` is
|
||||
slower and flakier to no benefit across three. That argument is about those four
|
||||
tests and was never made about the two beside them: the **standalone** block
|
||||
mocks nothing, it is the deployment improvise-bck's criterion (1) names in so
|
||||
many words ("Chrome and Firefox"), and the engine's OPFS is already exercised by
|
||||
`opfs.spec.js`, so the path is known to be there. It runs on Firefox too since
|
||||
improvise-zkkw — `testMatch` widened, `grepInvert: /@mocked-peer/` — measured at
|
||||
**2.0s** for the pair and 10/10 green under `--repeat-each=5`. The split is a
|
||||
tag rather than a third copy of the file, and widening the whole file and then
|
||||
quieting what flaked would be adopting a reason nobody gave.
|
||||
|
||||
Every other spec is Chromium-only, including the three added by improvise-plcx,
|
||||
78v6 and bgtf: a fixture document, a help overlay and a landing page decide
|
||||
nothing per engine.
|
||||
|
||||
### `harness/opfs-contract`
|
||||
|
||||
@@ -1296,14 +1308,17 @@ to check that `refresh` skips one, has to happen outside it.
|
||||
**E2E and integration tests are rare and pin boundary contracts; unit tests
|
||||
assume those contracts and test between them.** (User ruling, 2026-09-20.) So
|
||||
the question is not "can this assertion be made cheaper" but "which boundary is
|
||||
it about", and the browser half has four:
|
||||
it about", and the browser half has six:
|
||||
|
||||
| boundary | the contract | where it is pinned |
|
||||
|---|---|---|
|
||||
| `RenderModel` → DOM | every field the render model projects reaches an element | `web/tests/render_surface.test.js` (statically) **and** `harness/tests/contract.spec.js` (on the screen) |
|
||||
| JS → wasm | the calls the page makes exist, with the arity the bundle declares, and their answers are painted | `web/tests/bundle/wasm_surface.test.js` (the surface) **and** `contract.spec.js`'s pointer test (the wiring) |
|
||||
| page → peer | the handshake goes out first, a frame is answered whole, a failure is named, **and the page routes each failure through the call that disarms the watchdog** | `harness/tests/shell.spec.js`, `harness/tests/drill.spec.js` |
|
||||
| `help::pages()` → DOM | the overlay's rows are the shared content — the keys off the keymap and the words beside them — and not the right number of blank rows | `harness/tests/help.spec.js` |
|
||||
| worker → OPFS | `storage_contract`, run against a real `OpfsFs` | `harness/tests/opfs.spec.js` |
|
||||
| `AxisFrame` → DOM | the grid's *frame* — header levels, group spans, the collapse indicator and the group row that carries no cells — is drawn as the frame says, not slash-joined | `harness/tests/grouped.spec.js`, over `harness/fixtures/grouped.improv`; `web/tests/grid_plan.test.js` pins the plan it draws from |
|
||||
| the site → the app | the landing page's casts are really fetched and really play, and its front door reaches `./app/` | `harness/tests/landing.spec.js` |
|
||||
|
||||
**A unit test of a rule does not test the wiring that reaches it**, and the
|
||||
sharpest case in this file is the watchdog. `shell.test.js` proves that
|
||||
@@ -1399,12 +1414,19 @@ trap below). Two runs each; the pair is the variance.
|
||||
|---|---|---|---|---|
|
||||
| Playwright, before (`1af51c9`) | 23 | 26 passed + 3 skipped | 36.2s / 36.3s | ~4m30s in CI |
|
||||
| Playwright, after | 14 | 17 passed + 3 skipped | 15.1s / 14.9s | ~4m30s in CI |
|
||||
| Playwright, + wave 4 | 21 | 26 passed + 3 skipped | 34.4s / 33.7s | ~4m30s in CI |
|
||||
| `node --test web/tests` | — | 26 | 0.12s | seconds |
|
||||
|
||||
**Two columns because one number was ambiguous three times.** Every spec but
|
||||
`opfs.spec.js` is Chromium-only, so a count taken off a run is three larger than
|
||||
the count of tests in the files (Firefox's OPFS battery) with three more skipped
|
||||
(WebKit's). The columns must also **agree**: chromium + 3 is the passed total,
|
||||
The last row is improvise-78v6/igxp/plcx/bgtf/zkkw: seven more Chromium tests
|
||||
and two Firefox ones. **Most of the 19 seconds is one test** — the landing
|
||||
page's four asciinema casts are played until each terminal draws, which is the
|
||||
only assertion that can tell a cast from a 404 (evidence file). The Firefox pair
|
||||
is 2.0s of it.
|
||||
|
||||
**Two columns because one number was ambiguous three times.** Firefox runs
|
||||
`opfs.spec.js` and the standalone half of `shell.spec.js` and every other spec
|
||||
is Chromium-only, so a count taken off a run is five larger than the chromium
|
||||
count, with three more skipped (WebKit's OPFS battery). The columns must also **agree**: chromium + 3 is the passed total,
|
||||
and a row where they do not is a row built from two different trees.
|
||||
`npx playwright test --list` prints the per-project truth; quote which one, and
|
||||
this is the same "two numbers, not one" the Rust suite learned in
|
||||
|
||||
@@ -1703,3 +1703,125 @@ not the text, so a cell drawn blank passed. The assertion was strengthened until
|
||||
the injection failed, which is the only reason it is worth having — and it is
|
||||
the same lesson as the eleven tests above: a class is a flag, and a flag is not
|
||||
a pixel.
|
||||
|
||||
---
|
||||
|
||||
## `harness/tests/help.spec.js` — the overlay, asserted as words
|
||||
|
||||
PR #61 deleted `"? opens help, and its rows are the shared ones"` and nothing
|
||||
replaced it, so `paintHelp`, `help`, `help_page` and `help_titles` reached no
|
||||
assertion at all (improvise-78v6). What `contract.spec.js` carried in its place
|
||||
said only that a `dt` was VISIBLE — and an empty `dt` is a visible `dt`
|
||||
(improvise-igxp), which is the same shape as the caret bug above: a row is a
|
||||
frame, and a frame is not a word.
|
||||
|
||||
**The injection both tickets name**: blank `keys.textContent` in `paintHelp`,
|
||||
patched in `harness/serve/app/app.js`. Before this work it left all three
|
||||
contract tests passing. It now fails both tests in this file (the Welcome
|
||||
banner reads empty; the advertised key reads empty) **and** `contract.spec.js`,
|
||||
whose two help lines were strengthened from `toBeVisible` to the banner's text
|
||||
and the first `dt.help-keys`'s text.
|
||||
|
||||
**What the pairs helper is for.** The overlay is a `<dl>`, and the association
|
||||
between a key row and its description is document order — `dt` then `dd`. Read
|
||||
as two lists they zip wrong, because a prose row emits **no** `dd` at all
|
||||
(deliberately: an empty one would leave a blank grid cell). Welcome is prose end
|
||||
to end, so `dd` count zero there is an assertion about that rule and not a
|
||||
coincidence.
|
||||
|
||||
#### "a key the overlay advertises does what the overlay says it does"
|
||||
|
||||
The assertions above are still about a string on a screen; this one asks whether
|
||||
the string is **true**. The key for `Jump to last row` is read OUT OF THE DOM,
|
||||
the overlay closed, the key pressed, and the selection asserted onto the last
|
||||
row. A page drawing a plausible key it does not have — what a hand-spelled table
|
||||
decays into — fails here, and the spec holds no second copy of the keymap.
|
||||
|
||||
It presses the FIRST key the row names because a row is a compaction of every
|
||||
binding that reaches the command (`h j k l ← ↑ → ↓` is eight).
|
||||
|
||||
## `harness/tests/grouped.spec.js` — the first browser run of improvise-anr9
|
||||
|
||||
`web/demo.improv` has no groups and one category per visible axis, deliberately
|
||||
(gotcha 46), so the multi-level header, the group span, the collapse indicator
|
||||
and the inline group row — exactly the code improvise-anr9 added — had never
|
||||
been rendered anywhere a test could see (improvise-plcx). The fixture is
|
||||
`harness/fixtures/grouped.improv`, served beside the shipped demo by
|
||||
`harness/build.sh`, and the shipped document is unchanged.
|
||||
|
||||
**A missing seed is not an error**: `web/worker.js` answers a fetch that 404s
|
||||
with an empty model. So a typo in the filename yields a page that draws a grid,
|
||||
and "a grid is drawn" would pass on it. Every assertion here is about labels the
|
||||
fixture really carries.
|
||||
|
||||
**Five injections, five distinct failures**, patched in `harness/serve/` so no
|
||||
rebuild stands between the fault and the run:
|
||||
|
||||
- `gridPlan` slash-joins the level tuples into one line (the exact defect anr9
|
||||
fixed) — fails the header-levels test and the span test;
|
||||
- `th.colSpan = 1` on every column group header — fails the span test alone;
|
||||
- a stray `tr.insertCell()` on the inline group row — fails the group-row test
|
||||
alone, which is the DOM half of `RenderRow::Group` carrying no cells;
|
||||
- `indicator(false)` hard-coded — fails on `▶ Upland` while `▼ Inland` still
|
||||
passes, which is why the fixture has one collapsed group and one open one;
|
||||
- `collapsed: Region/Upland` removed from the served document — fails on the
|
||||
rows that reappear, so the assertions see the collapse and not just the glyph.
|
||||
|
||||
**A bug this fixture surfaced rather than predicted.** `Coastal` is the row
|
||||
frame's FIRST entry and its header is never drawn: the spec asserts **two**
|
||||
group rows where the document has three, with improvise-0o6x named on the line.
|
||||
That count is a record of an open bug, not an endorsement — when 0o6x lands it
|
||||
becomes 3 and this line is where the change must be argued.
|
||||
|
||||
## `harness/tests/landing.spec.js` — the front door
|
||||
|
||||
`docs/index.html` is copied to the site root by `web/assemble.sh`, so the
|
||||
harness has been serving it all along with nothing asserted about it
|
||||
(improvise-bgtf).
|
||||
|
||||
**Counting the four `.ap-player` elements proves nothing**, and that is the
|
||||
whole reason this is a spec rather than a one-liner: `autoPlay` is false, so
|
||||
asciinema-player fetches no cast until play is pressed. A page whose every cast
|
||||
path 404s constructs four players, throws no page error, and looks identical to
|
||||
a working one. Injected while the page was being written (`./casts/` ->
|
||||
`./cats/`) the construction-only check passed on the broken page.
|
||||
|
||||
Re-injected into `harness/serve/index.html` now, the spec fails at the first
|
||||
player: `.ap-line` elements exist only once there is a cast to lay out. The spec
|
||||
also asserts, BEFORE pressing anything, that no `.cast` has been requested —
|
||||
without which the network evidence below could be satisfied by a page that
|
||||
fetched them eagerly, which is not the page we ship.
|
||||
|
||||
Two assertions where one looks sufficient: the rendering says a cast played, and
|
||||
the recorded responses say **four distinct** casts answered 200. Three players
|
||||
showing the same recording satisfies the first and fails the second.
|
||||
|
||||
`"the front door reaches the app it advertises"` follows the footer link rather
|
||||
than asserting its text, because whether `./app/` resolves is a fact about the
|
||||
assembled tree. Injecting `href="./aplication/"` fails it.
|
||||
|
||||
## `harness/tests/contract.spec.js` — the ninth injection, and a lost guard
|
||||
|
||||
**The lost guard.** improvise-78v6 asks for the test-name diff over PR #61's
|
||||
range, and it finds a fourth loss beyond the help test: `"a click on an item row
|
||||
filters, as Enter does"`. `contract.spec.js` asserts the CATEGORY arm of
|
||||
`click_category_row` (the row expands) and nothing asserts the ITEM arm, whose
|
||||
rule is "do what Enter does here" — filter. Only `#status` distinguishes them,
|
||||
so a click wired to the wrong arm passed the whole file. Restored inside the
|
||||
click test, and proved by injecting `click_category(row.is_category ? index :
|
||||
0)`, which fails that line alone.
|
||||
|
||||
The diff that finds this is two commands, and is worth running on any PR whose
|
||||
subject is a lost guard:
|
||||
|
||||
```sh
|
||||
M=$(gh pr view 61 --json mergeCommit -q .mergeCommit.oid)
|
||||
for r in "$M^1" "$M"; do
|
||||
git ls-tree -r --name-only "$r" -- harness/tests web/tests |
|
||||
while read -r f; do git show "$r:$f" | grep -oE 'test\("[^"]*"'; done
|
||||
done
|
||||
```
|
||||
|
||||
The other two losses are deliberate: `pointer.spec.js`'s sidebar test was
|
||||
superseded by the open-at-launch ruling, and the widgets file was merged into
|
||||
this one.
|
||||
|
||||
Reference in New Issue
Block a user