15 Commits

Author SHA1 Message Date
Claude (dbsp-backend) a020ac669f feat(incremental): Sum and Avg through an exact sum in both backends (improvise-hrjz)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-10-10 15:14:20 -07:00
Ed L 6dbbfdb6cb chore: more updates 2026-10-10 11:52:45 -07:00
Claude (agent-adr007) 7ddc50ac62 Merge commit '62066e1e1c972218b099a36f3f10440eb93d430e' into adr007/plan
# Conflicts:
#	context/code/core.md
#	context/test-evidence/core.md
2026-09-25 10:05:02 -07:00
Edward Langley 0ae25b99c5 chore: updating missing dep 2026-09-25 01:08:55 -07:00
Claude (agent-adr007) e56148fa64 feat(core): slugify and the gensym id allocator for ADR 007 (improvise-cq1q.2)
model::id adds slugify, which case folds, turns each run of spaces into one
`_`, strips everything outside a-z, 0-9, `-` and `_`, then drops leading
underscores. It also adds IdAllocator, which keeps one increment-only counter
per document plus every id seen this session; claim admits hand-written ids
verbatim. Full case folding comes from the unicase crate. Nothing calls it yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:02:32 -07:00
Edward Langley 6d5b4e328a improvise-km71: serde back in improvise-formula for FormulaId alone
tt22 removed the formula crate's serde dependency with the legacy JSON loader,
keeping serde only where a type crosses the wire. FormulaId crosses the wire, so
it is the third such type. No derive goes back on the formula AST.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:50:53 -07:00
Edward Langley f85fddbb2f harness/opfs-contract: refresh Cargo.lock for the dropped serde features (improvise-tt22) 2026-09-23 08:13:25 -07:00
Edward Langley 1550d390cb pr-reviews re-polls after a slow tick; the harness lock fails loudly
improvise-qnpi: `pr-reviews --wait` checked its deadline before the first
re-poll, so a second boundary between two reads of the clock skipped the
only poll (exit 1 with polls=1, reproduced under CPU load). The deadline
is now checked after each poll. A new case drives it with a PATH shim
whose `date` ticks one second per read, so it fails deterministically on
the old loop rather than under load.

improvise-mkbp: harness/opfs-contract/Cargo.lock pinned pest 2.9.1 while
its path deps require 2.9.2, so every build rewrote it. The resolved lock
is committed and build.sh starts with `cargo metadata --locked`; the
wasm-pack `-- --locked` route was tried and does not work (wasm-pack
re-resolves first). opfs.spec.js now names the fix when harness/serve/opfs
is missing, which is what running web/assemble.sh alone leaves.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-22 23:36:00 -07:00
Edward Langley 836323cafa style: cargo fmt the harness crate, which --all does not reach
`harness/opfs-contract` carries its own [workspace] table, so it is not a
member of the root workspace -- deliberately, since it names
improvise_io::opfs and a member would break `cargo test --workspace`. The
consequence is that `cargo fmt --all -- --check` passes over it without
looking, so every gate I ran was honest and green while the file I had
edited was never visited.

It needs its own gate, as CI runs it:

    cd harness/opfs-contract
    cargo fmt -p opfs-contract -- --check
    cargo clippy --target wasm32-unknown-unknown --all-targets -- -D warnings

The gap is bounded: grepping for crates carrying their own [workspace]
table finds exactly this one and the root.

And a fmt failure is the FIRST step of that job, so the clippy behind it
never ran and nobody knew whether it was clean. It is -- verified locally
at this head, no warnings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
2026-09-21 15:22:47 -07:00
Edward Langley 12714c7527 feat(browser): a boot failure is a state with a reason, not silence
improvise-1wft and the five beads it blocked were one failure: nothing
carried a boot failure's REASON from the worker to the page. The channel
is one rule -- a protocol frame crosses the port as a STRING, a shell
notice as an OBJECT -- so it needs no protocol change and the
discriminator is total.

A boot failure is a STATE, not a rejected promise. A document another
context holds the exclusive handle to now OPENS read-only: measured first
(harness probe, Chromium + Firefox), getFile() reads the bytes while the
handle is held, with the second-handle refusal asserted so the probe
cannot pass vacuously.

- improvise-1wft  the channel: Notice/NoticeKind, boot_notice, take_notice
- improvise-i0v9  read-only boot over vfs::ReadOnly + OpfsFs::read_unattached
- improvise-86nw  start() rejecting for any reason now posts BootFailed
- improvise-9cma  boot::document_name -- and a/b/c.improv was ACCEPTED as
                  c.improv, not rejected, which is worse than the report
- improvise-iobc  vfs::Origin: a seed error is a notice only when the
                  document really had to be created without it
- improvise-vep8  the watchdog arms BEFORE transport.ready; its message
                  stopped claiming the connection was accepted
- improvise-7gip  DedicatedWorkerGlobalScope (needs a web-sys feature)
- improvise-zre1  a persist that fails raises a notice; the trigger is the
                  dirty flag surviving, observed rather than predicted
- improvise-7gf8  the render breaker trips once instead of retrying forever
- improvise-ociy  non-OPEN sends are loud; a viewport announcement is not
- improvise-y01k  the three ws-server drops are logged
- improvise-r4nf  serde failures throw instead of reading as "nothing to send"

Every notice routes through shell.fail, so gotcha 48's disarm half is not
re-broken. Two comments in shell.spec.js stated the opposite of the new
arming order and were corrected rather than left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
2026-09-21 14:51:34 -07:00
Edward Langley 3d3bec43e0 docs(web,harness): cap the JavaScript and the browser harness
web/app.js's transport/watchdog/frameCost arguments and web/worker.js's two
ordering rules — including the one nothing here can test, that an `async`
message listener does not serialize — are now prose in context/code/clients.md.
The harness keeps its own section: why a worker, why a report, the engine
coverage and what `wipe` deliberately cannot clear.

harness/tests/*.spec.js's comments were the record of what each assertion
proves; they are in context/test-evidence/clients.md under the exact test
titles, including the `test.fail` measurement that could NOT become a CI guard
because its trigger is the host's network stack, and the closed-invalid
improvise-xps8 report that a simulated fix appeared to confirm.

ws-server/src/main.rs's ```text fence was prose claiming the doctest
exemption, not a runnable doctest; the synopsis is now two lines and the full
form is in the prose file. No fences remain in this unit.

Verified: node --check on every JS file touched; the non-comment diff for both
'*.rs' and '*.js' is empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 10:02:03 -07:00
Edward Langley f12a2d3517 docs: reconcile the docs with the code after wave 1
An independent review checked every factual claim in the docs against the
code at 22d43b2, after eight PRs landed in one wave. This applies its
replacement text.

context/repo-map.md
  - gotcha 44 rewritten: the cache DOES frame the grid now (improvise-anr9,
    AxisFrame/FrameEntry/header_level_widths). The surviving half — the TUI
    builds at offsets zero, so a cache position is absolute — is kept,
    because four code comments cite gotcha 44 for exactly that.
  - gotcha 45: ClientFrame::Save exists (improvise-368h); MarkDirty is gone
    and dirtiness is read off EffectKind::Model(_); Rc<T>: Storage lives in
    improvise-io (improvise-hmws); OpfsFs::list joins dir (improvise-vhcz);
    OpfsFs is run by harness/ and bindings.rs is now the dark corner.
  - gotcha 47: Host::commands answers the frame whole (improvise-955n); the
    open hole is clause 4, not a per-command Checked. set_viewport "must"
    -> "does".
  - gotcha 43: the frame, not the command, is what is answered; two of three
    tiers hoist.
  - gotcha 21 widened: three effects abort, and tidy-up goes at the head.
  - gotcha 48: web/app.js is executed by harness/tests/shell.spec.js;
    improvise-m18f -> improvise-taxx (fixed); the open half is improvise-xps8.
  - gotcha 42: the never-rendered / improvise-hwpi sentence pair deleted.
  - gotcha 18: live_records and analyze_records are both decided now.
  - gotcha 1: the records columns are [coords…, +cat, measures…, +].
  - AppMode is 17 variants, ModeKey is 17, eight text-entry modes; the keymap
    table moved to improvise-protocol/src/keymap.rs (six references).
  - the Wasm section: the mirror is gone; the wasm gate covers eight crates.
  - the File Inventory regenerated rather than patched (57/79 rows were
    stale); the duplicate ws-server/host.rs collapsed to a pointer at the
    23-line shim; harness/, protocol keymap.rs/viewmodel.rs, improvise-gp
    fit.rs/dataset.rs, cmd/fit.rs, model/record.rs and the integration tests
    added.
  - the suite speed re-measured rather than re-asserted.

README.md — the four "no browser here" sites narrowed to what is still true;
17 modes; the browser section names the Playwright harness.

AGENTS.md — the dead-ID audit recipe was itself defective: two paths, a
character class blind to a commit SHA, and a two-name exclusion list. Widened,
made runnable under the worktree shell guard, and its own prose no longer
spells any id.

prolog-sessions/kb.pl — seven invariants replaced in place (they were false),
five location/3 facts re-sited plus three new, six doctrines and seven
invariants added. doctrine 48->54, invariant 44->51, decision 47 unchanged,
location 31->34. kb_consistent passes.

Comment-only Rust changes: a duplicated sentence in worker.rs, records_columns'
doc omitting NewCategory, improvise-709f2df -> commit 709f2df, and three more
copies of "OpfsFs has never been run".

cargo test --workspace: 1309 passed, 0 failed, 2 ignored — unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 00:56:33 -07:00
Edward Langley fb507b9855 fix(opfs): a directory is not a document refresh could not measure
Copilot's second round on PR #24, and it found the hole in the previous
commit's own argument. That commit said an entry `refresh` cannot measure
counts as a document, "because the one cause is another tab holding the
exclusive handle". There is a second cause, and I missed it: **`keys()`
yields directories as well as files**, and `getFileHandle` on a directory
rejects with `TypeMismatchError`. So a directory would have been listed as
a document, and one can never be attached or read — the exact picker bug
the zero-byte filter exists to remove, reintroduced by the fix for it.

The two failures now get opposite answers, which is the whole of
`holds_bytes`:

- **`getFileHandle` fails** — the name is not a file. A directory, or an
  entry removed between `keys()` and here. Not a document; skip it.
- **`getFile` fails on a handle `getFileHandle` already returned** — the
  exclusive-lock case, a second tab of the same origin. That one still
  fails open, for the reasons the previous commit gives.

Argued *and* tested this time. `a_directory_entry_is_not_a_document` in the
harness makes a real directory in OPFS — `OpfsFs` has no directory
vocabulary at all, so the harness reaches `getDirectoryHandle` itself — and
asserts it is not listed, with a real file written beside it as the
precondition, so the check cannot pass by the directory never having been
created or by `keys()` walking nothing. Proved discriminating by flipping
the `None` arm back to `true`: it fails with `refresh() listed a directory
as a document: ["a-folder"]`.

Adding it also exposed that three checks were reading the leftovers of the
check before them — the battery broke the moment one was inserted in the
middle. Every check now wipes first, so the order of the battery is not
load-bearing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 23:03:34 -07:00
Edward Langley 29962b113f test(harness): name the root check for what it asserts, and lint the crate
`open_and_refresh_see_an_empty_origin_root` claimed more than it checked:
since the previous commit an empty *entry* is not a document, so `list`
being empty means "no documents", not "no entries". A check whose name
overstates it is the shape this repo keeps finding — a green tick against a
claim nobody verified. Renamed, and `wipe`'s doc now says out loud that it
cannot clear a zero-byte entry, and why that costs nothing.

`harness/opfs-contract` is not a workspace member, so the `test` job's
workspace-wide `cargo fmt --check` and clippy never reach it. The browser
job now runs both against it — it was, in fact, unformatted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 22:46:19 -07:00
Edward Langley 7e991360df test(harness): a headless browser, and the first execution of OpfsFs
improvise-vvl3, absorbing improvise-xf1. The condition the user set when
deferring browser testing — "WE HAVE TO HAVE SOMETHING TO TEST" — has been
met: after improvise-9x6, djm, hwpi and the demo model there is a complete
self-contained application whose entire untested surface is what a harness
covers.

`python3 -m http.server` plus one Playwright project, in `harness/`:

  harness/opfs-contract/   a wasm crate that runs
                           `improvise_io::storage::contract::storage_contract`
                           against `OpfsFs` — the shared function, not a
                           transcription of it
  harness/www/opfs/        the page and the dedicated Web Worker it runs in;
                           `createSyncAccessHandle` exists nowhere else
  harness/tests/           two specs
  harness/build.sh         site + harness bundle -> harness/serve

It lives at the repo root rather than under `web/`, because `web/` is the
*source of what ships* and every file in it is copied to the site. The
harness serves the **assembled** tree (`web/assemble.sh`, extracted from
the CI workflow in this commit so there is one answer to "how is the site
arranged" rather than two) plus its own `/opfs/` — a harness that runs
against a different arrangement of files is testing something nobody
deploys. `harness/opfs-contract` is deliberately not a workspace member: it
only builds for wasm32, so a member would break `cargo test --workspace`.

What this runs that nothing has ever run:

- `storage_contract` against `OpfsFs`, at `""` **and** at `"docs"`. Two
  bugs fell straight out — improvise-vhcz's `list` path shape, whose fix
  the `"docs"` run guards (reverting it fails with `lists the one
  document`), and the zero-byte entry fix in the previous commit, which
  this found by failing on `starts empty`.
- The `keys()` async-iterator walk in `refresh`, hand-rolled through
  `js_sys::Reflect` and named by xf1 as the least certain code in the file.
  Three entries written out of order, so one passing by accident cannot
  carry it.
- `FileSystemReadWriteOptions`' `at` on read and write: a long write then a
  short one, which is where an append or a non-zero start shows up.
- `commit_deletes`, both halves — queued but still in OPFS before, gone
  after — and **across a reload**, which is the only way to ask whether
  OPFS persisted. That reload is also improvise-bck's persistence criterion
  at the storage layer.
- improvise-hwpi's three one-assertion shell tests, which that ticket said
  it could not write: status after a bad frame, status with no peer, one
  repaint on commit. Plus the first-frame watchdog, which was found by
  reading and only because someone went looking.

Check the checker, throughout, because a harness that passes against a stub
— or that silently skips because the worker never started — is exactly this
repo's characteristic defect wearing a green tick:

- `run_canary` deliberately fails, through the *same* reporting path every
  real check uses, and the spec asserts it is reported as a failure with its
  message intact.
- The spec asserts the report contains exactly the expected check names, in
  order, so a battery that quietly shrinks is a failure.
- Every shell assertion was injected and watched to fail: removing the
  bad-frame catch, the watchdog's `setTimeout`, the socket `error` listener
  and the `transport.send(frame)` on commit each fails its own test and
  only its own.
- The commit test types `20000` and asserts `20,000`, so a page that echoed
  what was typed instead of drawing what the host returned would fail.

One live bug is pinned rather than fixed. `a peer that cannot be reached`
asserts the *history* of `#status`, because improvise-m18f is real and
reproduced here: the socket's `close` listener overwrites the precise
"cannot reach" with "disconnected". That is filed and deliberately not the
same fix as the watchdog's (gotcha 48), so the test asserts both halves and
will speak up whichever one moves.

CI gets a `browser` job that can fail the build, and `pages-deploy` now
needs it. Playwright's Chromium download is ~80 MB and well under a minute
on ubuntu-latest, next to the two wasm-pack builds the job already needs —
so "impractical in this CI" does not arise. A harness nobody runs is worse
than none, because it reads as coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 22:45:03 -07:00