fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #24 from fiddlerwoaroof/wave/j-browser-harness

Browse Source 
WU-J: a headless browser harness, and the first execution of OpfsFs (improvise-vvl3, improvise-vhcz, improvise-hmws; improvise-xf1 stays open for Safari).

storage_contract had been cited for months as the written-down spec OpfsFs must satisfy in a browser. The first time OpfsFs was handed to it, it failed on its first assertion.
This commit is contained in:
Ed Langley
2026-09-19 23:44:15 -07:00
committed by GitHub
parent 391599476b 4abadb8e6d
commit 8528843341
20 changed files with 2146 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+79 -44
View File
@@ -74,15 +74,83 @@ jobs:
nix shell nixpkgs#trealla --command tpl prolog-sessions/kb.pl \
-g "(kb_consistent -> halt(0) ; (kb_report, halt(1)))"
# ---------------------------------------------------------------------------
# The browser harness (improvise-vvl3, absorbing improvise-xf1).
#
# This is the only job that RUNS any of the browser half. `OpfsFs`,
# `web/app.js` and `web/worker.js` were previously compile-checked,
# syntax-checked and read — never executed — so every bug in them was found
# by reading or by a person pressing keys, neither of which is repeatable.
#
# It can fail the build, deliberately: a harness nobody runs is worse than
# none, because it reads as coverage. Playwright's browser download is ~80 MB
# and takes well under a minute on ubuntu-latest, which is affordable next to
# the two wasm-pack builds this job already needs.
browser:
name: Browser harness (OPFS contract + shell)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
# Named rather than assumed: this job lints harness/opfs-contract,
# which the `test` job's workspace-wide fmt and clippy cannot reach.
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Install wasm-pack
env:
WASM_PACK_VERSION: v0.15.0
run: |
set -eu
url="https://github.com/rustwasm/wasm-pack/releases/download/${WASM_PACK_VERSION}/wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl.tar.gz"
curl -sSfL "$url" | tar -xz --strip-components=1 -C /usr/local/bin \
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
wasm-pack --version
# Not a workspace member (it only builds for wasm32), so the `test` job's
# workspace-wide fmt and clippy do not reach it.
- name: Lint the harness crate
working-directory: harness/opfs-contract
run: |
cargo fmt --all -- --check
cargo clippy --target wasm32-unknown-unknown --all-targets -- -D warnings
- name: Install the harness
working-directory: harness
run: npm ci
# All three, because improvise-xf1's acceptance criterion names Chrome,
# Firefox and Safari and `createSyncAccessHandle` is implemented
# separately by each engine. WebKit is installed even though its OPFS is
# absent today: the spec skips on a runtime probe, so the day that build
# gains OPFS the check starts running without a CI change.
- name: Install browsers
working-directory: harness
run: npx playwright install --with-deps chromium firefox webkit
# Builds both site bundles, the harness's own wasm bundle, and assembles
# `harness/serve` — the same tree `pages-build` publishes, plus /opfs/.
- name: Build the harness
run: ./harness/build.sh
- name: Run the harness
working-directory: harness
run: npx playwright test
- if: failure()
uses: actions/upload-artifact@v4
with:
name: playwright-traces
path: harness/test-results
retention-days: 7
# ---------------------------------------------------------------------------
# GitHub Pages delivery (improvise-d31).
#
# Two jobs on purpose. `pages-build` runs on every push and pull request, so
# a change that breaks `web/build.sh` or the bundle fails on the PR that made
# it — the build is a gate. `pages-deploy` runs only on a push to main and
# `needs` the two check jobs above, so nothing that failed `cargo test` or the
# wasm32 gate can reach the published site. The deploy depends on those checks
# rather than repeating them (improvise-d31 AC 2).
# `needs` the check jobs above, so nothing that failed `cargo test`, the
# wasm32 gate or the browser harness can reach the published site. The deploy
# depends on those checks rather than repeating them (improvise-d31 AC 2).
#
# Not path-filtered, deliberately. The bundles are `improvise-wasm-client` and
# `improvise-worker-server` plus, between them, most of the workspace — the
@@ -123,47 +191,14 @@ jobs:
- name: Build the wasm bundle
run: ./web/build.sh
# `web/` is the source directory: it also holds build.sh and .gitignore,
# which have no business being served. Assemble what ships.
# `web/assemble.sh`, not an inline copy of it: the browser harness job
# below assembles the same tree, and a harness that runs against a
# different arrangement of files is testing something nobody deploys.
- name: Assemble the site
run: |
set -eu
mkdir -p site
cp web/index.html web/app.js web/style.css web/worker.js site/
# The document the page opens on a browser that has never seen
# one. web/worker.js fetches it by the ?seed= URL and hands the
# bytes to Rust, which writes them through the VFS — so leaving
# it out is an empty grid, not a 404 the page could report.
cp web/demo.improv site/
cp -rf web/pkg site/pkg
cp -rf web/worker-pkg site/worker-pkg
rm -f site/pkg/.gitignore site/pkg/package.json
rm -f site/worker-pkg/.gitignore site/worker-pkg/package.json
# Cache-bust the two generated bundles. Pages serves everything
# `max-age=600` and the wasm-pack filenames carry no content hash,
# so for ten minutes after a deploy a returning visitor runs the
# PREVIOUS build -- which during improvise-9x6 debugging made a
# shipped fix look like it had not shipped, twice. The specifiers
# are static import strings, so stamping the copies in site/ is
# enough; the tracked sources are left alone and a local
# `http.server` is unaffected.
# Both halves, because a fresh glue file does NOT pull a fresh
# .wasm: the glue locates it with
# `new URL('..._bg.wasm', import.meta.url)`, and a URL constructed
# against a base does not inherit the base's query string. Stamping
# only the import specifier would refresh the wrapper and keep the
# stale binary -- the failure mode this is here to end.
sed -i "s|\./pkg/improvise_wasm_client\.js|&?v=${GITHUB_SHA}|" site/app.js
sed -i "s|\./worker-pkg/improvise_worker_server\.js|&?v=${GITHUB_SHA}|" site/worker.js
sed -i "s|'improvise_wasm_client_bg\.wasm'|'improvise_wasm_client_bg.wasm?v=${GITHUB_SHA}'|" \
site/pkg/improvise_wasm_client.js
sed -i "s|'improvise_worker_server_bg\.wasm'|'improvise_worker_server_bg.wasm?v=${GITHUB_SHA}'|" \
site/worker-pkg/improvise_worker_server.js
grep -n "?v=" site/app.js site/worker.js \
site/pkg/improvise_wasm_client.js \
site/worker-pkg/improvise_worker_server.js
find site -type f | sort
du -sh site
run: ./web/assemble.sh site "$GITHUB_SHA"
- run: find site -type f | sort
- run: du -sh site
- uses: actions/upload-pages-artifact@v3
with:
@@ -171,7 +206,7 @@ jobs:
pages-deploy:
name: Deploy to GitHub Pages
needs: [test, wasm, pages-build]
needs: [test, wasm, browser, pages-build]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
crates/improvise-io/Cargo.toml
+2
View File
@@ -29,6 +29,8 @@ wasm-bindgen-futures = "0.4"
[target.'cfg(target_arch = "wasm32")'.dependencies.web-sys]
version = "0.3"
features = [
"Blob",
"File",
"FileSystemDirectoryHandle",
"FileSystemFileHandle",
"FileSystemGetFileOptions",
crates/improvise-io/src/opfs.rs
+127 -13
View File
@@ -20,13 +20,26 @@
//! [`OpfsFs::refresh`], [`OpfsFs::commit_deletes`]. The worker calls
//! these from its message loop, where awaiting is free.
//! - **sync, via the trait**: `read`/`write` use an already-attached
//! handle; `list` answers from the index `refresh` populated; `delete`
//! drops the document from the index and queues the removal for
//! `commit_deletes`.
//! handle; `list` answers from an index; `delete` drops the document from
//! that index and queues the removal for `commit_deletes`.
//!
//! A `delete` is therefore visible to `read` and `list` immediately and
//! lands in OPFS at the end of the batch. That ordering is the price of
//! keeping the trait synchronous, and it is invisible to a single worker.
//!
//! # The index holds documents, not entries
//!
//! `Storage::list` promises *document* names, and the two stopped being the
//! same thing the moment `attach` had to create a file to get a handle for a
//! document that does not exist yet (improvise-vvl3). So three writers, and
//! the split between them is the whole rule:
//!
//! - [`OpfsFs::attach`] indexes **nothing**. What it creates is zero bytes.
//! - [`Storage::write`] indexes the name, and un-queues any pending delete
//! of it. A written document is the first thing there is to list.
//! - [`OpfsFs::refresh`] rebuilds the index from OPFS, skipping zero-byte
//! entries — and counting an entry it cannot measure as a document, since
//! the one cause of that is another tab holding the exclusive handle.
use std::cell::RefCell;
use std::collections::HashMap;
@@ -59,6 +72,20 @@ fn entry_name(path: &Path) -> Result<String> {
.ok_or_else(|| anyhow!("{} is not a usable document name", path.display()))
}
/// The size of a file, or `None` if it cannot be read — which, on a handle
/// `getFileHandle` already returned, means somebody else holds the exclusive
/// sync access handle. See [`OpfsFs::holds_bytes`].
async fn file_size(file: &web_sys::FileSystemFileHandle) -> Option<f64> {
Some(
JsFuture::from(file.get_file())
.await
.ok()?
.dyn_into::<web_sys::File>()
.ok()?
.size(),
)
}
pub struct OpfsFs {
root: FileSystemDirectoryHandle,
/// Sync access handles for attached documents. Exclusive per file:
@@ -98,6 +125,17 @@ impl OpfsFs {
/// Acquire the sync access handle for one document, creating the file
/// if it is not there. Must be awaited before `read` or `write` names
/// that document.
///
/// **It does not index what it created**, and that is the whole of
/// improvise-9x6's first browser failure read the other way round. The
/// file it creates is zero bytes, and a zero-byte entry is not a
/// document — `Storage::list` promises *document* names. Indexing here
/// made a fresh origin list `demo.improv` before anything had been
/// written to it, so the seed was skipped as "already there" and
/// `load_from` got an empty buffer. `vfs::holds_a_document` works around
/// that by reading each candidate back; this removes the need.
/// [`Self::write`] is what puts a name in the index, because a written
/// document is the first thing there is to list.
pub async fn attach(&self, path: &Path) -> Result<()> {
let name = entry_name(path)?;
if let Some(old) = self.handles.borrow_mut().remove(&name) {
@@ -115,12 +153,7 @@ impl OpfsFs {
.map_err(|e| js_err(&format!("createSyncAccessHandle({name})"), e))?
.dyn_into::<FileSystemSyncAccessHandle>()
.map_err(|_| anyhow!("createSyncAccessHandle({name}) did not yield a handle"))?;
self.handles.borrow_mut().insert(name.clone(), handle);
let mut index = self.index.borrow_mut();
if !index.contains(&name) {
index.push(name);
index.sort();
}
self.handles.borrow_mut().insert(name, handle);
Ok(())
}
@@ -135,6 +168,12 @@ impl OpfsFs {
}
/// Re-read the directory listing that `list` answers from.
///
/// Zero-byte entries are skipped: `Storage::list` promises *document*
/// names, and [`Self::attach`] creates an empty file for a document that
/// does not exist yet. Listing those made a browser offer an entry that
/// cannot be opened (improvise-vvl3 found it by running
/// `storage_contract` here for the first time).
pub async fn refresh(&self) -> Result<()> {
let mut names = Vec::new();
let iter = js_sys::Reflect::get(&self.root, &"keys".into())
@@ -164,7 +203,9 @@ impl OpfsFs {
.ok()
.and_then(|v| v.as_string())
{
names.push(name);
if self.holds_bytes(&name).await {
names.push(name);
}
}
}
names.sort();
@@ -186,6 +227,47 @@ impl OpfsFs {
Ok(())
}
/// Whether one entry is a document with content in it.
///
/// Read off the sync access handle when this instance has one, and only
/// otherwise through the promise-based API, because a file under an
/// exclusive sync access handle is not reliably readable that way.
///
/// Two ways the measurement can fail, and they get opposite answers —
/// which is the whole of the function:
///
/// - **`getFileHandle` fails**: the name is not a file. `keys()` yields
/// directories as well as files, and a directory rejects with
/// `TypeMismatchError`; an entry removed between `keys()` and here
/// rejects too. Neither is a document, and neither can ever be
/// `attach`ed or `read`, so listing one would be the same picker bug
/// this filter exists to remove (found by Copilot on PR #24).
/// - **`getFile` fails on a real file handle**: another context of the
/// same origin holds the exclusive sync access handle — two tabs of the
/// same site. An entry somebody else has open is certainly a document,
/// so this one **fails open**. Dropping it would hide a document that
/// exists, and failing the whole `refresh` would let a second tab break
/// the first one's picker. Over-reporting costs one row, the same
/// direction `Dirty::of` is deliberately sloppy in.
async fn holds_bytes(&self, name: &str) -> bool {
if let Some(handle) = self.handles.borrow().get(name) {
return handle.get_size().is_ok_and(|size| size > 0.0);
}
match self.file_handle(name).await {
None => false,
Some(file) => file_size(&file).await.is_none_or(|size| size > 0.0),
}
}
/// The file handle for `name`, or `None` if it does not name a file.
async fn file_handle(&self, name: &str) -> Option<web_sys::FileSystemFileHandle> {
JsFuture::from(self.root.get_file_handle(name))
.await
.ok()?
.dyn_into::<web_sys::FileSystemFileHandle>()
.ok()
}
fn with_handle<T>(
&self,
path: &Path,
@@ -229,11 +311,43 @@ impl Storage for OpfsFs {
.map_err(|e| js_err("write()", e))?;
handle.flush().map_err(|e| js_err("flush()", e))?;
Ok(())
})
})?;
// A written document is the first thing there is to list. `attach`
// deliberately does not do this — see its doc comment.
let name = entry_name(path)?;
// Writing a name back un-queues its removal. Otherwise a delete
// followed by a write of the same document would be listed and
// readable, and then removed from OPFS by `commit_deletes` at the
// end of the batch — the one ordering gotcha 28's deferral can get
// wrong.
self.pending_deletes.borrow_mut().retain(|n| *n != name);
let mut index = self.index.borrow_mut();
if !index.contains(&name) {
index.push(name);
index.sort();
}
Ok(())
}
fn list(&self, _dir: &Path) -> Result<Vec<PathBuf>> {
Ok(self.index.borrow().iter().map(PathBuf::from).collect())
/// Entries in the origin root, presented under `dir`.
///
/// The join is not cosmetic (improvise-vhcz). `MemoryFs` and `PhysicalFs`
/// answer `list` with `dir`-joined paths — paths a caller can hand
/// straight back to `read` — and this used to answer with bare names. At
/// `Path::new("")` the two shapes coincide, which is the only `dir` the
/// worker ever passes, so the disagreement was invisible: a worker that
/// organised documents under `docs/` would have got a **full picker in
/// every host test and an empty one in a browser**, and nothing would have
/// said why.
///
/// `dir` is decoration rather than a filter because this backend is flat
/// by construction: [`entry_name`] reduces every path to its file name, so
/// `docs/a.improv` and `a.improv` are the same OPFS entry, and an entry
/// name may not contain a separator at all. That is the trait's own
/// "backends that are not trees treat `dir` as a name prefix" — the doc
/// was right and the impl was wrong.
fn list(&self, dir: &Path) -> Result<Vec<PathBuf>> {
Ok(self.index.borrow().iter().map(|n| dir.join(n)).collect())
}
fn delete(&self, path: &Path) -> Result<()> {
crates/improvise-io/src/storage.rs
+133 -33
View File
@@ -30,6 +30,107 @@ pub trait Storage {
fn delete(&self, path: &Path) -> Result<()>;
}
/// The behaviour every backend owes, as a function rather than as a test.
///
/// It reads like a test helper and it was one — `#[cfg(test)]` in this file,
/// run against `MemoryFs` and `PhysicalFs` so they could not drift apart. The
/// one backend that most needed it could never be handed to it: `OpfsFs` is
/// `cfg(target_arch = "wasm32")` and its sync access handles exist in no host
/// harness, so the contract described it only in a doc comment. Moving it out
/// of `cfg(test)` is what lets a browser run **this** sequence rather than a
/// second transcription of it (improvise-vvl3/xf1) — a second copy of a
/// contract is the disease this codebase keeps curing.
///
/// It returns `Result` rather than asserting, because a failure has to travel
/// out of a Web Worker as a string; the host tests `.unwrap()` it and get the
/// same message.
pub mod contract {
use std::path::Path;
use anyhow::{Result, ensure};
use super::Storage;
/// Write / read / replace / list-sorted / delete / read-fails /
/// delete-twice-errors / shorter-write-truncates, at `dir`.
///
/// `dir` is a parameter and not a constant because the three backends
/// must agree at **every** `dir`, not only at the flat root the browser
/// happens to use (improvise-vhcz). A backend whose `list` answered in a
/// different path shape would pass at `Path::new("")` — where joining is
/// the identity — and fail here at anything else.
pub fn storage_contract(fs: &dyn Storage, dir: &Path) -> Result<()> {
let a = dir.join("a.improv");
let b = dir.join("b.improv");
ensure!(fs.list(dir)?.is_empty(), "starts empty");
fs.write(&a, b"first")?;
ensure!(fs.read(&a)? == b"first", "reads back what was written");
ensure!(fs.list(dir)? == vec![a.clone()], "lists the one document");
fs.write(&a, b"second")?;
ensure!(fs.read(&a)? == b"second", "write replaces");
ensure!(fs.list(dir)? == vec![a.clone()], "no duplicate entry");
fs.write(&b, b"other")?;
ensure!(fs.list(dir)? == vec![a.clone(), b.clone()], "sorted");
fs.delete(&a)?;
ensure!(fs.read(&a).is_err(), "deleted document is unreadable");
ensure!(
fs.list(dir)? == vec![b.clone()],
"delete drops it from list"
);
ensure!(fs.delete(&a).is_err(), "deleting twice is an error");
// A shorter document must not leave the tail of a longer one.
fs.write(&b, b"0123456789")?;
fs.write(&b, b"xy")?;
ensure!(fs.read(&b)? == b"xy", "a shorter write truncates");
fs.delete(&b)?;
ensure!(fs.list(dir)?.is_empty(), "ends as it started");
Ok(())
}
}
/// A shared backend is a backend.
///
/// This exists because of a seam that only a browser has, and it lives here
/// because of the orphan rule (improvise-hmws). `OpfsFs` keeps four genuinely
/// async operations — `open`, `attach`, `refresh`, `commit_deletes` — off the
/// trait, deliberately, so that [`Storage`] can stay synchronous (gotcha 28).
/// But `App::storage` is a `Box<dyn Storage>`, and a trait object can name
/// none of the four: **handing the backend to the `App` puts the async seam
/// permanently out of reach**. The caller therefore needs to hold the concrete
/// backend *and* give the `App` a `Storage`, which means sharing one.
///
/// `impl Storage for Rc<OpfsFs>` is refused in any *other* crate — two foreign
/// types — so improvise-9x6 wrote a forwarding newtype in the worker, and
/// every future worker would have written it again. Here the trait is local,
/// so the impl is allowed, and it is blanket rather than `OpfsFs`-specific
/// because nothing about it is OPFS-shaped: it is the general statement that a
/// shared handle to a backend is usable as that backend.
///
/// `Rc` and not `Arc`: the one caller is a Web Worker, which is a single
/// thread, and `wasm32-unknown-unknown` has no others. An `Arc` impl would be
/// speculative — add it when something needs it.
impl<T: Storage + ?Sized> Storage for std::rc::Rc<T> {
fn read(&self, path: &Path) -> Result<Vec<u8>> {
(**self).read(path)
}
fn write(&self, path: &Path, bytes: &[u8]) -> Result<()> {
(**self).write(path, bytes)
}
fn list(&self, dir: &Path) -> Result<Vec<PathBuf>> {
(**self).list(dir)
}
fn delete(&self, path: &Path) -> Result<()> {
(**self).delete(path)
}
}
/// The real filesystem. What the native build wires up.
#[derive(Debug, Default, Clone, Copy)]
pub struct PhysicalFs;
@@ -140,47 +241,46 @@ impl Storage for MemoryFs {
mod tests {
use super::*;
/// The behaviour every backend owes, exercised against each host
/// backend so they cannot drift apart. `OpfsFs` has to satisfy the same
/// sequence in a browser, where this harness cannot reach it — so this
/// doubles as the written-down contract for it (improvise-i34).
fn storage_contract(fs: &dyn Storage, dir: &Path) {
let a = dir.join("a.improv");
let b = dir.join("b.improv");
assert!(fs.list(dir).unwrap().is_empty(), "starts empty");
fs.write(&a, b"first").unwrap();
assert_eq!(fs.read(&a).unwrap(), b"first");
assert_eq!(fs.list(dir).unwrap(), vec![a.clone()]);
fs.write(&a, b"second").unwrap();
assert_eq!(fs.read(&a).unwrap(), b"second", "write replaces");
assert_eq!(fs.list(dir).unwrap(), vec![a.clone()], "no duplicate entry");
fs.write(&b, b"other").unwrap();
assert_eq!(fs.list(dir).unwrap(), vec![a.clone(), b.clone()], "sorted");
fs.delete(&a).unwrap();
assert!(fs.read(&a).is_err(), "deleted document is unreadable");
assert_eq!(fs.list(dir).unwrap(), vec![b.clone()]);
assert!(fs.delete(&a).is_err(), "deleting twice is an error");
// A shorter document must not leave the tail of a longer one.
fs.write(&b, b"0123456789").unwrap();
fs.write(&b, b"xy").unwrap();
assert_eq!(fs.read(&b).unwrap(), b"xy");
}
use super::contract::storage_contract;
#[test]
fn memory_fs_satisfies_the_storage_contract() {
storage_contract(&MemoryFs::new(), Path::new("docs"));
storage_contract(&MemoryFs::new(), Path::new("docs")).unwrap();
}
#[test]
fn physical_fs_satisfies_the_storage_contract() {
let dir = tempfile::tempdir().unwrap();
storage_contract(&PhysicalFs, dir.path());
storage_contract(&PhysicalFs, dir.path()).unwrap();
}
/// improvise-hmws: a worker must hold the concrete backend (for the four
/// async methods the trait does not carry) *and* hand the `App` a
/// `Storage`. Before the blanket impl above, `Rc<OpfsFs>` was not a
/// `Storage` and could not be made one outside this crate — the orphan
/// rule — so `improvise-worker-server` wrote a forwarding newtype, and so
/// would every worker after it.
///
/// The guard is the **signature**: this does not compile without the
/// impl. What it asserts beyond compiling is the part a newtype could
/// still get wrong — that the clone the `App` holds and the handle the
/// message loop keeps are one backend, not two.
#[test]
fn a_shared_backend_is_a_backend_and_a_clone_is_the_same_one() {
use std::rc::Rc;
let fs = Rc::new(MemoryFs::new());
storage_contract(&Rc::clone(&fs), Path::new("docs")).unwrap();
let boxed: Box<dyn Storage> = Box::new(Rc::clone(&fs));
boxed
.write(Path::new("shared.improv"), b"through the box")
.unwrap();
assert_eq!(
fs.read(Path::new("shared.improv")).unwrap(),
b"through the box",
"the App's copy and the loop's copy must be one backend"
);
}
#[test]
crates/improvise-worker-server/src/bindings.rs
+18 -45
View File
@@ -5,9 +5,11 @@
//! is told lives in [`crate::worker`], and everything that could be wrong
//! about *which document* is open lives in [`crate::vfs`]. Both are driven by
//! host tests. What is left here is awaiting four promises and moving strings,
//! and it is the only part of this crate that no test in the repository can
//! reach — there is no browser here and `createSyncAccessHandle` exists in no
//! harness.
//! and it is the only part of this crate that no test reaches — `harness/`
//! now runs `OpfsFs` itself in a real browser (improvise-vvl3), but it drives
//! the backend directly and never instantiates this shim, so the four
//! `await`s below and the `#[wasm_bindgen]` surface around them are still
//! read rather than run.
//!
//! # The async seam, and the `Rc` it forces
//!
@@ -21,11 +23,15 @@
//! `App::storage` is a `Box<dyn Storage>`, and the trait has none of those four
//! methods, so **handing `OpfsFs` to the `App` puts the async seam out of
//! reach**. A boxed backend can be read and written and never attached again.
//! The fix is local and small — keep an `Rc<OpfsFs>` for the async half and
//! give the `App` a newtype over a clone of it — but it cannot live in
//! `improvise-io`'s callers generically, because `impl Storage for Rc<OpfsFs>`
//! is two foreign types and the orphan rule refuses it. Hence [`Shared`].
//! Reported as a VFS gap rather than routed around silently.
//! So the worker keeps an `Rc<OpfsFs>` for the async half and gives the `App` a
//! clone of it for the sync half.
//!
//! That clone is a `Storage` because `improvise-io` says so
//! (`impl<T: Storage> Storage for Rc<T>`, improvise-hmws). This module used to
//! carry a forwarding newtype, because `impl Storage for Rc<OpfsFs>` *here* is
//! two foreign types and the orphan rule refuses it — and so would every
//! worker written after this one. In `improvise-io` the trait is local and the
//! impl is allowed, so the seam costs a `Rc::clone` and nothing else.
//!
//! # `attach` before anything
//!
@@ -36,42 +42,17 @@
//! that is the seam being explicit rather than a lazy `attach` hidden inside a
//! synchronous call, which could not await anything anyway.
use std::path::{Path, PathBuf};
use std::path::PathBuf;
use std::rc::Rc;
use anyhow::Result;
use improvise_command::ui::app::App;
use improvise_io::opfs::OpfsFs;
use improvise_io::storage::Storage;
use wasm_bindgen::prelude::*;
use crate::vfs::{self, Opening};
use crate::worker::Worker;
/// An `OpfsFs` the worker and the `App` both hold.
///
/// The `App` needs a `Box<dyn Storage>` while the message loop needs the
/// backend's four async methods, and the trait carries neither the methods nor
/// a way to downcast back. A local newtype over the shared pointer is the
/// smallest thing that gives both; `Rc` rather than `Arc` because a worker is
/// one thread and `wasm32-unknown-unknown` has no others.
struct Shared(Rc<OpfsFs>);
impl Storage for Shared {
fn read(&self, path: &Path) -> Result<Vec<u8>> {
self.0.read(path)
}
fn write(&self, path: &Path, bytes: &[u8]) -> Result<()> {
self.0.write(path, bytes)
}
fn list(&self, dir: &Path) -> Result<Vec<PathBuf>> {
self.0.list(dir)
}
fn delete(&self, path: &Path) -> Result<()> {
self.0.delete(path)
}
}
/// The worker's handle for the lifetime of the page.
#[wasm_bindgen]
pub struct WorkerServer {
@@ -121,9 +102,9 @@ async fn boot(document: &str, seed: Option<&[u8]>) -> Result<WorkerServer> {
Some(bytes) => Opening::named(path.clone()).with_seed(bytes),
None => Opening::named(path.clone()),
};
let workbook = vfs::open(&Shared(Rc::clone(&storage)), &opening)?;
let workbook = vfs::open(&*storage, &opening)?;
let mut app = App::with_storage(workbook, Some(path), Box::new(Shared(Rc::clone(&storage))));
let mut app = App::with_storage(workbook, Some(path), Box::new(Rc::clone(&storage)));
app.rebuild_layout();
Ok(WorkerServer {
worker: Worker::new(app),
@@ -159,7 +140,7 @@ impl WorkerServer {
/// (improvise-i34): there is no OS file manager behind this.
#[wasm_bindgen]
pub fn documents(&self) -> Result<Vec<String>, JsError> {
vfs::documents(&Shared(Rc::clone(&self.storage)))
vfs::documents(&*self.storage)
.map(|paths| {
paths
.iter()
@@ -175,11 +156,3 @@ impl WorkerServer {
self.storage.refresh().await.map_err(report)
}
}
/// Assert the shim is reachable at all. Not a test of behaviour — there is
/// nothing here to test without a browser — but it does keep the module from
/// rotting out of the wasm build unnoticed.
#[allow(dead_code)]
fn _shim_is_a_storage(fs: Rc<OpfsFs>) -> Box<dyn Storage> {
Box::new(Shared(fs))
}
crates/improvise-worker-server/src/lib.rs
+10 -5
View File
@@ -33,11 +33,16 @@
//! [`bindings`] is the shim, and it is deliberately thin enough to read: it
//! awaits `OpfsFs::open`/`attach`/`refresh`, builds an `App` through
//! [`vfs::open`], and forwards strings. It is compile-checked for wasm32 and
//! nothing more — `createSyncAccessHandle` exists in no test harness here, and
//! browser verification (improvise-xf1) is deferred by decision, so the
//! `storage_contract` that `OpfsFs` owes has still never been run against it
//! (gotcha 28). **If persistence misbehaves in a browser, `OpfsFs` is the
//! first place to look, not the last.**
//! nothing more — `createSyncAccessHandle` exists in no *host* harness.
//!
//! **What `OpfsFs` owes is no longer unrun.** `harness/` (improvise-vvl3)
//! executes `improvise_io::storage::contract::storage_contract` against it
//! inside a real dedicated Web Worker, under Chromium and Firefox, and the
//! browser CI job can fail the build. Two bugs fell out of the first run.
//! Still uncovered: Safari, because Playwright's WebKit build provides no
//! OPFS at all; and `bindings.rs` itself, which the harness does not
//! instantiate. So if persistence misbehaves in a browser, `OpfsFs` is a much
//! better-lit place to look than it was, and this shim is the dark one.
//!
//! # Nothing depends on this crate
//!
crates/improvise-worker-server/src/vfs.rs
+41 -26
View File
@@ -41,13 +41,18 @@
//!
//! [`root`] is `Path::new("")`, so a document name is a bare `demo.improv`
//! whose `parent()` is the root. `OpfsFs` flattens by construction — it stores
//! one entry per document in the origin's root directory and its `list`
//! returns bare names — so the flat root is the one `dir` at which `OpfsFs`
//! and `MemoryFs` answer `list` identically, and therefore the one at which
//! `storage_contract` describes both. See `the_flat_root_is_where_the_backends
//! _agree`, which is the closest a host-side test can get to exercising the
//! browser backend (improvise-xf1 is deferred, so `OpfsFs` remains
//! compile-checked and not browser-tested — gotcha 28).
//! one entry per document in the origin's root directory, and an OPFS entry
//! name may not contain a separator at all — so `docs/a.improv` and
//! `a.improv` are the *same* entry there and different ones in `MemoryFs`.
//! The flat root is the one `dir` at which that collapse cannot be observed.
//!
//! It used to be load-bearing for a second, sharper reason: `OpfsFs::list`
//! answered with bare names where the host backends answer with `dir`-joined
//! ones, so the flat root was the only `dir` at which `list` agreed at all.
//! That is fixed (improvise-vhcz) and the contract now describes all three
//! everywhere; `the_flat_root_is_where_the_backends_agree` below pins the
//! shape this worker relies on, and `harness/` runs the contract itself
//! against `OpfsFs` in a real browser (improvise-vvl3).
use std::path::{Path, PathBuf};
@@ -115,18 +120,24 @@ pub fn open(storage: &dyn Storage, opening: &Opening<'_>) -> Result<Workbook> {
/// Whether the backend holds a *document* under `path` — not merely an entry.
///
/// The distinction is the whole of improvise-9x6's first browser failure, and
/// it exists because `list` answers "is there an entry" while this asks "is
/// there something to load". `OpfsFs::attach` must create the file to obtain a
/// sync access handle for a document that does not exist yet, and it indexes
/// what it created — so by the time `boot` gets here, a **fresh origin already
/// lists `demo.improv`**, as a zero-byte file. Asking `documents` alone then
/// skips the seed and hands `load_from` an empty buffer, which is the
/// `Cannot open demo.improv` the deployed page reported.
/// The distinction is the whole of improvise-9x6's first browser failure.
/// `OpfsFs::attach` must create the file to obtain a sync access handle for a
/// document that does not exist yet, and it used to **index what it created**
/// — so a fresh origin already listed `demo.improv` as a zero-byte file,
/// `documents` said it was there, the seed was skipped, and `load_from` got an
/// empty buffer: the `Cannot open demo.improv` the deployed page reported.
///
/// `MemoryFs` has no `attach` — it is not on `Storage` — so no host test could
/// reach this, and the test below builds the empty entry by hand to stand in
/// for what `attach` does.
/// That cause is gone (improvise-vvl3): `attach` indexes nothing and `refresh`
/// skips zero-byte entries, so `list` names documents rather than entries.
/// This check is kept as the second half of the same rule — `list` answers
/// "is there an entry the backend will admit to" and this asks "is there
/// something to load" — and it is now belt and braces rather than the
/// mechanism. Removing it is a separate argument, since a backend that has
/// not been told the rule would put the bug straight back.
///
/// `MemoryFs` has no `attach` — it is not on `Storage` — so no host test can
/// reach the browser path, and the test below builds the empty entry by hand
/// to stand in for what `attach` used to leave behind.
fn holds_a_document(storage: &dyn Storage, path: &Path) -> Result<bool> {
if !documents(storage)?.iter().any(|p| p == path) {
return Ok(false);
@@ -171,12 +182,17 @@ mod tests {
/// test could see it.
///
/// `OpfsFs::attach` creates the file — it must, to get a sync access
/// handle for a document that does not exist yet — and indexes what it
/// created. `boot` attaches *before* it opens, so on a fresh origin
/// handle for a document that does not exist yet — and used to index what
/// it created. `boot` attaches *before* it opens, so on a fresh origin
/// `documents()` already contained `demo.improv` as a **zero-byte entry**,
/// the seed was skipped as "already there", and `load_from` got an empty
/// buffer: `Cannot open demo.improv`.
///
/// `OpfsFs` no longer indexes an attached-but-unwritten entry
/// (improvise-vvl3, asserted in `harness/` against a real browser), so
/// that cause is gone. This stays as the guard for any backend that lists
/// an entry it cannot load.
///
/// `MemoryFs` has no `attach` (it is not a `Storage` method), so the
/// seeding tests above passed for a reason that cannot hold in a browser.
/// This builds the empty entry by hand to stand in for it.
@@ -272,11 +288,10 @@ mod tests {
);
}
/// `documents` is `Storage::list` at [`root`], and the flat root is the
/// one `dir` at which `OpfsFs` — which stores bare names in the origin
/// root — and the host backends answer alike. Nothing here can reach a
/// browser (improvise-xf1 is deferred), so this pins the *shape* the
/// worker relies on rather than the backend.
/// `documents` is `Storage::list` at [`root`], and a flat root is what
/// keeps `OpfsFs`'s collapse of `dir` unobservable: it stores one entry
/// per document in the origin root. This pins the *shape* the worker
/// relies on; the backend itself is exercised by `harness/`.
#[test]
fn the_flat_root_is_where_the_backends_agree() {
let fs = MemoryFs::new();
@@ -285,7 +300,7 @@ mod tests {
assert_eq!(
documents(&fs).unwrap(),
vec![PathBuf::from("a.improv"), PathBuf::from("b.improv")],
"sorted bare names, exactly what OpfsFs::list returns"
"sorted names under the flat root, exactly what OpfsFs::list answers"
);
assert_eq!(
Path::new("a.improv").parent(),
harness/.gitignore
+8
View File
@@ -0,0 +1,8 @@
# Built by build.sh; never committed.
serve/
www/opfs/pkg/
# Playwright
node_modules/
test-results/
playwright-report/
harness/build.sh
Executable
+27
View File
@@ -0,0 +1,27 @@
#!/bin/sh
# Build everything the browser harness serves, into harness/serve/.
#
# harness/serve/ the assembled site, exactly as `web/assemble.sh`
# produces it for GitHub Pages
# harness/serve/opfs/ this harness's own page, worker and wasm bundle
#
# One server root, so Playwright runs one `python3 -m http.server`. The site
# half is the *deployed* arrangement rather than `web/` itself, because a
# harness that runs against a different tree from the one that ships is
# testing something nobody deploys (improvise-vvl3).
set -eu
root=$(cd "$(dirname "$0")/.." && pwd)
serve="$root/harness/serve"
"$root/web/build.sh" >/dev/null
"$root/web/assemble.sh" "$serve" harness >/dev/null
wasm-pack build "$root/harness/opfs-contract" \
--target web --out-dir "$root/harness/www/opfs/pkg"
mkdir -p "$serve/opfs"
cp -f "$root/harness/www/opfs/index.html" "$root/harness/www/opfs/worker.js" "$serve/opfs/"
cp -rf "$root/harness/www/opfs/pkg" "$serve/opfs/pkg"
rm -f "$serve/opfs/pkg/.gitignore" "$serve/opfs/pkg/package.json"
echo "harness/serve ready"
harness/opfs-contract/Cargo.lock
Generated
+622
View File
@@ -0,0 +1,622 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "android_system_properties"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae221649c9976a6f6c56ae1facf410f3ddb33cc661c4b7b61020a912d4237fbc"
dependencies = [
"libc",
]
[[package]]
name = "anyhow"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "54413ede23c2daf518f35156dfde027feb2374004d63bd497f983c8db9c0e313"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
[[package]]
name = "chrono"
version = "0.4.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327"
dependencies = [
"iana-time-zone",
"js-sys",
"num-traits",
"serde",
"wasm-bindgen",
"windows-link",
]
[[package]]
name = "core-foundation-sys"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "crc32fast"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
dependencies = [
"cfg-if",
]
[[package]]
name = "csv"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938"
dependencies = [
"csv-core",
"itoa",
"ryu",
"serde_core",
]
[[package]]
name = "csv-core"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782"
dependencies = [
"memchr",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "find-msvc-tools"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef25905e51abafe4dcea6c15fec58c57b601cdbd0ee53d22ea1d3016c587d39b"
[[package]]
name = "flate2"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
name = "futures-core"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
[[package]]
name = "futures-task"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
[[package]]
name = "futures-util"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "iana-time-zone"
version = "0.1.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
dependencies = [
"android_system_properties",
"core-foundation-sys",
"iana-time-zone-haiku",
"js-sys",
"log",
"wasm-bindgen",
"windows-core",
]
[[package]]
name = "iana-time-zone-haiku"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
dependencies = [
"cc",
]
[[package]]
name = "improvise-core"
version = "0.1.0-rc2"
dependencies = [
"anyhow",
"improvise-formula",
"indexmap",
"serde",
"siphasher",
]
[[package]]
name = "improvise-formula"
version = "0.1.0-rc2"
dependencies = [
"anyhow",
"pest",
"pest_derive",
"serde",
]
[[package]]
name = "improvise-io"
version = "0.1.0-rc2"
dependencies = [
"anyhow",
"chrono",
"csv",
"flate2",
"improvise-core",
"improvise-formula",
"indexmap",
"js-sys",
"pest",
"pest_derive",
"serde",
"serde_json",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
]
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.105"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e"
dependencies = [
"cfg-if",
"futures-util",
"wasm-bindgen",
]
[[package]]
name = "libc"
version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "log"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "miniz_oxide"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "opfs-contract"
version = "0.0.0"
dependencies = [
"anyhow",
"improvise-io",
"js-sys",
"serde",
"serde_json",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
]
[[package]]
name = "pest"
version = "2.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d45aeb61b4bf818e12d4205f2466f8c4748f85f4fce0146d1c03d69d753f0ad"
dependencies = [
"memchr",
"ucd-trie",
]
[[package]]
name = "pest_derive"
version = "2.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "89cc5a242e25ed4e7704d0be240f2cfbe20a8c27e7e252d94835be93d92dc39f"
dependencies = [
"pest",
"pest_generator",
]
[[package]]
name = "pest_generator"
version = "2.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7abf21475cc3820fe4b2ca2dc2142902f67a02189f3b5b3a229f4febc01a43e5"
dependencies = [
"pest",
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "pest_meta"
version = "2.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adba4db388f687393c18c51348d44a41d870ca9df71a2c98172ea3035dc6936e"
dependencies = [
"pest",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "ryu"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "siphasher"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649"
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "ucd-trie"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2896d95c02a80c6d6a5d6e953d479f5ddf2dfdb6a244441010e373ac0fb88971"
[[package]]
name = "unicode-ident"
version = "1.0.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
[[package]]
name = "wasm-bindgen"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.78"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ef4c5d3d2cdf5c54f4231181768f5510842e350db025faf1f7163b1030ed928"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 3.0.6",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.128"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e"
dependencies = [
"unicode-ident",
]
[[package]]
name = "web-sys"
version = "0.3.105"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fbddc4a036f00ec4f18c83445bd3115cb306a91da554919a099d9222fe4a7f8"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "windows-core"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
dependencies = [
"windows-link",
]
[[package]]
name = "zlib-rs"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
harness/opfs-contract/Cargo.toml
+45
View File
@@ -0,0 +1,45 @@
[package]
name = "opfs-contract"
version = "0.0.0"
edition = "2024"
description = "Runs improvise-io's storage_contract against OpfsFs in a real browser"
license = "Apache-2.0"
publish = false
# Deliberately NOT a workspace member. It builds only for wasm32 — it names
# `improvise_io::opfs`, which is `cfg(target_arch = "wasm32")` — so a member
# would break `cargo test --workspace` and `cargo clippy --workspace`. An empty
# `[workspace]` table makes it its own root, which is what keeps it out without
# editing the root manifest (and without a Cargo.toml conflict with every other
# agent).
[workspace]
[lib]
crate-type = ["cdylib", "rlib"]
[dependencies]
improvise-io = { path = "../../crates/improvise-io" }
anyhow = "1"
js-sys = "0.3"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.4"
# Only in order to CREATE a directory in OPFS, which `OpfsFs` never does.
# `keys()` yields directories alongside files and one must not be listed as a
# document; without this the fix for that would be argued rather than tested.
[dependencies.web-sys]
version = "0.3"
features = [
"FileSystemDirectoryHandle",
"FileSystemGetDirectoryOptions",
"FileSystemRemoveOptions",
"Navigator",
"StorageManager",
"WorkerGlobalScope",
"WorkerNavigator",
]
[package.metadata.wasm-pack.profile.release]
wasm-opt = ["-O", "--enable-bulk-memory", "--enable-nontrapping-float-to-int"]
harness/opfs-contract/src/lib.rs
+417
View File
@@ -0,0 +1,417 @@
//! Run `improvise_io::storage::contract::storage_contract` against `OpfsFs`,
//! inside a dedicated Web Worker, in a real browser.
//!
//! `OpfsFs` has been compile-checked and never run (gotcha 45). This is the
//! instrument that changes that (improvise-vvl3, absorbing improvise-xf1). It
//! is a *harness* crate, not part of the application: it ships in no bundle
//! and is not a workspace member.
//!
//! # Why a worker
//!
//! `FileSystemFileHandle.createSyncAccessHandle()` exists only in a dedicated
//! Web Worker, and it is the whole reason `Storage` could stay synchronous
//! (gotcha 28). A check that ran on the main thread would fail for a reason
//! that has nothing to do with `OpfsFs`.
//!
//! # Why a report rather than a panic
//!
//! Every check runs and its outcome is reported, so a failure names *which*
//! step and the rest still run. More importantly it is what lets the spec
//! check the checker: Playwright asserts the report contains exactly the
//! checks it expects, so a worker that never started, or a build that dropped
//! a check, is a failure rather than a silent pass.
use std::path::Path;
use anyhow::{Result, anyhow, ensure};
use improvise_io::opfs::OpfsFs;
use improvise_io::storage::Storage;
use improvise_io::storage::contract::storage_contract;
use serde::Serialize;
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;
/// One named check and what happened to it.
#[derive(Serialize)]
struct Check {
name: String,
ok: bool,
detail: String,
}
#[derive(Default)]
struct Report(Vec<Check>);
impl Report {
async fn run<F>(&mut self, name: &str, f: F)
where
F: AsyncFnOnce() -> Result<String>,
{
let (ok, detail) = match f().await {
Ok(detail) => (true, detail),
Err(e) => (false, format!("{e:#}")),
};
self.0.push(Check {
name: name.to_string(),
ok,
detail,
});
}
fn finish(self) -> String {
serde_json::to_string(&self.0).expect("a Vec<Check> serializes")
}
}
/// The origin's root directory, reached the way `OpfsFs::open` reaches it.
///
/// Only the harness needs this. `OpfsFs` has no directory vocabulary at all —
/// it stores one flat entry per document — so making a directory, to check
/// that `refresh` skips one, has to happen outside it.
async fn opfs_root() -> Result<web_sys::FileSystemDirectoryHandle> {
let scope = js_sys::global()
.dyn_into::<web_sys::WorkerGlobalScope>()
.map_err(|_| anyhow!("not in a dedicated Web Worker"))?;
JsFuture::from(scope.navigator().storage().get_directory())
.await
.map_err(|e| anyhow!("getDirectory(): {e:?}"))?
.dyn_into::<web_sys::FileSystemDirectoryHandle>()
.map_err(|_| anyhow!("getDirectory() did not yield a directory handle"))
}
async fn make_directory(name: &str) -> Result<()> {
let opts = web_sys::FileSystemGetDirectoryOptions::new();
opts.set_create(true);
JsFuture::from(
opfs_root()
.await?
.get_directory_handle_with_options(name, &opts),
)
.await
.map_err(|e| anyhow!("getDirectoryHandle({name}): {e:?}"))?;
Ok(())
}
async fn drop_directory(name: &str) -> Result<()> {
let opts = web_sys::FileSystemRemoveOptions::new();
opts.set_recursive(true);
JsFuture::from(opfs_root().await?.remove_entry_with_options(name, &opts))
.await
.map_err(|e| anyhow!("removeEntry({name}): {e:?}"))?;
Ok(())
}
/// Remove every *document* in the origin root, so a run does not inherit the
/// last one's leftovers.
///
/// Uses only the backend's own vocabulary — `refresh`, `delete`,
/// `commit_deletes` — rather than reaching for `removeEntry` directly, so a
/// broken `delete` shows up here as well as in the contract.
///
/// It therefore cannot clear a zero-byte entry, because `list` no longer
/// names one. That is deliberate and costs nothing: `attach` recreates and
/// `write` fills whatever a previous run left behind, and a leftover empty
/// entry is invisible to every assertion here for the same reason it is
/// invisible to a document picker.
async fn wipe(fs: &OpfsFs) -> Result<usize> {
fs.refresh().await?;
let names = fs.list(Path::new(""))?;
for name in &names {
fs.delete(name)?;
}
fs.commit_deletes().await?;
fs.refresh().await?;
ensure!(
fs.list(Path::new(""))?.is_empty(),
"wipe left {:?} behind",
fs.list(Path::new(""))?
);
Ok(names.len())
}
/// The whole battery. Returns a JSON array of `Check`.
#[wasm_bindgen]
pub async fn run_checks() -> Result<String, JsError> {
let mut report = Report::default();
// `open()` is the first thing that can fail, and everything after it needs
// the handle, so it is not a `Check` — a failure here is a thrown error.
let fs = OpfsFs::open()
.await
.map_err(|e| JsError::new(&format!("OpfsFs::open(): {e:#}")))?;
wipe(&fs)
.await
.map_err(|e| JsError::new(&format!("wipe: {e:#}")))?;
report
.run(
"open_and_refresh_see_an_origin_root_holding_no_documents",
async || {
fs.refresh().await?;
let listed = fs.list(Path::new(""))?;
ensure!(listed.is_empty(), "a wiped root listed {listed:?}");
Ok("empty".into())
},
)
.await;
// `attach` must create the file to obtain a sync access handle for a
// document that does not exist yet — and that created file is zero bytes,
// which is NOT a document. `Storage::list` promises document names, so an
// attached-but-unwritten name must not be listed. Indexing it is exactly
// the improvise-9x6 browser failure (the seed was skipped as "already
// there" and `load_from` got an empty buffer).
report
.run(
"an_attached_but_unwritten_entry_is_not_a_document",
async || {
wipe(&fs).await?;
fs.attach(Path::new("attached.improv")).await?;
let listed = fs.list(Path::new(""))?;
ensure!(
listed.is_empty(),
"attach listed {listed:?} before anything was written"
);
// ...and a second instance, which shares no index, must not see
// it either: the filter has to be in `refresh`, not only in the
// omission from `attach`.
let other = OpfsFs::open().await?;
other.refresh().await?;
let seen = other.list(Path::new(""))?;
ensure!(
seen.is_empty(),
"refresh() listed the empty entry: {seen:?}"
);
// The handle is real, though: writing through it works, and then
// it IS a document.
fs.write(Path::new("attached.improv"), b"now it is one")?;
ensure!(
fs.list(Path::new(""))? == vec![std::path::PathBuf::from("attached.improv")],
"a written document was not listed"
);
Ok("created, unlisted until written".into())
},
)
.await;
// `keys()` yields DIRECTORIES as well as files, and `getFileHandle` on a
// directory rejects with `TypeMismatchError`. Treating that as "could not
// measure, so assume a document" would list an entry that can never be
// attached or read — the same picker bug the zero-byte filter removes
// (found by Copilot on PR #24). `OpfsFs` never creates a directory, so
// the harness makes one itself.
report
.run("a_directory_entry_is_not_a_document", async || {
wipe(&fs).await?;
make_directory("a-folder").await?;
let fresh = OpfsFs::open().await?;
fresh.refresh().await?;
let listed = fresh.list(Path::new(""))?;
ensure!(
listed.is_empty(),
"refresh() listed a directory as a document: {listed:?}"
);
// The precondition, so this cannot pass by the directory never
// having been created: a real file beside it IS listed, which
// also says `keys()` walked the root at all.
fs.attach(Path::new("beside.improv")).await?;
fs.write(Path::new("beside.improv"), b"a real document")?;
let again = OpfsFs::open().await?;
again.refresh().await?;
let seen = again.list(Path::new(""))?;
ensure!(
seen == vec![std::path::PathBuf::from("beside.improv")],
"the file beside the directory was not listed alone: {seen:?}"
);
drop_directory("a-folder").await?;
Ok("skipped".into())
})
.await;
// The hand-rolled `keys()` async-iterator walk in `refresh` is the least
// certain code in opfs.rs (improvise-xf1 named it). One entry could pass
// by accident; several, out of order, cannot.
report
.run("refresh_walks_every_key_and_sorts_them", async || {
// Wipes first, like every other check. A battery whose steps
// depend on the leftovers of the step before breaks when one is
// inserted, which is exactly what the directory check above did.
wipe(&fs).await?;
for name in ["c.improv", "a.improv", "b.improv"] {
fs.attach(Path::new(name)).await?;
fs.write(Path::new(name), name.as_bytes())?;
}
let fresh = OpfsFs::open().await?;
fresh.refresh().await?;
let listed = fresh.list(Path::new(""))?;
let names: Vec<String> = listed.iter().map(|p| p.display().to_string()).collect();
ensure!(
names == ["a.improv", "b.improv", "c.improv"],
"refresh() saw {names:?}"
);
Ok(format!("{names:?}"))
})
.await;
// `FileSystemReadWriteOptions`' `at` handling, the other thing xf1 called
// out. A `write` that appended instead of overwriting, or a `read` that
// started anywhere but 0, is invisible on a single short write and obvious
// across a long one followed by a short one.
report
.run("read_and_write_are_absolute_at_offset_zero", async || {
wipe(&fs).await?;
let p = Path::new("offsets.improv");
fs.attach(p).await?;
fs.write(p, b"0123456789ABCDEF")?;
ensure!(
fs.read(p)? == b"0123456789ABCDEF",
"long write did not round-trip"
);
fs.write(p, b"xy")?;
let short = fs.read(p)?;
ensure!(
short == b"xy",
"a shorter write left {:?} behind — truncate or `at` is wrong",
String::from_utf8_lossy(&short)
);
Ok("absolute".into())
})
.await;
// The point of the whole exercise. `attach` is hoisted out because
// acquiring a handle is the async half and `Storage::write` cannot await;
// the contract itself is the shared function, not a transcription of it.
for dir in ["", "docs"] {
report
.run(&format!("storage_contract_at_{:?}", dir), async || {
wipe(&fs).await?;
let d = Path::new(dir);
fs.attach(&d.join("a.improv")).await?;
fs.attach(&d.join("b.improv")).await?;
storage_contract(&fs, d)?;
Ok("passed".into())
})
.await;
}
// Gotcha 28: a delete is visible to `read` and `list` at once and lands in
// OPFS at the end of the batch. Both halves, because either one alone
// would pass against a backend that got the timing exactly backwards.
report
.run(
"a_delete_is_visible_at_once_and_lands_on_commit",
async || {
wipe(&fs).await?;
let p = Path::new("doomed.improv");
fs.attach(p).await?;
fs.write(p, b"bytes")?;
fs.delete(p)?;
ensure!(fs.read(p).is_err(), "a deleted document was still readable");
ensure!(
fs.list(Path::new(""))?.is_empty(),
"a deleted document was still listed"
);
let before = OpfsFs::open().await?;
before.refresh().await?;
ensure!(
before.list(Path::new(""))?.iter().any(|q| q == p),
"the entry left OPFS before commit_deletes — the queue is not a queue"
);
fs.commit_deletes().await?;
let after = OpfsFs::open().await?;
after.refresh().await?;
ensure!(
!after.list(Path::new(""))?.iter().any(|q| q == p),
"commit_deletes did not remove the entry"
);
Ok("queued then removed".into())
},
)
.await;
Ok(report.finish())
}
/// Write the state the reload check expects, then hand control back.
///
/// Split from [`after_reload`] because the one thing no in-page check can
/// establish is that OPFS *persisted* — and a reload is the only way to ask.
#[wasm_bindgen]
pub async fn before_reload() -> Result<String, JsError> {
inner_before().await.map_err(js)
}
async fn inner_before() -> Result<String> {
let fs = OpfsFs::open().await?;
wipe(&fs).await?;
let survivor = Path::new("survivor.improv");
fs.attach(survivor).await?;
fs.write(survivor, b"persisted across a reload")?;
let doomed = Path::new("doomed.improv");
fs.attach(doomed).await?;
fs.write(doomed, b"should not survive")?;
fs.delete(doomed)?;
fs.commit_deletes().await?;
Ok("staged".into())
}
/// Assert what [`before_reload`] left is still there, in a fresh page.
#[wasm_bindgen]
pub async fn after_reload() -> Result<String, JsError> {
inner_after().await.map_err(js)
}
async fn inner_after() -> Result<String> {
let fs = OpfsFs::open().await?;
fs.refresh().await?;
let listed = fs.list(Path::new(""))?;
let names: Vec<String> = listed.iter().map(|p| p.display().to_string()).collect();
ensure!(
names == ["survivor.improv"],
"after a reload the origin held {names:?}"
);
let survivor = Path::new("survivor.improv");
fs.attach(survivor).await?;
let bytes = fs.read(survivor)?;
ensure!(
bytes == b"persisted across a reload",
"the surviving document read back as {:?}",
String::from_utf8_lossy(&bytes)
);
Ok("survived".into())
}
fn js(e: anyhow::Error) -> JsError {
JsError::new(&format!("{e:#}"))
}
/// Deliberately fails, so the harness can prove it can see a failure.
///
/// A green run against a stub, or one that silently skipped because the worker
/// never started, is exactly the check-the-checker failure this repo has a
/// doctrine about. The spec runs this and asserts it is reported as a failure
/// with its message intact, which exercises the *same* reporting path every
/// real check uses.
#[wasm_bindgen]
pub async fn run_canary() -> String {
let mut report = Report::default();
report
.run("the_canary_must_fail", async || {
Err(anyhow!("the harness can see a failure"))
})
.await;
report.finish()
}
harness/package-lock.json
Generated
+76
View File
@@ -0,0 +1,76 @@
{
"name": "improvise-harness",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "improvise-harness",
"devDependencies": {
"@playwright/test": "1.56.1"
}
},
"node_modules/@playwright/test": {
"version": "1.56.1",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.56.1.tgz",
"integrity": "sha512-vSMYtL/zOcFpvJCW71Q/OEGQb7KYBPAdKh35WNSkaZA75JlAO8ED8UN6GUNTm3drWomcbcqRPFqQbLae8yBTdg==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.56.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.56.1",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.56.1.tgz",
"integrity": "sha512-aFi5B0WovBHTEvpM3DzXTUaeN6eN0qWnTkKx4NQaH4Wvcmc153PdaY2UBdSYKaGYw+UyWXSVyxDUg5DoPEttjw==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.56.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.56.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.56.1.tgz",
"integrity": "sha512-hutraynyn31F+Bifme+Ps9Vq59hKuUCz7H1kDOcBs+2oGguKkWTU50bBWrtz34OUWmIwpBTWDxaRPXrIXkgvmQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}
harness/package.json
+13
View File
@@ -0,0 +1,13 @@
{
"name": "improvise-harness",
"private": true,
"type": "module",
"description": "The first instrument pointed at the browser half of improvise (improvise-vvl3)",
"scripts": {
"build": "./build.sh",
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "1.56.1"
}
}
harness/playwright.config.js
+77
View File
@@ -0,0 +1,77 @@
import { defineConfig, devices } from "@playwright/test";
// `python3 -m http.server` and one spec directory. Deliberately small
// (improvise-vvl3): the point is to be able to run `storage_contract` against
// `OpfsFs` and to read `#status` out of a real page, not to grow a second
// test framework beside `cargo test`.
//
// # This is the one place a line of web/*.js is executed
//
// It is not only the OPFS harness. Before this there was no JavaScript test
// infrastructure in the repository at all — no package.json, no runner,
// nothing in CI running a single line of `web/app.js` or `web/worker.js` — so
// every bug in them was found by reading (gotcha 48) and the bug-first rule
// could not be honoured for a change to either. PR #21 declined to add
// regression tests for its close-listener fix on exactly that ground, and
// that decline is only honest because those assertions live here instead:
// see `tests/shell.spec.js`.
//
// So a change to `web/*.js` belongs in this spec directory, not in a second
// ad-hoc surface beside it — two answers to one question is the defect this
// codebase keeps curing. Playwright is the right vehicle rather than a
// headless-DOM runner (jsdom and friends) because the things that go wrong in
// these files are a real Worker, a real WebSocket, real OPFS and real event
// ordering, none of which a DOM shim has.
//
// The server root is `harness/serve`, built by `build.sh` — the assembled
// site plus this harness's own page. A module script will not load from
// `file://`, so an http server is not optional.
export default defineConfig({
testDir: "./tests",
fullyParallel: false,
workers: 1,
forbidOnly: !!process.env.CI,
retries: 0,
reporter: process.env.CI ? [["github"], ["list"]] : "list",
timeout: 60_000,
use: {
baseURL: "http://127.0.0.1:8099",
trace: "retain-on-failure",
},
// improvise-xf1's acceptance criterion names Chrome, Firefox and Safari,
// and the OPFS spec is the reason: `createSyncAccessHandle` is the one API
// this whole design rests on (gotcha 28), its sync read/write/truncate
// signatures were added later than the handle itself, and each engine
// implemented it separately. Chromium alone would be one engine's opinion.
//
// The shell spec is Chromium-only on purpose — it is about `web/app.js`'s
// own logic, which is engine-independent, and `routeWebSocket` plus the
// worker handshake are slower and flakier to no benefit across three.
//
// Result today: Chromium and Firefox run the whole battery and pass.
// **Playwright's WebKit provides no OPFS at all** — `getDirectory()` itself
// rejects with `UnknownError` — so Safari is NOT covered and xf1's
// criterion is met for two engines of three. The project is kept rather
// than deleted so that it starts running the day that build gains OPFS;
// `opfs.spec.js` skips on a runtime probe, and an unexpected skip on any
// other engine is a failure.
projects: [
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
{
name: "firefox",
use: { ...devices["Desktop Firefox"] },
testMatch: /opfs\.spec\.js/,
},
{
name: "webkit",
use: { ...devices["Desktop Safari"] },
testMatch: /opfs\.spec\.js/,
},
],
webServer: {
command: "python3 -m http.server 8099 --bind 127.0.0.1 --directory serve",
url: "http://127.0.0.1:8099/index.html",
reuseExistingServer: !process.env.CI,
timeout: 30_000,
},
});
harness/tests/opfs.spec.js
+102
View File
@@ -0,0 +1,102 @@
// `storage_contract` against `OpfsFs`, in a browser (improvise-vvl3/xf1).
//
// `OpfsFs` has been compile-checked and never run. Everything found in a
// browser so far was found by a person pressing keys, which is not a
// repeatable check. This is.
//
// The checks themselves are in Rust (harness/opfs-contract) because that is
// where `OpfsFs` is and where the contract is; this file drives them and reads
// the report. Assertions here are about *the report*, deliberately, so a
// worker that never started or a build that dropped a check fails rather than
// passing quietly.
import { test, expect } from "@playwright/test";
/** Run one entry point in the harness worker. */
async function run(page, cmd) {
return JSON.parse(await page.evaluate((c) => window.opfs(c), cmd));
}
/** The checks the report must contain, in order. Named here as well as in
* Rust on purpose: a check deleted on one side has to be deleted on both,
* which is what stops the battery quietly shrinking. */
const EXPECTED = [
"open_and_refresh_see_an_origin_root_holding_no_documents",
"an_attached_but_unwritten_entry_is_not_a_document",
"a_directory_entry_is_not_a_document",
"refresh_walks_every_key_and_sorts_them",
"read_and_write_are_absolute_at_offset_zero",
'storage_contract_at_""',
'storage_contract_at_"docs"',
"a_delete_is_visible_at_once_and_lands_on_commit",
];
test.describe("OpfsFs in a real browser", () => {
test.beforeEach(async ({ page, browserName }, testInfo) => {
await page.goto("/opfs/index.html");
// improvise-xf1's acceptance criterion names Chrome, Firefox and Safari.
// Chromium and Firefox run the whole battery. Playwright's WebKit build
// provides **no OPFS at all** — `navigator.storage.getDirectory()` itself
// rejects with `UnknownError` — so there is nothing there for `OpfsFs` to
// be right or wrong about, and Safari stays uncovered. See the hand-back
// and `playwright.config.js`.
//
// The skip is conditional on the PROBE, not on the browser name, so it
// cannot quietly swallow a real regression: if Chromium or Firefox ever
// stops providing OPFS, this fails rather than skipping. A silently
// skipped check is exactly the failure this harness exists to prevent, so
// an unexpected skip also fails.
const opfs = await page.evaluate(async () => {
try {
await navigator.storage.getDirectory();
return "yes";
} catch (err) {
return String(err?.name ?? err);
}
});
if (opfs !== "yes") {
expect(
browserName,
`${browserName} lost OPFS (${opfs}) — that is a regression, not a known gap`,
).toBe("webkit");
testInfo.skip(true, `this WebKit build provides no OPFS (${opfs})`);
}
});
// Check the checker, and do it first: a green run against a worker that
// never started is the failure this repo has a doctrine about. This drives
// the *same* reporting path every real check uses.
test("the harness can see a failure", async ({ page }) => {
const report = await run(page, "run_canary");
expect(report).toHaveLength(1);
expect(report[0].name).toBe("the_canary_must_fail");
expect(report[0].ok).toBe(false);
expect(report[0].detail).toContain("the harness can see a failure");
});
test("every check in the battery runs and passes", async ({ page }) => {
const report = await run(page, "run_checks");
// The battery is complete before any of it is believed.
expect(report.map((c) => c.name)).toEqual(EXPECTED);
const failed = report.filter((c) => !c.ok);
expect(
failed,
`failing checks:\n${failed.map((c) => ` ${c.name}: ${c.detail}`).join("\n")}`,
).toEqual([]);
});
// The one thing no in-page check can establish. gotcha 28: a delete is
// visible immediately and lands in OPFS at the end of the batch — so
// "commit_deletes actually removed it" is only answered by a fresh page.
// This is also the storage half of improvise-bck's persistence criterion.
test("what OPFS held survives a reload, and a committed delete does not", async ({
page,
}) => {
expect(await page.evaluate(() => window.opfs("before_reload"))).toBe("staged");
await page.reload();
expect(await page.evaluate(() => window.opfs("after_reload"))).toBe("survived");
});
});
harness/tests/shell.spec.js
+199
View File
@@ -0,0 +1,199 @@
// The browser shell: the three assertions improvise-hwpi could not write.
//
// That ticket fixed four things and said of itself that the fixes were "as
// unverified as the bugs", because three of them are one-assertion tests that
// needed a browser. These are those three, plus the watchdog — which was found
// by reading, and only because someone went looking.
//
// `web/app.js` and `web/worker.js` have never been rendered or run (gotcha 48).
// Everything below is the first execution of either.
import { test, expect } from "@playwright/test";
const status = (page) => page.locator("#status");
/// A peer that accepts the TCP connection and then refuses the WebSocket
/// upgrade — the harness's own static file server. Deterministic where a
/// closed port is not: a refused port's `close` timing is a property of the
/// host's network stack, and on the GitHub runner it did not arrive inside
/// the window, which made two expected-failure tests report "expected to
/// fail, but passed". The page cannot tell the two apart and neither can
/// `web/app.js`: in both cases the socket never opened.
const NO_WEBSOCKET_HERE = "127.0.0.1:8099";
const json = (v) => JSON.stringify(v);
/// `web/app.js`'s own HANDSHAKE_TIMEOUT_MS. Named here because two tests have
/// to outlast it, and a number that silently disagrees with the page would
/// make them pass for the wrong reason.
const HANDSHAKE_TIMEOUT_MS = 8_000;
/// Record every distinct #status the page shows, not just the last one.
/// Several of these are about a message being REPLACED, which the final value
/// cannot see.
function logStatus() {
window.__statusLog = [];
new MutationObserver(() => {
const text = document.getElementById("status")?.textContent ?? "";
if (text && window.__statusLog.at(-1) !== text) window.__statusLog.push(text);
}).observe(document, { childList: true, subtree: true, characterData: true });
}
test.describe("the shell reports what its peer is doing", () => {
// improvise-hwpi: `on_frame` returns a Result, which wasm-bindgen throws.
// Thrown in a bare listener it reached the devtools console and nothing
// else: the grid simply stopped updating with nothing on screen saying why.
//
// The fault is injected at the *peer*, not in the page — a mocked socket
// that answers the handshake with a frame the client cannot fold — so this
// exercises the real `on_frame` and the real catch.
test("a frame the client cannot fold says so on screen", async ({ page }) => {
await page.routeWebSocket("ws://127.0.0.1:8787/", (ws) => {
ws.onMessage(() => ws.send("{\"NotAFrame\":{}}"));
});
await page.goto("/index.html?ws=127.0.0.1:8787");
await expect(status(page)).toContainText("bad frame", { timeout: 15_000 });
});
// improvise-hwpi: without an `error` listener, a connection that never
// opened left #status blank, and "the server is not running" read exactly
// like "still connecting". Port 9 (discard) refuses immediately.
//
// The watchdog half is asserted on the same page load rather than in its
// own test, because it needs the same 9-second wait. It holds for a reason
// its own comment does not lead you to expect: `armWatchdog` runs only
// after `transport.ready` resolves, which happens on `open` — so on a
// socket that never opened the watchdog is never armed at all.
test("a peer that cannot be reached is named, and the watchdog stays out of it", async ({
page,
}) => {
await page.addInitScript(logStatus);
await page.goto(`/index.html?ws=${NO_WEBSOCKET_HERE}`);
await expect(status(page)).not.toHaveText("", { timeout: 15_000 });
await page.waitForTimeout(HANDSHAKE_TIMEOUT_MS + 1_000);
const log = await page.evaluate(() => window.__statusLog);
expect(log.some((t) => t.includes("cannot reach")), `#status said: ${json(log)}`).toBe(
true,
);
expect(log.some((t) => t.includes("sent no document")), `#status said: ${json(log)}`).toBe(
false,
);
});
// The other arm, which must keep working when the first one is fixed: a
// socket that really did open and then dropped IS a disconnect, and saying
// so is the whole reason `close` cannot simply be suppressed.
test("a socket that opened and then dropped reads as disconnected", async ({ page }) => {
// Closes on the page's first frame, so "it opened first" is true by
// construction rather than by timer.
await page.routeWebSocket("ws://127.0.0.1:8790/", (ws) => {
ws.onMessage(() => ws.close());
});
await page.goto("/index.html?ws=127.0.0.1:8790");
await expect(status(page)).toContainText("connected", { timeout: 15_000 });
await expect(status(page)).toContainText("disconnected", { timeout: 15_000 });
});
// TWO BUGS THIS FILE DELIBERATELY DOES NOT ASSERT, and the reason is worth
// more than the assertions would have been.
//
// 1. improvise-m18f: the socket's `close` listener is unconditional, so on
// a connection that never opened it overwrites "cannot reach …" with
// "disconnected".
// 2. On a connection that DID open and then dropped, nothing disarms the
// handshake watchdog, so 8s later "disconnected" is replaced by
// "accepted the connection but sent no document" — which is false. Same
// class as the bug `setShellFailure` fixed (a watchdog must be disarmed,
// not pre-empted, gotcha 48), and it SURVIVES the m18f fix, because that
// fix is about which message `close` writes rather than about the timer
// still running.
//
// Both are real and both were measured, on macOS/Chromium:
//
// never-opened: connecting… -> cannot reach -> disconnected
// open-then-close: connecting… -> connected -> disconnected
// -> accepted the connection but sent no document
//
// They were asserted here as `test.fail` expected-failures and **had to be
// removed**: neither reproduces on the GitHub runner, where `close` does
// not follow `error` within the window and the watchdog does not land. An
// expected-failure is green when the bug is present and red when it is
// absent, so one whose trigger is platform-dependent does not merely flake
// — it **inverts CI's meaning**, and it failed the build twice saying
// "expected to fail, but passed".
//
// What that leaves is a genuine limit rather than an oversight: the trigger
// for both is the ORDER AND TIMING of `error`/`close`/timer on the host's
// network stack, which is not a property of `web/app.js` and not something
// a harness can pin. The four assertions that remain in this file are the
// portable half — every message the page must write, and the one case where
// the watchdog must stay silent. Filed for the orchestrator with the
// recipe above; fixing them is `web/app.js`, which is not this unit's file.
// The first-frame watchdog (gotcha 46), found by reading. The condition is
// one no per-transport check would catch: the peer accepted the connection
// and then said nothing. The mock accepts and never answers.
test("a peer that accepts and then says nothing trips the watchdog", async ({
page,
}) => {
await page.routeWebSocket("ws://127.0.0.1:8788/", () => {});
await page.goto("/index.html?ws=127.0.0.1:8788");
await expect(status(page)).toContainText("sent no document", { timeout: 20_000 });
});
});
test.describe("the standalone deployment", () => {
// No `?ws=`, so `web/app.js` spawns the in-page worker over OPFS: the whole
// application, with no server process. If this draws a grid, then
// worker.js, bindings.rs, OpfsFs, vfs::open and the seed fetch all ran.
test("the page draws the demo document out of the worker", async ({ page }) => {
await page.goto("/index.html");
await expect(page.locator("#grid table")).toBeVisible({ timeout: 20_000 });
await expect(status(page)).toContainText("in-page worker");
// web/demo.improv: Product on row, _Measure on column (gotcha 46).
await expect(page.locator("#grid th", { hasText: "Revenue" })).toBeVisible();
await expect(page.locator("#grid td.editable").first()).toBeVisible();
});
// A commit round-trips through the host, and what lands on screen is the
// HOST's value rather than the string that was typed. `20000` is typed and
// `20,000` asserted: the number format is applied by the host and relayed
// back, so a page that echoed the input would fail.
//
// This is deliberately NOT named for improvise-hwpi's third fix (the
// `closed` re-entry guard in `edit`), which Copilot asked for on PR #24.
// That guard is **not observable from the DOM**, measured rather than
// assumed: with it removed, `close` still runs exactly twice, the frame is
// still sent exactly once, `#grid` still takes exactly two childList
// mutations, and the final text is identical. The only difference is one
// extra *entry* into `repaint()` that produces no output — because the
// second close is `close(false)`, so it does not re-send, and everything
// else it does is idempotent. That is precisely why hwpi called it
// "harmless before, but only by accident".
//
// Counting repaint() entries would need a test hook inside `web/app.js`,
// which would be a page that knows it is being tested. So the name says
// what is checked and this comment says what is not.
test("committing an edit sends one frame and shows the host's value", async ({
page,
}) => {
await page.goto("/index.html?doc=commit-test.improv");
await expect(page.locator("#grid table")).toBeVisible({ timeout: 20_000 });
const cell = page.locator("#grid td.editable").first();
const before = await cell.textContent();
await cell.click();
const input = cell.locator("input");
await expect(input).toBeFocused();
await input.fill("20000");
await input.press("Enter");
await expect(cell).toHaveText("20,000", { timeout: 10_000 });
expect(before).not.toBe("20,000");
// The editor is gone and did not come back. This is the half of the
// re-entry guard that IS observable: whatever `close` ran, no stray
// <input> is left behind and the host's value is not overwritten by the
// `original` text the second close writes before repainting.
await expect(page.locator("#grid input")).toHaveCount(0);
});
});
harness/www/opfs/index.html
+48
View File
@@ -0,0 +1,48 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>improvise — OPFS contract harness</title>
</head>
<body>
<!--
Not part of the application. This page exists so Playwright can drive
`improvise_io::storage::contract::storage_contract` against `OpfsFs`
inside a dedicated Web Worker, which is the only context where OPFS
sync access handles exist (improvise-vvl3).
-->
<h1>OPFS contract harness</h1>
<pre id="out">idle</pre>
<script type="module">
const worker = new Worker("./worker.js", { type: "module" });
let next = 0;
const pending = new Map();
worker.addEventListener("message", (event) => {
const { id, ok, value, error } = event.data;
const settle = pending.get(id);
pending.delete(id);
if (!settle) return;
ok ? settle.resolve(value) : settle.reject(new Error(error));
});
worker.addEventListener("error", (event) => {
for (const { reject } of pending.values()) {
reject(new Error(`worker error: ${event.message ?? "see console"}`));
}
pending.clear();
});
// The page's whole API. Returns whatever the Rust entry point returned,
// and rejects — loudly, with the message intact — if it threw.
window.opfs = (cmd) =>
new Promise((resolve, reject) => {
const id = next++;
pending.set(id, { resolve, reject });
worker.postMessage({ id, cmd });
}).then((value) => {
document.getElementById("out").textContent = value;
return value;
});
</script>
</body>
</html>
harness/www/opfs/worker.js
+30
View File
@@ -0,0 +1,30 @@
// The dedicated Web Worker the OPFS contract runs inside.
//
// `createSyncAccessHandle` exists only here — that is the whole reason
// `Storage` could stay synchronous (gotcha 28), and the reason this file
// exists rather than the page calling the bundle directly.
//
// Every decision is in Rust (harness/opfs-contract). This moves strings.
import init, {
run_checks,
run_canary,
before_reload,
after_reload,
} from "./pkg/opfs_contract.js";
const ready = init();
const ENTRY = { run_checks, run_canary, before_reload, after_reload };
self.onmessage = async (event) => {
const { id, cmd } = event.data;
try {
await ready;
const fn = ENTRY[cmd];
if (!fn) throw new Error(`no such entry point: ${cmd}`);
self.postMessage({ id, ok: true, value: await fn() });
} catch (err) {
self.postMessage({ id, ok: false, error: String(err?.message ?? err) });
}
};
web/assemble.sh
Executable
+72
View File
@@ -0,0 +1,72 @@
#!/bin/sh
# Assemble the static site from `web/` into `site/`.
#
# `web/` is the *source* directory: it also holds build.sh, assemble.sh and
# .gitignore, which have no business being served. This copies what ships.
#
# Extracted from the CI workflow so that the Pages job and the browser harness
# assemble the same tree — a harness that runs against a different arrangement
# of files is testing something nobody deploys (improvise-vvl3). Same reason
# `web/build.sh` exists rather than two copies of the wasm-pack invocation.
#
# Usage: web/assemble.sh [OUT_DIR] [CACHE_BUST_TOKEN]
#
# The token cache-busts the two generated bundles. Pages serves everything
# `max-age=600` and the wasm-pack filenames carry no content hash, so for ten
# minutes after a deploy a returning visitor runs the PREVIOUS build — which
# during improvise-9x6 debugging made a shipped fix look like it had not
# shipped, twice. CI passes the commit SHA; a local run defaults to `dev`.
#
# Both halves are stamped, because a fresh glue file does NOT pull a fresh
# .wasm: the glue locates it with `new URL('..._bg.wasm', import.meta.url)`,
# and a URL constructed against a base does not inherit the base's query
# string. Stamping only the import specifier would refresh the wrapper and
# keep the stale binary — the failure mode this is here to end.
set -eu
root=$(cd "$(dirname "$0")/.." && pwd)
out=${1:-"$root/site"}
token=${2:-dev}
test -d "$root/web/pkg" || { echo "web/pkg missing — run web/build.sh first" >&2; exit 1; }
test -d "$root/web/worker-pkg" || { echo "web/worker-pkg missing — run web/build.sh first" >&2; exit 1; }
rm -rf "$out"
mkdir -p "$out"
cp -f "$root/web/index.html" "$root/web/app.js" "$root/web/style.css" "$root/web/worker.js" "$out/"
# The document the page opens on a browser that has never seen one.
# web/worker.js fetches it by the ?seed= URL and hands the bytes to Rust,
# which writes them through the VFS — so leaving it out is an empty grid, not
# a 404 the page could report.
cp -f "$root/web/demo.improv" "$out/"
cp -rf "$root/web/pkg" "$out/pkg"
cp -rf "$root/web/worker-pkg" "$out/worker-pkg"
rm -f "$out/pkg/.gitignore" "$out/pkg/package.json"
rm -f "$out/worker-pkg/.gitignore" "$out/worker-pkg/package.json"
# BSD sed (macOS) needs an argument to -i; GNU sed must not be given one.
sedi() { sed "$1" "$2" > "$2.tmp" && mv -f "$2.tmp" "$2"; }
sedi "s|\./pkg/improvise_wasm_client\.js|&?v=${token}|" "$out/app.js"
sedi "s|\./worker-pkg/improvise_worker_server\.js|&?v=${token}|" "$out/worker.js"
sedi "s|'improvise_wasm_client_bg\.wasm'|'improvise_wasm_client_bg.wasm?v=${token}'|" \
"$out/pkg/improvise_wasm_client.js"
sedi "s|'improvise_worker_server_bg\.wasm'|'improvise_worker_server_bg.wasm?v=${token}'|" \
"$out/worker-pkg/improvise_worker_server.js"
# One `grep` over all four files succeeds when ANY one of them matches, so
# three failed substitutions could still produce a "successful" assembly under
# `set -e` (found by Copilot on PR #24). These checks are the only guard
# against the generated glue changing shape, so each stamped reference is
# verified on its own and names itself when it is the one missing.
stamped() {
grep -q "$2" "$1" || {
echo "assemble: $1 is missing the cache-bust stamp: $2" >&2
exit 1
}
grep -n "$2" "$1"
}
stamped "$out/app.js" "pkg/improvise_wasm_client.js?v=${token}"
stamped "$out/worker.js" "worker-pkg/improvise_worker_server.js?v=${token}"
stamped "$out/pkg/improvise_wasm_client.js" "improvise_wasm_client_bg.wasm?v=${token}"
stamped "$out/worker-pkg/improvise_worker_server.js" "improvise_worker_server_bg.wasm?v=${token}"