fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ci: build the bundles once, and take the harness lint off the critical path

Browse Source 
Measured on run 35535225637 (push to main, caches warm). The critical path
was the browser job at 263s plus an 11s deploy; everything else finished
inside it. Its decomposition: harness lint 37s, Playwright browser download
50s, `./harness/build.sh` 110s, `npx playwright test` 50s.

Three wasm-pack builds ran in that 110s, and two of them — the client and
the worker bundle — were the same two `pages-build` ran in its own 68s step,
in a parallel job with a separate rust-cache key (`add-job-id-key` defaults
on). Same work, twice per run.

- One `build` job now runs `./harness/build.sh` (all three bundles plus the
  `harness/serve` tree) and then `./web/assemble.sh site $GITHUB_SHA` from
  the pkg dirs it just produced. It uploads `harness/serve`, the two raw
  bundles, and the Pages artifact. `pages-build` is folded into it.
- `browser` downloads `harness/serve` whole and needs no Rust toolchain and
  no wasm-pack. Downloading the tree rather than reassembling it keeps one
  answer to "what does the harness serve" (improvise-vvl3): `harness/build.sh`.
  Verified locally that the two assemblies differ in exactly the four
  cache-bust-stamped files and nothing else — the .wasm bytes are identical.
- `harness-lint` is its own job. It gated nothing the harness needs and cost
  37s of the critical path, and `cargo fmt --all` run from inside
  `harness/opfs-contract` formats the WHOLE outer workspace (114 files,
  measured) despite that crate's empty `[workspace]` table, so a rustfmt
  violation anywhere in the repo failed the *Browser harness* job.
  `-p opfs-contract` scopes it to the one file it is about.
- Playwright's browser download is cached on the resolved package version.
- rust-cache in `build` and `harness-lint` covers `harness/opfs-contract`'s
  own target dir, which the default single-workspace config never did.

No gate is weakened. `pages-deploy` still needs every check that could
previously fail it; `harness-lint` is named explicitly because it used to be
a step inside `browser`. `kb`, `wasm` and `test` gain no dependency on
`build` — none of them consumes an artifact.

On improvise-864a: the gap between the Pages artifact upload and the deploy
narrows from ~182s to ~115s on this run's shape, so the window in which a
superseding run can expire it is smaller. The race is not closed, and not
widened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edward Langley
2026-09-20 13:40:08 -07:00
parent 1af51c99c6
commit 3c9668f9e2
3 changed files with 179 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+177 -82
View File
@@ -13,6 +13,107 @@ env:
CARGO_TERM_COLOR: always
jobs:
# ---------------------------------------------------------------------------
# The one place wasm-pack runs (improvise CI dedup).
#
# Before this job there were two: `pages-build` ran `web/build.sh`, and the
# browser harness ran `harness/build.sh`, which CALLS `web/build.sh`. The two
# wasm-pack builds were therefore done twice per run, in parallel jobs, with
# separate rust-cache keys (`add-job-id-key` is on by default, so `browser`
# and `pages-build` never shared a target dir). Measured on run 35535225637
# (push to main, caches warm): 68s inside `pages-build`, and the same 68s
# inside the browser job's 110s build step.
#
# Everything downstream now DOWNLOADS. The invariant that buys is artifact
# identity: the harness and Pages are served bytes from one wasm-pack run,
# so "the harness runs against the tree Pages publishes" (improvise-vvl3) is
# true by construction rather than because two scripts agree.
#
# The native binary is deliberately NOT built here. Nothing downstream
# consumes one — `cargo test --workspace` in the `test` job builds its own,
# for its own target dir and its own cache key, and a `needs:` on this job
# would serialise a 109s job behind a 126s one for no artifact at all.
build:
name: Build all bundles
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
# Two workspaces, because `harness/opfs-contract` carries an empty
# `[workspace]` table and so has its own target dir, which the default
# single-workspace configuration never cached (it recompiled its whole
# dependency tree every run).
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
. -> target
harness/opfs-contract -> target
# Pinned release binary rather than `cargo install` (a three-minute
# source build) or a third-party action. wasm-pack runs its own bundled
# `wasm-opt`, which is why improvise-wasm-client's Cargo.toml names
# `--enable-bulk-memory`: without it the optimizer rejects the
# `memory.copy` LLVM emits regardless.
# v0.15.0 is what `nix develop` supplies, and the bundle was built and
# measured against it locally (346 KB, 120 KB gzipped). Pinning the same
# version is what makes that local run evidence about this one.
- name: Install wasm-pack
env:
WASM_PACK_VERSION: v0.15.0
run: |
set -eu
url="https://github.com/rustwasm/wasm-pack/releases/download/${WASM_PACK_VERSION}/wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl.tar.gz"
curl -sSfL "$url" | tar -xz --strip-components=1 -C /usr/local/bin \
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
wasm-pack --version
# The same script a developer runs locally, so there is one answer to
# "how is the bundle built" rather than two that can drift. It builds all
# three wasm bundles (client, worker, opfs-contract) and assembles
# `harness/serve` — the deployed arrangement plus /opfs/.
- name: Build the bundles and the harness tree
run: ./harness/build.sh
# Re-assembles from the pkg dirs the step above already produced — no
# wasm-pack runs here. A second assembly rather than a copy of
# `harness/serve` because the cache-bust token differs: the harness tree
# is stamped `harness` and the published site the commit SHA, which is
# what makes a returning visitor pick up a fresh bundle (improvise-9x6).
- name: Assemble the site
run: ./web/assemble.sh site "$GITHUB_SHA"
- run: find site -type f | sort
- run: du -sh site
# The tree the browser harness serves. Uploaded whole rather than
# reassembled downstream: a second assembly path is a second answer to
# "what does the harness serve", which is the thing improvise-vvl3 exists
# to prevent.
- name: Upload the harness tree
uses: actions/upload-artifact@v7
with:
name: harness-serve
path: harness/serve
retention-days: 1
include-hidden-files: true
# For a job that needs the raw wasm-pack output rather than the
# assembled site — the Node bundle suite under `web/tests/bundle/`.
- name: Upload the wasm bundles
uses: actions/upload-artifact@v7
with:
name: wasm-bundles
path: |
web/pkg
web/worker-pkg
retention-days: 1
include-hidden-files: true
- uses: actions/upload-pages-artifact@v5
with:
path: site
test:
name: Test and lint (host)
runs-on: ubuntu-latest
@@ -111,6 +212,37 @@ jobs:
nix shell nixpkgs#trealla --command tpl prolog-sessions/kb.pl \
-g "(kb_consistent -> halt(0) ; (kb_report, halt(1)))"
# ---------------------------------------------------------------------------
# Lints the one crate the `test` job cannot reach.
#
# `harness/opfs-contract` carries an empty `[workspace]` table, so it is its
# own root and `cargo clippy --workspace` at the repo root never sees it.
#
# Its own job rather than a step inside `browser`, for two reasons. It took
# 37s of the browser job's 263s on run 35535225637 while gating nothing the
# harness needs — a lint failure and a harness failure are different reports,
# and here the lint failure wore the harness's name. And `cargo fmt --all`
# from inside that directory formats the WHOLE outer workspace (114 files,
# measured) rather than the one crate, because cargo-fmt's `--all` walks past
# the nested workspace boundary — so a rustfmt violation anywhere in the repo
# failed the *Browser harness* job. `-p opfs-contract` scopes it to 1 file.
harness-lint:
name: Lint the harness crate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: harness/opfs-contract -> target
- working-directory: harness/opfs-contract
run: |
cargo fmt -p opfs-contract -- --check
cargo clippy --target wasm32-unknown-unknown --all-targets -- -D warnings
# ---------------------------------------------------------------------------
# The browser harness (improvise-vvl3, absorbing improvise-xf1).
#
@@ -120,55 +252,56 @@ jobs:
# by reading or by a person pressing keys, neither of which is repeatable.
#
# It can fail the build, deliberately: a harness nobody runs is worse than
# none, because it reads as coverage. Playwright's browser download is ~80 MB
# and takes well under a minute on ubuntu-latest, which is affordable next to
# the two wasm-pack builds this job already needs.
# none, because it reads as coverage.
#
# It now needs NO Rust toolchain: the tree it serves is downloaded from
# `build`, which is what makes "the harness runs against the tree Pages
# publishes" (improvise-vvl3) a fact about bytes rather than about two
# scripts agreeing. The download is `harness/serve` whole, so there is still
# exactly one place that arrangement is decided — `harness/build.sh`.
browser:
name: Browser harness (OPFS contract + shell)
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
# Named rather than assumed: this job lints harness/opfs-contract,
# which the `test` job's workspace-wide fmt and clippy cannot reach.
components: clippy, rustfmt
- uses: Swatinem/rust-cache@v2
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Install wasm-pack
env:
WASM_PACK_VERSION: v0.15.0
run: |
set -eu
url="https://github.com/rustwasm/wasm-pack/releases/download/${WASM_PACK_VERSION}/wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl.tar.gz"
curl -sSfL "$url" | tar -xz --strip-components=1 -C /usr/local/bin \
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
wasm-pack --version
# Not a workspace member (it only builds for wasm32), so the `test` job's
# workspace-wide fmt and clippy do not reach it.
- name: Lint the harness crate
working-directory: harness/opfs-contract
run: |
cargo fmt --all -- --check
cargo clippy --target wasm32-unknown-unknown --all-targets -- -D warnings
- name: Install the harness
working-directory: harness
run: npm ci
# Keyed on the resolved @playwright/test version, so a version bump is a
# miss and anything else is a hit. The download was 50s on run
# 35535225637 and is pure I/O — nothing about it depends on this commit.
- name: Playwright version
id: pw
working-directory: harness
run: echo "v=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v4
id: pw-cache
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ steps.pw.outputs.v }}
# All three, because improvise-xf1's acceptance criterion names Chrome,
# Firefox and Safari and `createSyncAccessHandle` is implemented
# separately by each engine. WebKit is installed even though its OPFS is
# absent today: the spec skips on a runtime probe, so the day that build
# gains OPFS the check starts running without a CI change.
# `--with-deps` installs apt packages OUTSIDE the cached directory, so it
# runs on a cache hit too; only the browser download is skipped.
- name: Install browsers
working-directory: harness
run: npx playwright install --with-deps chromium firefox webkit
# Builds both site bundles, the harness's own wasm bundle, and assembles
# `harness/serve` — the same tree `pages-build` publishes, plus /opfs/.
- name: Build the harness
run: ./harness/build.sh
# The tree `build` assembled, not one this job rebuilds. Same bytes as
# the Pages artifact, modulo the cache-bust token `web/assemble.sh`
# stamps — the harness tree is stamped `harness` and the site the commit
# SHA, which was already true when both jobs built their own.
- name: Download the harness tree
uses: actions/download-artifact@v7
with:
name: harness-serve
path: harness/serve
- name: Run the harness
working-directory: harness
run: npx playwright test
@@ -182,12 +315,13 @@ jobs:
# ---------------------------------------------------------------------------
# GitHub Pages delivery (improvise-d31).
#
# Two jobs on purpose. `pages-build` runs on every push and pull request, so
# a change that breaks `web/build.sh` or the bundle fails on the PR that made
# it — the build is a gate. `pages-deploy` runs only on a push to main and
# `needs` the check jobs above, so nothing that failed `cargo test`, the
# wasm32 gate or the browser harness can reach the published site. The deploy
# depends on those checks rather than repeating them (improvise-d31 AC 2).
# The artifact is uploaded by `build`, which runs on every push and pull
# request, so a change that breaks `web/build.sh` or the bundle fails on the
# PR that made it — the build is a gate. `pages-deploy` runs only on a push
# to main and `needs` the check jobs, so nothing that failed `cargo test`,
# the wasm32 gate or the browser harness can reach the published site. The
# deploy depends on those checks rather than repeating them (improvise-d31
# AC 2).
#
# Not path-filtered, deliberately. The bundles are `improvise-wasm-client` and
# `improvise-worker-server` plus, between them, most of the workspace — the
@@ -196,54 +330,15 @@ jobs:
# that meant "skip when the bundle cannot have changed" would have to name
# most of the workspace, and the failure mode of getting it wrong is a stale
# site with no signal that it is stale. A redundant build is the cheaper error.
pages-build:
name: Browser bundle + static site
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@v2
# Pinned release binary rather than `cargo install` (a three-minute
# source build) or a third-party action. wasm-pack runs its own bundled
# `wasm-opt`, which is why improvise-wasm-client's Cargo.toml names
# `--enable-bulk-memory`: without it the optimizer rejects the
# `memory.copy` LLVM emits regardless.
# v0.15.0 is what `nix develop` supplies, and the bundle was built and
# measured against it locally (346 KB, 120 KB gzipped). Pinning the same
# version is what makes that local run evidence about this one.
- name: Install wasm-pack
env:
WASM_PACK_VERSION: v0.15.0
run: |
set -eu
url="https://github.com/rustwasm/wasm-pack/releases/download/${WASM_PACK_VERSION}/wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl.tar.gz"
curl -sSfL "$url" | tar -xz --strip-components=1 -C /usr/local/bin \
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
wasm-pack --version
# The same script a developer runs locally, so there is one answer to
# "how is the bundle built" rather than two that can drift.
- name: Build the wasm bundle
run: ./web/build.sh
# `web/assemble.sh`, not an inline copy of it: the browser harness job
# below assembles the same tree, and a harness that runs against a
# different arrangement of files is testing something nobody deploys.
- name: Assemble the site
run: ./web/assemble.sh site "$GITHUB_SHA"
- run: find site -type f | sort
- run: du -sh site
- uses: actions/upload-pages-artifact@v5
with:
path: site
#
# On improvise-864a (concurrent merges racing on one Pages environment, the
# superseded run failing "Found 0 artifacts"): building once shortens the gap
# between upload and deploy — 182s before, ~115s after, on the run 35535225637
# shape — so the window in which a superseding run can expire this run's
# artifact is smaller. It does not close the race; it does not widen it.
pages-deploy:
name: Deploy to GitHub Pages
needs: [test, wasm, browser, pages-build]
needs: [test, wasm, browser, harness-lint, build]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions: