Edward Langley
d6706774a2
improvise-st8q, improvise-tvte, from the #228 re-review at c1417359. The
maax parser still failed open. A bare checkers.sh inside a heredoc or a
folded run: value counted as every suite, and step and job if: were
ignored. Every round of parsing met another shell or YAML form, so the
parsing is gone.
checkers.sh holds a job-to-suites table (ci_suites), and each CI job
makes one call, `checkers.sh --ci <job>`. check-ci-graph.py counts a step
only when all of these hold: it sits inside steps: at step-key
indentation, its whole run: value is that call, the call names its own
job, and no if:, continue-on-error:, shell: or defaults: covers it. Every
other mention of scripts/checkers.sh is reported. A key with an inline
value owns deeper lines, and an open quote owns lines until it closes, so
heredocs, folds and quoted values are content, never structure.
There are 19 suite self-test cases and 7 rivals, each failing a case only
it can. The reviewer's heredoc, fold and if: attacks on the real ci.yml
now exit 1. checkers-test.sh gains five --ci cases. context/code/tooling.md
corrects two overstated claims ("--would-run anywhere is a usage error",
"fails closed").
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
448 lines
21 KiB
YAML
448 lines
21 KiB
YAML
name: CI
|
|
|
|
on:
|
|
# `push` unrestricted plus `pull_request` ran every job TWICE on any branch
|
|
# with an open PR — visible as the (push) / (pull_request) pairs in the checks
|
|
# list. Restricting push to main means a PR branch is covered once by
|
|
# pull_request, and main once by push, which is also what pages-deploy needs.
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
|
|
jobs:
|
|
# ---------------------------------------------------------------------------
|
|
# The one place wasm-pack runs (improvise CI dedup).
|
|
#
|
|
# Before this job there were two: `pages-build` ran `web/build.sh`, and the
|
|
# browser harness ran `harness/build.sh`, which CALLS `web/build.sh`. The two
|
|
# wasm-pack builds were therefore done twice per run, in parallel jobs, with
|
|
# separate rust-cache keys (`add-job-id-key` is on by default, so `browser`
|
|
# and `pages-build` never shared a target dir). Measured on run 35535225637
|
|
# (push to main, caches warm): 68s inside `pages-build`, and the same 68s
|
|
# inside the browser job's 110s build step.
|
|
#
|
|
# Everything downstream now DOWNLOADS. The invariant that buys is artifact
|
|
# identity: the harness and Pages are served bytes from one wasm-pack run,
|
|
# so "the harness runs against the tree Pages publishes" (improvise-vvl3) is
|
|
# true by construction rather than because two scripts agree.
|
|
#
|
|
# The native binary is deliberately NOT built here. Nothing downstream
|
|
# consumes one — `cargo test --workspace` in the `test` job builds its own,
|
|
# for its own target dir and its own cache key, and a `needs:` on this job
|
|
# would serialise a 109s job behind a 126s one for no artifact at all.
|
|
build:
|
|
name: Build all bundles
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
# First, because it needs nothing and takes a second: the gate that stops
|
|
# an unstamped bundle shipping was deleted by injection on PR #69 and the
|
|
# assembly still exited 0 (improvise-w8ih). It guards the assemble step
|
|
# below, so it fails under this job's name rather than another's.
|
|
- name: The stamp gate refuses what it claims to
|
|
run: ./scripts/checkers.sh --ci build
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
targets: wasm32-unknown-unknown
|
|
# Two workspaces, because `harness/opfs-contract` carries an empty
|
|
# `[workspace]` table and so has its own target dir, which the default
|
|
# single-workspace configuration never cached (it recompiled its whole
|
|
# dependency tree every run).
|
|
- uses: Swatinem/rust-cache@v2
|
|
with:
|
|
workspaces: |
|
|
. -> target
|
|
harness/opfs-contract -> target
|
|
# One home for the two pins, so the release capture
|
|
# (scripts/build-web-release.sh, improvise-oujb) provisions the same
|
|
# versions rather than a second copy free to go stale. The grep keeps a
|
|
# comment or a blank line out of $GITHUB_ENV, which takes NAME=VALUE only.
|
|
- name: Pinned web toolchain versions
|
|
run: grep -E '^[A-Z_]+=' scripts/web-toolchain-versions.env >> "$GITHUB_ENV"
|
|
|
|
# Pinned release binary rather than `cargo install` (a three-minute
|
|
# source build) or a third-party action. wasm-pack runs its own bundled
|
|
# `wasm-opt`, which is why improvise-wasm-client's Cargo.toml names
|
|
# `--enable-bulk-memory`: without it the optimizer rejects the
|
|
# `memory.copy` LLVM emits regardless.
|
|
# v0.15.0 is what `nix develop` supplies, and the bundle was built and
|
|
# measured against it locally (346 KB, 120 KB gzipped). Pinning the same
|
|
# version is what makes that local run evidence about this one.
|
|
- name: Install wasm-pack
|
|
run: |
|
|
set -eu
|
|
url="https://github.com/rustwasm/wasm-pack/releases/download/${WASM_PACK_VERSION}/wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl.tar.gz"
|
|
curl -sSfL "$url" | tar -xz --strip-components=1 -C /usr/local/bin \
|
|
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
|
|
wasm-pack --version
|
|
|
|
# Both assemblies below generate the prose site with node. The runner
|
|
# image ships one, but pinning it is what makes a local run evidence.
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: ${{ env.NODE_VERSION }}
|
|
|
|
# The same script a developer runs locally, so there is one answer to
|
|
# "how is the bundle built" rather than two that can drift. It builds all
|
|
# three wasm bundles (client, worker, opfs-contract) and assembles
|
|
# `harness/serve` — the deployed arrangement plus /opfs/.
|
|
- name: Build the bundles and the harness tree
|
|
run: ./harness/build.sh
|
|
|
|
# Re-assembles from the pkg dirs the step above already produced — no
|
|
# wasm-pack runs here. A second assembly rather than a copy of
|
|
# `harness/serve` because the cache-bust token differs: the harness tree
|
|
# is stamped `harness` and the published site the commit SHA, which is
|
|
# what makes a returning visitor pick up a fresh bundle (improvise-9x6).
|
|
- name: Assemble the site
|
|
run: ./web/assemble.sh site "$GITHUB_SHA"
|
|
|
|
- run: find site -type f | sort
|
|
- run: du -sh site
|
|
|
|
# The tree the browser harness serves. Uploaded whole rather than
|
|
# reassembled downstream: a second assembly path is a second answer to
|
|
# "what does the harness serve", which is the thing improvise-vvl3 exists
|
|
# to prevent.
|
|
- name: Upload the harness tree
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: harness-serve
|
|
path: harness/serve
|
|
retention-days: 1
|
|
include-hidden-files: true
|
|
|
|
# For a job that needs the raw wasm-pack output rather than the
|
|
# assembled site — the Node bundle suite under `web/tests/bundle/`.
|
|
- name: Upload the wasm bundles
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: wasm-bundles
|
|
path: |
|
|
web/pkg
|
|
web/worker-pkg
|
|
retention-days: 1
|
|
include-hidden-files: true
|
|
|
|
- uses: actions/upload-pages-artifact@v5
|
|
with:
|
|
path: site
|
|
|
|
test:
|
|
name: Test and lint (host)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
components: clippy, rustfmt
|
|
- uses: Swatinem/rust-cache@v2
|
|
# NOT `cargo fmt --all`, which reaches only ONE of the two workspace
|
|
# roots. Exit 2 here is a usage error, never a verdict (improvise-b4ks).
|
|
- name: Every workspace root is formatted
|
|
run: ./scripts/fmt-workspaces.sh
|
|
- run: cargo clippy --workspace --all-targets -- -D warnings
|
|
- run: cargo test --workspace
|
|
# The comment cap and the test-evidence keys (improvise-9c8c); without
|
|
# this step the rule is advisory. See context/design-principles.md §6.
|
|
- name: The comment checker catches what it has been fixed for
|
|
run: python3 scripts/check-comment-runs.py --self-test
|
|
- name: Comment runs capped at two lines
|
|
run: python3 scripts/check-comment-runs.py
|
|
- name: Test-evidence headings name real tests
|
|
run: python3 scripts/check-comment-runs.py --check-evidence
|
|
# `pages-deploy` needs every other job, and saying so was not enough: a
|
|
# job added during review of PR #83 was ungated and nothing noticed.
|
|
- name: The deploy gate refuses what it claims to
|
|
run: python3 scripts/check-ci-graph.py --self-test
|
|
- name: Every check job gates the deploy
|
|
run: python3 scripts/check-ci-graph.py
|
|
|
|
# Too slow on GitHub Actions
|
|
#
|
|
# nix-test:
|
|
# name: Nix Test
|
|
# runs-on: ubuntu-latest
|
|
# steps:
|
|
# - name: Free up disk space
|
|
# run: |
|
|
# sudo rm -rf /usr/share/dotnet \
|
|
# /usr/local/lib/android
|
|
# sudo df -h
|
|
|
|
# - uses: actions/checkout@v7
|
|
# - name: Install Nix
|
|
# uses: DeterminateSystems/nix-installer-action@v14
|
|
|
|
# - name: Magic Nix Cache
|
|
# uses: DeterminateSystems/magic-nix-cache-action@v8
|
|
|
|
# - name: Check Nix health
|
|
# run: nix doctor
|
|
|
|
# - run: nix build
|
|
|
|
wasm:
|
|
name: wasm32 check (browser bundle crates)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
targets: wasm32-unknown-unknown
|
|
- uses: Swatinem/rust-cache@v2
|
|
# Only the crates that go into the browser bundle. The root `improvise`
|
|
# crate is the terminal binary and depends on crossterm, which does not
|
|
# build for wasm — that is correct, not a gap (improvise-ywd).
|
|
- name: improvise-core
|
|
run: cargo check -p improvise-core --target wasm32-unknown-unknown
|
|
- name: improvise-formula
|
|
run: cargo check -p improvise-formula --target wasm32-unknown-unknown
|
|
- name: improvise-protocol
|
|
run: cargo check -p improvise-protocol --target wasm32-unknown-unknown
|
|
- name: improvise-io
|
|
run: cargo check -p improvise-io --target wasm32-unknown-unknown
|
|
- name: improvise-command
|
|
run: cargo check -p improvise-command --target wasm32-unknown-unknown
|
|
# The browser client itself. This is the one crate for which a wasm
|
|
# build is the *point* rather than a property worth keeping, so a
|
|
# failure here is not a portability regression but a broken deliverable
|
|
# (improvise-gsw).
|
|
- name: improvise-wasm-client
|
|
run: cargo check -p improvise-wasm-client --target wasm32-unknown-unknown
|
|
# The worker half of the standalone deployment (improvise-9x6). Same
|
|
# terms as the client: a wasm build is the point, so a failure here is a
|
|
# broken deliverable rather than a portability regression.
|
|
- name: improvise-worker-server
|
|
run: cargo check -p improvise-worker-server --target wasm32-unknown-unknown
|
|
# Not in the browser bundle yet — nothing depends on improvise-gp. Gated
|
|
# anyway because it is a candidate (fitting a tree in the browser) and
|
|
# staying wasm-clean is free now, expensive to retrofit (improvise-z5eg).
|
|
- name: improvise-gp
|
|
run: cargo check -p improvise-gp --target wasm32-unknown-unknown
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The browser JavaScript's unit tests. Its own job, deliberately: a failure
|
|
# here is a decision `web/shell.js` or `web/serial.js` got wrong, and
|
|
# reporting that as "Browser harness" would put it behind an 80 MB browser
|
|
# download and two wasm-pack builds. It needs neither — `node --test` is in
|
|
# the runtime, and this job installs nothing at all.
|
|
js:
|
|
name: Browser JS unit tests (node --test)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
# The glob is the SCRIPT's, not the shell's and not node's. Under bash an
|
|
# unmatched glob survives as a literal, node globs it, matches nothing and
|
|
# exits 0 having run no test -- so this step gated nothing the moment
|
|
# `web/tests/` moved (improvise-vz54). `web/tests/bundle/` needs a built
|
|
# wasm bundle and runs in the browser job instead.
|
|
- name: What the page decides between its boundaries
|
|
run: ./scripts/node-test.sh 'web/tests/*.test.js'
|
|
# The guard above is only worth its line if its refusals are themselves
|
|
# checked -- the step it replaces was green while asserting nothing.
|
|
- name: The node test runner refuses what it claims to
|
|
run: ./scripts/checkers.sh --ci js
|
|
|
|
kb:
|
|
name: Knowledge base consistency
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: cachix/install-nix-action@v31
|
|
# kb.pl is executable knowledge, so it gets checked like code: rules
|
|
# must reject what they should, domains must close, and functional keys
|
|
# must not carry conflicting values. See kb_report/0 in the file.
|
|
- name: kb_consistent
|
|
run: |
|
|
nix shell nixpkgs#trealla --command tpl prolog-sessions/kb.pl \
|
|
-g "(kb_consistent -> halt(0) ; (kb_report, halt(1)))"
|
|
# Every suite goes through scripts/checkers.sh, which pins a bash >= 5
|
|
# and scrubs the environment (improvise-f9qf). Which suites this job runs
|
|
# is the `kb` row of the CI table in checkers.sh, deliberately not listed
|
|
# here. check-ci-graph.py holds that table to `checkers.sh --list`.
|
|
- name: Every checker in scripts/ refuses what it claims to
|
|
run: nix shell nixpkgs#trealla --command ./scripts/checkers.sh --ci kb
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Lints the one crate the `test` job cannot reach.
|
|
#
|
|
# `harness/opfs-contract` carries an empty `[workspace]` table, so it is its
|
|
# own root and `cargo clippy --workspace` at the repo root never sees it.
|
|
#
|
|
# Its own job rather than a step inside `browser`, for two reasons. It took
|
|
# 37s of the browser job's 263s on run 35535225637 while gating nothing the
|
|
# harness needs — a lint failure and a harness failure are different reports,
|
|
# and here the lint failure wore the harness's name. And `cargo fmt --all`
|
|
# from inside that directory formats the WHOLE outer workspace (114 files,
|
|
# measured) rather than the one crate, because cargo-fmt's `--all` walks past
|
|
# the nested workspace boundary — so a rustfmt violation anywhere in the repo
|
|
# failed the *Browser harness* job. `-p opfs-contract` scopes it to 1 file.
|
|
harness-lint:
|
|
name: Lint the harness crate
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: dtolnay/rust-toolchain@stable
|
|
with:
|
|
targets: wasm32-unknown-unknown
|
|
components: clippy, rustfmt
|
|
- uses: Swatinem/rust-cache@v2
|
|
with:
|
|
workspaces: harness/opfs-contract -> target
|
|
- working-directory: harness/opfs-contract
|
|
run: |
|
|
cargo fmt -p opfs-contract -- --check
|
|
cargo clippy --target wasm32-unknown-unknown --all-targets -- -D warnings
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The browser harness (improvise-vvl3, absorbing improvise-xf1).
|
|
#
|
|
# This is the only job that RUNS any of the browser half. `OpfsFs`,
|
|
# `web/app.js` and `web/worker.js` were previously compile-checked,
|
|
# syntax-checked and read — never executed — so every bug in them was found
|
|
# by reading or by a person pressing keys, neither of which is repeatable.
|
|
#
|
|
# It can fail the build, deliberately: a harness nobody runs is worse than
|
|
# none, because it reads as coverage.
|
|
#
|
|
# It now needs NO Rust toolchain: the tree it serves is downloaded from
|
|
# `build`, which is what makes "the harness runs against the tree Pages
|
|
# publishes" (improvise-vvl3) a fact about bytes rather than about two
|
|
# scripts agreeing. The download is `harness/serve` whole, so there is still
|
|
# exactly one place that arrangement is decided — `harness/build.sh`.
|
|
browser:
|
|
name: Browser harness (OPFS contract + shell)
|
|
needs: build
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
- name: Install the harness
|
|
working-directory: harness
|
|
run: npm ci
|
|
# Keyed on the resolved @playwright/test version, so a version bump is a
|
|
# miss and anything else is a hit. The download was 50s on run
|
|
# 35535225637 and is pure I/O — nothing about it depends on this commit.
|
|
- name: Playwright version
|
|
id: pw
|
|
working-directory: harness
|
|
run: echo "v=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT"
|
|
- uses: actions/cache@v4
|
|
id: pw-cache
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-${{ runner.os }}-${{ steps.pw.outputs.v }}
|
|
# All three, because improvise-xf1's acceptance criterion names Chrome,
|
|
# Firefox and Safari and `createSyncAccessHandle` is implemented
|
|
# separately by each engine. WebKit is installed even though its OPFS is
|
|
# absent today: the spec skips on a runtime probe, so the day that build
|
|
# gains OPFS the check starts running without a CI change.
|
|
# `--with-deps` installs apt packages OUTSIDE the cached directory, so it
|
|
# runs on a cache hit too; only the browser download is skipped.
|
|
- name: Install browsers
|
|
working-directory: harness
|
|
run: npx playwright install --with-deps chromium firefox webkit
|
|
# The tree `build` assembled, not one this job rebuilds. Same bytes as
|
|
# the Pages artifact, modulo the cache-bust token `web/assemble.sh`
|
|
# stamps — the harness tree is stamped `harness` and the site the commit
|
|
# SHA, which was already true when both jobs built their own.
|
|
- name: Download the harness tree
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: harness-serve
|
|
path: harness/serve
|
|
# `web/pkg` and `web/worker-pkg` as wasm-pack left them, for the .d.ts
|
|
# the step below reads. The artifact's root is `web/`, so this restores
|
|
# both directories at the paths the test resolves relative to itself.
|
|
- name: Download the wasm bundles
|
|
uses: actions/download-artifact@v7
|
|
with:
|
|
name: wasm-bundles
|
|
path: web
|
|
# The JS↔wasm boundary: every call web/app.js makes is declared by the
|
|
# bundle, with that arity. It needs the bundle and nothing else, so it
|
|
# runs here rather than in the `js` job, and before Playwright — a
|
|
# renamed export is cheaper to read here than as a blank page.
|
|
- name: The page's calls match the bundle it calls
|
|
run: ./scripts/node-test.sh 'web/tests/bundle/*.test.js'
|
|
- name: Run the harness
|
|
working-directory: harness
|
|
run: npx playwright test
|
|
- if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: playwright-traces
|
|
path: harness/test-results
|
|
retention-days: 7
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# GitHub Pages delivery (improvise-d31).
|
|
#
|
|
# The artifact is uploaded by `build`, which runs on every push and pull
|
|
# request, so a change that breaks `web/build.sh` or the bundle fails on the
|
|
# PR that made it — the build is a gate. `pages-deploy` runs only on a push
|
|
# to main and `needs` the check jobs, so nothing that failed `cargo test`,
|
|
# the wasm32 gate or the browser harness can reach the published site. The
|
|
# deploy depends on those checks rather than repeating them (improvise-d31
|
|
# AC 2).
|
|
#
|
|
# Not path-filtered, deliberately. The bundles are `improvise-wasm-client` and
|
|
# `improvise-worker-server` plus, between them, most of the workspace — the
|
|
# worker bundle is the whole App: core, formula, io, protocol and command. A
|
|
# filter
|
|
# that meant "skip when the bundle cannot have changed" would have to name
|
|
# most of the workspace, and the failure mode of getting it wrong is a stale
|
|
# site with no signal that it is stale. A redundant build is the cheaper error.
|
|
#
|
|
# On improvise-864a: the mechanism is NOT supersession and NOT queue depth —
|
|
# another run's SUCCESSFUL deployment invalidates this run's `github-pages`
|
|
# artifact, measured over 100 runs. Building once shortens the upload→deploy
|
|
# window (182s → ~115s on the run 35535225637 shape), and the window is where
|
|
# the whole risk lives. It narrows the race; it does not close it. The
|
|
# evidence and what it rules out: context/code/tooling.md §3.
|
|
# `needs` is EVERY check job, which is the only rule that does not go stale.
|
|
# `kb` was the one absent, on the unstated ground that kb.pl is not shipped by
|
|
# the site — but neither is `harness-lint`'s crate, and it was in the list, so
|
|
# the ground proved too much (improvise-x7cq). A failing consistency check
|
|
# means main is broken, and publishing from a broken main is what this list
|
|
# exists to stop.
|
|
pages-deploy:
|
|
name: Deploy to GitHub Pages
|
|
needs: [test, wasm, js, browser, harness-lint, kb, build]
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
pages: write
|
|
id-token: write
|
|
# A superseded deploy is allowed to finish rather than being cancelled:
|
|
# `deploy-pages` cancelled midway can leave the Pages deployment in a
|
|
# half-applied state, and the next push republishes anyway.
|
|
concurrency:
|
|
group: pages
|
|
cancel-in-progress: false
|
|
environment:
|
|
name: github-pages
|
|
url: ${{ steps.deployment.outputs.page_url }}
|
|
steps:
|
|
# `enablement: false` (the default) configures the environment URL
|
|
# without attempting to create or modify the Pages site via the API.
|
|
# GitHub Pages must be enabled once in the repository settings
|
|
# (Settings → Pages → Source: GitHub Actions) before this job will
|
|
# produce a live URL. Using `enablement: true` fails when the workflow
|
|
# token lacks admin permission to create the site (improvise-3gy).
|
|
- uses: actions/configure-pages@v6
|
|
with:
|
|
enablement: false
|
|
- id: deployment
|
|
uses: actions/deploy-pages@v5
|