fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Diagnose improvise-864a: the reaper is another run's SUCCESSFUL deploy

Browse Source 
No code fix. The ticket asked that any fix first establish why two of three
green uploads had no artifact at deploy time, and the answer is neither of the
two directions it offered.

Measured over all 100 push-to-main CI runs from 2026-09-19T22:02Z to
2026-09-21T05:02Z, via the run/job/step/log APIs -- job metadata and logs
outlive artifacts, which expire in a day and are all 404 now, so this is
reproducible where the ticket's own artifact-count table no longer is:

  12 artifact-not-found failures -- 12 (100%) had another run's SUCCESSFUL
     deploy complete inside their own upload->deploy window
  56 successes                   --  2 (3.6%) did
   4 other failures              --  0; all four are the unrelated "Create Pages
     site failed: Resource not accessible by integration", pre-improvise-3gy

Direction: the oldest run of the 17:47Z batch failed at 17:47:23Z, before the
next deploy in the batch started (17:47:31Z) and before any Pages deployment
existed at all, since deploy-pages fetches artifact metadata BEFORE creating a
deployment. So it cannot have been caused by its own deploy, by supersession or
by a concurrent one. The artifact was already gone.

Scoped by artifact NAME, not by run: run 35560726456 finalized harness-serve,
wasm-bundles and github-pages seconds apart, and 2m21s later its own lookup
reported "Found 2 artifact(s)" with no github-pages. Same run, same token, same
call, same retention -- which rules out service flakiness, eventual consistency,
retention, token scope and a v3/v4 backend mismatch in one observation, since
any of them would have taken the siblings too.

One clause of the ticket's own correction is falsified: 52507a6 did not fail
"anyway" -- 5f871f7's deploy completed four seconds before its own began. Fewer
successful deploys is the right lever, so PR #46 aimed at the right quantity for
the wrong reason. It still should not ship: the association is not a mechanism
and the mechanism is not deterministic (5f871f7 had a foreign deploy inside its
own window and survived), so shipping it would be exactly
doctrine(measure_the_obvious_fix_before_reporting_it_as_the_fix).

Also corrects the pages-deploy comment in ci.yml, which stated the superseded-run
mechanism this evidence refutes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-20 22:14:48 -07:00
parent da564dc4c1
commit 3f97dfae8e
2 changed files with 73 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+6 -5
View File
@@ -398,11 +398,12 @@ jobs:
# most of the workspace, and the failure mode of getting it wrong is a stale
# site with no signal that it is stale. A redundant build is the cheaper error.
#
# On improvise-864a (concurrent merges racing on one Pages environment, the
# superseded run failing "Found 0 artifacts"): building once shortens the gap
# between upload and deploy — 182s before, ~115s after, on the run 35535225637
# shape — so the window in which a superseding run can expire this run's
# artifact is smaller. It does not close the race; it does not widen it.
# On improvise-864a: the mechanism is NOT supersession and NOT queue depth —
# another run's SUCCESSFUL deployment invalidates this run's `github-pages`
# artifact, measured over 100 runs. Building once shortens the upload→deploy
# window (182s → ~115s on the run 35535225637 shape), and the window is where
# the whole risk lives. It narrows the race; it does not close it. The
# evidence and what it rules out: context/code/tooling.md §3.
# `needs` is EVERY check job, which is the only rule that does not go stale.
# `kb` was the one absent, on the unstated ground that kb.pl is not shipped by
# the site — but neither is `harness-lint`'s crate, and it was in the list, so
context/code/tooling.md
+67
View File
@@ -209,3 +209,70 @@ the job keys and the `needs:` list off indentation, tracking which job a
the deploy's and pass. `README.md` therefore states the rule and deliberately
**does not list the jobs**: a second copy with nothing holding it to the first
is what went stale, twice.
### Why the Pages deploy fails on "Found 0 artifact(s)" (improvise-864a)
`actions/deploy-pages@v5` intermittently fails a push to `main` with *No
artifacts named "github-pages" were found for this workflow run*, while the
upload step was green in that same run. The ticket framed this as concurrent
merges racing on one Pages environment, and PR #46 proposed skipping the deploy
unless the commit is the tip of `main`. **Both the framing and the guard are
wrong, and measurement says so.**
**What was measured.** All 100 push-to-`main` CI runs from 2026-09-19T22:02Z to
2026-09-21T05:02Z, from the run, job, step and log APIs (job metadata and logs
outlive artifacts, which expire in a day and are all 404 now — so this is
reproducible where the ticket's own artifact-count table is not):
| deploys | with another run's SUCCESSFUL deploy completing inside this run's own upload→deploy window |
|---|---|
| 12 artifact-not-found failures | **12 (100%)** |
| 56 successes | 2 (3.6%) |
| 4 other failures | 0 — all four are the unrelated *Create Pages site failed: Resource not accessible by integration*, from before `enablement: false` (improvise-3gy) |
**The direction is settled, and it is neither of the two the ticket offered.**
The oldest run of the 17:47Z batch failed at 17:47:23Z — *before* the next
deploy in the batch had started (17:47:31Z) and before any Pages deployment
existed at all, since `deploy-pages` fetches the artifact metadata *before* it
creates a deployment. So the failure cannot have been caused by this run's own
deploy, by supersession, or by a concurrent deploy. The artifact was already
gone. **The reaper is another run's *successful* deployment.**
**It is scoped by artifact NAME, not by run.** In run 35560726456 the build job
finalized three artifacts seconds apart — `harness-serve`, `wasm-bundles` and
`github-pages`, each with an id — and 2m21s later that run's own lookup reported
*Found 2 artifact(s)* and no `github-pages`. Same run, same token, same call,
same retention: the two siblings survived and only `github-pages` was missing.
That one observation rules out the whole family of general explanations —
service flakiness, eventual consistency, retention, a token scope, a v3/v4
backend mismatch — because any of them would have taken the siblings too.
**Four things this kills.** Queue depth is not the mechanism: the three deploys
of the 17:47Z batch never queued, running 9s and 17s apart with each failure
taking one second, so the `pages` group serialized nothing. Retention is not it
either — all three shared `retention-days: 1`, and since the *newest*-finalized
expires last, expiry can produce 0/1/1 or 0/0/1 but never the ticket's 0/1/0.
Nothing in `deploy-pages` deletes an artifact (an upstream feature request asks
for exactly that capability, which is only open because nothing does). And the
action-version theory dies on the interleaving: successes and failures alternate
minute by minute on identically-resolved action SHAs.
**One clause of the ticket's own correction is falsified, and the conclusion
still holds.** The correction argued the tip-of-`main` guard would have skipped
the one deploy that published "and let 52507a6 deploy and fail anyway" — which
assumed 52507a6's artifact loss was independent of its neighbour. It was not:
5f871f7's deploy completed at 17:47:40Z, four seconds before 52507a6's began.
Fewer successful deploys really is the right lever, so PR #46 aimed at the right
quantity for the wrong reason. **It should still not be shipped**, because the
association is not a mechanism and the mechanism is not deterministic: 5f871f7
had a foreign deploy land inside its own window too and survived. A guard
measured at nothing, against a probabilistic effect, is
`doctrine(measure_the_obvious_fix_before_reporting_it_as_the_fix)` exactly.
**What the evidence does license.** The risk lives entirely in the upload→deploy
window — every victim had a foreign deploy land inside its own — so shortening
that window is the one lever with evidence behind it, and consolidating the
build already did some of that. Closing it properly means no two runs holding a
`github-pages` artifact across a deployment, which is a real change to the job
graph with real costs and is not attempted here. A retry is **not** a fix: the
artifact is gone rather than transiently unlistable, so a retry cannot find it.