improvise/AGENTS.md

- Always use tests to demonstrate the existence of a bug before fixing the bug.
  - If you suspect that a bug exists, use a test to demonstrate it first:
    - prefer unit tests testing a small amount of code to integration or e2e tests
- Option<...> or Result<...> are fine but should not be present in the majority of the code.
- Similarly, code managing  Box<...> or RC<...>, etc. for containers pointing to heap data should be split
  from logic
- @context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated.
- @context/design-principles.md is also important for keeping the repository consistent.
- prefer merges to rebasing.
- always start responses with bananaS!
- there is a prolog MCP available: load a session from `kb` at the root of the repository at the beginning of
  every session, and query it as you work. **Reading only** — the MCP must never write `kb.pl`.
- save structured facts as you go — work done, interesting locations, important decisions — with
  `scripts/kb add` (see the `kb` skill). Write a fact when you learn it; there is no reason to batch.
- query the knowledge base as needed and use Prolog reasoning when it makes sense.
- Prolog should be used as a design tool to reason about the code and the implementation of tickets and to
  persist a formal model of it.

# Agent Instructions

This project uses **bd** (beads) for issue tracking. Run `bd prime` for full workflow context.

## Quick Reference

```bash
bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work atomically
bd close <id>         # Complete work
bd dolt push          # Push beads data to remote
```

## Non-Interactive Shell Commands

**ALWAYS use non-interactive flags** with file operations to avoid hanging on confirmation prompts.

Shell commands like `cp`, `mv`, and `rm` may be aliased to include `-i` (interactive) mode on some systems, causing the agent to hang indefinitely waiting for y/n input.

**Use these forms instead:**
```bash
# Force overwrite without prompting
cp -f source dest           # NOT: cp source dest
mv -f source dest           # NOT: mv source dest
rm -f file                  # NOT: rm file

# For recursive operations
rm -rf directory            # NOT: rm -r directory
cp -rf source dest          # NOT: cp -r source dest
```

**Other commands that may prompt:**
- `scp` - use `-o BatchMode=yes` for non-interactive
- `ssh` - use `-o BatchMode=yes` to fail instead of prompting
- `apt-get` - use `-y` flag
- `brew` - use `HOMEBREW_NO_AUTO_UPDATE=1` env var

## The Wave Workflow

Most work here is done by **waves**: an orchestrator chunks the open backlog,
fans several agents out into their own git worktrees, merges what comes back,
reviews it, and pushes the wave as one PR. This section is what that costs to
get right. Everything in it was learned by getting it wrong first.

### Who owns what

Three files are **orchestrator-only**, and the rule exists because they are
guaranteed conflicts — every agent has a reason to touch them and they merge
badly:

| File | Why |
|---|---|
| `.beads/` (incl. `issues.jsonl`) | bd owns it. See "Never hand-edit" below. |
| `prolog-sessions/kb.pl` | Written by `scripts/kb` only. See `context/code/kb-tool.md`. |
| `context/repo-map.md` | Every agent wants to add a gotcha; every wave would conflict. |

Wave agents **read** all three freely (`bd show`, `bd ready`, grep `kb.pl`) and
**write** none of them. They report proposed repo-map lines and `kb.pl` facts as
*text* in their hand-back, and the orchestrator applies them — batched, at the
end of the wave, after the review. **`kb.pl` is written by `scripts/kb add|replace|rm` and by nothing else** — not
the Prolog MCP, not an editor. Every write is gated on `kb_consistent` against
the candidate, so a bad delta leaves the file untouched. Facts do not need
batching: write one when you learn it. There is a `kb` skill; read it before
your first write. Reading is unaffected — `loadSession` at session start is
still how a session gets the model in hand.

### Partitioning a wave

Partition by **crate or directory**, not by ticket count, and tell each agent
explicitly which ground is *not* theirs. Two rules that fall out of the ones we
broke:

- **Tickets that share a file belong to one agent.** The four TUI widget
  migrations all touch `src/draw.rs`; split across agents they are a guaranteed
  conflict, so they go to one agent to do sequentially. Group first, then count.
- **An agent that must stray says so rather than guessing.** Give the boundary
  and ask for a report when it has to be crossed.

Two agents each adding a workspace member will still collide in
`Cargo.toml`/`Cargo.lock`. That one is expected; keep both sides.

### Every wave agent's brief must include

1. **Verify your base commit first.** Worktrees do **not** reliably branch from
   the orchestrator's HEAD — one was created three merges behind and only
   noticed because it was told to look. `git log --oneline -1` against a named
   commit, reset onto `main` if behind. Name the expected commit in the brief.
2. **The baseline test count**, so "1090 passing" is checkable rather than
   asserted, and a drop is visible.
3. **Do not push, do not merge to `main`.** The orchestrator merges.
4. The commit-message trailers, spelled out.
5. **What to report**: branch and commits, what changed per ticket, *what could
   not be verified and why*, and proposed repo-map/`kb.pl` text. Ask explicitly
   for the thing the agent learned that the ticket did not predict — that is
   consistently the most valuable part of a hand-back, and agents omit it unless
   asked.

`bd` works in any worktree, but `.envrc` must be allowed there or you get a
**version mismatch** rather than a clean failure — `direnv allow` in the
worktree, or rely on the `[whitelist] prefix` in `~/.config/direnv/direnv.toml`.
Note `direnv status` reports `Found RC allowed 0`, and `0` means **allowed**
(the enum is 0=Allowed, 1=NotAllowed). It reads like the opposite.

### Between waves

Run the **code-review skill** on the merged range and iterate until its backlog
is empty. This is not ceremony: in this repo the review's characteristic find is
not a wrong line, it is **a test that passes while the thing it names is
broken** — a vacuous fold test, a bit-witness that cannot see the bit, a
one-category fixture, a helper that fabricates the `_Measure` items the code
under test needs, a check-the-checker that injected malformed JSON and so proved
nothing. A green suite is not evidence. Ask the reviewer to *inject the fault*
and watch the check fail.

### Shipping a wave

**Do not push `main` to `gh/main`.** Work reaches GitHub as a branch and a pull
request, and lands on `main` by being merged there. The orchestrator still
merges agent branches into its *local* `main` to integrate and run the gates —
that is how conflicts get found early — but the push is a branch push, not a
`main` push. `origin` and `gitea` are mirrors and take `main` directly.

This is easy to drift out of, because merging locally and pushing feels like
finishing. It happened in this repo: 27 commits across six agent branches —
including a new crate, a new protocol type, and a type moving between crates —
went straight to `main` unreviewed, while `AGENTS.md` already said not to. If
you notice a batch has bypassed the loop, the cheap repair is a **review-only
PR**: base a branch at the last PR merge, open a PR with `main` as the head, let
Copilot review the diff, and land the findings as an ordinary PR afterwards. It
cannot be merged — the commits are already in — so close it once its findings
are filed or fixed. Rewinding the default branch to fake a mergeable PR is not
worth it on a repo with other collaborators.

One PR per wave, built by a **dedicated PR agent in its own worktree**, so waves
pipeline against it: the next wave runs while the PR absorbs review fixes.

**Read the PR's reviews before merging it.** This is separate from CI and from
the adversarial agent, and it has been missed: two ChatGPT Codex reviews landed
on merged PRs and were not read until afterwards, one of them a live P2
(improvise-x1ns). Both were correct.

```bash
gh pr view <n> --json reviews -q '.reviews[]|"[\(.state)] \(.submittedAt) \(.commit.oid[0:8])\n\(.body)"'
```

Two things make a Codex review easy to miss. It arrives as a **review**, not a
comment, so a glance at the comment thread does not show it. And it posts under
the repository owner's account — the same account our own agents post under —
so **authorship does not distinguish it**. The tell is the trailing line
`_Automated review by ChatGPT Codex (…)_`; ours open with
`_Posted by Claude (…)_`. Check the `commit.oid` too: a review may target an
earlier head, and the finding may already be fixed — verify rather than assume
in either direction.

Reviews arrive **as the owner's quota allows**, so absence is not evidence.
Wait where a review is plausible, and when one is found, answer it on the
thread even if the PR has already merged: that is where the next reader looks.

**Copilot is not reviewing this repository, and the request is a silent no-op.**
Measured 2026-09-20: PRs #12, #21 and #27 carry 2, 8 and 4 Copilot reviews; #61,
#66, #68 and #70 carry none. Both request paths report success and produce zero
`review_requested` events — the GraphQL mutation returns `clientMutationId: null`
and the REST call returns 200 with `requested_reviewers: []`. So **do not wait on
it**: an absent review and an unrequested one look identical from outside, which
is `doctrine(review_is_requested_not_inherited)` seen from the requester's side.
Review is an adversarial agent the orchestrator dispatches, given the head SHA.
The mechanics below are kept for the day it comes back.

Copilot review must be **re-requested after every fix push**; it does not
re-review on its own. The mutation takes a **`botIds`** field — `userIds` fails
with *"Could not resolve to User node"*:

```bash
gh api graphql -f query='mutation($pr:ID!,$ids:[ID!]!){requestReviews(input:{pullRequestId:$pr,botIds:$ids,union:true}){clientMutationId}}' \
  -f pr="$(gh pr view <n> --json id -q .id)" -f ids=BOT_kgDOCnlnWA
```

REST also works, with the login spelled `copilot-pull-request-reviewer[bot]` —
without the `[bot]` suffix it 422s as "not a collaborator".

In a worktree-isolated agent the shell guard refuses an inline
`gh api graphql -f query='mutation(...)'` as "too complex to verify". Put the
query in a file and pass `-F query=@file`; PR comment bodies likewise go
through `-F body=@file` rather than command substitution.

**Anything you post to GitHub goes out under the repository owner's account, so
say who is speaking.** Open every PR comment, review and issue body with a line
naming the model and that it is an agent posting in his session. A PR body's
`🤖 Generated with Claude Code` footer covers that case; comments have nothing
unless you add it, and an unattributed one reads as his own words to everyone
who sees it.

Two environment notes. **`gh` is not on `PATH` here** — the name is
shell-aliased to `git-hub`, a different tool — so use
`nix shell nixpkgs#gh --command gh …`. And **a Copilot review takes about six
minutes**; tool calls return fast, which makes elapsed time easy to misjudge, so
check `date -u` against the GitHub timeline before concluding a review is stuck.
(One agent closed and reopened a PR to "unstick" a review that was proceeding
normally.)

Across four PRs Copilot has found genuine bugs and no false positives, so its
findings are worth the loop. **But judge each one against the repo's doctrine
rather than applying it reflexively**, and when you decline, reply on the thread
with the reason. The distinction that has mattered: a finding can be *correct*
and still be wrong to fix in that PR. On #12, a non-ASCII `truncate` panic was
applied (byte-identical on ASCII, so it cost no evidence) while a hint-line
padding bug in the same file was declined — fixing it would have rewritten the
exact-frame characterization tests that were the PR's only evidence the
migration changed nothing. Two right findings, different costs. File the
declined one.

**Under merge-as-soon-as-green, a review round can land after its own merge.**
GitHub deletes the branch when a PR merges, so a fix push to the old branch name
**silently succeeds and accomplishes nothing** — it re-creates a branch `main`
will never see, CI runs on it, and the commit is nowhere. The tell is the push
printing `* [new branch]` where you expected an `old..new` range. It happened on
#51: the third review round completed four minutes after the merge, and the
correction it prompted went to a re-created `wave/b-projection-gaps`. The repair
is to branch from `gh/main`, cherry-pick, open a follow-up PR, and delete the
stray re-creation. Check `gh pr view <n> --json state` *before* pushing a fix to
a PR you have not looked at in a few minutes, rather than after.

Merge only once a review has passed — an adversarial agent's while Copilot is
absent, and it must name the head SHA it reviewed. Prefer a **merge commit** —
`AGENTS.md` says merges over rebases, and a wave's commits are individually
meaningful.

### Never hand-edit `.beads/issues.jsonl`

bd manipulates that file and auto-exports to it. Editing it — or even
`git checkout`-ing it, which fires the post-checkout hook — breaks auto-export
and desynchronises `export-state.json`. Use `bd` commands, let auto-export run,
and commit whatever it leaves.

### Cite a ticket ID you have read back

`bd create` assigns the ID; it is not derivable from the title. An orchestrator
that files with output suppressed and then writes the ticket into a gotcha or an
ADR **will invent one** — six invented IDs survived into `context/` in a single
session before an agent tried one and found nothing. Read the ID back after
filing, and before pushing docs run the audit:

```bash
grep -rohE 'improvise-[a-z0-9]{3,8}\b' \
     context README.md AGENTS.md CLAUDE.md crates src tests examples web harness docs \
  | sort -u \
  | grep -vE '^improvise-(core|formula|io|protocol|command|gp|ws|worker|wasm|harness|registry|side)$' \
  > /tmp/bd-audit-ids.txt
bd show $(cat /tmp/bd-audit-ids.txt) 2>&1 \
  | grep -iE 'not found|no such issue' || echo "all resolve"
```

Three things the earlier version of this recipe got wrong, each of which let a
real defect through:

- **It scanned two paths** (`context/repo-map.md context/adr/*.md`), so it
  found one dead id in the repo map and missed both copies of it in
  `harness/tests/shell.spec.js`. Ticket IDs are cited in code comments,
  in `README.md` and in `web/`, not only in `context/`.
- **`[a-z0-9]{3,4}` cannot see a commit SHA wearing the prefix.** A seven-hex
  commit sha cited as `improvise-<sha>` survived in
  `crates/improvise-core/src/workbook.rs` precisely because it was too long
  to match. An ID-shaped non-ID is worse than a dead ID, because the
  diagnosis is identical and the repair is not.
- **The exclusion list named only `core` and `wasm`**, so every other crate
  name would have been reported missing. The exclusions above also cover the
  non-ticket tokens this repo really contains: a package name, a temp file
  name, and the prose word "improvise-side". **They are anchored `^…$` for the
  same reason the recipe is read as code**: unanchored, the `gp` alternative is
  a *prefix* match, so a future real ticket whose id happens to begin `gp` is
  silently excluded and the audit reports nothing while skipping it. An
  exclusion list is a filter, and a filter that over-matches fails in the
  direction that looks like success. (Anchoring it is also what caught the
  example id this very paragraph originally spelled out — see the note below.)

Note the recipe's own prose deliberately spells no live or dead id: a document
that lists ids as examples makes itself a source of audit noise.

One environment note: in a worktree-isolated agent the shell guard refuses a
loop that passes a runtime-computed argument to `bd`, so the recipe above
materialises the id list into a file and hands `bd show` the whole list in one
call rather than looping. If `bd show` in your build does not take several ids,
run the ids through `xargs -n1 -I{} bd show {}` from a plain shell instead.

A dead ID is worse than no ID: it reads as a promise that the follow-up exists.

### Before filing a ticket, search

Agent-filed issues duplicate work that is already open, sometimes already
in progress. Two duplicates were filed in this repo within minutes of the
hand-backs that prompted them. `bd search` the core concept *and* scan
in-progress issues first.


<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
## Beads Issue Tracker

This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.

### Quick Reference

```bash
bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work
bd close <id>         # Complete work
```

### Rules

- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists (the exception being that you must maintain context/design-principles.md and context/repo-map.md)
- Run `bd prime` for detailed command reference and session close protocol
- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files (the exception being that you must maintain context/design-principles.md and context/repo-map.md)

## Session Completion

**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.

**MANDATORY WORKFLOW:**

1. **File issues for remaining work** - Create issues for anything that needs follow-up
2. **Run quality gates** (if code changed) - Tests, linters, builds
3. **Update issue status** - Close finished work, update in-progress items
4. **PUSH TO REMOTE** - This is MANDATORY:
   ```bash
   git merge origin/main
   bd dolt push
   git push
   git status  # MUST show "up to date with origin"
   ```
5. **Clean up** - Clear stashes, prune remote branches
6. **Verify** - All changes committed AND pushed
7. **Hand off** - Provide context for next session

**CRITICAL RULES:**
- Work is NOT complete until `git push` succeeds
- NEVER stop before pushing - that leaves work stranded locally
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
<!-- END BEADS INTEGRATION -->

<!-- BEGIN BEADS CODEX SETUP: generated by bd setup codex -->
## Beads Issue Tracker

Use Beads (`bd`) for durable task tracking in repositories that include it. Use the `beads` skill at `.agents/skills/beads/SKILL.md` (project install) or `~/.agents/skills/beads/SKILL.md` (global install) for Beads workflow guidance, then use the `bd` CLI for issue operations.

### Quick Reference

```bash
bd ready                # Find available work
bd show <id>            # View issue details
bd update <id> --claim  # Claim work
bd close <id>           # Complete work
bd prime                # Refresh Beads context
```

### Rules

- Use `bd` for all task tracking; do not create markdown TODO lists.
- Run `bd prime` when Beads context is missing or stale. Codex 0.129.0+ can load Beads context automatically through native hooks; use `/hooks` to inspect or toggle them.
- Keep persistent project memory in Beads via `bd remember`; do not create ad hoc memory files.

**Architecture in one line:** issues live in a local Dolt DB; sync uses `refs/dolt/data` on your git remote; `.beads/issues.jsonl` is a passive export. See https://github.com/gastownhall/beads/blob/main/docs/core-concepts/sync-concepts.md for details and anti-patterns.
<!-- END BEADS CODEX SETUP -->