improvise/AGENTS.md

- Always use tests to demonstrate the existence of a bug before fixing the bug.
  - If you suspect that a bug exists, use a test to demonstrate it first:
    - prefer unit tests testing a small amount of code to integration or e2e tests
- Option<...> or Result<...> are fine but should not be present in the majority of the code.
- Similarly, code managing  Box<...> or RC<...>, etc. for containers pointing to heap data should be split
  from logic
- @context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated.
- @context/design-principles.md is also important for keeping the repository consistent.
- prefer merges to rebasing.
- always start responses with bananaS!
- there is a prolog MCP available, load a session from `kb` at the root of the repository at the beginning of
  every session and save structured facts to the knowledge base as you go. (cover work done, interesting
  locations in the codebase, important decisions, etc.)
- query the knowledge base as needed and use Prolog reasoning when it makes sense.
- Prolog should be used as a design tool to reason about the code and the implementation of tickets and to
  persist a formal model of it.

# Agent Instructions

This project uses **bd** (beads) for issue tracking. Run `bd prime` for full workflow context.

## Quick Reference

```bash
bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work atomically
bd close <id>         # Complete work
bd dolt push          # Push beads data to remote
```

## Non-Interactive Shell Commands

**ALWAYS use non-interactive flags** with file operations to avoid hanging on confirmation prompts.

Shell commands like `cp`, `mv`, and `rm` may be aliased to include `-i` (interactive) mode on some systems, causing the agent to hang indefinitely waiting for y/n input.

**Use these forms instead:**
```bash
# Force overwrite without prompting
cp -f source dest           # NOT: cp source dest
mv -f source dest           # NOT: mv source dest
rm -f file                  # NOT: rm file

# For recursive operations
rm -rf directory            # NOT: rm -r directory
cp -rf source dest          # NOT: cp -r source dest
```

**Other commands that may prompt:**
- `scp` - use `-o BatchMode=yes` for non-interactive
- `ssh` - use `-o BatchMode=yes` to fail instead of prompting
- `apt-get` - use `-y` flag
- `brew` - use `HOMEBREW_NO_AUTO_UPDATE=1` env var

## The Wave Workflow

Most work here is done by **waves**: an orchestrator chunks the open backlog,
fans several agents out into their own git worktrees, merges what comes back,
reviews it, and pushes the wave as one PR. This section is what that costs to
get right. Everything in it was learned by getting it wrong first.

### Who owns what

Three files are **orchestrator-only**, and the rule exists because they are
guaranteed conflicts — every agent has a reason to touch them and they merge
badly:

| File | Why |
|---|---|
| `.beads/` (incl. `issues.jsonl`) | bd owns it. See "Never hand-edit" below. |
| `prolog-sessions/kb.pl` | MCP-maintained, whole-predicate replacement. |
| `context/repo-map.md` | Every agent wants to add a gotcha; every wave would conflict. |

Wave agents **read** all three freely (`bd show`, `bd ready`, grep `kb.pl`) and
**write** none of them. They report proposed repo-map lines and `kb.pl` facts as
*text* in their hand-back, and the orchestrator applies them — batched, at the
end of the wave, after the review. Batching matters for `kb.pl` specifically:
`loadProgram` **replaces a whole predicate**, so every addition means re-sending
every existing clause of `doctrine/2`, `decision/3`, etc. Doing that once per
wave instead of once per fact is the difference between cheap and painful. It
has already silently wiped 22 decisions and 19 doctrines once — count with
`findall/length` before and after, and check the count before saving.

`saveSession` re-adds the MCP's own `host_rpc` shims to `kb.pl` every time.
Strip them after each save.

### Partitioning a wave

Partition by **crate or directory**, not by ticket count, and tell each agent
explicitly which ground is *not* theirs. Two rules that fall out of the ones we
broke:

- **Tickets that share a file belong to one agent.** The four TUI widget
  migrations all touch `src/draw.rs`; split across agents they are a guaranteed
  conflict, so they go to one agent to do sequentially. Group first, then count.
- **An agent that must stray says so rather than guessing.** Give the boundary
  and ask for a report when it has to be crossed.

Two agents each adding a workspace member will still collide in
`Cargo.toml`/`Cargo.lock`. That one is expected; keep both sides.

### Every wave agent's brief must include

1. **Verify your base commit first.** Worktrees do **not** reliably branch from
   the orchestrator's HEAD — one was created three merges behind and only
   noticed because it was told to look. `git log --oneline -1` against a named
   commit, reset onto `main` if behind. Name the expected commit in the brief.
2. **The baseline test count**, so "1090 passing" is checkable rather than
   asserted, and a drop is visible.
3. **Do not push, do not merge to `main`.** The orchestrator merges.
4. The commit-message trailers, spelled out.
5. **What to report**: branch and commits, what changed per ticket, *what could
   not be verified and why*, and proposed repo-map/`kb.pl` text. Ask explicitly
   for the thing the agent learned that the ticket did not predict — that is
   consistently the most valuable part of a hand-back, and agents omit it unless
   asked.

`bd` works in any worktree, but `.envrc` must be allowed there or you get a
**version mismatch** rather than a clean failure — `direnv allow` in the
worktree, or rely on the `[whitelist] prefix` in `~/.config/direnv/direnv.toml`.
Note `direnv status` reports `Found RC allowed 0`, and `0` means **allowed**
(the enum is 0=Allowed, 1=NotAllowed). It reads like the opposite.

### Between waves

Run the **code-review skill** on the merged range and iterate until its backlog
is empty. This is not ceremony: in this repo the review's characteristic find is
not a wrong line, it is **a test that passes while the thing it names is
broken** — a vacuous fold test, a bit-witness that cannot see the bit, a
one-category fixture, a helper that fabricates the `_Measure` items the code
under test needs, a check-the-checker that injected malformed JSON and so proved
nothing. A green suite is not evidence. Ask the reviewer to *inject the fault*
and watch the check fail.

### Shipping a wave

**Do not push `main` to `gh/main`.** Work reaches GitHub as a branch and a pull
request, and lands on `main` by being merged there. The orchestrator still
merges agent branches into its *local* `main` to integrate and run the gates —
that is how conflicts get found early — but the push is a branch push, not a
`main` push. `origin` and `gitea` are mirrors and take `main` directly.

This is easy to drift out of, because merging locally and pushing feels like
finishing. It happened in this repo: 27 commits across six agent branches —
including a new crate, a new protocol type, and a type moving between crates —
went straight to `main` unreviewed, while `AGENTS.md` already said not to. If
you notice a batch has bypassed the loop, the cheap repair is a **review-only
PR**: base a branch at the last PR merge, open a PR with `main` as the head, let
Copilot review the diff, and land the findings as an ordinary PR afterwards. It
cannot be merged — the commits are already in — so close it once its findings
are filed or fixed. Rewinding the default branch to fake a mergeable PR is not
worth it on a repo with other collaborators.

One PR per wave, built by a **dedicated PR agent in its own worktree**, so waves
pipeline against it: the next wave runs while the PR absorbs review fixes.

Copilot review must be **re-requested after every fix push**; it does not
re-review on its own. The mutation takes a **`botIds`** field — `userIds` fails
with *"Could not resolve to User node"*:

```bash
gh api graphql -f query='mutation($pr:ID!,$ids:[ID!]!){requestReviews(input:{pullRequestId:$pr,botIds:$ids,union:true}){clientMutationId}}' \
  -f pr="$(gh pr view <n> --json id -q .id)" -f ids=BOT_kgDOCnlnWA
```

REST also works, with the login spelled `copilot-pull-request-reviewer[bot]` —
without the `[bot]` suffix it 422s as "not a collaborator".

In a worktree-isolated agent the shell guard refuses an inline
`gh api graphql -f query='mutation(...)'` as "too complex to verify". Put the
query in a file and pass `-F query=@file`; PR comment bodies likewise go
through `-F body=@file` rather than command substitution.

Two environment notes. **`gh` is not on `PATH` here** — the name is
shell-aliased to `git-hub`, a different tool — so use
`nix shell nixpkgs#gh --command gh …`. And **a Copilot review takes about six
minutes**; tool calls return fast, which makes elapsed time easy to misjudge, so
check `date -u` against the GitHub timeline before concluding a review is stuck.
(One agent closed and reopened a PR to "unstick" a review that was proceeding
normally.)

Across four PRs Copilot has found genuine bugs and no false positives, so its
findings are worth the loop. **But judge each one against the repo's doctrine
rather than applying it reflexively**, and when you decline, reply on the thread
with the reason. The distinction that has mattered: a finding can be *correct*
and still be wrong to fix in that PR. On #12, a non-ASCII `truncate` panic was
applied (byte-identical on ASCII, so it cost no evidence) while a hint-line
padding bug in the same file was declined — fixing it would have rewritten the
exact-frame characterization tests that were the PR's only evidence the
migration changed nothing. Two right findings, different costs. File the
declined one.

Merge only once Copilot passes. Prefer a **merge commit** — `AGENTS.md` says
merges over rebases, and a wave's commits are individually meaningful.

### Never hand-edit `.beads/issues.jsonl`

bd manipulates that file and auto-exports to it. Editing it — or even
`git checkout`-ing it, which fires the post-checkout hook — breaks auto-export
and desynchronises `export-state.json`. Use `bd` commands, let auto-export run,
and commit whatever it leaves.

### Cite a ticket ID you have read back

`bd create` assigns the ID; it is not derivable from the title. An orchestrator
that files with output suppressed and then writes the ticket into a gotcha or an
ADR **will invent one** — six invented IDs survived into `context/` in a single
session before an agent tried one and found nothing. Read the ID back after
filing, and before pushing docs run the audit:

```bash
grep -rohE 'improvise-[a-z0-9]{3,8}\b' \
     context README.md AGENTS.md CLAUDE.md crates src tests examples web harness docs \
  | sort -u \
  | grep -vE '^improvise-(core|formula|io|protocol|command|gp|ws|worker|wasm|harness|registry|side)$' \
  > /tmp/bd-audit-ids.txt
bd show $(cat /tmp/bd-audit-ids.txt) 2>&1 \
  | grep -iE 'not found|no such issue' || echo "all resolve"
```

Three things the earlier version of this recipe got wrong, each of which let a
real defect through:

- **It scanned two paths** (`context/repo-map.md context/adr/*.md`), so it
  found one dead id in the repo map and missed both copies of it in
  `harness/tests/shell.spec.js`. Ticket IDs are cited in code comments,
  in `README.md` and in `web/`, not only in `context/`.
- **`[a-z0-9]{3,4}` cannot see a commit SHA wearing the prefix.** A seven-hex
  commit sha cited as `improvise-<sha>` survived in
  `crates/improvise-core/src/workbook.rs` precisely because it was too long
  to match. An ID-shaped non-ID is worse than a dead ID, because the
  diagnosis is identical and the repair is not.
- **The exclusion list named only `core` and `wasm`**, so every other crate
  name would have been reported missing. The exclusions above also cover the
  non-ticket tokens this repo really contains: a package name, a temp file
  name, and the prose word "improvise-side". **They are anchored `^…$` for the
  same reason the recipe is read as code**: unanchored, the `gp` alternative is
  a *prefix* match, so a future real ticket whose id happens to begin `gp` is
  silently excluded and the audit reports nothing while skipping it. An
  exclusion list is a filter, and a filter that over-matches fails in the
  direction that looks like success. (Anchoring it is also what caught the
  example id this very paragraph originally spelled out — see the note below.)

Note the recipe's own prose deliberately spells no live or dead id: a document
that lists ids as examples makes itself a source of audit noise.

One environment note: in a worktree-isolated agent the shell guard refuses a
loop that passes a runtime-computed argument to `bd`, so the recipe above
materialises the id list into a file and hands `bd show` the whole list in one
call rather than looping. If `bd show` in your build does not take several ids,
run the ids through `xargs -n1 -I{} bd show {}` from a plain shell instead.

A dead ID is worse than no ID: it reads as a promise that the follow-up exists.

### Before filing a ticket, search

Agent-filed issues duplicate work that is already open, sometimes already
in progress. Two duplicates were filed in this repo within minutes of the
hand-backs that prompted them. `bd search` the core concept *and* scan
in-progress issues first.


<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
## Beads Issue Tracker

This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.

### Quick Reference

```bash
bd ready              # Find available work
bd show <id>          # View issue details
bd update <id> --claim  # Claim work
bd close <id>         # Complete work
```

### Rules

- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists (the exception being that you must maintain context/design-principles.md and context/repo-map.md)
- Run `bd prime` for detailed command reference and session close protocol
- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files (the exception being that you must maintain context/design-principles.md and context/repo-map.md)

## Session Completion

**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.

**MANDATORY WORKFLOW:**

1. **File issues for remaining work** - Create issues for anything that needs follow-up
2. **Run quality gates** (if code changed) - Tests, linters, builds
3. **Update issue status** - Close finished work, update in-progress items
4. **PUSH TO REMOTE** - This is MANDATORY:
   ```bash
   git merge origin/main
   bd dolt push
   git push
   git status  # MUST show "up to date with origin"
   ```
5. **Clean up** - Clear stashes, prune remote branches
6. **Verify** - All changes committed AND pushed
7. **Hand off** - Provide context for next session

**CRITICAL RULES:**
- Work is NOT complete until `git push` succeeds
- NEVER stop before pushing - that leaves work stranded locally
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
<!-- END BEADS INTEGRATION -->

<!-- BEGIN BEADS CODEX SETUP: generated by bd setup codex -->
## Beads Issue Tracker

Use Beads (`bd`) for durable task tracking in repositories that include it. Use the `beads` skill at `.agents/skills/beads/SKILL.md` (project install) or `~/.agents/skills/beads/SKILL.md` (global install) for Beads workflow guidance, then use the `bd` CLI for issue operations.

### Quick Reference

```bash
bd ready                # Find available work
bd show <id>            # View issue details
bd update <id> --claim  # Claim work
bd close <id>           # Complete work
bd prime                # Refresh Beads context
```

### Rules

- Use `bd` for all task tracking; do not create markdown TODO lists.
- Run `bd prime` when Beads context is missing or stale. Codex 0.129.0+ can load Beads context automatically through native hooks; use `/hooks` to inspect or toggle them.
- Keep persistent project memory in Beads via `bd remember`; do not create ad hoc memory files.

**Architecture in one line:** issues live in a local Dolt DB; sync uses `refs/dolt/data` on your git remote; `.beads/issues.jsonl` is a passive export. See https://github.com/gastownhall/beads/blob/main/docs/core-concepts/sync-concepts.md for details and anti-patterns.
<!-- END BEADS CODEX SETUP -->