improvise/context/code/cli.md

The headless CLI

improvise cmd and improvise script are one function — run_headless_commands in src/main.rs; run_headless_script reads the file and delegates. Every claim below therefore holds for both, and a fix to one is a fix to both by construction rather than by remembering.

1 — the status line is the only channel, so it must be printed

App has no return channel for "what happened". A command's outcome — a refusal, a warning, a fit-measure's reproducing seed — is written to ViewState::status_msg and rendered by the TUI's bottom bar. Headless, nothing read it, so success and refusal were byte-identical: silence and exit 0 (improvise-q914, improvise-j6x8). Two things followed that are worth keeping in mind, because both were mistaken for correct behaviour by agents working above this layer:

• A P1's carefully-worded refusal can be written, tested at the effect, and still reach nobody, because the test asserts the effect's status and the surface that would show it drops it. That is this repo's characteristic defect one layer out, and the only cure is a test at the process boundary — tests/headless_cli.rs, which spawns the real binary.
• improvise cmd 'fit-measure …' looked like it did nothing whether it worked or not, which made the whole GP feature read as broken from a script.

The loop takes the message with std::mem::take and clears it per command, so one command's status cannot be reported twice or attributed to its successor.

2 — stderr, all of it, and why no split was possible

The tempting design is stdout for a report and stderr for a refusal. It cannot be built honestly, because status_msg is a bare String with no severity, and both candidate classifiers are disproved by code in the tree:

• "A refusal is a batch that aborted." Only three effects set abort_effects (gotcha 21). AddItem writes workbook.add_item's Err straight to the status and does not abort — and that is q914's own worked example of a refusal. So this classifier puts a refusal on stdout labelled a success.
• "A refusal is anything that spoke." cmd/fit.rs sets a status on success, carrying the seed that reproduces the fit. So this one labels a deliverable a failure.

Guessing between them would be precisely the adjacent-check defect context/design-principles.md §7 names. So everything goes to stderr: it is the diagnostics stream, it leaves stdout free for a future data output, and it matches every other diagnostic main.rs already emits (Saved to …, warning: …, Parse error: …).

No --quiet flag. Silence is never the default and 2>/dev/null is the standard suppression; a flag would be a second knob with nothing asking for it.

3 — the exit code (superseded by §10)

This section used to say: exit 1 on a parse error or on App::last_batch_aborted(), and a refusal that only writes a status exits 0, because closing that needed a severity beside status_msg — a ViewState change, and so a wire change (gotcha 25). Both halves were wrong in a way worth keeping. The abort half made a success exit 1 (improvise-c5ip), and the severity half named the wrong home: the question an exit code asks is about one batch, not about a message, so it belongs beside batch_aborted on App and never reaches the wire. §10 is what replaced it.

4 — a run that changed nothing does not write

persistence::save used to run unconditionally, so improvise cmd normalised the document on every invocation — 1e20 expanded to twenty digits. The cost was not cosmetic: an agent editing a fixture by string-replacing a value after one run got a silent no-op, and two misleading test results out of it.

The gate is changed, accumulated from model_state.dirty after each command, not read once at the end. Reading it at the end is wrong because effect::Save clears the flag, so [add-item …, w other.improv] would look unchanged and drop the edit from the -f document. !path.exists() is or-ed in so that naming a new file still creates it.

Over-approximation is the safe direction here, the same one Dirty::of takes (gotcha 35): a Model(_) effect that happens to write the value already there still costs one redundant save.

5 — the refusal names the word the user typed

CmdRegistry::resolve maps formula → add-formula before the arity check runs, so every validator that spelled its own name reported a command nobody typed (improvise-z05h). The fix moves the name to the one place that knows which of the two the user wrote: require_args no longer takes a word, and CmdRegistry::parse prefixes "{typed}: " onto any Err.

That removed 41 name literals from cmd/, which is the same argument gotcha 15 makes about registry.rs having no name literals — a name spelled twice can be spelled wrong once. interactive deliberately does not prefix: dispatch in command/keymap.rs already formats "{name}: {e}" with the bound name (improvise-55l5, PR #126), and prefixing in both would double it.

The message states the shape rather than the arity, because "requires 1-2 argument(s), got 6" does not tell anyone to quote anything.

6 — the help's ex-commands were the unguarded surface

help.rs derives HelpLine::Keys rows from the keymap and drops any row whose command does not resolve, so a key row cannot go stale. HelpLine::Literal (an ex-command only, since improvise-wk18: built by ex!) and HelpLine::Accent rows are hand-written prose and had nothing checking them — which is how all three documented spellings of :formula came to be ones the parser refuses.

every_ex_command_the_help_pages_show_is_one_the_parser_accepts closes that surface rather than the one instance: every help line whose trimmed text opens with : is split on its colons and each piece handed to parse_line. The split exists because one row shows two examples side by side.

What it still cannot see, and should be closed if the welcome text moves again: draw.rs::draw_welcome builds its lines as a local array inside a ratatui render function, so nothing can walk them. Its :formula line was corrected by hand.

7 — a script's command is built against the cursor it will run against

improvise cmd and improvise script used to resolve every command through CmdRegistry::parse — the pure arm, which builds CursorState::default(): row 0, col 0, and a grid of zero rows and zero columns. The interactive arm, which builds CursorState::from_ctx(ctx), was never reached from a script.

So every Move (move-selection, scroll-rows, page-scroll, the four jump-*) and enter-advance snapshotted a zeroed cursor at parse time, clamped to (0, 0), and emitted SetSelected(0, 0). That is worse than a no-op: it resets the cursor. Nothing could position it, so every cursor-reading command — clear-cell, yank, paste, cycle-axis-at-cursor, hide-selected-row-item, edit-or-drill, drill-into-cell, toggle-group-under-cursor, the three delete-*-at-cursor, commit-and-advance — was pinned to the origin (improvise-6uva). It was introduced with CursorState and the two-arm registry in April and predates the browser work by five months.

A registry with two construction arms silently demotes every caller of the weaker arm, and nothing type-checks which arm a caller deserves. That is the shape to remember, and it has a second instance in the same fix: the register_pure entries had an interactive arm that refused empty args while their pure arm accepted them, so :w (save to the current path) parsed one way and refused the other. Both arms are now parse, which is what "needs no interactive context" was supposed to mean; a_pure_commands_two_arms_agree_on_no_arguments is the guard.

The repair is parse::segments plus parse::parse_segment_interactive, and the loop re-parses per segment, because command n must see command n-1's cursor — which means construction cannot be hoisted out of the loop. Line atomicity is preserved by keeping the existing whole-line parse_line as a syntax gate before anything in the line runs: a line that parsed before still parses, and a line that did not still does not, so the only behavioural change is which cursor the commands see.

That atomicity claim was false as written (improvise-2bac), and §8 is what makes it true. The gate runs the pure arm; the loop runs the interactive arm. A gate is only a gate if every argument list the pure arm accepts, the interactive arm accepts too — and commit-cell-edit broke it: add-category Foo . commit-cell-edit editing 5 Foo/x on an empty document passed the gate, applied the first segment, and then refused the second with no cell at cursor. Half the line landed.

The TUI's : line had the same defect and is fixed the same way, with one limitation that is not new: ExecuteCommand returns one effect batch, so the segments of :a . b all see the pre-batch cursor, exactly as a Binding::Sequence does (gotchas 21 and 23).

8 — the interactive arm is parse, then the cursor; it is never a second parser

#151 fixed three instances of a registry with two construction arms demotes whichever caller reaches the weaker one, one patch each. A post-merge review found the fourth (improvise-2bac): commit-cell-edit <mode> <value> <coords>, whose interactive arm read only <mode>, took the value from the edit buffer and the key from the cursor. So a script that wrote 5 to Region/East deleted the cell under the cursor — the empty buffer, committed — with exit 0 and nothing on stderr. The fifth was already in the tree: toggle-panel-and-focus <panel> false false ignored both flags and toggled.

The shape rather than the instances is what changed. CmdRegistry::register used to take two free functions, parse(args) and interactive(args, ctx), and nothing related them; each command re-derived its arguments twice and every disagreement was silent. The interactive arm is now built from parse, not beside it:

• parse(args) -> C — the only function that reads arguments a caller supplied.
• fallback: Option<(args, ctx) -> C> — reached only when parse refused. This is where "no coordinates means the cursor" lives: clear-cell bare, and commit-cell-edit <mode> bare, which is what the keymap binds.
• For register_at_cursor only, the registry itself then sets the command's cursor — its CursorState, reached through AtCursor::cursor_mut — to the live one. That is the whole of what the interactive arm adds to a parsed command, and no registration supplies it.

Three consequences hold by construction rather than by test. Whatever parse accepts, the interactive arm accepts — nothing after parse can fail — so the §7 gate cannot pass a segment the loop then refuses. An accepted argument list cannot be overridden by the cursor, because the only function that sees both args and context never runs on an accepted list. And no field parse set can be overwritten from context, because nothing a registration writes can name one: the cursor slot is typed &mut CursorState, and no command parses a CursorState out of its arguments. register_pure and register_nullary already had one arm; they are register with no fallback.

That last one is improvise-yf7p, and it is why the registry lost a parameter. #171 took a third function, contextualize(C, &CmdContext) -> C, handed the built command and not the args, and stated honestly that it could still overwrite a parsed field — naming the field was "a visible line in review". Review then injected five such overwrites on commands other than the one with a test (yank, paste and drill-into-cell taking their key from the cursor over the parsed one; toggle-panel-visibility ignoring its parsed panel; move-panel-cursor clamping its parsed delta to ±1) and the whole workspace stayed green. A hole guarded by review discipline plus one command's test is not guarded. Every contextualizer in the tree fell into one of two shapes, and each had an answer that removes the function rather than checking it:

• The cursor (Move, EnterAdvance, CommitAndAdvance). Always one CursorState field, always CursorState::from_ctx. So the registry does it, through a slot whose type is the guard: an AtCursor impl pointing at a CellKey or a delta is a compile error, which was tried.
• A read of context that belonged in execute (toggle-panel-visibility's currently_open, cycle-panel-focus's three flags, move-panel-cursor's cursor and bound). These copied context into the command only to read it back; execute now reads ctx directly, the same answer toggle-panel-and-focus got below. Those three are register/register_nullary now, so their interactive arm is exactly parse.

What execute does with a parsed field is still the command's own logic and is tested as such — toggle_panel_visibility_reads_the_panel_it_names and move_panel_cursor_moves_from_the_named_panels_own_cursor fail if the panel or the delta is ignored there. The difference is that the registry is no longer a second place such a bug can be written.

What is still representable, stated rather than hidden:

• A fallback sees the args parse refused and could ignore some of them. commit-cell-edit's refuses anything but a lone mode, so the malformed commit-cell-edit editing 5 is an error on the : line — which has no pure gate — rather than a commit of the edit buffer to the cursor cell.
• The gate is stricter than the loop wherever a fallback exists: a bare clear-cell is refused in a script. That is the safe direction — the whole line is refused — and it is unchanged from before #151.
• A new command could declare a second CursorState field and parse it from args; cursor_mut could then point at the wrong one. No command has one, and a parsed cursor would be a new idea deserving its own argument.

toggle-panel-and-focus did not fit #171's contextualize either, because its pure arm defaulted the omitted flags to true and a contextualizer cannot tell an omitted flag from an explicit one. So the omission is now in the type — target: Option<(bool, bool)>, None meaning toggle — and the toggle is resolved in execute against the same context. That is the honest home: the decision depends on context, and execute is where context is.

Guards: every_argument_list_the_gate_accepts_the_run_accepts_too enumerates every registration against every candidate argument list, in a context whose cursor sits on no cell — the context in which the old commit-cell-edit arm refused. It lists no commands, so a new registration is covered the day it lands. It is backed by the process-level tests in tests/headless_cli.rs.

9 — the extent is fixed, because it reaches the document

A headless run used to read crossterm::terminal::size(), and the page size derived from it decides where page-scroll lands and therefore which record the next cursor-reading command writes (improvise-up8g). So one script wrote a different document in a different window, and with no tty — CI, cron — the read failed and left App's default, so CI and a shell disagreed too. It was invisible until §7 let a script see its cursor.

HEADLESS_EXTENT in main.rs pins it to 80x24 — the size CI already had, so nothing that passed there moves. There is no flag to change it, for §2's reason: a second knob with nothing asking for it. A script that wants a particular row says so with move-selection or jump-*, which do not depend on any window.

What this does not close: the TUI still takes its extent from the terminal, and that is correct there, because the window really is an input to an interactive session. The ticket's wider claim — that a replayable log needs the extent in it (improvise-ilvl) — is about that log, which does not exist yet; a headless run has nothing to replay except its script, so pinning is the whole fix for this surface.

10 — the exit code asks whether the change happened, never whether a chain stopped

App::last_batch_aborted() answered the wrong question for an exit code, in both directions at once.

• A success that exited 1 (improvise-c5ip). commit-cell-edit ends in EnterAdvance, which emits AbortChain when the cursor is already bottom-right or the grid is empty: the advance's premise no longer holds. The write had already landed and was saved, and the run exited 1 with nothing on stderr. Before §7 that was every scripted commit.
• A refusal that exited 0 (improvise-udt2). AddItem into a missing category writes the error and does not abort, because nothing after it depends on the item. So the flag never saw it.

An abort is a statement about the chain; a refusal is a statement about the request. AbortChain stops a chain without refusing anything, and AddItem refuses without stopping anything. The two coincide only for AddCategory and the file writes, which is why reading one for the other looked right for so long.

So App carries a second per-batch outcome, batch_refused, reset where batch_aborted is and written by exactly one method, App::refuse(msg), which sets the status and the flag together — a refusal cannot write its message and forget the flag. Commands reach it through effect::refuse, the Refuse effect, beside effect::set_status for reports. main.rs exits 1 on last_batch_refused() and no longer reads last_batch_aborted() at all.

Why not a severity on status_msg, which udt2 proposed. A severity on the message would live in ViewState, so it would be portable state sent to every client in a Snapshot, and it would go stale: every one of the ~60 writers of status_msg would have to reset it or leave the previous message's severity in place. A per-batch flag cannot go stale, because apply_effects clears it before every batch, and it is not in ViewState, so nothing on the wire carries it or has to agree with it. The TUI colouring the ticket hoped for is still possible later, and would want the severity on the message; that is a separate change with a separate staleness problem.

The classification is per writer, and it is not total. Every refusal in ui/effect.rs and in cmd/{commit,fit,grid,mode,search,text_buffer}.rs goes through refuse. Those still written with set_status exit 0 as before, and are listed in improvise-i3o5. q on a dirty document is deliberately left a report: headless, its text is false — the CLI saves anyway (improvise-ep5o). The two rivals udt2 named both stay green: a successful :w and a successful fit-measure report through set_status and exit 0.