fix(cli): the headless commands were silent whether they worked or not
improvise-q914 and improvise-j6x8 are one bug seen twice, and the second
measured it: `run_headless_commands` applied effects and never read
`app.view_state.status_msg`, setting `exit_code` only for parse errors. A
successful command and a refused one therefore produced byte-identical
output — nothing — and exit 0 for both. `improvise script` is the same
function, so it shared the hole exactly.
Three decisions, argued in `context/code/cli.md`:
**Everything to stderr.** The tempting split — stdout for a report, stderr
for a refusal — cannot be built honestly, because `status_msg` is a bare
`String` with no severity and both candidate classifiers are disproved by
code already in the tree. "A refusal aborted" is false: `AddItem` writes its
`Err` to the status and does not abort, and that is q914's own example. "A
refusal is anything that spoke" is false too: `cmd/fit.rs` reports the seed
that reproduces a fit on **success**. Guessing between them is the adjacent
check design-principles §7 names.
**Exit 1 on a parse error or an aborted batch, and no further.** Both are
real flags. A refusal that only writes a status is now printed but still
exits 0, because nothing can classify it; the README says so plainly rather
than over-promising, and closing it needs a severity on `ViewState`, which
is a wire change reaching the browser.
**No quiet flag.** Silence is never the default and `2>/dev/null` already
exists.
Also: a run that changed nothing no longer rewrites the document. Saving
unconditionally normalised it — `1e20` to its twenty digits — which turns a
hand-edited fixture into a silent no-op. The gate accumulates
`model_state.dirty` per command rather than reading it once at the end,
because `effect::Save` clears the flag and `[add-item, w elsewhere]` would
otherwise drop the edit.
And the three clap doc strings advertised JSON, which the parser refuses.
`tests/headless_cli.rs` pins all of it at the process boundary, where an
exit code and a stream exist. Six of its twelve tests fail at bf83294. The
rival worth injecting — print on the happy path, stay silent on the
refusal — fails five of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
@@ -0,0 +1,268 @@
|
||||
//! The `improvise cmd` / `improvise script` contract, pinned at the process
|
||||
//! boundary. Why a spawned binary: `context/test-evidence/cli.md`.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Output};
|
||||
|
||||
fn bin() -> &'static str {
|
||||
env!("CARGO_BIN_EXE_improvise")
|
||||
}
|
||||
|
||||
struct Scratch(PathBuf);
|
||||
|
||||
impl Scratch {
|
||||
fn new(tag: &str) -> Self {
|
||||
let dir = std::env::temp_dir().join(format!("improvise-cli-{tag}-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).expect("a scratch directory");
|
||||
Self(dir)
|
||||
}
|
||||
|
||||
fn path(&self, name: &str) -> PathBuf {
|
||||
self.0.join(name)
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Scratch {
|
||||
fn drop(&mut self) {
|
||||
let _ = std::fs::remove_dir_all(&self.0);
|
||||
}
|
||||
}
|
||||
|
||||
fn run_cmd(args: &[&str], file: &Path) -> Output {
|
||||
Command::new(bin())
|
||||
.arg("cmd")
|
||||
.args(args)
|
||||
.arg("-f")
|
||||
.arg(file)
|
||||
.output()
|
||||
.expect("the improvise binary runs")
|
||||
}
|
||||
|
||||
fn stderr_of(out: &Output) -> String {
|
||||
String::from_utf8_lossy(&out.stderr).to_string()
|
||||
}
|
||||
|
||||
fn stdout_of(out: &Output) -> String {
|
||||
String::from_utf8_lossy(&out.stdout).to_string()
|
||||
}
|
||||
|
||||
/// A document with one category, so `add-item` has somewhere legitimate to go.
|
||||
fn seeded(scratch: &Scratch, name: &str) -> PathBuf {
|
||||
let doc = scratch.path(name);
|
||||
let out = run_cmd(&["add-category Region"], &doc);
|
||||
assert!(doc.exists(), "the seed run wrote a document: {out:?}");
|
||||
doc
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_refused_command_says_what_went_wrong() {
|
||||
let scratch = Scratch::new("refused");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let out = run_cmd(&["add-item Nonexistent Thing"], &doc);
|
||||
let said = stderr_of(&out);
|
||||
assert!(
|
||||
!said.trim().is_empty(),
|
||||
"a refused command said nothing at all; stdout={:?}",
|
||||
stdout_of(&out)
|
||||
);
|
||||
assert!(
|
||||
said.contains("Nonexistent"),
|
||||
"the refusal does not name the category that does not exist: {said:?}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The rival this separates from: a fix that reports on the happy path and
|
||||
/// stays silent on the refusal. `context/test-evidence/cli.md`.
|
||||
#[test]
|
||||
fn a_command_with_nothing_to_say_says_nothing() {
|
||||
let scratch = Scratch::new("quiet");
|
||||
let doc = scratch.path("m.improv");
|
||||
let out = run_cmd(&["add-category Region"], &doc);
|
||||
assert!(
|
||||
stderr_of(&out).trim().is_empty(),
|
||||
"a command that succeeded with nothing to report spoke anyway: {:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"a command that succeeded did not exit 0"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_aborted_batch_exits_non_zero_and_says_why() {
|
||||
let scratch = Scratch::new("abort");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let bad = scratch.path("no-such-dir").join("out.improv");
|
||||
let out = run_cmd(&[&format!("w {}", bad.display())], &doc);
|
||||
let said = stderr_of(&out);
|
||||
assert!(
|
||||
!said.trim().is_empty(),
|
||||
"a failed save said nothing: stdout={:?}",
|
||||
stdout_of(&out)
|
||||
);
|
||||
assert!(
|
||||
!out.status.success(),
|
||||
"a failed save exited 0; stderr={said:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_parse_error_still_exits_non_zero() {
|
||||
let scratch = Scratch::new("parse");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let out = run_cmd(&["no-such-command-at-all"], &doc);
|
||||
assert!(!out.status.success(), "an unparseable line exited 0");
|
||||
assert!(
|
||||
stderr_of(&out).contains("no-such-command-at-all"),
|
||||
"the parse error does not name the word typed: {:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
}
|
||||
|
||||
/// Diagnostics go to stderr so that stdout stays free for a data output, and
|
||||
/// so `2>/dev/null` is the whole suppression story. `context/code/cli.md`.
|
||||
#[test]
|
||||
fn what_a_command_says_goes_to_stderr_and_stdout_stays_empty() {
|
||||
let scratch = Scratch::new("stream");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let out = run_cmd(&["add-item Nonexistent Thing"], &doc);
|
||||
assert!(
|
||||
stdout_of(&out).trim().is_empty(),
|
||||
"a diagnostic reached stdout: {:?}",
|
||||
stdout_of(&out)
|
||||
);
|
||||
assert!(!stderr_of(&out).trim().is_empty(), "stderr was empty too");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_script_reports_exactly_as_cmd_does() {
|
||||
let scratch = Scratch::new("script");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let script = scratch.path("s.txt");
|
||||
std::fs::write(&script, "# a comment\nadd-item Nonexistent Thing\n").expect("a script file");
|
||||
let out = Command::new(bin())
|
||||
.arg("script")
|
||||
.arg(&script)
|
||||
.arg("-f")
|
||||
.arg(&doc)
|
||||
.output()
|
||||
.expect("the improvise binary runs");
|
||||
assert!(
|
||||
stderr_of(&out).contains("Nonexistent"),
|
||||
"`script` swallowed the refusal `cmd` reports: {:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
}
|
||||
|
||||
/// A run that changed nothing must leave the bytes alone: a fixture edited by
|
||||
/// hand between runs is otherwise silently normalised. `context/code/cli.md`.
|
||||
#[test]
|
||||
fn a_run_that_changes_nothing_does_not_rewrite_the_document() {
|
||||
let scratch = Scratch::new("noop");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let before = std::fs::read(&doc).expect("the seeded document");
|
||||
let out = run_cmd(&["move-cursor down"], &doc);
|
||||
let after = std::fs::read(&doc).expect("the document after a view-only command");
|
||||
assert_eq!(
|
||||
before,
|
||||
after,
|
||||
"a view-only command rewrote the document; stderr={:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
}
|
||||
|
||||
/// The reported instance: `1e20` is expanded to its twenty digits by a run that
|
||||
/// was asked to change nothing at all.
|
||||
#[test]
|
||||
fn a_no_op_run_leaves_an_unnormalised_literal_alone() {
|
||||
let scratch = Scratch::new("literal");
|
||||
let doc = scratch.path("m.improv");
|
||||
std::fs::write(
|
||||
&doc,
|
||||
"v2025-04-09\n# Big\nInitial View: Default\n\n\
|
||||
## View: Default\nRegion: row\n_Measure: column\n\n\
|
||||
## Category: Region\n- North\n\n\
|
||||
## Category: _Measure\n- Score\n\n\
|
||||
## Data\nRegion=North, _Measure=Score = 1e20\n",
|
||||
)
|
||||
.expect("a hand-written document");
|
||||
let before = std::fs::read_to_string(&doc).expect("the document");
|
||||
assert!(before.contains("1e20"), "the fixture itself is wrong");
|
||||
let out = run_cmd(&["move-cursor down"], &doc);
|
||||
let after = std::fs::read_to_string(&doc).expect("the document after");
|
||||
assert!(
|
||||
after.contains("1e20"),
|
||||
"a no-op run normalised the literal; stderr={:?}, after={after:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_cmd_help_does_not_advertise_a_language_it_refuses() {
|
||||
for sub in ["cmd", "script"] {
|
||||
let out = Command::new(bin())
|
||||
.args([sub, "--help"])
|
||||
.output()
|
||||
.expect("the improvise binary runs");
|
||||
let text = format!("{}{}", stdout_of(&out), stderr_of(&out));
|
||||
assert!(
|
||||
!text.contains("JSON") && !text.contains("json"),
|
||||
"`{sub} --help` still advertises JSON, which it refuses: {text}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The language it does accept, demonstrated end to end so the help above has
|
||||
/// something true to point at.
|
||||
#[test]
|
||||
fn the_language_the_help_names_is_the_one_it_accepts() {
|
||||
let scratch = Scratch::new("lang");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let out = run_cmd(&["add-item Region East"], &doc);
|
||||
assert!(out.status.success(), "the command line form was refused");
|
||||
let text = std::fs::read_to_string(&doc).expect("the document");
|
||||
assert!(text.contains("East"), "the item was not added: {text}");
|
||||
|
||||
let json = run_cmd(&[r#"{"cmd":"add-item","args":["Region","West"]}"#], &doc);
|
||||
assert!(
|
||||
!json.status.success(),
|
||||
"the JSON form the help used to advertise was accepted after all"
|
||||
);
|
||||
}
|
||||
|
||||
/// The rival this separates from: making any emitted status a non-zero exit.
|
||||
/// `fit-measure` reports its reproducing seed the same way — `context/code/cli.md`.
|
||||
#[test]
|
||||
fn a_command_that_succeeds_while_reporting_still_exits_zero() {
|
||||
let scratch = Scratch::new("reports");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let save = format!("w {}", doc.display());
|
||||
let out = run_cmd(&["add-item Region East", &save], &doc);
|
||||
assert!(
|
||||
!stderr_of(&out).trim().is_empty(),
|
||||
"a successful save reported nothing, so this test proves nothing"
|
||||
);
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"a successful command that reported was treated as a failure: {:?}",
|
||||
stderr_of(&out)
|
||||
);
|
||||
}
|
||||
|
||||
/// A run that did change the document still writes it, even when the last thing
|
||||
/// it did was save somewhere else and clear the dirty flag.
|
||||
#[test]
|
||||
fn a_run_that_changed_the_document_still_writes_it() {
|
||||
let scratch = Scratch::new("elsewhere");
|
||||
let doc = seeded(&scratch, "m.improv");
|
||||
let other = scratch.path("other.improv");
|
||||
let save = format!("w {}", other.display());
|
||||
run_cmd(&["add-item Region East", &save], &doc);
|
||||
let text = std::fs::read_to_string(&doc).expect("the -f document");
|
||||
assert!(
|
||||
text.contains("East"),
|
||||
"the -f document lost an edit because `:w` elsewhere cleared the flag: {text}"
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user