docs(tooling): cite each headless gap's ticket, and record the empty window and the tools package (improvise-1xfb.5.1)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
This commit is contained in:
committed by
Edward Langley
parent
2dcf001b87
commit
e32a4de406
+53
-20
@@ -1048,7 +1048,9 @@ pipeline is also an end-to-end run of improvise's headless product: `import`,
|
||||
2. code-maat runs `coupling` (with `--min-revs`/`--min-shared-revs`), `abs-churn`,
|
||||
`age` and `soc`. Each writes one CSV into the output directory, named after
|
||||
the analysis. code-maat is pinned in `flake.nix` as a `fetchurl` of the
|
||||
1.0.4 standalone jar, with a JRE, in both the dev and `ci` shells.
|
||||
1.0.4 standalone jar, run by a JRE. It is in the dev shell and in the
|
||||
`.#repo-stats-tools` package, but not in the `ci` shell: the JRE is about
|
||||
0.6 GB on Linux, and every CI job enters that shell.
|
||||
3. The script adds `churn.csv`, one row per (commit, file) with the date, the ISO
|
||||
week, the path (`entity`, code-maat's own name for it), the area, the kind
|
||||
(prose, code or other), and the lines added and deleted. It also adds two
|
||||
@@ -1068,6 +1070,14 @@ pipeline is also an end-to-end run of improvise's headless product: `import`,
|
||||
The `.improv` file is the artifact. It opens in the TUI, and CI uploads it with
|
||||
the CSVs.
|
||||
|
||||
**A window in which no file changed builds no model.** Its commits are all
|
||||
empty (`--allow-empty`), so there is no churn to import and nothing for
|
||||
code-maat to analyse. The script writes `churn.csv` with its header alone, skips
|
||||
code-maat and improvise, and the report says so. It exits 0. Running improvise
|
||||
over an empty CSV was the other option. It was not taken, because it would test
|
||||
only how improvise treats a CSV with no records, which is not what this pipeline
|
||||
exists to exercise.
|
||||
|
||||
**No collisions, by construction.** A `churn.csv` record is keyed by commit and
|
||||
file, which `--no-renames` makes unique. code-maat's records are keyed by
|
||||
entity, by entity pair or by date. `File` separates the sources. So the import
|
||||
@@ -1091,12 +1101,14 @@ and age) are read from code-maat's own CSVs, because improvise cannot sort rows
|
||||
|
||||
- **Areas**: first match wins, in the order of `AREAS` in the script.
|
||||
`generated` comes first, so `harness/opfs-contract/Cargo.lock` counts as
|
||||
generated and not as harness. A Rust test is a `.rs` file under a `tests/` or
|
||||
generated and not as harness. `.beads/` and `site/` are generated too.
|
||||
`docs/*.md`, `README.md` and `RELEASING.md` are "other docs", and `.codex/`
|
||||
is agent docs and skills. A Rust test is a `.rs` file under a `tests/` or
|
||||
`benches/` directory, or one named `tests.rs` or `*_test(s).rs`. A
|
||||
`#[cfg(test)] mod tests` inside a source file counts as Rust src, because no
|
||||
path can tell the two apart.
|
||||
- **Prose** is `context/code`, `context/test-evidence`, other `context/`,
|
||||
`repo-map`, and the agent docs and skills. **Code** is Rust src, Rust tests,
|
||||
- **Prose** is `context/code`, `context/test-evidence`, other `context/`, other
|
||||
docs, `repo-map`, and the agent docs and skills. **Code** is Rust src, Rust tests,
|
||||
scripts/CI, and web and harness. `kb.pl`, generated files and other files are
|
||||
neither.
|
||||
- **The deletion ratio** is deleted lines over added lines. It is 0 when nothing
|
||||
@@ -1128,8 +1140,11 @@ and age) are read from code-maat's own CSVs, because improvise cannot sort rows
|
||||
`$GITHUB_STEP_SUMMARY`, and the directory is uploaded as the `repo-stats`
|
||||
artifact.
|
||||
- The report runs in `nix-build` because that job already builds
|
||||
`.#improvise`, and `nix shell .#improvise` reuses that build. The job's
|
||||
checkout fetches full history (`fetch-depth: 0`) for the report.
|
||||
`.#improvise`. Its steps run under `nix shell .#repo-stats-tools`, which joins
|
||||
that same build with code-maat, so nothing is built twice. It is one package
|
||||
rather than two installables because `check-ci-graph.py` reads a counted
|
||||
step as `nix shell <one> --command`. The job's checkout fetches full history
|
||||
(`fetch-depth: 0`) for the report.
|
||||
|
||||
### Gaps in the headless product, and the workarounds used
|
||||
|
||||
@@ -1140,39 +1155,43 @@ names it.
|
||||
1. **A forced `--category` value is still parsed as a number.** The commit
|
||||
`00565796` imported as item `565796`, and `220944e2` imported as
|
||||
`22094400.0`. Workaround: `churn.csv` writes commits as `g<sha>`.
|
||||
improvise-8jk6.4.
|
||||
2. **`import --formula` drops a formula that fails to parse, and exits 0.**
|
||||
`--formula 'X = ('` saves a model with no formula and says nothing.
|
||||
`add-formula` in a script refuses it and exits 1, so the pipeline's own
|
||||
category formula is added there.
|
||||
category formula is added there. improvise-8jk6.5.
|
||||
3. **A record that lacks any category's value is skipped.** A merge of CSVs with
|
||||
different headers, which is the user's sparse model, imports **no records**.
|
||||
Workaround: every CSV is padded with `-` for the categories it lacks. The
|
||||
padding shows up as a `-` item in each category.
|
||||
padding shows up as a `-` item in each category. improvise-lpzu makes
|
||||
the import sparse.
|
||||
4. **An import that skipped records exits 0**, even when it skipped all of
|
||||
them. It saves an empty model and prints only a warning.
|
||||
them. It saves an empty model and prints only a warning. improvise-8jk6.6.
|
||||
5. **`set-axis` and `set-page` accept names that do not exist, and exit 0.** An
|
||||
unknown category, an unknown axis name (`rwo`) and an unknown item are all
|
||||
silent no-ops.
|
||||
silent no-ops. improvise-31oe.2.
|
||||
6. **The command language does not read `|pipe quoting|`.**
|
||||
`create-view |By area|` made a view named `|By`, and the second word was
|
||||
ignored. Commands that take a fixed number of arguments ignore extra ones
|
||||
without saying so. Workaround: `"double quotes"`.
|
||||
without saying so. Workaround: `"double quotes"`. improvise-31oe.3.
|
||||
7. **`create-view` puts every category it is not told about on Page**, with its
|
||||
first item selected. That silently filters the view. Every view has to set
|
||||
every category's axis, one `set-axis` line each.
|
||||
every category's axis, one `set-axis` line each. improvise-31oe.4.
|
||||
8. **A view cannot hide a category's column.** A per-file records view shows
|
||||
every category and measure, including the `-` padding.
|
||||
every category and measure, including the `-` padding. improvise-31oe.5.
|
||||
9. **`export` of a records view writes the editor's `+cat` and `+` columns.**
|
||||
improvise-8jk6.7.
|
||||
10. **`export` writes display text.** It uses the view's number format, so the
|
||||
import's default `,.2` gives `"17,512.00"`. Workaround: `set-format .0` on
|
||||
every exported view, and `FORMAT .3` on the ratio formulas.
|
||||
improvise-8jk6.8.
|
||||
11. **Nothing sorts.** Items stay in first-seen order, so weeks come out
|
||||
newest first. A top-N table cannot be produced, so code-maat's rankings are
|
||||
read from its CSVs.
|
||||
read from its CSVs. improvise-31oe.6.
|
||||
12. **`--extract` has no ISO week.** It has year, month and quarter. Workaround:
|
||||
`churn.csv` carries a `week` column.
|
||||
`churn.csv` carries a `week` column. improvise-8jk6.9.
|
||||
13. **`import --formula` targets only `_Measure`.** A formula on another
|
||||
category needs `add-formula <category> …` in a script.
|
||||
category needs `add-formula <category> …` in a script. improvise-8jk6.10.
|
||||
14. **The saved model opens on the import's `Default` view**, which puts `File`
|
||||
on rows and `area` on columns, and not on the view the script left active.
|
||||
That is improvise-azhb.
|
||||
@@ -1180,12 +1199,26 @@ names it.
|
||||
folder import, and it will also make the per-file views. Both of the
|
||||
script's steps for these are marked interim.
|
||||
16. **There is no headless output but a file.** `export` writes a CSV, and the
|
||||
tables are laid out in Python.
|
||||
tables are laid out in Python. improvise-31oe.7.
|
||||
17. **`create-view` of an existing name replaces that view, and exits 0.**
|
||||
`Workbook::create_view` inserts into an `IndexMap`, so it overwrites without
|
||||
a word. It matters here once folder import makes its own per-file views: the
|
||||
script's interim `create-view` of the same name would replace them.
|
||||
improvise-31oe.8.
|
||||
|
||||
### The suite
|
||||
|
||||
`scripts/repo-stats-test.sh` builds a fixture repository whose every number is
|
||||
known, runs the whole pipeline, code-maat and improvise included, and asserts
|
||||
improvise's exported tables. It runs in `nix-build`, which puts improvise on
|
||||
`PATH`. Locally it runs `cargo build --bin improvise` when improvise is not on
|
||||
`PATH`. The evidence and the rivals are in `context/test-evidence/tooling.md`.
|
||||
improvise's exported tables. It runs in `nix-build`, which puts improvise and
|
||||
code-maat on `PATH`. Locally it uses improvise from `PATH`, or else
|
||||
`target/debug/improvise`, and it fails with a message when there is neither.
|
||||
|
||||
It does not build improvise itself. `checkers.sh` scrubs the environment to an
|
||||
allowlist, and on macOS that removes what the linker needs: a `cargo build`
|
||||
inside the suite failed with `library not found for -liconv`. Build first with
|
||||
`cargo build --bin improvise`; `cargo test --workspace` also leaves the binary
|
||||
there. The suite is the slowest in `checkers.sh`, at about 25 seconds, most of
|
||||
it JVM start-up for code-maat's four analyses per run.
|
||||
|
||||
The evidence and the rivals are in `context/test-evidence/tooling.md`.
|
||||
|
||||
@@ -1847,3 +1847,35 @@ through `checkers.sh repo-stats`:
|
||||
pipeline runs fail with exit 3.
|
||||
- Months instead of ISO weeks: both week cases fail.
|
||||
- A `warning:` from improvise ignored: the warning-stub case fails.
|
||||
|
||||
### A window of empty commits, and four faults the first fixture missed — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
|
||||
|
||||
**What broke.** ChatGPT Codex found it on PR #313, and the adversarial review
|
||||
reproduced it. When every non-merge commit in the window is `--allow-empty`,
|
||||
the log has commit headers and no files. `build_dir` took `churn.csv`'s header
|
||||
from `rows[0]`, raised `IndexError` and exited 1. That exit is outside the
|
||||
pipeline's failure split: it is neither a number nor improvise's exit 3.
|
||||
|
||||
**Fix.** `churn.csv` has a fixed header. When no file changed, the script
|
||||
skips code-maat and improvise, writes the header alone, and exits 0.
|
||||
|
||||
**Red first.** The new case is a second fixture repository whose only
|
||||
in-window commit is empty. Restoring the `rows[0]` header fails its run case
|
||||
with the traceback.
|
||||
|
||||
**The review's four surviving faults, each now killed:**
|
||||
- **`--no-renames` dropped.** The fixture now renames `scripts/foo.sh` to
|
||||
`scripts/bar.sh`. Without the flag git reports one `{foo.sh => bar.sh}` entity
|
||||
with no lines changed, not a delete and an add. Four cases fail: the scripts/CI
|
||||
row, the code row, the ratio row and the coupling count.
|
||||
- **`crate_of` forgetting the root crate.** `src/main.rs` now co-changes with
|
||||
both crates. The crossing list loses three pairs.
|
||||
- **A failing `script` call ignored.** A third stub exits 0 for `import` and
|
||||
refuses `script`. The pipeline then reads exports that were never written and
|
||||
exits 1, not 3.
|
||||
- **`.beads/` and `site/` left out of generated.** Both are in the fixture now.
|
||||
The generated row and the other row both fail.
|
||||
|
||||
**Also counted now:** `docs/guide.md` as other docs and `.codex/config.toml` as
|
||||
agent docs. The coupling count is 16, not 15: `web/app.js` and
|
||||
`scripts/foo.sh` change together in two commits, the 06-04 one and the rename.
|
||||
|
||||
Reference in New Issue
Block a user