Merge pull request #313 from fiddlerwoaroof/chore/1xfb-5-1-repo-stats
chore(tooling): scripts/repo-stats — churn, coupling and doc-growth report (improvise-1xfb.5.1)
This commit is contained in:
Ed Langley
GitHub
@@ -146,13 +146,39 @@ jobs:
|
||||
# dependency crate each run. The NixOS runners substitute from titan's
|
||||
# harmonia first, and titan builds gh/main's `#improvise` hourly, so a PR that
|
||||
# leaves Cargo.lock alone rebuilds only the workspace crates it changed.
|
||||
#
|
||||
# The repo-stats pipeline runs here because it needs this job's binary:
|
||||
# `.#repo-stats-tools` is that build plus code-maat and its JRE, which stay
|
||||
# out of the ci shell every other job enters.
|
||||
# Its suite gates; the report on a push to main never fails the build, and
|
||||
# its step fails only when improvise refuses the pipeline's input
|
||||
# (context/code/tooling.md section 7). Full history is for the report.
|
||||
nix-build:
|
||||
name: Nix build (crate2nix)
|
||||
name: Nix build (crate2nix) and the repo-stats pipeline
|
||||
runs-on: [self-hosted, nixos, microvm]
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: nix build .#improvise
|
||||
run: nix build --print-build-logs .#improvise
|
||||
- name: The repo-stats pipeline computes what it claims to
|
||||
run: nix shell .#repo-stats-tools --command ./scripts/checkers.sh --ci nix-build
|
||||
- name: Churn, coupling and doc-growth report (report only)
|
||||
if: github.event_name == 'push'
|
||||
continue-on-error: true
|
||||
run: |
|
||||
rc=0
|
||||
nix shell .#repo-stats-tools --command ./scripts/repo-stats --out repo-stats > repo-stats.txt || rc=$?
|
||||
{ echo '```'; cat repo-stats.txt; echo '```'; } >> "$GITHUB_STEP_SUMMARY"
|
||||
exit "$rc"
|
||||
- name: Upload the repo-stats model and its CSVs
|
||||
if: github.event_name == 'push'
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: repo-stats
|
||||
path: repo-stats
|
||||
retention-days: 30
|
||||
|
||||
wasm:
|
||||
name: wasm32 check (browser bundle crates)
|
||||
|
||||
@@ -1032,3 +1032,212 @@ changes. Two rivals were run:
|
||||
- reading every failure as stale fails the offline case (exit 1, wanted 2);
|
||||
- dropping `--locked` fails the stale case (exit 0, wanted 1). Without
|
||||
`--locked`, cargo quietly rewrites the lock and reports success.
|
||||
|
||||
## 7. The repo-stats pipeline (improvise-1xfb.5.1)
|
||||
|
||||
`scripts/repo-stats` reports churn, change coupling and doc growth over a window
|
||||
of history: by default the last 7 days of `gh/main`, falling back to `HEAD`,
|
||||
without merges. It reports and never gates. The user asked for code-maat to do
|
||||
the analyses it has, and for improvise to be the engine for the rest. So the
|
||||
pipeline is also an end-to-end run of improvise's headless product: `import`,
|
||||
`script` and `export`.
|
||||
|
||||
### The shape
|
||||
|
||||
1. `git log --numstat --no-renames --no-merges` in code-maat's `git2` format.
|
||||
2. code-maat runs `coupling` (with `--min-revs`/`--min-shared-revs`), `abs-churn`,
|
||||
`age` and `soc`. Each writes one CSV into the output directory, named after
|
||||
the analysis. code-maat is pinned in `flake.nix` as a `fetchurl` of the
|
||||
1.0.4 standalone jar, run by a JRE. It is in the dev shell and in the
|
||||
`.#repo-stats-tools` package, but not in the `ci` shell: the JRE is about
|
||||
0.6 GB on Linux, and every CI job enters that shell.
|
||||
3. The script adds `churn.csv`, one row per (commit, file) with the date, the ISO
|
||||
week, the path (`entity`, code-maat's own name for it), the area, the kind
|
||||
(prose, code or other), and the lines added and deleted. It also adds two
|
||||
flags to code-maat's `coupling.csv`: `crosses-crate` and `agent-rules`.
|
||||
4. `improvise import <dir>` (improvise-lpzu) merges the directory's five CSVs
|
||||
into **one** sparse model, `repo-stats.improv`. The merge's `File` category
|
||||
names each source, and the import makes one view per file, named after its
|
||||
stem and filtered to it. That is the user's ruling: "Records are already
|
||||
sparse: the model can just leave null/empty values for columns not in a
|
||||
given CSV file." A record keeps no coordinate in a category its file lacks.
|
||||
5. `improvise script` runs the checked-in `scripts/repo-stats.improvise`. It
|
||||
adds the prose-to-code formula, builds four views on top of the per-file
|
||||
ones and `export`s each to `report/*.csv`. The four names (`By area`,
|
||||
`By kind`, `Area by week`, `By day`) must never be a file's stem:
|
||||
`create-view` of an existing name replaces that view silently (gap 17). The
|
||||
suite's view count, 10, fails if one does. None of the report reads the
|
||||
merged `Default` view, which can be empty for a typical code-maat pair
|
||||
(improvise-8jk6.15).
|
||||
6. The text report is those exports laid out as tables. The code-maat rankings,
|
||||
doc sizes and the coupling flags are added to it. `--json` gives the same
|
||||
data as JSON.
|
||||
|
||||
The `.improv` file is the artifact. It opens in the TUI, and CI uploads it with
|
||||
the CSVs.
|
||||
|
||||
**A window in which no file changed builds no model.** Its commits are all
|
||||
empty (`--allow-empty`), so there is no churn to import and nothing for
|
||||
code-maat to analyse. The script writes `churn.csv` with its header alone, skips
|
||||
code-maat and improvise, and the report says so. It exits 0. Running improvise
|
||||
over an empty CSV was the other option. It was not taken, because it would test
|
||||
only how improvise treats a CSV with no records, which is not what this pipeline
|
||||
exists to exercise.
|
||||
|
||||
**`--out` must hold no CSV the script did not write.** `improvise import .`
|
||||
merges every CSV in the directory, so a stale or foreign one would silently
|
||||
join the model. The script refuses one with exit 2 and names it. It refuses
|
||||
rather than deleting, because deleting files in a directory the caller named is
|
||||
the wrong default. Its own five CSVs are overwritten on every run, so a rerun
|
||||
into the same `--out` works.
|
||||
|
||||
**No collisions, by construction.** A `churn.csv` record is keyed by commit and
|
||||
file, which `--no-renames` makes unique. code-maat's records are keyed by
|
||||
entity, by entity pair or by date. `File` separates the sources. So the import
|
||||
keeps its default, `--on-duplicate refuse`. A collision would be a bug in this
|
||||
script, and the import refuses it.
|
||||
|
||||
**What improvise computes and what it does not.** Everything that is a sum or a
|
||||
ratio of sums is a view:
|
||||
- churn by area, with `DeletionRatio = deleted / added` as a `_Measure` formula;
|
||||
- prose, code and other, with `ProseToCode = prose / code` as a formula on the
|
||||
`kind` category;
|
||||
- area by ISO week;
|
||||
- lines added per day from `churn.csv` next to code-maat's `abs-churn`. The two
|
||||
columns must agree, and the suite checks that they do.
|
||||
|
||||
Two kinds of output stay in Python. The ranked code-maat tables (coupling, soc
|
||||
and age) are read from code-maat's own CSVs, because improvise cannot sort rows
|
||||
(gap 11 below). Doc sizes are a property of files at one ref, not of the window.
|
||||
|
||||
### Definitions
|
||||
|
||||
- **Areas**: first match wins, in the order of `AREAS` in the script.
|
||||
`generated` comes first, so `harness/opfs-contract/Cargo.lock` counts as
|
||||
generated and not as harness. `.beads/` and `site/` are generated too.
|
||||
`docs/*.md`, `README.md` and `RELEASING.md` are "other docs", and `.codex/`
|
||||
is agent docs and skills. A Rust test is a `.rs` file under a `tests/` or
|
||||
`benches/` directory, or one named `tests.rs` or `*_test(s).rs`. A
|
||||
`#[cfg(test)] mod tests` inside a source file counts as Rust src, because no
|
||||
path can tell the two apart.
|
||||
- **Prose** is `context/code`, `context/test-evidence`, other `context/`, other
|
||||
docs, `repo-map`, and the agent docs and skills. **Code** is Rust src, Rust tests,
|
||||
scripts/CI, and web and harness. `kb.pl`, generated files and other files are
|
||||
neither.
|
||||
- **The deletion ratio** is deleted lines over added lines. It is 0 when nothing
|
||||
was deleted, and blank when nothing was added.
|
||||
- **A crate crossing**: both paths are in a crate (`crates/<name>/`, the root
|
||||
crate's `src/`, `tests/`, `benches/`, `examples/` or `Cargo.toml`, or
|
||||
`harness/opfs-contract/`), and the two crates differ. A pair with one side
|
||||
outside every crate is not flagged.
|
||||
- **Agent rules**: both paths are in the agent docs and skills area. Such a pair
|
||||
means one rule stated in several places.
|
||||
- **Gotchas** are the numbered items between `## Gotchas` and the next `---` or
|
||||
`##`. The longest gotcha is measured in characters. Most gotchas are one long
|
||||
line, so a line count would say little.
|
||||
- **Test-evidence entries** are the `###` and `####` headings.
|
||||
|
||||
### What fails, and what never does
|
||||
|
||||
- **The numbers never fail anything.** No budget is checked. Budgets are a
|
||||
separate decision under the compaction epic, improvise-1xfb.5.
|
||||
- **improvise refusing the pipeline's input exits 3.** That covers a non-zero
|
||||
exit from `import` or `script`, and any `warning:` on their stderr. The
|
||||
warning rule is there because a skipped record is data loss that exits 0
|
||||
(gaps 3 and 4). It is a product regression, not a statistic.
|
||||
- **In CI**, the suite runs on every PR in `nix-build` and gates like any other
|
||||
suite. The report runs only on a push to `main`. Its step has
|
||||
`continue-on-error`, so an improvise failure there marks the step failed and
|
||||
the job and the deploy go on. Whether that failure should block the deploy
|
||||
is a question for the user on the PR. The report appends to
|
||||
`$GITHUB_STEP_SUMMARY`, and the directory is uploaded as the `repo-stats`
|
||||
artifact.
|
||||
- The report runs in `nix-build` because that job already builds
|
||||
`.#improvise`. Its steps run under `nix shell .#repo-stats-tools`, which joins
|
||||
that same build with code-maat, so nothing is built twice. It is one package
|
||||
rather than two installables because `check-ci-graph.py` reads a counted
|
||||
step as `nix shell <one> --command`. The job's checkout fetches full history
|
||||
(`fetch-depth: 0`) for the report.
|
||||
|
||||
### Gaps in the headless product, and the workarounds used
|
||||
|
||||
Each of these was found by running the pipeline, and each is a finding against
|
||||
improvise's command and model layer. Where a workaround exists, the script
|
||||
names it.
|
||||
|
||||
1. **A forced `--category` value is still parsed as a number.** The commit
|
||||
`00565796` imported as item `565796`, and `220944e2` imported as
|
||||
`22094400.0`. Workaround: `churn.csv` writes commits as `g<sha>`.
|
||||
improvise-8jk6.4.
|
||||
2. **`import --formula` drops a formula that fails to parse, and exits 0.**
|
||||
`--formula 'X = ('` saves a model with no formula and says nothing.
|
||||
`add-formula` in a script refuses it and exits 1, so the pipeline's own
|
||||
category formula is added there. improvise-8jk6.5.
|
||||
3. **A record that lacks any category's value is skipped.** A merge of CSVs with
|
||||
different headers, which is the user's sparse model, imports **no records**.
|
||||
Closed by improvise-lpzu, whose sparse import this pipeline now uses. Until
|
||||
it landed, every CSV was padded with `-` for the categories it lacked. A
|
||||
related hole is still open: a record with no measure value stores nothing
|
||||
(improvise-8jk6.14).
|
||||
4. **An import that skipped records exits 0**, even when it skipped all of
|
||||
them. It saves an empty model and prints only a warning. improvise-8jk6.6.
|
||||
5. **`set-axis` and `set-page` accept names that do not exist, and exit 0.** An
|
||||
unknown category, an unknown axis name (`rwo`) and an unknown item are all
|
||||
silent no-ops. improvise-31oe.2.
|
||||
6. **The command language does not read `|pipe quoting|`.**
|
||||
`create-view |By area|` made a view named `|By`, and the second word was
|
||||
ignored. Commands that take a fixed number of arguments ignore extra ones
|
||||
without saying so. Workaround: `"double quotes"`. improvise-31oe.3.
|
||||
7. **`create-view` puts every category it is not told about on Page**, with its
|
||||
first item selected. That silently filters the view. Every view has to set
|
||||
every category's axis, one `set-axis` line each. improvise-31oe.4.
|
||||
8. **A view cannot hide a category's column.** A per-file records view shows
|
||||
every category and measure in the model. improvise-31oe.5.
|
||||
9. **`export` of a records view writes the editor's `+cat` and `+` columns.**
|
||||
improvise-8jk6.7.
|
||||
10. **`export` writes display text.** It uses the view's number format, so the
|
||||
import's default `,.2` gives `"17,512.00"`. Workaround: `set-format .0` on
|
||||
every exported view, and `FORMAT .3` on the ratio formulas.
|
||||
improvise-8jk6.8.
|
||||
11. **Nothing sorts.** Items stay in first-seen order, so weeks come out
|
||||
newest first. A top-N table cannot be produced, so code-maat's rankings are
|
||||
read from its CSVs. improvise-31oe.6.
|
||||
12. **`--extract` has no ISO week.** It has year, month and quarter. Workaround:
|
||||
`churn.csv` carries a `week` column. improvise-8jk6.9.
|
||||
13. **`import --formula` targets only `_Measure`.** A formula on another
|
||||
category needs `add-formula <category> …` in a script. improvise-8jk6.10.
|
||||
14. **The saved model opens on the import's `Default` view**, which puts `File`
|
||||
on rows and `area` on columns, and not on the view the script left active.
|
||||
That is improvise-azhb.
|
||||
15. **No folder import.** Closed by improvise-lpzu (PR #314), which this
|
||||
pipeline now uses. Until it landed, the script named the five files and
|
||||
generated its own per-file views.
|
||||
16. **There is no headless output but a file.** `export` writes a CSV, and the
|
||||
tables are laid out in Python. improvise-31oe.7.
|
||||
17. **`create-view` of an existing name replaces that view, and exits 0.**
|
||||
`Workbook::create_view` inserts into an `IndexMap`, so it overwrites without
|
||||
a word. Folder import makes a view per file stem, so a report view named
|
||||
like a stem would replace that file's view.
|
||||
improvise-31oe.8.
|
||||
|
||||
### The suite
|
||||
|
||||
`scripts/repo-stats-test.sh` builds a fixture repository whose every number is
|
||||
known, runs the whole pipeline, code-maat and improvise included, and asserts
|
||||
improvise's exported tables. It runs in `nix-build`, which puts improvise and
|
||||
code-maat on `PATH`. Locally it uses improvise from `PATH`, or else
|
||||
`target/debug/improvise`, and it fails with a message when there is neither.
|
||||
The fallback does not check how fresh that binary is, so locally the suite can
|
||||
test a `target/debug/improvise` older than the tree. Rebuild after changing
|
||||
improvise.
|
||||
|
||||
It does not build improvise itself. `checkers.sh` scrubs the environment to an
|
||||
allowlist, and on macOS that removes what the linker needs: a `cargo build`
|
||||
inside the suite failed with `library not found for -liconv`. That is a bug in
|
||||
`checkers.sh`: its allowlist drops the Nix linker variables, and the
|
||||
orchestrator has filed it separately. Build first with
|
||||
`cargo build --bin improvise`; `cargo test --workspace` also leaves the binary
|
||||
there. The suite is the slowest in `checkers.sh`, at about 25 seconds, most of
|
||||
it JVM start-up for code-maat's four analyses per run.
|
||||
|
||||
The evidence and the rivals are in `context/test-evidence/tooling.md`.
|
||||
|
||||
@@ -1811,3 +1811,107 @@ that failed would leave the owner-authored base at HEAD and fail the case too.
|
||||
`GIT_CONFIG_GLOBAL=/dev/null` and `GIT_CONFIG_NOSYSTEM=1`, and every `GIT_*`
|
||||
identity variable is unset first. So a caller's own identity cannot make the
|
||||
author or committer assertions agree by accident.
|
||||
|
||||
### The repo-stats pipeline computes what it claims to — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
|
||||
|
||||
**What it pins.** The fixture repository has one commit before the window, a
|
||||
side branch merged with `--no-ff`, a shrink in the next ISO week, and one file
|
||||
in every area. So each number the report prints is known by construction. The
|
||||
suite runs the real pipeline, code-maat and improvise included, and asserts
|
||||
improvise's exported tables: each area's added, deleted and deletion ratio;
|
||||
prose, code and the `ProseToCode` row; the week split; and the per-day totals.
|
||||
It also checks the doc sizes and the coupling flags, and that an improvise that
|
||||
refuses or warns makes the pipeline exit 3.
|
||||
|
||||
**Two fixture choices are load-bearing.**
|
||||
- The repo map's `4.` item sits after the `---` that closes the Gotchas
|
||||
section, with no `##` heading between them. In an earlier fixture a `##`
|
||||
heading followed the `---`, so the `---` rule could be deleted and the suite
|
||||
stayed green.
|
||||
- `churn.csv` must equal code-maat's `abs-churn` on every day. That compares
|
||||
two independent derivations of one number, one of them improvise's
|
||||
aggregation. The fixture has a binary-free, rename-free history, so the two
|
||||
agree exactly.
|
||||
|
||||
**Rivals**, each injected into the script or into `repo-stats.improvise` and run
|
||||
through `checkers.sh repo-stats`:
|
||||
- Rust src matched before Rust tests: the rust src and rust tests rows fail.
|
||||
- `DeletionRatio = deleted / (added + deleted)`: six ratio rows fail.
|
||||
- `ProseToCode = code / prose`: the `ProseToCode` row fails.
|
||||
- `--no-merges` dropped: the commit count fails (6, wanted 5).
|
||||
- The `---` end of the Gotchas section ignored: the gotcha count fails (4, wanted 3).
|
||||
- A crate crossing as "the crates differ", so a crate against no crate counts:
|
||||
the crate-crossing case fails.
|
||||
- `kb.pl` counted as prose: the prose row and the ratio fail.
|
||||
- The category padding removed: the import skipped every record, and both
|
||||
pipeline runs failed with exit 3. That rival is retired: folder import
|
||||
(improvise-lpzu) keeps sparse records, and the padding is gone. The suite
|
||||
passes 41 of 41 without it.
|
||||
- Months instead of ISO weeks: both week cases fail.
|
||||
- A `warning:` from improvise ignored: the warning-stub case fails.
|
||||
|
||||
### A window of empty commits, and four faults the first fixture missed — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
|
||||
|
||||
**What broke.** ChatGPT Codex found it on PR #313, and the adversarial review
|
||||
reproduced it. When every non-merge commit in the window is `--allow-empty`,
|
||||
the log has commit headers and no files. `build_dir` took `churn.csv`'s header
|
||||
from `rows[0]`, raised `IndexError` and exited 1. That exit is outside the
|
||||
pipeline's failure split: it is neither a number nor improvise's exit 3.
|
||||
|
||||
**Fix.** `churn.csv` has a fixed header. When no file changed, the script
|
||||
skips code-maat and improvise, writes the header alone, and exits 0.
|
||||
|
||||
**Red first.** The new case is a second fixture repository whose only
|
||||
in-window commit is empty. Restoring the `rows[0]` header fails its run case
|
||||
with the traceback.
|
||||
|
||||
**The review's four surviving faults, each now killed:**
|
||||
- **`--no-renames` dropped.** The fixture now renames `scripts/foo.sh` to
|
||||
`scripts/bar.sh`. Without the flag git reports one `{foo.sh => bar.sh}` entity
|
||||
with no lines changed, not a delete and an add. Four cases fail: the scripts/CI
|
||||
row, the code row, the ratio row and the coupling count.
|
||||
- **`crate_of` forgetting the root crate.** `src/main.rs` now co-changes with
|
||||
both crates. The crossing list loses three pairs.
|
||||
- **A failing `script` call ignored.** A third stub exits 0 for `import` and
|
||||
refuses `script`. The pipeline then reads exports that were never written and
|
||||
exits 1, not 3.
|
||||
- **`.beads/` and `site/` left out of generated.** Both are in the fixture now.
|
||||
The generated row and the other row both fail.
|
||||
|
||||
**Also counted now:** `docs/guide.md` as other docs and `.codex/config.toml` as
|
||||
agent docs. The coupling count is 16, not 15: `web/app.js` and
|
||||
`scripts/foo.sh` change together in two commits, the 06-04 one and the rename.
|
||||
|
||||
### The pipeline on folder import — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
|
||||
|
||||
Once #314 merged, `improvise import <dir>` replaced the file list, the `-`
|
||||
padding and the generated per-file views. The suite needed no change of
|
||||
expectations and still passed 41 of 41. That shows only that the numbers
|
||||
agree. It does **not** show that the model is right: the suite could not tell
|
||||
sparse from padded. The delta review re-added `pad_categories` and the suite
|
||||
still passed 41 of 41, while the model then had a `|-|` item in 13 places and
|
||||
the `churn` view seated `agent-rules` on rows and paged everything else. The
|
||||
view case counted 10 views either way.
|
||||
|
||||
**What separates them now:**
|
||||
- **"the sparse model has no padding item"**: no line of the model names `|-|`.
|
||||
- **"the churn view seats …"**: four cases, one per seat. `date` is on rows, and
|
||||
`agent-rules`, `coupled` and `crosses-crate`, which are coupling-only, are on
|
||||
`none`. The churn view puts `entity` on columns, not `area` as the review
|
||||
guessed. What discriminates is `date` on rows and the coupling-only
|
||||
categories on `none`, which a padded CSV cannot give, because it claims all
|
||||
nine categories.
|
||||
|
||||
**Rivals:**
|
||||
- **Padding re-added** (the review's own injection): 5 cases fail. They are
|
||||
the `|-|` case, with 161 lines naming it in the fixture model, and all four
|
||||
seat cases.
|
||||
- **A report view named like a file stem**: `By day` renamed to `churn` leaves
|
||||
9 views, and the view-count case fails.
|
||||
- **The foreign-CSV guard removed**: "a foreign CSV in --out is refused with
|
||||
exit 2" fails with exit 0. The guard is the answer to the review's
|
||||
non-blocking finding that `import .` picks up any CSV in `--out`. It refuses
|
||||
instead of clearing. Deleting files in a directory the caller named is the
|
||||
wrong default, and the script's own five CSVs are overwritten on every run,
|
||||
so a rerun into the same `--out` is still accepted. The same fixture run
|
||||
writes JSON and then text into one directory, which shows that.
|
||||
|
||||
@@ -85,6 +85,17 @@
|
||||
pkgs.jq
|
||||
];
|
||||
|
||||
# scripts/repo-stats runs code-maat (improvise-1xfb.5.1). The JRE is about
|
||||
# 0.6 GB, so it is kept out of ciInputs; only `nix-build` takes it.
|
||||
|
||||
codeMaatJar = pkgs.fetchurl {
|
||||
url = "https://github.com/adamtornhill/code-maat/releases/download/v1.0.4/code-maat-1.0.4-standalone.jar";
|
||||
hash = "sha256-QoeuIDSQGERFD3aVR7xEl6DaGm0cf6b63IWHBeZ0O18=";
|
||||
};
|
||||
codeMaat = pkgs.writeShellScriptBin "code-maat" ''
|
||||
exec ${pkgs.jre_headless}/bin/java -jar ${codeMaatJar} "$@"
|
||||
'';
|
||||
|
||||
# Playwright's downloaded browsers are generic-Linux builds that cannot
|
||||
# start on NixOS, so nixpkgs' build is used. harness/package.json pins
|
||||
# @playwright/test to this playwright-driver's version: they must match.
|
||||
@@ -114,6 +125,7 @@
|
||||
pkgs.global
|
||||
pkgs.universal-ctags
|
||||
beads.packages.${system}.default
|
||||
codeMaat
|
||||
|
||||
# Demo recording and release tooling
|
||||
pkgs.asciinema
|
||||
@@ -129,6 +141,13 @@
|
||||
// {
|
||||
improvise = cargoNix.workspaceMembers.improvise.build;
|
||||
default = self.packages.${system}.improvise;
|
||||
code-maat = codeMaat;
|
||||
# What the repo-stats pipeline runs besides the ci shell: one
|
||||
# installable, because check-ci-graph reads `nix shell <one> --command`.
|
||||
repo-stats-tools = pkgs.symlinkJoin {
|
||||
name = "repo-stats-tools";
|
||||
paths = [self.packages.${system}.improvise codeMaat];
|
||||
};
|
||||
};
|
||||
|
||||
# `nix flake check` builds every CI build: crane's checks plus the
|
||||
|
||||
@@ -101,6 +101,8 @@ ci_suites() {
|
||||
case "$1" in
|
||||
build) echo assemble-stamp ;;
|
||||
js) echo node-test ;;
|
||||
# Needs improvise and code-maat, which only this job has (tooling.md section 7).
|
||||
nix-build) echo repo-stats ;;
|
||||
# check-harness-lock must never read "could not resolve" as "stale" (improvise-zs2r).
|
||||
kb) echo kb checkers pr-watch release-preflight pr-reviews fmt-workspaces bd-audit-ids check-harness-lock agent-git ;;
|
||||
*) return 1 ;;
|
||||
|
||||
Executable
+321
@@ -0,0 +1,321 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Churn, coupling and doc-growth report over a window of history.
|
||||
|
||||
code-maat's analyses and this script's per-file churn go into one directory of
|
||||
CSVs; improvise imports them into one model and computes the report's tables.
|
||||
See context/code/tooling.md section 7.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
# First match wins, so the order is part of the definition.
|
||||
AREAS = [
|
||||
("generated", re.compile(r"^(\.beads/|site/|Cargo\.nix$|roadmap\.org$)"
|
||||
r"|\.(lock|cast)$|(^|/)package-lock\.json$")),
|
||||
("repo-map", re.compile(r"^context/repo-map\.md$")),
|
||||
("kb.pl", re.compile(r"^prolog-sessions/kb\.pl$")),
|
||||
("context/code", re.compile(r"^context/code/")),
|
||||
("context/test-evidence", re.compile(r"^context/test-evidence/")),
|
||||
("other context", re.compile(r"^context/")),
|
||||
("other docs", re.compile(r"^(docs/.*\.md|README\.md|RELEASING\.md)$")),
|
||||
("agent docs and skills", re.compile(r"^(AGENTS\.md|CLAUDE\.md|\.agents/|\.claude/|\.codex/)")),
|
||||
("scripts/CI", re.compile(r"^(scripts/|\.github/|nix/|flake\.nix$)")),
|
||||
("web and harness", re.compile(r"^(web|harness)/")),
|
||||
("rust tests", re.compile(r"(^|/)(tests|benches)/.*\.rs$|(^|/|_)tests?\.rs$")),
|
||||
("rust src", re.compile(r"\.(rs|pest)$")),
|
||||
]
|
||||
OTHER = "other"
|
||||
PROSE = {"context/code", "context/test-evidence", "other context", "other docs", "repo-map",
|
||||
"agent docs and skills"}
|
||||
CODE = {"rust src", "rust tests", "scripts/CI", "web and harness"}
|
||||
AGENT_RULES = "agent docs and skills"
|
||||
|
||||
# One CSV per code-maat analysis, plus churn.csv. Each stem becomes an item of
|
||||
# the model's File category, which the .improvise script pages on.
|
||||
ANALYSES = ["coupling", "abs-churn", "age", "soc"]
|
||||
CATEGORIES = ["commit", "date", "week", "entity", "area", "kind", "coupled",
|
||||
"crosses-crate", "agent-rules"]
|
||||
CHURN_HEADER = ["commit", "date", "week", "entity", "area", "kind", "added", "deleted"]
|
||||
MEASURES = ["added", "deleted", "commits", "degree", "average-revs", "age-months", "soc"]
|
||||
TABLES = ["by-area", "by-kind", "area-by-week", "by-day"]
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
class ImproviseFailed(Exception):
|
||||
"""improvise refused the pipeline's input: a product regression, not a statistic."""
|
||||
|
||||
|
||||
def area_of(path):
|
||||
for name, rx in AREAS:
|
||||
if rx.search(path):
|
||||
return name
|
||||
return OTHER
|
||||
|
||||
|
||||
def kind_of(area):
|
||||
return "prose" if area in PROSE else "code" if area in CODE else "other"
|
||||
|
||||
|
||||
def crate_of(path):
|
||||
"""The crate a path belongs to, or None for a path in no crate."""
|
||||
m = re.match(r"^crates/([^/]+)/", path)
|
||||
if m:
|
||||
return m.group(1)
|
||||
if path.startswith("harness/opfs-contract/"):
|
||||
return "opfs-contract"
|
||||
if re.match(r"^(src|tests|benches|examples)/", path) or path == "Cargo.toml":
|
||||
return "improvise"
|
||||
return None
|
||||
|
||||
|
||||
def git(repo, *args):
|
||||
return subprocess.run(["git", "-C", repo, *args], capture_output=True, text=True,
|
||||
check=True).stdout
|
||||
|
||||
|
||||
def resolve_ref(repo, ref):
|
||||
"""The ref asked for; the default gh/main falls back to HEAD where it is absent."""
|
||||
for r in ([ref] if ref else ["gh/main", "HEAD"]):
|
||||
out = subprocess.run(["git", "-C", repo, "rev-parse", "--verify", "--quiet",
|
||||
r + "^{commit}"], capture_output=True, text=True)
|
||||
if out.returncode == 0:
|
||||
return r, out.stdout.strip()
|
||||
sys.exit(f"repo-stats: cannot resolve {ref or 'gh/main or HEAD'}")
|
||||
|
||||
|
||||
def git2_log(repo, sha, since):
|
||||
"""The log in code-maat's git2 format: the ref's history, without merges."""
|
||||
return git(repo, "log", sha, "--no-merges", f"--since={since}", "--numstat",
|
||||
"--date=short", "--pretty=format:--%h--%ad--%aN", "--no-renames")
|
||||
|
||||
|
||||
def is_header(line):
|
||||
return line.startswith("--") and line.count("--") >= 3
|
||||
|
||||
|
||||
def churn_rows(log):
|
||||
"""One row per (commit, file). A commit is written `g<sha>`, because improvise
|
||||
reads an all-digit or exponent-shaped sha as a number (tooling.md section 7)."""
|
||||
rows, commit, date = [], None, None
|
||||
for line in log.split("\n"):
|
||||
if is_header(line):
|
||||
_, commit, date, _ = line.split("--", 3)
|
||||
elif "\t" in line:
|
||||
a, d, path = line.split("\t", 2)
|
||||
year, week, _ = datetime.date.fromisoformat(date).isocalendar()
|
||||
area = area_of(path)
|
||||
rows.append({"commit": "g" + commit, "date": date,
|
||||
"week": f"{year}-W{week:02d}", "entity": path, "area": area,
|
||||
"kind": kind_of(area), "added": 0 if a == "-" else int(a),
|
||||
"deleted": 0 if d == "-" else int(d)})
|
||||
return rows
|
||||
|
||||
|
||||
def write_csv(path, header, rows):
|
||||
with open(path, "w", newline="") as f:
|
||||
w = csv.DictWriter(f, fieldnames=header)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
|
||||
|
||||
def read_csv(path):
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
|
||||
|
||||
def code_maat(logfile, analysis, out, *extra):
|
||||
subprocess.run(["code-maat", "-l", logfile, "-c", "git2", "-a", analysis,
|
||||
"-o", out, *extra], capture_output=True, text=True, check=True)
|
||||
|
||||
|
||||
def flag_coupling(path):
|
||||
"""code-maat's coupling.csv, plus the two flags code-maat has no notion of."""
|
||||
rows = read_csv(path)
|
||||
for r in rows:
|
||||
a, b = crate_of(r["entity"]), crate_of(r["coupled"])
|
||||
r["crosses-crate"] = "yes" if a and b and a != b else "no"
|
||||
both = area_of(r["entity"]) == area_of(r["coupled"]) == AGENT_RULES
|
||||
r["agent-rules"] = "yes" if both else "no"
|
||||
write_csv(path, ["entity", "coupled", "degree", "average-revs", "crosses-crate",
|
||||
"agent-rules"], rows)
|
||||
|
||||
|
||||
def build_dir(out, log, rows, min_revs, today):
|
||||
"""The directory improvise imports: churn.csv and one CSV per analysis."""
|
||||
with tempfile.NamedTemporaryFile("w", suffix=".log", delete=False) as f:
|
||||
f.write(log + "\n")
|
||||
try:
|
||||
revs = ["--min-revs", str(min_revs), "--min-shared-revs", str(min_revs)]
|
||||
extra = {"coupling": revs, "age": ["--age-time-now", today]}
|
||||
for a in ANALYSES:
|
||||
code_maat(f.name, a, os.path.join(out, f"{a}.csv"), *extra.get(a, []))
|
||||
finally:
|
||||
os.unlink(f.name)
|
||||
flag_coupling(os.path.join(out, "coupling.csv"))
|
||||
|
||||
|
||||
def improvise(out, *args):
|
||||
exe = os.environ.get("IMPROVISE", "improvise")
|
||||
p = subprocess.run([exe, *args], cwd=out, capture_output=True, text=True)
|
||||
if p.returncode != 0 or "warning:" in p.stderr:
|
||||
raise ImproviseFailed(f"improvise {' '.join(args)} exited {p.returncode}:\n"
|
||||
f"{p.stderr}")
|
||||
|
||||
|
||||
def run_improvise(out):
|
||||
"""Import the directory into one model, a view per file included, then build
|
||||
the report's views and export them."""
|
||||
flags = [f"--category={c}" for c in CATEGORIES] + [f"--measure={m}" for m in MEASURES]
|
||||
improvise(out, "import", ".", "--no-wizard", *flags, "--name=Repo stats",
|
||||
"--formula=DeletionRatio = deleted / added FORMAT .3",
|
||||
"-o", "repo-stats.improv")
|
||||
os.makedirs(os.path.join(out, "report"), exist_ok=True)
|
||||
improvise(out, "script", os.path.join(HERE, "repo-stats.improvise"),
|
||||
"-f", "repo-stats.improv")
|
||||
return {t: read_rows(os.path.join(out, "report", f"{t}.csv")) for t in TABLES}
|
||||
|
||||
|
||||
def read_rows(path):
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.reader(f))
|
||||
|
||||
|
||||
def gotchas(text):
|
||||
"""(number, lines, chars) for each numbered item of repo-map's Gotchas section."""
|
||||
out, cur, inside = [], None, False
|
||||
for line in text.split("\n"):
|
||||
if line.startswith("## "):
|
||||
inside = line.startswith("## Gotchas")
|
||||
continue
|
||||
if inside and line.startswith("---"):
|
||||
inside = False
|
||||
if not inside:
|
||||
continue
|
||||
m = re.match(r"^(\d+)\. ", line)
|
||||
if m:
|
||||
cur = [int(m.group(1)), 0, 0]
|
||||
out.append(cur)
|
||||
if cur is not None and line.strip():
|
||||
cur[1] += 1
|
||||
cur[2] += len(line)
|
||||
return [tuple(g) for g in out]
|
||||
|
||||
|
||||
def doc_sizes(repo, sha):
|
||||
files = [p for p in git(repo, "ls-tree", "-r", "--name-only", sha, "--",
|
||||
"context").split("\n") if p.endswith(".md")]
|
||||
show = lambda p: git(repo, "show", f"{sha}:{p}")
|
||||
lines = {p: len(show(p).splitlines()) for p in sorted(files)}
|
||||
gs = gotchas(show("context/repo-map.md")) if "context/repo-map.md" in files else []
|
||||
longest = max(gs, key=lambda g: g[2], default=None)
|
||||
evidence = {p: sum(1 for l in show(p).split("\n") if re.match(r"^#{3,4} ", l))
|
||||
for p in sorted(files) if p.startswith("context/test-evidence/")}
|
||||
return {
|
||||
"context_lines": lines,
|
||||
"repo_map": {"gotchas": len(gs), "longest": longest and
|
||||
{"number": longest[0], "lines": longest[1], "chars": longest[2]}},
|
||||
"test_evidence": {"entries": sum(evidence.values()), "per_file": evidence},
|
||||
}
|
||||
|
||||
|
||||
def report(repo, ref, since, min_revs, out):
|
||||
name, sha = resolve_ref(repo, ref)
|
||||
log = git2_log(repo, sha, since)
|
||||
commits = sum(1 for line in log.split("\n") if is_header(line))
|
||||
today = git(repo, "show", "-s", "--date=short", "--format=%cd", sha).strip()
|
||||
r = {"ref": name, "sha": sha, "since": since, "commits": commits, "out": out,
|
||||
"docs": doc_sizes(repo, sha), "tables": {}, "coupling": [], "soc": [], "age": []}
|
||||
rows = churn_rows(log)
|
||||
write_csv(os.path.join(out, "churn.csv"), CHURN_HEADER, rows)
|
||||
# Empty commits alone change no file: nothing to analyse, so no model is built.
|
||||
if rows:
|
||||
build_dir(out, log, rows, min_revs, today)
|
||||
r["tables"] = run_improvise(out)
|
||||
for a in ("coupling", "soc", "age"):
|
||||
r[a] = read_csv(os.path.join(out, f"{a}.csv"))
|
||||
return r
|
||||
|
||||
|
||||
def table(rows):
|
||||
"""A CSV improvise exported, as aligned text; the first column left-aligned."""
|
||||
n = max(len(r) for r in rows)
|
||||
widths = [max(len(r[i]) for r in rows if i < len(r)) for i in range(n)]
|
||||
return [(" " + " ".join((c.ljust if i == 0 else c.rjust)(widths[i])
|
||||
for i, c in enumerate(r))).rstrip() for r in rows]
|
||||
|
||||
|
||||
TITLES = {
|
||||
"by-area": "Churn by area (improvise)",
|
||||
"by-kind": "Prose, code and other; ProseToCode = prose / code (improvise)",
|
||||
"area-by-week": "Lines added by area and ISO week (improvise)",
|
||||
"by-day": "Lines added per day, churn.csv beside code-maat's abs-churn (improvise)",
|
||||
}
|
||||
|
||||
|
||||
def text(r, top):
|
||||
out = [f"repo-stats: {r['ref']} ({r['sha'][:10]}), since {r['since']}, "
|
||||
f"{r['commits']} non-merge commits"]
|
||||
if not r["tables"]:
|
||||
out += ["", "No file changed in the window, so no model was built."]
|
||||
for t in TABLES:
|
||||
if r["tables"].get(t):
|
||||
out += ["", TITLES[t]] + table(r["tables"][t])
|
||||
out += ["", "Doc sizes (lines)"]
|
||||
out += [f" {n:>6} {path}" for path, n in r["docs"]["context_lines"].items()]
|
||||
rm, te = r["docs"]["repo_map"], r["docs"]["test_evidence"]
|
||||
lg = rm["longest"]
|
||||
out.append(f" repo-map gotchas: {rm['gotchas']}" + (
|
||||
f"; longest is {lg['number']} ({lg['lines']} lines, {lg['chars']} chars)"
|
||||
if lg else ""))
|
||||
out.append(f" test-evidence entries: {te['entries']}")
|
||||
out += ["", f"Change coupling (code-maat; top {top})"]
|
||||
for c in r["coupling"][:top]:
|
||||
flags = [f for f, k in (("CROSSES-CRATE", "crosses-crate"),
|
||||
("AGENT-RULES", "agent-rules")) if c[k] == "yes"]
|
||||
out.append(f" {c['degree']:>4}% {c['average-revs']:>4} revs {c['entity']}"
|
||||
f" <-> {c['coupled']}" + (f" [{', '.join(flags)}]" if flags else ""))
|
||||
out += ["", f"Sum of coupling (code-maat; top {top})"]
|
||||
out += [f" {s['soc']:>6} {s['entity']}" for s in r["soc"][:top]]
|
||||
out += ["", f"Code age in months (code-maat; first {top})"]
|
||||
out += [f" {a['age-months']:>4} {a['entity']}" for a in r["age"][:top]]
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("--ref", help="the ref to report on (default gh/main, else HEAD)")
|
||||
ap.add_argument("--since", default="7 days ago", help="git --since (default 7 days ago)")
|
||||
ap.add_argument("--min-revs", type=int, default=5,
|
||||
help="code-maat --min-revs and --min-shared-revs (default 5)")
|
||||
ap.add_argument("--top", type=int, default=15, help="rows per ranked table")
|
||||
ap.add_argument("--out", default="repo-stats",
|
||||
help="the directory for the CSVs, the model and its exports")
|
||||
ap.add_argument("--json", action="store_true", help="emit JSON instead of text")
|
||||
ap.add_argument("-C", dest="repo", default=".", help="the repository (default .)")
|
||||
a = ap.parse_args()
|
||||
os.makedirs(a.out, exist_ok=True)
|
||||
ours = {f"{n}.csv" for n in ["churn"] + ANALYSES}
|
||||
foreign = sorted(f for f in os.listdir(a.out) if f.lower().endswith(".csv") and f not in ours)
|
||||
if foreign:
|
||||
print(f"repo-stats: {a.out} holds CSVs this script did not write, and "
|
||||
f"`improvise import` would merge them: {', '.join(foreign)}", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
r = report(a.repo, a.ref, a.since, a.min_revs, os.path.abspath(a.out))
|
||||
except ImproviseFailed as e:
|
||||
print(f"repo-stats: {e}", file=sys.stderr)
|
||||
return 3
|
||||
print(json.dumps(r, indent=2) if a.json else text(r, a.top))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Executable
+233
@@ -0,0 +1,233 @@
|
||||
#!/usr/bin/env bash
|
||||
# check_the_checker for scripts/repo-stats (improvise-1xfb.5.1): the whole pipeline,
|
||||
# code-maat and improvise included, over a fixture repo whose every number is known.
|
||||
# See context/test-evidence/tooling.md.
|
||||
set -uo pipefail
|
||||
|
||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
STATS="$here/repo-stats"
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
|
||||
pass=0; fail=0
|
||||
ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); }
|
||||
bad() { printf ' FAIL %s (%s)\n' "$1" "$2"; fail=$((fail+1)); }
|
||||
|
||||
# The improvise under test: on PATH in CI, else this tree's debug build. No build
|
||||
# here: checkers.sh scrubs the linker's environment (context/code/tooling.md §7).
|
||||
if command -v improvise >/dev/null; then
|
||||
IMPROVISE="$(command -v improvise)"
|
||||
elif [ -x "$here/../target/debug/improvise" ]; then
|
||||
IMPROVISE="$here/../target/debug/improvise"
|
||||
else
|
||||
echo "repo-stats-test: no improvise; run \`cargo build --bin improvise\` first" >&2
|
||||
exit 1
|
||||
fi
|
||||
export IMPROVISE
|
||||
|
||||
# $1 name, $2 jq expression over $work/out.json, $3 the value it must print
|
||||
is() {
|
||||
local got
|
||||
got="$(jq -c "$2" "$work/out.json" 2>&1)"
|
||||
[ "$got" = "$3" ] && ok "$1" || bad "$1" "got $got, wanted $3"
|
||||
}
|
||||
|
||||
# $1 name, $2 table, $3 row label, $4 the row's cells after the label, as JSON
|
||||
row() {
|
||||
is "$1" ".tables[\"$2\"][]|select(.[0]==\"$3\")|.[1:]" "$4"
|
||||
}
|
||||
|
||||
# $1 name, $2 fixed-string the text report must contain
|
||||
says() {
|
||||
grep -Fq -- "$2" "$work/out.txt" && ok "$1" || bad "$1" "no '$2' in the text report"
|
||||
}
|
||||
|
||||
export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null
|
||||
export GIT_AUTHOR_NAME=fixture GIT_AUTHOR_EMAIL=f@x GIT_COMMITTER_NAME=fixture GIT_COMMITTER_EMAIL=f@x
|
||||
repo="$work/repo"
|
||||
g() { git -C "$repo" "$@" >/dev/null; }
|
||||
|
||||
# $1 path, $2 line count, $3 tag. Rewriting with a new tag replaces every line.
|
||||
put() { mkdir -p "$(dirname "$repo/$1")"; seq -f "$3 %g" 1 "$2" > "$repo/$1"; }
|
||||
# $1 date; commits everything staged or not.
|
||||
commit() {
|
||||
g add -A
|
||||
GIT_AUTHOR_DATE="$1T12:00:00Z" GIT_COMMITTER_DATE="$1T12:00:00Z" g commit -q -m "at $1"
|
||||
}
|
||||
|
||||
git init -q -b main "$repo"
|
||||
|
||||
# Before the window: must count nowhere.
|
||||
put README.md 50 old; commit 2026-01-01
|
||||
|
||||
# Two commits co-change six files, so each pair couples at 100% over 2 revs.
|
||||
# src/ is the root crate's, so it crosses a boundary with both crates/.
|
||||
for f in crates/a/src/lib.rs crates/b/src/lib.rs; do put "$f" 10 x1; done
|
||||
put src/main.rs 5 x1; put crates/a/tests/t.rs 4 x1
|
||||
put AGENTS.md 3 x1; put .agents/skills/wave/agent-rules.md 3 x1
|
||||
commit 2026-06-02
|
||||
for f in crates/a/src/lib.rs crates/b/src/lib.rs; do put "$f" 6 x2; done
|
||||
put src/main.rs 5 x2; put crates/a/tests/t.rs 4 x2
|
||||
put AGENTS.md 3 x2; put .agents/skills/wave/agent-rules.md 3 x2
|
||||
commit 2026-06-03
|
||||
|
||||
# One file in each remaining area. The repo map is 13 lines.
|
||||
put context/code/x.md 8 x3
|
||||
mkdir -p "$repo/context/test-evidence"
|
||||
printf '### t_one\ntext\n#### t_two\n## not an entry\n### t_three\n' \
|
||||
> "$repo/context/test-evidence/y.md"
|
||||
cat > "$repo/context/repo-map.md" <<'EOF'
|
||||
# Map
|
||||
|
||||
## Gotchas (read before editing)
|
||||
|
||||
1. **One.** short.
|
||||
2. **Two.** a much longer gotcha, with plenty of words in it to be the longest one
|
||||
and a second line.
|
||||
|
||||
3. **Three.** middling length.
|
||||
|
||||
---
|
||||
|
||||
4. not a gotcha, because the section has ended
|
||||
EOF
|
||||
put context/plan.md 2 x3
|
||||
put prolog-sessions/kb.pl 5 x3
|
||||
put scripts/foo.sh 7 x3; put .github/workflows/ci.yml 1 x3
|
||||
put web/app.js 9 x3; put harness/x.js 2 x3
|
||||
put Cargo.lock 11 x3
|
||||
put crates/a/src/grammar.pest 3 x3
|
||||
put .beads/issues.jsonl 2 x3; put site/index.html 1 x3
|
||||
put docs/guide.md 2 x3; put .codex/config.toml 1 x3
|
||||
commit 2026-06-04
|
||||
g tag before-shrink
|
||||
|
||||
# A side branch merged with a merge commit: the side commit counts, the merge does not.
|
||||
g checkout -q -b side
|
||||
put notes.txt 4 x5; commit 2026-06-05
|
||||
g checkout -q main
|
||||
GIT_AUTHOR_DATE=2026-06-06T12:00:00Z GIT_COMMITTER_DATE=2026-06-06T12:00:00Z \
|
||||
g merge -q --no-ff -m merge side
|
||||
|
||||
# A shrink in the next ISO week, so one area deletes and one row spans two weeks,
|
||||
# and a rename, which --no-renames counts as a delete and an add.
|
||||
put web/app.js 3 x4
|
||||
g mv scripts/foo.sh scripts/bar.sh
|
||||
commit 2026-06-09
|
||||
|
||||
# $1 output dir, $2 "text" to also write the text report, rest: extra args.
|
||||
run() {
|
||||
local out="$1" text="$2"; shift 2
|
||||
"$STATS" -C "$repo" --since 2026-06-01 --min-revs 2 --out "$out" "$@" --json \
|
||||
> "$work/out.json" 2> "$work/err" \
|
||||
|| { bad "repo-stats $* ran" "$(cat "$work/err")"; return 1; }
|
||||
[ "$text" = text ] || return 0
|
||||
"$STATS" -C "$repo" --since 2026-06-01 --min-revs 2 --out "$out" "$@" \
|
||||
> "$work/out.txt" 2>&1 \
|
||||
|| { bad "repo-stats $* ran (text)" "$(cat "$work/out.txt")"; return 1; }
|
||||
}
|
||||
|
||||
if run "$work/stats" text; then
|
||||
is "no gh/main in the fixture, so the ref is HEAD" '.ref' '"HEAD"'
|
||||
is "the merge commit is not counted, the side commit is" '.commits' '5'
|
||||
row "rust src is .rs and .pest outside tests" by-area "rust src" '["45","25","0.556"]'
|
||||
row "rust tests are the tests/ files" by-area "rust tests" '["8","4","0.500"]'
|
||||
row "agent docs and skills take .codex" by-area "agent docs and skills" '["13","6","0.462"]'
|
||||
row "docs/*.md is other docs" by-area "other docs" '["2","0","0.000"]'
|
||||
row "context/code" by-area "context/code" '["8","0","0.000"]'
|
||||
row "context/test-evidence" by-area "context/test-evidence" '["5","0","0.000"]'
|
||||
row "repo-map is its own area" by-area "repo-map" '["13","0","0.000"]'
|
||||
row "other context" by-area "other context" '["2","0","0.000"]'
|
||||
row "kb.pl" by-area "kb.pl" '["5","0","0.000"]'
|
||||
row "scripts/CI takes .github, and a rename is a delete and an add" \
|
||||
by-area "scripts/CI" '["15","7","0.467"]'
|
||||
row "web and harness, and its deletion ratio" by-area "web and harness" '["14","9","0.643"]'
|
||||
row "a lock file, .beads/ and site/ are generated" by-area "generated" '["14","0","0.000"]'
|
||||
row "the pre-window README counts nowhere" by-area "other" '["4","0","0.000"]'
|
||||
row "prose is the context areas, other docs and agent docs" by-kind prose '["43","6","0.140"]'
|
||||
row "code is rust, scripts/CI and web" by-kind code '["82","45","0.549"]'
|
||||
row "prose to code is prose over code" by-kind ProseToCode '["0.524","0.133",""]'
|
||||
is "weeks are columns, first-seen first" '.tables["area-by-week"][0][1:]' '["2026-W24","2026-W23"]'
|
||||
row "an area is split by ISO week" area-by-week "web and harness" '["3","11"]'
|
||||
is "every day's churn.csv total equals code-maat's abs-churn" \
|
||||
'[.tables["by-day"][1:][]|.[1]==.[2]]|all' 'true'
|
||||
row "a day's total is every file changed that day" by-day 2026-06-04 '["72","72"]'
|
||||
is "gotchas end with their section" '.docs.repo_map.gotchas' '3'
|
||||
is "the longest gotcha is measured over its lines" \
|
||||
'.docs.repo_map.longest|[.number,.lines]' '[2,2]'
|
||||
is "test-evidence entries are ### and #### headings" '.docs.test_evidence.entries' '3'
|
||||
is "coupling finds the 15 pairs of six files, and web/app.js with scripts/foo.sh" \
|
||||
'.coupling|length' '16'
|
||||
is "files of two crates, the root crate's included, cross a boundary" \
|
||||
'[.coupling[]|select(.["crosses-crate"]=="yes")|[.entity,.coupled]|sort]|sort' \
|
||||
'[["crates/a/src/lib.rs","crates/b/src/lib.rs"],["crates/a/src/lib.rs","src/main.rs"],["crates/a/tests/t.rs","crates/b/src/lib.rs"],["crates/a/tests/t.rs","src/main.rs"],["crates/b/src/lib.rs","src/main.rs"]]'
|
||||
is "two agent-rule docs are flagged" \
|
||||
'[.coupling[]|select(.["agent-rules"]=="yes")|[.entity,.coupled]|sort]' \
|
||||
'[[".agents/skills/wave/agent-rules.md","AGENTS.md"]]'
|
||||
says "the text report flags a crate crossing" "[CROSSES-CRATE]"
|
||||
says "the text report flags agent-rule docs" "[AGENT-RULES]"
|
||||
says "the text report prints improvise's ratio row" "ProseToCode"
|
||||
views="$(grep -c '^## View: ' "$work/stats/repo-stats.improv")"
|
||||
[ "$views" = 10 ] && ok "the model keeps a view per file and the report's four" \
|
||||
|| bad "the model keeps a view per file and the report's four" "$views views"
|
||||
# Sparse, not padded: a padded merge has the same numbers and views, but a
|
||||
# `-` item in every category and per-file views seated on the padding.
|
||||
dashes="$(grep -c '|-|' "$work/stats/repo-stats.improv")"
|
||||
[ "$dashes" = 0 ] && ok "the sparse model has no padding item" \
|
||||
|| bad "the sparse model has no padding item" "$dashes lines name |-|"
|
||||
churn_view="$(awk '/^## View: churn$/,/^$/' "$work/stats/repo-stats.improv")"
|
||||
for seat in "date: row" "agent-rules: none" "coupled: none" "crosses-crate: none"; do
|
||||
grep -qx -- "$seat" <<< "$churn_view" && ok "the churn view seats $seat" \
|
||||
|| bad "the churn view seats $seat" "$(tr '\n' ' ' <<< "$churn_view")"
|
||||
done
|
||||
fi
|
||||
|
||||
# An --out holding a CSV this script did not write is refused, since the folder
|
||||
# import would merge it. The script's own CSVs, from a rerun, are not foreign.
|
||||
mkdir -p "$work/foreign"; echo "x,y" > "$work/foreign/other.csv"
|
||||
"$STATS" -C "$repo" --since 2026-06-01 --out "$work/foreign" > /dev/null 2>&1
|
||||
got=$?
|
||||
[ "$got" = 2 ] && ok "a foreign CSV in --out is refused with exit 2" \
|
||||
|| bad "a foreign CSV in --out is refused with exit 2" "exit $got"
|
||||
|
||||
# The window is scoped to the ref: at the tag, the shrink has not happened.
|
||||
if run "$work/at-tag" json --ref before-shrink; then
|
||||
row "--ref scopes the log" by-area "web and harness" '["11","0","0.000"]'
|
||||
is "--ref names the ref it read" '.ref' '"before-shrink"'
|
||||
fi
|
||||
|
||||
# A window whose only commits are empty changes no file: exit 0, and no model.
|
||||
empty="$work/empty"
|
||||
git init -q -b main "$empty"
|
||||
repo="$empty"; put README.md 5 old; commit 2026-01-01
|
||||
GIT_AUTHOR_DATE=2026-06-02T12:00:00Z GIT_COMMITTER_DATE=2026-06-02T12:00:00Z \
|
||||
g commit -q --allow-empty -m empty
|
||||
if run "$work/empty-out" text; then
|
||||
is "an empty commit is counted" '.commits' '1'
|
||||
is "and builds no tables" '.tables' '{}'
|
||||
lines="$(wc -l < "$work/empty-out/churn.csv" | tr -d ' ')"
|
||||
[ "$lines" = 1 ] && ok "churn.csv is written with its header alone" \
|
||||
|| bad "churn.csv is written with its header alone" "$lines lines"
|
||||
says "the text report says why there are no tables" "No file changed in the window"
|
||||
fi
|
||||
repo="$work/repo"
|
||||
|
||||
# improvise refusing the pipeline is exit 3, whatever the numbers were. The
|
||||
# third stub imports cleanly and refuses only the scripts.
|
||||
mkdir -p "$work/bin"
|
||||
printf '#!/usr/bin/env bash\necho "Parse error: no" >&2\nexit 1\n' > "$work/bin/refuses"
|
||||
printf '#!/usr/bin/env bash\necho "warning: Skipped 3 of 3 records" >&2\nexit 0\n' \
|
||||
> "$work/bin/warns"
|
||||
printf '#!/usr/bin/env bash\n[ "$1" = script ] || exit 0\necho "Parse error: no" >&2\nexit 1\n' \
|
||||
> "$work/bin/refuses-a-script"
|
||||
chmod +x "$work/bin/refuses" "$work/bin/warns" "$work/bin/refuses-a-script"
|
||||
for stub in refuses warns refuses-a-script; do
|
||||
IMPROVISE="$work/bin/$stub" "$STATS" -C "$repo" --since 2026-06-01 --out "$work/$stub" \
|
||||
> /dev/null 2>&1
|
||||
got=$?
|
||||
[ "$got" = 3 ] && ok "an improvise that $stub fails the pipeline with exit 3" \
|
||||
|| bad "an improvise that $stub fails the pipeline with exit 3" "exit $got"
|
||||
done
|
||||
|
||||
printf '\nrepo-stats: %d passed, %d failed\n' "$pass" "$fail"
|
||||
[ "$fail" = 0 ]
|
||||
@@ -0,0 +1,87 @@
|
||||
# The views scripts/repo-stats builds over the model it imported, run by
|
||||
# `improvise script` with the output directory as the working directory.
|
||||
# Every `export` here is a table the text report prints. See
|
||||
# context/code/tooling.md section 7.
|
||||
|
||||
# The prose-to-code ratio is a formula over the kind category's own items.
|
||||
add-formula kind "ProseToCode = prose / code FORMAT .3"
|
||||
|
||||
# Churn by area, with the deletion ratio (a _Measure formula from the import).
|
||||
create-view "By area"
|
||||
switch-view "By area"
|
||||
set-axis File page
|
||||
set-page File churn
|
||||
set-axis area row
|
||||
set-axis _Measure column
|
||||
set-axis kind none
|
||||
set-axis week none
|
||||
set-axis date none
|
||||
set-axis commit none
|
||||
set-axis entity none
|
||||
set-axis coupled none
|
||||
set-axis crosses-crate none
|
||||
set-axis agent-rules none
|
||||
toggle-prune-empty
|
||||
set-format .0
|
||||
export report/by-area.csv
|
||||
|
||||
# Prose, code and other, and the ratio of the first two.
|
||||
create-view "By kind"
|
||||
switch-view "By kind"
|
||||
set-axis File page
|
||||
set-page File churn
|
||||
set-axis kind row
|
||||
set-axis _Measure column
|
||||
set-axis area none
|
||||
set-axis week none
|
||||
set-axis date none
|
||||
set-axis commit none
|
||||
set-axis entity none
|
||||
set-axis coupled none
|
||||
set-axis crosses-crate none
|
||||
set-axis agent-rules none
|
||||
toggle-prune-empty
|
||||
set-format .0
|
||||
export report/by-kind.csv
|
||||
|
||||
# Lines added per area per ISO week.
|
||||
create-view "Area by week"
|
||||
switch-view "Area by week"
|
||||
set-axis File page
|
||||
set-page File churn
|
||||
set-axis area row
|
||||
set-axis week column
|
||||
set-axis _Measure page
|
||||
set-page _Measure added
|
||||
set-axis kind none
|
||||
set-axis date none
|
||||
set-axis commit none
|
||||
set-axis entity none
|
||||
set-axis coupled none
|
||||
set-axis crosses-crate none
|
||||
set-axis agent-rules none
|
||||
toggle-prune-empty
|
||||
set-format .0
|
||||
export report/area-by-week.csv
|
||||
|
||||
# Lines added per day, from both sources side by side: this script's churn
|
||||
# rows and code-maat's abs-churn. The two columns must agree.
|
||||
create-view "By day"
|
||||
switch-view "By day"
|
||||
set-axis date row
|
||||
set-axis File column
|
||||
set-axis _Measure page
|
||||
set-page _Measure added
|
||||
set-axis area none
|
||||
set-axis kind none
|
||||
set-axis week none
|
||||
set-axis commit none
|
||||
set-axis entity none
|
||||
set-axis coupled none
|
||||
set-axis crosses-crate none
|
||||
set-axis agent-rules none
|
||||
toggle-prune-empty
|
||||
set-format .0
|
||||
export report/by-day.csv
|
||||
|
||||
switch-view "By area"
|
||||
Reference in New Issue
Block a user