improvise/context/test-evidence/gp.md

# Test evidence: `gp-tree` and `improvise-gp`

What each test's bug was, what fault was injected to prove the test can fail,
and why the obvious weaker version of it proved nothing. Keyed by exact test
name — a rename orphans the entry, which
`scripts/check-comment-runs.py --check-evidence` catches.

The architectural arguments these tests defend are in `context/code/gp.md`.

---

## `gp-tree/src/lib.rs`

### splitmix64_matches_the_reference_vectors

The vendored RNG's constants had nothing pinning them. Changing the gamma from
`0x9E3779B97F4A7C15` to `…7C17` left **every other test in the workspace
green**, because they only ever assert that two runs at one seed agree.

The crate's promise is stronger: a fit's seed is printed to the user and saved
in `data::Model`, so a *later* run at an old seed must reproduce the old fit.
These are therefore the published SplitMix64 vectors for seed 0, not a golden
capture of this code — which is what makes them evidence rather than a
tautology.

### splitmix64_matches_the_reference_vectors_from_a_nonzero_seed

Seed 0 alone cannot catch a change to how the seed enters the state
(`state = seed` versus `state = 0`), because both give the same stream at 0.
Also asserts `seed()` still answers the seed after the generator has advanced.

---

## `gp-tree/src/fit.rs`

### mse_averages_over_exactly_the_rows_it_was_given

`mse` used to take parallel `x`/`y` slices, sum over `zip(x, y)` — which stops
at the shorter — and divide by `x.len()`. A caller that got the two out of step
was handed a mean that was quietly **too small** rather than an error: the
improvise-n3wt hazard, a count supplied alongside the data rather than taken
from it.

The remedy was not a length check but a type: one slice of `Row`, so the shorter
slice cannot be built (improvise-dkqn). This test is what is left to say about
it — numerator and denominator count the same rows, for any prefix.

### scoring_no_rows_is_zero_error_rather_than_nan

Every mean in the module reached `0.0 / 0.0` on an empty slice and handed back a
NaN a caller could not tell from a score: it propagates through
`FitConfig::cost` and sorts unpredictably against real costs. One answer, in
`mean_over`, covers `mse`, `misclass_rate` and `standardize`, and the test
asserts all three plus `std == 1e-6` (a degenerate spread still never divides by
zero).

### asking_for_no_folds_yields_one_rather_than_dividing_by_zero

`k` reaches `folds` from a CLI flag, so zero is a value a caller can supply
rather than an impossible state. `out[pos % k]` divided by it and panicked — a
public library function taking its host down with it.

### cross_validating_without_a_held_out_fold_pools_nothing

The sharper half of the same parameter (improvise-ry89). A cross-validated
prediction is one made by a tree that never saw the row it scores, so `k < 2`
asks for something that cannot exist: the single fold holds every index, the
training complement is empty, and the tree is fit and standardized on nothing
before predicting every row.

**This used to announce itself** — `standardize(&[])` gave a NaN mean and every
pooled prediction came back NaN. Once `mean_over` answered the empty row set
with `0.0`, the same call handed back near-zero numbers indistinguishable from a
model's output, and an R² computed off them is a number rather than a NaN. So
fixing one bug hid another, and the count is the contract: no held-out row means
no pair. The test also asserts the generator is **untouched**, because declining
before the shuffle is what keeps a caller's next draw the one it would have made.

### the_training_split_is_what_the_contains_scan_gave

The O(n²) fix (improvise-t3ux) replaced `test.contains(i)` per row with a bool
mask. It is only safe because the mask yields the *same indices in the same
order* — the training rows are what every downstream draw from the generator
depends on. Pinned directly, at every fold count, rather than inferred from a
fit that happened to agree.

### a_fit_improves_on_the_tree_it_started_from

Without it, a `fit` that returned its seed tree unchanged would satisfy both
`the_same_seed_and_config_fit_the_same_tree` and
`a_different_seed_fits_a_different_tree`, because the initial `random_tree`
already differs across seeds.

### fit_detailed_starts_from_the_best_of_generation_zero

What `start` has to mean for the question it exists to answer: the best of
generation 0, not the seed ancestor it grew from (improvise-ut7p). The two
differ whenever a mutated member of the initial population beats member 0, and
only the former makes "the search improved on where it started" a claim about
**breeding**. Before this, `improvise-gp` kept a second copy of the whole GA
loop just to reach that starting point.

### a_tiny_lone_importance_is_still_the_whole_of_it

Copilot on PR #25. `fit_csv` divided by `total.max(1e-9)`, which is right for
the only input it ever had and wrong as a contract: one feature carrying an
importance of `1e-10` carries all of what the model knows and was reported at
**10%**. Importances this small are ordinary — they are error differences over a
*standardized* target. `the_shares_sum_to_a_hundred_at_any_scale` is the general
form across four orders of magnitude.

### denormalize_rescales_the_leaves_and_leaves_the_splits_alone

The negative arm is the load-bearing one: `standardize` rescales the **target**,
so a threshold is already in its feature's own units and re-scaling it with the
leaves would be the bug.
`a_tree_fitted_on_standardized_rows_predicts_original_units_after_denormalize`
pins the arithmetic against a hand-built tree rather than a search, so it does
not depend on the GA.

---

## `gp-tree/src/data.rs`

### an_entirely_missing_column_is_skipped_and_reported_as_data

`preprocess` used to say so with an `eprintln!`, which was correct while this
code was `examples/fit_csv.rs`. A library does not own its caller's stderr: a
TUI on an alternate screen, a wasm build, or a caller wanting the notice in its
own status line could none of them suppress, redirect or translate it. The
notice now comes back as data.

### seed_zero_is_distinguishable_from_no_seed

`0` is a legal seed, so "unknown" and "fitted at 0" have to be two different
values — anything reporting the seed of a pre-seed model would otherwise state a
seed that never ran. That is why `Model::seed` is an `Option` and not a
sentinel.

### a_saved_model_round_trips_with_its_whole_ga_config

The seed alone is necessary and **not sufficient**: the same seed under a
different `FitConfig` searches differently and lands somewhere else, so a model
carrying only a seed looks reproducible while being reproducible only by whoever
still remembers the flags.

The fixture `a_fit_config()` differs from `FitConfig::default()` in **every**
field, and the test asserts that explicitly. A round trip that silently
substituted the default would otherwise pass against a default-constructed
value — the failure mode this test exists for.

### a_model_saved_before_configs_still_loads

A backward-compatibility test aimed at the **absent field**, not at the
attribute. `#[serde(default)]` on an `Option` is documentation rather than
mechanism — serde defaults an `Option` anyway, so a test that merely deletes the
attribute fails nothing and is not discriminating. This one removes `config`
from the serialized JSON and asserts it loads as `None` (unknown), not as
`Some(FitConfig::default())`. `a_model_saved_before_seeds_still_loads` is the
same shape for `seed`.

---

## `gp-tree/tests/scoring_known_answers.rs`

Whole-file note: these are `f32` **bit patterns**, not approximate values,
because a change that merely moves an accumulation around silently re-points
every seed already saved at a different tree. `assert!((a - b).abs() < 1e-6)`
cannot see that, and neither can the CLI's output, rounded to three or four
decimals. The numbers were captured from the parallel-`x`/`y` code *before*
improvise-dkqn reshaped the signatures and improvise-t3ux replaced the
`contains` scan, and are unchanged by both. If you are changing the search on
purpose these numbers change with it — but then every seed recorded by an older
`--save` names a different tree, and that is the decision this file forces you
to make out loud.

### standardize_is_bit_for_bit_what_it_was

**A witness that could not see the bit.** The first and last rows plus
`mean`/`std` were the whole witness, and four numbers cannot separate two ways
of computing the same quantity: injecting `r.target / std - mean / std` for
`(r.target - mean) / std` — algebraically identical, a different rounding — left
this test **and both cross-validation vectors** green (improvise-ry89). All
thirty-seven values do separate them: that reassociation moves 21 of the rows,
and none of them were rows 0, 36 or 19.

Note what is *not* evidence here. Reversing the mean's accumulation order also
left every vector green, but on this fixture the two orders sum to the same
`f32` — so that injection changed no behaviour to catch. **A perturbation has to
move a bit before a bit-level witness can be blamed for missing it.**

### cv_regress_pools_the_same_predictions_and_leaves_the_rng_where_it_was

The `rng_after` witness is the part that pins the *order* of the generator's
calls: the next `u64` after a cross-validation depends on exactly how many draws
that run made, so a reshaped loop that drew differently would move it even if
every pooled prediction happened to agree. 37 rows is a prime, so no fold count
divides it evenly and the last fold is short — where an off-by-one in the
train/test split shows.

### cv_classify_pools_the_same_codes_and_leaves_the_rng_where_it_was

The classification half of the same guard. The mask replaced `contains` in both
loops, and one arm passing says nothing about the other
(`doctrine(one_arm_of_a_symmetric_rule_is_evidence_about_that_arm_only)`).

---

## `improvise-gp/src/lib.rs`

### improvise_evaluation_matches_gp_tree_eval

The acceptance property: over random trees and random feature vectors,
improvise's evaluation of the emitted formula matches `gp_tree::eval`
**exactly** — widening f32 to f64 is lossless and the thresholds are exact, so
there is no tolerance to allow.

#### The `improvise_eval` helper, and the shape to remember

This test and `a_value_sitting_on_the_threshold_falls_the_same_side` both run
through the `improvise_eval` helper, and its scope is what was once read as
covering more than it does. **It fabricates a `_Measure` item per feature
name**, so it asserts the emitter and `gp_tree::eval` agree **against a model
that could not exist**. It says `Ok` to `outlook=rainy` because it just made an
item called that. A helper that builds the world the code wants is not evidence
that the world exists.

So the awkward names above are a *quoting* fixture and nothing more:
`outlook=rainy` is there because it is awkward to spell, not because a document
could hold it — it could not, and `fitted_formula` refuses it. The helper is
still the right tool for the arithmetic-and-quoting claims; the property it
could not state is asserted separately, against a real document, in
`evaluability_in_a_real_document`.

### an_awkward_target_name_is_quoted_and_round_trips

This test used to assert `parsed.target == "|Total Revenue|"` and call the pipes
correct. They are not: the target names a `_Measure` item, so a target carrying
its delimiters writes the fit into an item literally named `|Total Revenue|`,
which is not the item the user has. What the emitter owes its caller is a
*semantic* round trip — the name goes in, the same name comes back out.
`an_awkward_target_names_the_users_measure_item` is the consequence that makes
it a data bug rather than a cosmetic one.

### an_infinite_threshold_is_refused_rather_than_emitted_as_inf

`number_text(f32::INFINITY)` emitted `"inf"`, which `"inf".parse::<f64>()`
accepts — so the round-trip loop returned it and a `formula_text` caller got
text the grammar rejects, with nothing on the way saying so. `fitted_formula`
would surface it as a parse error; a bare `formula_text` caller would write it
into a file.

### a_nan_threshold_is_refused_by_name

NaN fell through the loop to `unreachable!`, so the panic blamed the decimal
expansion instead of the caller's value.

### a_threshold_is_the_shortest_decimal_that_reads_back_the_same

The doc used to overclaim the emitted text as the f32's full decimal expansion.
It is the *shortest* decimal that reads back as the same `f64` — the weaker
claim, and the one the routing argument actually needs: equal text means equal
`f64` means the same branch.

### a_value_sitting_on_the_threshold_falls_the_same_side

The boundary case the whole `number_text` design exists for. `0.7f32` is really
0.699999988…; the shortest text that reads back as that **f32** is `"0.7"`,
which parses as an `f64` strictly *greater* than it. A row whose feature value
equals the threshold routes `hi` under `eval` and would route `lo` through a
formula carrying the rounded threshold. The test asserts the trap is live —
`"0.7".parse::<f64>() > thresh as f64` — before relying on it.

### a_split_on_a_one_hot_feature_is_spelled_as_a_coordinate

improvise-8xhy's defect, and improvise-4zl7's answer to it. `preprocess`
one-hots a categorical column into features named `Region=North`, an encoding
artefact no document has an item for, so `find_item_category` returned `None`
and every cell of the fitted measure came back as `Error("ref:Region=North")` —
silently, with the emitter having said nothing. 8xhy shipped a refusal; 4zl7
replaced the refusal with a spelling, `Region IS North`.

The assertion is the emitted **text**, exactly, and not merely that the call is
`Ok`: the whole failure mode here was a well-formed formula meaning the wrong
thing, so "it parsed" is the assertion that missed it the first time. The
fault to inject is `Spelling::Coordinate` rendering as `quote_name(name)` —
byte-identical for every measure, and wrong for exactly this family.

### a_coordinate_naming_no_category_of_this_document_is_refused

The negative arm of the same rule, and it is not redundant with the artefact
refusal: a `Coordinate` is *spellable* and may still be unresolvable. Without
it, `resolves_in`'s `Coordinate` arm could return `true` unconditionally and
every test above would still pass.

### a_measure_that_merely_looks_like_a_one_hot_is_accepted

The negative arm, and why the check asks the **document** rather than refusing
names that look like encoding artefacts: a measure literally named
`Region=North` is a legal document. The two are indistinguishable as strings.
`the_missing_flag_and_the_date_sub_features_are_refused_too` covers the other
two artefact families with nothing enumerating them — the check asks the exact
precondition `Expr::Ref` has, so a future naming scheme is covered without this
crate learning about it.

### a_formula_the_emitter_accepted_evaluates_everywhere

The property the lying test could not state: **if `fitted_formula` says `Ok`, no
cell of the target is a `ref:` error.** It is also what stops the refusal tests
above from passing by refusing everything.

### a_tree_indexing_past_the_spec_says_so

improvise-hkt6. A tree whose `feat` runs past the spec violates the same
precondition in two places, and the two answered differently: `expr_text` said
what was wrong, `mark_features` said `index out of bounds`. Whichever ran first
decided what the caller was told, and `fitted_formula` runs `mark_features`
first — so the legible message was the one nobody ever saw. Driven through
`fitted_formula` because both private functions are reached that way, and the
**message** is asserted rather than merely the panic, since an out-of-bounds
index panics too.

### a_record_added_after_the_fit_is_scored_by_the_formula_alone

The load-bearing half of improvise-7na's answer: scoring a *new* row needs no
saved encoding plan. `fit_csv --predict` calls `encode_row` because its new row
arrives as untyped CSV text; improvise's new row is a record in the same
document, scored through the stored formula from its own measures. So nothing
about the `--predict` path argues for putting a `gp_tree::data::Model` in the
file.

### breeding_improves_on_generation_zero

What rules out a GA whose `breed` returned its input unchanged.
`a_different_seed_fits_a_different_formula` is weak evidence on its own, since
the initial `random_tree` already differs across seeds.

---

## `improvise-gp/src/dataset.rs`

### a_measure_on_the_page_axis_does_not_filter_away_the_training_rows

improvise-vx3. `Workbook::new`'s default view puts `_Measure` on `Axis::Page`,
so `filtering_cats()` named it, `filter_coords` pinned it to the first measure,
and every "record" then carried exactly one measure cell — never the target. The
whole table came back as "no visible record has a Revenue value to learn from"
on a document plainly full of them, which reads as a data problem rather than as
an axis one.

**Every test in the module used `View::new` + `on_category_added`**, under which
a leading-underscore category lands on `Axis::None` — so the fixture was quietly
building the one world the code worked in, and no test could see the default a
real `Workbook` hands out.

The assertion compares the **contents**, not the shape. Pinning only the header
and the row count passes on this fixture even with the bug present: `_Measure`'s
first item is `Revenue`, so every row still has a target and only the feature
columns come back blank — the shape survives while the data does not.

### a_paged_measure_that_is_not_the_target_does_not_empty_the_table

The sharper half: `filter_coords` pinned to the *first* item, so on a document
whose target is not first the table came back empty.

### the_feature_table_is_one_row_per_record_with_the_target_last

The literal row **order** is half of what this pins, and f32 accumulation over
the rows is why: a reordered table makes a seeded fit irreproducible in a way
nothing prints. Since improvise-shyt `matching_cells` yields insertion order, so
the assertion now holds by construction; before it, a test in one process could
not tell "sorted" from "this process happened to hash them in that order".

### a_formula_measure_is_not_a_feature

The fixture lists `Profit` as an explicit `_Measure` item **as well as** making
it a formula target, which is legal and is the only case this test can see: a
target that is only implied by its formula never reaches the explicit item list,
so a fixture without the explicit item passes whether the rule is implemented or
not.

---

## `improvise-gp/src/fit.rs`

Whole-module fixture note — `document()`. **Every category is a `Label`, and
that is the only thing that makes the success path deterministic.** An earlier
version left `Region` regular with a single item and claimed the constant
one-hot had a degenerate `(1, 1)` range that `simplify` would prune. **That
claim is false**: `data.rs::build_categorical` pushes `(0.0, 1.0)` for every
one-hot whatever the column contains, so `simplify` cannot know it is constant
and the champion was free to split on `Region=North`. These tests were passing
on seed luck, and the comment asserting otherwise was the worst part (Copilot on
PR #25). The targets are in the **thousands** so a fit that forgets to
de-normalize is visibly wrong rather than subtly so. The view puts `_Measure` on
`Axis::Page` because that is what `Workbook::new` does — fitting under the real
default is not an extra case, it is the only honest one.

### the_fitted_formula_predicts_in_the_documents_own_units

The de-normalization regression. `fit` runs against a standardized target, so a
champion written down untouched predicts standard deviations: a well-formed
formula, parsing cleanly, evaluating to ±5 where the user's data is 1000–4000.
Nothing downstream can see that — it is a number in a cell.

**Fault injected:** dropping the `denormalize` call in `fit_measure` fails this
and only this test. The loop `panic!`s rather than `continue`s on a non-numeric
cell, because a loop that skips every record asserts nothing and "no cell
evaluated" is itself the improvise-8xhy failure. The claim is then read a second
way off the formula **text**, which does not depend on evaluation at all.

### a_category_and_a_measure_of_the_same_name_are_refused

Copilot on PR #25, and the sharpest finding of the round: asking the *document*
whether it can resolve a feature name is not the same question as asking whether
that name means the column the tree was fitted on. A numeric category called
`Cost` encodes to a feature called `Cost`; with a `_Measure` item of the same
name the emitted `Expr::Ref("Cost")` resolves cleanly — **to the measure** — and
the stored rule silently reads a different column from the one it learned.

**The first version of this test never built the collision.** Adding a category
`Cost` with items puts no `Cost` coordinate on the existing records, so
`feature_table` emits an empty column, `preprocess` drops it, and there was no
duplicate name to find; it passed on the *positional* role bug instead. The
records here carry the coordinate, and the test asserts the fixture really
produces two columns named `Cost` before asserting the refusal.

### an_empty_category_before_a_measure_does_not_break_the_role_check

Copilot on PR #25. `preprocess` `continue`s past an entirely-missing column
without pushing a `ColumnSpec`, so `Dataset::columns` is not index-aligned with
the header once anything is dropped. A positional `i >= category_columns` role
test then read the surviving numeric measure as a category and refused a
perfectly good fit. The test asserts the fixture really drops a column first.

### a_coordinate_is_spelled_as_one_even_when_an_item_shares_the_one_hots_name

The collision a *name* cannot resolve, now closed by construction rather than by
a refusal. The fixture adds a `_Measure` item literally named `Region=North` and
asserts `find_item_category` resolves it — so a check that matched the one-hot's
feature name against the document would have found the wrong column.

Under improvise-4zl7 the one-hot is never spelled by its feature name at all:
`Features::of` reads the **role** off `preprocess`'s plan and emits
`Region IS North`, which cannot collide with any measure however the document is
named. So the assertion is the *absence* of `|Region=North|` in the emitted text
as much as the presence of `Region IS`. Injecting the fault means having
`spell` fall through to `Spelling::Measure` for a `OneHot` role: the fit then
succeeds, the formula reads `|Region=North|`, and it silently evaluates against
the wrong column — which is the original bug wearing the new API.

### a_high_cardinality_numeric_measure_is_one_feature_not_many

The preflight approximates `preprocess`'s type inference, so it can disagree in
both directions. A numeric measure with more distinct values than `MAX_FEATURES`
encodes to **one** feature and must not be refused for having many values.

### a_category_with_a_distinct_value_per_record_is_refused_not_searched

`FitConfig` bounds the search; nothing bounded the width. A regular category
with one distinct value per record one-hots to one column per record, and
permutation importance is `repeats × rows × features` evaluations on the draw
thread.

**The fixture had to be pushed past `MAX_FEATURES`**, and the assertion names
the *width* refusal specifically. At twelve records this document is refused
too — but by `fitted_formula` noticing the champion split on `Tag=r0`, a
different rule, which would have made the test pass with the guard removed
entirely.

### more_records_than_the_budget_is_refused_before_any_search

The width bound does not bound the work: every stage scales with rows. The
assertion also requires the message to say *what to do* (`Page or Filter`).

### a_document_with_no_usable_feature_is_refused_rather_than_panicking

`preprocess` drops an entirely-missing column, so a document whose only
candidate feature is empty encodes to **zero** features — and `random_tree` then
draws `rng.below(0)`, which `SplitMix64` answers with `0`, and `sample_thresh`
indexes `spec.ranges[0]`. A panic, on a legal document, for any seed that does
not happen to open on a leaf.

This needs the all-`Label` fixture rather than merely an empty measure: with a
regular category present its one-hot survives — range `(0.0, 1.0)` whatever it
holds — and the fit returns a constant leaf instead of crashing.

### a_non_finite_cell_is_refused_rather_than_panicking

A `.improv` cell holding `inf` parses as an `f64`, so this is reachable from a
document rather than only from a bug. `standardize` turns an infinite target
into NaN leaves, an `inf..inf` range makes `sample_thresh` produce a NaN
threshold, and `number_text` then **asserts** — a panic out of a function whose
signature promised a `Result`.

**Verified by disabling the guard, and the two directions differ**: an infinite
*target* really does panic in `number_text` ("wants a finite value, got inf"),
while an infinite *feature* poisons its range and quietly yields a constant
leaf. A panic and a meaningless rule; both are refused.

`NaN` is deliberately absent from the cases: `f64::NAN.to_string()` is `"NaN"`,
which `is_missing` already spells as *missing*, so it is imputed away and never
reaches the search.

### a_name_containing_a_newline_cannot_split_the_status_line

`.improv` pipe quoting makes a newline legal inside a name, so a document can
hold a measure whose name would split the TUI status bar in two.

### a_fit_that_needs_a_category_now_spells_it_as_a_coordinate

improvise-8xhy reaching a real user, and what improvise-4zl7 bought: the same
fixture, the same champion, and a formula instead of a refusal. The fixture
clears **both** measures on every North record: clearing only `Cost` would leave
North carrying both target values, so the one-hot would explain nothing and the
champion would have no reason to split on it.

The second assertion is the one that earns its place. Checking only that the
text says `Region IS` would pass for a rule that routed both groups to the same
leaf — which is what a tree that could not see the coordinate would produce. The
fixture's two groups are 1000 and 4000, so requiring one leaf under 2000 and one
over 3000 separates "spelled it" from "spelled it and meant it".

The `\n`-and-double-space assertions moved with the refusal they guarded; the
one-line rule itself is unchanged and is still asserted by
`a_name_containing_a_newline_cannot_split_the_status_line`.

### the_fold_count_moves_the_champion_and_so_is_recorded

Copilot on PR #25, and it is right: the outcome used to carry `seed` plus
`config`, which is the pair gotcha 37b names — and that pair is **not sufficient
here**. Cross-validation draws from the same generator the fit then draws from,
so changing `cv_folds` alone moves the champion. Recording two of four fields
documented the reproducibility contract without meeting it. The test asserts the
seed and config are *equal* across the two requests first, so the difference can
only be the fold count.

### the_same_seed_under_a_different_config_is_a_different_search

The knob varied is `pop`, not `generations`. More generations over an
exactly-learnable fixture can land on the same optimum and `simplify` normalises
it, so that version of the test would fail one day for a reason that is not a
regression. `pop` changes how many RNG draws each generation makes, so the two
searches diverge from the first one.

### a_fit_named_after_a_category_item_is_refused

Copilot on PR #25. The check used to ask only about *measures*, and `Expr::Ref`
does not: `find_item_category` finds a regular category's item **before**
falling back into `_Measure` (gotcha 7). So naming the fit after some category's
item installed cleanly and then made every later formula referring to it read
the category coordinate. `a_later_formula_referring_to_the_fit_reads_the_fit` is
the property that refusal protects, asserted against a real document.

### a_fitted_rule_at_an_aggregated_cell_routes_per_record_and_sums

The characterization PR #25 left here, now replaced by the rule it was waiting
for. A tree is fitted on individual records; an improvise formula is evaluated
at a `CellKey`, and `evaluate_aggregated` used to aggregate each `Expr::Ref`
*first* and evaluate the `IF` once on the totals. Linear formulas are unaffected
— `SUM(Revenue) - SUM(Cost)` is `SUM(Revenue - Cost)` — and a tree is not, so
the stored rule meant one thing on a record cell and a different thing one row
up. improvise-4zl7's ruling was to aggregate the rule over the members.

The old test asserted the **negative** and carried a message telling whoever
broke it to replace it with the rule. It fired on the first run of the fold.

What this one asserts is the property the ticket names: the aggregated cell
equals the sum of the per-record predictions. That compares two *different*
code paths — a fully-specified key evaluates one member, an aggregating key
folds twelve — rather than one path against itself, but it is still an
end-to-end agreement rather than a hand-computed number. **The hand-computed
evidence is in `improvise-core`**, under
`route_then_aggregate_tests`, on a four-record fixture whose every expected
value is written out in the test name or the assertion.

The third assertion is what stops this one being satisfiable by accident:
routing once on the total lands on a single leaf, so the test refuses a fixture
in which a leaf happens to equal the right answer.

### a_missing_flag_a_date_part_and_a_measures_value_are_refused_differently

improvise-wd6n. `Spelling::Artefact` was one variant standing for **five**
distinct families — `Missing`, `DatePart`, a one-hot over a measure column, the
`(missing)` pseudo-item, and a `Numeric` role over a category column — the last
five reached through a `_ =>` wildcard in `spell`. The single refusal they all
fed said the tree "splits on a `col.missing` flag or a date part", so three of
the five were declined in words naming something they are not. A user fitting
over a text-valued measure column was told about missing-value flags.

The test drives `Features::of` over a real `preprocess` output rather than
fabricating `Spelling`s — which is the point, since what is under test is
`spell`'s classification, and the helper `artefacts()` beside it fabricates
exactly what `spell` decides. That helper now takes the family as an argument
so a fabricated spelling at least names one on purpose; `Features::of` is still
what really decides, and this test is the one that asks it.

Each family's assertion has two clauses, and the second is what earns it: the
error must contain that family's reason **and not the other two's**. The first
clause alone passes against one message naming every family at once, which is
precisely the state the ticket describes. Fault injected: the five arms of
`Artefact::reason` collapsed to one shared string — the test fails on the first
family, and with the injection removed all three arms pass.

The fixture reaches three families from one `preprocess` call (a text measure
column, a measure with a hole in it, a column of ISO dates), which is why the
module is named for what it reaches rather than for "each family". The
remaining two are not reachable from it and are not asserted here.
`AbsentCoordinate` is a permanent refusal; `NumericCategory` was **not**, and
the first draft of this entry and of `gp.md` §11 both called it one — review
found that `fit_measure` simply never tells `preprocess` which columns are
categories. improvise-zv8t has since closed it (below). Worth recording as the
failure mode: the branch that *invented* a classification also wrote the prose
declaring one of its arms unfixable, and nothing but a reader could have caught
that — the suite was green on either reading.

### fit::tests::a_category_whose_items_are_numbers_is_still_spelled_as_a_coordinate
### fit::tests::a_category_whose_items_are_dates_is_still_spelled_as_a_coordinate

improvise-zv8t. `fit_measure` handed `preprocess` an `Overrides::default()`, so
it never said which header columns are document categories — although the very
next line told `Features::of`. `preprocess` infers `Numeric` → `Date` →
`Categorical` from the values, so a **`Year` category with items `2024`/`2025`**
became a numeric column and a **category of ISO dates** became a date column,
and both were refused as encoding artefacts with `Year IS 2024` spellable all
along. User-visible: a fit over such a document could not be stored at all.

No fault had to be injected, because the bug was live. On the unfixed tree both
tests fail inside `fit_measure` with the real refusals, which are the two
arms the ticket named and are worth quoting:

- `["Year"] names a category whose items all parse as numbers, so a reference
  to it would resolve to a `_Measure` item of the same name`
- `["Year.epoch_day", "Year.year"] is a part of a date, and date arithmetic is
  something improvise does not have anywhere`

The second is why `gp.md` §11's date decline was narrower than it read.

**Why each fixture is discriminating.** `year_document` gives the model **one**
regular category and **no other feature column**, and makes the target a
function of that category alone — so a tree that does not split on `Year` is a
poor fit and the GA does not settle for one. That matters because the refusal is
conditioned on `used_features`: a fit whose champion happened to be a bare leaf
would be *accepted* on the unfixed tree and prove nothing. Asserting
`formula.raw.contains("Year IS")` rather than merely `is_ok()` closes that hole
in both directions. Note what the unfixed tree actually does, because a first
draft of this line got it wrong: it produces **no text at all** —
`fitted_formula` refuses before emitting, which is what the messages quoted
above are. The only unfixed path that emits anything is the bare-leaf champion
named in the previous paragraph, and its text is `SUM(<constant>)`, carrying no
`Year` in any spelling. So the assertion closes the hole; "it would have said
`Expr::Ref(Year)`" was a guess about a branch that does not run.

The numeric test then adds the property the refusal exists to protect: the
formula goes in through `add_formula` and
every record evaluates to a `Number`, not `Error("ref:…")` (improvise-8xhy).

### fit::tests::the_preflight_counts_a_numeric_category_at_its_cardinality

The other half of improvise-zv8t, and the one that is a *cost* rather than a
fix. Forcing a category categorical widens it from one numeric feature to one
per distinct value, so `encoded_width` — the cheap preflight that exists to
refuse a too-wide table **before** `preprocess` materialises it (§14) — had to
learn the same `header[..category_columns]` slice, or it would under-count
exactly the column this ticket is about.

Fault injected: the `col >= category_columns` clause dropped from `encoded_width`,
leaving the pre-ticket inference. The test fails by name, and the failure message
is the evidence — it shows the **exact** check speaking where the preflight
should:

> the PREFLIGHT must refuse it, before anything is materialised: fitting "Score"
> encoded 70 features, over the 64 this host will search.

A test asserting only `expect_err` would have passed under that injection, since
both guards refuse; the discriminator is *which message*, because the two guards
differ in what they have already paid for. The fixture also asserts its own
precondition — that `Year` really is the header's first column — so a
`feature_table` change that reorders the header fails here rather than silently
slicing the wrong names.

### fit::tests::a_wide_numeric_category_now_refuses_a_fit_that_used_to_ignore_it

The **cost** of improvise-zv8t, pinned rather than argued — and it exists
because the first draft of `gp.md` §11 claimed there was none. That draft said
"no document that fitted before stops fitting", reasoning that a tree splitting
on a numeric category was refused as an artefact anyway. The refusal is
conditioned on `used_features`, so a fit that never split on the column
succeeded, and the wide columns are exactly the ones a fit has least reason to
split on.

Measured both ways with a throwaway `#[ignore]` probe before this test was
written, on a fixture of 100 records with a 100-item numeric `Year` beside a
`Cost` that predicts `Score` on its own:

- pre-fix: `FIT LANDS -> |Score (fit)| = SUM(IF(Cost < 48.0150032043457, 970.3…, 3684.7…))`
  — `Year` never appears.
- post-fix: `REFUSED -> fitting "Score" would encode 101 features from 100
  records`.

So no fault needed injecting; the *change itself* is what the test observes, and
it fails on either side if the behaviour moves. The second half is what keeps it
from being a bare characterization: it marks `Year` a `Label` and asserts the
fit then lands on `Cost`, which is the repair the refusal's own message names.
A test asserting only the refusal would leave that advice unchecked — and advice
in an error message is a promise, so it is the half worth asserting.

The generalisable part is the review finding, not the test: **prose claiming a
change has no cost is a claim, and it is checkable.** The suite was green with
the false sentence in place, exactly as it was green with improvise-wd6n's
"permanent refusal" — the same section, the same failure mode, one paragraph
apart. See improvise-e8no for the case the trade genuinely loses.

### fit::tests::a_duplicate_column_name_is_refused_before_the_matrix_is_built

Found by adversarial review of improvise-zv8t, and **introduced by it**:
`as_categories` forces by column **name**, while `encoded_width` counts by
column **index**. A document may legally name a regular category and a raw
measure the same thing -- `a_category_and_a_measure_of_the_same_name_are_refused`
already ships that fixture -- and then the *measure* column is forced
categorical by the name match while the preflight still counts it numeric by
index.

Measured on the fixture here (40 records, a 40-item `Dup` category beside a
`Dup` measure): the preflight answers **41** where forcing materialises **80**.
The refusal that catches the document, `no_ambiguous_column_names`, ran *after*
`preprocess`, so the expansion the preflight exists to prevent was allocated and
only then refused. At the default `max_records = 20_000` that is a 20k-row
one-hot matrix built before the error -- `doctrine(guard_the_cost_not_the_result)`
defeated by an inconsistency the ticket itself introduced. The bounded drift
that `doctrine(a_budget_check_is_advisory_or_exact_and_must_say_which)` licenses
is *approximation*, not this.

**The repair is a data dependency, not a moved line.** `as_categories` now runs
the check itself and returns `Result<Overrides>`, so the `Overrides` that
`preprocess` requires cannot exist until the header has been accepted. That is
`doctrine(an_invariant_is_made_unrepresentable_not_validated)` -- the answer
`prune_axis_items`, `Row<T>` and `Checked` each got.

**So the regression guard is the signature, and the injection is a compile
error.** Swapping the two statements back gives
`error[E0425]: cannot find value 'overrides' in this scope`, verified. What that
does *not* cover, stated rather than implied: the check could still be deleted
from `as_categories` deliberately. The accidental reordering -- the way this
actually happened -- is what is now unrepresentable.

The test therefore asserts the **hazard** rather than the ordering: that the
fixture really names one column twice, that the preflight's 41 really disagrees
with what forcing would build, and that `as_categories` itself refuses. A test
asserting only the error message would pass on either ordering, since the
message is identical -- the whole defect was in work done, not in the result.

### the_preflight_counts_the_missing_category_a_forced_column_materialises

improvise-x1ns, found by ChatGPT Codex reviewing PR #90 and live on `main` at
`ce895ff`. `encoded_width` computed `has_missing` and used it in the **numeric**
arm only:

```
true  => 1 + usize::from(has_missing),
false => present.iter().collect::<HashSet<_>>().len().max(1),
```

`build_categorical` maps every missing cell to `MISSING_CAT` and includes it in
`distinct(&mapped)`, so a categorical column with N distinct *present* values and
at least one absent cell materialises **N+1** one-hots. The preflight promised N.

It under-reports by exactly one per such column, and it under-reports in the
direction that costs: the preflight runs *before* `preprocess`, the exact check
on `feature_names.len()` runs *after*, so the exact check bounds the error
message and not the allocation — which is the whole reason the preflight exists.
The previous wave's review recorded this shape as "read, not injected" and
"bounded by the exact check"; the second half is false and is now corrected in
`context/code/gp.md` §14.

**Fails before the fix** with

```
the preflight under-reported: it promised 5 features and `preprocess`
materialised 6 — ["Year=(missing)", "Year=2020", "Year=2021", "Year=2022",
"Year=2023", "Cost"]
```

The fixture is Codex's suggested case, and the reason it is the interesting one:
a **forced numeric category with a missing coordinate**. Forcing is what sends a
numeric column down the categorical branch, so only a *forced* column reaches the
arm that was wrong, and only a missing cell makes the arms disagree.

The test does not assert a number. It asks `preprocess` what it actually built
and requires `encoded_width >= data.feature_names.len()`, so it cannot pass by
agreeing with a stale constant, and it keeps holding if the encoding widens for
some other reason. Three preconditions are asserted first, because each would
make it vacuous: that exactly one record lacks the coordinate, that the present
cells really are numeric (otherwise the column would be categorical by inference
and forcing would prove nothing), and that a `Year=(missing)` one-hot is really
among the materialised features.

The `.max(1)` is deliberately outside the addition. A column whose cells are
*entirely* missing has no present values, and `preprocess` skips it with a
warning rather than encoding it; keeping the clamp on the sum leaves that case
counted as 1 exactly as before, so the fix moves only the case it names.

### a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored

The evidence improvise-e8no is declined on, and the only coverage
`Artefact::NumericCategory` has ever had. It was **one of three** families with
no test, not the only one: `MissingFlag` and `DatePart` are covered by
`the_missing_flag_and_the_date_sub_features_are_refused_too`, and
`ValueOfAMeasure` and `AbsentCoordinate` still have none — filed, not fixed
here.

improvise-e8no observes that forcing a document category to `Forced::Categorical`
(improvise-zv8t) widens the encoding, so a high-cardinality numeric category now
exceeds `MAX_FEATURES` where inference used to read it as one numeric column and
the fit succeeded. The decline turns on what that success was worth: a tree that
splits on a numeric column over a *category* column produces a formula
`fitted_formula` **refuses**, because a bare reference to the category's name
would resolve to a `_Measure` item of the same name (improvise-8xhy). So the
pre-zv8t success was either a fit that never touched the column — for which
marking it a `Label` gives a fit over the same remaining features, within budget
— or one that could not be stored. Note what this does *not* establish: an
ordinal encoding is spellable as a weighted indicator sum, so it is declined on
semantics and formula size rather than on spellability (`context/code/gp.md`).

The test asks `preprocess` with `Forced::Numeric`, which is the pre-zv8t reading
reproduced deliberately rather than described, and then asks `Features::of` and
`fitted_formula`. It does **not** use the `artefacts()` helper: that helper
fabricates a `Spelling`, which is the shape gotcha 37(d) warns about — it would
assert that the emitter refuses what the test told it to refuse. Here the
document and the encoder decide.

Injected fault: change `spell()`'s `FeatureRole::Numeric { .. }` arm from
`Spelling::Artefact(Artefact::NumericCategory)` to `Spelling::Measure(column.clone())`
— the "fix" someone would reach for on reading improvise-e8no, and the one that
silently reopens improvise-8xhy. Fails with

```
assertion `left == right` failed: a numeric column over a category column is unspellable
  left: Measure("Year")
 right: Artefact(NumericCategory)
```

### an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum

Not a bug fix: this pins a **grammar fact a prose argument rests on**, after that
argument got it wrong. improvise-e8no's decline was first written claiming that
one-hot is the only encoding of a category whose splits the formula language can
spell, and therefore that ordinal encoding was ruled out by spellability. An
adversarial review disproved it by running `parse_formula` instead of reading the
rule that looked relevant: `is_expr` is a `primary`, so it is an `Expr` yielding
1/0, and `cmp_op`'s operands are full `expr`s — so
`IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, 1, 2)` parses.

The decline survives on other grounds (a category's items carry insertion order,
not magnitude; and the emitted formula still spells one `IS` per item, so the
width is not saved) — but the *stated reason* was false, and a decline is the one
artefact nothing checks. This test is the check: if the grammar ever narrows so
that an indicator sum stops parsing, the paragraph that says it does becomes
discoverable rather than quietly wrong.

The negative arm is the incidental fact that explains the emitter: `Year IS 2020`
does **not** parse, because `filter_value` is `string | pipe_quoted | bare_ident`
and a bare number is none of them. That is why `formula_text` pipe-quotes a
numeric item name through `quote_name`, and asserting it here keeps the positive
arm from passing for the trivial reason that everything parses.

### the_preflight_counts_the_parts_a_date_measure_expands_into

improvise-x1ns, **second instance** — found by ChatGPT Codex on PR #97, live at
`f128087`, i.e. in the very change that fixed the first one.

`preprocess` infers `Numeric → Date → Categorical` (`data.rs:406`), and
`encoded_width` had **no date arm**: numeric-or-categorical only. A date string
does not parse as `f64`, so a measure column holding dates took the categorical
arm and was counted by its distinct strings. `build_date` emits
`epoch_day/year/month/day/dow` — seven parts once any cell has a time
(`data.rs:326`) — so a column with **one** distinct timestamp promised 1 and
materialised 5.

Fails before the fix, on the plain-date case:

```
a plain date: the preflight under-reported: it promised 4 features and
`preprocess` materialised 8 — ["Region=East", "Region=North", "Region=South",
"When.epoch_day", "When.year", "When.month", "When.day", "When.dow"]
```

**The fixture carries a time case because of the rival fix, not because of the
bug.** Two implementations repair the plain-date failure: "a date column is 5
features", and "5, or 7 when any present cell carries a time". They differ only
on a column containing a time, so the second sub-case is what separates them.
Injected, `date_parts` hard-coded to 5:

```
a date with a time: the preflight under-reported: it promised 8 features and
`preprocess` materialised 10 — [… "When.hour", "When.minute"]
```

Note the plain-date sub-case **passes** under that injection — so without the
time fixture this test would have been satisfied by either reading, and would
have pinned the bug without pinning the rule.

Both arms compare against `data.feature_names.len()` rather than against a width
computed a second way, so the test cannot pass by agreeing with a stale constant
or by reimplementing the thing under test. Two preconditions guard vacuity:
every `When` cell really is the same string (so the categorical count really
would be 1), and the fixture's timestamp really does or does not carry a time,
asked of `parse_datetime` rather than of the literal's shape.

The column must be a **measure** — `as_categories` forces every document
category to `Forced::Categorical`, so a category of dates never reaches the date
branch at all. That is gotcha 37(d)'s "the date decline is narrower than it
read": this is the genuine date *measure*.