Keyed by test name per design-principles section 7, recording for each one the
bug it reproduced, the fault injected to watch it fail, and -- where a test has
a neighbour that looks like it -- the input that separates the two.
Three entries are about that separation rather than about a bug:
- a_view_that_does_not_prune_writes_no_prune_line is the direction the default
supplies, so a writer emitting "prune: true" unconditionally passes the real
test and fails this one.
- an_interleaved_category_keeps_its_item_order explains why the ticket's own
two-item reproduction cannot discriminate: with one item of each kind, both
the right and the wrong order are a single bare run followed by a single
grouped one.
- slash_in_hidden_name_survives_roundtrip records that improvise-nvq4 does NOT
reproduce, and is kept as a passing control so the claim cannot be re-filed
without someone first watching it pass.
core.md is touched for the improvise-qx1z entries, which are View-level and
belong there rather than in formula-and-io.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
A sweep for the SHAPE of improvise-vyrx rather than the site it named. Both
eval_bool and eval_bool_cached also wildcarded the outer Expr match as
`_ => Err("type")`, so a new Expr variant that ought to be usable as a
condition would have become a silent type error in both. Not vyrx's asymmetry
-- the two agreed -- but vyrx's defect: a new variant answered wrongly with
nothing to notice. Adding an Expr::And now fails the build at four sites where
it previously failed at two; verified by adding the variant.
The sweep's other half is which wildcards to LEAVE, recorded in the evidence:
eval_agg's `match inner { Expr::Ref(..) => .., _ => false }` falls through to
the GENERAL path (the per-record fold), which is right for any expression that
is not a bare ref, including one that does not exist yet. The dangerous
wildcard is the one falling through to a refusal or to a specialisation.
CellValue::as_f64 and CategoryKind::of_name clear on the same reading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-s08t: eval_agg narrowed its partial key only where
find_item_category resolved, so SUM(Typo) left the key unconstrained and
aggregated every numeric measure matching the context. An unknown name now
errors ref:{name} before the raw fast path.
improvise-s2w3: the same fast path assumed a bare Ref names a STORED cell. A
formula target has no raw cell behind it (improvise-nrs), so SUM(Double)
summed nothing and answered 0 where SUM(Double + 0) folded and answered 20.
core.md §13's defence of the specialisation holds — what failed is its
premise — so eval_agg asks formula_for, the one home of "is this cell
derived", and a derived ref falls into the fold.
improvise-7iww: recompute_formulas stopped at MAX_EVAL_DEPTH and left the last
iteration in formula_cache, so a circular formula read Error("circular")
directly and Number(26) through the cached path — which is the DISPLAYED path,
since default views put the virtual categories on Axis::None. Cells still
moving when the bound runs out are now marked circular, which is the same
bound evaluate_depth already enforces.
improvise-vyrx: eval_bool_cached wildcarded BinOp where eval_bool lists its
arms. Verified by adding a BinOp::Mod: four sites now fail the build where
three did before.
improvise-pikk: round_half_away scaled by 10^decimals before truncating, so
format_f64(1e307, false, 2) rendered "inf" for a finite number.
improvise-f86q: add_category checked MAX_CATEGORIES before contains_key, so
re-adding an existing name at the limit was refused with a false message — and
per gotcha 21 aborted the rest of the batch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-yae4. grid_body narrowed every measured width to u16 and only then
added the one-character gap and clamped to MAX_COL_WIDTH, so the arithmetic ran
at full width before the cap that makes it safe. A stored 65535-column text
value panicked at `w + 1` in debug; in release the sum wrapped to 0 and the
column was drawn at MIN_COL_WIDTH — the narrowest column, for the widest value.
A 65536-column value never reached the addition: `as u16` truncated it to 0
first, so it wrapped in debug builds too, silently.
Widths are now capped while still usize, in the one place every measurement
enters (`widen`), which is why header_level_seed folds through it rather than
keeping a second `as u16`. The answer is unchanged for every input that did not
overflow: (min(w,32) + 1).clamp(5,32) and (w + 1).clamp(5,32) agree everywhere,
both saturating at 32 for any w >= 31.
improvise_protocol::header_level_widths carried the identical defect on the
row-header axis — a 65535-column item label — and is fixed the same way. It is
the one home for that rule (gotcha 44), so the row header and the column widths
cannot end up with two answers.
wide_cell_clamps_without_overflow is un-ignored and extended to the 65536 arm
and to an ordinary 20-column value. It asserts `widths == vec![expect]` rather
than `all(<= 32)`: the length pins that the fixture really produces ONE column,
and the ordinary arm pins that the single column is the one carrying the value —
a fixture in which no column was wide would read MIN_COL_WIDTH there and pass a
`<= 32` check. Both injected faults are recorded in
context/test-evidence/core.md.
Not fixed, and out of this ticket's scope: compute_visible_cols accumulates
`acc + w` in u16, reachable only from a terminal ~65535 columns wide.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`parse_number_format` read the precision as `fmt[i+1..].parse::<u8>()` over
everything after the last `.`, so `,.2f` — the printf spelling the repo-map's
own `.improv` example carries — failed to parse and fell through to zero
decimals, silently, in the TUI, the browser projection and CSV export.
The precision is now the leading digit run after the last `.`, with any
conversion letter ignored. The spellings that are legal are stated once, in
`context/code/core.md`, along with why the trailing `f` is accepted rather than
refused (the grammar takes the format line as an opaque `rest_of_line`, so
refusing it rejects documents this repo already writes) and why the reading is
printf's rather than Excel's (`.00` is the spelling the two disagree on).
`export_csv_quotes_commas_per_rfc4180` asserted the value `starts_with("1,200")`
against a `,.2f` fixture, so it could not tell two decimals from none. It is now
an exact equality, and reverting the fix fails it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The parameter was none_cats: &[String] and the body consulted it exactly once,
for emptiness -- a boolean wearing a list's type, since the aggregation runs
over key.0 through matching_values. git log -S back to the commit that
introduced Axis::None shows it never examined an element, so the name promised a
filter the function has never performed.
Aggregation::{Exact, OverHidden} admits only the two states the body
distinguishes, and Aggregation::of reduces the list once at the boundary that
holds it. The regression guard is the signature: two lists with the same
emptiness and different contents is no longer a constructible call.
recompute_formulas and aggregate_raw keep &[String] -- they walk the elements.
improvise-5jm5
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`View::on_category_added`'s early return read
cat_name.starts_with('_') || self.is_records_layout()
— the same expression mixing both derivations one operator apart, the left a
name prefix and the right `CategoryKind`. A reader who checks this line sees
the enum being used and has no reason to look at the operand beside it. The
second site, `holds_a_regular`, had the same split.
`Model::add_category` does not validate names, so a user category literally
named `_Foo` was `CategoryKind::Regular` to `records_columns`, `cell_id` and
`is_empty_model`, and "virtual" to these two lines — seated on `Axis::None`,
never appearing in a pivot, while every other consumer treated it as ordinary.
Decided explicitly, as the ticket asks: `add_category` is NOT made to refuse a
leading underscore. The three reserved names are the ones
`CategoryKind::of_name` knows, `_Foo` is a legal document, and a user-visible
restriction is a bigger change than making one answer out of two.
Both sites now ask `CategoryKind::of_name(..).is_virtual()`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Review of #54 found the fold reintroducing exactly the divergence the ruling
exists to remove, and the same line turned out to be the 244 ms stall. One
defect, two symptoms.
`eval_expr_cached`'s `Expr::Ref` looks the value up in `formula_cache` at
`context.with(cat, name)`. Inside the fold `context` is a MEMBER-RECORD key
carrying the none_cat coordinates, while `formula_cache` is keyed by stems with
none_cats STRIPPED — so the lookup could never hit, fell through to
`aggregate_raw`, found no raw cell behind a formula target, and answered
Err("ref:…"). A routing formula over a derived measure gave Number(11.0)
uncached and Error("ref:Margin") cached, and the cached path is the one a pivot
grid runs whenever any category sits on Axis::None.
`eval_expr`/`eval_bool` are hoisted to module level and BOTH evaluators now hand
the fold the same record-level `eval_expr`. The two agree by construction rather
than by a test. Note the repair not taken: teaching the fold to hit the cache by
stripping none_cats off the member key returns the GROUP's value to every
member — aggregate-then-route through a cache lookup, and silent.
That same per-member `aggregate_raw` was one poorly-narrowed scan per member, an
O(n^2) shape. Measured with the new `examples/fold_bench.rs`, five runs a point,
at 18,050 cells: 51.1-53.3 ms before, 18.1-19.2 ms after, against 7.1-9.0 ms for
a linear formula. Scaling went from superlinear (45x cells, 85x time) to linear
(45x cells, 45x time). Debug 2.75 s -> 132.9-135.1 ms. The fixed point runs
exactly 2 passes, measured by counting, so hoisting the member set has a 2x
ceiling and is declined on improvise-9iyv — a memo keyed by less than the whole
`partial` is a second answer to what a cell depends on.
`Expr::Is` has an arm in each evaluator and the cached one was untested: the
earlier fault was injected into the shared `eval_is`, which both arms call.
Replacing the cached arm alone left the suite green. The first replacement test
ALSO stayed green, which is the interesting part — once the fold routes through
`eval_expr`, an `Is` inside a SUM never reaches the cached arm, so that arm is
reachable only by a bare `Is` outside an aggregate. That is what the new test
uses, and injecting into that arm alone now fails it.
`IS` gained a word boundary: `Region ISLAND` parsed as Is(Region, "LAND"),
silently wrong where it was a parse error. `is_expr` is compound-atomic with
`sp+` written out, because a negative lookahead in an ordinary rule tests the
character AFTER pest's auto-inserted whitespace and fails backwards. The
lookahead was then redundant and removed — deleting it left the suite green,
while relaxing `sp+` to `sp*` fails the test, so `sp+` is the mechanism. `WHERE`
has the same hole, is far less exposed, and is filed as improvise-r08u.
Also: `ambiguous.sort_unstable()` before `dedup`, which only removes consecutive
duplicates, so the refusal could name a column twice.
Every fault re-injected after the refactor: no-fold 10 failures, shared eval_is
5, record-level Is arm 4, cached Is arm 1, feature_roles order 2, context-vs-
partial 1, the new cached-fold divergence 1, `sp+` relaxed 1. Baseline 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Self-review of the commit before it. The record key the fold evaluates at was
built by folding the record's coordinates onto `context`, which still carries
the formula's own target coordinate. Harmless for a `_Measure` target, because
`Expr::Ref` overwrites `_Measure` on the way in — and not harmless for a formula
targeting any other category: the record key keeps `Levy=Tax`, `evaluate_depth`
matches the same formula again at that key, and the cell converges to
`Error("circular")`. A formula silently self-referencing through its own
aggregate.
`partial` is `context.without(target_category)` plus any inline WHERE, so
folding onto it drops the target coordinate by construction. Every member
satisfies `partial` by definition — that is what selected them — so nothing
constraining is lost.
`a_formula_targeting_another_category_still_folds_over_its_records` is the
guard; restoring the `context` fold fails it with `Error("circular")` against
`11.0` and leaves the other eight passing. Evidence in
`context/test-evidence/core.md`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
improvise-4zl7. A tree is fitted on individual records; an improvise formula is
evaluated at a CellKey, and an aggregating cell used to aggregate each Expr::Ref
first and evaluate the IF once on the totals. That is the same answer for a
linear formula and a different one for anything that routes, so every fitted
tree meant one thing on a record row and something else one row up. The user
ruled: aggregate the rule over the members.
The rule needed no new aggregate vocabulary. SUM(e) already means "fold e over
the members" and simply did not: eval_agg specialised Expr::Ref and, for every
other inner expression, summed the raw matching cells and ignored the expression
entirely. It now folds — one entry per member record, the expression evaluated
at that record's own coordinates. eval_agg is shared by eval_expr and
eval_expr_cached, so the fold has exactly one home; each evaluator passes its
own recursion in.
Expr::Is(Filter) is the new arm in both evaluators: `Cat IS Item` is 1.0 when
the cell's own Cat coordinate is Item, keeping the evaluator f64-only and
reusing Formula::filter's own context.get/unwrap_or(false). A grammar production
in primary, before ref_expr, with both operands through
improvise_formula::quote's one home; IS joins the keyword guard.
The emitter reads feature index -> role off preprocess's plan rather than
reconstructing it: gp_tree::data::feature_roles is the one statement of the
expansion order and encode_row now drives off it too. improvise_gp::Features
carries one Spelling per feature — Measure, Coordinate or Artefact — decided by
role and column, never by the name's shape. A one-hot over a real category is
emitted as `Region IS North` and accepted; col.missing and the date parts stay
Artefact and stay refused (improvise-wd6n).
fit-measure on a document a category explains now lands instead of refusing, and
the refusal no longer depends on which features the search happened to pick.
Evidence: eight hand-computed tests in improvise-core over a four-record fixture
whose Cost straddles the threshold inside each region, so route-then-aggregate
and aggregate-then-route differ at every level. Two faults injected and watched
to fail — reverting the fold fails 7 of 8, and a constant-false eval_is fails
exactly the 3 coordinate tests. Reordering feature_roles fails both it and
encode_row_replays_the_training_expansion.
The fold is O(members) per evaluation per fixed-point pass and unbenchmarked:
improvise-9iyv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
improvise-core's comments are the oldest in the repository and carry the most
expensive artefact in it. This lands the destination before the cap, so the
next commits are a move and not a deletion.
context/code/core.md carries the chains that cannot survive two lines: the
ordering-is-correctness argument for DataStore (improvise-shyt/2yc2), the
unanimity rule and its branch order (improvise-l4qx), the surrogate width and
byte-format argument (ADR 002), the wide-records column vocabulary, the
virtual-on-an-axis rule (improvise-2zq), and the two-evaluator hazard behind
improvise-0bf.
context/test-evidence/core.md carries the bug narratives, keyed by exact test
name, per design-principles §7. 52 entries, all verified against
`--check-evidence`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Edward Langley