A collapsed group is written to CSV as its own total row, because an export
writes what the grid draws (improvise-iot6). core.md §14 carries the argument;
the evidence entries record each rival and where it failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
Found by ChatGPT Codex reviewing PR #150 at af7fae0e, and correct. The tally was
incremented while walking a record's categories, before the record was known to
survive that walk. A row whose Date is unparseable and whose LATER required
category is missing is skipped entirely — it never lands in the imported model —
yet it had already been charged to the derived column. So the warning counted a
row nothing else in the import reports, and its value depended on the order the
categories happen to be walked in.
Mismatches are now collected per record and merged only after missing.is_none().
The regression asserts the fixture's category order explicitly, because it only
discriminates while Date precedes the missing category: a later change to
proposal order would otherwise make the test pass by never reaching the bad
date. Before the fix it reports three warnings where one is correct; the rival —
merging before the missing check rather than after — reproduces exactly that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
extract_date_component returns None for a value the detected format cannot
parse, and the caller's `if let Some(..)` simply did not push the derived
coordinate — so the row silently left the derived Date_Month/Date_Year category
with nothing reported. Same shape as improvise-2cw9/jc95/5hfh: a value the code
cannot interpret became an absence rather than a report.
Narrower than the ticket reads, and the reason is a property of one function.
detect_date_format accepts a format only when EVERY distinct non-empty sample
parses, and extract_date_component makes the identical parse call — so through
the shipped CLI and TUI wizard a mismatch cannot occur. The reachable door is
ImportPipeline's `pub proposals`, where a library caller sets date_format
itself. a_column_of_mixed_date_formats_is_not_proposed_as_a_time_category guards
that reachability claim rather than leaving it as prose.
One warning per derived column, in the house wording csv_parser already uses.
The row still lands with the coordinates it does have: this reports, it does not
relocate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-dy3j (P1). CategoryKind::Label had no spelling in `.improv` at
all: format_md wrote every category as a bare `## Category: Name` and
build_category read every section back through add_category. The kind was
dropped on the way out AND on the way in, so a round-trip equality test
could not see it.
A label is outside MAX_CATEGORIES = 12 by design and the import wizard
scores it as zero, so twelve categories plus a label is legally
constructible. On load every label came back Regular, the thirteenth was
refused, and parse_md failed: any such document was permanently
unloadable. Below the limit the loss was quieter — a label silently
became a pivot category.
Grammar first. The ticket proposed `## Category: Note [label]`, and
improvise-pczv forbids it: a section header takes its name as the WHOLE
line, so the suffix is indistinguishable from a category named
`Note [label]`. The kind is therefore an entry inside the section,
`kind: label`, spelled with a real token rather than a lookahead — the
grammar-walking generator skips predicates. Absence means `regular`, so
every existing file parses unchanged and a pivot category's section is
byte-identical to before.
Tests assert the parsed Category::kind, never the emitted line. Both
rival faults were injected against the fixed tree and both fail two
tests: a writer that omits the line, and a reader that discards the kind
it parsed (whose file text is correct, so any text-comparing test passes
it). Reasoning in context/code/formula-and-io.md; evidence in
context/test-evidence/formula-and-io.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-e0ex. `Margin = Profit / Revenue` drew as `1`: `View::new` seeded
`,.0` — zero decimals — so a 0.66 ratio rounded half-away-from-zero. The cause
is the format, not the arithmetic; exact decimal math would not change the
rendering by one character, so nothing here touches the evaluator and gotcha 8
is unchanged.
Two halves, per the user's ruling:
* `View::new` seeds `,.2`. Revenue draws `18,400.00` and Margin `0.66`. The
noise on whole numbers is the cheaper error — a currency column with two
redundant zeroes is legible, a ratio with none is wrong — and a format cannot
be guessed from a value.
* A formula may carry its own format, which wins for its target; a raw measure
keeps the view's. `formula.pest` gains
`format_clause = ${ ^"FORMAT" ~ sp+ ~ format_spec }`, with `sp+` for `WHERE`'s
reason (a predicate is invisible to the grammar-walking generator) and
`FORMAT` added to `quote::KEYWORDS`.
`improv.pest` is deliberately unchanged: `formula_raw` already carries the
`## Formulas` line verbatim, so the format rides in the formula text with
nothing to add — and the generator, which reads `improv.pest`, sees no new
spelling to violate.
Resolution has one home. `FormatPlan::of(model, view)` replaces the
`(comma, decimals)` pair every render path threaded — `GridLayout::display_text`,
`viewport::grid_body`/`compute_col_widths`, `project::grid_in` and
`persistence/export.rs::render_csv` — rather than being added beside it, which
would be two answers to "what format is this cell". It is asked of the cell's
`CellKey`, so a cell aggregating over `_Measure` keeps the view's format by
construction. `GridBody::col_formats` carries the answer to the totals strip,
which is formatted on its own path.
Persistence is untouched except for metadata: `format_f64` is display,
`persistence/format.rs::format_number` writes round-trip precision, and the two
have no call between them. Newly created views write `format: ,.2`; existing
documents are unaffected, because `format_md` has always written the line
explicitly. `web/demo.improv` moves to `,.2` on both views.
The trap this sets is gotcha 34's: `format_md` writes `Formula::raw`, so a
`FORMAT` suffix round-trips whether or not the walker parsed it. The round-trip
test therefore asserts the parsed field — injecting a walker that drops the
clause fails that one test and nothing else in `improvise-io`, 242 tests
including the whole characterization suite and the grammar properties.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Integration review, cross-branch against #108. My asymmetry argument was right
-- a &str dispatch can only have GRAMMAR_INVARIANT, a Rule dispatch can have a
compile error -- and I then applied it to the two &str sites and NOT to the
Rule site where the stronger technique was available. build_primary kept its
unlabelled `else`.
Demonstrated, not argued. With `pct_literal = { "#" ~ identifier }` first in
`primary`: the tree built clean, all 85 crate tests passed including my three
new 492x guards, and parse_formula("X = #Revenue") returned Ref("Revenue") --
the `#` discarded, the formula silently wrong. The injection must be
NON-ATOMIC: an atomic rule is caught by accident in first_inner's own
GRAMMAR_INVARIANT and would have reported the code safe.
Sweeping for the shape found three more in the same file, two genuinely silent:
build_formula's if-chain-in-a-loop dropped an unhandled part (its comment said
"Rule::EOI and any silent rules are ignored"; EOI is now an ARM, so ignoring is
a decision), filter_value_to_string returned unprocessed text, and build_unary
was caught only incidentally downstream. All four now match over Rule with an
or-pattern over the rules that cannot appear, so ANY new rule in formula.pest
is a build failure at all four until someone decides what it means. Verified by
injecting three more probe rules: every site names all three.
The greppable invariant: `== Rule::` now appears nowhere in parser.rs.
each_primary_alternative_builds_its_own_expr is the anti-clamp guard -- a
mechanical if/else-to-match conversion can transpose two arms and still
compile, since the compiler checks that every rule is NAMED, never that it is
named against the right body. Proved by swapping the string and pipe_quoted
arms: it fails and nothing else in the suite does.
Also corrects context/code/formula-and-io.md, which still called the final
`else` arms "unlabelled-but-safe" on the grounds that COUNT was the only
remaining agg_func and >= the only remaining cmp_op. Two of those three clauses
were made false by my own earlier commit. Each was true OF THE GRAMMAR AS IT
STOOD -- a snapshot silently re-read as an invariant, which is the whole defect
in one sentence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Self-review by fault injection, not reading. Three findings, one a real bug
this branch would otherwise have shipped and two vacuous tests of my own.
1. A REAL BUG THIS BRANCH CREATED THE DURABILITY OF. Model::remove_item left
measure_agg holding an entry for a measure it had just deleted -- exactly
improvise-qx1z's defect one layer down, in the same branch that fixed the
View half and did not look at the Model half. Harmless while improvise-ppru
was unfixed, because nothing persisted the map; persisting it turned a
stale entry into a permanent one, so the document grows an Aggregation line
naming a measure it does not have and re-adding that measure brings back an
aggregation nobody asked for. A fix that introduces a format for state that
outlives what it names is how a latent bug becomes a permanent one, so the
sweep ships with the writer.
2. A VACUOUS CONTROL. a_file_naming_no_view_keeps_the_seeded_default PASSED
under the over-broad "always drop Default" it exists to catch, because
delete_view refuses the LAST view -- a guard one level away from the
condition under test. It answered "does delete_view refuse the last view"
where it meant "does drop_unnamed_default decline to fire". Split in two:
a discriminating case naming Default AND a sibling, where the wrong
condition really deletes, and the legacy no-views case kept separate and
marked as the weaker statement it is.
3. A VACUOUS NEW TEST. a_category_named_prune_is_not_read_as_the_prune_flag
passed under the rejected presence-style grammar that swallows "prune: row"
whole, because the fixture used Row and on_category_added RE-DERIVES Row for
the first regular category on reload -- the axis matched by reinvention, not
by being read. Now uses Page plus a page selection, neither of which any
default supplies.
Also verified and unchanged: the generator reaches all three new grammar
branches, including the scalar data line, each proved by a separate injection;
awkward measure names (|A: B|, |sum|, true) round-trip through the Aggregation
section; and improvise-nvq4 still does not reproduce.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Self-review with the code-review skill, against kb.pl's doctrine. All three are
tests that passed while the thing they named was not pinned.
F1. `doctrine(inject_the_rival_fix_not_only_the_bug)`. The jcfs test used one
twelve-value fixture, and `take(100)` — raising the cap rather than removing it
— passed the ENTIRE suite. That pins a behaviour, not the rule: the rule is that
detection does not depend on how many values there are, and any fixed fixture
pins one side of some cap. It now runs at 11 and 500 leading values, so every
`take(N)` fails one arm. Renamed to say what it asserts.
F2. `doctrine(an_injection_that_does_not_fail_names_an_untested_path)`.
`an_all_null_column_is_a_label` was never proved able to fail. Under the l1f5
fault it PASSES — an all-null column was Label before the fix too — so its
neighbours' injection says nothing about it. Injecting its own fault (dropping
the `!present.is_empty()` conjunct) fails it alone. The rival fix for l1f5 was
also injected: `is_number() || is_null()` over the full slice repairs Revenue
and leaves Region broken, which is why both `all_` bindings had to move and
`a_blank_value_does_not_demote_a_string_column` is the separating input.
F3. `doctrine(an_order_test_needs_two_histories)`. The cell-order test used one
insertion history five times, which cannot tell "listed its own order" from
"emitted one canonical order that matches this fixture". Same six regions now
imported forwards and backwards, each asserting its own. Reaching the second
direction needed the loop re-ordered under the fault, since an injection that
fails at iteration 1 leaves iteration 2 untested.
Also: `duplicate_csv_headers_not_silent` asserted its two values by map
ITERATION ORDER; now by key, which is also what it means.
Four controls were verified able to fail rather than assumed:
an_ordinary_float_is_still_a_number (float branch deleted),
distinct_headers_are_not_renamed_and_warn_nothing (rename every header),
an_all_null_column_is_a_label (F2), and the long-row control.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Keyed by test name per design-principles section 7, recording for each one the
bug it reproduced, the fault injected to watch it fail, and -- where a test has
a neighbour that looks like it -- the input that separates the two.
Three entries are about that separation rather than about a bug:
- a_view_that_does_not_prune_writes_no_prune_line is the direction the default
supplies, so a writer emitting "prune: true" unconditionally passes the real
test and fails this one.
- an_interleaved_category_keeps_its_item_order explains why the ticket's own
two-item reproduction cannot discriminate: with one item of each kind, both
the right and the wrong order are a single bare run followed by a single
grouped one.
- slash_in_hidden_name_survives_roundtrip records that improvise-nvq4 does NOT
reproduce, and is kept as a passing control so the claim cannot be re-filed
without someone first watching it pass.
core.md is touched for the improvise-qx1z entries, which are View-level and
belong there rather than in formula-and-io.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
The five reproductions for improvise-jc95, 5hfh, l1f5, 2cw9 and the long-row
half of k8i now run. Removing `#[ignore]` is part of the fix: an ignored
reproduction is an open bug, and the module doc now says so rather than claiming
the file is report-only.
Three of the five asserted warning PROSE against a module that emitted no
warnings at all, so they specified wording with no convention to match — and
`"overwritten"` does not contain `"overwrite"`, so the most natural wording
would have failed its own assertion. Each now asserts the OBSERVABLE outcome
first and the prose only as a companion:
- duplicate_csv_headers_not_silent: three fields with values 1, 2, 3 in order.
- repeated_import_coordinates_not_silently_overwritten: exactly one cell at 30.0.
- long_csv_row_is_not_silently_dropped: no row discarded, plus a warning that
counts the affected rows and names the widest, because the values here really
are still dropped and there is no observable outcome to assert instead.
Every fix was verified by injecting its own fault and watching the test fail:
the `unwrap_or(0)` restored, `disambiguate_headers` short-circuited, the
long-row counter disabled, both `all_` bindings restored to the full slice,
`take(10)` restored, the accumulate arm changed back to last-wins, and the cell
write order sorted. Each named test failed; each control did not.
Evidence in context/test-evidence/formula-and-io.md, keyed by test name,
including the correction to `the_detected_date_format_does_not_depend_on_the_hash_seed`
— its fixture had made the ten-sample truncation a specification.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-492x. parse_agg_func ended `_ => AggFunc::Count` and parse_cmp_op
ended `_ => BinOp::Ge`, so a new alternative in formula.pest would have parsed
as COUNT or as >= with no error: MEDIAN(Revenue) evaluating as a count.
One of three beads that are the same defect (with improvise-vyrx and
improvise-4lgj); design-principles §2 is the rule all three break. A compile
error is not on offer here, because agg_func is one pest rule over five string
alternatives and pest_derive generates no per-alternative Rule — so the walker
gets the idiom this file already uses for a grammar/walker desync,
GRAMMAR_INVARIANT, and the accepted set is written down in Rust as a test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Found by an external Grok Bot adversarial review (improvise-jkxr), verified by
injection rather than taken on report.
The test parses two `## Category: Type` sections -- `- A` then `- B` -- and
asserted only `!item_names.is_empty()`. push_category concatenates on a
duplicate section name, and a merge keeping EITHER half alone still leaves the
list non-empty, so only a total wipe could ever have failed it.
Fault injected: `existing.items.extend(items)` -> `existing.items = items`, so
a second section replaces the first. The old assertion PASSED; the new one
fails with left ["B"], right ["A", "B"].
Pins order as well as membership -- insertion order is load-bearing (gotcha 18)
-- matching the standard its neighbour already holds groups to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
An adversarial review of this branch found that the prose was worse than the
diff — including, twice, in the exact genre the branch was congratulating itself
for catching. Recorded rather than quietly rewritten, because that is the shape.
Must-fix, all verified before correcting:
- `fit.rs`'s comment on the exact check still read "this is the check that
enforces the bound in both directions of a drift" — verbatim the sentence
gp.md §14 now spends 25 lines refuting. Gotcha 48's third clause exactly: a
fact stated twice is changed by grep, not by edit. I refuted the copy in the
prose and left the copy in the code.
- "false for eight months" was fabricated. `encoded_width` landed 2026-09-19,
the sentence 2026-09-20; the repository itself is six months old. A paragraph
whose point is that unchecked cost claims are checkable closed on one.
- "NumericCategory was the one artefact family with no coverage" is false: it
was one of three, and `ValueOfAMeasure` and `AbsentCoordinate` still have
none. Corrected in both files that said it, and the remaining gap is named.
The decline's stated reason was wrong, which matters most because a decline is
the artefact nothing checks. It claimed one-hot is the only encoding of a
category whose splits are spellable; `is_expr` is a `primary`, so an ordinal
split IS spellable as a weighted indicator sum. Verified by running
`parse_formula`, not by re-reading the grammar, and now pinned by
`an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum` so the
paragraph cannot go quietly wrong again. e8no stays declined on grounds that
survive: a category's items carry insertion order rather than magnitude, and the
emitted formula spells one IS per item either way, so the width is not saved.
Also weakened two overclaims — "marking it a Label yields the same fit" (false:
the feature count moves the RNG draw sequence, gotcha 37b) and "asserted rather
than argued" (the dichotomy is argued from spell()'s exhaustive match; the test
asserts one instance) — narrowed "the only coordinate predicate is is_expr" to
what is true of it, corrected an uncheckable test count, named which two of the
three consumers of parse_number_format are actually guarded, and moved an
evidence block that had been inserted under the wrong module heading.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`parse_number_format` read the precision as `fmt[i+1..].parse::<u8>()` over
everything after the last `.`, so `,.2f` — the printf spelling the repo-map's
own `.improv` example carries — failed to parse and fell through to zero
decimals, silently, in the TUI, the browser projection and CSV export.
The precision is now the leading digit run after the last `.`, with any
conversion letter ignored. The spellings that are legal are stated once, in
`context/code/core.md`, along with why the trailing `f` is accepted rather than
refused (the grammar takes the format line as an opaque `rest_of_line`, so
refusing it rejects documents this repo already writes) and why the reading is
printf's rather than Excel's (`.00` is the spelling the two disagree on).
`export_csv_quotes_commas_per_rfc4180` asserted the value `starts_with("1,200")`
against a `,.2f` fixture, so it could not tell two decimals from none. It is now
an exact equality, and reverting the fix fails it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Adversarial review round on the branch. One real finding, two prose/comment
corrections, one fixture widened.
F2 (the real one, and introduced by zv8t). `as_categories` forces by column
NAME; `encoded_width` counts by column INDEX. A document may legally name a
regular category and a raw measure the same thing — the repo already ships that
fixture in `a_category_and_a_measure_of_the_same_name_are_refused` — and the
measure column was then forced categorical by the name match while the preflight
still counted it numeric by index. Measured at 40 records: preflight 41, actual
80. `no_ambiguous_column_names` ran AFTER `preprocess`, so the expansion the
preflight exists to prevent was materialised and only then refused; at the
default max_records that is a 20k-row one-hot matrix built before the error.
That is `doctrine(guard_the_cost_not_the_result)` defeated by an inconsistency
this ticket introduced — not the bounded approximation that
`doctrine(a_budget_check_is_advisory_or_exact_and_must_say_which)` licenses.
The repair is a data dependency rather than a moved line: `as_categories` runs
the check and returns `Result<Overrides>`, so the Overrides `preprocess` needs
cannot exist until the header has been accepted
(`doctrine(an_invariant_is_made_unrepresentable_not_validated)`). The regression
guard is therefore the SIGNATURE and the injection is a compile error — swapping
the two statements back gives E0425, verified. What it does not cover is stated
in the evidence rather than implied: a deliberate deletion of the check from
`as_categories` is still possible; the accidental reorder is not.
F5. The `encoded_width` comment claimed the preflight "must count it the same
way". It does not, quite: `build_categorical` counts MISSING_CAT as a distinct
value and the preflight does not, so it under-counts by one per category with an
absent coordinate. Bounded by the exact check and pre-existing for genuinely
categorical columns — but the comment asserted an identity that does not hold.
F4. The evidence for the two spelling tests said that on the unfixed tree the
text "could only have been `Expr::Ref(\"Year\")`". There is no text on the
unfixed tree at all — `fitted_formula` refuses before emitting. The argument for
the assertion survives; the stated reason was a guess about a branch that does
not run, and an evidence file is the one place that should be exact.
And the iot6 fixture: `prune_empty` prunes BOTH axes and the test asserted rows
only. `Feb` is now the empty column, the precondition is a pair on each layout,
and the CSV header is asserted to carry `Jan` alone.
1464 passing, up from 1463.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
`render_csv` built its layout with `GridLayout::new`, which does not apply
`prune_empty` — only `with_frozen_records` does. So `:export` in the TUI and
the CLI export path both wrote rows the grid does not draw. User-visible, in
production, and shipping for as long as the two constructors have differed.
The fourth instance of this trap; the third was `project::grid_in` in PR #78,
and this is that fix line for line.
The fixture is the difficulty, and the ticket says so. The sibling property
written for improvise-t6db passed on the buggy tree in its first version,
because it made its two views differ by turning pruning *off* — vacuous, since
with nothing to prune both code paths agree. So this one contains a row that is
actually empty, leaves `prune_empty` ON, and asserts the precondition in both
directions before exporting: `with_frozen_records` draws 2 rows where `new`
holds 3. If pruning ever becomes a no-op the precondition fails and says so,
rather than the assertion passing vacuously.
No fault needed injecting — the bug was live. On the unfixed tree the test
fails with `["Food", "Gas", "Rent"]` against `["Food", "Gas"]`. None of
`export.rs`'s three pre-existing tests mentions pruning at all, which is
exactly why this shipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-8shu. improvise-io's lib binary was the run's outlier at 1.42s
against a 0.01s median. The ticket's diagnosis was an unbounded grammar walk
generating deep documents; measuring the two halves separately falsified it.
load_grammar() x1000 was 2.50s and the walk x1000 was 15.7ms — improv_file()
called load_grammar() inside its prop_map, so pest_meta re-parsed the grammar
once per generated case.
The strategy's input space is unchanged. The grammar is read once through a
OnceLock, and a guard counts parses rather than calls so it is independent of
which test in the binary runs first. Case counts go 500 -> 256, the default
design-principles §7 prescribes and never below it.
1.42s -> 0.21s; improvise-io is no longer the run's longest binary.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Adversarial review injected the other fault -- Some(_existing) => {}, which
drops the duplicate section outright -- and the whole improvise-io suite stayed
green at 2000 proptest cases while the fixture's item and its group line
vanished from the document. A test that compares two printed forms cannot tell
unstable-but-lossless from stable-and-lossy, because both sides of the equality
lose the same thing. Three contains assertions now run before the equality, and
they fail under that injection.
Two more from the same review. push_category destructures PCategory, so a fourth
field has to be named rather than silently dropped on the duplicate path only.
And design-principles.md's formula-tokenizer section described a tokenizer that
no longer exists -- a split_where pre-parse step and Token::Ident/Token::Str,
none of which are in the tree -- and claimed bare identifiers allow spaces,
which bare_ident forbids. This branch changes that production, so correcting the
sibling copy is the grep gotcha 48 asks for.
improvise-349m improvise-r08u
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-l8rh carries the user's ruling of 2026-09-20: leave the first-match
heuristic as it is and track it as a known limit. The prose still read as a
filed-and-open question, and it did not record what the ruling rests on -- that
a day-first default only moves which half of the world imports silently wrong,
that the honest fix changes FieldProposal's shape, and that the limit is
tolerable chiefly because improvise-7yx made it deterministic.
No behaviour change; the two pinning tests are unchanged.
improvise-l8rh
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
context/code/formula-and-io.md records why the category merge happens in pass 1
rather than pass 2, and why the WHERE boundary is sp+ rather than a lookahead --
the grammar-walking generator cannot honour a predicate, which is the part that
generalises.
context/test-evidence records the fault injected for each test, including the
one that made the WHERE test carry two negative shapes: compound-atomicity alone
already refuses the ticket's own reproducer, so a test built only on it passes
over a rule still spelled sp*.
improvise-349m improvise-r08u
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Review of #54 found the fold reintroducing exactly the divergence the ruling
exists to remove, and the same line turned out to be the 244 ms stall. One
defect, two symptoms.
`eval_expr_cached`'s `Expr::Ref` looks the value up in `formula_cache` at
`context.with(cat, name)`. Inside the fold `context` is a MEMBER-RECORD key
carrying the none_cat coordinates, while `formula_cache` is keyed by stems with
none_cats STRIPPED — so the lookup could never hit, fell through to
`aggregate_raw`, found no raw cell behind a formula target, and answered
Err("ref:…"). A routing formula over a derived measure gave Number(11.0)
uncached and Error("ref:Margin") cached, and the cached path is the one a pivot
grid runs whenever any category sits on Axis::None.
`eval_expr`/`eval_bool` are hoisted to module level and BOTH evaluators now hand
the fold the same record-level `eval_expr`. The two agree by construction rather
than by a test. Note the repair not taken: teaching the fold to hit the cache by
stripping none_cats off the member key returns the GROUP's value to every
member — aggregate-then-route through a cache lookup, and silent.
That same per-member `aggregate_raw` was one poorly-narrowed scan per member, an
O(n^2) shape. Measured with the new `examples/fold_bench.rs`, five runs a point,
at 18,050 cells: 51.1-53.3 ms before, 18.1-19.2 ms after, against 7.1-9.0 ms for
a linear formula. Scaling went from superlinear (45x cells, 85x time) to linear
(45x cells, 45x time). Debug 2.75 s -> 132.9-135.1 ms. The fixed point runs
exactly 2 passes, measured by counting, so hoisting the member set has a 2x
ceiling and is declined on improvise-9iyv — a memo keyed by less than the whole
`partial` is a second answer to what a cell depends on.
`Expr::Is` has an arm in each evaluator and the cached one was untested: the
earlier fault was injected into the shared `eval_is`, which both arms call.
Replacing the cached arm alone left the suite green. The first replacement test
ALSO stayed green, which is the interesting part — once the fold routes through
`eval_expr`, an `Is` inside a SUM never reaches the cached arm, so that arm is
reachable only by a bare `Is` outside an aggregate. That is what the new test
uses, and injecting into that arm alone now fails it.
`IS` gained a word boundary: `Region ISLAND` parsed as Is(Region, "LAND"),
silently wrong where it was a parse error. `is_expr` is compound-atomic with
`sp+` written out, because a negative lookahead in an ordinary rule tests the
character AFTER pest's auto-inserted whitespace and fails backwards. The
lookahead was then redundant and removed — deleting it left the suite green,
while relaxing `sp+` to `sp*` fails the test, so `sp+` is the mechanism. `WHERE`
has the same hole, is far less exposed, and is filed as improvise-r08u.
Also: `ambiguous.sort_unstable()` before `dedup`, which only removes consecutive
duplicates, so the refusal could name a column twice.
Every fault re-injected after the refactor: no-fold 10 failures, shared eval_is
5, record-level Is arm 4, cached Is arm 1, feature_roles order 2, context-vs-
partial 1, the new cached-fold divergence 1, `sp+` relaxed 1. Baseline 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review finding [P2]: four headings named unannotated helpers —
load_corrupt_gzip, storage_fixture, populated_model, awkward_name. The
current --check-evidence accepts every `fn`, so it reported them as
satisfied; a corrected checker would have failed this branch.
They are now `####` subordinate headings, each saying it is not a test and
naming the tests it supports. The narratives are worth keeping — the
preconditions in load_corrupt_gzip and the reason populated_model is not a
fresh workbook are both "why this test is not vacuous" — so they are demoted
rather than deleted.
Checked against a stricter local checker that accepts only a fn preceded by
`#[test]` or inside a `proptest!` block: 64 headings, 0 orphans.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
improvise-formula's comments carried three arguments worth keeping and
several restatements of the code. The arguments move to
context/code/formula-and-io.md (pipe quoting's one home, the keyword guard
and why PEG rule ordering is not a rule, the unquoted formula target, and
why a new grammar production needs generator coverage) and the bug
narratives to context/test-evidence/formula-and-io.md, keyed by test name.
No behaviour change: comments only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Edward Langley