ImportPipeline::from_merge seeds the File items and every file's columns
from Merged::sources, so a header-only CSV keeps its File item and its
columns (unseen columns are proposed as Labels, as an all-null column
is). A file with no measure column gets a records view filtered to it
instead of paging onto another file's hidden measure. Four per-file-view
rules the review found untested now each have a test that fails when
the rule is broken.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The user's ruling on PR #314: a folder always merges, so even a single
CSV in it records its file identity. A CSV named directly still imports
alone. The "(none)" question is answered as no position, per the user.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
Review of #307 at 7b9e868b: the duplicate-step evidence and argument still
said 12, tui.md bounded a collision tuple by twelve, and the coupling
inventory named no beads. The page-bar entry now notes its unclipped
set_string can overwrite the right border.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
The wizard's private copy of the limit is gone; it reads the model's
now-public constant. Tests that fill to the cap derive it. Why 12 was
chosen and what 24 costs: context/code/core.md section 9.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
Every wizard write goes through Panel, which clips to the panel and saturates
its height arithmetic, so no step panics on a small terminal. The bottom row
truncates in cells, so a wrapped message with a multibyte character is drawn
whole. A scrolling list's range is named on its header row, where no item can
cover it. Tests: a multibyte message at every width, wrap bytes vs cells, a
no-panic sweep of every step at 20-100 x 3-20, the first item filling the list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
When source records collide on every coordinate, the import wizard now asks:
sum them, keep them separate with a generated occurrence category, or go
back. Collisions are found before the build by the same placement pass the
build uses, and are carried to the TUI as structured data (WizardCache) and
listed in full, scrollable. At the 12-category cap, "keep separate" is shown
disabled with its reason and remedy.
Every wizard warning and message now wraps to the panel instead of being
truncated, and long lists scroll.
Headless paths no longer sum silently: `import --no-wizard` takes
`--on-duplicate refuse|sum|separate` (default refuse), and `:import-json`
refuses and points at `:import`.
Reasoning: context/code/formula-and-io.md, "Colliding records are a
question, not a default"; context/code/tui.md; context/code/cli.md §11.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
improvise-tt22 merged into gh/main while this branch was open and dropped
serde from core's document types, CategoryKind included, keeping only the
wire types (Axis, PageAxis). TileEntry now carries a CategoryKind, so it
joins them. formula-and-io.md's list of what remains is updated to say so.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
tt22 removed the formula crate's serde dependency with the legacy JSON loader,
keeping serde only where a type crosses the wire. FormulaId crosses the wire, so
it is the third such type. No derive goes back on the formula AST.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
context/code/formula-and-io.md already says a new grammar production needs
generator coverage or the proptests quietly stop describing the format.
`kind_line` is an alternative of `category_entry`, so the grammar walk
reaches it with no generator change — but that is precisely the claim the
section exists to say must be checked rather than assumed.
Probed over 500 files from generator::improv_file(): 14 carried a
`kind: label` line and all 14 parsed to a CategoryKind::Label, so each
256-case proptest run exercises the production about seven times.
The probe is not kept. Its rate depends on proptest's internal RNG, so as
a standing assertion it would fail on a dependency bump for a reason that
has nothing to do with the format — the measurement belongs in the
argument, where it can be re-run and corrected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-dy3j (P1). CategoryKind::Label had no spelling in `.improv` at
all: format_md wrote every category as a bare `## Category: Name` and
build_category read every section back through add_category. The kind was
dropped on the way out AND on the way in, so a round-trip equality test
could not see it.
A label is outside MAX_CATEGORIES = 12 by design and the import wizard
scores it as zero, so twelve categories plus a label is legally
constructible. On load every label came back Regular, the thirteenth was
refused, and parse_md failed: any such document was permanently
unloadable. Below the limit the loss was quieter — a label silently
became a pivot category.
Grammar first. The ticket proposed `## Category: Note [label]`, and
improvise-pczv forbids it: a section header takes its name as the WHOLE
line, so the suffix is indistinguishable from a category named
`Note [label]`. The kind is therefore an entry inside the section,
`kind: label`, spelled with a real token rather than a lookahead — the
grammar-walking generator skips predicates. Absence means `regular`, so
every existing file parses unchanged and a pivot category's section is
byte-identical to before.
Tests assert the parsed Category::kind, never the emitted line. Both
rival faults were injected against the fixed tree and both fail two
tests: a writer that omits the line, and a reader that discards the kind
it parsed (whose file text is correct, so any text-comparing test passes
it). Reasoning in context/code/formula-and-io.md; evidence in
context/test-evidence/formula-and-io.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
Integration review, cross-branch against #108. My asymmetry argument was right
-- a &str dispatch can only have GRAMMAR_INVARIANT, a Rule dispatch can have a
compile error -- and I then applied it to the two &str sites and NOT to the
Rule site where the stronger technique was available. build_primary kept its
unlabelled `else`.
Demonstrated, not argued. With `pct_literal = { "#" ~ identifier }` first in
`primary`: the tree built clean, all 85 crate tests passed including my three
new 492x guards, and parse_formula("X = #Revenue") returned Ref("Revenue") --
the `#` discarded, the formula silently wrong. The injection must be
NON-ATOMIC: an atomic rule is caught by accident in first_inner's own
GRAMMAR_INVARIANT and would have reported the code safe.
Sweeping for the shape found three more in the same file, two genuinely silent:
build_formula's if-chain-in-a-loop dropped an unhandled part (its comment said
"Rule::EOI and any silent rules are ignored"; EOI is now an ARM, so ignoring is
a decision), filter_value_to_string returned unprocessed text, and build_unary
was caught only incidentally downstream. All four now match over Rule with an
or-pattern over the rules that cannot appear, so ANY new rule in formula.pest
is a build failure at all four until someone decides what it means. Verified by
injecting three more probe rules: every site names all three.
The greppable invariant: `== Rule::` now appears nowhere in parser.rs.
each_primary_alternative_builds_its_own_expr is the anti-clamp guard -- a
mechanical if/else-to-match conversion can transpose two arms and still
compile, since the compiler checks that every rule is NAMED, never that it is
named against the right body. Proved by swapping the string and pipe_quoted
arms: it fails and nothing else in the suite does.
Also corrects context/code/formula-and-io.md, which still called the final
`else` arms "unlabelled-but-safe" on the grounds that COUNT was the only
remaining agg_func and >= the only remaining cmp_op. Two of those three clauses
were made false by my own earlier commit. Each was true OF THE GRAMMAR AS IT
STOOD -- a snapshot silently re-read as an invariant, which is the whole defect
in one sentence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-l8rh carries the user's ruling of 2026-09-20: leave the first-match
heuristic as it is and track it as a known limit. The prose still read as a
filed-and-open question, and it did not record what the ruling rests on -- that
a day-first default only moves which half of the world imports silently wrong,
that the honest fix changes FieldProposal's shape, and that the limit is
tolerable chiefly because improvise-7yx made it deterministic.
No behaviour change; the two pinning tests are unchanged.
improvise-l8rh
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
context/code/formula-and-io.md records why the category merge happens in pass 1
rather than pass 2, and why the WHERE boundary is sp+ rather than a lookahead --
the grammar-walking generator cannot honour a predicate, which is the part that
generalises.
context/test-evidence records the fault injected for each test, including the
one that made the WHERE test carry two negative shapes: compound-atomicity alone
already refuses the ticket's own reproducer, so a test built only on it passes
over a rule still spelled sp*.
improvise-349m improvise-r08u
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
improvise-4zl7. A tree is fitted on individual records; an improvise formula is
evaluated at a CellKey, and an aggregating cell used to aggregate each Expr::Ref
first and evaluate the IF once on the totals. That is the same answer for a
linear formula and a different one for anything that routes, so every fitted
tree meant one thing on a record row and something else one row up. The user
ruled: aggregate the rule over the members.
The rule needed no new aggregate vocabulary. SUM(e) already means "fold e over
the members" and simply did not: eval_agg specialised Expr::Ref and, for every
other inner expression, summed the raw matching cells and ignored the expression
entirely. It now folds — one entry per member record, the expression evaluated
at that record's own coordinates. eval_agg is shared by eval_expr and
eval_expr_cached, so the fold has exactly one home; each evaluator passes its
own recursion in.
Expr::Is(Filter) is the new arm in both evaluators: `Cat IS Item` is 1.0 when
the cell's own Cat coordinate is Item, keeping the evaluator f64-only and
reusing Formula::filter's own context.get/unwrap_or(false). A grammar production
in primary, before ref_expr, with both operands through
improvise_formula::quote's one home; IS joins the keyword guard.
The emitter reads feature index -> role off preprocess's plan rather than
reconstructing it: gp_tree::data::feature_roles is the one statement of the
expansion order and encode_row now drives off it too. improvise_gp::Features
carries one Spelling per feature — Measure, Coordinate or Artefact — decided by
role and column, never by the name's shape. A one-hot over a real category is
emitted as `Region IS North` and accepted; col.missing and the date parts stay
Artefact and stay refused (improvise-wd6n).
fit-measure on a document a category explains now lands instead of refusing, and
the refusal no longer depends on which features the search happened to pick.
Evidence: eight hand-computed tests in improvise-core over a four-record fixture
whose Cost straddles the threshold inside each region, so route-then-aggregate
and aggregate-then-route differ at every level. Two faults injected and watched
to fail — reverting the fold fails 7 of 8, and a constant-false eval_is fails
exactly the 3 coordinate tests. Reordering feature_roles fails both it and
encode_row_replays_the_training_expansion.
The fold is O(members) per evaluation per fixed-point pass and unbenchmarked:
improvise-9iyv.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
improvise-formula's comments carried three arguments worth keeping and
several restatements of the code. The arguments move to
context/code/formula-and-io.md (pipe quoting's one home, the keyword guard
and why PEG rule ordering is not a rule, the unquoted formula target, and
why a new grammar production needs generator coverage) and the bug
narratives to context/test-evidence/formula-and-io.md, keyed by test name.
No behaviour change: comments only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude (agent-hier)