docs: the duplicate-section merge and the WHERE boundary, with their evidence

context/code/formula-and-io.md records why the category merge happens in pass 1
rather than pass 2, and why the WHERE boundary is sp+ rather than a lookahead --
the grammar-walking generator cannot honour a predicate, which is the part that
generalises.

context/test-evidence records the fault injected for each test, including the
one that made the WHERE test carry two negative shapes: compound-atomicity alone
already refuses the ticket's own reproducer, so a test built only on it passes
over a rule still spelled sp*.

improvise-349m improvise-r08u

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
2026-09-20 21:09:49 -07:00
parent 7d2e489c62
commit 35adf66757
2 changed files with 76 additions and 4 deletions
+34
View File
@@ -92,6 +92,28 @@ round-tripped from the day it exists.
What the *evaluator* does with it, and why a fitted tree is always wrapped in a
`SUM`, is `context/code/core.md` §13.
### `WHERE` carries the same boundary, and the mechanism has to be generatable
`where_clause` and `inline_where` are compound-atomic with `sp+` after the
keyword, exactly as `is_expr` is (improvise-r08u). Non-atomic, pest auto-skipped
`WHITESPACE` after `^"WHERE"`, so `SUM(Rev WHEREAS x = y)` read as a `WHERE` on
a category `AS` — a silently wrong filter where it should have been a syntax
error, the same class of defect `IS` had.
Two things decided the shape, and the second is the one worth carrying. A
negative lookahead `!bare_char` with `sp*` would have been *strictly* more
permissive — it keeps `WHERE|Region|=East`, which the old rule accepted — but
**the grammar-walking generator cannot honour a predicate**: it walks
`formula.pest` through `pest_meta` and emits `WHEREa=""`, text the grammar then
refuses, so `generated_formula_parses` goes red. A rule whose only guard is a
lookahead is a rule the generator can violate. `sp+` is generatable, matches
`IS`, and costs exactly one accepted spelling: a category may no longer abut the
keyword, so `WHERE |Region|` needs its space.
Note compound-atomicity *alone* closes the ticket's own reproducer, because
`AS x` cannot be one identifier once implicit whitespace is gone. `sp+` is what
closes the residue, `WHEREAS=y`, and the negative test names both.
## A formula target is unquoted, exactly as a reference is
`parse_formula` strips the pipes off the target the same way it does off an
@@ -158,6 +180,18 @@ format order-independent — a `## View:` may name a category defined further
down. `PCategory::items` is a `Vec` in file order because `IndexMap` insertion
order is load-bearing (gotcha 18), so pass 2 must replay the list as written.
Two `## Category:` sections naming one category are **one** category, and the
merge is a concatenation performed in pass 1 (`ParsedFile::push_category`,
improvise-349m). Merging in pass 2 — building each section in turn — was not
idempotent. A group is created either by the item that names it or by a `>`
line, and `build_category` creates its own section's item-groups *before* its
own `>` lines; so the first section's `>` groups landed ahead of the second
section's item-groups, while `format_md` writes one section with all the items
before all the `>` lines. The reparse then built the other order, and the two
`format_md` outputs differed. Concatenating first makes the built order the
written order by construction. Refusing a duplicate header was the alternative
and would have made a legal hand-written file an error for no gain.
### A section header takes its name as the whole line, totally
`header_name = { pipe_quoted ~ &NEWLINE | line_name }` (improvise-pczv). The