docs: group hierarchies and fold lines, with the evidence for each rival (improvise-felo.6)

The harness fixtures open their groups explicitly now that groups start closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze

context/code/core.md

@@ -258,9 +258,11 @@ user drilled in to see all the raw data.
 
 On an axis with two grouped categories the runs **nest**: `North` in `Upper`
 holds `Jan` in `Q1`, and after `Q1` ends `North/Mar` is still in `Upper`. So a
-marker names its level. `GroupHeader { level, .. }` opens a run at `level` and
-closes every deeper one; `GroupEnd { level }` closes `level` and every deeper
-one. `expand_category` reads the level off its prefix, which holds one item per
+marker names its level. `GroupHeader { level, depth, .. }` opens a run at
+`(level, depth)` and closes every deeper one; `GroupEnd { level, depth }` closes
+that run and every deeper one. "Deeper" is the ordering of the pair: the level
+first, then the depth inside one category's hierarchy (§19, improvise-felo.6).
+`expand_category` reads the level off its prefix, which holds one item per
 outer category, so it is derived rather than passed. A row's group, for
 `row_group_for`/`col_group_for` and so for `z`, is the **innermost** run open
 at it. That is the user's ruling on the ticket: membership is per level, and
@@ -1556,3 +1558,128 @@ editor stays open over the user's text (gotcha 21). `was` is not redundant
 with a stable id: a concurrent *replace* keeps the id and changes the text,
 which only `was` can see.
 
+
+## 19. Group hierarchies: parents nest runs, and every group starts closed (improvise-felo.6)
+
+`Group::parent` was parsed (`> Q1[2024]`), written, round-tripped and hashed,
+and nothing read it: `Q1` was never drawn inside `2024`. It is now the
+hierarchy.
+
+### The user's rulings
+
+- "I think hierarchies should start closed unless we somehow opt into a
+  specific one opened to a certain depth (view config maybe?)" — and the
+  reason: "in most cases, the closed by default start is preferable because it
+  makes the grid more dense."
+- "Everything opens closed, including old files." There is no version gate: a
+  file written before this change opens with its groups closed, and its
+  `collapsed:` lines simply agree with the default.
+- "I think we'll need a depth for expanded, something like expanded(2): ...".
+- "expanded+collapsed in the same group is an error".
+
+### The model
+
+- **A group's chain** is the group and its ancestors, root first
+  (`Category::group_chain`). An item's chain is its group's
+  (`Category::item_chain`); an ungrouped item has none. A group's depth is its
+  number of ancestors.
+- **A group covers every leaf beneath it.** `items_in_group` reads each item's
+  chain, so `2024` covers the months of `Q1-2024` and `Q2-2024`. That is the
+  whole change to evaluation: `evaluate_covering` already expanded a group
+  coordinate through `items_in_group` (§14), so a year's fold is the fold over
+  its months with no second path.
+- **An item may sit directly in a parent group, beside its child groups.**
+  Chosen because nothing is simpler: the chain handles it (`Jun[H1]` after a
+  run of `Q2` inside `H1` ends the `Q2` run and stays in `H1`), and refusing it
+  would be a rule with nothing to protect. A date import never produces one.
+- **A cycle is refused on load, naming its members** (`Refusal::HierarchyCycle`,
+  `persistence/parse.rs`). A cycle built in code cannot hang anything either:
+  `group_chain` stops where it would repeat.
+- **There is no such thing as a dangling parent in this format.** A root group
+  is declared *only* by being named as a parent: the writer emits `>` lines
+  only for groups that have a parent, and nothing else can name `2024`. So
+  "a parent that names no group" and "a root" are the same line, and refusing
+  one refuses the other.
+
+### Which groups are open
+
+State lives on the `View`, in two places that cannot disagree about one group:
+
+- `expand_depth: cat → N` — `expanded(N): Cat`, the category's top N levels
+  open.
+- `group_folds: cat → group → GroupFold` — ONE map, so a group has at most one
+  fold. `GroupFold::Collapsed` is `collapsed: Cat/Group`;
+  `GroupFold::Expanded(N)` is `expanded(N): Cat/Group`, the group open and
+  N-1 levels beneath it. Two sets would have made "both" representable, which
+  is the state the user ruled an error
+  (`doctrine(an_invariant_is_made_unrepresentable_not_validated)`).
+
+`View::is_group_collapsed(cat, chain)` answers, in order:
+
+1. the group's own fold, if it has one — so a `collapsed:` group inside an
+   `expanded(N)` region stays collapsed, and an `expanded:` group below the
+   category's depth opens;
+2. open, if an ancestor's `Expanded(n)` reaches it (the ancestor `k` levels up
+   reaches it when `k < n`), or the category's depth does (`depth < N`);
+3. closed.
+
+**`z` writes one fold, or none.** `toggle_group_collapse` flips the answer and
+records the flip as `Collapsed` or `Expanded(1)` — unless the flip lands on
+what the group would inherit from rule 2, in which case it removes the group's
+fold. So `z z` leaves the view as it was, and no sequence of toggles writes a
+file that names one group twice
+(`no_sequence_of_toggles_writes_a_conflicting_file`). Closing a group that
+carried `Expanded(3)` drops the 3: reopening it with `z` opens one level.
+
+### The layout
+
+`expand_category` walks each drawn item's chain. The **outermost** collapsed
+group in the chain folds everything beneath it: the row's tuple is that group's
+name, it folds once however non-contiguous its members are (§14), and the
+headers above it are drawn. The run markers move from item to item exactly as
+`remark` moves them after pruning — past the shared prefix of the two chains,
+end the deeper runs innermost-out, then open what differs — so an unpruned axis
+survives `prune_axis_items` marker for marker, and the enumeration in
+`group_membership` (now with a `NESTED` fixture: a non-contiguous quarter, an
+item directly in a half-year, an outer grouped category) asserts it.
+
+A header carries its own `collapsed`, computed once in `expand_category`, so
+the frame projection and the frame hash read it rather than asking the view
+again with a chain they would have to rebuild.
+
+### Persistence
+
+`expanded(N): Cat`, `expanded(N): Cat/Group`, bare `expanded:` for N = 1, and
+the existing `collapsed: Cat/Group` (`improv.pest`, `expanded_line`). N is at
+least 1 by the grammar (`'1'..'9' ~ ASCII_DIGIT*`), so `expanded(0)` is a parse
+error rather than a value to explain. Two further decisions:
+
+- **The writer keeps what the view holds.** An old file's `collapsed:` lines
+  load as `Collapsed` folds and are written back verbatim, though they agree
+  with the default. They are the user's explicit closes, and they survive a
+  later `expanded(N): Cat` that would otherwise open those groups; dropping
+  them on save would change what that later line means. A `z` round trip
+  removes such a line, because `z` normalises (above).
+- **A category depth is never written bare.** `axis_line` is tried before
+  `expanded_line`, so a category literally named `expanded` keeps its
+  `expanded: row`. The price is that `expanded: row` meaning "category `row`,
+  depth 1" would read as that axis line, so the writer always spells the
+  category form `expanded(N): Cat`. The group form has a `/` and is never
+  ambiguous.
+
+A view naming one group with two different folds, or one category with two
+depths, is refused on load, naming the view and the group
+(`Refusal::FoldConflict`). The same line twice is not a conflict.
+
+The grammar-walking generator emits cycles and conflicts with ordinary
+probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a
+`Refusal` — a typed error, so it cannot accept a parse failure by accident.
+
+### Known limits
+
+- The TUI draws nested headers exactly as it draws any other: one line per
+  header, not indented by depth, and the column band shows each column's
+  innermost group only. Both are pre-existing presentation choices; the frame
+  carries `depth` for the day a front end wants to indent.
+- Two `>` lines giving one group two different parents: the last wins, as it
+  did before this change. That makes the hierarchy a function of line order.

context/code/formula-and-io.md

@@ -218,6 +218,32 @@ times. The probe was removed rather than kept: its rate depends on proptest's
 internal RNG, so as a permanent assertion it would fail on a dependency bump
 for a reason unrelated to the format.
 
+### Folds are view lines: `collapsed:` and `expanded(N):` (improvise-felo.6)
+
+Groups start collapsed, so a view persists what it opens. `expanded(N): Cat`
+opens a category's top N levels, `expanded(N): Cat/Group` opens that group and
+N-1 levels beneath it, a bare `expanded:` is N = 1, and `collapsed: Cat/Group`
+keeps its old meaning. The semantics, the precedence and the refusals are
+`context/code/core.md` §19; what is particular to the format:
+
+- **No version bump.** The line is additive, and the user ruled that old files
+  open closed like new ones, so `v2025-04-09` still reads every file it did.
+- **N >= 1 is grammar, not validation**: `depth_number = @{ '1'..'9' ~
+  ASCII_DIGIT* }`. Spelled with a `Range` and `ASCII_DIGIT` rather than
+  `ASCII_NONZERO_DIGIT`, because the generator knows the first two and would
+  emit nothing for the third — `expanded(): ` — a file its own grammar refuses.
+- **`axis_line` comes before `expanded_line`** in `view_entry`, so a category
+  named `expanded` keeps its axis line, and the writer never spells a category
+  depth bare (`expanded: row` would be that axis line).
+- **Two refusals the grammar cannot express** — a cycle among `>` lines and one
+  group given two folds — are a typed `Refusal`, which `generated_file_parses`
+  accepts and nothing else does. The generator produces both at ordinary rates.
+- **The generator reaches the new rule**, asserted by
+  `the_generator_writes_expanded_lines_with_and_without_a_depth` over 512
+  deterministic files. Unlike the kind-line probe above, it asserts that each
+  spelling occurs at all, not a rate, so a proptest bump fails it only if the
+  walk stopped reaching the rule.
+
 ### The tree walkers are infallible by construction
 
 Every `build_*` in `parser.rs` operates on a `Pair` pest has already

context/code/protocol.md

@@ -1474,6 +1474,22 @@ sequence. The header's level also gives a client the group's category,
 the dead field improvise-7xyx removed: `level` is read by `groups()`, and the
 category name is not carried twice.
 
+**Within one category runs nest by `depth` (improvise-felo.6).** A group's
+parent (`> Q1[2024]`) puts the quarter's run inside the year's on the SAME
+level, so both markers also carry `depth`, the group's number of ancestors.
+A run's key is `(level, depth)`, compared in that order: a header at a key
+closes every run with a greater key, and so does a `GroupEnd`. With `Date`
+alone on an axis, `2024-03` after `Q1-2024` ends sits in `2024`, and after
+`2024` ends in nothing. Depth rides beside `level` rather than being folded
+into one flat index, because a flat index would need every category's tree
+height to be known before the first marker, and `level` would stop naming
+`cats[level]`. Both front ends draw a nested header exactly as they draw any
+other: one inline row per `Group` entry, the innermost group in the column
+band. `the_two_front_ends_draw_the_same_nested_hierarchy_on_rows` and
+`..._on_columns` in `tests/grid_frame_agreement.rs` hold them together, and
+`the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy` does the same for the
+windowed frame against the layout.
+
 `group_runs` is unchanged. It coalesces whatever `groups()` answers, by header
 identity (improvise-h0iv), so the band still paints one group per position, now
 the innermost. One band per level would be a separate feature.
@@ -1554,14 +1570,19 @@ runs. Reversing it is one comparison, and the test is renamed
 
 ## §3b. A frame's levels are checked as it arrives
 
-A `FrameGroup.level` and a `GroupEnd { level }` name a category: they index
-`cats`. `AxisFrame::groups` keeps one open run per level and sizes that stack
-with `open.resize(g.level, None)`. So a level arriving over the wire decides how
-much it allocates. A frame whose level is `4294967295` would ask for a
+A `FrameGroup.level` and a `GroupEnd`'s `level` name a category: they index
+`cats`. `AxisFrame::groups` used to keep one open run per level and size that
+stack with `open.resize(g.level, None)`, so a level arriving over the wire
+decided how much it allocated. A frame whose level is `4294967295` would ask for a
 multi-gigabyte vector, and the client would be taken down instead of the frame
 being declined. ChatGPT Codex found this on #250, the PR that gave levels to the
 wire (improvise-e265).
 
+Since improvise-felo.6 `groups()` keeps only the headers it has received,
+ordered by `(level, depth)`, so no number on the wire sizes anything and the
+new `depth` field needs no bound. The level check stays for its other reason:
+`cats[level]` names the group's category, which a browser `z` reads.
+
 The bound is enforced at deserialization, by `#[serde(try_from =
 "UncheckedAxisFrame")]`. That is the one place a frame enters from outside. A
 level that is not below `cats.len()` fails to parse, with an error that names the

context/test-evidence/command.md

@@ -7214,3 +7214,16 @@ cleared the cell cannot pass.
   `the_shipped_demo_seeds_a_worker_whose_raw_cells_are_settable` wrote with
   `view: "Default"` against `web/demo.improv`, whose view is `Plan`. It passed
   only because a stored record ignored the view name. It now names `Plan`.
+
+### the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy
+
+improvise-felo.6. The windowed frame carries the runs open above the window;
+those are now keyed by `(level, depth)`. Rival: `axis_frame` closing carried
+runs by level alone. Only this test fails; the existing agreement test has one
+group per level, where the two keyings coincide.
+
+### z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again
+
+`z` through a closed Year > Quarter > Month hierarchy, cursor at row 1. Rival:
+`toggle_group_collapse` never removing a fold; the final assertion (no fold left
+behind once the quarter is closed again) fails.

context/test-evidence/core.md

@@ -1865,5 +1865,56 @@ were run against the whole `improvise-core` suite:
 **What it cannot see is the allocation itself.** A visitor that clones the
 stack at every row returns the same answers, so no assertion here can tell the
 two apart without a counting allocator. The copy-free property is held by the
-shape: the visitor receives a borrowed `&[Option<usize>]`, and no owned stack
+shape: the visitor receives a borrowed slice of open runs (`&[OpenRun]` since
+improvise-felo.6), and no owned stack
 leaves the walk. It is not held by a test.
+
+### a_parent_group_covers_every_leaf_beneath_its_child_groups
+
+improvise-felo.6. `items_in_group` read only the items whose own `group` named
+the group, so a year covered nothing: its months sit in quarters. Rival run
+against `improvise-core`: the direct-only filter restored. Four tests fail —
+this one, `a_hierarchy_opens_closed_as_one_fold_per_root` (the 2024 fold reads
+empty), `collapsing_a_parent_folds_its_children_whatever_their_state` and
+`hidden_items_leave_every_fold_above_them`. The fixture adds `2024-03` last, so
+the year and its first quarter are non-contiguous in insertion order.
+
+### a_year_quarter_month_hierarchy_nests_its_runs
+
+improvise-felo.6: the whole marker sequence of a Year > Quarter > Month axis
+opened to depth 2, compared exactly. Rival: every header inside a category
+emitted at depth 0 (a flat hierarchy). This test fails, and so do
+`pruning_keeps_every_surviving_rows_runs_at_every_level` (the `NESTED` fixture),
+`the_nested_fixture_draws_runs_inside_one_category` and
+`pruning_drops_an_empty_inner_group_and_keeps_its_parent`.
+
+### a_hierarchy_opens_closed_as_one_fold_per_root
+
+The user's ruling on improvise-felo.6: groups start closed. Two rivals fail it.
+The old open default (`inherits_collapsed` always false) fails twelve tests
+across `improvise-core` and `improvise-io`, this one among them. The INNERMOST
+collapsed group folding instead of the outermost (`.rev()` on the fold search)
+fails this one, `hidden_items_leave_every_fold_above_them` and
+`the_group_under_the_cursor_is_the_innermost_at_every_depth`: under it a closed
+year shows its quarters as rows.
+
+### a_groups_depth_opens_it_and_n_minus_one_levels_beneath
+
+`expanded(2): Time/2024` opens 2024 and its quarters, not its months. Rival: an
+ancestor reaching `k <= n` levels instead of `k < n`. Only this test fails, which
+is why it exists: the boundary is the whole semantics of N.
+
+### a_toggle_overrides_what_a_group_inherits_either_way_and_toggling_back_clears_it
+
+`z` writes `Collapsed` or `Expanded(1)`, or removes the fold where the flip
+lands on what the group inherits. Rival: never remove. This test and
+`command::cmd::grid::tests::z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again`
+fail; every layout test passes, since a redundant fold draws the same grid. What
+the rival breaks is the file (a line per toggle, forever) and `z z` being the
+identity on the view.
+
+### pruning_drops_an_empty_inner_group_and_keeps_its_parent
+
+`prune_empty` at depth: an empty quarter goes, header and all, and its year's
+header stays for the sibling quarter that has data. Rival: `prune_axis_items`
+keying open runs by level alone. This test and the `NESTED` enumeration fail.

context/test-evidence/formula-and-io.md

@@ -1617,3 +1617,34 @@ that name, which must be refused. It was green on arrival, so its evidence is
 the reviewer's rival, **date-derived names dropped from the taken names**,
 which fails it. That fault would have let `add_category` reuse the existing
 category, and every cell key would then name `when_Year` twice.
+
+### a_group_both_expanded_and_collapsed_is_refused_naming_view_and_group
+
+The user's ruling on improvise-felo.6. Three spellings of one group given two
+folds, each refused as `Refusal::FoldConflict` naming the view and the group,
+plus a category given two depths. Rival: a second fold for a group silently
+wins. Only this test fails. Its control is
+`a_collapsed_group_inside_an_expanded_region_loads`: a DIFFERENT group
+collapsed inside an expanded region, and the same line twice, both load.
+
+### a_cycle_among_group_parents_is_refused_naming_it
+
+`> Q1-2025[|2025|]` and `> |2025|[Q1-2025]`. Rival: the cycle check disabled;
+only this test fails. The fixture's first version spelled `2024-01` bare, which
+the grammar refuses (a bare name starts with a letter), so every test on it
+passed or failed for the parse error, `a_malformed_depth_is_a_parse_error`
+vacuously. It now carries a control that parses.
+
+### an_axis_line_for_a_category_named_expanded_is_still_an_axis_line
+
+`axis_line` is tried before `expanded_line`, so the writer must never spell a
+category depth bare. Rival: the writer writing `expanded: Cat` for depth 1. This
+test fails: the category `row` at depth 1 reads back as an axis line for a
+category named `expanded`.
+
+### an_old_file_with_only_collapsed_lines_loads_closed_and_keeps_its_lines
+
+"Everything opens closed, including old files." Every group of the file opens
+closed, the one its `collapsed:` line names and the three it does not, and the
+line is written back. Rival: the old open default; this test fails with the
+other eleven in that rival's list.

context/test-evidence/protocol.md

@@ -1259,3 +1259,11 @@ the Enter commit.
   - **Rival** (the commit naming `state.view.active_view`, the view at
     `Enter`), run across the whole workspace: fails **only** this test, with
     `["Later"]` against `["Other"]`.
+
+### a_quarters_end_leaves_its_year_open_on_the_same_level
+
+improvise-felo.6: runs inside one category nest by `(level, depth)`. `Direct`
+sits in `2024` after `Q1`'s `GroupEnd { level: 0, depth: 1 }`, and `Undated`
+in nothing after `2024` ends. Rival: `groups()` closing by level alone, as it did
+before depth existed. Only this test fails in `improvise-protocol`: every other
+frame fixture puts one group per level.

context/test-evidence/tui.md

@@ -1429,3 +1429,18 @@ to the list's header row, which no item can cover. This test fills three rows
 with a three-line item and a second item after it. It asserts the first item
 is drawn whole and the header reads `1–1 of 2`. The rival **the header claims
 every item shown** fails it.
+
+### the_two_front_ends_draw_the_same_nested_hierarchy_on_rows
+
+improvise-felo.6: Year > Quarter > Month in one category, under a grouped outer
+category, with a year open two levels, a quarter closed and a year folded. The
+precondition pins the header sequence, so the agreement cannot pass on a frame
+that drew no nested header. What makes the two ends agree is that neither
+derives membership: both walk the same `FrameEntry` list. Rival: the browser's
+`render.rs` skipping a `Group` entry whose `depth > 0`. This test fails on the
+row count (the TUI draws the quarter line, the browser does not); the column
+test does not, since the band shows only each column's innermost group, which
+the rival leaves alone. Group names are short because the TUI
+truncates a band label to its column's width, which the browser does not: the
+first version named its years `Y2024` and the column test failed on `▶ Y2…`
+against `▶ Y2025`, a truncation, not a disagreement.

harness/fixtures/grouped.improv

@@ -11,6 +11,8 @@ _Measure: page, Revenue
 _Index: none
 _Dim: none
 collapsed: Region/Upland
+expanded(1): Month
+expanded(1): Region
 format: ,.0
 
 ## Category: Region

harness/fixtures/tall.improv

@@ -8,6 +8,7 @@ _Dim: none
 _Measure: page
 Region: row
 format: ,.0
+expanded(1): Region
 
 ## View: Tall
 _Index: none
@@ -15,6 +16,7 @@ _Dim: none
 _Measure: column
 Region: row
 format: ,.0
+expanded(1): Region
 
 ## Category: _Measure
 - Revenue, Cost