From 051c2389f6f4d9fca335a49f738909a388a7d1eb Mon Sep 17 00:00:00 2001 From: "Claude (agent-adr007)" Date: Fri, 25 Sep 2026 11:58:41 -0700 Subject: [PATCH 01/27] docs(adr): ADR 007 revision 9, answers 30-32 and the review text fixes (improvise-cq1q.1) - Folds in the user's answers on improvise-m56i (items 2 and 6) and improvise-hexb (item 2) as answers 30-32. The m56i readings (items 1, 3, 4 and 5) go in clause 5 as the formula line. - Applies the planned text fixes: le4p, p003, the i0n7 remainder, hexb items 1, 3 and 4, 8vs2 item 4, the 8tk5 merge sentence, and PR #318's two slugify readings. - Asks one new question, Q23: how a formula line spells its target. - Changes no earlier decision. As an amendment to an accepted ADR, it needs the user's approval. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 199 ++++++++++++++++++++++++--- 1 file changed, 178 insertions(+), 21 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 20c98836..694fa401 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -4,7 +4,12 @@ followed the adversarial review of that head: it adds clause 12, the ADR's own reading of when a formula edit's names become ids, and corrects stale facts about main. The user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17:10Z: -"looks good, approved"). · +"looks good, approved"). +**Revision 9 is a proposed amendment, not yet approved.** It folds in the user's +answers on the implementation tickets (answers 30–32) and clarifies text the +reviews found unclear or stale (improvise-cq1q.1). It changes no earlier decision. +It asks one new question, Q23 (a formula line's spelling), and flags three places +where it states the ADR's own reading. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -321,6 +326,35 @@ data key and reference in that file spells it. Clause 4 says what follows: id keeps its spelling; - two identical hand-written ids in one file are refused at load, not repaired. +### Seventh round: answers on the implementation tickets + +After the ADR was accepted, planning its implementation (epic improvise-cq1q) +raised three questions the ADR does not answer. The user answered them on +2026-09-25. The answers are recorded on the tickets and quoted here verbatim. + +**30. A formula target outside `_Measure`** (improvise-m56i item 2): + +> "Real item + separate formula id" + +`Model::add_formula` keeps creating a real item for a target outside `_Measure`. +That item has its own `ItemId`, the formula has its own `FormulaId`, and by +answer 22 the two differ. A formula line names such a target by the item's id. +This was chosen over the recommended alternative, which was to give every formula +target the `_Measure` treatment (no real item, handled by its `FormulaId`). + +**31. A typed name that is both one thing's display and another thing's id** +(improvise-m56i item 6): + +> "Display first; client holds it" + +A typed name is matched against displays first, then against ids. A display that +matches more than one thing is refused, and the refusal lists the candidates' ids. + +**32. Which side holds clause 12's capture** (improvise-hexb item 2): the +**client**, as it holds `OpenedCell`. It resolves typed names against its +id-bearing projection, the wire carries only id-form text, and the host never +resolves a display name. Clause 12 is rewritten to match. + ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. @@ -341,9 +375,11 @@ first message names what that costs: **What already exists.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize` aliases, assigned from monotone counters. **No production caller uses either value.** -`Model::add_category` and `Category::add_item` return them on the idempotent path, and -nothing reads the returned value; the only reader is one test. Neither has ever been written to `.improv`. `View` has no -id. +`Category::add_item` returns the id on every call, and `Model::add_category` returns +one too. Tests read them (`category.rs`'s `add_item_returns_sequential_ids` and +`add_item_duplicate_returns_same_id`, and one in `types.rs`), but no production +caller does: `parse.rs` discards the value. Neither has ever been written to +`.improv`. `View` has no id. **`FormulaId` exists, as an interim.** km71 landed an opaque `FormulaId` whose private fields are `(target, target_category)`, built only by `Formula::id()` @@ -390,7 +426,23 @@ categories. An operation that creates an item still names the parent category. with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** A `_Measure` formula target has no item of its own. Its handle is its formula's `FormulaId`, which is why the category panel's `_Measure` rows need -`ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). +`ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). **A formula +target outside `_Measure` is a real item** (answer 30). It has its own `ItemId`, +and its formula has a separate `FormulaId`. Generated ids follow clause 4, so a +formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-`. +The panel lists that row as `ItemHandle::Item`, like any other item. + +**The virtual views have reserved ids too:** `_records` and `_drill`, with the +displays `_Records` and `_Drill`. They are never persisted, but they are +addressed on the wire (`SwitchView`, improvise-obtu). Answer 23 reserves a +leading `_` for virtual things, and a slugified `_Drill` would otherwise be +`drill`, which could collide with a user's view (improvise-p003 item 1). + +**The ADR's reading, not a user answer: `_Index` and `_Dim` coordinates are +outside the id namespace.** They are synthetic record positions (`_Dim` carries a +column index, gotcha 1). They are never persisted and never typed, so they take no +part in answer 22's uniqueness, and `_index`'s `0` cannot collide with a user item +whose id is `0` (improvise-p003 item 2). ### 2. Storage keys by id; `display` is the only place a human name lives @@ -402,7 +454,10 @@ Following answers 1 to 3: canonical form is kept with a different sort key. `DataStore` interns ids exactly as it interns names today. - **View configuration:** the axis maps, page selections, and hidden and collapsed sets - are keyed by ids. + are keyed by ids. The one exception is a collapsed entry, which names a group: + groups have no id (answer 9), so it is `(CategoryId, group name)`, and + `collapsed:` writes the category id, then the group name (improvise-p003 + item 4). - **Formulas:** a formula's references are ids (clause 5). The formula panel renders them as display names (answer 19; clause 6). @@ -473,6 +528,16 @@ Examples: `East` → `east`, `Time Period` → `time_period`, `Coastal_East` → 22), the second is `east-`. Here `` is the counter's next value, for example `east-1`. +**Two readings of the steps, the ADR's own and not user answers**, chosen by the +implementation (PR #318; `context/code/core.md` §20): +- **"Space" means U+0020.** Only a run of U+0020 becomes `_`. A tab, a no-break + space (U+00A0) and an ideographic space (U+3000) are not spaces here, so step 3 + strips them: `Time\tPeriod` → `timeperiod`, which collides with `TimePeriod`, + not with `Time Period`. A tab-separated import header is the realistic case. +- **A slug of only `-` is a legal bare id.** `-` and `---` slug to themselves. + Step 5's "if nothing is left" means the empty string only. The id alphabet + admits them, so a line such as `-: row` is legal but hard to read. + The remaining rules: - **A file has ids on everything or on nothing** (answer 21). An id-less file (old @@ -511,6 +576,11 @@ The remaining rules: `east` again. An in-flight command from before the reload is refused by the generation tier. The one remaining risk is an outside reference that outlives the file, and none exists today. +- **Two branches of one file can merge into a file that will not load.** If each + branch adds a different thing whose id generates to the same slug, the merged + file declares that id twice. Its load is refused (answers 11 and 29), and the + error names both declarations, so the merge must be repaired by hand. Stable + identity across copies and branches of a file is not promised (improvise-8tk5). ### 5. Persistence: ids on declarations *and* references @@ -558,6 +628,32 @@ id_ref = { id_token } and takes an `id_ref`. A group reference (`[Coastal]`) stays a name while groups have no id (answer 9). +**An axis line is tried before `format:`** (improvise-le4p). `view_entry` is an +ordered choice, and `format_line` takes the rest of the line, so a category whose +id is `format` would lose its axis line to the view's number format. In the +new-version arm, `axis_line`, which requires an axis keyword after `: `, comes +before `format_line`. No number format is spelled as an axis keyword, so neither +rule loses a line. The id `format` needs no quoting, and `id_ref` keeps no quoting +arm. The required test is a category displayed `Format`, on a row axis, that +round-trips. + +**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). The shape is +`- () = []`: +- **The parenthesised id is the formula's own `FormulaId`** (answer 5's + placement). +- **`` is the target item's id when the target is an item** (outside + `_Measure`, answer 30). For a `_Measure` target, which has no item, it is the + target's display. + - `Formula` carries that display, and a display rename of a `_Measure` target + writes it. + - Whether one line should spell both cases this way is **Q23**, below. +- **The expression is over ids**, spelled through `quote::quote_name` as above. +- **A `WHERE`/`IS` filter value is an item id**, bare or quoted (answer 3: every + reference is an id). The formula panel shows it as a display. +- **A formula cell's `CellKey`** carries the target item's `ItemId` outside + `_Measure`. For a `_Measure` target, it carries the `FormulaId` text under + `_measure`, which answer 22 makes unique. + **`header_display` is guarded only by a lookahead**, which the generator skips (gotcha 34). So the generator may emit a display ending in something that later parses as an id. Round-trip property 3 still holds, because the first parse settles the split. But @@ -646,9 +742,19 @@ pipe-quotes any display name that would otherwise read as display plus id: - `ViewSummary { id, display, … }`. `AxisFrame` labels stay display strings. They gain ids when a consumer needs them. -- **`ViewState` keys by id**: the cursors, `expanded_cats`, and the view history - (`view_back_stack` / `view_forward_stack`). Answer 1 makes this natural rather than a - migration of its own. A view or category rename then orphans nothing. +- **`ViewState` keys by id**: the cursors, `expanded_cats`, the view history + (`view_back_stack` / `view_forward_stack`) and `active_view` itself, as does + `Workbook::active_view` (improvise-p003 item 3). Answer 1 makes this natural + rather than a migration of its own. A view or category rename then orphans + nothing. +- **A `_Measure` formula-target row** (`ItemHandle::FormulaTarget(FormulaId)`): + removing it means `RemoveFormula`, and renaming it means a display rename of + the formula's target (improvise-i0n7). +- **CheckedBatch's created-record exemption goes** (improvise-p003 item 5). It + exempts a record that neither resolves nor is claimed, for frames like + `[AddItem(East), SetCell(East)]`. Under this clause a frame cannot address what + it creates, so nothing produces such a frame any more. The wire step removes + the exemption rather than leaving dead reasoning in place. ### 7. ADR 001: an id that no longer resolves is gone @@ -750,6 +856,14 @@ Following answers 15 and 25: 5. **Projections and `ViewState`** (clause 6). 6. **Vocabulary:** the id-addressed variants, `Rejection::Gone`, `ids_named` and `broken_chain`'s id arm. +7. **Clause 12's capture:** the client-side display→id mapping taken when the + formula editor opens, and the resolver that turns typed names into id-form + text (answers 31 and 32). + +The implementation epic (improvise-cq1q) orders these steps to keep `main` +shippable after each one, which the list above does not: the display field +before the re-key, and projection ids before the re-key. That ordering is the +epic's, not a change to this list. ### 12. A formula edit's names are fixed when the editor opens @@ -772,22 +886,34 @@ is the capture-at-open rule the cell editor already follows: `OpenedCell` in `improvise-protocol/src/client.rs` records the record, its version, the document generation and the view at open, and the commit answers for those rather than for whatever the document holds by then. +- **The client holds the mapping** (answer 32), as it holds `OpenedCell`. The TUI + and the browser both use the same resolver in `improvise-protocol`. - The mapping covers every display name the panel showed in that formula's text, including the target's. -- A name the user types that was **not** in the captured mapping is new to the edit. It - is resolved against the document at commit, as any typed name is. If it does not - resolve, the edit is refused and says which name. -- A captured id that no longer exists at commit is `Rejection::Gone` (clause 7). +- A name the user types that was **not** in the captured mapping is new to the + edit. The client resolves it against its own id-bearing projection (clause 6) at + commit, in answer 31's order: displays first, then ids. If a display matches more + than one thing, the edit is refused and the refusal lists the candidates' ids. If + the name does not resolve at all, the edit is refused and names it. +- **The wire carries only id-form text.** `ReplaceFormula.raw` and + `AddFormula.raw` are in the file's id form, and the host never resolves a display + name (answer 32). The host parses ids, and one that no longer exists is + `Rejection::Gone` (clause 7). - `was` is still compared against the stored id-form text (clause 6), so a concurrent *replace* of the formula is still seen. -**Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. From -another client, rename the item `Revenue` to `Sales` and rename a different item to -`Revenue`. Commit the unedited text. The formula must still reference the original +**Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. +From another client, rename the item `Revenue` to `Sales` and rename a different +item to `Revenue`. Then edit something that is **not a name**: change the text to +`Revenue - Cost * 1` and commit it. The stored formula must reference the original item's id, and the commit must not be refused. The test must fail under commit-time resolution, which binds to the new `Revenue`, and it must fail under a refusal on rename. Both are the rivals this clause rules out. +Revision 8's version committed the *unedited* text. A client that treats an +unchanged text as a no-op passes it without resolving any name, so the test could +not tell a correct capture from no capture at all (improvise-hexb item 1). + ## What this means for you ### improvise-km71 — editing a formula (landed as an interim) @@ -852,7 +978,8 @@ fdk1 landed in PR #221, before this ADR. What exists on main: The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14, revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All -are answered.** +of those are answered.** After acceptance, three implementation questions were +answered on their tickets (answers 30–32). Revision 9 adds Q23, which is open. | # | question | my recommendation | status | |---|---|---|---| @@ -880,6 +1007,10 @@ are answered.** | 21 | An old file naming an undeclared item | (a) declare it at load | **answered: (a)** (answer 24) | | — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal | | 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) | +| — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation | +| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31) | +| — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32) | +| 23 | A formula line's spelling (revision 9) | see "Still open" | **open** | ## Still open @@ -887,8 +1018,33 @@ Nothing is open among Q1–Q22, and answer 29 settles the one case it raises: tw identical hand-written ids are refused at load. **The user approved revision 7. Revision 8 adds clause 12, which is the ADR's own reading, and the user approved it ("looks good, approved", 2026-09-25 00:17:10Z).** Three follow-ups from the same -review stay as tickets, not ADR text: improvise-le4p, improvise-m56i and -improvise-p003. +review were left as tickets rather than ADR text: improvise-le4p, improvise-m56i and +improvise-p003. Revision 9 folds all three in, and improvise-m56i's two +questions are answers 30 and 31. + +**Revision 9 needs the user's approval, and it asks one question.** + +**Q23. How does a formula line spell its target?** Answer 30 makes a target +outside `_Measure` a real item, and says a formula line names it "by the item's +id". A `_Measure` target has no item and so no item id. Clause 5 above therefore +reads the line as `- () = …`, where `` is an item id +outside `_Measure` and a display inside it. That puts two different kinds of +thing in one position, and the category after `[` is what tells them apart. +- **(a) Recommended: as clause 5 reads it.** No extra syntax. The one asymmetry + is that a `_Measure` target's display appears in the file bare, while every + other name is an id. +- **(b) Always a display, and the item id in a second tag:** + `- Tax(tax-1)[tax] = …`. The two positions never mix kinds, but a formula line + outside `_Measure` gains a third bracketed token. +- **(c) Always the target's id**, giving a `_Measure` target an id of its own + that is distinct from the formula's. This reverses clause 1's "its handle is its + formula's `FormulaId`". + +Three places in revision 9 state the ADR's own reading rather than a user +answer. Each is flagged where it appears: +- the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); +- "space" means U+0020; +- a slug of only `-` is a legal id (both in clause 4). ## Consequences @@ -907,8 +1063,9 @@ improvise-p003. save. - `Rejection` loses `Copy`. - `Formula::raw` is no longer verbatim: the file carries formula text over ids. -- The formula panel is a second rendering of formula text, over display names, so the - host must resolve display names typed there back to ids (answer 19). +- The formula panel is a second rendering of formula text, over display names, so + the **client** must resolve display names typed there back to ids (answers 19, + 31 and 32). The host sees only id-form text. - Ids are lower case, so an old file's data keys change spelling on its first save. - ADR 002's prose needs a wording update when this lands. From f6e5145aa5b0d495b454038505441adf9eaf27f4 Mon Sep 17 00:00:00 2001 From: "Claude (agent-adr007)" Date: Fri, 25 Sep 2026 12:14:14 -0700 Subject: [PATCH 02/27] docs(adr): a CellKey coordinate is (CategoryId, ItemHandle) (improvise-cq1q.1) Revision 9 put a FormulaId in a CellKey's item position while clause 2 typed that position ItemId, a contradiction Codex found on PR #326. Clause 2, the formula-cell key in clause 5 and clause 8's RecordHash now use clause 6's own ItemHandle. Answer 22 keeps the id text unambiguous. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 694fa401..a2e385e9 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -450,9 +450,16 @@ Following answers 1 to 3: - **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id, and each `Category`, `Item` and `View` gains a `display: String`. - **Cells and derived surrogates:** a `CellKey` is a sorted list of - `(CategoryId, ItemId)` pairs. It stays sorted, now **by category id**, so gotcha 5's - canonical form is kept with a different sort key. `DataStore` interns ids exactly as it - interns names today. + `(CategoryId, ItemHandle)` pairs, where `ItemHandle` is clause 6's + `{Item(ItemId), FormulaTarget(FormulaId)}`. Outside `_Measure` every coordinate is + an `Item`. A `_Measure` formula cell's coordinate is `FormulaTarget`, because + that target has no item. The key stays sorted, now **by category id**, so gotcha + 5's canonical form is kept with a different sort key. `DataStore` interns ids + exactly as it interns names today. Answer 22 makes every id unique across kinds, + so an id's text alone names one thing; the tag is there so that the types + never claim an `ItemId` the document does not hold. Revision 9's first draft + wrote this pair as `(CategoryId, ItemId)` while also putting a `FormulaId` in it + (Codex on PR #326). - **View configuration:** the axis maps, page selections, and hidden and collapsed sets are keyed by ids. The one exception is a collapsed entry, which names a group: groups have no id (answer 9), so it is `(CategoryId, group name)`, and @@ -650,9 +657,9 @@ round-trips. - **The expression is over ids**, spelled through `quote::quote_name` as above. - **A `WHERE`/`IS` filter value is an item id**, bare or quoted (answer 3: every reference is an id). The formula panel shows it as a display. -- **A formula cell's `CellKey`** carries the target item's `ItemId` outside - `_Measure`. For a `_Measure` target, it carries the `FormulaId` text under - `_measure`, which answer 22 makes unique. +- **A formula cell's `CellKey`** carries `ItemHandle::Item()` + outside `_Measure`. For a `_Measure` target it carries + `ItemHandle::FormulaTarget()` under `_measure` (clause 2). **`header_display` is guarded only by a lookahead**, which the generator skips (gotcha 34). So the generator may emit a display ending in something that later parses as an id. @@ -773,7 +780,11 @@ Per-entity versions are not added (answer 10). ### 8. ADR 002: `RecordHash` is now defined over ids Following answer 3, `RecordHash` hashes the explicit serialization of the sorted -`(CategoryId, ItemId)` text pairs, the same function over different strings. +`(CategoryId, ItemHandle)` pairs, spelled as their id text, the same function +over different strings. The text is enough: answer 22 makes ids unique across +kinds, so no `Item` and `FormulaTarget` can spell alike, and the hash stays +canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate +is never part of a record in the first place. - **ADR 002's decision is unchanged in substance.** It still hashes an explicit serialization of a normalized record, with a fixed key and a width argued for. - **Its prose will be out of date once this lands.** ADR 002 says `CellKey::new` From 24ee72aff979384fe9eb5d2f5304153cbef2d63b Mon Sep 17 00:00:00 2001 From: "Claude (agent-adr007)" Date: Fri, 2 Oct 2026 15:51:58 -0700 Subject: [PATCH 03/27] docs(adr): a format that reads as an axis line is pipe-quoted, and flag every reading (improvise-cq1q.1) The le4p fix said no number format is spelled as an axis keyword. That is false: set-format stores its argument verbatim and the writer emits `format: `, so `:set-format none` writes `format: none`, which the axis-first order would read as the axis line of a category `format`. Clause 5 now states one rule, as the ADR's own reading: the writer pipe-quotes a format that would also match axis_line, and the new arm's format_line unquotes it. Also: - answer 32 quotes the recorded comment whole, and the section says what its quotations are and why authorship in the tracker does not settle them; - the readings that are not user answers are flagged in place and listed under Q23; - clause 11 step 2 lists ItemHandle; - Q23 states how an expression refers to a formula target and that the target position's pipe-quoting covers a `_Measure` display that looks like `Name(x)`. Nothing is renumbered, and no earlier decision changes. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 120 +++++++++++++++++++-------- 1 file changed, 86 insertions(+), 34 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index a2e385e9..7b88e513 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -329,8 +329,14 @@ data key and reference in that file spells it. Clause 4 says what follows: ### Seventh round: answers on the implementation tickets After the ADR was accepted, planning its implementation (epic improvise-cq1q) -raised three questions the ADR does not answer. The user answered them on -2026-09-25. The answers are recorded on the tickets and quoted here verbatim. +raised three questions the ADR does not answer. They were answered on +2026-09-25, and the answers are recorded as comments on the tickets +(improvise-m56i and improvise-hexb). The tracker shows the repository owner's +account as the author of those comments, and agents write to it under the same +account, so authorship there does not show whose words they are. The user's +"Approved" on this PR is what confirms them. Answers 30 and 31 quote the option +chosen, as recorded, and the sentences after each are the ADR's own statement of +what it means. Answer 32 was recorded as one sentence and is quoted whole. **30. A formula target outside `_Measure`** (improvise-m56i item 2): @@ -350,10 +356,14 @@ target the `_Measure` treatment (no real item, handled by its `FormulaId`). A typed name is matched against displays first, then against ids. A display that matches more than one thing is refused, and the refusal lists the candidates' ids. -**32. Which side holds clause 12's capture** (improvise-hexb item 2): the -**client**, as it holds `OpenedCell`. It resolves typed names against its -id-bearing projection, the wire carries only id-form text, and the host never -resolves a display name. Clause 12 is rewritten to match. +**32. Which side holds clause 12's capture** (improvise-hexb item 2), as recorded: + +> "USER DECISION 2026-09-25 (item 2): the CLIENT holds clause 12's display->id +> capture (like OpenedCell) and resolves names against its id-bearing +> projection; the wire carries only id-form text; the host never resolves a +> display name." + +Clause 12 is rewritten to match. ## Context @@ -432,11 +442,12 @@ and its formula has a separate `FormulaId`. Generated ids follow clause 4, so a formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-`. The panel lists that row as `ItemHandle::Item`, like any other item. -**The virtual views have reserved ids too:** `_records` and `_drill`, with the -displays `_Records` and `_Drill`. They are never persisted, but they are -addressed on the wire (`SwitchView`, improvise-obtu). Answer 23 reserves a -leading `_` for virtual things, and a slugified `_Drill` would otherwise be -`drill`, which could collide with a user's view (improvise-p003 item 1). +**The virtual views have reserved ids too** (the ADR's own reading, not a user +answer): `_records` and `_drill`, with the displays `_Records` and `_Drill`. They +are never persisted, but they are addressed on the wire (`SwitchView`, +improvise-obtu). Answer 23 reserves a leading `_` for the virtual *categories*, +and this extends it to the virtual views, because a slugified `_Drill` would +otherwise be `drill`, which could collide with a user's view (improvise-p003 item 1). **The ADR's reading, not a user answer: `_Index` and `_Dim` coordinates are outside the id namespace.** They are synthetic record positions (`_Dim` carries a @@ -450,7 +461,8 @@ Following answers 1 to 3: - **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id, and each `Category`, `Item` and `View` gains a `display: String`. - **Cells and derived surrogates:** a `CellKey` is a sorted list of - `(CategoryId, ItemHandle)` pairs, where `ItemHandle` is clause 6's + `(CategoryId, ItemHandle)` pairs (the ADR's own reading of answer 30, not a + user answer), where `ItemHandle` is clause 6's `{Item(ItemId), FormulaTarget(FormulaId)}`. Outside `_Measure` every coordinate is an `Item`. A `_Measure` formula cell's coordinate is `FormulaTarget`, because that target has no item. The key stays sorted, now **by category id**, so gotcha @@ -463,8 +475,8 @@ Following answers 1 to 3: - **View configuration:** the axis maps, page selections, and hidden and collapsed sets are keyed by ids. The one exception is a collapsed entry, which names a group: groups have no id (answer 9), so it is `(CategoryId, group name)`, and - `collapsed:` writes the category id, then the group name (improvise-p003 - item 4). + `collapsed:` writes the category id, then the group name (the ADR's own + reading, not a user answer; improvise-p003 item 4). - **Formulas:** a formula's references are ids (clause 5). The formula panel renders them as display names (answer 19; clause 6). @@ -635,16 +647,36 @@ id_ref = { id_token } and takes an `id_ref`. A group reference (`[Coastal]`) stays a name while groups have no id (answer 9). -**An axis line is tried before `format:`** (improvise-le4p). `view_entry` is an -ordered choice, and `format_line` takes the rest of the line, so a category whose -id is `format` would lose its axis line to the view's number format. In the -new-version arm, `axis_line`, which requires an axis keyword after `: `, comes -before `format_line`. No number format is spelled as an axis keyword, so neither -rule loses a line. The id `format` needs no quoting, and `id_ref` keeps no quoting -arm. The required test is a category displayed `Format`, on a row axis, that -round-trips. +**An axis line is tried before `format:`, and a format that reads as an axis line is +pipe-quoted** (improvise-le4p; the ADR's own reading, not a user answer). +`view_entry` is an ordered choice, and `format_line` takes the rest of the line, +so a category whose id is `format` would lose its axis line to the view's number +format. In the new-version arm, `axis_line` therefore comes before `format_line`. +That moves the collision to the other side. `set-format` stores its argument +verbatim and the writer emits `format: {number_format}`, so `:set-format none` +writes `format: none`, which would now read as the axis line of a category +`format`: the format is lost, or the id is unknown and the load is refused +(answer 11), so a document that saves would not load. -**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). The shape is +The rule is that **the writer pipe-quotes a number format whenever the line would +also match `axis_line`**, that is, when the value is an axis keyword (`row`, +`column`, `page`, `filter` or `none`) alone or followed by `, `, or when it +begins with `|`. The new arm's `format_line` takes +`pipe_quoted ~ &NEWLINE | rest_of_line`, as `header_name` does (gotcha 34). So +`format: |none|` is the format `none`, and nothing is refused. An old-version +file's `format: none` is read as before, because the old arm keeps `format_line` +first, and it is written quoted on its first save. The alternatives were to refuse +such a format in `set-format`, which does nothing for a file that already holds +one, or in the writer, which leaves a document that cannot be saved. The id +`format` needs no quoting, and `id_ref` keeps no quoting arm. + +Required tests: a category displayed `Format`, on a row axis, round-trips; and a +view whose number format is each axis keyword round-trips, in a document where +the same view also holds a category `format` on an axis. + +**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). The shape is the ADR's +own reading, not a user answer, except that the formula id's place is answer 5's and +an item target's id is answer 30's. It is `- () = []`: - **The parenthesised id is the formula's own `FormulaId`** (answer 5's placement). @@ -652,8 +684,9 @@ round-trips. `_Measure`, answer 30). For a `_Measure` target, which has no item, it is the target's display. - `Formula` carries that display, and a display rename of a `_Measure` target - writes it. - - Whether one line should spell both cases this way is **Q23**, below. + writes it. Where that display lives is also the ADR's own reading. + - Whether one line should spell both cases this way is **Q23**, below, and so + are how an expression refers to a target and how a `_Measure` display is quoted. - **The expression is over ids**, spelled through `quote::quote_name` as above. - **A `WHERE`/`IS` filter value is an item id**, bare or quoted (answer 3: every reference is an id). The formula panel shows it as a display. @@ -757,7 +790,8 @@ pipe-quotes any display name that would otherwise read as display plus id: - **A `_Measure` formula-target row** (`ItemHandle::FormulaTarget(FormulaId)`): removing it means `RemoveFormula`, and renaming it means a display rename of the formula's target (improvise-i0n7). -- **CheckedBatch's created-record exemption goes** (improvise-p003 item 5). It +- **CheckedBatch's created-record exemption goes** (the ADR's own reading, not a + user answer; improvise-p003 item 5). It exempts a record that neither resolves nor is claimed, for frames like `[AddItem(East), SetCell(East)]`. Under this clause a frame cannot address what it creates, so nothing produces such a frame any more. The wire step removes @@ -857,8 +891,9 @@ Following answers 15 and 25: ### 11. Migration 1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224. -2. **Types:** the id newtypes over `Symbol`, the `display` fields, and `FormulaId` and - `ViewId`. +2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId`, + `ViewId`, and `ItemHandle`, which the re-key (step 3) needs for `CellKey` before + clause 6's step (5) uses it in projections. 3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with display→id resolution moved to every input edge. 4. **Persistence:** the new version literal (clause 10), the grammar in clause 5, the @@ -1045,17 +1080,34 @@ thing in one position, and the category after `[` is what tells them apart. is that a `_Measure` target's display appears in the file bare, while every other name is an id. - **(b) Always a display, and the item id in a second tag:** - `- Tax(tax-1)[tax] = …`. The two positions never mix kinds, but a formula line - outside `_Measure` gains a third bracketed token. + `- Tax(tax-1)(tax) [customcat] = …`. The two positions never mix kinds, but a + formula line outside `_Measure` gains a second parenthesised tag. - **(c) Always the target's id**, giving a `_Measure` target an id of its own that is distinct from the formula's. This reverses clause 1's "its handle is its formula's `FormulaId`". -Three places in revision 9 state the ADR's own reading rather than a user -answer. Each is flagged where it appears: +Two consequences of (a) that the answer should take in, both the ADR's own reading: +- **References to a formula target.** An expression, and a `WHERE`/`IS` value, + names a target outside `_Measure` by its **item id**, and a `_Measure` target by + its **formula id**. That follows from `ItemHandle` (clause 2). +- **Quoting the target position.** A `_Measure` display in the target position can + itself look like `Name(x)`, which would read back as a display and a formula id. + The writer's rule that pipe-quotes such a display therefore covers a + formula-line target too: `- |Margin(%)|(margin) = …`. An item id never contains + a parenthesis, so only a `_Measure` display can need it. + +Revision 9 states these readings of its own, which are not user answers. Each is +flagged where it appears, and approving the PR approves them: +- the virtual views' reserved ids `_records` and `_drill` (clause 1); - the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); -- "space" means U+0020; -- a slug of only `-` is a legal id (both in clause 4). +- a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2); +- `collapsed:` is `(CategoryId, group name)` (clause 2); +- "space" means U+0020, and a slug of only `-` is a legal id (clause 4); +- the axis-before-`format:` order and the quoting of a format that reads as an axis + line (clause 5); +- the formula-line shape, and a `_Measure` target's display living on the `Formula` + (clause 5); +- CheckedBatch's created-record exemption is removed (clause 6). ## Consequences From f786f835aca659d61d4439c55a1c6d767c8f1c65 Mon Sep 17 00:00:00 2001 From: "Claude (agent-adr007)" Date: Fri, 2 Oct 2026 16:24:05 -0700 Subject: [PATCH 04/27] docs(adr): one rule for a reference to a formula target, and the right count of readings (improvise-cq1q.1) Copilot found two inconsistencies in the previous push: - clause 5 said a WHERE/IS value is an item id while Q23 said it could be a formula id. Clause 5 now states one rule, the id text of an ItemHandle, for an expression and a filter value alike, and Q23 points to it instead of restating it; - the status header still said three readings were flagged, where Q23 lists eight. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 7b88e513..b6ca9e81 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -8,8 +8,8 @@ user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17: **Revision 9 is a proposed amendment, not yet approved.** It folds in the user's answers on the implementation tickets (answers 30–32) and clarifies text the reviews found unclear or stale (improvise-cq1q.1). It changes no earlier decision. -It asks one new question, Q23 (a formula line's spelling), and flags three places -where it states the ADR's own reading. · +It asks one new question, Q23 (a formula line's spelling), and flags eight readings +of its own that are not user answers, listed under Q23. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -687,9 +687,13 @@ an item target's id is answer 30's. It is writes it. Where that display lives is also the ADR's own reading. - Whether one line should spell both cases this way is **Q23**, below, and so are how an expression refers to a target and how a `_Measure` display is quoted. -- **The expression is over ids**, spelled through `quote::quote_name` as above. -- **A `WHERE`/`IS` filter value is an item id**, bare or quoted (answer 3: every - reference is an id). The formula panel shows it as a display. +- **The expression is over ids**, spelled through `quote::quote_name` as above. A + reference to a formula target is the id text of its `ItemHandle`: the item id + outside `_Measure`, and the formula id for a `_Measure` target. +- **A `WHERE`/`IS` filter value follows the same rule** (answer 3: every reference + is an id). It is the id text of an `ItemHandle` in the filtered category, bare or + quoted: an item id, or a formula id when the category is `_Measure` and the value + names a formula target. The formula panel shows it as a display. - **A formula cell's `CellKey`** carries `ItemHandle::Item()` outside `_Measure`. For a `_Measure` target it carries `ItemHandle::FormulaTarget()` under `_measure` (clause 2). @@ -1087,9 +1091,10 @@ thing in one position, and the category after `[` is what tells them apart. formula's `FormulaId`". Two consequences of (a) that the answer should take in, both the ADR's own reading: -- **References to a formula target.** An expression, and a `WHERE`/`IS` value, - names a target outside `_Measure` by its **item id**, and a `_Measure` target by - its **formula id**. That follows from `ItemHandle` (clause 2). +- **References to a formula target** are as clause 5 states them, for an expression + and a filter value alike: the id text of an `ItemHandle`, which is the item id + outside `_Measure` and the formula id for a `_Measure` target. That holds under + (a) and (b). Under (c) a `_Measure` target would be named by its own id instead. - **Quoting the target position.** A `_Measure` display in the target position can itself look like `Name(x)`, which would read back as a display and a formula id. The writer's rule that pipe-quotes such a display therefore covers a From 4aed8c071f8aab2bf4e832206592b4ade71d57e0 Mon Sep 17 00:00:00 2001 From: "Claude (agent-adr007)" Date: Fri, 2 Oct 2026 17:26:42 -0700 Subject: [PATCH 05/27] docs(adr): put option (b)'s category tag after the expression, and count places (improvise-cq1q.1) Option (b)'s example had `[customcat]` before the `=`, while clause 5's formula-line shape puts the category tag after the expression. It was the only copy of the slip. The status header and the list under Q23 now say eight places, since three of the eight bullets each hold two readings. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index b6ca9e81..d1151945 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -8,8 +8,8 @@ user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17: **Revision 9 is a proposed amendment, not yet approved.** It folds in the user's answers on the implementation tickets (answers 30–32) and clarifies text the reviews found unclear or stale (improvise-cq1q.1). It changes no earlier decision. -It asks one new question, Q23 (a formula line's spelling), and flags eight readings -of its own that are not user answers, listed under Q23. · +It asks one new question, Q23 (a formula line's spelling), and flags eight places +where it states a reading of its own that is not a user answer, listed under Q23. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -1084,7 +1084,7 @@ thing in one position, and the category after `[` is what tells them apart. is that a `_Measure` target's display appears in the file bare, while every other name is an id. - **(b) Always a display, and the item id in a second tag:** - `- Tax(tax-1)(tax) [customcat] = …`. The two positions never mix kinds, but a + `- Tax(tax-1)(tax) = … [customcat]`. The two positions never mix kinds, but a formula line outside `_Measure` gains a second parenthesised tag. - **(c) Always the target's id**, giving a `_Measure` target an id of its own that is distinct from the formula's. This reverses clause 1's "its handle is its @@ -1101,8 +1101,9 @@ Two consequences of (a) that the answer should take in, both the ADR's own readi formula-line target too: `- |Margin(%)|(margin) = …`. An item id never contains a parenthesis, so only a `_Measure` display can need it. -Revision 9 states these readings of its own, which are not user answers. Each is -flagged where it appears, and approving the PR approves them: +Revision 9 states a reading of its own, which is not a user answer, in each of these +eight places (three of them hold two readings). Each is flagged where it appears, and +approving the PR approves them: - the virtual views' reserved ids `_records` and `_drill` (clause 1); - the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); - a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2); From e9659903d80d13bb2e85e873c9d011949d652ede Mon Sep 17 00:00:00 2001 From: "Claude (adr7-steward)" Date: Mon, 5 Oct 2026 13:28:07 -0700 Subject: [PATCH 06/27] docs(adr): fold answers 33 and 34 into ADR 007 (improvise-cq1q.1) Answer 33: typed names match ids first until a view-layer autocomplete exists, reversing answer 31's order. Answer 34: "space" means whitespace and a run of it becomes one underscore, overriding the ADR's U+0020 reading. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 162 ++++++++++++++++++++++----- 1 file changed, 132 insertions(+), 30 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index d1151945..f7c3522b 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -6,10 +6,13 @@ of when a formula edit's names become ids, and corrects stale facts about main. user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17:10Z: "looks good, approved"). **Revision 9 is a proposed amendment, not yet approved.** It folds in the user's -answers on the implementation tickets (answers 30–32) and clarifies text the -reviews found unclear or stale (improvise-cq1q.1). It changes no earlier decision. -It asks one new question, Q23 (a formula line's spelling), and flags eight places -where it states a reading of its own that is not a user answer, listed under Q23. · +answers on the implementation tickets (answers 30–32), the user's two inline +comments on its head `4aed8c07` and a clarification in chat (answers 33 and 34), +and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 +reverses the matching order that answer 31 recorded, and answer 34 overrides the +ADR's own reading of "space". It asks one new question, Q23 (a formula line's +spelling), and flags nine places where it states a reading of its own that is not +a user answer, listed under Q23. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -224,7 +227,8 @@ run of spaces collapse to one `_`?"): > all or always have IDs)" This decides three things: -- a run of spaces collapses to one `_`, as recommended; +- a run of spaces collapses to one `_`, as recommended (a run of whitespace, by + answer 34); - a duplicate slug gets a numeric suffix after a `-`; - **a file has ids on everything or on nothing.** A new-version file with a declaration missing its id is refused, like any file that cannot be resaved safely (answer 11). @@ -353,8 +357,12 @@ target the `_Measure` treatment (no real item, handled by its `FormulaId`). > "Display first; client holds it" -A typed name is matched against displays first, then against ids. A display that -matches more than one thing is refused, and the refusal lists the candidates' ids. +The option is quoted as recorded. **The order it names was reversed by the user's own +comment on this answer (answer 33): until autocomplete exists, a typed name is +matched against ids first, then against displays.** What stands from answer 31 is +that the client holds the resolution (answer 32), and that a display matching more +than one thing is refused, with the refusal listing the candidates' ids. Where +answers 31 and 33 differ, answer 33 governs. **32. Which side holds clause 12's capture** (improvise-hexb item 2), as recorded: @@ -365,6 +373,43 @@ matches more than one thing is refused, and the refusal lists the candidates' id Clause 12 is rewritten to match. +### Eighth round: answers on revision 9 + +The user answered revision 9 (`context/adr/007-stable-identities.md` at `4aed8c07`) +with two inline comments on PR #326, posted 2026-10-04 02:48Z and 02:49Z, and then +clarified the second one in chat (not on the PR, relayed to the steward). Each is +mapped to the line it is attached to, and quoted verbatim. + +**33. The order of answer 31.** On line 356, the sentence under answer 31, "A typed +name is matched against displays first, then against ids": + +> "This should be handled by separating the view layer autocomplete of typed display names from the model layer reference to IDs: but, before we have autocomplete, IDs are matched first." + +This decides two things: +- **The long-term design.** A view-layer autocomplete resolves typed display text to + ids. The model layer refers to ids only, and never matches a display. +- **The interim rule, until autocomplete exists.** A typed name is matched against + **ids first, then displays.** This reverses answer 31's "displays first". + +Clause 12 states the rule and what it means for a document where one thing's display +equals another's id. Nothing built yet implements either order (clause 11, step 7). + +**34. What "space" means in slugify.** On line 553, the bullet that read "Space" +as U+0020 only, so that a tab was stripped and `Time\tPeriod` slugged to +`timeperiod`: + +> "This shouldn’t matter because consecutive spaces should normalize to a single underscore" + +and, in chat: + +> "space should generally mean "whitespace"" + +This **overrides the ADR's own reading** (revision 9's, which was not a user answer). +"Space" in answers 18 and 21 means whitespace, and a run of whitespace becomes one +`_`. The user did not enumerate which characters count, so the definition in clause +4 is **the ADR's own reading of "whitespace"**, flagged there. Answers 18 and 21 +keep their wording, which this reads as "space, meaning whitespace". + ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. @@ -496,8 +541,9 @@ be renaming field access"), and for the field that is right. The call sites are cost. **Display names stay unique within their scope**, as names are today. So a human-typed -name resolves to exactly one id, and a rename to a display name already in use is -refused. An item's scope is its category, so two categories may each have an item +display resolves to at most one id within a scope, and a rename to a display name +already in use is refused. Typed text is matched against ids before displays +(answer 33; clause 12). An item's scope is its category, so two categories may each have an item displayed `Revenue`. Formula text crosses categories, which is where answer 19's "ambiguous" arises. Their ids differ (answer 22), so the panel resolves the ambiguity by showing the ids. @@ -527,8 +573,9 @@ space with `_`, strip"): `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped), so this needs a case-folding table or a small crate in `improvise-core`. It must pass the `wasm` gate, because that crate is in the browser bundle. -2. **Replace each run of spaces with one `_`** (answers 18 and 21; answer 18 changed - answer 8's `-`). +2. **Replace each run of whitespace with one `_`** (answers 18, 21 and 34; answer 18 + changed answer 8's `-`). A run is any unbroken sequence of whitespace + characters, whatever mix, so it becomes one `_`, never one per character. 3. **Strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`** (answers 16(c) and 16(d)). This strips non-ASCII letters, so an id never needs quoting in an `.improv` reference (clause 5 notes where formula text differs). @@ -547,16 +594,36 @@ Examples: `East` → `east`, `Time Period` → `time_period`, `Coastal_East` → 22), the second is `east-`. Here `` is the counter's next value, for example `east-1`. -**Two readings of the steps, the ADR's own and not user answers**, chosen by the -implementation (PR #318; `context/code/core.md` §20): -- **"Space" means U+0020.** Only a run of U+0020 becomes `_`. A tab, a no-break - space (U+00A0) and an ideographic space (U+3000) are not spaces here, so step 3 - strips them: `Time\tPeriod` → `timeperiod`, which collides with `TimePeriod`, - not with `Time Period`. A tab-separated import header is the realistic case. +**Two readings of the steps, the ADR's own and not user answers.** The first is +the ADR's reading of the word "whitespace", which answer 34 requires and does not +define. The second was chosen by the implementation (PR #318; `context/code/core.md` +§20): +- **"Whitespace" means the Unicode `White_Space` property**, the set Rust's + `char::is_whitespace` tests. That is U+0009–U+000D, U+0020, U+0085, U+00A0, + U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. It needs no + table in `improvise-core`. A tab, a no-break space (U+00A0) and an ideographic + space (U+3000) are whitespace, so each becomes `_` like a space does: + `Time\tPeriod`, `Time\u{A0}Period` and `Time Period` all slug to `time_period`, + and `Time \t Period`, one run of three characters, does too. A zero-width space + (U+200B) and a byte-order mark (U+FEFF) are **not** `White_Space`, so step 3 + strips them. The realistic case is a header pasted from a web page, where a + no-break space stands for a space. The ADR chose this set because it is the standard + definition and the user asked for "whitespace", not for a list. The user may + prefer a narrower one (ASCII whitespace only, say), and that is a one-line change. + An underscore the display already had is still never merged with the one a run + makes (`a_ b` → `a__b`), because the user said *consecutive spaces* collapse, not + consecutive underscores. - **A slug of only `-` is a legal bare id.** `-` and `---` slug to themselves. Step 5's "if nothing is left" means the empty string only. The id alphabet admits them, so a line such as `-: row` is legal but hard to read. +**What this changes in built code (a follow-up, not made here).** PR #318's +`slugify` (`crates/improvise-core/src/model/id.rs`, `underscore_space_runs`) tests +`c == ' '`, and `context/code/core.md` §20 states the U+0020-only reading as a +rule. So today `Time\tPeriod` slugs to `timeperiod`. Answer 34 makes that wrong, and +the implementation needs a ticket to test whitespace instead and to repair §20 and +its tests. Nothing calls `slugify` yet (§20), so no stored id is affected. + The remaining rules: - **A file has ids on everything or on nothing** (answer 21). An id-less file (old @@ -778,7 +845,10 @@ pipe-quotes any display name that would otherwise read as display plus id: - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows (answer 19): the target's display name with its id in parentheses, then the expression over display names, with an id wherever a display name is ambiguous - (ids are unique, answer 22). `stored` is the id-form text the file carries, and it is what + (ids are unique, answer 22). A display that equals another thing's id is + ambiguous in this sense (the ADR's own reading of answers 19 and 33, flagged + under Q23), so the panel never shows a bare name that the ids-first rule would + bind elsewhere. `stored` is the id-form text the file carries, and it is what `ReplaceFormula`'s `was` is compared against. So a rename of `Revenue` elsewhere changes what the panel shows but does not refuse an edit of `Profit`, which is the "avoid glitches" of answer 3. What makes that hold for an edit already open @@ -908,7 +978,9 @@ Following answers 15 and 25: `broken_chain`'s id arm. 7. **Clause 12's capture:** the client-side display→id mapping taken when the formula editor opens, and the resolver that turns typed names into id-form - text (answers 31 and 32). + text (answers 32 and 33). Neither the resolver nor any display-matching code + exists yet, so the interim order is built ids first from the start and nothing + has to be migrated. The autocomplete that retires it is not in this list. The implementation epic (improvise-cq1q) orders these steps to keep `main` shippable after each one, which the list above does not: the display field @@ -942,9 +1014,17 @@ whatever the document holds by then. including the target's. - A name the user types that was **not** in the captured mapping is new to the edit. The client resolves it against its own id-bearing projection (clause 6) at - commit, in answer 31's order: displays first, then ids. If a display matches more - than one thing, the edit is refused and the refusal lists the candidates' ids. If - the name does not resolve at all, the edit is refused and names it. + commit, in answer 33's order: **ids first, then displays**. A typed name equal to + an id names that id's thing, even where it is also another thing's display. + Otherwise, if exactly one thing in scope has that display, it names that thing. If + a display matches more than one thing, the edit is refused and the refusal lists + the candidates' ids. If the name does not resolve at all, the edit is refused and + names it. An id is matched exactly: no case folding. +- **The ids-first order is interim** (answer 33). The long-term design separates two + layers. A view-layer autocomplete of typed display names turns text into ids + before it is committed, and the model layer refers to ids only. When it exists, + this resolver's display stage and the ids-first order go with it. This ADR does not + specify the autocomplete. - **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are in the file's id form, and the host never resolves a display name (answer 32). The host parses ids, and one that no longer exists is @@ -952,6 +1032,16 @@ whatever the document holds by then. - `was` is still compared against the stored id-form text (clause 6), so a concurrent *replace* of the formula is still seen. +**What the interim order means for a document where an item's display equals another +item's id.** Item A is displayed `East` and has id `east`. Item B is displayed `east`, +so slugify finds `east` taken and gives it `east-1` (clause 4). Typing `east` names +**A**, because the id wins. B is reached by typing `east-1`, or by autocomplete once +it exists. A user who typed `east` meaning B is bound to A without being told, which +is the cost of the interim order and the reason the panel shows B as `east-1`, not as +a bare `east` (clause 6). The captured mapping cannot disagree with the rule, since +it holds only names the panel showed bare, and the panel shows none that ids-first +would bind elsewhere. + **Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. From another client, rename the item `Revenue` to `Sales` and rename a different item to `Revenue`. Then edit something that is **not a name**: change the text to @@ -960,6 +1050,13 @@ item's id, and the commit must not be refused. The test must fail under commit-time resolution, which binds to the new `Revenue`, and it must fail under a refusal on rename. Both are the rivals this clause rules out. +**Required test for the interim order** (answer 33). In a document with items A +(display `East`, id `east`) and B (display `east`, id `east-1`), type `east` into a +formula editor, in text not captured at open. The stored formula must reference A. +The test must fail under displays-first, which binds B. Typing `east-1` must bind B, +and typing a display that is nobody's id must still resolve by display. A fixture +where no display equals an id passes under either order and proves nothing. + Revision 8's version committed the *unedited* text. A client that treats an unchanged text as a no-op passes it without resolving any name, so the test could not tell a correct capture from no capture at all (improvise-hexb item 1). @@ -1029,7 +1126,8 @@ fdk1 landed in PR #221, before this ADR. What exists on main: The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14, revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All of those are answered.** After acceptance, three implementation questions were -answered on their tickets (answers 30–32). Revision 9 adds Q23, which is open. +answered on their tickets (answers 30–32), and the user's comments on revision 9 are +answers 33 and 34. Revision 9 adds Q23, which is open. | # | question | my recommendation | status | |---|---|---|---| @@ -1047,10 +1145,10 @@ answered on their tickets (answers 30–32). Revision 9 adds Q23, which is open. | 12 | Old files and the new header rule | (a) no | **answered: (a), with the version marked unambiguously** (answer 15) | | 13 | Slugify details | all generation; keep case; keep `_` `-`; strip non-ASCII; `-` | **answered** (answer 16): all generation, **lower case only** (reversing the recommendation), keep both, strip, `-` | | 14 | Formula text: ids or display names? | ids in the file | **answered**: ids in the file (answer 3); the panel is Q16 | -| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-` | +| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-`, and "space" means whitespace (answer 34) | | 16 | What the formula panel shows | none | **answered: display names, target shown as `Profit (profit)`, ids where a name is ambiguous** (answer 19) | | 17 | Capitals in a hand-written id | (a) no | **answered: no** (answer 20) | -| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse**, plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | +| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse** (a run of whitespace, answer 34), plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | | 18(b) | What "casefold" means | full case folding | **answered: full Unicode case folding** (answer 26) | | 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answer 22), stronger than (a) | | 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23) | @@ -1058,7 +1156,7 @@ answered on their tickets (answers 30–32). Revision 9 adds Q23, which is open. | — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal | | 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) | | — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation | -| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31) | +| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31). **Order reversed: ids first until there is autocomplete** (answer 33) | | — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32) | | 23 | A formula line's spelling (revision 9) | see "Still open" | **open** | @@ -1102,18 +1200,21 @@ Two consequences of (a) that the answer should take in, both the ADR's own readi a parenthesis, so only a `_Measure` display can need it. Revision 9 states a reading of its own, which is not a user answer, in each of these -eight places (three of them hold two readings). Each is flagged where it appears, and +nine places (three of them hold two readings). Each is flagged where it appears, and approving the PR approves them: - the virtual views' reserved ids `_records` and `_drill` (clause 1); - the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); - a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2); - `collapsed:` is `(CategoryId, group name)` (clause 2); -- "space" means U+0020, and a slug of only `-` is a legal id (clause 4); +- "whitespace" means the Unicode `White_Space` property (replacing the U+0020 reading, + by answer 34), and a slug of only `-` is a legal id (clause 4); - the axis-before-`format:` order and the quoting of a format that reads as an axis line (clause 5); - the formula-line shape, and a `_Measure` target's display living on the `Formula` (clause 5); -- CheckedBatch's created-record exemption is removed (clause 6). +- CheckedBatch's created-record exemption is removed (clause 6); +- a display equal to another thing's id counts as ambiguous in the formula panel + (clause 6). ## Consequences @@ -1134,7 +1235,8 @@ approving the PR approves them: - `Formula::raw` is no longer verbatim: the file carries formula text over ids. - The formula panel is a second rendering of formula text, over display names, so the **client** must resolve display names typed there back to ids (answers 19, - 31 and 32). The host sees only id-form text. + 32 and 33), ids first until an autocomplete exists. The host sees only id-form + text. - Ids are lower case, so an old file's data keys change spelling on its first save. - ADR 002's prose needs a wording update when this lands. From 5385bb36bc2d007080ca0425eb7e212be1e0d4fa Mon Sep 17 00:00:00 2001 From: "Claude (adr7-steward)" Date: Mon, 5 Oct 2026 13:31:14 -0700 Subject: [PATCH 07/27] docs(adr): fold answers 35 and 36, reopen Q23, add Q24 and Q25 (improvise-cq1q.1) Measures get the id/display split and are the normal formula target; dimension targets are rare. The mechanism is left to Q23 as options. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 162 ++++++++++++++++++++------- 1 file changed, 123 insertions(+), 39 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index f7c3522b..b6776588 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -8,11 +8,14 @@ user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17: **Revision 9 is a proposed amendment, not yet approved.** It folds in the user's answers on the implementation tickets (answers 30–32), the user's two inline comments on its head `4aed8c07` and a clarification in chat (answers 33 and 34), -and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 +its two inline comments on `e9659903` (answers 35 and 36), and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 reverses the matching order that answer 31 recorded, and answer 34 overrides the -ADR's own reading of "space". It asks one new question, Q23 (a formula line's -spelling), and flags nine places where it states a reading of its own that is not -a user answer, listed under Q23. · +ADR's own reading of "space". Answers 35 and 36 make measures the normal formula +target and give them an id/display split, which reopens Q23. It asks three new +questions, Q23 (how a formula-target measure carries its id, and how a formula line +spells its target), Q24 and Q25 (two details of slugify's whitespace rule), and +flags nine places where it states a reading of its own that is not a user answer, +listed under Q23. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -410,6 +413,42 @@ This **overrides the ADR's own reading** (revision 9's, which was not a user ans 4 is **the ADR's own reading of "whitespace"**, flagged there. Answers 18 and 21 keep their wording, which this reads as "space, meaning whitespace". +### Ninth round: answers on revision 9's head `e9659903` + +The user posted two inline comments on PR #326 at 2026-10-05 20:22:52Z and 20:23:40Z, +both on the line holding Q23's option (c). Each is quoted verbatim. + +**35. Measures get the id/display split too.** + +> "Measures should have ID/display split too. Measures are the normal case for formulas, anyways: something like `profit = revenue - cost`" + +**36. Dimensions are rarely formula targets.** + +> "Dimensions are mainly in formulas when you're calculating some kind of derived dimension, but usually dimensions should be relatively static and not interesting in formulas." + +These decide two things, and no more: +- **A measure has an id and a display**, as every other thing does. +- **The normal formula target is a measure** (`profit = revenue - cost`), and a + dimension (a non-`_Measure` category) target is the rare, derived case. + +They do **not** decide what a formula-target measure's id is, how its formula line +names it, or whether it is an item. That is **Q23**, reopened and reworded. Four +things in earlier text meet them, and none is resolved here: +- **Clause 1 and revision 9's flagged readings (clauses 1, 2, 5, 8)** say a `_Measure` + formula target "has no item of its own" and is handled by `FormulaTarget(FormulaId)`. + A stored measure (a non-formula `_Measure` item) is an ordinary item and already has + an id and a display. A measure that exists only because a formula targets it has + its display on the `Formula` and its id in the `FormulaId`. Whether that is the + split the user wants, or whether the target should be an item, is Q23. +- **Answer 3** ("every reference is an id") sits badly with revision 9's line for the + normal case, where the target position holds a bare display. +- **Answers 5 and 19** picture the measure case as a display plus one id + (`Profit(3)`, `Profit (profit)`), which fits an id/display split. +- **Answer 30** and the ADR's worked example (`Tax` in `CustomCat`) are about the + *rare* case. The decision stands. The emphasis of the surrounding text changed: + revision 9 wrote the dimension target as the main case and the `_Measure` target as + the exception. + ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. @@ -479,10 +518,13 @@ categories. An operation that creates an item still names the parent category. **Virtual categories have reserved ids** (answer 23): `_index`, `_dim` and `_measure`, with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** -A `_Measure` formula target has no item of its own. Its handle is its -formula's `FormulaId`, which is why the category panel's `_Measure` rows need +**Measures are the normal formula target** (answers 35 and 36), and **a measure +has an id and a display**. A stored measure is an ordinary item. Revision 9 says a +`_Measure` formula target has no item of its own and that its handle is its +formula's `FormulaId`; that is **under question again (Q23)**, because it is +the normal case. On revision 9's reading, this is why the category panel's `_Measure` rows need `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). **A formula -target outside `_Measure` is a real item** (answer 30). It has its own `ItemId`, +target outside `_Measure`, the rare case (answer 36), is a real item** (answer 30). It has its own `ItemId`, and its formula has a separate `FormulaId`. Generated ids follow clause 4, so a formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-`. The panel lists that row as `ItemHandle::Item`, like any other item. @@ -612,7 +654,8 @@ define. The second was chosen by the implementation (PR #318; `context/code/core prefer a narrower one (ASCII whitespace only, say), and that is a one-line change. An underscore the display already had is still never merged with the one a run makes (`a_ b` → `a__b`), because the user said *consecutive spaces* collapse, not - consecutive underscores. + consecutive underscores. That is **Q24**. A trailing run slugs to a trailing + `_` (`East ` → `east_`), which is **Q25**. - **A slug of only `-` is a legal bare id.** `-` and `---` slug to themselves. Step 5's "if nothing is left" means the empty string only. The id alphabet admits them, so a line such as `-: row` is legal but hard to read. @@ -741,7 +784,9 @@ Required tests: a category displayed `Format`, on a row axis, round-trips; and a view whose number format is each axis keyword round-trips, in a document where the same view also holds a category `format` on an axis. -**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). The shape is the ADR's +**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). **Under question +again by answers 35 and 36, because the `_Measure` target is the normal case (Q23).** +The shape below is revision 9's. It is the ADR's own reading, not a user answer, except that the formula id's place is answer 5's and an item target's id is answer 30's. It is `- () = []`: @@ -861,7 +906,8 @@ pipe-quotes any display name that would otherwise read as display plus id: `Workbook::active_view` (improvise-p003 item 3). Answer 1 makes this natural rather than a migration of its own. A view or category rename then orphans nothing. -- **A `_Measure` formula-target row** (`ItemHandle::FormulaTarget(FormulaId)`): +- **A `_Measure` formula-target row** (`ItemHandle::FormulaTarget(FormulaId)`; under + question again, Q23): removing it means `RemoveFormula`, and renaming it means a display rename of the formula's target (improvise-i0n7). - **CheckedBatch's created-record exemption goes** (the ADR's own reading, not a @@ -1127,7 +1173,7 @@ The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q1 revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All of those are answered.** After acceptance, three implementation questions were answered on their tickets (answers 30–32), and the user's comments on revision 9 are -answers 33 and 34. Revision 9 adds Q23, which is open. +answers 33 and 34, and its two later comments are answers 35 and 36. Revision 9 adds Q23 to Q25, which are open. | # | question | my recommendation | status | |---|---|---|---| @@ -1158,7 +1204,10 @@ answers 33 and 34. Revision 9 adds Q23, which is open. | — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation | | — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31). **Order reversed: ids first until there is autocomplete** (answer 33) | | — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32) | -| 23 | A formula line's spelling (revision 9) | see "Still open" | **open** | +| — | Whether a measure has an id/display split, and which formula targets are normal (user comments on revision 9) | — | **answered in principle**: measures have the split (answer 35), and the normal target is a measure (answers 35, 36). The mechanism is Q23 | +| 23 | How a formula-target measure carries its id, and how a formula line spells its target (revision 9; reworded after answers 35 and 36) | none until the user chooses | **open** | +| 24 | Does `a_ b` slug to `a__b`? (answer 34) | (a) keep | **open** | +| 25 | Does a trailing whitespace run slug to a trailing `_`? | (a) keep | **open** | ## Still open @@ -1170,34 +1219,69 @@ review were left as tickets rather than ADR text: improvise-le4p, improvise-m56i improvise-p003. Revision 9 folds all three in, and improvise-m56i's two questions are answers 30 and 31. -**Revision 9 needs the user's approval, and it asks one question.** +**Revision 9 needs the user's approval, and it asks three questions**, Q23 to Q25. -**Q23. How does a formula line spell its target?** Answer 30 makes a target -outside `_Measure` a real item, and says a formula line names it "by the item's -id". A `_Measure` target has no item and so no item id. Clause 5 above therefore -reads the line as `- () = …`, where `` is an item id -outside `_Measure` and a display inside it. That puts two different kinds of -thing in one position, and the category after `[` is what tells them apart. -- **(a) Recommended: as clause 5 reads it.** No extra syntax. The one asymmetry - is that a `_Measure` target's display appears in the file bare, while every - other name is an id. -- **(b) Always a display, and the item id in a second tag:** - `- Tax(tax-1)(tax) = … [customcat]`. The two positions never mix kinds, but a - formula line outside `_Measure` gains a second parenthesised tag. -- **(c) Always the target's id**, giving a `_Measure` target an id of its own - that is distinct from the formula's. This reverses clause 1's "its handle is its - formula's `FormulaId`". +**Q23. How does a formula-target measure carry its id, and how does a formula line +spell its target?** (Reworded after answers 35 and 36. Revision 9 asked only how +the line spelled its target, and it had the dimension case in view.) -Two consequences of (a) that the answer should take in, both the ADR's own reading: -- **References to a formula target** are as clause 5 states them, for an expression - and a filter value alike: the id text of an `ItemHandle`, which is the item id - outside `_Measure` and the formula id for a `_Measure` target. That holds under - (a) and (b). Under (c) a `_Measure` target would be named by its own id instead. -- **Quoting the target position.** A `_Measure` display in the target position can - itself look like `Name(x)`, which would read back as a display and a formula id. - The writer's rule that pipe-quotes such a display therefore covers a - formula-line target too: `- |Margin(%)|(margin) = …`. An item id never contains - a parenthesis, so only a `_Measure` display can need it. +The user's words: measures have an id and a display, and measures are the normal +formula target. What is open is the *mechanism*. A stored measure is already an item +with both. A measure created by a formula (`profit`) has, in revision 9, no item: +its display lives on the `Formula`, and its id is the formula's own id. Clause 5 +therefore spells the line `- Profit() = … [_measure]` (the display bare) +for the normal case, and `- () = … []` for the rare +dimension case, so the two cases put different kinds of thing in one position. + +- **(a) Keep the handle: the target's id is the formula's id.** The display lives on + the `Formula`. The id is generated from the target's display, and with no item to + collide with it comes out bare: `Profit` → `profit`, not `profit-`. + `- Profit(profit) = revenue - cost [_measure]`, and a dimension target is + `- tax(tax-1) = … [customcat]`. References to the measure are `profit`. This is + the smallest change, and matches the pictures in answers 5 and 19. Whether it + *is* the split the user asks for is the thing to decide. Its cost is that the + normal case writes a bare display in a position that holds an id for the rare one, + which sits against answer 3. +- **(b) Always a display, then ids in tags.** Measure: `- Profit(profit) = … [_measure]`. + Dimension: `- Tax(tax-1)(tax) = … [customcat]`. The target position never mixes + kinds. A dimension line gains a second tag. +- **(c) A formula-target measure is a real `_Measure` item**, like a dimension target + under answer 30: its own `ItemId` and display, and a separate `FormulaId` + (`profit`, then `profit-`). The target position is the item id in every case: + `- profit(profit-1) = revenue - cost [_measure]`. This is the reading closest to + "measures have an id/display split too". It removes `ItemHandle::FormulaTarget`, + returns `CellKey` to `(CategoryId, ItemId)`, and ends the `CellKey` typing problem + Codex raised on this PR. It reverses clause 1's "no item of its own" and revision + 9's readings in clauses 1, 2, 5 and 8. It also needs the item and its display + written somewhere: today only non-formula `_Measure` items persist, and formula + targets are rebuilt from `## Formulas`. + +**No recommendation is made until the user chooses.** Revision 9 recommended (a), +written when the dimension case was the one in view. (c) is the literal reading of +answer 35 and (a) is the minimal change. + +Two consequences, both the ADR's own reading, for whichever is chosen: +- **References to a formula target** are the id text of its handle: under (a) and + (b) the item id outside `_Measure` and the formula id for a `_Measure` target; + under (c) always the target's item id. +- **Quoting the target position.** Where a display sits in the target position + (a, b), a display that looks like `Name(x)` would read back as a display and an id, + so the writer pipe-quotes it: `- |Margin(%)|(margin) = …`. An item id never + contains a parenthesis, so only a display needs it. (c) has no such position. + +**Q24. Does `a_ b` slug to `a__b`?** Answer 34 collapses a run of whitespace to one +`_`, and the ADR applies the steps as written, so an underscore the display already +had is not merged with the one a run makes. The user did not say. +- **(a) Keep it.** `a_ b` → `a__b`, and a deliberate `a__b` stays. +- **(b) Collapse any run of whitespace and `_` to one `_`.** `a_ b` → `a_b`, and + `a__b` → `a_b`. That changes ids for displays that hold a double underscore. + +**Q25. Does a trailing run slug to a trailing `_`?** Step 4 removes leading `_` and +nothing removes trailing ones, so `East ` → `east_`. The user did not say. +- **(a) Keep it.** It is what PR #318 does. +- **(b) Remove trailing `_` as step 4 removes leading `_`.** `East ` → `east`, and + also `East_` → `east`. +- **(c) Trim whitespace before step 2.** `East ` → `east`, and `East_` stays `east_`. Revision 9 states a reading of its own, which is not a user answer, in each of these nine places (three of them hold two readings). Each is flagged where it appears, and @@ -1211,7 +1295,7 @@ approving the PR approves them: - the axis-before-`format:` order and the quoting of a format that reads as an axis line (clause 5); - the formula-line shape, and a `_Measure` target's display living on the `Formula` - (clause 5); + (clause 5; **reopened by answers 35 and 36, Q23**); - CheckedBatch's created-record exemption is removed (clause 6); - a display equal to another thing's id counts as ambiguous in the formula panel (clause 6). From 769f7b251f4fad26126c430f88d1635b5325309e Mon Sep 17 00:00:00 2001 From: "Claude (adr7-steward)" Date: Mon, 5 Oct 2026 13:49:46 -0700 Subject: [PATCH 08/27] docs(adr): put answers 35 and 36 on their true head and mark Q23's premises (improvise-cq1q.1) Merge the eighth and ninth rounds (all four comments are on 4aed8c07), quote the anchored line, drop the lean toward option (c), add pending-Q23 markers, and record the review's non-blocking points as flagged text. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 101 +++++++++++++++++++-------- 1 file changed, 70 insertions(+), 31 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index b6776588..e2d3533d 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -8,13 +8,13 @@ user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17: **Revision 9 is a proposed amendment, not yet approved.** It folds in the user's answers on the implementation tickets (answers 30–32), the user's two inline comments on its head `4aed8c07` and a clarification in chat (answers 33 and 34), -its two inline comments on `e9659903` (answers 35 and 36), and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 +its two later inline comments, also on `4aed8c07` (answers 35 and 36), and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 reverses the matching order that answer 31 recorded, and answer 34 overrides the ADR's own reading of "space". Answers 35 and 36 make measures the normal formula target and give them an id/display split, which reopens Q23. It asks three new questions, Q23 (how a formula-target measure carries its id, and how a formula line spells its target), Q24 and Q25 (two details of slugify's whitespace rule), and -flags nine places where it states a reading of its own that is not a user answer, +flags eleven places where it states a reading of its own that is not a user answer, listed under Q23. · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · @@ -376,12 +376,16 @@ answers 31 and 33 differ, answer 33 governs. Clause 12 is rewritten to match. -### Eighth round: answers on revision 9 +### Eighth round: answers on revision 9 (head `4aed8c07`) The user answered revision 9 (`context/adr/007-stable-identities.md` at `4aed8c07`) -with two inline comments on PR #326, posted 2026-10-04 02:48Z and 02:49Z, and then -clarified the second one in chat (not on the PR, relayed to the steward). Each is -mapped to the line it is attached to, and quoted verbatim. +with four inline comments on PR #326: two posted 2026-10-04 02:48Z and 02:49Z, and +two posted 2026-10-05 20:22:52Z and 20:23:40Z. The GitHub API gives all four the +original commit `4aed8c07`. **All four are one round**: the user wrote the last two +before `e9659903`, which folded in the first two, existed, so they had not seen +answers 33 and 34 written up. The second comment was also clarified in chat (not on +the PR, relayed to the steward). Each is mapped to the line it was attached to on +`4aed8c07`, and quoted verbatim. **33. The order of answer 31.** On line 356, the sentence under answer 31, "A typed name is matched against displays first, then against ids": @@ -413,16 +417,14 @@ This **overrides the ADR's own reading** (revision 9's, which was not a user ans 4 is **the ADR's own reading of "whitespace"**, flagged there. Answers 18 and 21 keep their wording, which this reads as "space, meaning whitespace". -### Ninth round: answers on revision 9's head `e9659903` - -The user posted two inline comments on PR #326 at 2026-10-05 20:22:52Z and 20:23:40Z, -both on the line holding Q23's option (c). Each is quoted verbatim. - -**35. Measures get the id/display split too.** +**35. Measures get the id/display split too.** On line 1091, the end of Q23's +option (c), "Always the target's id, giving a `_Measure` target an id of its own that is +distinct from the formula's. This reverses clause 1's 'its handle is its formula's +`FormulaId`'": > "Measures should have ID/display split too. Measures are the normal case for formulas, anyways: something like `profit = revenue - cost`" -**36. Dimensions are rarely formula targets.** +**36. Dimensions are rarely formula targets.** Also on line 1091, 48 seconds later: > "Dimensions are mainly in formulas when you're calculating some kind of derived dimension, but usually dimensions should be relatively static and not interesting in formulas." @@ -440,8 +442,10 @@ things in earlier text meet them, and none is resolved here: an id and a display. A measure that exists only because a formula targets it has its display on the `Formula` and its id in the `FormulaId`. Whether that is the split the user wants, or whether the target should be an item, is Q23. -- **Answer 3** ("every reference is an id") sits badly with revision 9's line for the - normal case, where the target position holds a bare display. +- **Answer 3** ("every reference is an id") is about references. Clause 5 calls a + formula line a declaration, so the bare display in its target position is a + declaration's display and not a reference. Whether the normal case should still + spell an id there is part of Q23. - **Answers 5 and 19** picture the measure case as a display plus one id (`Profit(3)`, `Profit (profit)`), which fits an id/display split. - **Answer 30** and the ADR's worked example (`Tax` in `CustomCat`) are about the @@ -552,7 +556,8 @@ Following answers 1 to 3: user answer), where `ItemHandle` is clause 6's `{Item(ItemId), FormulaTarget(FormulaId)}`. Outside `_Measure` every coordinate is an `Item`. A `_Measure` formula cell's coordinate is `FormulaTarget`, because - that target has no item. The key stays sorted, now **by category id**, so gotcha + that target has no item (**pending Q23**: option (c) gives it an item, and this + reading falls). The key stays sorted, now **by category id**, so gotcha 5's canonical form is kept with a different sort key. `DataStore` interns ids exactly as it interns names today. Answer 22 makes every id unique across kinds, so an id's text alone names one thing; the tag is there so that the types @@ -938,7 +943,8 @@ Following answer 3, `RecordHash` hashes the explicit serialization of the sorted over different strings. The text is enough: answer 22 makes ids unique across kinds, so no `Item` and `FormulaTarget` can spell alike, and the hash stays canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate -is never part of a record in the first place. +is never part of a record in the first place. **Pending Q23**: under option (c) no +`FormulaTarget` exists, and this paragraph reduces to ids over `ItemId`s. - **ADR 002's decision is unchanged in substance.** It still hashes an explicit serialization of a normalized record, with a fixed key and a width argued for. - **Its prose will be out of date once this lands.** ADR 002 says `CellKey::new` @@ -1013,7 +1019,8 @@ Following answers 15 and 25: 1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224. 2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId`, `ViewId`, and `ItemHandle`, which the re-key (step 3) needs for `CellKey` before - clause 6's step (5) uses it in projections. + clause 6's step (5) uses it in projections (**pending Q23**: under option (c) + there is no `ItemHandle`). 3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with display→id resolution moved to every input edge. 4. **Persistence:** the new version literal (clause 10), the grammar in clause 5, the @@ -1238,27 +1245,49 @@ dimension case, so the two cases put different kinds of thing in one position. collide with it comes out bare: `Profit` → `profit`, not `profit-`. `- Profit(profit) = revenue - cost [_measure]`, and a dimension target is `- tax(tax-1) = … [customcat]`. References to the measure are `profit`. This is - the smallest change, and matches the pictures in answers 5 and 19. Whether it - *is* the split the user asks for is the thing to decide. Its cost is that the - normal case writes a bare display in a position that holds an id for the rare one, - which sits against answer 3. + the smallest change. Its cost is that the normal case writes a display in a + position that holds an id for the rare one. Under (a) a measure does have an id + and a display; whether that is the split the user asks for is the thing to decide. - **(b) Always a display, then ids in tags.** Measure: `- Profit(profit) = … [_measure]`. Dimension: `- Tax(tax-1)(tax) = … [customcat]`. The target position never mixes kinds. A dimension line gains a second tag. - **(c) A formula-target measure is a real `_Measure` item**, like a dimension target under answer 30: its own `ItemId` and display, and a separate `FormulaId` (`profit`, then `profit-`). The target position is the item id in every case: - `- profit(profit-1) = revenue - cost [_measure]`. This is the reading closest to - "measures have an id/display split too". It removes `ItemHandle::FormulaTarget`, + `- profit(profit-1) = revenue - cost [_measure]`. It removes `ItemHandle::FormulaTarget`, returns `CellKey` to `(CategoryId, ItemId)`, and ends the `CellKey` typing problem Codex raised on this PR. It reverses clause 1's "no item of its own" and revision 9's readings in clauses 1, 2, 5 and 8. It also needs the item and its display written somewhere: today only non-formula `_Measure` items persist, and formula targets are rebuilt from `## Formulas`. +Answers 5 and 19 picture the measure case as `Profit(3)` and `Profit (profit)`. Both +pictures fit (a), (b) and (c): each gives the measure a display and an id, and under +(c) the `profit` in answer 19's picture is the item id. + **No recommendation is made until the user chooses.** Revision 9 recommended (a), -written when the dimension case was the one in view. (c) is the literal reading of -answer 35 and (a) is the minimal change. +written when the dimension case was the one in view. Any ranking of how well an option +fits answer 35 would be the ADR's own judgement and not the user's, since (a) and (b) +also give a measure an id and a display. (a) is the smallest change. + +Four further points the choice must settle, all the ADR's own reading and none a +user answer: +- **A formula targeting a stored measure of the same name.** `measure_item_names` + merges them today, so a stored `Profit` column and a formula `Profit = …` are one + measure. Under (a) and (b) that one column has two ids (the item's `profit` and the + formula's `profit-`), and the ADR must say which one keys and references use. + Under (c) it is the natural case, and the formula attaches to the existing item. +- **Removing a formula under (c).** `RemoveFormula` drops the formula and leaves its + `_Measure` item, as it leaves a dimension target's item today (answer 30). The + measure column then has no formula and possibly no data. +- **The `[_measure]` tag.** Today's writer omits the tag for a `_Measure` formula + (`- {raw}`), and the user's own example, `profit = revenue - cost`, has none. The + examples above write it only to show the category. Whether the tag is omitted for + `_measure` is not decided here. +- **Other ADRs.** ADR 008 (PR #308) anchors the cursor by item identity, which under + (a) or (b) a formula-measure column does not have, so it would need `ItemHandle`. + ADR 009 (PR #309) relies on `find_item_category`'s `_Measure` fallback for formula + targets, which (c) would remove. Neither ADR is edited here. Two consequences, both the ADR's own reading, for whichever is chosen: - **References to a formula target** are the id text of its handle: under (a) and @@ -1271,24 +1300,31 @@ Two consequences, both the ADR's own reading, for whichever is chosen: **Q24. Does `a_ b` slug to `a__b`?** Answer 34 collapses a run of whitespace to one `_`, and the ADR applies the steps as written, so an underscore the display already -had is not merged with the one a run makes. The user did not say. +had is not merged with the one a run makes. The user did not say. The author's +lean, not the user's, is (a): it is what the user's wording ("consecutive spaces") +and PR #318 give. - **(a) Keep it.** `a_ b` → `a__b`, and a deliberate `a__b` stays. - **(b) Collapse any run of whitespace and `_` to one `_`.** `a_ b` → `a_b`, and `a__b` → `a_b`. That changes ids for displays that hold a double underscore. +A neighbouring edge: step 2 runs before step 3 strips non-alphabet characters, so +`Time \u{200B} Period` (a zero-width space between two spaces) slugs to +`time__period`, two runs and not one. + **Q25. Does a trailing run slug to a trailing `_`?** Step 4 removes leading `_` and -nothing removes trailing ones, so `East ` → `east_`. The user did not say. +nothing removes trailing ones, so `East ` → `east_`. The user did not say. The +author's lean, not the user's, is (a). - **(a) Keep it.** It is what PR #318 does. - **(b) Remove trailing `_` as step 4 removes leading `_`.** `East ` → `east`, and also `East_` → `east`. - **(c) Trim whitespace before step 2.** `East ` → `east`, and `East_` stays `east_`. Revision 9 states a reading of its own, which is not a user answer, in each of these -nine places (three of them hold two readings). Each is flagged where it appears, and +eleven places (three of them hold two readings). Each is flagged where it appears, and approving the PR approves them: - the virtual views' reserved ids `_records` and `_drill` (clause 1); - the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); -- a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2); +- a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2; **pending Q23**); - `collapsed:` is `(CategoryId, group name)` (clause 2); - "whitespace" means the Unicode `White_Space` property (replacing the U+0020 reading, by answer 34), and a slug of only `-` is a legal id (clause 4); @@ -1298,7 +1334,10 @@ approving the PR approves them: (clause 5; **reopened by answers 35 and 36, Q23**); - CheckedBatch's created-record exemption is removed (clause 6); - a display equal to another thing's id counts as ambiguous in the formula panel - (clause 6). + (clause 6); +- an id is matched exactly, with no case folding (clause 12); +- the display stage of the typed-name resolver goes once autocomplete exists + (clause 12). ## Consequences From 068e4130714ccb08d45e34564af381de6f3a4a78 Mon Sep 17 00:00:00 2001 From: "Claude (adr7-steward)" Date: Mon, 5 Oct 2026 14:07:05 -0700 Subject: [PATCH 09/27] docs(adr): mark CategoryRow pending Q23 and qualify answer 5's fit (improvise-cq1q.1) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index e2d3533d..bc22f671 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -447,7 +447,8 @@ things in earlier text meet them, and none is resolved here: declaration's display and not a reference. Whether the normal case should still spell an id there is part of Q23. - **Answers 5 and 19** picture the measure case as a display plus one id - (`Profit(3)`, `Profit (profit)`), which fits an id/display split. + (`Profit(3)`, `Profit (profit)`), which fits an id/display split (answer 5's + file-line picture fits option (c) only loosely; see Q23). - **Answer 30** and the ADR's worked example (`Tax` in `CustomCat`) are about the *rare* case. The decision stands. The emphasis of the surrounding text changed: revision 9 wrote the dimension target as the main case and the `_Measure` target as @@ -891,7 +892,8 @@ pipe-quotes any display name that would otherwise read as display plus id: `ClearCell { view, … }`, which improvise-wpxr (PR #281, merged) gave a view named by its name, like every other view-scoped variant (improvise-lltt). - **Projections carry id and display together:** - - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }`; + - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }` + (**pending Q23**: under option (c) there is no `ItemHandle`); - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows (answer 19): the target's display name with its id in parentheses, then the expression over display names, with an id wherever a display name is ambiguous @@ -1261,9 +1263,11 @@ dimension case, so the two cases put different kinds of thing in one position. written somewhere: today only non-formula `_Measure` items persist, and formula targets are rebuilt from `## Formulas`. -Answers 5 and 19 picture the measure case as `Profit(3)` and `Profit (profit)`. Both -pictures fit (a), (b) and (c): each gives the measure a display and an id, and under -(c) the `profit` in answer 19's picture is the item id. +Answers 5 and 19 picture the measure case as `Profit(3)` and `Profit (profit)`. +Answer 19's panel picture fits (a), (b) and (c): each gives the measure a display and +an id, and under (c) the `profit` is the item id. Answer 5's file-line picture, +`- Profit(3) = …`, fits (a) and (b) exactly, and (c) only loosely, because (c) writes +the item id in the target position where answer 5 shows the display. **No recommendation is made until the user chooses.** Revision 9 recommended (a), written when the dimension case was the one in view. Any ranking of how well an option From 74efa481964e00f04f6eb186179ab04fec057ed7 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Thu, 8 Oct 2026 22:38:52 -0700 Subject: [PATCH 10/27] docs(adr): fold the user's nine answers on 068e4130 into ADR 007 revision 9 (improvise-cq1q.1) Answers 37-45 in a ninth round of "What the user decided". Q24 and Q25 are answered, slugify's steps are rewritten (whitespace to _, strip, collapse runs of _, trim leading and trailing _ and -), groups get ids (overriding answer 9), and Q23 is restated around "a formula is a derived measure" with a worked example. Q23 stays open; Q26-Q34 are new. core.md section 20 notes that slugify predates the proposed rules. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 727 ++++++++++++++++++++------- context/code/core.md | 17 +- 2 files changed, 559 insertions(+), 185 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index bc22f671..0f6dfd17 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -11,11 +11,16 @@ comments on its head `4aed8c07` and a clarification in chat (answers 33 and 34), its two later inline comments, also on `4aed8c07` (answers 35 and 36), and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 reverses the matching order that answer 31 recorded, and answer 34 overrides the ADR's own reading of "space". Answers 35 and 36 make measures the normal formula -target and give them an id/display split, which reopens Q23. It asks three new +target and give them an id/display split, which reopens Q23. It asked three new questions, Q23 (how a formula-target measure carries its id, and how a formula line -spells its target), Q24 and Q25 (two details of slugify's whitespace rule), and -flags eleven places where it states a reading of its own that is not a user answer, -listed under Q23. · +spells its target), Q24 and Q25 (two details of slugify's whitespace rule). +The user's nine inline comments on head `068e4130` are answers 37–45 (the ninth +round). **They answer Q24 and Q25**, rewrite slugify's steps, **give groups ids +(overriding answer 9)**, reject two of the ADR's flagged readings (the `_Index`/`_Dim` +one and the legal `-` slug) and override a third (`collapsed:`), and ask for Q23 to be +stated more clearly, which it now is. **Q23 stays open.** The ninth round adds nine +questions, Q26–Q34. The places where the ADR states a reading of its own that is not +a user answer are listed at the end of "Still open". · **Decided by:** the user; see "What the user decided" · **Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and @@ -117,7 +122,8 @@ exception to 'every reference is an id'"): > "This seems right" -Groups get no id in this ADR. +Groups get no id in this ADR. **Overridden by answer 44** (ninth round): a group +name is an id, and the friendly name is separate. **10. Q6, per-entity versions.** On line 469, Q6 ("Recommendation: not in this ADR"): @@ -231,7 +237,8 @@ run of spaces collapse to one `_`?"): This decides three things: - a run of spaces collapses to one `_`, as recommended (a run of whitespace, by - answer 34); + answer 34; answer 41 makes each whitespace character a `_` first and then collapses + runs of `_`); - a duplicate slug gets a numeric suffix after a `-`; - **a file has ids on everything or on nothing.** A new-version file with a declaration missing its id is refused, like any file that cannot be resaved safely (answer 11). @@ -260,6 +267,8 @@ capitalised displays"): The ids are `_index`, `_dim` and `_measure`, and the displays stay `_Index`, `_Dim` and `_Measure`. A leading `_` is reserved for them. Clause 4 says how slugify keeps a generated id out of that space, and clause 5 says what happens to a hand-written one. +Answer 42 widens the reservation to "virtual categories and measures and other +entities", and answer 43 restates `_index` and `_dim`. **24. Q21, an old file naming an undeclared item.** On line 708, option (a) ("Declare the item at load"): @@ -354,6 +363,8 @@ That item has its own `ItemId`, the formula has its own `FormulaId`, and by answer 22 the two differ. A formula line names such a target by the item's id. This was chosen over the recommended alternative, which was to give every formula target the `_Measure` treatment (no real item, handled by its `FormulaId`). +The user's comment on this sentence is answer 37: a formula id that would collide +takes the counter's suffix. **31. A typed name that is both one thing's display and another thing's id** (improvise-m56i item 6): @@ -365,7 +376,8 @@ comment on this answer (answer 33): until autocomplete exists, a typed name is matched against ids first, then against displays.** What stands from answer 31 is that the client holds the resolution (answer 32), and that a display matching more than one thing is refused, with the refusal listing the candidates' ids. Where -answers 31 and 33 differ, answer 33 governs. +answers 31 and 33 differ, answer 33 governs. The user's comment on this sentence is +answer 38: ids "should always be preferred when there's ambiguity". **32. Which side holds clause 12's capture** (improvise-hexb item 2), as recorded: @@ -374,7 +386,8 @@ answers 31 and 33 differ, answer 33 governs. > projection; the wire carries only id-form text; the host never resolves a > display name." -Clause 12 is rewritten to match. +Clause 12 is rewritten to match. The user confirmed this on the PR and widened it +from the formula editor to every reference (answer 39). ### Eighth round: answers on revision 9 (head `4aed8c07`) @@ -415,7 +428,9 @@ This **overrides the ADR's own reading** (revision 9's, which was not a user ans "Space" in answers 18 and 21 means whitespace, and a run of whitespace becomes one `_`. The user did not enumerate which characters count, so the definition in clause 4 is **the ADR's own reading of "whitespace"**, flagged there. Answers 18 and 21 -keep their wording, which this reads as "space, meaning whitespace". +keep their wording, which this reads as "space, meaning whitespace". **Answer 41 +refines the mechanism:** each whitespace character becomes `_`, and runs of `_` are +collapsed afterwards, so a run of whitespace still ends as one `_`. **35. Measures get the id/display split too.** On line 1091, the end of Q23's option (c), "Always the target's id, giving a `_Measure` target an id of its own that is @@ -454,6 +469,145 @@ things in earlier text meet them, and none is resolved here: revision 9 wrote the dimension target as the main case and the `_Measure` target as the exception. +### Ninth round: answers on revision 9 (head `068e4130`) + +The user answered revision 9 at `068e4130` with nine inline comments on PR #326, +posted 2026-10-09 05:17:38Z–05:26:39Z. The GitHub API gives all nine the original +commit `068e4130`. Each is mapped to the line it was attached to on `068e4130`, +quoted verbatim, and given the comment id. Where the ADR reads more into an answer +than its words say, the text says it is the ADR's reading and asks a numbered +question. + +**37. An id names one thing, and a colliding formula id takes the counter.** Comment +4226862456, on line 356, the last sentence of answer 30 ("This was chosen over the +recommended alternative, which was to give every formula target the `_Measure` +treatment …"): + +> "An id always refers to the same thing and so must be globally unique: if a formula has an id that would conflict, increment the counter to disambiguate it. Formulas may refer to anything in a typed context: e.g. a formula might count the number of times a particular letter appears in a category value." + +This decides: +- **A formula id that would collide is suffixed from the counter**, like any other + generated id. This restates answers 22 and 28 for the formula case, and clause 1's + example already follows it: a formula `Tax` in `CustomCat` gets item id `tax`, then + formula id `tax-`. +- **Formulas may refer to any kind of thing**, and the context of the reference + gives its type. This is why ids are unique across kinds (answer 22): a bare id in + formula text must name one thing whatever its kind. + +The example (count a letter in a category value) has no function in today's formula +language. Whether it is in this ADR's scope, what a reference may name, and whether +"a category value" means an item's display or its id are **Q26**. + +**38. Ids win under ambiguity.** Comment 4226867495, on line 368, the end of the +paragraph under answer 31 ("a display matching more than one thing is refused … +Where answers 31 and 33 differ, answer 33 governs"): + +> "IDs are unambiguous so they should always be preferred when there's ambiguity. " + +This confirms answer 33's ids-first order. The display-versus-display refusal is +untouched: no id is involved there, so there is nothing to prefer. Whether "always" +makes ids-first permanent rather than interim (clause 12 calls it interim until an +autocomplete exists) is **Q27**. + +**39. Display names live only in the client.** Comment 4226873976, on line 377, +"Clause 12 is rewritten to match.", under answer 32: + +> "Yes, the client should be the only place that allows any reference to something by a human-friendly name and this sort of reference should be minimized to the UI layer and converted to IDs ASAP" + +This confirms answer 32 and **widens it from the formula editor to every +reference**: only a client accepts a human-friendly name, it does so in the UI layer +only, and it converts the name to an id as early as it can. Clause 2's "resolved to +ids at the input edge" and clause 6's "every operand naming an existing thing is an +id" already say this. Two places in the ADR resolve a display outside a client: the +old-version loader (clause 5) and headless `improvise cmd`/`script`. Whether they +are exceptions is **Q28**. + +**40. A formula is a derived measure.** Comment 4226895692, on line 1303, the end of +Q23 ("Quoting the target position …"): + +> "Formulas are a derived measure. I don't really understand the problem here, this whole section needs to be stated more clearly." + +**Q23 is rewritten** to state the problem in those terms, with a worked example and +a plain split between what is decided and what is open. It stays **open**: the +answer says what a formula *is*, and the ADR does not read it as choosing an option. +"Still open" says why. + +**41. Whitespace becomes `_` first, then runs of `_` collapse.** Comment 4226897815, +on line 1316, Q24's "neighbouring edge", where `Time \u{200B} Period` slugged to +`time__period`: + +> "The spaces should have been normalized to _ before collapsing runs of _s" + +This **answers Q24 with option (b)**. It changes slugify's step 2 and adds a +collapse step (clause 4, steps 2 and 4): +- each whitespace character becomes `_`; +- **runs of `_` collapse to one `_`**, whether they came from whitespace or were + already in the display. So `a_ b` → `a_b` and `a__b` → `a_b`; +- `Time \u{200B} Period` → `time_period`. + +That last result needs the collapse to run **after** step 3 strips the zero-width +space. The answer says the collapse comes after the normalisation, and the example it +is attached to gives one `_` only if it also comes after the strip. So clause 4 puts +it there. That placement is the ADR's reading, and **Q29** asks the user to confirm +it. This overrides revision 9's reading and the code's comment in +`context/code/core.md` §20 ("no extra underscore-collapsing step exists"). + +**42. A leading `_` is reserved; a trailing one is dropped.** Comment 4226901843, on +line 1324, Q25's option (c): + +> "A leading _ is reserved for virtual categories and measures and other entities. A trailing one should be dropped to: disambiguate with -#" + +This **answers Q25 with option (b)**: slugify drops a trailing `_` as it drops a +leading one, so `East ` → `east` and `East_` → `east`. The ADR reads "to:" as "too;": +drop it too, and if the shorter slug then collides, disambiguate it with `-` +(answers 27 and 28). The reservation of a leading `_` now covers "virtual categories +and measures and other entities", which is wider than answer 23's three categories. +The ADR reads the virtual views `_records` and `_drill` (clause 1) as among the +"other entities". What a virtual *measure* is, and that reading, are **Q30**. + +**43. `_Index` and `_Dim` have ids.** Comment 4226904733, on line 1330, the flagged +reading "the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1)": + +> "No, they have ids _index and _dim " + +This **rejects the reading as it was worded**, and restates answer 23: the categories' +ids are `_index` and `_dim`. The reading was about their *coordinates*, the +synthetic positions in records mode (`_Dim` carries a column index, gotcha 1), not +the categories. Whether those positions are ids in the document's one namespace, and +how they would be spelled so that `_index`'s `0` does not collide with a user's item +`0`, is **Q31**. Today the repository spells the virtual categories `_Index`, `_Dim` +and `_Measure`, as keys, on the wire and in `.improv` files. What the move to lower +case ids costs is **Q32**. + +**44. Groups get ids.** Comment 4226908008, on line 1332, the flagged reading +"`collapsed:` is `(CategoryId, group name)` (clause 2)": + +> "Group name should be an id for the group, not a friendly name " + +This **overrides answer 9** ("not now") and the reading it was attached to. A group +has an id, and its friendly name is a separate display. A `collapsed:` entry names a +group by its id. Clause 1's table, clause 2 and clause 5 are changed to match. How a +group's id is made and spelled, and what the wire carries, are **Q33**. The ADR +does not decide them. +- **Relation, not a dependency.** PR #336 (improvise-cq1q.12, branch + `adr007/display-2b`, open, not on `main`) gives `Group` a private display beside + its key. On the ADR's reading, under answer 44 that key becomes the group's id, as + a category's does (clause 2). Nothing here depends on #336 landing. + +**45. A slug of `-` is not an id; no id begins or ends with `-`.** Comment +4226910518, on line 1334, the flagged reading "'whitespace' means the Unicode +`White_Space` property … and a slug of only `-` is a legal id (clause 4)": + +> "A slug of - should be invalid as an id and there should be no leading or trailing -s" + +This **rejects the second half of that reading**. Slugify removes leading and +trailing `-`, so `-` and `---` leave nothing and take `-` (step 6), and +`-East-` → `east`. The first half of the bullet, the meaning of "whitespace", was not +addressed and stays the ADR's flagged reading. That a **hand-written** id with a +leading or trailing `-` is refused at load, as one with a leading `_` is, and how the +trims interleave (`-_x` → `x`), are the ADR's reading of "invalid as an id" and are +**Q34**. + ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. @@ -514,15 +668,19 @@ Twelve clauses. | formula | yes | the document, across every kind (answer 22) | `FormulaId` | | view | yes (the user's example) | the document, across every kind (answer 22) | `ViewId` | | record | `RecordHash`, now over ids (clause 8) | the document | unchanged shape | -| group | **no**, not in this ADR (answer 9) | | | +| group | **yes** (answer 44, overriding answer 9) | the document, across every kind (the ADR's reading of answers 22 and 37; Q33) | to be decided (Q33) | -**Ids are unique in the whole document** (answer 22). One namespace covers categories, -items, formulas and views, so an item id names one item without its category. A +**Ids are unique in the whole document** (answers 22 and 37: "An id always refers to +the same thing and so must be globally unique"). One namespace covers categories, +items, formulas and views, and on the ADR's reading groups too (Q33), so an item id +names one item without its category. A `CellKey` still pairs each item with its category, because the key needs the categories. An operation that creates an item still names the parent category. -**Virtual categories have reserved ids** (answer 23): `_index`, `_dim` and `_measure`, -with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** +**Virtual categories have reserved ids** (answers 23 and 43): `_index`, `_dim` and +`_measure`, with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin +with `_`**: a leading `_` "is reserved for virtual categories and measures and other +entities" (answer 42). Moving today's spellings to these ids is Q32. **Measures are the normal formula target** (answers 35 and 36), and **a measure has an id and a display**. A stored measure is an ordinary item. Revision 9 says a `_Measure` formula target has no item of its own and that its handle is its @@ -531,21 +689,27 @@ the normal case. On revision 9's reading, this is why the category panel's `_Mea `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). **A formula target outside `_Measure`, the rare case (answer 36), is a real item** (answer 30). It has its own `ItemId`, and its formula has a separate `FormulaId`. Generated ids follow clause 4, so a -formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-`. +formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-` +(answer 37: "if a formula has an id that would conflict, increment the counter"). The panel lists that row as `ItemHandle::Item`, like any other item. -**The virtual views have reserved ids too** (the ADR's own reading, not a user -answer): `_records` and `_drill`, with the displays `_Records` and `_Drill`. They -are never persisted, but they are addressed on the wire (`SwitchView`, -improvise-obtu). Answer 23 reserves a leading `_` for the virtual *categories*, -and this extends it to the virtual views, because a slugified `_Drill` would -otherwise be `drill`, which could collide with a user's view (improvise-p003 item 1). +**The virtual views have reserved ids too**: `_records` and `_drill`, with the +displays `_Records` and `_Drill`. They are never persisted, but they are addressed on +the wire (`SwitchView`, improvise-obtu). Revision 9 introduced this as its own +reading, extending answer 23's reservation from the virtual categories to the +virtual views, because a slugified `_Drill` would otherwise be `drill`, which could +collide with a user's view (improvise-p003 item 1). Answer 42 now reserves a leading +`_` for "virtual categories and measures and other entities". **That the virtual +views are among the "other entities" is the ADR's reading** (Q30). -**The ADR's reading, not a user answer: `_Index` and `_Dim` coordinates are -outside the id namespace.** They are synthetic record positions (`_Dim` carries a -column index, gotcha 1). They are never persisted and never typed, so they take no -part in answer 22's uniqueness, and `_index`'s `0` cannot collide with a user item -whose id is `0` (improvise-p003 item 2). +**The `_Index` and `_Dim` categories have ids `_index` and `_dim`** (answers 23 and +43). Revision 9 read their *coordinates*, the synthetic record positions (`_Dim` +carries a column index, gotcha 1), as outside the id namespace, so that `_index`'s +`0` could not collide with a user item whose id is `0` (improvise-p003 item 2). **The +user rejected that reading as worded** (answer 43, "No, they have ids _index and +_dim"). The ADR does not read the answer as deciding what the positions are: that +is **Q31**. Until it is answered, nothing in this ADR depends on the positions' +spelling, because they are never persisted and never typed. ### 2. Storage keys by id; `display` is the only place a human name lives @@ -566,10 +730,12 @@ Following answers 1 to 3: wrote this pair as `(CategoryId, ItemId)` while also putting a `FormulaId` in it (Codex on PR #326). - **View configuration:** the axis maps, page selections, and hidden and collapsed sets - are keyed by ids. The one exception is a collapsed entry, which names a group: - groups have no id (answer 9), so it is `(CategoryId, group name)`, and - `collapsed:` writes the category id, then the group name (the ADR's own - reading, not a user answer; improvise-p003 item 4). + are keyed by ids, with no exception. A collapsed entry names a group, and a + group name is an id (answer 44), so the entry is the category id and the group's + id. Revision 9 had made it `(CategoryId, group name)`, because answer 9 gave groups + no id (improvise-p003 item 4); answer 44 overrides both. Whether the category id + is still needed beside a group id that is unique in the document, and how the + line is spelled, are part of Q33. - **Formulas:** a formula's references are ids (clause 5). The formula panel renders them as display names (answer 19; clause 6). @@ -615,63 +781,88 @@ Following answers 8, 16 and 18, **every** id the app generates comes from the di name by **slugify**. That covers import, interactive creation, and assigning ids to an id-less `.improv` file at load (answer 16(a), "All id generation"). So a thing's id never depends on how it was made. The steps, in order (answer 18, "casefold, replace -space with `_`, strip"): +space with `_`, strip", as answers 34, 41, 42 and 45 refine it): 1. **Casefold, with Unicode full case folding** (answers 16(b), 18 and 26). Rust's `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped), so this needs a case-folding table or a small crate in `improvise-core`. It must pass the `wasm` gate, because that crate is in the browser bundle. -2. **Replace each run of whitespace with one `_`** (answers 18, 21 and 34; answer 18 - changed answer 8's `-`). A run is any unbroken sequence of whitespace - characters, whatever mix, so it becomes one `_`, never one per character. +2. **Replace each whitespace character with `_`** (answers 18, 34 and 41; answer 18 + changed answer 8's `-`). "The spaces should have been normalized to _ before + collapsing runs of _s" (answer 41), so this step makes one `_` per character, and + step 4 does the collapsing. 3. **Strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`** (answers 16(c) and 16(d)). This strips non-ASCII letters, so an id never needs quoting in an `.improv` reference (clause 5 notes where formula text differs). -4. **Remove any leading `_`** (answer 23: no non-virtual id may begin with `_`). The - answer states the constraint. Removing the underscores is this ADR's way of meeting - it, so `_Draft` → `draft`. -5. **If nothing is left, use `-`** (answer 16(e)), such as `category-1` or +4. **Collapse each run of `_` to one `_`** (answers 21 and 41). It does not matter + whether an underscore came from whitespace or was in the display already. **The + step comes after the strip, which is the ADR's reading** (Q29): it is the order in + which answer 41's example, `Time \u{200B} Period`, gives one `_`. +5. **Remove every leading and trailing `_` and `-`**, until the slug neither begins + nor ends with either. + - A leading `_` "is reserved for virtual categories and measures and other + entities" (answers 23 and 42). Removing it is this ADR's way of meeting that + constraint, so `_Draft` → `draft`. + - A trailing `_` "should be dropped" (answer 42). + - "There should be no leading or trailing -s" (answer 45). + - Repeating the removal until neither character is left at either end (`-_x` → + `x`) is the ADR's reading (Q34). +6. **If nothing is left, use `-`** (answer 16(e)), such as `category-1` or `item-4`. `n` comes from the same counter as every other suffix (answer 28), not - from the lowest free number. + from the lowest free number. This covers a display that is only `-` or only `_`, + because "a slug of - should be invalid as an id" (answer 45). -Examples: `East` → `east`, `Time Period` → `time_period`, `Coastal_East` → -`coastal_east`, `Q1-2024` → `q1-2024`, `Region(code)` → `regioncode`, `Café` → `caf`, -`Ü` → `category-`, `Straße` → `strasse` (answer 26). `Time Period` and -`Time_Period` slug alike, so whichever comes second gets `time_period-`. `East` in -`Region` and `East` in `Direction` slug alike too, and since ids are global (answer -22), the second is `east-`. Here `` is the counter's next value, for example -`east-1`. +Examples: `East` → `east`, `Time Period` → `time_period`, `Time Period` → +`time_period`, `Coastal_East` → `coastal_east`, `Q1-2024` → `q1-2024`, +`Region(code)` → `regioncode`, `Café` → `caf`, `Ü` → `category-`, `Straße` → +`strasse` (answer 26). By answer 41, `a_ b` → `a_b`, `a__b` → `a_b` and +`Time \u{200B} Period` → `time_period`. By answer 42, `East ` → `east` and `East_` → +`east`. By answer 45, `-East-` → `east`, and `-` and `---` → `-`. Step 4 +collapses only `_`, so a `-` between two spaces survives: `Q1 - 2024` → +`q1_-_2024`. The user has not said otherwise, and this ADR adds no rule for it. -**Two readings of the steps, the ADR's own and not user answers.** The first is -the ADR's reading of the word "whitespace", which answer 34 requires and does not -define. The second was chosen by the implementation (PR #318; `context/code/core.md` -§20): +`Time Period` and `Time_Period` slug alike, so whichever comes second gets +`time_period-`. So do `East`, `East ` and `East_`, and the second and third each +take a suffix ("disambiguate with -#", answer 42). `East` in `Region` and `East` in +`Direction` slug alike too, and since ids are global (answers 22 and 37), the second is +`east-`. Here `` is the counter's next value, for example `east-1`. The suffix +never puts a `-` at either end, so a suffixed id still meets step 5. + +**The ADR's own reading of the word "whitespace"**, which answer 34 requires and does +not define. It is not a user answer. Revision 9 had a second reading here, that a slug +of only `-` is a legal id. Answer 45 rejects it, so step 5 removes it. - **"Whitespace" means the Unicode `White_Space` property**, the set Rust's `char::is_whitespace` tests. That is U+0009–U+000D, U+0020, U+0085, U+00A0, U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. It needs no table in `improvise-core`. A tab, a no-break space (U+00A0) and an ideographic space (U+3000) are whitespace, so each becomes `_` like a space does: `Time\tPeriod`, `Time\u{A0}Period` and `Time Period` all slug to `time_period`, - and `Time \t Period`, one run of three characters, does too. A zero-width space + and so does `Time \t Period`, whose three `_` collapse at step 4. A zero-width space (U+200B) and a byte-order mark (U+FEFF) are **not** `White_Space`, so step 3 strips them. The realistic case is a header pasted from a web page, where a no-break space stands for a space. The ADR chose this set because it is the standard definition and the user asked for "whitespace", not for a list. The user may prefer a narrower one (ASCII whitespace only, say), and that is a one-line change. - An underscore the display already had is still never merged with the one a run - makes (`a_ b` → `a__b`), because the user said *consecutive spaces* collapse, not - consecutive underscores. That is **Q24**. A trailing run slugs to a trailing - `_` (`East ` → `east_`), which is **Q25**. -- **A slug of only `-` is a legal bare id.** `-` and `---` slug to themselves. - Step 5's "if nothing is left" means the empty string only. The id alphabet - admits them, so a line such as `-: row` is legal but hard to read. + Revision 9 asked here whether `a_ b` keeps two underscores (Q24) and whether a + trailing run leaves a trailing `_` (Q25). Answers 41 and 42 settle both, and steps + 4 and 5 follow them. **What this changes in built code (a follow-up, not made here).** PR #318's -`slugify` (`crates/improvise-core/src/model/id.rs`, `underscore_space_runs`) tests -`c == ' '`, and `context/code/core.md` §20 states the U+0020-only reading as a -rule. So today `Time\tPeriod` slugs to `timeperiod`. Answer 34 makes that wrong, and -the implementation needs a ticket to test whitespace instead and to repair §20 and -its tests. Nothing calls `slugify` yet (§20), so no stored id is affected. +`slugify` (`crates/improvise-core/src/model/id.rs`) and `context/code/core.md` §20 +predate answers 34, 41, 42 and 45, and differ from steps 2–6 in five ways: +- `underscore_space_runs` tests `c == ' '`, so `Time\tPeriod` slugs to + `timeperiod` (answer 34); +- it collapses runs of spaces before the strip and never collapses `_`, so `a_ b` → + `a__b` and `Time \u{200B} Period` → `time__period` (answer 41); +- it removes only leading `_`, so `East ` → `east_` (answer 42); +- it never removes `-`, so `-` slugs to `-` and `-East-` to `-east-` (answer 45); +- `IdAllocator::claim` accepts `-`, `-x` and `x-`, which Q34 would refuse. + +The tests `slugify_keeps_an_underscore_the_display_already_had` and +`slugify_collapses_a_run_of_spaces_to_one_underscore` assert the superseded results. +Answers 34, 41, 42 and 45 are all folded into revision 9, which the user has not yet +approved, so the follow-up should wait for that approval. Nothing calls `slugify` +yet (§20), so no stored id is affected. The remaining rules: @@ -679,7 +870,7 @@ The remaining rules: version, or a CSV or JSON import) gets every id from slugify. A new-version file must carry an id on every declaration, and one that does not is refused (clause 5). So no file mixes hand-written and generated ids. -- **Any colliding id gets a suffix, `gensym`-style** (answers 21, 27 and 28). +- **Any colliding id gets a suffix, `gensym`-style** (answers 21, 27, 28, 37 and 42). - **When:** whenever a generated slug is already taken, whatever kind of thing holds it. That applies on import, when an id-less file loads, and when the host creates something in a session. An id that does not collide stays bare. @@ -734,7 +925,13 @@ parenthesised run that contains a capital, such as `Sales(Copy)`, does not match `id_tag`, and neither does an empty or trailing-space tag. In a new-version file that declaration has no id, so the load is refused (answer 21), and the error names the line. Revision 4 generated an id and warned. A hand-written id beginning with `_`, -other than the three virtual ids, is refused too (answer 23). +other than the reserved virtual ids, is refused too (answers 23 and 42). A +hand-written id that is `-`, or that begins or ends with `-`, is refused the same +way. That is the ADR's reading of answer 45 ("a slug of - should be invalid as an id +and there should be no leading or trailing -s") for an id the program did not +generate (Q34). Whether a hand-written id ending in `_` is refused is also Q34: +answer 42 drops a trailing `_` from a slug, and does not say whether a written id may +carry one. **Inside formula text an id may still need quoting.** The formula grammar's `bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So @@ -760,8 +957,11 @@ id_ref = { id_token } ``` `model_name` has no id, so it keeps `header_name`. `Initial View:` becomes a reference -and takes an `id_ref`. A group reference (`[Coastal]`) stays a name while groups have no -id (answer 9). +and takes an `id_ref`. **A group reference (`[Coastal]`, and the group in a +`collapsed:` line) becomes an id** (answer 44, overriding answer 9's "stays a name"). +The grammar above does not yet spell it, and neither does the group definition +(`> Coastal`), which becomes a declaration carrying a display and an id. Those +spellings are Q33. **An axis line is tried before `format:`, and a format that reads as an axis line is pipe-quoted** (improvise-le4p; the ADR's own reading, not a user answer). @@ -823,7 +1023,8 @@ property 1 must be stated over the parsed result, not over the generated text. **Which regime a file is read under is decided by its version line alone** (clause 10). In an old-version file, a header is the whole line, a reference resolves against -display names, and ids are assigned by slugify. In a new-version file, declarations +display names, and ids are assigned by slugify. That resolution happens in the loader, +not in a client, which answer 39 may or may not allow (Q28). In a new-version file, declarations carry ids and a reference is an id. **An old file naming an item it never declares** declares it at load (answer 24). This @@ -874,6 +1075,12 @@ pipe-quotes any display name that would otherwise read as display plus id: - `AddItem { category: CategoryId, display }`, `AddItemInGroup { category: CategoryId, display, group }` and `AddFormula { target_category: CategoryId, raw }`. + - Since groups have ids (answer 44), `AddItemInGroup`'s `group` names an existing + group by its id, or creates one by its display. Which, or both, is Q33. +- **The client is the only place a display name is accepted as a reference** + (answers 32 and 39). It converts the name to an id as soon as it can, in the UI + layer, and the wire never carries a display as a reference. The one display an + operand carries is the name of the thing being created. - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. So a create-then-act compound becomes **one command** carrying both halves. `create-and-switch-view` sends `[CreateView, SwitchView]` today and @@ -1064,22 +1271,29 @@ is the capture-at-open rule the cell editor already follows: `OpenedCell` in generation and the view at open, and the commit answers for those rather than for whatever the document holds by then. - **The client holds the mapping** (answer 32), as it holds `OpenedCell`. The TUI - and the browser both use the same resolver in `improvise-protocol`. + and the browser both use the same resolver in `improvise-protocol`. The user + confirmed this and widened it: the client "should be the only place that allows any + reference to something by a human-friendly name", and converts it "to IDs ASAP" + (answer 39). - The mapping covers every display name the panel showed in that formula's text, including the target's. - A name the user types that was **not** in the captured mapping is new to the edit. The client resolves it against its own id-bearing projection (clause 6) at - commit, in answer 33's order: **ids first, then displays**. A typed name equal to + commit, in answer 33's order: **ids first, then displays** ("IDs are unambiguous so + they should always be preferred when there's ambiguity", answer 38). A typed name equal to an id names that id's thing, even where it is also another thing's display. Otherwise, if exactly one thing in scope has that display, it names that thing. If a display matches more than one thing, the edit is refused and the refusal lists the candidates' ids. If the name does not resolve at all, the edit is refused and names it. An id is matched exactly: no case folding. -- **The ids-first order is interim** (answer 33). The long-term design separates two - layers. A view-layer autocomplete of typed display names turns text into ids - before it is committed, and the model layer refers to ids only. When it exists, - this resolver's display stage and the ids-first order go with it. This ADR does not - specify the autocomplete. +- **The ids-first order was stated as interim** (answer 33). The long-term design + separates two layers. A view-layer autocomplete of typed display names turns text + into ids before it is committed, and the model layer refers to ids only. Revision 9 + read that, when the autocomplete exists, this resolver's display stage and the + ids-first order go with it. **Answer 38 says ids "should always be preferred when + there's ambiguity"**, so the preference itself is not temporary. Whether it carries + into the autocomplete (an id match offered first, say), or ends with this resolver, + is Q27. This ADR does not specify the autocomplete. - **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are in the file's id form, and the host never resolves a display name (answer 32). The host parses ids, and one that no longer exists is @@ -1180,9 +1394,11 @@ fdk1 landed in PR #221, before this ADR. What exists on main: The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14, revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All -of those are answered.** After acceptance, three implementation questions were -answered on their tickets (answers 30–32), and the user's comments on revision 9 are -answers 33 and 34, and its two later comments are answers 35 and 36. Revision 9 adds Q23 to Q25, which are open. +of those are answered**, and answer 44 later overrode Q5's answer. After acceptance, +three implementation questions were answered on their tickets (answers 30–32). The +user's comments on revision 9 are answers 33–36 (eighth round) and 37–45 (ninth +round). Revision 9 added Q23 to Q25: answers 41 and 42 settle Q24 and Q25, and Q23 is +still open. The ninth round adds Q26 to Q34, which are open. | # | question | my recommendation | status | |---|---|---|---| @@ -1190,7 +1406,7 @@ answers 33 and 34, and its two later comments are answers 35 and 36. Revision 9 | 2 | Formula id placement | F1, `- Profit(3) = …` | **answered: F1** (answer 5) | | 3 | Headers in old files | first draft: accept the rare break; revision 1: gate on version | **answered**: the space is optional (answer 6), and old files keep the whole-line reading (answer 15, Q12) | | 4 | Do `## Data` keys carry ids? | no | **answered: yes, ids** (answer 7), reversing the recommendation | -| 5 | Group ids? | not now | **answered: not now** (answer 9) | +| 5 | Group ids? | not now | **answered: not now** (answer 9). **Overridden: a group name is an id** (answer 44); the mechanism is Q33 | | 6 | Per-entity versions? | not in this ADR | **answered: not in this ADR** (answer 10) | | 7 | `RecordHash` over ids? | no, keep names | **answered: over ids** (answer 3), reversing the recommendation | | 8 | Duplicate id: repair or refuse? | revision 2: refuse | **answered: refuse the load** (answer 11) | @@ -1200,23 +1416,35 @@ answers 33 and 34, and its two later comments are answers 35 and 36. Revision 9 | 12 | Old files and the new header rule | (a) no | **answered: (a), with the version marked unambiguously** (answer 15) | | 13 | Slugify details | all generation; keep case; keep `_` `-`; strip non-ASCII; `-` | **answered** (answer 16): all generation, **lower case only** (reversing the recommendation), keep both, strip, `-` | | 14 | Formula text: ids or display names? | ids in the file | **answered**: ids in the file (answer 3); the panel is Q16 | -| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-`, and "space" means whitespace (answer 34) | +| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-`, "space" means whitespace (answer 34), and each whitespace character becomes `_` before runs of `_` collapse (answer 41) | | 16 | What the formula panel shows | none | **answered: display names, target shown as `Profit (profit)`, ids where a name is ambiguous** (answer 19) | | 17 | Capitals in a hand-written id | (a) no | **answered: no** (answer 20) | -| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse** (a run of whitespace, answer 34), plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | +| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse** (a run of whitespace, answer 34; by collapsing runs of `_`, answer 41), plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | | 18(b) | What "casefold" means | full case folding | **answered: full Unicode case folding** (answer 26) | -| 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answer 22), stronger than (a) | -| 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23) | +| 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answers 22 and 37), stronger than (a) | +| 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23); the reservation covers "virtual categories and measures and other entities" (answer 42), and `_index`/`_dim` are restated (answer 43) | | 21 | An old file naming an undeclared item | (a) declare it at load | **answered: (a)** (answer 24) | | — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal | | 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) | -| — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation | -| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31). **Order reversed: ids first until there is autocomplete** (answer 33) | -| — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32) | -| — | Whether a measure has an id/display split, and which formula targets are normal (user comments on revision 9) | — | **answered in principle**: measures have the split (answer 35), and the normal target is a measure (answers 35, 36). The mechanism is Q23 | -| 23 | How a formula-target measure carries its id, and how a formula line spells its target (revision 9; reworded after answers 35 and 36) | none until the user chooses | **open** | -| 24 | Does `a_ b` slug to `a__b`? (answer 34) | (a) keep | **open** | -| 25 | Does a trailing whitespace run slug to a trailing `_`? | (a) keep | **open** | +| — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation; a formula id that collides takes the counter's suffix (answer 37) | +| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31). **Order reversed: ids first until there is autocomplete** (answer 33). Ids are always preferred under ambiguity (answer 38); whether that outlives the interim resolver is Q27 | +| — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32), widened to every reference by a human-friendly name (answer 39) | +| — | Whether a measure has an id/display split, and which formula targets are normal (user comments on revision 9) | — | **answered in principle**: measures have the split (answer 35), the normal target is a measure (answers 35, 36), and a formula is a derived measure (answer 40). The mechanism is Q23 | +| — | Revision 9's reading that `_Index`/`_Dim` coordinates are outside the id namespace | (the ADR's reading) | **rejected as worded** (answer 43); what the positions are is Q31 | +| — | Revision 9's reading that `collapsed:` names a group by name | (the ADR's reading) | **overridden: a group name is an id** (answer 44) | +| — | Revision 9's reading that a slug of only `-` is a legal id | (the ADR's reading) | **rejected: `-` is not an id, and no id begins or ends with `-`** (answer 45) | +| 23 | Is a formula's derived measure one thing with one id, or two things with two ids? (revision 9; reworded after answers 35 and 36; restated after answer 40) | none until the user chooses | **open** | +| 24 | Does `a_ b` slug to `a__b`? (answer 34) | (a) keep | **answered: (b), runs of `_` collapse**, so `a_ b` → `a_b` (answer 41), not the recommendation | +| 25 | Does a trailing whitespace run slug to a trailing `_`? | (a) keep | **answered: (b), a trailing `_` is dropped** (answer 42), not the recommendation | +| 26 | What may a formula refer to, and what is "a category value"? (answer 37) | (iii) only keep ids unique | **open** | +| 27 | Does ids-first outlive the interim resolver? (answer 38) | none | **open** | +| 28 | May the old-version loader and headless `cmd`/`script` resolve displays? (answer 39) | yes, both | **open** | +| 29 | Does step 4's collapse come after the strip? (answer 41) | (a) after | **open** | +| 30 | Which "other entities" share the reserved `_`? (answer 42) | the virtual views are among them | **open** | +| 31 | Are `_Index`'s and `_Dim`'s positions ids? (answer 43) | none | **open** | +| 32 | Moving the virtual categories to lower-case ids (answers 23 and 43) | as written there | **open** | +| 33 | How groups carry ids (answer 44) | none, beyond the readings stated | **open** | +| 34 | Hand-written ids with `-` or a trailing `_`, and how the trims repeat (answer 45) | refuse `-` at either end; repeat the trims | **open** | ## Still open @@ -1228,120 +1456,251 @@ review were left as tickets rather than ADR text: improvise-le4p, improvise-m56i improvise-p003. Revision 9 folds all three in, and improvise-m56i's two questions are answers 30 and 31. -**Revision 9 needs the user's approval, and it asks three questions**, Q23 to Q25. +**Revision 9 needs the user's approval.** It asked Q23 to Q25, and the ninth round +answered Q24 and Q25. Q23 and the ninth round's Q26 to Q34 are open. -**Q23. How does a formula-target measure carry its id, and how does a formula line -spell its target?** (Reworded after answers 35 and 36. Revision 9 asked only how -the line spelled its target, and it had the dimension case in view.) +**Q23. A formula is a derived measure. Is the measure and its formula one thing with +one id, or two things with two ids?** (Restated after answer 40, which asked for this +question to be stated more clearly. Revision 9 asked how a formula line spells its +target; after answers 35 and 36 it asked how a formula-target measure carries its id. +It is the same question, stated in the user's terms.) -The user's words: measures have an id and a display, and measures are the normal -formula target. What is open is the *mechanism*. A stored measure is already an item -with both. A measure created by a formula (`profit`) has, in revision 9, no item: -its display lives on the `Formula`, and its id is the formula's own id. Clause 5 -therefore spells the line `- Profit() = … [_measure]` (the display bare) -for the normal case, and `- () = … []` for the rare -dimension case, so the two cases put different kinds of thing in one position. +**What is decided.** +- A formula is a derived measure (answer 40), and a measure is the normal formula + target (answers 35 and 36). +- A measure has an id and a display (answer 35). +- Every reference is an id (answer 3), and no two ids in a document are equal + (answers 22 and 37). +- A formula carries an id in its line, after the target and before the `=` + (answer 5), and `ReplaceFormula` and `RemoveFormula` address a formula by it + (clause 6). +- The rare dimension target is a real item with a separate formula id (answer 30), + and a formula id that collides takes the counter's suffix (answer 37). -- **(a) Keep the handle: the target's id is the formula's id.** The display lives on - the `Formula`. The id is generated from the target's display, and with no item to - collide with it comes out bare: `Profit` → `profit`, not `profit-`. - `- Profit(profit) = revenue - cost [_measure]`, and a dimension target is - `- tax(tax-1) = … [customcat]`. References to the measure are `profit`. This is - the smallest change. Its cost is that the normal case writes a display in a - position that holds an id for the rare one. Under (a) a measure does have an id - and a display; whether that is the split the user asks for is the thing to decide. -- **(b) Always a display, then ids in tags.** Measure: `- Profit(profit) = … [_measure]`. - Dimension: `- Tax(tax-1)(tax) = … [customcat]`. The target position never mixes - kinds. A dimension line gains a second tag. -- **(c) A formula-target measure is a real `_Measure` item**, like a dimension target - under answer 30: its own `ItemId` and display, and a separate `FormulaId` - (`profit`, then `profit-`). The target position is the item id in every case: - `- profit(profit-1) = revenue - cost [_measure]`. It removes `ItemHandle::FormulaTarget`, - returns `CellKey` to `(CategoryId, ItemId)`, and ends the `CellKey` typing problem - Codex raised on this PR. It reverses clause 1's "no item of its own" and revision - 9's readings in clauses 1, 2, 5 and 8. It also needs the item and its display - written somewhere: today only non-formula `_Measure` items persist, and formula - targets are rebuilt from `## Formulas`. +**The worked example.** A document has two stored measures, `Revenue` and `Cost`. +Each is an item of `_Measure`, with ids `revenue` and `cost`, and each holds data. +The user adds `Profit = Revenue - Cost`. `Profit` holds no data of its own: its +column is computed by the formula, which is what makes it a derived measure. Two +kinds of thing must then name Profit by an id: +1. **Things that refer to the measure.** Another formula (`Margin = Profit / + Revenue`), the `CellKey` of each Profit cell, a `hidden:` line, and ADR 008's + cursor anchor. +2. **Things that refer to the formula.** `ReplaceFormula` and `RemoveFormula`, which + edit or delete the formula. -Answers 5 and 19 picture the measure case as `Profit(3)` and `Profit (profit)`. -Answer 19's panel picture fits (a), (b) and (c): each gives the measure a display and -an id, and under (c) the `profit` is the item id. Answer 5's file-line picture, -`- Profit(3) = …`, fits (a) and (b) exactly, and (c) only loosely, because (c) writes -the item id in the target position where answer 5 shows the display. +**The question is whether 1 and 2 use the same id.** -**No recommendation is made until the user chooses.** Revision 9 recommended (a), -written when the dimension case was the one in view. Any ranking of how well an option -fits answer 35 would be the ADR's own judgement and not the user's, since (a) and (b) -also give a measure an id and a display. (a) is the smallest change. +- **(a) One thing, one id.** The formula *is* the measure. Profit's id is `profit` + and its display is `Profit`, both held on the formula. `Margin = profit / revenue`, + each Profit cell's key holds `profit`, and `ReplaceFormula` addresses `profit`. + The file line is `- Profit(profit) = revenue - cost`. No `_Measure` item exists for + Profit, so in code the column's coordinate is `ItemHandle::FormulaTarget(profit)` + (clause 2). This is revision 9's model, and the smallest change. +- **(c) Two things, two ids.** The measure is a `_Measure` item with id `profit` and + display `Profit`. The formula that computes it is a separate thing, with id + `profit-1`, suffixed because `profit` is taken (answer 37). `Margin = profit / + revenue` and each Profit cell's key holds `profit`, while `ReplaceFormula` + addresses `profit-1`. The file line is `- profit(profit-1) = revenue - cost`. This + is answer 30's model applied to measures too, so a measure target and a dimension + target work the same way. +- **(b) is not a third model.** It is (a)'s model, with the rare dimension line + spelled `- Tax(tax-1)(tax) = … [customcat]` (display, formula id, item id), so that + the target position always holds a display. Under (a) and (c) the dimension line + is `- tax(tax-1) = … [customcat]`, with the item id in the target position. -Four further points the choice must settle, all the ADR's own reading and none a -user answer: -- **A formula targeting a stored measure of the same name.** `measure_item_names` - merges them today, so a stored `Profit` column and a formula `Profit = …` are one - measure. Under (a) and (b) that one column has two ids (the item's `profit` and the - formula's `profit-`), and the ADR must say which one keys and references use. - Under (c) it is the natural case, and the formula attaches to the existing item. -- **Removing a formula under (c).** `RemoveFormula` drops the formula and leaves its - `_Measure` item, as it leaves a dimension target's item today (answer 30). The - measure column then has no formula and possibly no data. +**Where (a) and (c) differ.** + +| | (a) one id (and (b)) | (c) two ids | +|---|---|---| +| A Profit cell's coordinate | the formula's id, as `ItemHandle::FormulaTarget` | an ordinary item id | +| `CellKey` | `(CategoryId, ItemHandle)` | `(CategoryId, ItemId)`, and `ItemHandle` goes | +| A stored `Profit` column and a formula `Profit = …`, merged into one measure today by `measure_item_names` | one column with two ids (the item's `profit`, the formula's `profit-1`), and the ADR must say which keys the cells | the formula attaches to the existing item | +| `RemoveFormula` on Profit | the measure goes too | the item `profit` stays, with no formula and maybe no data, as a dimension target's item stays today | +| What the file must newly write | nothing: the formula line declares the measure | the item `profit` and its display (today a formula target is not written as an item, and is rebuilt from `## Formulas`) | +| A dimension target (answer 30) | works differently from a measure target | works the same way | +| ADR 008 (PR #308), which anchors the cursor by item identity | needs `ItemHandle` | works as it is | +| ADR 009 (PR #309), which uses `find_item_category`'s `_Measure` fallback for formula targets | keeps it | loses it | + +Neither ADR 008 nor ADR 009 is edited here. + +**What is open.** The choice between (a) and (c) and, under (a), whether the +dimension line is spelled (a)'s way or (b)'s. **No recommendation is made.** Answer +40, "Formulas are a derived measure", reads most literally as (a), where the formula +*is* the measure. But (c) also models a derived measure, an item whose values a +formula derives, and the answer was a request for clarity rather than a choice. So +reading it as choosing (a) would be the ADR's judgement, not the user's. + +Smaller points, each the ADR's own reading and none a user answer: - **The `[_measure]` tag.** Today's writer omits the tag for a `_Measure` formula (`- {raw}`), and the user's own example, `profit = revenue - cost`, has none. The - examples above write it only to show the category. Whether the tag is omitted for - `_measure` is not decided here. -- **Other ADRs.** ADR 008 (PR #308) anchors the cursor by item identity, which under - (a) or (b) a formula-measure column does not have, so it would need `ItemHandle`. - ADR 009 (PR #309) relies on `find_item_category`'s `_Measure` fallback for formula - targets, which (c) would remove. Neither ADR is edited here. - -Two consequences, both the ADR's own reading, for whichever is chosen: -- **References to a formula target** are the id text of its handle: under (a) and - (b) the item id outside `_Measure` and the formula id for a `_Measure` target; - under (c) always the target's item id. + examples above omit it too. Whether the new writer keeps omitting it is not decided + here. +- **Answers 5 and 19** picture the measure case as `Profit(3)` (the file line) and + `Profit (profit)` (the panel). The panel picture fits every option. The file-line + picture, a display before the id, fits (a) exactly and (c) only loosely, because + (c) writes the item id where answer 5 shows the display. +- **References to a formula target** are the id text of its handle: under (a) the + item id outside `_Measure` and the formula id for a `_Measure` target; under (c) + always the target's item id. - **Quoting the target position.** Where a display sits in the target position - (a, b), a display that looks like `Name(x)` would read back as a display and an id, - so the writer pipe-quotes it: `- |Margin(%)|(margin) = …`. An item id never - contains a parenthesis, so only a display needs it. (c) has no such position. + ((a) for a measure, and (b)), a display that looks like `Name(x)` would read back + as a display and an id, so the writer pipe-quotes it: `- |Margin(%)|(margin) = …`. + An item id never contains a parenthesis, so only a display needs it. (c) has no + such position. -**Q24. Does `a_ b` slug to `a__b`?** Answer 34 collapses a run of whitespace to one -`_`, and the ADR applies the steps as written, so an underscore the display already -had is not merged with the one a run makes. The user did not say. The author's -lean, not the user's, is (a): it is what the user's wording ("consecutive spaces") -and PR #318 give. -- **(a) Keep it.** `a_ b` → `a__b`, and a deliberate `a__b` stays. -- **(b) Collapse any run of whitespace and `_` to one `_`.** `a_ b` → `a_b`, and - `a__b` → `a_b`. That changes ids for displays that hold a double underscore. +**Q24. Does `a_ b` slug to `a__b`?** **Answered by answer 41: no.** Revision 9's +options were (a) keep it, its lean, and (b) collapse any run of whitespace and `_` +to one `_`. Answer 41, "The spaces should have been normalized to _ before +collapsing runs of _s", is (b): `a_ b` → `a_b` and `a__b` → `a_b`, so ids change for +displays that hold a double underscore. Its example, `Time \u{200B} Period`, now slugs +to `time_period`. Clause 4's steps 2 and 4 state it, and where step 4 goes is Q29. -A neighbouring edge: step 2 runs before step 3 strips non-alphabet characters, so -`Time \u{200B} Period` (a zero-width space between two spaces) slugs to -`time__period`, two runs and not one. +**Q25. Does a trailing run slug to a trailing `_`?** **Answered by answer 42: no.** +Revision 9's options were (a) keep it, its lean, (b) remove a trailing `_` as a +leading one is removed, and (c) trim whitespace before step 2. Answer 42 is (b): "A +trailing one should be dropped", so `East ` → `east` and `East_` → `east`, and a +slug that then collides takes the suffix. Clause 4's step 5 states it. -**Q25. Does a trailing run slug to a trailing `_`?** Step 4 removes leading `_` and -nothing removes trailing ones, so `East ` → `east_`. The user did not say. The -author's lean, not the user's, is (a). -- **(a) Keep it.** It is what PR #318 does. -- **(b) Remove trailing `_` as step 4 removes leading `_`.** `East ` → `east`, and - also `East_` → `east`. -- **(c) Trim whitespace before step 2.** `East ` → `east`, and `East_` stays `east_`. +**Q26. What may a formula refer to, and what is "a category value"?** Answer 37 says +"Formulas may refer to anything in a typed context", with the example of a formula +counting "the number of times a particular letter appears in a category value". +Today a formula reference (`Expr::Ref`) names an item or a formula target, and +`find_item_category` finds its category. The answer does not say: +- (i) whether a bare id in formula text may also name a category, a view or a group, + with its kind given by where it appears (an aggregate's argument, a `WHERE` clause, + a function's parameter); +- (ii) whether the text a function like that reads is the item's display (`Café`) + or its id (`caf`). The two differ, because slugify folds case and strips; +- (iii) whether this ADR must specify any of it, or only keep ids unique so that it + can be added later. -Revision 9 states a reading of its own, which is not a user answer, in each of these -eleven places (three of them hold two readings). Each is flagged where it appears, and +The ADR's lean, not the user's, is (iii): the formula language has no string +function today, and answer 22 already makes a reference to any kind unambiguous. + +**Q27. Does ids-first outlive the interim resolver?** Answer 33 made ids-first the +order "before we have autocomplete". Answer 38 says ids "should always be preferred +when there's ambiguity". +- **(a) Yes.** Wherever typed text could name both an id and a display, the id wins, + and that includes the autocomplete's ranking (an id match offered first, say). + Clause 12's "interim" wording goes. +- **(b) No.** Once the autocomplete turns display text into ids, nothing is matched + and nothing is ambiguous, so answer 38 governs only until then. + +No recommendation. + +**Q28. Two places resolve a display outside a client.** Answer 39: "the client should +be the only place that allows any reference to something by a human-friendly name". +- **The old-version loader** resolves an old file's references against display + names (clause 5), because in an old file the name is the only identity there is. +- **Headless `improvise cmd` and `improvise script`** run command lines that name + things by display, with no client attached. + +The ADR's reading: both are allowed. The loader migrates a file and is not a person +referring to something. The CLI is the UI layer of its own invocation, and it +converts names to ids at its edge, as a client does. Is that right? + +**Q29. Does step 4's collapse come after the strip?** Clause 4 puts it after step 3. +That order is what makes answer 41's example, `Time \u{200B} Period`, give +`time_period`. Before the strip, the zero-width character keeps the two `_` apart, and +the result would still be `time__period`. +- **(a) After the strip**, as clause 4 is written. The ADR's lean. +- **(b) Before the strip.** + +**Q30. Which "other entities" share the reserved `_`?** Answer 42: "A leading _ is +reserved for virtual categories and measures and other entities." +- (i) No virtual measure exists today. Is one planned, or does "measures" mean the + `_measure` category? +- (ii) The ADR reads the virtual views, `_records` and `_drill` (clause 1), as among + the "other entities". Is that right? +- (iii) The ADR reads "dropped to: disambiguate with -#" as "dropped too; + disambiguate with -#": a slug that collides once its `_` is dropped takes the + suffix. Is that right? + +**Q31. Are the positions under `_Index` and `_Dim` ids?** Records mode gives each row +a synthetic coordinate under `_Index`, and each column one under `_Dim` (a column +index, gotcha 1). They are never persisted and never typed. Answer 43 says the two +categories have ids `_index` and `_dim`, as answer 23 said, but revision 9's reading +was about the positions, and the answer does not say what they are. +- **(a) Outside the id namespace**, revision 9's reading: a position `0` under + `_index` does not collide with a user's item `0`. +- **(b) Inside it**, spelled so that they cannot collide, such as `_index-0`. That + needs answer 42's reservation to cover them. +- **(c) Something else.** + +No recommendation. + +**Q32. Moving the virtual categories to lower-case ids.** Today `_Index`, `_Dim` and +`_Measure` are keys in the model, on the wire, in `ViewState` and in `.improv` files: +`web/demo.improv` has `_Measure: column`, `_Index: none`, `## Category: _Measure` +and `_Measure=Revenue`. Under answers 23 and 43 they become `_index`, `_dim` and +`_measure`, and the displays stay as they are. The ADR's reading of the migration: +- an old file's `_Measure` resolves to `_measure` through its display at load, like + any other name (clause 5). The first save writes `_measure: column` and + `_measure=revenue`; +- a new-version file must spell the id, and `_Measure` there is refused (answer 20); +- the virtual category's header carries its id like any other: + `## Category: _Measure(_measure)`; +- every string literal that names them in code, on the wire and in the browser client + moves in the re-key step (clause 11, step 3). + +Is that the intended migration? + +**Q33. How do groups carry ids?** Answer 44 decides that a group has an id and a +separate friendly name. It does not decide: +- (i) **Namespace.** Is a group id unique in the whole document, like every other id + (the ADR's reading of answers 22 and 37), or only within its category? +- (ii) **Generation.** Is it the slug of the display, with `group` as an empty slug's + `` and the shared counter for collisions (the ADR's reading of answer 16(a), + "All id generation")? +- (iii) **The file.** For example a definition `> Coastal(coastal)`, a membership + `- Coastal_East(coastal_east)[coastal]`, a parent group (improvise-felo.6's + hierarchies) named by id, and a `collapsed:` line holding the group id, with or + without its category id. +- (iv) **The wire.** Does `AddItemInGroup`'s `group` name an existing group by id, + create one by display, or either? Is there a `RenameGroup { group, to }`? +- (v) **The frame.** `AxisFrame`'s `FrameGroup` carries a group name today. PR #336 + (improvise-cq1q.12, open) adds a display beside it. + +No recommendation beyond the readings stated. + +**Q34. Hand-written ids, and how step 5's trims repeat.** Answer 45: "A slug of - +should be invalid as an id and there should be no leading or trailing -s". +- (i) Is a hand-written id that is `-`, or begins or ends with `-`, refused at load, + as one beginning with `_` is? The ADR's reading is yes (clause 5). +- (ii) Is a hand-written id ending in `_` refused too? Answer 42 drops a trailing `_` + from a slug, and does not say whether a written id may carry one. +- (iii) Step 5 repeats until neither end holds `_` or `-`, so `-_x` → `x` and `x_-` → + `x`. One pass of each would leave `_x`, which is reserved, or `x_`. The ADR's + reading is to repeat. + +**Where revision 9 states a reading of its own.** In each of these fourteen places it +states a reading that is not a user answer. Each is flagged where it appears, and approving the PR approves them: -- the virtual views' reserved ids `_records` and `_drill` (clause 1); -- the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1); +- the virtual views' reserved ids `_records` and `_drill`, now read as among answer + 42's "other entities" (clause 1; Q30); +- groups share the document's one id namespace (clause 1; Q33); - a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2; **pending Q23**); -- `collapsed:` is `(CategoryId, group name)` (clause 2); - "whitespace" means the Unicode `White_Space` property (replacing the U+0020 reading, - by answer 34), and a slug of only `-` is a legal id (clause 4); + by answer 34) (clause 4); +- step 4's collapse comes after the strip (clause 4; Q29); +- step 5's trims repeat until neither end holds `_` or `-` (clause 4; Q34); +- a hand-written id that is `-` or begins or ends with `-` is refused at load + (clause 5; Q34); - the axis-before-`format:` order and the quoting of a format that reads as an axis line (clause 5); - the formula-line shape, and a `_Measure` target's display living on the `Formula` - (clause 5; **reopened by answers 35 and 36, Q23**); + (clause 5; **pending Q23**); - CheckedBatch's created-record exemption is removed (clause 6); - a display equal to another thing's id counts as ambiguous in the formula panel (clause 6); - an id is matched exactly, with no case folding (clause 12); - the display stage of the typed-name resolver goes once autocomplete exists - (clause 12). + (clause 12; Q27, since answer 38); +- answer 42's "dropped to:" is read as "dropped too;" (ninth round; Q30). + +Three readings that revision 9 listed here are gone. The user rejected the +`_Index`/`_Dim` reading as worded (answer 43, Q31), gave groups ids, which overrides +the `collapsed:` reading (answer 44), and rejected the legal `-` slug (answer 45). ## Consequences @@ -1362,10 +1721,10 @@ approving the PR approves them: - `Formula::raw` is no longer verbatim: the file carries formula text over ids. - The formula panel is a second rendering of formula text, over display names, so the **client** must resolve display names typed there back to ids (answers 19, - 32 and 33), ids first until an autocomplete exists. The host sees only id-form - text. + 32, 33 and 39), ids first (answers 33 and 38). The host sees only id-form text. - Ids are lower case, so an old file's data keys change spelling on its first save. - ADR 002's prose needs a wording update when this lands. -**Not addressed.** Per-entity versions (Q6), group ids (Q5), and ids in `AxisFrame` and -`CellId::Derived`. +**Not addressed.** Per-entity versions (Q6), and ids in `AxisFrame` and +`CellId::Derived`. Group ids are now in scope (answer 44, overriding Q5's answer), +but how they are made, spelled and carried is Q33. diff --git a/context/code/core.md b/context/code/core.md index d17f45df..49e6c119 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1712,6 +1712,20 @@ slugs to `a__b`: the steps are applied as written, and no extra underscore-collapsing step exists. A trailing run survives, so `East ` slugs to `east_`. Only *leading* underscores are removed (step 4). +**ADR 007 revision 9 proposes different rules, and this code predates them.** +Revision 9 is not yet approved. Its answers 34, 41, 42 and 45 make clause 4: +- each whitespace character becomes `_`, not only U+0020; +- runs of `_` collapse after the strip, so `a_ b` and `a__b` both slug to `a_b` + and `Time \u{200B} Period` to `time_period`; +- leading and trailing `_` and `-` are removed, so `East ` → `east`, `-East-` → + `east`, and `-` alone takes `-`. + +This section, `slugify`, and the tests +`slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and +`slugify_keeps_an_underscore_the_display_already_had` describe the earlier rule +above. They change in a follow-up once revision 9 is approved. The ADR's clause 4 +lists every difference. + **The allocator records every id it admits and releases none.** That is the whole of retirement (clause 4, "within a session a removed id is never handed out again"). No release operation exists, so a removed id cannot be reissued @@ -1734,7 +1748,8 @@ the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading `_` (only `_index`, `_dim` and `_measure` may carry one, per answer 23, and the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other id may start with `_`"), and -duplicates. It never renames. Refusing is the loader's job, and the loader +duplicates. It accepts `-`, `-x` and `x-`. Revision 9's Q34 asks whether a +hand-written id like that must be refused (answer 45). It never renames. Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the loader keeps the line numbers. The virtual views `_records` and `_drill` are not yet in `VIRTUAL_IDS`. improvise-p003 item 1 asks for them, and they arrive From fa26ec1e417ab581cc7e725122246cfe9480e464 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Thu, 8 Oct 2026 23:09:04 -0700 Subject: [PATCH 11/27] docs(adr): condense ADR 007 to status, context, decision and open questions (improvise-cq1q.1) On the user's instruction the ADR no longer carries the decision process: no answer rounds, quotes, answer numbers or revision history. The accepted decisions and the proposed amendment are stated as design, with each amendment rule marked [Amendment] and a table of what it replaces. Settled questions are deleted; the eleven open ones are renumbered from 1 and written for a first-time reader. The verbatim record of every answer moves to a provenance comment on PR #326. Folds in the delta review of 74efa481: a group's identifier is an id, and whether it also has a display is now an open question; formulas referring to anything in a typed context is stated without deciding what that allows; the Decision's opening and Rejection::Gone include groups. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 2288 ++++++++------------------ context/code/core.md | 15 +- 2 files changed, 725 insertions(+), 1578 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 0f6dfd17..8f9ec480 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -1,1045 +1,374 @@ -# ADR 007 — Stable identities for categories, items, formulas and views - -**Status:** accepted (2026-09-25). The user approved revision 7. Revision 8 then -followed the adversarial review of that head: it adds clause 12, the ADR's own reading -of when a formula edit's names become ids, and corrects stale facts about main. The -user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17:10Z: -"looks good, approved"). -**Revision 9 is a proposed amendment, not yet approved.** It folds in the user's -answers on the implementation tickets (answers 30–32), the user's two inline -comments on its head `4aed8c07` and a clarification in chat (answers 33 and 34), -its two later inline comments, also on `4aed8c07` (answers 35 and 36), and clarifies text the reviews found unclear or stale (improvise-cq1q.1). Answer 33 -reverses the matching order that answer 31 recorded, and answer 34 overrides the -ADR's own reading of "space". Answers 35 and 36 make measures the normal formula -target and give them an id/display split, which reopens Q23. It asked three new -questions, Q23 (how a formula-target measure carries its id, and how a formula line -spells its target), Q24 and Q25 (two details of slugify's whitespace rule). -The user's nine inline comments on head `068e4130` are answers 37–45 (the ninth -round). **They answer Q24 and Q25**, rewrite slugify's steps, **give groups ids -(overriding answer 9)**, reject two of the ADR's flagged readings (the `_Index`/`_Dim` -one and the legal `-` slug) and override a third (`collapsed:`), and ask for Q23 to be -stated more clearly, which it now is. **Q23 stays open.** The ninth round adds nine -questions, Q26–Q34. The places where the ADR states a reading of its own that is not -a user answer are listed at the end of "Still open". · -**Decided by:** the user; see "What the user decided" · -**Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · -**Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and -improvise-fdk1's `RenameRecordCoordinate` (landed in PR #221) · -**Builds on:** ADR 001, ADR 002 · **Depended on:** improvise-tt22 (remove legacy JSON), -which landed in PR #224 - -**Numbering.** The ADR was first filed as 005, but improvise-l2b8 already claims ADR 005 -(exact decimal arithmetic) and improvise-ilvl claims 006. The review tickets filed -against the first draft still say "ADR 005" in their titles; they mean this document. - -**Direction from** the user, 2026-09-23, in three messages: - -> "there would be a lot of benefit to categories/formulas having stable identities, even if -> they were only transient: it would help pin editing operations and distinguish them from -> delete/remove" - -> "It's probably worth a file format change (in a opt-in way, that gets rectified on save if -> the input file was missing them)" - -> "Maybe something like `View: Default(default)` and the standard -> `View: |Default (default)|(default)` for quoting." - -## What the user decided - -The user answered the **first draft** (`context/adr/005-stable-identities.md` at -`9d97664c`) with inline comments on PR #210, and then posted one comment on the thread. -Each answer is quoted verbatim below, with where it was attached and what it overturned. -**Five of the first draft's positions were reversed.** The rest of this document is -rewritten to match. - -**1. Storage keys by id; the human name becomes a display field.** On first-draft -line 92, the clause "`CellKey`, `DataStore`, views and formula text **stay keyed by -name**": - -> "Shouldn't it be easy to replace this to key by ID? There's no type change so, at worst, -> it would be renaming field access. We could also add the name as something like -> `display` and reserve `name` or whatever it is currently for the id." - -**2. The point of ids is that keys use them.** On first-draft line 97, "ids do not make a -rename cheap. They make it *expressible*": - -> "This is a lot lower value if CellKeys, etc. don't refer to the new id" - -**3. Every reference uses the id.** On first-draft line 275, the heading of clause 8, -"ADR 002: `RecordHash` stays defined over names": - -> "No, anything referring to something with an id should refer to the id as the stable -> identity to avoid glitches." - -This answers Q7 by reversing it, and reverses clause 5's "references carry names" too. - -**4. Q1, the id's internal type.** On first-draft line 469, the recommendation to wrap "a -small string": - -> "We probably can reuse symbols as the internal type with interning?" - -Ids are strings, interned through the existing `SymbolTable`. - -**5. Q2, formula id placement.** On first-draft line 473, recommendation F1 -(`- Profit(3) = Revenue - Cost`): - -> "Yes, before the equal to the right of the name" - -**6. Q3, header syntax.** On first-draft line 480, the claim that "a header with a space -before its parenthesis (`Sales (copy)`) is unaffected": - -> "Space before the open paren is optional: document this edge case that might break -> existing docs." - -**7. Q4, data keys.** On first-draft line 485, the recommendation "no — names only": - -> "In Data, we should prefer ids that are guaranteed unique to names that might not be." - -**8. Import generates ids; legacy JSON goes.** Thread comment, 2026-09-23 06:51: - -> "For CSV or JSON import, the dimensions are to be treated as the human friendly name and -> never parsed for IDs: during import, ids should be generated by a slugify algorithm that -> strips non-alphanumeric characters and replaces spaces with hyphens(collapsing runs). -> The only supported native file format is .improv, cleanup any legacy JSON format." - -Legacy JSON removal is its own implementation ticket, **improvise-tt22**. This ADR -depended on it and did not implement it; it landed in PR #224. It also dissolves -improvise-65kf: with no legacy JSON to load, there are no numeric JSON ids to accept. - -**The answers change the ADR's centre of gravity.** The first draft put an id *beside* -the name as an index. The user's answers make the **id the key** and the name a -**display** string. So a rename becomes cheap: it changes one display field and moves no -key, no surrogate and no basis tier. The first draft had called that impossible. - -### Second round: answers on revision 2 - -The user answered revision 2 (`context/adr/007-stable-identities.md` at `5e46ce03`) -with thirteen inline comments on PR #210, posted 2026-09-23 15:03–15:11Z. Each answer -is mapped to the question **on the line it is attached to**, and quoted verbatim. - -**9. Q5, group ids.** On line 468, inside Q5 ("Recommendation: not now … the one -exception to 'every reference is an id'"): - -> "This seems right" - -Groups get no id in this ADR. **Overridden by answer 44** (ninth round): a group -name is an id, and the friendly name is separate. - -**10. Q6, per-entity versions.** On line 469, Q6 ("Recommendation: not in this ADR"): - -> "Agreed" - -**11. Q8, a duplicate or unknown id.** On line 473, the recommendation to refuse the -load: - -> "Yeah, don't load the file because you can't safely resave it" - -**12. Q9, the reading of "opt-in".** On line 476, the recommendation that an id-less -file is read forever and every save writes ids: - -> "Yes" - -**13. Q10, id reuse after a reload.** On line 485, inside option **(b)**, "Accept -reuse across a reload (recommended)": - -> "Yes, b" - -This is option (b), and (b) was the recommendation. Within a session, retirement still -prevents reuse. Across a save and reload, a removed name that comes back gets its old -slug back. - -**14. Q11, who assigns ids.** On line 492, inside option (a): - -> "The host assigns" - -**15. Q12, old-version files and the new header rule.** On line 502, inside option (a), -"No (recommended)": - -> "Yes, a as long as version is unambiguously marked" - -Old files are read as they always were, on the condition that the version is marked -unambiguously. Clause 9 says how the loader meets that condition. - -**16. Q13, slugify.** Five answers, one per item: -- **(a) where it applies.** On line 510: - > "All id generation" -- **(b) case.** On line 512, against the recommendation to keep case: - > "lower case only" -- **(c) `_` and `-`.** On line 515: - > "Yeah, keep both" -- **(d) non-ASCII letters.** On line 518: - > "Strip non lower alphanumeric or -_" -- **(e) the empty slug.** On line 520, the recommendation `-`: - > "Yes" - -So an id is lower case, and its alphabet is `a`–`z`, `0`–`9`, `-` and `_`. Clause 4 -states the rule. Q15 asked the user to confirm the order of its steps (answer 18). - -**17. Q14, formula text.** On line 529, the recommendation "ids in the file": - -> "Display in the formula panel but not as column/row/page headers." - -This does not contradict ids in the file, which answer 3 already requires. It answers -**where formula text is displayed**, and it could be read two ways. That was Q16, -which answer 19 settles. - -### Third round: answers on revision 3 - -The user answered revision 3 (`context/adr/007-stable-identities.md` at `18c82e1c`) -with three inline comments on PR #210, posted 2026-09-24 00:34–00:36Z. Each is mapped -to the question on the line it is attached to, and quoted verbatim. - -**18. Q15, the order of slugify's steps.** On line 571, Q15's recommendation "fold, then -strip": - -> "casefold, replace space with `_`, strip" - -The fold comes first, as recommended. The answer also **changes answer 8's middle -step**: a space becomes `_`, not `-`. Two details of that sentence are Q18. - -**19. Q16, what the formula panel shows.** On line 584, the paragraph that declined to -recommend a reading: - -> "It should be at least `Profit (profit) = Revenue - Cost` but if `Revenue` is -> ambiguous, you'll have to display the slugs. Working with slugs internally and making -> the friendly names view-layer is almost always the right move here." - -So the panel shows formula text over **display names**, with the target's id beside it -in parentheses. Where a display name in the text is ambiguous, the panel shows the id -instead. Ids are what the file and the model hold, and display names belong to the view -layer. This also settles answer 17: the thing displayed "in the formula panel but not as -column/row/page headers" is the target's **id**. Whether an id is enough to resolve an -ambiguous name is Q19. - -**20. Q17, capitals in a hand-written id.** On line 589, the question "May a -hand-written id contain capitals?": - -> "no." - -This is option (a). `id_token` is lower case only, and a header ending in a -parenthesised run that contains a capital loads as a whole-line display with a load -warning. The virtual categories are spelled with capitals today. That is Q20. -(Answer 21 later made a new-version file all-or-nothing, so that header is now refused -rather than warned about. See clause 5.) - -### Fourth round: answers on revision 4 - -The user answered revision 4 (`context/adr/007-stable-identities.md` at `024aa2d3`) -with five inline comments on PR #210, posted 2026-09-24 06:05–06:08Z. Each is mapped -to the question on the line it is attached to, and quoted verbatim. - -**21. Q18(a), runs of spaces, and the collision suffix.** On line 658, Q18(a) ("Does a -run of spaces collapse to one `_`?"): - -> "runs of spaces are collapsed to one `_` and then `-###` numeric suffix is added if -> duplicate (only when importing a file without IDs: a file must either have no IDs at -> all or always have IDs)" - -This decides three things: -- a run of spaces collapses to one `_`, as recommended (a run of whitespace, by - answer 34; answer 41 makes each whitespace character a `_` first and then collapses - runs of `_`); -- a duplicate slug gets a numeric suffix after a `-`; -- **a file has ids on everything or on nothing.** A new-version file with a declaration - missing its id is refused, like any file that cannot be resaved safely (answer 11). - So the earlier rule that a hand-written id wins over a generated one in the same file - is gone. - -The parenthesis also limits the suffix to generating ids for a file that has none. Two -details of that are Q22. Q18(b), what "casefold" means, was not answered in this round -(answer 26 settles it). - -**22. Q19, whether an id disambiguates formula text.** On line 678, option (a) ("Make -item ids unique in the document"): - -> "Ids have to be globally unique." - -This is stronger than option (a). **No two ids in a document are equal, whatever their -kind**, so a category, an item, a formula and a view never share one. An id in formula -text or on the wire therefore always names exactly one thing. - -**23. Q20, the virtual categories.** On line 691, option (a) ("Lower-case ids, -capitalised displays"): - -> "Virtual categories have IDs: we need to add a "slug" version such as `_index`, but -> no non-virtual id can start with `_`" - -The ids are `_index`, `_dim` and `_measure`, and the displays stay `_Index`, `_Dim` and -`_Measure`. A leading `_` is reserved for them. Clause 4 says how slugify keeps a -generated id out of that space, and clause 5 says what happens to a hand-written one. -Answer 42 widens the reservation to "virtual categories and measures and other -entities", and answer 43 restates `_index` and `_dim`. - -**24. Q21, an old file naming an undeclared item.** On line 708, option (a) ("Declare -the item at load"): - -> "yes, this is right." - -**25. The file version.** On line 731, the Consequences line "A new file version.": - -> "File version should be added explciitly if it's not already part of the format." - -It is already part of the format. The first line of every `.improv` file is a -mandatory, exact version literal (`version_line = { "v2025-04-09" ~ NEWLINE ~ … }` in -`improv.pest`), and the writer emits it. So the new format is marked by a **new -literal**, and nothing is inferred. Clause 10, new in this revision, states the rule in -one place. - -### Fifth round: answers on revision 5 - -The user answered revision 5 (`context/adr/007-stable-identities.md` at `e044c968`) -with three inline comments on PR #210, posted 2026-09-24 19:50–19:51Z. Each is mapped -to the question on the line it is attached to, and quoted verbatim. - -**26. Q18(b), what "casefold" means.** On line 766, the recommendation to take -"casefold" literally, as full case folding: - -> "full unicode" - -Slugify's first step is **Unicode full case folding**, so `Straße` → `strasse`. - -**27. Q22(a), how the suffix is spelled.** On line 771, the recommendation "a plain -number": - -> "Just a plane number." - -The suffix is `-` followed by a plain decimal number, with no padding. - -**28. Q22(b), collisions outside import.** On line 782, option (i), "Suffix it there -too": - -> "Any ambiguous id gets a suffix, if if the id is between different types of things -> identified. Don't bother implementing a strict order here: `east` `east-1` -> `east-2`, etc. just do what gensym does in CL: global increment-only counter appended -> when dedupe needed." - -This decides three things: -- **every** colliding id gets a suffix: on import, in a session, and across kinds (a - category and an item that slug alike); -- the number comes from **one counter for the document that only ever increments**, as - Common Lisp's `gensym` does, so it is not "the lowest free number"; -- the suffix is appended **only when a dedupe is needed**, so an id that does not - collide stays bare. - -Clause 4 is rewritten to match. The rules this replaces (lowest free number, and -suffixing only on import) were revision 5's. - -### Sixth round: answer on revision 6 - -The user replied at 2026-09-24 22:09Z to their own answer 28, on line 782 of -`e044c968`: - -**29. Hand-written ids are canonical.** - -> "You have to treat the file's hand-written ids as canonical, doing otherwise will -> corrupt the data section." - -This constrains answer 28. The `gensym` suffix applies only to an id the program is -**generating**, on import, when an id-less file loads, or when the host creates -something in a session. **An id written in a file is never renamed**, because every -data key and reference in that file spells it. Clause 4 says what follows: -- a generated id that would collide with a file's id takes the suffix, and the file's - id keeps its spelling; -- two identical hand-written ids in one file are refused at load, not repaired. - -### Seventh round: answers on the implementation tickets - -After the ADR was accepted, planning its implementation (epic improvise-cq1q) -raised three questions the ADR does not answer. They were answered on -2026-09-25, and the answers are recorded as comments on the tickets -(improvise-m56i and improvise-hexb). The tracker shows the repository owner's -account as the author of those comments, and agents write to it under the same -account, so authorship there does not show whose words they are. The user's -"Approved" on this PR is what confirms them. Answers 30 and 31 quote the option -chosen, as recorded, and the sentences after each are the ADR's own statement of -what it means. Answer 32 was recorded as one sentence and is quoted whole. - -**30. A formula target outside `_Measure`** (improvise-m56i item 2): - -> "Real item + separate formula id" - -`Model::add_formula` keeps creating a real item for a target outside `_Measure`. -That item has its own `ItemId`, the formula has its own `FormulaId`, and by -answer 22 the two differ. A formula line names such a target by the item's id. -This was chosen over the recommended alternative, which was to give every formula -target the `_Measure` treatment (no real item, handled by its `FormulaId`). -The user's comment on this sentence is answer 37: a formula id that would collide -takes the counter's suffix. - -**31. A typed name that is both one thing's display and another thing's id** -(improvise-m56i item 6): - -> "Display first; client holds it" - -The option is quoted as recorded. **The order it names was reversed by the user's own -comment on this answer (answer 33): until autocomplete exists, a typed name is -matched against ids first, then against displays.** What stands from answer 31 is -that the client holds the resolution (answer 32), and that a display matching more -than one thing is refused, with the refusal listing the candidates' ids. Where -answers 31 and 33 differ, answer 33 governs. The user's comment on this sentence is -answer 38: ids "should always be preferred when there's ambiguity". - -**32. Which side holds clause 12's capture** (improvise-hexb item 2), as recorded: - -> "USER DECISION 2026-09-25 (item 2): the CLIENT holds clause 12's display->id -> capture (like OpenedCell) and resolves names against its id-bearing -> projection; the wire carries only id-form text; the host never resolves a -> display name." - -Clause 12 is rewritten to match. The user confirmed this on the PR and widened it -from the formula editor to every reference (answer 39). - -### Eighth round: answers on revision 9 (head `4aed8c07`) - -The user answered revision 9 (`context/adr/007-stable-identities.md` at `4aed8c07`) -with four inline comments on PR #326: two posted 2026-10-04 02:48Z and 02:49Z, and -two posted 2026-10-05 20:22:52Z and 20:23:40Z. The GitHub API gives all four the -original commit `4aed8c07`. **All four are one round**: the user wrote the last two -before `e9659903`, which folded in the first two, existed, so they had not seen -answers 33 and 34 written up. The second comment was also clarified in chat (not on -the PR, relayed to the steward). Each is mapped to the line it was attached to on -`4aed8c07`, and quoted verbatim. - -**33. The order of answer 31.** On line 356, the sentence under answer 31, "A typed -name is matched against displays first, then against ids": - -> "This should be handled by separating the view layer autocomplete of typed display names from the model layer reference to IDs: but, before we have autocomplete, IDs are matched first." - -This decides two things: -- **The long-term design.** A view-layer autocomplete resolves typed display text to - ids. The model layer refers to ids only, and never matches a display. -- **The interim rule, until autocomplete exists.** A typed name is matched against - **ids first, then displays.** This reverses answer 31's "displays first". - -Clause 12 states the rule and what it means for a document where one thing's display -equals another's id. Nothing built yet implements either order (clause 11, step 7). - -**34. What "space" means in slugify.** On line 553, the bullet that read "Space" -as U+0020 only, so that a tab was stripped and `Time\tPeriod` slugged to -`timeperiod`: - -> "This shouldn’t matter because consecutive spaces should normalize to a single underscore" - -and, in chat: - -> "space should generally mean "whitespace"" - -This **overrides the ADR's own reading** (revision 9's, which was not a user answer). -"Space" in answers 18 and 21 means whitespace, and a run of whitespace becomes one -`_`. The user did not enumerate which characters count, so the definition in clause -4 is **the ADR's own reading of "whitespace"**, flagged there. Answers 18 and 21 -keep their wording, which this reads as "space, meaning whitespace". **Answer 41 -refines the mechanism:** each whitespace character becomes `_`, and runs of `_` are -collapsed afterwards, so a run of whitespace still ends as one `_`. - -**35. Measures get the id/display split too.** On line 1091, the end of Q23's -option (c), "Always the target's id, giving a `_Measure` target an id of its own that is -distinct from the formula's. This reverses clause 1's 'its handle is its formula's -`FormulaId`'": - -> "Measures should have ID/display split too. Measures are the normal case for formulas, anyways: something like `profit = revenue - cost`" - -**36. Dimensions are rarely formula targets.** Also on line 1091, 48 seconds later: - -> "Dimensions are mainly in formulas when you're calculating some kind of derived dimension, but usually dimensions should be relatively static and not interesting in formulas." - -These decide two things, and no more: -- **A measure has an id and a display**, as every other thing does. -- **The normal formula target is a measure** (`profit = revenue - cost`), and a - dimension (a non-`_Measure` category) target is the rare, derived case. - -They do **not** decide what a formula-target measure's id is, how its formula line -names it, or whether it is an item. That is **Q23**, reopened and reworded. Four -things in earlier text meet them, and none is resolved here: -- **Clause 1 and revision 9's flagged readings (clauses 1, 2, 5, 8)** say a `_Measure` - formula target "has no item of its own" and is handled by `FormulaTarget(FormulaId)`. - A stored measure (a non-formula `_Measure` item) is an ordinary item and already has - an id and a display. A measure that exists only because a formula targets it has - its display on the `Formula` and its id in the `FormulaId`. Whether that is the - split the user wants, or whether the target should be an item, is Q23. -- **Answer 3** ("every reference is an id") is about references. Clause 5 calls a - formula line a declaration, so the bare display in its target position is a - declaration's display and not a reference. Whether the normal case should still - spell an id there is part of Q23. -- **Answers 5 and 19** picture the measure case as a display plus one id - (`Profit(3)`, `Profit (profit)`), which fits an id/display split (answer 5's - file-line picture fits option (c) only loosely; see Q23). -- **Answer 30** and the ADR's worked example (`Tax` in `CustomCat`) are about the - *rare* case. The decision stands. The emphasis of the surrounding text changed: - revision 9 wrote the dimension target as the main case and the `_Measure` target as - the exception. - -### Ninth round: answers on revision 9 (head `068e4130`) - -The user answered revision 9 at `068e4130` with nine inline comments on PR #326, -posted 2026-10-09 05:17:38Z–05:26:39Z. The GitHub API gives all nine the original -commit `068e4130`. Each is mapped to the line it was attached to on `068e4130`, -quoted verbatim, and given the comment id. Where the ADR reads more into an answer -than its words say, the text says it is the ADR's reading and asks a numbered -question. - -**37. An id names one thing, and a colliding formula id takes the counter.** Comment -4226862456, on line 356, the last sentence of answer 30 ("This was chosen over the -recommended alternative, which was to give every formula target the `_Measure` -treatment …"): - -> "An id always refers to the same thing and so must be globally unique: if a formula has an id that would conflict, increment the counter to disambiguate it. Formulas may refer to anything in a typed context: e.g. a formula might count the number of times a particular letter appears in a category value." - -This decides: -- **A formula id that would collide is suffixed from the counter**, like any other - generated id. This restates answers 22 and 28 for the formula case, and clause 1's - example already follows it: a formula `Tax` in `CustomCat` gets item id `tax`, then - formula id `tax-`. -- **Formulas may refer to any kind of thing**, and the context of the reference - gives its type. This is why ids are unique across kinds (answer 22): a bare id in - formula text must name one thing whatever its kind. - -The example (count a letter in a category value) has no function in today's formula -language. Whether it is in this ADR's scope, what a reference may name, and whether -"a category value" means an item's display or its id are **Q26**. - -**38. Ids win under ambiguity.** Comment 4226867495, on line 368, the end of the -paragraph under answer 31 ("a display matching more than one thing is refused … -Where answers 31 and 33 differ, answer 33 governs"): - -> "IDs are unambiguous so they should always be preferred when there's ambiguity. " - -This confirms answer 33's ids-first order. The display-versus-display refusal is -untouched: no id is involved there, so there is nothing to prefer. Whether "always" -makes ids-first permanent rather than interim (clause 12 calls it interim until an -autocomplete exists) is **Q27**. - -**39. Display names live only in the client.** Comment 4226873976, on line 377, -"Clause 12 is rewritten to match.", under answer 32: - -> "Yes, the client should be the only place that allows any reference to something by a human-friendly name and this sort of reference should be minimized to the UI layer and converted to IDs ASAP" - -This confirms answer 32 and **widens it from the formula editor to every -reference**: only a client accepts a human-friendly name, it does so in the UI layer -only, and it converts the name to an id as early as it can. Clause 2's "resolved to -ids at the input edge" and clause 6's "every operand naming an existing thing is an -id" already say this. Two places in the ADR resolve a display outside a client: the -old-version loader (clause 5) and headless `improvise cmd`/`script`. Whether they -are exceptions is **Q28**. - -**40. A formula is a derived measure.** Comment 4226895692, on line 1303, the end of -Q23 ("Quoting the target position …"): - -> "Formulas are a derived measure. I don't really understand the problem here, this whole section needs to be stated more clearly." - -**Q23 is rewritten** to state the problem in those terms, with a worked example and -a plain split between what is decided and what is open. It stays **open**: the -answer says what a formula *is*, and the ADR does not read it as choosing an option. -"Still open" says why. - -**41. Whitespace becomes `_` first, then runs of `_` collapse.** Comment 4226897815, -on line 1316, Q24's "neighbouring edge", where `Time \u{200B} Period` slugged to -`time__period`: - -> "The spaces should have been normalized to _ before collapsing runs of _s" - -This **answers Q24 with option (b)**. It changes slugify's step 2 and adds a -collapse step (clause 4, steps 2 and 4): -- each whitespace character becomes `_`; -- **runs of `_` collapse to one `_`**, whether they came from whitespace or were - already in the display. So `a_ b` → `a_b` and `a__b` → `a_b`; -- `Time \u{200B} Period` → `time_period`. - -That last result needs the collapse to run **after** step 3 strips the zero-width -space. The answer says the collapse comes after the normalisation, and the example it -is attached to gives one `_` only if it also comes after the strip. So clause 4 puts -it there. That placement is the ADR's reading, and **Q29** asks the user to confirm -it. This overrides revision 9's reading and the code's comment in -`context/code/core.md` §20 ("no extra underscore-collapsing step exists"). - -**42. A leading `_` is reserved; a trailing one is dropped.** Comment 4226901843, on -line 1324, Q25's option (c): - -> "A leading _ is reserved for virtual categories and measures and other entities. A trailing one should be dropped to: disambiguate with -#" - -This **answers Q25 with option (b)**: slugify drops a trailing `_` as it drops a -leading one, so `East ` → `east` and `East_` → `east`. The ADR reads "to:" as "too;": -drop it too, and if the shorter slug then collides, disambiguate it with `-` -(answers 27 and 28). The reservation of a leading `_` now covers "virtual categories -and measures and other entities", which is wider than answer 23's three categories. -The ADR reads the virtual views `_records` and `_drill` (clause 1) as among the -"other entities". What a virtual *measure* is, and that reading, are **Q30**. - -**43. `_Index` and `_Dim` have ids.** Comment 4226904733, on line 1330, the flagged -reading "the `_Index`/`_Dim` coordinates are outside the id namespace (clause 1)": - -> "No, they have ids _index and _dim " - -This **rejects the reading as it was worded**, and restates answer 23: the categories' -ids are `_index` and `_dim`. The reading was about their *coordinates*, the -synthetic positions in records mode (`_Dim` carries a column index, gotcha 1), not -the categories. Whether those positions are ids in the document's one namespace, and -how they would be spelled so that `_index`'s `0` does not collide with a user's item -`0`, is **Q31**. Today the repository spells the virtual categories `_Index`, `_Dim` -and `_Measure`, as keys, on the wire and in `.improv` files. What the move to lower -case ids costs is **Q32**. - -**44. Groups get ids.** Comment 4226908008, on line 1332, the flagged reading -"`collapsed:` is `(CategoryId, group name)` (clause 2)": - -> "Group name should be an id for the group, not a friendly name " - -This **overrides answer 9** ("not now") and the reading it was attached to. A group -has an id, and its friendly name is a separate display. A `collapsed:` entry names a -group by its id. Clause 1's table, clause 2 and clause 5 are changed to match. How a -group's id is made and spelled, and what the wire carries, are **Q33**. The ADR -does not decide them. -- **Relation, not a dependency.** PR #336 (improvise-cq1q.12, branch - `adr007/display-2b`, open, not on `main`) gives `Group` a private display beside - its key. On the ADR's reading, under answer 44 that key becomes the group's id, as - a category's does (clause 2). Nothing here depends on #336 landing. - -**45. A slug of `-` is not an id; no id begins or ends with `-`.** Comment -4226910518, on line 1334, the flagged reading "'whitespace' means the Unicode -`White_Space` property … and a slug of only `-` is a legal id (clause 4)": - -> "A slug of - should be invalid as an id and there should be no leading or trailing -s" - -This **rejects the second half of that reading**. Slugify removes leading and -trailing `-`, so `-` and `---` leave nothing and take `-` (step 6), and -`-East-` → `east`. The first half of the bullet, the meaning of "whitespace", was not -addressed and stays the ADR's flagged reading. That a **hand-written** id with a -leading or trailing `-` is refused at load, as one with a leading `_` is, and how the -trims interleave (`-_x` → `x`), are the ADR's reading of "invalid as an id" and are -**Q34**. +# ADR 007 — Stable identities for categories, items, formulas, views and groups + +**Ticket:** improvise-sboz (implementation epic improvise-cq1q; this amendment +improvise-cq1q.1) · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's +interim `FormulaId` (PR #214) and improvise-fdk1's `RenameRecordCoordinate` +(PR #221) · **Builds on:** ADR 001, ADR 002 · Review tickets filed against an early +draft call this document "ADR 005". + +## Status + +**Accepted** on 2026-09-25 (PR #210). **An amendment is proposed and not yet +approved** (PR #326). The amendment's text is marked **[Amendment]** wherever it +appears. Everything without that mark is accepted and binding. + +Where the amendment replaces accepted text, the accepted rule binds until the +amendment is approved: + +| clause | accepted rule | the amendment proposes | +|---|---|---| +| 1 | groups have no id | a group's identifier is an id | +| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | the virtual views also have reserved `_` ids: `_records`, `_drill` | +| 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` (pending open question 1) | +| 2, 5 | a collapsed entry and a group reference name a group by name | by its id | +| 4 | a run of U+0020 spaces becomes one `_`; only a leading `_` is removed | each whitespace character becomes `_`, runs of `_` collapse after the strip, and leading and trailing `_` and `-` are removed | +| 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | + +The amendment also adds rules where the accepted text says nothing: +- a measure has an id and a display; +- the axis-line/`format:` order; +- `ViewState` and `Workbook` key the active view by id; +- CheckedBatch's created-record exemption is removed; +- two branches of one file can merge into a file that will not load. + +Each is marked where it appears. Eleven questions are open (last section). ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. `CellKey` and `DataStore`'s interned symbols are names. Views, axis maps, page -selections, hidden and collapsed sets, and formula text all refer by name. The user's -first message names what that costs: +selections, hidden and collapsed sets, and formula text all refer by name. That costs +three things: +- **An edit cannot be told apart from a delete followed by an add.** No rename + operation exists for a category or an item. A rename is remove plus add, which loses + every cell (improvise-fz77). A formula is identified by `(target, target_category)`, + and `Model::add_formula` upserts on that pair. +- **An edit whose target was concurrently replaced cannot be told apart from an edit + of the replacement.** +- **A records-mode coordinate rename names its category and item by name** + (`RenameRecordCoordinate { record, category, item }`, `String` operands). -- **An edit cannot be told apart from a delete followed by an add.** No rename operation - exists for a category or an item. A rename is remove plus add, which loses every cell - (improvise-fz77). A formula is effectively identified by `(target, target_category)`, - and `Model::add_formula` *upserts* on that pair. improvise-km71 (PR #214) made formula - editing its own operation, with an interim id built from that pair (below). -- **An edit whose target was concurrently replaced cannot be told apart from an edit of - the replacement.** -- **A records-mode coordinate rename names its category and item by name.** - improvise-fdk1 (PR #221) landed `RenameRecordCoordinate { record, category, item }` - with `String` operands (below). - -**What already exists.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize` -aliases, assigned from monotone counters. **No production caller uses either value.** -`Category::add_item` returns the id on every call, and `Model::add_category` returns -one too. Tests read them (`category.rs`'s `add_item_returns_sequential_ids` and -`add_item_duplicate_returns_same_id`, and one in `types.rs`), but no production -caller does: `parse.rs` discards the value. Neither has ever been written to -`.improv`. `View` has no id. - -**`FormulaId` exists, as an interim.** km71 landed an opaque `FormulaId` whose private -fields are `(target, target_category)`, built only by `Formula::id()` -(`context/code/core.md` §18). It is on the wire in -`ModelCommand::ReplaceFormula { id, was, raw }`, in `FormulaSlot::Existing` and in -`FormulaEntry`. It was made opaque so that this ADR changes one struct rather than every -signature. It is still a name pair, so a formula whose target is renamed gets a -different `FormulaId`. `Model::replace_formula` therefore refuses to rename a target -outside `_Measure` until this ADR lands (improvise-kb1d). - -**Legacy JSON is gone.** improvise-tt22 landed in PR #224, so `.improv` is the only -native format. +**What exists already.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize` +aliases from monotone counters. No production caller reads them, they have never been +written to `.improv`, and `View` has no id. km71 landed an opaque interim `FormulaId` +over `(target, target_category)`, built only by `Formula::id()` (`context/code/core.md` +§18). It is on the wire in `ModelCommand::ReplaceFormula { id, was, raw }`, +`FormulaSlot::Existing` and `FormulaEntry`. Because it is still a name pair, +`Model::replace_formula` refuses to rename a target outside `_Measure` +(improvise-kb1d). Legacy JSON is gone (improvise-tt22, PR #224), so `.improv` is the +only native format. ## Decision -**Categories, items, formulas and views each have an id, and the id is their key.** A -category or item's current `name` field *becomes* the id (the user's suggestion), and the +**Categories, items, formulas and views each have an id, and so do groups +[Amendment]. The id is the key.** The current `name` field becomes the id, and the human name moves to a new `display` field. -- **Every reference is an id.** That covers `CellKey`s, axis maps, page selections, - hidden and collapsed sets, formula references, data keys, and wire operands that name - an existing thing. -- **Display names are for people.** They appear on screen, are typed into minibuffers, - and are resolved to ids at the input edge. - -Twelve clauses. +- **Every reference is an id**: `CellKey`s, axis maps, page selections, hidden and + collapsed sets, formula references, data keys, and every wire operand that names an + existing thing. A reference to the stable identity cannot glitch when something is + renamed or replaced. +- **Display names are for people.** They appear on screen, are typed into + minibuffers, and are turned into ids at the input edge. ### 1. What carries an identity -| kind | id | unique within | wire address | -|---|---|---|---| -| category | yes | the document, across every kind (answer 22) | `CategoryId` | -| item | yes | the document, across every kind (answer 22) | `ItemId` | -| formula | yes | the document, across every kind (answer 22) | `FormulaId` | -| view | yes (the user's example) | the document, across every kind (answer 22) | `ViewId` | -| record | `RecordHash`, now over ids (clause 8) | the document | unchanged shape | -| group | **yes** (answer 44, overriding answer 9) | the document, across every kind (the ADR's reading of answers 22 and 37; Q33) | to be decided (Q33) | +| kind | id | wire address | +|---|---|---| +| category | yes | `CategoryId` | +| item | yes | `ItemId` | +| formula | yes | `FormulaId` | +| view | yes | `ViewId` | +| group | **[Amendment]** yes (open questions 9 and 10) | open question 10 | +| record | `RecordHash`, over ids (clause 8) | unchanged shape | -**Ids are unique in the whole document** (answers 22 and 37: "An id always refers to -the same thing and so must be globally unique"). One namespace covers categories, -items, formulas and views, and on the ADR's reading groups too (Q33), so an item id -names one item without its category. A -`CellKey` still pairs each item with its category, because the key needs the -categories. An operation that creates an item still names the parent category. +**An id always names the same thing, so ids are unique in the whole document**, across +every kind. An item id names one item without its category. A `CellKey` still pairs +each item with its category, and an operation that creates an item still names the +parent category. A formula whose generated id would collide takes a suffix like any +other generated id (clause 4). A formula `Tax` in `CustomCat` gets item id `tax`, then +formula id `tax-`. **[Amendment]** Formulas may refer to anything in a typed +context. What exactly that allows is open question 2. -**Virtual categories have reserved ids** (answers 23 and 43): `_index`, `_dim` and -`_measure`, with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin -with `_`**: a leading `_` "is reserved for virtual categories and measures and other -entities" (answer 42). Moving today's spellings to these ids is Q32. -**Measures are the normal formula target** (answers 35 and 36), and **a measure -has an id and a display**. A stored measure is an ordinary item. Revision 9 says a -`_Measure` formula target has no item of its own and that its handle is its -formula's `FormulaId`; that is **under question again (Q23)**, because it is -the normal case. On revision 9's reading, this is why the category panel's `_Measure` rows need -`ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). **A formula -target outside `_Measure`, the rare case (answer 36), is a real item** (answer 30). It has its own `ItemId`, -and its formula has a separate `FormulaId`. Generated ids follow clause 4, so a -formula `Tax` in `CustomCat` gets item id `tax` and then formula id `tax-` -(answer 37: "if a formula has an id that would conflict, increment the counter"). -The panel lists that row as `ItemHandle::Item`, like any other item. +**Reserved ids.** The virtual categories have ids `_index`, `_dim` and `_measure`, +displayed `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** +**[Amendment]** A leading `_` is reserved for virtual categories, measures and other +virtual entities. The virtual views have ids `_records` and `_drill`, displayed +`_Records` and `_Drill`. They are never persisted, but they are addressed on the wire +(`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and could collide +with a user's view. Which entities the reservation covers is open question 6. Whether +records mode's synthetic positions under `_Index` and `_Dim` are ids is open question 7. +Moving today's capitalised keys to these ids is open question 8. -**The virtual views have reserved ids too**: `_records` and `_drill`, with the -displays `_Records` and `_Drill`. They are never persisted, but they are addressed on -the wire (`SwitchView`, improvise-obtu). Revision 9 introduced this as its own -reading, extending answer 23's reservation from the virtual categories to the -virtual views, because a slugified `_Drill` would otherwise be `drill`, which could -collide with a user's view (improvise-p003 item 1). Answer 42 now reserves a leading -`_` for "virtual categories and measures and other entities". **That the virtual -views are among the "other entities" is the ADR's reading** (Q30). - -**The `_Index` and `_Dim` categories have ids `_index` and `_dim`** (answers 23 and -43). Revision 9 read their *coordinates*, the synthetic record positions (`_Dim` -carries a column index, gotcha 1), as outside the id namespace, so that `_index`'s -`0` could not collide with a user item whose id is `0` (improvise-p003 item 2). **The -user rejected that reading as worded** (answer 43, "No, they have ids _index and -_dim"). The ADR does not read the answer as deciding what the positions are: that -is **Q31**. Until it is answered, nothing in this ADR depends on the positions' -spelling, because they are never persisted and never typed. +**Formula targets.** **[Amendment]** A measure is the normal formula target +(`profit = revenue - cost`). A dimension (a category other than `_Measure`) is a rare +one, for a derived dimension. +- **A target outside `_Measure` is a real item**, with its own `ItemId`, and its + formula has a separate `FormulaId`. +- **A `_Measure` target has no item of its own**, and its handle is its formula's + `FormulaId`. So the category panel's `_Measure` rows use + `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). +- **[Amendment]** A measure has an id and a display, as everything else does. + Whether a formula-target measure keeps the formula's id or becomes an item with + its own is open question 1. The `ItemHandle` text in clauses 2, 5, 6, 8 and 11 + depends on its answer. ### 2. Storage keys by id; `display` is the only place a human name lives -Following answers 1 to 3: - **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id, and each `Category`, `Item` and `View` gains a `display: String`. -- **Cells and derived surrogates:** a `CellKey` is a sorted list of - `(CategoryId, ItemHandle)` pairs (the ADR's own reading of answer 30, not a - user answer), where `ItemHandle` is clause 6's - `{Item(ItemId), FormulaTarget(FormulaId)}`. Outside `_Measure` every coordinate is - an `Item`. A `_Measure` formula cell's coordinate is `FormulaTarget`, because - that target has no item (**pending Q23**: option (c) gives it an item, and this - reading falls). The key stays sorted, now **by category id**, so gotcha - 5's canonical form is kept with a different sort key. `DataStore` interns ids - exactly as it interns names today. Answer 22 makes every id unique across kinds, - so an id's text alone names one thing; the tag is there so that the types - never claim an `ItemId` the document does not hold. Revision 9's first draft - wrote this pair as `(CategoryId, ItemId)` while also putting a `FormulaId` in it - (Codex on PR #326). +- **Cells:** a `CellKey` is a sorted list of `(CategoryId, ItemId)` pairs. + **[Amendment]** The pairs are `(CategoryId, ItemHandle)`, so that a `_Measure` + formula cell's coordinate is `FormulaTarget` without claiming an `ItemId` the + document does not hold (pending open question 1). + - The key stays sorted, now **by category id**, so the canonical form (gotcha 5) + is kept with a different sort key. + - `DataStore` interns ids exactly as it interns names today. + - An id's text names one thing, because ids are unique across kinds. - **View configuration:** the axis maps, page selections, and hidden and collapsed sets - are keyed by ids, with no exception. A collapsed entry names a group, and a - group name is an id (answer 44), so the entry is the category id and the group's - id. Revision 9 had made it `(CategoryId, group name)`, because answer 9 gave groups - no id (improvise-p003 item 4); answer 44 overrides both. Whether the category id - is still needed beside a group id that is unique in the document, and how the - line is spelled, are part of Q33. -- **Formulas:** a formula's references are ids (clause 5). The formula panel renders - them as display names (answer 19; clause 6). + are keyed by ids. **[Amendment]** A collapsed entry names its group by the group's + id. Whether the category id stays beside it is part of open question 10. +- **Formulas:** a formula's references are ids (clause 5), and the formula panel + renders them as display names (clause 6). -**What a rename now costs:** one `display` write. It moves no `CellKey`, no `RecordHash` -and no `layout_version`, so an edit a client composed before the rename still lands after -it. That is the "avoid glitches" the user's rule is for. The grid's labels are display -strings, so the rename's effect must still dirty the grid and the panels -(`Dirty::of` gotcha 35), but no basis tier moves. +**A rename costs one `display` write.** It moves no `CellKey`, no `RecordHash` and no +`layout_version`, so an edit a client composed before the rename still lands after it. +The grid's labels are display strings, so a rename must still dirty the grid and the +panels (`Dirty::of`, gotcha 35), but no basis tier moves. -**What the migration costs, stated honestly.** Every API that takes a name today -(`Model::category(name)`, `Workbook::add_item(cat, item)`, `CellKey::new` callers, the -formula evaluator's `Expr::Ref` lookup, `View` methods) takes an id instead. The -display→id resolution moves to the places a human types, such as the minibuffer commits, -the formula parser's reference resolution, and import. The change is mechanical but wide, -and it is the bulk of the implementation. The user judged it cheap ("at worst, it would -be renaming field access"), and for the field that is right. The call sites are the -cost. +**The migration cost is the call sites.** Every API that takes a name today takes an +id instead: `Model::category(name)`, `Workbook::add_item(cat, item)`, `CellKey::new`'s +callers, the evaluator's `Expr::Ref` lookup and the `View` methods. Display→id +resolution moves to the places a human types. The field rename itself is cheap. The +change is mechanical but wide, and it is the bulk of the implementation. -**Display names stay unique within their scope**, as names are today. So a human-typed -display resolves to at most one id within a scope, and a rename to a display name -already in use is refused. Typed text is matched against ids before displays -(answer 33; clause 12). An item's scope is its category, so two categories may each have an item -displayed `Revenue`. Formula text crosses categories, which is where answer 19's -"ambiguous" arises. Their ids differ (answer 22), so the panel resolves the ambiguity -by showing the ids. +**Display names stay unique within their scope**, as names are today, and a rename to +a display already in use there is refused. An item's scope is its category, so two +categories may each have an item displayed `Revenue`. Formula text crosses +categories, so a display may be ambiguous there. The ids differ, so the formula panel +shows the ids where a display is ambiguous. ### 3. Ids are interned symbols -Following answer 4, an id is a **string**, and in memory it is a `Symbol` from the -existing `SymbolTable`: copyable, with integer comparison. Two consequences. - -- **A `Symbol` never crosses the wire or reaches the file.** It is an index into one - process's table. The wire and `.improv` carry the id's **text**, and each end interns - it on arrival. -- **`Rejection` stops being `Copy`**, because `Rejection::Gone` (clause 7) carries an id's - text. `basis.rs`'s `SaveFailed` carries no message *because* `Rejection` was `Copy`, so - that reason lapses. Whether `SaveFailed` now gains its message is the implementer's - call. +An id is a **string**, and in memory a `Symbol` from the existing `SymbolTable`: +copyable, compared as an integer. +- **A `Symbol` never crosses the wire or reaches the file.** It indexes one process's + table. The wire and `.improv` carry the id's text, and each end interns it on + arrival. +- **`Rejection` stops being `Copy`**, because `Rejection::Gone` (clause 7) carries an + id's text. `SaveFailed` carries no message only because `Rejection` was `Copy`. + Whether it gains one is the implementer's call. ### 4. How an id is made: slugify, one rule everywhere -Following answers 8, 16 and 18, **every** id the app generates comes from the display -name by **slugify**. That covers import, interactive creation, and assigning ids to an -id-less `.improv` file at load (answer 16(a), "All id generation"). So a thing's id -never depends on how it was made. The steps, in order (answer 18, "casefold, replace -space with `_`, strip", as answers 34, 41, 42 and 45 refine it): +**Every** id the app generates comes from the display name by **slugify**: on +import, on interactive creation, and when an id-less `.improv` file is loaded. So a +thing's id never depends on how it was made. An id is lower case, in the alphabet +`a`–`z`, `0`–`9`, `-` and `_`. -1. **Casefold, with Unicode full case folding** (answers 16(b), 18 and 26). Rust's - `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped), so this - needs a case-folding table or a small crate in `improvise-core`. It must pass the - `wasm` gate, because that crate is in the browser bundle. -2. **Replace each whitespace character with `_`** (answers 18, 34 and 41; answer 18 - changed answer 8's `-`). "The spaces should have been normalized to _ before - collapsing runs of _s" (answer 41), so this step makes one `_` per character, and - step 4 does the collapsing. -3. **Strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`** (answers 16(c) - and 16(d)). This strips non-ASCII letters, so an id never needs quoting in an - `.improv` reference (clause 5 notes where formula text differs). -4. **Collapse each run of `_` to one `_`** (answers 21 and 41). It does not matter - whether an underscore came from whitespace or was in the display already. **The - step comes after the strip, which is the ADR's reading** (Q29): it is the order in - which answer 41's example, `Time \u{200B} Period`, gives one `_`. -5. **Remove every leading and trailing `_` and `-`**, until the slug neither begins - nor ends with either. - - A leading `_` "is reserved for virtual categories and measures and other - entities" (answers 23 and 42). Removing it is this ADR's way of meeting that - constraint, so `_Draft` → `draft`. - - A trailing `_` "should be dropped" (answer 42). - - "There should be no leading or trailing -s" (answer 45). - - Repeating the removal until neither character is left at either end (`-_x` → - `x`) is the ADR's reading (Q34). -6. **If nothing is left, use `-`** (answer 16(e)), such as `category-1` or - `item-4`. `n` comes from the same counter as every other suffix (answer 28), not - from the lowest free number. This covers a display that is only `-` or only `_`, - because "a slug of - should be invalid as an id" (answer 45). +The steps, in order, as the amendment states them **[Amendment]**: +1. **Casefold, with Unicode full case folding**, so `Straße` → `strasse`. Rust's + `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped). The folding + table or crate must pass the `wasm` gate, because `improvise-core` is in the browser + bundle. +2. **Replace each whitespace character with `_`.** "Whitespace" is the Unicode + `White_Space` property (`char::is_whitespace`): U+0009–U+000D, U+0020, U+0085, + U+00A0, U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. + - A tab, a no-break space and an ideographic space each become `_`. + - A zero-width space (U+200B) and a byte-order mark (U+FEFF) are not whitespace, + and step 3 strips them. + - The realistic case is a header pasted from a web page, where a no-break space + stands for a space. +3. **Strip every character outside the id alphabet.** This strips non-ASCII letters, + so an id never needs quoting in an `.improv` reference. +4. **Collapse each run of `_` to one `_`**, whether the underscores came from + whitespace or were in the display. The step comes after the strip (open + question 5). +5. **Remove every leading and trailing `_` and `-`**, repeating until neither end + holds either (open question 11). + - A leading `_` is reserved (clause 1). + - A trailing `_` is dropped. + - An id never begins or ends with `-`. +6. **If nothing is left, use `-`**, such as `category-1` or `item-4`. `n` + comes from the shared counter below. A display of only `-`, `_` or non-ASCII letters + ends here, because `-` alone is not an id. -Examples: `East` → `east`, `Time Period` → `time_period`, `Time Period` → -`time_period`, `Coastal_East` → `coastal_east`, `Q1-2024` → `q1-2024`, -`Region(code)` → `regioncode`, `Café` → `caf`, `Ü` → `category-`, `Straße` → -`strasse` (answer 26). By answer 41, `a_ b` → `a_b`, `a__b` → `a_b` and -`Time \u{200B} Period` → `time_period`. By answer 42, `East ` → `east` and `East_` → -`east`. By answer 45, `-East-` → `east`, and `-` and `---` → `-`. Step 4 -collapses only `_`, so a `-` between two spaces survives: `Q1 - 2024` → -`q1_-_2024`. The user has not said otherwise, and this ADR adds no rule for it. +The accepted text has five steps instead: casefold; replace each **run of spaces** +with one `_`; strip; remove any **leading** `_`; `-` if empty. -`Time Period` and `Time_Period` slug alike, so whichever comes second gets -`time_period-`. So do `East`, `East ` and `East_`, and the second and third each -take a suffix ("disambiguate with -#", answer 42). `East` in `Region` and `East` in -`Direction` slug alike too, and since ids are global (answers 22 and 37), the second is -`east-`. Here `` is the counter's next value, for example `east-1`. The suffix -never puts a `-` at either end, so a suffixed id still meets step 5. +Examples under the amendment: -**The ADR's own reading of the word "whitespace"**, which answer 34 requires and does -not define. It is not a user answer. Revision 9 had a second reading here, that a slug -of only `-` is a legal id. Answer 45 rejects it, so step 5 removes it. -- **"Whitespace" means the Unicode `White_Space` property**, the set Rust's - `char::is_whitespace` tests. That is U+0009–U+000D, U+0020, U+0085, U+00A0, - U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. It needs no - table in `improvise-core`. A tab, a no-break space (U+00A0) and an ideographic - space (U+3000) are whitespace, so each becomes `_` like a space does: - `Time\tPeriod`, `Time\u{A0}Period` and `Time Period` all slug to `time_period`, - and so does `Time \t Period`, whose three `_` collapse at step 4. A zero-width space - (U+200B) and a byte-order mark (U+FEFF) are **not** `White_Space`, so step 3 - strips them. The realistic case is a header pasted from a web page, where a - no-break space stands for a space. The ADR chose this set because it is the standard - definition and the user asked for "whitespace", not for a list. The user may - prefer a narrower one (ASCII whitespace only, say), and that is a one-line change. - Revision 9 asked here whether `a_ b` keeps two underscores (Q24) and whether a - trailing run leaves a trailing `_` (Q25). Answers 41 and 42 settle both, and steps - 4 and 5 follow them. +| display | id | +|---|---| +| `East` | `east` | +| `Time Period`, `Time Period`, `Time\tPeriod`, `Time\u{A0}Period`, `Time \u{200B} Period` | `time_period` | +| `Coastal_East` | `coastal_east` | +| `a_ b`, `a__b` | `a_b` | +| `Q1-2024` | `q1-2024` | +| `Q1 - 2024` | `q1_-_2024` (only `_` runs collapse) | +| `East `, `East_`, `-East-` | `east` | +| `_Draft` | `draft` | +| `Region(code)` | `regioncode` | +| `Café` | `caf` | +| `Straße` | `strasse` | +| `Ü`, `-`, `---` | `-` | -**What this changes in built code (a follow-up, not made here).** PR #318's -`slugify` (`crates/improvise-core/src/model/id.rs`) and `context/code/core.md` §20 -predate answers 34, 41, 42 and 45, and differ from steps 2–6 in five ways: -- `underscore_space_runs` tests `c == ' '`, so `Time\tPeriod` slugs to - `timeperiod` (answer 34); -- it collapses runs of spaces before the strip and never collapses `_`, so `a_ b` → - `a__b` and `Time \u{200B} Period` → `time__period` (answer 41); -- it removes only leading `_`, so `East ` → `east_` (answer 42); -- it never removes `-`, so `-` slugs to `-` and `-East-` to `-east-` (answer 45); -- `IdAllocator::claim` accepts `-`, `-x` and `x-`, which Q34 would refuse. +**Collisions.** `Time Period` and `Time_Period` slug alike, and so do `East`, `East ` +and `East_`. Whichever comes later takes a suffix. `East` in `Region` and `East` in +`Direction` also slug alike, and ids are global, so the second is `east-`. +- **Any colliding generated id gets a suffix**, whatever kind of thing holds the + slug: on import, when an id-less file loads, and when the host creates something in + a session. An id that does not collide stays bare. +- **The suffix is `-` plus the next value of one increment-only counter per + document**, as a plain number (`east-1`, not `east-001`). The counter works like + Common Lisp's `gensym`: it never goes down and is never reused, so no order holds + among suffixes (`east-7` may follow `east` directly). A suffix ends in a digit, so a + suffixed id still meets step 5. +- **Retirement.** Within a session a removed id is never handed out again. The + counter is past any suffix it issued, and a removed bare slug stays reserved in + memory. +- **Determinism.** The counter advances in file order or creation order, never in + hash order (gotcha 18), so the same input gives the same ids. +- **The counter** lives in memory, starts at 1 when a document loads, and is not + written to the file. -The tests `slugify_keeps_an_underscore_the_display_already_had` and -`slugify_collapses_a_run_of_spaces_to_one_underscore` assert the superseded results. -Answers 34, 41, 42 and 45 are all folded into revision 9, which the user has not yet -approved, so the follow-up should wait for that approval. Nothing calls `slugify` -yet (§20), so no stored id is affected. +**Hand-written ids are canonical.** An id written in a file is never renamed, because +every data key and reference in that file spells it. So: +- **a file has ids on everything or on nothing.** An id-less file (an old version, + or a CSV or JSON import) gets every id from slugify. A new-version file must carry + an id on every declaration, and one that does not is refused (clause 5); +- the suffix only ever goes on the id being generated. When a generated candidate + `slug-` is taken by a file's id, the counter advances until a candidate is free; +- **two identical hand-written ids in one file are refused at load**, even when they + declare different kinds of thing. Renaming one would corrupt the data section, and a + file with a duplicate cannot be resaved safely. The error names both lines. -The remaining rules: +**Reuse across a save and reload is accepted.** Retired ids are not persisted. So +remove `East`, save, reload and add `East`, and the new item gets `east` again. An +in-flight command from before the reload is refused by the generation tier. -- **A file has ids on everything or on nothing** (answer 21). An id-less file (old - version, or a CSV or JSON import) gets every id from slugify. A new-version file must - carry an id on every declaration, and one that does not is refused (clause 5). So no - file mixes hand-written and generated ids. -- **Any colliding id gets a suffix, `gensym`-style** (answers 21, 27, 28, 37 and 42). - - **When:** whenever a generated slug is already taken, whatever kind of thing holds - it. That applies on import, when an id-less file loads, and when the host creates - something in a session. An id that does not collide stays bare. - - **What:** `-` and the next value of **one increment-only counter per document**, - as a plain number (`east-1`, not `east-001`). The counter never goes down and is - never reused, so no order is enforced among suffixes: `east-7` may follow `east` - directly. - - **Retirement:** within a session a removed id is never handed out again. The - counter has already moved past any suffix it issued, and a bare slug that was - removed is still reserved in memory. - - **Determinism:** the counter advances in file order or creation order, so the same - input gives the same ids, never hash order (gotcha 18). - - **Hand-written ids are canonical** (answer 29). The suffix only ever goes on the - id being generated. An id read from a new-version file keeps its spelling, even - when it looks like a slug or a suffixed one (`east-3`), because the file's data - keys and references spell it. So when a generated candidate `slug-` is already - taken by a file's id, the counter advances again until the candidate is free. - - **Two identical hand-written ids in one file are refused at load** (answers 11, - 22 and 29). By answer 22 they are a duplicate even when they declare different - kinds of thing, such as a category and an item. Renaming either one is exactly - what answer 29 forbids, and answer 11 already refuses a duplicate because such a - file "can't safely resave". The error names both lines (clause 5). This follows - from the answers, so it is not asked as a new question. - - **The counter** lives in memory and starts at 1 when a document loads. It is not - written to the file, since retired ids are not persisted (answer 13). That is - this ADR's reading. -- **Across a save and reload, reuse is accepted** (answer 13, Q10(b)). Retired ids are - not persisted. So remove `East`, save, reload and add `East`, and the new item gets - `east` again. An in-flight command from before the reload is refused by the - generation tier. The one remaining risk is an outside reference that outlives the - file, and none exists today. -- **Two branches of one file can merge into a file that will not load.** If each - branch adds a different thing whose id generates to the same slug, the merged - file declares that id twice. Its load is refused (answers 11 and 29), and the - error names both declarations, so the merge must be repaired by hand. Stable - identity across copies and branches of a file is not promised (improvise-8tk5). +**[Amendment] Two branches of one file can merge into a file that will not load.** If +each branch adds a different thing that generates the same slug, the merged file +declares that id twice. The load is refused and names both declarations, so the merge +is repaired by hand. Stable identity across copies and branches of a file is not +promised (improvise-8tk5). + +**Built code.** `crates/improvise-core/src/model/id.rs` (`slugify`, `IdAllocator`) +implements the accepted steps, and `context/code/core.md` §20 describes them. Once +the amendment is approved, a follow-up must: +- test whitespace, not only U+0020; +- collapse `_` after the strip; +- trim a trailing `_` and both ends' `-`; +- settle what `claim` refuses (open question 11); +- add the virtual views to `VIRTUAL_IDS`; +- add `IdKind::Group` if groups are generated (open question 10). + +Nothing calls `slugify` yet. ### 5. Persistence: ids on declarations *and* references -Following answers 3 and 7, the file now carries ids everywhere: -- **Declarations** (section headers, item lines, formula lines) carry the display name - and the id. -- **Every reference** carries the id: axis lines, `hidden:` and `collapsed:`, page - selections, `## Aggregation`, `Initial View:`, `## Data` keys, and names inside formula - text (answer 3). The formula panel shows display names instead (answer 19). +- **Declarations** (section headers, item lines, formula lines) carry the display and + the id. +- **Every reference** is the id: axis lines, `hidden:` and `collapsed:`, page + selections, `## Aggregation`, `Initial View:`, `## Data` keys (ids are guaranteed + unique, names might not be), and names inside formula text. -**Id alphabet.** Every id, generated or hand-written, is spelled only in `a`–`z`, -`0`–`9`, `_` and `-` (clause 4; answer 20). The grammar spells them **with ranges**, -because the generator drops `ASCII_ALPHANUMERIC` (improvise-rxz9). So an id never needs -pipe quoting in an `.improv` declaration or reference. A header ending in a -parenthesised run that contains a capital, such as `Sales(Copy)`, does not match -`id_tag`, and neither does an empty or trailing-space tag. In a new-version file that -declaration has no id, so the load is refused (answer 21), and the error names the -line. Revision 4 generated an id and warned. A hand-written id beginning with `_`, -other than the reserved virtual ids, is refused too (answers 23 and 42). A -hand-written id that is `-`, or that begins or ends with `-`, is refused the same -way. That is the ADR's reading of answer 45 ("a slug of - should be invalid as an id -and there should be no leading or trailing -s") for an id the program did not -generate (Q34). Whether a hand-written id ending in `_` is refused is also Q34: -answer 42 drops a trailing `_` from a slug, and does not say whether a written id may -carry one. +**The id alphabet.** Every id is spelled only in `a`–`z`, `0`–`9`, `_` and `-`. The +grammar spells it **with ranges**, because the generator drops `ASCII_ALPHANUMERIC` +(improvise-rxz9). So an id never needs pipe quoting in a declaration or reference. + +**What the loader refuses** in a new-version file: +- a declaration with no id. A header ending in a parenthesised run with a capital + (`Sales(Copy)`), an empty tag or a tag with a trailing space does not match + `id_tag`, so it has no id. The error names the line; +- a hand-written id beginning with `_`, other than a reserved one; +- **[Amendment]** a hand-written id that is `-` (open question 11 asks whether a + leading or trailing `-`, or a trailing `_`, is refused too). **Inside formula text an id may still need quoting.** The formula grammar's `bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So -the writer spells an id in formula text through `quote::quote_name`, as it spells a -name today. An id beginning with a digit (`2024`) or equal to a keyword (`sum`, `if`, -`where`, `is`) is written `|2024|` or `|sum|`. No new rule is needed, but "never needs -quoting" does not extend to formula text. +the writer spells an id in formula text through `quote::quote_name`. An id beginning +with a digit (`2024`) or equal to a keyword (`sum`, `if`, `where`, `is`) is written +`|2024|` or `|sum|`. ```pest id_token = @{ ('a'..'z' | '0'..'9' | "_" | "-")+ } -// The space is optional (answer 6). +// The space before the parenthesis is optional. id_tag = ${ " "* ~ "(" ~ id_token ~ ")" } -// Declarations: display name, then its id. The id is optional in the grammar so the -// loader, not the parser, refuses a missing one and can name the line (answer 21). +// Declarations: display, then id. The id is optional in the grammar so that the +// loader, not the parser, refuses a missing one and can name the line. header_decl = { pipe_quoted ~ id_tag ~ &NEWLINE | header_display ~ id_tag ~ &NEWLINE | line_name } header_display = @{ (!(id_tag ~ NEWLINE) ~ !NEWLINE ~ ANY)+ } item_decl = { name ~ id_tag? } grouped_item = { "- " ~ name ~ id_tag? ~ "[" ~ name ~ "]" ~ NEWLINE ~ blank_lines } -// References: an id, never a display name. +// References: an id, never a display. id_ref = { id_token } ``` -`model_name` has no id, so it keeps `header_name`. `Initial View:` becomes a reference -and takes an `id_ref`. **A group reference (`[Coastal]`, and the group in a -`collapsed:` line) becomes an id** (answer 44, overriding answer 9's "stays a name"). -The grammar above does not yet spell it, and neither does the group definition -(`> Coastal`), which becomes a declaration carrying a display and an id. Those -spellings are Q33. +`model_name` has no id and keeps `header_name`. `Initial View:` is a reference and +takes an `id_ref`. -**An axis line is tried before `format:`, and a format that reads as an axis line is -pipe-quoted** (improvise-le4p; the ADR's own reading, not a user answer). -`view_entry` is an ordered choice, and `format_line` takes the rest of the line, -so a category whose id is `format` would lose its axis line to the view's number -format. In the new-version arm, `axis_line` therefore comes before `format_line`. -That moves the collision to the other side. `set-format` stores its argument -verbatim and the writer emits `format: {number_format}`, so `:set-format none` -writes `format: none`, which would now read as the axis line of a category -`format`: the format is lost, or the id is unknown and the load is refused -(answer 11), so a document that saves would not load. +**Group references.** A group reference (`[Coastal]`) is a name. +**[Amendment]** It is the group's id. So is the group in a `collapsed:` line. The +group definition (`> Coastal`) declares the id. The spellings are open question 10. -The rule is that **the writer pipe-quotes a number format whenever the line would -also match `axis_line`**, that is, when the value is an axis keyword (`row`, -`column`, `page`, `filter` or `none`) alone or followed by `, `, or when it -begins with `|`. The new arm's `format_line` takes -`pipe_quoted ~ &NEWLINE | rest_of_line`, as `header_name` does (gotcha 34). So -`format: |none|` is the format `none`, and nothing is refused. An old-version -file's `format: none` is read as before, because the old arm keeps `format_line` -first, and it is written quoted on its first save. The alternatives were to refuse -such a format in `set-format`, which does nothing for a file that already holds -one, or in the writer, which leaves a document that cannot be saved. The id -`format` needs no quoting, and `id_ref` keeps no quoting arm. +**[Amendment] An axis line is tried before `format:`, and a format that reads as an +axis line is pipe-quoted** (improvise-le4p). `view_entry` is an ordered choice, and +`format_line` takes the rest of the line. So a category whose id is `format` would +lose its axis line to the view's number format, and the new-version arm puts +`axis_line` first. +- **The collision then moves.** `set-format` stores its argument verbatim, so + `:set-format none` writes `format: none`, which would now read as the axis line of a + category `format`. +- **So the writer pipe-quotes a number format whenever the line would also match + `axis_line`**: an axis keyword (`row`, `column`, `page`, `filter`, `none`) alone or + followed by `, `, or a value beginning with `|`. The new arm's `format_line` is + `pipe_quoted ~ &NEWLINE | rest_of_line`, so `format: |none|` is the format `none`. +- **An old-version file's `format: none`** is read as before (its arm keeps + `format_line` first), and it is written quoted on its first save. +- **Required tests:** + - a category displayed `Format`, on a row axis, round-trips; + - a view whose number format is each axis keyword round-trips in a document where + that view also holds a category `format` on an axis. -Required tests: a category displayed `Format`, on a row axis, round-trips; and a -view whose number format is each axis keyword round-trips, in a document where -the same view also holds a category `format` on an axis. - -**A formula line** (improvise-m56i; answers 3, 5, 22 and 30). **Under question -again by answers 35 and 36, because the `_Measure` target is the normal case (Q23).** -The shape below is revision 9's. It is the ADR's -own reading, not a user answer, except that the formula id's place is answer 5's and -an item target's id is answer 30's. It is -`- () = []`: -- **The parenthesised id is the formula's own `FormulaId`** (answer 5's - placement). +**[Amendment] A formula line** is +`- () = []` (pending open +question 1): +- **The parenthesised id is the formula's `FormulaId`**, between the target and the + `=`. - **`` is the target item's id when the target is an item** (outside - `_Measure`, answer 30). For a `_Measure` target, which has no item, it is the - target's display. - - `Formula` carries that display, and a display rename of a `_Measure` target - writes it. Where that display lives is also the ADR's own reading. - - Whether one line should spell both cases this way is **Q23**, below, and so - are how an expression refers to a target and how a `_Measure` display is quoted. -- **The expression is over ids**, spelled through `quote::quote_name` as above. A - reference to a formula target is the id text of its `ItemHandle`: the item id - outside `_Measure`, and the formula id for a `_Measure` target. -- **A `WHERE`/`IS` filter value follows the same rule** (answer 3: every reference - is an id). It is the id text of an `ItemHandle` in the filtered category, bare or - quoted: an item id, or a formula id when the category is `_Measure` and the value - names a formula target. The formula panel shows it as a display. -- **A formula cell's `CellKey`** carries `ItemHandle::Item()` - outside `_Measure`. For a `_Measure` target it carries - `ItemHandle::FormulaTarget()` under `_measure` (clause 2). + `_Measure`). For a `_Measure` target, which has no item, it is the target's + display, held on the `Formula`. +- **The expression is over ids**, spelled through `quote::quote_name`. A reference to + a formula target is the id text of its `ItemHandle`: the item id outside `_Measure`, + the formula id for a `_Measure` target. +- **A `WHERE`/`IS` value follows the same rule**. It is the id text of an + `ItemHandle` in the filtered category, and the formula panel shows it as a display. +- **A display in the target position that looks like `Name(x)` is pipe-quoted**, as in + `- |Margin(%)|(margin) = …`. An item id never contains a parenthesis. -**`header_display` is guarded only by a lookahead**, which the generator skips (gotcha -34). So the generator may emit a display ending in something that later parses as an id. -Round-trip property 3 still holds, because the first parse settles the split. But +**`header_display` is guarded only by a lookahead**, which the generator skips +(gotcha 34). So the generator may emit a display ending in something that parses as an +id. Round-trip property 3 still holds, because the first parse settles the split, but property 1 must be stated over the parsed result, not over the generated text. -**Which regime a file is read under is decided by its version line alone** (clause -10). In an old-version file, a header is the whole line, a reference resolves against -display names, and ids are assigned by slugify. That resolution happens in the loader, -not in a client, which answer 39 may or may not allow (Q28). In a new-version file, declarations -carry ids and a reference is an id. +**The version line alone picks the regime** (clause 10). +- **In an old-version file** a header is the whole line, a reference resolves against + display names, and ids are assigned by slugify. +- **In a new-version file** declarations carry ids and a reference is an id. -**An old file naming an item it never declares** declares it at load (answer 24). This -happens today when a `## Data` line names an undeclared item (improvise-98pd, second -source). The item gets a slug like any other, and the first save writes it, so a file -that loads today keeps loading (the lesson of improvise-80rm). The NonItem work -(improvise-gk51, PR #265) projects such a cell as `CellId::NonItem`. From a loaded file -that case no longer arises. Ids do not settle 98pd's other two sources, `set-cell` and -a stale page selection: a page selection holds an item id, and removing the item leaves -it dangling exactly as a removed name does today. +**An old file naming an item it never declares** declares it at load. This happens +today when a `## Data` line names an undeclared item (improvise-98pd). The item gets a +slug like any other, and the first save writes it, so a file that loads today keeps +loading (improvise-80rm). Ids do not settle 98pd's other two sources, `set-cell` and a +stale page selection. A page selection holds an item id, and removing the item leaves +it dangling, as a removed name does today. -**Slugify folds case (answer 16(b)), so old data keys change spelling on first save.** -An old file's `Region=East` resolves at load through the display `East`. The first -save writes the id: `Region=East` becomes `region=east`. The data is the same, but a -`git diff` across that first save touches every data line. +**Old data keys change spelling on the first save.** Slugify folds case, so an old +file's `Region=East` resolves at load through the display `East` and is written +`region=east`. The data is the same, but a `git diff` across that save touches every +data line. **Round-trip properties:** 1. an id-less (old-version) file loads, gets ids, and is written in the new version on @@ -1047,137 +376,114 @@ save writes the id: `Region=East` becomes `region=east`. The data is the same, b 2. a new-version file loads with every id kept verbatim; 3. `format(parse(format(parse(x)))) == format(parse(x))`. -A **formula-line generator strategy** is required. The grammar walker emits no `=` and +A **formula-line generator strategy** is required. The grammar walker emits no `=`, and the walker drops such lines, so without its own strategy every formula property is vacuous (improvise-rxz9). -**A duplicate, unknown or missing id in a new-version file refuses the load** (answers -11, 21, 22 and 29; a duplicate across two kinds of thing counts, and a hand-written id -is never renamed to resolve one; answer 11: -"don't load the file because you can't safely resave it"). The error names the id, -and for a duplicate it names both declarations. Repairing silently is ruled out, -because references carry ids (answer 3): reassigning a duplicate would silently change -what its references point at. +**A duplicate, unknown or missing id in a new-version file refuses the load.** The +error names the id, and for a duplicate both declarations. A silent repair is ruled +out, because reassigning an id would silently change what its references point at. -**Legacy JSON is removed** (improvise-tt22), so it needs no migration here. - -**Import** (CSV, JSON) never parses ids out of dimension names (answer 8). A header -`Region(code)` is the display name `Region(code)`, and its id is its slug. The writer -pipe-quotes any display name that would otherwise read as display plus id: +**Import** (CSV, JSON) never parses ids out of dimension names. A header `Region(code)` +is the display `Region(code)`, and its id is its slug. The writer pipe-quotes any +display that would otherwise read as display plus id: `## Category: |Region(code)|(regioncode)`. -### 6. The wire: address by id, create by display name, learn ids from projections +### 6. The wire: address by id, create by display, learn ids from projections -- **The host assigns every id** (answer 14). A client never proposes one. +- **The host assigns every id.** A client never proposes one. - **Every operand naming an existing thing is an id.** The one operand naming the thing - being created is a display name, and the host slugifies it: + being created is a display, and the host slugifies it: - `AddCategory(display)` and `CreateView(display)`; - - `AddItem { category: CategoryId, display }`, - `AddItemInGroup { category: CategoryId, display, group }` and - `AddFormula { target_category: CategoryId, raw }`. - - Since groups have ids (answer 44), `AddItemInGroup`'s `group` names an existing - group by its id, or creates one by its display. Which, or both, is Q33. -- **The client is the only place a display name is accepted as a reference** - (answers 32 and 39). It converts the name to an id as soon as it can, in the UI - layer, and the wire never carries a display as a reference. The one display an - operand carries is the name of the thing being created. + - `AddItem { category: CategoryId, display }`; + - `AddItemInGroup { category: CategoryId, display, group }`. Whether `group` is an + existing group's id or a new group's display is open question 10 **[Amendment]**; + - `AddFormula { target_category: CategoryId, raw }`. +- **[Amendment] Only a client accepts a display name as a reference**, in its UI + layer, and it converts the name to an id as early as it can. The wire never carries + a display as a reference. Whether the old-version loader and headless `improvise + cmd`/`script` are exceptions is open question 4. - **A frame cannot address a thing it creates**, because it does not know the id the - host will assign. So a create-then-act compound becomes **one command** carrying - both halves. `create-and-switch-view` sends `[CreateView, SwitchView]` today and - becomes `CreateView { display, switch: true }`. Any other compound of this shape - gets the same treatment. -- **New operations are born with ids:** + host will assign. A create-then-act compound becomes **one command**: + `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes + `CreateView { display, switch: true }`. Any other compound of that shape gets the + same treatment. +- **New operations carry ids:** - `RenameCategory { category, to: display }`; - `RenameItem { category, item, to: display }`; - - `ReplaceFormula { id, was, raw }`, which km71 already landed with an interim id - (below); - - fdk1's `RenameRecordCoordinate`, which landed with name operands (below); + - `ReplaceFormula { id, was, raw }`; + - `RenameRecordCoordinate`; - the removes (`RemoveCategory`, `RemoveItem`, `RemoveFormula`, `DeleteView`). -- **Older variants** (`SetAxis`, `SetPageSelection`, `HideItem`, …) **must also move to - ids**. Their operands are references, and a reference is now an id. They may move one - whole variant at a time. That includes `SetCell { view, … }` and - `ClearCell { view, … }`, which improvise-wpxr (PR #281, merged) gave a view named by - its name, like every other view-scoped variant (improvise-lltt). +- **Older variants move to ids too** (`SetAxis`, `SetPageSelection`, `HideItem`, …, + including `SetCell { view, … }` and `ClearCell { view, … }`), one whole variant at a + time. - **Projections carry id and display together:** - - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }` - (**pending Q23**: under option (c) there is no `ItemHandle`); - - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows - (answer 19): the target's display name with its id in parentheses, then the - expression over display names, with an id wherever a display name is ambiguous - (ids are unique, answer 22). A display that equals another thing's id is - ambiguous in this sense (the ADR's own reading of answers 19 and 33, flagged - under Q23), so the panel never shows a bare name that the ids-first rule would - bind elsewhere. `stored` is the id-form text the file carries, and it is what - `ReplaceFormula`'s `was` is compared against. So a rename of `Revenue` elsewhere - changes what the panel shows but does not refuse an edit of `Profit`, which is - the "avoid glitches" of answer 3. What makes that hold for an edit already open - is clause 12; + - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }` (pending open + question 1); + - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows: + the target's display with its id in parentheses (`Profit (profit) = Revenue - + Cost`), then the expression over display names, with an id wherever a display is + ambiguous. **[Amendment]** A display equal to another thing's id counts as + ambiguous, so the panel never shows a bare name that the ids-first rule (clause 12) + would bind elsewhere. `stored` is the id-form text the file carries, and + `ReplaceFormula`'s `was` is compared against it. So a rename of `Revenue` elsewhere + changes what the panel shows, but does not refuse an edit of `Profit`; - `ViewSummary { id, display, … }`. `AxisFrame` labels stay display strings. They gain ids when a consumer needs them. -- **`ViewState` keys by id**: the cursors, `expanded_cats`, the view history - (`view_back_stack` / `view_forward_stack`) and `active_view` itself, as does - `Workbook::active_view` (improvise-p003 item 3). Answer 1 makes this natural - rather than a migration of its own. A view or category rename then orphans - nothing. -- **A `_Measure` formula-target row** (`ItemHandle::FormulaTarget(FormulaId)`; under - question again, Q23): - removing it means `RemoveFormula`, and renaming it means a display rename of - the formula's target (improvise-i0n7). -- **CheckedBatch's created-record exemption goes** (the ADR's own reading, not a - user answer; improvise-p003 item 5). It - exempts a record that neither resolves nor is claimed, for frames like - `[AddItem(East), SetCell(East)]`. Under this clause a frame cannot address what - it creates, so nothing produces such a frame any more. The wire step removes - the exemption rather than leaving dead reasoning in place. +- **`ViewState` keys by id**: the cursors, `expanded_cats` and the view history + (`view_back_stack` / `view_forward_stack`). **[Amendment]** So do `active_view` + itself and `Workbook::active_view` (improvise-p003). A rename then orphans nothing. +- **[Amendment] A `_Measure` formula-target row** (`ItemHandle::FormulaTarget`) is + removed by `RemoveFormula` and renamed by a display rename of the formula's target + (improvise-i0n7; pending open question 1). +- **[Amendment] CheckedBatch's created-record exemption is removed** (improvise-p003). + It exempts a record that neither resolves nor is claimed, for frames like + `[AddItem(East), SetCell(East)]`. A frame cannot address what it creates, so nothing + produces such a frame any more. ### 7. ADR 001: an id that no longer resolves is gone -A command naming an id that does not resolve is rejected before anything is applied, as -`Rejection::Gone(Named)`: -- `Named` is one of `Category`, `Item`, `Formula` or `View`, and carries the id's text. -- It is **structural**: force never creates structure. +A command naming an id that does not resolve is rejected before anything is applied, +as `Rejection::Gone(Named)`: +- `Named` is `Category`, `Item`, `Formula` or `View`, and carries the id's text. + **[Amendment]** It gains `Group`. +- The rejection is **structural**: force never creates structure. The check has two halves, like the record tier: - **Before the frame:** an exhaustive `ids_named` feeds `Checked::against`. - **Within the frame:** an **id arm in `broken_chain`** refuses `[RemoveFormula(f), ReplaceFormula(f)]` (improvise-gkyo). -Per-entity versions are not added (answer 10). +Per-entity versions are not added. -### 8. ADR 002: `RecordHash` is now defined over ids +### 8. ADR 002: `RecordHash` is defined over ids -Following answer 3, `RecordHash` hashes the explicit serialization of the sorted -`(CategoryId, ItemHandle)` pairs, spelled as their id text, the same function -over different strings. The text is enough: answer 22 makes ids unique across -kinds, so no `Item` and `FormulaTarget` can spell alike, and the hash stays -canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate -is never part of a record in the first place. **Pending Q23**: under option (c) no -`FormulaTarget` exists, and this paragraph reduces to ids over `ItemId`s. -- **ADR 002's decision is unchanged in substance.** It still hashes an explicit - serialization of a normalized record, with a fixed key and a width argued for. -- **Its prose will be out of date once this lands.** ADR 002 says `CellKey::new` - "sorts coordinates by category name", and a record is spelled in names. Both become - ids. ADR 002 is **not edited here**. Updating that wording belongs to the - implementation PR. +`RecordHash` hashes the explicit serialization of the sorted `(CategoryId, ItemId)` +text pairs: the same function over different strings. **[Amendment]** The pairs are +`(CategoryId, ItemHandle)` spelled as id text. Ids are unique across kinds, so the hash +stays canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate +is never part of a record (pending open question 1). -**What changes in behaviour:** a display rename no longer changes any `RecordHash` or -bumps `layout_version`, because `view_shape` hashes ids too. So a client's surrogate -survives a rename, and its in-flight edit still lands. A **coordinate** change, as in -fdk1 or a records-mode move, still produces a different record and a different hash, as -it should: it *is* a different record. +ADR 002's decision is unchanged in substance. Its prose ("sorts coordinates by +category name", records spelled in names) is out of date once this lands, and the +implementation PR updates it. -### 9. Headers that change meaning: the edge case answer 6 asks to document +**In behaviour:** a display rename changes no `RecordHash` and bumps no +`layout_version`, because `view_shape` hashes ids too. So a client's surrogate survives +a rename, and its in-flight edit lands. A **coordinate** change, as in fdk1 or a +records-mode move, produces a different record and a different hash, because it is a +different record. -Under answer 6, `header_decl` reads a trailing `(token)`, with or without a space before -it, as the id. **It applies only to new-version files** (answer 15, Q12(a)). An -old-version file is read as it always was, so its headers never split. In a new-version -file the writer never produces an ambiguous header, because it pipe-quotes any display -that would split. So the edge cases below arise **only when a new-version file is -written by hand**. The table shows what the header rule would do to lines that exist -in old files today. That is why old files do not get the rule. +### 9. Headers that change meaning -| header line | whole-line reading (today) | under `header_decl` | +In a new-version file, `header_decl` reads a trailing `(token)`, with or without a +space before it, as the id. **Old-version files never get this rule**, so their headers +never split. The writer pipe-quotes any display that would split, so the cases below +arise only in a new-version file written by hand. They show why old files keep the +whole-line reading. + +| header line | whole-line reading (old version) | under `header_decl` (new version) | |---|---|---| | `## View: Default(default)` | `Default(default)` | display `Default`, id `default` | | `## View: \|Default (default)\|(default)` | pipes literal | display `Default (default)`, id `default` | @@ -1186,545 +492,385 @@ in old files today. That is why old files do not get the rule. | `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** | | `## Category: Time Period` | `Time Period` | unchanged; id `time_period` generated | -**The documents that change meaning** are those with a view or category header ending in -a parenthesised run of letters, digits, `_` or `-`. **The app writes such headers -itself**: `improvise import` on a CSV with headers like `Region(code)` or `Size(cm)` -saves `## Category: Region(code)` today (improvise-80rm). Any view someone named -`Sales (copy)` does the same. +The app itself writes such headers: `improvise import` on a CSV with headers like +`Region(code)` saves `## Category: Region(code)` today (improvise-80rm). -**What it would cost if old files got the new rule** (the case answer 15 rules out): -- The category is renamed on load: display `Region`, id `code`. -- Every reference in that old file spells `Region(code)`, which resolves to neither the - display nor the id. -- So the axis lines and every data key for that category do not resolve. -- Today's loader would silently drop the axis lines and orphan the data, so nothing is - drawn. +If old files got the rule, that category would load as display `Region`, id `code`. +Every reference in the file spells `Region(code)`, which resolves to neither, so the +axis lines and data keys would not resolve, and nothing would be drawn. Because old +files keep the whole-line reading, none of this happens. On the first save the writer +pipe-quotes such a display: `## Category: |Region(code)|(regioncode)`. -**Decided (answer 15):** old-version files keep the whole-line reading, so none of -this happens to a file that exists today. On the first save the writer pipe-quotes -such a display, for example `## Category: |Region(code)|(regioncode)`, so the new file -cannot split it either. +### 10. The file version alone picks the regime -### 10. The file version is explicit, and it alone picks the regime - -Following answers 15 and 25: - **The version is already explicit.** The first line of every `.improv` file is a mandatory version line, matched as an **exact literal** - (`version_line = { "v2025-04-09" ~ NEWLINE ~ blank_lines }` in `improv.pest`), and - `format_md` writes it. Nothing is added to make the version explicit. What is added is - a second literal. -- **The new format gets a new literal.** The grammar's `file` rule gets one arm per - literal, and each arm uses its own header and reference rules. The new literal's - exact spelling is left to the implementation. -- **The regime is chosen only by the first line**, never inferred from what the body - looks like. So the old literal means no ids anywhere, and the new literal means ids - on every declaration (answer 21). -- **A file whose first line is neither literal is refused**, as it is today. -- **Every save writes the new literal** (answer 12). So an old file changes version on - its first save. + (`version_line = { "v2025-04-09" ~ NEWLINE ~ blank_lines }`), and `format_md` writes + it. +- **The new format gets a second literal.** The grammar's `file` rule gets one arm per + literal, each with its own header and reference rules. The literal's spelling is the + implementation's choice. +- **The regime is chosen only by the first line**, never inferred from the body. The + old literal means no ids anywhere. The new literal means ids on every declaration. +- **A file whose first line is neither literal is refused**, as today. +- **Every save writes the new literal**, so an old file changes version on its first + save. ### 11. Migration 1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224. -2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId`, - `ViewId`, and `ItemHandle`, which the re-key (step 3) needs for `CellKey` before - clause 6's step (5) uses it in projections (**pending Q23**: under option (c) - there is no `ItemHandle`). +2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId` and + `ViewId`. **[Amendment]** Also `ItemHandle`, which the re-key needs for `CellKey` + (pending open question 1). 3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with display→id resolution moved to every input edge. -4. **Persistence:** the new version literal (clause 10), the grammar in clause 5, the - loader and its refusals, slugify, the writer, and the round-trip properties with a - formula-line strategy. +4. **Persistence:** the new literal (clause 10), clause 5's grammar, the loader and its + refusals, slugify, the writer, and the round-trip properties with a formula-line + strategy. 5. **Projections and `ViewState`** (clause 6). 6. **Vocabulary:** the id-addressed variants, `Rejection::Gone`, `ids_named` and `broken_chain`'s id arm. -7. **Clause 12's capture:** the client-side display→id mapping taken when the - formula editor opens, and the resolver that turns typed names into id-form - text (answers 32 and 33). Neither the resolver nor any display-matching code - exists yet, so the interim order is built ids first from the start and nothing - has to be migrated. The autocomplete that retires it is not in this list. +7. **[Amendment] Clause 12's client-side capture and resolver.** No display-matching + code exists yet, so the resolver is built ids first from the start. -The implementation epic (improvise-cq1q) orders these steps to keep `main` -shippable after each one, which the list above does not: the display field -before the re-key, and projection ids before the re-key. That ordering is the -epic's, not a change to this list. +The implementation epic (improvise-cq1q) orders these steps to keep `main` shippable +after each one: the display field before the re-key, and projection ids before the +re-key. ### 12. A formula edit's names are fixed when the editor opens -**This clause is the ADR's own reading, not a user answer.** It closes a contradiction -the adversarial review of `4830fa30` found between clause 6 and the km71 section, which -is also ChatGPT Codex's P1 on `29ad14f4`. - -**The problem.** The formula panel shows display names (answer 19), and the user edits -that text. If the host maps those names to ids at **commit**, a concurrent rename breaks -the edit one of two ways: -- the name the user typed no longer resolves, so the edit is refused, which contradicts - clause 6's "does not refuse an edit of `Profit`"; +**The problem.** The formula panel shows display names, and the user edits that text. +If those names were mapped to ids at **commit**, a concurrent rename would break the +edit one of two ways: +- the typed name no longer resolves, and the edit is refused; - or the name now belongs to something else (`Revenue` renamed away, another item - renamed to `Revenue`), so the edit silently binds to a different thing. That is the - glitch answer 3 exists to prevent. + renamed to `Revenue`), and the edit silently binds to a different thing. -**The rule.** The display-name-to-id mapping is **captured when the formula editor -opens**, and the commit resolves the edited text through that captured mapping. This -is the capture-at-open rule the cell editor already follows: `OpenedCell` in -`improvise-protocol/src/client.rs` records the record, its version, the document -generation and the view at open, and the commit answers for those rather than for -whatever the document holds by then. -- **The client holds the mapping** (answer 32), as it holds `OpenedCell`. The TUI - and the browser both use the same resolver in `improvise-protocol`. The user - confirmed this and widened it: the client "should be the only place that allows any - reference to something by a human-friendly name", and converts it "to IDs ASAP" - (answer 39). -- The mapping covers every display name the panel showed in that formula's text, - including the target's. -- A name the user types that was **not** in the captured mapping is new to the - edit. The client resolves it against its own id-bearing projection (clause 6) at - commit, in answer 33's order: **ids first, then displays** ("IDs are unambiguous so - they should always be preferred when there's ambiguity", answer 38). A typed name equal to - an id names that id's thing, even where it is also another thing's display. - Otherwise, if exactly one thing in scope has that display, it names that thing. If - a display matches more than one thing, the edit is refused and the refusal lists - the candidates' ids. If the name does not resolve at all, the edit is refused and - names it. An id is matched exactly: no case folding. -- **The ids-first order was stated as interim** (answer 33). The long-term design - separates two layers. A view-layer autocomplete of typed display names turns text - into ids before it is committed, and the model layer refers to ids only. Revision 9 - read that, when the autocomplete exists, this resolver's display stage and the - ids-first order go with it. **Answer 38 says ids "should always be preferred when - there's ambiguity"**, so the preference itself is not temporary. Whether it carries - into the autocomplete (an id match offered first, say), or ends with this resolver, - is Q27. This ADR does not specify the autocomplete. -- **The wire carries only id-form text.** `ReplaceFormula.raw` and - `AddFormula.raw` are in the file's id form, and the host never resolves a display - name (answer 32). The host parses ids, and one that no longer exists is - `Rejection::Gone` (clause 7). -- `was` is still compared against the stored id-form text (clause 6), so a concurrent +**The rule.** The display→id mapping is **captured when the formula editor opens**, and +the commit resolves the edited text through it. This is the cell editor's +capture-at-open rule: `OpenedCell` (`improvise-protocol/src/client.rs`) records the +record, its version, the document generation and the view at open, and the commit +answers for those. +- The mapping covers every display the panel showed in that formula's text, including + the target's. +- `was` is compared against the stored id-form text (clause 6), so a concurrent *replace* of the formula is still seen. +- A captured id that no longer exists at commit is `Rejection::Gone` (clause 7). +- **A typed name that was not in the captured mapping** is resolved against the + document at commit, and refused, naming it, if it does not resolve. -**What the interim order means for a document where an item's display equals another -item's id.** Item A is displayed `East` and has id `east`. Item B is displayed `east`, -so slugify finds `east` taken and gives it `east-1` (clause 4). Typing `east` names -**A**, because the id wins. B is reached by typing `east-1`, or by autocomplete once -it exists. A user who typed `east` meaning B is bound to A without being told, which -is the cost of the interim order and the reason the panel shows B as `east-1`, not as -a bare `east` (clause 6). The captured mapping cannot disagree with the rule, since -it holds only names the panel showed bare, and the panel shows none that ids-first -would bind elsewhere. +**[Amendment] The client does the resolving:** +- **The client holds the mapping**, as it holds `OpenedCell`. The TUI and the browser + share one resolver in `improvise-protocol`. +- **A name new to the edit** is resolved by the client against its id-bearing + projection (clause 6), **ids first, then displays**. Ids are unambiguous, so they are + preferred wherever there is ambiguity. + - A typed name equal to an id names that id's thing, even where it is also another + thing's display. + - Otherwise a display that exactly one thing in scope has names that thing. + - A display matching several things is refused, and the refusal lists their ids. + - A name that resolves to nothing is refused, and the refusal names it. + - Ids are matched exactly, with no case folding. +- **The long-term design separates two layers.** A view-layer autocomplete turns + typed display text into ids before commit, and the model layer refers to ids only. + What becomes of the ids-first order once that exists is open question 3. This ADR + does not specify the autocomplete. +- **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are + in the file's id form, and the host never resolves a display. The host parses ids, + and one that no longer exists is `Rejection::Gone`. -**Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. -From another client, rename the item `Revenue` to `Sales` and rename a different -item to `Revenue`. Then edit something that is **not a name**: change the text to -`Revenue - Cost * 1` and commit it. The stored formula must reference the original -item's id, and the commit must not be refused. The test must fail under -commit-time resolution, which binds to the new `Revenue`, and it must fail under a -refusal on rename. Both are the rivals this clause rules out. +**[Amendment] What ids-first costs.** Suppose item A is displayed `East` with id `east`, +and item B is displayed `east`, so slugify gave it `east-1`. Typing `east` names **A**. +B is reached by typing `east-1`. A user who typed `east` meaning B is bound to A +without being told. That is why the panel shows B as `east-1` and never as a bare +`east`. -**Required test for the interim order** (answer 33). In a document with items A -(display `East`, id `east`) and B (display `east`, id `east-1`), type `east` into a -formula editor, in text not captured at open. The stored formula must reference A. -The test must fail under displays-first, which binds B. Typing `east-1` must bind B, -and typing a display that is nobody's id must still resolve by display. A fixture -where no display equals an id passes under either order and proves nothing. +**Required regression test.** +1. Open a formula editor on `Profit = Revenue - Cost`. +2. From another client, rename `Revenue` to `Sales`, and rename a different item to + `Revenue`. +3. Edit something that is not a name (`Revenue - Cost * 1`) and commit. -Revision 8's version committed the *unedited* text. A client that treats an -unchanged text as a no-op passes it without resolving any name, so the test could -not tell a correct capture from no capture at all (improvise-hexb item 1). +The stored formula must reference the original item's id, and the commit must not be +refused. The test must fail under commit-time resolution and under a refusal on +rename. Committing the *unedited* text proves nothing, because a client may treat it +as a no-op without resolving any name. -## What this means for you +**[Amendment] Required test for ids-first.** With items A (`East`, `east`) and B +(`east`, `east-1`), type `east` into an editor, in text not captured at open: +- it must bind A, and the test must fail under displays-first; +- `east-1` must bind B; +- a display that is nobody's id must still resolve by display. -### improvise-km71 — editing a formula (landed as an interim) +A fixture where no display equals an id proves nothing. -km71 landed in PR #214, before this ADR, and was built to be revised by it -(`context/code/core.md` §18): -- **What landed.** `ModelCommand::ReplaceFormula { id: FormulaId, was, raw }`. - `FormulaId` is opaque, with private fields `(target, target_category)`. - `Model::replace_formula` writes in place, keeping the position. It refuses a new - target that another formula owns, and it refuses any target change outside - `_Measure` (improvise-kb1d). `was` is the text the editor opened on. It makes a - concurrent *replace* visible, which even a stable id cannot do, so **`was` stays**. -- **What this ADR changes.** - - `FormulaId`'s fields become one id. The signatures that carry `FormulaId` do not - change, which is what making it opaque was for. - - A target change becomes a **display** rename of the target. The id does not move, - and nothing that references the target is rewritten. So kb1d's refusal outside - `_Measure` can be lifted, and so can the dangling-reference report core.md §18 - describes. - - The panel shows `Profit (profit) = Revenue - Cost` (answer 19). The user edits that - text. Its display names are resolved to ids through the mapping captured when the - editor opened (clause 12), not through the names the document holds at commit, and - `was` is compared against the stored id-form text (clause 6). The parenthesised id is not the user's to - change, since the host assigns ids (answer 14). An edit that changes it is refused. - This is the ADR's reading, not a user answer. -- **Rejections.** Another client's deletion gives `Rejection::Gone(Formula(id))`. A - deletion earlier in the same frame gives the broken-chain refusal. +## What this means for the tickets -### improvise-fz77 — renaming a category or item +**improvise-km71 (editing a formula), landed as an interim.** +`ReplaceFormula { id: FormulaId, was, raw }` writes in place. It refuses a new target +that another formula owns, and any target change outside `_Measure` (improvise-kb1d). +`was` makes a concurrent *replace* visible, which even a stable id cannot, so `was` +stays. -- **The operations.** `RenameCategory { category, to }` and - `RenameItem { category, item, to }`. The panel rows carry the ids. -- **It is now cheap** (answers 1 and 2): one `display` write. No `CellKey`, axis map, - page selection, hidden or collapsed set, or formula text is rewritten, because all of - them hold ids. No `RecordHash` or `layout_version` moves, so in-flight edits survive. -- **What it still owes:** - - **Uniqueness.** Refuse a display name already used in the same scope. - - **Dirtiness.** Mark the effect so the grid labels and panels are re-projected - (gotcha 35). The labels are display strings. - - **Tests.** Assert that the *labels* changed and the *surrogates did not*. That is - the property this design exists for. -- **It depends on step 3 of the migration**, the re-keying. Until then there is no cheap - rename to build on. +This ADR changes three things: +- **`FormulaId`'s fields become one id.** The signatures do not change. +- **A target change becomes a display rename of the target.** The id does not move, + so kb1d's refusal and core.md §18's dangling-reference report can be lifted. +- **The panel's text is resolved through clause 12's captured mapping.** The + parenthesised id is not the user's to change, so an edit that changes it is + refused. -### improvise-fdk1 — renaming a records coordinate +Rejections: +- Another client's deletion gives `Rejection::Gone(Formula(id))`. +- A deletion earlier in the same frame gives the broken-chain refusal. -fdk1 landed in PR #221, before this ADR. What exists on main: -- `ModelCommand::RenameRecordCoordinate { record: RecordHash, category: String, item: - String }`, which moves a whole record to a new coordinate and creates the item if need - be; -- `CellId::Coordinate { record: RecordHash, category: String }`, the surrogate a - records-mode coordinate cell carries. +**improvise-fz77 (renaming a category or item).** It adds `RenameCategory` and +`RenameItem`. A rename is one `display` write, and no key, set, formula text, +`RecordHash` or `layout_version` moves. It still owes three things: +- refuse a display already used in the scope; +- mark the effect so labels and panels re-project (gotcha 35); +- tests asserting that the *labels* changed and the *surrogates did not*. -**What this ADR changes**, as it does for km71's `FormulaId`: -- `category` becomes a `CategoryId`, in the command and in `CellId::Coordinate`. -- `item` becomes an `ItemRef`: `Existing(ItemId)` or `New(display)`. That is the - create-by-display half of clause 6, since the host assigns the new item's id. -- Unlike a display rename, this **changes a coordinate**, so the record's `CellKey`s and - `RecordHash` change. It is a different record, and that does not change. +It depends on migration step 3. -## Questions as asked, and where each now stands - -The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14, -revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All -of those are answered**, and answer 44 later overrode Q5's answer. After acceptance, -three implementation questions were answered on their tickets (answers 30–32). The -user's comments on revision 9 are answers 33–36 (eighth round) and 37–45 (ninth -round). Revision 9 added Q23 to Q25: answers 41 and 42 settle Q24 and Q25, and Q23 is -still open. The ninth round adds Q26 to Q34, which are open. - -| # | question | my recommendation | status | -|---|---|---|---| -| 1 | Integer or word ids? | first draft: words; revision 1: integers only | **answered**: interned symbols (answer 4), generated by slugify (answers 8, 16). Words. | -| 2 | Formula id placement | F1, `- Profit(3) = …` | **answered: F1** (answer 5) | -| 3 | Headers in old files | first draft: accept the rare break; revision 1: gate on version | **answered**: the space is optional (answer 6), and old files keep the whole-line reading (answer 15, Q12) | -| 4 | Do `## Data` keys carry ids? | no | **answered: yes, ids** (answer 7), reversing the recommendation | -| 5 | Group ids? | not now | **answered: not now** (answer 9). **Overridden: a group name is an id** (answer 44); the mechanism is Q33 | -| 6 | Per-entity versions? | not in this ADR | **answered: not in this ADR** (answer 10) | -| 7 | `RecordHash` over ids? | no, keep names | **answered: over ids** (answer 3), reversing the recommendation | -| 8 | Duplicate id: repair or refuse? | revision 2: refuse | **answered: refuse the load** (answer 11) | -| 9 | Reading of "opt-in" | read optional, write always | **answered: yes** (answer 12) | -| 10 | Id reuse after a reload | revision 2: (b) accept reuse | **answered: (b)** (answer 13) | -| 11 | Who assigns ids? | the host | **answered: the host** (answer 14) | -| 12 | Old files and the new header rule | (a) no | **answered: (a), with the version marked unambiguously** (answer 15) | -| 13 | Slugify details | all generation; keep case; keep `_` `-`; strip non-ASCII; `-` | **answered** (answer 16): all generation, **lower case only** (reversing the recommendation), keep both, strip, `-` | -| 14 | Formula text: ids or display names? | ids in the file | **answered**: ids in the file (answer 3); the panel is Q16 | -| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-`, "space" means whitespace (answer 34), and each whitespace character becomes `_` before runs of `_` collapse (answer 41) | -| 16 | What the formula panel shows | none | **answered: display names, target shown as `Profit (profit)`, ids where a name is ambiguous** (answer 19) | -| 17 | Capitals in a hand-written id | (a) no | **answered: no** (answer 20) | -| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse** (a run of whitespace, answer 34; by collapsing runs of `_`, answer 41), plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | -| 18(b) | What "casefold" means | full case folding | **answered: full Unicode case folding** (answer 26) | -| 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answers 22 and 37), stronger than (a) | -| 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23); the reservation covers "virtual categories and measures and other entities" (answer 42), and `_index`/`_dim` are restated (answer 43) | -| 21 | An old file naming an undeclared item | (a) declare it at load | **answered: (a)** (answer 24) | -| — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal | -| 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) | -| — | A formula target outside `_Measure` (improvise-m56i item 2) | no real item; a `FormulaTarget` handle | **answered: a real item and a separate formula id** (answer 30), reversing the recommendation; a formula id that collides takes the counter's suffix (answer 37) | -| — | A typed name matching one display and another id (improvise-m56i item 6) | display first, then id; refuse ambiguity | **answered: display first; the client holds it** (answer 31). **Order reversed: ids first until there is autocomplete** (answer 33). Ids are always preferred under ambiguity (answer 38); whether that outlives the interim resolver is Q27 | -| — | Which side holds clause 12's capture (improvise-hexb item 2) | the client | **answered: the client; id-form text on the wire** (answer 32), widened to every reference by a human-friendly name (answer 39) | -| — | Whether a measure has an id/display split, and which formula targets are normal (user comments on revision 9) | — | **answered in principle**: measures have the split (answer 35), the normal target is a measure (answers 35, 36), and a formula is a derived measure (answer 40). The mechanism is Q23 | -| — | Revision 9's reading that `_Index`/`_Dim` coordinates are outside the id namespace | (the ADR's reading) | **rejected as worded** (answer 43); what the positions are is Q31 | -| — | Revision 9's reading that `collapsed:` names a group by name | (the ADR's reading) | **overridden: a group name is an id** (answer 44) | -| — | Revision 9's reading that a slug of only `-` is a legal id | (the ADR's reading) | **rejected: `-` is not an id, and no id begins or ends with `-`** (answer 45) | -| 23 | Is a formula's derived measure one thing with one id, or two things with two ids? (revision 9; reworded after answers 35 and 36; restated after answer 40) | none until the user chooses | **open** | -| 24 | Does `a_ b` slug to `a__b`? (answer 34) | (a) keep | **answered: (b), runs of `_` collapse**, so `a_ b` → `a_b` (answer 41), not the recommendation | -| 25 | Does a trailing whitespace run slug to a trailing `_`? | (a) keep | **answered: (b), a trailing `_` is dropped** (answer 42), not the recommendation | -| 26 | What may a formula refer to, and what is "a category value"? (answer 37) | (iii) only keep ids unique | **open** | -| 27 | Does ids-first outlive the interim resolver? (answer 38) | none | **open** | -| 28 | May the old-version loader and headless `cmd`/`script` resolve displays? (answer 39) | yes, both | **open** | -| 29 | Does step 4's collapse come after the strip? (answer 41) | (a) after | **open** | -| 30 | Which "other entities" share the reserved `_`? (answer 42) | the virtual views are among them | **open** | -| 31 | Are `_Index`'s and `_Dim`'s positions ids? (answer 43) | none | **open** | -| 32 | Moving the virtual categories to lower-case ids (answers 23 and 43) | as written there | **open** | -| 33 | How groups carry ids (answer 44) | none, beyond the readings stated | **open** | -| 34 | Hand-written ids with `-` or a trailing `_`, and how the trims repeat (answer 45) | refuse `-` at either end; repeat the trims | **open** | - -## Still open - -Nothing is open among Q1–Q22, and answer 29 settles the one case it raises: two -identical hand-written ids are refused at load. **The user approved revision 7. -Revision 8 adds clause 12, which is the ADR's own reading, and the user approved it -("looks good, approved", 2026-09-25 00:17:10Z).** Three follow-ups from the same -review were left as tickets rather than ADR text: improvise-le4p, improvise-m56i and -improvise-p003. Revision 9 folds all three in, and improvise-m56i's two -questions are answers 30 and 31. - -**Revision 9 needs the user's approval.** It asked Q23 to Q25, and the ninth round -answered Q24 and Q25. Q23 and the ninth round's Q26 to Q34 are open. - -**Q23. A formula is a derived measure. Is the measure and its formula one thing with -one id, or two things with two ids?** (Restated after answer 40, which asked for this -question to be stated more clearly. Revision 9 asked how a formula line spells its -target; after answers 35 and 36 it asked how a formula-target measure carries its id. -It is the same question, stated in the user's terms.) - -**What is decided.** -- A formula is a derived measure (answer 40), and a measure is the normal formula - target (answers 35 and 36). -- A measure has an id and a display (answer 35). -- Every reference is an id (answer 3), and no two ids in a document are equal - (answers 22 and 37). -- A formula carries an id in its line, after the target and before the `=` - (answer 5), and `ReplaceFormula` and `RemoveFormula` address a formula by it - (clause 6). -- The rare dimension target is a real item with a separate formula id (answer 30), - and a formula id that collides takes the counter's suffix (answer 37). - -**The worked example.** A document has two stored measures, `Revenue` and `Cost`. -Each is an item of `_Measure`, with ids `revenue` and `cost`, and each holds data. -The user adds `Profit = Revenue - Cost`. `Profit` holds no data of its own: its -column is computed by the formula, which is what makes it a derived measure. Two -kinds of thing must then name Profit by an id: -1. **Things that refer to the measure.** Another formula (`Margin = Profit / - Revenue`), the `CellKey` of each Profit cell, a `hidden:` line, and ADR 008's - cursor anchor. -2. **Things that refer to the formula.** `ReplaceFormula` and `RemoveFormula`, which - edit or delete the formula. - -**The question is whether 1 and 2 use the same id.** - -- **(a) One thing, one id.** The formula *is* the measure. Profit's id is `profit` - and its display is `Profit`, both held on the formula. `Margin = profit / revenue`, - each Profit cell's key holds `profit`, and `ReplaceFormula` addresses `profit`. - The file line is `- Profit(profit) = revenue - cost`. No `_Measure` item exists for - Profit, so in code the column's coordinate is `ItemHandle::FormulaTarget(profit)` - (clause 2). This is revision 9's model, and the smallest change. -- **(c) Two things, two ids.** The measure is a `_Measure` item with id `profit` and - display `Profit`. The formula that computes it is a separate thing, with id - `profit-1`, suffixed because `profit` is taken (answer 37). `Margin = profit / - revenue` and each Profit cell's key holds `profit`, while `ReplaceFormula` - addresses `profit-1`. The file line is `- profit(profit-1) = revenue - cost`. This - is answer 30's model applied to measures too, so a measure target and a dimension - target work the same way. -- **(b) is not a third model.** It is (a)'s model, with the rare dimension line - spelled `- Tax(tax-1)(tax) = … [customcat]` (display, formula id, item id), so that - the target position always holds a display. Under (a) and (c) the dimension line - is `- tax(tax-1) = … [customcat]`, with the item id in the target position. - -**Where (a) and (c) differ.** - -| | (a) one id (and (b)) | (c) two ids | -|---|---|---| -| A Profit cell's coordinate | the formula's id, as `ItemHandle::FormulaTarget` | an ordinary item id | -| `CellKey` | `(CategoryId, ItemHandle)` | `(CategoryId, ItemId)`, and `ItemHandle` goes | -| A stored `Profit` column and a formula `Profit = …`, merged into one measure today by `measure_item_names` | one column with two ids (the item's `profit`, the formula's `profit-1`), and the ADR must say which keys the cells | the formula attaches to the existing item | -| `RemoveFormula` on Profit | the measure goes too | the item `profit` stays, with no formula and maybe no data, as a dimension target's item stays today | -| What the file must newly write | nothing: the formula line declares the measure | the item `profit` and its display (today a formula target is not written as an item, and is rebuilt from `## Formulas`) | -| A dimension target (answer 30) | works differently from a measure target | works the same way | -| ADR 008 (PR #308), which anchors the cursor by item identity | needs `ItemHandle` | works as it is | -| ADR 009 (PR #309), which uses `find_item_category`'s `_Measure` fallback for formula targets | keeps it | loses it | - -Neither ADR 008 nor ADR 009 is edited here. - -**What is open.** The choice between (a) and (c) and, under (a), whether the -dimension line is spelled (a)'s way or (b)'s. **No recommendation is made.** Answer -40, "Formulas are a derived measure", reads most literally as (a), where the formula -*is* the measure. But (c) also models a derived measure, an item whose values a -formula derives, and the answer was a request for clarity rather than a choice. So -reading it as choosing (a) would be the ADR's judgement, not the user's. - -Smaller points, each the ADR's own reading and none a user answer: -- **The `[_measure]` tag.** Today's writer omits the tag for a `_Measure` formula - (`- {raw}`), and the user's own example, `profit = revenue - cost`, has none. The - examples above omit it too. Whether the new writer keeps omitting it is not decided - here. -- **Answers 5 and 19** picture the measure case as `Profit(3)` (the file line) and - `Profit (profit)` (the panel). The panel picture fits every option. The file-line - picture, a display before the id, fits (a) exactly and (c) only loosely, because - (c) writes the item id where answer 5 shows the display. -- **References to a formula target** are the id text of its handle: under (a) the - item id outside `_Measure` and the formula id for a `_Measure` target; under (c) - always the target's item id. -- **Quoting the target position.** Where a display sits in the target position - ((a) for a measure, and (b)), a display that looks like `Name(x)` would read back - as a display and an id, so the writer pipe-quotes it: `- |Margin(%)|(margin) = …`. - An item id never contains a parenthesis, so only a display needs it. (c) has no - such position. - -**Q24. Does `a_ b` slug to `a__b`?** **Answered by answer 41: no.** Revision 9's -options were (a) keep it, its lean, and (b) collapse any run of whitespace and `_` -to one `_`. Answer 41, "The spaces should have been normalized to _ before -collapsing runs of _s", is (b): `a_ b` → `a_b` and `a__b` → `a_b`, so ids change for -displays that hold a double underscore. Its example, `Time \u{200B} Period`, now slugs -to `time_period`. Clause 4's steps 2 and 4 state it, and where step 4 goes is Q29. - -**Q25. Does a trailing run slug to a trailing `_`?** **Answered by answer 42: no.** -Revision 9's options were (a) keep it, its lean, (b) remove a trailing `_` as a -leading one is removed, and (c) trim whitespace before step 2. Answer 42 is (b): "A -trailing one should be dropped", so `East ` → `east` and `East_` → `east`, and a -slug that then collides takes the suffix. Clause 4's step 5 states it. - -**Q26. What may a formula refer to, and what is "a category value"?** Answer 37 says -"Formulas may refer to anything in a typed context", with the example of a formula -counting "the number of times a particular letter appears in a category value". -Today a formula reference (`Expr::Ref`) names an item or a formula target, and -`find_item_category` finds its category. The answer does not say: -- (i) whether a bare id in formula text may also name a category, a view or a group, - with its kind given by where it appears (an aggregate's argument, a `WHERE` clause, - a function's parameter); -- (ii) whether the text a function like that reads is the item's display (`Café`) - or its id (`caf`). The two differ, because slugify folds case and strips; -- (iii) whether this ADR must specify any of it, or only keep ids unique so that it - can be added later. - -The ADR's lean, not the user's, is (iii): the formula language has no string -function today, and answer 22 already makes a reference to any kind unambiguous. - -**Q27. Does ids-first outlive the interim resolver?** Answer 33 made ids-first the -order "before we have autocomplete". Answer 38 says ids "should always be preferred -when there's ambiguity". -- **(a) Yes.** Wherever typed text could name both an id and a display, the id wins, - and that includes the autocomplete's ranking (an id match offered first, say). - Clause 12's "interim" wording goes. -- **(b) No.** Once the autocomplete turns display text into ids, nothing is matched - and nothing is ambiguous, so answer 38 governs only until then. - -No recommendation. - -**Q28. Two places resolve a display outside a client.** Answer 39: "the client should -be the only place that allows any reference to something by a human-friendly name". -- **The old-version loader** resolves an old file's references against display - names (clause 5), because in an old file the name is the only identity there is. -- **Headless `improvise cmd` and `improvise script`** run command lines that name - things by display, with no client attached. - -The ADR's reading: both are allowed. The loader migrates a file and is not a person -referring to something. The CLI is the UI layer of its own invocation, and it -converts names to ids at its edge, as a client does. Is that right? - -**Q29. Does step 4's collapse come after the strip?** Clause 4 puts it after step 3. -That order is what makes answer 41's example, `Time \u{200B} Period`, give -`time_period`. Before the strip, the zero-width character keeps the two `_` apart, and -the result would still be `time__period`. -- **(a) After the strip**, as clause 4 is written. The ADR's lean. -- **(b) Before the strip.** - -**Q30. Which "other entities" share the reserved `_`?** Answer 42: "A leading _ is -reserved for virtual categories and measures and other entities." -- (i) No virtual measure exists today. Is one planned, or does "measures" mean the - `_measure` category? -- (ii) The ADR reads the virtual views, `_records` and `_drill` (clause 1), as among - the "other entities". Is that right? -- (iii) The ADR reads "dropped to: disambiguate with -#" as "dropped too; - disambiguate with -#": a slug that collides once its `_` is dropped takes the - suffix. Is that right? - -**Q31. Are the positions under `_Index` and `_Dim` ids?** Records mode gives each row -a synthetic coordinate under `_Index`, and each column one under `_Dim` (a column -index, gotcha 1). They are never persisted and never typed. Answer 43 says the two -categories have ids `_index` and `_dim`, as answer 23 said, but revision 9's reading -was about the positions, and the answer does not say what they are. -- **(a) Outside the id namespace**, revision 9's reading: a position `0` under - `_index` does not collide with a user's item `0`. -- **(b) Inside it**, spelled so that they cannot collide, such as `_index-0`. That - needs answer 42's reservation to cover them. -- **(c) Something else.** - -No recommendation. - -**Q32. Moving the virtual categories to lower-case ids.** Today `_Index`, `_Dim` and -`_Measure` are keys in the model, on the wire, in `ViewState` and in `.improv` files: -`web/demo.improv` has `_Measure: column`, `_Index: none`, `## Category: _Measure` -and `_Measure=Revenue`. Under answers 23 and 43 they become `_index`, `_dim` and -`_measure`, and the displays stay as they are. The ADR's reading of the migration: -- an old file's `_Measure` resolves to `_measure` through its display at load, like - any other name (clause 5). The first save writes `_measure: column` and - `_measure=revenue`; -- a new-version file must spell the id, and `_Measure` there is refused (answer 20); -- the virtual category's header carries its id like any other: - `## Category: _Measure(_measure)`; -- every string literal that names them in code, on the wire and in the browser client - moves in the re-key step (clause 11, step 3). - -Is that the intended migration? - -**Q33. How do groups carry ids?** Answer 44 decides that a group has an id and a -separate friendly name. It does not decide: -- (i) **Namespace.** Is a group id unique in the whole document, like every other id - (the ADR's reading of answers 22 and 37), or only within its category? -- (ii) **Generation.** Is it the slug of the display, with `group` as an empty slug's - `` and the shared counter for collisions (the ADR's reading of answer 16(a), - "All id generation")? -- (iii) **The file.** For example a definition `> Coastal(coastal)`, a membership - `- Coastal_East(coastal_east)[coastal]`, a parent group (improvise-felo.6's - hierarchies) named by id, and a `collapsed:` line holding the group id, with or - without its category id. -- (iv) **The wire.** Does `AddItemInGroup`'s `group` name an existing group by id, - create one by display, or either? Is there a `RenameGroup { group, to }`? -- (v) **The frame.** `AxisFrame`'s `FrameGroup` carries a group name today. PR #336 - (improvise-cq1q.12, open) adds a display beside it. - -No recommendation beyond the readings stated. - -**Q34. Hand-written ids, and how step 5's trims repeat.** Answer 45: "A slug of - -should be invalid as an id and there should be no leading or trailing -s". -- (i) Is a hand-written id that is `-`, or begins or ends with `-`, refused at load, - as one beginning with `_` is? The ADR's reading is yes (clause 5). -- (ii) Is a hand-written id ending in `_` refused too? Answer 42 drops a trailing `_` - from a slug, and does not say whether a written id may carry one. -- (iii) Step 5 repeats until neither end holds `_` or `-`, so `-_x` → `x` and `x_-` → - `x`. One pass of each would leave `_x`, which is reserved, or `x_`. The ADR's - reading is to repeat. - -**Where revision 9 states a reading of its own.** In each of these fourteen places it -states a reading that is not a user answer. Each is flagged where it appears, and -approving the PR approves them: -- the virtual views' reserved ids `_records` and `_drill`, now read as among answer - 42's "other entities" (clause 1; Q30); -- groups share the document's one id namespace (clause 1; Q33); -- a `CellKey` coordinate is `(CategoryId, ItemHandle)` (clause 2; **pending Q23**); -- "whitespace" means the Unicode `White_Space` property (replacing the U+0020 reading, - by answer 34) (clause 4); -- step 4's collapse comes after the strip (clause 4; Q29); -- step 5's trims repeat until neither end holds `_` or `-` (clause 4; Q34); -- a hand-written id that is `-` or begins or ends with `-` is refused at load - (clause 5; Q34); -- the axis-before-`format:` order and the quoting of a format that reads as an axis - line (clause 5); -- the formula-line shape, and a `_Measure` target's display living on the `Formula` - (clause 5; **pending Q23**); -- CheckedBatch's created-record exemption is removed (clause 6); -- a display equal to another thing's id counts as ambiguous in the formula panel - (clause 6); -- an id is matched exactly, with no case folding (clause 12); -- the display stage of the typed-name resolver goes once autocomplete exists - (clause 12; Q27, since answer 38); -- answer 42's "dropped to:" is read as "dropped too;" (ninth round; Q30). - -Three readings that revision 9 listed here are gone. The user rejected the -`_Index`/`_Dim` reading as worded (answer 43, Q31), gave groups ids, which overrides -the `collapsed:` reading (answer 44), and rejected the legal `-` slug (answer 45). +**improvise-fdk1 (renaming a records coordinate).** +`RenameRecordCoordinate { record, category: String, item: String }` and +`CellId::Coordinate { record, category: String }` exist on main. This ADR changes +three things: +- `category` becomes a `CategoryId`. +- `item` becomes `ItemRef::{Existing(ItemId), New(display)}`. +- The rename **changes a coordinate**, so the record's `CellKey`s and `RecordHash` + change. It is a different record. ## Consequences **Good.** - Edit, rename and replace are distinct, addressed operations. -- A rename is one display write: it moves no key, surrogate or basis tier, and in-flight - edits survive it. +- A rename is one display write, and in-flight edits survive it. - `## Data` keys are unique by construction. -- km71's `FormulaId` becomes a real id by changing one struct, and kb1d's refusal can be - lifted. +- km71's `FormulaId` becomes a real id by changing one struct. **Bad.** -- Re-keying storage is wide: every name-taking API moves to ids, and display→id - resolution moves to the input edges. -- A second version literal (clause 10), and every old file changes version on its first - save. +- Re-keying storage is wide. Every name-taking API moves to ids. +- Old files change version, and their data keys change case, on the first save. - `Rejection` loses `Copy`. -- `Formula::raw` is no longer verbatim: the file carries formula text over ids. -- The formula panel is a second rendering of formula text, over display names, so - the **client** must resolve display names typed there back to ids (answers 19, - 32, 33 and 39), ids first (answers 33 and 38). The host sees only id-form text. -- Ids are lower case, so an old file's data keys change spelling on its first save. -- ADR 002's prose needs a wording update when this lands. +- `Formula::raw` is no longer verbatim. The file carries formula text over ids. +- The formula panel is a second rendering of formula text, so display names typed + there must be resolved back to ids (by the client, under the amendment). +- ADR 002's prose needs a wording update. -**Not addressed.** Per-entity versions (Q6), and ids in `AxisFrame` and -`CellId::Derived`. Group ids are now in scope (answer 44, overriding Q5's answer), -but how they are made, spelled and carried is Q33. +**Not addressed.** Per-entity versions, ids in `AxisFrame` and `CellId::Derived`, and +(under the accepted text) group ids. + +## Open questions + +These are open, and nothing above depends on an answer except where it says so. + +### 1. Is a formula and the measure it derives one thing with one id, or two things with two ids? + +**The question.** A formula is a derived measure. Take `Revenue` and `Cost`, stored +measures with ids `revenue` and `cost`, and add `Profit = Revenue - Cost`. Profit holds +no data: the formula computes its column. Two kinds of thing must name Profit by an +id: +- things that refer to the **measure**: another formula (`Margin = Profit / Revenue`), + each Profit cell's `CellKey`, a `hidden:` line, and ADR 008's cursor anchor; +- things that refer to the **formula**: `ReplaceFormula` and `RemoveFormula`. + +Do they use the same id? + +**Why it matters.** It decides the `CellKey` type (clause 2), the formula line's +shape (clause 5), `RecordHash` (clause 8), the `_Measure` panel rows (clause 6) and +what `.improv` must write. + +**Options.** +- **(a) One id.** The formula *is* the measure. Profit is `profit` everywhere, with + its display on the formula, and the line is `- Profit(profit) = revenue - cost`. + - A Profit cell's coordinate is `ItemHandle::FormulaTarget(profit)`. + - The cost: a stored `Profit` column and a formula `Profit` (merged into one measure + today by `measure_item_names`) would have two ids, and one must be chosen to key + cells. + - `RemoveFormula` removes the measure. + - The rare dimension target (an item `tax` plus formula `tax-1`) works differently. + - ADR 008 needs `ItemHandle`. + - This is the smallest change from the text above. +- **(b)** is (a) with the dimension line spelled `- Tax(tax-1)(tax) = … [customcat]` + (display, formula id, item id), so that the target position always holds a display. +- **(c) Two ids.** Profit is a `_Measure` item `profit`, and its formula is `profit-1`. + The line is `- profit(profit-1) = revenue - cost`. + - `CellKey` returns to `(CategoryId, ItemId)` and `ItemHandle` goes. + - A same-named stored measure is simply the item the formula attaches to. + - Measure and dimension targets work alike. + - The cost: the file must newly write the item and its display (today a formula + target is rebuilt from `## Formulas`). + - `RemoveFormula` leaves an item with no formula and maybe no data. + - ADR 009's `_Measure` fallback in `find_item_category` goes. + +Also to settle: whether the writer keeps omitting the `[_measure]` tag on a `_Measure` +formula line, as today's writer does. + +**Recommendation:** none. Both (a) and (c) model a derived measure, and the choice is a +product judgement about whether a formula and its column are one thing. + +### 2. What may a formula refer to? + +**The question.** Formulas may refer to things in a typed context. An example is a +function counting how often a letter appears in a category value. Today a formula +reference (`Expr::Ref`) names an item or a formula target. Three things are open: +- (i) may a bare id in formula text also name a category, a view or a group, with + its kind given by where it appears? +- (ii) does a text function read an item's display (`Café`) or its id (`caf`)? They + differ, because slugify folds case and strips; +- (iii) must this ADR specify that, or only keep ids unique so it can be added later? + +**Why it matters.** It decides whether the formula grammar and evaluator change with +this ADR. + +**Author's recommendation:** (iii). The formula language has no string function +today, and global uniqueness already makes a reference to any kind unambiguous. + +### 3. Does ids-first matching outlive the interim resolver? + +**The question.** Clause 12's resolver matches typed text against ids before displays, +and ids are preferred wherever there is ambiguity. Once a view-layer autocomplete +turns display text into ids, does the preference carry into it? + +**Options.** +- **(a) Yes.** Wherever typed text could name both an id and a display, the id wins, + including in the autocomplete's ranking. +- **(b) No.** Once the autocomplete exists, nothing is matched at commit, so the rule + ends with the interim resolver. + +**Recommendation:** none. It matters only once an autocomplete is designed. + +### 4. May anything other than a client resolve a display name? + +**The question.** Only a client accepts a display as a reference (clause 6). Two +places would resolve displays outside one: +- the **old-version loader**, which resolves an old file's references by display + (clause 5), because in an old file the name is the only identity; +- **headless `improvise cmd` and `improvise script`**, which run command lines naming + things by display, with no client attached. + +**Why it matters.** If neither is allowed, old files cannot be migrated and the +headless CLI needs a client layer of its own. + +**Author's recommendation:** allow both. The loader is migrating a file, not a person +referring to something. The CLI is the UI layer of its own invocation and converts at +its edge, as a client does. + +### 5. Does the `_` collapse come after the strip? + +**The question.** Clause 4 collapses runs of `_` (step 4) after stripping characters +outside the alphabet (step 3). + +**Why it matters.** `Time \u{200B} Period` (a zero-width space between two spaces) +gives `time_period` only if the collapse comes after the strip. Before it, the +zero-width character still separates the two `_`, and the result is `time__period`. + +**Options:** (a) after the strip, as written; (b) before it. +**Author's recommendation:** (a). + +### 6. Which entities share the reserved leading `_`? + +**The question.** A leading `_` is reserved for virtual categories, measures and other +virtual entities. +- (i) No virtual measure exists today. Is one planned, or does "measures" mean the + `_measure` category? +- (ii) Are the virtual views, `_records` and `_drill` (clause 1), among the reserved + entities? + +**Why it matters.** It fixes `VIRTUAL_IDS` and what `claim` refuses. + +**Author's recommendation:** (ii) yes, as clause 1 is written. Otherwise `_Drill` +slugs to `drill` and can collide with a user's view. + +### 7. Are records mode's positions under `_Index` and `_Dim` ids? + +**The question.** Records mode gives each row a synthetic coordinate under `_Index`, +and each column one under `_Dim` (a column index, gotcha 1). They are never persisted +and never typed. + +**Options.** +- **(a)** Outside the id namespace: a position `0` under `_index` does not collide + with a user's item `0`. +- **(b)** Inside it, spelled so that it cannot collide (for example `_index-0`), under + the reservation of a leading `_`. +- **(c)** Something else. + +**Why it matters.** Under (b), uniqueness is checked for synthetic coordinates, and +`claim`'s exact match on reserved ids must change. + +**Recommendation:** none. + +### 8. How do the virtual categories move to their lower-case ids? + +**The question.** Today `_Index`, `_Dim` and `_Measure` are keys in the model, on the +wire, in `ViewState` and in `.improv` files. For example, `web/demo.improv` has +`_Measure: column`, `_Index: none`, `## Category: _Measure` and `_Measure=Revenue`. +Under clause 1 they become `_index`, `_dim` and `_measure`, with the displays +unchanged. The proposed migration: +- an old file's `_Measure` resolves to `_measure` through its display at load, like + any name, and the first save writes `_measure: column` and `_measure=revenue`; +- a new-version file must spell the id, and `_Measure` there is refused (ids are + lower case); +- the header becomes `## Category: _Measure(_measure)`; +- every string literal naming them in code, on the wire and in the browser client + moves in the re-key step. + +**Why it matters.** Every shipped document and much test code spells these names. + +**Author's recommendation:** the migration above. + +### 9. Does a group also carry a display name? + +**The question.** A group's identifier is an id, not a friendly name (clause 1). Does +a group also have a separate display, as categories, items and views do? + +**Why it matters.** +- If it does not, the grid and panels show the id: `Coastal East` would appear as + `coastal_east`. +- If it does, groups need a display rename like everything else, and the file + declares both. +- The code is already moving: PR #336 (improvise-cq1q.12, open) adds a private + display to `Group`, beside its key. + +**Options:** (a) yes, a display beside the id; (b) no, the id is all a group has. +**Author's recommendation:** (a), for consistency with every other id-bearing kind and +so that users see what they typed. + +### 10. How do groups carry ids? + +**The question.** A group's identifier is an id. Not yet decided: +- (i) **Namespace.** Is a group id unique in the whole document, like every other id, + or only within its category? +- (ii) **Generation.** Is it the slug of a display (if question 9 gives it one), with + `group` as an empty slug's `` and the shared counter for collisions? +- (iii) **The file.** For example a definition `> Coastal(coastal)`, a membership + `- Coastal_East(coastal_east)[coastal]`, and a parent group (group hierarchies, + improvise-felo.6) named by id. Does a `collapsed:` line hold the group id with or + without its category id? +- (iv) **The wire.** Does `AddItemInGroup`'s `group` name an existing group by id, + create one by display, or either? Is there a `RenameGroup { group, to }`? +- (v) **The frame.** `AxisFrame`'s `FrameGroup` carries a group name today. What does + it carry? + +**Why it matters.** Clauses 2, 5, 6 and 7 cannot be implemented for groups without +these answers. + +**Author's recommendation:** (i) document-wide, as every other id is; (ii) slugify, +like every generated id. No recommendation on (iii)–(v). + +### 11. What may a hand-written id look like at its ends, and do slugify's trims repeat? + +**The question.** No id may be `-`, and no id begins or ends with `-`. +- (i) Is a hand-written id that begins or ends with `-` refused at load, as one + beginning with `_` is? +- (ii) Slugify drops a trailing `_`. Is a hand-written id ending in `_` refused too? +- (iii) Clause 4's step 5 repeats until neither end holds `_` or `-`, so `-_x` → `x`. + One pass of each would leave `_x`, which is reserved, or `x_`. Is repeating right? + +**Why it matters.** It decides what `IdAllocator::claim` refuses. Today it accepts +`-`, `-x` and `x-`. + +**Author's recommendation:** (i) yes; (ii) yes, so that a hand-written id obeys the +same shape as a generated one; (iii) repeat. diff --git a/context/code/core.md b/context/code/core.md index 49e6c119..04cbdf89 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1691,7 +1691,8 @@ probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a `slugify` and `IdAllocator` are the first piece of ADR 007. They are pure functions of the display and the allocator's own history. Nothing calls them yet: the creation sites move onto them in the re-key step (improvise-cq1q.4). -Each rule traces to a numbered answer in the ADR's "What the user decided". +Each rule traces to a numbered answer of the user's. The ADR no longer carries +them; they are listed, with comment ids, in PR #326's provenance comment. **Case folding is full Unicode folding, and it comes from `unicase`.** `str::to_lowercase` is not folding: `ß` lowercases to itself and is then @@ -1712,8 +1713,8 @@ slugs to `a__b`: the steps are applied as written, and no extra underscore-collapsing step exists. A trailing run survives, so `East ` slugs to `east_`. Only *leading* underscores are removed (step 4). -**ADR 007 revision 9 proposes different rules, and this code predates them.** -Revision 9 is not yet approved. Its answers 34, 41, 42 and 45 make clause 4: +**ADR 007's proposed amendment changes these rules, and this code predates it.** +The amendment (PR #326) is not yet approved. Its clause 4: - each whitespace character becomes `_`, not only U+0020; - runs of `_` collapse after the strip, so `a_ b` and `a__b` both slug to `a_b` and `Time \u{200B} Period` to `time_period`; @@ -1723,8 +1724,8 @@ Revision 9 is not yet approved. Its answers 34, 41, 42 and 45 make clause 4: This section, `slugify`, and the tests `slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and `slugify_keeps_an_underscore_the_display_already_had` describe the earlier rule -above. They change in a follow-up once revision 9 is approved. The ADR's clause 4 -lists every difference. +above. They change in a follow-up once the amendment is approved. The ADR's +clause 4 lists every difference. **The allocator records every id it admits and releases none.** That is the whole of retirement (clause 4, "within a session a removed id is never handed @@ -1748,8 +1749,8 @@ the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading `_` (only `_index`, `_dim` and `_measure` may carry one, per answer 23, and the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other id may start with `_`"), and -duplicates. It accepts `-`, `-x` and `x-`. Revision 9's Q34 asks whether a -hand-written id like that must be refused (answer 45). It never renames. Refusing is the loader's job, and the loader +duplicates. It accepts `-`, `-x` and `x-`. ADR 007's open question 11 asks +whether a hand-written id like that must be refused. It never renames. Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the loader keeps the line numbers. The virtual views `_records` and `_drill` are not yet in `VIRTUAL_IDS`. improvise-p003 item 1 asks for them, and they arrive From 1190c80efe24d005209e1fcddf97c34327b1c38d Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Thu, 8 Oct 2026 23:22:01 -0700 Subject: [PATCH 12/27] docs(adr): tag every ADR 007 change to accepted text as [Amendment] (improvise-cq1q.1) The Status now says the tag is the complete list of proposed rules, and its table gains the regression test's step 3. Clause 12's test keeps the accepted "commit the unedited text" with the amendment's edit-a-non-name step tagged; the outside-_Measure real-item rule, the formula-collision example and the East/East_ collision sentences are tagged; the "exactly one id" sentence in clause 2 is restored; "other entities" loses the added "virtual"; the title drops "and groups". core.md section 20 and the id.rs test evidence cite ADR clauses instead of answer numbers. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 65 ++++++++++++++-------------- context/code/core.md | 20 +++++---- context/test-evidence/core.md | 15 ++++--- 3 files changed, 52 insertions(+), 48 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 8f9ec480..79c29a4a 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -1,4 +1,4 @@ -# ADR 007 — Stable identities for categories, items, formulas, views and groups +# ADR 007 — Stable identities for categories, items, formulas and views **Ticket:** improvise-sboz (implementation epic improvise-cq1q; this amendment improvise-cq1q.1) · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's @@ -9,8 +9,9 @@ draft call this document "ADR 005". ## Status **Accepted** on 2026-09-25 (PR #210). **An amendment is proposed and not yet -approved** (PR #326). The amendment's text is marked **[Amendment]** wherever it -appears. Everything without that mark is accepted and binding. +approved** (PR #326). **Every rule tagged [Amendment] is proposed; everything +without the tag is accepted and binding.** The tag is the complete list of what the +amendment adds. Where the amendment replaces accepted text, the accepted rule binds until the amendment is approved: @@ -23,15 +24,9 @@ amendment is approved: | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | a run of U+0020 spaces becomes one `_`; only a leading `_` is removed | each whitespace character becomes `_`, runs of `_` collapse after the strip, and leading and trailing `_` and `-` are removed | | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | +| 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | -The amendment also adds rules where the accepted text says nothing: -- a measure has an id and a display; -- the axis-line/`format:` order; -- `ViewState` and `Workbook` key the active view by id; -- CheckedBatch's created-record exemption is removed; -- two branches of one file can merge into a file that will not load. - -Each is marked where it appears. Eleven questions are open (last section). +Eleven questions are open (last section). ## Context @@ -84,15 +79,15 @@ human name moves to a new `display` field. **An id always names the same thing, so ids are unique in the whole document**, across every kind. An item id names one item without its category. A `CellKey` still pairs each item with its category, and an operation that creates an item still names the -parent category. A formula whose generated id would collide takes a suffix like any -other generated id (clause 4). A formula `Tax` in `CustomCat` gets item id `tax`, then -formula id `tax-`. **[Amendment]** Formulas may refer to anything in a typed -context. What exactly that allows is open question 2. +parent category. **[Amendment]** A formula whose generated id would collide takes a +suffix like any other generated id (clause 4): a formula `Tax` in `CustomCat` gets +item id `tax`, then formula id `tax-`. **[Amendment]** Formulas may refer to +anything in a typed context. What exactly that allows is open question 2. **Reserved ids.** The virtual categories have ids `_index`, `_dim` and `_measure`, displayed `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** **[Amendment]** A leading `_` is reserved for virtual categories, measures and other -virtual entities. The virtual views have ids `_records` and `_drill`, displayed +entities. The virtual views have ids `_records` and `_drill`, displayed `_Records` and `_Drill`. They are never persisted, but they are addressed on the wire (`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and could collide with a user's view. Which entities the reservation covers is open question 6. Whether @@ -102,8 +97,8 @@ Moving today's capitalised keys to these ids is open question 8. **Formula targets.** **[Amendment]** A measure is the normal formula target (`profit = revenue - cost`). A dimension (a category other than `_Measure`) is a rare one, for a derived dimension. -- **A target outside `_Measure` is a real item**, with its own `ItemId`, and its - formula has a separate `FormulaId`. +- **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, + and its formula has a separate `FormulaId`. - **A `_Measure` target has no item of its own**, and its handle is its formula's `FormulaId`. So the category panel's `_Measure` rows use `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). @@ -141,8 +136,9 @@ callers, the evaluator's `Expr::Ref` lookup and the `View` methods. Display→id resolution moves to the places a human types. The field rename itself is cheap. The change is mechanical but wide, and it is the bulk of the implementation. -**Display names stay unique within their scope**, as names are today, and a rename to -a display already in use there is refused. An item's scope is its category, so two +**Display names stay unique within their scope**, as names are today. So a +human-typed name resolves to exactly one id within its scope, and a rename to a +display already in use there is refused. An item's scope is its category, so two categories may each have an item displayed `Revenue`. Formula text crosses categories, so a display may be ambiguous there. The ids differ, so the formula panel shows the ids where a display is ambiguous. @@ -212,8 +208,9 @@ Examples under the amendment: | `Straße` | `strasse` | | `Ü`, `-`, `---` | `-` | -**Collisions.** `Time Period` and `Time_Period` slug alike, and so do `East`, `East ` -and `East_`. Whichever comes later takes a suffix. `East` in `Region` and `East` in +**Collisions.** `Time Period` and `Time_Period` slug alike, and whichever comes later +takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, which slug alike +only under the amendment's steps (under the accepted steps `East ` slugs to `east_`). `East` in `Region` and `East` in `Direction` also slug alike, and ids are global, so the second is `east-`. - **Any colliding generated id gets a suffix**, whatever kind of thing holds the slug: on import, when an id-less file loads, and when the host creates something in @@ -221,8 +218,8 @@ and `East_`. Whichever comes later takes a suffix. `East` in `Region` and `East` - **The suffix is `-` plus the next value of one increment-only counter per document**, as a plain number (`east-1`, not `east-001`). The counter works like Common Lisp's `gensym`: it never goes down and is never reused, so no order holds - among suffixes (`east-7` may follow `east` directly). A suffix ends in a digit, so a - suffixed id still meets step 5. + among suffixes (`east-7` may follow `east` directly). **[Amendment]** A suffix ends + in a digit, so a suffixed id still meets the amendment's step 5. - **Retirement.** Within a session a removed id is never handed out again. The counter is past any suffix it issued, and a removed bare slug stays reserved in memory. @@ -589,12 +586,16 @@ without being told. That is why the panel shows B as `east-1` and never as a bar 1. Open a formula editor on `Profit = Revenue - Cost`. 2. From another client, rename `Revenue` to `Sales`, and rename a different item to `Revenue`. -3. Edit something that is not a name (`Revenue - Cost * 1`) and commit. +3. Commit the unedited text. The stored formula must reference the original item's id, and the commit must not be refused. The test must fail under commit-time resolution and under a refusal on -rename. Committing the *unedited* text proves nothing, because a client may treat it -as a no-op without resolving any name. +rename. + +**[Amendment]** Step 3 becomes: edit something that is not a name +(`Revenue - Cost * 1`) and commit. Committing the unedited text cannot tell a correct +capture from no capture at all, because a client may treat unchanged text as a no-op +without resolving any name (improvise-hexb). **[Amendment] Required test for ids-first.** With items A (`East`, `east`) and B (`east`, `east-1`), type `east` into an editor, in text not captured at open: @@ -773,11 +774,11 @@ zero-width character still separates the two `_`, and the result is `time__perio ### 6. Which entities share the reserved leading `_`? **The question.** A leading `_` is reserved for virtual categories, measures and other -virtual entities. -- (i) No virtual measure exists today. Is one planned, or does "measures" mean the - `_measure` category? -- (ii) Are the virtual views, `_records` and `_drill` (clause 1), among the reserved - entities? +entities. +- (i) Does "measures" mean virtual measures (none exists today), the `_measure` + category, or something else? +- (ii) Which "other entities" are covered? In particular, are the virtual views, + `_records` and `_drill` (clause 1), among them? **Why it matters.** It fixes `VIRTUAL_IDS` and what `claim` refuses. diff --git a/context/code/core.md b/context/code/core.md index 04cbdf89..ffcee2e4 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1691,12 +1691,12 @@ probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a `slugify` and `IdAllocator` are the first piece of ADR 007. They are pure functions of the display and the allocator's own history. Nothing calls them yet: the creation sites move onto them in the re-key step (improvise-cq1q.4). -Each rule traces to a numbered answer of the user's. The ADR no longer carries -them; they are listed, with comment ids, in PR #326's provenance comment. +Each rule traces to ADR 007's clause 4 (slugify, the counter, hand-written +ids) or clause 1 (reserved ids). **Case folding is full Unicode folding, and it comes from `unicase`.** `str::to_lowercase` is not folding: `ß` lowercases to itself and is then -stripped, so `Straße` would slug to `strae` (answer 26). The same goes for `ſ` +stripped, so `Straße` would slug to `strae` (clause 4, step 1: full case folding). The same goes for `ſ` (long s) and the `fi` ligature. `unicase` 2.9 is `no_std` and has no dependencies, so it passes the wasm gate. Its `to_folded_case` applies the C+F mappings of Unicode's CaseFolding.txt, and its own tests pin `Maße` → @@ -1705,8 +1705,8 @@ letter survives step 3. Deriving that ASCII-producing subset of the table by hand was rejected. It is a second copy of the Unicode data, and nothing would say when it drifted. -**A "space" is U+0020.** Answer 18 says "replace space with `_`", and answer 21 -says runs collapse. A tab or a no-break space is not a space here, so it is +**A "space" is U+0020.** The accepted clause 4 says "replace each run of spaces +with one `_`". A tab or a no-break space is not a space here, so it is stripped by step 3. Only a run of U+0020 becomes one `_`. An underscore the display already had is never merged with the one a space run makes. So `a_ b` slugs to `a__b`: the steps are applied as written, and no extra @@ -1732,7 +1732,8 @@ whole of retirement (clause 4, "within a session a removed id is never handed out again"). No release operation exists, so a removed id cannot be reissued by construction; no check refuses it. The counter starts at 1, is shared by every kind, and advances each time a suffix is tried, including a candidate -that turns out to be taken by a hand-written id (answers 28 and 29). So +that turns out to be taken by a hand-written id (clause 4: the `gensym` counter, and hand-written +ids are canonical). So `east-7` may follow `east` directly, and no order is enforced among suffixes. **The walk past taken candidates is a loop, never recursion.** A new-version file saved from a non-ASCII import holds `item-1` … `item-N`, and @@ -1741,16 +1742,17 @@ version recursed once per candidate, not in tail position, and aborted the process on a stack overflow: in debug at N ≈ 11,000, in release at 50,000. An abort, not a panic, so the TUI lost unsaved edits (Codex P2 on PR #318). An empty slug always takes a suffix (`category-1`), even when the bare kind -name happens to be free, because answer 16(e) spells the result +name happens to be free, because clause 4 spells the result `-`. **`claim` is the only door for a hand-written id.** It checks, in this order: the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading -`_` (only `_index`, `_dim` and `_measure` may carry one, per answer 23, and +`_` (only `_index`, `_dim` and `_measure` may carry one, per clause 1, and the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other id may start with `_`"), and duplicates. It accepts `-`, `-x` and `x-`. ADR 007's open question 11 asks -whether a hand-written id like that must be refused. It never renames. Refusing is the loader's job, and the loader +whether a hand-written id like that must be refused. It never renames. +Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the loader keeps the line numbers. The virtual views `_records` and `_drill` are not yet in `VIRTUAL_IDS`. improvise-p003 item 1 asks for them, and they arrive diff --git a/context/test-evidence/core.md b/context/test-evidence/core.md index 354bceba..d4ae4c48 100644 --- a/context/test-evidence/core.md +++ b/context/test-evidence/core.md @@ -1935,14 +1935,14 @@ Rival: every space becomes `_`, with runs not collapsed. Only this test fails: ### model::id::tests::slugify_keeps_an_underscore_the_display_already_had -Rival: `.replace("__", "_")` after the space step, which reads answer 21 as -"no doubled underscores". Only this test fails: `left: "a_b"`. The rule +Rival: `.replace("__", "_")` after the space step, which reads the accepted +clause 4 as "no doubled underscores". Only this test fails: `left: "a_b"`. The rule collapses runs of *spaces*. An underscore that was in the display is kept. ### model::id::tests::the_suffix_is_the_next_counter_value_not_the_lowest_free_number -Rival: the lowest free `-` for each base, which revision 5 of the ADR -proposed and answer 28 replaced. Three tests fail: this one (`"west-1"` against +Rival: the lowest free `-` for each base, which an early draft of +ADR 007 proposed and clause 4's shared `gensym` counter replaced. Three tests fail: this one (`"west-1"` against `"west-2"`), `an_empty_slug_takes_its_kind_and_the_shared_counter` (`"item-1"`) and `a_slug_taken_by_another_kind_takes_a_suffix` (`"time_period-1"`). Only interleaving two bases separates one shared counter from per-base numbering, @@ -1952,8 +1952,9 @@ and this test is the one that does it on purpose. Rival: `claim` validates the id but does not record it. This test and `claim_refuses_a_duplicate_a_malformed_and_a_reserved_id` fail. The second -`East` is issued `east-1`, the file's own id, which is the collision answer 29 -forbids: "doing otherwise will corrupt the data section". +`East` is issued `east-1`, the file's own id, which is the collision clause 4 +forbids: hand-written ids are canonical, and renaming one would corrupt the +data section. ### model::id::tests::generated_ids_are_distinct_well_formed_and_never_reserved @@ -1990,7 +1991,7 @@ exempt any id that begins with a virtual id (`VIRTUAL_IDS.iter().any(|v| id.starts_with(v))`), all 11 earlier tests passed, because `claim_refuses_a_duplicate_a_malformed_and_a_reserved_id` tries only `_draft`. This test fails on that rival (`left: Ok(())` for `_indexer`). The match is -exact by the ADR's own wording: answer 23 says no other id may begin with `_`. +exact by the ADR's own wording: clause 1 says no other id may begin with `_`. ### model::id::tests::a_display_with_leading_spaces_never_slugs_into_the_reserved_space From 6e4678788785dee2b9c1c31c9e20d409b54e3643 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:01:10 -0700 Subject: [PATCH 13/27] docs(adr): settle ADR 007's eleven open questions and state the decisions (improvise-cq1q.1) The decisions are written into the clauses as plain design, tagged [Amendment]: a measure formula's id is the measure's id; formulas may refer to any id-bearing thing; ids always win, with the display as a preview; user interfaces accept display names and scripts use ids and display_for; slugify replaces every non-alphanumeric character with _, collapses _, trims _ at both ends and prefixes n before a digit; a hand-written id obeys the generated syntax; a leading _ is reserved for virtual ids, which are lower case and never written to .improv; records-mode positions are values; groups have a display and ordinary ids. All eleven questions are deleted. Five new ones are open: the - suffix in hand-written ids, ASCII "alphanumeric", how a file records the virtual categories, the dimension formula line and [_measure] tag, and which headless entry points are scripts. core.md section 20's note follows the new slug rules. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 539 ++++++++++++--------------- context/code/core.md | 15 +- 2 files changed, 247 insertions(+), 307 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 79c29a4a..36e719e0 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -18,15 +18,16 @@ amendment is approved: | clause | accepted rule | the amendment proposes | |---|---|---| -| 1 | groups have no id | a group's identifier is an id | -| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | the virtual views also have reserved `_` ids: `_records`, `_drill` | -| 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` (pending open question 1) | +| 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | +| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; virtual ids are never written to `.improv` | +| 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | -| 4 | a run of U+0020 spaces becomes one `_`; only a leading `_` is removed | each whitespace character becomes `_`, runs of `_` collapse after the strip, and leading and trailing `_` and `-` are removed | +| 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every non-alphanumeric character becomes `_`; runs of `_` collapse; leading and trailing `_` are removed; `n` is prepended to a slug that starts with a digit | +| 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` and may begin with `_` only if reserved | a hand-written id obeys every syntax rule of a generated id | | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | | 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | -Eleven questions are open (last section). +Five questions are open (last section). ## Context @@ -73,55 +74,79 @@ human name moves to a new `display` field. | item | yes | `ItemId` | | formula | yes | `FormulaId` | | view | yes | `ViewId` | -| group | **[Amendment]** yes (open questions 9 and 10) | open question 10 | +| group | **[Amendment]** yes, with a display | `GroupId` | | record | `RecordHash`, over ids (clause 8) | unchanged shape | **An id always names the same thing, so ids are unique in the whole document**, across every kind. An item id names one item without its category. A `CellKey` still pairs each item with its category, and an operation that creates an item still names the -parent category. **[Amendment]** A formula whose generated id would collide takes a -suffix like any other generated id (clause 4): a formula `Tax` in `CustomCat` gets -item id `tax`, then formula id `tax-`. **[Amendment]** Formulas may refer to -anything in a typed context. What exactly that allows is open question 2. +parent category. + +**[Amendment] Groups.** A group has an id and a display, like a category or an item, +and its id behaves like every other id: it is unique in the document, generated by +slugify (`group` is an empty slug's ``), declared beside the display in the file, +referred to by id, created by display, and renamed by a display rename. In code, +PR #336 (improvise-cq1q.12, open) already adds a private display to `Group` beside its +key; under this amendment that key becomes the group's id. **Reserved ids.** The virtual categories have ids `_index`, `_dim` and `_measure`, displayed `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** -**[Amendment]** A leading `_` is reserved for virtual categories, measures and other -entities. The virtual views have ids `_records` and `_drill`, displayed -`_Records` and `_Drill`. They are never persisted, but they are addressed on the wire -(`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and could collide -with a user's view. Which entities the reservation covers is open question 6. Whether -records mode's synthetic positions under `_Index` and `_Dim` are ids is open question 7. -Moving today's capitalised keys to these ids is open question 8. +**[Amendment]** A leading `_` is reserved for virtual ids in general. The virtual views +have ids `_records` and `_drill`, displayed `_Records` and `_Drill`; they are addressed +on the wire (`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and +could collide with a user's view. Slugify never produces a leading `_` (clause 4), so +the reservation needs no check in the grammar. +- **[Amendment] Virtual ids are lower case and are never written to `.improv`.** The + code, the wire, `ViewState` and the browser client spell the virtual categories + `_Index`, `_Dim` and `_Measure` today, and move to the lower-case ids in the re-key + step (clause 11). How the file records what it records today under `_Measure`, + `_Index` and `_Dim` is open question 3. +- **[Amendment] Records mode's synthetic positions are values, not ids.** The row + coordinate under `_index` and the column coordinate under `_dim` (a column index, + gotcha 1) are values of those virtual categories. They are never persisted or typed, + take no part in id uniqueness, and cannot collide with a user's item. -**Formula targets.** **[Amendment]** A measure is the normal formula target +**Formulas and measures.** **[Amendment]** A measure is the normal formula target (`profit = revenue - cost`). A dimension (a category other than `_Measure`) is a rare one, for a derived dimension. -- **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, - and its formula has a separate `FormulaId`. - **A `_Measure` target has no item of its own**, and its handle is its formula's `FormulaId`. So the category panel's `_Measure` rows use `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). -- **[Amendment]** A measure has an id and a display, as everything else does. - Whether a formula-target measure keeps the formula's id or becomes an item with - its own is open question 1. The `ItemHandle` text in clauses 2, 5, 6, 8 and 11 - depends on its answer. +- **[Amendment] A formula is a kind of measure, so a `_Measure` formula's id is the + measure's id.** A measure has an id and a display, as everything else does. + `Profit = Revenue - Cost` is one thing, `profit`: another formula refers to it as + `profit`, each Profit cell's `CellKey` holds `profit` under `_measure`, and + `ReplaceFormula` and `RemoveFormula` address `profit`. A formula that derives a + measure already stored under that display takes the stored measure's id, so the + column has one id, not two. Removing the formula removes a measure that exists only + through it; a measure that also holds stored data stays, as a stored measure. +- **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, + and its formula has a separate `FormulaId`. A formula whose generated id would + collide takes a suffix like any other generated id (clause 4): a formula `Tax` in + `CustomCat` gets item id `tax`, then formula id `tax-`. +- **[Amendment] Formulas may refer to anything by id.** A function in a formula may + accept any id-bearing thing (a category, an item, a measure, a formula, a group), + referred to by its id. Whether a given function takes the thing itself or its value + (an item's display, say) is part of that function's signature. Global uniqueness + makes every such reference unambiguous, and a function may read any part of a record, + so nothing further is needed here. ### 2. Storage keys by id; `display` is the only place a human name lives - **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id, - and each `Category`, `Item` and `View` gains a `display: String`. + and each `Category`, `Item` and `View` gains a `display: String`. **[Amendment]** So + does `Group`. - **Cells:** a `CellKey` is a sorted list of `(CategoryId, ItemId)` pairs. **[Amendment]** The pairs are `(CategoryId, ItemHandle)`, so that a `_Measure` formula cell's coordinate is `FormulaTarget` without claiming an `ItemId` the - document does not hold (pending open question 1). + document does not hold. - The key stays sorted, now **by category id**, so the canonical form (gotcha 5) is kept with a different sort key. - `DataStore` interns ids exactly as it interns names today. - An id's text names one thing, because ids are unique across kinds. - **View configuration:** the axis maps, page selections, and hidden and collapsed sets are keyed by ids. **[Amendment]** A collapsed entry names its group by the group's - id. Whether the category id stays beside it is part of open question 10. + id. - **Formulas:** a formula's references are ids (clause 5), and the formula panel renders them as display names (clause 6). @@ -158,68 +183,68 @@ copyable, compared as an integer. **Every** id the app generates comes from the display name by **slugify**: on import, on interactive creation, and when an id-less `.improv` file is loaded. So a -thing's id never depends on how it was made. An id is lower case, in the alphabet -`a`–`z`, `0`–`9`, `-` and `_`. +thing's id never depends on how it was made. An id is lower case. -The steps, in order, as the amendment states them **[Amendment]**: +**The accepted steps:** +1. casefold, with Unicode full case folding; +2. replace each run of spaces with one `_`; +3. strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`; +4. remove any leading `_`; +5. if nothing is left, use `-`. + +**[Amendment] The steps become:** 1. **Casefold, with Unicode full case folding**, so `Straße` → `strasse`. Rust's - `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped). The folding - table or crate must pass the `wasm` gate, because `improvise-core` is in the browser - bundle. -2. **Replace each whitespace character with `_`.** "Whitespace" is the Unicode - `White_Space` property (`char::is_whitespace`): U+0009–U+000D, U+0020, U+0085, - U+00A0, U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F and U+3000. - - A tab, a no-break space and an ideographic space each become `_`. - - A zero-width space (U+200B) and a byte-order mark (U+FEFF) are not whitespace, - and step 3 strips them. - - The realistic case is a header pasted from a web page, where a no-break space - stands for a space. -3. **Strip every character outside the id alphabet.** This strips non-ASCII letters, - so an id never needs quoting in an `.improv` reference. -4. **Collapse each run of `_` to one `_`**, whether the underscores came from - whitespace or were in the display. The step comes after the strip (open - question 5). -5. **Remove every leading and trailing `_` and `-`**, repeating until neither end - holds either (open question 11). - - A leading `_` is reserved (clause 1). - - A trailing `_` is dropped. - - An id never begins or ends with `-`. + `str::to_lowercase` is not enough (`ß` stays `ß`). The folding table or crate must + pass the `wasm` gate, because `improvise-core` is in the browser bundle. +2. **Replace every character that is not alphanumeric with `_`.** That includes + spaces, tabs and every other whitespace, punctuation and `-`. Whether + "alphanumeric" means ASCII only is open question 2. +3. **Collapse each run of `_` to one `_`**, whether the underscores came from step 2 + or were in the display. +4. **Remove every leading and trailing `_`.** A leading `_` is reserved for virtual + ids (clause 1), and a trailing one is dropped. +5. **If the slug starts with a digit, prepend `n`.** An id is never purely numeric + and never starts with a digit, so `2024` becomes `n2024`. 6. **If nothing is left, use `-`**, such as `category-1` or `item-4`. `n` - comes from the shared counter below. A display of only `-`, `_` or non-ASCII letters - ends here, because `-` alone is not an id. + comes from the shared counter below. -The accepted text has five steps instead: casefold; replace each **run of spaces** -with one `_`; strip; remove any **leading** `_`; `-` if empty. +So a generated id is a lower-case letter followed by letters, digits and single +underscores, never ending in `_`, plus, only when it collided, a `-` suffix; or +`-`. Whether a hand-written id may carry that suffix is open question 1. -Examples under the amendment: +Examples under the amendment, with "alphanumeric" read as ASCII: | display | id | |---|---| | `East` | `east` | -| `Time Period`, `Time Period`, `Time\tPeriod`, `Time\u{A0}Period`, `Time \u{200B} Period` | `time_period` | +| `Time Period`, `Time Period`, `Time\tPeriod`, `Time \u{200B} Period` | `time_period` | | `Coastal_East` | `coastal_east` | | `a_ b`, `a__b` | `a_b` | -| `Q1-2024` | `q1-2024` | -| `Q1 - 2024` | `q1_-_2024` (only `_` runs collapse) | +| `Q1-2024`, `Q1 - 2024` | `q1_2024` | | `East `, `East_`, `-East-` | `east` | | `_Draft` | `draft` | -| `Region(code)` | `regioncode` | +| `Region(code)` | `region_code` | | `Café` | `caf` | | `Straße` | `strasse` | -| `Ü`, `-`, `---` | `-` | +| `2024`, `_2024` | `n2024` | +| `2024 Budget` | `n2024_budget` | +| `Ü`, `-`, `%` | `-` | + +Under the accepted steps, the ones that differ are `Q1-2024` → `q1-2024`, +`Q1 - 2024` → `q1_-_2024`, `East ` → `east_`, `a_ b` → `a__b`, `Region(code)` → +`regioncode`, `2024` → `2024` and `-` → `-`. **Collisions.** `Time Period` and `Time_Period` slug alike, and whichever comes later -takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, which slug alike -only under the amendment's steps (under the accepted steps `East ` slugs to `east_`). `East` in `Region` and `East` in -`Direction` also slug alike, and ids are global, so the second is `east-`. +takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, and `Q1-2024` and +`Q1 2024`, which slug alike only under the amendment's steps. `East` in `Region` and +`East` in `Direction` also slug alike, and ids are global, so the second is `east-`. - **Any colliding generated id gets a suffix**, whatever kind of thing holds the slug: on import, when an id-less file loads, and when the host creates something in a session. An id that does not collide stays bare. - **The suffix is `-` plus the next value of one increment-only counter per document**, as a plain number (`east-1`, not `east-001`). The counter works like Common Lisp's `gensym`: it never goes down and is never reused, so no order holds - among suffixes (`east-7` may follow `east` directly). **[Amendment]** A suffix ends - in a digit, so a suffixed id still meets the amendment's step 5. + among suffixes (`east-7` may follow `east` directly). - **Retirement.** Within a session a removed id is never handed out again. The counter is past any suffix it issued, and a removed bare slug stays reserved in memory. @@ -238,6 +263,9 @@ every data key and reference in that file spells it. So: - **two identical hand-written ids in one file are refused at load**, even when they declare different kinds of thing. Renaming one would corrupt the data section, and a file with a duplicate cannot be resaved safely. The error names both lines. +- **[Amendment] A hand-written id obeys every syntax rule of a generated id**: lower + case, never starting with `_` or a digit, never ending in `_`, no runs of `_`, and no + character slugify would replace. One that does not is refused at load (clause 5). **Reuse across a save and reload is accepted.** Retired ids are not persisted. So remove `East`, save, reload and add `East`, and the new item gets `east` again. An @@ -252,19 +280,18 @@ promised (improvise-8tk5). **Built code.** `crates/improvise-core/src/model/id.rs` (`slugify`, `IdAllocator`) implements the accepted steps, and `context/code/core.md` §20 describes them. Once the amendment is approved, a follow-up must: -- test whitespace, not only U+0020; -- collapse `_` after the strip; -- trim a trailing `_` and both ends' `-`; -- settle what `claim` refuses (open question 11); -- add the virtual views to `VIRTUAL_IDS`; -- add `IdKind::Group` if groups are generated (open question 10). +- replace every non-alphanumeric character, not only spaces, with `_`, and collapse + `_` runs; +- trim a trailing `_`, and prepend `n` before a leading digit; +- make `claim` refuse every hand-written id a generated one could not be; +- add the virtual views to `VIRTUAL_IDS`, and `IdKind::Group`. Nothing calls `slugify` yet. ### 5. Persistence: ids on declarations *and* references -- **Declarations** (section headers, item lines, formula lines) carry the display and - the id. +- **Declarations** (section headers, item lines, formula lines, and **[Amendment]** + group definitions) carry the display and the id. - **Every reference** is the id: axis lines, `hidden:` and `collapsed:`, page selections, `## Aggregation`, `Initial View:`, `## Data` keys (ids are guaranteed unique, names might not be), and names inside formula text. @@ -278,14 +305,17 @@ grammar spells it **with ranges**, because the generator drops `ASCII_ALPHANUMER (`Sales(Copy)`), an empty tag or a tag with a trailing space does not match `id_tag`, so it has no id. The error names the line; - a hand-written id beginning with `_`, other than a reserved one; -- **[Amendment]** a hand-written id that is `-` (open question 11 asks whether a - leading or trailing `-`, or a trailing `_`, is refused too). +- **[Amendment]** a hand-written id that breaks any syntax rule of a generated id + (clause 4). Virtual ids are never written to a file, so no hand-written id begins + with `_`. The grammar's `id_token` stays permissive, and the loader refuses, so the + error can name the line. **Inside formula text an id may still need quoting.** The formula grammar's `bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So -the writer spells an id in formula text through `quote::quote_name`. An id beginning -with a digit (`2024`) or equal to a keyword (`sum`, `if`, `where`, `is`) is written -`|2024|` or `|sum|`. +the writer spells an id in formula text through `quote::quote_name`. An id equal to a +keyword (`sum`, `if`, `where`, `is`) is written `|sum|`, and under the accepted rules +an id beginning with a digit (`2024`) is written `|2024|`. **[Amendment]** No id begins +with a digit, so only keyword ids need quoting. ```pest id_token = @{ ('a'..'z' | '0'..'9' | "_" | "-")+ } @@ -306,9 +336,10 @@ id_ref = { id_token } `model_name` has no id and keeps `header_name`. `Initial View:` is a reference and takes an `id_ref`. -**Group references.** A group reference (`[Coastal]`) is a name. -**[Amendment]** It is the group's id. So is the group in a `collapsed:` line. The -group definition (`> Coastal`) declares the id. The spellings are open question 10. +**Group references.** A group reference (`[Coastal]`) is a name. **[Amendment]** It is +the group's id, as is the group in a `collapsed:` line, and the group definition +(`> Coastal`) declares a display and an id like any other declaration +(`> Coastal(coastal)`). **[Amendment] An axis line is tried before `format:`, and a format that reads as an axis line is pipe-quoted** (improvise-le4p). `view_entry` is an ordered choice, and @@ -330,20 +361,19 @@ lose its axis line to the view's number format, and the new-version arm puts that view also holds a category `format` on an axis. **[Amendment] A formula line** is -`- () = []` (pending open -question 1): +`- () = []`: - **The parenthesised id is the formula's `FormulaId`**, between the target and the - `=`. -- **`` is the target item's id when the target is an item** (outside - `_Measure`). For a `_Measure` target, which has no item, it is the target's - display, held on the `Formula`. + `=`. For a measure formula it is also the measure's id. +- **For a measure formula, `` is the measure's display**, held on the + `Formula`: `- Profit(profit) = revenue - cost`. How a dimension formula's line spells + its target, and whether the `[_measure]` tag is written, is open question 4. - **The expression is over ids**, spelled through `quote::quote_name`. A reference to a formula target is the id text of its `ItemHandle`: the item id outside `_Measure`, - the formula id for a `_Measure` target. + the formula's (and measure's) id for a `_Measure` target. - **A `WHERE`/`IS` value follows the same rule**. It is the id text of an `ItemHandle` in the filtered category, and the formula panel shows it as a display. - **A display in the target position that looks like `Name(x)` is pipe-quoted**, as in - `- |Margin(%)|(margin) = …`. An item id never contains a parenthesis. + `- |Margin(%)|(margin) = …`. An id never contains a parenthesis. **`header_display` is guarded only by a lookahead**, which the generator skips (gotcha 34). So the generator may emit a display ending in something that parses as an @@ -352,7 +382,8 @@ property 1 must be stated over the parsed result, not over the generated text. **The version line alone picks the regime** (clause 10). - **In an old-version file** a header is the whole line, a reference resolves against - display names, and ids are assigned by slugify. + display names, and ids are assigned by slugify. This resolution migrates a file + whose names were its only identity; it is not a reference by a person. - **In a new-version file** declarations carry ids and a reference is an id. **An old file naming an item it never declares** declares it at load. This happens @@ -384,7 +415,7 @@ out, because reassigning an id would silently change what its references point a **Import** (CSV, JSON) never parses ids out of dimension names. A header `Region(code)` is the display `Region(code)`, and its id is its slug. The writer pipe-quotes any display that would otherwise read as display plus id: -`## Category: |Region(code)|(regioncode)`. +`## Category: |Region(code)|(regioncode)` (**[Amendment]** `(region_code)`). ### 6. The wire: address by id, create by display, learn ids from projections @@ -393,13 +424,17 @@ display that would otherwise read as display plus id: being created is a display, and the host slugifies it: - `AddCategory(display)` and `CreateView(display)`; - `AddItem { category: CategoryId, display }`; - - `AddItemInGroup { category: CategoryId, display, group }`. Whether `group` is an - existing group's id or a new group's display is open question 10 **[Amendment]**; + - `AddItemInGroup { category: CategoryId, display, group }`. **[Amendment]** `group` + is the group's id, and a new group is created by its display under the same + create-by-display rule; - `AddFormula { target_category: CategoryId, raw }`. -- **[Amendment] Only a client accepts a display name as a reference**, in its UI - layer, and it converts the name to an id as early as it can. The wire never carries - a display as a reference. Whether the old-version loader and headless `improvise - cmd`/`script` are exceptions is open question 4. +- **[Amendment] Only a user interface accepts a display name as a reference**, and it + converts the name to an id as early as it can. The TUI and browser clients are user + interfaces, and so is the headless command line. The wire never carries a display as + a reference. +- **[Amendment] Scripts refer to things by id.** A script that names something by its + display fails. A script gets a display only through a dedicated function such as + `display_for(id)`. Which headless entry points count as scripts is open question 5. - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. A create-then-act compound becomes **one command**: `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes @@ -408,6 +443,7 @@ display that would otherwise read as display plus id: - **New operations carry ids:** - `RenameCategory { category, to: display }`; - `RenameItem { category, item, to: display }`; + - **[Amendment]** a group rename by group id and new display; - `ReplaceFormula { id, was, raw }`; - `RenameRecordCoordinate`; - the removes (`RemoveCategory`, `RemoveItem`, `RemoveFormula`, `DeleteView`). @@ -415,8 +451,7 @@ display that would otherwise read as display plus id: including `SetCell { view, … }` and `ClearCell { view, … }`), one whole variant at a time. - **Projections carry id and display together:** - - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }` (pending open - question 1); + - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }`; - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows: the target's display with its id in parentheses (`Profit (profit) = Revenue - Cost`), then the expression over display names, with an id wherever a display is @@ -425,15 +460,17 @@ display that would otherwise read as display plus id: would bind elsewhere. `stored` is the id-form text the file carries, and `ReplaceFormula`'s `was` is compared against it. So a rename of `Revenue` elsewhere changes what the panel shows, but does not refuse an edit of `Profit`; - - `ViewSummary { id, display, … }`. + - `ViewSummary { id, display, … }`; + - **[Amendment]** the grid frame's group entries (`FrameGroup`) carry the group's id + and display. `AxisFrame` labels stay display strings. They gain ids when a consumer needs them. - **`ViewState` keys by id**: the cursors, `expanded_cats` and the view history (`view_back_stack` / `view_forward_stack`). **[Amendment]** So do `active_view` itself and `Workbook::active_view` (improvise-p003). A rename then orphans nothing. - **[Amendment] A `_Measure` formula-target row** (`ItemHandle::FormulaTarget`) is - removed by `RemoveFormula` and renamed by a display rename of the formula's target - (improvise-i0n7; pending open question 1). + removed by `RemoveFormula` and renamed by a display rename of the measure + (improvise-i0n7). - **[Amendment] CheckedBatch's created-record exemption is removed** (improvise-p003). It exempts a record that neither resolves nor is claimed, for frames like `[AddItem(East), SetCell(East)]`. A frame cannot address what it creates, so nothing @@ -460,7 +497,7 @@ Per-entity versions are not added. text pairs: the same function over different strings. **[Amendment]** The pairs are `(CategoryId, ItemHandle)` spelled as id text. Ids are unique across kinds, so the hash stays canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate -is never part of a record (pending open question 1). +is never part of a record. ADR 002's decision is unchanged in substance. Its prose ("sorts coordinates by category name", records spelled in names) is out of date once this lands, and the @@ -486,7 +523,7 @@ whole-line reading. | `## View: \|Default (default)\|(default)` | pipes literal | display `Default (default)`, id `default` | | `## View: Sales (copy)` | `Sales (copy)` | **display `Sales`, id `copy`** | | `## Category: Region(code)` | `Region(code)` | **display `Region`, id `code`** | -| `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** | +| `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** (**[Amendment]** refused: an id may not start with a digit) | | `## Category: Time Period` | `Time Period` | unchanged; id `time_period` generated | The app itself writes such headers: `improvise import` on a CSV with headers like @@ -496,7 +533,8 @@ If old files got the rule, that category would load as display `Region`, id `cod Every reference in the file spells `Region(code)`, which resolves to neither, so the axis lines and data keys would not resolve, and nothing would be drawn. Because old files keep the whole-line reading, none of this happens. On the first save the writer -pipe-quotes such a display: `## Category: |Region(code)|(regioncode)`. +pipe-quotes such a display: `## Category: |Region(code)|(regioncode)` +(**[Amendment]** `(region_code)`). ### 10. The file version alone picks the regime @@ -517,10 +555,12 @@ pipe-quotes such a display: `## Category: |Region(code)|(regioncode)`. 1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224. 2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId` and - `ViewId`. **[Amendment]** Also `ItemHandle`, which the re-key needs for `CellKey` - (pending open question 1). + `ViewId`. **[Amendment]** Also `GroupId`, a display on `Group`, and `ItemHandle`, + which the re-key needs for `CellKey`. 3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with - display→id resolution moved to every input edge. + display→id resolution moved to every input edge. **[Amendment]** The virtual + categories' keys move from `_Index`, `_Dim` and `_Measure` to `_index`, `_dim` and + `_measure` here, in the model, on the wire, in `ViewState` and in the browser client. 4. **Persistence:** the new literal (clause 10), clause 5's grammar, the loader and its refusals, slugify, the writer, and the round-trip properties with a formula-line strategy. @@ -556,22 +596,22 @@ answers for those. - **A typed name that was not in the captured mapping** is resolved against the document at commit, and refused, naming it, if it does not resolve. -**[Amendment] The client does the resolving:** +**[Amendment] The client does the resolving, and ids always win:** - **The client holds the mapping**, as it holds `OpenedCell`. The TUI and the browser share one resolver in `improvise-protocol`. - **A name new to the edit** is resolved by the client against its id-bearing projection (clause 6), **ids first, then displays**. Ids are unambiguous, so they are - preferred wherever there is ambiguity. + preferred wherever there is ambiguity, now and once an autocomplete exists. - A typed name equal to an id names that id's thing, even where it is also another thing's display. - Otherwise a display that exactly one thing in scope has names that thing. - A display matching several things is refused, and the refusal lists their ids. - A name that resolves to nothing is refused, and the refusal names it. - Ids are matched exactly, with no case folding. -- **The long-term design separates two layers.** A view-layer autocomplete turns - typed display text into ids before commit, and the model layer refers to ids only. - What becomes of the ids-first order once that exists is open question 3. This ADR - does not specify the autocomplete. +- **Typing prefers ids and previews displays.** A view-layer autocomplete turns typed + text into ids before commit, preferring a match on an id, and shows the matched + thing's display as a preview. The model layer refers to ids only. This ADR does not + specify the autocomplete further. - **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are in the file's id form, and the host never resolves a display. The host parses ids, and one that no longer exists is `Rejection::Gone`. @@ -579,8 +619,8 @@ answers for those. **[Amendment] What ids-first costs.** Suppose item A is displayed `East` with id `east`, and item B is displayed `east`, so slugify gave it `east-1`. Typing `east` names **A**. B is reached by typing `east-1`. A user who typed `east` meaning B is bound to A -without being told. That is why the panel shows B as `east-1` and never as a bare -`east`. +without being told, unless the preview shows it. That is why the panel shows B as +`east-1` and never as a bare `east`. **Required regression test.** 1. Open a formula editor on `Profit = Revenue - Cost`. @@ -659,219 +699,118 @@ three things: - The formula panel is a second rendering of formula text, so display names typed there must be resolved back to ids (by the client, under the amendment). - ADR 002's prose needs a wording update. +- **[Amendment]** Scripts that name things by display break and must be rewritten + over ids. -**Not addressed.** Per-entity versions, ids in `AxisFrame` and `CellId::Derived`, and -(under the accepted text) group ids. +**Not addressed.** Per-entity versions, ids in `AxisFrame` labels and +`CellId::Derived`, and (under the accepted text) group ids. ## Open questions These are open, and nothing above depends on an answer except where it says so. -### 1. Is a formula and the measure it derives one thing with one id, or two things with two ids? +### 1. May a hand-written id carry the collision suffix? -**The question.** A formula is a derived measure. Take `Revenue` and `Cost`, stored -measures with ids `revenue` and `cost`, and add `Profit = Revenue - Cost`. Profit holds -no data: the formula computes its column. Two kinds of thing must name Profit by an -id: -- things that refer to the **measure**: another formula (`Margin = Profit / Revenue`), - each Profit cell's `CellKey`, a `hidden:` line, and ADR 008's cursor anchor; -- things that refer to the **formula**: `ReplaceFormula` and `RemoveFormula`. +**The question.** A hand-written id obeys every syntax rule of a generated id +(clause 4), and slugify turns `-` into `_`, so a slug never contains `-`. But a +generated id that collided carries a `-` suffix (`east-1`), and an empty slug +becomes `-` (`item-4`). Every save writes those ids into the file, so the +next load reads them as hand-written. -Do they use the same id? - -**Why it matters.** It decides the `CellKey` type (clause 2), the formula line's -shape (clause 5), `RecordHash` (clause 8), the `_Measure` panel rows (clause 6) and -what `.improv` must write. +**Why it matters.** If a hand-written id may never contain `-`, the app refuses to load +a file it saved itself. **Options.** -- **(a) One id.** The formula *is* the measure. Profit is `profit` everywhere, with - its display on the formula, and the line is `- Profit(profit) = revenue - cost`. - - A Profit cell's coordinate is `ItemHandle::FormulaTarget(profit)`. - - The cost: a stored `Profit` column and a formula `Profit` (merged into one measure - today by `measure_item_names`) would have two ids, and one must be chosen to key - cells. - - `RemoveFormula` removes the measure. - - The rare dimension target (an item `tax` plus formula `tax-1`) works differently. - - ADR 008 needs `ItemHandle`. - - This is the smallest change from the text above. -- **(b)** is (a) with the dimension line spelled `- Tax(tax-1)(tax) = … [customcat]` - (display, formula id, item id), so that the target position always holds a display. -- **(c) Two ids.** Profit is a `_Measure` item `profit`, and its formula is `profit-1`. - The line is `- profit(profit-1) = revenue - cost`. - - `CellKey` returns to `(CategoryId, ItemId)` and `ItemHandle` goes. - - A same-named stored measure is simply the item the formula attaches to. - - Measure and dimension targets work alike. - - The cost: the file must newly write the item and its display (today a formula - target is rebuilt from `## Formulas`). - - `RemoveFormula` leaves an item with no formula and maybe no data. - - ADR 009's `_Measure` fallback in `find_item_category` goes. +- **(a)** The syntax of an id is a slug, optionally followed by `-` and a number: + `[a-z][a-z0-9]*(_[a-z0-9]+)*` then optionally `-[0-9]+` (which also covers + `-`). A `-` anywhere else is refused. +- **(b)** Change the suffix to avoid `-`, for example `east_1`. That collides with a + display such as `East 1`, which also slugs to `east_1`, so the counter must advance + past it. -Also to settle: whether the writer keeps omitting the `[_measure]` tag on a `_Measure` -formula line, as today's writer does. +**Author's recommendation:** (a). It keeps the counter's suffix visibly distinct from +anything a display can slug to. -**Recommendation:** none. Both (a) and (c) model a derived measure, and the choice is a -product judgement about whether a formula and its column are one thing. +### 2. Does "alphanumeric" mean ASCII? -### 2. What may a formula refer to? - -**The question.** Formulas may refer to things in a typed context. An example is a -function counting how often a letter appears in a category value. Today a formula -reference (`Expr::Ref`) names an item or a formula target. Three things are open: -- (i) may a bare id in formula text also name a category, a view or a group, with - its kind given by where it appears? -- (ii) does a text function read an item's display (`Café`) or its id (`caf`)? They - differ, because slugify folds case and strips; -- (iii) must this ADR specify that, or only keep ids unique so it can be added later? - -**Why it matters.** It decides whether the formula grammar and evaluator change with -this ADR. - -**Author's recommendation:** (iii). The formula language has no string function -today, and global uniqueness already makes a reference to any kind unambiguous. - -### 3. Does ids-first matching outlive the interim resolver? - -**The question.** Clause 12's resolver matches typed text against ids before displays, -and ids are preferred wherever there is ambiguity. Once a view-layer autocomplete -turns display text into ids, does the preference carry into it? +**The question.** Slugify replaces every non-alphanumeric character with `_`. Unicode +counts `é`, `ü` and `日` as alphanumeric, but an id's alphabet is `a`–`z`, `0`–`9`, `_` +(and `-`). **Options.** -- **(a) Yes.** Wherever typed text could name both an id and a display, the id wins, - including in the autocomplete's ranking. -- **(b) No.** Once the autocomplete exists, nothing is matched at commit, so the rule - ends with the interim resolver. +- **(a) ASCII.** A non-ASCII letter becomes `_`: `Café` → `caf`, `Ü` → `-`, + `Crème Brûlée` → `cr_me_br_l_e`. +- **(b) Unicode, then drop what is outside the alphabet.** A non-ASCII letter + disappears without splitting the word: `Café` → `caf`, `Crème Brûlée` → + `crme_brle`. +- **(c) Unicode, then transliterate** (`é` → `e`). `Café` → `cafe`, `Crème Brûlée` → + `creme_brulee`. This needs a transliteration table that passes the `wasm` gate. -**Recommendation:** none. It matters only once an autocomplete is designed. +**Why it matters.** It decides every id generated from a non-English display. -### 4. May anything other than a client resolve a display name? +**Author's recommendation:** (a): it is the simplest reading of "non-alphanumeric +characters are replaced", and it needs no table. -**The question.** Only a client accepts a display as a reference (clause 6). Two -places would resolve displays outside one: -- the **old-version loader**, which resolves an old file's references by display - (clause 5), because in an old file the name is the only identity; -- **headless `improvise cmd` and `improvise script`**, which run command lines naming - things by display, with no client attached. +### 3. How does a file record the virtual categories, if virtual ids are never written? -**Why it matters.** If neither is allowed, old files cannot be migrated and the -headless CLI needs a client layer of its own. +**The question.** Virtual ids are never written to `.improv` (clause 1). But today's +files record three things under the virtual categories (`web/demo.improv` has all of +them): +- the virtual categories' axis placements, such as `_Measure: column` and + `_Index: none`; +- the stored measures, under `## Category: _Measure`; +- each data key's measure, as `_Measure=Revenue`. -**Author's recommendation:** allow both. The loader is migrating a file, not a person -referring to something. The CLI is the UI layer of its own invocation and converts at -its edge, as a client does. - -### 5. Does the `_` collapse come after the strip? - -**The question.** Clause 4 collapses runs of `_` (step 4) after stripping characters -outside the alphabet (step 3). - -**Why it matters.** `Time \u{200B} Period` (a zero-width space between two spaces) -gives `time_period` only if the collapse comes after the strip. Before it, the -zero-width character still separates the two `_`, and the result is `time__period`. - -**Options:** (a) after the strip, as written; (b) before it. -**Author's recommendation:** (a). - -### 6. Which entities share the reserved leading `_`? - -**The question.** A leading `_` is reserved for virtual categories, measures and other -entities. -- (i) Does "measures" mean virtual measures (none exists today), the `_measure` - category, or something else? -- (ii) Which "other entities" are covered? In particular, are the virtual views, - `_records` and `_drill` (clause 1), among them? - -**Why it matters.** It fixes `VIRTUAL_IDS` and what `claim` refuses. - -**Author's recommendation:** (ii) yes, as clause 1 is written. Otherwise `_Drill` -slugs to `drill` and can collide with a user's view. - -### 7. Are records mode's positions under `_Index` and `_Dim` ids? - -**The question.** Records mode gives each row a synthetic coordinate under `_Index`, -and each column one under `_Dim` (a column index, gotcha 1). They are never persisted -and never typed. +**Why it matters.** Every document with data has measures, so every file needs some +way to say which measure a cell holds and where `_Measure` sits. **Options.** -- **(a)** Outside the id namespace: a position `0` under `_index` does not collide - with a user's item `0`. -- **(b)** Inside it, spelled so that it cannot collide (for example `_index-0`), under - the reservation of a leading `_`. -- **(c)** Something else. +- **(a) Reserved keywords.** The file keeps fixed spellings for the virtual categories + (`_Measure`, `_Index`, `_Dim`, or new lower-case keywords). They are part of the + format, not ids, so no id is written. +- **(b) Write the virtual ids** (`_measure: column`, `_measure=revenue`), an exception + to "never written". +- **(c) Leave the virtual categories implicit.** Their axis placements take defaults, + the measures section has no category header, and a data key's measure is written + without its category. -**Why it matters.** Under (b), uniqueness is checked for synthetic coordinates, and -`claim`'s exact match on reserved ids must change. +**Author's recommendation:** (a). The file needs these facts, and a keyword says so +without turning a virtual id into file content. -**Recommendation:** none. +### 4. How does a dimension formula's line spell its target, and is the `[_measure]` tag written? -### 8. How do the virtual categories move to their lower-case ids? +**The question.** A measure formula's line is `- Profit(profit) = revenue - cost`: the +measure's display, then its id (clause 5). A dimension formula (the rare case) has an +item id and a separate formula id. Its line may be: +- **(a)** `- tax(tax-1) = … [customcat]`: the item id in the target position. The + target position then holds a display for a measure and an id for a dimension. +- **(b)** `- Tax(tax-1)(tax) = … [customcat]`: display, formula id, item id. The target + position always holds a display. -**The question.** Today `_Index`, `_Dim` and `_Measure` are keys in the model, on the -wire, in `ViewState` and in `.improv` files. For example, `web/demo.improv` has -`_Measure: column`, `_Index: none`, `## Category: _Measure` and `_Measure=Revenue`. -Under clause 1 they become `_index`, `_dim` and `_measure`, with the displays -unchanged. The proposed migration: -- an old file's `_Measure` resolves to `_measure` through its display at load, like - any name, and the first save writes `_measure: column` and `_measure=revenue`; -- a new-version file must spell the id, and `_Measure` there is refused (ids are - lower case); -- the header becomes `## Category: _Measure(_measure)`; -- every string literal naming them in code, on the wire and in the browser client - moves in the re-key step. +Today's writer omits the category tag for a `_Measure` formula. Should the new writer +keep omitting `[_measure]`? -**Why it matters.** Every shipped document and much test code spells these names. +**Why it matters.** It fixes the formula-line grammar and its generator strategy. -**Author's recommendation:** the migration above. +**Author's recommendation:** (b), so one position never holds two kinds of thing; +keep omitting `[_measure]`, which also keeps the virtual id out of the file +(question 3). -### 9. Does a group also carry a display name? +### 5. Which headless entry points are scripts? -**The question.** A group's identifier is an id, not a friendly name (clause 1). Does -a group also have a separate display, as categories, items and views do? +**The question.** User interfaces accept display names, and scripts refer to things by +id only (clause 6). The headless front ends are `improvise cmd ''`, which runs +one or more command lines given on the command line, and `improvise script `, +which runs a file of them. -**Why it matters.** -- If it does not, the grid and panels show the id: `Coastal East` would appear as - `coastal_east`. -- If it does, groups need a display rename like everything else, and the file - declares both. -- The code is already moving: PR #336 (improvise-cq1q.12, open) adds a private - display to `Group`, beside its key. +**Why it matters.** It decides whether `improvise cmd 'add-item Region East'`-style +invocations that name existing things by display keep working. -**Options:** (a) yes, a display beside the id; (b) no, the id is all a group has. -**Author's recommendation:** (a), for consistency with every other id-bearing kind and -so that users see what they typed. +**Options.** +- **(a)** `improvise cmd` is a user interface and resolves displays; a script file is + a script. +- **(b)** Both are scripts: only the TUI and browser clients resolve displays. -### 10. How do groups carry ids? - -**The question.** A group's identifier is an id. Not yet decided: -- (i) **Namespace.** Is a group id unique in the whole document, like every other id, - or only within its category? -- (ii) **Generation.** Is it the slug of a display (if question 9 gives it one), with - `group` as an empty slug's `` and the shared counter for collisions? -- (iii) **The file.** For example a definition `> Coastal(coastal)`, a membership - `- Coastal_East(coastal_east)[coastal]`, and a parent group (group hierarchies, - improvise-felo.6) named by id. Does a `collapsed:` line hold the group id with or - without its category id? -- (iv) **The wire.** Does `AddItemInGroup`'s `group` name an existing group by id, - create one by display, or either? Is there a `RenameGroup { group, to }`? -- (v) **The frame.** `AxisFrame`'s `FrameGroup` carries a group name today. What does - it carry? - -**Why it matters.** Clauses 2, 5, 6 and 7 cannot be implemented for groups without -these answers. - -**Author's recommendation:** (i) document-wide, as every other id is; (ii) slugify, -like every generated id. No recommendation on (iii)–(v). - -### 11. What may a hand-written id look like at its ends, and do slugify's trims repeat? - -**The question.** No id may be `-`, and no id begins or ends with `-`. -- (i) Is a hand-written id that begins or ends with `-` refused at load, as one - beginning with `_` is? -- (ii) Slugify drops a trailing `_`. Is a hand-written id ending in `_` refused too? -- (iii) Clause 4's step 5 repeats until neither end holds `_` or `-`, so `-_x` → `x`. - One pass of each would leave `_x`, which is reserved, or `x_`. Is repeating right? - -**Why it matters.** It decides what `IdAllocator::claim` refuses. Today it accepts -`-`, `-x` and `x-`. - -**Author's recommendation:** (i) yes; (ii) yes, so that a hand-written id obeys the -same shape as a generated one; (iii) repeat. +**Author's recommendation:** (a). A person types `improvise cmd` lines as they type +`:` lines in the TUI; a script file is a program that should not break when something +is renamed. diff --git a/context/code/core.md b/context/code/core.md index ffcee2e4..5969e0f3 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1715,11 +1715,12 @@ to `east_`. Only *leading* underscores are removed (step 4). **ADR 007's proposed amendment changes these rules, and this code predates it.** The amendment (PR #326) is not yet approved. Its clause 4: -- each whitespace character becomes `_`, not only U+0020; -- runs of `_` collapse after the strip, so `a_ b` and `a__b` both slug to `a_b` - and `Time \u{200B} Period` to `time_period`; -- leading and trailing `_` and `-` are removed, so `East ` → `east`, `-East-` → - `east`, and `-` alone takes `-`. +- every non-alphanumeric character becomes `_` (not only U+0020, and `-` too), + so `Q1-2024` slugs to `q1_2024` and `Region(code)` to `region_code`; +- runs of `_` collapse, so `a_ b` and `a__b` both slug to `a_b`; +- leading and trailing `_` are removed, so `East ` → `east` and `-East-` → `east`; +- `n` is prepended to a slug that starts with a digit, so `2024` → `n2024`; +- a hand-written id must obey every syntax rule of a generated one. This section, `slugify`, and the tests `slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and @@ -1750,8 +1751,8 @@ the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading `_` (only `_index`, `_dim` and `_measure` may carry one, per clause 1, and the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other id may start with `_`"), and -duplicates. It accepts `-`, `-x` and `x-`. ADR 007's open question 11 asks -whether a hand-written id like that must be refused. It never renames. +duplicates. It accepts `-`, `-x`, `x-` and `2024`, all of which the amendment +refuses (a hand-written id obeys the syntax of a generated one). It never renames. Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the loader keeps the line numbers. The virtual views `_records` and `_drill` are From 2dfe088668720b6338f5fd32a58bbdacbc549a72 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:05:01 -0700 Subject: [PATCH 14/27] docs(adr): state that a hand-written id may carry the - suffix, and keep slugs in the id alphabet (improvise-cq1q.1) Codex's review of 6e467878. The suffix question followed from two decided rules (collisions take -; a hand-written id obeys the generated syntax), so clause 4 now states the id syntax and the question is deleted. Step 2 says the slug ends in a-z, 0-9 and _ whichever reading of "alphanumeric" is chosen. The remaining questions are renumbered 1-4. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 56 ++++++++++------------------ 1 file changed, 20 insertions(+), 36 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 36e719e0..a8f54484 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -27,7 +27,7 @@ amendment is approved: | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | | 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | -Five questions are open (last section). +Four questions are open (last section). ## Context @@ -100,7 +100,7 @@ the reservation needs no check in the grammar. code, the wire, `ViewState` and the browser client spell the virtual categories `_Index`, `_Dim` and `_Measure` today, and move to the lower-case ids in the re-key step (clause 11). How the file records what it records today under `_Measure`, - `_Index` and `_Dim` is open question 3. + `_Index` and `_Dim` is open question 2. - **[Amendment] Records mode's synthetic positions are values, not ids.** The row coordinate under `_index` and the column coordinate under `_dim` (a column index, gotcha 1) are values of those virtual categories. They are never persisted or typed, @@ -198,7 +198,9 @@ thing's id never depends on how it was made. An id is lower case. pass the `wasm` gate, because `improvise-core` is in the browser bundle. 2. **Replace every character that is not alphanumeric with `_`.** That includes spaces, tabs and every other whitespace, punctuation and `-`. Whether - "alphanumeric" means ASCII only is open question 2. + "alphanumeric" means ASCII only is open question 1. Whichever it decides, the slug + ends up in `a`–`z`, `0`–`9` and `_`: a Unicode reading needs a further step that + drops or transliterates what is outside that alphabet, and question 1 picks it. 3. **Collapse each run of `_` to one `_`**, whether the underscores came from step 2 or were in the display. 4. **Remove every leading and trailing `_`.** A leading `_` is reserved for virtual @@ -210,7 +212,9 @@ thing's id never depends on how it was made. An id is lower case. So a generated id is a lower-case letter followed by letters, digits and single underscores, never ending in `_`, plus, only when it collided, a `-` suffix; or -`-`. Whether a hand-written id may carry that suffix is open question 1. +`-`. The syntax of an id is therefore a slug, +`[a-z][a-z0-9]*(_[a-z0-9]+)*`, optionally followed by `-` and a number; a `-` anywhere +else never occurs. Examples under the amendment, with "alphanumeric" read as ASCII: @@ -263,9 +267,11 @@ every data key and reference in that file spells it. So: - **two identical hand-written ids in one file are refused at load**, even when they declare different kinds of thing. Renaming one would corrupt the data section, and a file with a duplicate cannot be resaved safely. The error names both lines. -- **[Amendment] A hand-written id obeys every syntax rule of a generated id**: lower - case, never starting with `_` or a digit, never ending in `_`, no runs of `_`, and no - character slugify would replace. One that does not is refused at load (clause 5). +- **[Amendment] A hand-written id obeys every syntax rule of a generated id**: a slug + (lower case, starting with a letter, single underscores, not ending in `_`), + optionally followed by the `-` suffix. So `east-1` and `item-4`, which every save + writes, load back; `-east`, `east-`, `2024` and `_draft` do not. One that breaks the + syntax is refused at load (clause 5). **Reuse across a save and reload is accepted.** Retired ids are not persisted. So remove `East`, save, reload and add `East`, and the new item gets `east` again. An @@ -366,7 +372,7 @@ lose its axis line to the view's number format, and the new-version arm puts `=`. For a measure formula it is also the measure's id. - **For a measure formula, `` is the measure's display**, held on the `Formula`: `- Profit(profit) = revenue - cost`. How a dimension formula's line spells - its target, and whether the `[_measure]` tag is written, is open question 4. + its target, and whether the `[_measure]` tag is written, is open question 3. - **The expression is over ids**, spelled through `quote::quote_name`. A reference to a formula target is the id text of its `ItemHandle`: the item id outside `_Measure`, the formula's (and measure's) id for a `_Measure` target. @@ -434,7 +440,7 @@ display that would otherwise read as display plus id: a reference. - **[Amendment] Scripts refer to things by id.** A script that names something by its display fails. A script gets a display only through a dedicated function such as - `display_for(id)`. Which headless entry points count as scripts is open question 5. + `display_for(id)`. Which headless entry points count as scripts is open question 4. - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. A create-then-act compound becomes **one command**: `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes @@ -709,29 +715,7 @@ three things: These are open, and nothing above depends on an answer except where it says so. -### 1. May a hand-written id carry the collision suffix? - -**The question.** A hand-written id obeys every syntax rule of a generated id -(clause 4), and slugify turns `-` into `_`, so a slug never contains `-`. But a -generated id that collided carries a `-` suffix (`east-1`), and an empty slug -becomes `-` (`item-4`). Every save writes those ids into the file, so the -next load reads them as hand-written. - -**Why it matters.** If a hand-written id may never contain `-`, the app refuses to load -a file it saved itself. - -**Options.** -- **(a)** The syntax of an id is a slug, optionally followed by `-` and a number: - `[a-z][a-z0-9]*(_[a-z0-9]+)*` then optionally `-[0-9]+` (which also covers - `-`). A `-` anywhere else is refused. -- **(b)** Change the suffix to avoid `-`, for example `east_1`. That collides with a - display such as `East 1`, which also slugs to `east_1`, so the counter must advance - past it. - -**Author's recommendation:** (a). It keeps the counter's suffix visibly distinct from -anything a display can slug to. - -### 2. Does "alphanumeric" mean ASCII? +### 1. Does "alphanumeric" mean ASCII? **The question.** Slugify replaces every non-alphanumeric character with `_`. Unicode counts `é`, `ü` and `日` as alphanumeric, but an id's alphabet is `a`–`z`, `0`–`9`, `_` @@ -751,7 +735,7 @@ counts `é`, `ü` and `日` as alphanumeric, but an id's alphabet is `a`–`z`, **Author's recommendation:** (a): it is the simplest reading of "non-alphanumeric characters are replaced", and it needs no table. -### 3. How does a file record the virtual categories, if virtual ids are never written? +### 2. How does a file record the virtual categories, if virtual ids are never written? **The question.** Virtual ids are never written to `.improv` (clause 1). But today's files record three things under the virtual categories (`web/demo.improv` has all of @@ -777,7 +761,7 @@ way to say which measure a cell holds and where `_Measure` sits. **Author's recommendation:** (a). The file needs these facts, and a keyword says so without turning a virtual id into file content. -### 4. How does a dimension formula's line spell its target, and is the `[_measure]` tag written? +### 3. How does a dimension formula's line spell its target, and is the `[_measure]` tag written? **The question.** A measure formula's line is `- Profit(profit) = revenue - cost`: the measure's display, then its id (clause 5). A dimension formula (the rare case) has an @@ -794,9 +778,9 @@ keep omitting `[_measure]`? **Author's recommendation:** (b), so one position never holds two kinds of thing; keep omitting `[_measure]`, which also keeps the virtual id out of the file -(question 3). +(question 2). -### 5. Which headless entry points are scripts? +### 4. Which headless entry points are scripts? **The question.** User interfaces accept display names, and scripts refer to things by id only (clause 6). The headless front ends are `improvise cmd ''`, which runs From 8405f5c18baa0ba357bc4929774a18cd38bb2bdf Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:18:17 -0700 Subject: [PATCH 15/27] docs(adr): fold the user's answers on ASCII ids, virtual categories and scripts into ADR 007 (improvise-cq1q.1) - Id alphanumerics are ASCII and include -: slugify maps everything outside a-z, 0-9 and - to _, collapses _, trims _ and - at both ends, prefixes n before a digit; the id syntax, examples and the suffix / hyphenated-slug collision rule follow. - A file never declares a virtual category and refers to one by its id. - Only CSV import and the .improv loaders read display names in input text; command-language text, from any source, names things by id. - The formula line is shown before and after; the dimension spelling and the [_measure] tag follow from the rules, so that question is deleted. - The Opus review of 2dfe0886: each measure is keyed by one handle, and the shared-display case moves to an open question; views join the things a formula may name. Open questions are now three: a formula and a stored measure with the same display, where stored measures are declared, and whether interactive typing keeps a display stage. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 321 +++++++++++++++------------ context/code/core.md | 12 +- 2 files changed, 188 insertions(+), 145 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index a8f54484..5f09f406 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -19,15 +19,16 @@ amendment is approved: | clause | accepted rule | the amendment proposes | |---|---|---| | 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | -| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; virtual ids are never written to `.improv` | +| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; a file never declares a virtual category and refers to one by its id (`_measure`) | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | -| 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every non-alphanumeric character becomes `_`; runs of `_` collapse; leading and trailing `_` are removed; `n` is prepended to a slug that starts with a digit | +| 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended to a slug that starts with a digit | | 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` and may begin with `_` only if reserved | a hand-written id obeys every syntax rule of a generated id | +| 6 | display names typed into minibuffers are turned into ids at the input edge | only CSV import and the `.improv` loaders turn display names in input text into ids; command-language text, from any source, names things by id | | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | | 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | -Four questions are open (last section). +Three questions are open (last section). ## Context @@ -64,7 +65,8 @@ human name moves to a new `display` field. existing thing. A reference to the stable identity cannot glitch when something is renamed or replaced. - **Display names are for people.** They appear on screen, are typed into - minibuffers, and are turned into ids at the input edge. + minibuffers, and are turned into ids at the input edge. **[Amendment]** Clause 6 narrows + which readers turn display text into ids. ### 1. What carries an identity @@ -96,15 +98,21 @@ have ids `_records` and `_drill`, displayed `_Records` and `_Drill`; they are ad on the wire (`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and could collide with a user's view. Slugify never produces a leading `_` (clause 4), so the reservation needs no check in the grammar. -- **[Amendment] Virtual ids are lower case and are never written to `.improv`.** The - code, the wire, `ViewState` and the browser client spell the virtual categories - `_Index`, `_Dim` and `_Measure` today, and move to the lower-case ids in the re-key - step (clause 11). How the file records what it records today under `_Measure`, - `_Index` and `_Dim` is open question 2. +- **[Amendment] Virtual ids are lower case.** The code, the wire, `ViewState` and the + browser client spell the virtual categories `_Index`, `_Dim` and `_Measure` today, + and move to the lower-case ids in the re-key step (clause 11). +- **[Amendment] A file never declares a virtual category; it refers to one by its + id.** The virtual categories exist in every document, so they are not required in + the file. Where a line refers to one, it spells the virtual id: + `_measure: column`, `_index: none`, `region=east, _measure=revenue = 1200`. An + old-version file's `_Measure` resolves through its display at load, like every name + in an old file (clause 5), and the first save writes `_measure`. Where the file + declares the stored measures themselves (`Revenue`, `Cost`) is open question 2. - **[Amendment] Records mode's synthetic positions are values, not ids.** The row - coordinate under `_index` and the column coordinate under `_dim` (a column index, - gotcha 1) are values of those virtual categories. They are never persisted or typed, - take no part in id uniqueness, and cannot collide with a user's item. + coordinate under `_index` is a value of that virtual category. So, by the same + reasoning, is the column coordinate under `_dim` (a column index, gotcha 1). They are + never persisted or typed, take no part in id uniqueness, and cannot collide with a + user's item. **Formulas and measures.** **[Amendment]** A measure is the normal formula target (`profit = revenue - cost`). A dimension (a category other than `_Measure`) is a rare @@ -112,21 +120,23 @@ one, for a derived dimension. - **A `_Measure` target has no item of its own**, and its handle is its formula's `FormulaId`. So the category panel's `_Measure` rows use `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). -- **[Amendment] A formula is a kind of measure, so a `_Measure` formula's id is the - measure's id.** A measure has an id and a display, as everything else does. - `Profit = Revenue - Cost` is one thing, `profit`: another formula refers to it as - `profit`, each Profit cell's `CellKey` holds `profit` under `_measure`, and - `ReplaceFormula` and `RemoveFormula` address `profit`. A formula that derives a - measure already stored under that display takes the stored measure's id, so the - column has one id, not two. Removing the formula removes a measure that exists only - through it; a measure that also holds stored data stays, as a stored measure. +- **[Amendment] A formula is a kind of measure, so there is no distinction between a + `_Measure` formula's id and its measure's id.** A measure has an id and a display, + as everything else does. `Profit = Revenue - Cost` is one thing, `profit`: another + formula refers to it as `profit`, each Profit cell's `CellKey` holds `profit` under + `_measure`, and `ReplaceFormula` and `RemoveFormula` address `profit`. +- **[Amendment] Each measure's cells are keyed by one handle.** A stored measure is + keyed `ItemHandle::Item(id)`. A measure that exists only through its formula is + keyed `ItemHandle::FormulaTarget(id)`. The id is the measure's either way, and no + column is keyed by both. What happens when a formula's measure and a stored measure + share a display, and what removing that formula does, is open question 1. - **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, and its formula has a separate `FormulaId`. A formula whose generated id would collide takes a suffix like any other generated id (clause 4): a formula `Tax` in `CustomCat` gets item id `tax`, then formula id `tax-`. - **[Amendment] Formulas may refer to anything by id.** A function in a formula may - accept any id-bearing thing (a category, an item, a measure, a formula, a group), - referred to by its id. Whether a given function takes the thing itself or its value + accept any id-bearing thing (a category, an item, a measure, a formula, a view, a + group), referred to by its id. Whether a given function takes the thing itself or its value (an item's display, say) is part of that function's signature. Global uniqueness makes every such reference unambiguous, and a function may read any part of a record, so nothing further is needed here. @@ -196,27 +206,27 @@ thing's id never depends on how it was made. An id is lower case. 1. **Casefold, with Unicode full case folding**, so `Straße` → `strasse`. Rust's `str::to_lowercase` is not enough (`ß` stays `ß`). The folding table or crate must pass the `wasm` gate, because `improvise-core` is in the browser bundle. -2. **Replace every character that is not alphanumeric with `_`.** That includes - spaces, tabs and every other whitespace, punctuation and `-`. Whether - "alphanumeric" means ASCII only is open question 1. Whichever it decides, the slug - ends up in `a`–`z`, `0`–`9` and `_`: a Unicode reading needs a further step that - drops or transliterates what is outside that alphabet, and question 1 picks it. +2. **Replace every character outside `a`–`z`, `0`–`9` and `-` with `_`.** The id's + alphanumerics are ASCII and include `-`. Every other character, including spaces, + tabs and other whitespace, punctuation and every non-ASCII letter, becomes `_`. 3. **Collapse each run of `_` to one `_`**, whether the underscores came from step 2 or were in the display. -4. **Remove every leading and trailing `_`.** A leading `_` is reserved for virtual - ids (clause 1), and a trailing one is dropped. +4. **Remove every leading and trailing `_` and `-`**, repeating until neither end + holds either. A leading `_` is reserved for virtual ids (clause 1), a trailing `_` + is dropped, and no id begins or ends with `-`. 5. **If the slug starts with a digit, prepend `n`.** An id is never purely numeric and never starts with a digit, so `2024` becomes `n2024`. 6. **If nothing is left, use `-`**, such as `category-1` or `item-4`. `n` - comes from the shared counter below. + comes from the shared counter below. A display of only `-`, only `_` or only + non-ASCII letters ends here, because `-` alone is not an id. -So a generated id is a lower-case letter followed by letters, digits and single -underscores, never ending in `_`, plus, only when it collided, a `-` suffix; or -`-`. The syntax of an id is therefore a slug, -`[a-z][a-z0-9]*(_[a-z0-9]+)*`, optionally followed by `-` and a number; a `-` anywhere -else never occurs. +**The syntax of an id** is therefore: a lower-case ASCII letter, then any of `a`–`z`, +`0`–`9`, `-` and single underscores, never ending in `_` or `-`: +`[a-z]([a-z0-9-]|_(?=[a-z0-9-]))*`, not ending in `-`. Slugify's output already has +that shape, so slugifying an id gives the same id back. The collision suffix `-` +(below) keeps the shape too, because it ends in a digit. -Examples under the amendment, with "alphanumeric" read as ASCII: +Examples under the amendment: | display | id | |---|---| @@ -224,24 +234,40 @@ Examples under the amendment, with "alphanumeric" read as ASCII: | `Time Period`, `Time Period`, `Time\tPeriod`, `Time \u{200B} Period` | `time_period` | | `Coastal_East` | `coastal_east` | | `a_ b`, `a__b` | `a_b` | -| `Q1-2024`, `Q1 - 2024` | `q1_2024` | +| `Q1-2024` | `q1-2024` | +| `Q1 - 2024` | `q1_-_2024` | +| `East-1` | `east-1` | | `East `, `East_`, `-East-` | `east` | | `_Draft` | `draft` | | `Region(code)` | `region_code` | | `Café` | `caf` | +| `Crème Brûlée` | `cr_me_br_l_e` | | `Straße` | `strasse` | | `2024`, `_2024` | `n2024` | -| `2024 Budget` | `n2024_budget` | +| `2024-Q1` | `n2024-q1` | | `Ü`, `-`, `%` | `-` | -Under the accepted steps, the ones that differ are `Q1-2024` → `q1-2024`, -`Q1 - 2024` → `q1_-_2024`, `East ` → `east_`, `a_ b` → `a__b`, `Region(code)` → -`regioncode`, `2024` → `2024` and `-` → `-`. +Under the accepted steps, the ones that differ are `East ` → `east_`, `a_ b` → +`a__b`, `Time\tPeriod` → `timeperiod`, `Time \u{200B} Period` → `time__period`, +`-East-` → `-east-`, +`Region(code)` → `regioncode`, `Crème Brûlée` → `crme_brle`, `2024` → `2024`, +`2024-Q1` → `2024-q1` and `-` → `-`. **Collisions.** `Time Period` and `Time_Period` slug alike, and whichever comes later -takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, and `Q1-2024` and -`Q1 2024`, which slug alike only under the amendment's steps. `East` in `Region` and -`East` in `Direction` also slug alike, and ids are global, so the second is `east-`. +takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, which slug alike +only under the amendment's steps. `East` in `Region` and `East` in `Direction` also +slug alike, and ids are global, so the second is `east-`. + +**[Amendment] A suffix and a hyphenated slug can look alike.** The display `East-1` +slugs to `east-1`, which is also what the counter's suffix may make of `east`. They +never collide, because every id a document holds is taken, whoever made it: +- if `east-1` is already held (as a slug or as a suffix), a display `East-1` that slugs + to it is a collision, and becomes `east-1-`; +- if the counter's candidate `east-` is already held, the counter advances until a + candidate is free. + +So an id says nothing about how it was made: `east-1` may be a suffixed `east` or the +slug of `East-1`. - **Any colliding generated id gets a suffix**, whatever kind of thing holds the slug: on import, when an id-less file loads, and when the host creates something in a session. An id that does not collide stays bare. @@ -267,11 +293,9 @@ every data key and reference in that file spells it. So: - **two identical hand-written ids in one file are refused at load**, even when they declare different kinds of thing. Renaming one would corrupt the data section, and a file with a duplicate cannot be resaved safely. The error names both lines. -- **[Amendment] A hand-written id obeys every syntax rule of a generated id**: a slug - (lower case, starting with a letter, single underscores, not ending in `_`), - optionally followed by the `-` suffix. So `east-1` and `item-4`, which every save - writes, load back; `-east`, `east-`, `2024` and `_draft` do not. One that breaks the - syntax is refused at load (clause 5). +- **[Amendment] A hand-written id obeys every syntax rule of a generated id** (the + syntax above). So `east-1`, `item-4` and `q1_-_2024` load; `-east`, `east-`, `2024`, + `a__b` and `_draft` are refused at load (clause 5). **Reuse across a save and reload is accepted.** Retired ids are not persisted. So remove `East`, save, reload and add `East`, and the new item gets `east` again. An @@ -286,9 +310,9 @@ promised (improvise-8tk5). **Built code.** `crates/improvise-core/src/model/id.rs` (`slugify`, `IdAllocator`) implements the accepted steps, and `context/code/core.md` §20 describes them. Once the amendment is approved, a follow-up must: -- replace every non-alphanumeric character, not only spaces, with `_`, and collapse - `_` runs; -- trim a trailing `_`, and prepend `n` before a leading digit; +- replace every character outside `a`–`z`, `0`–`9` and `-`, not only spaces, with + `_` instead of stripping it, and collapse `_` runs; +- trim a trailing `_` and both ends' `-`, and prepend `n` before a leading digit; - make `claim` refuse every hand-written id a generated one could not be; - add the virtual views to `VIRTUAL_IDS`, and `IdKind::Group`. @@ -312,16 +336,17 @@ grammar spells it **with ranges**, because the generator drops `ASCII_ALPHANUMER `id_tag`, so it has no id. The error names the line; - a hand-written id beginning with `_`, other than a reserved one; - **[Amendment]** a hand-written id that breaks any syntax rule of a generated id - (clause 4). Virtual ids are never written to a file, so no hand-written id begins - with `_`. The grammar's `id_token` stays permissive, and the loader refuses, so the - error can name the line. + (clause 4). A file never declares a virtual category, so no declared id begins with + `_`; a reference may still spell a virtual id such as `_measure`. The grammar's + `id_token` stays permissive, and the loader refuses, so the error can name the line. **Inside formula text an id may still need quoting.** The formula grammar's `bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So the writer spells an id in formula text through `quote::quote_name`. An id equal to a keyword (`sum`, `if`, `where`, `is`) is written `|sum|`, and under the accepted rules an id beginning with a digit (`2024`) is written `|2024|`. **[Amendment]** No id begins -with a digit, so only keyword ids need quoting. +with a digit, so only keyword ids need quoting. A `-` inside an id needs none, because +the formula grammar's bare identifier already admits `-` after its first character. ```pest id_token = @{ ('a'..'z' | '0'..'9' | "_" | "-")+ } @@ -366,16 +391,39 @@ lose its axis line to the view's number format, and the new-version arm puts - a view whose number format is each axis keyword round-trips in a document where that view also holds a category `format` on an axis. -**[Amendment] A formula line** is -`- () = []`: -- **The parenthesised id is the formula's `FormulaId`**, between the target and the - `=`. For a measure formula it is also the measure's id. -- **For a measure formula, `` is the measure's display**, held on the - `Formula`: `- Profit(profit) = revenue - cost`. How a dimension formula's line spells - its target, and whether the `[_measure]` tag is written, is open question 3. +**[Amendment] A formula line** declares the formula: its target, its id, and its +expression over ids. Today's `web/demo.improv` and the format's own dimension example +are written: + +``` +## Formulas +- Profit = Revenue - Cost +- Tax = Revenue * 0.1 [CustomCat] +``` + +Under the amendment the same two formulas are written: + +``` +## Formulas +- Profit(profit) = revenue - cost +- tax(tax-1) = revenue * 0.1 [customcat] +``` + +- **A measure formula** (no category tag) is the normal case. The formula is the + measure (clause 1), so the line is the measure's only declaration: `Profit` is its + display, and `(profit)` is the id it shares with its formula. No `[_measure]` tag is + written, because a file does not spell a virtual category where it is not needed + (clause 1); a line without a tag targets `_measure`. +- **A dimension formula** (a `[]` tag) targets a real item, already + declared with its display in its category's section (`## Category: CustomCat(customcat)` + lists `- Tax(tax)`). So the line names the target by its item id, `tax`, and + `(tax-1)` is the formula's own id. The display is not repeated, because a display + lives in one place. +- **So the loader can always tell the two apart**: with a tag the target is an item + id; without one it is a measure's display. - **The expression is over ids**, spelled through `quote::quote_name`. A reference to a formula target is the id text of its `ItemHandle`: the item id outside `_Measure`, - the formula's (and measure's) id for a `_Measure` target. + the measure's (and its formula's) id for a `_Measure` target. - **A `WHERE`/`IS` value follows the same rule**. It is the id text of an `ItemHandle` in the filtered category, and the formula panel shows it as a display. - **A display in the target position that looks like `Name(x)` is pipe-quoted**, as in @@ -434,13 +482,17 @@ display that would otherwise read as display plus id: is the group's id, and a new group is created by its display under the same create-by-display rule; - `AddFormula { target_category: CategoryId, raw }`. -- **[Amendment] Only a user interface accepts a display name as a reference**, and it - converts the name to an id as early as it can. The TUI and browser clients are user - interfaces, and so is the headless command line. The wire never carries a display as - a reference. -- **[Amendment] Scripts refer to things by id.** A script that names something by its - display fails. A script gets a display only through a dedicated function such as - `display_for(id)`. Which headless entry points count as scripts is open question 4. +- **[Amendment] Only two readers turn display names in input text into ids: CSV + import and the `.improv` loaders.** They read text people wrote, where names are + all there is. The wire never carries a display as a reference. +- **[Amendment] Command-language text names things by id.** Anything that uses the + command language as a scripting language is a script, wherever the text comes from + (a script file, `improvise cmd`, another program). A script that names something by + its display fails. A script gets a display only through a dedicated function such as + `display_for(id)`. +- **[Amendment] Interactive typing prefers ids and previews displays** (clause 12). + Whether an interactive input still resolves typed text that matches no id by + display is open question 3. - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. A create-then-act compound becomes **one command**: `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes @@ -610,8 +662,9 @@ answers for those. preferred wherever there is ambiguity, now and once an autocomplete exists. - A typed name equal to an id names that id's thing, even where it is also another thing's display. - - Otherwise a display that exactly one thing in scope has names that thing. - - A display matching several things is refused, and the refusal lists their ids. + - Otherwise a display that exactly one thing in scope has names that thing, and a + display matching several things is refused with a list of their ids. Whether this + display stage stays is open question 3. - A name that resolves to nothing is refused, and the refusal names it. - Ids are matched exactly, with no case folding. - **Typing prefers ids and previews displays.** A view-layer autocomplete turns typed @@ -647,7 +700,8 @@ without resolving any name (improvise-hexb). (`east`, `east-1`), type `east` into an editor, in text not captured at open: - it must bind A, and the test must fail under displays-first; - `east-1` must bind B; -- a display that is nobody's id must still resolve by display. +- a display that is nobody's id must still resolve by display (pending open + question 3). A fixture where no display equals an id proves nothing. @@ -715,86 +769,73 @@ three things: These are open, and nothing above depends on an answer except where it says so. -### 1. Does "alphanumeric" mean ASCII? +### 1. A formula and a stored measure with the same display: one measure, and what does removal do? -**The question.** Slugify replaces every non-alphanumeric character with `_`. Unicode -counts `é`, `ü` and `日` as alphanumeric, but an id's alphabet is `a`–`z`, `0`–`9`, `_` -(and `-`). +**The question.** A document stores a `Profit` measure: an item of `_measure` with id +`profit`, display `Profit`, holding data. The user then adds the formula +`Profit = Revenue - Cost`. A formula is a kind of measure with one id (clause 1), and +displays are unique within their scope (clause 2), so this cannot simply make a second +`Profit`. Today `measure_item_names` merges the two into one column. + +**Why it matters.** It decides which handle keys the column (`Item(profit)` or +`FormulaTarget(profit)`, clause 1), what `RemoveFormula` leaves behind, and whether +documents that rely on today's merge keep loading. **Options.** -- **(a) ASCII.** A non-ASCII letter becomes `_`: `Café` → `caf`, `Ü` → `-`, - `Crème Brûlée` → `cr_me_br_l_e`. -- **(b) Unicode, then drop what is outside the alphabet.** A non-ASCII letter - disappears without splitting the word: `Café` → `caf`, `Crème Brûlée` → - `crme_brle`. -- **(c) Unicode, then transliterate** (`é` → `e`). `Café` → `cafe`, `Crème Brûlée` → - `creme_brulee`. This needs a transliteration table that passes the `wasm` gate. +- **(a) The formula attaches to the stored measure.** One measure, id `profit`, keyed + `Item(profit)`; the formula computes it. Removing the formula leaves the stored + measure and its data. +- **(b) Refuse it.** Adding a formula whose display is a stored measure's is refused, + as any duplicate display is. Existing files that hold both must be repaired at load. +- **(c) The formula replaces the stored measure.** Its stored data is dropped or kept + hidden. -**Why it matters.** It decides every id generated from a non-English display. +**Author's recommendation:** (a). It keeps today's behaviour, so existing documents +load unchanged, and a measure stays one thing with one id. -**Author's recommendation:** (a): it is the simplest reading of "non-alphanumeric -characters are replaced", and it needs no table. +### 2. Where does a file declare the stored measures? -### 2. How does a file record the virtual categories, if virtual ids are never written? +**The question.** A file never declares a virtual category (clause 1). But the stored +measures, such as `Revenue` and `Cost`, are items of `_measure`, and each needs a +declaration that carries its display and id. Today they are listed under +`## Category: _Measure`: -**The question.** Virtual ids are never written to `.improv` (clause 1). But today's -files record three things under the virtual categories (`web/demo.improv` has all of -them): -- the virtual categories' axis placements, such as `_Measure: column` and - `_Index: none`; -- the stored measures, under `## Category: _Measure`; -- each data key's measure, as `_Measure=Revenue`. +``` +## Category: _Measure +- Revenue, Cost +``` -**Why it matters.** Every document with data has measures, so every file needs some -way to say which measure a cell holds and where `_Measure` sits. +**Why it matters.** Without a declaration a measure has no display or id in the file, +and a new-version file refuses any undeclared id (clause 5). **Options.** -- **(a) Reserved keywords.** The file keeps fixed spellings for the virtual categories - (`_Measure`, `_Index`, `_Dim`, or new lower-case keywords). They are part of the - format, not ids, so no id is written. -- **(b) Write the virtual ids** (`_measure: column`, `_measure=revenue`), an exception - to "never written". -- **(c) Leave the virtual categories implicit.** Their axis placements take defaults, - the measures section has no category header, and a data key's measure is written - without its category. +- **(a) Keep a section for the virtual category's items only**, spelled by its id: + `## Category: _measure` with `- Revenue(revenue), Cost(cost)`. The section declares + items, not the category. +- **(b) A dedicated section**, such as `## Measures`, with the same item lines. +- **(c) Declare a measure where it is first used**, in a data key. A data key holds + only an id, so the display would be lost. -**Author's recommendation:** (a). The file needs these facts, and a keyword says so -without turning a virtual id into file content. +**Author's recommendation:** (b). It names what the section holds, and keeps the +virtual category out of the file entirely. -### 3. How does a dimension formula's line spell its target, and is the `[_measure]` tag written? +### 3. Does interactive typing still resolve a display name that matches no id? -**The question.** A measure formula's line is `- Profit(profit) = revenue - cost`: the -measure's display, then its id (clause 5). A dimension formula (the rare case) has an -item id and a separate formula id. Its line may be: -- **(a)** `- tax(tax-1) = … [customcat]`: the item id in the target position. The - target position then holds a display for a measure and an id for a dimension. -- **(b)** `- Tax(tax-1)(tax) = … [customcat]`: display, formula id, item id. The target - position always holds a display. +**The question.** Only CSV import and the `.improv` loaders turn display names in input +text into ids (clause 6). Interactive typing (the TUI's minibuffers and `:` line, the +formula editor, the browser) prefers ids and shows the display as a preview +(clause 12). Clause 12's resolver also has a display stage: typed text that matches no +id but is exactly one thing's display names that thing. Does that stage stay? -Today's writer omits the category tag for a `_Measure` formula. Should the new writer -keep omitting `[_measure]`? - -**Why it matters.** It fixes the formula-line grammar and its generator strategy. - -**Author's recommendation:** (b), so one position never holds two kinds of thing; -keep omitting `[_measure]`, which also keeps the virtual id out of the file -(question 2). - -### 4. Which headless entry points are scripts? - -**The question.** User interfaces accept display names, and scripts refer to things by -id only (clause 6). The headless front ends are `improvise cmd ''`, which runs -one or more command lines given on the command line, and `improvise script `, -which runs a file of them. - -**Why it matters.** It decides whether `improvise cmd 'add-item Region East'`-style -invocations that name existing things by display keep working. +**Why it matters.** Without it, a person typing `Revenue` in the formula editor must +type or pick the id `revenue`. With it, a typed display still works, and interactive +input is a third place that reads display names. **Options.** -- **(a)** `improvise cmd` is a user interface and resolves displays; a script file is - a script. -- **(b)** Both are scripts: only the TUI and browser clients resolve displays. +- **(a) Keep the display stage** for interactive typing only. Command-language text + from scripts never gets it. +- **(b) Ids only.** Interactive inputs match ids; displays appear only as previews and + autocomplete suggestions, which insert the id. -**Author's recommendation:** (a). A person types `improvise cmd` lines as they type -`:` lines in the TUI; a script file is a program that should not break when something -is renamed. +**Recommendation:** none. "Prefer ids" leaves room for (a); "only import and the +loaders read display names" points to (b). diff --git a/context/code/core.md b/context/code/core.md index 5969e0f3..533b02b5 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1715,10 +1715,12 @@ to `east_`. Only *leading* underscores are removed (step 4). **ADR 007's proposed amendment changes these rules, and this code predates it.** The amendment (PR #326) is not yet approved. Its clause 4: -- every non-alphanumeric character becomes `_` (not only U+0020, and `-` too), - so `Q1-2024` slugs to `q1_2024` and `Region(code)` to `region_code`; +- every character outside `a`–`z`, `0`–`9` and `-` becomes `_` instead of being + stripped (not only U+0020), so `Region(code)` slugs to `region_code`, while + `Q1-2024` keeps its `-`; - runs of `_` collapse, so `a_ b` and `a__b` both slug to `a_b`; -- leading and trailing `_` are removed, so `East ` → `east` and `-East-` → `east`; +- leading and trailing `_` and `-` are removed, so `East ` → `east`, `-East-` → + `east`, and `-` alone takes `-`; - `n` is prepended to a slug that starts with a digit, so `2024` → `n2024`; - a hand-written id must obey every syntax rule of a generated one. @@ -1751,8 +1753,8 @@ the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading `_` (only `_index`, `_dim` and `_measure` may carry one, per clause 1, and the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other id may start with `_`"), and -duplicates. It accepts `-`, `-x`, `x-` and `2024`, all of which the amendment -refuses (a hand-written id obeys the syntax of a generated one). It never renames. +duplicates. It accepts `-`, `-x`, `x-`, `a__b` and `2024`, all of which the +amendment refuses (a hand-written id obeys the syntax of a generated one). It never renames. Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the loader keeps the line numbers. The virtual views `_records` and `_drill` are From ccc675188662e92c930f77ca2ad95a5fa081aa88 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:20:20 -0700 Subject: [PATCH 16/27] docs(adr): say plainly that a formula line's category tag is an id and _measure is implicit (improvise-cq1q.1) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 5f09f406..1e6fb637 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -419,6 +419,8 @@ Under the amendment the same two formulas are written: lists `- Tax(tax)`). So the line names the target by its item id, `tax`, and `(tax-1)` is the formula's own id. The display is not repeated, because a display lives in one place. +- **The tag is always an id**, the target category's (`[customcat]`, never + `[CustomCat]`), and `_measure` is implicit: it is never written as a tag. - **So the loader can always tell the two apart**: with a tag the target is an item id; without one it is a measure's display. - **The expression is over ids**, spelled through `quote::quote_name`. A reference to From 209270542ccf61f3eee819276127dcea78dc4cef Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:38:06 -0700 Subject: [PATCH 17/27] docs(adr): a formula never merges into a stored measure; it takes profit- (improvise-cq1q.1) Clause 1 states it, with what removal leaves. Open question 1 is replaced by the residue: two measures then share a display, which clause 2's uniqueness rule forbids. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 52 ++++++++++++++++------------ 1 file changed, 29 insertions(+), 23 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 1e6fb637..56cad442 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -21,6 +21,7 @@ amendment is approved: | 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | | 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; a file never declares a virtual category and refers to one by its id (`_measure`) | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | +| 1 | (silent; today `measure_item_names` merges a formula into a stored measure of the same name) | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended to a slug that starts with a digit | | 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` and may begin with `_` only if reserved | a hand-written id obeys every syntax rule of a generated id | @@ -126,10 +127,16 @@ one, for a derived dimension. formula refers to it as `profit`, each Profit cell's `CellKey` holds `profit` under `_measure`, and `ReplaceFormula` and `RemoveFormula` address `profit`. - **[Amendment] Each measure's cells are keyed by one handle.** A stored measure is - keyed `ItemHandle::Item(id)`. A measure that exists only through its formula is - keyed `ItemHandle::FormulaTarget(id)`. The id is the measure's either way, and no - column is keyed by both. What happens when a formula's measure and a stored measure - share a display, and what removing that formula does, is open question 1. + keyed `ItemHandle::Item(id)`. A formula's measure is keyed + `ItemHandle::FormulaTarget(id)`. The id is the measure's either way, and no column is + keyed by both. +- **[Amendment] A formula never merges into a stored measure.** If a stored measure + `profit` (displayed `Profit`) exists and the user adds `Profit = Revenue - Cost`, + the new formula is a second measure with a fresh id, `profit-` from the shared + counter (clause 4), like any colliding generated id. The stored measure keeps + `profit` and its data. Removing the formula removes `profit-` and leaves `profit` + untouched. Both are displayed `Profit`, which clause 2's display-uniqueness rule + does not allow; open question 1 asks how that is resolved. - **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, and its formula has a separate `FormulaId`. A formula whose generated id would collide takes a suffix like any other generated id (clause 4): a formula `Tax` in @@ -176,7 +183,9 @@ human-typed name resolves to exactly one id within its scope, and a rename to a display already in use there is refused. An item's scope is its category, so two categories may each have an item displayed `Revenue`. Formula text crosses categories, so a display may be ambiguous there. The ids differ, so the formula panel -shows the ids where a display is ambiguous. +shows the ids where a display is ambiguous. **[Amendment]** A formula and a stored +measure may end up sharing a display (clause 1); whether that is an exception to this +rule is open question 1. ### 3. Ids are interned symbols @@ -771,29 +780,26 @@ three things: These are open, and nothing above depends on an answer except where it says so. -### 1. A formula and a stored measure with the same display: one measure, and what does removal do? +### 1. Two measures with the same display: is that allowed? -**The question.** A document stores a `Profit` measure: an item of `_measure` with id -`profit`, display `Profit`, holding data. The user then adds the formula -`Profit = Revenue - Cost`. A formula is a kind of measure with one id (clause 1), and -displays are unique within their scope (clause 2), so this cannot simply make a second -`Profit`. Today `measure_item_names` merges the two into one column. +**The question.** A formula never merges into a stored measure (clause 1): with a stored +`Profit` (id `profit`) in the document, adding the formula `Profit = Revenue - Cost` +creates a second measure, `profit-1`. Both are displayed `Profit`. But clause 2 keeps +displays unique within their scope, and both measures are in `_measure`'s scope. -**Why it matters.** It decides which handle keys the column (`Item(profit)` or -`FormulaTarget(profit)`, clause 1), what `RemoveFormula` leaves behind, and whether -documents that rely on today's merge keep loading. +**Why it matters.** Under the uniqueness rule, adding that formula would be refused, +which contradicts clause 1. **Options.** -- **(a) The formula attaches to the stored measure.** One measure, id `profit`, keyed - `Item(profit)`; the formula computes it. Removing the formula leaves the stored - measure and its data. -- **(b) Refuse it.** Adding a formula whose display is a stored measure's is refused, - as any duplicate display is. Existing files that hold both must be repaired at load. -- **(c) The formula replaces the stored measure.** Its stored data is dropped or kept - hidden. +- **(a) Allow duplicate displays among measures only.** The formula panel already + shows ids wherever a display is ambiguous (clause 6), and typing prefers ids + (clause 12), so the two stay distinguishable. +- **(b) Drop display uniqueness everywhere.** Ids carry identity, and every place that + shows displays already falls back to ids when they clash. +- **(c) Keep uniqueness and give the new formula a different display**, such as + `Profit 1`, chosen by the host. -**Author's recommendation:** (a). It keeps today's behaviour, so existing documents -load unchanged, and a measure stays one thing with one id. +**Author's recommendation:** (a). It is the smallest change that makes clause 1 hold. ### 2. Where does a file declare the stored measures? From 4778d9741b627f4f47a9caedc499d3f8ec100920 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:39:23 -0700 Subject: [PATCH 18/27] docs(adr): virtual categories and measures are defined by criteria, never declared (improvise-cq1q.1) Their ids and displays are hardcoded; an old file's _Measure names are recognised, not migrated as declarations. Open question 2 narrows to where a new-version file declares the stored measures. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 50 ++++++++++++++++------------ 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 56cad442..7b2fda5c 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -19,7 +19,7 @@ amendment is approved: | clause | accepted rule | the amendment proposes | |---|---|---| | 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | -| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; a file never declares a virtual category and refers to one by its id (`_measure`) | +| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; virtual categories and measures are defined by criteria with hardcoded ids and displays, are never declared in a file, and are referred to by id (`_measure`) | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 1 | (silent; today `measure_item_names` merges a formula into a stored measure of the same name) | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | @@ -102,13 +102,18 @@ the reservation needs no check in the grammar. - **[Amendment] Virtual ids are lower case.** The code, the wire, `ViewState` and the browser client spell the virtual categories `_Index`, `_Dim` and `_Measure` today, and move to the lower-case ids in the re-key step (clause 11). -- **[Amendment] A file never declares a virtual category; it refers to one by its - id.** The virtual categories exist in every document, so they are not required in - the file. Where a line refers to one, it spells the virtual id: - `_measure: column`, `_index: none`, `region=east, _measure=revenue = 1200`. An - old-version file's `_Measure` resolves through its display at load, like every name - in an old file (clause 5), and the first save writes `_measure`. Where the file - declares the stored measures themselves (`Revenue`, `Cost`) is open question 2. +- **[Amendment] Virtual categories and virtual measures are defined by criteria, not + declared.** Each contains whatever in the document meets its criterion: `_measure` + holds the measures, and `_index` and `_dim` hold the record positions. Their ids and + displays are hardcoded in the application, so a file never declares one. Where a + line refers to one, it spells the virtual id: `_measure: column`, `_index: none`, + `region=east, _measure=revenue = 1200`. +- **[Amendment] An old file's virtual names are recognised, not migrated as + declarations.** The loader recognises `_Measure`, `_Index` and `_Dim` by their + hardcoded displays, and an old `## Category: _Measure` section is read for the + stored measures it lists, not as a declaration of `_measure`. The first save writes + `_measure` wherever the file refers to it. Where a new-version file declares the + stored measures themselves (`Revenue`, `Cost`) is open question 2. - **[Amendment] Records mode's synthetic positions are values, not ids.** The row coordinate under `_index` is a value of that virtual category. So, by the same reasoning, is the column coordinate under `_dim` (a column index, gotcha 1). They are @@ -803,29 +808,30 @@ which contradicts clause 1. ### 2. Where does a file declare the stored measures? -**The question.** A file never declares a virtual category (clause 1). But the stored -measures, such as `Revenue` and `Cost`, are items of `_measure`, and each needs a -declaration that carries its display and id. Today they are listed under -`## Category: _Measure`: +**The question.** `_measure` is never declared: it holds whatever meets its criterion, +and its id and display are hardcoded (clause 1). But the stored measures in it, such as +`Revenue` and `Cost`, are ordinary things with a user's display and a generated id, so +the file must declare each one. Today they are listed under the virtual category's own +header, which a new-version file no longer writes: ``` ## Category: _Measure - Revenue, Cost ``` -**Why it matters.** Without a declaration a measure has no display or id in the file, -and a new-version file refuses any undeclared id (clause 5). +**Why it matters.** Without a declaration a stored measure has no display or id in the +file, and a new-version file refuses any undeclared id (clause 5). The declaration is +also what puts a stored measure into `_measure`'s criterion. **Options.** -- **(a) Keep a section for the virtual category's items only**, spelled by its id: - `## Category: _measure` with `- Revenue(revenue), Cost(cost)`. The section declares - items, not the category. -- **(b) A dedicated section**, such as `## Measures`, with the same item lines. -- **(c) Declare a measure where it is first used**, in a data key. A data key holds - only an id, so the display would be lost. +- **(a) A dedicated section**, such as `## Measures`, with item lines + (`- Revenue(revenue), Cost(cost)`). Being declared there is what makes something a + stored measure. +- **(b) A marker on an ordinary item line**, wherever the measure is listed, for + example `- Revenue(revenue) [measure]`. -**Author's recommendation:** (b). It names what the section holds, and keeps the -virtual category out of the file entirely. +**Author's recommendation:** (a). It names what the section holds, and keeps the +virtual category's header out of the file. ### 3. Does interactive typing still resolve a display name that matches no id? From c7f8d0a3cde39b75d7a4a32cdba07bf45729aa16 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:40:36 -0700 Subject: [PATCH 19/27] docs(adr): delete ADR 007's remaining open questions (improvise-cq1q.1) The duplicate measure display follows from the profit- rule and is stated as its consequence; where stored measures are listed is the format's existing concern; interactive typing is UI behaviour, not identity. The Open questions section now says None. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 97 ++++------------------------ 1 file changed, 13 insertions(+), 84 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 7b2fda5c..8ca77b63 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -29,7 +29,7 @@ amendment is approved: | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | | 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | -Three questions are open (last section). +No questions are open. ## Context @@ -112,8 +112,8 @@ the reservation needs no check in the grammar. declarations.** The loader recognises `_Measure`, `_Index` and `_Dim` by their hardcoded displays, and an old `## Category: _Measure` section is read for the stored measures it lists, not as a declaration of `_measure`. The first save writes - `_measure` wherever the file refers to it. Where a new-version file declares the - stored measures themselves (`Revenue`, `Cost`) is open question 2. + `_measure` wherever the file refers to it. The stored measures themselves + (`Revenue`, `Cost`) are listed as the format lists them today, each with its id. - **[Amendment] Records mode's synthetic positions are values, not ids.** The row coordinate under `_index` is a value of that virtual category. So, by the same reasoning, is the column coordinate under `_dim` (a column index, gotcha 1). They are @@ -140,8 +140,10 @@ one, for a derived dimension. the new formula is a second measure with a fresh id, `profit-` from the shared counter (clause 4), like any colliding generated id. The stored measure keeps `profit` and its data. Removing the formula removes `profit-` and leaves `profit` - untouched. Both are displayed `Profit`, which clause 2's display-uniqueness rule - does not allow; open question 1 asks how that is resolved. + untouched. The formula keeps the display the user typed, so both are displayed + `Profit` and are told apart by id: the formula panel shows the ids where the display + is ambiguous (clause 6), and a typed `Profit` that matches both is refused with both + ids listed (clause 12). - **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, and its formula has a separate `FormulaId`. A formula whose generated id would collide takes a suffix like any other generated id (clause 4): a formula `Tax` in @@ -188,9 +190,9 @@ human-typed name resolves to exactly one id within its scope, and a rename to a display already in use there is refused. An item's scope is its category, so two categories may each have an item displayed `Revenue`. Formula text crosses categories, so a display may be ambiguous there. The ids differ, so the formula panel -shows the ids where a display is ambiguous. **[Amendment]** A formula and a stored -measure may end up sharing a display (clause 1); whether that is an exception to this -rule is open question 1. +shows the ids where a display is ambiguous. **[Amendment]** The one exception is a formula +whose measure shares a stored measure's display (clause 1): it is created anyway, and +the two are told apart by id. ### 3. Ids are interned symbols @@ -507,8 +509,6 @@ display that would otherwise read as display plus id: its display fails. A script gets a display only through a dedicated function such as `display_for(id)`. - **[Amendment] Interactive typing prefers ids and previews displays** (clause 12). - Whether an interactive input still resolves typed text that matches no id by - display is open question 3. - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. A create-then-act compound becomes **one command**: `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes @@ -679,8 +679,7 @@ answers for those. - A typed name equal to an id names that id's thing, even where it is also another thing's display. - Otherwise a display that exactly one thing in scope has names that thing, and a - display matching several things is refused with a list of their ids. Whether this - display stage stays is open question 3. + display matching several things is refused with a list of their ids. - A name that resolves to nothing is refused, and the refusal names it. - Ids are matched exactly, with no case folding. - **Typing prefers ids and previews displays.** A view-layer autocomplete turns typed @@ -716,8 +715,7 @@ without resolving any name (improvise-hexb). (`east`, `east-1`), type `east` into an editor, in text not captured at open: - it must bind A, and the test must fail under displays-first; - `east-1` must bind B; -- a display that is nobody's id must still resolve by display (pending open - question 3). +- a display that is nobody's id must still resolve by display. A fixture where no display equals an id proves nothing. @@ -783,73 +781,4 @@ three things: ## Open questions -These are open, and nothing above depends on an answer except where it says so. - -### 1. Two measures with the same display: is that allowed? - -**The question.** A formula never merges into a stored measure (clause 1): with a stored -`Profit` (id `profit`) in the document, adding the formula `Profit = Revenue - Cost` -creates a second measure, `profit-1`. Both are displayed `Profit`. But clause 2 keeps -displays unique within their scope, and both measures are in `_measure`'s scope. - -**Why it matters.** Under the uniqueness rule, adding that formula would be refused, -which contradicts clause 1. - -**Options.** -- **(a) Allow duplicate displays among measures only.** The formula panel already - shows ids wherever a display is ambiguous (clause 6), and typing prefers ids - (clause 12), so the two stay distinguishable. -- **(b) Drop display uniqueness everywhere.** Ids carry identity, and every place that - shows displays already falls back to ids when they clash. -- **(c) Keep uniqueness and give the new formula a different display**, such as - `Profit 1`, chosen by the host. - -**Author's recommendation:** (a). It is the smallest change that makes clause 1 hold. - -### 2. Where does a file declare the stored measures? - -**The question.** `_measure` is never declared: it holds whatever meets its criterion, -and its id and display are hardcoded (clause 1). But the stored measures in it, such as -`Revenue` and `Cost`, are ordinary things with a user's display and a generated id, so -the file must declare each one. Today they are listed under the virtual category's own -header, which a new-version file no longer writes: - -``` -## Category: _Measure -- Revenue, Cost -``` - -**Why it matters.** Without a declaration a stored measure has no display or id in the -file, and a new-version file refuses any undeclared id (clause 5). The declaration is -also what puts a stored measure into `_measure`'s criterion. - -**Options.** -- **(a) A dedicated section**, such as `## Measures`, with item lines - (`- Revenue(revenue), Cost(cost)`). Being declared there is what makes something a - stored measure. -- **(b) A marker on an ordinary item line**, wherever the measure is listed, for - example `- Revenue(revenue) [measure]`. - -**Author's recommendation:** (a). It names what the section holds, and keeps the -virtual category's header out of the file. - -### 3. Does interactive typing still resolve a display name that matches no id? - -**The question.** Only CSV import and the `.improv` loaders turn display names in input -text into ids (clause 6). Interactive typing (the TUI's minibuffers and `:` line, the -formula editor, the browser) prefers ids and shows the display as a preview -(clause 12). Clause 12's resolver also has a display stage: typed text that matches no -id but is exactly one thing's display names that thing. Does that stage stay? - -**Why it matters.** Without it, a person typing `Revenue` in the formula editor must -type or pick the id `revenue`. With it, a typed display still works, and interactive -input is a third place that reads display names. - -**Options.** -- **(a) Keep the display stage** for interactive typing only. Command-language text - from scripts never gets it. -- **(b) Ids only.** Interactive inputs match ids; displays appear only as previews and - autocomplete suggestions, which insert the id. - -**Recommendation:** none. "Prefer ids" leaves room for (a); "only import and the -loaders read display names" points to (b). +None. From 98db007e716bee6127cb7eeb90ea7186c58d5aec Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:41:00 -0700 Subject: [PATCH 20/27] docs(adr): interactive input matches a display or an id when the pick is unambiguous (improvise-cq1q.1) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 8ca77b63..7f86354c 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -508,7 +508,9 @@ display that would otherwise read as display plus id: (a script file, `improvise cmd`, another program). A script that names something by its display fails. A script gets a display only through a dedicated function such as `display_for(id)`. -- **[Amendment] Interactive typing prefers ids and previews displays** (clause 12). +- **[Amendment] Interactive input matches either a display or an id**, in the TUI + and the browser, as long as it is unambiguous which thing the user is picking. Ids + win when it is not, and the display is shown as a preview (clause 12). - **A frame cannot address a thing it creates**, because it does not know the id the host will assign. A create-then-act compound becomes **one command**: `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes @@ -670,7 +672,8 @@ answers for those. - **A typed name that was not in the captured mapping** is resolved against the document at commit, and refused, naming it, if it does not resolve. -**[Amendment] The client does the resolving, and ids always win:** +**[Amendment] The client does the resolving. Interactive input may match a display or +an id as long as the pick is unambiguous, and ids win when it is not:** - **The client holds the mapping**, as it holds `OpenedCell`. The TUI and the browser share one resolver in `improvise-protocol`. - **A name new to the edit** is resolved by the client against its id-bearing From 38058ec73f1198b6c3b8d14c0784dd70e6ff759c Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:43:16 -0700 Subject: [PATCH 21/27] docs(adr): move ADR 007's amendment to its own file, stating only what it changes (improvise-cq1q.1) On the user's instruction an ADR states only what its decision changes, and mentions an existing convention only where it conflicts. The accepted ADR 007 is restored to main's text plus a one-line pointer, so merging this PR keeps it; the amendment is context/adr/007-stable-identities- amendment-1.md (280 lines, from 784), with a table of the accepted rules it replaces. The doc site lists the new file, and core.md section 20 points at it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- .../adr/007-stable-identities-amendment-1.md | 280 ++++ context/adr/007-stable-identities.md | 1400 +++++++++-------- context/code/core.md | 5 +- scripts/build-doc-site.mjs | 1 + 4 files changed, 1050 insertions(+), 636 deletions(-) create mode 100644 context/adr/007-stable-identities-amendment-1.md diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md new file mode 100644 index 00000000..c4bf95a1 --- /dev/null +++ b/context/adr/007-stable-identities-amendment-1.md @@ -0,0 +1,280 @@ +# ADR 007, amendment 1 — Stable identities + +**Ticket:** improvise-sboz (this amendment improvise-cq1q.1; implementation epic +improvise-cq1q) · **Builds on:** ADR 001, ADR 002 + +## Status + +**ADR 007 is accepted** (PR #210, 2026-09-25), and `007-stable-identities.md` stands. **This +amendment is proposed and not yet approved** (PR #326). This file states only what the +amendment changes or adds, numbered by the accepted ADR's clauses; everything else in +the accepted ADR is still true. Where the amendment conflicts with an accepted rule or an existing +behaviour, the accepted rule binds until the amendment is approved: + +| clause | accepted rule or existing behaviour | the amendment | +|---|---|---| +| 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | +| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill` | +| 1 | the virtual categories are declared and named in the file (`## Category: _Measure`, `_Measure: column`) | they are defined by criteria with hardcoded ids and displays, never declared, and referred to by id (`_measure`) | +| 1 | `measure_item_names` merges a formula into a stored measure of the same name | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | +| 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | +| 2, 5 | a collapsed entry and a group reference name a group by name | by its id | +| 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended before a leading digit | +| 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` | a hand-written id obeys every syntax rule of a generated id | +| 6 | display names typed into minibuffers are turned into ids at the input edge | only CSV import and the `.improv` loaders read display names in input text; command-language text names things by id | +| 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | +| 12 | the regression test commits the unedited text | the test edits a non-name token and commits | + +## Context + +Implementing the accepted ADR (epic improvise-cq1q) and using it raised points the +accepted text does not settle: whether groups and measures carry ids, what a virtual +category is in a file, exactly what slugify produces, which inputs may read display +names, and which side resolves names in a formula edit. This amendment settles them. + +## Decision + +### 1. What carries an identity + +**Groups have ids.** A group has an id and a display, like a category or an item, and +its id is an ordinary id: unique in the document, generated by slugify (`group` is an +empty slug's ``), declared beside its display, referred to by id, created by +display, and renamed by a display rename. Its wire address is `GroupId`. In code, +PR #336 (improvise-cq1q.12) already adds a display to `Group` beside its key; that key +becomes the group's id. + +**Virtual ids.** +- **A leading `_` is reserved for virtual ids in general.** The virtual views have ids + `_records` and `_drill`, displayed `_Records` and `_Drill`: they are addressed on the + wire (`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and could + collide with a user's view. Slugify never produces a leading `_`, so the grammar does + not check the reservation. +- **Virtual ids are lower case.** The code, the wire, `ViewState` and the browser + client spell the virtual categories `_Index`, `_Dim` and `_Measure` today, and move + to `_index`, `_dim` and `_measure` in the re-key step. +- **Virtual categories and measures are defined by criteria, not declared.** Each + contains whatever in the document meets its criterion: `_measure` holds the measures, + and `_index` and `_dim` hold the record positions. Their ids and displays are + hardcoded in the application, so a file never declares one, and a line that refers to + one spells its id: `_measure: column`, `region=east, _measure=revenue = 1200`. +- **Conflict with old files:** they declare and name `_Measure`. The loader recognises + `_Measure`, `_Index` and `_Dim` by their hardcoded displays, reads an old + `## Category: _Measure` section only for the stored measures it lists, and the first + save writes `_measure`. +- **Records mode's synthetic positions are values, not ids.** The row coordinate under + `_index`, and by the same reasoning the column coordinate under `_dim`, are values of + those virtual categories. They take no part in id uniqueness and cannot collide with + a user's item. + +**Formulas and measures.** A measure is the normal formula target (`profit = revenue - +cost`); a dimension target is rare, for a derived dimension. +- **A formula is a kind of measure, so a `_Measure` formula's id is its measure's id.** + `Profit = Revenue - Cost` is one thing, `profit`: other formulas, each Profit cell's + `CellKey`, and `ReplaceFormula`/`RemoveFormula` all name `profit`. +- **Each measure's cells are keyed by one handle:** `ItemHandle::Item(id)` for a stored + measure, `ItemHandle::FormulaTarget(id)` for a formula's measure, never both. +- **A formula never merges into a stored measure.** With a stored `profit` (displayed + `Profit`), adding `Profit = Revenue - Cost` creates a second measure with a fresh id, + `profit-`, by the ordinary collision rule. The stored measure keeps `profit` and + its data; removing the formula removes only `profit-`. + - **Conflict with display uniqueness:** both are displayed `Profit` in `_measure`'s + scope. This is the one permitted case; the two are told apart by id (the formula + panel shows ids where a display is ambiguous, and a typed `Profit` matching both is + refused with both ids listed, clause 12). +- **A target outside `_Measure` is a real item** with its own `ItemId`, and its formula + has a separate `FormulaId`: a formula `Tax` in `CustomCat` gets item id `tax`, then + formula id `tax-`. +- **Formulas may refer to anything by id.** A function in a formula may accept any + id-bearing thing (category, item, measure, formula, view, group) by its id. Whether a + function takes the thing or its value (an item's display, say) is part of that + function's signature. + +### 2. Storage keys + +- **`CellKey` pairs are `(CategoryId, ItemHandle)`**, so a `_Measure` formula cell's + coordinate is `FormulaTarget` without claiming an `ItemId` the document does not + hold. +- **A collapsed entry names its group by the group's id**, and `Group` gains a + `display`. + +### 4. How an id is made + +**Slugify's steps:** +1. **Casefold**, with Unicode full case folding (`Straße` → `strasse`). +2. **Replace every character outside `a`–`z`, `0`–`9` and `-` with `_`.** Ids' + alphanumerics are ASCII and include `-`; whitespace, punctuation and every non-ASCII + letter become `_`. +3. **Collapse each run of `_` to one `_`**, whether the underscores came from step 2 or + were in the display. +4. **Remove every leading and trailing `_` and `-`**, repeating until neither end holds + either. +5. **If the slug starts with a digit, prepend `n`**, so no id is purely numeric or + starts with a digit: `2024` → `n2024`. +6. **If nothing is left, use `-`** from the shared counter. A display of only + `-`, only `_` or only non-ASCII letters ends here. + +**The syntax of an id:** a lower-case ASCII letter, then `a`–`z`, `0`–`9`, `-` and +single underscores, never ending in `_` or `-`. Slugify's output has this shape, so +slugifying an id returns it unchanged, and a collision suffix `-` keeps the shape. + +| display | id | +|---|---| +| `Time Period`, `Time\tPeriod`, `Time \u{200B} Period` | `time_period` | +| `a_ b`, `a__b` | `a_b` | +| `Q1 - 2024` | `q1_-_2024` | +| `East `, `East_`, `-East-` | `east` | +| `Region(code)` | `region_code` | +| `Crème Brûlée` | `cr_me_br_l_e` | +| `2024`, `_2024` | `n2024` | +| `2024-Q1` | `n2024-q1` | +| `Ü`, `-`, `%` | `-` | + +**A suffix and a hyphenated slug can look alike.** `East-1` slugs to `east-1`, which is +also what the counter may make of `east`. They never collide, because every id a +document holds is taken however it was made: a display slugging to a held `east-1` +becomes `east-1-`, and a held counter candidate is skipped. + +**Hand-written ids obey every syntax rule of a generated id.** `east-1`, `item-4` and +`q1_-_2024` load; `-east`, `east-`, `2024`, `a__b` and `_draft` are refused at load. + +**Two branches of one file can merge into a file that will not load**, if each adds a +different thing that generates the same slug: the merged file declares that id twice, +and the load is refused naming both. Stable identity across copies of a file is not +promised (improvise-8tk5). + +**Built code.** `crates/improvise-core/src/model/id.rs` implements the accepted steps +(`context/code/core.md` §20). After approval it must replace rather than strip, collapse +`_`, trim a trailing `_` and both ends' `-`, prepend `n`, make `claim` refuse what a +generated id could not be, and add the virtual views and `IdKind::Group`. + +### 5. Persistence + +- **The loader also refuses a hand-written id that breaks the syntax above**, naming + the line; the grammar's `id_token` stays permissive. A declared id never begins with + `_`, since a file never declares a virtual category. +- **Formula text quotes only keyword ids** (`|sum|`). No id starts with a digit, and + the formula grammar's bare identifier already admits `-`. +- **Groups:** a group reference (`[coastal]`) and the group in a `collapsed:` line are + ids, and a group definition declares a display and an id (`> Coastal(coastal)`). +- **A header such as `## Category: Budget (2024)`** in a new-version file gives the id + `2024`, which is refused. Import pipe-quotes `Region(code)` as + `## Category: |Region(code)|(region_code)`. + +**An axis line is tried before `format:`, and a format that reads as an axis line is +pipe-quoted** (improvise-le4p). `view_entry` is an ordered choice and `format_line` +takes the rest of the line, so a category with id `format` would lose its axis line to +the number format; the new-version arm puts `axis_line` first. Then `:set-format none` +would write `format: none`, read as an axis line, so the writer pipe-quotes a format +that would also match `axis_line` (an axis keyword alone or followed by `, `, or a +value starting with `|`), and the new arm's `format_line` is +`pipe_quoted ~ &NEWLINE | rest_of_line`. An old file's `format: none` is read as before +and written quoted on first save. Required tests: a category displayed `Format` on a +row axis round-trips; a view whose format is each axis keyword round-trips beside a +category `format` on an axis. + +**A formula line.** Today: + +``` +## Formulas +- Profit = Revenue - Cost +- Tax = Revenue * 0.1 [CustomCat] +``` + +Under the amendment: + +``` +## Formulas +- Profit(profit) = revenue - cost +- tax(tax-1) = revenue * 0.1 [customcat] +``` + +- **A measure formula** has no tag: `_measure` is implicit and never written as a tag. + The line is the measure's only declaration: `Profit` is its display and `(profit)` + the id it shares with its formula. +- **A dimension formula** has a tag, which is always the target category's id. The + target is already declared with its display in its category, so the line names it by + item id (`tax`), and `(tax-1)` is the formula's own id. +- So the loader tells them apart by the tag. The expression is over ids; a reference to + a formula target, and a `WHERE`/`IS` value, is the id text of its `ItemHandle`. A + target display that looks like `Name(x)` is pipe-quoted: `- |Margin(%)|(margin) = …`. + +### 6. Display names and the wire + +- **Only CSV import and the `.improv` loaders turn display names in input text into + ids.** They read text people wrote. An old file's names are resolved there as a + migration. +- **Command-language text names things by id.** Anything that uses the command language + as a scripting language is a script, wherever the text comes from (a file, + `improvise cmd`, another program). A script naming something by display fails; it + gets a display only through a dedicated function such as `display_for(id)`. +- **Interactive input** (TUI, browser) matches either a display or an id, as long as it + is unambiguous which thing the user is picking. Ids win when it is not, and the + display is shown as a preview (clause 12). +- **Groups on the wire:** `AddItemInGroup`'s `group` is a group id, a new group is + created by display, a group is renamed by id and new display, and `FrameGroup` + carries the group's id and display. +- **The formula panel** treats a display equal to another thing's id as ambiguous, so it + never shows a bare name that ids-first would bind elsewhere. +- **`ViewState::active_view` and `Workbook::active_view`** are keyed by id + (improvise-p003). +- **A `_Measure` formula-target row** is removed by `RemoveFormula` and renamed by a + display rename of the measure (improvise-i0n7). +- **CheckedBatch's created-record exemption is removed** (improvise-p003). It exists for + frames like `[AddItem(East), SetCell(East)]`, and a frame cannot address what it + creates, so nothing produces one. + +### 7. Rejections + +`Rejection::Gone(Named)` gains `Named::Group`. + +### 8. `RecordHash` + +`RecordHash` hashes `(CategoryId, ItemHandle)` pairs spelled as id text. Ids are unique +across kinds, so it stays canonical; a record excludes `_measure`, so a `FormulaTarget` +is never part of one. + +### 11. Migration + +- Step 2 adds `GroupId`, a display on `Group`, and `ItemHandle`. +- Step 3 moves the virtual categories' keys to `_index`, `_dim` and `_measure`. +- A new step 7 builds clause 12's client-side capture and resolver, ids first from the + start. + +### 12. Who resolves names in a formula edit + +**The client resolves; the host never sees a display.** +- The client holds the display→id mapping captured when the editor opened, as it holds + `OpenedCell`. The TUI and the browser share one resolver in `improvise-protocol`. +- **A name new to the edit** is resolved by the client against its id-bearing + projection: a typed name equal to an id names that thing, even if it is also another + thing's display; otherwise a display exactly one thing has names it; a display + matching several things is refused with their ids; a name matching nothing is + refused. Ids are matched exactly. Ids win wherever there is ambiguity, now and once an + autocomplete exists; the autocomplete prefers id matches and previews the display. +- **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are + in id form; an id that no longer exists is `Rejection::Gone`. + +**What ids-first costs.** Item A is displayed `East` (id `east`) and item B is displayed +`east`, so it got `east-1`. Typing `east` names A; B is reached as `east-1`. The panel +therefore shows B as `east-1`, never as a bare `east`. + +**Tests.** +- The accepted regression test's step 3 becomes: edit something that is not a name + (`Revenue - Cost * 1`) and commit. Committing unedited text cannot tell a correct + capture from none, because a client may pass unchanged text as a no-op + (improvise-hexb). +- Ids-first: with A (`East`, `east`) and B (`east`, `east-1`), typing `east` in text not + captured at open must bind A (and fail under displays-first), `east-1` must bind B, + and a display that is nobody's id must still resolve by display. A fixture where no + display equals an id proves nothing. + +## Consequences + +- Scripts that name things by display break and must be rewritten over ids. +- Two measures can share a display; every place that shows displays falls back to ids. +- Old files' `_Measure` declarations and hyphen- or digit-led names change on first save. +- `id.rs` and its tests need the follow-up in clause 4. + +## Open questions + +None. diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index 7f86354c..e044be5c 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -1,474 +1,589 @@ # ADR 007 — Stable identities for categories, items, formulas and views -**Ticket:** improvise-sboz (implementation epic improvise-cq1q; this amendment -improvise-cq1q.1) · **Unblocks:** improvise-fz77 · **Revises:** improvise-km71's -interim `FormulaId` (PR #214) and improvise-fdk1's `RenameRecordCoordinate` -(PR #221) · **Builds on:** ADR 001, ADR 002 · Review tickets filed against an early -draft call this document "ADR 005". +**Amended:** an amendment is proposed in `007-stable-identities-amendment-1.md` +(PR #326, not yet approved). Until it is approved, this text binds. -## Status +**Status:** accepted (2026-09-25). The user approved revision 7. Revision 8 then +followed the adversarial review of that head: it adds clause 12, the ADR's own reading +of when a formula edit's names become ids, and corrects stale facts about main. The +user approved revision 8 on PR #210 (owner comment 5824468104, 2026-09-25 00:17:10Z: +"looks good, approved"). · +**Decided by:** the user; see "What the user decided" · +**Ticket:** improvise-sboz · **Unblocks:** improvise-fz77 · +**Revises:** improvise-km71's interim `FormulaId` (landed in PR #214) and +improvise-fdk1's `RenameRecordCoordinate` (landed in PR #221) · +**Builds on:** ADR 001, ADR 002 · **Depended on:** improvise-tt22 (remove legacy JSON), +which landed in PR #224 -**Accepted** on 2026-09-25 (PR #210). **An amendment is proposed and not yet -approved** (PR #326). **Every rule tagged [Amendment] is proposed; everything -without the tag is accepted and binding.** The tag is the complete list of what the -amendment adds. +**Numbering.** The ADR was first filed as 005, but improvise-l2b8 already claims ADR 005 +(exact decimal arithmetic) and improvise-ilvl claims 006. The review tickets filed +against the first draft still say "ADR 005" in their titles; they mean this document. -Where the amendment replaces accepted text, the accepted rule binds until the -amendment is approved: +**Direction from** the user, 2026-09-23, in three messages: -| clause | accepted rule | the amendment proposes | -|---|---|---| -| 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | -| 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill`; virtual categories and measures are defined by criteria with hardcoded ids and displays, are never declared in a file, and are referred to by id (`_measure`) | -| 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | -| 1 | (silent; today `measure_item_names` merges a formula into a stored measure of the same name) | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | -| 2, 5 | a collapsed entry and a group reference name a group by name | by its id | -| 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended to a slug that starts with a digit | -| 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` and may begin with `_` only if reserved | a hand-written id obeys every syntax rule of a generated id | -| 6 | display names typed into minibuffers are turned into ids at the input edge | only CSV import and the `.improv` loaders turn display names in input text into ids; command-language text, from any source, names things by id | -| 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | -| 12 | the regression test commits the **unedited** text after the concurrent renames | the test edits a non-name token and commits, because a client may pass unedited text as a no-op without resolving any name | +> "there would be a lot of benefit to categories/formulas having stable identities, even if +> they were only transient: it would help pin editing operations and distinguish them from +> delete/remove" -No questions are open. +> "It's probably worth a file format change (in a opt-in way, that gets rectified on save if +> the input file was missing them)" + +> "Maybe something like `View: Default(default)` and the standard +> `View: |Default (default)|(default)` for quoting." + +## What the user decided + +The user answered the **first draft** (`context/adr/005-stable-identities.md` at +`9d97664c`) with inline comments on PR #210, and then posted one comment on the thread. +Each answer is quoted verbatim below, with where it was attached and what it overturned. +**Five of the first draft's positions were reversed.** The rest of this document is +rewritten to match. + +**1. Storage keys by id; the human name becomes a display field.** On first-draft +line 92, the clause "`CellKey`, `DataStore`, views and formula text **stay keyed by +name**": + +> "Shouldn't it be easy to replace this to key by ID? There's no type change so, at worst, +> it would be renaming field access. We could also add the name as something like +> `display` and reserve `name` or whatever it is currently for the id." + +**2. The point of ids is that keys use them.** On first-draft line 97, "ids do not make a +rename cheap. They make it *expressible*": + +> "This is a lot lower value if CellKeys, etc. don't refer to the new id" + +**3. Every reference uses the id.** On first-draft line 275, the heading of clause 8, +"ADR 002: `RecordHash` stays defined over names": + +> "No, anything referring to something with an id should refer to the id as the stable +> identity to avoid glitches." + +This answers Q7 by reversing it, and reverses clause 5's "references carry names" too. + +**4. Q1, the id's internal type.** On first-draft line 469, the recommendation to wrap "a +small string": + +> "We probably can reuse symbols as the internal type with interning?" + +Ids are strings, interned through the existing `SymbolTable`. + +**5. Q2, formula id placement.** On first-draft line 473, recommendation F1 +(`- Profit(3) = Revenue - Cost`): + +> "Yes, before the equal to the right of the name" + +**6. Q3, header syntax.** On first-draft line 480, the claim that "a header with a space +before its parenthesis (`Sales (copy)`) is unaffected": + +> "Space before the open paren is optional: document this edge case that might break +> existing docs." + +**7. Q4, data keys.** On first-draft line 485, the recommendation "no — names only": + +> "In Data, we should prefer ids that are guaranteed unique to names that might not be." + +**8. Import generates ids; legacy JSON goes.** Thread comment, 2026-09-23 06:51: + +> "For CSV or JSON import, the dimensions are to be treated as the human friendly name and +> never parsed for IDs: during import, ids should be generated by a slugify algorithm that +> strips non-alphanumeric characters and replaces spaces with hyphens(collapsing runs). +> The only supported native file format is .improv, cleanup any legacy JSON format." + +Legacy JSON removal is its own implementation ticket, **improvise-tt22**. This ADR +depended on it and did not implement it; it landed in PR #224. It also dissolves +improvise-65kf: with no legacy JSON to load, there are no numeric JSON ids to accept. + +**The answers change the ADR's centre of gravity.** The first draft put an id *beside* +the name as an index. The user's answers make the **id the key** and the name a +**display** string. So a rename becomes cheap: it changes one display field and moves no +key, no surrogate and no basis tier. The first draft had called that impossible. + +### Second round: answers on revision 2 + +The user answered revision 2 (`context/adr/007-stable-identities.md` at `5e46ce03`) +with thirteen inline comments on PR #210, posted 2026-09-23 15:03–15:11Z. Each answer +is mapped to the question **on the line it is attached to**, and quoted verbatim. + +**9. Q5, group ids.** On line 468, inside Q5 ("Recommendation: not now … the one +exception to 'every reference is an id'"): + +> "This seems right" + +Groups get no id in this ADR. + +**10. Q6, per-entity versions.** On line 469, Q6 ("Recommendation: not in this ADR"): + +> "Agreed" + +**11. Q8, a duplicate or unknown id.** On line 473, the recommendation to refuse the +load: + +> "Yeah, don't load the file because you can't safely resave it" + +**12. Q9, the reading of "opt-in".** On line 476, the recommendation that an id-less +file is read forever and every save writes ids: + +> "Yes" + +**13. Q10, id reuse after a reload.** On line 485, inside option **(b)**, "Accept +reuse across a reload (recommended)": + +> "Yes, b" + +This is option (b), and (b) was the recommendation. Within a session, retirement still +prevents reuse. Across a save and reload, a removed name that comes back gets its old +slug back. + +**14. Q11, who assigns ids.** On line 492, inside option (a): + +> "The host assigns" + +**15. Q12, old-version files and the new header rule.** On line 502, inside option (a), +"No (recommended)": + +> "Yes, a as long as version is unambiguously marked" + +Old files are read as they always were, on the condition that the version is marked +unambiguously. Clause 9 says how the loader meets that condition. + +**16. Q13, slugify.** Five answers, one per item: +- **(a) where it applies.** On line 510: + > "All id generation" +- **(b) case.** On line 512, against the recommendation to keep case: + > "lower case only" +- **(c) `_` and `-`.** On line 515: + > "Yeah, keep both" +- **(d) non-ASCII letters.** On line 518: + > "Strip non lower alphanumeric or -_" +- **(e) the empty slug.** On line 520, the recommendation `-`: + > "Yes" + +So an id is lower case, and its alphabet is `a`–`z`, `0`–`9`, `-` and `_`. Clause 4 +states the rule. Q15 asked the user to confirm the order of its steps (answer 18). + +**17. Q14, formula text.** On line 529, the recommendation "ids in the file": + +> "Display in the formula panel but not as column/row/page headers." + +This does not contradict ids in the file, which answer 3 already requires. It answers +**where formula text is displayed**, and it could be read two ways. That was Q16, +which answer 19 settles. + +### Third round: answers on revision 3 + +The user answered revision 3 (`context/adr/007-stable-identities.md` at `18c82e1c`) +with three inline comments on PR #210, posted 2026-09-24 00:34–00:36Z. Each is mapped +to the question on the line it is attached to, and quoted verbatim. + +**18. Q15, the order of slugify's steps.** On line 571, Q15's recommendation "fold, then +strip": + +> "casefold, replace space with `_`, strip" + +The fold comes first, as recommended. The answer also **changes answer 8's middle +step**: a space becomes `_`, not `-`. Two details of that sentence are Q18. + +**19. Q16, what the formula panel shows.** On line 584, the paragraph that declined to +recommend a reading: + +> "It should be at least `Profit (profit) = Revenue - Cost` but if `Revenue` is +> ambiguous, you'll have to display the slugs. Working with slugs internally and making +> the friendly names view-layer is almost always the right move here." + +So the panel shows formula text over **display names**, with the target's id beside it +in parentheses. Where a display name in the text is ambiguous, the panel shows the id +instead. Ids are what the file and the model hold, and display names belong to the view +layer. This also settles answer 17: the thing displayed "in the formula panel but not as +column/row/page headers" is the target's **id**. Whether an id is enough to resolve an +ambiguous name is Q19. + +**20. Q17, capitals in a hand-written id.** On line 589, the question "May a +hand-written id contain capitals?": + +> "no." + +This is option (a). `id_token` is lower case only, and a header ending in a +parenthesised run that contains a capital loads as a whole-line display with a load +warning. The virtual categories are spelled with capitals today. That is Q20. +(Answer 21 later made a new-version file all-or-nothing, so that header is now refused +rather than warned about. See clause 5.) + +### Fourth round: answers on revision 4 + +The user answered revision 4 (`context/adr/007-stable-identities.md` at `024aa2d3`) +with five inline comments on PR #210, posted 2026-09-24 06:05–06:08Z. Each is mapped +to the question on the line it is attached to, and quoted verbatim. + +**21. Q18(a), runs of spaces, and the collision suffix.** On line 658, Q18(a) ("Does a +run of spaces collapse to one `_`?"): + +> "runs of spaces are collapsed to one `_` and then `-###` numeric suffix is added if +> duplicate (only when importing a file without IDs: a file must either have no IDs at +> all or always have IDs)" + +This decides three things: +- a run of spaces collapses to one `_`, as recommended; +- a duplicate slug gets a numeric suffix after a `-`; +- **a file has ids on everything or on nothing.** A new-version file with a declaration + missing its id is refused, like any file that cannot be resaved safely (answer 11). + So the earlier rule that a hand-written id wins over a generated one in the same file + is gone. + +The parenthesis also limits the suffix to generating ids for a file that has none. Two +details of that are Q22. Q18(b), what "casefold" means, was not answered in this round +(answer 26 settles it). + +**22. Q19, whether an id disambiguates formula text.** On line 678, option (a) ("Make +item ids unique in the document"): + +> "Ids have to be globally unique." + +This is stronger than option (a). **No two ids in a document are equal, whatever their +kind**, so a category, an item, a formula and a view never share one. An id in formula +text or on the wire therefore always names exactly one thing. + +**23. Q20, the virtual categories.** On line 691, option (a) ("Lower-case ids, +capitalised displays"): + +> "Virtual categories have IDs: we need to add a "slug" version such as `_index`, but +> no non-virtual id can start with `_`" + +The ids are `_index`, `_dim` and `_measure`, and the displays stay `_Index`, `_Dim` and +`_Measure`. A leading `_` is reserved for them. Clause 4 says how slugify keeps a +generated id out of that space, and clause 5 says what happens to a hand-written one. + +**24. Q21, an old file naming an undeclared item.** On line 708, option (a) ("Declare +the item at load"): + +> "yes, this is right." + +**25. The file version.** On line 731, the Consequences line "A new file version.": + +> "File version should be added explciitly if it's not already part of the format." + +It is already part of the format. The first line of every `.improv` file is a +mandatory, exact version literal (`version_line = { "v2025-04-09" ~ NEWLINE ~ … }` in +`improv.pest`), and the writer emits it. So the new format is marked by a **new +literal**, and nothing is inferred. Clause 10, new in this revision, states the rule in +one place. + +### Fifth round: answers on revision 5 + +The user answered revision 5 (`context/adr/007-stable-identities.md` at `e044c968`) +with three inline comments on PR #210, posted 2026-09-24 19:50–19:51Z. Each is mapped +to the question on the line it is attached to, and quoted verbatim. + +**26. Q18(b), what "casefold" means.** On line 766, the recommendation to take +"casefold" literally, as full case folding: + +> "full unicode" + +Slugify's first step is **Unicode full case folding**, so `Straße` → `strasse`. + +**27. Q22(a), how the suffix is spelled.** On line 771, the recommendation "a plain +number": + +> "Just a plane number." + +The suffix is `-` followed by a plain decimal number, with no padding. + +**28. Q22(b), collisions outside import.** On line 782, option (i), "Suffix it there +too": + +> "Any ambiguous id gets a suffix, if if the id is between different types of things +> identified. Don't bother implementing a strict order here: `east` `east-1` +> `east-2`, etc. just do what gensym does in CL: global increment-only counter appended +> when dedupe needed." + +This decides three things: +- **every** colliding id gets a suffix: on import, in a session, and across kinds (a + category and an item that slug alike); +- the number comes from **one counter for the document that only ever increments**, as + Common Lisp's `gensym` does, so it is not "the lowest free number"; +- the suffix is appended **only when a dedupe is needed**, so an id that does not + collide stays bare. + +Clause 4 is rewritten to match. The rules this replaces (lowest free number, and +suffixing only on import) were revision 5's. + +### Sixth round: answer on revision 6 + +The user replied at 2026-09-24 22:09Z to their own answer 28, on line 782 of +`e044c968`: + +**29. Hand-written ids are canonical.** + +> "You have to treat the file's hand-written ids as canonical, doing otherwise will +> corrupt the data section." + +This constrains answer 28. The `gensym` suffix applies only to an id the program is +**generating**, on import, when an id-less file loads, or when the host creates +something in a session. **An id written in a file is never renamed**, because every +data key and reference in that file spells it. Clause 4 says what follows: +- a generated id that would collide with a file's id takes the suffix, and the file's + id keeps its spelling; +- two identical hand-written ids in one file are refused at load, not repaired. ## Context **Today the name is the identity.** `Model::categories` is an `IndexMap` keyed by name. `CellKey` and `DataStore`'s interned symbols are names. Views, axis maps, page -selections, hidden and collapsed sets, and formula text all refer by name. That costs -three things: -- **An edit cannot be told apart from a delete followed by an add.** No rename - operation exists for a category or an item. A rename is remove plus add, which loses - every cell (improvise-fz77). A formula is identified by `(target, target_category)`, - and `Model::add_formula` upserts on that pair. -- **An edit whose target was concurrently replaced cannot be told apart from an edit - of the replacement.** -- **A records-mode coordinate rename names its category and item by name** - (`RenameRecordCoordinate { record, category, item }`, `String` operands). +selections, hidden and collapsed sets, and formula text all refer by name. The user's +first message names what that costs: -**What exists already.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize` -aliases from monotone counters. No production caller reads them, they have never been -written to `.improv`, and `View` has no id. km71 landed an opaque interim `FormulaId` -over `(target, target_category)`, built only by `Formula::id()` (`context/code/core.md` -§18). It is on the wire in `ModelCommand::ReplaceFormula { id, was, raw }`, -`FormulaSlot::Existing` and `FormulaEntry`. Because it is still a name pair, -`Model::replace_formula` refuses to rename a target outside `_Measure` -(improvise-kb1d). Legacy JSON is gone (improvise-tt22, PR #224), so `.improv` is the -only native format. +- **An edit cannot be told apart from a delete followed by an add.** No rename operation + exists for a category or an item. A rename is remove plus add, which loses every cell + (improvise-fz77). A formula is effectively identified by `(target, target_category)`, + and `Model::add_formula` *upserts* on that pair. improvise-km71 (PR #214) made formula + editing its own operation, with an interim id built from that pair (below). +- **An edit whose target was concurrently replaced cannot be told apart from an edit of + the replacement.** +- **A records-mode coordinate rename names its category and item by name.** + improvise-fdk1 (PR #221) landed `RenameRecordCoordinate { record, category, item }` + with `String` operands (below). + +**What already exists.** `Category.id: CategoryId` and `Item.id: ItemId` are `usize` +aliases, assigned from monotone counters. **No production caller uses either value.** +`Model::add_category` and `Category::add_item` return them on the idempotent path, and +nothing reads the returned value; the only reader is one test. Neither has ever been written to `.improv`. `View` has no +id. + +**`FormulaId` exists, as an interim.** km71 landed an opaque `FormulaId` whose private +fields are `(target, target_category)`, built only by `Formula::id()` +(`context/code/core.md` §18). It is on the wire in +`ModelCommand::ReplaceFormula { id, was, raw }`, in `FormulaSlot::Existing` and in +`FormulaEntry`. It was made opaque so that this ADR changes one struct rather than every +signature. It is still a name pair, so a formula whose target is renamed gets a +different `FormulaId`. `Model::replace_formula` therefore refuses to rename a target +outside `_Measure` until this ADR lands (improvise-kb1d). + +**Legacy JSON is gone.** improvise-tt22 landed in PR #224, so `.improv` is the only +native format. ## Decision -**Categories, items, formulas and views each have an id, and so do groups -[Amendment]. The id is the key.** The current `name` field becomes the id, and the +**Categories, items, formulas and views each have an id, and the id is their key.** A +category or item's current `name` field *becomes* the id (the user's suggestion), and the human name moves to a new `display` field. -- **Every reference is an id**: `CellKey`s, axis maps, page selections, hidden and - collapsed sets, formula references, data keys, and every wire operand that names an - existing thing. A reference to the stable identity cannot glitch when something is - renamed or replaced. -- **Display names are for people.** They appear on screen, are typed into - minibuffers, and are turned into ids at the input edge. **[Amendment]** Clause 6 narrows - which readers turn display text into ids. +- **Every reference is an id.** That covers `CellKey`s, axis maps, page selections, + hidden and collapsed sets, formula references, data keys, and wire operands that name + an existing thing. +- **Display names are for people.** They appear on screen, are typed into minibuffers, + and are resolved to ids at the input edge. + +Twelve clauses. ### 1. What carries an identity -| kind | id | wire address | -|---|---|---| -| category | yes | `CategoryId` | -| item | yes | `ItemId` | -| formula | yes | `FormulaId` | -| view | yes | `ViewId` | -| group | **[Amendment]** yes, with a display | `GroupId` | -| record | `RecordHash`, over ids (clause 8) | unchanged shape | +| kind | id | unique within | wire address | +|---|---|---|---| +| category | yes | the document, across every kind (answer 22) | `CategoryId` | +| item | yes | the document, across every kind (answer 22) | `ItemId` | +| formula | yes | the document, across every kind (answer 22) | `FormulaId` | +| view | yes (the user's example) | the document, across every kind (answer 22) | `ViewId` | +| record | `RecordHash`, now over ids (clause 8) | the document | unchanged shape | +| group | **no**, not in this ADR (answer 9) | | | -**An id always names the same thing, so ids are unique in the whole document**, across -every kind. An item id names one item without its category. A `CellKey` still pairs -each item with its category, and an operation that creates an item still names the -parent category. +**Ids are unique in the whole document** (answer 22). One namespace covers categories, +items, formulas and views, so an item id names one item without its category. A +`CellKey` still pairs each item with its category, because the key needs the +categories. An operation that creates an item still names the parent category. -**[Amendment] Groups.** A group has an id and a display, like a category or an item, -and its id behaves like every other id: it is unique in the document, generated by -slugify (`group` is an empty slug's ``), declared beside the display in the file, -referred to by id, created by display, and renamed by a display rename. In code, -PR #336 (improvise-cq1q.12, open) already adds a private display to `Group` beside its -key; under this amendment that key becomes the group's id. - -**Reserved ids.** The virtual categories have ids `_index`, `_dim` and `_measure`, -displayed `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** -**[Amendment]** A leading `_` is reserved for virtual ids in general. The virtual views -have ids `_records` and `_drill`, displayed `_Records` and `_Drill`; they are addressed -on the wire (`SwitchView`), and a slugified `_Drill` would otherwise be `drill` and -could collide with a user's view. Slugify never produces a leading `_` (clause 4), so -the reservation needs no check in the grammar. -- **[Amendment] Virtual ids are lower case.** The code, the wire, `ViewState` and the - browser client spell the virtual categories `_Index`, `_Dim` and `_Measure` today, - and move to the lower-case ids in the re-key step (clause 11). -- **[Amendment] Virtual categories and virtual measures are defined by criteria, not - declared.** Each contains whatever in the document meets its criterion: `_measure` - holds the measures, and `_index` and `_dim` hold the record positions. Their ids and - displays are hardcoded in the application, so a file never declares one. Where a - line refers to one, it spells the virtual id: `_measure: column`, `_index: none`, - `region=east, _measure=revenue = 1200`. -- **[Amendment] An old file's virtual names are recognised, not migrated as - declarations.** The loader recognises `_Measure`, `_Index` and `_Dim` by their - hardcoded displays, and an old `## Category: _Measure` section is read for the - stored measures it lists, not as a declaration of `_measure`. The first save writes - `_measure` wherever the file refers to it. The stored measures themselves - (`Revenue`, `Cost`) are listed as the format lists them today, each with its id. -- **[Amendment] Records mode's synthetic positions are values, not ids.** The row - coordinate under `_index` is a value of that virtual category. So, by the same - reasoning, is the column coordinate under `_dim` (a column index, gotcha 1). They are - never persisted or typed, take no part in id uniqueness, and cannot collide with a - user's item. - -**Formulas and measures.** **[Amendment]** A measure is the normal formula target -(`profit = revenue - cost`). A dimension (a category other than `_Measure`) is a rare -one, for a derived dimension. -- **A `_Measure` target has no item of its own**, and its handle is its formula's - `FormulaId`. So the category panel's `_Measure` rows use - `ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). -- **[Amendment] A formula is a kind of measure, so there is no distinction between a - `_Measure` formula's id and its measure's id.** A measure has an id and a display, - as everything else does. `Profit = Revenue - Cost` is one thing, `profit`: another - formula refers to it as `profit`, each Profit cell's `CellKey` holds `profit` under - `_measure`, and `ReplaceFormula` and `RemoveFormula` address `profit`. -- **[Amendment] Each measure's cells are keyed by one handle.** A stored measure is - keyed `ItemHandle::Item(id)`. A formula's measure is keyed - `ItemHandle::FormulaTarget(id)`. The id is the measure's either way, and no column is - keyed by both. -- **[Amendment] A formula never merges into a stored measure.** If a stored measure - `profit` (displayed `Profit`) exists and the user adds `Profit = Revenue - Cost`, - the new formula is a second measure with a fresh id, `profit-` from the shared - counter (clause 4), like any colliding generated id. The stored measure keeps - `profit` and its data. Removing the formula removes `profit-` and leaves `profit` - untouched. The formula keeps the display the user typed, so both are displayed - `Profit` and are told apart by id: the formula panel shows the ids where the display - is ambiguous (clause 6), and a typed `Profit` that matches both is refused with both - ids listed (clause 12). -- **[Amendment] A target outside `_Measure` is a real item**, with its own `ItemId`, - and its formula has a separate `FormulaId`. A formula whose generated id would - collide takes a suffix like any other generated id (clause 4): a formula `Tax` in - `CustomCat` gets item id `tax`, then formula id `tax-`. -- **[Amendment] Formulas may refer to anything by id.** A function in a formula may - accept any id-bearing thing (a category, an item, a measure, a formula, a view, a - group), referred to by its id. Whether a given function takes the thing itself or its value - (an item's display, say) is part of that function's signature. Global uniqueness - makes every such reference unambiguous, and a function may read any part of a record, - so nothing further is needed here. +**Virtual categories have reserved ids** (answer 23): `_index`, `_dim` and `_measure`, +with the displays `_Index`, `_Dim` and `_Measure`. **No other id may begin with `_`.** +A `_Measure` formula target has no item of its own. Its handle is its +formula's `FormulaId`, which is why the category panel's `_Measure` rows need +`ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)}` (clause 6). ### 2. Storage keys by id; `display` is the only place a human name lives +Following answers 1 to 3: - **Categories and views:** `Model::categories` and `Workbook::views` are keyed by id, - and each `Category`, `Item` and `View` gains a `display: String`. **[Amendment]** So - does `Group`. -- **Cells:** a `CellKey` is a sorted list of `(CategoryId, ItemId)` pairs. - **[Amendment]** The pairs are `(CategoryId, ItemHandle)`, so that a `_Measure` - formula cell's coordinate is `FormulaTarget` without claiming an `ItemId` the - document does not hold. - - The key stays sorted, now **by category id**, so the canonical form (gotcha 5) - is kept with a different sort key. - - `DataStore` interns ids exactly as it interns names today. - - An id's text names one thing, because ids are unique across kinds. + and each `Category`, `Item` and `View` gains a `display: String`. +- **Cells and derived surrogates:** a `CellKey` is a sorted list of + `(CategoryId, ItemId)` pairs. It stays sorted, now **by category id**, so gotcha 5's + canonical form is kept with a different sort key. `DataStore` interns ids exactly as it + interns names today. - **View configuration:** the axis maps, page selections, and hidden and collapsed sets - are keyed by ids. **[Amendment]** A collapsed entry names its group by the group's - id. -- **Formulas:** a formula's references are ids (clause 5), and the formula panel - renders them as display names (clause 6). + are keyed by ids. +- **Formulas:** a formula's references are ids (clause 5). The formula panel renders + them as display names (answer 19; clause 6). -**A rename costs one `display` write.** It moves no `CellKey`, no `RecordHash` and no -`layout_version`, so an edit a client composed before the rename still lands after it. -The grid's labels are display strings, so a rename must still dirty the grid and the -panels (`Dirty::of`, gotcha 35), but no basis tier moves. +**What a rename now costs:** one `display` write. It moves no `CellKey`, no `RecordHash` +and no `layout_version`, so an edit a client composed before the rename still lands after +it. That is the "avoid glitches" the user's rule is for. The grid's labels are display +strings, so the rename's effect must still dirty the grid and the panels +(`Dirty::of` gotcha 35), but no basis tier moves. -**The migration cost is the call sites.** Every API that takes a name today takes an -id instead: `Model::category(name)`, `Workbook::add_item(cat, item)`, `CellKey::new`'s -callers, the evaluator's `Expr::Ref` lookup and the `View` methods. Display→id -resolution moves to the places a human types. The field rename itself is cheap. The -change is mechanical but wide, and it is the bulk of the implementation. +**What the migration costs, stated honestly.** Every API that takes a name today +(`Model::category(name)`, `Workbook::add_item(cat, item)`, `CellKey::new` callers, the +formula evaluator's `Expr::Ref` lookup, `View` methods) takes an id instead. The +display→id resolution moves to the places a human types, such as the minibuffer commits, +the formula parser's reference resolution, and import. The change is mechanical but wide, +and it is the bulk of the implementation. The user judged it cheap ("at worst, it would +be renaming field access"), and for the field that is right. The call sites are the +cost. -**Display names stay unique within their scope**, as names are today. So a -human-typed name resolves to exactly one id within its scope, and a rename to a -display already in use there is refused. An item's scope is its category, so two -categories may each have an item displayed `Revenue`. Formula text crosses -categories, so a display may be ambiguous there. The ids differ, so the formula panel -shows the ids where a display is ambiguous. **[Amendment]** The one exception is a formula -whose measure shares a stored measure's display (clause 1): it is created anyway, and -the two are told apart by id. +**Display names stay unique within their scope**, as names are today. So a human-typed +name resolves to exactly one id, and a rename to a display name already in use is +refused. An item's scope is its category, so two categories may each have an item +displayed `Revenue`. Formula text crosses categories, which is where answer 19's +"ambiguous" arises. Their ids differ (answer 22), so the panel resolves the ambiguity +by showing the ids. ### 3. Ids are interned symbols -An id is a **string**, and in memory a `Symbol` from the existing `SymbolTable`: -copyable, compared as an integer. -- **A `Symbol` never crosses the wire or reaches the file.** It indexes one process's - table. The wire and `.improv` carry the id's text, and each end interns it on - arrival. -- **`Rejection` stops being `Copy`**, because `Rejection::Gone` (clause 7) carries an - id's text. `SaveFailed` carries no message only because `Rejection` was `Copy`. - Whether it gains one is the implementer's call. +Following answer 4, an id is a **string**, and in memory it is a `Symbol` from the +existing `SymbolTable`: copyable, with integer comparison. Two consequences. + +- **A `Symbol` never crosses the wire or reaches the file.** It is an index into one + process's table. The wire and `.improv` carry the id's **text**, and each end interns + it on arrival. +- **`Rejection` stops being `Copy`**, because `Rejection::Gone` (clause 7) carries an id's + text. `basis.rs`'s `SaveFailed` carries no message *because* `Rejection` was `Copy`, so + that reason lapses. Whether `SaveFailed` now gains its message is the implementer's + call. ### 4. How an id is made: slugify, one rule everywhere -**Every** id the app generates comes from the display name by **slugify**: on -import, on interactive creation, and when an id-less `.improv` file is loaded. So a -thing's id never depends on how it was made. An id is lower case. +Following answers 8, 16 and 18, **every** id the app generates comes from the display +name by **slugify**. That covers import, interactive creation, and assigning ids to an +id-less `.improv` file at load (answer 16(a), "All id generation"). So a thing's id +never depends on how it was made. The steps, in order (answer 18, "casefold, replace +space with `_`, strip"): -**The accepted steps:** -1. casefold, with Unicode full case folding; -2. replace each run of spaces with one `_`; -3. strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`; -4. remove any leading `_`; -5. if nothing is left, use `-`. +1. **Casefold, with Unicode full case folding** (answers 16(b), 18 and 26). Rust's + `str::to_lowercase` is not enough (`ß` stays `ß` and is then stripped), so this + needs a case-folding table or a small crate in `improvise-core`. It must pass the + `wasm` gate, because that crate is in the browser bundle. +2. **Replace each run of spaces with one `_`** (answers 18 and 21; answer 18 changed + answer 8's `-`). +3. **Strip every character that is not `a`–`z`, `0`–`9`, `-` or `_`** (answers 16(c) + and 16(d)). This strips non-ASCII letters, so an id never needs quoting in an + `.improv` reference (clause 5 notes where formula text differs). +4. **Remove any leading `_`** (answer 23: no non-virtual id may begin with `_`). The + answer states the constraint. Removing the underscores is this ADR's way of meeting + it, so `_Draft` → `draft`. +5. **If nothing is left, use `-`** (answer 16(e)), such as `category-1` or + `item-4`. `n` comes from the same counter as every other suffix (answer 28), not + from the lowest free number. -**[Amendment] The steps become:** -1. **Casefold, with Unicode full case folding**, so `Straße` → `strasse`. Rust's - `str::to_lowercase` is not enough (`ß` stays `ß`). The folding table or crate must - pass the `wasm` gate, because `improvise-core` is in the browser bundle. -2. **Replace every character outside `a`–`z`, `0`–`9` and `-` with `_`.** The id's - alphanumerics are ASCII and include `-`. Every other character, including spaces, - tabs and other whitespace, punctuation and every non-ASCII letter, becomes `_`. -3. **Collapse each run of `_` to one `_`**, whether the underscores came from step 2 - or were in the display. -4. **Remove every leading and trailing `_` and `-`**, repeating until neither end - holds either. A leading `_` is reserved for virtual ids (clause 1), a trailing `_` - is dropped, and no id begins or ends with `-`. -5. **If the slug starts with a digit, prepend `n`.** An id is never purely numeric - and never starts with a digit, so `2024` becomes `n2024`. -6. **If nothing is left, use `-`**, such as `category-1` or `item-4`. `n` - comes from the shared counter below. A display of only `-`, only `_` or only - non-ASCII letters ends here, because `-` alone is not an id. +Examples: `East` → `east`, `Time Period` → `time_period`, `Coastal_East` → +`coastal_east`, `Q1-2024` → `q1-2024`, `Region(code)` → `regioncode`, `Café` → `caf`, +`Ü` → `category-`, `Straße` → `strasse` (answer 26). `Time Period` and +`Time_Period` slug alike, so whichever comes second gets `time_period-`. `East` in +`Region` and `East` in `Direction` slug alike too, and since ids are global (answer +22), the second is `east-`. Here `` is the counter's next value, for example +`east-1`. -**The syntax of an id** is therefore: a lower-case ASCII letter, then any of `a`–`z`, -`0`–`9`, `-` and single underscores, never ending in `_` or `-`: -`[a-z]([a-z0-9-]|_(?=[a-z0-9-]))*`, not ending in `-`. Slugify's output already has -that shape, so slugifying an id gives the same id back. The collision suffix `-` -(below) keeps the shape too, because it ends in a digit. +The remaining rules: -Examples under the amendment: - -| display | id | -|---|---| -| `East` | `east` | -| `Time Period`, `Time Period`, `Time\tPeriod`, `Time \u{200B} Period` | `time_period` | -| `Coastal_East` | `coastal_east` | -| `a_ b`, `a__b` | `a_b` | -| `Q1-2024` | `q1-2024` | -| `Q1 - 2024` | `q1_-_2024` | -| `East-1` | `east-1` | -| `East `, `East_`, `-East-` | `east` | -| `_Draft` | `draft` | -| `Region(code)` | `region_code` | -| `Café` | `caf` | -| `Crème Brûlée` | `cr_me_br_l_e` | -| `Straße` | `strasse` | -| `2024`, `_2024` | `n2024` | -| `2024-Q1` | `n2024-q1` | -| `Ü`, `-`, `%` | `-` | - -Under the accepted steps, the ones that differ are `East ` → `east_`, `a_ b` → -`a__b`, `Time\tPeriod` → `timeperiod`, `Time \u{200B} Period` → `time__period`, -`-East-` → `-east-`, -`Region(code)` → `regioncode`, `Crème Brûlée` → `crme_brle`, `2024` → `2024`, -`2024-Q1` → `2024-q1` and `-` → `-`. - -**Collisions.** `Time Period` and `Time_Period` slug alike, and whichever comes later -takes a suffix. **[Amendment]** So do `East`, `East ` and `East_`, which slug alike -only under the amendment's steps. `East` in `Region` and `East` in `Direction` also -slug alike, and ids are global, so the second is `east-`. - -**[Amendment] A suffix and a hyphenated slug can look alike.** The display `East-1` -slugs to `east-1`, which is also what the counter's suffix may make of `east`. They -never collide, because every id a document holds is taken, whoever made it: -- if `east-1` is already held (as a slug or as a suffix), a display `East-1` that slugs - to it is a collision, and becomes `east-1-`; -- if the counter's candidate `east-` is already held, the counter advances until a - candidate is free. - -So an id says nothing about how it was made: `east-1` may be a suffixed `east` or the -slug of `East-1`. -- **Any colliding generated id gets a suffix**, whatever kind of thing holds the - slug: on import, when an id-less file loads, and when the host creates something in - a session. An id that does not collide stays bare. -- **The suffix is `-` plus the next value of one increment-only counter per - document**, as a plain number (`east-1`, not `east-001`). The counter works like - Common Lisp's `gensym`: it never goes down and is never reused, so no order holds - among suffixes (`east-7` may follow `east` directly). -- **Retirement.** Within a session a removed id is never handed out again. The - counter is past any suffix it issued, and a removed bare slug stays reserved in - memory. -- **Determinism.** The counter advances in file order or creation order, never in - hash order (gotcha 18), so the same input gives the same ids. -- **The counter** lives in memory, starts at 1 when a document loads, and is not - written to the file. - -**Hand-written ids are canonical.** An id written in a file is never renamed, because -every data key and reference in that file spells it. So: -- **a file has ids on everything or on nothing.** An id-less file (an old version, - or a CSV or JSON import) gets every id from slugify. A new-version file must carry - an id on every declaration, and one that does not is refused (clause 5); -- the suffix only ever goes on the id being generated. When a generated candidate - `slug-` is taken by a file's id, the counter advances until a candidate is free; -- **two identical hand-written ids in one file are refused at load**, even when they - declare different kinds of thing. Renaming one would corrupt the data section, and a - file with a duplicate cannot be resaved safely. The error names both lines. -- **[Amendment] A hand-written id obeys every syntax rule of a generated id** (the - syntax above). So `east-1`, `item-4` and `q1_-_2024` load; `-east`, `east-`, `2024`, - `a__b` and `_draft` are refused at load (clause 5). - -**Reuse across a save and reload is accepted.** Retired ids are not persisted. So -remove `East`, save, reload and add `East`, and the new item gets `east` again. An -in-flight command from before the reload is refused by the generation tier. - -**[Amendment] Two branches of one file can merge into a file that will not load.** If -each branch adds a different thing that generates the same slug, the merged file -declares that id twice. The load is refused and names both declarations, so the merge -is repaired by hand. Stable identity across copies and branches of a file is not -promised (improvise-8tk5). - -**Built code.** `crates/improvise-core/src/model/id.rs` (`slugify`, `IdAllocator`) -implements the accepted steps, and `context/code/core.md` §20 describes them. Once -the amendment is approved, a follow-up must: -- replace every character outside `a`–`z`, `0`–`9` and `-`, not only spaces, with - `_` instead of stripping it, and collapse `_` runs; -- trim a trailing `_` and both ends' `-`, and prepend `n` before a leading digit; -- make `claim` refuse every hand-written id a generated one could not be; -- add the virtual views to `VIRTUAL_IDS`, and `IdKind::Group`. - -Nothing calls `slugify` yet. +- **A file has ids on everything or on nothing** (answer 21). An id-less file (old + version, or a CSV or JSON import) gets every id from slugify. A new-version file must + carry an id on every declaration, and one that does not is refused (clause 5). So no + file mixes hand-written and generated ids. +- **Any colliding id gets a suffix, `gensym`-style** (answers 21, 27 and 28). + - **When:** whenever a generated slug is already taken, whatever kind of thing holds + it. That applies on import, when an id-less file loads, and when the host creates + something in a session. An id that does not collide stays bare. + - **What:** `-` and the next value of **one increment-only counter per document**, + as a plain number (`east-1`, not `east-001`). The counter never goes down and is + never reused, so no order is enforced among suffixes: `east-7` may follow `east` + directly. + - **Retirement:** within a session a removed id is never handed out again. The + counter has already moved past any suffix it issued, and a bare slug that was + removed is still reserved in memory. + - **Determinism:** the counter advances in file order or creation order, so the same + input gives the same ids, never hash order (gotcha 18). + - **Hand-written ids are canonical** (answer 29). The suffix only ever goes on the + id being generated. An id read from a new-version file keeps its spelling, even + when it looks like a slug or a suffixed one (`east-3`), because the file's data + keys and references spell it. So when a generated candidate `slug-` is already + taken by a file's id, the counter advances again until the candidate is free. + - **Two identical hand-written ids in one file are refused at load** (answers 11, + 22 and 29). By answer 22 they are a duplicate even when they declare different + kinds of thing, such as a category and an item. Renaming either one is exactly + what answer 29 forbids, and answer 11 already refuses a duplicate because such a + file "can't safely resave". The error names both lines (clause 5). This follows + from the answers, so it is not asked as a new question. + - **The counter** lives in memory and starts at 1 when a document loads. It is not + written to the file, since retired ids are not persisted (answer 13). That is + this ADR's reading. +- **Across a save and reload, reuse is accepted** (answer 13, Q10(b)). Retired ids are + not persisted. So remove `East`, save, reload and add `East`, and the new item gets + `east` again. An in-flight command from before the reload is refused by the + generation tier. The one remaining risk is an outside reference that outlives the + file, and none exists today. ### 5. Persistence: ids on declarations *and* references -- **Declarations** (section headers, item lines, formula lines, and **[Amendment]** - group definitions) carry the display and the id. -- **Every reference** is the id: axis lines, `hidden:` and `collapsed:`, page - selections, `## Aggregation`, `Initial View:`, `## Data` keys (ids are guaranteed - unique, names might not be), and names inside formula text. +Following answers 3 and 7, the file now carries ids everywhere: +- **Declarations** (section headers, item lines, formula lines) carry the display name + and the id. +- **Every reference** carries the id: axis lines, `hidden:` and `collapsed:`, page + selections, `## Aggregation`, `Initial View:`, `## Data` keys, and names inside formula + text (answer 3). The formula panel shows display names instead (answer 19). -**The id alphabet.** Every id is spelled only in `a`–`z`, `0`–`9`, `_` and `-`. The -grammar spells it **with ranges**, because the generator drops `ASCII_ALPHANUMERIC` -(improvise-rxz9). So an id never needs pipe quoting in a declaration or reference. - -**What the loader refuses** in a new-version file: -- a declaration with no id. A header ending in a parenthesised run with a capital - (`Sales(Copy)`), an empty tag or a tag with a trailing space does not match - `id_tag`, so it has no id. The error names the line; -- a hand-written id beginning with `_`, other than a reserved one; -- **[Amendment]** a hand-written id that breaks any syntax rule of a generated id - (clause 4). A file never declares a virtual category, so no declared id begins with - `_`; a reference may still spell a virtual id such as `_measure`. The grammar's - `id_token` stays permissive, and the loader refuses, so the error can name the line. +**Id alphabet.** Every id, generated or hand-written, is spelled only in `a`–`z`, +`0`–`9`, `_` and `-` (clause 4; answer 20). The grammar spells them **with ranges**, +because the generator drops `ASCII_ALPHANUMERIC` (improvise-rxz9). So an id never needs +pipe quoting in an `.improv` declaration or reference. A header ending in a +parenthesised run that contains a capital, such as `Sales(Copy)`, does not match +`id_tag`, and neither does an empty or trailing-space tag. In a new-version file that +declaration has no id, so the load is refused (answer 21), and the error names the +line. Revision 4 generated an id and warned. A hand-written id beginning with `_`, +other than the three virtual ids, is refused too (answer 23). **Inside formula text an id may still need quoting.** The formula grammar's `bare_ident` must begin with a letter or `_`, and keywords are guarded (gotcha 34). So -the writer spells an id in formula text through `quote::quote_name`. An id equal to a -keyword (`sum`, `if`, `where`, `is`) is written `|sum|`, and under the accepted rules -an id beginning with a digit (`2024`) is written `|2024|`. **[Amendment]** No id begins -with a digit, so only keyword ids need quoting. A `-` inside an id needs none, because -the formula grammar's bare identifier already admits `-` after its first character. +the writer spells an id in formula text through `quote::quote_name`, as it spells a +name today. An id beginning with a digit (`2024`) or equal to a keyword (`sum`, `if`, +`where`, `is`) is written `|2024|` or `|sum|`. No new rule is needed, but "never needs +quoting" does not extend to formula text. ```pest id_token = @{ ('a'..'z' | '0'..'9' | "_" | "-")+ } -// The space before the parenthesis is optional. +// The space is optional (answer 6). id_tag = ${ " "* ~ "(" ~ id_token ~ ")" } -// Declarations: display, then id. The id is optional in the grammar so that the -// loader, not the parser, refuses a missing one and can name the line. +// Declarations: display name, then its id. The id is optional in the grammar so the +// loader, not the parser, refuses a missing one and can name the line (answer 21). header_decl = { pipe_quoted ~ id_tag ~ &NEWLINE | header_display ~ id_tag ~ &NEWLINE | line_name } header_display = @{ (!(id_tag ~ NEWLINE) ~ !NEWLINE ~ ANY)+ } item_decl = { name ~ id_tag? } grouped_item = { "- " ~ name ~ id_tag? ~ "[" ~ name ~ "]" ~ NEWLINE ~ blank_lines } -// References: an id, never a display. +// References: an id, never a display name. id_ref = { id_token } ``` -`model_name` has no id and keeps `header_name`. `Initial View:` is a reference and -takes an `id_ref`. +`model_name` has no id, so it keeps `header_name`. `Initial View:` becomes a reference +and takes an `id_ref`. A group reference (`[Coastal]`) stays a name while groups have no +id (answer 9). -**Group references.** A group reference (`[Coastal]`) is a name. **[Amendment]** It is -the group's id, as is the group in a `collapsed:` line, and the group definition -(`> Coastal`) declares a display and an id like any other declaration -(`> Coastal(coastal)`). - -**[Amendment] An axis line is tried before `format:`, and a format that reads as an -axis line is pipe-quoted** (improvise-le4p). `view_entry` is an ordered choice, and -`format_line` takes the rest of the line. So a category whose id is `format` would -lose its axis line to the view's number format, and the new-version arm puts -`axis_line` first. -- **The collision then moves.** `set-format` stores its argument verbatim, so - `:set-format none` writes `format: none`, which would now read as the axis line of a - category `format`. -- **So the writer pipe-quotes a number format whenever the line would also match - `axis_line`**: an axis keyword (`row`, `column`, `page`, `filter`, `none`) alone or - followed by `, `, or a value beginning with `|`. The new arm's `format_line` is - `pipe_quoted ~ &NEWLINE | rest_of_line`, so `format: |none|` is the format `none`. -- **An old-version file's `format: none`** is read as before (its arm keeps - `format_line` first), and it is written quoted on its first save. -- **Required tests:** - - a category displayed `Format`, on a row axis, round-trips; - - a view whose number format is each axis keyword round-trips in a document where - that view also holds a category `format` on an axis. - -**[Amendment] A formula line** declares the formula: its target, its id, and its -expression over ids. Today's `web/demo.improv` and the format's own dimension example -are written: - -``` -## Formulas -- Profit = Revenue - Cost -- Tax = Revenue * 0.1 [CustomCat] -``` - -Under the amendment the same two formulas are written: - -``` -## Formulas -- Profit(profit) = revenue - cost -- tax(tax-1) = revenue * 0.1 [customcat] -``` - -- **A measure formula** (no category tag) is the normal case. The formula is the - measure (clause 1), so the line is the measure's only declaration: `Profit` is its - display, and `(profit)` is the id it shares with its formula. No `[_measure]` tag is - written, because a file does not spell a virtual category where it is not needed - (clause 1); a line without a tag targets `_measure`. -- **A dimension formula** (a `[]` tag) targets a real item, already - declared with its display in its category's section (`## Category: CustomCat(customcat)` - lists `- Tax(tax)`). So the line names the target by its item id, `tax`, and - `(tax-1)` is the formula's own id. The display is not repeated, because a display - lives in one place. -- **The tag is always an id**, the target category's (`[customcat]`, never - `[CustomCat]`), and `_measure` is implicit: it is never written as a tag. -- **So the loader can always tell the two apart**: with a tag the target is an item - id; without one it is a measure's display. -- **The expression is over ids**, spelled through `quote::quote_name`. A reference to - a formula target is the id text of its `ItemHandle`: the item id outside `_Measure`, - the measure's (and its formula's) id for a `_Measure` target. -- **A `WHERE`/`IS` value follows the same rule**. It is the id text of an - `ItemHandle` in the filtered category, and the formula panel shows it as a display. -- **A display in the target position that looks like `Name(x)` is pipe-quoted**, as in - `- |Margin(%)|(margin) = …`. An id never contains a parenthesis. - -**`header_display` is guarded only by a lookahead**, which the generator skips -(gotcha 34). So the generator may emit a display ending in something that parses as an -id. Round-trip property 3 still holds, because the first parse settles the split, but +**`header_display` is guarded only by a lookahead**, which the generator skips (gotcha +34). So the generator may emit a display ending in something that later parses as an id. +Round-trip property 3 still holds, because the first parse settles the split. But property 1 must be stated over the parsed result, not over the generated text. -**The version line alone picks the regime** (clause 10). -- **In an old-version file** a header is the whole line, a reference resolves against - display names, and ids are assigned by slugify. This resolution migrates a file - whose names were its only identity; it is not a reference by a person. -- **In a new-version file** declarations carry ids and a reference is an id. +**Which regime a file is read under is decided by its version line alone** (clause +10). In an old-version file, a header is the whole line, a reference resolves against +display names, and ids are assigned by slugify. In a new-version file, declarations +carry ids and a reference is an id. -**An old file naming an item it never declares** declares it at load. This happens -today when a `## Data` line names an undeclared item (improvise-98pd). The item gets a -slug like any other, and the first save writes it, so a file that loads today keeps -loading (improvise-80rm). Ids do not settle 98pd's other two sources, `set-cell` and a -stale page selection. A page selection holds an item id, and removing the item leaves -it dangling, as a removed name does today. +**An old file naming an item it never declares** declares it at load (answer 24). This +happens today when a `## Data` line names an undeclared item (improvise-98pd, second +source). The item gets a slug like any other, and the first save writes it, so a file +that loads today keeps loading (the lesson of improvise-80rm). The NonItem work +(improvise-gk51, PR #265) projects such a cell as `CellId::NonItem`. From a loaded file +that case no longer arises. Ids do not settle 98pd's other two sources, `set-cell` and +a stale page selection: a page selection holds an item id, and removing the item leaves +it dangling exactly as a removed name does today. -**Old data keys change spelling on the first save.** Slugify folds case, so an old -file's `Region=East` resolves at load through the display `East` and is written -`region=east`. The data is the same, but a `git diff` across that save touches every -data line. +**Slugify folds case (answer 16(b)), so old data keys change spelling on first save.** +An old file's `Region=East` resolves at load through the display `East`. The first +save writes the id: `Region=East` becomes `region=east`. The data is the same, but a +`git diff` across that first save touches every data line. **Round-trip properties:** 1. an id-less (old-version) file loads, gets ids, and is written in the new version on @@ -476,312 +591,329 @@ data line. 2. a new-version file loads with every id kept verbatim; 3. `format(parse(format(parse(x)))) == format(parse(x))`. -A **formula-line generator strategy** is required. The grammar walker emits no `=`, and +A **formula-line generator strategy** is required. The grammar walker emits no `=` and the walker drops such lines, so without its own strategy every formula property is vacuous (improvise-rxz9). -**A duplicate, unknown or missing id in a new-version file refuses the load.** The -error names the id, and for a duplicate both declarations. A silent repair is ruled -out, because reassigning an id would silently change what its references point at. +**A duplicate, unknown or missing id in a new-version file refuses the load** (answers +11, 21, 22 and 29; a duplicate across two kinds of thing counts, and a hand-written id +is never renamed to resolve one; answer 11: +"don't load the file because you can't safely resave it"). The error names the id, +and for a duplicate it names both declarations. Repairing silently is ruled out, +because references carry ids (answer 3): reassigning a duplicate would silently change +what its references point at. -**Import** (CSV, JSON) never parses ids out of dimension names. A header `Region(code)` -is the display `Region(code)`, and its id is its slug. The writer pipe-quotes any -display that would otherwise read as display plus id: -`## Category: |Region(code)|(regioncode)` (**[Amendment]** `(region_code)`). +**Legacy JSON is removed** (improvise-tt22), so it needs no migration here. -### 6. The wire: address by id, create by display, learn ids from projections +**Import** (CSV, JSON) never parses ids out of dimension names (answer 8). A header +`Region(code)` is the display name `Region(code)`, and its id is its slug. The writer +pipe-quotes any display name that would otherwise read as display plus id: +`## Category: |Region(code)|(regioncode)`. -- **The host assigns every id.** A client never proposes one. +### 6. The wire: address by id, create by display name, learn ids from projections + +- **The host assigns every id** (answer 14). A client never proposes one. - **Every operand naming an existing thing is an id.** The one operand naming the thing - being created is a display, and the host slugifies it: + being created is a display name, and the host slugifies it: - `AddCategory(display)` and `CreateView(display)`; - - `AddItem { category: CategoryId, display }`; - - `AddItemInGroup { category: CategoryId, display, group }`. **[Amendment]** `group` - is the group's id, and a new group is created by its display under the same - create-by-display rule; - - `AddFormula { target_category: CategoryId, raw }`. -- **[Amendment] Only two readers turn display names in input text into ids: CSV - import and the `.improv` loaders.** They read text people wrote, where names are - all there is. The wire never carries a display as a reference. -- **[Amendment] Command-language text names things by id.** Anything that uses the - command language as a scripting language is a script, wherever the text comes from - (a script file, `improvise cmd`, another program). A script that names something by - its display fails. A script gets a display only through a dedicated function such as - `display_for(id)`. -- **[Amendment] Interactive input matches either a display or an id**, in the TUI - and the browser, as long as it is unambiguous which thing the user is picking. Ids - win when it is not, and the display is shown as a preview (clause 12). + - `AddItem { category: CategoryId, display }`, + `AddItemInGroup { category: CategoryId, display, group }` and + `AddFormula { target_category: CategoryId, raw }`. - **A frame cannot address a thing it creates**, because it does not know the id the - host will assign. A create-then-act compound becomes **one command**: - `create-and-switch-view` sends `[CreateView, SwitchView]` today and becomes - `CreateView { display, switch: true }`. Any other compound of that shape gets the - same treatment. -- **New operations carry ids:** + host will assign. So a create-then-act compound becomes **one command** carrying + both halves. `create-and-switch-view` sends `[CreateView, SwitchView]` today and + becomes `CreateView { display, switch: true }`. Any other compound of this shape + gets the same treatment. +- **New operations are born with ids:** - `RenameCategory { category, to: display }`; - `RenameItem { category, item, to: display }`; - - **[Amendment]** a group rename by group id and new display; - - `ReplaceFormula { id, was, raw }`; - - `RenameRecordCoordinate`; + - `ReplaceFormula { id, was, raw }`, which km71 already landed with an interim id + (below); + - fdk1's `RenameRecordCoordinate`, which landed with name operands (below); - the removes (`RemoveCategory`, `RemoveItem`, `RemoveFormula`, `DeleteView`). -- **Older variants move to ids too** (`SetAxis`, `SetPageSelection`, `HideItem`, …, - including `SetCell { view, … }` and `ClearCell { view, … }`), one whole variant at a - time. +- **Older variants** (`SetAxis`, `SetPageSelection`, `HideItem`, …) **must also move to + ids**. Their operands are references, and a reference is now an id. They may move one + whole variant at a time. That includes `SetCell { view, … }` and + `ClearCell { view, … }`, which improvise-wpxr (PR #281, merged) gave a view named by + its name, like every other view-scoped variant (improvise-lltt). - **Projections carry id and display together:** - `CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis }`; - - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows: - the target's display with its id in parentheses (`Profit (profit) = Revenue - - Cost`), then the expression over display names, with an id wherever a display is - ambiguous. **[Amendment]** A display equal to another thing's id counts as - ambiguous, so the panel never shows a bare name that the ids-first rule (clause 12) - would bind elsewhere. `stored` is the id-form text the file carries, and - `ReplaceFormula`'s `was` is compared against it. So a rename of `Revenue` elsewhere - changes what the panel shows, but does not refuse an edit of `Profit`; - - `ViewSummary { id, display, … }`; - - **[Amendment]** the grid frame's group entries (`FrameGroup`) carry the group's id - and display. + - `FormulaCache` rows `{ id, stored, display }`. `display` is what the panel shows + (answer 19): the target's display name with its id in parentheses, then the + expression over display names, with an id wherever a display name is ambiguous + (ids are unique, answer 22). `stored` is the id-form text the file carries, and it is what + `ReplaceFormula`'s `was` is compared against. So a rename of `Revenue` elsewhere + changes what the panel shows but does not refuse an edit of `Profit`, which is + the "avoid glitches" of answer 3. What makes that hold for an edit already open + is clause 12; + - `ViewSummary { id, display, … }`. `AxisFrame` labels stay display strings. They gain ids when a consumer needs them. -- **`ViewState` keys by id**: the cursors, `expanded_cats` and the view history - (`view_back_stack` / `view_forward_stack`). **[Amendment]** So do `active_view` - itself and `Workbook::active_view` (improvise-p003). A rename then orphans nothing. -- **[Amendment] A `_Measure` formula-target row** (`ItemHandle::FormulaTarget`) is - removed by `RemoveFormula` and renamed by a display rename of the measure - (improvise-i0n7). -- **[Amendment] CheckedBatch's created-record exemption is removed** (improvise-p003). - It exempts a record that neither resolves nor is claimed, for frames like - `[AddItem(East), SetCell(East)]`. A frame cannot address what it creates, so nothing - produces such a frame any more. +- **`ViewState` keys by id**: the cursors, `expanded_cats`, and the view history + (`view_back_stack` / `view_forward_stack`). Answer 1 makes this natural rather than a + migration of its own. A view or category rename then orphans nothing. ### 7. ADR 001: an id that no longer resolves is gone -A command naming an id that does not resolve is rejected before anything is applied, -as `Rejection::Gone(Named)`: -- `Named` is `Category`, `Item`, `Formula` or `View`, and carries the id's text. - **[Amendment]** It gains `Group`. -- The rejection is **structural**: force never creates structure. +A command naming an id that does not resolve is rejected before anything is applied, as +`Rejection::Gone(Named)`: +- `Named` is one of `Category`, `Item`, `Formula` or `View`, and carries the id's text. +- It is **structural**: force never creates structure. The check has two halves, like the record tier: - **Before the frame:** an exhaustive `ids_named` feeds `Checked::against`. - **Within the frame:** an **id arm in `broken_chain`** refuses `[RemoveFormula(f), ReplaceFormula(f)]` (improvise-gkyo). -Per-entity versions are not added. +Per-entity versions are not added (answer 10). -### 8. ADR 002: `RecordHash` is defined over ids +### 8. ADR 002: `RecordHash` is now defined over ids -`RecordHash` hashes the explicit serialization of the sorted `(CategoryId, ItemId)` -text pairs: the same function over different strings. **[Amendment]** The pairs are -`(CategoryId, ItemHandle)` spelled as id text. Ids are unique across kinds, so the hash -stays canonical. A record is the key minus `_measure`, so a `FormulaTarget` coordinate -is never part of a record. +Following answer 3, `RecordHash` hashes the explicit serialization of the sorted +`(CategoryId, ItemId)` text pairs, the same function over different strings. +- **ADR 002's decision is unchanged in substance.** It still hashes an explicit + serialization of a normalized record, with a fixed key and a width argued for. +- **Its prose will be out of date once this lands.** ADR 002 says `CellKey::new` + "sorts coordinates by category name", and a record is spelled in names. Both become + ids. ADR 002 is **not edited here**. Updating that wording belongs to the + implementation PR. -ADR 002's decision is unchanged in substance. Its prose ("sorts coordinates by -category name", records spelled in names) is out of date once this lands, and the -implementation PR updates it. +**What changes in behaviour:** a display rename no longer changes any `RecordHash` or +bumps `layout_version`, because `view_shape` hashes ids too. So a client's surrogate +survives a rename, and its in-flight edit still lands. A **coordinate** change, as in +fdk1 or a records-mode move, still produces a different record and a different hash, as +it should: it *is* a different record. -**In behaviour:** a display rename changes no `RecordHash` and bumps no -`layout_version`, because `view_shape` hashes ids too. So a client's surrogate survives -a rename, and its in-flight edit lands. A **coordinate** change, as in fdk1 or a -records-mode move, produces a different record and a different hash, because it is a -different record. +### 9. Headers that change meaning: the edge case answer 6 asks to document -### 9. Headers that change meaning +Under answer 6, `header_decl` reads a trailing `(token)`, with or without a space before +it, as the id. **It applies only to new-version files** (answer 15, Q12(a)). An +old-version file is read as it always was, so its headers never split. In a new-version +file the writer never produces an ambiguous header, because it pipe-quotes any display +that would split. So the edge cases below arise **only when a new-version file is +written by hand**. The table shows what the header rule would do to lines that exist +in old files today. That is why old files do not get the rule. -In a new-version file, `header_decl` reads a trailing `(token)`, with or without a -space before it, as the id. **Old-version files never get this rule**, so their headers -never split. The writer pipe-quotes any display that would split, so the cases below -arise only in a new-version file written by hand. They show why old files keep the -whole-line reading. - -| header line | whole-line reading (old version) | under `header_decl` (new version) | +| header line | whole-line reading (today) | under `header_decl` | |---|---|---| | `## View: Default(default)` | `Default(default)` | display `Default`, id `default` | | `## View: \|Default (default)\|(default)` | pipes literal | display `Default (default)`, id `default` | | `## View: Sales (copy)` | `Sales (copy)` | **display `Sales`, id `copy`** | | `## Category: Region(code)` | `Region(code)` | **display `Region`, id `code`** | -| `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** (**[Amendment]** refused: an id may not start with a digit) | +| `## Category: Budget (2024)` | `Budget (2024)` | **display `Budget`, id `2024`** | | `## Category: Time Period` | `Time Period` | unchanged; id `time_period` generated | -The app itself writes such headers: `improvise import` on a CSV with headers like -`Region(code)` saves `## Category: Region(code)` today (improvise-80rm). +**The documents that change meaning** are those with a view or category header ending in +a parenthesised run of letters, digits, `_` or `-`. **The app writes such headers +itself**: `improvise import` on a CSV with headers like `Region(code)` or `Size(cm)` +saves `## Category: Region(code)` today (improvise-80rm). Any view someone named +`Sales (copy)` does the same. -If old files got the rule, that category would load as display `Region`, id `code`. -Every reference in the file spells `Region(code)`, which resolves to neither, so the -axis lines and data keys would not resolve, and nothing would be drawn. Because old -files keep the whole-line reading, none of this happens. On the first save the writer -pipe-quotes such a display: `## Category: |Region(code)|(regioncode)` -(**[Amendment]** `(region_code)`). +**What it would cost if old files got the new rule** (the case answer 15 rules out): +- The category is renamed on load: display `Region`, id `code`. +- Every reference in that old file spells `Region(code)`, which resolves to neither the + display nor the id. +- So the axis lines and every data key for that category do not resolve. +- Today's loader would silently drop the axis lines and orphan the data, so nothing is + drawn. -### 10. The file version alone picks the regime +**Decided (answer 15):** old-version files keep the whole-line reading, so none of +this happens to a file that exists today. On the first save the writer pipe-quotes +such a display, for example `## Category: |Region(code)|(regioncode)`, so the new file +cannot split it either. +### 10. The file version is explicit, and it alone picks the regime + +Following answers 15 and 25: - **The version is already explicit.** The first line of every `.improv` file is a mandatory version line, matched as an **exact literal** - (`version_line = { "v2025-04-09" ~ NEWLINE ~ blank_lines }`), and `format_md` writes - it. -- **The new format gets a second literal.** The grammar's `file` rule gets one arm per - literal, each with its own header and reference rules. The literal's spelling is the - implementation's choice. -- **The regime is chosen only by the first line**, never inferred from the body. The - old literal means no ids anywhere. The new literal means ids on every declaration. -- **A file whose first line is neither literal is refused**, as today. -- **Every save writes the new literal**, so an old file changes version on its first - save. + (`version_line = { "v2025-04-09" ~ NEWLINE ~ blank_lines }` in `improv.pest`), and + `format_md` writes it. Nothing is added to make the version explicit. What is added is + a second literal. +- **The new format gets a new literal.** The grammar's `file` rule gets one arm per + literal, and each arm uses its own header and reference rules. The new literal's + exact spelling is left to the implementation. +- **The regime is chosen only by the first line**, never inferred from what the body + looks like. So the old literal means no ids anywhere, and the new literal means ids + on every declaration (answer 21). +- **A file whose first line is neither literal is refused**, as it is today. +- **Every save writes the new literal** (answer 12). So an old file changes version on + its first save. ### 11. Migration 1. **Remove legacy JSON** (improvise-tt22). Done, in PR #224. -2. **Types:** the id newtypes over `Symbol`, the `display` fields, `FormulaId` and - `ViewId`. **[Amendment]** Also `GroupId`, a display on `Group`, and `ItemHandle`, - which the re-key needs for `CellKey`. +2. **Types:** the id newtypes over `Symbol`, the `display` fields, and `FormulaId` and + `ViewId`. 3. **Re-key storage by id** (clause 2). This is the wide, mechanical step, with - display→id resolution moved to every input edge. **[Amendment]** The virtual - categories' keys move from `_Index`, `_Dim` and `_Measure` to `_index`, `_dim` and - `_measure` here, in the model, on the wire, in `ViewState` and in the browser client. -4. **Persistence:** the new literal (clause 10), clause 5's grammar, the loader and its - refusals, slugify, the writer, and the round-trip properties with a formula-line - strategy. + display→id resolution moved to every input edge. +4. **Persistence:** the new version literal (clause 10), the grammar in clause 5, the + loader and its refusals, slugify, the writer, and the round-trip properties with a + formula-line strategy. 5. **Projections and `ViewState`** (clause 6). 6. **Vocabulary:** the id-addressed variants, `Rejection::Gone`, `ids_named` and `broken_chain`'s id arm. -7. **[Amendment] Clause 12's client-side capture and resolver.** No display-matching - code exists yet, so the resolver is built ids first from the start. - -The implementation epic (improvise-cq1q) orders these steps to keep `main` shippable -after each one: the display field before the re-key, and projection ids before the -re-key. ### 12. A formula edit's names are fixed when the editor opens -**The problem.** The formula panel shows display names, and the user edits that text. -If those names were mapped to ids at **commit**, a concurrent rename would break the -edit one of two ways: -- the typed name no longer resolves, and the edit is refused; +**This clause is the ADR's own reading, not a user answer.** It closes a contradiction +the adversarial review of `4830fa30` found between clause 6 and the km71 section, which +is also ChatGPT Codex's P1 on `29ad14f4`. + +**The problem.** The formula panel shows display names (answer 19), and the user edits +that text. If the host maps those names to ids at **commit**, a concurrent rename breaks +the edit one of two ways: +- the name the user typed no longer resolves, so the edit is refused, which contradicts + clause 6's "does not refuse an edit of `Profit`"; - or the name now belongs to something else (`Revenue` renamed away, another item - renamed to `Revenue`), and the edit silently binds to a different thing. + renamed to `Revenue`), so the edit silently binds to a different thing. That is the + glitch answer 3 exists to prevent. -**The rule.** The display→id mapping is **captured when the formula editor opens**, and -the commit resolves the edited text through it. This is the cell editor's -capture-at-open rule: `OpenedCell` (`improvise-protocol/src/client.rs`) records the -record, its version, the document generation and the view at open, and the commit -answers for those. -- The mapping covers every display the panel showed in that formula's text, including - the target's. -- `was` is compared against the stored id-form text (clause 6), so a concurrent - *replace* of the formula is still seen. +**The rule.** The display-name-to-id mapping is **captured when the formula editor +opens**, and the commit resolves the edited text through that captured mapping. This +is the capture-at-open rule the cell editor already follows: `OpenedCell` in +`improvise-protocol/src/client.rs` records the record, its version, the document +generation and the view at open, and the commit answers for those rather than for +whatever the document holds by then. +- The mapping covers every display name the panel showed in that formula's text, + including the target's. +- A name the user types that was **not** in the captured mapping is new to the edit. It + is resolved against the document at commit, as any typed name is. If it does not + resolve, the edit is refused and says which name. - A captured id that no longer exists at commit is `Rejection::Gone` (clause 7). -- **A typed name that was not in the captured mapping** is resolved against the - document at commit, and refused, naming it, if it does not resolve. +- `was` is still compared against the stored id-form text (clause 6), so a concurrent + *replace* of the formula is still seen. -**[Amendment] The client does the resolving. Interactive input may match a display or -an id as long as the pick is unambiguous, and ids win when it is not:** -- **The client holds the mapping**, as it holds `OpenedCell`. The TUI and the browser - share one resolver in `improvise-protocol`. -- **A name new to the edit** is resolved by the client against its id-bearing - projection (clause 6), **ids first, then displays**. Ids are unambiguous, so they are - preferred wherever there is ambiguity, now and once an autocomplete exists. - - A typed name equal to an id names that id's thing, even where it is also another - thing's display. - - Otherwise a display that exactly one thing in scope has names that thing, and a - display matching several things is refused with a list of their ids. - - A name that resolves to nothing is refused, and the refusal names it. - - Ids are matched exactly, with no case folding. -- **Typing prefers ids and previews displays.** A view-layer autocomplete turns typed - text into ids before commit, preferring a match on an id, and shows the matched - thing's display as a preview. The model layer refers to ids only. This ADR does not - specify the autocomplete further. -- **The wire carries only id-form text.** `ReplaceFormula.raw` and `AddFormula.raw` are - in the file's id form, and the host never resolves a display. The host parses ids, - and one that no longer exists is `Rejection::Gone`. +**Required regression test.** Open a formula editor on `Profit = Revenue - Cost`. From +another client, rename the item `Revenue` to `Sales` and rename a different item to +`Revenue`. Commit the unedited text. The formula must still reference the original +item's id, and the commit must not be refused. The test must fail under +commit-time resolution, which binds to the new `Revenue`, and it must fail under a +refusal on rename. Both are the rivals this clause rules out. -**[Amendment] What ids-first costs.** Suppose item A is displayed `East` with id `east`, -and item B is displayed `east`, so slugify gave it `east-1`. Typing `east` names **A**. -B is reached by typing `east-1`. A user who typed `east` meaning B is bound to A -without being told, unless the preview shows it. That is why the panel shows B as -`east-1` and never as a bare `east`. +## What this means for you -**Required regression test.** -1. Open a formula editor on `Profit = Revenue - Cost`. -2. From another client, rename `Revenue` to `Sales`, and rename a different item to - `Revenue`. -3. Commit the unedited text. +### improvise-km71 — editing a formula (landed as an interim) -The stored formula must reference the original item's id, and the commit must not be -refused. The test must fail under commit-time resolution and under a refusal on -rename. +km71 landed in PR #214, before this ADR, and was built to be revised by it +(`context/code/core.md` §18): +- **What landed.** `ModelCommand::ReplaceFormula { id: FormulaId, was, raw }`. + `FormulaId` is opaque, with private fields `(target, target_category)`. + `Model::replace_formula` writes in place, keeping the position. It refuses a new + target that another formula owns, and it refuses any target change outside + `_Measure` (improvise-kb1d). `was` is the text the editor opened on. It makes a + concurrent *replace* visible, which even a stable id cannot do, so **`was` stays**. +- **What this ADR changes.** + - `FormulaId`'s fields become one id. The signatures that carry `FormulaId` do not + change, which is what making it opaque was for. + - A target change becomes a **display** rename of the target. The id does not move, + and nothing that references the target is rewritten. So kb1d's refusal outside + `_Measure` can be lifted, and so can the dangling-reference report core.md §18 + describes. + - The panel shows `Profit (profit) = Revenue - Cost` (answer 19). The user edits that + text. Its display names are resolved to ids through the mapping captured when the + editor opened (clause 12), not through the names the document holds at commit, and + `was` is compared against the stored id-form text (clause 6). The parenthesised id is not the user's to + change, since the host assigns ids (answer 14). An edit that changes it is refused. + This is the ADR's reading, not a user answer. +- **Rejections.** Another client's deletion gives `Rejection::Gone(Formula(id))`. A + deletion earlier in the same frame gives the broken-chain refusal. -**[Amendment]** Step 3 becomes: edit something that is not a name -(`Revenue - Cost * 1`) and commit. Committing the unedited text cannot tell a correct -capture from no capture at all, because a client may treat unchanged text as a no-op -without resolving any name (improvise-hexb). +### improvise-fz77 — renaming a category or item -**[Amendment] Required test for ids-first.** With items A (`East`, `east`) and B -(`east`, `east-1`), type `east` into an editor, in text not captured at open: -- it must bind A, and the test must fail under displays-first; -- `east-1` must bind B; -- a display that is nobody's id must still resolve by display. +- **The operations.** `RenameCategory { category, to }` and + `RenameItem { category, item, to }`. The panel rows carry the ids. +- **It is now cheap** (answers 1 and 2): one `display` write. No `CellKey`, axis map, + page selection, hidden or collapsed set, or formula text is rewritten, because all of + them hold ids. No `RecordHash` or `layout_version` moves, so in-flight edits survive. +- **What it still owes:** + - **Uniqueness.** Refuse a display name already used in the same scope. + - **Dirtiness.** Mark the effect so the grid labels and panels are re-projected + (gotcha 35). The labels are display strings. + - **Tests.** Assert that the *labels* changed and the *surrogates did not*. That is + the property this design exists for. +- **It depends on step 3 of the migration**, the re-keying. Until then there is no cheap + rename to build on. -A fixture where no display equals an id proves nothing. +### improvise-fdk1 — renaming a records coordinate -## What this means for the tickets +fdk1 landed in PR #221, before this ADR. What exists on main: +- `ModelCommand::RenameRecordCoordinate { record: RecordHash, category: String, item: + String }`, which moves a whole record to a new coordinate and creates the item if need + be; +- `CellId::Coordinate { record: RecordHash, category: String }`, the surrogate a + records-mode coordinate cell carries. -**improvise-km71 (editing a formula), landed as an interim.** -`ReplaceFormula { id: FormulaId, was, raw }` writes in place. It refuses a new target -that another formula owns, and any target change outside `_Measure` (improvise-kb1d). -`was` makes a concurrent *replace* visible, which even a stable id cannot, so `was` -stays. +**What this ADR changes**, as it does for km71's `FormulaId`: +- `category` becomes a `CategoryId`, in the command and in `CellId::Coordinate`. +- `item` becomes an `ItemRef`: `Existing(ItemId)` or `New(display)`. That is the + create-by-display half of clause 6, since the host assigns the new item's id. +- Unlike a display rename, this **changes a coordinate**, so the record's `CellKey`s and + `RecordHash` change. It is a different record, and that does not change. -This ADR changes three things: -- **`FormulaId`'s fields become one id.** The signatures do not change. -- **A target change becomes a display rename of the target.** The id does not move, - so kb1d's refusal and core.md §18's dangling-reference report can be lifted. -- **The panel's text is resolved through clause 12's captured mapping.** The - parenthesised id is not the user's to change, so an edit that changes it is - refused. +## Questions as asked, and where each now stands -Rejections: -- Another client's deletion gives `Rejection::Gone(Formula(id))`. -- A deletion earlier in the same frame gives the broken-chain refusal. +The first draft asked Q1–Q9. Revision 1 added Q10 and Q11, revision 2 added Q12–Q14, +revision 3 added Q15–Q17, revision 4 added Q18–Q21 and revision 5 added Q22. **All +are answered.** -**improvise-fz77 (renaming a category or item).** It adds `RenameCategory` and -`RenameItem`. A rename is one `display` write, and no key, set, formula text, -`RecordHash` or `layout_version` moves. It still owes three things: -- refuse a display already used in the scope; -- mark the effect so labels and panels re-project (gotcha 35); -- tests asserting that the *labels* changed and the *surrogates did not*. +| # | question | my recommendation | status | +|---|---|---|---| +| 1 | Integer or word ids? | first draft: words; revision 1: integers only | **answered**: interned symbols (answer 4), generated by slugify (answers 8, 16). Words. | +| 2 | Formula id placement | F1, `- Profit(3) = …` | **answered: F1** (answer 5) | +| 3 | Headers in old files | first draft: accept the rare break; revision 1: gate on version | **answered**: the space is optional (answer 6), and old files keep the whole-line reading (answer 15, Q12) | +| 4 | Do `## Data` keys carry ids? | no | **answered: yes, ids** (answer 7), reversing the recommendation | +| 5 | Group ids? | not now | **answered: not now** (answer 9) | +| 6 | Per-entity versions? | not in this ADR | **answered: not in this ADR** (answer 10) | +| 7 | `RecordHash` over ids? | no, keep names | **answered: over ids** (answer 3), reversing the recommendation | +| 8 | Duplicate id: repair or refuse? | revision 2: refuse | **answered: refuse the load** (answer 11) | +| 9 | Reading of "opt-in" | read optional, write always | **answered: yes** (answer 12) | +| 10 | Id reuse after a reload | revision 2: (b) accept reuse | **answered: (b)** (answer 13) | +| 11 | Who assigns ids? | the host | **answered: the host** (answer 14) | +| 12 | Old files and the new header rule | (a) no | **answered: (a), with the version marked unambiguously** (answer 15) | +| 13 | Slugify details | all generation; keep case; keep `_` `-`; strip non-ASCII; `-` | **answered** (answer 16): all generation, **lower case only** (reversing the recommendation), keep both, strip, `-` | +| 14 | Formula text: ids or display names? | ids in the file | **answered**: ids in the file (answer 3); the panel is Q16 | +| 15 | Slugify order | fold, then strip | **answered: casefold, space → `_`, strip** (answer 18); the space character changed from `-` | +| 16 | What the formula panel shows | none | **answered: display names, target shown as `Profit (profit)`, ids where a name is ambiguous** (answer 19) | +| 17 | Capitals in a hand-written id | (a) no | **answered: no** (answer 20) | +| 18(a) | Does a run of spaces collapse? | collapse | **answered: collapse**, plus a numeric suffix on a duplicate, and ids on everything or nothing (answer 21) | +| 18(b) | What "casefold" means | full case folding | **answered: full Unicode case folding** (answer 26) | +| 19 | Does an id disambiguate formula text? | (a) item ids unique in the document | **answered: ids are globally unique** (answer 22), stronger than (a) | +| 20 | Virtual category ids | (a) lower-case ids, capitalised displays | **answered: (a)**, and no other id may begin with `_` (answer 23) | +| 21 | An old file naming an undeclared item | (a) declare it at load | **answered: (a)** (answer 24) | +| — | "A new file version" (a Consequences line) | — | **answered**: the version must be explicit (answer 25); it already is, and clause 10 adds a second literal | +| 22 | The suffix's spelling, and interactive collisions | (a) plain number; (b)(i) suffix there too | **answered**: a plain number (answer 27); any collision is suffixed, across kinds, from a `gensym`-style increment-only counter (answer 28), and only a generated id is ever suffixed, since a hand-written one is canonical (answer 29) | -It depends on migration step 3. +## Still open -**improvise-fdk1 (renaming a records coordinate).** -`RenameRecordCoordinate { record, category: String, item: String }` and -`CellId::Coordinate { record, category: String }` exist on main. This ADR changes -three things: -- `category` becomes a `CategoryId`. -- `item` becomes `ItemRef::{Existing(ItemId), New(display)}`. -- The rename **changes a coordinate**, so the record's `CellKey`s and `RecordHash` - change. It is a different record. +Nothing is open among Q1–Q22, and answer 29 settles the one case it raises: two +identical hand-written ids are refused at load. **The user approved revision 7. +Revision 8 adds clause 12, which is the ADR's own reading, and the user approved it +("looks good, approved", 2026-09-25 00:17:10Z).** Three follow-ups from the same +review stay as tickets, not ADR text: improvise-le4p, improvise-m56i and +improvise-p003. ## Consequences **Good.** - Edit, rename and replace are distinct, addressed operations. -- A rename is one display write, and in-flight edits survive it. +- A rename is one display write: it moves no key, surrogate or basis tier, and in-flight + edits survive it. - `## Data` keys are unique by construction. -- km71's `FormulaId` becomes a real id by changing one struct. +- km71's `FormulaId` becomes a real id by changing one struct, and kb1d's refusal can be + lifted. **Bad.** -- Re-keying storage is wide. Every name-taking API moves to ids. -- Old files change version, and their data keys change case, on the first save. +- Re-keying storage is wide: every name-taking API moves to ids, and display→id + resolution moves to the input edges. +- A second version literal (clause 10), and every old file changes version on its first + save. - `Rejection` loses `Copy`. -- `Formula::raw` is no longer verbatim. The file carries formula text over ids. -- The formula panel is a second rendering of formula text, so display names typed - there must be resolved back to ids (by the client, under the amendment). -- ADR 002's prose needs a wording update. -- **[Amendment]** Scripts that name things by display break and must be rewritten - over ids. +- `Formula::raw` is no longer verbatim: the file carries formula text over ids. +- The formula panel is a second rendering of formula text, over display names, so the + host must resolve display names typed there back to ids (answer 19). +- Ids are lower case, so an old file's data keys change spelling on its first save. +- ADR 002's prose needs a wording update when this lands. -**Not addressed.** Per-entity versions, ids in `AxisFrame` labels and -`CellId::Derived`, and (under the accepted text) group ids. - -## Open questions - -None. +**Not addressed.** Per-entity versions (Q6), group ids (Q5), and ids in `AxisFrame` and +`CellId::Derived`. diff --git a/context/code/core.md b/context/code/core.md index 533b02b5..4655a987 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1714,7 +1714,8 @@ underscore-collapsing step exists. A trailing run survives, so `East ` slugs to `east_`. Only *leading* underscores are removed (step 4). **ADR 007's proposed amendment changes these rules, and this code predates it.** -The amendment (PR #326) is not yet approved. Its clause 4: +The amendment (`context/adr/007-stable-identities-amendment-1.md`, PR #326) is not +yet approved. Its clause 4: - every character outside `a`–`z`, `0`–`9` and `-` becomes `_` instead of being stripped (not only U+0020), so `Region(code)` slugs to `region_code`, while `Q1-2024` keeps its `-`; @@ -1727,7 +1728,7 @@ The amendment (PR #326) is not yet approved. Its clause 4: This section, `slugify`, and the tests `slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and `slugify_keeps_an_underscore_the_display_already_had` describe the earlier rule -above. They change in a follow-up once the amendment is approved. The ADR's +above. They change in a follow-up once the amendment is approved. The amendment's clause 4 lists every difference. **The allocator records every id it admits and releases none.** That is the diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index 3ae5941d..55c91ba1 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -47,6 +47,7 @@ const SECTIONS = [ ["context/adr/004-faces-and-themes.md", "adr/004-faces-and-themes", "ADR 004 — Faces"], ["context/adr/005-exact-decimal-arithmetic.md", "adr/005-exact-decimal-arithmetic", "ADR 005 — Exact decimal arithmetic (proposed)"], ["context/adr/007-stable-identities.md", "adr/007-stable-identities", "ADR 007 — Stable identities"], + ["context/adr/007-stable-identities-amendment-1.md", "adr/007-stable-identities-amendment-1", "ADR 007, amendment 1 (proposed)"], ], }, { From 1df5f99e356ea5abcd24a9960860863d4087d8e5 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:50:15 -0700 Subject: [PATCH 22/27] docs(adr): equal displays in a scope are shown with a gapless " #2" suffix in id order (improvise-cq1q.1) Replaces the "one permitted exception" to display uniqueness for a formula beside a stored measure of the same name: Profit and Profit #2. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- .../adr/007-stable-identities-amendment-1.md | 30 +++++++++++++++---- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index c4bf95a1..5930f4c4 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -17,6 +17,7 @@ behaviour, the accepted rule binds until the amendment is approved: | 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill` | | 1 | the virtual categories are declared and named in the file (`## Category: _Measure`, `_Measure: column`) | they are defined by criteria with hardcoded ids and displays, never declared, and referred to by id (`_measure`) | | 1 | `measure_item_names` merges a formula into a stored measure of the same name | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | +| 2 | a display already in use in a scope is refused | equal displays in a scope are shown with a gapless ` #2`, ` #3`, … suffix in id order | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended before a leading digit | @@ -77,10 +78,8 @@ cost`); a dimension target is rare, for a derived dimension. `Profit`), adding `Profit = Revenue - Cost` creates a second measure with a fresh id, `profit-`, by the ordinary collision rule. The stored measure keeps `profit` and its data; removing the formula removes only `profit-`. - - **Conflict with display uniqueness:** both are displayed `Profit` in `_measure`'s - scope. This is the one permitted case; the two are told apart by id (the formula - panel shows ids where a display is ambiguous, and a typed `Profit` matching both is - refused with both ids listed, clause 12). + - Both are named `Profit` in `_measure`'s scope, so they are shown as `Profit` and + `Profit #2` (clause 2's suffix rule). - **A target outside `_Measure` is a real item** with its own `ItemId`, and its formula has a separate `FormulaId`: a formula `Tax` in `CustomCat` gets item id `tax`, then formula id `tax-`. @@ -96,6 +95,27 @@ cost`); a dimension target is rare, for a derived dimension. hold. - **A collapsed entry names its group by the group's id**, and `Group` gains a `display`. +- **Displays stay unique within a scope by suffixing.** The accepted text refuses a + display already in use in the scope; a formula's measure beside a stored measure of + the same name (clause 1) cannot be refused. So when several things in one scope have + the same display, they are shown with a suffix: the first keeps the bare display, and + the rest get ` #2`, ` #3`, … (a space, then `#` and the number) as a gapless + sequence in the order their ids sort. + - The order is the ids' string order, the order `CellKey` already sorts ids in. So + `profit`, `profit-1` and `profit-2` show as `Profit`, `Profit #2` and `Profit #3`, + and `profit-10` sorts before `profit-2`. + - Gapless: if the thing shown as `Profit #2` is removed, `Profit #3` becomes + `Profit #2`. + - The suffix is computed when a display is shown, from the things that share it, and + is never stored or written to a file. + - A suffixed display is a display like any other: interactive input that types + `Profit #2` picks that thing (clause 6), and the ids stay the unambiguous name. + - Old files cannot hold two things with one name in a scope, so loading one adds no + suffix. + - A display can also be literally `Profit #2`. Beside two things displayed `Profit`, + it shows the same text as the suffixed one; the two stay distinct by id, and typing + `Profit #2` then matches both, so the pick is ambiguous and is refused with both ids + listed (clause 12). ### 4. How an id is made @@ -271,7 +291,7 @@ therefore shows B as `east-1`, never as a bare `east`. ## Consequences - Scripts that name things by display break and must be rewritten over ids. -- Two measures can share a display; every place that shows displays falls back to ids. +- Two measures can share a display, and are shown as `Profit` and `Profit #2`. - Old files' `_Measure` declarations and hyphen- or digit-led names change on first save. - `id.rs` and its tests need the follow-up in clause 4. From 46c8a967bc6fee5a8bba0649757cb77e5afcfecd Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 09:52:14 -0700 Subject: [PATCH 23/27] docs(adr): a display suffix skips a number a literal display already holds (improvise-cq1q.1) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities-amendment-1.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index 5930f4c4..dc8ff825 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -112,10 +112,11 @@ cost`); a dimension target is rare, for a derived dimension. `Profit #2` picks that thing (clause 6), and the ids stay the unambiguous name. - Old files cannot hold two things with one name in a scope, so loading one adds no suffix. - - A display can also be literally `Profit #2`. Beside two things displayed `Profit`, - it shows the same text as the suffixed one; the two stay distinct by id, and typing - `Profit #2` then matches both, so the pick is ambiguous and is refused with both ids - listed (clause 12). + - A suffix never repeats a display already in the scope. If something is literally + displayed `Profit #2`, the shared `Profit`s take the next free numbers instead: + `Profit`, `Profit #3`, `Profit #4`, still in id order. "Gapless" counts only the + suffixes actually assigned, so a number taken by a literal display is skipped, + like an id collision candidate that is already held. ### 4. How an id is made From c10eca98e0f4d797f8207a1ab0355c5028e4065c Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 10:04:28 -0700 Subject: [PATCH 24/27] docs(adr): keep the _Measure section for stored measures, and mark ADR 007's amendment accepted (improvise-cq1q.1) The Opus review of 46c8a967: a new-version file had nowhere to declare its stored measures. The ## Category: _Measure section stays and lists them with displays and ids; its header declares nothing. Clause 6 says input text means files and command-language text, distinct from interactive typing. The sort note says plain string order, not numeric. The user approved the amendment on PR #326 (2026-10-09), so its Status, the accepted ADR's pointer, core.md section 20 and the doc-site title no longer say proposed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- .../adr/007-stable-identities-amendment-1.md | 34 ++++++++++--------- context/adr/007-stable-identities.md | 4 +-- context/code/core.md | 8 ++--- scripts/build-doc-site.mjs | 2 +- 4 files changed, 25 insertions(+), 23 deletions(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index dc8ff825..6f11596c 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -5,24 +5,23 @@ improvise-cq1q) · **Builds on:** ADR 001, ADR 002 ## Status -**ADR 007 is accepted** (PR #210, 2026-09-25), and `007-stable-identities.md` stands. **This -amendment is proposed and not yet approved** (PR #326). This file states only what the -amendment changes or adds, numbered by the accepted ADR's clauses; everything else in -the accepted ADR is still true. Where the amendment conflicts with an accepted rule or an existing -behaviour, the accepted rule binds until the amendment is approved: +**Accepted** (2026-10-09, PR #326). It amends ADR 007 (accepted 2026-09-25, PR #210, +`007-stable-identities.md`). This file states only what the amendment changes or +adds, numbered by the accepted ADR's clauses; everything else in ADR 007 is still +true. Where the amendment replaces an ADR 007 rule or an existing behaviour: | clause | accepted rule or existing behaviour | the amendment | |---|---|---| | 1 | groups have no id | a group has an id and a display, and its id behaves like every other id | | 1 | `_index`, `_dim`, `_measure` are the only ids beginning with `_` | a leading `_` is reserved for virtual ids in general, including the virtual views `_records` and `_drill` | -| 1 | the virtual categories are declared and named in the file (`## Category: _Measure`, `_Measure: column`) | they are defined by criteria with hardcoded ids and displays, never declared, and referred to by id (`_measure`) | +| 1 | the virtual categories are declared and named in the file (`## Category: _Measure`, `_Measure: column`) | they are defined by criteria with hardcoded ids and displays and referred to by id (`_measure`); the `## Category: _Measure` header lists the stored measures but declares nothing | | 1 | `measure_item_names` merges a formula into a stored measure of the same name | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | | 2 | a display already in use in a scope is refused | equal displays in a scope are shown with a gapless ` #2`, ` #3`, … suffix in id order | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended before a leading digit | | 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` | a hand-written id obeys every syntax rule of a generated id | -| 6 | display names typed into minibuffers are turned into ids at the input edge | only CSV import and the `.improv` loaders read display names in input text; command-language text names things by id | +| 6 | display names typed into minibuffers are turned into ids at the input edge | only CSV import and the `.improv` loaders read display names in files and command-language text, which names things by id; interactive input matches a display or an id | | 12 | a typed name new to a formula edit is resolved against the document at commit | the client resolves it, ids first, and the wire carries only id-form text | | 12 | the regression test commits the unedited text | the test edits a non-name token and commits | @@ -58,10 +57,11 @@ becomes the group's id. and `_index` and `_dim` hold the record positions. Their ids and displays are hardcoded in the application, so a file never declares one, and a line that refers to one spells its id: `_measure: column`, `region=east, _measure=revenue = 1200`. -- **Conflict with old files:** they declare and name `_Measure`. The loader recognises - `_Measure`, `_Index` and `_Dim` by their hardcoded displays, reads an old - `## Category: _Measure` section only for the stored measures it lists, and the first - save writes `_measure`. +- **The stored measures are still listed in a `## Category: _Measure` section**, each + with its display and id (`- Revenue(revenue), Cost(cost)`). Its header names the + hardcoded virtual category and is not a declaration of it. The loader recognises + `_Measure`, `_Index` and `_Dim` by their hardcoded displays, and references are + written `_measure`. - **Records mode's synthetic positions are values, not ids.** The row coordinate under `_index`, and by the same reasoning the column coordinate under `_dim`, are values of those virtual categories. They take no part in id uniqueness and cannot collide with @@ -101,7 +101,8 @@ cost`); a dimension target is rare, for a derived dimension. the same display, they are shown with a suffix: the first keeps the bare display, and the rest get ` #2`, ` #3`, … (a space, then `#` and the number) as a gapless sequence in the order their ids sort. - - The order is the ids' string order, the order `CellKey` already sorts ids in. So + - The order is the ids' plain string order, the order `CellKey` already sorts ids + in, not the counter's numeric order. So `profit`, `profit-1` and `profit-2` show as `Profit`, `Profit #2` and `Profit #3`, and `profit-10` sorts before `profit-2`. - Gapless: if the thing shown as `Profit #2` is removed, `Profit #3` becomes @@ -164,7 +165,7 @@ and the load is refused naming both. Stable identity across copies of a file is promised (improvise-8tk5). **Built code.** `crates/improvise-core/src/model/id.rs` implements the accepted steps -(`context/code/core.md` §20). After approval it must replace rather than strip, collapse +(`context/code/core.md` §20). A follow-up must replace rather than strip, collapse `_`, trim a trailing `_` and both ends' `-`, prepend `n`, make `claim` refuse what a generated id could not be, and add the virtual views and `IdKind::Group`. @@ -222,13 +223,14 @@ Under the amendment: ### 6. Display names and the wire - **Only CSV import and the `.improv` loaders turn display names in input text into - ids.** They read text people wrote. An old file's names are resolved there as a - migration. + ids**, where input text means files and command-language text. They read text people + wrote. An old file's names are resolved there as a migration. - **Command-language text names things by id.** Anything that uses the command language as a scripting language is a script, wherever the text comes from (a file, `improvise cmd`, another program). A script naming something by display fails; it gets a display only through a dedicated function such as `display_for(id)`. -- **Interactive input** (TUI, browser) matches either a display or an id, as long as it +- **Interactive input** (typing in the TUI or the browser, which is not input text) + matches either a display or an id, as long as it is unambiguous which thing the user is picking. Ids win when it is not, and the display is shown as a preview (clause 12). - **Groups on the wire:** `AddItemInGroup`'s `group` is a group id, a new group is diff --git a/context/adr/007-stable-identities.md b/context/adr/007-stable-identities.md index e044be5c..7bf92117 100644 --- a/context/adr/007-stable-identities.md +++ b/context/adr/007-stable-identities.md @@ -1,7 +1,7 @@ # ADR 007 — Stable identities for categories, items, formulas and views -**Amended:** an amendment is proposed in `007-stable-identities-amendment-1.md` -(PR #326, not yet approved). Until it is approved, this text binds. +**Amended:** by `007-stable-identities-amendment-1.md` (accepted 2026-10-09, PR #326), +which states every rule it replaces. **Status:** accepted (2026-09-25). The user approved revision 7. Revision 8 then followed the adversarial review of that head: it adds clause 12, the ADR's own reading diff --git a/context/code/core.md b/context/code/core.md index 4655a987..c00deb72 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1713,9 +1713,9 @@ slugs to `a__b`: the steps are applied as written, and no extra underscore-collapsing step exists. A trailing run survives, so `East ` slugs to `east_`. Only *leading* underscores are removed (step 4). -**ADR 007's proposed amendment changes these rules, and this code predates it.** -The amendment (`context/adr/007-stable-identities-amendment-1.md`, PR #326) is not -yet approved. Its clause 4: +**ADR 007's amendment changes these rules, and this code predates it.** The +amendment (`context/adr/007-stable-identities-amendment-1.md`, accepted 2026-10-09) +says in its clause 4: - every character outside `a`–`z`, `0`–`9` and `-` becomes `_` instead of being stripped (not only U+0020), so `Region(code)` slugs to `region_code`, while `Q1-2024` keeps its `-`; @@ -1728,7 +1728,7 @@ yet approved. Its clause 4: This section, `slugify`, and the tests `slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and `slugify_keeps_an_underscore_the_display_already_had` describe the earlier rule -above. They change in a follow-up once the amendment is approved. The amendment's +above. They change in a follow-up the amendment calls for. The amendment's clause 4 lists every difference. **The allocator records every id it admits and releases none.** That is the diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index 55c91ba1..1cf88745 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -47,7 +47,7 @@ const SECTIONS = [ ["context/adr/004-faces-and-themes.md", "adr/004-faces-and-themes", "ADR 004 — Faces"], ["context/adr/005-exact-decimal-arithmetic.md", "adr/005-exact-decimal-arithmetic", "ADR 005 — Exact decimal arithmetic (proposed)"], ["context/adr/007-stable-identities.md", "adr/007-stable-identities", "ADR 007 — Stable identities"], - ["context/adr/007-stable-identities-amendment-1.md", "adr/007-stable-identities-amendment-1", "ADR 007, amendment 1 (proposed)"], + ["context/adr/007-stable-identities-amendment-1.md", "adr/007-stable-identities-amendment-1", "ADR 007, amendment 1"], ], }, { From cb73ad3d55976ed413cd311a149beebf0ef5567c Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 10:06:14 -0700 Subject: [PATCH 25/27] docs(adr): the _Measure section lists every stored measure (improvise-cq1q.1) Formula measures stay declared by their ## Formulas lines. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities-amendment-1.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index 6f11596c..61a54de7 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -59,7 +59,10 @@ becomes the group's id. one spells its id: `_measure: column`, `region=east, _measure=revenue = 1200`. - **The stored measures are still listed in a `## Category: _Measure` section**, each with its display and id (`- Revenue(revenue), Cost(cost)`). Its header names the - hardcoded virtual category and is not a declaration of it. The loader recognises + hardcoded virtual category and is not a declaration of it. The section is complete: + it lists every stored measure, so a file never relies on its data lines to discover + one. A formula's measure is not listed there; it is declared by its `## Formulas` + line, which carries the id it shares with its formula. The loader recognises `_Measure`, `_Index` and `_Dim` by their hardcoded displays, and references are written `_measure`. - **Records mode's synthetic positions are values, not ids.** The row coordinate under From 12642e40b62f513171ca163cab436bb85dc07865 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 10:06:59 -0700 Subject: [PATCH 26/27] docs(adr): ids sort in natural order everywhere (improvise-cq1q.1) Defines natural order, applies it to the display suffix, and adds a Status row for the conflict with CellKey's String::cmp order: RecordHash values change and saved data lines reorder; the loader canonicalises. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities-amendment-1.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index 61a54de7..560a046c 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -18,6 +18,7 @@ true. Where the amendment replaces an ADR 007 rule or an existing behaviour: | 1 | `measure_item_names` merges a formula into a stored measure of the same name | a formula never merges: it gets its own id, `profit-` beside a stored `profit` | | 2 | a display already in use in a scope is refused | equal displays in a scope are shown with a gapless ` #2`, ` #3`, … suffix in id order | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | +| 4 | ids sort by plain string order (`CellKey::new` sorts with `String::cmp`, and the writer orders data lines by their text) | ids sort in natural order. `CellKey`'s canonical coordinate order changes with it, so `RecordHash` values change (they are on the wire, not in files), and a saved file's data lines and their coordinates are written in the new order; the loader canonicalises keys, so the data read back is the same | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended before a leading digit | | 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` | a hand-written id obeys every syntax rule of a generated id | @@ -104,10 +105,9 @@ cost`); a dimension target is rare, for a derived dimension. the same display, they are shown with a suffix: the first keeps the bare display, and the rest get ` #2`, ` #3`, … (a space, then `#` and the number) as a gapless sequence in the order their ids sort. - - The order is the ids' plain string order, the order `CellKey` already sorts ids - in, not the counter's numeric order. So - `profit`, `profit-1` and `profit-2` show as `Profit`, `Profit #2` and `Profit #3`, - and `profit-10` sorts before `profit-2`. + - The order is the ids' natural order (clause 4), so `profit`, `profit-1`, + `profit-2` and `profit-10` show as `Profit`, `Profit #2`, `Profit #3` and + `Profit #4`. - Gapless: if the thing shown as `Profit #2` is removed, `Profit #3` becomes `Profit #2`. - The suffix is computed when a display is shown, from the things that share it, and @@ -159,6 +159,12 @@ also what the counter may make of `east`. They never collide, because every id a document holds is taken however it was made: a display slugging to a held `east-1` becomes `east-1-`, and a held counter candidate is skipped. +**Ids are always sorted in natural order**, wherever ids are ordered. Split each id into +alternating runs of digits and non-digits; compare the runs pairwise, a digit run by its +numeric value and a non-digit run by byte order; when two digit runs have equal value, +the shorter (fewer leading zeros) sorts first; when one id runs out of runs first, it +sorts first. So `profit`, `profit-1`, `profit-2`, `profit-10`, and `a-7` before `a-007`. + **Hand-written ids obey every syntax rule of a generated id.** `east-1`, `item-4` and `q1_-_2024` load; `-east`, `east-`, `2024`, `a__b` and `_draft` are refused at load. From bc08a1e41ece185db0e7875376e2b59bafada663 Mon Sep 17 00:00:00 2001 From: "Claude (adr007-steward)" Date: Fri, 9 Oct 2026 10:26:20 -0700 Subject: [PATCH 27/27] docs(adr): list every place natural order changes, and define digit-run value (improvise-cq1q.1) Status rows for records-mode order and fold order, the writer's other sorted lines, GP fit row order, and two internal fingerprints. A digit run compares by length after stripping leading zeros, then by bytes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/adr/007-stable-identities-amendment-1.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/context/adr/007-stable-identities-amendment-1.md b/context/adr/007-stable-identities-amendment-1.md index 560a046c..940b8e38 100644 --- a/context/adr/007-stable-identities-amendment-1.md +++ b/context/adr/007-stable-identities-amendment-1.md @@ -19,6 +19,10 @@ true. Where the amendment replaces an ADR 007 rule or an existing behaviour: | 2 | a display already in use in a scope is refused | equal displays in a scope are shown with a gapless ` #2`, ` #3`, … suffix in id order | | 2 | `CellKey` pairs are `(CategoryId, ItemId)` | `(CategoryId, ItemHandle)` | | 4 | ids sort by plain string order (`CellKey::new` sorts with `String::cmp`, and the writer orders data lines by their text) | ids sort in natural order. `CellKey`'s canonical coordinate order changes with it, so `RecordHash` values change (they are on the wire, not in files), and a saved file's data lines and their coordinates are written in the new order; the loader canonicalises keys, so the data read back is the same | +| 4 | records mode string-sorts the store on entry (`DataStore::sort_by_key`, run by `SortData`) | records-mode rows come out in natural order, and so does the order `matching_positions` folds in (gotcha 18), so a `SUM`/`AVG` can change in the last bits | +| 4 | the writer string-sorts `hidden:`, `expanded(…)`, `collapsed:` and `## Aggregation` lines | those lines are written in natural order too | +| 4 | a GP fit string-sorts its records (`improvise-gp` `dataset.rs`) | the feature matrix's row order, and so a seeded fit's folds, follow natural order: a seeded fit can produce a different tree | +| 4 | `app.rs` view-shape fingerprints and `Model`'s `Debug` string-sort ids | they sort naturally too; nothing visible changes | | 2, 5 | a collapsed entry and a group reference name a group by name | by its id | | 4 | casefold; a run of U+0020 spaces becomes one `_`; strip characters outside `a`–`z`, `0`–`9`, `-`, `_`; remove a leading `_` | casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of `_` collapse; leading and trailing `_` and `-` are removed; `n` is prepended before a leading digit | | 5 | a hand-written id uses the alphabet `a`–`z`, `0`–`9`, `-`, `_` | a hand-written id obeys every syntax rule of a generated id | @@ -160,10 +164,11 @@ document holds is taken however it was made: a display slugging to a held `east- becomes `east-1-`, and a held counter candidate is skipped. **Ids are always sorted in natural order**, wherever ids are ordered. Split each id into -alternating runs of digits and non-digits; compare the runs pairwise, a digit run by its -numeric value and a non-digit run by byte order; when two digit runs have equal value, -the shorter (fewer leading zeros) sorts first; when one id runs out of runs first, it -sorts first. So `profit`, `profit-1`, `profit-2`, `profit-10`, and `a-7` before `a-007`. +alternating runs of digits and non-digits; compare the runs pairwise, a non-digit run by +byte order and a digit run by value: strip its leading zeros, then the shorter run is +smaller, then compare bytes (so runs of any length work, with no fixed-width parse); when +two digit runs have equal value, the one with fewer leading zeros sorts first; when one +id runs out of runs first, it sorts first. So `profit`, `profit-1`, `profit-2`, `profit-10`, and `a-7` before `a-007`. **Hand-written ids obey every syntax rule of a generated id.** `east-1`, `item-4` and `q1_-_2024` load; `-east`, `east-`, `2024`, `a__b` and `_draft` are refused at load.