improvise/context/adr/007-stable-identities.md

ADR 007 — Stable identities for categories, items, formulas and views

Status: proposed (2026-09-23). Revision 2 folds in the user's answers; several questions are still open (listed under "Still open"). This ADR is not accepted until the user says it is. · Decided by: the user, in part; see "What the user decided" · Ticket: improvise-sboz · Unblocks: improvise-km71, improvise-fz77, improvise-fdk1 · Builds on: ADR 001, ADR 002 · Depends on: improvise-tt22 (remove legacy JSON)

Numbering. The ADR was first filed as 005, but improvise-l2b8 already claims ADR 005 (exact decimal arithmetic) and improvise-ilvl claims 006. The review tickets filed against the first draft still say "ADR 005" in their titles; they mean this document.

Direction from the user, 2026-09-23, in three messages:

"there would be a lot of benefit to categories/formulas having stable identities, even if they were only transient: it would help pin editing operations and distinguish them from delete/remove"

"It's probably worth a file format change (in a opt-in way, that gets rectified on save if the input file was missing them)"

"Maybe something like View: Default(default) and the standard View: |Default (default)|(default) for quoting."

What the user decided

The user answered the first draft (context/adr/005-stable-identities.md at 9d97664c) with inline comments on PR #210, and then posted one comment on the thread. Each answer is quoted verbatim below, with where it was attached and what it overturned. Five of the first draft's positions were reversed. The rest of this document is rewritten to match.

1. Storage keys by id; the human name becomes a display field. On first-draft line 92, the clause "CellKey, DataStore, views and formula text stay keyed by name":

"Shouldn't it be easy to replace this to key by ID? There's no type change so, at worst, it would be renaming field access. We could also add the name as something like display and reserve name or whatever it is currently for the id."

2. The point of ids is that keys use them. On first-draft line 97, "ids do not make a rename cheap. They make it expressible":

"This is a lot lower value if CellKeys, etc. don't refer to the new id"

3. Every reference uses the id. On first-draft line 275, the heading of clause 8, "ADR 002: RecordHash stays defined over names":

"No, anything referring to something with an id should refer to the id as the stable identity to avoid glitches."

This answers Q7 by reversing it, and reverses clause 5's "references carry names" too.

4. Q1, the id's internal type. On first-draft line 469, the recommendation to wrap "a small string":

"We probably can reuse symbols as the internal type with interning?"

Ids are strings, interned through the existing SymbolTable.

5. Q2, formula id placement. On first-draft line 473, recommendation F1 (- Profit(3) = Revenue - Cost):

"Yes, before the equal to the right of the name"

6. Q3, header syntax. On first-draft line 480, the claim that "a header with a space before its parenthesis (Sales (copy)) is unaffected":

"Space before the open paren is optional: document this edge case that might break existing docs."

7. Q4, data keys. On first-draft line 485, the recommendation "no — names only":

"In Data, we should prefer ids that are guaranteed unique to names that might not be."

8. Import generates ids; legacy JSON goes. Thread comment, 2026-09-23 06:51:

"For CSV or JSON import, the dimensions are to be treated as the human friendly name and never parsed for IDs: during import, ids should be generated by a slugify algorithm that strips non-alphanumeric characters and replaces spaces with hyphens(collapsing runs). The only supported native file format is .improv, cleanup any legacy JSON format."

Legacy JSON removal is its own implementation ticket, improvise-tt22. This ADR depends on it and does not implement it. It also dissolves improvise-65kf: with no legacy JSON to load, there are no numeric JSON ids to accept.

The answers change the ADR's centre of gravity. The first draft put an id beside the name as an index. The user's answers make the id the key and the name a display string. So a rename becomes cheap: it changes one display field and moves no key, no surrogate and no basis tier. The first draft had called that impossible.

Context

Today the name is the identity. Model::categories is an IndexMap keyed by name. CellKey and DataStore's interned symbols are names. Views, axis maps, page selections, hidden and collapsed sets, and formula text all refer by name. The user's first message names what that costs:

• An edit cannot be told apart from a delete followed by an add. No rename operation exists (grep 'fn rename' finds nothing). A rename is remove plus add, which loses every cell (improvise-fz77). A formula is effectively identified by (target, target_category), and Model::add_formula upserts on that pair (improvise-km71).
• An edit whose target was concurrently replaced cannot be told apart from an edit of the replacement.
• A records-mode coordinate rename has no wire form (improvise-fdk1).

What already exists. Category.id: CategoryId and Item.id: ItemId are usize aliases, assigned from monotone counters. No production code reads either value; the only reader is one test. Neither has ever been written to .improv. Formula and View have no id.

Decision

Categories, items, formulas and views each have an id, and the id is their key. A category or item's current name field becomes the id (the user's suggestion), and the human name moves to a new display field.

• Every reference is an id. That covers CellKeys, axis maps, page selections, hidden and collapsed sets, formula references, data keys, and wire operands that name an existing thing.
• Display names are for people. They appear on screen, are typed into minibuffers, and are resolved to ids at the input edge.

Ten clauses.

1. What carries an identity

kind	id	unique within	wire address
category	yes	the document	CategoryId
item	yes	its category	(CategoryId, ItemId)
formula	yes	the document	FormulaId
view	yes (the user's example)	the document	ViewId
record	RecordHash, now over ids (clause 8)	the document	unchanged shape
group	still open (Q5)

Virtual categories keep their reserved names as their ids: _Index, _Dim and _Measure. A _Measure formula target has no item of its own. Its handle is its formula's FormulaId, which is why the category panel's _Measure rows need ItemHandle::{Item(ItemId), FormulaTarget(FormulaId)} (clause 6).

2. Storage keys by id; display is the only place a human name lives

Following answers 1 to 3:

• Categories and views: Model::categories and Workbook::views are keyed by id, and each Category, Item and View gains a display: String.
• Cells and derived surrogates: a CellKey is a sorted list of (CategoryId, ItemId) pairs. It stays sorted, now by category id, so gotcha 5's canonical form is kept with a different sort key. DataStore interns ids exactly as it interns names today.
• View configuration: the axis maps, page selections, and hidden and collapsed sets are keyed by ids.
• Formulas: a formula's references are ids (clause 5, and Q14).

What a rename now costs: one display write. It moves no CellKey, no RecordHash and no layout_version, so an edit a client composed before the rename still lands after it. That is the "avoid glitches" the user's rule is for. The grid's labels are display strings, so the rename's effect must still dirty the grid and the panels (Dirty::of gotcha 35), but no basis tier moves.

What the migration costs, stated honestly. Every API that takes a name today (Model::category(name), Workbook::add_item(cat, item), CellKey::new callers, the formula evaluator's Expr::Ref lookup, View methods) takes an id instead. The display→id resolution moves to the places a human types, such as the minibuffer commits, the formula parser's reference resolution, and import. The change is mechanical but wide, and it is the bulk of the implementation. The user judged it cheap ("at worst, it would be renaming field access"), and for the field that is right. The call sites are the cost.

Display names stay unique within their scope, as names are today. So a human-typed name resolves to exactly one id, and a rename to a display name already in use is refused.

3. Ids are interned symbols

Following answer 4, an id is a string, and in memory it is a Symbol from the existing SymbolTable: copyable, with integer comparison. Two consequences.

• A Symbol never crosses the wire or reaches the file. It is an index into one process's table. The wire and .improv carry the id's text, and each end interns it on arrival.
• Rejection stops being Copy, because Rejection::Gone (clause 7) carries an id's text. basis.rs's SaveFailed carries no message because Rejection was Copy, so that reason lapses. Whether SaveFailed now gains its message is the implementer's call.

4. How an id is made: slugify, one rule everywhere

Following answer 8, an id is generated from the display name by slugify: strip non-alphanumeric characters, replace spaces with hyphens, and collapse runs. The user stated this for CSV and JSON import. This ADR proposes the same rule for every other creation too: interactive creation, and assigning ids to an id-less .improv file at load. That way a thing's id never depends on how it was made. Whether to extend it that far is Q13(a).

The user's rule leaves details open. They are gathered in Q13, with a recommendation for each:

• case: keep it or fold it;
• _ and -: whether they are "non-alphanumeric";
• non-ASCII letters;
• the empty slug: what id a name made only of stripped characters gets.

Whatever Q13 settles, three rules are needed and are decided here:

• A slug collision gets a suffix. Within the id's scope (clause 1), the first thing in creation or file order keeps the slug, and later ones get -2, -3 and so on, the lowest free number. Assignment is deterministic, never in hash order (gotcha 18).
• A hand-written id always wins over a generated one. Load is two-pass: first every explicit id in the file is collected, then slugs are generated for the rest, suffixed past anything already taken.
• Within a session an id is never reused. Removing east and later adding East gives east-2 while east is retired in memory. Whether that retirement survives a save and reload is Q10.

5. Persistence: ids on declarations and references

Following answers 3 and 7, the file now carries ids everywhere:

• Declarations (section headers, item lines, formula lines) carry the display name and the id.
• Every reference carries the id: axis lines, hidden: and collapsed:, page selections, ## Aggregation, Initial View:, ## Data keys, and names inside formula text (Q14).

Id alphabet. An id is spelled in slug characters only. Whatever Q13 settles, those are a subset of ('A'..'Z' | 'a'..'z' | '0'..'9' | "_" | "-"), written with ranges, because the generator drops ASCII_ALPHANUMERIC (improvise-rxz9). So an id never needs pipe quoting, in a declaration or a reference.

id_token = @{ ('A'..'Z' | 'a'..'z' | '0'..'9' | "_" | "-")+ }
// The space is optional (answer 6).
id_tag   = ${ " "* ~ "(" ~ id_token ~ ")" }

// Declarations: display name, then its id.
header_decl  = { pipe_quoted ~ id_tag ~ &NEWLINE | header_display ~ id_tag ~ &NEWLINE | line_name }
header_display = @{ (!(id_tag ~ NEWLINE) ~ !NEWLINE ~ ANY)+ }
item_decl    = { name ~ id_tag? }
grouped_item = { "- " ~ name ~ id_tag? ~ "[" ~ name ~ "]" ~ NEWLINE ~ blank_lines }

// References: an id, never a display name.
id_ref       = { id_token }

model_name has no id, so it keeps header_name. Initial View: becomes a reference and takes an id_ref. A group reference ([Coastal]) stays a name while groups have no id (Q5).

header_display is guarded only by a lookahead, which the generator skips (gotcha 34). So the generator may emit a display ending in something that later parses as an id. Round-trip property 3 still holds, because the first parse settles the split. But property 1 must be stated over the parsed result, not over the generated text.

The file version changes. A file written after this change starts with a new version line.

• Old files carry no ids, and in them every reference is a display name. So the loader must know which regime it is reading, and the version line is how.
• In an old file, a reference resolves against display names, and ids are assigned by slugify.
• In a new file, a reference is an id.

Under Q13's recommendation, slugify keeps case, _ and -, so every bare name slugifies to itself. Most of an old document's ## Data therefore reads identically in the new format. Pipe-quoted names are the ones that change.

Round-trip properties:

1. an id-less (old-version) file loads, gets ids, and is written in the new version on its first save;
2. a new-version file loads with every id kept verbatim;
3. format(parse(format(parse(x)))) == format(parse(x)).

A formula-line generator strategy is required. The grammar walker emits no = and the walker drops such lines, so without its own strategy every formula property is vacuous (improvise-rxz9).

A duplicate or unknown id in a hand-edited file can no longer be repaired silently. The first draft's repair argument rested on "no id is ever referred to", and answer 3 reversed that. Reassigning a duplicate would silently change what its references point at. So Q8 is restated below, and the recommendation changes to refusing the load with an error naming the id.

Legacy JSON is removed (improvise-tt22), so it needs no migration here.

Import (CSV, JSON) never parses ids out of dimension names (answer 8). A header Region(code) is the display name Region(code), and its id is its slug. The writer pipe-quotes any display name that would otherwise read as display plus id: ## Category: |Region(code)|(Regioncode).

6. The wire: address by id, create by display name, learn ids from projections

• Every operand naming an existing thing is an id. The one operand naming the thing being created is a display name, and the host slugifies it:

  • AddCategory(display) and CreateView(display);
  • AddItem { category: CategoryId, display }, AddItemInGroup { category: CategoryId, display, group } and AddFormula { target_category: CategoryId, raw }.

• New operations are born with ids:

  • RenameCategory { category, to: display };
  • RenameItem { category, item, to: display };
  • EditFormula { formula, raw };
  • fdk1's coordinate rename;
  • the removes (RemoveCategory, RemoveItem, RemoveFormula, DeleteView).

• Older variants (SetAxis, SetPageSelection, HideItem, …) must also move to ids. Their operands are references, and a reference is now an id. They may move one whole variant at a time.

• Projections carry id and display together:

  • CategoryRow { id, display, items: Vec<(ItemHandle, String)>, axis };
  • FormulaCache rows { id, text }, where the text is rendered with display names for people;
  • ViewSummary { id, display, … }.

  AxisFrame labels stay display strings. They gain ids when a consumer needs them.

• ViewState keys by id: the cursors, expanded_cats, and the view history (view_back_stack / view_forward_stack). Answer 1 makes this natural rather than a migration of its own. A view or category rename then orphans nothing.

7. ADR 001: an id that no longer resolves is gone

A command naming an id that does not resolve is rejected before anything is applied, as Rejection::Gone(Named):

• Named is one of Category, Item, Formula or View, and carries the id's text.
• It is structural: force never creates structure.

The check has two halves, like the record tier:

• Before the frame: an exhaustive ids_named feeds Checked::against.
• Within the frame: an id arm in broken_chain refuses [RemoveFormula(f), EditFormula(f)] (improvise-gkyo).

Per-entity versions are still not added (Q6).

8. ADR 002: RecordHash is now defined over ids

Following answer 3, RecordHash hashes the explicit serialization of the sorted (CategoryId, ItemId) text pairs, the same function over different strings.

• ADR 002's decision is unchanged in substance. It still hashes an explicit serialization of a normalized record, with a fixed key and a width argued for.
• Its prose will be out of date once this lands. ADR 002 says CellKey::new "sorts coordinates by category name", and a record is spelled in names. Both become ids. ADR 002 is not edited here. Updating that wording belongs to the implementation PR.

What changes in behaviour: a display rename no longer changes any RecordHash or bumps layout_version, because view_shape hashes ids too. So a client's surrogate survives a rename, and its in-flight edit still lands. A coordinate change, as in fdk1 or a records-mode move, still produces a different record and a different hash, as it should: it is a different record.

9. Headers that change meaning: the edge case answer 6 asks to document

Under answer 6, header_decl reads a trailing (token), with or without a space before it, as the id. In a new-version file the writer never produces an ambiguous header, because it pipe-quotes any display that would split. So the edge cases below arise only when a new-version file is hand-written, or when an old-version file is read with the new header rule. Whether old files get that rule is Q12.

header line	whole-line reading (today)	under header_decl
## View: Default(default)	Default(default)	display Default, id default
## View: |Default (default)|(default)	pipes literal	display Default (default), id default
## View: Sales (copy)	Sales (copy)	display Sales, id copy
## Category: Region(code)	Region(code)	display Region, id code
## Category: Budget (2024)	Budget (2024)	display Budget, id 2024
## Category: Time Period	Time Period	unchanged; id Time-Period generated

The documents that change meaning are those with a view or category header ending in a parenthesised run of letters, digits, _ or -. The app writes such headers itself: improvise import on a CSV with headers like Region(code) or Size(cm) saves ## Category: Region(code) today (improvise-80rm). Any view someone named Sales (copy) does the same.

What it costs, if old files get the new rule:

• The category is renamed on load: display Region, id code.
• Every reference in that old file spells Region(code), which resolves to neither the display nor the id.
• So the axis lines and every data key for that category do not resolve.
• Today's loader would silently drop the axis lines and orphan the data, so nothing is drawn.

What a user would have to do with such a file is pipe-quote the header before opening it (## Category: |Region(code)|). That fixes it, because a pipe-quoted header without an id reads as the whole name.

Silent data loss on load is a real hazard, so this ADR does not decide it. It is Q12.

10. Migration

1. Remove legacy JSON (improvise-tt22).
2. Types: the id newtypes over Symbol, the display fields, and FormulaId and ViewId.
3. Re-key storage by id (clause 2). This is the wide, mechanical step, with display→id resolution moved to every input edge.
4. Persistence: the new version line, the grammar in clause 5, the two-pass loader, slugify, the writer, and the round-trip properties with a formula-line strategy.
5. Projections and ViewState (clause 6).
6. Vocabulary: the id-addressed variants, Rejection::Gone, ids_named and broken_chain's id arm.

What this means for you

improvise-km71 — editing a formula

• The operation. The formula panel's rows carry FormulaId, and the row text is rendered with display names. Commit sends EditFormula { formula, raw }.
  • The user types raw with display names, and the host resolves them to ids.
  • The formula is replaced in place, keeping its position and id.
• A changed target is the same formula. Identity is the id. If another formula already owns the new target, the edit is refused. It must not fall through to add_formula's upsert.
• A target change can leave an item behind. A non-_Measure target is a real item, which add_formula creates. Decide whether the old one is removed when nothing else references it, and test the direction you choose.
• Rejections. Another client's deletion gives Rejection::Gone(Formula(id)). A deletion earlier in the same frame gives the broken-chain refusal.

improvise-fz77 — renaming a category or item

• The operations. RenameCategory { category, to } and RenameItem { category, item, to }. The panel rows carry the ids.
• It is now cheap (answers 1 and 2): one display write. No CellKey, axis map, page selection, hidden or collapsed set, or formula text is rewritten, because all of them hold ids. No RecordHash or layout_version moves, so in-flight edits survive.
• What it still owes:
  • Uniqueness. Refuse a display name already used in the same scope.
  • Dirtiness. Mark the effect so the grid labels and panels are re-projected (gotcha 35). The labels are display strings.
  • Tests. Assert that the labels changed and the surrogates did not. That is the property this design exists for.
• It depends on step 3 of the migration, the re-keying. Until then there is no cheap rename to build on.

improvise-fdk1 — renaming a records coordinate

• The operation is RenameRecordCoordinate { record: RecordHash, category: CategoryId, to: ItemRef }, where ItemRef is Existing(ItemId) or New(display), the create-by-display half of clause 6.
• Unlike a display rename, this changes a coordinate, so the record's CellKeys and RecordHash change. It is a different record.
• Map it onto RecordRewrite, which the drill and the plain records path already share (improvise-uo2p).
• CellId::Coordinate grows { record, category: CategoryId }.
• The producer blocker (a browser cannot enter records editing) is untouched.

Questions as asked, and where each now stands

The first draft asked Q1–Q9, and revision 1 added Q10 and Q11. Each original question and recommendation is kept below, with its status.

#	question	my recommendation	status
1	Integer or word ids?	first draft: words; revision 1: integers only	answered: interned symbols (answer 4), generated by slugify (answer 8). Words.
2	Formula id placement	F1, - Profit(3) = …	answered: F1 (answer 5)
3	Headers in old files	first draft: accept the rare break; revision 1: gate on version	answered in part: the space is optional (answer 6). Whether old files get the rule is still open (Q12).
4	Do ## Data keys carry ids?	no	answered: yes, ids (answer 7), reversing the recommendation
5	Group ids?	not now	open
6	Per-entity versions?	not in this ADR	open
7	RecordHash over ids?	no, keep names	answered: over ids (answer 3), reversing the recommendation
8	Duplicate id: repair or refuse?	repair	open, restated below
9	Reading of "opt-in"	read optional, write always	open
10	Id reuse after a reload	persist a high-water mark	open, restated below for slug ids
11	Who assigns ids?	the host	open, restated below

Still open

5. Do groups get ids? Recommendation: not now. Nothing addresses a group on the wire. The cost of deferring is that group references ([Coastal], collapsed:) stay display names, the one exception to "every reference is an id".
6. Per-entity versions? Recommendation: not in this ADR.
7. A duplicate or unknown id in a hand-edited file: repair or refuse? (Restated.) References now carry ids, so reassigning a duplicate silently re-points every reference to it. Recommendation: refuse the load, with an error naming the id and both declarations.
8. "Opt-in": confirm the reading. Recommendation: reading an id-less file is supported forever, as an old-version file whose ids are assigned by slugify, and every save writes the new version with ids.
9. May an id be reused after a save and reload? (Restated for slug ids.) Slug ids are only reused when a removed thing's name comes back: remove East, save, reload, add East, and it gets East again.
   • (a) Persist the retired ids in the file, so the new East becomes East-2. The list grows forever.
   • (b) Accept reuse across a reload (recommended). Within a session, retirement still prevents reuse. An in-flight command from before the reload is refused by the generation tier. The one remaining risk is an outside reference that outlives the file, and none exists today.
   • (c) No retirement even within a session. That brings back in-session glitches.
10. Who assigns ids? (Restated.) Ids are now derived from names, so a client could compute one. create-and-switch-view sends [CreateView, SwitchView], a frame that addresses what it creates.
    • (a) The host assigns (recommended). A create-then-act compound becomes one command, for example CreateView { display, switch: true }. The host alone resolves collisions and suffixes.
    • (b) The client proposes the slug and the host refuses one that is taken. The frame can then address what it creates, but a collision becomes a refusal the user sees.
11. Do old-version files get the new header rule? (New; the hazard answer 6 asked to document.) See clause 9.
    • (a) No (recommended). An old-version file is read as it always was: a header is the whole line. That is safe because no file written before this change can contain an id, so any (token) in its headers is part of the name. The optional-space rule applies to new-version files, where the writer quotes every ambiguous display. Nothing that exists today changes meaning.
    • (b) Yes. Headers like Region(code) and Sales (copy) split on load, and the data under them silently disappears unless the user pipe-quotes the header first. If this is chosen, the loader should at least refuse such a file with an error naming the header, rather than orphaning data silently.
12. Slugify details. (New; the rule as stated leaves these open.)
    • (a) Where slugify applies. The recommendation is all id generation: interactive creation and old-file load as well as import. That gives one rule, and bare names slugify to themselves.
    • (b) Case. Recommendation: keep it, so East stays East and old data keys read the same.
    • (c) _ and -. Recommendation: keep both. They are already legal in bare names, so every existing bare name is its own slug. The literal "strip non-alphanumeric" would turn Coastal_East into CoastalEast and Q1-2024 into Q12024.
    • (d) Non-ASCII letters. Recommendation: strip them (ASCII only), so an id never needs quoting. A name made only of such letters falls to (e).
    • (e) The empty slug. Recommendation: <kind>-<n> (for example category-1 or item-4), with the lowest free n.
13. Formula text in the file: ids or display names? (New; a consequence of answer 3 that is visible in the file.) Answer 3 says every reference is an id, so a formula line would be written - Profit(Profit) = Revenue - Cost over ids. With Q13's recommendations the ids equal the bare display names, so most formulas read the same. A display rename changes the UI rendering and not the file. The cost is that Formula::raw is no longer stored verbatim: the file gets a normalized rendering over ids, and the UI a rendering over display names. Recommendation: ids in the file, per answer 3. Otherwise every rename must rewrite formula text, which is exactly the glitch answer 3 is avoiding.

Consequences

Good.

• Edit, rename and replace are distinct, addressed operations.
• A rename is one display write: it moves no key, surrogate or basis tier, and in-flight edits survive it.
• ## Data keys are unique by construction.
• The format loses a legacy branch (improvise-tt22).

Bad.

• Re-keying storage is wide: every name-taking API moves to ids, and display→id resolution moves to the input edges.
• A new file version.
• Rejection loses Copy.
• Formula::raw is no longer verbatim (if Q14 holds).
• ADR 002's prose needs a wording update when this lands.

Not addressed. Per-entity versions (Q6), group ids (Q5), and ids in AxisFrame and CellId::Derived.