From 2e1f0f65508f4a18adfcfbf11ad99b3b6f28dcce Mon Sep 17 00:00:00 2001 From: "Claude (adr007-slug)" Date: Fri, 9 Oct 2026 12:49:09 -0700 Subject: [PATCH] docs(context): core.md section 20 and test evidence for the amended id rules (improvise-cq1q.14) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/code/core.md | 89 ++++++++++++++------------ context/test-evidence/core.md | 92 +++++++++++++++++++++++---- crates/improvise-core/src/model/id.rs | 16 ++--- 3 files changed, 136 insertions(+), 61 deletions(-) diff --git a/context/code/core.md b/context/code/core.md index c00deb72..67a004ef 100644 --- a/context/code/core.md +++ b/context/code/core.md @@ -1688,16 +1688,16 @@ probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a ## 20. How an id is made — `model/id.rs` (ADR 007 clause 4, improvise-cq1q.2) -`slugify` and `IdAllocator` are the first piece of ADR 007. They are pure -functions of the display and the allocator's own history. Nothing calls them -yet: the creation sites move onto them in the re-key step (improvise-cq1q.4). +`slugify`, `IdAllocator`, `natural_cmp` and `display_suffixes` are the first +piece of ADR 007. They are pure functions of their arguments and the +allocator's own history. Nothing calls them yet: the creation sites move onto them in the re-key step (improvise-cq1q.4). Each rule traces to ADR 007's clause 4 (slugify, the counter, hand-written ids) or clause 1 (reserved ids). **Case folding is full Unicode folding, and it comes from `unicase`.** -`str::to_lowercase` is not folding: `ß` lowercases to itself and is then -stripped, so `Straße` would slug to `strae` (clause 4, step 1: full case folding). The same goes for `ſ` -(long s) and the `fi` ligature. `unicase` 2.9 is `no_std` and has no +`str::to_lowercase` is not folding: `ß` lowercases to itself and then +becomes `_`, so `Straße` would slug to `stra_e` (clause 4, step 1: full case +folding). The same goes for `ſ` (long s) and the `fi` ligature. `unicase` 2.9 is `no_std` and has no dependencies, so it passes the wasm gate. Its `to_folded_case` applies the C+F mappings of Unicode's CaseFolding.txt, and its own tests pin `Maße` → `masse`. Most of what folding produces is stripped anyway: only an ASCII @@ -1705,31 +1705,45 @@ letter survives step 3. Deriving that ASCII-producing subset of the table by hand was rejected. It is a second copy of the Unicode data, and nothing would say when it drifted. -**A "space" is U+0020.** The accepted clause 4 says "replace each run of spaces -with one `_`". A tab or a no-break space is not a space here, so it is -stripped by step 3. Only a run of U+0020 becomes one `_`. An underscore the -display already had is never merged with the one a space run makes. So `a_ b` -slugs to `a__b`: the steps are applied as written, and no extra -underscore-collapsing step exists. A trailing run survives, so `East ` slugs -to `east_`. Only *leading* underscores are removed (step 4). +**Slugify is the amendment's six steps** (ADR 007 amendment 1, clause 4): +casefold; every character outside `a`–`z`, `0`–`9` and `-` becomes `_`; runs of +`_` collapse; leading and trailing `_` and `-` are removed, all of them (the +trim is `trim_matches` over both characters, so `_-_East-_-` is `east`); +`n` is prepended to a slug that starts with a digit; nothing left is the +caller's `-`. The order matters in one place: the trim runs before +the `n`, so `-_7up` is `n7up` and a leading `-` can never be hidden behind the +prefix. Whitespace of every kind, punctuation and every non-ASCII letter +become `_`, so `Region(code)` is `region_code` and `Crème Brûlée` is +`cr_me_br_l_e`. Folding is `unicase`'s, which maps `ß` to `ss`; a character +that folds to a combining mark (`İ` folds to `i` plus U+0307) leaves the `i` +and turns the mark into `_`, which the trim then removes. A hyphen between +underscores survives (`a - b` is `a_-_b`). -**ADR 007's amendment changes these rules, and this code predates it.** The -amendment (`context/adr/007-stable-identities-amendment-1.md`, accepted 2026-10-09) -says in its clause 4: -- every character outside `a`–`z`, `0`–`9` and `-` becomes `_` instead of being - stripped (not only U+0020), so `Region(code)` slugs to `region_code`, while - `Q1-2024` keeps its `-`; -- runs of `_` collapse, so `a_ b` and `a__b` both slug to `a_b`; -- leading and trailing `_` and `-` are removed, so `East ` → `east`, `-East-` → - `east`, and `-` alone takes `-`; -- `n` is prepended to a slug that starts with a digit, so `2024` → `n2024`; -- a hand-written id must obey every syntax rule of a generated one. +**The id syntax has one validator, `is_id_syntax`**: a lower-case ASCII +letter, then `a`–`z`, `0`–`9`, `-` and single underscores, never ending in +`_` or `-`. Slugify's non-empty output always satisfies it and is a fixed point +of itself, and a `-` suffix keeps the shape, so a generated id always +validates (properties in `model::id::tests`). A virtual id fails it by design, +because the leading `_` is the reservation. -This section, `slugify`, and the tests -`slugify_collapses_a_run_of_spaces_to_one_underscore` (its `east_` case) and -`slugify_keeps_an_underscore_the_display_already_had` describe the earlier rule -above. They change in a follow-up the amendment calls for. The amendment's -clause 4 lists every difference. +**Natural order is `natural_cmp`** (clause 4, "Ids are always sorted in +natural order"). An id is split into alternating digit and non-digit runs, and +the runs are compared pairwise: non-digit runs by bytes; digit runs by value +(strip leading zeros, shorter is smaller, then bytes), and a tie on value goes +to the run with fewer leading zeros, immediately and not after the later runs. +The id that runs out of runs first sorts first. No digit run is parsed into an +integer, so a run of any length compares correctly. Where two ids disagree +about the kind of run at a position (only possible for text that is not an +id), the runs compare as bytes. It is a total order and equal only for equal +strings. **Nothing calls it yet**: `CellKey`, `DataStore`, the writer and the GP +dataset still sort with `String::cmp` until the re-key step. + +**Equal displays are suffixed by `display_suffixes`** (clause 2). Given +`(id, display)` pairs it returns what each is shown as, in input order: the +first of a group of equal displays in natural id order keeps the bare display, +the rest get ` #2`, ` #3`, …, and a number is skipped when a literal display +already reads ` #` anywhere in the scope. Displays are equal +byte for byte, not case-folded. It is pure and nothing renders through it yet. **The allocator records every id it admits and releases none.** That is the whole of retirement (clause 4, "within a session a removed id is never handed @@ -1750,17 +1764,14 @@ name happens to be free, because clause 4 spells the result `-`. **`claim` is the only door for a hand-written id.** It checks, in this order: -the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading -`_` (only `_index`, `_dim` and `_measure` may carry one, per clause 1, and -the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other -id may start with `_`"), and -duplicates. It accepts `-`, `-x`, `x-`, `a__b` and `2024`, all of which the -amendment refuses (a hand-written id obeys the syntax of a generated one). It never renames. -Refusing is the loader's job, and the loader +the reserved leading `_` (only `_index`, `_dim`, `_measure`, `_records` and +`_drill` may carry one, and the match is exact: `_indexer` and `_measure-1` +are Reserved), the syntax (`is_id_syntax`; `-x`, `x-`, `2024`, `a__b` and `x_` +are Malformed, `east-1`, `item-4`, `a-7` and `q1_-_2024` pass), and +duplicates. A virtual id skips the syntax check, since it fails it by design. +It never renames. Refusing is the loader's job, and the loader must name both lines of a duplicate; `ClaimError` carries only the id, so the -loader keeps the line numbers. The virtual views `_records` and `_drill` are -not yet in `VIRTUAL_IDS`. improvise-p003 item 1 asks for them, and they arrive -with the ADR text fix (improvise-cq1q.1) rather than ahead of it. +loader keeps the line numbers. `IdKind::Group` spells `group-`. ## 21. A display beside the key (ADR 007 clause 2, improvise-cq1q.3) diff --git a/context/test-evidence/core.md b/context/test-evidence/core.md index d4ae4c48..b25328ca 100644 --- a/context/test-evidence/core.md +++ b/context/test-evidence/core.md @@ -1914,30 +1914,96 @@ identity on the view. header stays for the sibling quarter that has data. Rival: `prune_axis_items` keying open runs by level alone. This test and the `NESTED` enumeration fail. -## `model/id.rs` (ADR 007 clause 4, improvise-cq1q.2) +## `model/id.rs` (ADR 007 clause 4 and amendment 1, improvise-cq1q.2, improvise-cq1q.14) -These tests were written before `slugify` and `IdAllocator` had bodies. With -`todo!()` bodies all eleven failed. Each rival below was then injected into the -finished module, run against `cargo test -p improvise-core --lib model::id`, and +Each rival below was injected into the finished module, run against `cargo test -p improvise-core --lib model::id`, and reverted. The reasoning is `context/code/core.md` §20. ### model::id::tests::slugify_case_folds_rather_than_lowercases Rival: `display.to_lowercase()` in place of `unicase`'s full folding. Only -this test fails: `left: "strae"`, `right: "strasse"`. `ß` lowercases to itself -and step 3 strips it. `slugify_follows_the_adrs_examples` cannot see this rival, +this test fails: `ß` lowercases to itself and becomes `_`, so `stra_e` against +`strasse`. `slugify_follows_the_amendments_table` cannot see this rival, because each of its displays lowercases and folds alike. -### model::id::tests::slugify_collapses_a_run_of_spaces_to_one_underscore +### model::id::tests::slugify_follows_the_amendments_table -Rival: every space becomes `_`, with runs not collapsed. Only this test fails: -`"time___period"`. No other fixture has two spaces in a row. +Every row of the amendment's clause 4 table, plus `-`, `%` and the empty +display. Rivals, each of which fails this test: trimming only leading `_` +(`East_` stays), trimming `_` and not `-` (`-East-`), not collapsing runs of +`_` (`a__b`), not prepending `n` (`2024`), and stripping instead of replacing +(`Region(code)` is `regioncode`). The last also fails +`slugify_keeps_a_hyphen_between_underscores` and +`a_slug_taken_by_another_kind_takes_a_suffix`. -### model::id::tests::slugify_keeps_an_underscore_the_display_already_had +### model::id::tests::slugify_trims_hyphens_and_underscores_until_neither_is_at_an_end -Rival: `.replace("__", "_")` after the space step, which reads the accepted -clause 4 as "no doubled underscores". Only this test fails: `left: "a_b"`. The rule -collapses runs of *spaces*. An underscore that was in the display is kept. +Rival: trim leading `_` only, or `_` only at both ends. `_-_East-_-` keeps a +hyphen, and `-_-` is not empty. A single `trim_matches` over both characters +passes; two sequential single-character trims would not. + +### model::id::tests::slugify_trims_before_it_prepends_the_n + +Rival: no `n` prefix, which fails this test, the table and the property below. +`-_7up` is `n7up` only if the trim runs first; prepending first would leave the +`-_` in front of the `n`. + +### model::id::tests::slugify_output_has_the_id_syntax_and_is_a_fixed_point + +The property over arbitrary displays: a non-empty slug satisfies +`is_id_syntax` and slugifying it returns it. Rivals that fail it (and +`generated_ids_are_distinct_well_formed_and_never_reserved`): no `_` +collapse, no `n` prefix, keeping a trailing `_` or `-`. + +### model::id::tests::id_syntax_agrees_with_its_regular_expression + +`is_id_syntax` against the spelled-out rule over `[a-c0-9_-]{0,8}`. Rivals: +allowing `__`, allowing a leading digit and allowing a trailing `-` each fail +here and in `claim_refuses_what_a_generated_id_could_not_be`. + +### model::id::tests::claim_refuses_what_a_generated_id_could_not_be + +Rival: `claim` keeps the old alphabet-only check. `-x`, `x-`, `2024`, `a__b`, +`x_` and `-` are admitted, and the test fails. It also pins the accepted ids +`east-1`, `item-4`, `a-7` and `q1_-_2024`. + +### model::id::tests::claim_admits_each_virtual_id_once + +Rival: `VIRTUAL_IDS` without `_records` and `_drill`. The test spells the five +ids out instead of iterating the constant, because an iteration over the +constant passes with the constant shortened (the first version did). + +### model::id::tests::natural_order_puts_a_counter_suffix_in_numeric_order + +Rivals: comparing digit runs as strings (`profit-10` before `profit-2`) fails +this test, `natural_order_compares_digit_runs_of_any_length_by_value` and +`a_suffix_numbers_equal_displays_in_natural_id_order`; plain `str::cmp` fails +those and `natural_order_ties_on_value_go_to_fewer_leading_zeros`. + +### model::id::tests::natural_order_ties_on_value_go_to_fewer_leading_zeros + +Rivals: more zeros first, or no tie-break (`a-7` equal to `a-007`). Both fail this test and the any-length test (`x000` after `x0`). + +### model::id::tests::natural_order_puts_the_id_that_runs_out_of_runs_first + +Rival: the longer id first. Only this test fails. + +### model::id::tests::natural_order_is_a_total_order + +Antisymmetry, equality only for equal strings and transitivity over +`[a-c0-3_-]{0,7}`, which includes ids starting with a digit. + +### model::id::tests::a_suffix_numbers_equal_displays_in_natural_id_order + +Rival: ordering the group by `str::cmp` on the ids. `profit-10` is numbered +before `profit-2` and this test fails alone. + +### model::id::tests::a_suffix_skips_a_number_a_literal_display_already_holds + +Rival: numbering without the skip. A literal `Profit #2` and a generated +`Profit #2` are then both shown, and this test fails alone. A rival that starts +numbering at ` #1` also fails it and +`a_suffix_is_gapless_when_a_thing_goes_away`. ### model::id::tests::the_suffix_is_the_next_counter_value_not_the_lowest_free_number diff --git a/crates/improvise-core/src/model/id.rs b/crates/improvise-core/src/model/id.rs index 5340cc09..dfdea0f1 100644 --- a/crates/improvise-core/src/model/id.rs +++ b/crates/improvise-core/src/model/id.rs @@ -239,6 +239,7 @@ mod tests { assert_eq!(slugify("Straße"), "strasse"); assert_eq!(slugify("ſtate"), "state"); assert_eq!(slugify("file"), "file"); + assert_eq!(slugify("İ"), "i"); } #[test] @@ -352,7 +353,7 @@ mod tests { #[test] fn claim_admits_each_virtual_id_once() { let mut ids = IdAllocator::new(); - for v in VIRTUAL_IDS { + for v in ["_index", "_dim", "_measure", "_records", "_drill"] { assert_eq!(ids.claim(v), Ok(())); assert_eq!(ids.claim(v), Err(ClaimError::Duplicate(v.into()))); } @@ -447,13 +448,7 @@ mod tests { ]; assert_eq!( display_suffixes(&items), - [ - "Profit #4", - "Profit", - "Profit #3", - "Profit #2", - "Cost" - ] + ["Profit #4", "Profit", "Profit #3", "Profit #2", "Cost"] ); } @@ -474,7 +469,10 @@ mod tests { #[test] fn a_suffix_is_gapless_when_a_thing_goes_away() { let three = [("p", "Profit"), ("p-1", "Profit"), ("p-2", "Profit")]; - assert_eq!(display_suffixes(&three), ["Profit", "Profit #2", "Profit #3"]); + assert_eq!( + display_suffixes(&three), + ["Profit", "Profit #2", "Profit #3"] + ); let two = [("p", "Profit"), ("p-2", "Profit")]; assert_eq!(display_suffixes(&two), ["Profit", "Profit #2"]); }