docs(adr-002): the surrogate is a hash of the normalized record, resolved via a maintained index
CI / Test and lint (host) (push) Has been cancelled
CI / wasm32 check (browser bundle crates) (push) Has been cancelled
CI / Knowledge base consistency (push) Has been cancelled

Decided: hash the record (coord tuple minus _Measure), not the cell — it is
already ADR 001's atomicity unit and what record_versions is keyed by, so one
identity serves addressing and staleness. Resolution is a maintained index in
DataStore, which already interns names and already keeps a (Symbol,Symbol)
secondary index that makes invalidation tractable.

Records the three ways this breaks silently: a per-process-seeded hasher, a
derived Hash with no stability guarantee, and a width chosen for convenience
rather than for the consequence of a collision.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
This commit is contained in:
Edward Langley
2026-09-19 16:56:13 -07:00
parent 666423d5c8
commit f74800c799
2 changed files with 72 additions and 16 deletions
+1 -1
View File
File diff suppressed because one or more lines are too long
+71 -15
View File
@@ -58,13 +58,46 @@ such fact.
These are deliberately unresolved; the implementer should decide them with evidence.
**What the surrogate *is*.** A hash of the `CellKey` is the obvious candidate: stateless,
stable across a reconnect, nothing to keep in sync. But a hash is one-way, so the server
still needs to get back to a `CellKey` — see below — and it needs enough width that a
collision is not a silent wrong-cell write. The alternative is an interned id, for which
this codebase has precedent: `DataStore` already interns names into `Symbol` through a
`SymbolTable`. That has no collision risk and can be small, at the cost of a table with a
lifecycle.
**What the surrogate is — DECIDED: a hash of the normalized record.** Not of the cell:
the record is the coordinate tuple minus `_Measure`, which is already ADR 001's unit of
atomicity and already what `record_versions` is keyed by. A cell is then
`(record hash, measure)`.
The reason is that it makes **stale edits cheap to identify**, and it does so three ways
at once. The surrogate a client already holds *is* the lookup key for its own version, so
`record_versions` becomes `HashMap<RecordHash, u64>` — fixed-size, no allocation, no
`CellKey` reconstruction on the hot path. A basis is then a list of
`(record hash, version)` pairs, which is compact on the wire and O(1) to check. And a
record that no longer exists simply **fails to resolve**, so that whole class of
staleness needs no version comparison at all. One identity serves addressing and
staleness instead of each ADR carrying its own.
It is well-defined because `CellKey` already has a canonical form: `CellKey::new` sorts
coordinates by category name and equality and hashing rely on it (gotcha 5). "Normalized"
means that canonical sorted form with `_Measure` removed — the normalization is a
property the type already guarantees, not a new convention to maintain.
An interned id was rejected: it would need a table with a lifetime nobody has specified,
and it buys only collision-freedom, which width buys more cheaply.
**Three ways this silently breaks, all avoidable.** They are listed because each produces
a *wrong write* rather than an error, which is the failure mode this ADR exists to
prevent:
- **Do not use `DefaultHasher` or anything built on `RandomState`.** `std`'s default
`HashMap` hasher is randomly seeded **per process**, so hashes would differ across a
server restart and between any two processes. Specify a fixed hash (a keyed SipHash
with a constant key, or FxHash/xxHash/blake3) and write the choice down.
- **Do not use `#[derive(Hash)]` as the definition.** Rust gives no stability guarantee
for derived `Hash` across compiler versions or platforms. Hash an **explicit byte
serialization** of the sorted `(category, item)` pairs, so the surrogate is defined by
the format rather than by whatever the compiler did.
- **Pick the width by consequence, not by convenience.** A collision is a silent write to
the wrong record — precisely why `_Index` was rejected. 64 bits reaches birthday risk
around 2³² records, which a pivot tool will not see; 128 bits costs 8 more bytes per
*visible* cell, so on a 50×20 viewport that is 8 KB a frame. The cost is small and the
failure is silent, so argue the width explicitly rather than defaulting to `u64` because
it is convenient.
**`_Index` was considered and rejected — right ontology, unsafe representation.** It is
the coordinate that identifies a *record*, and a record is exactly the thing that is
@@ -84,14 +117,37 @@ record, not the cell.** A record is the coordinate tuple minus `_Measure`
then one identity serves both addressing *and* staleness, and the two ADRs share a
mechanism instead of each inventing one.
**How the server resolves a surrogate back to a `CellKey`.** Two shapes. A table
populated as it projects — simple, but it is per-session state with a lifetime nobody has
specified. Or **regenerate from the current layout on demand**: the server already has
the `GridLayout`, so it can recompute the visible keys and their surrogates and look the
incoming one up. That is attractive for a reason beyond avoiding state: **a surrogate
minted against a layout that has since changed simply fails to resolve** — which is
exactly ADR 001's stale-basis rejection, arriving at cell granularity for free. The two
designs compose rather than merely coexist.
**How the server resolves a surrogate back — DECIDED: a maintained index, in
`DataStore`.** A hash is one-way, so resolution needs a reverse map, and regenerating it
from the layout on demand would be O(visible cells) per resolve *and* would only ever
cover the visible window — a client may legitimately hold a surrogate for a record that
has scrolled out of view.
`DataStore` is the right home because it already does exactly this. It interns every
category and item name into a `SymbolTable`, and it already maintains a secondary index
`HashMap<(Symbol, Symbol), HashSet<InternedKey>>` for hot-path lookups. A
`RecordHash → record` index is the same shape, updated at the same points, and the
existing `(Symbol, Symbol)` index is what makes invalidation tractable: when a category or
item disappears, the records that mention it are findable without a scan.
Two consequences worth stating, because they fall out and are easy to miss.
**The index and `record_versions` want to be one structure.** Both are keyed by the record
hash and both are invalidated at the same moments, so `RecordHash → (record, version)`
gives ADR 001's basis check and this ADR's resolution in a single lookup.
**Structural and data staleness separate cleanly**, which is better than the
regenerate-on-demand shape would have given:
| incoming surrogate | means | answer |
|---|---|---|
| does not resolve | the record is **gone** — a category or item it named was removed | reject; there is nothing to write to |
| resolves, version matches | the client's basis is current | apply |
| resolves, version differs | the record **changed** since the client read it | reject as stale (ADR 001), offer force |
Regenerating from the layout would have conflated the first two rows, because a record
scrolled out of the viewport would have looked identical to a record that no longer
exists.
**What drill uses.** If `key` is `None` the client wants to *drill*, and this ADR gives
it nothing to drill *with*. Whether an aggregate needs its own surrogate, or whether