improvise/context/code/zset.md

zset: the reasoning

crates/zset is ADR 009 step 1 (improvise-tk51, epic improvise-bw4p): the Z-set group, first and alone. Nothing depends on it yet, so it changes no behaviour. The decisions are the ADR's (context/adr/009-incremental-evaluation.md, clause 1). This file records how the crate reads them, where the ADR left a choice open, and the evidence discipline. The evidence for each test is in context/test-evidence/zset.md.

The shape

item	what it is
ZSet<K: Ord>	a map from K to a non-zero i64 weight; an abelian group within i64 (interpretation 5)
+, -, unary -, * i64, scale	the group operations and scalar multiplication
new, singleton, insert, FromIterator, Extend	construction; weights of equal keys sum
weight, is_zero, len, iter, IntoIterator	reads; iteration is ascending by key
map, filter, distinct	the three per-element operators
integrate, differentiate	the stream operators, over a slice

There are no dependencies: not serde (clause 9 keeps the wire form in improvise-protocol), not even the workspace's own crates. The crate is wasm32-clean by construction and is in nix/ci.nix's wasmCrates.

Canonical form

A key with weight zero is never stored. insert removes a key whose sum reaches zero, and singleton(k, 0) is the zero Z-set. Two Z-sets are therefore == exactly when every key has the same weight, so the derived PartialEq, Hash and Ord are the group's equality. The clause 9 correction target - held relies on it: a correct state gives == the zero Z-set, and is_zero is len() == 0, not a scan.

The representation is a BTreeMap. Iteration order is the key order, the same in every process and for every insertion order, which clause 3 needs (eager folds in CellKey order) and which a HashMap would give up (context/code/core.md, gotcha 18's argument).

Interpretations

The ADR is one paragraph on the crate, and says more about what it is for than what it is. These are the readings this step takes.

1. Clause 1 lists a Circuit and more operators; this step builds the pure part. The ADR's list is delay, integrate, differentiate, map, filter, flat_map, index_by, join_indexed, distinct and aggregate, run by a synchronous Circuit, plus resumable rebuilds and a trace interface. Step 1 as ticketed is the group and the operators that are functions of a Z-set or of a stream of them: map, filter, distinct, integrate, differentiate. The rest needs the circuit (a clock, state held between steps), or a second key type (index_by, join_indexed, aggregate), and is left to the steps that use it rather than designed without a caller. flat_map is map into several images and has no user yet either.

2. integrate and differentiate are slice functions, not circuit nodes. integrate(&[d_0, d_1, ..]) is the running sums d_0 + .. + d_i; differentiate is its inverse, s_0 then s_i - s_(i-1). The state before the first element is zero, as in DBSP. delay is not provided: over a slice it is a shift, and a circuit gives it meaning.

3. distinct is not linear, and is not a group homomorphism. It keeps weight 1 on the keys whose weight is positive and drops every other key, negative included. That is the set-semantics operator of DBSP. The ADR says nothing about negative weights; dropping them is the standard reading, and the one that makes distinct idempotent.

4. Operators are methods on &self, group operations are std::ops traits. K: Clone is required only where a result copies keys (filter, distinct, scale, the borrowed operators); map and the by-value operators need only Ord.

5. Overflow. Neither clause 1 nor clause 8 states a policy for weights (clause 8 is about f64 sums). A weight that wraps would turn a retraction into an insertion with nothing to show it, which is the failure the ADR's exactness argument is against. So every weight operation uses checked_* and panics with zset: weight overflow in <op>, in debug and in release (checked_* does not depend on overflow-checks). The operations are add (including insert, Extend, FromIterator), sub, neg (so i64::MIN panics when it has to be negated) and scale. Scaling by zero cannot overflow and returns the zero Z-set. A panic, not a Result: Result does not belong in the majority of the code (AGENTS.md), and a weight near i64::MAX is a defect in the caller, not an input to recover from. The weights clause 3 produces are small (one per cell).

   The group is partial, and that is the stated reading. The ADR calls a Z-set an abelian group. With i64 weights and no wrapping, the laws hold wherever no weight, intermediate ones included, leaves i64; outside that, an operation panics rather than return a different value. So associativity is order-dependent at the boundary: (MAX + 1) + (-1) panics where MAX + (1 + (-1)) is MAX. The result is never wrong, only sometimes refused. Wrapping arithmetic would make the type a true group (the integers mod 2^64), and (w + x) - x == w would still hold exactly, so a retraction does cancel its insertion under wrapping. (An earlier version of this paragraph said otherwise; that was wrong.) What wrapping breaks is the intermediate reads. MAX + 1 stores i64::MIN, so distinct drops a key whose true weight is positive and weight() has the wrong sign. A true weight of 2^64 stores as 0, so the key vanishes and is_zero, len and == report a set that does not exist. Those are wrong answers returned silently, which is why the crate panics instead. A wider weight (i128) moves the boundary and does not remove it. The ADR's "abelian group" wording needs amending whichever policy is chosen: checked-and-panic gives a group only within range, wrapping gives a group whose reads can be wrong, and i128 is partial too. Which one is a ruling for the user, not for this step; the crate currently panics, and overflow_depends_on_the_order_of_evaluation_and_never_gives_a_wrong_value states it. sub is checked directly against the held weight, so a - a is zero for every a, i64::MIN included, and only an absent key's 0 - w can overflow (w is i64::MIN).

6. Stable error text. The panic message names the operation and both operands, so a failure in a long-running host is diagnosable from the log.

What the tests are

crates/zset/tests/properties.rs holds one checker per property, parameterised by an Alg trait. The property's test runs it on ZSet. A companion runs it on Model(Fault::None), a second implementation over a sorted vector, so the checker is shown not to reject a correct implementation that is not ZSet. Each rival is Model(Fault::X), run through the same checker with a deterministic seed, and the failure message must carry the property's tag ([inverse], [map], ..). A rival that fails for another reason does not count. The group laws, canonical form, scalar distributivity, the map homomorphism, filter linearity, distinct's definition, iteration order, the clause 9 correction identity and the integrate/differentiate inverses are all properties; each runs at the default 256 cases over keys from a range of six, so collisions and cancelling pairs are the common case. Weights come from -3..=3, an explicit zero, and ±2^40, so sums of three cannot wrap.

The overflow policy is not a property over the generators (they cannot reach it by design) and is tested by example in src/lib.rs, with should_panic(expected = ..).