Files
improvise/context/adr/005-exact-decimal-arithmetic.md

129 KiB
Raw Permalink Blame History

ADR 005 — Exact decimal arithmetic: a three-level numeric tower for formulas

Status: proposed. Only the user accepts an ADR. All sixteen numbered questions are answered (question 16 last, in revision 8, K = 8); the ADR waits for the user's "Approved". · Ticket: improvise-l2b8 (parent improvise-felo) · Direction from: the user, 2026-09-22 (below) · Relates to: ADR 009 (incremental evaluation, improvise-felo.4, PR #309), which now carries an exact f64 accumulator; ADR 007 (stable identities), whose new file-version literal this ADR may share · Governed by: context/design-principles.md §3 and §5

"Formulas should default to decimal math (ideally following Scheme's numeric tower rules for promotion)."

— the user, 2026-09-22, recorded on improvise-l2b8

Numbering. This is the ADR that improvise-l2b8 claimed as 005 before ADR 007 was written. improvise-8tk5 and improvise-80rm say "ADR 005" in their titles only because ADR 007 was first filed under this number. They belong to ADR 007, not to this document.

Citations. Every file:line below is at gh/main b639266b. Line numbers are dated, so re-derive them before citing this document elsewhere. "ADR 009" means the head of PR #309 at 4973845f (revision 6; it was 25ebf641 through revision 5).

Revision 2 answers the adversarial review of 1fe3a85d (a BLOCK) and the four ChatGPT Codex findings on the same head. The review's two blocking points are both correct, and both were reproduced here before the text changed: an exact SUM was not independent of order at the 28-digit bound (clause 8 now sums exactly in a wide integer), and rust_decimal's conversion to f64 is not correctly rounded (clause 4 now names one that is). Questions 1–9 keep their numbers; 10 and 11 are new.

Revision 3 adds option (d) to question 2, at the user's request: "I think (d): exact fractions with a size limit. should be added." It is specified and measured in Option (d) for question 2. The recommendation for question 2 changes from (a) to (d), in a hybrid form: the decimal arm stays, and a bounded fraction arm is added only for the values (a) would have made inexact. The clauses below are still written to (a). That section lists what changes in them if the user answers (d). Question 12 is new.

Revision 4 moves the numeric tower into its own workspace crate, at the user's direction: "I think ADR 005 should implement the math operations in a separate crate". That is new clause 14. Clause 2 and the migration now place Num there, not in improvise-core. It also adds questions 13–15: where the fraction bound applies (13), the crate's name (14), and where ADR 009's ExactSum lives (15). The same revision answers the review of e4acf643 (BLOCK D1 and D2, and its should-fix list) and Codex's P2 on aligning CSV entry with PR #321.

Revision 5 folds in the user's answers to questions 1–12, which they left inline on e4acf643 at 06:49–06:57 UTC on 2026-09-25. Revision 4 missed them, and that was the review of 87ae21ca's first blocking point. They are quoted in What the user decided, and the clauses are now written to them: three levels, option (d), exact comparison plus an EQUAL(a, b, epsilon) function, a string encoding on the wire, and n/d read as a fraction only where the type is forced. The same revision answers the rest of that review (B2–B4, D1's correction, question 13, the crate homes). Questions 10 and 13–15 stay open, and nothing is renumbered.

Revision 6 folds in the user's answers to questions 10, 13, 14 and 15, left inline on 33833f28 at 17:45–17:48 UTC on 2026-09-25: the 320-bit decimal partition and option (a) for question 10, (b) for question 13, numtower for question 14, and (a) for question 15. It answers the review of 33833f28 (issuecomment-5836990563):

  • N1. The number language's gate is now the finite results of f64::from_str, so it is #321's gate. nan, inf and 1e999 stay text at import, as improvise-jc95 decided. Reversing jc95 is named as the rejected alternative (clause 3).
  • N2. Entry tries the fraction arm before f64, so a typed value is exactly as exact as the same value computed (clause 3).
  • The should-fix items: raw_value's cost to the import pipeline is stated (clause 12, step 5); the stale text is corrected; the ceiling is sized to the grouped read; and the exact read's cost is now measured. It is about 50× the revision 5 estimate, which is why question 16 is new.

It also corrects when a last-bit miss is visible to =. The bound is on the terms, not the total, as ADR 009 now says at 4973845f (Codex on #309).

Revision 7 folded in the user's first answer to question 16, left inline on b232c5a9 at 03:37 UTC on 2026-10-09: "let's do k = 32". It made K 32, option (a). Revision 8 supersedes that answer (below). Every question 1–16 was then answered, and the ADR stayed proposed until the user says "Approved". The change is confined to the value of K and the text that waited on it; the figures that depend on K were re-derived from the appendix's own measurements and none is new (question 16's "Re-derived" notes list them, now for K = 8). The revision also adds "Relations to other ADRs", a note on what ADRs 007, 008 and 009 ask of this one since it was written. It changes none of the user's decisions, and it lists the places where the other ADRs' text and this one's disagree.

Revision 8 folds in the user's second answer to question 16, a PR-conversation comment at 04:04 UTC on 2026-10-09: "Ok, let's drop K back to 8". It supersedes "let's do k = 32": K is 8, option (b), which was the recommendation of question 16. The earlier answer stays on the record, quoted in "What the user decided". The user gave no reason for the change, and this ADR does not supply or guess one. Every dependent figure was re-derived from the appendix's own measurements (question 16 lists them), and none is new. The K = 8 cutoff is still the second exception to "exact iff it fits" (question 13), and still overrides representability.

What the user decided

The user answered inline on e4acf643, one comment per question. Each answer is quoted verbatim, with what it changed.

1. The tower's shape.

"I think, with bounded fractions, we need at least 3"

Decided: three levels. Exact decimal, exact bounded fraction and inexact f64 (clauses 1 and 2).

2. Non-terminating exact division.

"(d)"

Decided: option (d), bounded exact fractions, in the hybrid form. A non-terminating quotient that fits is exact (clause 5, and the option (d) section, now normative).

3. A tolerance on inexact =.

"Exact comparison everywhere but add equal(a,b,epsilon) to formula language"

Decided: no tolerance on =. A new formula function, EQUAL(a, b, epsilon), is the explicit tolerant comparison (clause 6). It replaces the earlier "an APPROX if it is ever wanted".

4. The .improv spelling of an inexact finite number.

"#i works"

Decided: #i (clause 10).

5. The version literal.

"share with ADR 007"

Decided: share ADR 007 clause 10's new literal, unconditionally (clause 10).

6. JSON import precision, and the wire.

"I think we either have to add our own encoding as strings on the wire or use arbitrary precision: the round trip through f64 seems dangerous (unless maybe we tag f64s as exact somehow like NaN boxing)"

Decided: no value crosses as a JSON number, and no number is recovered by a round trip through f64.

  • The wire spells every number as a string in its own encoding (clause 11).
  • JSON import reads each number's source lexeme, not the f64 serde_json made of it (clause 12).
  • NaN boxing is not needed once the encoding is a string: the string carries the exactness.
  • serde_json's arbitrary_precision stays rejected, for a reason the string encoding does not remove: it breaks serde's buffered deserialization of numbers. The workspace has an internally tagged enum (improvise-wasm-client/src/render.rs:86) and a #[serde(flatten)] (gp-tree/src/data.rs:64), and the feature is unified across the workspace. So the lexeme is read with serde_json's raw_value feature, which is additive and changes nothing else.

7. Comparing an exact value with an inexact one.

"Exactly, but add equal(a,b,epsilon) to formula language."

Decided: exact comparison, with the same EQUAL function as question 3 (clause 6).

8. The dependency.

"yes, this seems fine."

Decided: yes. One figure changed after the answer, and it is flagged here.

  • The user answered against option (a)'s +15.4 KB gzipped for rust_decimal alone.
  • Under (d), numtower also carries num-rational and num-bigint. The probe measured those at about +20.8 KB gzipped on their own; together with rust_decimal they are unprobed (clause 14).
  • If that changes the answer, question 8 reopens.

9. Version skew on the wire.

"yeah, accept and add a ticket for handshake version."

Decided: accept the skew. The handshake version is filed as improvise-szi8.

10. The exact-sum accumulator.

"Don't we need a much bigger accumulator (2000ish bits) to handle exact accumulation generically?"

That was a question back to the ADR, answered with numbers in clause 8, "How wide, generically". In short:

  • Decimals need 320 bits, because their range is small.
  • f64 values need about 2,100 bits. That is ADR 009's accumulator.
  • Fractions have no fixed generic width. Combining k arbitrary i128 denominators needs up to 127·k bits. The user's ~2,000 bits is exactly k = 16.

Then, on 33833f28, on the line "decimals need 320 bits":

"320bits is fine"

and on option (a):

"(a)"

Decided: (a). A 320-bit decimal partition, a 256-bit numerator per fraction group, and a read that combines at most K groups in a bigint rational and then applies the stored bound. One figure changed after the answer, and it is flagged here: the exact read was an estimate of 16–34 µs at K = 32, and it measures 0.67–1.9 ms in the worst case (option (d), "What that costs"). K is a constant inside numtower, not a format or wire fact, so its value was split out as question 16 rather than reopening this one. The user decided K = 8 there (first K = 32, then K = 8; the later answer governs).

11. The MIN/MAX tie-break.

"Yeah, exact."

Decided: prefer the exact value (clause 8).

12. Typing a fraction.

"keep as text for type inference from strings because of dates, read as fraction in places where the type is forced."

Decided: n/d is text wherever a type is inferred, and a fraction wherever the type is forced. So a typed or imported 3/4 whose type is being guessed stays text, which keeps dates. A column the user has forced to a measure reads 3/4 as a fraction. Clause 12 defines the one number language this uses.

13. Where the fraction bound applies.

"(b)"

Decided: (b). The bound applies to the stored result, and every intermediate inside one formula evaluation is unbounded up to the ceiling. The ceiling is the first of two exceptions to "exact iff it fits", sized to the grouped read (question 13). The second, added by revision 7, is the K cutoff on a SUM, K = 8 since revision 8 (question 13, "The K cutoff").

14. The numeric crate's name.

"this is fine"

Decided: numtower (clause 14).

15. Where ADR 009's ExactSum lives.

"(a)"

Decided: (a), in numtower, beside the exact decimal accumulator. ADR 009's seam crate adapts it to zset's group trait through a newtype, and numtower can land holding ExactSum alone, ahead of this ADR (clause 14).

16. The value of K. The user answered twice. Both are quoted verbatim.

First, inline on b232c5a9, on question 16's options (2026-10-09T03:37:34Z, comment 4226381911):

"let's do k = 32"

Then, in a PR-conversation comment (2026-10-09T04:04:36Z, comment 6074029223):

"Ok, let's drop K back to 8"

Decided: (b), K = 8. The later answer governs, and the first is superseded, not deleted. K = 8 is the recommendation of question 16, and not the value revision 5 proposed and question 10 was answered with. The worst-case exact read is 52–127 µs (52 µs is the lcm method, the one numtower uses; 127 µs reduces after every add). The combined numerator is at most K·127 + 256 = 1,272 bits (before the up to 3 carry bits of summing 8 terms), which the proposed 8,192-bit ceiling clears by 6,920, enough to absorb them. A SUM over more than 8 distinct non-decimal denominators is inexact, and the cutoff overrides representability (question 13). K remains a constant inside numtower, so it can still be tuned later against the evaluator benchmark without a new ADR.

The decision this ADR writes down

  1. Three levels (decided, question 1). A number is an exact decimal, an exact bounded fraction (an i128 numerator and denominator), or inexact (an f64). There are no unbounded rationals. They were measured, and they fail on the most ordinary pivot-table question, the sum of ratios (clause 1).
  2. Exact is the default. Every number a person types, every number in a .improv file, every CSV or JSON number and every formula literal is exact when it fits either exact arm: a decimal first, then a bounded fraction (clause 3).
  3. Exactness is lost only in named places. +, -, * and / keep it: a quotient is a decimal when it terminates and fits, and otherwise a bounded fraction (decided, question 2). POW keeps it for an integer exponent. A non-integer exponent, a result that fits in neither exact form, and any inexact operand make the result inexact. That is Scheme's contagion rule.
  4. The epsilon goes. = and != compare exactly, and the division-by-zero guard tests for exactly zero. The two still share one predicate, which is the property gotcha 17 exists to keep. A tolerant comparison is explicit: EQUAL(a, b, epsilon) (decided, questions 3 and 7; clause 6).
  5. The stored type is CellValue::Number(Num), where Num is Exact(Decimal), Fraction(Box<Ratio<i128>>) or Inexact(f64). The crates are rust_decimal and num-rational. Num and everything numeric live in a problem-agnostic workspace crate of their own, numtower (decided, question 14; clause 14).
  6. Each kind of number has its own exact accumulator. ADR 009's f64 accumulator makes SUM over inexact values order-free, and this ADR does not re-specify it. The exact values get a wide fixed-point integer of their own (clause 8), because rust_decimal's own addition rounds at its bound and so is not order-free there. What this ADR adds beyond summation is everything the f64 accumulator cannot reach: per-cell arithmetic, typed money-like values, exact comparison, and showing a typed value back as it was typed. Clause 8 states the boundary, and the measurement behind it.

Context

How a number flows today

Every number is an f64 from the moment it is parsed until the moment it is drawn.

where what file:line
the stored type CellValue::Number(f64), serialized through wire_f64 crates/improvise-core/src/model/cell.rs:116-122
non-finite tokens nonfinite_token / parse_nonfinite (inf, -inf, nan) cell.rs:63-82
the wire adaptor JSON number when finite, token string when not (improvise-55v2) cell.rs:84-114
editor seed CellValue::edit_text: Rust's shortest round-trip spelling (improvise-umi4) cell.rs:137-145
screen spelling Display: integer, else {:.4} cell.rs:148-161
cell aggregation apply_agg over &[f64]; aggregate_cell_values filters as_f64 crates/improvise-core/src/model/types.rs:20-44
formula aggregation eval_agg, values.iter().sum() in store order types.rs:57-87
the epsilon FLOAT_EQ_EPSILON = 1e-10 (improvise-0bf) types.rs:16
record-level eval eval_expr: lv + rv, lv / rv behind the epsilon, lv.powf(rv) types.rs:229-282
comparison eval_bool: (lv - rv).abs() < FLOAT_EQ_EPSILON types.rs:314-315
the cached twin eval_expr_cached, the same arithmetic again types.rs:1090-1104, 1156-1157
formula literal Expr::Number(f64), from number = ASCII_DIGIT+ ("." ASCII_DIGIT*)? | "." ASCII_DIGIT+: no sign, no exponent, no digit bound, and 1. and .5 are legal crates/improvise-formula/src/ast.rs:35, formula.pest:105-108, parser.rs:192
typed input typed(): text.parse::<f64>(), else text (improvise-j7v6) crates/improvise-protocol/src/resolve.rs:1438-1446
display round_half_away scales by 10^d in f64, then format! crates/improvise-core/src/format.rs:125-160
.improv write format_number: integer, else shortest Display, else {:?} crates/improvise-io/src/persistence/format.rs:38-54
.improv grammar number = "-"? digits ("." digits)? (exponent)? crates/improvise-io/src/persistence/improv.pest:110-112
.improv read parse_number_cell: s.parse() into f64 crates/improvise-io/src/persistence/parse.rs:141-146
CSV field i64, else f64, into a serde_json::Number crates/improvise-io/src/import/csv_parser.rs:167-186
import measures read back with as_f64, which drops an exact i64 of more than 15 digits crates/improvise-io/src/import/wizard.rs:425
record fingerprint hashes a number's bits, not its PartialEq (ADR 001's record tier) crates/improvise-command/src/ui/app.rs:231-250
JSON import serde_json::Value, numbers already f64 (float_roundtrip) src/main.rs:365, crates/improvise-command/src/ui/effect.rs:1151, 1280
import collisions the "sum" policy adds in f64, in record order (improvise-cgqt) crates/improvise-io/src/import/wizard.rs:307, 330
the wire ModelCommand::SetCell { value: CellValue }, CellDisplay::value, SetYanked crates/improvise-protocol/src/reduce.rs:33, 88-92; render.rs:88-104
gp seam cell_text: a number crosses as text, n.to_string() crates/improvise-gp/src/dataset.rs:50-56

grep -rn 'CellValue::Number' --include='*.rs' . finds 672 occurrences in 47 files, tests and benches/ included. That is the blast radius of any change to the stored type.

What is already wrong, measured

The measurements in this ADR were taken on an Apple M4 Max (64 GB), rustc 1.98.1, release builds, by a scratch program described in the appendix. They were not committed, for the reason the appendix gives.

Per-cell arithmetic is wrong in the last place, and the user sees it.

  • 0.1 + 0.2 is 0.30000000000000004, and ten 0.1s sum to 0.9999999999999999.
  • 1234.56 - 1234.55 is 0.009999999999990905, and 100.10 - 100 is 0.09999999999999432.

A display format hides most of this. It does not hide it from =, from an editor, or from the next formula.

Display rounding is wrong for a typed value. format.rs:127-136 rounds by scaling the f64. A typed 1.005 is stored as 1.00499999999999989…, so at two places the grid shows 1.00. A person who typed 1.005 and asked for two places expects 1.01. Half-away-from-zero on the decimal gives 1.01. 2.675, 0.125 and 1.115 happen to agree on both paths. The review found more that miss: 1.015 draws 1.01, 1.025 draws 1.02 and 0.285 draws 0.28. So the bug is real but intermittent, which is the worst kind for a spreadsheet. It is filed on its own as improvise-felo.9, because it can be fixed without this ADR: round the shortest round-trip spelling as a decimal.

SUM depends on order, and on how far it is from what was typed. SUM over 18,000 typed two-place decimals (0.01 to 999,999.99, seed 1):

  • the forward and the reverse f64 fold differ by 5.0e-5;
  • the forward fold is 5.3e-5 from the exact decimal total (1.4e-4 at 90,000 cells);
  • the decimal sum (rust_decimal) is identical in both orders for this data. It is not identical in general: at the 28-digit bound rust_decimal rounds, and then order matters again (clause 8).

ADR 009's own measurement found the same thing with five-place values: the eager sum is up to 5.8e-10 from the exact decimal sum, above FLOAT_EQ_EPSILON at 7 of 8 checkpoints (improvise-l2b8's comment). Gotcha 18 recorded the order dependence as a real bug (improvise-shyt), fixed by pinning iteration order rather than by removing it.

The epsilon is a patch, and it does not scale. It was chosen because it "absorbs accumulated f64 rounding noise" (context/code/core.md §4). But one ulp of 1,000,000 is 1.16e-10. So once a value is above about 2¹⁹ (524,288), a one-ulp error in it is already larger than the epsilon, and = sees it. Below about 1e-10, the epsilon calls different numbers equal.

What ADR 009's exact accumulator fixes, and what it does not

ADR 009 now proposes a superaccumulator: a fixed-point integer wide enough for every finite f64. It sums f64 values exactly and rounds once, when the sum is read. It makes SUM independent of order, and its retraction exact, without this ADR. That changes what this ADR is for, so it was measured rather than assumed.

The question: after a perfect f64 accumulator, does the sum of typed decimals equal the typed total? The stand-in for the accumulator was an exact rational sum of the stored f64 values, rounded once (appendix).

The miss rate depends on the values' distribution, not only on how many are summed, so two distributions were measured. The first is positive two-place values uniform in 0.01–999,999.99, as the rest of this ADR uses. The second is two-place values uniform in ±1,000.00. The ADR 009 agent found it with an independent Python Fraction check (about 37–40%), and it was reproduced here with the same stand-in (seed 7).

typed two-place values summed trials accumulator ≠ typed total, 0.01–999,999.99 accumulator ≠ typed total, ±1,000.00 naive f64 fold ≠ typed total, 0.01–999,999.99
2 10,000 2,120 (21%) 3,792 (38%) 2,120 (21%)
3 10,000 1,621 (16%) 3,836 (38%) 2,683 (27%)
10 10,000 752 (7.5%) 3,950 (40%) 4,039 (40%)
100 10,000 250 (2.5%) 4,018 (40%) 7,944 (79%)
18,000 8 seeds 0 5 of 8 seeds (the review) (all 8: see above)
90,000 8 seeds 0 2 of 8 seeds (the review) (all 8)

The signed column's large-N cells were measured by the review of e4acf643, with the same generator and stand-in. It also measured signed N = 1,000: 84 of 200 trials (42%). At 18k it found 16 of 40 trials with seed 7, and 5 of 8 across seeds 1–8. So "about 40% at every length" holds out to 18k.

The accumulator helps only where the total is large beside its terms. With positive values the total grows with the count, its ulp grows with it, and the input rounding errors cancel far below that ulp. So the one rounding lands on the typed total, and over thousands of cells it always did. With signed values the total stays small, because the terms cancel. The input errors are then comparable to the total's ulp at every count, and about 40% of sums miss however many terms there are. Revision 2 said "the accumulator solves the large sum". That holds only for same-sign data, and it is corrected here. For two summands a superaccumulator is exactly +, because one f64 addition is already correctly rounded. So 0.1 + 0.2 is still 0.30000000000000004 under the accumulator.

The misses are in the last bit of the largest term, not of the total. Each stored term is off from its typed value by at most half an ulp of that term. So the exact sum of the stored terms is off from the typed total by at most about ½·Σ ulp(term), plus the total's own half-ulp at the final rounding (revision 6; revision 5 put the condition on the total, and ADR 009 made the same correction at 4973845f, from ChatGPT Codex on #309). When terms cancel, a miss can be far larger than the total's ulp:

  • The large-term, small-total case. 1000000000000000.10 + (-1000000000000000.00) has a typed total of 0.10. The first term is stored as 1e15 + 0.125, because the ulp at 1e15 is 0.125. So the exact sum of the stored terms is 0.125, and the miss is 0.025 on a total of 0.1.
  • When = sees one today. = is an absolute 1e-10 (gotcha 17), so a miss can reach it only when ½·Σ ulp(term) reaches 1e-10. That needs at least one term of about 2¹⁹ ≈ 5.2e5 or larger, or many terms just below it. It does not depend on the total. The ±1,000 values are far below 2¹⁹, so today's epsilon hides their misses; the positive values up to 999,999.99 pass it.
  • Once this ADR deletes the epsilon (clause 6), every miss is visible to =.

A person reading a subtotal at two places usually sees the right number, because the miss is below the displayed digits unless the terms are large. The miss also shows in the editor, which opens on the shortest round-trip spelling (0.30000000000000004). That is where 16% of positive 3-term sums, and about 40% of signed sums of any length, go wrong.

So the division of labour is:

ADR 009 accumulator this ADR
SUM/AVG independent of order, exact retraction yes, for inexact values yes for exact values, through its own wide fixed-point accumulator; it uses ADR 009's for inexact ones (clause 8)
a large same-sign SUM equal to its typed total yes, measured above yes
a small SUM, or a signed SUM of any size, equal to its typed total no: 16% of positive 3-term sums miss, and about 40% of signed sums yes
per-cell arithmetic (Revenue - Cost, * 1.1, 0.1 + 0.2) no yes
exact = against a typed value, with no epsilon only where the sum happens to round right yes
a typed value shown back as typed (12.50, 1.005 at two places) no yes
cost 568 bytes per group, 2.0–3.6 ns per update, 0.6–1.1 µs per read (ADR 009's figures) 16 bytes per value; about 5–8 ns per add, 60–110 ns per division; 40 bytes per summed group (clause 8, question 10; an estimate, not measured)

This ADR does not re-specify the f64 accumulator. Where an inexact value must be summed, it is summed by ADR 009's accumulator, and this ADR only says so. ADR 009's "what each exact domain covers" table says that accumulator makes = against a typed decimal right "only for sums". The measurement above narrows that to large same-sign sums. ADR 009 says "no, in general" since 8ebcf889, and cites both distributions, so the two tables agree.

The crates

Three crates were evaluated. All three build for wasm32-unknown-unknown.

rust_decimal 1.43.0 bigdecimal 0.4.10 num-rational 0.4.2 (BigRational)
value 96-bit integer and a scale up to 28 BigInt and an i64 scale BigInt / BigInt
inline size 16 bytes, Copy, no heap 40 bytes and a heap buffer 64 bytes and two heap buffers
+ - * exact within 28 significant digits; beyond that it rounds silently (1000 + 1/3 comes back at scale 25) always exact always exact
/ rounds to 28 significant digits (1/3*3 ≠ 1) rounds to 100 digits by default exact
parsing past the bound from_str rounds silently (30 digits come back as 28); from_str_exact refuses exact exact
to f64 to_f64 is not correctly rounded (clause 4) — —
normal-edge deps arrayvec, num-traits, serde libm, num-bigint, num-integer, num-traits num-bigint, num-integer, num-traits
wasm size, marginal +35.9 KB, +15.4 KB gzipped +77.9 KB, +30.7 KB gzipped +59.6 KB, +20.8 KB gzipped
SUM, ns per element (18k / 90k) 5.7–8.1 / 4.5–8.5 14.5–22.9 / 10.8–20.7 609–1,057 / 491–989
(Rev − Cost) / Rev, summed, ns per record 65–111 / 58–112 2,429–5,353 5.3–6.8 ms, abandoned (below)

The f64 references are 0.54–1.18 ns per element for SUM and 0.56–1.24 ns per record for the formula. Each range is two full runs. 1.5 == 1.50 and the two hash equally in rust_decimal. The wasm figures are wasm-opt -Oz over a module that already carries f64 parse and print, as the real bundle does, so they are the marginal cost. They are from a five-operation probe, not the real bundle, and the real cost should be re-read when the dependency lands (question 8).

Exact rationals fail on the sum of ratios, and it is not close. The ticket asked what bounds denominator growth. Summing (Revenue − Cost) / Revenue over random typed records, the denominator of the running sum grew as follows:

ratios summed 256 512 1,024 2,048 stopped (20 s budget)
denominator bits 4,699 9,103 17,015 31,697 44–55 k, after 2,936–3,741 records

Nothing bounds it. Each new revenue brings new prime factors into the least common denominator, so it grows linearly in bits and each addition costs more than the last. At the stopping point one addition cost milliseconds. An 18,000-record average margin does not finish. That is the most ordinary question a pivot table is asked, so exact rationals as the default are rejected on measurement, not taste.

Decision

1. Three levels: exact decimal, exact bounded fraction, inexact binary

Scheme's tower is integer ⊂ rational ⊂ real ⊂ complex, and each level has an exact and an inexact form. Improvise keeps three levels (decided, questions 1 and 2):

  • Exact decimal: rust_decimal::Decimal. Integers are decimals with scale 0. Typed values, money and a typed scale such as 12.50 live here.
  • Exact bounded fraction: a reduced Ratio<i128>. It holds only exact values that are not a finite decimal that fits, such as 10/3 (the option (d) section).
  • Inexact: an f64, Scheme's flonum.

There are no unbounded rationals, for the reason measured above, and no complex numbers, because nothing produces one. The exact levels are bounded. A decimal has 28 significant digits and a magnitude below about 7.9 × 10²⁸, and a fraction has an i128 numerator and denominator. Scheme's exact numbers are unbounded, so this is the one place the tower deviates from Scheme by design. Clause 4 and question 13 say what happens at the bound.

2. The stored type: CellValue::Number(Num)

pub enum Num {
    Exact(Decimal),
    Fraction(Box<Ratio<i128>>), // boxed: keeps Num at 24 bytes (option (d))
    Inexact(f64),
}

pub enum CellValue {
    Number(Num),
    Text(String),
    Error(String),
}

Num lives in the numeric crate (clause 14), and CellValue in improvise-core wraps it. The arithmetic and the contagion rules are methods on Num, in that one crate. Both evaluators (eval_expr and eval_expr_cached) call the same methods, so the rules cannot drift between them the way the epsilon once did (improvise-0bf).

A new arm beside Number(f64) was rejected. It would double every numeric match at the 672 sites, and "is this cell a number" could then be asked two ways. CellValue::as_f64 stays, as the explicitly lossy accessor for the seams that want a binary float (clause 12). It uses clause 4's correctly rounded conversion.

Expr::Number(f64) becomes Expr::Number(Num), not Expr::Number(Decimal). The formula grammar has no digit bound, so a literal such as 999999999999999999999999999999 is legal formula text and has no Decimal. It takes clause 3's order, like every other spelled number: a decimal if it fits, else a fraction if it fits (that literal is below 2¹²⁷, so it is the exact Fraction n/1), else the nearest f64. A literal is a number by the grammar, so clause 3's gate, which decides number-or-text for data, does not apply to it. A literal that overflows f64 (the formula grammar has no exponent, so a digit string of more than 309 digits) keeps today's meaning, inexact infinity, the same as an inexact overflow in *. Bounding the grammar instead was rejected, because it would make a formula that parses today stop parsing, over a limit of the representation rather than of the language. (Codex, P2 on 1fe3a85d.)

3. What is exact on entry

Everything a person or a file spells in decimal is exact, when it fits. That covers typed input (typed()), .improv numbers, CSV fields, formula literals and pasted values.

This deviates from Scheme on purpose. Scheme reads 1.5 as inexact and needs #e1.5 for exact. Improvise reads 1.5 as exact, because a pivot table's inputs are decimals a person typed, and "decimal math by default" is the user's instruction. So Scheme's #e is improvise's default.

A spelled number is inexact only when it cannot be exact: it fits neither a Decimal nor an i128 fraction (for example 1e300, 1e-40, or a 40-digit integer). Then it reads as the nearest f64. The tokens inf, -inf and nan are inexact too, but they are clause 7's tokens, not numbers of the language (below).

The parse must refuse, not round. Decimal::from_str rounds a 30-digit input to 28 digits silently (0.12345678901234567890123456789 comes back as 0.1234567890123456789012345679). That is a rounded value calling itself exact, the one outcome clause 4 forbids. Under the order below, that input is the exact fraction 12345678901234567890123456789/10²⁹ instead.

One number language, for every entry point (revision 5, answering B4 of the review of 87ae21ca). Revision 4 said "parse with from_str_exact", but the gate that decides whether a string is a number at all accepts different strings. from_str_exact rejects 1.5e3 and accepts 1_000, and #321's number_in gate does the opposite. So the language is defined once, and the gate and the parse cannot disagree. numtower owns it.

  • What is a number. The gate is the strings Rust's f64::from_str accepts and reads as a finite value (revision 6, N1 of the review of 33833f28). That is #321's number_in gate, which drops anything Number::from_f64 cannot hold. So a number is an optional sign; digits with an optional fraction (1., .5); and an optional exponent e/E with its own sign, whose value does not overflow f64.

    • 1_000 is not a number, so it is text.
    • nan, inf, infinity in any case, and a finite lexeme that overflows f64 (1e999, -1e999) are not numbers either, so they are text. That is improvise-jc95's decision, which #321's a_nonfinite_numeric_token_stays_text_rather_than_becoming_zero pins for NaN, inf, -inf, 1e999 and -1e999. A finite lexeme must never read as infinity.
    • A lexeme that underflows (1e-999) passes, because f64 reads it as a finite zero. It is then too small for either exact arm, and it reads as that zero, inexact. #321 treats it the same way.
  • The non-finite values are clause 7's tokens, not numbers. typed(), the .improv reader and the wire accept exactly inf, -inf and nan, the three spellings nonfinite_token writes (cell.rs:65), as inexact values. Import does not, so a CSV nan stays text, as jc95 decided. This is a behaviour change for typed(), and it is stated: today it hands any string to str::parse::<f64> (resolve.rs:1438), so a typed Infinity or 1e999 becomes infinity and a typed NaN becomes NaN. Under this clause each is text.

  • How it parses. A lexeme that passes the gate is parsed exactly from its digits and exponent: a mantissa and a scale shift, never through an f64. Then it takes the first arm it fits, in this order (revision 6, N2 of the same review):

    • A decimal. If the exact value fits a Decimal, it is Exact, with the scale the lexeme implies: 1.50 has scale 2, 1.5e3 is 1500 at scale 0, and 1.5e-3 is 0.0015.
    • A fraction. Otherwise, if its reduced value m / 10^k fits Ratio<i128>, it is a Fraction. So a typed 1e-30 is the exact 1/10³⁰, and an integer between Decimal's maximum (about 7.9 × 10²⁸) and 2¹²⁷ is the exact n/1. This is the order clause 4 already gives arithmetic, so the same value is equally exact typed and computed: 1e-15 * 1e-15 was already the exact 1/10³⁰. A fraction carries no scale (the option (d) section).
    • The nearest f64. Otherwise it is f64::from_str's correctly rounded value, Inexact. The gate has already made it finite.
    • A lexeme that passes the gate is never refused. Failing to fit falls back; failing to parse cannot happen.
  • Zero has no sign at the exact level. -0.0 reads as exact 0.0. The sign of zero is an f64 artefact. This is a behaviour change, and it is stated: an f64 -0.0 never survives entry as exact.

  • n/d is a number only where the type is forced (decided, question 12). n/d means an optional sign, digits, /, and digits with d > 0, and it is a fraction or, once normalized, a decimal. The places where the type is forced are:

    • the .improv fraction arm (clause 10);
    • the wire's exact arm (clause 11);
    • an import column the user has forced to a measure (clause 12).

    Everywhere a type is inferred it is text: typed(), and the import analyzer deciding a column's kind. That keeps 3/4 as a date or a label.

The entry points are typed(), the .improv reader (whose number rule is a subset of the gate before the finite check), formula literals, the wire and import.

Reversing jc95 was rejected. The alternative was a gate of every string f64::from_str accepts, so that a CSV nan imports as Inexact(NaN) and a CSV 1e999 as Inexact(inf). It would make a finite lexeme read silently as infinity, which is the invented-value class jc95 closed, and it would flip #321's test. The gain, a measure column that can hold a NaN, is not worth either: a column carrying nan is shown as text in the wizard, where the user can see it (jc95's close reason).

A JSON number is read from its source lexeme, never through its f64 (decided, question 6: "the round trip through f64 seems dangerous"). serde_json normally parses a number to f64 before improvise sees it. The import therefore reads each record's values with serde_json's raw_value feature, which hands over the number's exact source text, and parses that text by the language above. So a JSON 12.50 keeps its scale, and a 20-digit JSON literal stays exact. That costs a restructuring of the import pipeline, stated in clause 12. arbitrary_precision was the other way to do this, and it stays rejected, because it breaks serde's buffered deserialization of numbers (the user's answer to question 6). Revision 4's shortest-spelling recovery of a JSON number is withdrawn.

4. How exactness propagates

operation exact ∘ exact any inexact operand
+, -, *, unary - exact, if the result is exactly representable; else inexact inexact
/ clause 5 inexact
POW clause 5 inexact
IS, IF not arithmetic: IS yields exact 0 or 1 —

Contagion is Scheme's rule. An inexact operand makes the result inexact, and an exact result never comes from an inexact input.

"If exactly representable" needs care, because rust_decimal rounds silently. An add whose result needs more than 28 significant digits comes back rounded, with no signal. The measurement caught it: 1000 + 0.3333333333333333333333333333 returns scale 25. Overflow of the magnitude is reported (checked_add returns None), but precision loss is not. So Num's arithmetic must detect either one and return Inexact. It must never return a rounded Exact. A rounded value that calls itself exact is the one outcome this design exists to prevent.

Detection reads the scale, then checks that the dropped digits were not zeros (revision 7, Copilot on 66f317e1). The exact result of a + b or a - b has scale max(sa, sb), and of a * b has scale sa + sb. A returned scale that is smaller is a hint of rounding, not proof. 1000 + 0.333… (28 places) comes back at scale 25, and 0.1234567890123456² at 28 against an exact 32. Revision 6 demoted on the hint alone and called the zero-digit case a safe false positive. It is not safe: it breaks question 13's decided rule that a stored value is exact iff it fits. 1.2500000000000000 * 4.0000000000000000 (scale 16 each) has the exact value 5 at scale 32, which is 5·10³² as an integer and does not fit 96 bits, yet 5 fits a Decimal. So on the hint the arithmetic recomputes the exact integer result in the 256-bit integer clause 8 already uses (a product of two 96-bit mantissas is below 2¹⁹², and an aligned sum below 2¹⁹⁰), strips trailing zeros, and returns exact iff the stripped mantissa fits 96 bits at a scale of at most 28. The stored scale (flagged reading, revision 7, so the fingerprint of clause 9 is a function of the value and the operands alone): the nominal scale above capped at 28, min(nominal, 28), padded back with zeros while the mantissa still fits 96 bits, and otherwise the largest scale at which it fits. So the example is 5 at scale 28 (5·10²⁸ fits), not scale 0 and not 32. A result that never fired the hint keeps its returned scale, so the common path is unchanged. Otherwise the value is not a Decimal, and goes to the fraction arm and then f64 as below. The recompute runs only when the scale hint fires, so the common path is unchanged. This is a design for the implementer and is not measured. Division has its own test (clause 5).

This is not rare in practice, and the review showed why. Clause 10 loads an old file's 17-digit f64 spellings as exact, so 0.30000000000000004 + 1e12 needs 29 significant digits and is rounded. That is the case clause 8's accumulator exists for.

The one conversion to f64 is correctly rounded, and it is not the crate's. Demotion, mixed comparison (clause 6), as_f64 and the reading of a mixed sum (clause 8) all turn an exact value into an f64. rust_decimal's to_f64 is not correctly rounded. In 200,000 random Decimals with 96-bit mantissas and scales 0–28, it disagreed with a correctly rounded parse of the same value's text 17,392 times in the review's sample and 17,375 times in this one (about 8.7%). For example, 63294.607176721429540940386554 gives the f64 ending …144, where the nearest is …143. With mantissas of at most 15 digits there were no disagreements, but exact products and sums leave that range quickly. So Num converts through the decimal string: d.to_string(), then Rust's str::parse::<f64>, which is correctly rounded. That costs an allocation, paid only where an exact value leaves the exact level. A correctly rounded integer algorithm can replace it later, and it must agree with the string path on a property test.

So "computed in f64" above means: convert each exact operand by this conversion, then do the f64 operation. A demoted sum or product is therefore rounded twice, once per operand and once in the operation. That is deterministic, and it is the Scheme reading of an operation that could not stay exact.

5. Division, AVG and POW

Division. Exact ÷ exact is always exact when the result fits (decided, question 2).

  • It is a decimal when the quotient is a finite decimal that fits: 1 / 4 = 0.25.
  • Otherwise it is a bounded fraction when the reduced fraction fits i128: 1 / 3, so 1 / 3 * 3 is exactly 1.
  • Otherwise it is inexact: the correctly rounded f64 of the exact quotient.

The implementation must decide termination and fit without trusting rust_decimal's division, which rounds a terminating quotient silently when it needs more than 28 places (below).

The termination test is arithmetic, not a round trip. Write a / b as the fraction (ma · 10^sb) / (mb · 10^sa), with mantissas ma, mb and scales sa, sb. The powers of ten contribute only 2s and 5s, so the quotient is a finite decimal exactly when mb / gcd(ma, mb) is 2^p · 5^q. That takes one i128 gcd over the mantissas (each below 2⁹⁶, so nothing overflows) and a loop dividing out 2s and 5s, with no rounding anywhere.

Terminating is not enough: the quotient must also fit, and that test is specified here (revision 4, answering the review of e4acf643). Clause 4's scale test covers + − × only, because a quotient's scale is not a function of its operands' scales. The review measured 1 / 2^40: it terminates, but it needs 40 fractional digits. rust_decimal returns 0.0000000000009094947017729282 at scale 28, with no signal, and an exact quotient can legitimately have scale 28 too. So the result alone cannot tell the two apart. The rule:

  • Write the quotient as the reduced fraction n / (2^p · 5^q), computing it in 256 bits, because ma · 10^sb can reach 2¹⁹⁰.
  • Its exact scale is k = max(p, q).
  • The quotient is an exact Decimal iff k ≤ 28 and n · 2^(k−p) · 5^(k−q) fits in 96 bits.
  • Otherwise, including every quotient that terminates but needs more than 28 places, it is a Fraction when the reduced fraction fits i128.
  • Otherwise it is inexact. The one conversion is the reduced 256-bit fraction, converted once, correctly rounded (clause 4's interval-checked rational → f64). It is not a conversion of each operand followed by an f64 division. Revision 4 said "the correctly rounded f64 … through each operand's conversion and one f64 division", which contradicted itself: two roundings are not a correct rounding. The fraction is already in hand, so it is converted once.

The direct test above is normative (revision 5, the review's correction to D1). Multiplying back with clause 4's scale detection is sound: q × b exact and equal to a implies q = a / b. But it is incomplete, because it demotes some exact quotients. 1e-28 / 0.5 = 2e-28 is exact and fits (k = 28, mantissa 2), yet q × 0.5 needs scale 29, so the check flags it. The review measured this case. If the method were left to the implementer, exactness would depend on which method was picked. So the direct test is the rule, and multiplying back is not an allowed substitute.

Scheme would answer 1/3 exactly, and so does improvise, within the bound. Unbounded rationals are ruled out by the measurement above.

AVG is SUM / COUNT under the division rule. The average of 10.00, 20.00 and 40.00 is 70/3, an exact fraction, which displays as 23.33 at two places. The average of 10.00, 20.00, 30.00 and 40.00 is exactly the decimal 25.

POW. x ^ n with exact x and an exact integer n ≥ 0 is exact (repeated multiplication, with clause 4's overflow rule). A negative integer n is 1 / x^|n|, under the division rule. The size is checked before computing. The result of (n/d)^k has about k times the operand's width, which is known in advance. An exponent whose result cannot fit, or would pass question 13's ceiling, is decided before any multiplication, so POW(x, 10^9) allocates nothing and demotes to powf at once. Any non-integer exponent, or any inexact operand, is powf, inexact. This is the ticket's "obvious candidate" for an inexact operation, and it is the only one besides division: improvise has no transcendental functions.

Division by zero is Error("div/0") for every combination of exactness, as today. The test is exactly zero, with no epsilon (clause 6).

6. Comparison, and the end of FLOAT_EQ_EPSILON

=, !=, <, >, <= and >= compare exactly. FLOAT_EQ_EPSILON and its six uses are deleted.

  • Exact with exact: exact comparison. Two decimals compare as decimals, and a fraction against a decimal or a fraction by cross-multiplying in 256 bits. 1.5 = 1.50 is true, because scale is presentation (clause 9), not value.
  • Inexact with inexact: f64 comparison, as today but with no epsilon (decided, question 3).
  • Exact with inexact: compared exactly, as values. A finite f64 is a finite binary fraction, so this is well defined. Revision 1 converted the exact operand to f64 instead, and the review showed that rule is non-transitive for ordinary values, not only at a tie: any two distinct decimals with the same nearest f64 break it. Exact(0.1) = Inexact(0.1) and Exact(0.10000000000000000001) = Inexact(0.1) would both be true, while the two exacts differ. A non-transitive = cannot order a MIN, and it cannot key a sorted map. So comparison is exact (decided, question 7). The cost is that Exact(0.1) = Inexact(0.1) is false: the f64 is 0.1000000000000000055…. That is Scheme's own answer (R7RS requires = to be transitive), and it is the honest one.

Exact mixed comparison is cheap in the common case. Convert the exact operand by clause 4's correctly rounded conversion. If the result differs from the f64, it already decides the order, because rounding is monotone. Only when the decimal rounds to that very f64 is an exact tie-check needed, against the f64's exact decimal expansion, which is finite.

EQUAL(a, b, epsilon): the explicit tolerant comparison (decided, questions 3 and 7: "Exact comparison everywhere but add equal(a,b,epsilon) to formula language").

  • Meaning. True iff |a − b| ≤ epsilon, an absolute tolerance. A relative tolerance is expressible as EQUAL(a / b, 1, epsilon), so no second function is needed.
  • Evaluation. The subtraction, the absolute value and the ≤ use the tower's own rules. So with exact operands the test is itself exact: EQUAL(0.1 + 0.2, 0.3, 0) is true. Any inexact operand makes the arithmetic inexact, but the final ≤ is still clause 6's exact comparison.
  • Errors. A negative epsilon is Error("type"). So is a non-number in any position.
  • Where it may appear. Wherever a comparison may: an IF condition (eval_bool). It is also usable as a value of 1 or 0, as IS already is.
  • Spelling. It is a new keyword. Like the others it is case-insensitive (^"EQUAL") and is followed by (, so a bare EQUAL still names an item (PEG falls through to ref_expr). The writer's keyword guard (quote::quote_name) must add it, so an item literally named EQUAL is written |EQUAL| (gotcha 34).
  • Home. The grammar is in improvise-formula (formula.pest, the AST) and the evaluation in improvise-core (both evaluators). The arithmetic it calls is numtower's. It is migration step 3.

It replaces the epsilon's one legitimate use, a tolerant test on an inexact result, by putting the tolerance in the formula where a reader can see it.

Gotcha 17's invariant survives in a new form. The epsilon was shared by = and the division guard so that IF(x = 0, a, y / x) is coherent: the condition and the guard must agree on what zero is (improvise-0bf). Under this ADR they agree because both test exactly zero. The property is "one predicate for zero", not "one epsilon".

Both assertions of the test that pins it flip. near_zero_divisor_consistent_with_equality_epsilon (types.rs:2709) and its cached twin set X = 1e-11 and assert IF(X = 0, 1, 0) is 1 and 100 / X is div/0. Under this ADR X is not zero on either side. So IsZero becomes 0, and Ratio becomes 1e13, the very output improvise-0bf called the bug. It is coherent now, because the condition and the guard agree, and the test is rewritten to assert that agreement at X = 0 and at X = 1e-11, not either old value. The name must change too, since it names the epsilon. The change reaches existing documents as well (Consequences).

7. NaN and the infinities live only at the inexact level

An exact number is always finite. inf, -inf and nan are inexact, as in Scheme (+inf.0, +nan.0). They keep their one spelling (improvise-55v2) in the file, on the wire, and in typed(): inf, -inf and nan, and no other (clause 3). Import keeps them as text (improvise-jc95). Where they come from is unchanged: stored data, and inexact overflow in * or POW. Exact overflow becomes inexact (clause 4), and it may overflow further to inf only at the inexact level.

8. SUM and the other aggregates

apply_agg and eval_agg partition their values by exactness:

  • Exact values are summed in a wide fixed-point integer, not with Decimal's own +. The sum is independent of order and never rounds. That removes gotcha 18's order-dependent SUM as a class for exact data, which is nearly all data.
  • Inexact values are summed by ADR 009's exact f64 accumulator when it lands, and by today's fold until then.
  • The result is exact if every value was exact and the exact total fits the stored bound: a Decimal, or, under option (d), an i128 fraction (a sum of fractions can fit the fraction arm without fitting a Decimal). A second condition applies to fractions: if the sum holds more than K = 8 non-decimal denominator groups it reads the inexact f64 accumulator, so it is inexact even when its exact total would have fitted (clause 8's recommendation below, question 16). Otherwise it is inexact. It must not be a fold with contagion part-way through. Such a fold switches to f64 at the first inexact value in store order, so the answer would depend on where that value sits.

Why Decimal's own addition is not enough. Revision 1 said exact values were "summed exactly, as a decimal", and that is false at the 28-digit bound. The review measured it, and it was reproduced here:

(0.30000000000000004 + 1e12) - 1e12 = 0.3000000000000000    (scale 16: rounded)
(1e12 - 1e12) + 0.30000000000000004 = 0.30000000000000004   (scale 17: exact)

Under clause 4's detect-or-demote, the first order demotes to f64 part-way through and the second stays exact. So SUM over [p, 1e12, −1e12] would differ by order in both its value and its exactness. Codex found the same with Decimal::MAX: [M, 1, −1] demotes on M + 1, and [M, −1, 1] does not. And clause 10 loads old files' 17-digit spellings as exact, so the bound is reached by ordinary data, not only at 7.9 × 10²⁸. Once a maintained sum has demoted it cannot come back, so retraction breaks too.

The fix is an accumulator for the exact partition. Every Decimal is m · 10⁻ˢ with |m| < 2⁹⁶ and s ≤ 28. So at the fixed scale 28 it is the integer m · 10²⁸⁻ˢ, below 2⁹⁶ · 10²⁸ < 2¹⁹⁰. Adding and subtracting those integers is exact, associative and commutative, so the partition is a true abelian group whatever the order. Detect-or-demote is then applied once, when the sum is read:

  • if the total, with trailing zeros stripped, fits a Decimal, the result is that exact value;
  • otherwise it demotes, by clause 4's correctly rounded conversion of the integer over 10²⁸.

Width. For a Decimal term, a 256-bit integer holds about 2⁶⁵ unweighted terms of 2¹⁹⁰. ADR 009 feeds its aggregators consolidated Z-set deltas whose i64 weights multiply into each term, so a weighted Decimal term can reach 2²⁵³. 256 bits is therefore enough only if weights are split into chunks, as ADR 009's ExactSum already does at 2³⁰. 320 bits (five 64-bit limbs, 40 bytes) holds any i64-weighted Decimal term with 2⁶⁶ of headroom and needs no chunking.

Revision 5 corrects that bound for option (d). A Fraction whose denominator is 2^p·5^q with k ≤ 28 also joins this partition (the option (d) section), and its numerator can pass 2⁹⁶. For example POW(10, 35) is Fraction(10^35/1) with k = 0, about 2²⁰⁹ at scale 28. Such a term reaches about 2²²⁰, or 2²⁸³ weighted, so 320 bits then holds about 2³⁶ weighted terms. That is still ample: a document holds far fewer cells. But the earlier "2⁶⁶ of headroom" holds only for Decimal terms. The figures are estimates, not measured, and the partition is still far smaller than the f64 accumulator's 568 bytes.

How wide, generically (the user's question 10: "Don't we need a much bigger accumulator (2000ish bits) to handle exact accumulation generically?"). It depends on what is summed, and three answers differ by orders of magnitude:

what is summed exact width needed why
decimals (and 2^p·5^q fractions with k ≤ 28) 320 bits, fixed a common scale of 28 exists, and every term is below 2²²⁰ (2²⁸³ weighted)
f64 values about 2,100 bits, fixed a common unit of 2⁻¹⁰⁷⁴ exists, and the exponent range spans 2⁻¹⁰⁷⁴ to 2¹⁰²⁴. This is ADR 009's ExactSum
bounded fractions with arbitrary denominators no fixed width the common denominator is the lcm, which grows by up to 127 bits per new i128 denominator

So the user's figure is right in two places. It is right for f64 values, which is why ADR 009's accumulator is that size. And it is right for combining 16 arbitrary fraction denominators: 16 × 127 = 2,032 bits. No fixed register handles fractions generically. That is why option (d) keeps one group per distinct denominator, where each group is an integer:

  • a group's numerator is at most i128 × an i64 weight × 2⁶⁴ terms, so 256 bits per group;
  • the only unbounded step is the read, which combines the groups over their lcm;
  • with at most K = 8 groups that lcm is at most about 1,016 bits (K·127), or 2,030 at K = 16 and 4,100 at K = 32, and the combined numerator at most K·127 + 256 ≈ 1,272 bits at K = 8 (measured lcm: 988 bits for 8 random 126-bit denominators; 1,957 at 16 and 3,924 at 32; the combined numerator was measured only at K = 32, 4,053 bits).

Under question 13 (b) that read runs in a bigint rational (num-bigint), so it needs no fixed register at all. Its result is then held to the stored bound: it is exact if it fits a Decimal or an i128 fraction, and otherwise demoted by one correctly rounded conversion. Question 10 is revised to this shape.

A sum's scale is normalized. Summing 12.50 and 1.25 reads back as 13.75, with no memory of which scales went in, because tracking the largest scale would not survive retraction. That costs nothing a person can see: a sum was never typed, so there is no typed spelling to show back.

A mixed sum is rounded twice, not once. Revision 1 said "rounded once", and that was wrong. The exact part is converted to f64 (one rounding) and added into the f64 accumulator, whose read is the second. Both steps are deterministic and independent of order, which is the property that matters. One rounding is possible, by combining the two exact totals as a rational at read time. It is not proposed, because the case is rare.

COUNT is exact (an integer). MIN and MAX order by clause 6's exact comparison, which is total over the finite values. They need one tie-break: an exact and an inexact value can be equal as values, for example Exact(0.5) and Inexact(0.5), and then "the first one met" would depend on store order. The tie-break is to prefer the exact one, so the result keeps the most information (decided, question 11; Codex, P2 on 1fe3a85d). NaN is skipped, as f64::min skips it today. Text unanimity and mixed are unchanged (gotcha 7).

For ADR 009: its accumulator stays the mechanism for inexact values. The exact part is an abelian group through this clause's wide accumulator, not through Decimal addition. Revision 1 told ADR 009 that a maintained exact Sum was bit-equal to eager "with no accumulator at all", and that was false. It is bit-equal to eager on every path provided both backends use the same wide accumulator, exactly as ADR 009 already requires of its own f64 ExactSum. Z-set weights multiply into each term, and the width above allows for that.

9. Display, edit text and typed scale

Display of an exact value rounds the decimal, half away from zero, with round_dp_with_strategy(d, MidpointAwayFromZero), then groups. So a typed 1.005 shows as 1.01 at two places. An inexact value is rounded over its shortest round-trip spelling (clause 12, "Format"). format_value and NumberFormat::number take &Num. Gotcha 8 is unchanged: rounding is view-only and never fed back.

Edit text of an exact value is its decimal with its scale. A typed 12.50 opens as 12.50, not 12.5, so a typed value is shown back as it was typed, up to canonical spelling: +1, 1e3 and 012 become 1, 1000 and 12. An inexact value keeps today's shortest round-trip spelling (improvise-umi4). A non-finite value keeps its token.

Scale is not value, but it is state. 12.5 and 12.50 are equal under =, because they are the same number. But a typed scale is kept, persisted and shown back in the editor, so it is part of what a person typed. Revision 1 concluded that rewriting 12.5 as 12.50 "does not bump the record's version". Codex showed the cost: a client holding the old spelling could then pass the stale-basis check and overwrite the newer one without being refused.

So a record's fingerprint includes the scale. It already hashes a number's bits rather than its PartialEq (app.rs:231-250), which is why -0.0 and 0.0 fingerprint differently today. Under this ADR hash_value hashes the Num's representation: exactness, mantissa and scale. So a scale-only edit bumps the record's version, and a stale client is refused. CellValue's PartialEq stays numeric, since it answers a different question: whether two values are the same number (gotcha 7's unanimity). rust_decimal hashes 1.5 and 1.50 equally, so the fingerprint cannot use the crate's Hash. It hashes the representation explicitly.

10. .improv persistence and migration

Reading. Every number token reads as clause 3 says: a decimal if it fits, else a fraction if it fits, else the nearest f64. The grammar's number rule (improv.pest:110-112) already accepts everything a decimal needs, including an exponent. A number token that overflows f64 (1e999, which only a hand-edited file can hold) is text, as a CSV field is (clause 3). inf, -inf and nan read through the bare_value arm as clause 7's tokens, as they do today (parse.rs:268-273).

Writing. An exact decimal is written as its decimal, with its scale. An exact fraction is written n/d, with the sign on n (-10/3), through the grammar's fraction arm, which is a place where the type is forced (clause 3). An inexact finite number needs a spelling that reads back as inexact, or a save and reload would silently make it exact. The format must say which one it holds: the ticket's 1/3 against 0.333… point. It is Scheme's own marker, #i (decided, question 4): #i0.3333333333333333. The non-finite tokens are already inexact and need no marker.

Inexact stored values are rare. A ## Data line holds raw cells: typed, imported or pasted. None of them is inexact unless it was pasted from an inexact formula result, is non-finite, or is out of Decimal's range. The marker is a correctness requirement, not a common sight.

Migration: an existing number becomes the exact decimal it spells. Every existing .improv loads unchanged in the sense that matters: every value reads as the number its text names. That is a behaviour change at the 17th significant digit, and it goes toward what the text says. An old file containing 0.30000000000000004 (possible where a computed f64 was pasted and saved) loads as exactly that decimal, not as the binary fraction the old program held. No document changes in any digit a person can see.

The version literal. #i and n/d are grammar changes. An old reader would take #i0.5 or 10/3 as text (bare_value), so a file that uses either must carry a new version literal. It shares ADR 007 clause 10's new literal (decided, question 5: "share with ADR 007"): one bump, and the first line alone picks the regime, as ADR 007 decided. A file with neither spelling reads the same under both literals.

#i and n/d are recognised only under the new literal. Under the old one, a hand-written file may hold bare text #i0.5 in a data cell, and that must keep reading as text, or an old file changes meaning on load. The writer always pipe-quotes text (persistence/format.rs:224), so no file this program wrote carries it. The grammar needs only a plain sequence arm ordered before bare_value, with no predicate, so gotcha 34's trap (the generator skips predicates) does not apply.

The round-trip tests must be re-read, as the ticket says. format_number's three spellings collapse to two (decimal, #i shortest), and the grammar-walking generator gains the #i arm. The property parse(format(parse(x))) stable is unchanged, and it is now joined by exactness preserved, which the old format could not state.

11. The wire

CellValue crosses the wire in ModelCommand::SetCell, SetYanked and CellDisplay::value. CellDisplay::text and edit_text are strings and are unaffected. So a thin client is affected, and the ticket was right to ask for confirmation rather than assume.

Every number crosses as a string in improvise's own encoding (decided, question 6: "add our own encoding as strings on the wire … the round trip through f64 seems dangerous"). No new peer ever writes a JSON number for a cell value:

value JSON string
exact decimal its decimal, with its scale: "12.50"
exact fraction "n/d": "10/3"
inexact, finite #i and the shortest round-trip spelling: "#i0.3333333333333333"
inexact, non-finite the token, as today: "inf", "-inf", "nan"
  • The string carries the exactness. So nothing is recovered from an f64, and no NaN boxing is needed. The four spellings are disjoint. A decimal has no / and no #, and the tokens are neither.
  • One parser. It is numtower's: clause 3's number language, with the wire's exact arm as a place where the type is forced, so n/d is accepted there. It replaces today's untagged wire_f64 adaptor (cell.rs:84-114), which was the workspace's only #[serde(untagged)]. So the question of which untagged arm a JSON number lands in no longer arises.
  • A finite inexact cell's bytes change, from a number to a #i string. That is the cost, and it is the point.

Version skew is accepted, and the handshake version is improvise-szi8 (decided, question 9). An old peer reading any new string refuses the frame, which is loud. A new peer reading an old peer's bare JSON number reads it as inexact, which is silent, and then that old peer's 0.1 compares unequal to a typed 0.1. This matters only when a deployed page talks to a ws-server of another version, and improvise-szi8 closes it at the handshake.

typed() stays the one home of text → value (improvise-j7v6). It parses by clause 3's number language, with the type inferred, so 3/4 typed into a cell stays text (decided, question 12). A browser composes exact values with no second rule.

12. The seams

gp-tree is out of scope and needs no change. A number crosses into a fit as text (dataset.rs:50-56), which preprocess parses into f32. An exact 12.50 crosses as "12.50", and it parses to the same f32. The emitted thresholds are decimal text (improvise-gp/src/lib.rs:285-298), so they become exact literals, and exact comparison against exact data routes the same way as the f64 comparison does today, except within f32 rounding of a threshold. That boundary difference already exists between f32 and f64. scoring_known_answers.rs's bit patterns (gotcha 37c) are untouched, because gp-tree never sees a Num.

Import. Two sources, two rules. This follows Codex's P2 on 962d7f90, which aligns the contract with PR #321.

  • CSV. PR #321 keeps every non-empty CSV field as its verbatim text (Value::String, carried as Scalars::Text). It does so to preserve numeric-looking category identifiers. So a CSV measure is parsed from that text by clause 3's number language. #321's number_in gate is that language's gate, finite values only (revision 6), so the analyzer and the parse agree on every string: 1.5e3 is the exact 1500, and 1_000, nan, inf and 1e999 are text. It keeps its significant digits and its scale, so a CSV 12.50 opens in the editor as 12.50. It falls back to the nearest f64 only when it fits neither exact arm, never because it failed to parse.
    • Inferred and forced columns differ on n/d (decided, question 12). While the analyzer is inferring a column's kind, 3/4 is text, so a column of dates stays a column of dates. Once the user forces a column to a measure (a wizard override, or --measure), 3/4 in it reads as the exact fraction 3/4.
    • Revision 3's description of parse_csv_field (csv_parser.rs:167-186: an i64, else an f64 Number) is today's code at b639266b, and #321 replaces it. The ADR does not reintroduce that normalization.
    • If this ADR lands before #321, the same contract holds by reading the field's text before parse_csv_field turns it into a number.
  • JSON. A JSON number is read from its source lexeme through serde_json's raw_value, and never through the f64 as_f64 (wizard.rs:425) reads today (decided, question 6; clause 3). So a JSON 12.50 keeps its scale, and a 19-digit integer stays exact. Revision 4's shortest-spelling rule for JSON is withdrawn.
    • This restructures the JSON half of the import pipeline (revision 6, from the review of 33833f28). ImportPipeline carries serde_json::Value (ten uses in wizard.rs, and more in analyzer.rs), and a Value's numbers are already f64. raw_value hands over a lexeme only to a reader that deserializes into RawValue through a hand-written visitor. It does not work through a Value, or through any buffered path such as an untagged enum. So the JSON reader becomes a hand-written deserializer that keeps each number as its source text, and the pipeline carries that text to the number language, as #321 already does for CSV.
    • That is the restructuring revision 3 cited as a reason not to carry the text. It is paid here because the user ruled out the round trip through f64 (question 6), and this is the only way to honour that for JSON. It is migration step 5's largest part.

The collision "sum" policy (wizard.rs:307) then sums exactly, by clause 8.

Format. FormatPlan is unaffected: it chooses a format, not a value. The rounding itself moves to numtower (clause 14), and it covers all three arms:

  • a decimal or fraction is rounded exactly;
  • an inexact value is rounded over its shortest round-trip spelling, as a decimal. That is the fix improvise-felo.9 names, and it makes a typed-then-computed 1.005 behave.

The rounding mode is a parameter, so improvise's half-away-from-zero policy (gotcha 8) stays improvise's and does not become the crate's.

13. The crate: rust_decimal

Chosen on the table above. Three reasons:

  • Size and speed. It is Copy and 16 bytes, with no heap. So a CellValue does not start allocating for every number. It is about 2.5× faster than bigdecimal on SUM, and about 40× faster on the formula workload.
  • Bundle. It is the smallest marginal addition to the wasm bundle.
  • Bound. It is bounded, and the bound is the right size for this domain. 28 significant digits holds any money amount and any count. bigdecimal's unbounded add costs a heap allocation for a precision no pivot table uses. Its division still rounds, so it does not buy exact division either.

Its serde support is not a reason. numtower has no serde (clause 14), and the wire spells a value in its own encoding (clause 11). Revision 5 still listed it, and that contradicted both.

The cost is three duties the crate does not do for us, each named where it applies: detecting rounding at the bound (clause 4), parsing (clause 3 parses the digits and exponent itself, because from_str rounds silently and from_str_exact rejects 1.5e3), and converting to f64 by a correctly rounded path rather than to_f64 (clause 4). A fourth is avoided rather than done: its serde deserializer accepts JSON numbers, so the wire does not use it. The wire parses its own string encoding through numtower (clause 11). num-rational's unbounded BigRational is rejected as the default on measurement (above); its bounded Ratio<i128> is the fraction arm (question 2). bigdecimal is the fallback if the bound ever matters. Other crates (fastnum, dashu) were not evaluated.

14. The numeric tower is its own crate

At the user's direction (revision 4): "I think ADR 005 should implement the math operations in a separate crate". The numbers are a problem-agnostic workspace crate, like gp-tree and ADR 009's zset, and it knows nothing about improvise.

Name. numtower (decided, question 14). It names the idea, Scheme's numeric tower, rather than this program.

What it holds:

  • Num, with its Exact(Decimal), Fraction (question 2, decided (d)) and Inexact(f64) arms;
  • the arithmetic and contagion (clause 4);
  • the rounding and overflow detection: the scale test, and the 2^p·5^q termination test (clauses 4 and 5);
  • the correctly rounded conversion to f64 (clause 4);
  • exact comparison, the MIN/MAX tie-break included (clauses 6 and 8);
  • the exact-sum accumulator: the 320-bit decimal partition, plus the per-denominator groups under (d) (clause 8);
  • rounding any Num to n places for display, with the rounding mode a parameter: exact values exactly, and inexact values over their shortest round-trip spelling (clause 12, "Format", and improvise-felo.9);
  • the f64 → decimal shortest-spelling conversion, which the display rounding and the #i spelling on the wire and in the file all use;
  • parsing that never rounds silently: digits and exponent to an exact value, then the decimal, fraction and f64 arms in that order (clause 3);
  • the canonical text of a value: the decimal with its scale, n/d, #i…, and the non-finite tokens;
  • clause 3's number language, both its gate and its parse, including the forced n/d arm;
  • the wide integers: a fixed-width [u64; 5] for the 320-bit accumulator, and 256-bit helpers for the division reduction and the cross-multiply. These are hand-rolled, because they sit on hot paths and must not allocate.

What it depends on (revision 5 corrects revision 4, answering B3 of the review):

  • rust_decimal, with default features off except std;

  • num-rational, with default-features = false, for Ratio<i128>;

  • num-bigint, always. Revision 4 said "only if question 13 is answered 'unbounded intermediates'", and that was false by the ADR's own text. Three paths need a bigint whatever question 13 says:

    • display beyond 38 places;
    • converting a grouped sum's 256-bit numerator over an i128 denominator;
    • question 13 (b)'s widened intermediates and the grouped read.

    (num-rational's default features would pull it in anyway.)

  • not serde: the wire is improvise's concern (below).

Bundle cost under (d). rust_decimal probed at +15.4 KB gzipped marginal. num-rational with num-bigint probed at +20.8 KB gzipped on its own. The two together are unprobed, and are estimated at up to about +36 KB gzipped. The user's "yes" to question 8 was given against the first figure alone. The decided-questions section flags that, and migration step 1 measures the real bundle.

All of these build for wasm32-unknown-unknown: each was built there in the size probe. The crate is added to the wasm CI job's list, and checked with cargo tree --edges normal.

The dependency direction is one way. improvise-core depends on numtower, and numtower depends on no improvise crate. It is the gp-tree/improvise-gp shape: the general thing below, the seam above.

Improvise's own concerns stay in improvise crates and call into it:

concern stays in calls
CellValue::Number(Num), as_f64, edit_text, the record fingerprint's hashing of a Num's representation (clause 9) improvise-core (fingerprint: improvise-command) Num's accessors for exactness, mantissa, scale and fraction parts
the formula evaluator, FLOAT_EQ_EPSILON's deletion, aggregates' use of the accumulator improvise-core Num's arithmetic, comparison and accumulator
formula literals: Expr::Number(Num) improvise-formula numtower's parse
the .improv grammar arms (#i, n/d), the writer, the version literal improvise-io numtower's parse and canonical text; the #i prefix and the grammar are improvise's
serde and the wire encoding (clause 11) improvise-core, replacing CellValue's untagged adaptor the canonical text of every arm, and the number language to read it back
display formats (,.2f), comma grouping improvise-core's format.rs numtower's round-to-n-places; the grouping stays in improvise
typed() improvise-protocol numtower's number language, with the type inferred (question 12: n/d stays text)

Where the tests live. Anything about numbers lives in numtower's own tests, where no improvise type can make them pass by accident:

  • the property tests of migration step 1 (no rounded Exact; total contagion; exact comparison is transitive; n/d normalization);
  • the differential tests against an exact BigRational oracle: every operation's result, the f64 conversion, and the accumulator's read against a direct exact sum;
  • the order-independence and retraction tests for the accumulator, seeded with the review's [0.30000000000000004, 1e12, −1e12].

Improvise's crates keep the tests about improvise:

  • improvise-core: the evaluator, the improvise-0bf zero predicate, and the scale-aware fingerprint;
  • improvise-io: the grammar round trip with exactness preserved;
  • the protocol: which wire arm a JSON number or string lands in.

The benchmark example lives in numtower/examples/, where its dependencies are already normal ones. That is what lets it be committed at all (appendix).

ADR 009's ExactSum is an f64 superaccumulator, in the proposed zset crate. Clause 8 needs it for the inexact partition, and a mixed sum's read combines it with the exact partition. ExactSum moves into numtower (decided, question 15 (a)), the one home for exact summation. zset stays independent of numbers: it keeps its abelian-group trait, and ADR 009's seam crate (improvise-incremental, its question 2) wraps numtower's accumulators in a newtype that implements that trait. The orphan rule forbids implementing zset's trait for numtower's type anywhere else, and the newtype costs nothing. The alternative, numtower depending on zset for ExactSum, would make the number crate depend on a stream-processing crate.

numtower can land holding ExactSum alone, ahead of this ADR and independent of it. ADR 009's question 3 recommends its exact accumulator "without waiting for ADR 005". Under question 15 (a) that accumulator lives in a crate this ADR's migration creates, which would make ADR 009 wait after all. It does not have to. Nothing in ExactSum depends on Num, so numtower may first land with ExactSum and its tests only, before this ADR is accepted, and Num joins it in migration step 1. ADR 009 then depends on numtower and not on ADR 005.

Option (d) for question 2: bounded exact fractions

The user asked for this option (revision 3): "I think (d): exact fractions with a size limit. should be added." It is specified here and measured the same way as the others: same machine (Apple M4 Max, 64 GB, rustc 1.98.1, release), same typed two-place data (0.01–999,999.99, SplitMix64), and num-rational 0.4.2's checked operations. Those reduce by the gcd on every operation.

The number

An exact non-decimal value is a reduced fraction n/d with n and d each an i128 and d > 0. The alternative bound, i64, was measured and is too small:

what stops being exact i64 i128 rust_decimal
running sum of 18k or 90k margins (Rev − Cost) / Rev, first overflowing term (1-based: the term whose addition overflows), 8 seeds 3rd–4th 5th–6th not exact (a non-terminating quotient)
1.05^k (5% compound growth) k = 15 k = 29 k = 15

i128 doubles every horizon, for 2–3× the cost of i64 (below). A value is 32 bytes.

Decimals are a subset, and they keep their own arm. A decimal is a fraction whose denominator is 2^p · 5^q, so (d) could hold everything as fractions. It should not. A fraction has no scale, so 12.50 would need a scale field beside it. And every operation would pay the gcd, which the measurements put at 30–300× a decimal add. So the recommended shape is a hybrid:

pub enum Num {
    Exact(Decimal),             // typed values, money, `12.50`: scale as today
    Fraction(Box<Ratio<i128>>), // exact, not a finite decimal: `10/3`
    Inexact(f64),
}
  • A Fraction is only ever produced where option (a) would have produced Inexact:
    • a non-terminating quotient;
    • a decimal operation that exceeds 28 digits but whose reduced fraction fits (1.05^k for 15 ≤ k ≤ 28).
  • A Fraction whose denominator turns out to be 2^p · 5^q and fits is normalized back to Exact(Decimal), so each value has one representation.
  • Scale lives only on the decimal arm. Typed values are decimals, so a fraction was never typed and has no spelling to preserve. That answers "how is scale carried" without a scale field.
  • The fraction arm is boxed. That keeps Num at 24 bytes, the same as option (a); measured, an inline Ratio<i128> makes it 48. The heap allocation falls only on fraction values, which are formula results. Per AGENTS.md, the Box handling lives in Num's constructors, apart from the arithmetic.

Operations

  • Division stays exact. 10/3 is Fraction(10/3), and 10/3 × 3 is exactly 10, which normalizes back to Exact(10). Both measured.

  • Overflow demotes at the store (question 13, decided (b); revision 7 replaced the per-operation wording, which Copilot found contradicted it). The fast path uses checked i128 arithmetic after the gcd reduction (num-rational's checked_add/checked_mul/checked_div do exactly this). A None may mean the reduced result does not fit, and may mean only that an intermediate overflowed (checked_add goes through the lcm of the denominators) while the reduced result fits. So a None widens that step to a bigint rational and the stored result is tested against the i128 bound: exact iff it fits, else the correctly rounded f64 of the exact fraction, which is Inexact. It is deterministic. Past question 13's ceiling, the intermediate demotes (the first of the two stated exceptions).

    Revision 3 cited Ratio<i128>::to_f64 agreeing with BigRational's conversion. That was circular: both share num-rational's one ratio_to_f64 routine. The review ran an independent check instead. It took 200,000 random reduced Ratio<i128>s, with numerator and denominator widths from 20 to 126 bits and both signs. For each, it asked whether the exact BigRational lies inside the round-to-nearest-even interval of the returned f64. There were 0 failures. That is evidence, not proof, and migration step 1's property test uses this interval form.

  • Contagion is unchanged. Exact ∘ fraction is a fraction (or a decimal after normalization, or f64 on overflow). Anything with an inexact operand is inexact.

  • POW with an integer exponent is exact while it fits. That is why 1.05^k reaches k = 28.

  • Comparison is exact. Two fractions are compared by cross-multiplying in 256 bits (an i128 times an i128 fits). A fraction against an f64 follows clause 6's exact rule.

  • Display rounds the exact fraction to the view's places, half away from zero, exactly: n · 10^places / d, then rounded. So 10/3 at two places shows 3.33, and 2/3 shows 0.67. In 256 bits that holds up to about 38 places, because places is a u8 (parse_number_format) and n · 10^places passes 2²⁵⁵ above that. Beyond 38 places the computation takes the bigint path. That is display only, and rare.

Sums: the one real design choice

Clause 8's accumulator holds decimals only: an integer at the fixed scale 28. A fraction whose denominator is not 2^p · 5^q has no finite value at any fixed scale. There are three ways to sum one:

  1. Demote each fraction term to f64 into ADR 009's accumulator. Simple, and independent of order. But SUM(v / 3) is then inexact, even though SUM(v) / 3 is exact.

  2. Group by denominator. Keep one wide integer numerator per distinct reduced denominator; decimals are the one group clause 8 already has. Adding or retracting a term touches only its own group. So the state is independent of order, and retraction is exact.

    How it is read is the recommendation below: exactly, in a bigint rational, when there are at most K non-decimal groups (question 13 (b)); otherwise from a maintained f64 accumulator of the groups' values, with the decimal partition converted once. Both paths are deterministic and independent of the order the terms arrived in. (Revision 5 still described a checked-i128 read here, with a fallback that summed each group's f64 at read. That contradicted the maintained accumulator below.)

    The cost is memory. It is O(distinct denominators) per summed group, bounded by the cells, the same order as ADR 009's multisets.

  3. Demote the whole sum on mixed denominators. Rejected. Whether two denominators ever met depends on nothing a user can see.

Two rules the grouping needs.

  • A group whose numerator returns to 0 is dropped. A retracted denominator then cannot affect the read, and "the current groups" is a function of state alone.
  • A Fraction whose denominator is 2^p · 5^q joins the decimal partition when its exact scale k = max(p, q) is at most 28. An example is one born from a decimal overflow with denominator 10^17. At scale 28 its value is an exact integer, which the 320-bit accumulator holds. With k > 28 (for example 1/2^40) it is a group of its own, keyed by its denominator like any other.

Recommendation: (2), with a bound on the exact read (revision 4, answering review BLOCK D2). Revision 3 said (2) "costs nothing where it cannot help", and that was false in exactly that case. The sum of margins has one denominator per record: 17,996 to 17,999 distinct at 18k, and 89,923 to 89,956 at 90k. So its state is O(records), and so is every read. Under ADR 009 a maintained SUM whose read is O(records) is no longer incremental: every one-cell edit would re-read 90k groups, where (a) holds a 40-byte integer and reads in O(1). The review proposed a mitigation, and it is adopted:

  • Beside the non-decimal groups, keep one ADR 009 f64 accumulator of each such group's correctly rounded value. Updating a group retracts its old f64 and inserts its new one, so it stays order-free and retractable.
  • The decimal partition is exempt (revision 5, B2 of the review). Revision 4's wording put every group, the decimal partition included, into the parallel accumulator, which would have charged an ordinary decimal SUM about 1 µs per edit where (a) pays a 40-byte integer add. The decimal partition stays in its 320-bit accumulator alone, and is converted to f64 once, at an f64 read (only when there are more than K non-decimal groups). So a pure-decimal SUM pays nothing for (d).
  • Read exactly only when the current number of non-decimal groups is at most K = 8 (decided, question 16; revision 8). Otherwise read the f64 accumulator, and the result is inexact. The count is a function of the current state, not of history, so the choice is independent of order.

What that costs.

  • Update of a non-decimal group (an estimate): O(log D) for the group map, plus one correctly rounded conversion of the group's new value. That conversion is a 256-bit numerator over an i128 denominator, estimated at about 1 µs through a bigint rational. Plus two accumulator updates at 2.0–3.6 ns (ADR 009's figures). An update of the decimal partition is one 320-bit add.

  • Exact read (measured in revision 6). At most K combinations of groups that hold 256-bit numerators, in a bigint rational (question 13 (b)). The worst case is K distinct denominators near i128's size, where the lcm grows about 127 bits per group. Measured with 250-bit numerators and random 126-bit denominators, 200 reads each (appendix):

    K lcm bits BigRational add per group lcm, then one reduce
    2 252 16 µs 7 µs
    8 988 127 µs 52 µs
    16 1,957 448 µs 174 µs
    32 3,924 1.9 ms 0.67 ms
    • Revision 5 estimated 16–34 µs at K = 32 from the SUM benchmark's rate. Those operands had small typed denominators, so the estimate was about 50× too low at this size. The cost grows roughly as K², because each step works on an lcm that grows linearly.
    • Combining over the lcm and reducing once is 2.3–2.8× faster than reducing after every add, and it gives the same value (the benchmark asserts it). numtower's read should do that.
    • Denominators a person produces (/ 3, / 12, a percentage) are small, and the read is then far cheaper. The table is the bound, not the common case. The value of K is 8, decided in question 16.
  • f64 read: ADR 009's 0.6–1.1 µs.

  • Memory is still O(distinct denominators), because the map must survive retraction to know when the count falls back under K. For a 90k-record margin sum that is about 90k × (16-byte key + 32-byte numerator + map overhead): an estimate of 6–8 MB for that one summed group, the same order as ADR 009's per-cell multisets. Not measured.

This keeps (d)'s wins, because SUM(v / 3), averages and percentages have one or a handful of denominators, and it bounds the rest. The sum of margins reads the f64 accumulator and is inexact, as under (a). Its exact combination would overflow by the 6th term anyway (below). The 32 of revisions 5 to 7 was a judgement and was not tuned. The read's measured cost above is why question 16 recommended a smaller one (K = 8). The user first chose 32 and then 8, so 52–127 µs is the accepted worst case. It is paid per read, and under ADR 009 per edit, by a maintained SUM that holds between 2 and 8 distinct non-decimal denominators near i128 size; a sum with more groups reads the f64 accumulator, and one with few or small denominators is far cheaper (the table is the bound).

The file and the wire

  • .improv. A fraction is written n/d, with a sign on n, for example -10/3. The number rule does not accept /, and an old reader would take 10/3 as text (bare_value), silently. So it needs a grammar arm ordered before bare_value, and the new version literal. That is the same bump as #i (question 5), so (d) adds no second bump.
  • The wire. A fraction is a string "10/3" in the exact arm of clause 11's adaptor. An old peer tries it as a non-finite token and refuses the frame, loudly.
  • Typing 3/4 stays text. That is question 12.

Measured

Cost per operation, 18,000 operand pairs, ns, best of 20, the range over two runs:

add mul div
rust_decimal (checked) 0.9–1.3 1.5–2.2 40–55
Ratio<i64>, two-place operands 55–68 144–179 78–113
Ratio<i128>, two-place operands 123–178 324–472 198–288
Ratio<i128>, margin operands (unrelated denominators) 313–457 407–652 223–325

f64 is not in the table, because this harness's f64 loop was optimized away. The SUM figures below give f64's per-add cost. num-rational's gcd over i128 is generic. A hand-written one with a 64-bit fast path would likely be several times faster, but that was not measured.

SUM over typed two-place values, as fractions, ns per element, best of 20:

cells f64 rust_decimal Ratio<i64> Ratio<i128>
18,000 0.8–1.2 4.6–6.4 65–76 155–226
90,000 0.8–1.2 4.5–6.5 69–87 163–239

Both bounds held the whole sum with no overflow. The sum was the same in both orders and equal to the decimal sum. But Ratio<i128> is 25–50× slower than rust_decimal: 14–22 ms for a 90k SUM, against 0.4–0.6 ms. That is why the hybrid keeps decimals in their own arm. Under it, this SUM costs what it costs under (a).

How often realistic formulas overflow i128, 8 seeds, 18k and 90k records:

formula overflows
(Rev − Cost) / Rev per record never (864,000 records: 8 seeds × (18k + 90k), both bounds)
running SUM of those margins at the 5th–6th term in every seed, counting 1-based (i64: 3rd–4th)
v / SUM(v) × 100 per cell (a percentage of the total) never (0 of 144,000 and 0 of 720,000, both bounds)
SUM(v) / N (the average) never
SUM(v / 3) against SUM(v) / 3 never; equal exactly
1.05^k at k = 29 (i64 and rust_decimal: k = 15)

Where (d) beats (a), and where it does not

(d) is better:

  • a non-terminating quotient stays exact: 10/3 × 3 = 10 (measured), and so Margin × Revenue = Profit holds exactly;
  • the average over any count;
  • SUM(v / 3) = SUM(v) / 3;
  • exact comparison of ratios;
  • compound growth exact to k = 28 rather than 14.

(d) is no better:

  • the sum of ratios, which overflows by the 6th term and ends inexact as under (a);
  • anything already exact under (a), which is nearly all typed data.

(d) costs:

  • a third numeric arm;
  • a grouped sum accumulator;
  • a grammar arm and the wire spelling n/d;
  • question 12;
  • 120–650 ns per fraction operation, paid only where (a) would have demoted;
  • num-rational and num-bigint in the browser bundle: +20.8 KB gzipped on their own, and up to about +36 KB together with rust_decimal (unprobed together; clause 14).

Scale after the fraction arm (revision 5, the review's point on question 13). A result that passed through the fraction arm and normalized back to a decimal carries its minimal scale. A result that stayed decimal keeps clause 4's scale. So a × (b + c) and a × b + a × c can be = and still differ in scale. The review's example (reasoned, not run) is a = 123456789012345.6780, b = 12345678901234.5678 and c = −b + 0.0001. The first stays decimal at scale 8. In the second, a × b overflows 96 bits, so the sum comes back through the fraction arm at scale 7. The editor seed and the scale-aware fingerprint (clause 9) then differ between the two spellings. This is true under (d) whatever question 13 says. It is stated rather than fixed, because scale is presentation, and a computed value was never typed.

Folded in: what changed in the clauses when the answer became (d)

Revision 3 listed these changes as conditional. Revision 5 makes them normative, because the user answered question 2 "(d)":

  • Clause 1: three levels: exact decimal, exact bounded fraction, inexact.
  • Clause 2: Num gains the boxed Fraction arm.
  • Clause 3: entry tries the fraction arm before f64, as arithmetic does (revision 6).
  • Clause 4: a decimal operation that exceeds 28 digits tries the fraction arm before f64.
  • Clause 5: a non-terminating quotient is a Fraction. AVG is exact.
  • Clause 6: fraction comparison is by cross-multiplication.
  • Clause 8: sums group by denominator, as above.
  • Clause 10: the n/d spelling.
  • Clause 11: the wire spells a fraction "n/d".
  • Question 12 decided typed input: n/d is text where the type is inferred, and a fraction where it is forced (clause 3).

Alternatives rejected

  • Exact rationals as the default (Scheme proper). Measured above: an 18,000-record average of ratios does not finish. It would be the right answer for a symbolic calculator and the wrong one for a pivot table.
  • A rounded decimal that still calls itself exact (SQL NUMERIC: 1/3 becomes 0.3333… at 28 places, marked exact). It keeps "decimal math" true everywhere, but then "exact" means "exact except after a division". Equality, ADR 009's group law and every future reasoning about exactness would carry that exception silently. It is question 2's option (b), for the user to overrule.
  • Keep f64 and adopt only ADR 009's accumulator. It fixes large same-sign sums. It leaves per-cell arithmetic, small sums (16% of positive 3-term sums), signed sums (about 40% at every length), display of typed values and the epsilon exactly as they are (Context).
  • Three levels (exact decimal, inexact decimal, f64). An "inexact decimal" arm for rounded quotients would add a third arm to every numeric match. It would buy only a 28-digit rather than a 17-digit approximation of a number that is not a decimal anyway.
  • A new CellValue arm beside Number(f64). Rejected in clause 2.

Migration

In order. Each step lands on its own, with the suite green.

  1. The numtower crate, first and alone (clause 14). It holds Num and its arithmetic, with the contagion table (clause 4) and the scale-based rounding detection. It holds the correctly rounded conversion to f64, the division and POW rules (clause 5), exact comparison (clause 6), the exact-sum accumulator (clause 8), rounding to places, and clause 3's number language. It also holds the fraction arm and its grouped sums (question 2 is decided (d)), and the widening rule with its ceiling (question 13 is decided (b); the ceiling's value, 8,192 bits, is the steward's proposal and not the user's decision). Nothing in improvise depends on it yet, so it lands with its own tests and changes no behaviour. It may be preceded by a numtower holding only ExactSum, for ADR 009 (clause 14).

    Its property tests:

    • an Exact result is never a rounded one (inject a silently-rounding add and watch it fail);
    • contagion is total;
    • the conversion agrees with str::parse::<f64> of the decimal text. Inject to_f64 and watch it fail on the review's 63294.60… case. The random generator spans 96-bit mantissas and scales 0–28: with 15 or fewer digits the broken to_f64 agreed 200,000 times out of 200,000, so a narrower range is vacuous;
    • a fraction's conversion lies inside the round-to-nearest-even interval of the exact value (the review's independent form);
    • exact mixed comparison is transitive;
    • the accumulator is independent of order (clause 8's seed case);
    • division's direct fit test: 1/2^40 and 1/5^30 are fractions, 1e-28 / 0.5 is the exact decimal 2e-28, and multiplying back is not used (clause 5);
    • the number language's gate and parse agree on every generated lexeme. The seed cases are:
      • 1.5e3 (exact 1500), 1_000 (text), -0.0 (exact 0.0), and 3/4 (text when inferred, a fraction when forced);
      • 1e999 and -1e999 (text: a finite lexeme never reads as infinity), nan, inf and Infinity (text in the gate), and 1e-999 (an inexact zero);
      • 1e-30 (the exact fraction 1/10³⁰), 0.12345678901234567890123456789 (an exact fraction, not a rounded decimal), an integer between Decimal's maximum and 2¹²⁷ (an exact n/1), and 1e-40 (inexact: it fits neither arm);
    • a typed value and the same value computed are equally exact: 1e-30 parsed equals 1e-15 * 1e-15, arm and value both;
    • the grouped read combines over the lcm and reduces once, and agrees with a BigRational sum of the groups.

    The differential tests run against a BigRational oracle. The benchmark goes in numtower/examples/. Add the crate to the wasm CI job. Measure one whole recompute at 18,000 and 90,000 cells against today once step 3 lands, and name the machine. The arithmetic figures above are a microbenchmark, not the evaluator.

  2. improvise-core depends on numtower, and CellValue::Number(Num) replaces the f64 across the 672 sites. This is mechanical, and as_f64 keeps the seams compiling. hash_value hashes the representation, scale included (clause 9).

  3. Both evaluators move to Num. FLOAT_EQ_EPSILON is deleted. The improvise-0bf tests are rewritten to assert one zero predicate (clause 6). EQUAL(a, b, epsilon) lands here: its grammar and AST in improvise-formula, its evaluation in both evaluators, and EQUAL in quote_name's keyword guard (clause 6).

  4. Aggregates partition by exactness and use numtower's accumulator (clause 8). Its own order-independence test is in numtower (step 1). Here the test is that apply_agg and eval_agg route through it: seed the review's [0.30000000000000004, 1e12, −1e12] through the evaluator. Decimal's own + must fail it.

  5. Entry points: typed(), the import pipeline's measure read, formula literals (clauses 3 and 12). The largest part is the JSON reader: a hand-written deserializer that keeps each number's source text through raw_value, because a serde_json::Value has already made it an f64 (clause 12). The import tests assert:

    • a CSV 12.50 and a JSON 12.50 both keep their scale, because both are parsed from their lexeme;
    • 1.5e3 imports as the exact 1500, and 1_000 imports as text;
    • nan, inf, 1e999 and -1e999 import as text, so #321's a_nonfinite_numeric_token_stays_text_rather_than_becoming_zero stays green unchanged;
    • typed() reads inf, -inf and nan as clause 7's tokens, and Infinity, NaN and 1e999 as text (the stated behaviour change);
    • 3/4 is text in an inferred column and the fraction 3/4 in a column forced to a measure.
  6. Display and edit text (clause 9). The 1.005 case is the regression test, and it fails at HEAD today. If improvise-felo.9 lands first, this step reuses its test.

  7. .improv: read exact, write with scale, n/d for a fraction, #i for inexact finite, and ADR 007's version literal (clause 10).

  8. The wire: the string encoding replaces the untagged adaptor (clause 11), with the retired-spelling discipline of gotcha 42: assert the error kind, not merely that an error occurred. The version handshake is improvise-szi8, separately.

Open questions

The user answered questions 1–9, 11 and 12 inline on e4acf643, and 10, 13, 14 and 15 inline on 33833f28, and question 16 inline on b232c5a9. Each is marked decided below, and quoted in full under "What the user decided". No question is open. Nothing is renumbered.

  1. The tower's shape. Decided (the user): three levels: exact decimal, exact bounded fraction, inexact f64 (clause 1). The original recommendation, two levels, was written before option (d) existed.

  2. Non-terminating exact division. (a) inexact f64; (b) a 28-digit decimal that stays marked exact (SQL NUMERIC); (c) an exact rational (Scheme proper); (d) an exact fraction with a size limit. Decided (the user): (d), in the hybrid form (the option (d) section).

  3. Tolerance on inexact equality. Decided (the user): no tolerance on =. A formula function EQUAL(a, b, epsilon) is added instead (clause 6). It replaces this question's earlier "an APPROX(a, b, tol) if it is ever wanted".

  4. The .improv spelling of an inexact finite number. Decided (the user): #i (clause 10).

  5. The version literal. Decided (the user): share ADR 007 clause 10's literal (clause 10).

  6. JSON import precision, and the wire. Decided (the user): no round trip through f64.

    • The wire spells every number as a string in improvise's own encoding (clause 11).
    • JSON import reads each number's source lexeme through serde_json's raw_value (clauses 3 and 12).
    • arbitrary_precision stays rejected, because it breaks serde's buffered deserialization of numbers under the workspace's internally tagged enum and its flatten.
    • The earlier recommendation, "shortest spelling", is withdrawn.
  7. Mixed-exactness comparison. Decided (the user): exactly, plus EQUAL(a, b, epsilon) (clause 6).

  8. The dependency. Decided (the user): yes. That answer was given against rust_decimal's +15.4 KB gzipped. Under (d), num-rational and num-bigint add about +20.8 KB on their own, up to about +36 KB together (unprobed). If that changes the answer, this question reopens (clause 14).

  9. Version skew on the wire. Decided (the user): accept it, and the handshake version is filed as improvise-szi8 (clause 11).

  10. The exact-sum accumulator's width. Decided (the user): (a), with "320bits is fine" for the decimal partition. The value of K is split out as question 16, because the read's cost was measured after the answer. The question as it was put (revised in revision 5, after the user asked: "Don't we need a much bigger accumulator (2000ish bits) to handle exact accumulation generically?"). Clause 8's "How wide, generically" answers with numbers:

    • decimals need 320 bits;
    • f64 values need about 2,100 bits, which is ADR 009's ExactSum;
    • fractions need no fixed width at all, because their common denominator grows by up to 127 bits per distinct i128 denominator. 2,032 bits holds 16.

    Options:

    • (a) a 320-bit decimal partition, plus a 256-bit numerator per fraction group, plus a read that combines at most K groups in a bigint rational and then applies the stored bound;
    • (b) one fixed 2,048-bit exact register, which holds K = 16 arbitrary denominators, with no bigint on the read;
    • (c) Decimal's own +, accepting demotion that depends on order.

    Recommendation: (a), with K = 32 as proposed here. Question 16 later set K = 8.

    • A fixed register cannot be generic for fractions at any width.
    • 2,048 bits pays 256 bytes per summed group for a combination that rarely fits the stored bound anyway.
    • The bigint read is paid only when a sum holds 2 to K distinct fraction denominators: about 16–34 µs at K = 32, an estimate. (Revision 6 measured it: 0.67–1.9 ms at K = 32 in the worst case, 52–127 µs at K = 8, and the user chose K = 8. See question 16.)
    • (c) breaks order-independence (revision 2).
  11. The MIN/MAX tie-break. Decided (the user): prefer the exact value (clause 8).

  12. Typing a fraction. Decided (the user): n/d is text where the type is inferred and a fraction where it is forced (clause 3). So typed() and the import analyzer keep 3/4 as text, and a column forced to a measure reads it as a fraction. An editor opened on a fraction shows 10/3, and an untouched commit writes nothing, on both front ends. The browser already skips an untouched commit (resolve.rs:1421); the TUI must be held to the same rule (the improvise-umi4 lesson).

  13. Where the fraction bound applies. Decided (the user): (b). Added in revision 4, and put as follows. Is the i128 bound checked (a) after every operation, or (b) only on the stored result, with every intermediate inside one formula evaluation unbounded? Under (b) an intermediate uses the i128 fast path and widens to a bigint rational only when a step overflows. Recommendation: (b). It was the orchestrator's recommendation in a discussion with the user.

    The argument. Under (a), whether a result is exact depends on how the formula is written. a × (b + c) and a × b + a × c are the same number, but an intermediate can overflow in one spelling and not the other, so one is exact and the other a demoted f64. Under (b) the rule is one sentence: a stored value is exact if and only if it fits. Algebraically equal formulas give equal values. (Two stated exceptions: the ceiling below, and a SUM of more than K = 8 non-decimal denominator groups, which reads inexact by question 16 whatever its value.)

    The rule has two exceptions, and they are stated rather than hedged (the ceiling from revision 5, the review of 87ae21ca; the K cutoff from revision 7, Codex on be9de2ee):

    • The ceiling. Widening carries a size ceiling so that a pathological expression cannot grow a bigint without limit. It must be at least K·127 + 256 bits, so that the grouped read (below) never reaches it: 1,272 bits at K = 8 (it was 4,320 at K = 32). Revision 5's example of 4,096 bits was below the K = 32 figure, and a worst-case K = 32 read would have hit it (the review of 33833f28). Proposed: 8,192 bits. That value is the steward's proposal from revision 6, commit 02bcfcb1 (it first appears after the user's answer "(b)", which was given against the text's 4,096-bit example), so it is not part of the user's decision, and approval of the ADR confirms it. It covers K = 8 by 6,920 bits and K = 32 as well, and costs nothing until a value is that large. A smaller ceiling would now suffice. Past it, the intermediate demotes, and by contagion the stored result is inexact even when the exact result would fit. That is spelling-dependent, so distributivity fails past the ceiling. It is the first exception to "exact iff it fits", and no ledger reaches it. POW checks its result size before computing (clause 5), so the ceiling is never reached by allocating first.
    • The K cutoff, which does override representability. A SUM holding more than K = 8 non-decimal denominator groups reads the inexact f64 accumulator, so its stored result is inexact even when the exact total would fit the bound. That is the second exception. It is a policy about the read's cost (question 16), not about whether the value is representable, and it is the consequence of the user's K = 8. Flagged reading (revision 7, restated in revision 8): the user decided K on question 16's text, which says "Above it the read is the inexact f64 accumulator", and answered question 10 (a) with "at most K groups". No "What the user decided" item says in terms that the cutoff overrides representability. That sentence is the ADR's reading of those two answers, so approval confirms it.
    • Not an exception: the grouped read at K groups or fewer. Revision 4 described the grouped sum's read as checked i128, and that can overflow in an lcm step while the reduced result fits. Under (b) the read is widened like any intermediate (clause 8, "How wide, generically"), so it is exact iff its result fits.
    • What (b) does not change. The bound still applies at every stored value, so a calculation split across two formula cells is bounded at the cell in between. Equal values may still differ in scale after passing through the fraction arm (the option (d) section). Scale is not value.

    Cost. This is an estimate, not measured on this path.

    • The common case is unchanged: no intermediate overflows i128.
    • Only a cell whose expression overflows i128 part-way through pays for bigint rational steps. A widened step works on operands near 256 bits, not on the SUM benchmark's small typed values: combining two such fractions measured 7–16 µs (option (d), "What that costs"). So a widened step costs microseconds, not the 490–1,060 ns revision 5 cited.
    • Such a cell usually demotes at the store anyway.
  14. The numeric crate's name. Decided (the user): numtower ("this is fine"). Added in revision 4, with the recommendation numtower. It is problem-agnostic like gp-tree and zset, so it should not carry improvise's name.

  15. Where ADR 009's ExactSum lives. Decided (the user): (a). Added in revision 4, and put as follows.

    • (a) In numtower, beside the exact decimal accumulator, with ADR 009's seam crate adapting it to zset's group trait through a newtype.
    • (b) In zset, as ADR 009 proposes, with numtower depending on zset to combine a mixed sum.

    Recommendation: (a). Exact summation then has one home, a mixed sum's read needs no second crate, and zset stays free of numbers. numtower can land holding ExactSum alone, ahead of this ADR, so ADR 009 does not wait for it (clause 14). This question is shared with ADR 009 (PR #309).

  16. The value of K. Decided (the user): (b), K = 8 ("Ok, let's drop K back to 8", superseding an earlier "let's do k = 32"; both quoted under "What the user decided"). Added in revision 6, and put as follows. K is the most non-decimal groups a SUM reads exactly (question 10). Above it the read is the inexact f64 accumulator. Revision 5 proposed K = 32 on an estimate of 16–34 µs per read, and the measured worst case is 0.67–1.9 ms (option (d), "What that costs"). Under ADR 009 a maintained SUM reads after every edit that touches it, so that is a per-edit cost, paid by each such cell.

    • (a) K = 32, as decided with question 10: worst case 0.67–1.9 ms per read. Chosen first, then superseded.
    • (b) K = 8: worst case 52–127 µs per read. A SUM over more than 8 distinct denominators is inexact. Chosen (the later answer).
    • (c) K = 16: worst case 174–448 µs per read.

    Recommendation (taken in the end): (b), K = 8, with the read combining over the lcm and reducing once (the lower figure in each range). The user chose (a), then (b).

    • The ceiling (question 13) stays at 8,192 bits under any of the three.
    • The common sums keep exactness under K = 8: SUM(v / 3) has one denominator, and an average or a percentage has a handful.
    • The sum of margins has one denominator per record, so it passes any of these Ks long before it is large, and reads inexact under each of them.
    • K is a constant inside numtower, not a format or wire fact, so it can be tuned later against the evaluator benchmark (migration step 1) without a new ADR.

    Re-derived under K = 8 (revision 8), from this ADR's own measurements, with nothing new measured (the K = 32 figures of revision 7 are superseded):

    • Read cost: the K = 8 row of the table, 52 µs (lcm, then one reduce) to 127 µs (reduce after every add). numtower reads by the first method, so the lower figure is the working number. The 16–34 µs of revision 5 stays withdrawn.
    • Combined width: lcm up to K·127 = 1,016 bits (measured 988), numerator up to K·127 + 256 = 1,272 bits. The combined numerator was measured only at K = 32 (4,053 bits), so 1,272 is a bound and not a measurement. It excludes the up to 3 carry bits of summing 8 terms, which the headroom below absorbs.
    • Ceiling: the proposed 8,192 bits clears 1,272 by 6,920. Unchanged, so question 13's decision is untouched.
    • Memory: O(distinct denominators), which K does not change (clause 8).
    • Not re-derived: an end-to-end per-edit latency for a maintained SUM. The benchmark times the read alone. Whether 52–127 µs per edit is acceptable on a page of such cells is a question for the evaluator benchmark, not a figure here.

Relations to other ADRs

Added in revision 7. This lists what the three ADRs written since ask of this one, as read at their heads, and changes none of the user's decisions here. Each ADR is cited at the head named, and none was built or run.

ADR 007 (stable identities, accepted at revision 8; merged via #210). The points below are read from revision 8, as on gh/main. Revision 9 is not part of it: see the last bullet.

  • The version literal. ADR 007 clause 10 adds a second .improv literal and says its exact spelling is left to the implementation. This ADR shares it (clause 10, question 5), so #i and n/d ride on the same bump. ADR 007 says nothing about numbers, so nothing in its text conflicts.
  • Flag, ordering. If ADR 007's migration ships the new literal before this ADR's grammar arms, a reader of that literal would not know #i or n/d, and "shared" would mean one literal with two grammars. The two should either land the literal together or this ADR's arms should be added under the literal ADR 007 has already shipped, with the old readers' behaviour on #i stated. That is a merge-order matter for the implementer, not a decision the user has made.
  • Ids and display. This ADR's name-quoting and the formula-target handling are about names, and ADR 007 moves keys from names to ids. Numbers are not names, so clause 10's number spellings are unaffected; any clause here that quotes a category or item by name should be read as "by display name" once ADR 007 lands. Not re-read clause by clause.
  • Revision 9 is a proposed amendment, not accepted (PR #326, open, head 068e4130; it needs the user's explicit approval, and its Q23–Q25 are open). Nothing in this ADR depends on it. It touches this ADR only at the edges: answers 33 and 34 (a typed name matches ids first, then displays; whitespace in a slug) bear on how a name in formula text is resolved, and answers 35 and 36 reopen Q23 (whether a measure or formula target has an id and a display of its own), which would change what "a formula target" means where this ADR names one. Read as a proposal, and re-check this ADR if #326 lands.

ADR 008 (virtualized grid, PR #308, proposed). Its Q28 answer, "bigint is fine, we don't need to reinvent integers > safe size", makes a grid position a bigint above 2^53 (its Q36 spells it as a decimal string in JSON).

  • Dependency. It adds num-bigint to improvise-protocol, a wasm-bundle crate. This ADR already accepts num-bigint in numtower (+20.8 KB gzipped for num-rational plus num-bigint, probed; question 8). If improvise-core depends on numtower (clause 2), improvise-protocol already reaches num-bigint through it, so the cost is paid once, not twice. ADR 008 says its cargo tree --edges normal check has not been run; neither has one here for the bundle.
  • Flag, a new consumer of numtower. Positions are integers, not numbers in the tower's sense. Whether they use numtower's type or num-bigint directly is open in ADR 008. The tower's n/d and #i spellings must not leak into a position's wire spelling.

ADR 009 (incremental evaluation, PR #309, proposed).

  • ExactSum lives in numtower (question 15, (a)), which ADR 009 cites. numtower can land holding it alone, so ADR 009 does not wait for this ADR.
  • Sum is exact only under ADR 009's question 3 (c) or under this ADR's wide accumulator. This ADR's accumulator is exact for decimals (320 bits) and for up to K = 8 fraction denominators (clause 8). For f64 terms the exact accumulator is ADR 009's ExactSum. A mixed sum is read from both (clause 8), so "exact under this ADR" holds for decimal and bounded-fraction data, not for f64 data.
  • The per-edit cost is a lower bound, and the larger exposure is Eager. ADR 009 reads a maintained SUM after every edit that touches it. With K = 8 (question 16) the read alone is up to 52–127 µs for a cell holding several large distinct denominators (see "Re-derived under K = 8"), and the update cost comes on top. ADR 009 says Eager "pays the read on every aggregate evaluation", and today's TUI projects the whole grid every frame, so it pays one read per summed cell. Against ADR 009's budget (8 ms per operation, a 2 ms slice for a job; ADR 008 clause 11), as arithmetic from these figures and not a finding: 154 such cells at 52 µs (154 × 52 = 8.0 ms; 153 stay under) or 63 at 127 µs (63 × 127 = 8.0 ms) exceed 8 ms, and 39 at 52 µs or 16 at 127 µs exceed a 2 ms slice. At the earlier K = 32 (superseded) it was five and twelve cells per 8 ms, and one read nearly filled a slice. K = 8 is 13 to 15 times cheaper per read than 32 (0.67/0.052, 1.9/0.127). It remains a question for ADR 009's Eager and cap design, but a far smaller one. The worst case needs 2 to 8 large distinct denominators in one sum; a sum of small denominators is far cheaper. ADR 009's own ExactSum figures (2.0–3.6 ns per update, 0.6–1.1 µs per read) are for the f64 accumulator and are not comparable.
  • Flag, stale text in ADR 009. At its head it describes ADR 005's cost as rust_decimal alone, "not a bignum, +15.4 KB", with 40 bytes per summed group. Under option (d) the crate also carries num-rational and num-bigint (up to about +36 KB together, unprobed together) and a per-denominator group map. It also gives ExactSum 568 bytes per group where this ADR gives the decimal partition 40 bytes; those are two different accumulators and are not a conflict.

Consequences

  • Typed values and their + - * are exact. 0.1 + 0.2 = 0.3, 1234.56 - 1234.55 = 0.01, and a typed 1.005 shows as 1.01 at two places (improvise-felo.9 may fix the last one sooner, and without this ADR).
  • SUM over exact data is independent of order, through a 40-byte wide accumulator per group (clause 8). Gotcha 18's SUM drift goes as a class for exact data, not just by pinned iteration order.
  • FLOAT_EQ_EPSILON is gone, and gotcha 17 is rewritten as "one zero predicate".
  • Deleting the epsilon changes existing documents, not only the tests. Both assertions of the improvise-0bf test flip (clause 6). And in a document today, a formula whose inputs make an inexact residual changes answer: A / 3 * 3 - A is a few ulps from zero, so IF(… = 0, …) flips from true to false, and dividing by such a residual yields a huge value where it yields div/0 today. Exact data is unaffected, because its residuals are exactly zero. The user decided that policy (question 3), and the escape hatch is explicit: EQUAL(a, b, epsilon) (clause 6). Plain examples of formulas that change, each with the question that causes it:
    • Question 3 (no epsilon): a comparison of two inexact residuals, such as IF(POW(A, 0.5) * POW(A, 0.5) - A = 0, …), was true within the epsilon and is now false. The fix is EQUAL(…, 0, 1e-9).
    • Question 7 (exact mixed comparison): IF(POW(x, 0.5) = 1.1, …) with x = 1.21. In f64, pow(1.21, 0.5) is exactly the f64 nearest 1.1, so this is true today even without the epsilon (the review measured it). It becomes false because it compares that inexact value with the exact decimal 1.1. Revision 4 filed this under question 3, and that was wrong. The fix is EQUAL(POW(x, 0.5), 1.1, 1e-9).
    • Option (d): IF(A / 3 * 3 = A, …) on typed data is true today within the epsilon, and it stays true, now exactly, because A / 3 is an exact fraction.
    • Question 9 (version skew): an old peer's 0.1 arrives as a bare JSON number and reads as Inexact(0.1). That is unequal to every typed 0.1, though the two look identical on screen. improvise-szi8 closes it.
  • A scale-only edit (12.5 to 12.50) bumps the record's version (clause 9).
  • Exact(0.1) = Inexact(0.1) is false, because comparison between exact and inexact is exact (clause 6, question 7).
  • A typed 12.50 stays 12.50 in the editor.
  • Division that does not terminate, and AVG over any count, are exact fractions within the i128 bound (decided, question 2). POW with a fractional exponent is inexact.
  • .improv gains #i, n/d and ADR 007's version literal. Existing files load as the decimals they spell.
  • The wire spells every number as a string in improvise's own encoding (decided, question 6). A thin client needs no new rule, because typed() and the number language are shared.
  • n/d is text wherever a type is inferred and a fraction wherever it is forced (decided, question 12).
  • nan, inf and a finite lexeme that overflows f64 (1e999) are text at import, as improvise-jc95 decided. typed() narrows to the three canonical tokens, so a typed Infinity, NaN or 1e999 becomes text where today it is a number (clause 3).
  • A value typed past Decimal's range is an exact fraction when it fits i128 (1e-30, a 30-digit integer), as the same value computed already is (clause 3).
  • A SUM reads exactly over at most K distinct fraction denominators, at a worst-case cost that grows about as K² (question 16: K = 8, decided), and past K it is inexact.
  • Formulas gain EQUAL(a, b, epsilon), and EQUAL becomes a keyword the writer quotes.
  • Costs: Num is 24 bytes where an f64 is 8, but CellValue stays 32 bytes under both (a) and (d), because its String arm already sets the size (the review measured it). The bundle gains rust_decimal (+15.4 KB gzipped, probed) and, under (d), num-rational and num-bigint (+20.8 KB gzipped probed alone, up to about +36 KB together, unprobed together). Arithmetic is about 8× slower per add and about 100× per division than f64 (measured, microbenchmark). A 90,000-record formula that divides for every record pays about 5–10 ms in arithmetic alone. Whether the evaluator notices is migration step 1's measurement, and it is not measured here.
  • Not covered: the order-free sum of inexact values, which is ADR 009's accumulator; the protocol version, which is improvise-szi8; conversion functions such as EXACT()/INEXACT(), which no ticket asks for.
  • Option (d)'s own consequences apply, now that it is decided: a third numeric arm, a grouped sum accumulator, the n/d spelling, exact quotients, and a result's scale depending on whether it passed through the fraction arm.

Appendix: the measurements

A scratch crate outside the workspace, run with cargo run --release. It was not committed. Committing it would add three decimal crates to the workspace's dev dependencies, and every cargo test --workspace would then build them, to hold a proposal's evidence. If the user accepts this ADR, migration step 1 commits the rust_decimal and fraction halves of the measurement as an example of the numeric crate (clause 14), whose normal dependencies already include what they need. So it adds no dependency beyond the ones the ADR adds.

Data. A typed value is a two-place decimal from 0.01 to 999,999.99, drawn by SplitMix64 (the same generator as ADR 009's retraction_drift.rs), and spelled as the text a person types. Each representation parses that text: f64 by str::parse, Decimal/BigDecimal by from_str, and BigRational as mantissa over 10^scale.

SUM. A fold over N parsed values, N = 18,000 and 90,000, best of 50 runs (20 for bigdecimal, 5 for BigRational), reported per element. The order check folds forward and reversed.

Formula. For N records with independent Revenue and Cost, SUM((Rev − Cost) / Rev). bigdecimal is run at its default precision (100 digits) and at 28. BigRational is summed incrementally, printing the running denominator's bit length at powers of two, and abandoned after 20 seconds.

Accumulator stand-in. Each stored f64 is converted to its exact rational (BigRational::from_float), summed exactly, and rounded once to f64. That is precisely what a correct superaccumulator returns. It is compared with the typed total, the decimal sum of the texts converted to f64 once. Small N: 10,000 trials each at N = 2, 3, 10 and 100, seed 7. Large N: seeds 1 to 8. The signed distribution draws cents uniformly from −100,000 to 100,000 (±1,000.00), with the same N, trials and seed; it was not run at large N.

Display. format.rs:127-136's round_half_away, copied verbatim, against Decimal::round_dp_with_strategy(2, MidpointAwayFromZero), for 2.675, 1.005, 0.125 and 1.115.

Wasm size. A cdylib exporting one function that parses two strings, then does (a + b) − b, divides by b and prints, in each representation. It also parses and prints an f64 in every variant, so f64 formatting is in the baseline as it is in the real bundle. Profile opt-level = "s", lto = true, codegen-units = 1, panic = "abort". Then wasm-opt -Oz --enable-bulk-memory --enable-nontrapping-float-to-int, and gzip -9 for the compressed figure.

Revision 2's checks of the review's findings, in the same scratch crate:

  • B1. (p + 1e12) − 1e12 with p = 0.30000000000000004 gives 0.3000000000000000 at scale 16. (1e12 − 1e12) + p gives 0.30000000000000004 at scale 17.
  • B2. 200,000 Decimals from from_i128_with_scale, with a 96-bit mantissa and a scale drawn from 0–28 (SplitMix64, seed 11). to_f64() differed from to_string().parse::<f64>() in 17,375 of them. The review's own sample found 17,392.
  • Parsing. Decimal::from_str("0.12345678901234567890123456789") returned 0.1234567890123456789012345679, and from_str_exact refused it.

Everywhere this appendix converts a decimal total to f64 ("the typed total"), it does so through to_string().parse::<f64>(), the correctly rounded path, not to_f64. So the accumulator table above does not depend on the conversion B2 found wrong.

Option (d) (revision 3) uses the same crate, with num-rational 0.4.2's Ratio<i64> and Ratio<i128> checked operations. Operation costs are measured over 18,000 pairs of typed values: seeds 21 and 22 for two-place operands, and those pairs' margins for unrelated denominators. The overflow cases use seeds 1–8, drawing Revenue and Cost from seeds 2s and 2s + 1. Sizes are size_of of the three Num shapes. The to_f64 check compared Ratio<i128>::to_f64 with BigRational::to_f64 on 200,000 fractions (seed 5). That comparison is circular, because the two share one routine, so the ADR now relies on the review's independent interval check instead (option (d), "Operations"). Every figure is the range over two runs on the machine above.

The grouped read (revision 6), in the same crate. For K = 2, 8, 16 and 32, 200 reads each. A read combines K groups whose numerators are random 250-bit integers and whose denominators are random odd 126-bit integers (xorshift64, one fixed seed), sorted ascending by denominator. Two methods are timed on the same groups: BigRational addition group by group, which reduces by a gcd after every add, and an lcm accumulation with one gcd reduction at the end. The two results are asserted equal. One run on the machine above; a second run agreed within 6% (within 2.7% above K = 2). Where the text gives a range such as 0.67–1.9 ms, it runs from the lcm method to the per-add method.