diff --git a/context/adr/005-exact-decimal-arithmetic.md b/context/adr/005-exact-decimal-arithmetic.md new file mode 100644 index 00000000..ae57cfab --- /dev/null +++ b/context/adr/005-exact-decimal-arithmetic.md @@ -0,0 +1,2097 @@ +# ADR 005 — Exact decimal arithmetic: a three-level numeric tower for formulas + +**Status:** proposed. Only the user accepts an ADR. All sixteen numbered questions are +answered (question 16 last, in revision 8, K = 8); the ADR waits for the user's "Approved". · +**Ticket:** improvise-l2b8 (parent improvise-felo) · +**Direction from:** the user, 2026-09-22 (below) · +**Relates to:** ADR 009 (incremental evaluation, improvise-felo.4, PR #309), which now +carries an exact f64 accumulator; ADR 007 (stable identities), whose new file-version +literal this ADR may share · +**Governed by:** `context/design-principles.md` §3 and §5 + +> "Formulas should default to decimal math (ideally following Scheme's numeric tower +> rules for promotion)." +> +> — the user, 2026-09-22, recorded on improvise-l2b8 + +**Numbering.** This is the ADR that improvise-l2b8 claimed as 005 before ADR 007 was +written. improvise-8tk5 and improvise-80rm say "ADR 005" in their titles only because +ADR 007 was first filed under this number. They belong to ADR 007, not to this document. + +**Citations.** Every `file:line` below is at `gh/main` `b639266b`. Line numbers are +dated, so re-derive them before citing this document elsewhere. "ADR 009" means the +head of PR #309 at `4973845f` (revision 6; it was `25ebf641` through revision 5). + +**Revision 2** answers the adversarial review of `1fe3a85d` (a BLOCK) and the four +ChatGPT Codex findings on the same head. The review's two blocking points are both +correct, and both were reproduced here before the text changed: an exact `SUM` was +**not** independent of order at the 28-digit bound (clause 8 now sums exactly in a wide +integer), and `rust_decimal`'s conversion to f64 is **not** correctly rounded (clause 4 +now names one that is). Questions 1–9 keep their numbers; 10 and 11 are new. + +**Revision 3** adds option (d) to question 2, at the user's request: "I think (d): exact +fractions with a size limit. should be added." It is specified and measured in +[Option (d) for question 2](#option-d-for-question-2-bounded-exact-fractions). **The +recommendation for question 2 changes from (a) to (d)**, in a hybrid form: the decimal +arm stays, and a bounded fraction arm is added only for the values (a) would have made +inexact. The clauses below are still written to (a). That section lists what changes in +them if the user answers (d). Question 12 is new. + +**Revision 4** moves the numeric tower into **its own workspace crate**, at the user's +direction: "I think ADR 005 should implement the math operations in a separate crate". +That is new clause 14. Clause 2 and the migration now place `Num` there, not in +`improvise-core`. It also adds questions 13–15: where the fraction bound applies (13), +the crate's name (14), and where ADR 009's `ExactSum` lives (15). The same revision +answers the review of `e4acf643` (BLOCK D1 and D2, and its should-fix list) and Codex's +P2 on aligning CSV entry with PR #321. + +**Revision 5** folds in **the user's answers** to questions 1–12, which they left +inline on `e4acf643` at 06:49–06:57 UTC on 2026-09-25. Revision 4 missed them, and that +was the review of `87ae21ca`'s first blocking point. They are quoted in +[What the user decided](#what-the-user-decided), and the clauses are now written to +them: three levels, option (d), exact comparison plus an `EQUAL(a, b, epsilon)` +function, a string encoding on the wire, and `n/d` read as a fraction only where the +type is forced. The same revision answers the rest of that review (B2–B4, D1's +correction, question 13, the crate homes). Questions 10 and 13–15 stay open, and nothing +is renumbered. + +**Revision 6** folds in **the user's answers to questions 10, 13, 14 and 15**, left +inline on `33833f28` at 17:45–17:48 UTC on 2026-09-25: the 320-bit decimal partition +and option (a) for question 10, (b) for question 13, `numtower` for question 14, and (a) +for question 15. It answers the review of `33833f28` (issuecomment-5836990563): +- **N1.** The number language's gate is now the **finite** results of `f64::from_str`, + so it is #321's gate. `nan`, `inf` and `1e999` stay text at import, as improvise-jc95 + decided. Reversing jc95 is named as the rejected alternative (clause 3). +- **N2.** Entry tries the **fraction arm before f64**, so a typed value is exactly as + exact as the same value computed (clause 3). +- The should-fix items: `raw_value`'s cost to the import pipeline is stated (clause 12, + step 5); the stale text is corrected; the ceiling is sized to the grouped read; and + the exact read's cost is now **measured**. It is about 50× the revision 5 estimate, + which is why question 16 is new. + +It also corrects when a last-bit miss is visible to `=`. The bound is on the **terms**, +not the total, as ADR 009 now says at `4973845f` (Codex on #309). + +**Revision 7** folded in **the user's first answer to question 16**, left inline on +`b232c5a9` at 03:37 UTC on 2026-10-09: "let's do k = 32". It made K **32**, option (a). +**Revision 8 supersedes that answer** (below). Every question 1–16 was then answered, and the ADR stayed +**proposed** until the user says "Approved". The change is confined to the value of K +and the text that waited on it; the figures that depend on K were re-derived from the +appendix's own measurements and none is new (question 16's "Re-derived" notes list them, now for K = 8). The revision also adds "Relations to other ADRs", a note on what ADRs +007, 008 and 009 ask of this one since it was written. It changes none of the user's +decisions, and it lists the places where the other ADRs' text and this one's disagree. + +**Revision 8** folds in **the user's second answer to question 16**, a PR-conversation +comment at 04:04 UTC on 2026-10-09: "Ok, let's drop K back to 8". It **supersedes** "let's +do k = 32": K is **8**, option (b), which was the recommendation of question 16. The +earlier answer stays on the record, quoted in "What the user decided". **The user gave no +reason for the change, and this ADR does not supply or guess one.** Every dependent figure was re-derived from the appendix's own +measurements (question 16 lists them), and none is new. The K = 8 cutoff is still the +second exception to "exact iff it fits" (question 13), and still overrides +representability. + +## What the user decided + +The user answered inline on `e4acf643`, one comment per question. Each answer is quoted +verbatim, with what it changed. + +**1. The tower's shape.** +> "I think, with bounded fractions, we need at least 3" + +Decided: **three levels**. Exact decimal, exact bounded fraction and inexact f64 +(clauses 1 and 2). + +**2. Non-terminating exact division.** +> "(d)" + +Decided: **option (d)**, bounded exact fractions, in the hybrid form. A non-terminating +quotient that fits is exact (clause 5, and the option (d) section, now normative). + +**3. A tolerance on inexact `=`.** +> "Exact comparison everywhere but add `equal(a,b,epsilon)` to formula language" + +Decided: **no tolerance on `=`**. A new formula function, `EQUAL(a, b, epsilon)`, is +the explicit tolerant comparison (clause 6). It replaces the earlier "an `APPROX` if it +is ever wanted". + +**4. The `.improv` spelling of an inexact finite number.** +> "`#i` works" + +Decided: **`#i`** (clause 10). + +**5. The version literal.** +> "share with ADR 007" + +Decided: **share ADR 007 clause 10's new literal**, unconditionally (clause 10). + +**6. JSON import precision, and the wire.** +> "I think we either have to add our own encoding as strings on the wire or use +> arbitrary precision: the round trip through f64 seems dangerous (unless maybe we tag +> f64s as exact somehow like NaN boxing)" + +Decided: **no value crosses as a JSON number, and no number is recovered by a round +trip through f64.** +- The wire spells every number as a string in its own encoding (clause 11). +- JSON import reads each number's **source lexeme**, not the f64 `serde_json` made of + it (clause 12). +- NaN boxing is not needed once the encoding is a string: the string carries the + exactness. +- `serde_json`'s `arbitrary_precision` stays rejected, for a reason the string encoding + does not remove: it breaks serde's buffered deserialization of numbers. The + workspace has an internally tagged enum (`improvise-wasm-client/src/render.rs:86`) + and a `#[serde(flatten)]` (`gp-tree/src/data.rs:64`), and the feature is unified + across the workspace. So the lexeme is read with `serde_json`'s `raw_value` feature, + which is additive and changes nothing else. + +**7. Comparing an exact value with an inexact one.** +> "Exactly, but add `equal(a,b,epsilon)` to formula language." + +Decided: **exact comparison**, with the same `EQUAL` function as question 3 (clause 6). + +**8. The dependency.** +> "yes, this seems fine." + +Decided: **yes**. One figure changed after the answer, and it is flagged here. +- The user answered against option (a)'s +15.4 KB gzipped for `rust_decimal` alone. +- Under (d), `numtower` also carries `num-rational` and `num-bigint`. The probe + measured those at about +20.8 KB gzipped on their own; together with `rust_decimal` + they are unprobed (clause 14). +- If that changes the answer, question 8 reopens. + +**9. Version skew on the wire.** +> "yeah, accept and add a ticket for handshake version." + +Decided: **accept the skew**. The handshake version is filed as **improvise-szi8**. + +**10. The exact-sum accumulator.** +> "Don't we need a much bigger accumulator (2000ish bits) to handle exact accumulation +> generically?" + +That was a question back to the ADR, answered with numbers in clause 8, "How wide, +generically". In short: +- **Decimals need 320 bits**, because their range is small. +- **f64 values need about 2,100 bits.** That is ADR 009's accumulator. +- **Fractions have no fixed generic width.** Combining k arbitrary `i128` denominators + needs up to 127·k bits. The user's ~2,000 bits is exactly k = 16. + +Then, on `33833f28`, on the line "decimals need 320 bits": +> "320bits is fine" + +and on option (a): +> "(a)" + +Decided: **(a)**. A 320-bit decimal partition, a 256-bit numerator per fraction group, +and a read that combines at most K groups in a bigint rational and then applies the +stored bound. One figure changed after the answer, and it is flagged here: the exact +read was an estimate of 16–34 µs at K = 32, and it measures **0.67–1.9 ms** in the worst +case (option (d), "What that costs"). K is a constant inside `numtower`, not a format +or wire fact, so its value was split out as question 16 rather than reopening this one. +The user decided K = 8 there (first K = 32, then K = 8; the later answer governs). + +**11. The `MIN`/`MAX` tie-break.** +> "Yeah, exact." + +Decided: **prefer the exact value** (clause 8). + +**12. Typing a fraction.** +> "keep as text for type inference from strings because of dates, read as fraction in +> places where the type is forced." + +Decided: **`n/d` is text wherever a type is inferred, and a fraction wherever the type +is forced.** So a typed or imported `3/4` whose type is being guessed stays text, which +keeps dates. A column the user has forced to a measure reads `3/4` as a fraction. +Clause 12 defines the one number language this uses. + +**13. Where the fraction bound applies.** +> "(b)" + +Decided: **(b)**. The bound applies to the **stored** result, and every intermediate +inside one formula evaluation is unbounded up to the ceiling. The ceiling is the first +of two exceptions to "exact iff it fits", sized to the grouped read (question 13). The +second, added by revision 7, is the K cutoff on a `SUM`, K = 8 since revision 8 +(question 13, "The K cutoff"). + +**14. The numeric crate's name.** +> "this is fine" + +Decided: **`numtower`** (clause 14). + +**15. Where ADR 009's `ExactSum` lives.** +> "(a)" + +Decided: **(a)**, in `numtower`, beside the exact decimal accumulator. ADR 009's seam +crate adapts it to `zset`'s group trait through a newtype, and `numtower` can land +holding `ExactSum` alone, ahead of this ADR (clause 14). + +**16. The value of K.** The user answered twice. Both are quoted verbatim. + +First, inline on `b232c5a9`, on question 16's options (2026-10-09T03:37:34Z, comment +4226381911): +> "let's do k = 32" + +Then, in a PR-conversation comment (2026-10-09T04:04:36Z, comment 6074029223): +> "Ok, let's drop K back to 8" + +Decided: **(b), K = 8.** The later answer governs, and the first is superseded, not +deleted. K = 8 is the recommendation of question 16, and not the value revision 5 +proposed and question 10 was answered with. The worst-case exact read is **52–127 µs** +(52 µs is the lcm method, the one `numtower` uses; 127 µs reduces after every add). The +combined numerator is at most K·127 + 256 = 1,272 bits (before the up to 3 carry bits +of summing 8 terms), which the proposed 8,192-bit ceiling clears by 6,920, enough to +absorb them. A `SUM` over more than 8 distinct non-decimal denominators is +inexact, and the cutoff overrides representability (question 13). K remains a constant +inside `numtower`, so it can still be tuned later against the evaluator benchmark +without a new ADR. + +## The decision this ADR writes down + +1. **Three levels** (decided, question 1). A number is an **exact decimal**, an **exact + bounded fraction** (an `i128` numerator and denominator), or **inexact** (an f64). + There are no unbounded rationals. They were measured, and they fail on the most + ordinary pivot-table question, the sum of ratios (clause 1). +2. **Exact is the default.** Every number a person types, every number in a `.improv` + file, every CSV or JSON number and every formula literal is exact when it fits + either exact arm: a decimal first, then a bounded fraction (clause 3). +3. **Exactness is lost only in named places.** `+`, `-`, `*` and `/` keep it: a + quotient is a decimal when it terminates and fits, and otherwise a bounded fraction + (decided, question 2). `POW` keeps it for an integer exponent. A non-integer exponent, + a result that fits in neither exact form, and any inexact operand make the result + inexact. That is Scheme's contagion rule. +4. **The epsilon goes.** `=` and `!=` compare exactly, and the division-by-zero guard + tests for exactly zero. The two still share one predicate, which is the property + gotcha 17 exists to keep. A tolerant comparison is explicit: `EQUAL(a, b, epsilon)` + (decided, questions 3 and 7; clause 6). +5. **The stored type is `CellValue::Number(Num)`**, where `Num` is `Exact(Decimal)`, + `Fraction(Box>)` or `Inexact(f64)`. The crates are `rust_decimal` and + `num-rational`. `Num` and everything numeric live in a + problem-agnostic workspace crate of their own, `numtower` (decided, question 14; clause 14). +6. **Each kind of number has its own exact accumulator.** ADR 009's f64 accumulator + makes `SUM` over *inexact* values order-free, and this ADR does not re-specify it. + The *exact* values get a wide fixed-point integer of their own (clause 8), because + `rust_decimal`'s own addition rounds at its bound and so is not order-free there. + What this ADR adds beyond summation is everything the f64 accumulator cannot reach: + per-cell arithmetic, typed money-like values, exact comparison, and showing a typed + value back as it was typed. Clause 8 states the boundary, and the measurement behind + it. + +## Context + +### How a number flows today + +Every number is an `f64` from the moment it is parsed until the moment it is drawn. + +| where | what | `file:line` | +|---|---|---| +| the stored type | `CellValue::Number(f64)`, serialized through `wire_f64` | `crates/improvise-core/src/model/cell.rs:116-122` | +| non-finite tokens | `nonfinite_token` / `parse_nonfinite` (`inf`, `-inf`, `nan`) | `cell.rs:63-82` | +| the wire adaptor | JSON number when finite, token string when not (improvise-55v2) | `cell.rs:84-114` | +| editor seed | `CellValue::edit_text`: Rust's shortest round-trip spelling (improvise-umi4) | `cell.rs:137-145` | +| screen spelling | `Display`: integer, else `{:.4}` | `cell.rs:148-161` | +| cell aggregation | `apply_agg` over `&[f64]`; `aggregate_cell_values` filters `as_f64` | `crates/improvise-core/src/model/types.rs:20-44` | +| formula aggregation | `eval_agg`, `values.iter().sum()` in store order | `types.rs:57-87` | +| the epsilon | `FLOAT_EQ_EPSILON = 1e-10` (improvise-0bf) | `types.rs:16` | +| record-level eval | `eval_expr`: `lv + rv`, `lv / rv` behind the epsilon, `lv.powf(rv)` | `types.rs:229-282` | +| comparison | `eval_bool`: `(lv - rv).abs() < FLOAT_EQ_EPSILON` | `types.rs:314-315` | +| the cached twin | `eval_expr_cached`, the same arithmetic again | `types.rs:1090-1104`, `1156-1157` | +| formula literal | `Expr::Number(f64)`, from `number = ASCII_DIGIT+ ("." ASCII_DIGIT*)? \| "." ASCII_DIGIT+`: no sign, no exponent, no digit bound, and `1.` and `.5` are legal | `crates/improvise-formula/src/ast.rs:35`, `formula.pest:105-108`, `parser.rs:192` | +| typed input | `typed()`: `text.parse::()`, else text (improvise-j7v6) | `crates/improvise-protocol/src/resolve.rs:1438-1446` | +| display | `round_half_away` scales by `10^d` in f64, then `format!` | `crates/improvise-core/src/format.rs:125-160` | +| `.improv` write | `format_number`: integer, else shortest `Display`, else `{:?}` | `crates/improvise-io/src/persistence/format.rs:38-54` | +| `.improv` grammar | `number = "-"? digits ("." digits)? (exponent)?` | `crates/improvise-io/src/persistence/improv.pest:110-112` | +| `.improv` read | `parse_number_cell`: `s.parse()` into f64 | `crates/improvise-io/src/persistence/parse.rs:141-146` | +| CSV field | `i64`, else `f64`, into a `serde_json::Number` | `crates/improvise-io/src/import/csv_parser.rs:167-186` | +| import measures | read back with `as_f64`, which drops an exact `i64` of more than 15 digits | `crates/improvise-io/src/import/wizard.rs:425` | +| record fingerprint | hashes a number's **bits**, not its `PartialEq` (ADR 001's record tier) | `crates/improvise-command/src/ui/app.rs:231-250` | +| JSON import | `serde_json::Value`, numbers already f64 (`float_roundtrip`) | `src/main.rs:365`, `crates/improvise-command/src/ui/effect.rs:1151`, `1280` | +| import collisions | the "sum" policy adds in f64, in record order (improvise-cgqt) | `crates/improvise-io/src/import/wizard.rs:307`, `330` | +| the wire | `ModelCommand::SetCell { value: CellValue }`, `CellDisplay::value`, `SetYanked` | `crates/improvise-protocol/src/reduce.rs:33`, `88-92`; `render.rs:88-104` | +| gp seam | `cell_text`: a number crosses **as text**, `n.to_string()` | `crates/improvise-gp/src/dataset.rs:50-56` | + +`grep -rn 'CellValue::Number' --include='*.rs' .` finds 672 occurrences in 47 files, +tests and `benches/` included. That is the blast radius of any change to the stored +type. + +### What is already wrong, measured + +The measurements in this ADR were taken on an **Apple M4 Max (64 GB), rustc 1.98.1, +release builds**, by a scratch program described in the appendix. They were not +committed, for the reason the appendix gives. + +**Per-cell arithmetic is wrong in the last place, and the user sees it.** + +- `0.1 + 0.2` is `0.30000000000000004`, and ten `0.1`s sum to `0.9999999999999999`. +- `1234.56 - 1234.55` is `0.009999999999990905`, and `100.10 - 100` is + `0.09999999999999432`. + +A display format hides most of this. It does not hide it from `=`, from an editor, or +from the next formula. + +**Display rounding is wrong for a typed value.** `format.rs:127-136` rounds by scaling +the f64. A typed `1.005` is stored as `1.00499999999999989…`, so at two places the grid +shows **`1.00`**. A person who typed `1.005` and asked for two places expects `1.01`. +Half-away-from-zero on the decimal gives `1.01`. `2.675`, `0.125` and `1.115` happen to +agree on both paths. The review found more that miss: `1.015` draws `1.01`, `1.025` +draws `1.02` and `0.285` draws `0.28`. So the bug is real but intermittent, which is the +worst kind for a spreadsheet. It is filed on its own as **improvise-felo.9**, because it +can be fixed without this ADR: round the shortest round-trip spelling as a decimal. + +**`SUM` depends on order, and on how far it is from what was typed.** SUM over +18,000 typed two-place decimals (0.01 to 999,999.99, seed 1): + +- the forward and the reverse f64 fold differ by **5.0e-5**; +- the forward fold is **5.3e-5** from the exact decimal total (1.4e-4 at 90,000 cells); +- the decimal sum (`rust_decimal`) is identical in both orders **for this data**. It is + not identical in general: at the 28-digit bound `rust_decimal` rounds, and then order + matters again (clause 8). + +ADR 009's own measurement found the same thing with five-place values: the eager sum is +up to 5.8e-10 from the exact decimal sum, above `FLOAT_EQ_EPSILON` at 7 of 8 +checkpoints (improvise-l2b8's comment). Gotcha 18 recorded the order dependence as a real +bug (improvise-shyt), fixed by pinning iteration order rather than by removing it. + +**The epsilon is a patch, and it does not scale.** It was chosen because it "absorbs +accumulated f64 rounding noise" (`context/code/core.md` §4). But one ulp of 1,000,000 is +1.16e-10. So once a value is above about 2¹⁹ (524,288), a one-ulp error in it is +already larger than the epsilon, and `=` sees it. Below about 1e-10, the epsilon calls +different numbers equal. + +### What ADR 009's exact accumulator fixes, and what it does not + +ADR 009 now proposes a **superaccumulator**: a fixed-point integer wide enough for every +finite f64. It sums f64 values exactly and rounds once, when the sum is read. It makes +`SUM` independent of order, and its retraction exact, **without this ADR**. That changes +what this ADR is for, so it was measured rather than assumed. + +The question: after a perfect f64 accumulator, does the sum of typed decimals equal the +typed total? The stand-in for the accumulator was an exact rational sum of the stored +f64 values, rounded once (appendix). + +The miss rate depends on **the values' distribution**, not only on how many are summed, +so two distributions were measured. The first is positive two-place values uniform in +0.01–999,999.99, as the rest of this ADR uses. The second is two-place values uniform in +±1,000.00. The ADR 009 agent found it with an independent Python `Fraction` check (about +37–40%), and it was reproduced here with the same stand-in (seed 7). + +| typed two-place values summed | trials | accumulator ≠ typed total, 0.01–999,999.99 | accumulator ≠ typed total, ±1,000.00 | naive f64 fold ≠ typed total, 0.01–999,999.99 | +|---|---|---|---|---| +| 2 | 10,000 | **2,120** (21%) | **3,792** (38%) | 2,120 (21%) | +| 3 | 10,000 | **1,621** (16%) | **3,836** (38%) | 2,683 (27%) | +| 10 | 10,000 | **752** (7.5%) | **3,950** (40%) | 4,039 (40%) | +| 100 | 10,000 | **250** (2.5%) | **4,018** (40%) | 7,944 (79%) | +| 18,000 | 8 seeds | **0** | **5 of 8** seeds (the review) | (all 8: see above) | +| 90,000 | 8 seeds | **0** | **2 of 8** seeds (the review) | (all 8) | + +The signed column's large-N cells were measured by the review of `e4acf643`, with the +same generator and stand-in. It also measured signed N = 1,000: 84 of 200 trials +(42%). At 18k it found 16 of 40 trials with seed 7, and 5 of 8 across seeds 1–8. So +"about 40% at every length" holds out to 18k. + +**The accumulator helps only where the total is large beside its terms.** With positive +values the total grows with the count, its ulp grows with it, and the input rounding +errors cancel far below that ulp. So the one rounding lands on the typed total, and over +thousands of cells it always did. With signed values the total stays small, because the +terms cancel. The input errors are then comparable to the total's ulp at every count, +and about 40% of sums miss however many terms there are. Revision 2 said "the +accumulator solves the large sum". That holds only for same-sign data, and it is +corrected here. For two summands a superaccumulator is exactly `+`, because one f64 +addition is already correctly rounded. So `0.1 + 0.2` is still `0.30000000000000004` +under the accumulator. + +**The misses are in the last bit of the largest term, not of the total.** Each stored +term is off from its typed value by at most half an ulp of that term. So the exact sum of +the stored terms is off from the typed total by at most about ½·Σ ulp(term), plus the +total's own half-ulp at the final rounding (revision 6; revision 5 put the condition on +the total, and ADR 009 made the same correction at `4973845f`, from ChatGPT Codex on +#309). When terms cancel, a miss can be far larger than the total's ulp: +- **The large-term, small-total case.** `1000000000000000.10 + (-1000000000000000.00)` + has a typed total of 0.10. The first term is stored as `1e15 + 0.125`, because the ulp + at 1e15 is 0.125. So the exact sum of the stored terms is **0.125**, and the miss is + 0.025 on a total of 0.1. +- **When `=` sees one today.** `=` is an absolute `1e-10` (gotcha 17), so a miss can + reach it only when ½·Σ ulp(term) reaches `1e-10`. That needs at least one term of + about 2¹⁹ ≈ 5.2e5 or larger, or many terms just below it. It does not depend on the + total. The ±1,000 values are far below 2¹⁹, so today's epsilon hides their misses; + the positive values up to 999,999.99 pass it. +- **Once this ADR deletes the epsilon** (clause 6), every miss is visible to `=`. + +A person reading a subtotal at two places usually sees the right number, because the +miss is below the displayed digits unless the terms are large. The miss also shows in +the editor, which opens on the shortest round-trip spelling (`0.30000000000000004`). +That is where 16% of positive 3-term sums, and about 40% of signed sums of any length, +go wrong. + +So the division of labour is: + +| | ADR 009 accumulator | this ADR | +|---|---|---| +| `SUM`/`AVG` independent of order, exact retraction | **yes**, for inexact values | yes for exact values, through its **own** wide fixed-point accumulator; it uses ADR 009's for inexact ones (clause 8) | +| a large same-sign `SUM` equal to its typed total | yes, measured above | yes | +| a small `SUM`, or a signed `SUM` of any size, equal to its typed total | **no**: 16% of positive 3-term sums miss, and about 40% of signed sums | yes | +| per-cell arithmetic (`Revenue - Cost`, `* 1.1`, `0.1 + 0.2`) | **no** | yes | +| exact `=` against a typed value, with no epsilon | only where the sum happens to round right | yes | +| a typed value shown back as typed (`12.50`, `1.005` at two places) | no | yes | +| cost | 568 bytes per group, 2.0–3.6 ns per update, 0.6–1.1 µs per read (ADR 009's figures) | 16 bytes per value; about 5–8 ns per add, 60–110 ns per division; 40 bytes per summed group (clause 8, question 10; an estimate, not measured) | + +**This ADR does not re-specify the f64 accumulator.** Where an inexact value must be +summed, it is summed by ADR 009's accumulator, and this ADR only says so. ADR 009's +"what each exact domain covers" table says that accumulator makes `=` against a typed +decimal right "only for sums". The measurement above narrows that to *large same-sign* +sums. ADR 009 says "no, in general" since `8ebcf889`, and cites both distributions, +so the two tables agree. + +### The crates + +Three crates were evaluated. All three build for `wasm32-unknown-unknown`. + +| | `rust_decimal` 1.43.0 | `bigdecimal` 0.4.10 | `num-rational` 0.4.2 (`BigRational`) | +|---|---|---|---| +| value | 96-bit integer and a scale up to 28 | `BigInt` and an `i64` scale | `BigInt` / `BigInt` | +| inline size | **16 bytes, `Copy`**, no heap | 40 bytes and a heap buffer | 64 bytes and two heap buffers | +| `+ - *` | exact within 28 significant digits; beyond that it **rounds silently** (1000 + 1/3 comes back at scale 25) | always exact | always exact | +| `/` | rounds to 28 significant digits (`1/3*3 ≠ 1`) | rounds to 100 digits by default | exact | +| parsing past the bound | `from_str` **rounds silently** (30 digits come back as 28); `from_str_exact` refuses | exact | exact | +| to f64 | `to_f64` is **not** correctly rounded (clause 4) | — | — | +| normal-edge deps | `arrayvec`, `num-traits`, `serde` | `libm`, `num-bigint`, `num-integer`, `num-traits` | `num-bigint`, `num-integer`, `num-traits` | +| wasm size, marginal | **+35.9 KB, +15.4 KB gzipped** | +77.9 KB, +30.7 KB gzipped | +59.6 KB, +20.8 KB gzipped | +| SUM, ns per element (18k / 90k) | **5.7–8.1 / 4.5–8.5** | 14.5–22.9 / 10.8–20.7 | 609–1,057 / 491–989 | +| `(Rev − Cost) / Rev`, summed, ns per record | **65–111 / 58–112** | 2,429–5,353 | **5.3–6.8 ms**, abandoned (below) | + +The f64 references are 0.54–1.18 ns per element for SUM and 0.56–1.24 ns per record for +the formula. Each range is two full runs. `1.5 == 1.50` and the two hash equally in +`rust_decimal`. The wasm figures are `wasm-opt -Oz` over a module that already carries +f64 parse and print, as the real bundle does, so they are the marginal cost. They are +from a five-operation probe, not the real bundle, and the real cost should be re-read +when the dependency lands (question 8). + +**Exact rationals fail on the sum of ratios, and it is not close.** The ticket asked +what bounds denominator growth. Summing `(Revenue − Cost) / Revenue` over random typed +records, the denominator of the running sum grew as follows: + +| ratios summed | 256 | 512 | 1,024 | 2,048 | stopped (20 s budget) | +|---|---|---|---|---|---| +| denominator bits | 4,699 | 9,103 | 17,015 | 31,697 | 44–55 k, after 2,936–3,741 records | + +Nothing bounds it. Each new revenue brings new prime factors into the least common +denominator, so it grows linearly in bits and each addition costs more than the last. +At the stopping point one addition cost milliseconds. **An 18,000-record average margin +does not finish.** That is the most ordinary question a pivot table is asked, so exact +rationals as the *default* are rejected on measurement, not taste. + +## Decision + +### 1. Three levels: exact decimal, exact bounded fraction, inexact binary + +Scheme's tower is integer ⊂ rational ⊂ real ⊂ complex, and each level has an exact and an +inexact form. Improvise keeps **three** levels (decided, questions 1 and 2): + +- **Exact decimal:** `rust_decimal::Decimal`. Integers are decimals with scale 0. Typed + values, money and a typed scale such as `12.50` live here. +- **Exact bounded fraction:** a reduced `Ratio`. It holds only exact values that + are not a finite decimal that fits, such as `10/3` (the option (d) section). +- **Inexact:** an `f64`, Scheme's flonum. + +There are no unbounded rationals, for the reason measured above, and no complex numbers, +because nothing produces one. The exact levels are **bounded**. A decimal has 28 +significant digits and a magnitude below about 7.9 × 10²⁸, and a fraction has an `i128` +numerator and denominator. Scheme's exact numbers are unbounded, so this is the one +place the tower deviates from Scheme by design. Clause 4 and question 13 say what +happens at the bound. + +### 2. The stored type: `CellValue::Number(Num)` + +```rust +pub enum Num { + Exact(Decimal), + Fraction(Box>), // boxed: keeps Num at 24 bytes (option (d)) + Inexact(f64), +} + +pub enum CellValue { + Number(Num), + Text(String), + Error(String), +} +``` + +`Num` lives in the numeric crate (clause 14), and `CellValue` in `improvise-core` +wraps it. The arithmetic and the contagion rules are methods on `Num`, in that one +crate. Both evaluators (`eval_expr` and +`eval_expr_cached`) call the same methods, so the rules cannot drift between them the +way the epsilon once did (improvise-0bf). + +A new arm beside `Number(f64)` was rejected. It would double every numeric match at the +672 sites, and "is this cell a number" could then be asked two ways. `CellValue::as_f64` +stays, as the explicitly lossy accessor for the seams that want a binary float (clause +12). It uses clause 4's correctly rounded conversion. + +`Expr::Number(f64)` becomes **`Expr::Number(Num)`**, not `Expr::Number(Decimal)`. The +formula grammar has no digit bound, so a literal such as +`999999999999999999999999999999` is legal formula text and has no `Decimal`. It takes +clause 3's order, like every other spelled number: a decimal if it fits, else a fraction +if it fits (that literal is below 2¹²⁷, so it is the exact `Fraction` n/1), else the +nearest f64. A literal is a number by the grammar, so clause 3's gate, which decides +number-or-text for data, does not apply to it. A literal that overflows f64 (the formula +grammar has no exponent, so a digit string of more than 309 digits) keeps today's +meaning, inexact infinity, the same as an inexact overflow in `*`. +Bounding the grammar instead was rejected, because it would make a formula +that parses today stop parsing, over a limit of the representation rather than of the +language. (Codex, P2 on `1fe3a85d`.) + +### 3. What is exact on entry + +**Everything a person or a file spells in decimal is exact, when it fits.** That covers +typed input (`typed()`), `.improv` numbers, CSV fields, formula literals and pasted +values. + +**This deviates from Scheme on purpose.** Scheme reads `1.5` as *inexact* and needs +`#e1.5` for exact. Improvise reads `1.5` as exact, because a pivot table's inputs are +decimals a person typed, and "decimal math by default" is the user's instruction. So +Scheme's `#e` is improvise's default. + +A spelled number is **inexact** only when it cannot be exact: it fits neither a +`Decimal` nor an `i128` fraction (for example `1e300`, `1e-40`, or a 40-digit integer). +Then it reads as the nearest f64. The tokens `inf`, `-inf` and `nan` are inexact too, +but they are clause 7's tokens, not numbers of the language (below). + +**The parse must refuse, not round.** `Decimal::from_str` rounds a 30-digit input to 28 +digits **silently** (`0.12345678901234567890123456789` comes back as +`0.1234567890123456789012345679`). That is a rounded value calling itself exact, the one +outcome clause 4 forbids. Under the order below, that input is the exact fraction +`12345678901234567890123456789/10²⁹` instead. + +**One number language, for every entry point** (revision 5, answering B4 of the review +of `87ae21ca`). Revision 4 said "parse with `from_str_exact`", but the gate that decides +whether a string is a number at all accepts different strings. `from_str_exact` rejects +`1.5e3` and accepts `1_000`, and #321's `number_in` gate does the opposite. So the +language is defined once, and the gate and the parse cannot disagree. `numtower` owns +it. + +- **What is a number.** The gate is the strings Rust's `f64::from_str` accepts **and + reads as a finite value** (revision 6, N1 of the review of `33833f28`). That is #321's + `number_in` gate, which drops anything `Number::from_f64` cannot hold. So a number is + an optional sign; digits with an optional fraction (`1.`, `.5`); and an optional + exponent `e`/`E` with its own sign, whose value does not overflow f64. + - **`1_000` is not a number**, so it is text. + - **`nan`, `inf`, `infinity` in any case, and a finite lexeme that overflows f64** + (`1e999`, `-1e999`) are not numbers either, so they are text. That is + improvise-jc95's decision, which #321's + `a_nonfinite_numeric_token_stays_text_rather_than_becoming_zero` pins for `NaN`, + `inf`, `-inf`, `1e999` and `-1e999`. A finite lexeme must never read as infinity. + - A lexeme that **underflows** (`1e-999`) passes, because f64 reads it as a finite + zero. It is then too small for either exact arm, and it reads as that zero, + inexact. #321 treats it the same way. +- **The non-finite values are clause 7's tokens, not numbers.** `typed()`, the `.improv` + reader and the wire accept exactly `inf`, `-inf` and `nan`, the three spellings + `nonfinite_token` writes (`cell.rs:65`), as inexact values. Import does not, so a + CSV `nan` stays text, as jc95 decided. This is a behaviour change for `typed()`, and + it is stated: today it hands any string to `str::parse::` (`resolve.rs:1438`), so + a typed `Infinity` or `1e999` becomes infinity and a typed `NaN` becomes NaN. Under + this clause each is text. +- **How it parses.** A lexeme that passes the gate is parsed **exactly** from its digits + and exponent: a mantissa and a scale shift, never through an f64. Then it takes the + first arm it fits, in this order (revision 6, N2 of the same review): + - **A decimal.** If the exact value fits a `Decimal`, it is `Exact`, with the scale + the lexeme implies: `1.50` has scale 2, `1.5e3` is `1500` at scale 0, and `1.5e-3` + is `0.0015`. + - **A fraction.** Otherwise, if its reduced value `m / 10^k` fits `Ratio`, it is + a `Fraction`. So a typed `1e-30` is the exact `1/10³⁰`, and an integer between + `Decimal`'s maximum (about 7.9 × 10²⁸) and 2¹²⁷ is the exact `n/1`. This is the + order clause 4 already gives arithmetic, so the same value is equally exact typed + and computed: `1e-15 * 1e-15` was already the exact `1/10³⁰`. A fraction carries + no scale (the option (d) section). + - **The nearest f64.** Otherwise it is `f64::from_str`'s correctly rounded value, + `Inexact`. The gate has already made it finite. + - A lexeme that passes the gate is never refused. **Failing to fit falls back; failing + to parse cannot happen.** +- **Zero has no sign at the exact level.** `-0.0` reads as exact `0.0`. The sign of zero + is an f64 artefact. This is a behaviour change, and it is stated: an f64 `-0.0` never + survives entry as exact. +- **`n/d` is a number only where the type is forced** (decided, question 12). `n/d` + means an optional sign, digits, `/`, and digits with `d > 0`, and it is a fraction or, + once normalized, a decimal. The places where the type is forced are: + - the `.improv` fraction arm (clause 10); + - the wire's exact arm (clause 11); + - an import column the user has forced to a measure (clause 12). + + Everywhere a type is **inferred** it is text: `typed()`, and the import analyzer + deciding a column's kind. That keeps `3/4` as a date or a label. + +The entry points are `typed()`, the `.improv` reader (whose `number` rule is a subset of +the gate before the finite check), formula literals, the wire and import. + +**Reversing jc95 was rejected.** The alternative was a gate of every string +`f64::from_str` accepts, so that a CSV `nan` imports as `Inexact(NaN)` and a CSV `1e999` +as `Inexact(inf)`. It would make a finite lexeme read silently as infinity, which is the +invented-value class jc95 closed, and it would flip #321's test. The gain, a measure +column that can hold a NaN, is not worth either: a column carrying `nan` is shown as +text in the wizard, where the user can see it (jc95's close reason). + +**A JSON number is read from its source lexeme, never through its f64** (decided, +question 6: "the round trip through f64 seems dangerous"). `serde_json` normally parses +a number to f64 before improvise sees it. The import therefore reads each record's +values with `serde_json`'s `raw_value` feature, which hands over the number's exact +source text, and parses that text by the language above. So a JSON `12.50` keeps its +scale, and a 20-digit JSON literal stays exact. That costs a restructuring of the import +pipeline, stated in clause 12. `arbitrary_precision` was the other way +to do this, and it stays rejected, because it breaks serde's buffered deserialization of +numbers (the user's answer to question 6). Revision 4's shortest-spelling recovery of a +JSON number is withdrawn. + +### 4. How exactness propagates + +| operation | exact ∘ exact | any inexact operand | +|---|---|---| +| `+`, `-`, `*`, unary `-` | exact, **if** the result is exactly representable; else inexact | inexact | +| `/` | clause 5 | inexact | +| `POW` | clause 5 | inexact | +| `IS`, `IF` | not arithmetic: `IS` yields exact 0 or 1 | — | + +**Contagion is Scheme's rule.** An inexact operand makes the result inexact, and an +exact result never comes from an inexact input. + +**"If exactly representable" needs care, because `rust_decimal` rounds silently.** An add +whose result needs more than 28 significant digits comes back rounded, with no signal. +The measurement caught it: `1000 + 0.3333333333333333333333333333` returns scale 25. +Overflow of the magnitude is reported (`checked_add` returns `None`), but precision loss +is not. So `Num`'s arithmetic must **detect** either one and return `Inexact`. It must +never return a rounded `Exact`. A rounded value that calls itself exact is the one +outcome this design exists to prevent. + +**Detection reads the scale, then checks that the dropped digits were not zeros** +(revision 7, Copilot on `66f317e1`). The exact result of `a + b` or `a - b` has scale +`max(sa, sb)`, and of `a * b` has scale `sa + sb`. A returned scale that is smaller is a +*hint* of rounding, not proof. `1000 + 0.333…` (28 places) comes back at scale 25, and +`0.1234567890123456²` at 28 against an exact 32. Revision 6 demoted on the hint alone and +called the zero-digit case a safe false positive. It is not safe: it breaks question 13's +decided rule that a stored value is exact iff it fits. `1.2500000000000000 * +4.0000000000000000` (scale 16 each) has the exact value `5` at scale 32, which is +`5·10³²` as an integer and does not fit 96 bits, yet `5` fits a `Decimal`. So on the hint +the arithmetic recomputes the exact integer result in the 256-bit integer clause 8 +already uses (a product of two 96-bit mantissas is below 2¹⁹², and an aligned sum below +2¹⁹⁰), strips trailing zeros, and returns exact iff the stripped mantissa fits 96 +bits at a scale of at most 28. **The stored scale** (flagged reading, revision 7, so the +fingerprint of clause 9 is a function of the value and the operands alone): the +nominal scale above capped at 28, `min(nominal, 28)`, padded back with zeros while the +mantissa still fits 96 bits, and otherwise the largest scale at which it fits. So the +example is `5` at scale 28 (`5·10²⁸` fits), not scale 0 and not 32. A result that never +fired the hint keeps its returned scale, so the common path is unchanged. Otherwise the value is not a `Decimal`, and goes to the +fraction arm and then f64 as below. The recompute runs only when the scale hint fires, so +the common path is unchanged. This is a design for the implementer and is **not +measured**. Division has its own test (clause 5). + +This is not rare in practice, and the review showed why. Clause 10 loads an old file's +17-digit f64 spellings as exact, so `0.30000000000000004 + 1e12` needs 29 significant +digits and is rounded. That is the case clause 8's accumulator exists for. + +**The one conversion to f64 is correctly rounded, and it is not the crate's.** Demotion, +mixed comparison (clause 6), `as_f64` and the reading of a mixed sum (clause 8) all turn +an exact value into an f64. `rust_decimal`'s `to_f64` is **not** correctly rounded. In +200,000 random `Decimal`s with 96-bit mantissas and scales 0–28, it disagreed with a +correctly rounded parse of the same value's text 17,392 times in the review's sample +and 17,375 times in this one (about 8.7%). For example, `63294.607176721429540940386554` +gives the f64 ending `…144`, where the nearest is `…143`. With mantissas of at most 15 +digits there were no disagreements, but exact products and sums leave that range +quickly. So `Num` converts through the **decimal string**: `d.to_string()`, then Rust's +`str::parse::`, which is correctly rounded. That costs an allocation, paid only +where an exact value leaves the exact level. A correctly rounded integer algorithm can +replace it later, and it must agree with the string path on a property test. + +So "computed in f64" above means: convert each exact operand by this conversion, then do +the f64 operation. A demoted sum or product is therefore rounded twice, once per +operand and once in the operation. That is deterministic, and it is the Scheme reading +of an operation that could not stay exact. + +### 5. Division, `AVG` and `POW` + +**Division.** Exact ÷ exact is **always exact when the result fits** (decided, question +2). +- It is a **decimal** when the quotient is a finite decimal that fits: `1 / 4 = 0.25`. +- Otherwise it is a **bounded fraction** when the reduced fraction fits `i128`: + `1 / 3`, so `1 / 3 * 3` is exactly `1`. +- Otherwise it is **inexact**: the correctly rounded f64 of the exact quotient. + +The implementation must decide termination and fit without trusting `rust_decimal`'s +division, which rounds a terminating quotient silently when it needs more than 28 +places (below). + +**The termination test is arithmetic, not a round trip.** Write `a / b` as the fraction +`(ma · 10^sb) / (mb · 10^sa)`, with mantissas `ma`, `mb` and scales `sa`, `sb`. The +powers of ten contribute only 2s and 5s, so the quotient is a finite decimal exactly +when `mb / gcd(ma, mb)` is `2^p · 5^q`. That takes one `i128` gcd over the mantissas +(each below 2⁹⁶, so nothing overflows) and a loop dividing out 2s and 5s, with no +rounding anywhere. + +**Terminating is not enough: the quotient must also fit, and that test is specified +here** (revision 4, answering the review of `e4acf643`). Clause 4's scale test covers +`+ − ×` only, because a quotient's scale is not a function of its operands' scales. The +review measured `1 / 2^40`: it terminates, but it needs 40 fractional digits. +`rust_decimal` returns `0.0000000000009094947017729282` at scale 28, with no signal, and +an exact quotient can legitimately have scale 28 too. So the result alone cannot tell +the two apart. The rule: +- Write the quotient as the reduced fraction `n / (2^p · 5^q)`, computing it in 256 bits, + because `ma · 10^sb` can reach 2¹⁹⁰. +- Its exact scale is `k = max(p, q)`. +- The quotient is an exact `Decimal` **iff `k ≤ 28` and `n · 2^(k−p) · 5^(k−q)` fits + in 96 bits.** +- Otherwise, including every quotient that terminates but needs more than 28 places, it + is a `Fraction` when the reduced fraction fits `i128`. +- Otherwise it is inexact. **The one conversion is the reduced 256-bit fraction, + converted once, correctly rounded** (clause 4's interval-checked rational → f64). It + is not a conversion of each operand followed by an f64 division. Revision 4 said "the + correctly rounded f64 … through each operand's conversion and one f64 division", + which contradicted itself: two roundings are not a correct rounding. The fraction is + already in hand, so it is converted once. + +**The direct test above is normative** (revision 5, the review's correction to D1). +Multiplying back with clause 4's scale detection is *sound*: `q × b` exact and equal to +`a` implies `q = a / b`. But it is **incomplete**, because it demotes some exact +quotients. `1e-28 / 0.5 = 2e-28` is exact and fits (k = 28, mantissa 2), yet `q × 0.5` +needs scale 29, so the check flags it. The review measured this case. If the method +were left to the implementer, exactness would depend on which method was picked. So the +direct test is the rule, and multiplying back is not an allowed substitute. + +Scheme would answer `1/3` exactly, and so does improvise, within the bound. Unbounded +rationals are ruled out by the measurement above. + +**`AVG`** is `SUM / COUNT` under the division rule. The average of `10.00`, `20.00` and +`40.00` is `70/3`, an exact fraction, which displays as `23.33` at two places. The +average of `10.00`, `20.00`, `30.00` and `40.00` is exactly the decimal `25`. + +**`POW`.** `x ^ n` with exact `x` and an exact integer `n ≥ 0` is exact (repeated +multiplication, with clause 4's overflow rule). A negative integer `n` is `1 / x^|n|`, +under the division rule. **The size is checked before computing.** The result of +`(n/d)^k` has about k times the operand's width, which is known in advance. An exponent +whose result cannot fit, or would pass question 13's ceiling, is decided before any +multiplication, so `POW(x, 10^9)` allocates nothing and demotes to `powf` at once. Any non-integer exponent, or any inexact operand, is `powf`, +inexact. This is the ticket's "obvious candidate" for an inexact operation, and it is +the only one besides division: improvise has no transcendental functions. + +**Division by zero** is `Error("div/0")` for every combination of exactness, as today. +The test is **exactly zero**, with no epsilon (clause 6). + +### 6. Comparison, and the end of `FLOAT_EQ_EPSILON` + +`=`, `!=`, `<`, `>`, `<=` and `>=` compare **exactly**. `FLOAT_EQ_EPSILON` and its six +uses are deleted. + +- **Exact with exact:** exact comparison. Two decimals compare as decimals, and a + fraction against a decimal or a fraction by cross-multiplying in 256 bits. `1.5 = 1.50` + is true, because scale is presentation (clause 9), not value. +- **Inexact with inexact:** f64 comparison, as today but with no epsilon (decided, + question 3). +- **Exact with inexact:** compared **exactly**, as values. A finite f64 is a finite + binary fraction, so this is well defined. Revision 1 converted the exact operand to + f64 instead, and the review showed that rule is **non-transitive for ordinary + values**, not only at a tie: any two distinct decimals with the same nearest f64 + break it. `Exact(0.1) = Inexact(0.1)` and `Exact(0.10000000000000000001) = + Inexact(0.1)` would both be true, while the two exacts differ. A non-transitive `=` + cannot order a `MIN`, and it cannot key a sorted map. So comparison is exact (decided, + question 7). The cost is that `Exact(0.1) = Inexact(0.1)` is + **false**: the f64 is `0.1000000000000000055…`. That is Scheme's own answer (R7RS + requires `=` to be transitive), and it is the honest one. + +**Exact mixed comparison is cheap in the common case.** Convert the exact operand by +clause 4's correctly rounded conversion. If the result differs from the f64, it already +decides the order, because rounding is monotone. Only when the decimal rounds to that +very f64 is an exact tie-check needed, against the f64's exact decimal expansion, which +is finite. + +**`EQUAL(a, b, epsilon)`: the explicit tolerant comparison** (decided, questions 3 and +7: "Exact comparison everywhere but add `equal(a,b,epsilon)` to formula language"). +- **Meaning.** True iff `|a − b| ≤ epsilon`, an **absolute** tolerance. A relative + tolerance is expressible as `EQUAL(a / b, 1, epsilon)`, so no second function is + needed. +- **Evaluation.** The subtraction, the absolute value and the `≤` use the tower's own + rules. So with exact operands the test is itself exact: `EQUAL(0.1 + 0.2, 0.3, 0)` is + true. Any inexact operand makes the arithmetic inexact, but the final `≤` is still + clause 6's exact comparison. +- **Errors.** A negative `epsilon` is `Error("type")`. So is a non-number in any + position. +- **Where it may appear.** Wherever a comparison may: an `IF` condition (`eval_bool`). + It is also usable as a value of `1` or `0`, as `IS` already is. +- **Spelling.** It is a new keyword. Like the others it is case-insensitive + (`^"EQUAL"`) and is followed by `(`, so a bare `EQUAL` still names an item (PEG falls + through to `ref_expr`). The writer's keyword guard (`quote::quote_name`) must add it, + so an item literally named `EQUAL` is written `|EQUAL|` (gotcha 34). +- **Home.** The grammar is in `improvise-formula` (`formula.pest`, the AST) and the + evaluation in `improvise-core` (both evaluators). The arithmetic it calls is + `numtower`'s. It is migration step 3. + +It replaces the epsilon's one legitimate use, a tolerant test on an inexact result, by +putting the tolerance in the formula where a reader can see it. + +**Gotcha 17's invariant survives in a new form.** The epsilon was shared by `=` and the +division guard so that `IF(x = 0, a, y / x)` is coherent: the condition and the guard +must agree on what zero is (improvise-0bf). Under this ADR they agree because both test +exactly zero. The property is "one predicate for zero", not "one epsilon". + +**Both assertions of the test that pins it flip.** `near_zero_divisor_consistent_with_equality_epsilon` +(`types.rs:2709`) and its cached twin set `X = 1e-11` and assert `IF(X = 0, 1, 0)` is 1 +and `100 / X` is `div/0`. Under this ADR `X` is not zero on either side. So `IsZero` +becomes 0, and `Ratio` becomes `1e13`, the very output improvise-0bf called the bug. It +is coherent now, because the condition and the guard agree, and the test is rewritten +to assert that agreement at `X = 0` and at `X = 1e-11`, not either old value. The name +must change too, since it names the epsilon. The change reaches existing documents as +well (Consequences). + +### 7. NaN and the infinities live only at the inexact level + +An exact number is always finite. `inf`, `-inf` and `nan` are inexact, as in Scheme +(`+inf.0`, `+nan.0`). They keep their one spelling (improvise-55v2) in the file, on the +wire, and in `typed()`: `inf`, `-inf` and `nan`, and no other (clause 3). Import keeps +them as text (improvise-jc95). Where they come from is unchanged: stored data, and inexact +overflow in `*` or `POW`. Exact overflow becomes inexact (clause 4), and it may overflow +further to `inf` only at the inexact level. + +### 8. `SUM` and the other aggregates + +`apply_agg` and `eval_agg` partition their values by exactness: + +- **Exact values are summed in a wide fixed-point integer**, not with `Decimal`'s own + `+`. The sum is independent of order and never rounds. That removes gotcha 18's + order-dependent `SUM` as a class for exact data, which is nearly all data. +- **Inexact values are summed by ADR 009's exact f64 accumulator** when it lands, and by + today's fold until then. +- **The result is exact if every value was exact** and the exact total fits the stored + bound: a `Decimal`, or, under option (d), an `i128` fraction (a sum of fractions can + fit the fraction arm without fitting a `Decimal`). **A second condition applies to + fractions:** if the sum holds more than K = 8 non-decimal denominator groups it reads + the inexact f64 accumulator, so it is inexact **even when its exact total would have + fitted** (clause 8's recommendation below, question 16). Otherwise it is inexact. **It must not be a fold with contagion part-way through.** + Such a fold switches to f64 at the first inexact value in store order, so the answer + would depend on where that value sits. + +**Why `Decimal`'s own addition is not enough.** Revision 1 said exact values were +"summed exactly, as a decimal", and that is false at the 28-digit bound. The review +measured it, and it was reproduced here: + +``` +(0.30000000000000004 + 1e12) - 1e12 = 0.3000000000000000 (scale 16: rounded) +(1e12 - 1e12) + 0.30000000000000004 = 0.30000000000000004 (scale 17: exact) +``` + +Under clause 4's detect-or-demote, the first order demotes to f64 part-way through and +the second stays exact. So `SUM` over `[p, 1e12, −1e12]` would differ by order in both +its value and its exactness. Codex found the same with `Decimal::MAX`: `[M, 1, −1]` +demotes on `M + 1`, and `[M, −1, 1]` does not. And clause 10 loads old files' 17-digit +spellings as exact, so the bound is reached by ordinary data, not only at 7.9 × 10²⁸. +Once a maintained sum has demoted it cannot come back, so retraction breaks too. + +**The fix is an accumulator for the exact partition.** Every `Decimal` is `m · 10⁻ˢ` +with `|m| < 2⁹⁶` and `s ≤ 28`. So at the fixed scale 28 it is the integer +`m · 10²⁸⁻ˢ`, below 2⁹⁶ · 10²⁸ < 2¹⁹⁰. Adding and subtracting those integers is exact, +associative and commutative, so the partition is a true abelian group whatever the +order. Detect-or-demote is then applied **once, when the sum is read**: + +- if the total, with trailing zeros stripped, fits a `Decimal`, the result is that exact + value; +- otherwise it demotes, by clause 4's correctly rounded conversion of the integer over + 10²⁸. + +**Width.** For a `Decimal` term, a 256-bit integer holds about 2⁶⁵ unweighted terms of +2¹⁹⁰. ADR 009 feeds its aggregators consolidated Z-set deltas whose `i64` weights +multiply into each term, so a weighted `Decimal` term can reach 2²⁵³. 256 bits is +therefore enough only if weights are split into chunks, as ADR 009's `ExactSum` already +does at 2³⁰. 320 bits (five 64-bit limbs, 40 bytes) holds any `i64`-weighted `Decimal` +term with 2⁶⁶ of headroom and needs no chunking. + +**Revision 5 corrects that bound for option (d).** A `Fraction` whose denominator is +`2^p·5^q` with `k ≤ 28` also joins this partition (the option (d) section), and its +numerator can pass 2⁹⁶. For example `POW(10, 35)` is `Fraction(10^35/1)` with `k = 0`, +about 2²⁰⁹ at scale 28. Such a term reaches about 2²²⁰, or 2²⁸³ weighted, so 320 bits +then holds about 2³⁶ weighted terms. That is still ample: a document holds far fewer +cells. But the earlier "2⁶⁶ of headroom" holds only for `Decimal` terms. The figures are +estimates, not measured, and the partition is still far smaller than the f64 +accumulator's 568 bytes. + +**How wide, generically** (the user's question 10: "Don't we need a much bigger +accumulator (2000ish bits) to handle exact accumulation generically?"). It depends on +what is summed, and three answers differ by orders of magnitude: + +| what is summed | exact width needed | why | +|---|---|---| +| decimals (and `2^p·5^q` fractions with `k ≤ 28`) | **320 bits**, fixed | a common scale of 28 exists, and every term is below 2²²⁰ (2²⁸³ weighted) | +| f64 values | **about 2,100 bits**, fixed | a common unit of 2⁻¹⁰⁷⁴ exists, and the exponent range spans 2⁻¹⁰⁷⁴ to 2¹⁰²⁴. This is ADR 009's `ExactSum` | +| bounded fractions with arbitrary denominators | **no fixed width** | the common denominator is the lcm, which grows by up to 127 bits per new `i128` denominator | + +So the user's figure is right in two places. It is right for f64 values, which is why +ADR 009's accumulator is that size. And it is right for combining **16** arbitrary +fraction denominators: 16 × 127 = 2,032 bits. No fixed register handles fractions +generically. That is why option (d) keeps one **group per distinct denominator**, where +each group is an integer: +- a group's numerator is at most `i128` × an `i64` weight × 2⁶⁴ terms, so **256 bits** + per group; +- the only unbounded step is the **read**, which combines the groups over their lcm; +- with at most K = 8 groups that lcm is at most about 1,016 bits (K·127), or 2,030 at + K = 16 and 4,100 at K = 32, and the combined numerator at most K·127 + 256 ≈ 1,272 + bits at K = 8 (measured lcm: 988 bits for 8 random 126-bit denominators; 1,957 at 16 + and 3,924 at 32; the combined numerator was measured only at K = 32, 4,053 bits). + +Under question 13 (b) that read runs in a bigint rational (`num-bigint`), so it needs no +fixed register at all. Its result is then held to the stored bound: it is exact if it +fits a `Decimal` or an `i128` fraction, and otherwise demoted by one correctly rounded +conversion. Question 10 is revised to this shape. + +**A sum's scale is normalized.** Summing `12.50` and `1.25` reads back as `13.75`, with +no memory of which scales went in, because tracking the largest scale would not survive +retraction. That costs nothing a person can see: a sum was never typed, so there is no +typed spelling to show back. + +**A mixed sum is rounded twice, not once.** Revision 1 said "rounded once", and that was +wrong. The exact part is converted to f64 (one rounding) and added into the f64 +accumulator, whose read is the second. Both steps are deterministic and independent of +order, which is the property that matters. One rounding is possible, by combining the two +exact totals as a rational at read time. It is not proposed, because the case is rare. + +**`COUNT`** is exact (an integer). **`MIN` and `MAX`** order by clause 6's exact +comparison, which is total over the finite values. They need one tie-break: an exact and +an inexact value can be **equal as values**, for example `Exact(0.5)` and +`Inexact(0.5)`, and then "the first one met" would depend on store order. The tie-break +is to prefer the exact one, so the result keeps the most information (decided, question +11; Codex, P2 on `1fe3a85d`). NaN is skipped, as `f64::min` skips it today. Text +unanimity and `mixed` are unchanged (gotcha 7). + +**For ADR 009:** its accumulator stays the mechanism for inexact values. The exact part is +an abelian group **through this clause's wide accumulator**, not through `Decimal` +addition. Revision 1 told ADR 009 that a maintained exact `Sum` was bit-equal to eager +"with no accumulator at all", and that was false. It is bit-equal to eager on every path +**provided both backends use the same wide accumulator**, exactly as ADR 009 already +requires of its own f64 `ExactSum`. Z-set weights multiply into each term, and the width +above allows for that. + +### 9. Display, edit text and typed scale + +**Display** of an exact value rounds the decimal, half away from zero, with +`round_dp_with_strategy(d, MidpointAwayFromZero)`, then groups. So a typed `1.005` shows +as `1.01` at two places. An inexact value is rounded over its shortest round-trip +spelling (clause 12, "Format"). `format_value` and +`NumberFormat::number` take `&Num`. Gotcha 8 is unchanged: rounding is view-only and +never fed back. + +**Edit text** of an exact value is its decimal **with its scale**. A typed `12.50` opens +as `12.50`, not `12.5`, so a typed value is shown back as it was typed, up to canonical +spelling: `+1`, `1e3` and `012` become `1`, `1000` and `12`. An inexact value keeps +today's shortest round-trip spelling (improvise-umi4). A non-finite value keeps its token. + +**Scale is not value, but it is state.** `12.5` and `12.50` are equal under `=`, because +they are the same number. But a typed scale is kept, persisted and shown back in the +editor, so it is part of what a person typed. Revision 1 concluded that rewriting +`12.5` as `12.50` "does not bump the record's version". Codex showed the cost: a client +holding the old spelling could then pass the stale-basis check and overwrite the newer +one without being refused. + +So **a record's fingerprint includes the scale.** It already hashes a number's +**bits** rather than its `PartialEq` (`app.rs:231-250`), which is why `-0.0` and `0.0` +fingerprint differently today. Under this ADR `hash_value` hashes the `Num`'s +representation: exactness, mantissa and scale. So a scale-only edit bumps the record's +version, and a stale client is refused. `CellValue`'s `PartialEq` stays numeric, since it +answers a different question: whether two values are the same number (gotcha 7's +unanimity). `rust_decimal` hashes `1.5` and `1.50` equally, so the fingerprint cannot +use the crate's `Hash`. It hashes the representation explicitly. + +### 10. `.improv` persistence and migration + +**Reading.** Every `number` token reads as clause 3 says: a decimal if it fits, else a +fraction if it fits, else the nearest f64. The grammar's `number` rule +(`improv.pest:110-112`) already accepts everything a decimal needs, including an +exponent. A `number` token that overflows f64 (`1e999`, which only a hand-edited file +can hold) is text, as a CSV field is (clause 3). `inf`, `-inf` and `nan` read through +the `bare_value` arm as clause 7's tokens, as they do today (`parse.rs:268-273`). + +**Writing.** An exact decimal is written as its decimal, with its scale. An exact +fraction is written `n/d`, with the sign on `n` (`-10/3`), through the grammar's +fraction arm, which is a place where the type is forced (clause 3). An inexact *finite* +number needs a spelling that reads back as inexact, or a save and reload would silently +make it exact. The format must say which one it holds: the ticket's `1/3` against +`0.333…` point. It is Scheme's own marker, **`#i`** (decided, question 4): +`#i0.3333333333333333`. The non-finite tokens are already inexact and need no marker. + +**Inexact stored values are rare.** A `## Data` line holds raw cells: typed, imported or +pasted. None of them is inexact unless it was pasted from an inexact formula result, is +non-finite, or is out of `Decimal`'s range. The marker is a correctness requirement, +not a common sight. + +**Migration: an existing number becomes the exact decimal it spells.** Every existing +`.improv` loads unchanged in the sense that matters: every value reads as the number its +text names. That is a behaviour change at the 17th significant digit, and it goes +toward what the text says. An old file containing `0.30000000000000004` (possible where a +computed f64 was pasted and saved) loads as exactly that decimal, not as the binary +fraction the old program held. No document changes in any digit a person can see. + +**The version literal.** `#i` and `n/d` are grammar changes. An old reader would take +`#i0.5` or `10/3` as text (`bare_value`), so a file that uses either must carry a new +version literal. It **shares ADR 007 clause 10's new literal** (decided, question 5: +"share with ADR 007"): one bump, and the first line alone picks the regime, as ADR 007 +decided. A file with neither spelling reads the same under both literals. + +**`#i` and `n/d` are recognised only under the new literal.** Under the old one, a hand-written +file may hold bare text `#i0.5` in a data cell, and that must keep reading as text, or +an old file changes meaning on load. The writer always pipe-quotes text +(`persistence/format.rs:224`), so no file this program wrote carries it. The grammar +needs only a plain sequence arm ordered before `bare_value`, with no predicate, so gotcha +34's trap (the generator skips predicates) does not apply. + +**The round-trip tests** must be re-read, as the ticket says. `format_number`'s three +spellings collapse to two (decimal, `#i` shortest), and the grammar-walking generator +gains the `#i` arm. The property `parse(format(parse(x)))` stable is unchanged, and it +is now joined by **exactness preserved**, which the old format could not state. + +### 11. The wire + +`CellValue` crosses the wire in `ModelCommand::SetCell`, `SetYanked` and +`CellDisplay::value`. `CellDisplay::text` and `edit_text` are strings and are unaffected. +So a thin client *is* affected, and the ticket was right to ask for confirmation rather +than assume. + +**Every number crosses as a string in improvise's own encoding** (decided, question 6: +"add our own encoding as strings on the wire … the round trip through f64 seems +dangerous"). No new peer ever writes a JSON number for a cell value: + +| value | JSON string | +|---|---| +| exact decimal | its decimal, with its scale: `"12.50"` | +| exact fraction | `"n/d"`: `"10/3"` | +| inexact, finite | `#i` and the shortest round-trip spelling: `"#i0.3333333333333333"` | +| inexact, non-finite | the token, as today: `"inf"`, `"-inf"`, `"nan"` | + +- **The string carries the exactness.** So nothing is recovered from an f64, and no + NaN boxing is needed. The four spellings are disjoint. A decimal has no `/` and no + `#`, and the tokens are neither. +- **One parser.** It is `numtower`'s: clause 3's number language, with the wire's + exact arm as a place where the type is forced, so `n/d` is accepted there. It + replaces today's untagged `wire_f64` adaptor (`cell.rs:84-114`), which was the + workspace's only `#[serde(untagged)]`. So the question of which untagged arm a JSON + number lands in no longer arises. +- **A finite inexact cell's bytes change**, from a number to a `#i` string. That is the + cost, and it is the point. + +**Version skew is accepted, and the handshake version is improvise-szi8** (decided, +question 9). An old peer reading any new string refuses the frame, which is loud. A new +peer reading an old peer's bare JSON number reads it as **inexact**, which is silent, +and then that old peer's `0.1` compares unequal to a typed `0.1`. This matters only +when a deployed page talks to a `ws-server` of another version, and improvise-szi8 closes +it at the handshake. + +`typed()` stays the one home of text → value (improvise-j7v6). It parses by clause 3's +number language, with the type **inferred**, so `3/4` typed into a cell stays text +(decided, question 12). A browser composes exact values with no second rule. + +### 12. The seams + +**gp-tree is out of scope and needs no change.** A number crosses into a fit **as text** +(`dataset.rs:50-56`), which `preprocess` parses into f32. An exact `12.50` crosses as +`"12.50"`, and it parses to the same f32. The emitted thresholds are decimal text +(`improvise-gp/src/lib.rs:285-298`), so they become exact literals, and exact comparison +against exact data routes the same way as the f64 comparison does today, except within +f32 rounding of a threshold. That boundary difference already exists between f32 and +f64. `scoring_known_answers.rs`'s bit patterns (gotcha 37c) are untouched, because +gp-tree never sees a `Num`. + +**Import.** Two sources, two rules. This follows Codex's P2 on `962d7f90`, which +aligns the contract with PR #321. + +- **CSV.** PR #321 keeps every non-empty CSV field as its verbatim text + (`Value::String`, carried as `Scalars::Text`). It does so to preserve numeric-looking + category identifiers. So a CSV measure is parsed **from that text** by clause 3's + number language. #321's `number_in` gate *is* that language's gate, finite values only + (revision 6), so the analyzer and the parse agree on every string: `1.5e3` is the + exact `1500`, and `1_000`, `nan`, `inf` and `1e999` are text. It keeps its significant + digits and its **scale**, so a CSV `12.50` opens in the editor as `12.50`. It falls + back to the nearest f64 only when it fits neither exact arm, never because it failed + to parse. + - **Inferred and forced columns differ on `n/d`** (decided, question 12). While the + analyzer is inferring a column's kind, `3/4` is text, so a column of dates stays a + column of dates. Once the user **forces** a column to a measure (a wizard override, + or `--measure`), `3/4` in it reads as the exact fraction `3/4`. + - Revision 3's description of `parse_csv_field` (`csv_parser.rs:167-186`: an `i64`, + else an `f64` `Number`) is today's code at `b639266b`, and #321 replaces it. The + ADR does not reintroduce that normalization. + - If this ADR lands before #321, the same contract holds by reading the field's text + before `parse_csv_field` turns it into a number. +- **JSON.** A JSON number is read from its **source lexeme** through `serde_json`'s + `raw_value`, and never through the f64 `as_f64` (`wizard.rs:425`) reads today + (decided, question 6; clause 3). So a JSON `12.50` keeps its scale, and a 19-digit + integer stays exact. Revision 4's shortest-spelling rule for JSON is withdrawn. + - **This restructures the JSON half of the import pipeline** (revision 6, from the + review of `33833f28`). `ImportPipeline` carries `serde_json::Value` (ten uses in + `wizard.rs`, and more in `analyzer.rs`), and a `Value`'s numbers are already f64. + `raw_value` hands over a lexeme only to a reader that deserializes into `RawValue` + through a hand-written visitor. It does not work through a `Value`, or through any + buffered path such as an untagged enum. So the JSON reader becomes a hand-written + deserializer that keeps each number as its source text, and the pipeline carries + that text to the number language, as #321 already does for CSV. + - That is the restructuring revision 3 cited as a reason not to carry the text. It is + paid here because the user ruled out the round trip through f64 (question 6), and + this is the only way to honour that for JSON. It is migration step 5's largest + part. + +The collision "sum" policy (`wizard.rs:307`) then sums exactly, by clause 8. + +**Format.** `FormatPlan` is unaffected: it chooses a format, not a value. The rounding +itself moves to `numtower` (clause 14), and it covers all three arms: +- a decimal or fraction is rounded exactly; +- an **inexact** value is rounded over its **shortest round-trip spelling**, as a + decimal. That is the fix improvise-felo.9 names, and it makes a typed-then-computed + `1.005` behave. + +The rounding mode is a parameter, so improvise's half-away-from-zero policy (gotcha 8) +stays improvise's and does not become the crate's. + +### 13. The crate: `rust_decimal` + +Chosen on the table above. Three reasons: + +- **Size and speed.** It is `Copy` and 16 bytes, with no heap. So a `CellValue` does + not start allocating for every number. It is about 2.5× faster than `bigdecimal` on + SUM, and about 40× faster on the formula workload. +- **Bundle.** It is the smallest marginal addition to the wasm bundle. +- **Bound.** It is bounded, and the bound is the right size for this domain. 28 + significant digits holds any money amount and any count. `bigdecimal`'s unbounded + add costs a heap allocation for a precision no pivot table uses. Its division still + rounds, so it does not buy exact division either. + +Its serde support is not a reason. `numtower` has no serde (clause 14), and the wire +spells a value in its own encoding (clause 11). Revision 5 still listed it, and that +contradicted both. + +The cost is three duties the crate does not do for us, each named where it applies: +detecting rounding at the bound (clause 4), parsing (clause 3 parses the digits and +exponent itself, because `from_str` rounds silently and `from_str_exact` rejects +`1.5e3`), and converting to f64 by a correctly rounded path rather +than `to_f64` (clause 4). A fourth is avoided rather than done: its serde deserializer +accepts JSON numbers, so the wire does not use it. The wire parses its own string +encoding through `numtower` (clause 11). +`num-rational`'s unbounded `BigRational` is rejected as the default on measurement +(above); its bounded `Ratio` is the fraction arm (question 2). `bigdecimal` is the fallback if the +bound ever matters. Other crates (`fastnum`, `dashu`) were not evaluated. + +### 14. The numeric tower is its own crate + +At the user's direction (revision 4): "I think ADR 005 should implement the math +operations in a separate crate". The numbers are a **problem-agnostic workspace crate**, +like `gp-tree` and ADR 009's `zset`, and it knows nothing about improvise. + +**Name.** **`numtower`** (decided, question 14). It names the idea, Scheme's numeric +tower, rather than this program. + +**What it holds:** +- `Num`, with its `Exact(Decimal)`, `Fraction` (question 2, decided (d)) and `Inexact(f64)` + arms; +- the arithmetic and contagion (clause 4); +- the rounding and overflow detection: the scale test, and the `2^p·5^q` termination + test (clauses 4 and 5); +- the correctly rounded conversion to f64 (clause 4); +- exact comparison, the `MIN`/`MAX` tie-break included (clauses 6 and 8); +- the exact-sum accumulator: the 320-bit decimal partition, plus the per-denominator + groups under (d) (clause 8); +- rounding any `Num` to *n* places for display, with the rounding mode a parameter: + exact values exactly, and inexact values over their shortest round-trip spelling + (clause 12, "Format", and improvise-felo.9); +- the **f64 → decimal shortest-spelling** conversion, which the display rounding and the + `#i` spelling on the wire and in the file all use; +- parsing that never rounds silently: digits and exponent to an exact value, then the + decimal, fraction and f64 arms in that order (clause 3); +- the canonical text of a value: the decimal with its scale, `n/d`, `#i…`, and the + non-finite tokens; +- clause 3's **number language**, both its gate and its parse, including the forced + `n/d` arm; +- the wide integers: a fixed-width `[u64; 5]` for the 320-bit accumulator, and 256-bit + helpers for the division reduction and the cross-multiply. These are hand-rolled, + because they sit on hot paths and must not allocate. + +**What it depends on** (revision 5 corrects revision 4, answering B3 of the review): +- `rust_decimal`, with default features off except `std`; +- `num-rational`, with `default-features = false`, for `Ratio`; +- **`num-bigint`, always.** Revision 4 said "only if question 13 is answered + 'unbounded intermediates'", and that was false by the ADR's own text. Three paths need + a bigint whatever question 13 says: + - display beyond 38 places; + - converting a grouped sum's 256-bit numerator over an `i128` denominator; + - question 13 (b)'s widened intermediates and the grouped read. + + (`num-rational`'s default features would pull it in anyway.) +- **not** `serde`: the wire is improvise's concern (below). + +**Bundle cost under (d).** `rust_decimal` probed at +15.4 KB gzipped marginal. +`num-rational` with `num-bigint` probed at +20.8 KB gzipped on its own. The two +together are **unprobed**, and are estimated at up to about +36 KB gzipped. The user's +"yes" to question 8 was given against the first figure alone. The decided-questions +section flags that, and migration step 1 measures the real bundle. + +All of these build for `wasm32-unknown-unknown`: each was built there in the size probe. +The crate is added to the `wasm` CI job's list, and checked with +`cargo tree --edges normal`. + +**The dependency direction is one way.** `improvise-core` depends on `numtower`, and +`numtower` depends on no improvise crate. It is the `gp-tree`/`improvise-gp` shape: the +general thing below, the seam above. + +**Improvise's own concerns stay in improvise crates and call into it:** + +| concern | stays in | calls | +|---|---|---| +| `CellValue::Number(Num)`, `as_f64`, `edit_text`, the record fingerprint's hashing of a `Num`'s representation (clause 9) | `improvise-core` (fingerprint: `improvise-command`) | `Num`'s accessors for exactness, mantissa, scale and fraction parts | +| the formula evaluator, `FLOAT_EQ_EPSILON`'s deletion, aggregates' use of the accumulator | `improvise-core` | `Num`'s arithmetic, comparison and accumulator | +| formula literals: `Expr::Number(Num)` | `improvise-formula` | `numtower`'s parse | +| the `.improv` grammar arms (`#i`, `n/d`), the writer, the version literal | `improvise-io` | `numtower`'s parse and canonical text; the `#i` prefix and the grammar are improvise's | +| serde and the wire encoding (clause 11) | `improvise-core`, replacing `CellValue`'s untagged adaptor | the canonical text of every arm, and the number language to read it back | +| display formats (`,.2f`), comma grouping | `improvise-core`'s `format.rs` | `numtower`'s round-to-*n*-places; the grouping stays in improvise | +| `typed()` | `improvise-protocol` | `numtower`'s number language, with the type inferred (question 12: `n/d` stays text) | + +**Where the tests live.** Anything about numbers lives in `numtower`'s own tests, where +no improvise type can make them pass by accident: +- the property tests of migration step 1 (no rounded `Exact`; total contagion; exact + comparison is transitive; `n/d` normalization); +- the **differential tests** against an exact `BigRational` oracle: every operation's + result, the f64 conversion, and the accumulator's read against a direct exact sum; +- the order-independence and retraction tests for the accumulator, seeded with the + review's `[0.30000000000000004, 1e12, −1e12]`. + +Improvise's crates keep the tests about improvise: +- `improvise-core`: the evaluator, the improvise-0bf zero predicate, and the scale-aware + fingerprint; +- `improvise-io`: the grammar round trip with exactness preserved; +- the protocol: which wire arm a JSON number or string lands in. + +The benchmark example lives in `numtower/examples/`, where its dependencies are already +normal ones. That is what lets it be committed at all (appendix). + +**ADR 009's `ExactSum`** is an f64 superaccumulator, in the proposed `zset` crate. +Clause 8 needs it for the inexact partition, and a mixed sum's read combines it with the +exact partition. **`ExactSum` moves into `numtower`** (decided, question 15 (a)), the +one home for exact summation. `zset` stays independent of numbers: it +keeps its abelian-group trait, and ADR 009's seam crate (`improvise-incremental`, its +question 2) wraps `numtower`'s accumulators in a newtype that implements that trait. The +orphan rule forbids implementing `zset`'s trait for `numtower`'s type anywhere else, and +the newtype costs nothing. The alternative, `numtower` depending on `zset` for +`ExactSum`, would make the number crate depend on a stream-processing crate. + +**`numtower` can land holding `ExactSum` alone, ahead of this ADR and independent of +it.** ADR 009's question 3 recommends its exact accumulator "without waiting for ADR +005". Under question 15 (a) that accumulator lives in a crate this ADR's migration +creates, which would make ADR 009 wait after all. It does not have to. Nothing in +`ExactSum` depends on `Num`, so `numtower` may first land with `ExactSum` and its tests +only, before this ADR is accepted, and `Num` joins it in migration step 1. ADR 009 then +depends on `numtower` and not on ADR 005. + +## Option (d) for question 2: bounded exact fractions + +The user asked for this option (revision 3): "I think (d): exact fractions with a size +limit. should be added." It is specified here and measured the same way as the others: +same machine (Apple M4 Max, 64 GB, rustc 1.98.1, release), same typed two-place data +(0.01–999,999.99, SplitMix64), and `num-rational` 0.4.2's checked operations. Those +reduce by the gcd on every operation. + +### The number + +An exact non-decimal value is a **reduced fraction `n/d` with `n` and `d` each an +`i128`** and `d > 0`. The alternative bound, `i64`, was measured and is too small: + +| what stops being exact | `i64` | `i128` | `rust_decimal` | +|---|---|---|---| +| running sum of 18k or 90k margins `(Rev − Cost) / Rev`, first overflowing term (1-based: the term whose addition overflows), 8 seeds | 3rd–4th | **5th–6th** | not exact (a non-terminating quotient) | +| `1.05^k` (5% compound growth) | k = 15 | **k = 29** | k = 15 | + +`i128` doubles every horizon, for 2–3× the cost of `i64` (below). A value is 32 bytes. + +**Decimals are a subset, and they keep their own arm.** A decimal is a fraction whose +denominator is `2^p · 5^q`, so (d) could hold everything as fractions. It should not. A +fraction has no scale, so `12.50` would need a scale field beside it. And every +operation would pay the gcd, which the measurements put at 30–300× a decimal add. So +the recommended shape is a **hybrid**: + +```rust +pub enum Num { + Exact(Decimal), // typed values, money, `12.50`: scale as today + Fraction(Box>), // exact, not a finite decimal: `10/3` + Inexact(f64), +} +``` + +- A `Fraction` is only ever produced where option (a) would have produced `Inexact`: + - a non-terminating quotient; + - a decimal operation that exceeds 28 digits but whose reduced fraction fits (`1.05^k` + for 15 ≤ k ≤ 28). +- A `Fraction` whose denominator turns out to be `2^p · 5^q` and fits is normalized back + to `Exact(Decimal)`, so each value has one representation. +- **Scale lives only on the decimal arm.** Typed values are decimals, so a fraction was + never typed and has no spelling to preserve. That answers "how is scale carried" + without a scale field. +- **The fraction arm is boxed.** That keeps `Num` at 24 bytes, the same as option (a); + measured, an inline `Ratio` makes it 48. The heap allocation falls only on + fraction values, which are formula results. Per `AGENTS.md`, the `Box` handling lives + in `Num`'s constructors, apart from the arithmetic. + +### Operations + +- **Division stays exact.** `10/3` is `Fraction(10/3)`, and `10/3 × 3` is exactly `10`, + which normalizes back to `Exact(10)`. Both measured. +- **Overflow demotes at the store** (question 13, decided (b); revision 7 replaced the + per-operation wording, which Copilot found contradicted it). The fast path uses + checked `i128` arithmetic after the gcd reduction (`num-rational`'s + `checked_add`/`checked_mul`/`checked_div` do exactly this). A `None` **may** mean the + reduced result does not fit, and **may** mean only that an intermediate overflowed + (`checked_add` goes through the lcm of the denominators) while the reduced result + fits. So a `None` widens that step to a bigint rational and the **stored** result is + tested against the `i128` bound: exact iff it fits, else the correctly rounded f64 of + the exact fraction, which is `Inexact`. It is deterministic. Past question 13's + ceiling, the intermediate demotes (the first of the two stated exceptions). + + Revision 3 cited `Ratio::to_f64` agreeing with `BigRational`'s conversion. That + was **circular**: both share `num-rational`'s one `ratio_to_f64` routine. The review + ran an independent check instead. It took 200,000 random reduced `Ratio`s, with + numerator and denominator widths from 20 to 126 bits and both signs. For each, it + asked whether the exact `BigRational` lies inside the round-to-nearest-even interval + of the returned f64. There were **0 failures**. That is evidence, not proof, and + migration step 1's property test uses this interval form. +- **Contagion is unchanged.** Exact ∘ fraction is a fraction (or a decimal after + normalization, or f64 on overflow). Anything with an inexact operand is inexact. +- **`POW`** with an integer exponent is exact while it fits. That is why `1.05^k` + reaches k = 28. +- **Comparison** is exact. Two fractions are compared by cross-multiplying in 256 bits + (an `i128` times an `i128` fits). A fraction against an f64 follows clause 6's exact + rule. +- **Display** rounds the exact fraction to the view's places, half away from zero, + exactly: `n · 10^places / d`, then rounded. So `10/3` at two places shows `3.33`, and + `2/3` shows `0.67`. In 256 bits that holds up to about 38 places, because `places` is + a `u8` (`parse_number_format`) and `n · 10^places` passes 2²⁵⁵ above that. Beyond 38 + places the computation takes the bigint path. That is display only, and rare. + +### Sums: the one real design choice + +Clause 8's accumulator holds decimals only: an integer at the fixed scale 28. A +fraction whose denominator is not `2^p · 5^q` has no finite value at any fixed scale. +There are three ways to sum one: + +1. **Demote each fraction term to f64** into ADR 009's accumulator. Simple, and + independent of order. But `SUM(v / 3)` is then inexact, even though `SUM(v) / 3` + is exact. +2. **Group by denominator.** Keep one wide integer numerator per distinct reduced + denominator; decimals are the one group clause 8 already has. Adding or retracting a + term touches only its own group. So the state is independent of order, and + retraction is exact. + + How it is read is the recommendation below: exactly, in a bigint rational, when + there are at most K non-decimal groups (question 13 (b)); otherwise from a maintained + f64 accumulator of the groups' values, with the decimal partition converted once. + Both paths are deterministic and independent of the order the terms arrived in. + (Revision 5 still described a checked-`i128` read here, with a fallback that summed + each group's f64 at read. That contradicted the maintained accumulator below.) + + The cost is memory. It is O(distinct denominators) per summed group, bounded by + the cells, the same order as ADR 009's multisets. +3. **Demote the whole sum on mixed denominators.** Rejected. Whether two denominators + ever met depends on nothing a user can see. + +**Two rules the grouping needs.** +- **A group whose numerator returns to 0 is dropped.** A retracted denominator then + cannot affect the read, and "the current groups" is a function of state alone. +- **A `Fraction` whose denominator is `2^p · 5^q` joins the decimal partition when its + exact scale `k = max(p, q)` is at most 28.** An example is one born from a decimal + overflow with denominator `10^17`. At scale 28 its value is an exact integer, which + the 320-bit accumulator holds. With `k > 28` (for example `1/2^40`) it is a group of + its own, keyed by its denominator like any other. + +**Recommendation: (2), with a bound on the exact read** (revision 4, answering review +BLOCK D2). Revision 3 said (2) "costs nothing where it cannot help", and that was +false in exactly that case. The sum of margins has one denominator per record: 17,996 to +17,999 distinct at 18k, and 89,923 to 89,956 at 90k. So its state is O(records), and +so is every read. Under ADR 009 a maintained `SUM` whose read is O(records) is no longer +incremental: every one-cell edit would re-read 90k groups, where (a) holds a 40-byte +integer and reads in O(1). The review proposed a mitigation, and it is adopted: + +- **Beside the non-decimal groups, keep one ADR 009 f64 accumulator of each such + group's correctly rounded value.** Updating a group retracts its old f64 and inserts + its new one, so it stays order-free and retractable. +- **The decimal partition is exempt** (revision 5, B2 of the review). Revision 4's + wording put every group, the decimal partition included, into the parallel + accumulator, which would have charged an ordinary decimal `SUM` about 1 µs per edit + where (a) pays a 40-byte integer add. The decimal partition stays in its 320-bit + accumulator alone, and is converted to f64 **once, at an f64 read** (only when there + are more than K non-decimal groups). So a pure-decimal `SUM` pays nothing for (d). +- **Read exactly only when the current number of non-decimal groups is at most + K = 8** (decided, question 16; revision 8). Otherwise read the f64 accumulator, and the result is inexact. The count is + a function of the current state, not of history, so the choice is independent of + order. + +**What that costs.** +- **Update of a non-decimal group** (an estimate): O(log D) for the group map, plus one + correctly rounded conversion of the group's new value. That conversion is a 256-bit + numerator over an `i128` denominator, estimated at about 1 µs through a bigint + rational. Plus two accumulator updates at 2.0–3.6 ns (ADR 009's figures). An update + of the decimal partition is one 320-bit add. +- **Exact read (measured in revision 6).** At most K combinations of groups that hold + 256-bit numerators, in a **bigint rational** (question 13 (b)). The worst case is K + distinct denominators near `i128`'s size, where the lcm grows about 127 bits per + group. Measured with 250-bit numerators and random 126-bit denominators, 200 reads + each (appendix): + + | K | lcm bits | `BigRational` add per group | lcm, then one reduce | + |---|---|---|---| + | 2 | 252 | 16 µs | 7 µs | + | 8 | 988 | 127 µs | 52 µs | + | 16 | 1,957 | 448 µs | 174 µs | + | 32 | 3,924 | 1.9 ms | 0.67 ms | + + - Revision 5 estimated **16–34 µs** at K = 32 from the SUM benchmark's rate. Those + operands had small typed denominators, so the estimate was about **50× too low** at + this size. The cost grows roughly as K², because each step works on an lcm that + grows linearly. + - **Combining over the lcm and reducing once** is 2.3–2.8× faster than reducing after + every add, and it gives the same value (the benchmark asserts it). `numtower`'s read + should do that. + - Denominators a person produces (`/ 3`, `/ 12`, a percentage) are small, and the read + is then far cheaper. The table is the bound, not the common case. The value of K is + 8, decided in question 16. +- **f64 read:** ADR 009's 0.6–1.1 µs. +- **Memory is still O(distinct denominators)**, because the map must survive + retraction to know when the count falls back under K. For a 90k-record margin sum + that is about 90k × (16-byte key + 32-byte numerator + map overhead): an estimate + of 6–8 MB for that one summed group, the same order as ADR 009's per-cell multisets. + Not measured. + +This keeps (d)'s wins, because `SUM(v / 3)`, averages and percentages have one or a +handful of denominators, and it bounds the rest. The sum of margins reads the f64 +accumulator and is inexact, as under (a). Its exact combination would overflow by the +6th term anyway (below). The 32 of revisions 5 to 7 was a judgement and was not tuned. The read's measured cost +above is why question 16 recommended a smaller one (K = 8). The user first chose 32 +and then 8, so **52–127 µs is the accepted worst case**. It is paid per read, and under +ADR 009 per edit, by a maintained `SUM` that holds between 2 and 8 distinct non-decimal +denominators near `i128` size; a sum with more groups reads the f64 accumulator, and one +with few or small denominators is far cheaper (the table is the bound). + +### The file and the wire + +- **`.improv`.** A fraction is written `n/d`, with a sign on `n`, for example `-10/3`. + The `number` rule does not accept `/`, and an old reader would take `10/3` as **text** + (`bare_value`), silently. So it needs a grammar arm ordered before `bare_value`, and + the new version literal. That is the same bump as `#i` (question 5), so (d) adds no + second bump. +- **The wire.** A fraction is a **string** `"10/3"` in the exact arm of clause 11's + adaptor. An old peer tries it as a non-finite token and refuses the frame, loudly. +- **Typing `3/4`** stays text. That is question 12. + +### Measured + +**Cost per operation**, 18,000 operand pairs, ns, best of 20, the range over two runs: + +| | add | mul | div | +|---|---|---|---| +| `rust_decimal` (checked) | 0.9–1.3 | 1.5–2.2 | 40–55 | +| `Ratio`, two-place operands | 55–68 | 144–179 | 78–113 | +| `Ratio`, two-place operands | 123–178 | 324–472 | 198–288 | +| `Ratio`, margin operands (unrelated denominators) | 313–457 | 407–652 | 223–325 | + +f64 is not in the table, because this harness's f64 loop was optimized away. The SUM +figures below give f64's per-add cost. `num-rational`'s gcd over `i128` is generic. A +hand-written one with a 64-bit fast path would likely be several times faster, but +that was **not measured**. + +**SUM over typed two-place values, as fractions**, ns per element, best of 20: + +| cells | f64 | `rust_decimal` | `Ratio` | `Ratio` | +|---|---|---|---|---| +| 18,000 | 0.8–1.2 | 4.6–6.4 | 65–76 | 155–226 | +| 90,000 | 0.8–1.2 | 4.5–6.5 | 69–87 | 163–239 | + +Both bounds held the whole sum with no overflow. The sum was the same in both orders +and equal to the decimal sum. But `Ratio` is 25–50× slower than `rust_decimal`: +14–22 ms for a 90k SUM, against 0.4–0.6 ms. **That is why the hybrid keeps decimals in +their own arm.** Under it, this SUM costs what it costs under (a). + +**How often realistic formulas overflow `i128`**, 8 seeds, 18k and 90k records: + +| formula | overflows | +|---|---| +| `(Rev − Cost) / Rev` per record | **never** (864,000 records: 8 seeds × (18k + 90k), both bounds) | +| running `SUM` of those margins | **at the 5th–6th term** in every seed, counting 1-based (`i64`: 3rd–4th) | +| `v / SUM(v) × 100` per cell (a percentage of the total) | **never** (0 of 144,000 and 0 of 720,000, both bounds) | +| `SUM(v) / N` (the average) | never | +| `SUM(v / 3)` against `SUM(v) / 3` | never; **equal exactly** | +| `1.05^k` | at k = 29 (`i64` and `rust_decimal`: k = 15) | + +### Where (d) beats (a), and where it does not + +**(d) is better:** +- a non-terminating quotient stays exact: `10/3 × 3 = 10` (measured), and so + `Margin × Revenue = Profit` holds exactly; +- the average over any count; +- `SUM(v / 3) = SUM(v) / 3`; +- exact comparison of ratios; +- compound growth exact to k = 28 rather than 14. + +**(d) is no better:** +- the sum of ratios, which overflows by the 6th term and ends inexact as under (a); +- anything already exact under (a), which is nearly all typed data. + +**(d) costs:** +- a third numeric arm; +- a grouped sum accumulator; +- a grammar arm and the wire spelling `n/d`; +- question 12; +- 120–650 ns per fraction operation, paid only where (a) would have demoted; +- **`num-rational` and `num-bigint` in the browser bundle**: +20.8 KB gzipped on their + own, and up to about +36 KB together with `rust_decimal` (unprobed together; clause + 14). + +**Scale after the fraction arm** (revision 5, the review's point on question 13). A +result that passed through the fraction arm and normalized back to a decimal carries its +**minimal** scale. A result that stayed decimal keeps clause 4's scale. So `a × (b + c)` +and `a × b + a × c` can be `=` and still differ in scale. The review's example (reasoned, +not run) is `a = 123456789012345.6780`, `b = 12345678901234.5678` and +`c = −b + 0.0001`. The first stays decimal at scale 8. In the second, `a × b` overflows +96 bits, so the sum comes back through the fraction arm at scale 7. The editor seed and +the scale-aware fingerprint (clause 9) then differ between the two spellings. This is +true under (d) whatever question 13 says. It is stated rather than fixed, because scale +is presentation, and a computed value was never typed. + +### Folded in: what changed in the clauses when the answer became (d) + +Revision 3 listed these changes as conditional. Revision 5 makes them normative, because +the user answered question 2 "(d)": + +- **Clause 1:** three levels: exact decimal, exact bounded fraction, inexact. +- **Clause 2:** `Num` gains the boxed `Fraction` arm. +- **Clause 3:** entry tries the fraction arm before f64, as arithmetic does + (revision 6). +- **Clause 4:** a decimal operation that exceeds 28 digits tries the fraction arm + before f64. +- **Clause 5:** a non-terminating quotient is a `Fraction`. `AVG` is exact. +- **Clause 6:** fraction comparison is by cross-multiplication. +- **Clause 8:** sums group by denominator, as above. +- **Clause 10:** the `n/d` spelling. +- **Clause 11:** the wire spells a fraction `"n/d"`. +- **Question 12** decided typed input: `n/d` is text where the type is inferred, and a + fraction where it is forced (clause 3). + +## Alternatives rejected + +- **Exact rationals as the default (Scheme proper).** Measured above: an 18,000-record + average of ratios does not finish. It would be the right answer for a symbolic + calculator and the wrong one for a pivot table. +- **A rounded decimal that still calls itself exact** (SQL `NUMERIC`: `1/3` becomes + `0.3333…` at 28 places, marked exact). It keeps "decimal math" true everywhere, but + then "exact" means "exact except after a division". Equality, ADR 009's group law and + every future reasoning about exactness would carry that exception silently. It is + question 2's option (b), for the user to overrule. +- **Keep f64 and adopt only ADR 009's accumulator.** It fixes large same-sign sums. It + leaves per-cell arithmetic, small sums (16% of positive 3-term sums), signed sums + (about 40% at every length), display of typed values and the + epsilon exactly as they are (Context). +- **Three levels (exact decimal, inexact decimal, f64).** An "inexact decimal" arm for + rounded quotients would add a third arm to every numeric match. It would buy only a + 28-digit rather than a 17-digit approximation of a number that is not a decimal + anyway. +- **A new `CellValue` arm beside `Number(f64)`.** Rejected in clause 2. + +## Migration + +In order. Each step lands on its own, with the suite green. + +1. **The `numtower` crate, first and alone** (clause 14). It holds `Num` and its + arithmetic, with the contagion table (clause 4) and the scale-based rounding + detection. It holds the correctly rounded conversion to f64, the division and `POW` + rules (clause 5), exact comparison (clause 6), the exact-sum accumulator (clause 8), + rounding to places, and clause 3's number language. It also holds the fraction arm and + its grouped sums (question 2 is decided (d)), and the widening rule with its + ceiling (question 13 is decided (b); the ceiling's value, 8,192 bits, is the + steward's proposal and not the user's decision). Nothing in improvise depends on it yet, so it lands + with its own tests and changes no behaviour. It may be preceded by a `numtower` + holding only `ExactSum`, for ADR 009 (clause 14). + + Its property tests: + - an `Exact` result is never a rounded one (inject a silently-rounding add and watch + it fail); + - contagion is total; + - the conversion agrees with `str::parse::` of the decimal text. Inject `to_f64` + and watch it fail on the review's `63294.60…` case. The random generator spans + **96-bit mantissas and scales 0–28**: with 15 or fewer digits the broken `to_f64` + agreed 200,000 times out of 200,000, so a narrower range is vacuous; + - a fraction's conversion lies inside the round-to-nearest-even interval of the + exact value (the review's independent form); + - exact mixed comparison is transitive; + - the accumulator is independent of order (clause 8's seed case); + - division's direct fit test: `1/2^40` and `1/5^30` are fractions, `1e-28 / 0.5` is + the exact decimal `2e-28`, and multiplying back is not used (clause 5); + - the number language's gate and parse agree on every generated lexeme. The seed + cases are: + - `1.5e3` (exact `1500`), `1_000` (text), `-0.0` (exact `0.0`), and `3/4` (text + when inferred, a fraction when forced); + - `1e999` and `-1e999` (text: a finite lexeme never reads as infinity), `nan`, + `inf` and `Infinity` (text in the gate), and `1e-999` (an inexact zero); + - `1e-30` (the exact fraction `1/10³⁰`), `0.12345678901234567890123456789` (an + exact fraction, not a rounded decimal), an integer between `Decimal`'s maximum + and 2¹²⁷ (an exact `n/1`), and `1e-40` (inexact: it fits neither arm); + - a typed value and the same value computed are equally exact: `1e-30` parsed + equals `1e-15 * 1e-15`, arm and value both; + - the grouped read combines over the lcm and reduces once, and agrees with a + `BigRational` sum of the groups. + + The differential tests run against a `BigRational` oracle. The benchmark goes in + `numtower/examples/`. Add the crate to the `wasm` CI job. Measure one whole recompute + at 18,000 and 90,000 cells against today once step 3 lands, and name the machine. + The arithmetic figures above are a microbenchmark, not the evaluator. +2. **`improvise-core` depends on `numtower`, and `CellValue::Number(Num)`** replaces the + f64 across the 672 sites. This is mechanical, and `as_f64` + keeps the seams compiling. `hash_value` hashes the representation, scale included + (clause 9). +3. **Both evaluators** move to `Num`. `FLOAT_EQ_EPSILON` is deleted. The improvise-0bf + tests are rewritten to assert one zero predicate (clause 6). `EQUAL(a, b, epsilon)` + lands here: its grammar and AST in `improvise-formula`, its evaluation in both + evaluators, and `EQUAL` in `quote_name`'s keyword guard (clause 6). +4. **Aggregates** partition by exactness and use `numtower`'s accumulator (clause 8). + Its own order-independence test is in `numtower` (step 1). Here the test is that + `apply_agg` and `eval_agg` route through it: seed the review's + `[0.30000000000000004, 1e12, −1e12]` through the evaluator. `Decimal`'s own `+` must + fail it. +5. **Entry points:** `typed()`, the import pipeline's measure read, formula literals + (clauses 3 and 12). The largest part is the JSON reader: a hand-written deserializer + that keeps each number's source text through `raw_value`, because a + `serde_json::Value` has already made it an f64 (clause 12). The import tests assert: + - a CSV `12.50` and a JSON `12.50` both keep their scale, because both are parsed + from their lexeme; + - `1.5e3` imports as the exact `1500`, and `1_000` imports as text; + - `nan`, `inf`, `1e999` and `-1e999` import as text, so #321's + `a_nonfinite_numeric_token_stays_text_rather_than_becoming_zero` stays green + unchanged; + - `typed()` reads `inf`, `-inf` and `nan` as clause 7's tokens, and `Infinity`, + `NaN` and `1e999` as text (the stated behaviour change); + - `3/4` is text in an inferred column and the fraction `3/4` in a column forced to + a measure. +6. **Display and edit text** (clause 9). The `1.005` case is the regression test, and + it fails at HEAD today. If improvise-felo.9 lands first, this step reuses its test. +7. **`.improv`:** read exact, write with scale, `n/d` for a fraction, `#i` for inexact + finite, and ADR 007's version literal (clause 10). +8. **The wire:** the string encoding replaces the untagged adaptor (clause 11), with the + retired-spelling discipline of gotcha 42: assert the error kind, not merely that an + error occurred. The version handshake is improvise-szi8, separately. + +## Open questions + +The user answered questions 1–9, 11 and 12 inline on `e4acf643`, and 10, 13, 14 and +15 inline on `33833f28`, and question 16 inline on `b232c5a9`. Each is marked +**decided** below, and quoted in full under "What the user decided". No question is +open. Nothing is renumbered. + +1. **The tower's shape.** **Decided (the user): three levels**: exact decimal, exact + bounded fraction, inexact f64 (clause 1). The original recommendation, two levels, + was written before option (d) existed. + +2. **Non-terminating exact division.** (a) inexact f64; (b) a 28-digit decimal that + stays marked exact (SQL `NUMERIC`); (c) an exact rational (Scheme proper); (d) an + exact fraction with a size limit. **Decided (the user): (d)**, in the hybrid form + (the option (d) section). + +3. **Tolerance on inexact equality.** **Decided (the user): no tolerance on `=`.** A + formula function `EQUAL(a, b, epsilon)` is added instead (clause 6). It replaces this + question's earlier "an `APPROX(a, b, tol)` if it is ever wanted". + +4. **The `.improv` spelling of an inexact finite number.** **Decided (the user): `#i`** + (clause 10). + +5. **The version literal.** **Decided (the user): share ADR 007 clause 10's literal** + (clause 10). + +6. **JSON import precision, and the wire.** **Decided (the user): no round trip through + f64.** + - The wire spells every number as a string in improvise's own encoding (clause 11). + - JSON import reads each number's source lexeme through `serde_json`'s `raw_value` + (clauses 3 and 12). + - `arbitrary_precision` stays rejected, because it breaks serde's buffered + deserialization of numbers under the workspace's internally tagged enum and its + `flatten`. + - The earlier recommendation, "shortest spelling", is withdrawn. + +7. **Mixed-exactness comparison.** **Decided (the user): exactly**, plus + `EQUAL(a, b, epsilon)` (clause 6). + +8. **The dependency.** **Decided (the user): yes.** That answer was given against + `rust_decimal`'s +15.4 KB gzipped. Under (d), `num-rational` and `num-bigint` add + about +20.8 KB on their own, up to about +36 KB together (unprobed). If that changes + the answer, this question reopens (clause 14). + +9. **Version skew on the wire.** **Decided (the user): accept it**, and the handshake + version is filed as **improvise-szi8** (clause 11). + +10. **The exact-sum accumulator's width.** **Decided (the user): (a)**, with "320bits + is fine" for the decimal partition. The value of K is split out as question 16, + because the read's cost was measured after the answer. The question as it was put + (revised in revision 5, after the user asked: + "Don't we need a much bigger accumulator (2000ish bits) to handle exact accumulation + generically?"). Clause 8's "How wide, generically" answers with numbers: + - **decimals need 320 bits**; + - **f64 values need about 2,100 bits**, which is ADR 009's `ExactSum`; + - **fractions need no fixed width at all**, because their common denominator grows + by up to 127 bits per distinct `i128` denominator. 2,032 bits holds 16. + + Options: + - **(a)** a 320-bit decimal partition, plus a 256-bit numerator per fraction group, + plus a read that combines at most K groups in a bigint rational and then applies + the stored bound; + - **(b)** one fixed 2,048-bit exact register, which holds K = 16 arbitrary + denominators, with no bigint on the read; + - **(c)** `Decimal`'s own `+`, accepting demotion that depends on order. + + *Recommendation:* (a), with K = 32 as proposed here. Question 16 later set K = 8. + - A fixed register cannot be generic for fractions at any width. + - 2,048 bits pays 256 bytes per summed group for a combination that rarely fits the + stored bound anyway. + - The bigint read is paid only when a sum holds 2 to K distinct fraction + denominators: about 16–34 µs at K = 32, an estimate. (Revision 6 measured it: + 0.67–1.9 ms at K = 32 in the worst case, 52–127 µs at K = 8, and the user chose + K = 8. See question 16.) + - (c) breaks order-independence (revision 2). + +11. **The `MIN`/`MAX` tie-break.** **Decided (the user): prefer the exact value** + (clause 8). + +12. **Typing a fraction.** **Decided (the user): `n/d` is text where the type is + inferred and a fraction where it is forced** (clause 3). So `typed()` and the import + analyzer keep `3/4` as text, and a column forced to a measure reads it as a + fraction. An editor opened on a fraction shows `10/3`, and an **untouched** commit + writes nothing, on both front ends. The browser already skips an untouched commit + (`resolve.rs:1421`); the TUI must be held to the same rule (the improvise-umi4 + lesson). + +13. **Where the fraction bound applies.** **Decided (the user): (b)**. Added in + revision 4, and put as follows. Is the `i128` + bound checked (a) after every operation, or (b) only on the **stored** result, with + every intermediate inside one formula evaluation unbounded? Under (b) an + intermediate uses the `i128` fast path and widens to a bigint rational only when a + step overflows. + *Recommendation:* (b). It was the orchestrator's recommendation in a discussion with + the user. + + **The argument.** Under (a), whether a result is exact depends on how the formula is + written. `a × (b + c)` and `a × b + a × c` are the same number, but an intermediate + can overflow in one spelling and not the other, so one is exact and the other a + demoted f64. Under (b) the rule is one sentence: **a stored value is exact if and + only if it fits.** Algebraically equal formulas give equal values. (Two stated + exceptions: the ceiling below, and a `SUM` of more than K = 8 non-decimal + denominator groups, which reads inexact by question 16 whatever its value.) + + **The rule has two exceptions, and they are stated rather than hedged** (the ceiling + from revision 5, the review of `87ae21ca`; the K cutoff from revision 7, Codex on + `be9de2ee`): + - **The ceiling.** Widening carries a size ceiling so that a pathological + expression cannot grow a bigint without limit. It must be at least + K·127 + 256 bits, so that the grouped read (below) never reaches it: 1,272 bits + at K = 8 (it was 4,320 at K = 32). Revision 5's example of 4,096 bits was below the + K = 32 figure, and a worst-case K = 32 read would have hit it (the review of + `33833f28`). Proposed: **8,192 bits**. That value is the steward's proposal from + revision 6, commit `02bcfcb1` (it first appears after the user's answer "(b)", which was given against + the text's 4,096-bit example), so it is not part of the user's decision, and + approval of the ADR confirms it. It covers K = 8 by 6,920 bits and K = 32 as + well, and costs nothing until a value is that large. A smaller ceiling would now + suffice. Past it, the + intermediate demotes, and by contagion the stored result is inexact **even when + the exact result would fit**. That is spelling-dependent, so distributivity fails + past the ceiling. It is the first exception to "exact iff it fits", and no ledger + reaches it. `POW` checks its result size **before** computing (clause 5), so the + ceiling is never reached by allocating first. + - **The K cutoff, which does override representability.** A `SUM` holding more than + K = 8 non-decimal denominator groups reads the inexact f64 accumulator, so its + stored result is inexact **even when the exact total would fit** the bound. That + is the second exception. It is a policy about the read's cost (question 16), not + about whether the value is representable, and it is the consequence of the user's + K = 8. **Flagged reading (revision 7, restated in revision 8):** the user decided K + on question 16's text, which says "Above it the read is the inexact f64 accumulator", and answered + question 10 (a) with "at most K groups". No "What the user decided" item says in + terms that the cutoff overrides representability. That sentence is the ADR's + reading of those two answers, so approval confirms it. + - **Not an exception: the grouped read at K groups or fewer.** Revision 4 described the grouped sum's + read as checked `i128`, and that can overflow in an lcm step while the reduced + result fits. Under (b) the read is widened like any intermediate (clause 8, "How + wide, generically"), so it is exact iff its result fits. + - **What (b) does not change.** The bound still applies at every *stored* value, so a + calculation split across two formula cells is bounded at the cell in between. + Equal values may still differ in **scale** after passing through the fraction arm + (the option (d) section). Scale is not value. + + **Cost.** This is an estimate, not measured on this path. + - The common case is unchanged: no intermediate overflows `i128`. + - Only a cell whose expression overflows `i128` part-way through pays for bigint + rational steps. A widened step works on operands near 256 bits, not on the SUM + benchmark's small typed values: combining two such fractions measured 7–16 µs + (option (d), "What that costs"). So a widened step costs microseconds, not the + 490–1,060 ns revision 5 cited. + - Such a cell usually demotes at the store anyway. + +14. **The numeric crate's name.** **Decided (the user): `numtower`** ("this is + fine"). Added in revision 4, with the recommendation `numtower`. It is problem-agnostic like `gp-tree` and `zset`, so it should not carry + improvise's name. + +15. **Where ADR 009's `ExactSum` lives.** **Decided (the user): (a)**. Added in + revision 4, and put as follows. + - (a) In `numtower`, beside the exact decimal accumulator, with ADR 009's seam crate + adapting it to `zset`'s group trait through a newtype. + - (b) In `zset`, as ADR 009 proposes, with `numtower` depending on `zset` to + combine a mixed sum. + + *Recommendation:* (a). Exact summation then has one home, a mixed sum's read needs + no second crate, and `zset` stays free of numbers. `numtower` can land holding + `ExactSum` alone, ahead of this ADR, so ADR 009 does not wait for it (clause 14). + This question is shared with ADR 009 (PR #309). + +16. **The value of K.** **Decided (the user): (b), K = 8** ("Ok, let's drop K back to 8", + superseding an earlier "let's do k = 32"; both quoted under "What the user + decided"). Added + in revision 6, and put as follows. K is the most non-decimal groups a + `SUM` reads exactly (question 10). Above it the read is the inexact f64 + accumulator. Revision 5 proposed K = 32 on an estimate of 16–34 µs per read, and + the measured worst case is **0.67–1.9 ms** (option (d), "What that costs"). Under + ADR 009 a maintained `SUM` reads after every edit that touches it, so that is a + per-edit cost, paid by each such cell. + - **(a)** K = 32, as decided with question 10: worst case 0.67–1.9 ms per read. + Chosen first, then **superseded**. + - **(b)** K = 8: worst case 52–127 µs per read. A `SUM` over more than 8 distinct + denominators is inexact. **Chosen (the later answer).** + - **(c)** K = 16: worst case 174–448 µs per read. + + *Recommendation (taken in the end):* **(b), K = 8**, with the read combining over the + lcm and reducing once (the lower figure in each range). The user chose (a), then (b). + - The ceiling (question 13) stays at 8,192 bits under any of the three. + - The common sums keep exactness under K = 8: `SUM(v / 3)` has one denominator, and + an average or a percentage has a handful. + - The sum of margins has one denominator per record, so it passes any of these Ks + long before it is large, and reads inexact under each of them. + - K is a constant inside `numtower`, not a format or wire fact, so it can be tuned + later against the evaluator benchmark (migration step 1) without a new ADR. + + **Re-derived under K = 8** (revision 8), from this ADR's own measurements, with + nothing new measured (the K = 32 figures of revision 7 are superseded): + - **Read cost:** the K = 8 row of the table, **52 µs** (lcm, then one reduce) to + **127 µs** (reduce after every add). `numtower` reads by the first method, so the + lower figure is the working number. The 16–34 µs of revision 5 stays withdrawn. + - **Combined width:** lcm up to K·127 = 1,016 bits (measured 988), numerator up to + K·127 + 256 = **1,272 bits**. The combined numerator was measured only at K = 32 + (4,053 bits), so 1,272 is a bound and not a measurement. It excludes the up to 3 + carry bits of summing 8 terms, which the headroom below absorbs. + - **Ceiling:** the proposed 8,192 bits clears 1,272 by 6,920. Unchanged, so + question 13's decision is untouched. + - **Memory:** O(distinct denominators), which K does not change (clause 8). + - **Not re-derived:** an end-to-end per-edit latency for a maintained `SUM`. The + benchmark times the read alone. Whether 52–127 µs per edit is acceptable on a + page of such cells is a question for the evaluator benchmark, not a figure here. + +## Relations to other ADRs + +Added in revision 7. This lists what the three ADRs written since ask of this one, as +read at their heads, and **changes none of the user's decisions** here. Each ADR is +cited at the head named, and none was built or run. + +**ADR 007 (stable identities, accepted at revision 8; merged via #210).** The points +below are read from revision 8, as on `gh/main`. Revision 9 is **not** part of it: see +the last bullet. +- **The version literal.** ADR 007 clause 10 adds a second `.improv` literal and says + its exact spelling is left to the implementation. This ADR shares it (clause 10, + question 5), so `#i` and `n/d` ride on the same bump. ADR 007 says nothing about + numbers, so nothing in its text conflicts. +- **Flag, ordering.** If ADR 007's migration ships the new literal **before** this ADR's + grammar arms, a reader of that literal would not know `#i` or `n/d`, and "shared" would + mean one literal with two grammars. The two should either land the literal together or + this ADR's arms should be added under the literal ADR 007 has already shipped, with the + old readers' behaviour on `#i` stated. That is a merge-order matter for the + implementer, not a decision the user has made. +- **Ids and display.** This ADR's name-quoting and the formula-target handling are about + names, and ADR 007 moves keys from names to ids. Numbers are not names, so clause 10's + number spellings are unaffected; any clause here that quotes a category or item by + name should be read as "by display name" once ADR 007 lands. Not re-read clause by + clause. +- **Revision 9 is a proposed amendment, not accepted** (PR #326, open, head `068e4130`; + it needs the user's explicit approval, and its Q23–Q25 are open). Nothing in this ADR + depends on it. It touches this ADR only at the edges: answers 33 and 34 (a typed name + matches ids first, then displays; whitespace in a slug) bear on how a name in formula + text is resolved, and answers 35 and 36 reopen Q23 (whether a measure or formula target + has an id and a display of its own), which would change what "a formula target" means + where this ADR names one. Read as a proposal, and re-check this ADR if #326 lands. + +**ADR 008 (virtualized grid, PR #308, proposed).** Its Q28 answer, "bigint is fine, we +don't need to reinvent integers > safe size", makes a grid position a bigint above 2^53 +(its Q36 spells it as a decimal string in JSON). +- **Dependency.** It adds `num-bigint` to `improvise-protocol`, a wasm-bundle crate. This + ADR already accepts `num-bigint` in `numtower` (+20.8 KB gzipped for `num-rational` + plus `num-bigint`, probed; question 8). If `improvise-core` depends on `numtower` (clause + 2), `improvise-protocol` already reaches `num-bigint` through it, so the cost is paid + once, not twice. ADR 008 says its `cargo tree --edges normal` check has not been run; + neither has one here for the bundle. +- **Flag, a new consumer of `numtower`.** Positions are integers, not numbers in the + tower's sense. Whether they use `numtower`'s type or `num-bigint` directly is open in + ADR 008. The tower's `n/d` and `#i` spellings must not leak into a position's wire + spelling. + +**ADR 009 (incremental evaluation, PR #309, proposed).** +- **`ExactSum` lives in `numtower`** (question 15, (a)), which ADR 009 cites. `numtower` + can land holding it alone, so ADR 009 does not wait for this ADR. +- **`Sum` is exact only under ADR 009's question 3 (c) or under this ADR's wide + accumulator.** This ADR's accumulator is exact for decimals (320 bits) and for up to + K = 8 fraction denominators (clause 8). For **f64** terms the exact accumulator is + ADR 009's `ExactSum`. A mixed sum is read from both (clause 8), so "exact under this + ADR" holds for decimal and bounded-fraction data, not for f64 data. +- **The per-edit cost is a lower bound, and the larger exposure is `Eager`.** ADR 009 + reads a maintained `SUM` after every edit that touches it. With K = 8 (question 16) + the read alone is up to 52–127 µs for a cell holding several large distinct + denominators (see "Re-derived under K = 8"), and the update cost comes on top. ADR + 009 says `Eager` "pays the read on every aggregate evaluation", and today's TUI + projects the whole grid every frame, so it pays one read per summed cell. Against ADR + 009's budget (8 ms per operation, a 2 ms slice for a job; ADR 008 clause 11), as + arithmetic from these figures and not a finding: **154** such cells at 52 µs + (154 × 52 = 8.0 ms; 153 stay under) or **63** at 127 µs (63 × 127 = 8.0 ms) exceed + 8 ms, and **39** at 52 µs or **16** at 127 µs exceed a 2 ms slice. At the earlier + K = 32 (superseded) it was five and twelve cells per 8 ms, and one read nearly filled a + slice. K = 8 is 13 to 15 times cheaper per read than 32 (0.67/0.052, 1.9/0.127). It remains a question for ADR + 009's `Eager` and cap design, but a far smaller one. The worst case needs 2 to 8 large + distinct denominators in one sum; a sum of small denominators is far cheaper. ADR + 009's own + `ExactSum` figures (2.0–3.6 ns per update, 0.6–1.1 µs per read) are for the f64 + accumulator and are not comparable. +- **Flag, stale text in ADR 009.** At its head it describes ADR 005's cost as + `rust_decimal` alone, "not a bignum, +15.4 KB", with 40 bytes per summed group. Under + option (d) the crate also carries `num-rational` and `num-bigint` (up to about +36 KB + together, unprobed together) and a per-denominator group map. It also gives `ExactSum` + 568 bytes per group where this ADR gives the decimal partition 40 bytes; those are two + different accumulators and are not a conflict. + +## Consequences + +- Typed values and their `+ - *` are exact. `0.1 + 0.2 = 0.3`, `1234.56 - 1234.55 = + 0.01`, and a typed `1.005` shows as `1.01` at two places (improvise-felo.9 may fix the + last one sooner, and without this ADR). +- `SUM` over exact data is independent of order, through a 40-byte wide accumulator per + group (clause 8). Gotcha 18's `SUM` drift goes as a class for exact data, not just by + pinned iteration order. +- `FLOAT_EQ_EPSILON` is gone, and gotcha 17 is rewritten as "one zero predicate". +- **Deleting the epsilon changes existing documents**, not only the tests. Both + assertions of the improvise-0bf test flip (clause 6). And in a document today, a + formula whose inputs make an *inexact* residual changes answer: `A / 3 * 3 - A` is a + few ulps from zero, so `IF(… = 0, …)` flips from true to false, and dividing by such a + residual yields a huge value where it yields `div/0` today. Exact data is unaffected, + because its residuals are exactly zero. The user decided that policy (question 3), and + the escape hatch is explicit: `EQUAL(a, b, epsilon)` (clause 6). Plain examples of + formulas that change, each with the question that causes it: + - **Question 3 (no epsilon):** a comparison of two *inexact* residuals, such as + `IF(POW(A, 0.5) * POW(A, 0.5) - A = 0, …)`, was true within the epsilon and is now + false. The fix is `EQUAL(…, 0, 1e-9)`. + - **Question 7 (exact mixed comparison):** `IF(POW(x, 0.5) = 1.1, …)` with `x = 1.21`. + In f64, `pow(1.21, 0.5)` is exactly the f64 nearest 1.1, so this is true today even + without the epsilon (the review measured it). It becomes false because it compares + that inexact value with the **exact** decimal `1.1`. Revision 4 filed this under + question 3, and that was wrong. The fix is `EQUAL(POW(x, 0.5), 1.1, 1e-9)`. + - **Option (d):** `IF(A / 3 * 3 = A, …)` on typed data is true today within the + epsilon, and it stays true, now exactly, because `A / 3` is an exact fraction. + - **Question 9 (version skew):** an old peer's `0.1` arrives as a bare JSON number and + reads as `Inexact(0.1)`. That is **unequal** to every typed `0.1`, though the two + look identical on screen. improvise-szi8 closes it. +- A scale-only edit (`12.5` to `12.50`) bumps the record's version (clause 9). +- `Exact(0.1) = Inexact(0.1)` is false, because comparison between exact and inexact is + exact (clause 6, question 7). +- A typed `12.50` stays `12.50` in the editor. +- Division that does not terminate, and `AVG` over any count, are **exact fractions** + within the `i128` bound (decided, question 2). `POW` with a fractional exponent is + inexact. +- `.improv` gains `#i`, `n/d` and ADR 007's version literal. Existing files load as the + decimals they spell. +- The wire spells **every** number as a string in improvise's own encoding (decided, + question 6). A thin client needs no new rule, because `typed()` and the number + language are shared. +- `n/d` is text wherever a type is inferred and a fraction wherever it is forced + (decided, question 12). +- `nan`, `inf` and a finite lexeme that overflows f64 (`1e999`) are text at import, as + improvise-jc95 decided. `typed()` narrows to the three canonical tokens, so a typed + `Infinity`, `NaN` or `1e999` becomes text where today it is a number (clause 3). +- A value typed past `Decimal`'s range is an exact fraction when it fits `i128` + (`1e-30`, a 30-digit integer), as the same value computed already is (clause 3). +- A `SUM` reads exactly over at most K distinct fraction denominators, at a worst-case + cost that grows about as K² (question 16: K = 8, decided), and past K it is inexact. +- Formulas gain `EQUAL(a, b, epsilon)`, and `EQUAL` becomes a keyword the writer quotes. +- **Costs:** `Num` is 24 bytes where an `f64` is 8, but **`CellValue` stays 32 bytes** + under both (a) and (d), because its `String` arm already sets the size (the review + measured it). The bundle gains `rust_decimal` (+15.4 KB gzipped, probed) and, under + (d), `num-rational` and `num-bigint` (+20.8 KB gzipped probed alone, up to about + +36 KB together, unprobed together). Arithmetic is about 8× slower per add and about 100× + per division than f64 (measured, microbenchmark). A 90,000-record formula that + divides for every record pays about 5–10 ms in arithmetic alone. Whether the + evaluator notices is migration step 1's measurement, and it is **not** measured here. +- **Not covered:** the order-free sum of *inexact* values, which is ADR 009's + accumulator; the protocol version, which is improvise-szi8; conversion functions such + as `EXACT()`/`INEXACT()`, which no ticket asks for. +- **Option (d)'s own consequences apply**, now that it is decided: a third numeric arm, + a grouped sum accumulator, the `n/d` spelling, exact quotients, and a result's scale + depending on whether it passed through the fraction arm. + +## Appendix: the measurements + +A scratch crate outside the workspace, run with `cargo run --release`. It was not +committed. Committing it would add three decimal crates to the workspace's dev +dependencies, and every `cargo test --workspace` would then build them, to hold a +proposal's evidence. If the user accepts this ADR, migration step 1 commits the +`rust_decimal` and fraction halves of the measurement as an example of the numeric +crate (clause 14), whose normal dependencies already include what they need. So it adds no dependency beyond the ones the ADR adds. + +**Data.** A typed value is a two-place decimal from 0.01 to 999,999.99, drawn by +SplitMix64 (the same generator as ADR 009's `retraction_drift.rs`), and spelled as the +text a person types. Each representation parses that text: f64 by `str::parse`, +`Decimal`/`BigDecimal` by `from_str`, and `BigRational` as mantissa over 10^scale. + +**SUM.** A fold over N parsed values, N = 18,000 and 90,000, best of 50 runs (20 for +`bigdecimal`, 5 for `BigRational`), reported per element. The order check folds forward +and reversed. + +**Formula.** For N records with independent Revenue and Cost, `SUM((Rev − Cost) / Rev)`. +`bigdecimal` is run at its default precision (100 digits) and at 28. `BigRational` is +summed incrementally, printing the running denominator's bit length at powers of two, +and abandoned after 20 seconds. + +**Accumulator stand-in.** Each stored f64 is converted to its exact rational +(`BigRational::from_float`), summed exactly, and rounded once to f64. That is precisely +what a correct superaccumulator returns. It is compared with the typed total, the +decimal sum of the texts converted to f64 once. Small N: 10,000 trials each at +N = 2, 3, 10 and 100, seed 7. Large N: seeds 1 to 8. The signed distribution draws cents uniformly +from −100,000 to 100,000 (±1,000.00), with the same N, trials and seed; it was not run at +large N. + +**Display.** `format.rs:127-136`'s `round_half_away`, copied verbatim, against +`Decimal::round_dp_with_strategy(2, MidpointAwayFromZero)`, for `2.675`, `1.005`, +`0.125` and `1.115`. + +**Wasm size.** A `cdylib` exporting one function that parses two strings, then does +`(a + b) − b`, divides by `b` and prints, in each representation. It also parses and +prints an f64 in every variant, so f64 formatting is in the baseline as it is in the real +bundle. Profile `opt-level = "s"`, `lto = true`, `codegen-units = 1`, +`panic = "abort"`. Then `wasm-opt -Oz --enable-bulk-memory +--enable-nontrapping-float-to-int`, and `gzip -9` for the compressed figure. + +**Revision 2's checks of the review's findings**, in the same scratch crate: + +- **B1.** `(p + 1e12) − 1e12` with `p = 0.30000000000000004` gives + `0.3000000000000000` at scale 16. `(1e12 − 1e12) + p` gives `0.30000000000000004` at + scale 17. +- **B2.** 200,000 `Decimal`s from `from_i128_with_scale`, with a 96-bit mantissa and a + scale drawn from 0–28 (SplitMix64, seed 11). `to_f64()` differed from + `to_string().parse::()` in 17,375 of them. The review's own sample found 17,392. +- **Parsing.** `Decimal::from_str("0.12345678901234567890123456789")` returned + `0.1234567890123456789012345679`, and `from_str_exact` refused it. + +Everywhere this appendix converts a decimal total to f64 ("the typed total"), it does +so through `to_string().parse::()`, the correctly rounded path, not `to_f64`. So +the accumulator table above does not depend on the conversion B2 found wrong. + +**Option (d)** (revision 3) uses the same crate, with `num-rational` 0.4.2's +`Ratio` and `Ratio` checked operations. Operation costs are measured over +18,000 pairs of typed values: seeds 21 and 22 for two-place operands, and those pairs' +margins for unrelated denominators. The overflow cases use seeds 1–8, drawing Revenue +and Cost from seeds `2s` and `2s + 1`. Sizes are `size_of` of the three `Num` shapes. +The `to_f64` check compared `Ratio::to_f64` with `BigRational::to_f64` on 200,000 +fractions (seed 5). That comparison is **circular**, because the two share one routine, +so the ADR now relies on the review's independent interval check instead (option (d), +"Operations"). Every figure is the range over two runs on the machine above. + +**The grouped read** (revision 6), in the same crate. For K = 2, 8, 16 and 32, 200 reads +each. A read combines K groups whose numerators are random 250-bit integers and whose +denominators are random odd 126-bit integers (xorshift64, one fixed seed), sorted +ascending by denominator. Two methods are timed on the same groups: `BigRational` +addition group by group, which reduces by a gcd after every add, and an lcm +accumulation with one gcd reduction at the end. The two results are asserted equal. +One run on the machine above; a second run agreed within 6% (within 2.7% above +K = 2). Where the text gives a range such as 0.67–1.9 ms, it runs from the lcm method +to the per-add method. diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index 8fe53cbf..484ba416 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -45,6 +45,7 @@ const SECTIONS = [ ["context/adr/002-naming-a-cell.md", "adr/002-naming-a-cell", "ADR 002 — Naming a cell"], ["context/adr/003-live-backed-models.md", "adr/003-live-backed-models", "ADR 003 — Live-backed models"], ["context/adr/004-faces-and-themes.md", "adr/004-faces-and-themes", "ADR 004 — Faces"], + ["context/adr/005-exact-decimal-arithmetic.md", "adr/005-exact-decimal-arithmetic", "ADR 005 — Exact decimal arithmetic (proposed)"], ["context/adr/007-stable-identities.md", "adr/007-stable-identities", "ADR 007 — Stable identities"], ["context/adr/009-incremental-evaluation.md", "adr/009-incremental-evaluation", "ADR 009 — Incremental evaluation (proposed)"], ],