refactor(core): a read-only CellSource trait, DataStore its first implementation (improvise-qddo)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze

context/code/source.md

@@ -0,0 +1,72 @@
+# `CellSource`: where cell values are read from
+
+`crates/improvise-core/src/model/source.rs`. ADR 003 asked for a source trait in core with the
+in-RAM `DataStore` as one implementation (improvise-qddo). ADR 009 step 7 (improvise-rnak) asks for
+the two live sources that need it. This file is the reasoning; the evidence for each test is in
+`context/test-evidence/core.md`. The ADRs are not amended: what follows are the readings taken where
+they leave a choice.
+
+## What is in the trait, and why
+
+| method | what the evaluator asks |
+|---|---|
+| `value_at(key)` | the value stored at exactly `key` (an exact read, the filter-mismatch fallback) |
+| `summarize(select)` | the summary of the cells a `Select` names: the aggregate, not the rows |
+| `keys(partial)` | the keys of the cells containing `partial`: stems, and the records behind an aggregate |
+| `cells(partial)` | the same with values: records mode and a drill |
+
+1. **Aggregation is the interface.** ADR 003 clause 1: `GridLayout` asks for a value, not for the
+   rows behind it, and the aggregates are SQL's. So the one read the evaluator makes in bulk,
+   `summarize`, returns a `Summary` and never a list of values. `keys` and `cells` exist because
+   some reads are of records, not of a fold (the stems a formula is evaluated at; records mode), and
+   a SQL source answers them with a `SELECT DISTINCT` or a plain select.
+2. **Writes are not in the trait.** `DataStore::set` and `remove` are inherent methods. A source that
+   cannot be written to has no method to fail in, which is ADR 003's "unrepresentable rather than
+   enforced". The model's own writes (`set_cell`, `clear_cell`) go to the owned store.
+3. **The trait has no `Result`.** A source that can fail (a connection) decides at its own boundary
+   what a failed read is, and `improvise-io` is where that decision belongs. AGENTS.md keeps
+   `Option` and `Result` at the edges; `Option` is the one the trait uses, for "no such cell".
+4. **`Select` is `spanned_values`' arguments.** A fold names a partial key, the covered members of
+   each group or omitted category it spans, and whether only an exact-width cell counts. The
+   covered members are decided in RAM from the categories and the view's hidden items, so a source
+   is handed explicit item lists and never a predicate. A hidden item therefore reaches a source
+   only as the absence of an item from a list.
+
+## `Summary` and `Numbers`
+
+A `Summary` is what a source reports about the values of a read: a `Numbers` (count, an exact sum,
+the least and greatest non-NaN, in `total_cmp` order so `-0.0` is below `+0.0`) and whether the rest
+(text, errors) agree on one value. It reduces exactly as `aggregate_cell_values` did: numbers by the
+function, anything else by unanimity, else `Error("mixed")`, none at all absent. Three properties
+carry the design:
+
+- **It merges.** A read of a collapsed group, a total, or several contexts of a formula aggregate is
+  the merge of the summaries of its parts, equal to the summary of the union. That is what lets an
+  engine answer a pooled read without sending rows, and what lets `eval_agg` pool `Numbers` instead
+  of concatenating `Vec<f64>`. The sum is `ExactSum`, so a merge is the same bits in any order.
+- **It is order-free.** Unanimity needs the set of non-number values, not their order, so a source
+  that has no insertion order loses nothing by it.
+- **The reference is kept.** `tests::reference` is the old reduction, copied into the test module;
+  `a_summary_reduces_as_the_values_did` compares every sequence of an eight-value pool, up to three
+  long, with `Summary` over all five functions. `characterization_survives_the_move`: the existing
+  evaluator tests (423 in the crate) are unchanged and pass.
+
+What a real engine must supply is stated here because the trait cannot enforce it: a `Numbers` whose
+sum is exact. SQLite's `SUM` is a float sum, so an engine that cannot return an exact accumulator
+either returns the numbers (and the adapter sums them) or accepts a last-bit difference, which
+`context/code/improvise-incremental.md` ("Step 6") calls a user-visible change.
+
+## `Model::source()`
+
+`Model::source()` returns the `&dyn CellSource` that the evaluator, the stems of `recompute_formulas`
+and the records of `GridLayout` read through. Today it is always the owned `DataStore`; the point of
+routing the reads is that PR 3 (improvise-rnak) changes only this accessor. Reads that bypass it
+(`model.data.*` in other crates, `Eager::occupancy`, `Shapes::of`) are the ones later steps move.
+
+## `TableSource`
+
+A scanned table of rows, behind `cfg(any(test, feature = "test-support"))`. It is the second
+implementation, written as a query engine with no index would answer, sharing only `Summary` with
+`DataStore`. `the_store_and_a_scanned_table_answer_every_select_alike` asks both every select over a
+fixture (every subset of every stored key as a base, with no span, one span of each of six item
+lists, and two), at both widths and for all five functions: over 5,000 triples.

context/test-evidence/core.md

@@ -2300,3 +2300,47 @@ key with the whole suite green. It is reached by `reopened_at` (an advance),
 `refused_here` (a commit) and `settle_declined_editor`. Rival: the arm passes
 `category`; the test fails on all three assertions, and the display is 36 characters
 against the key "Region".
+
+### model::source::tests::a_summary_reduces_as_the_values_did
+
+improvise-qddo. The reduction moved from `aggregate_cell_values` to `Summary::reduce`; the test keeps
+the old function as `reference` and compares all 585 value sequences of up to three values from an
+eight-value pool (NaN, both zeros, infinity, two texts, an error) on all five functions. Rivals, each
+failing it by name: unanimity ignored (`Others::One` kept for any second value), a number beside a
+unanimous text read as unanimous, and `<` for the order of the zeros (the sign of a zero extreme is
+lost). The first two also fail `aggregating_differing_text_values_is_an_explicit_error` and
+`aggregating_text_among_numbers_is_an_explicit_error`; the zero rival also fails
+`eager_math_tests::min_and_max_of_both_zeros_do_not_depend_on_their_order`.
+
+### model::source::tests::a_merge_of_summaries_is_the_summary_of_the_union
+
+A read of several parts is a merge. The test merges every ordered pair of sequences of up to two
+values and compares with the reference of the concatenation. Rivals: the least not merged, the
+greatest merged with the other side's least, the zeros unordered. Each fails it.
+
+### model::source::tests::numbers_pool_what_the_formula_aggregate_pools
+
+`eval_agg` pools `Numbers` now, not a concatenated `Vec<f64>`. Every split of a five-number list
+(a NaN and both zeros among them) pooled by merge must equal the whole, by bits for all five
+functions and for the total. Rivals: the least not merged, the greatest seeded from the other
+side's least. Both fail it.
+
+### model::source::tests::the_store_and_a_scanned_table_answer_every_select_alike
+
+`DataStore` and `TableSource` (a scan, sharing only `Summary`) are asked every select over the
+fixture, at both widths. Rivals, each failing it: the store ignores `exact` for a select with no
+span, and the store ignores `exact` for a spanned one. The first also fails
+`an_exact_select_with_no_span_names_one_cell`.
+
+### model::source::tests::the_store_and_a_scanned_table_list_the_same_cells
+
+`keys` and `cells` under every base of the fixture, and `value_at` for every row and an absent key.
+Rival: `DataStore::keys` ignores its partial. It fails here and in five evaluator tests that read
+the member records of an aggregate over an expression
+(`route_then_aggregate_tests::a_routing_formula_at_a_record_is_the_rule_itself` among them).
+
+### model::source::tests::an_exact_select_with_no_span_names_one_cell
+
+A select with `exact` and no span is the cell, not the cells containing the pairs. The store's
+`spanned_values` is never called for it (the evaluator reads an exact key through `value_at`), so
+this is the contract a second source implements. Rival: `exact` ignored when there is no span.