improvise/context/code/source.md

CellSource: where cell values are read from

crates/improvise-core/src/model/source.rs. ADR 003 asked for a source trait in core with the in-RAM DataStore as one implementation (improvise-qddo). ADR 009 step 7 (improvise-rnak) asks for the two live sources that need it. This file is the reasoning; the evidence for each test is in context/test-evidence/core.md. The ADRs are not amended: what follows are the readings taken where they leave a choice.

What is in the trait, and why

method	what the evaluator asks
value_at(key)	the value stored at exactly key (an exact read, the filter-mismatch fallback)
summarize(select)	the summary of the cells a Select names: the aggregate, not the rows
keys(partial)	the keys of the cells containing partial: stems, and the records behind an aggregate
cells(partial)	the same with values: records mode and a drill

1. Aggregation is the interface. ADR 003 clause 1: GridLayout asks for a value, not for the rows behind it, and the aggregates are SQL's. So the one read the evaluator makes in bulk, summarize, returns a Summary and never a list of values. keys and cells exist because some reads are of records, not of a fold (the stems a formula is evaluated at; records mode), and a SQL source answers them with a SELECT DISTINCT or a plain select.
2. Writes are not in the trait. DataStore::set and remove are inherent methods. A source that cannot be written to has no method to fail in, which is ADR 003's "unrepresentable rather than enforced". The model's own writes (set_cell, clear_cell) go to the owned store.
3. The trait has no Result. A source that can fail (a connection) decides at its own boundary what a failed read is, and improvise-io is where that decision belongs. AGENTS.md keeps Option and Result at the edges; Option is the one the trait uses, for "no such cell".
4. Select is spanned_values' arguments. A fold names a partial key, the covered members of each group or omitted category it spans, and whether only an exact-width cell counts. The covered members are decided in RAM from the categories and the view's hidden items, so a source is handed explicit item lists and never a predicate. A hidden item therefore reaches a source only as the absence of an item from a list.

Summary and Numbers

A Summary is what a source reports about the values of a read: a Numbers (count, an exact sum, the least and greatest non-NaN, in total_cmp order so -0.0 is below +0.0) and whether the rest (text, errors) agree on one value. It reduces exactly as aggregate_cell_values did: numbers by the function, anything else by unanimity, else Error("mixed"), none at all absent. Three properties carry the design:

• It merges. A read of a collapsed group, a total, or several contexts of a formula aggregate is the merge of the summaries of its parts, equal to the summary of the union. That is what lets an engine answer a pooled read without sending rows, and what lets eval_agg pool Numbers instead of concatenating Vec<f64>. The sum is ExactSum, so a merge is the same bits in any order.
• It is order-free. Unanimity needs the set of non-number values, not their order, so a source that has no insertion order loses nothing by it.
• The reference is kept. tests::reference is the old reduction, copied into the test module; a_summary_reduces_as_the_values_did compares every sequence of an eight-value pool, up to three long, with Summary over all five functions. characterization_survives_the_move: the existing evaluator tests (423 in the crate) are unchanged and pass.

What a real engine must supply is stated here because the trait cannot enforce it: a Numbers whose sum is exact. SQLite's SUM is a float sum, so an engine that cannot return an exact accumulator either returns the numbers (and the adapter sums them) or accepts a last-bit difference, which context/code/improvise-incremental.md ("Step 6") calls a user-visible change.

Model::source()

Model::source() returns the &dyn CellSource that the evaluator, the stems of recompute_formulas and the records of GridLayout read through. Today it is always the owned DataStore; the point of routing the reads is that PR 3 (improvise-rnak) changes only this accessor. Reads that bypass it (model.data.* in other crates, Eager::occupancy, Shapes::of) are the ones later steps move.

TableSource

A scanned table of rows, behind cfg(any(test, feature = "test-support")). It is the second implementation, written as a query engine with no index would answer, sharing only Summary with DataStore. the_store_and_a_scanned_table_answer_every_select_alike asks both every select over a fixture (every subset of every stored key as a base, with no span, one span of each of six item lists, and two), at both widths and for all five functions: over 5,000 triples.

Building a Summary from an engine's answer

Summary::from_parts(Numbers::from_parts(count, exact_sum, least, greatest), Agreement). Agreement is what the non-number values hold: None, One(v) or Mixed. from_parts drops extremes that contradict the count or are NaN, so a source cannot hand the evaluator a state add could not reach.