Claude (dbsp-sources)
103 lines
7.2 KiB
Markdown
103 lines
7.2 KiB
Markdown
# `CellSource`: where cell values are read from
|
|
|
|
`crates/improvise-core/src/model/source.rs`. ADR 003 asked for a source trait in core with the
|
|
in-RAM `DataStore` as one implementation (improvise-qddo). ADR 009 step 7 (improvise-rnak) asks for
|
|
the two live sources that need it. This file is the reasoning; the evidence for each test is in
|
|
`context/test-evidence/core.md`. The ADRs are not amended: what follows are the readings taken where
|
|
they leave a choice.
|
|
|
|
## What is in the trait, and why
|
|
|
|
| method | what the evaluator asks |
|
|
|---|---|
|
|
| `value_at(key)` | the value stored at exactly `key` (an exact read, the filter-mismatch fallback) |
|
|
| `summarize(select)` | the summary of the cells a `Select` names: the aggregate, not the rows |
|
|
| `keys(partial)` | the keys of the cells containing `partial`: stems, and the records behind an aggregate |
|
|
| `cells(partial)` | the same with values: records mode and a drill |
|
|
|
|
1. **Aggregation is the interface.** ADR 003 clause 1: `GridLayout` asks for a value, not for the
|
|
rows behind it, and the aggregates are SQL's. So the one read the evaluator makes in bulk,
|
|
`summarize`, returns a `Summary` and never a list of values. `keys` and `cells` exist because
|
|
some reads are of records, not of a fold (the stems a formula is evaluated at; records mode), and
|
|
a SQL source answers them with a `SELECT DISTINCT` or a plain select.
|
|
2. **Writes are not in the trait.** `DataStore::set` and `remove` are inherent methods. A source that
|
|
cannot be written to has no method to fail in, which is ADR 003's "unrepresentable rather than
|
|
enforced". The model's own writes (`set_cell`, `clear_cell`) go to the owned store.
|
|
3. **The trait has no `Result`.** A source that can fail (a connection) decides at its own boundary
|
|
what a failed read is, and `improvise-io` is where that decision belongs. AGENTS.md keeps
|
|
`Option` and `Result` at the edges; `Option` is the one the trait uses, for "no such cell".
|
|
4. **`Select` is `spanned_values`' arguments.** A fold names a partial key, the covered members of
|
|
each group or omitted category it spans, and whether only an exact-width cell counts. The
|
|
covered members are decided in RAM from the categories and the view's hidden items, so a source
|
|
is handed explicit item lists and never a predicate. A hidden item therefore reaches a source
|
|
only as the absence of an item from a list.
|
|
|
|
## `Summary` and `Numbers`
|
|
|
|
A `Summary` is what a source reports about the values of a read: a `Numbers` (count, an exact sum,
|
|
the least and greatest non-NaN, in `total_cmp` order so `-0.0` is below `+0.0`) and whether the rest
|
|
(text, errors) agree on one value. It reduces exactly as `aggregate_cell_values` did: numbers by the
|
|
function, anything else by unanimity, else `Error("mixed")`, none at all absent. Three properties
|
|
carry the design:
|
|
|
|
- **It merges.** A read of a collapsed group, a total, or several contexts of a formula aggregate is
|
|
the merge of the summaries of its parts, equal to the summary of the union. That is what lets an
|
|
engine answer a pooled read without sending rows, and what lets `eval_agg` pool `Numbers` instead
|
|
of concatenating `Vec<f64>`. The sum is `ExactSum`, so a merge is the same bits in any order.
|
|
- **It is order-free.** Unanimity needs the set of non-number values, not their order, so a source
|
|
that has no insertion order loses nothing by it.
|
|
- **The reference is kept.** `tests::reference` is the old reduction, copied into the test module;
|
|
`a_summary_reduces_as_the_values_did` compares every sequence of an eight-value pool, up to three
|
|
long, with `Summary` over all five functions. `characterization_survives_the_move`: the existing
|
|
evaluator tests (423 in the crate) are unchanged and pass.
|
|
|
|
What a real engine must supply is stated here because the trait cannot enforce it: a `Numbers` whose
|
|
sum is exact. SQLite's `SUM` is a float sum, so an engine that cannot return an exact accumulator
|
|
either returns the numbers (and the adapter sums them) or accepts a last-bit difference, which
|
|
`context/code/improvise-incremental.md` ("Step 6") calls a user-visible change.
|
|
|
|
## `Model::source()` and `Model::over(source)`
|
|
|
|
`Model::source()` returns the `&dyn CellSource` that the evaluator, the stems of `recompute_formulas`,
|
|
the member records of an aggregate over an expression and the records of `GridLayout` read through.
|
|
`Model::over(source)` points it at a `SharedSource` (an `Arc<dyn CellSource + Send + Sync>`) instead of
|
|
the owned `DataStore`. That is the whole evaluator hook (improvise-rnak, bw4p.14's prerequisite):
|
|
there is still one evaluator, `Model`'s, and it computes every formula value, over a source whose
|
|
cells it never owns.
|
|
|
|
1. **A model over a source is read-only, in every build.** `set_cell`, `clear_cell`, `remove_category` and
|
|
`remove_item` `assert!` that no source is set (a `debug_assert` first shipped and was lost in release:
|
|
the write went to the owned store, which nothing reads). A caller who holds a source writes to the
|
|
source (ADR 003: the trait has no writes). `Model::over` drops the model's own cells and the formula cells computed from them, so no stale copy
|
|
is left for `get_cell`, persistence or the harness to read. `data` is `pub`, so a caller can still write
|
|
it directly; it is not read, and nothing prevents that. Structural edits (a category or item removed)
|
|
are refused too, because the source's cells would still carry them.
|
|
2. **A shared source changes behind its readers by interior mutability of its own.** The model holds
|
|
an `Arc`, so a database connection or a test double behind a lock is the same object the
|
|
backend feeds from.
|
|
3. **What still reads the owned store**: `get_cell` (a production caller: `improvise-command/src/ui/effect.rs`,
|
|
the `old` value of a rename), `Model::data` (`pub`), persistence, the import wizard and the effects
|
|
in `improvise-command`. They are the facade of a document that owns its cells; a model over a source
|
|
has none of those to offer, and routing them would change `get_cell`'s `&CellValue` return for 129
|
|
call sites. `Eager::occupancy`, `Shapes::of` and `Stems::build` in `improvise-incremental` are
|
|
routed by the PRs that need them.
|
|
4. **`Debug` shows only whether a source is set.** A source is not necessarily printable, and two
|
|
models over different sources are not told apart by it.
|
|
|
|
## `TableSource`
|
|
|
|
A scanned table of rows, behind `cfg(any(test, feature = "test-support"))`. It is the second
|
|
implementation, written as a query engine with no index would answer, sharing only `Summary` with
|
|
`DataStore`. `the_store_and_a_scanned_table_answer_every_select_alike` asks both every select over a
|
|
fixture (every subset of every stored key as a base, with no span, one span of each of six item
|
|
lists, and two), at both widths and for all five functions: over 5,000 triples.
|
|
|
|
## Building a `Summary` from an engine's answer
|
|
|
|
`Summary::from_parts(Numbers::from_parts(count, exact_sum, least, greatest)?, Agreement)?`. `Agreement`
|
|
is what the non-number values hold: `None`, `One(v)` or `Mixed`. Both constructors return a `Result`
|
|
and **refuse** a state no sequence of values could produce (extremes with no numbers, a NaN extreme,
|
|
least above greatest, one extreme without the other, a sum with no numbers, no extremes beside a sum
|
|
that is not NaN, a number among the "others"), because repairing
|
|
it would hide an engine's bug.
|