docs(adr): ADR 009 makes the formula language the query language and states the incremental principle (improvise-felo.4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
This commit is contained in:
committed by
Edward Langley
parent
13b3dff35e
commit
e720f7ca50
@@ -21,6 +21,10 @@ cell write becomes a delta, aggregates are maintained, and only the formula cell
|
||||
inputs moved are re-evaluated by the existing evaluator. The incremental results must
|
||||
equal today's eager results exactly.
|
||||
|
||||
**Principle: everything is as incremental as possible**, so the work done per model
|
||||
mutation is minimised. Where this ADR does work that grows with more than the
|
||||
mutation, it says so and names the incremental path that normally applies.
|
||||
|
||||
### 1. The `zset` crate
|
||||
|
||||
A problem-agnostic workspace crate with no dependencies beyond `std` (so it is
|
||||
@@ -34,8 +38,11 @@ wasm32-clean and serde-free). It provides:
|
||||
- operator state behind one trace interface (look up, read a range, apply a delta), so
|
||||
later store-memory work can page or compress it.
|
||||
|
||||
There are no nested streams, no recursion operator, no timestamps beyond one clock, and
|
||||
no query language.
|
||||
There are no nested streams, no recursion operator and no timestamps beyond one clock.
|
||||
**The formula language is the query language.** What a request asks of the model beyond
|
||||
its axes and window (filters, aggregation, derived values) is written in formula text
|
||||
over ids (ADR 007). It goes through the same evaluator and circuit as a document
|
||||
formula, so there is no second query language.
|
||||
|
||||
### 2. The plug
|
||||
|
||||
@@ -85,7 +92,9 @@ shape key with the same None list, which removes the per-session model clone.
|
||||
|
||||
A group is read with eager's rules, including NaN and signed zero. `LAST`/`FIRST` and
|
||||
semi-additive measures (improvise-felo.7, felo.8) reuse the multiset shape, keyed by the
|
||||
ordering category's order. How that order is keyed is for those tickets.
|
||||
ordering category's order. Those tickets choose the key. It must keep a reorder or an
|
||||
insert incremental: only the entries under the moved items change, and nothing is
|
||||
rebuilt.
|
||||
|
||||
### 5. Formulas
|
||||
|
||||
@@ -118,7 +127,8 @@ before enumerating. This replaces `Expansion::members`.
|
||||
|
||||
DBSP's least fixed point is not improvise's circularity: `X = X + 1` has none, and the
|
||||
answer must be `Error("circular")`. The first version finds the formula graph's strongly
|
||||
connected components. Acyclic formulas are evaluated once each, in topological order. A
|
||||
connected components, recomputed when the formula set changes; how to keep that
|
||||
incremental belongs with open question 2. Acyclic formulas are evaluated once each, in topological order. A
|
||||
component is evaluated with today's bounded fixed point, restricted to that component.
|
||||
That reproduces the sixteen-pass bound for the differential test. It is to be replaced
|
||||
by a DBSP-native treatment (open question 2).
|
||||
@@ -143,7 +153,8 @@ changes no operator. The visible axis is a remap.
|
||||
arriving during the build are buffered and added at install. The finished result is the
|
||||
response to the request that asked for it, not a notice.
|
||||
- **A formula edit** is a formula delta fed to the circuit.
|
||||
- **A document swap** rebuilds everything.
|
||||
- **A document swap** rebuilds everything, and a new key's first build is a full pass.
|
||||
These are the only rebuilds; every other change is a delta.
|
||||
- **Saving a view definition** touches no operator.
|
||||
- **Live keys are a cache.** A key is evicted after a period in which no request names
|
||||
it (the period is set by measurement), and every eviction is announced (clause 9).
|
||||
@@ -243,10 +254,11 @@ an **edge adapter**. So every edge has the same semantics:
|
||||
keys, the host sends the current values of those keys, restricted to the client's
|
||||
window, and the client applies `target − held` over them. The log holds keys and
|
||||
flags, not values, so nothing per client is kept;
|
||||
- where the covered range holds an axis change, a relabel or a flag, which position
|
||||
and labels depend on, the host sends `target` for the whole window;
|
||||
- where the log no longer covers the watermark, the host sends `target` for the whole
|
||||
window, and the client applies `target − held`;
|
||||
- where the covered range also holds axis changes, relabels or flags, which positions
|
||||
and labels depend on, the correction carries them too, in order, as a merged notice
|
||||
does, so the client re-derives positions incrementally;
|
||||
- only where the log no longer covers the watermark does the host fall back to `target`
|
||||
for the whole window, and the client applies `target − held`;
|
||||
- a full refetch is `target − ∅`.
|
||||
|
||||
**Every read sees only the last committed state**, whether it is a correction, a
|
||||
@@ -294,7 +306,8 @@ notice carries:
|
||||
the circuit computes them. Moved records alone would miss formulas with cross-stem
|
||||
inputs.
|
||||
|
||||
A notice whose key lists exceed a fixed size carries "correct" instead.
|
||||
A notice whose key lists exceed a fixed size carries "correct" instead. That is a
|
||||
fallback, and it costs every reader a correction.
|
||||
|
||||
**What a client does.** It intersects the moved keys with its window. A fold row has no
|
||||
maintained key, so it maps member keys through its own remap first. It then fetches what
|
||||
@@ -405,7 +418,9 @@ Step 8's required tests:
|
||||
such as `A = IF(x, B, 1)`;
|
||||
- (c) a monotone reformulation, where one exists.
|
||||
|
||||
*Recommendation:* (a).
|
||||
Separately, the strongly connected components are recomputed whenever the formula set
|
||||
changes. Should they be maintained incrementally too? *Recommendation:* (a), and
|
||||
recompute the components only for the formulas an edit touches.
|
||||
|
||||
3. **What replaces `Eager`?** It still serves as the differential oracle and as the
|
||||
backend for live sources that can only answer aggregate queries. *Recommendation:* a
|
||||
|
||||
Reference in New Issue
Block a user