diff --git a/context/adr/009-incremental-evaluation.md b/context/adr/009-incremental-evaluation.md index 6099ec8b..834b849f 100644 --- a/context/adr/009-incremental-evaluation.md +++ b/context/adr/009-incremental-evaluation.md @@ -57,7 +57,7 @@ computes each value, so formulas are never compiled into operators. **`Eager` is deprecated.** Today's code moves behind the trait as `Eager`, unchanged. It serves as the oracle of a differential property test, which applies one random edit sequence to both backends and compares every drawn cell, total and occupancy count. It -is removed once its remaining uses are replaced (open question 3). +is removed once its remaining uses are replaced (open question 2). ### 3. The cell delta @@ -128,10 +128,10 @@ before enumerating. This replaces `Expansion::members`. DBSP's least fixed point is not improvise's circularity: `X = X + 1` has none, and the answer must be `Error("circular")`. The first version finds the formula graph's strongly connected components, recomputed when the formula set changes; how to keep that -incremental belongs with open question 2. Acyclic formulas are evaluated once each, in topological order. A +incremental belongs with open question 1. Acyclic formulas are evaluated once each, in topological order. A component is evaluated with today's bounded fixed point, restricted to that component. That reproduces the sixteen-pass bound for the differential test. It is to be replaced -by a DBSP-native treatment (open question 2). +by a DBSP-native treatment (open question 1). ### 7. Operator state follows the request's shape key @@ -297,20 +297,29 @@ query, and, when scrolling up or left, finds a new top-left by galloping backwar doubling jumps (about 14 to 28 probes at density 10⁻⁴). **The change notice.** Each command commits on its own (ADR 006), and after each -committed command the host emits one notice to every client, addressed to none. A frame +committed command the host emits one notice, addressed to no client. A frame whose later command fails keeps its earlier commands, and their notices, published. A notice carries: - the stamp `(host instance, document_generation, publication)`. The publication number increments once per committed command. The host instance changes on restart; it is a random id, never a wall-clock time; - axis changes, display relabels as `(id, display)`, and a flag when the formula set, - `measure_agg`, the view definitions or the default view moved. A shared display's ` #2` suffix is never sent: the client computes - it from the displays in scope; + `measure_agg`, the view definitions or the default view moved. A shared display's + ` #2` suffix is never sent: the client computes it from the displays in scope; - the shape keys evicted since the last notice; - for each live shape key, the moved output keys (cells, formula cells and totals), as the circuit computes them. Moved records alone would miss formulas with cross-stem inputs. +**A client receives only what it needs.** Filtering is part of the DBSP graph, not a +separate mechanism. A client either pulls just the state it needs, or pushes a filter to +the host's side of its connection: a filter operator on its shape key and window, written +in the query language, through which its notices pass. A pushed filter belongs to the open +connection, like that connection's timeout, and is released with it. On a sequence of +requests the client sends it with each request, so the host keeps no per-client state +beyond an open connection. While a connection's filter is open, it counts as a request +naming its shape, so its shape is not evicted. + A notice whose key lists exceed a fixed size carries "correct" instead. That is a fallback, and it costs every reader a correction. @@ -360,8 +369,8 @@ that applies. Numeric widths stay with ADR 008's view-layer rule. - **Compiling formulas into operators:** a second evaluator. - **An ad-hoc dirty-tracking cache:** one invalidation rule per derived fact, each a second description of evaluation. -- **Per-client filtering of notices, or pushed per-window deltas:** they save traffic - only by holding per-client state on the host (open question 1). +- **A topic or subscription mechanism beside the graph:** filtering is an operator in + the graph (clause 9). ## Consequences @@ -376,7 +385,6 @@ that applies. Numeric widths stay with ADR 008's view-layer rule. - Two backends exist until `Eager` goes. - Integrated state is O(cells) beside the store. - `ExactSum` costs 568 bytes per summed group. -- Every client receives every notice. - Each command snapshots the circuit's state for rollback. **Order** (each step is a ticket in the implementation epic, behind the differential @@ -405,16 +413,7 @@ Step 8's required tests: ## Open questions -1. **Should notices be filtered per client?** Clause 9 sends every client every live - shape's moved keys. That keeps the host free of per-client state, but traffic is - O(clients × all moved keys) per committed command. Filtering by topic would send each connection - only its own shape's key lists. It needs one subscription per connection, which pins - its shape in the cache. Is such a subscription "model-relevant information" the - backend may hold? - *Recommendation:* broadcast with the size cap. Add topics only if measured traffic - with many clients on different shapes justifies them. - -2. **How should cycles be handled natively?** Clause 6's component fixed point is +1. **How should cycles be handled natively?** Clause 6's component fixed point is interim. The options: - (a) a recursion operator bounded at sixteen iterations, writing `circular` where it has not converged, which keeps today's answers; @@ -426,12 +425,12 @@ Step 8's required tests: changes. Should they be maintained incrementally too? *Recommendation:* (a), and recompute the components only for the formulas an edit touches. -3. **What replaces `Eager`?** It still serves as the differential oracle and as the +2. **What replaces `Eager`?** It still serves as the differential oracle and as the backend for live sources that can only answer aggregate queries. *Recommendation:* a small from-scratch reference implementation, kept in the test crate, as the oracle, and a pushdown backend behind the same trait for query-only sources. -4. **Which columns get a host-maintained width?** Columns whose cells are text? What +3. **Which columns get a host-maintained width?** Columns whose cells are text? What about a column holding both text and numbers? *Recommendation:* text cells get the maintained width, and the view layer takes the larger of that and its own numeric width.