From e9a43ce657ff112b0fe223a09193e6a6ad69a93c Mon Sep 17 00:00:00 2001 From: "Claude (dbsp-probes)" Date: Sat, 10 Oct 2026 11:49:33 -0700 Subject: [PATCH] docs(incremental): the step 4 groundwork and its evidence; box the divergence's probe (improvise-bw4p.3, improvise-bw4p.4) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze --- context/code/improvise-incremental.md | 46 ++++++++++++++----- .../test-evidence/improvise-incremental.md | 36 +++++++++++++++ crates/improvise-incremental/src/harness.rs | 24 ++++++---- crates/improvise-incremental/tests/oracle.rs | 43 +++++++++++------ 4 files changed, 113 insertions(+), 36 deletions(-) diff --git a/context/code/improvise-incremental.md b/context/code/improvise-incremental.md index 32df1041..b23a4cc7 100644 --- a/context/code/improvise-incremental.md +++ b/context/code/improvise-incremental.md @@ -32,10 +32,8 @@ fold of the cells that match a partial key") and nothing else. These are the rea downstream, core cannot read through it, which is the clause's point. 2. **Two methods, the minimum.** `value(key, agg)` is the formula result at a key, or the cell as stored or folded; `raw_fold(partial)` is the fold of matching stored cells. `agg` is - `Aggregation`, the evaluator's own type. Hidden-item `covers`, totals and occupancy are not - in the trait, so the differential cannot probe them, and a hidden item (view state) cannot - be exercised at all. The ADR's differential compares "every drawn cell, total and occupancy - count"; improvise-bw4p.3 adds them. A group is exercised: a stored key with one coordinate + `Aggregation`, the evaluator's own type. Hidden-item `covers`, totals and occupancy joined the trait + in improvise-bw4p.3 (section "Step 4 groundwork"). A group is exercised: a stored key with one coordinate replaced by its item's group is a probe. 3. **`raw_fold` is never a formula, and `Eager` enforces it.** For a partial naming a formula target `Eager::raw_fold` is `None`; otherwise it is `evaluate_aggregated(partial, OverHidden)`, @@ -104,14 +102,38 @@ coordinates from a stored key is not asked about. `edit_probes` adds the key an edit names, so a cleared cell is asked about after it is gone. A divergence reads `divergence at step N: : eager != candidate `. -**A limit, stated plainly (improvise-bw4p.4).** The probe set is every stored key's -sub-partials, not the keys a view draws. A view with four or more categories on None draws a cell -that drops four or more coordinates of a stored key, and the harness checks none of those cells: -a rival wrong only at `{C0=a, _Measure=Revenue}` over a five-category model passes. Neither -shipped demo reaches the limit, so the corpus does not show it. Raising `MAX_DROPPED` is not the -answer, because `sub_partials` walks all 2^n subsets of a key whatever the limit is (93ms at 20 -categories and 225ms at 22 in a debug build, per call). The fix is to probe the keys each view -draws, with steps 3 and 4; until then a candidate on such a view is under-checked. +## Step 4 groundwork: what a view draws (improvise-bw4p.3, improvise-bw4p.4) + +The subset probes above cannot see a drawn cell on a view with four or more None categories +(it drops more than `MAX_DROPPED` coordinates of a stored key), and the trait could not be asked +about hidden items, totals or occupancy. Both are closed. These are the readings taken. + +1. **`Reading` is how a view reads.** The aggregation (`Aggregation::of` its None categories) + and the hidden `(category, item)` pairs of `View::hidden_items`. `covers(cat, item)` is the + predicate `GridLayout::evaluator` passes as its `covers`. It is a value, not a closure, so a + probe can name it and a divergence can print it. +2. **Three methods join the trait**: `covered(key, reading)` is `Model::evaluate_covering`; + `total(key, over, reading)` is `evaluate_spanning`, a column total as ONE fold over the + visible items of the omitted categories; `occupancy(partial, reading)` counts the stored + cells matching `partial` none of whose coordinates is hidden. Occupancy is a stored-cell + count, the quantity ADR 009 clause 7 keeps by axis item; "has a value" (what `prune_empty` + asks) is `covered(..).is_some()`, already probed. +3. **`drawn_probes(model, views)` asks what each view draws**, built from the view's own + `GridLayout` (pruned as production builds it, records mode skipped): for every drawn cell + the three subset questions plus `Covered` and `Occupancy`; for every column `Total` over + the row categories and the column line's occupancy; for every row the row line's occupancy. + `differential_in(model, views, ..)` adds them at every step with the rest; `differential` + is that with no views. A view is a `View`, not a document, because the model holds none. +4. **`sub_partials` is polynomial**: it enumerates the sets of at most `MAX_DROPPED` dropped + indices, not the 2^n masks. The probe set is the same; the cost at 26 coordinates falls from + about 800ms to under 70ms (debug build). `MAX_DROPPED` stays: it bounds the subset probes, + and the drawn probes carry the cells that drop more. +5. **The hidden fixture is a workbook the corpus now includes** (`hidden`): no shipped demo + hides an item, so a backend wrong about hidden items would have passed the direct + `Eager`-against-evaluator comparison as well. +6. **Not here**: a view's None categories fix the formulas' cache, and `Eager` holds one set. + The drawn probes of a view under another None set are read from that one cache, by both + sides alike, so they test the backend and not the evaluator's per-view recompute. The corpus the tests use is `examples/demo.improv` and `web/demo.improv`, loaded through `improvise_io::persistence::load`, and hand-built models (formulas, text, mixed values, a group). diff --git a/context/test-evidence/improvise-incremental.md b/context/test-evidence/improvise-incremental.md index b4a68838..9fe4a775 100644 --- a/context/test-evidence/improvise-incremental.md +++ b/context/test-evidence/improvise-incremental.md @@ -215,3 +215,39 @@ The differential for step 3. A proptest (256 cases, up to 12 edits drawn from se both zeros and three NaN spellings) checks that the delta `DataStore` logs for an edit equals the store as a Z-set after it minus the store before. No `Derived` backend exists yet, so this is the harness's step-3 form: the delta against the store it describes. + +## the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates + +improvise-bw4p.4. Six categories, four on None, a rival wrong only at `{C0=a, _Measure=Revenue}`. +Before the drawn probes the differential passed it (`is_err` failed); the cell drops four +coordinates of a stored key and `MAX_DROPPED` is three. Now the first divergence is the +drawn cell's exact value. Fault: `drawn_probes` skipping every view fails this and five more. + +## probing_a_key_of_twenty_six_coordinates_is_not_exponential + +The old `sub_partials` walked all 2^n masks (93ms at 20 categories, 225ms at 22). With the mask +walk put back, 26 coordinates take 807ms against a 250ms bound; the combination walk takes +under 70ms. The probe set is unchanged for keys of up to 3 coordinates over any bound. + +## the_harness_catches_a_fold_that_covers_a_hidden_item + +improvise-bw4p.3. East hidden, Coastal collapsed: the drawn Coastal/A is 5, not 15. A rival +whose `covered` reads with no hidden items diverges there. Fault in `Eager::covered` (covers +everything) fails this and `eager_matches_the_evaluator_on_the_shipped_documents`, which now +includes the `hidden` document, because no shipped demo hides an item. + +## the_harness_catches_a_total_that_counts_a_hidden_item + +A rival total reading with no hidden items diverges at a column total. The same fault in +`Eager::total` fails this, the next test and the shipped-documents comparison. + +## the_harness_catches_a_total_that_does_not_span_its_rows + +A rival answering `covered` for a column key (so the aggregate over its rows, not one fold over +the pooled rows) is told apart from `total`: with no None category the first has no cell. + +## the_harness_catches_an_occupancy_that_counts_a_hidden_item + +A rival counting stored cells with no hidden filter diverges at a drawn cell's or line's +occupancy. `Eager::occupancy` ignoring the hidden items fails this and the shipped-documents +comparison, whose count is taken independently from `iter_cells` and `matches_partial`. diff --git a/crates/improvise-incremental/src/harness.rs b/crates/improvise-incremental/src/harness.rs index 7d77858c..1e61ddee 100644 --- a/crates/improvise-incremental/src/harness.rs +++ b/crates/improvise-incremental/src/harness.rs @@ -40,7 +40,7 @@ impl Probe { pub struct Divergence { /// 0 is the model as built, `n` the state after edit `n`. pub step: usize, - pub probe: Probe, + pub probe: Box, pub eager: Option, pub candidate: Option, } @@ -123,10 +123,12 @@ fn drop_sets(n: usize, max: usize) -> Vec> { /// Every key `key` keeps after dropping at most `MAX_DROPPED` coordinates: /// polynomial in the key's length, not a walk of all its subsets. fn sub_partials(key: &CellKey) -> impl Iterator + '_ { - drop_sets(key.0.len(), MAX_DROPPED).into_iter().map(move |dropped| { - let kept = (0..key.0.len()).filter(|i| !dropped.contains(i)); - CellKey::new(kept.map(|i| key.0[i].clone()).collect()) - }) + drop_sets(key.0.len(), MAX_DROPPED) + .into_iter() + .map(move |dropped| { + let kept = (0..key.0.len()).filter(|i| !dropped.contains(i)); + CellKey::new(kept.map(|i| key.0[i].clone()).collect()) + }) } /// Every question worth asking of `model`; the bound and the key set are @@ -186,11 +188,13 @@ fn view_probes(view: &View, layout: &GridLayout) -> Vec { all_three(key).into_iter().chain([covered, occupied]) }); let columns = (0..layout.col_count()).flat_map(|c| { - let total = layout.column_key(c).map(|key| { - Probe::Total(key, layout.row_cats.clone(), reading.clone()) - }); + let total = layout + .column_key(c) + .map(|key| Probe::Total(key, layout.row_cats.clone(), reading.clone())); let line = axis_partial(&layout.col_cats, layout.col_tuple(c)); - total.into_iter().chain([Probe::Occupancy(line, reading.clone())]) + total + .into_iter() + .chain([Probe::Occupancy(line, reading.clone())]) }); let rows = (0..layout.row_count()).map(|r| { let line = axis_partial(&layout.row_cats, layout.row_tuple(r)); @@ -219,7 +223,7 @@ fn compare( if !same_value(&want, &got) { return Err(Divergence { step, - probe: probe.clone(), + probe: Box::new(probe.clone()), eager: want, candidate: got, }); diff --git a/crates/improvise-incremental/tests/oracle.rs b/crates/improvise-incremental/tests/oracle.rs index 380ec6d6..1b5d1dd5 100644 --- a/crates/improvise-incremental/tests/oracle.rs +++ b/crates/improvise-incremental/tests/oracle.rs @@ -171,6 +171,14 @@ fn corpus() -> Vec { } } out.push(("hand".into(), hand_model(), vec!["Region".into()], vec![])); + let hidden = hidden_workbook(); + let views = hidden.views.values().cloned().collect(); + out.push(( + "hidden".into(), + hidden.model, + vec!["_Measure".into()], + views, + )); out } @@ -412,7 +420,7 @@ fn the_harness_catches_a_backend_that_ignores_the_aggregation() { let d = run_rival(Fault::IgnoresAggregation).unwrap_err(); assert_eq!(d.step, 0); assert_eq!( - d.probe, + *d.probe, Probe::Value(key("East", "A", "Profit"), Aggregation::OverHidden) ); assert_eq!((d.eager, d.candidate), (None, num(6.0))); @@ -445,7 +453,7 @@ fn the_harness_catches_a_wrong_folded_formula_value() { let d = run_rival(Fault::WrongFoldedFormula).unwrap_err(); assert_eq!(d.step, 0); assert_eq!( - d.probe, + *d.probe, Probe::Value(partial("A", "Profit"), Aggregation::OverHidden) ); assert_eq!((d.eager, d.candidate), (num(9.0), num(-999.0))); @@ -463,7 +471,7 @@ fn the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula() { fn the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped() { let d = run_rival(Fault::WrongWhenTwoAreDropped).unwrap_err(); assert_eq!(d.step, 0); - assert_eq!(d.probe, Probe::Value(only_revenue(), Aggregation::Exact)); + assert_eq!(*d.probe, Probe::Value(only_revenue(), Aggregation::Exact)); assert_eq!(d.candidate, num(-999.0)); } @@ -471,7 +479,7 @@ fn the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped fn the_harness_catches_a_raw_fold_that_returns_a_formula() { let d = run_rival(Fault::RawFoldReturnsFormula).unwrap_err(); assert_eq!(d.step, 0); - assert!(matches!(d.probe, Probe::RawFold(_)), "{d}"); + assert!(matches!(*d.probe, Probe::RawFold(_)), "{d}"); assert_eq!(measure_of(&d.probe), Some("Profit"), "{d}"); assert!(d.eager.is_none() && d.candidate.is_some(), "{d}"); } @@ -486,7 +494,7 @@ fn the_harness_catches_a_wrong_group_fold() { &[], ) .unwrap_err(); - assert_eq!(d.probe, Probe::Value(coastal(), Aggregation::Exact)); + assert_eq!(*d.probe, Probe::Value(coastal(), Aggregation::Exact)); assert_eq!((d.eager, d.candidate), (num(15.0), num(-999.0))); } @@ -798,7 +806,7 @@ fn the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates() { let views: Vec<_> = wb.views.values().cloned().collect(); let d = differential_in(&wb.model, &views, &none, rival, &[]).unwrap_err(); assert_eq!(d.step, 0); - assert_eq!(d.probe, Probe::Value(wide_cell(), Aggregation::Exact)); + assert_eq!(*d.probe, Probe::Value(wide_cell(), Aggregation::Exact)); } /// East is hidden and Coastal is collapsed (a group is, by default), so the Coastal row folds West only; @@ -822,7 +830,8 @@ fn hidden_workbook() -> improvise_core::workbook::Workbook { ("Inland", "A", 2.0), ]; for (r, p, v) in cells { - wb.model.set_cell(key(r, p, "Revenue"), CellValue::Number(v)); + wb.model + .set_cell(key(r, p, "Revenue"), CellValue::Number(v)); } let view = wb.active_view_mut(); view.set_axis("Region", Axis::Row); @@ -897,26 +906,26 @@ fn the_harness_passes_a_view_reader_that_is_right() { #[test] fn the_harness_catches_a_fold_that_covers_a_hidden_item() { let d = run_blind(Some(Blindness::CoversIgnoreHidden)).unwrap_err(); - assert!(matches!(d.probe, Probe::Covered(..)), "{d}"); + assert!(matches!(*d.probe, Probe::Covered(..)), "{d}"); assert_eq!((d.eager, d.candidate), (num(5.0), num(15.0)), "Coastal/A"); } #[test] fn the_harness_catches_a_total_that_counts_a_hidden_item() { let d = run_blind(Some(Blindness::TotalsIgnoreHidden)).unwrap_err(); - assert!(matches!(d.probe, Probe::Total(..)), "{d}"); + assert!(matches!(*d.probe, Probe::Total(..)), "{d}"); } #[test] fn the_harness_catches_a_total_that_does_not_span_its_rows() { let d = run_blind(Some(Blindness::TotalsDoNotSpan)).unwrap_err(); - assert!(matches!(d.probe, Probe::Total(..)), "{d}"); + assert!(matches!(*d.probe, Probe::Total(..)), "{d}"); } #[test] fn the_harness_catches_an_occupancy_that_counts_a_hidden_item() { let d = run_blind(Some(Blindness::OccupancyCountsHidden)).unwrap_err(); - assert!(matches!(d.probe, Probe::Occupancy(..)), "{d}"); + assert!(matches!(*d.probe, Probe::Occupancy(..)), "{d}"); } #[test] @@ -933,12 +942,18 @@ fn a_view_is_drawn_as_cells_totals_and_occupancy() { } #[test] -fn probing_a_key_of_twenty_two_coordinates_is_not_exponential() { - let coords = (0..22).map(|i| (format!("C{i:02}"), "a".to_string())).collect(); +fn probing_a_key_of_twenty_six_coordinates_is_not_exponential() { + let coords = (0..26) + .map(|i| (format!("C{i:02}"), "a".to_string())) + .collect(); let mut m = Model::new("deep"); m.set_cell(CellKey::new(coords), CellValue::Number(1.0)); let started = std::time::Instant::now(); let n = probes(&m).len(); assert!(n > 0); - assert!(started.elapsed().as_millis() < 100, "{:?}", started.elapsed()); + assert!( + started.elapsed().as_millis() < 250, + "{:?}", + started.elapsed() + ); }