fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge gh/main into adr009/delivery

Browse Source
This commit is contained in:
Claude (dbsp-delivery)
2026-10-10 14:18:23 -07:00
committed by Edward Langley
parent a148ab34b3 29f1f85fe1
commit 97f7603c62
10 changed files with 892 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files
context/code/improvise-incremental.md
+43 -18
View File
@@ -32,10 +32,8 @@ fold of the cells that match a partial key") and nothing else. These are the rea
downstream, core cannot read through it, which is the clause's point.
2. **Two methods, the minimum.** `value(key, agg)` is the formula result at a key, or the cell
as stored or folded; `raw_fold(partial)` is the fold of matching stored cells. `agg` is
`Aggregation`, the evaluator's own type. Hidden-item `covers`, totals and occupancy are not
in the trait, so the differential cannot probe them, and a hidden item (view state) cannot
be exercised at all. The ADR's differential compares "every drawn cell, total and occupancy
count"; improvise-bw4p.3 adds them. A group is exercised: a stored key with one coordinate
`Aggregation`, the evaluator's own type. Hidden-item `covers`, totals and occupancy joined the trait
in improvise-bw4p.3 (section "Step 4 groundwork"). A group is exercised: a stored key with one coordinate
replaced by its item's group is a probe.
3. **`raw_fold` is never a formula, and `Eager` enforces it.** For a partial naming a formula
target `Eager::raw_fold` is `None`; otherwise it is `evaluate_aggregated(partial, OverHidden)`,
@@ -105,18 +103,45 @@ coordinates from a stored key is not asked about. `edit_probes`
adds the key an edit names, so a cleared cell is asked about after it is gone. A divergence
reads `divergence at step N: <probe>: eager <a> != candidate <b>`.
**A limit, stated plainly (improvise-bw4p.4).** The probe set is every stored key's
sub-partials, not the keys a view draws. A view with four or more categories on None draws a cell
that drops four or more coordinates of a stored key, and the harness checks none of those cells:
a rival wrong only at `{C0=a, _Measure=Revenue}` over a five-category model passes. Neither
shipped demo reaches the limit, so the corpus does not show it. Raising `MAX_DROPPED` is not the
answer, because `sub_partials` walks all 2^n subsets of a key whatever the limit is (93ms at 20
categories and 225ms at 22 in a debug build, per call). The fix is to probe the keys each view
draws, with steps 3 and 4; until then a candidate on such a view is under-checked.
## Step 4 groundwork: what a view draws (improvise-bw4p.3, improvise-bw4p.4)
The corpus the tests use is `examples/demo.improv` and `web/demo.improv`, loaded through
`improvise_io::persistence::load`, and hand-built models (formulas, text, mixed values, a group).
`eager_matches_the_evaluator_on_the_shipped_documents` takes every view's None categories; the
differential runs take the first per document, and the largest document only its first four
edits, to keep the binary under two seconds in a debug build. A proptest (256 cases) runs random
edit sequences against a candidate that rebuilds an `Eager` from scratch on each edit.
The subset probes above cannot see a drawn cell on a view with four or more None categories
(it drops more than `MAX_DROPPED` coordinates of a stored key), and the trait could not be asked
about hidden items, totals or occupancy. Both are closed. These are the readings taken.
1. **`Reading` is how a view reads.** The aggregation (`Aggregation::of` its None categories)
and the hidden `(category, item)` pairs of `View::hidden_items`. `covers(cat, item)` is the
predicate `GridLayout::evaluator` passes as its `covers`. It is a value, not a closure, so a
probe can name it and a divergence can print it. A fold over a group or a spanned category
leaves hidden members out; a read of a key naming neither does not (a hidden East still counts
in `{Product=A}` when Region is None), while an occupancy count always leaves them out. The
count follows the drawn items, not the value; `Eager` mirrors production in both.
2. **Three methods join the trait**: `covered(key, reading)` is `Model::evaluate_covering`;
`total(key, over, reading)` is `evaluate_spanning`, a column total as ONE fold over the
visible items of the omitted categories; `occupancy(partial, reading)` is ADR 009 clause 7's
count by axis item: the stored cells matching `partial`, none of whose coordinates is hidden,
where a coordinate naming a group matches every stored member of that group (a collapsed group
is a remap, so the Coastal line of two visible members counts 2). A key naming a formula
target has no stored cells and is not asked. "Has a value" (what `prune_empty` asks) is
`covered(..).is_some()`, already probed.
3. **`drawn_probes(model, views, none_cats)` asks what each view draws**, built from the view's
own `GridLayout` (pruned as production builds it): for every drawn cell the three subset
questions plus `Covered` and `Occupancy`; for every column its `column_key` occupancy and, where
`GridLayout::draws_totals` (the condition `grid_body` also reads), a `Total` over the row
categories; for every row the occupancy of its cell key less the column coordinates. Lines
carry the page coordinates, as a cell's key does, so a page is its own slice. A records view is
skipped: its keys are synthetic `_Index`/`_Dim` coordinates naming no stored cell and it has
no totals, while its stored keys are the subset probes' business. A view whose None categories
differ from the oracle's is skipped, since `Eager` recomputes its formulas under one set (a
backend keeping a cache per aggregation class is therefore not exercised on the others).
`differential_in(model, views, ..)` adds them at every step; `differential` is that with no
views. A view is a `View`, not a document, because the model holds none.
4. **`sub_partials` is polynomial**: it enumerates the sets of at most `MAX_DROPPED` dropped
indices, not the 2^n masks. The probe set is unchanged for every key length (a test compares
it with the old walk up to 12 coordinates, and counts it at 64). `MAX_DROPPED` stays: it
bounds the subset probes, and the drawn probes carry the cells that drop more.
5. **The hidden fixture is a workbook the corpus now includes** (`hidden`): no shipped demo
hides an item, so a backend wrong about hidden items would have passed the direct
`Eager`-against-evaluator comparison as well.
6. **`Probe` is `Hash + Eq`**, so `Asked` deduplicates on the probe itself (`Aggregation` gained
`Hash` in core for it).
context/test-evidence/improvise-incremental.md
+114 -26
View File
@@ -51,142 +51,142 @@ none did):
- `Edit::ClearCell` applied as `SetCell(0.0)`, which both sides of a differential share and so
only a direct `Eager` test can see: `eager_clear_cell_leaves_the_key_absent`.
## eager_raw_fold_sums_by_default
### eager_raw_fold_sums_by_default
Hand-computed: Revenue under Product A is 10 + 5 = 15. Fails on the `raw_fold`-reads-`Exact`
rival (a partial key names no stored cell, so the answer is `None`).
## eager_raw_fold_follows_the_measures_aggregation
### eager_raw_fold_follows_the_measures_aggregation
After `measure_agg[Revenue] = Max` the same fold is 10, and Cost, unchanged, still sums to 6.
A fold that always sums fails on the first; one that always takes the maximum fails on the second.
## eager_raw_fold_of_text_is_unanimous_or_mixed
### eager_raw_fold_of_text_is_unanimous_or_mixed
Note under A is two equal texts and reads as that text; under B it is two different texts and
reads `Error("mixed")`. A fold that drops text, which the evaluator used to do (gotcha 7),
returns `None` for both.
## eager_numbers_among_text_are_mixed
### eager_numbers_among_text_are_mixed
A text among numbers is `Error("mixed")`, not the sum of the numbers.
## eager_value_reads_a_formula_exactly_and_folded
### eager_value_reads_a_formula_exactly_and_folded
`Profit = Revenue - Cost` reads 6 at East/A exactly, and 9 at Product A with Region over
hidden: (10 + 5) - (4 + 2), the fold taken before the subtraction. Fails on the
`value`-ignores-`Aggregation` rival.
## eager_follows_an_edit_to_the_cell_a_fold_reads
### eager_follows_an_edit_to_the_cell_a_fold_reads
After East/A Revenue becomes 20 the fold is 25 and the folded Profit 19. The Profit half is
what fails when `apply` forgets to recompute formulas: the folded cache is then stale.
## eager_matches_the_evaluator_on_the_shipped_documents
### eager_matches_the_evaluator_on_the_shipped_documents
Both shipped documents, every view's None categories, and the hand model: every probe of
`Eager` equals a direct `evaluate_aggregated` on a model recomputed under the same None
categories, and the probe set is non-empty. Fails on both aggregation rivals.
## eager_agrees_with_itself_over_every_corpus_edit_sequence
### eager_agrees_with_itself_over_every_corpus_edit_sequence
`differential` over a generated edit sequence on each corpus model returns a `Report` with
`edits + 1` steps and a non-zero probe count. The count assertion is what a harness that
compared nothing would fail; the step count is what one that stopped early would fail.
## the_harness_passes_a_backend_that_is_right
### the_harness_passes_a_backend_that_is_right
The control for the `catches` tests: `Eager` as the candidate over the hand-built edits is a
pass, so a harness that rejects everything is seen.
## the_harness_catches_a_backend_that_ignores_the_aggregation
### the_harness_catches_a_backend_that_ignores_the_aggregation
A candidate that reads every `value` as `Exact` diverges at step 0, first at the full-key
formula cell `{East, A, Profit}` over hidden (eager `None`, candidate `6`).
## the_harness_catches_a_backend_that_ignores_the_measure_aggregation
### the_harness_catches_a_backend_that_ignores_the_measure_aggregation
A candidate that drops `SetMeasureAgg` diverges at step 2, the edit that sets it, on a Revenue key.
## the_harness_catches_a_backend_that_goes_stale_after_two_edits
### the_harness_catches_a_backend_that_goes_stale_after_two_edits
A candidate that stops applying edits after the second diverges at step 3, on a Revenue key.
A prefix of steps cannot see a backend that goes wrong late.
## the_harness_catches_a_backend_that_ignores_a_clear
### the_harness_catches_a_backend_that_ignores_a_clear
A candidate that keeps a cleared cell diverges at step 3 and the message names it.
## the_harness_catches_a_wrong_folded_formula_value
### the_harness_catches_a_wrong_folded_formula_value
Review of PR 343 (Codex P1): a rival wrong only at `value({Product=A, _Measure=Profit}, OverHidden)`,
the Profit cell a view with Region on None draws, passed 128 probes, because formula cells were
built over full keys only. Now the divergence is at step 0 on exactly that probe, eager `9`,
candidate `-999`.
## the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula
### the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula
Review of PR 343: probes were rebuilt from the post-edit model, so a candidate that ignores
`RemoveFormula` was never asked about Profit again. Probes now accumulate over every step. The
divergence is at the last step, on a Profit key, eager `None`, candidate `Some`.
## the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped
### the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped
Review of PR 343 (Copilot): only one-coordinate partials and the empty key were probed. A rival
wrong only at `{_Measure=Revenue}` now diverges at step 0 on `Value({_Measure=Revenue}, Exact)`.
The bound is `MAX_DROPPED = 3` coordinates; a fold dropping more is not asked about.
## the_harness_catches_a_raw_fold_that_returns_a_formula
### the_harness_catches_a_raw_fold_that_returns_a_formula
Review of PR 343 (Copilot): `raw_fold` is documented never to be a formula. A rival whose
`raw_fold` is `value(partial, OverHidden)` diverges at step 0 on a `RawFold` of a Profit partial
(eager `None`, candidate `Some`). It could not be seen while `Eager` itself returned the formula.
## the_harness_catches_a_wrong_group_fold
### the_harness_catches_a_wrong_group_fold
A group name stands in for an item in the probe keys. A rival wrong only at
`{Region=Coastal, Product=A, _Measure=Revenue}` diverges at step 0, eager `15`, candidate `-999`.
## eager_folds_a_group_it_is_asked_about
### eager_folds_a_group_it_is_asked_about
The group fold in the grouped model is East + West = 15 exactly, so the rival test above has a
true answer to differ from.
## eager_raw_fold_is_never_a_formula
### eager_raw_fold_is_never_a_formula
At `{Product=A, _Measure=Profit}` the folded value is `9` and the raw fold is `None`. Fails on the
fault that lets `raw_fold` read the formula cache.
## eager_add_and_remove_formula_change_what_a_key_reads
### eager_add_and_remove_formula_change_what_a_key_reads
Review of PR 343: `Edit::AddFormula` as a no-op left all 25 tests green, because an `Edit` fault
reaches both sides of a differential alike. `Double = Revenue * 2` reads `None` before the edit
and `20` at East/A after it, and `None` again after `RemoveFormula`. Fails on `AddFormula` as a
no-op.
## eager_clear_cell_leaves_the_key_absent
### eager_clear_cell_leaves_the_key_absent
After `ClearCell` the key reads `None` and the model holds no cell. A `ClearCell` applied as
`SetCell(0.0)` hits both sides of a differential identically, so only this direct test fails.
## same_value_follows_the_float_rules_and_is_exact_elsewhere
### same_value_follows_the_float_rules_and_is_exact_elsewhere
NaN equals NaN, `-0.0` equals `0.0`, `1.0` differs from `1.0000000000000002`, texts and errors
compare exactly, and `None` equals only `None`. Each of three one-line faults in `same_value`
(NaN unequal, signed zero by bits, any two texts equal) survived the whole suite until this test
existed.
## probes_cover_folds_formulas_measures_and_groups
### probes_cover_folds_formulas_measures_and_groups
`probes` includes a raw fold, a full key, a folded formula cell under both aggregations, the
two-dropped key `{_Measure=Revenue}`, the empty key and a group key. Review of PR 343 showed
`probes` omitting formula cells, `_Measure` keys or raw folds left every test green.
## an_edit_names_its_key_as_two_value_probes_and_a_fold
### an_edit_names_its_key_as_two_value_probes_and_a_fold
`edit_probes` of a `ClearCell` names the cleared key three ways, and of an edit naming no key
names nothing.
## a_maintained_eager_equals_a_fresh_one_over_random_edit_sequences
### a_maintained_eager_equals_a_fresh_one_over_random_edit_sequences
Clause 2 asks for a property test over a random edit sequence. A proptest (256 cases, up to four
edits of five kinds over the hand-built model) runs `differential` with a candidate that rebuilds
@@ -215,3 +215,91 @@ The differential for step 3. A proptest (256 cases, up to 12 edits drawn from se
both zeros and three NaN spellings) checks that the delta `DataStore` logs for an edit equals
the store as a Z-set after it minus the store before. No `Derived` backend exists yet, so
this is the harness's step-3 form: the delta against the store it describes.
### the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates
improvise-bw4p.4. Six categories, four on None, a rival wrong only at `{C0=a, _Measure=Revenue}`.
Before the drawn probes the differential passed it (`is_err` failed); the cell drops four
coordinates of a stored key and `MAX_DROPPED` is three. Now the first divergence is the
drawn cell's exact value. Fault: `drawn_probes` skipping every view fails this and five more.
### harness::tests::sub_partials_are_the_keys_the_old_mask_walk_gave
The subset walk no longer enumerates 2^n masks. For every key of up to 12 coordinates its keys
equal the old mask walk's as sorted sets, so only the order changed. Fault: dropping at most
two coordinates instead of three fails this and the next test. No wall clock is read.
### harness::tests::sub_partials_of_sixty_four_coordinates_number_the_choices_of_at_most_three
At 64 coordinates the old `1u64 << n` cannot run, so only the combination walk answers; its
count must be the sum of C(64, k) for k up to 3, 43745. This replaces a 250ms timing bound
(removed: a debug-build wall clock fails on a slow runner and the old walk passes in release).
### the_harness_catches_a_fold_that_covers_a_hidden_item
improvise-bw4p.3. East hidden, Coastal collapsed: the drawn Coastal/A is 5, not 15. A rival
whose `covered` reads with no hidden items diverges there. Fault in `Eager::covered` (covers
everything) fails this and `eager_matches_the_evaluator_on_the_shipped_documents`, which now
includes the `hidden` document, because no shipped demo hides an item.
### the_harness_catches_a_total_that_counts_a_hidden_item
A rival total reading with no hidden items diverges at a column total. The same fault in
`Eager::total` fails this, the next test and the shipped-documents comparison.
### the_harness_catches_a_total_that_does_not_span_its_rows
Nothing is hidden and Margin is `Revenue / Cost`, so only the span can separate the rival:
the pooled total of `{_Measure=Margin}` over the rows is 15 / 6 for eager and `None` for a
backend answering `covered` at a column key. The first version used the hidden fixture and
failed through the hidden East instead; it passed once nothing was hidden. Fault: `Eager::total`
ignoring hidden items does not fail this test, as it must not.
### the_harness_catches_an_occupancy_that_counts_a_hidden_item
A rival counting stored cells with no hidden filter diverges at a drawn cell's or line's
occupancy. `Eager::occupancy` ignoring the hidden items fails this and the shipped-documents
comparison, whose count is taken independently from `iter_cells` and `matches_partial`.
### a_collapsed_groups_occupancy_counts_its_visible_members
Occupancy is keyed by axis item and a collapsed group is a remap to its visible members. The
Coastal line draws two cells (West's, East being hidden) and `Eager` once counted 0 because it
matched the group name literally. Fault: matching literally fails this test.
### eager_reads_what_the_grid_evaluator_reads
Compares `Eager::covered` and `total` with `GridLayout::evaluator`'s own `cell` and
`column_total` for every drawn cell and total, so the `Reading` is checked against the layout's
hidden items and not against the same `Model` call. Fault: `Reading::of_view` dropping every
hidden item fails this and four others; before this test the shipped-documents comparison passed it.
### a_page_selection_narrows_the_line_occupancy_it_is_asked_under
Lines carry the page coordinates, as a drawn cell's key does. Fault: building the row line from
the row categories alone asks `{Product=A}` across every page and fails this test.
### every_row_and_column_line_is_asked_its_occupancy
Pins the row-line probes (Inland, and the collapsed Coastal) and the column line. Fault: dropping
the row lines fails this test and the pruning test.
### a_pruned_view_is_asked_only_about_the_lines_it_draws
A row with no cells is asked about unpruned and not once `prune_empty` is set. Fault: building the
layout with `GridLayout::new`, which does not prune, fails it.
### a_records_view_is_not_asked_about
A records view draws synthetic `_Index`/`_Dim` keys that name no stored cell and has no totals;
its stored keys are asked by the subset probes. Fault: not skipping it fails this test.
### no_total_is_asked_where_the_grid_draws_none
`GridLayout::draws_totals` is the one condition `grid_body` and `drawn_probes` share. With
`_Measure` on the rows a column mixes measures and has no total. Fault: asking totals always fails it.
### a_view_under_another_none_set_is_not_asked
The oracle recomputes its formulas under one None set; a view under another is not asked, since
the oracle would answer it from the wrong cache. Fault: dropping the filter fails this test.
crates/improvise-command/src/ui/viewport.rs
+1 -4
View File
@@ -7,7 +7,6 @@ use improvise_protocol::header_level_widths;
use crate::format::{FormatPlan, NumberFormat};
use crate::model::Model;
use crate::model::category::MEASURE_CATEGORY;
use crate::model::cell::{CellKey, CellValue};
use crate::view::{AxisEntry, GridEval, GridLayout};
@@ -50,9 +49,7 @@ pub fn grid_body(
// evaluated once (improvise-h6z1). The model does not change during it.
let eval = layout.evaluator(model);
// A column mixing measures has no total to show: `context/code/command.md`.
let measures_on_rows = layout.row_cats.iter().any(|c| c == MEASURE_CATEGORY);
let totalled = !records && !measures_on_rows && layout.row_count() > 0 && n > 0;
let totals: Vec<Option<CellValue>> = match totalled {
let totals: Vec<Option<CellValue>> = match layout.draws_totals() {
true => (0..n).map(|ci| eval.column_total(ci)).collect(),
false => Vec::new(),
};
crates/improvise-core/src/model/types.rs
+1 -1
View File
@@ -339,7 +339,7 @@ fn eval_is(coordinate: Option<&str>, filter: &Filter) -> f64 {
/// Whether a grid hides dimensions a cell key does not name. The emptiness of
/// the view's `Axis::None` list is the whole question — `context/code/core.md`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Aggregation {
/// The key names every dimension, so the cell is read as stored.
Exact,
crates/improvise-core/src/view/layout.rs
+7
View File
@@ -991,6 +991,13 @@ impl GridLayout {
Some(CellKey::new(coords))
}
/// Whether the grid draws a total under each column: not in records mode,
/// not when a column mixes measures (`_Measure` on the rows), not when empty.
pub fn draws_totals(&self) -> bool {
let measures_on_rows = self.row_cats.iter().any(|c| c == MEASURE_CATEGORY);
!self.is_records_mode() && !measures_on_rows && self.row_count() > 0 && self.col_count() > 0
}
/// The coordinates a COLUMN answers for once its rows are aggregated away:
/// the page/filter constraints and the column's own tuple (core.md §6).
pub fn column_key(&self, col: usize) -> Option<CellKey> {
crates/improvise-incremental/src/eager.rs
+37 -1
View File
@@ -1,6 +1,6 @@
//! Today's evaluator behind the trait. Temporary (ADR 009 clause 2).
use crate::{Derived, Edit, Maintained};
use crate::{Derived, Edit, Maintained, Reading};
use improvise_core::model::cell::{CellKey, CellValue};
use improvise_core::model::{Aggregation, Model};
@@ -23,6 +23,17 @@ impl Eager {
}
}
impl Eager {
/// Whether `item` names a group of `cat` that covers `held`, a stored item.
fn group_holds(&self, cat: &str, item: &str, held: Option<&str>) -> bool {
let Some(category) = self.model.category(cat) else {
return false;
};
let members = category.items_in_group(item);
!category.items.contains_key(item) && held.is_some_and(|h| members.contains(&h))
}
}
impl Derived for Eager {
fn value(&self, key: &CellKey, agg: Aggregation) -> Option<CellValue> {
self.model.evaluate_aggregated(key, agg)
@@ -36,6 +47,31 @@ impl Derived for Eager {
.evaluate_aggregated(partial, Aggregation::OverHidden),
}
}
fn covered(&self, key: &CellKey, reading: &Reading) -> Option<CellValue> {
let covers = |cat: &str, item: &str| reading.covers(cat, item);
self.model.evaluate_covering(key, reading.agg, &covers)
}
fn total(&self, key: &CellKey, over: &[String], reading: &Reading) -> Option<CellValue> {
let covers = |cat: &str, item: &str| reading.covers(cat, item);
self.model
.evaluate_spanning(key, over, reading.agg, &covers)
}
fn occupancy(&self, partial: &CellKey, reading: &Reading) -> usize {
let drawn = |(cat, item): &(String, String)| reading.covers(cat, item);
let named_by = |stored: &CellKey| {
partial.0.iter().all(|(cat, item)| {
let held = stored.get(cat);
held == Some(item.as_str()) || self.group_holds(cat, item, held)
})
};
let cells = self.model.data.iter_cells();
cells
.filter(|(k, _)| named_by(k) && k.0.iter().all(drawn))
.count()
}
}
impl Maintained for Eager {
crates/improvise-incremental/src/harness.rs
+147 -20
View File
@@ -1,8 +1,9 @@
//! The differential harness: one edit sequence, two backends, every probe after every step.
use crate::{Derived, Eager, Edit};
use crate::{Derived, Eager, Edit, Reading};
use improvise_core::model::cell::{CellKey, CellValue};
use improvise_core::model::{Aggregation, Model};
use improvise_core::view::{GridLayout, View};
use std::collections::HashSet;
use std::fmt;
@@ -12,10 +13,13 @@ pub trait Maintained: Derived {
}
/// One question put to both backends.
#[derive(Debug, Clone, PartialEq)]
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum Probe {
Value(CellKey, Aggregation),
RawFold(CellKey),
Covered(CellKey, Reading),
Total(CellKey, Vec<String>, Reading),
Occupancy(CellKey, Reading),
}
impl Probe {
@@ -23,6 +27,11 @@ impl Probe {
match self {
Probe::Value(key, agg) => backend.value(key, *agg),
Probe::RawFold(partial) => backend.raw_fold(partial),
Probe::Covered(key, reading) => backend.covered(key, reading),
Probe::Total(key, over, reading) => backend.total(key, over, reading),
Probe::Occupancy(key, reading) => {
Some(CellValue::Number(backend.occupancy(key, reading) as f64))
}
}
}
}
@@ -31,7 +40,7 @@ impl Probe {
pub struct Divergence {
/// 0 is the model as built, `n` the state after edit `n`.
pub step: usize,
pub probe: Probe,
pub probe: Box<Probe>,
pub eager: Option<CellValue>,
pub candidate: Option<CellValue>,
}
@@ -72,19 +81,14 @@ pub const MAX_DROPPED: usize = 3;
/// The questions asked of a backend, once each, in the order first added.
#[derive(Default)]
struct Asked {
seen: HashSet<(CellKey, u8)>,
seen: HashSet<Probe>,
order: Vec<Probe>,
}
impl Asked {
fn extend(&mut self, probes: impl IntoIterator<Item = Probe>) {
for p in probes {
let tag = match &p {
Probe::Value(k, Aggregation::Exact) => (k.clone(), 0),
Probe::Value(k, Aggregation::OverHidden) => (k.clone(), 1),
Probe::RawFold(k) => (k.clone(), 2),
};
if self.seen.insert(tag) {
if self.seen.insert(p.clone()) {
self.order.push(p);
}
}
@@ -99,16 +103,31 @@ fn all_three(key: CellKey) -> [Probe; 3] {
]
}
/// Every key `key` keeps after dropping at most `MAX_DROPPED` coordinates.
/// Every set of at most `max` indices below `n`, ascending within each set.
fn drop_sets(n: usize, max: usize) -> Vec<Vec<usize>> {
let mut all = vec![vec![]];
let mut level: Vec<Vec<usize>> = vec![vec![]];
for _ in 0..max {
level = level
.iter()
.flat_map(|set| {
let from = set.last().map_or(0, |last| last + 1);
(from..n).map(move |i| [set.as_slice(), &[i]].concat())
})
.collect();
all.extend(level.iter().cloned());
}
all
}
/// Every key `key` keeps after dropping at most `MAX_DROPPED` coordinates:
/// polynomial in the key's length, not a walk of all its subsets.
fn sub_partials(key: &CellKey) -> impl Iterator<Item = CellKey> + '_ {
let n = key.0.len();
(0..1u64 << n)
.filter(|mask| mask.count_ones() as usize <= MAX_DROPPED)
.map(move |mask| {
let kept = (0..n)
.filter(|i| mask >> i & 1 == 0)
.map(|i| key.0[i].clone());
CellKey::new(kept.collect())
drop_sets(key.0.len(), MAX_DROPPED)
.into_iter()
.map(move |dropped| {
let kept = (0..key.0.len()).filter(|i| !dropped.contains(i));
CellKey::new(kept.map(|i| key.0[i].clone()).collect())
})
}
@@ -140,6 +159,63 @@ pub fn probes(model: &Model) -> Vec<Probe> {
asked.order
}
/// What each view under `none_cats` draws of `model`: cells, totals, occupancy.
/// Why, and what is skipped: `context/code/improvise-incremental.md`, step 4.
pub fn drawn_probes(model: &Model, views: &[View], none_cats: &[String]) -> Vec<Probe> {
let sorted = |cats: &[String]| {
let mut cats = cats.to_vec();
cats.sort();
cats
};
let oracle_set = sorted(none_cats);
views
.iter()
.map(|view| (view, GridLayout::with_frozen_records(model, view, None)))
.filter(|(_, layout)| !layout.is_records_mode())
.filter(|(_, layout)| sorted(&layout.none_cats) == oracle_set)
.flat_map(|(view, layout)| view_probes(model, view, &layout))
.collect()
}
/// A stored cell count is asked of a key only if something is stored under it.
fn occupancy_probe(model: &Model, key: CellKey, reading: &Reading) -> Option<Probe> {
model
.formula_for(&key)
.is_none()
.then(|| Probe::Occupancy(key, reading.clone()))
}
fn view_probes(model: &Model, view: &View, layout: &GridLayout) -> Vec<Probe> {
let reading = Reading::of_view(view, &layout.none_cats);
let cells = (0..layout.row_count())
.flat_map(|r| (0..layout.col_count()).map(move |c| (r, c)))
.filter_map(|(r, c)| layout.cell_key(r, c))
.flat_map(|key| {
let covered = Probe::Covered(key.clone(), reading.clone());
let occupied = occupancy_probe(model, key.clone(), &reading);
all_three(key).into_iter().chain([covered]).chain(occupied)
});
let columns = (0..layout.col_count()).flat_map(|c| {
let key = layout.column_key(c);
let total = key.clone().filter(|_| layout.draws_totals());
let total = total.map(|k| Probe::Total(k, layout.row_cats.clone(), reading.clone()));
let line = key.and_then(|k| occupancy_probe(model, k, &reading));
total.into_iter().chain(line)
});
let rows = (0..layout.row_count()).flat_map(|r| {
let line = layout
.cell_key(r, 0)
.map(|k| without_all(k, &layout.col_cats));
line.and_then(|k| occupancy_probe(model, k, &reading))
});
cells.chain(columns).chain(rows).collect()
}
/// `key` less each of `cats`: a row's line is a cell without its column coordinates.
fn without_all(key: CellKey, cats: &[String]) -> CellKey {
cats.iter().fold(key, |k, cat| k.without(cat))
}
/// What an edit names: its key, asked all three ways, after the edit.
pub fn edit_probes(edit: &Edit) -> Vec<Probe> {
edit.key()
@@ -160,7 +236,7 @@ fn compare(
if !same_value(&want, &got) {
return Err(Divergence {
step,
probe: probe.clone(),
probe: Box::new(probe.clone()),
eager: want,
candidate: got,
});
@@ -174,17 +250,30 @@ fn compare(
pub fn differential<C: Maintained>(
model: &Model,
none_cats: &[String],
candidate: C,
edits: &[Edit],
) -> Result<Report, Divergence> {
differential_in(model, &[], none_cats, candidate, edits)
}
/// [`differential`], also asking everything `views` draw (`drawn_probes`).
pub fn differential_in<C: Maintained>(
model: &Model,
views: &[View],
none_cats: &[String],
mut candidate: C,
edits: &[Edit],
) -> Result<Report, Divergence> {
let mut eager = Eager::new(model.clone(), none_cats.to_vec());
let mut asked = Asked::default();
asked.extend(probes(eager.model()));
asked.extend(drawn_probes(eager.model(), views, none_cats));
let mut probes_run = compare(0, &eager, &candidate, &asked)?;
for (n, edit) in edits.iter().enumerate() {
eager.apply(edit);
candidate.apply(edit);
asked.extend(probes(eager.model()));
asked.extend(drawn_probes(eager.model(), views, none_cats));
asked.extend(edit_probes(edit));
probes_run += compare(n + 1, &eager, &candidate, &asked)?;
}
@@ -193,3 +282,41 @@ pub fn differential<C: Maintained>(
probes: probes_run,
})
}
#[cfg(test)]
mod tests {
use super::*;
fn old_mask_walk(key: &CellKey) -> Vec<CellKey> {
let n = key.0.len();
(0..1u64 << n)
.filter(|mask| mask.count_ones() as usize <= MAX_DROPPED)
.map(|mask| {
let kept = (0..n).filter(|i| mask >> i & 1 == 0);
CellKey::new(kept.map(|i| key.0[i].clone()).collect())
})
.collect()
}
fn key_of(n: usize) -> CellKey {
CellKey::new((0..n).map(|i| (format!("C{i:02}"), "a".into())).collect())
}
#[test]
fn sub_partials_are_the_keys_the_old_mask_walk_gave() {
for n in 0..=12 {
let mut now: Vec<_> = sub_partials(&key_of(n)).collect();
let mut was = old_mask_walk(&key_of(n));
now.sort();
was.sort();
assert_eq!(now, was, "{n} coordinates");
}
}
#[test]
fn sub_partials_of_sixty_four_coordinates_number_the_choices_of_at_most_three() {
let n = 64;
let choices = 1 + n + n * (n - 1) / 2 + n * (n - 1) * (n - 2) / 6;
assert_eq!(sub_partials(&key_of(n)).count(), choices);
}
}
crates/improvise-incremental/src/lib.rs
+16 -3
View File
@@ -5,19 +5,21 @@ mod delta;
mod eager;
mod edit;
mod harness;
mod reading;
pub use delta::{CellDelta, DeltaValue, cell_delta, moved_records};
pub use eager::Eager;
pub use edit::Edit;
pub use harness::{
Divergence, MAX_DROPPED, Maintained, Probe, Report, differential, edit_probes, probes,
same_value,
Divergence, MAX_DROPPED, Maintained, Probe, Report, differential, differential_in,
drawn_probes, edit_probes, probes, same_value,
};
pub use reading::Reading;
use improvise_core::model::Aggregation;
use improvise_core::model::cell::{CellKey, CellValue};
/// The two things the evaluator reads (ADR 009 clause 2).
/// What the evaluator reads (ADR 009 clause 2) and what a view draws from it.
pub trait Derived {
/// The value at `key`: the formula result when a formula owns it, else the
/// cell as stored or folded over what `agg` hides. `Model::evaluate_aggregated`.
@@ -26,4 +28,15 @@ pub trait Derived {
/// The raw fold of the stored cells that match `partial`, by the measure's
/// aggregation. Never a formula: `None` for a partial naming a formula target.
fn raw_fold(&self, partial: &CellKey) -> Option<CellValue>;
/// `key` as a view reads it: a collapsed group folds only the members
/// `reading` leaves visible. `Model::evaluate_covering`.
fn covered(&self, key: &CellKey, reading: &Reading) -> Option<CellValue>;
/// A column's total: `key` as ONE fold over every visible item of the
/// `over` categories it omits. `Model::evaluate_spanning`.
fn total(&self, key: &CellKey, over: &[String], reading: &Reading) -> Option<CellValue>;
/// How many stored cells match `partial` with no hidden coordinate.
fn occupancy(&self, partial: &CellKey, reading: &Reading) -> usize;
}
crates/improvise-incremental/src/reading.rs
+36
View File
@@ -0,0 +1,36 @@
//! What a view reads a backend under: the aggregation and the items it hides.
use improvise_core::model::Aggregation;
use improvise_core::view::View;
use std::collections::BTreeSet;
/// One view's aggregation and hidden `(category, item)` pairs; which reads
/// leave them out is in `context/code/improvise-incremental.md`, step 4.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Reading {
pub agg: Aggregation,
hidden: BTreeSet<(String, String)>,
}
impl Reading {
pub fn new(agg: Aggregation, hidden: impl IntoIterator<Item = (String, String)>) -> Self {
Reading {
agg,
hidden: hidden.into_iter().collect(),
}
}
/// The reading of `view`, whose None categories are `none_cats`.
pub fn of_view(view: &View, none_cats: &[String]) -> Self {
let hidden = view
.hidden_items
.iter()
.flat_map(|(cat, items)| items.iter().map(|i| (cat.clone(), i.clone())));
Self::new(Aggregation::of(none_cats), hidden)
}
/// Whether the view draws `item` of `cat`.
pub fn covers(&self, cat: &str, item: &str) -> bool {
!self.hidden.contains(&(cat.to_string(), item.to_string()))
}
}
crates/improvise-incremental/tests/oracle.rs
+490 -16
View File
@@ -5,10 +5,10 @@ use improvise_core::formula::ast::AggFunc;
use improvise_core::formula::parse_formula;
use improvise_core::model::cell::{CellKey, CellValue};
use improvise_core::model::{Aggregation, Model};
use improvise_core::view::Axis;
use improvise_core::view::{Axis, GridLayout, View};
use improvise_incremental::{
CellDelta, DeltaValue, Derived, Divergence, Eager, Edit, Maintained, Probe, cell_delta,
differential, probes, same_value,
CellDelta, DeltaValue, Derived, Divergence, Eager, Edit, Maintained, Probe, Reading,
cell_delta, differential, differential_in, drawn_probes, probes, same_value,
};
use proptest::prelude::*;
use std::path::Path;
@@ -149,7 +149,9 @@ fn eager_numbers_among_text_are_mixed() {
assert_eq!(eager.raw_fold(&partial("B", "Revenue")), mixed());
}
fn corpus() -> Vec<(String, Model, Vec<String>)> {
type Doc = (String, Model, Vec<String>, Vec<View>);
fn corpus() -> Vec<Doc> {
let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let mut out = Vec::new();
for file in ["examples/demo.improv", "web/demo.improv"] {
@@ -163,11 +165,16 @@ fn corpus() -> Vec<(String, Model, Vec<String>)> {
.collect();
if !seen.contains(&none) {
seen.push(none.clone());
out.push((format!("{file}:{name}"), wb.model.clone(), none));
let views = wb.views.values().cloned().collect();
out.push((format!("{file}:{name}"), wb.model.clone(), none, views));
}
}
}
out.push(("hand".into(), hand_model(), vec!["Region".into()]));
out.push(("hand".into(), hand_model(), vec!["Region".into()], vec![]));
let hidden = hidden_workbook();
let views = hidden.views.values().cloned().collect();
let none = none_of(&hidden);
out.push(("hidden".into(), hidden.model, none, views));
out
}
@@ -203,17 +210,36 @@ fn edits_for(model: &Model) -> Vec<Edit> {
#[test]
fn eager_matches_the_evaluator_on_the_shipped_documents() {
for (name, model, none) in corpus() {
for (name, model, none, views) in corpus() {
let eager = Eager::new(model.clone(), none.clone());
let mut direct = model.clone();
direct.recompute_formulas(&none);
let asked = probes(eager.model());
let mut asked = probes(eager.model());
assert!(!asked.is_empty(), "{name}: no probes");
asked.extend(drawn_probes(eager.model(), &views, &none));
for probe in asked {
let want = match &probe {
Probe::Value(k, a) => direct.evaluate_aggregated(k, *a),
Probe::RawFold(k) if direct.formula_for(k).is_some() => None,
Probe::RawFold(k) => direct.evaluate_aggregated(k, Aggregation::OverHidden),
Probe::Covered(k, r) => direct.evaluate_covering(k, r.agg, &|c, i| r.covers(c, i)),
Probe::Total(k, over, r) => {
direct.evaluate_spanning(k, over, r.agg, &|c, i| r.covers(c, i))
}
Probe::Occupancy(p, r) => {
let drawn = |(c, i): &(String, String)| r.covers(c, i);
let names = |k: &CellKey| {
p.0.iter().all(|(c, i)| {
let held = k.get(c);
let members = direct.category(c).map(|x| x.items_in_group(i));
held == Some(i.as_str())
|| held.is_some_and(|h| members.is_some_and(|m| m.contains(&h)))
})
};
let stored = direct.data.iter_cells().map(|(k, _)| k);
let n = stored.filter(|k| names(k) && k.0.iter().all(drawn));
Some(CellValue::Number(n.count() as f64))
}
};
assert_eq!(probe.ask(&eager), want, "{name}: {probe:?}");
}
@@ -223,7 +249,7 @@ fn eager_matches_the_evaluator_on_the_shipped_documents() {
#[test]
fn eager_agrees_with_itself_over_every_corpus_edit_sequence() {
let mut files_done: Vec<String> = Vec::new();
for (name, model, none) in corpus() {
for (name, model, none, views) in corpus() {
let file = name.split(':').next().unwrap().to_string();
if files_done.contains(&file) {
continue;
@@ -234,8 +260,9 @@ fn eager_agrees_with_itself_over_every_corpus_edit_sequence() {
if model.data.iter_cells().count() > 40 {
edits.truncate(4);
}
let report = differential(
let report = differential_in(
&model,
&views,
&none,
Eager::new(model.clone(), none.clone()),
&edits,
@@ -327,6 +354,18 @@ impl Derived for Rival {
_ => self.inner.raw_fold(p),
}
}
fn covered(&self, k: &CellKey, r: &Reading) -> Option<CellValue> {
self.inner.covered(k, r)
}
fn total(&self, k: &CellKey, over: &[String], r: &Reading) -> Option<CellValue> {
self.inner.total(k, over, r)
}
fn occupancy(&self, p: &CellKey, r: &Reading) -> usize {
self.inner.occupancy(p, r)
}
}
impl Maintained for Rival {
@@ -351,7 +390,11 @@ fn only_revenue() -> CellKey {
fn measure_of(probe: &Probe) -> Option<&str> {
match probe {
Probe::Value(k, _) | Probe::RawFold(k) => k.get("_Measure"),
Probe::Value(k, _)
| Probe::RawFold(k)
| Probe::Covered(k, _)
| Probe::Total(k, _, _)
| Probe::Occupancy(k, _) => k.get("_Measure"),
}
}
@@ -381,7 +424,7 @@ fn the_harness_catches_a_backend_that_ignores_the_aggregation() {
let d = run_rival(Fault::IgnoresAggregation).unwrap_err();
assert_eq!(d.step, 0);
assert_eq!(
d.probe,
*d.probe,
Probe::Value(key("East", "A", "Profit"), Aggregation::OverHidden)
);
assert_eq!((d.eager, d.candidate), (None, num(6.0)));
@@ -414,7 +457,7 @@ fn the_harness_catches_a_wrong_folded_formula_value() {
let d = run_rival(Fault::WrongFoldedFormula).unwrap_err();
assert_eq!(d.step, 0);
assert_eq!(
d.probe,
*d.probe,
Probe::Value(partial("A", "Profit"), Aggregation::OverHidden)
);
assert_eq!((d.eager, d.candidate), (num(9.0), num(-999.0)));
@@ -432,7 +475,7 @@ fn the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula() {
fn the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped() {
let d = run_rival(Fault::WrongWhenTwoAreDropped).unwrap_err();
assert_eq!(d.step, 0);
assert_eq!(d.probe, Probe::Value(only_revenue(), Aggregation::Exact));
assert_eq!(*d.probe, Probe::Value(only_revenue(), Aggregation::Exact));
assert_eq!(d.candidate, num(-999.0));
}
@@ -440,7 +483,7 @@ fn the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped
fn the_harness_catches_a_raw_fold_that_returns_a_formula() {
let d = run_rival(Fault::RawFoldReturnsFormula).unwrap_err();
assert_eq!(d.step, 0);
assert!(matches!(d.probe, Probe::RawFold(_)), "{d}");
assert!(matches!(*d.probe, Probe::RawFold(_)), "{d}");
assert_eq!(measure_of(&d.probe), Some("Profit"), "{d}");
assert!(d.eager.is_none() && d.candidate.is_some(), "{d}");
}
@@ -455,7 +498,7 @@ fn the_harness_catches_a_wrong_group_fold() {
&[],
)
.unwrap_err();
assert_eq!(d.probe, Probe::Value(coastal(), Aggregation::Exact));
assert_eq!(*d.probe, Probe::Value(coastal(), Aggregation::Exact));
assert_eq!((d.eager, d.candidate), (num(15.0), num(-999.0)));
}
@@ -569,6 +612,18 @@ impl Derived for Fresh {
fn raw_fold(&self, p: &CellKey) -> Option<CellValue> {
self.eager.raw_fold(p)
}
fn covered(&self, k: &CellKey, r: &Reading) -> Option<CellValue> {
self.eager.covered(k, r)
}
fn total(&self, k: &CellKey, over: &[String], r: &Reading) -> Option<CellValue> {
self.eager.total(k, over, r)
}
fn occupancy(&self, p: &CellKey, r: &Reading) -> usize {
self.eager.occupancy(p, r)
}
}
impl Maintained for Fresh {
@@ -676,3 +731,422 @@ proptest! {
prop_assert!(run.is_ok(), "{}", run.unwrap_err());
}
}
/// Six categories, four of them None in the one view: the cell it draws
/// drops four coordinates of every stored key.
fn wide_workbook() -> improvise_core::workbook::Workbook {
let mut wb = improvise_core::workbook::Workbook::new("wide");
for c in ["C0", "C1", "C2", "C3", "C4"] {
wb.add_category(c).unwrap();
wb.add_item(c, "a").unwrap();
}
wb.add_item("C0", "b").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
for (c0, v) in [("a", 1.0), ("b", 2.0)] {
let k = CellKey::new(vec![
("C0".into(), c0.into()),
("C1".into(), "a".into()),
("C2".into(), "a".into()),
("C3".into(), "a".into()),
("C4".into(), "a".into()),
("_Measure".into(), "Revenue".into()),
]);
wb.model.set_cell(k, CellValue::Number(v));
}
let view = wb.active_view_mut();
view.set_axis("C0", Axis::Row);
view.set_axis("_Measure", Axis::Column);
for c in ["C1", "C2", "C3", "C4"] {
view.set_axis(c, Axis::None);
}
wb
}
/// The None categories of the active view, virtual ones included: the set an
/// `Eager` standing for that view is built with.
fn none_of(wb: &improvise_core::workbook::Workbook) -> Vec<String> {
let none = wb.active_view().categories_on(Axis::None);
none.into_iter().map(String::from).collect()
}
fn wide_cell() -> CellKey {
CellKey::new(vec![
("C0".into(), "a".into()),
("_Measure".into(), "Revenue".into()),
])
}
struct WrongAt(Eager, CellKey);
impl Derived for WrongAt {
fn value(&self, k: &CellKey, agg: Aggregation) -> Option<CellValue> {
if *k == self.1 {
num(-999.0)
} else {
self.0.value(k, agg)
}
}
fn raw_fold(&self, p: &CellKey) -> Option<CellValue> {
self.0.raw_fold(p)
}
fn covered(&self, k: &CellKey, r: &Reading) -> Option<CellValue> {
self.0.covered(k, r)
}
fn total(&self, k: &CellKey, over: &[String], r: &Reading) -> Option<CellValue> {
self.0.total(k, over, r)
}
fn occupancy(&self, p: &CellKey, r: &Reading) -> usize {
self.0.occupancy(p, r)
}
}
impl Maintained for WrongAt {
fn apply(&mut self, edit: &Edit) {
self.0.apply(edit);
}
}
#[test]
fn the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates() {
let wb = wide_workbook();
let none = none_of(&wb);
let rival = WrongAt(Eager::new(wb.model.clone(), none.clone()), wide_cell());
let views: Vec<_> = wb.views.values().cloned().collect();
let d = differential_in(&wb.model, &views, &none, rival, &[]).unwrap_err();
assert_eq!(d.step, 0);
assert_eq!(*d.probe, Probe::Value(wide_cell(), Aggregation::Exact));
}
/// East is hidden and Coastal is collapsed (a group is, by default), so the Coastal row folds West only;
/// Inland has a cell under A only, so B's column holds just Coastal's.
fn hidden_workbook() -> improvise_core::workbook::Workbook {
let mut wb = improvise_core::workbook::Workbook::new("hidden");
for c in ["Region", "Product"] {
wb.add_category(c).unwrap();
}
wb.add_item_in_group("Region", "East", "Coastal").unwrap();
wb.add_item_in_group("Region", "West", "Coastal").unwrap();
wb.add_item("Region", "Inland").unwrap();
wb.add_item("Product", "A").unwrap();
wb.add_item("Product", "B").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
let cells = [
("East", "A", 10.0),
("West", "A", 5.0),
("East", "B", 20.0),
("West", "B", 1.0),
("Inland", "A", 2.0),
];
for (r, p, v) in cells {
wb.model
.set_cell(key(r, p, "Revenue"), CellValue::Number(v));
}
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Row);
view.set_axis("Product", Axis::Column);
view.set_axis("_Measure", Axis::None);
view.hide_item("Region", "East");
wb
}
/// A backend that reads the view wrongly in exactly one way.
#[derive(Clone, Copy)]
enum Blindness {
CoversIgnoreHidden,
TotalsIgnoreHidden,
TotalsDoNotSpan,
OccupancyCountsHidden,
}
struct Blind(Eager, Blindness);
impl Derived for Blind {
fn value(&self, k: &CellKey, agg: Aggregation) -> Option<CellValue> {
self.0.value(k, agg)
}
fn raw_fold(&self, p: &CellKey) -> Option<CellValue> {
self.0.raw_fold(p)
}
fn covered(&self, k: &CellKey, r: &Reading) -> Option<CellValue> {
match self.1 {
Blindness::CoversIgnoreHidden => self.0.covered(k, &Reading::new(r.agg, [])),
_ => self.0.covered(k, r),
}
}
fn total(&self, k: &CellKey, over: &[String], r: &Reading) -> Option<CellValue> {
match self.1 {
Blindness::TotalsIgnoreHidden => self.0.total(k, over, &Reading::new(r.agg, [])),
Blindness::TotalsDoNotSpan => self.0.covered(k, r),
_ => self.0.total(k, over, r),
}
}
fn occupancy(&self, p: &CellKey, r: &Reading) -> usize {
match self.1 {
Blindness::OccupancyCountsHidden => self.0.occupancy(p, &Reading::new(r.agg, [])),
_ => self.0.occupancy(p, r),
}
}
}
impl Maintained for Blind {
fn apply(&mut self, edit: &Edit) {
self.0.apply(edit);
}
}
fn run_blind(fault: Option<Blindness>) -> Result<improvise_incremental::Report, Divergence> {
let wb = hidden_workbook();
let edit = Edit::SetCell(key("West", "A", "Revenue"), CellValue::Number(6.0));
run_blind_on(&wb, fault, edit)
}
fn run_blind_on(
wb: &improvise_core::workbook::Workbook,
fault: Option<Blindness>,
edit: Edit,
) -> Result<improvise_incremental::Report, Divergence> {
let none = none_of(wb);
let eager = Eager::new(wb.model.clone(), none.clone());
let views: Vec<_> = wb.views.values().cloned().collect();
match fault {
Some(f) => differential_in(&wb.model, &views, &none, Blind(eager, f), &[edit]),
None => differential_in(&wb.model, &views, &none, eager, &[edit]),
}
}
/// Nothing is hidden and Margin is a ratio: the pooled total of Margin is
/// Revenue over Cost summed across the rows, which no single row's fold gives.
fn ratio_workbook() -> improvise_core::workbook::Workbook {
let mut wb = improvise_core::workbook::Workbook::new("ratio");
for c in ["Region", "Product"] {
wb.add_category(c).unwrap();
}
wb.add_item("Region", "East").unwrap();
wb.add_item("Region", "West").unwrap();
wb.add_item("Product", "A").unwrap();
for m in ["Revenue", "Cost"] {
wb.add_item("_Measure", m).unwrap();
}
for (r, rev, cost) in [("East", 10.0, 4.0), ("West", 5.0, 2.0)] {
wb.model
.set_cell(key(r, "A", "Revenue"), CellValue::Number(rev));
wb.model
.set_cell(key(r, "A", "Cost"), CellValue::Number(cost));
}
wb.model
.add_formula(parse_formula("Margin = Revenue / Cost", "_Measure").unwrap());
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Row);
view.set_axis("_Measure", Axis::Column);
view.set_axis("Product", Axis::None);
wb
}
#[test]
fn the_harness_passes_a_view_reader_that_is_right() {
run_blind(None).unwrap();
}
#[test]
fn the_harness_catches_a_fold_that_covers_a_hidden_item() {
let d = run_blind(Some(Blindness::CoversIgnoreHidden)).unwrap_err();
assert!(matches!(*d.probe, Probe::Covered(..)), "{d}");
assert_eq!((d.eager, d.candidate), (num(5.0), num(15.0)), "Coastal/A");
}
#[test]
fn the_harness_catches_a_total_that_counts_a_hidden_item() {
let d = run_blind(Some(Blindness::TotalsIgnoreHidden)).unwrap_err();
assert!(matches!(*d.probe, Probe::Total(..)), "{d}");
}
#[test]
fn the_harness_catches_a_total_that_does_not_span_its_rows() {
let edit = Edit::SetCell(key("East", "A", "Cost"), CellValue::Number(5.0));
let run = |fault| run_blind_on(&ratio_workbook(), fault, edit.clone());
run(None).unwrap();
let d = run(Some(Blindness::TotalsDoNotSpan)).unwrap_err();
assert!(matches!(*d.probe, Probe::Total(..)), "{d}");
assert_eq!(measure_of(&d.probe), Some("Margin"), "{d}");
let pooled = 15.0 / 6.0;
assert_eq!(
(d.step, d.eager.clone(), d.candidate.clone()),
(0, num(pooled), None),
"{d}"
);
}
#[test]
fn the_harness_catches_an_occupancy_that_counts_a_hidden_item() {
let d = run_blind(Some(Blindness::OccupancyCountsHidden)).unwrap_err();
assert!(matches!(*d.probe, Probe::Occupancy(..)), "{d}");
}
#[test]
fn a_view_is_drawn_as_cells_totals_and_occupancy() {
let wb = hidden_workbook();
let views: Vec<_> = wb.views.values().cloned().collect();
let drawn = drawn_probes(&wb.model, &views, &none_of(&wb));
let kinds = [
drawn.iter().any(|p| matches!(p, Probe::Covered(..))),
drawn.iter().any(|p| matches!(p, Probe::Total(..))),
drawn.iter().any(|p| matches!(p, Probe::Occupancy(..))),
];
assert_eq!(kinds, [true; 3]);
}
fn coastal_line() -> CellKey {
CellKey::new(vec![("Region".into(), "Coastal".into())])
}
#[test]
fn a_collapsed_groups_occupancy_counts_its_visible_members() {
let wb = hidden_workbook();
let eager = Eager::new(wb.model.clone(), none_of(&wb));
let reading = Reading::of_view(wb.active_view(), &none_of(&wb));
assert_eq!(eager.occupancy(&coastal_line(), &reading), 2);
let east = CellKey::new(vec![("Region".into(), "East".into())]);
assert_eq!(eager.occupancy(&east, &reading), 0, "East is hidden");
}
fn drawn_of(wb: &improvise_core::workbook::Workbook) -> Vec<Probe> {
let views: Vec<_> = wb.views.values().cloned().collect();
drawn_probes(&wb.model, &views, &none_of(wb))
}
fn occupancy_keys(probes: &[Probe]) -> Vec<CellKey> {
let keyed = |p: &Probe| match p {
Probe::Occupancy(k, _) => Some(k.clone()),
_ => None,
};
probes.iter().filter_map(keyed).collect()
}
fn line(coords: &[(&str, &str)]) -> CellKey {
CellKey::new(
coords
.iter()
.map(|(c, i)| (c.to_string(), i.to_string()))
.collect(),
)
}
/// What the grid's own evaluator reads for every cell and total each view draws.
#[test]
fn eager_reads_what_the_grid_evaluator_reads() {
let mut compared = 0;
for (name, model, none, views) in corpus() {
let eager = Eager::new(model, none.clone());
for view in &views {
let layout = GridLayout::with_frozen_records(eager.model(), view, None);
let mut sorted = layout.none_cats.clone();
sorted.sort();
let mut mine = none.clone();
mine.sort();
if layout.is_records_mode() || sorted != mine {
continue;
}
let reading = Reading::of_view(view, &layout.none_cats);
let grid = layout.evaluator(eager.model());
for (r, c) in
(0..layout.row_count()).flat_map(|r| (0..layout.col_count()).map(move |c| (r, c)))
{
let k = layout.cell_key(r, c).unwrap();
assert_eq!(eager.covered(&k, &reading), grid.cell(&k), "{name}: {k:?}");
compared += 1;
}
for c in (0..layout.col_count()).filter(|_| layout.draws_totals()) {
let k = layout.column_key(c).unwrap();
let want = grid.column_total(c);
assert_eq!(
eager.total(&k, &layout.row_cats, &reading),
want,
"{name}: {k:?}"
);
compared += 1;
}
}
}
assert!(compared > 100, "{compared}");
}
#[test]
fn a_page_selection_narrows_the_line_occupancy_it_is_asked_under() {
let mut wb = hidden_workbook();
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Page);
view.set_axis("Product", Axis::Row);
view.set_page_selection("Region", "West");
let keys = occupancy_keys(&drawn_of(&wb));
assert!(
keys.contains(&line(&[("Product", "A"), ("Region", "West")])),
"{keys:?}"
);
assert!(!keys.contains(&line(&[("Product", "A")])), "{keys:?}");
}
#[test]
fn every_row_and_column_line_is_asked_its_occupancy() {
let keys = occupancy_keys(&drawn_of(&hidden_workbook()));
let rows = [
line(&[("Region", "Inland")]),
line(&[("Region", "Coastal")]),
];
assert!(rows.iter().all(|k| keys.contains(k)), "{keys:?}");
assert!(keys.contains(&line(&[("Product", "B")])), "{keys:?}");
}
#[test]
fn a_pruned_view_is_asked_only_about_the_lines_it_draws() {
let mut wb = hidden_workbook();
wb.add_item("Region", "Empty").unwrap();
let empty = line(&[("Region", "Empty")]);
assert!(
occupancy_keys(&drawn_of(&wb)).contains(&empty),
"drawn unpruned"
);
wb.active_view_mut().prune_empty = true;
assert!(
!occupancy_keys(&drawn_of(&wb)).contains(&empty),
"pruned away"
);
}
#[test]
fn a_records_view_is_not_asked_about() {
let mut wb = hidden_workbook();
let view = wb.active_view_mut();
view.set_axis("_Index", Axis::Row);
view.set_axis("_Dim", Axis::Column);
for cat in ["Region", "Product", "_Measure"] {
view.set_axis(cat, Axis::None);
}
assert!(view.is_records_layout());
assert!(drawn_of(&wb).is_empty());
}
#[test]
fn no_total_is_asked_where_the_grid_draws_none() {
let mut wb = ratio_workbook();
let totals = |wb: &improvise_core::workbook::Workbook| {
drawn_of(wb)
.iter()
.filter(|p| matches!(p, Probe::Total(..)))
.count()
};
assert!(totals(&wb) > 0, "totals are drawn here");
let view = wb.active_view_mut();
view.set_axis("_Measure", Axis::Row);
view.set_axis("Region", Axis::Column);
assert_eq!(totals(&wb), 0, "a column mixing measures has no total");
}
#[test]
fn a_view_under_another_none_set_is_not_asked() {
let wb = hidden_workbook();
let views: Vec<_> = wb.views.values().cloned().collect();
assert!(drawn_probes(&wb.model, &views, &["Region".to_string()]).is_empty());
}