diff --git a/context/test-evidence/improvise-incremental.md b/context/test-evidence/improvise-incremental.md index 06a6399e..f47d8e60 100644 --- a/context/test-evidence/improvise-incremental.md +++ b/context/test-evidence/improvise-incremental.md @@ -51,142 +51,142 @@ none did): - `Edit::ClearCell` applied as `SetCell(0.0)`, which both sides of a differential share and so only a direct `Eager` test can see: `eager_clear_cell_leaves_the_key_absent`. -## eager_raw_fold_sums_by_default +### eager_raw_fold_sums_by_default Hand-computed: Revenue under Product A is 10 + 5 = 15. Fails on the `raw_fold`-reads-`Exact` rival (a partial key names no stored cell, so the answer is `None`). -## eager_raw_fold_follows_the_measures_aggregation +### eager_raw_fold_follows_the_measures_aggregation After `measure_agg[Revenue] = Max` the same fold is 10, and Cost, unchanged, still sums to 6. A fold that always sums fails on the first; one that always takes the maximum fails on the second. -## eager_raw_fold_of_text_is_unanimous_or_mixed +### eager_raw_fold_of_text_is_unanimous_or_mixed Note under A is two equal texts and reads as that text; under B it is two different texts and reads `Error("mixed")`. A fold that drops text, which the evaluator used to do (gotcha 7), returns `None` for both. -## eager_numbers_among_text_are_mixed +### eager_numbers_among_text_are_mixed A text among numbers is `Error("mixed")`, not the sum of the numbers. -## eager_value_reads_a_formula_exactly_and_folded +### eager_value_reads_a_formula_exactly_and_folded `Profit = Revenue - Cost` reads 6 at East/A exactly, and 9 at Product A with Region over hidden: (10 + 5) - (4 + 2), the fold taken before the subtraction. Fails on the `value`-ignores-`Aggregation` rival. -## eager_follows_an_edit_to_the_cell_a_fold_reads +### eager_follows_an_edit_to_the_cell_a_fold_reads After East/A Revenue becomes 20 the fold is 25 and the folded Profit 19. The Profit half is what fails when `apply` forgets to recompute formulas: the folded cache is then stale. -## eager_matches_the_evaluator_on_the_shipped_documents +### eager_matches_the_evaluator_on_the_shipped_documents Both shipped documents, every view's None categories, and the hand model: every probe of `Eager` equals a direct `evaluate_aggregated` on a model recomputed under the same None categories, and the probe set is non-empty. Fails on both aggregation rivals. -## eager_agrees_with_itself_over_every_corpus_edit_sequence +### eager_agrees_with_itself_over_every_corpus_edit_sequence `differential` over a generated edit sequence on each corpus model returns a `Report` with `edits + 1` steps and a non-zero probe count. The count assertion is what a harness that compared nothing would fail; the step count is what one that stopped early would fail. -## the_harness_passes_a_backend_that_is_right +### the_harness_passes_a_backend_that_is_right The control for the `catches` tests: `Eager` as the candidate over the hand-built edits is a pass, so a harness that rejects everything is seen. -## the_harness_catches_a_backend_that_ignores_the_aggregation +### the_harness_catches_a_backend_that_ignores_the_aggregation A candidate that reads every `value` as `Exact` diverges at step 0, first at the full-key formula cell `{East, A, Profit}` over hidden (eager `None`, candidate `6`). -## the_harness_catches_a_backend_that_ignores_the_measure_aggregation +### the_harness_catches_a_backend_that_ignores_the_measure_aggregation A candidate that drops `SetMeasureAgg` diverges at step 2, the edit that sets it, on a Revenue key. -## the_harness_catches_a_backend_that_goes_stale_after_two_edits +### the_harness_catches_a_backend_that_goes_stale_after_two_edits A candidate that stops applying edits after the second diverges at step 3, on a Revenue key. A prefix of steps cannot see a backend that goes wrong late. -## the_harness_catches_a_backend_that_ignores_a_clear +### the_harness_catches_a_backend_that_ignores_a_clear A candidate that keeps a cleared cell diverges at step 3 and the message names it. -## the_harness_catches_a_wrong_folded_formula_value +### the_harness_catches_a_wrong_folded_formula_value Review of PR 343 (Codex P1): a rival wrong only at `value({Product=A, _Measure=Profit}, OverHidden)`, the Profit cell a view with Region on None draws, passed 128 probes, because formula cells were built over full keys only. Now the divergence is at step 0 on exactly that probe, eager `9`, candidate `-999`. -## the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula +### the_harness_catches_a_backend_that_keeps_answering_for_a_removed_formula Review of PR 343: probes were rebuilt from the post-edit model, so a candidate that ignores `RemoveFormula` was never asked about Profit again. Probes now accumulate over every step. The divergence is at the last step, on a Profit key, eager `None`, candidate `Some`. -## the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped +### the_harness_catches_a_fault_that_only_shows_after_two_coordinates_are_dropped Review of PR 343 (Copilot): only one-coordinate partials and the empty key were probed. A rival wrong only at `{_Measure=Revenue}` now diverges at step 0 on `Value({_Measure=Revenue}, Exact)`. The bound is `MAX_DROPPED = 3` coordinates; a fold dropping more is not asked about. -## the_harness_catches_a_raw_fold_that_returns_a_formula +### the_harness_catches_a_raw_fold_that_returns_a_formula Review of PR 343 (Copilot): `raw_fold` is documented never to be a formula. A rival whose `raw_fold` is `value(partial, OverHidden)` diverges at step 0 on a `RawFold` of a Profit partial (eager `None`, candidate `Some`). It could not be seen while `Eager` itself returned the formula. -## the_harness_catches_a_wrong_group_fold +### the_harness_catches_a_wrong_group_fold A group name stands in for an item in the probe keys. A rival wrong only at `{Region=Coastal, Product=A, _Measure=Revenue}` diverges at step 0, eager `15`, candidate `-999`. -## eager_folds_a_group_it_is_asked_about +### eager_folds_a_group_it_is_asked_about The group fold in the grouped model is East + West = 15 exactly, so the rival test above has a true answer to differ from. -## eager_raw_fold_is_never_a_formula +### eager_raw_fold_is_never_a_formula At `{Product=A, _Measure=Profit}` the folded value is `9` and the raw fold is `None`. Fails on the fault that lets `raw_fold` read the formula cache. -## eager_add_and_remove_formula_change_what_a_key_reads +### eager_add_and_remove_formula_change_what_a_key_reads Review of PR 343: `Edit::AddFormula` as a no-op left all 25 tests green, because an `Edit` fault reaches both sides of a differential alike. `Double = Revenue * 2` reads `None` before the edit and `20` at East/A after it, and `None` again after `RemoveFormula`. Fails on `AddFormula` as a no-op. -## eager_clear_cell_leaves_the_key_absent +### eager_clear_cell_leaves_the_key_absent After `ClearCell` the key reads `None` and the model holds no cell. A `ClearCell` applied as `SetCell(0.0)` hits both sides of a differential identically, so only this direct test fails. -## same_value_follows_the_float_rules_and_is_exact_elsewhere +### same_value_follows_the_float_rules_and_is_exact_elsewhere NaN equals NaN, `-0.0` equals `0.0`, `1.0` differs from `1.0000000000000002`, texts and errors compare exactly, and `None` equals only `None`. Each of three one-line faults in `same_value` (NaN unequal, signed zero by bits, any two texts equal) survived the whole suite until this test existed. -## probes_cover_folds_formulas_measures_and_groups +### probes_cover_folds_formulas_measures_and_groups `probes` includes a raw fold, a full key, a folded formula cell under both aggregations, the two-dropped key `{_Measure=Revenue}`, the empty key and a group key. Review of PR 343 showed `probes` omitting formula cells, `_Measure` keys or raw folds left every test green. -## an_edit_names_its_key_as_two_value_probes_and_a_fold +### an_edit_names_its_key_as_two_value_probes_and_a_fold `edit_probes` of a `ClearCell` names the cleared key three ways, and of an edit naming no key names nothing. -## a_maintained_eager_equals_a_fresh_one_over_random_edit_sequences +### a_maintained_eager_equals_a_fresh_one_over_random_edit_sequences Clause 2 asks for a property test over a random edit sequence. A proptest (256 cases, up to four edits of five kinds over the hand-built model) runs `differential` with a candidate that rebuilds @@ -216,37 +216,37 @@ both zeros and three NaN spellings) checks that the delta `DataStore` logs for a the store as a Z-set after it minus the store before. No `Derived` backend exists yet, so this is the harness's step-3 form: the delta against the store it describes. -## the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates +### the_harness_checks_a_cell_a_view_draws_that_drops_four_coordinates improvise-bw4p.4. Six categories, four on None, a rival wrong only at `{C0=a, _Measure=Revenue}`. Before the drawn probes the differential passed it (`is_err` failed); the cell drops four coordinates of a stored key and `MAX_DROPPED` is three. Now the first divergence is the drawn cell's exact value. Fault: `drawn_probes` skipping every view fails this and five more. -## probing_a_key_of_twenty_six_coordinates_is_not_exponential +### probing_a_key_of_twenty_six_coordinates_is_not_exponential The old `sub_partials` walked all 2^n masks (93ms at 20 categories, 225ms at 22). With the mask walk put back, 26 coordinates take 807ms against a 250ms bound; the combination walk takes under 70ms. The probe set is unchanged for keys of up to 3 coordinates over any bound. -## the_harness_catches_a_fold_that_covers_a_hidden_item +### the_harness_catches_a_fold_that_covers_a_hidden_item improvise-bw4p.3. East hidden, Coastal collapsed: the drawn Coastal/A is 5, not 15. A rival whose `covered` reads with no hidden items diverges there. Fault in `Eager::covered` (covers everything) fails this and `eager_matches_the_evaluator_on_the_shipped_documents`, which now includes the `hidden` document, because no shipped demo hides an item. -## the_harness_catches_a_total_that_counts_a_hidden_item +### the_harness_catches_a_total_that_counts_a_hidden_item A rival total reading with no hidden items diverges at a column total. The same fault in `Eager::total` fails this, the next test and the shipped-documents comparison. -## the_harness_catches_a_total_that_does_not_span_its_rows +### the_harness_catches_a_total_that_does_not_span_its_rows A rival answering `covered` for a column key (so the aggregate over its rows, not one fold over the pooled rows) is told apart from `total`: with no None category the first has no cell. -## the_harness_catches_an_occupancy_that_counts_a_hidden_item +### the_harness_catches_an_occupancy_that_counts_a_hidden_item A rival counting stored cells with no hidden filter diverges at a drawn cell's or line's occupancy. `Eager::occupancy` ignoring the hidden items fails this and the shipped-documents