fix: resolve a cell's format through the formula that owns it
Both findings from ChatGPT Codex's review of PR #125. Both are real, and both are the same mistake in two places: answering "whose format is this" from something other than the rule that answers "whose value is this". **1. `FormatPlan` indexed formats in a `HashMap` keyed by target.** A target category need not be `_Measure` — `[CustomCat]` is in the format's own documentation — so a key can carry coordinates two formulas claim, and iterating the map answered from whichever category hashed first: the displayed format was nondeterministic per process while the value was not. That is gotcha 18's shape. Worse, only *formatted* formulas were indexed, so a later formula's `FORMAT` reached a cell whose value an earlier unformatted formula produces. `FormatPlan` now holds every formula's `(target_category, target, Option<format>)` in the model's own order and takes the first coordinate match — `Model::formula_for`'s rule and order exactly. A match naming no format falls through to the view; it is a match, not a miss. The list is dropped entirely when no formula names a format, so the common document still pays one `is_empty` per cell. **2. A column total took its format from row 0.** `context/code/core.md` argued that was safe because "where `_Measure` is on the row axis a column is not a measure at all". Wrong: a formula's target category is any category, so a formula can own a row-axis item — the total then inherited a row-local override for a row it had summed away, and re-ordering the rows changed its spelling. New `GridLayout::column_key(col)` gives the coordinates that survive the row aggregation (page/filter constraints plus the column's tuple), and `col_formats` is resolved from that, once per column, outside the row loop. Three regression tests, each with the rejected version injected as the fault: - `the_formula_that_owns_a_cell_names_its_format_even_when_it_names_none` puts the same two formulas in both orders, so preferring the formatted one fails one half and ignoring formats fails the other. - `the_owner_is_the_one_model_order_names_and_not_the_one_a_hash_does` rebuilds the plan five times, since each `HashMap` gets its own `RandomState`. - `a_column_total_does_not_inherit_a_row_local_format` builds one grid with the owned row first and one with it last and requires the strips to agree. Every raw cell holds the same value per measure, so the totals are numerically identical and only the format can differ. `context/code/core.md`'s wrong paragraph is replaced rather than amended, and says what it got wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
+33
-8
@@ -870,15 +870,40 @@ which falls out of the key rather than needing a rule. Deriving it from "is
|
||||
with itself while disagreeing with the cells.
|
||||
|
||||
**The empty plan short-circuits**, so a document with no formula format pays one
|
||||
`is_empty` per cell and nothing else; the lookup is otherwise one hash get per
|
||||
`(category, item)` pair in the plan, of which there is normally one.
|
||||
`is_empty` per cell and nothing else; the lookup is otherwise a scan of the
|
||||
formula list, which is short and must stay ordered for the reason below.
|
||||
|
||||
**A column total takes the format of its own column's first row.** `GridBody`
|
||||
carries `col_formats` beside `col_widths` so the totals strip and the cells
|
||||
above it cannot disagree. Where `_Measure` is on the *row* axis a column is not
|
||||
a measure at all, so no formula format applies to any cell in it and the first
|
||||
row's answer is the view's — the rule only has to be decided for the case it
|
||||
cannot arise in.
|
||||
**The owner is the formula `Model::formula_for` names, and nothing else.**
|
||||
`FormatPlan` holds every formula's `(target_category, target, Option<format>)`
|
||||
in the model's own order and takes the **first** coordinate match, which is the
|
||||
same rule and the same order evaluation uses to decide whose value the cell is.
|
||||
Two things that shape enforces, both of which a `HashMap` keyed by target got
|
||||
wrong and review caught (Codex, PR #125):
|
||||
|
||||
- **A key may be claimed by two formulas**, because a target category need not
|
||||
be `_Measure` — `- Tax = Revenue * 0.1 [CustomCat]` is in the format's own
|
||||
documentation. Iterating a `HashMap` answered from whichever category hashed
|
||||
first, so the *displayed* format was nondeterministic per process while the
|
||||
*value* was not. That is gotcha 18 exactly: an index whose values are hashed
|
||||
is for lookup, never for choosing between candidates.
|
||||
- **An owner that names no format still owns the cell.** Indexing only the
|
||||
formatted formulas let a later formula's `FORMAT` reach a cell whose value it
|
||||
does not produce. So the list holds *every* formula and a match with
|
||||
`format: None` falls through to the view — it is a match, not a miss.
|
||||
|
||||
The list is dropped entirely when no formula names a format, so the common
|
||||
document pays one `is_empty` per cell rather than a scan.
|
||||
|
||||
**A column total is resolved from the column, never from a row.**
|
||||
`GridLayout::column_key(col)` is the page/filter constraints plus the column's
|
||||
own tuple — the coordinates that survive the row aggregation — and `GridBody`
|
||||
carries the resulting `col_formats` so the strip and the cells cannot disagree.
|
||||
Reading it off row 0 was the first version and is wrong twice over, also found
|
||||
by review: a formula may target a **row-axis** item (its target category is any
|
||||
category), so the total inherited a row-local override for a row it had summed
|
||||
away, **and** re-ordering the rows changed the total's spelling. The guard is
|
||||
`a_column_total_does_not_inherit_a_row_local_format`, which builds the same grid
|
||||
with the owned row first and last and requires the two strips to agree.
|
||||
|
||||
### The grammar: `FORMAT <spec>` in the formula, not in the `.improv` line
|
||||
|
||||
|
||||
@@ -4244,3 +4244,27 @@ Two faults injected:
|
||||
The second is the one that earns the test its length: without it, the totals
|
||||
strip could round every formula column to the view's format and no assertion in
|
||||
the workspace would move.
|
||||
|
||||
### a_column_total_does_not_inherit_a_row_local_format
|
||||
|
||||
Found by review (ChatGPT Codex, PR #125). The first version of `grid_body` read
|
||||
each column's total format off **row 0**, and `context/code/core.md` argued that
|
||||
was safe because "where `_Measure` is on the row axis a column is not a measure
|
||||
at all". That argument is wrong: a formula's target category is *any* category,
|
||||
so a formula can own a **row-axis item**, and then row 0's format is a row-local
|
||||
override applied to a total that has summed that row away — and re-ordering the
|
||||
rows changes the total's spelling.
|
||||
|
||||
The fixture builds the same grid twice with `12.5 FORMAT .4` owning the **first**
|
||||
row in one and the **last** row in the other, and requires the two totals strips
|
||||
to agree. Every raw cell in `raw_workbook` holds the same value per measure, so
|
||||
the two grids' totals are numerically identical and the *only* thing that can
|
||||
differ is the format — the assertion has nowhere to hide.
|
||||
|
||||
It asserts the owned cell first (`12.5000`), so a plan that applies no format at
|
||||
all fails on the fixture rather than passing the comparison vacuously.
|
||||
|
||||
Fault injected: `plan.at(layout.cell_key(0, ci).as_ref())` in place of
|
||||
`layout.column_key(ci)` — the rejected version, restored. Fails with
|
||||
`left: ["14.5000", "16.5000"] right: ["14.50", "16.50"]`, which is the bug in
|
||||
one line: row order decided the spelling.
|
||||
|
||||
@@ -1107,3 +1107,39 @@ The fallback, asserted where it is easiest to lose: a formula target whose
|
||||
formula carries no format must be indistinguishable from a raw measure. It
|
||||
keeps `FormatPlan::of` from indexing every target with some default and then
|
||||
"overriding" it back, which would be a second copy of the view's answer.
|
||||
|
||||
### the_formula_that_owns_a_cell_names_its_format_even_when_it_names_none
|
||||
|
||||
Found by review (ChatGPT Codex, PR #125), not by the tests above, which is the
|
||||
entry's point: they used one formula and so could not see an ownership question
|
||||
at all.
|
||||
|
||||
A target category need not be `_Measure` — `- Tax = Revenue * 0.1 [CustomCat]`
|
||||
is in the format's own documentation — so a key may carry coordinates two
|
||||
formulas claim. The first `FormatPlan` indexed **only the formatted** formulas,
|
||||
so a later formula's `FORMAT` reached a cell whose value an earlier, unformatted
|
||||
formula produces: the number and its spelling came from different formulas.
|
||||
|
||||
The fixture puts the same two formulas in both orders and asserts the owner's
|
||||
answer each time, so a plan that prefers the formatted one fails the first half
|
||||
and a plan that ignores formats entirely fails the second.
|
||||
|
||||
Fault injected: `.filter(|t| t.format.is_some())` before the `find` — precisely
|
||||
the behaviour of the version review rejected. Fails with `left: "0.6667"
|
||||
right: "0.67"`.
|
||||
|
||||
### the_owner_is_the_one_model_order_names_and_not_the_one_a_hash_does
|
||||
|
||||
The other half of the same finding, and the one a single run cannot see.
|
||||
`by_target` was a `HashMap<category, HashMap<item, _>>` and `at` iterated it, so
|
||||
with two claiming categories the answer came from whichever hashed first —
|
||||
**a different answer per process**, gotcha 18's shape exactly (an index whose
|
||||
values are hashed is for lookup, never for choosing between candidates).
|
||||
|
||||
It rebuilds the plan five times rather than asserting once, because each
|
||||
`HashMap` gets its own `RandomState` and a single construction can be right by
|
||||
luck. It is the same shape as `analyze_records`' five-iteration guard.
|
||||
|
||||
This one cannot be made to fail by re-introducing a `HashMap` *reliably*, which
|
||||
is why the test that carries the load is its sibling above; kept because the
|
||||
repetition is what documents that hash order was the defect.
|
||||
|
||||
@@ -4537,6 +4537,53 @@ mod cell_identity {
|
||||
assert_eq!(grid.totals[revenue].text, "3.00");
|
||||
}
|
||||
|
||||
/// A formula owning one ROW, so its format is row-local and the totals have
|
||||
/// aggregated that row away. `context/test-evidence/command.md`.
|
||||
fn row_targeted_app(region: &str) -> App {
|
||||
let mut wb = raw_workbook();
|
||||
wb.model.add_formula(
|
||||
improvise_formula::parse_formula(&format!("{region} = 12.5 FORMAT .4"), "Region")
|
||||
.unwrap(),
|
||||
);
|
||||
App::new(wb, None)
|
||||
}
|
||||
|
||||
/// improvise-e0ex, found by review: reading a total's format off row 0 made
|
||||
/// it depend on row ORDER. `context/test-evidence/command.md`.
|
||||
#[test]
|
||||
fn a_column_total_does_not_inherit_a_row_local_format() {
|
||||
let first = row_targeted_app("North");
|
||||
let last = row_targeted_app("East");
|
||||
let (fg, lg) = (grid_of(&first, WHOLE), grid_of(&last, WHOLE));
|
||||
|
||||
// The row the formula owns really is drawn with the formula's format —
|
||||
// without this the test would pass against a plan that does nothing.
|
||||
let owned = fg.cell(0, 0).map(|c| c.text.clone());
|
||||
assert_eq!(
|
||||
owned,
|
||||
Some("12.5000".to_string()),
|
||||
"fixture: North is owned"
|
||||
);
|
||||
|
||||
// North is row 0 and East is the last row, so a total read off row 0
|
||||
// answers differently in the two grids. It must not.
|
||||
let totals = |g: &improvise_protocol::render::GridCache| -> Vec<String> {
|
||||
g.totals.iter().map(|t| t.text.clone()).collect()
|
||||
};
|
||||
assert_eq!(
|
||||
totals(&fg),
|
||||
totals(&lg),
|
||||
"which row a formula owns changed the totals' spelling"
|
||||
);
|
||||
for t in totals(&fg) {
|
||||
assert_eq!(
|
||||
t.split('.').nth(1).map(str::len),
|
||||
Some(2),
|
||||
"a total must keep the VIEW's two decimals, got {t:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn column_named(app: &App, label: &str) -> usize {
|
||||
(0..app.layout.col_count())
|
||||
.find(|&c| app.layout.col_label(c) == label)
|
||||
|
||||
@@ -23,8 +23,8 @@ pub use improvise_protocol::{MAX_ROW_HEADER_W, MIN_ROW_HEADER_W};
|
||||
pub struct GridBody {
|
||||
/// One per column of the WHOLE grid, clamped, never windowed.
|
||||
pub col_widths: Vec<u16>,
|
||||
/// The format each column's cells were drawn with, read off the top row so
|
||||
/// the totals strip agrees with them — `context/code/command.md`.
|
||||
/// The format each column's TOTAL is drawn with, resolved from the
|
||||
/// coordinates that survive the row aggregation — `context/code/core.md`.
|
||||
pub col_formats: Vec<NumberFormat>,
|
||||
/// Column sums over every row. **Empty** in records mode or an empty grid,
|
||||
/// which is "this grid has no totals" rather than a strip of zeroes.
|
||||
@@ -51,7 +51,11 @@ pub fn grid_body(
|
||||
false => Vec::new(),
|
||||
};
|
||||
let mut window: Vec<Vec<Option<CellValue>>> = Vec::with_capacity(rows.len());
|
||||
let mut col_formats: Vec<NumberFormat> = vec![plan.view_format(); n];
|
||||
// Off the COLUMN, not off row 0: a formula targeting a row-axis item is a
|
||||
// row-local override, and the total has aggregated that row away.
|
||||
let col_formats: Vec<NumberFormat> = (0..n)
|
||||
.map(|ci| plan.at(layout.column_key(ci).as_ref()))
|
||||
.collect();
|
||||
|
||||
for ri in 0..layout.row_count() {
|
||||
let drawn = rows.contains(&ri);
|
||||
@@ -62,9 +66,6 @@ pub fn grid_body(
|
||||
for ci in 0..n {
|
||||
let (key, value) = cell_value(model, layout, records, ri, ci);
|
||||
let fmt = plan.at(key.as_ref());
|
||||
if let Some(slot) = col_formats.get_mut(ci).filter(|_| ri == 0) {
|
||||
*slot = fmt;
|
||||
}
|
||||
let text = match records {
|
||||
true => value.as_ref().map(|v| v.to_string()).unwrap_or_default(),
|
||||
false => fmt.value(value.as_ref()),
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use crate::model::cell::{CellKey, CellValue};
|
||||
use crate::model::types::Model;
|
||||
use crate::view::types::View;
|
||||
@@ -27,28 +25,44 @@ impl NumberFormat {
|
||||
}
|
||||
}
|
||||
|
||||
/// Which format each cell is drawn with: the view's `format:`, unless one of the
|
||||
/// cell's coordinates names a formula carrying its own (`context/code/core.md`).
|
||||
/// One formula's claim on a cell: the coordinate it owns, and the format it
|
||||
/// asks for — `None` where it names none. `context/code/core.md`.
|
||||
#[derive(Clone, Debug)]
|
||||
struct TargetFormat {
|
||||
category: String,
|
||||
target: String,
|
||||
format: Option<NumberFormat>,
|
||||
}
|
||||
|
||||
/// Which format each cell is drawn with: the view's `format:`, unless the
|
||||
/// formula that OWNS the cell names its own (`context/code/core.md`).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct FormatPlan {
|
||||
view: NumberFormat,
|
||||
by_target: HashMap<String, HashMap<String, NumberFormat>>,
|
||||
/// Every formula, in the model's own order, or **empty** where none names a
|
||||
/// format — the ownership rule is `Model::formula_for`'s, not a hash's.
|
||||
targets: Vec<TargetFormat>,
|
||||
}
|
||||
|
||||
impl FormatPlan {
|
||||
pub fn of(model: &Model, view: &View) -> Self {
|
||||
let mut by_target: HashMap<String, HashMap<String, NumberFormat>> = HashMap::new();
|
||||
for f in model.formulas() {
|
||||
if let Some(fmt) = &f.format {
|
||||
by_target
|
||||
.entry(f.target_category.clone())
|
||||
.or_default()
|
||||
.insert(f.target.clone(), NumberFormat::parse(fmt));
|
||||
}
|
||||
}
|
||||
let targets: Vec<TargetFormat> = model
|
||||
.formulas()
|
||||
.iter()
|
||||
.map(|f| TargetFormat {
|
||||
category: f.target_category.clone(),
|
||||
target: f.target.clone(),
|
||||
format: f.format.as_deref().map(NumberFormat::parse),
|
||||
})
|
||||
.collect();
|
||||
Self {
|
||||
view: NumberFormat::parse(&view.number_format),
|
||||
by_target,
|
||||
// Where no formula names a format every answer is the view's, so
|
||||
// the list is dropped rather than scanned once per cell.
|
||||
targets: match targets.iter().any(|t| t.format.is_some()) {
|
||||
true => targets,
|
||||
false => Vec::new(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,11 +71,11 @@ impl FormatPlan {
|
||||
pub fn uniform(fmt: &str) -> Self {
|
||||
Self {
|
||||
view: NumberFormat::parse(fmt),
|
||||
by_target: HashMap::new(),
|
||||
targets: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// The view's own answer: what a cell naming no formula target is drawn with.
|
||||
/// The view's own answer: what a cell no formula owns is drawn with.
|
||||
pub fn view_format(&self) -> NumberFormat {
|
||||
self.view
|
||||
}
|
||||
@@ -69,12 +83,15 @@ impl FormatPlan {
|
||||
/// Asked of the *key*, never of the axes: a cell aggregating over `_Measure`
|
||||
/// carries no target coordinate and so keeps the view's format.
|
||||
pub fn at(&self, key: Option<&CellKey>) -> NumberFormat {
|
||||
match key.filter(|_| !self.by_target.is_empty()) {
|
||||
match key.filter(|_| !self.targets.is_empty()) {
|
||||
None => self.view,
|
||||
// The first match in the model's order — `formula_for`'s own rule,
|
||||
// an owner naming no format included. `context/code/core.md`.
|
||||
Some(key) => self
|
||||
.by_target
|
||||
.targets
|
||||
.iter()
|
||||
.find_map(|(cat, items)| items.get(key.get(cat)?).copied())
|
||||
.find(|t| key.get(&t.category) == Some(t.target.as_str()))
|
||||
.and_then(|t| t.format)
|
||||
.unwrap_or(self.view),
|
||||
}
|
||||
}
|
||||
@@ -421,6 +438,74 @@ mod plan_tests {
|
||||
assert_eq!(plan.at(None).value(Some(&ratio())), "0.67");
|
||||
}
|
||||
|
||||
fn two_owner_model(first: &str, second: &str) -> Model {
|
||||
let mut m = Model::new("m");
|
||||
m.add_category("Region").unwrap();
|
||||
m.add_formula(parse_formula(first, "_Measure").expect("parses"));
|
||||
m.add_formula(parse_formula(second, "Region").expect("parses"));
|
||||
m
|
||||
}
|
||||
|
||||
/// A key both formulas claim: `_Measure=Margin` and `Region=North`.
|
||||
fn contested() -> CellKey {
|
||||
CellKey::new(vec![
|
||||
("_Measure".to_string(), "Margin".to_string()),
|
||||
("Region".to_string(), "North".to_string()),
|
||||
])
|
||||
}
|
||||
|
||||
/// improvise-e0ex — see `context/test-evidence/core.md`.
|
||||
#[test]
|
||||
fn the_formula_that_owns_a_cell_names_its_format_even_when_it_names_none() {
|
||||
// Owner first, and it names NO format: the view answers, and the later
|
||||
// formula's `.4` must not reach a cell whose value it does not produce.
|
||||
let owner_unformatted = two_owner_model(
|
||||
"Margin = Profit / Revenue",
|
||||
"North = Profit / Revenue FORMAT .4",
|
||||
);
|
||||
let view = View::new("Default");
|
||||
assert_eq!(
|
||||
FormatPlan::of(&owner_unformatted, &view)
|
||||
.at(Some(&contested()))
|
||||
.value(Some(&ratio())),
|
||||
"0.67",
|
||||
"a later formula's format overrode the formula that owns the cell"
|
||||
);
|
||||
|
||||
// The same two formulas the other way round: now the owner names `.4`.
|
||||
let owner_formatted = two_owner_model(
|
||||
"Margin = Profit / Revenue FORMAT .4",
|
||||
"North = Profit / Revenue",
|
||||
);
|
||||
assert_eq!(
|
||||
FormatPlan::of(&owner_formatted, &view)
|
||||
.at(Some(&contested()))
|
||||
.value(Some(&ratio())),
|
||||
"0.6667",
|
||||
"the owner's own format must win"
|
||||
);
|
||||
}
|
||||
|
||||
/// improvise-e0ex — see `context/test-evidence/core.md`.
|
||||
#[test]
|
||||
fn the_owner_is_the_one_model_order_names_and_not_the_one_a_hash_does() {
|
||||
let model = two_owner_model(
|
||||
"Margin = Profit / Revenue",
|
||||
"North = Profit / Revenue FORMAT .4",
|
||||
);
|
||||
let view = View::new("Default");
|
||||
// A `HashMap` gets a fresh `RandomState` per process, so one run cannot
|
||||
// see hash order (gotcha 18); the plan is rebuilt to make that the point.
|
||||
for _ in 0..5 {
|
||||
assert_eq!(
|
||||
FormatPlan::of(&model, &view)
|
||||
.at(Some(&contested()))
|
||||
.value(Some(&ratio())),
|
||||
"0.67"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// improvise-e0ex — see `context/test-evidence/core.md`.
|
||||
#[test]
|
||||
fn a_formula_naming_no_format_leaves_its_target_on_the_views() {
|
||||
|
||||
@@ -586,6 +586,17 @@ impl GridLayout {
|
||||
Some(CellKey::new(coords))
|
||||
}
|
||||
|
||||
/// The coordinates a COLUMN answers for once its rows are aggregated away:
|
||||
/// the page/filter constraints and the column's own tuple (core.md §6).
|
||||
pub fn column_key(&self, col: usize) -> Option<CellKey> {
|
||||
let col_item = data_items(&self.col_items).nth(col)?;
|
||||
let mut coords = page_constraints(&self.page_coords);
|
||||
for (cat, item) in self.col_cats.iter().zip(col_item.iter()) {
|
||||
coords.push((cat.clone(), item.clone()));
|
||||
}
|
||||
Some(CellKey::new(coords))
|
||||
}
|
||||
|
||||
/// Visual index of the nth data row (skipping group headers).
|
||||
pub fn data_row_to_visual(&self, data_row: usize) -> Option<usize> {
|
||||
data_index_to_visual(&self.row_items, data_row)
|
||||
|
||||
Reference in New Issue
Block a user