The tokenizer already broke multi-word identifiers when the NEXT word
was a keyword, but not when the identifier collected SO FAR was a
keyword. This meant "WHERE Region" was merged into one token when
tokenizing "SUM(Revenue WHERE Region = East)".
Now the tokenizer also checks if the identifier built up to the current
space IS a keyword (WHERE, SUM, AVG, MIN, MAX, COUNT, IF), which
correctly produces separate tokens for "Revenue", "WHERE", "Region".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The tokenizer was greedily consuming spaces and potentially merging
identifiers with subsequent keywords. This change improves the tokenizer
by:
- Peeking ahead past spaces to find the next word/token.
- Breaking the identifier if the next word is a known keyword (WHERE, SUM,
AVG, MIN, MAX, COUNT, IF).
- Adding support for more delimiter characters (<, >, =, !, ").
This fixes a regression where "Revenue WHERE" was treated as a single
identifier instead of an identifier followed by a WHERE clause.
Includes a new regression test for inline WHERE filters in aggregate
functions.
Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
Add a comprehensive suite of unit tests for the formula parser, covering:
- Aggregate functions
- WHERE clauses
- Comparison operators
- Arithmetic operations
- Various error handling scenarios
Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
Previously Expr::BinOp(String, ...) accepted any string as an operator.
Invalid operators (e.g. "diagonal") would compile fine and silently
return CellValue::Empty at eval time.
Now BinOp is an enum with variants Add/Sub/Mul/Div/Pow/Eq/Ne/Lt/Gt/Le/Ge.
The parser produces enum variants directly; the evaluator pattern-matches
exhaustively with no fallback branch. An invalid operator is now a
compile error at the call site, and the compiler ensures every variant
is handled in both eval_expr and eval_bool.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- model.rs: division by zero now returns Empty instead of 0 so the cell
visually signals the error rather than silently showing a wrong value.
Updated the test to assert Empty.
- parser.rs: missing closing ')' in aggregate functions (SUM, AVG, etc.)
and IF(...) now returns a parse error instead of silently succeeding,
preventing malformed formulas from evaluating with unexpected results.
- dispatch.rs: SetCell validates that every category in the coords
exists before mutating anything; previously a typo in a category name
silently wrote an orphaned cell (stored but never visible in any grid)
and returned ok. Now returns an error immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
