Commit Graph

37 Commits

Author SHA1 Message Date
24c59fbf40 fix: Model::add_formula auto-adds target item; fix formula panel display
Model::add_formula now ensures the formula target exists as an item in
the target category (same fix as AddFormula effect, but at the model
level — covers file loading and headless paths).

Formula panel now shows raw formula text instead of debug-formatted
string with redundant target name and quotes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:07:59 -07:00
737d14a5c0 fix: add depth limit to formula evaluation, propagate errors
Circular or self-referencing formulas now return CellValue::Error
instead of stack overflowing. eval_expr uses Result<f64, String>
internally so errors (circular refs, div/0, missing refs) propagate
immediately through the expression tree via ?. The depth limit (16)
is checked per evaluate_depth call — normal 1-2 level chains are
unaffected.

Also adds CellValue::Error variant for displaying ERR:reason in the
grid, and handles it in format, persistence, and search.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:07:58 -07:00
6239ac83ad chore: fmt + clippy 2026-04-11 00:07:57 -07:00
db170a10b8 feat(model): add regular_category_names to Model
Updates `Model` to include `regular_category_names` , which returns
category names excluding virtual categories (_Index, _Dim). This allows
other parts of the application to distinguish between user-defined
categories and system-internal ones.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
2026-04-11 00:06:50 -07:00
da076eadc8 test(model,ui): add tests for precision and cat tree
Add unit tests for model precision and category tree rendering:

- `src/model/types.rs` : Added `formula_chain_preserves_full_precision` to
  ensure formulas use full `f64` precision for calculations, even when
  display is rounded.
- `src/ui/cat_tree.rs` : Added comprehensive tests for `build_cat_tree` ,
  covering empty models (virtual categories), expanded/collapsed states,
  and item rendering.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
2026-04-11 00:06:50 -07:00
3885fc19c8 refactor(test): split let statements and clean up vec! macro
Split multi-line let statements in command tests into separate lines.
Remove unnecessary commas and inline vec! macro. Clean up test code
formatting for readability. No functional changes, only style improvements.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (bartowski/nvidia_Nemotron-Cascade-2-30B-A3B-GGUF)
2026-04-11 00:06:48 -07:00
6f3af34056 chore: clippy 2026-04-11 00:06:48 -07:00
56838c0a61 style: cleanup formatting and code style across the project
Clean up formatting and code style across the project.

- Remove unnecessary whitespace and empty lines in `src/model/cell.rs` ,
  `src/model/types.rs` , and `src/draw.rs` .
- Reformat long lines and function calls in `src/command/cmd.rs` ,
  `src/ui/grid.rs` , and `src/ui/tile_bar.rs` for better readability.
- Consolidate imports and simplify expressions in `src/command/keymap.rs` and
  `src/ui/app.rs` .

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
2026-04-11 00:06:48 -07:00
9e02939b66 refactor: update TogglePanelAndFocus to use open/focused flags
Update TogglePanelAndFocus and related components to use open/focused flags.

Changed TogglePanelAndFocus from currently_open to open+focused flags.
Parser accepts optional [open] [focused] arguments.

Interactive mode toggles between open+focus and closed/unfocused.

Keymap updates: F/C/V in panel modes close panels when focused.

Model initialization: Virtual categories _Index/_Dim default to None
axis, regular categories auto-assign Row/Column/Page.

App test context updated with visible_rows/visible_cols.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-11 00:06:48 -07:00
55cad99ae1 feat: add new commands for records mode and category management
Add new commands for enhanced data entry and category management.

AddRecordRow: Adds a new record row in records mode with empty value.
TogglePruneEmpty: Toggles pruning of empty rows/columns in pivot mode.
ToggleRecordsMode: Switches between records and pivot layout.
DeleteCategoryAtCursor: Removes a category and all its cells.
ToggleCatExpand: Expands/collapses a category in the tree.
FilterToItem: Filters to show only items matching cursor position.

Model gains remove_category() and remove_item() to delete categories
and items along with all referencing cells and formulas.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-11 00:06:48 -07:00
34df174b9b feat(import): add Label field support for high-cardinality per-row data
Add support for Label-kind categories to handle high-cardinality
per-row fields like descriptions, IDs, and notes. These fields are
stored alongside regular categories but default to Axis::None and
are excluded from pivot category limits.

Changes:
- analyzer.rs: Label fields now default to accepted=true
- wizard.rs: Collect and process label fields during model building,
  attaching label values as coordinates for each cell
- category.rs: Add Label variant to CategoryKind enum
- types.rs: Add add_label_category() method and update category
  counting to only include Regular-kind categories

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-06 08:58:23 -07:00
19645a34cf feat: add records mode (long-format view) for drill-down
Implement records mode (long-format view) when drilling into aggregated cells.

Key changes:
- DrillIntoCell now creates views with _Index on Row and _Dim on Column
- GridLayout detects records mode and builds a records list instead of
  cross-product row/col items
- Added records_display() to render individual cell values in records mode
- GridWidget and CSV export updated to handle records mode rendering
- category_names() now includes virtual categories (_Index, _Dim)
- Tests updated to reflect virtual categories counting toward limits

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-05 11:10:41 -07:00
67041dd4a5 feat: add view history navigation and drill-into-cell
Add view navigation history with back/forward stacks (bound to < and >).

Introduce CategoryKind enum to distinguish regular categories from
virtual ones (_Index, _Dim) that are synthesized at query time.

Add DrillIntoCell command that creates a drill view showing raw data
for an aggregated cell, expanding categories on Axis::None into Row
and Column axes while filtering by the cell's fixed coordinates.

Virtual categories default to Axis::None and are automatically added
to all views when the model is initialized.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-05 10:57:28 -07:00
377d417e5a feat(model): add symbol table module
Add a new SymbolTable module for interned string identifiers.

The module implements a bidirectional mapping between strings and
Symbol IDs using a HashMap. Key functionality includes:

- intern(): Add a string to the table and return its Symbol ID
- get(): Look up a string by Symbol ID
- resolve(): Get the original string for a Symbol ID
- intern_pair() and intern_coords(): Helper functions for structured
  data interning

The implementation includes unit tests to verify correct behavior.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-05 01:09:17 -07:00
82ad459c4e feat: intern cell keys for O(1) comparison and indexing
Refactor DataStore to use interned keys (InternedKey) instead of
string-based CellKey for O(1) hash and compare operations.

Introduce SymbolTable-backed interning for all category and item
names, storing them as Symbol identifiers throughout the data structure.

Add secondary index mapping (category, item) pairs to sets of interned
keys, enabling efficient partial match queries without scanning all cells.

Optimize matching_values() to avoid allocating CellKey strings by
working directly with interned keys and intersecting index sets.

Update all callers to use new API: iter_cells(), matching_values(),
and internal lookup_key() helper.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/Qwen3.5-35B-A3B-GGUF:Q5_K_M)
2026-04-05 01:07:08 -07:00
a73fe160c7 feat: date parsing, component extraction, and wizard formulas
Extend FieldProposal with chrono-based date format detection and
configurable component extraction (Year, Month, Quarter). Add
ConfigureDates and DefineFormulas wizard steps to ImportPipeline.
build_model injects derived date categories and parses formula strings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:41:05 -07:00
5a251a1cbe feat: add Axis::None for hidden dimensions with implicit aggregation
Categories on the None axis are excluded from the grid and cell keys.
When evaluating cells, values across hidden dimensions are aggregated
using a per-measure function (default SUM). Adds evaluate_aggregated
to Model, none_cats to GridLayout, and 'n' shortcut in TileSelect.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:38:35 -07:00
edd6431444 refactor: use data_col_to_visual via group_for helpers, add column group toggle
Add row_group_for/col_group_for to GridLayout, replacing inline
backward-search logic. Refactor grid renderer to use col_group_for
instead of pre-filtering col_items. Add gz keybinding for column
group collapse toggle, symmetric with z for rows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 10:22:00 -07:00
5136aadd86 fix: remove tests for top_level_groups 2026-04-02 10:18:32 -07:00
edd33d6dee chore: cleanup dead code in category.rs 2026-04-02 10:07:49 -07:00
2c9d9c7de7 chore: move csv_path_p, restructure modules 2026-04-02 10:01:51 -07:00
183b2350f7 chore: reformat
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 00:07:22 -07:00
4fb97c89ed fix(tests): restore coverage for toggle_group_collapse, item group, and insertion order
Rewrites three commented-out tests to access public fields directly
instead of the removed item_by_name/item_index/is_group_collapsed methods:
- add_item_in_group_sets_group: uses items.get()
- item_index_reflects_insertion_order: uses items.get_index_of()
- toggle_group_collapse_toggles_twice + involutive proptest: inspect collapsed_groups directly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 23:08:06 -07:00
a1b17dc9af refactor: remove dead code, replace sum_matching tests with evaluate()
Removes unused methods (sum_matching, get_mut, item_by_name, item_index,
top_level_groups, is_group_collapsed, show_item) and unused constants
(LABEL_THRESHOLD, MIN_COL_WIDTH).

The sum_matching tests in model.rs were bypassing the formula evaluator
entirely. Replaced them with equivalent tests that call evaluate() against
the existing Total = SUM(Revenue) formula, exercising the real aggregation
code path.

Also fixes a compile error in view.rs prop_tests where View/Axis imports
and a doc comment were incorrectly commented out.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:56:04 -07:00
c8b88c63b3 refactor: eliminate Box<dyn Iterator> and Option sentinels in export_csv
The CSV export used Box<dyn Iterator<Item = Option<usize>>> to unify
empty and non-empty row iteration, violating the CLAUDE.md rule that
Box/Rc container management should be split from logic. Replaced with a
direct for loop over row indices, removing both the Box and the Option
sentinels used to represent "placeholder empty row/col".

Also removes unused pub use cell::CellKey re-export and an unused
import in cell.rs tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 09:00:37 -07:00
6038cb2d81 refactor: make active_view and axis_of infallible
Both functions previously returned Option despite their invariants
guaranteeing a value: active_view always names an existing view
(maintained by new/switch_view/delete_view), and axis_of only returns
None for categories never registered with the view (a programming error).

Callers no longer need to handle the impossible None case, eliminating
~15 match/if-let Option guards across app.rs, dispatch.rs, grid.rs,
tile_bar.rs, and category_panel.rs.

Also adds Model::evaluate_f64 (returns 0.0 for empty cells) and collapses
the double match-on-axis pattern in tile_bar/category_panel into a single
axis_display(Axis) helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 09:00:25 -07:00
a2e519efcc refactor: replace CellValue::Empty with Option<CellValue>
Previously CellValue had three variants: Number, Text, and Empty.
The Empty variant acted as a null sentinel, but the compiler could not
distinguish between "this is a real value" and "this might be empty".
Code that received a CellValue could use it without checking for Empty,
because there was no type-level enforcement.

Now CellValue has only Number and Text. The absence of a value is
represented as None at every API boundary:

  DataStore::get()    → Option<&CellValue>  (was &CellValue / Empty)
  Model::get_cell()   → Option<&CellValue>  (was &CellValue / Empty)
  Model::evaluate()   → Option<CellValue>   (was CellValue::Empty)
  eval_formula()      → Option<CellValue>   (was CellValue::Empty)

Model gains clear_cell() for explicit key removal; ClearCell dispatch
calls it instead of set_cell(key, CellValue::Empty).

The compiler now forces every caller of evaluate/get_cell to handle
the None case explicitly — accidental use of an empty value as if it
were real is caught at compile time rather than silently computing
wrong results.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 08:06:51 -07:00
e680b098ec refactor: remove Axis::Unassigned; axis_of returns Option<Axis>
Axis::Unassigned served two purposes that Option already covers:
  1. "this category has no assignment yet" → None
  2. "this category doesn't exist" → None

By removing the variant and changing axis_of to return Option<Axis>,
callers are forced by the compiler to handle the absent-category case
explicitly (via match or unwrap_or), rather than silently treating it
like a real axis value.

SetAxis { axis: String } also upgraded to SetAxis { axis: Axis }.
Previously, constructing SetAxis with an invalid string (e.g. "diagonal")
would compile and then silently fail at dispatch. Now the type only admits
valid axis values; the dispatch string-parser is gone.

Axis gains #[serde(rename_all = "lowercase")] so existing JSON command
files (smoke.jsonl, etc.) using "row"/"column"/"page" continue to work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 07:32:48 -07:00
599f1adcbd refactor: replace BinOp string with typed enum in Expr AST
Previously Expr::BinOp(String, ...) accepted any string as an operator.
Invalid operators (e.g. "diagonal") would compile fine and silently
return CellValue::Empty at eval time.

Now BinOp is an enum with variants Add/Sub/Mul/Div/Pow/Eq/Ne/Lt/Gt/Le/Ge.
The parser produces enum variants directly; the evaluator pattern-matches
exhaustively with no fallback branch. An invalid operator is now a
compile error at the call site, and the compiler ensures every variant
is handled in both eval_expr and eval_bool.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 00:20:08 -07:00
5434a60cc4 refactor: make Model::formulas private, expose read-only accessor
Previously `pub formulas: Vec<Formula>` allowed any code to call
`model.formulas.push(formula)` directly, bypassing the dedup logic in
`add_formula` that enforces the (target, target_category) uniqueness
invariant.

Making the field private means the only mutation paths are
`add_formula` and `remove_formula`, both of which maintain the invariant.
A `pub fn formulas(&self) -> &[Formula]` accessor preserves read access
for the UI and tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 00:18:43 -07:00
9ef0a72894 fix: formula evaluation and management bugs
Three bugs fixed, each with a failing regression test added first:

1. WHERE filter fallthrough: when the filter's category was absent from the
   cell key, the formula was applied unconditionally. Now returns the raw
   stored value (no formula applied) when the category is missing.

2. Agg inner/filter ignored: SUM(Revenue) was summing ALL cells in the
   partial slice rather than constraining to the Revenue item. Now resolves
   the inner Ref to its category and pins that coordinate before scanning.

3. Formula dedup by target only: add_formula and remove_formula keyed on
   target name alone, so two formulas with the same item name in different
   categories would collide. Both now key on (target, target_category).

RemoveFormula command updated to carry target_category.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 00:11:33 -07:00
09caf815d3 test: add proptest property-based tests
Add proptest dependency and property tests for:
- CellKey: key normalization invariants (sort order, dedup, round-trip,
  prefix non-equality, merge commutativity)
- View: axis exclusivity, set_axis, idempotency, page_selection roundtrip,
  hide/show roundtrip, toggle_group_collapse involution

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 00:10:44 -07:00
0cb491901d fix: formula correctness and command validation bugs
- model.rs: division by zero now returns Empty instead of 0 so the cell
  visually signals the error rather than silently showing a wrong value.
  Updated the test to assert Empty.
- parser.rs: missing closing ')' in aggregate functions (SUM, AVG, etc.)
  and IF(...) now returns a parse error instead of silently succeeding,
  preventing malformed formulas from evaluating with unexpected results.
- dispatch.rs: SetCell validates that every category in the coords
  exists before mutating anything; previously a typo in a category name
  silently wrote an orphaned cell (stored but never visible in any grid)
  and returned ok. Now returns an error immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:32:14 -07:00
cc072e192d Chore: remove unused imports and suppress unused variable warnings
Removes dead use statements across dispatch, formula, import, model, and
UI modules. Prefixes intentionally unused variables with _ in app.rs,
analyzer.rs, and grid.rs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 23:35:50 -07:00
56d11aee74 Test: add unit tests co-located with the code they cover
- model/cell.rs: CellKey (sorting, get, with, without, matches_partial,
  display) and DataStore (set/get, overwrite, empty-removes-key,
  sum_matching, matching_cells, text exclusion)
- model/category.rs: item ids, deduplication, group assignment,
  top_level_groups, item_index insertion order
- formula/parser.rs: subtraction, WHERE clause, SUM/AVG, IF,
  numeric literal, chained arithmetic, error on missing =
- view/view.rs: auto-axis assignment, set_axis, categories_on,
  page_selection, group collapse toggle, hide/show, cycle_axis
  (all transitions + scroll/selection reset)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 23:28:48 -07:00
6a9d28ecd6 Fix formula evaluator infinite recursion on unresolved item references
When a formula Ref resolves to an item that doesn't exist in any
category, find_item_category returned None and the fallback
unwrap_or(name) used the item name as a category name. The resulting
key still matched the formula target, causing infinite recursion and a
stack overflow. Now returns None immediately via ? so the expression
evaluates to Empty.

Adds unit tests for Model (structure, cell I/O, views), formula
evaluation (arithmetic, WHERE, aggregation, IF), and a full
five-category Region×Product×Channel×Time×Measure integration test.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 23:28:37 -07:00
eae00522e2 Initial implementation of Improvise TUI
Multi-dimensional data modeling terminal application with:
- Core data model: categories, items, groups, sparse cell store
- Formula system: recursive-descent parser, named formulas, WHERE clauses
- View system: Row/Column/Page axes, tile-based pivot, page slicing
- JSON import wizard (interactive TUI + headless auto-mode)
- Command layer: all mutations via typed Command enum for headless replay
- TUI: Ratatui grid, tile bar, formula/category/view panels, help overlay
- Persistence: .improv (JSON), .improv.gz (gzip), CSV export, autosave
- Static binary via x86_64-unknown-linux-musl + nix flake devShell
- Headless mode: --cmd '{"op":"..."}' and --script file.jsonl

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 21:11:55 -07:00