From fb85e98abec7ad110ad70152c8c0320bb6c9fceb Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Wed, 8 Apr 2026 22:27:35 -0700 Subject: [PATCH] docs: add repository context files Add new context files to assist with repository navigation and design consistency: - context/repo-map.md: A roadmap for the repository. - context/design-principles.md: Guidelines for maintaining repository consistency. Update CLAUDE.md to include instructions on using the new context files. Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL) --- CLAUDE.md | 3 +- context/design-principles.md | 228 ++++++++++++++++ context/repo-map.md | 488 +++++++++++++++++++++++++++++++++++ 3 files changed, 718 insertions(+), 1 deletion(-) create mode 100644 context/design-principles.md create mode 100644 context/repo-map.md diff --git a/CLAUDE.md b/CLAUDE.md index c6439e7..278ef18 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,8 @@ - Option<...> or Result<...> are fine but should not be present in the majority of the code. - Similarly, code managing Box<...> or RC<...>, etc. for containers pointing to heap data should be split from logic - +- context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated. +- context/design-principles.md is also important for keeping the repository consistent. ## Beads Issue Tracker diff --git a/context/design-principles.md b/context/design-principles.md new file mode 100644 index 0000000..4593b37 --- /dev/null +++ b/context/design-principles.md @@ -0,0 +1,228 @@ +# Improvise Design Principles + +## 1. Functional-First Architecture + +### Commands Are Pure, Effects Are Side-Effectful + +Every user action flows through a two-phase pipeline: + +1. **Command** (`Cmd` trait) — reads immutable context, returns a list of effects. + The `CmdContext` is a read-only snapshot: model, layout, mode, cursor position. + Commands never touch `&mut App`. All decision logic is pure. + +2. **Effect** (`Effect` trait) — a small struct with an `apply(&self, app: &mut App)` method. + Each effect is one discrete, debuggable state change. The app applies them in order. + +This separation means: +- Commands are testable without a terminal or an `App` instance. +- Effects can be logged, replayed, or composed. +- The only place `App` is mutated is inside `Effect::apply`. + +### Prefer Transformations to Mutation + +Where possible, build new values rather than mutating in place: +- `CellKey::with(cat, item)` returns a new key with an added/replaced coordinate. +- `CellKey::without(cat)` returns a new key with a coordinate removed. +- Viewport positioning is computed as a pure function (`viewport_effects`) that + returns a `Vec`, not a method that pokes at scroll offsets directly. + +### Compose Small Pieces + +Commands compose via `Binding::Sequence` — a keymap entry can chain multiple +commands, each contributing effects independently. The `o` key (add row + begin +editing) is two commands composed at the binding level, not a monolithic handler. + +--- + +## 2. Polymorphism Over Conditionals + +### Dispatch Through Traits and Registries, Not Match Blocks + +- **Commands**: 40+ types each implement `Cmd`. A `CmdRegistry` maps names to + constructor closures. Dispatching a key presses looks up the binding, resolves + the command name through the registry, and calls `execute`. No central + `match command_name { ... }` block. + +- **Effects**: 50+ types each implement `Effect`. Collected into a `Vec>` + and applied in order. No `match effect_kind { ... }`. + +- **Keymaps**: Each mode has its own `Keymap` (a `HashMap`). + Mode dispatch is one table lookup, not a nested `match (mode, key)`. + +### Use Enums to Make Invalid States Unrepresentable + +- `BinOp` is an enum (`Add | Sub | Mul | ...`), not a string. Invalid operators + are caught at parse time, not silently ignored at eval time. + +- `Axis` is `Row | Column | Page | None`. A category is on exactly one axis. + Cycling is a four-state rotation — no boolean flags, no "row_or_column" ambiguity. + +- `Binding` is `Cmd | Prefix | Sequence`. The keymap lookup returns one of these + three shapes; dispatch pattern-matches exhaustively. + +- `CategoryKind` is `Regular | VirtualIndex | VirtualDim | Label`. Business rules + (e.g., the 12-category limit counts only `Regular`) are enforced by matching + on the enum, not by checking name prefixes. + +### When You Add a Variant, the Compiler Finds Every Call Site + +Prefer exhaustive `match` over `if let` or `_ =>` wildcards. When a new `Axis` +variant or `AppMode` is added, non-exhaustive matches produce compile errors +that guide you to every place that needs updating. + +--- + +## 3. Correctness by Construction + +### Canonical Forms Prevent Equivalence Bugs + +`CellKey::new()` sorts coordinates by category name. Two keys that name the same +intersection but in different order are identical after construction. Equality, +hashing, and storage all work correctly without callers needing to remember to +sort. Property tests verify this invariant. + +### Smart Constructors Enforce Invariants + +- `CellKey::new()` is the only way to build a key — it always sorts. +- `Category::add_item()` deduplicates by name and auto-assigns IDs via a private + counter. External code cannot fabricate an `ItemId`. +- `Model::add_category()` checks the 12-category limit before insertion. +- `Formula::new()` takes all required fields; there is no default/empty formula + to accidentally leave half-initialized. + +### Type-Safe Identifiers + +`CategoryId` and `ItemId` are typed aliases. While they are `usize` underneath, +using named types signals intent and prevents accidentally passing an item count +where an item ID is expected. + +### Symbol Interning for Data Integrity + +`DataStore` interns category and item names into `Symbol` values (small copyable +handles). This means: +- String comparison is integer comparison — fast and allocation-free. +- A secondary index maps `(Symbol, Symbol)` pairs to cell sets, enabling O(1) + lookups for aggregation queries. +- Symbols can only be created through the `SymbolTable`, so misspelled names + produce a distinct symbol rather than silently matching a wrong cell. + +### Parse-Time Validation + +Formulas are parsed into a typed AST (`Expr` enum) at entry time. If the syntax +is invalid, the user gets an error immediately. The evaluator only sees +well-formed trees — it does not need to handle malformed input. + +--- + +## 4. Separation of Concerns + +### Four Layers + +| Layer | Directory | Responsibility | +|-------|-----------|----------------| +| **Model** | `src/model/` | Categories, items, groups, cell data, formulas. Pure data, no rendering. | +| **View** | `src/view/` | Axis assignments, page selection, hidden items, layout computation. Derived from model. | +| **Command / Effect** | `src/command/`, `src/ui/effect.rs` | Intent (commands) and state mutation (effects). Bridges user input to model changes. | +| **Rendering** | `src/draw.rs`, `src/ui/` | Terminal drawing. Reads model + view, writes pixels. No mutation. | + +### Formulas Are Data, Not Code + +A formula is a serializable struct: raw text, target name, category, AST, optional +filter. It is stored in the model alongside regular data. The evaluator walks the +AST at read time. Formulas never become closures or runtime-generated code. + +### Display Rounding Is View-Only + +Number formatting (`format_f64`) rounds for display. Formula evaluation always +operates on full `f64` precision. The rounding function is only called in +rendering paths — never in `eval_formula` or aggregation. + +### Drill State Isolates Edits + +When editing aggregated (drill-down) cells, a `DrillState` snapshot freezes the +current cell set. Pending edits accumulate in a staging map. On commit, +`ApplyAndClearDrill` writes them all atomically. On cancel, the snapshot is +discarded. No partial writes reach the model. + +--- + +## 5. Guidelines for New Code + +- **Add a command, not a special case.** If you need new behavior on a keypress, + implement `Cmd`, register it, and bind it in the keymap. Do not add an + `if key == 'x'` branch inside `handle_key`. + +- **Return effects, do not mutate.** Your command's `execute` receives `&CmdContext` + (immutable). Produce a `Vec>`. If you need a new kind of state + change, create a new `Effect` struct. + +- **Use the type system.** If a value can only be one of N things, make it an enum. + If an invariant must hold, enforce it in the constructor. If a field is + optional, use `Option` — do not use sentinel values. + +- **Test the logic, not the wiring.** Commands are pure functions of context; + test them by building a `CmdContext` and asserting on the returned effects. + You do not need a terminal. + +- **Keep `Option`/`Result`/`Box` at the boundaries.** Core logic should work with + concrete values. Wrap in `Option` at the edges (parsing, lookup, I/O) and + unwrap early. Do not thread `Option` through deep call chains. + +--- + +## 6. Testing + +### Coverage and ambition + +Aim for **~80% line and branch coverage** on logic code. This is a quality floor — +go higher where the code is tricky or load-bearing, but don't pad coverage by +testing trivial getters or chasing 100% on rendering widgets. The test suite +should remain fast (under 2 seconds). Slow tests erode the habit of running them. + +### Demonstrate bugs before fixing them + +Write a test that **fails on the current code** before writing the fix. Prefer a +small unit test targeting the broken function over an end-to-end test. After the +fix, the test stays as a regression guard. Document the bug in the test's +doc-comment (see `model/types.rs` → `formula_tests` for examples). + +### Use property tests judiciously + +Property tests (`proptest`) are for **invariants that must hold across all +inputs** — not a replacement for example-based tests. Good candidates: + +- Structural invariants: CellKey is always sorted, each category lives on exactly + one axis, toggle-collapse is involutive, hide/show roundtrips. +- Serialization roundtrips: save/load identity. +- Determinism: `evaluate` returns the same result for the same inputs. + +Keep case counts at the default (256). Don't crank them to thousands — if a +property needs more cases to feel safe, constrain the input space with better +strategies rather than brute-forcing. Property tests that take hundreds of +milliseconds each are a sign something is wrong. + +### What to test + +- **Model, formula, view**: the core logic. Unit tests for each operation and + edge case. Property tests for invariants. These are the highest-value tests. +- **Commands**: build a `CmdContext`, call `execute`, assert on the returned + effects. Pure functions — no terminal needed. +- **Persistence**: round-trip tests (`save → load → save` produces identical + output). Cover groups, formulas, views, hidden items, legacy JSON. +- **Format**: boundary cases for comma placement, rounding, negative numbers. +- **Import**: field classification heuristics, CSV quoting, multi-file merge. + +### What not to test + +- Ratatui `Widget::render` implementations — pure drawing code that changes + often. Test the data they consume (layout, cat_tree, format) instead. +- Trivial data definitions (`ast.rs`, `axis.rs`). +- Module re-export files. + +### Test the property, not the implementation + +A test like "calling `set_axis(cat, Row)` sets the internal map entry to `Row`" +is brittle — it mirrors the implementation and breaks if the storage changes. +Instead test the observable contract: "after `set_axis(cat, Row)`, +`axis_of(cat)` returns `Row` and `categories_on(Row)` includes `cat`." This +style survives refactoring and catches real bugs. diff --git a/context/repo-map.md b/context/repo-map.md new file mode 100644 index 0000000..e3e1075 --- /dev/null +++ b/context/repo-map.md @@ -0,0 +1,488 @@ +# Repository Map (LLM Reference) + +Terminal pivot-table modeling app. Rust, Ratatui TUI, command/effect architecture. +Crate `improvise` v0.1.0, MIT, edition 2021. + +--- + +## How to Find Things + +| I need to... | Look in | +|---------------------------------------|----------------------------------------------| +| Add a new keybinding | `command/keymap.rs` → `default_keymaps()` | +| Add a new user-facing command | `command/cmd.rs` → implement `Cmd`, register in `default_registry()` | +| Add a new state mutation | `ui/effect.rs` → implement `Effect` | +| Change formula evaluation | `model/types.rs` → `eval_formula()`, `eval_expr()` | +| Change how cells are stored/queried | `model/cell.rs` → `DataStore` | +| Change category/item behavior | `model/category.rs` → `Category` | +| Change view axis logic | `view/types.rs` → `View` | +| Change grid layout computation | `view/layout.rs` → `GridLayout` | +| Change .improv file format | `persistence/mod.rs` → `format_md()`, `parse_md()` | +| Change number display formatting | `format.rs` → `format_f64()` | +| Change CLI arguments | `main.rs` → clap structs | +| Change import wizard logic | `import/wizard.rs` → `ImportPipeline` | +| Change grid rendering | `ui/grid.rs` → `GridWidget` | +| Change TUI frame layout | `draw.rs` → `draw()` | +| Change app state / mode transitions | `ui/app.rs` → `App`, `AppMode` | +| Write a test for model logic | `model/types.rs` → `mod tests` / `mod formula_tests` | +| Write a test for a command | `command/cmd.rs` → `mod tests` | + +--- + +## Core Types and Traits + +### Command/Effect Pipeline (the central architecture pattern) + +``` +User keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec> → Effect::apply(&mut App) + (immutable) (pure, read-only) (state mutations) +``` + +```rust +// src/command/cmd.rs +pub trait Cmd: Debug + Send + Sync { + fn name(&self) -> &'static str; + fn execute(&self, ctx: &CmdContext) -> Vec>; +} + +pub struct CmdContext<'a> { + pub model: &'a Model, // immutable + pub layout: &'a GridLayout, // immutable + pub registry: &'a CmdRegistry, + pub mode: &'a AppMode, + pub selected: (usize, usize), // (row, col) cursor + pub row_offset: usize, + pub col_offset: usize, + pub search_query: &'a str, + pub search_mode: bool, + pub yanked: &'a Option, + pub key_code: KeyCode, // the key that triggered this command + pub buffers: &'a HashMap, + pub expanded_cats: &'a HashSet, + // panel cursors, tile cursor, visible dimensions... +} + +// src/ui/effect.rs +pub trait Effect: Debug { + fn apply(&self, app: &mut App); + fn changes_mode(&self) -> bool { false } // override if effect changes AppMode +} +``` + +**To add a command**: implement `Cmd`, then in `default_registry()` call `r.register(...)` or use the `effect_cmd!` macro for simple cases. Bind it in `default_keymaps()`. + +**To add an effect**: implement `Effect` in `effect.rs`, add a constructor function. + +### Data Model + +```rust +// src/model/types.rs +pub struct Model { + pub name: String, + pub categories: IndexMap, // ordered + pub data: DataStore, + pub formulas: Vec, + pub views: IndexMap, + pub active_view: String, + pub measure_agg: HashMap, // per-measure aggregation override +} +// Key methods: +// add_category(&mut self, name) -> Result [max 12 regular] +// category(&self, name) -> Option<&Category> +// category_mut(&mut self, name) -> Option<&mut Category> +// set_cell(&mut self, key: CellKey, value: CellValue) +// evaluate(&self, key: &CellKey) -> Option [formulas + raw data] +// evaluate_aggregated(&self, key, none_cats) -> Option [sums over hidden dims] +// add_formula(&mut self, formula: Formula) [replaces same target+category] +// remove_formula(&mut self, target, category) +// category_names(&self) -> impl Iterator + +const MAX_CATEGORIES: usize = 12; // virtual categories don't count +``` + +```rust +// src/model/cell.rs +#[derive(Clone, PartialEq, Eq, Hash)] +pub struct CellKey(pub Vec<(String, String)>); // always sorted by category name +// CellKey::new(coords) — sorts on construction, enforcing canonical form +// CellKey::with(cat, item) -> Self — returns new key with coord added/replaced +// CellKey::without(cat) -> Self — returns new key with coord removed +// CellKey::get(cat) -> Option<&str> + +#[derive(Clone, PartialEq)] +pub enum CellValue { + Number(f64), + Text(String), +} +// CellValue::as_f64() -> Option + +pub struct DataStore { + cells: HashMap, + pub symbols: SymbolTable, + index: HashMap<(Symbol, Symbol), HashSet>, // secondary index +} +// DataStore::set(&mut self, key: &CellKey, value: CellValue) +// DataStore::get(&self, key: &CellKey) -> Option<&CellValue> +// DataStore::matching_values(&self, partial: &[(String,String)]) -> Vec +// DataStore::matching_cells(&self, partial) -> Vec<(CellKey, CellValue)> +``` + +```rust +// src/model/category.rs +pub struct Category { + pub id: CategoryId, // usize + pub name: String, + pub kind: CategoryKind, + pub items: IndexMap, // ordered + pub groups: IndexMap, + next_item_id: ItemId, // private, auto-increment +} +// Category::add_item(&mut self, name) -> ItemId [deduplicates by name] +// Category::ordered_item_names(&self) -> Vec<&str> [respects group order] + +pub enum CategoryKind { Regular, VirtualIndex, VirtualDim, Label } +``` + +### Formula System + +```rust +// src/formula/ast.rs +pub enum Expr { + Number(f64), + Ref(String), // reference to an item name + BinOp(BinOp, Box, Box), + UnaryMinus(Box), + Agg(AggFunc, Box, Option), + If(Box, Box, Box), +} +pub enum BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge } +pub enum AggFunc { Sum, Avg, Min, Max, Count } +pub struct Formula { + pub raw: String, // "Profit = Revenue - Cost" + pub target: String, // "Profit" + pub target_category: String, // "Measure" + pub expr: Expr, + pub filter: Option, // WHERE clause +} + +// src/formula/parser.rs +pub fn parse_formula(raw: &str, target_category: &str) -> Result +``` + +Formula evaluation is in `model/types.rs` → `eval_formula()` / `eval_expr()`. Operates at full f64 precision. Display rounding in `format.rs` is view-only. + +### View and Layout + +```rust +// src/view/axis.rs +pub enum Axis { Row, Column, Page, None } + +// src/view/types.rs +pub struct View { + pub name: String, + pub category_axes: IndexMap, + pub page_selections: HashMap, + pub hidden_items: HashMap>, + pub collapsed_groups: HashMap>, + pub number_format: String, // e.g. ",.0" or ",.2f" + pub prune_empty: bool, + // scroll/selection state... +} +// View::set_axis(&mut self, cat, axis) +// View::axis_of(&self, cat) -> Axis +// View::cycle_axis(&mut self, cat) [Row→Column→Page→None→Row] +// View::transpose(&mut self) [swap Row↔Column] +// View::categories_on(&self, axis) -> Vec<&str> +// View::on_category_added(&mut self, cat) [auto-assigns axis] + +// src/view/layout.rs +pub struct GridLayout { /* computed from Model + View */ } +// GridLayout::new(model, view) -> Self +// GridLayout::cell_key(row, col) -> Option +// GridLayout::cell_value(row, col) -> Option +// GridLayout::row_label(row) -> &str +// GridLayout::col_label(col) -> &str +// GridLayout::drill_records(row, col) -> Vec<(CellKey, CellValue)> +// Records mode: auto-detected when _Index on Row + _Dim on Column +``` + +### App State + +```rust +// src/ui/app.rs +pub enum AppMode { + Normal, + Editing { minibuf: MinibufferConfig }, + FormulaEdit { minibuf: MinibufferConfig }, + FormulaPanel, + CategoryPanel, + ViewPanel, + TileSelect, + CategoryAdd { minibuf: MinibufferConfig }, + ItemAdd { minibuf: MinibufferConfig }, + ExportPrompt { minibuf: MinibufferConfig }, + CommandMode { minibuf: MinibufferConfig }, + ImportWizard, + Help, + Quit, +} +// Note: SearchMode is Normal + search_mode:bool flag, not a separate variant. + +pub struct App { + pub model: Model, + pub mode: AppMode, + pub file_path: Option, + pub dirty: bool, + pub help_page: usize, + pub transient_keymap: Option>, // for prefix keys + // layout cache, drill_state, wizard, buffers, panel cursors, etc. +} +// App::handle_key(&mut self, KeyEvent) -> Result<()> [main input dispatch] +// App::rebuild_layout(&mut self) +// App::is_empty_model(&self) -> bool [true when only virtual categories exist] +``` + +### Keymap System + +```rust +// src/command/keymap.rs +pub enum KeyPattern { Key(KeyCode, KeyModifiers), AnyChar, Any } +pub enum Binding { + Cmd { name: &'static str, args: Vec }, + Prefix(Arc), // Emacs-style sub-keymap + Sequence(Vec<(&'static str, Vec)>), // multi-command chain +} +pub enum ModeKey { + Normal, Help, FormulaPanel, CategoryPanel, ViewPanel, TileSelect, + Editing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode, + SearchMode, ImportWizard, +} + +// Keymap::lookup(&self, key, mods) -> Option<&Binding> +// Fallback chain: exact(key,mods) → Char with NONE mods → AnyChar → Any + +// KeymapSet::default_keymaps() -> Self [builds all 14 mode keymaps] +// KeymapSet::dispatch(&self, ctx, key, mods) -> Option>> +``` + +--- + +## File Format (.improv) + +Plain-text markdown-like. **Not JSON** (JSON is legacy, auto-detected by `{` prefix). + +``` +# Model Name + +## Category: Region +- North +- South +- East [Coastal] ← item in group "Coastal" +- West [Coastal] +> Coastal ← group definition + +## Category: Measure +- Revenue +- Cost +- Profit + +## Formulas +- Profit = Revenue - Cost [Measure] ← [TargetCategory] + +## Data +Region=East, Measure=Revenue = 1200 +Region=East, Measure=Cost = 800 +Region=West, Measure=Revenue = "pending" ← text value in quotes + +## View: Default (active) +Region: row +Measure: column +Time: page, Q1 ← page axis with selected item +hidden: Region/Internal +collapsed: Time/2024 +format: ,.2f +``` + +Gzip variant: `.improv.gz` (same content, gzipped). Persistence code: `persistence/mod.rs`. + +--- + +## CLI + +``` +improvise [model.improv] # open TUI (default) +improvise import data.csv [--no-wizard] [-o out] # import CSV/JSON +improvise cmd 'add-cat Region' -f model.improv # headless command(s) +improvise script setup.txt -f model.improv # run script file +``` + +Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`. + +--- + +## Key Dependencies + +| Crate | Purpose | +|-------|---------| +| ratatui 0.29 | TUI framework | +| crossterm 0.28 | Terminal backend | +| clap 4.6 (derive) | CLI parsing | +| serde + serde_json | Serialization | +| indexmap 2 | Ordered maps (categories, views) | +| anyhow | Error handling | +| chrono 0.4 | Date parsing in import | +| flate2 | Gzip for .improv.gz | +| csv | CSV parsing | +| enum_dispatch | CLI subcommand dispatch | +| **dev:** proptest, tempfile | Property testing, temp dirs | + +--- + +## File Inventory + +Lines / tests / path — grouped by layer. + +### Model layer +``` +1692 / 66t model/types.rs Model struct, formula eval, CRUD, MAX_CATEGORIES=12 + 621 / 28t model/cell.rs CellKey (canonical sort), CellValue, DataStore (interned) + 216 / 6t model/category.rs Category, Item, Group, CategoryKind + 79 / 3t model/symbol.rs Symbol interning (SymbolTable) + 6 / 0t model/mod.rs +``` + +### Formula layer +``` + 461 / 8t formula/parser.rs Recursive descent parser → Formula AST + 77 / 0t formula/ast.rs Expr, BinOp, AggFunc, Formula, Filter (data only) + 5 / 0t formula/mod.rs +``` + +### View layer +``` +1013 / 23t view/layout.rs GridLayout (pure fn of Model+View), records mode, drill + 521 / 28t view/types.rs View config (axes, pages, hidden, collapsed, format) + 21 / 0t view/axis.rs Axis enum {Row, Column, Page, None} + 7 / 0t view/mod.rs +``` + +### Command layer +``` +3373 / 21t command/cmd.rs Cmd trait, CmdContext, CmdRegistry, 40+ commands +1068 / 22t command/keymap.rs KeyPattern, Binding, Keymap, ModeKey, 14 mode keymaps + 236 / 19t command/parse.rs Script/command-line parser (prefix syntax) + 12 / 0t command/mod.rs +``` + +### UI layer +``` + 942 / 0t ui/effect.rs Effect trait, 50+ effect types (all state mutations) + 914 / 30t ui/app.rs App state, AppMode (15 variants), handle_key, autosave +1036 / 13t ui/grid.rs GridWidget (ratatui), col widths, rendering + 617 / 0t ui/help.rs 5-page help overlay, HELP_PAGE_COUNT=5 + 347 / 0t ui/import_wizard_ui.rs Import wizard overlay rendering + 165 / 6t ui/cat_tree.rs Category tree flattener for panel + 113 / 0t ui/view_panel.rs View list panel + 107 / 0t ui/category_panel.rs Category tree panel + 95 / 0t ui/tile_bar.rs Tile bar (axis assignment tiles) + 87 / 0t ui/panel.rs Generic panel frame widget + 81 / 0t ui/formula_panel.rs Formula list panel + 67 / 0t ui/which_key.rs Prefix-key hint popup + 12 / 0t ui/mod.rs +``` + +### Import layer +``` + 773 / 15t import/wizard.rs ImportPipeline + ImportWizard + 292 / 9t import/analyzer.rs Field kind detection (Category/Measure/Time/Skip) + 244 / 8t import/csv_parser.rs CSV parsing, multi-file merge + 3 / 0t import/mod.rs +``` + +### Top-level +``` + 400 / 0t draw.rs TUI event loop (run_tui), frame composition + 391 / 0t main.rs CLI entry (clap): open, import, cmd, script + 228 / 29t format.rs Number display formatting (view-only rounding) + 806 / 22t persistence/mod.rs .improv save/load (markdown format + gzip + legacy JSON) +``` + +### Context docs +``` +context/SPEC.md Feature specification +context/design-principles.md Architectural principles +context/plan.md Show HN launch plan +context/repo-map.md This file +``` + +**Total: ~15,875 lines, 356 tests.** + +--- + +## Testing Guidelines + +### Coverage target + +Aim for **~80% line and branch coverage** on logic code. This is a quality floor, not a +ceiling — go higher where the code warrants it, but don't chase 100% on rendering +widgets or write tests that just exercise trivial getters. Coverage should be run with +`cargo llvm-cov` (available via `nix develop`). + +### What to test and how + +| Layer | Approach | Notes | +|-------|----------|-------| +| **Model** (types, cell, category, symbol) | Unit tests + **proptest** | The data model is the foundation. Property tests catch invariant violations that hand-picked cases miss (see CellKey sort invariant, axis consistency). | +| **Formula** (parser, eval) | Unit tests per operator/construct | Cover each BinOp, AggFunc, IF, WHERE, unary minus, chained formulas, error cases (div-by-zero, missing ref). Ensure eval uses full f64 precision — never display-rounded values. | +| **View** (types, layout) | Unit tests + **proptest** | Property tests for axis assignment invariants (each category on exactly one axis, transpose is involutive, etc.). Unit tests for layout computation, records mode detection, drill. | +| **Command** (cmd, keymap, parse) | Unit tests | Test command execution by building a `CmdContext` and asserting on returned effects. Test keymap lookup fallback chain. Test script parser with edge cases (quoting, comments, dots). | +| **Persistence** | Round-trip tests | `save → load → save` must be identical. Cover groups, formulas, views, hidden items, legacy JSON detection. | +| **Format** | Unit tests | Boundary cases: comma placement at 3/4/7 digits, negative numbers, rounding half-away-from-zero (not banker's), zero, small fractions. | +| **Import** (analyzer, csv, wizard) | Unit tests | Field classification heuristics, CSV quoting (RFC 4180), multi-file merge, date extraction. | +| **UI rendering** (grid, panels, draw, help) | Generally skip | Ratatui widgets are hard to unit-test and change frequently. Test the *logic* they consume (layout, cat_tree, format) rather than the rendering itself. | +| **Effects** | Test indirectly | Effects are thin `apply` methods. Test via integration: send a key through `App::handle_key` and assert on resulting app state. The complex ones (drill reconciliation, import) deserve targeted unit tests. | + +### Property tests (proptest) + +Use property tests for **invariants that must hold across all inputs**, not as a +substitute for example-based tests. Good candidates: + +- Structural invariants: CellKey always sorted, each category on exactly one axis, + toggle-collapse is involutive, hide/show roundtrips. +- Serialization roundtrips: save/load identity. +- Determinism: `evaluate` returns the same value for the same inputs. + +Keep proptest case counts reasonable. The defaults (256 cases) are fine for most +properties. Don't crank them up to thousands — the test suite should complete in +under 2 seconds. If a property needs more cases to feel confident, that's a sign +the input space should be constrained with better strategies, not brute-forced. + +### Bug-fix workflow + +Per CLAUDE.md: **write a test that demonstrates the bug before fixing it.** Prefer +a small unit test targeting the specific function over an integration test. The test +should fail on the current code, then pass after the fix. Mark regression tests +with a doc-comment explaining the bug (see `model/types.rs` `formula_tests` for +examples). + +### What not to test + +- Trivial struct constructors and enum definitions (`ast.rs`, `axis.rs`). +- Ratatui `Widget::render` implementations — these are pure drawing code. +- Module re-export files (`mod.rs`). +- One-line delegation methods. + +--- + +## Patterns to Know + +1. **Commands never mutate.** They receive `&CmdContext` (read-only) and return `Vec>`. +2. **CellKey is always sorted.** Use `CellKey::new()` — never construct the inner Vec directly. +3. **`category_mut()` for adding items.** `Model` has no `add_item` method; get the category first: `m.category_mut("Region").unwrap().add_item("East")`. +4. **Virtual categories** `_Index` and `_Dim` always exist. `is_empty_model()` checks whether any *non-virtual* categories exist. +5. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full f64. +6. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the current mode, then the corresponding `Keymap` is looked up. Normal + `search_mode=true` maps to `SearchMode`. +7. **`effect_cmd!` macro** generates a command struct that just produces effects. Use for simple commands without complex logic. +8. **`.improv` format is markdown-like**, not JSON. See `persistence/mod.rs`. JSON is legacy only. +9. **`IndexMap`** is used for categories and views to preserve insertion order. +10. **`MAX_CATEGORIES = 12`** applies only to `CategoryKind::Regular`. Virtual/Label categories are exempt.