docs: add repository context files
Add new context files to assist with repository navigation and design consistency: - context/repo-map.md: A roadmap for the repository. - context/design-principles.md: Guidelines for maintaining repository consistency. Update CLAUDE.md to include instructions on using the new context files. Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
This commit is contained in:
Edward Langley
@ -4,7 +4,8 @@
|
||||
- Option<...> or Result<...> are fine but should not be present in the majority of the code.
|
||||
- Similarly, code managing Box<...> or RC<...>, etc. for containers pointing to heap data should be split
|
||||
from logic
|
||||
|
||||
- context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated.
|
||||
- context/design-principles.md is also important for keeping the repository consistent.
|
||||
|
||||
<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
|
||||
## Beads Issue Tracker
|
||||
|
||||
228
context/design-principles.md
Normal file
228
context/design-principles.md
Normal file
@ -0,0 +1,228 @@
|
||||
# Improvise Design Principles
|
||||
|
||||
## 1. Functional-First Architecture
|
||||
|
||||
### Commands Are Pure, Effects Are Side-Effectful
|
||||
|
||||
Every user action flows through a two-phase pipeline:
|
||||
|
||||
1. **Command** (`Cmd` trait) — reads immutable context, returns a list of effects.
|
||||
The `CmdContext` is a read-only snapshot: model, layout, mode, cursor position.
|
||||
Commands never touch `&mut App`. All decision logic is pure.
|
||||
|
||||
2. **Effect** (`Effect` trait) — a small struct with an `apply(&self, app: &mut App)` method.
|
||||
Each effect is one discrete, debuggable state change. The app applies them in order.
|
||||
|
||||
This separation means:
|
||||
- Commands are testable without a terminal or an `App` instance.
|
||||
- Effects can be logged, replayed, or composed.
|
||||
- The only place `App` is mutated is inside `Effect::apply`.
|
||||
|
||||
### Prefer Transformations to Mutation
|
||||
|
||||
Where possible, build new values rather than mutating in place:
|
||||
- `CellKey::with(cat, item)` returns a new key with an added/replaced coordinate.
|
||||
- `CellKey::without(cat)` returns a new key with a coordinate removed.
|
||||
- Viewport positioning is computed as a pure function (`viewport_effects`) that
|
||||
returns a `Vec<Effect>`, not a method that pokes at scroll offsets directly.
|
||||
|
||||
### Compose Small Pieces
|
||||
|
||||
Commands compose via `Binding::Sequence` — a keymap entry can chain multiple
|
||||
commands, each contributing effects independently. The `o` key (add row + begin
|
||||
editing) is two commands composed at the binding level, not a monolithic handler.
|
||||
|
||||
---
|
||||
|
||||
## 2. Polymorphism Over Conditionals
|
||||
|
||||
### Dispatch Through Traits and Registries, Not Match Blocks
|
||||
|
||||
- **Commands**: 40+ types each implement `Cmd`. A `CmdRegistry` maps names to
|
||||
constructor closures. Dispatching a key presses looks up the binding, resolves
|
||||
the command name through the registry, and calls `execute`. No central
|
||||
`match command_name { ... }` block.
|
||||
|
||||
- **Effects**: 50+ types each implement `Effect`. Collected into a `Vec<Box<dyn Effect>>`
|
||||
and applied in order. No `match effect_kind { ... }`.
|
||||
|
||||
- **Keymaps**: Each mode has its own `Keymap` (a `HashMap<KeyPattern, Binding>`).
|
||||
Mode dispatch is one table lookup, not a nested `match (mode, key)`.
|
||||
|
||||
### Use Enums to Make Invalid States Unrepresentable
|
||||
|
||||
- `BinOp` is an enum (`Add | Sub | Mul | ...`), not a string. Invalid operators
|
||||
are caught at parse time, not silently ignored at eval time.
|
||||
|
||||
- `Axis` is `Row | Column | Page | None`. A category is on exactly one axis.
|
||||
Cycling is a four-state rotation — no boolean flags, no "row_or_column" ambiguity.
|
||||
|
||||
- `Binding` is `Cmd | Prefix | Sequence`. The keymap lookup returns one of these
|
||||
three shapes; dispatch pattern-matches exhaustively.
|
||||
|
||||
- `CategoryKind` is `Regular | VirtualIndex | VirtualDim | Label`. Business rules
|
||||
(e.g., the 12-category limit counts only `Regular`) are enforced by matching
|
||||
on the enum, not by checking name prefixes.
|
||||
|
||||
### When You Add a Variant, the Compiler Finds Every Call Site
|
||||
|
||||
Prefer exhaustive `match` over `if let` or `_ =>` wildcards. When a new `Axis`
|
||||
variant or `AppMode` is added, non-exhaustive matches produce compile errors
|
||||
that guide you to every place that needs updating.
|
||||
|
||||
---
|
||||
|
||||
## 3. Correctness by Construction
|
||||
|
||||
### Canonical Forms Prevent Equivalence Bugs
|
||||
|
||||
`CellKey::new()` sorts coordinates by category name. Two keys that name the same
|
||||
intersection but in different order are identical after construction. Equality,
|
||||
hashing, and storage all work correctly without callers needing to remember to
|
||||
sort. Property tests verify this invariant.
|
||||
|
||||
### Smart Constructors Enforce Invariants
|
||||
|
||||
- `CellKey::new()` is the only way to build a key — it always sorts.
|
||||
- `Category::add_item()` deduplicates by name and auto-assigns IDs via a private
|
||||
counter. External code cannot fabricate an `ItemId`.
|
||||
- `Model::add_category()` checks the 12-category limit before insertion.
|
||||
- `Formula::new()` takes all required fields; there is no default/empty formula
|
||||
to accidentally leave half-initialized.
|
||||
|
||||
### Type-Safe Identifiers
|
||||
|
||||
`CategoryId` and `ItemId` are typed aliases. While they are `usize` underneath,
|
||||
using named types signals intent and prevents accidentally passing an item count
|
||||
where an item ID is expected.
|
||||
|
||||
### Symbol Interning for Data Integrity
|
||||
|
||||
`DataStore` interns category and item names into `Symbol` values (small copyable
|
||||
handles). This means:
|
||||
- String comparison is integer comparison — fast and allocation-free.
|
||||
- A secondary index maps `(Symbol, Symbol)` pairs to cell sets, enabling O(1)
|
||||
lookups for aggregation queries.
|
||||
- Symbols can only be created through the `SymbolTable`, so misspelled names
|
||||
produce a distinct symbol rather than silently matching a wrong cell.
|
||||
|
||||
### Parse-Time Validation
|
||||
|
||||
Formulas are parsed into a typed AST (`Expr` enum) at entry time. If the syntax
|
||||
is invalid, the user gets an error immediately. The evaluator only sees
|
||||
well-formed trees — it does not need to handle malformed input.
|
||||
|
||||
---
|
||||
|
||||
## 4. Separation of Concerns
|
||||
|
||||
### Four Layers
|
||||
|
||||
| Layer | Directory | Responsibility |
|
||||
|-------|-----------|----------------|
|
||||
| **Model** | `src/model/` | Categories, items, groups, cell data, formulas. Pure data, no rendering. |
|
||||
| **View** | `src/view/` | Axis assignments, page selection, hidden items, layout computation. Derived from model. |
|
||||
| **Command / Effect** | `src/command/`, `src/ui/effect.rs` | Intent (commands) and state mutation (effects). Bridges user input to model changes. |
|
||||
| **Rendering** | `src/draw.rs`, `src/ui/` | Terminal drawing. Reads model + view, writes pixels. No mutation. |
|
||||
|
||||
### Formulas Are Data, Not Code
|
||||
|
||||
A formula is a serializable struct: raw text, target name, category, AST, optional
|
||||
filter. It is stored in the model alongside regular data. The evaluator walks the
|
||||
AST at read time. Formulas never become closures or runtime-generated code.
|
||||
|
||||
### Display Rounding Is View-Only
|
||||
|
||||
Number formatting (`format_f64`) rounds for display. Formula evaluation always
|
||||
operates on full `f64` precision. The rounding function is only called in
|
||||
rendering paths — never in `eval_formula` or aggregation.
|
||||
|
||||
### Drill State Isolates Edits
|
||||
|
||||
When editing aggregated (drill-down) cells, a `DrillState` snapshot freezes the
|
||||
current cell set. Pending edits accumulate in a staging map. On commit,
|
||||
`ApplyAndClearDrill` writes them all atomically. On cancel, the snapshot is
|
||||
discarded. No partial writes reach the model.
|
||||
|
||||
---
|
||||
|
||||
## 5. Guidelines for New Code
|
||||
|
||||
- **Add a command, not a special case.** If you need new behavior on a keypress,
|
||||
implement `Cmd`, register it, and bind it in the keymap. Do not add an
|
||||
`if key == 'x'` branch inside `handle_key`.
|
||||
|
||||
- **Return effects, do not mutate.** Your command's `execute` receives `&CmdContext`
|
||||
(immutable). Produce a `Vec<Box<dyn Effect>>`. If you need a new kind of state
|
||||
change, create a new `Effect` struct.
|
||||
|
||||
- **Use the type system.** If a value can only be one of N things, make it an enum.
|
||||
If an invariant must hold, enforce it in the constructor. If a field is
|
||||
optional, use `Option` — do not use sentinel values.
|
||||
|
||||
- **Test the logic, not the wiring.** Commands are pure functions of context;
|
||||
test them by building a `CmdContext` and asserting on the returned effects.
|
||||
You do not need a terminal.
|
||||
|
||||
- **Keep `Option`/`Result`/`Box` at the boundaries.** Core logic should work with
|
||||
concrete values. Wrap in `Option` at the edges (parsing, lookup, I/O) and
|
||||
unwrap early. Do not thread `Option` through deep call chains.
|
||||
|
||||
---
|
||||
|
||||
## 6. Testing
|
||||
|
||||
### Coverage and ambition
|
||||
|
||||
Aim for **~80% line and branch coverage** on logic code. This is a quality floor —
|
||||
go higher where the code is tricky or load-bearing, but don't pad coverage by
|
||||
testing trivial getters or chasing 100% on rendering widgets. The test suite
|
||||
should remain fast (under 2 seconds). Slow tests erode the habit of running them.
|
||||
|
||||
### Demonstrate bugs before fixing them
|
||||
|
||||
Write a test that **fails on the current code** before writing the fix. Prefer a
|
||||
small unit test targeting the broken function over an end-to-end test. After the
|
||||
fix, the test stays as a regression guard. Document the bug in the test's
|
||||
doc-comment (see `model/types.rs` → `formula_tests` for examples).
|
||||
|
||||
### Use property tests judiciously
|
||||
|
||||
Property tests (`proptest`) are for **invariants that must hold across all
|
||||
inputs** — not a replacement for example-based tests. Good candidates:
|
||||
|
||||
- Structural invariants: CellKey is always sorted, each category lives on exactly
|
||||
one axis, toggle-collapse is involutive, hide/show roundtrips.
|
||||
- Serialization roundtrips: save/load identity.
|
||||
- Determinism: `evaluate` returns the same result for the same inputs.
|
||||
|
||||
Keep case counts at the default (256). Don't crank them to thousands — if a
|
||||
property needs more cases to feel safe, constrain the input space with better
|
||||
strategies rather than brute-forcing. Property tests that take hundreds of
|
||||
milliseconds each are a sign something is wrong.
|
||||
|
||||
### What to test
|
||||
|
||||
- **Model, formula, view**: the core logic. Unit tests for each operation and
|
||||
edge case. Property tests for invariants. These are the highest-value tests.
|
||||
- **Commands**: build a `CmdContext`, call `execute`, assert on the returned
|
||||
effects. Pure functions — no terminal needed.
|
||||
- **Persistence**: round-trip tests (`save → load → save` produces identical
|
||||
output). Cover groups, formulas, views, hidden items, legacy JSON.
|
||||
- **Format**: boundary cases for comma placement, rounding, negative numbers.
|
||||
- **Import**: field classification heuristics, CSV quoting, multi-file merge.
|
||||
|
||||
### What not to test
|
||||
|
||||
- Ratatui `Widget::render` implementations — pure drawing code that changes
|
||||
often. Test the data they consume (layout, cat_tree, format) instead.
|
||||
- Trivial data definitions (`ast.rs`, `axis.rs`).
|
||||
- Module re-export files.
|
||||
|
||||
### Test the property, not the implementation
|
||||
|
||||
A test like "calling `set_axis(cat, Row)` sets the internal map entry to `Row`"
|
||||
is brittle — it mirrors the implementation and breaks if the storage changes.
|
||||
Instead test the observable contract: "after `set_axis(cat, Row)`,
|
||||
`axis_of(cat)` returns `Row` and `categories_on(Row)` includes `cat`." This
|
||||
style survives refactoring and catches real bugs.
|
||||
488
context/repo-map.md
Normal file
488
context/repo-map.md
Normal file
@ -0,0 +1,488 @@
|
||||
# Repository Map (LLM Reference)
|
||||
|
||||
Terminal pivot-table modeling app. Rust, Ratatui TUI, command/effect architecture.
|
||||
Crate `improvise` v0.1.0, MIT, edition 2021.
|
||||
|
||||
---
|
||||
|
||||
## How to Find Things
|
||||
|
||||
| I need to... | Look in |
|
||||
|---------------------------------------|----------------------------------------------|
|
||||
| Add a new keybinding | `command/keymap.rs` → `default_keymaps()` |
|
||||
| Add a new user-facing command | `command/cmd.rs` → implement `Cmd`, register in `default_registry()` |
|
||||
| Add a new state mutation | `ui/effect.rs` → implement `Effect` |
|
||||
| Change formula evaluation | `model/types.rs` → `eval_formula()`, `eval_expr()` |
|
||||
| Change how cells are stored/queried | `model/cell.rs` → `DataStore` |
|
||||
| Change category/item behavior | `model/category.rs` → `Category` |
|
||||
| Change view axis logic | `view/types.rs` → `View` |
|
||||
| Change grid layout computation | `view/layout.rs` → `GridLayout` |
|
||||
| Change .improv file format | `persistence/mod.rs` → `format_md()`, `parse_md()` |
|
||||
| Change number display formatting | `format.rs` → `format_f64()` |
|
||||
| Change CLI arguments | `main.rs` → clap structs |
|
||||
| Change import wizard logic | `import/wizard.rs` → `ImportPipeline` |
|
||||
| Change grid rendering | `ui/grid.rs` → `GridWidget` |
|
||||
| Change TUI frame layout | `draw.rs` → `draw()` |
|
||||
| Change app state / mode transitions | `ui/app.rs` → `App`, `AppMode` |
|
||||
| Write a test for model logic | `model/types.rs` → `mod tests` / `mod formula_tests` |
|
||||
| Write a test for a command | `command/cmd.rs` → `mod tests` |
|
||||
|
||||
---
|
||||
|
||||
## Core Types and Traits
|
||||
|
||||
### Command/Effect Pipeline (the central architecture pattern)
|
||||
|
||||
```
|
||||
User keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
|
||||
(immutable) (pure, read-only) (state mutations)
|
||||
```
|
||||
|
||||
```rust
|
||||
// src/command/cmd.rs
|
||||
pub trait Cmd: Debug + Send + Sync {
|
||||
fn name(&self) -> &'static str;
|
||||
fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
|
||||
}
|
||||
|
||||
pub struct CmdContext<'a> {
|
||||
pub model: &'a Model, // immutable
|
||||
pub layout: &'a GridLayout, // immutable
|
||||
pub registry: &'a CmdRegistry,
|
||||
pub mode: &'a AppMode,
|
||||
pub selected: (usize, usize), // (row, col) cursor
|
||||
pub row_offset: usize,
|
||||
pub col_offset: usize,
|
||||
pub search_query: &'a str,
|
||||
pub search_mode: bool,
|
||||
pub yanked: &'a Option<CellValue>,
|
||||
pub key_code: KeyCode, // the key that triggered this command
|
||||
pub buffers: &'a HashMap<String, String>,
|
||||
pub expanded_cats: &'a HashSet<String>,
|
||||
// panel cursors, tile cursor, visible dimensions...
|
||||
}
|
||||
|
||||
// src/ui/effect.rs
|
||||
pub trait Effect: Debug {
|
||||
fn apply(&self, app: &mut App);
|
||||
fn changes_mode(&self) -> bool { false } // override if effect changes AppMode
|
||||
}
|
||||
```
|
||||
|
||||
**To add a command**: implement `Cmd`, then in `default_registry()` call `r.register(...)` or use the `effect_cmd!` macro for simple cases. Bind it in `default_keymaps()`.
|
||||
|
||||
**To add an effect**: implement `Effect` in `effect.rs`, add a constructor function.
|
||||
|
||||
### Data Model
|
||||
|
||||
```rust
|
||||
// src/model/types.rs
|
||||
pub struct Model {
|
||||
pub name: String,
|
||||
pub categories: IndexMap<String, Category>, // ordered
|
||||
pub data: DataStore,
|
||||
pub formulas: Vec<Formula>,
|
||||
pub views: IndexMap<String, View>,
|
||||
pub active_view: String,
|
||||
pub measure_agg: HashMap<String, AggFunc>, // per-measure aggregation override
|
||||
}
|
||||
// Key methods:
|
||||
// add_category(&mut self, name) -> Result<CategoryId> [max 12 regular]
|
||||
// category(&self, name) -> Option<&Category>
|
||||
// category_mut(&mut self, name) -> Option<&mut Category>
|
||||
// set_cell(&mut self, key: CellKey, value: CellValue)
|
||||
// evaluate(&self, key: &CellKey) -> Option<CellValue> [formulas + raw data]
|
||||
// evaluate_aggregated(&self, key, none_cats) -> Option<CellValue> [sums over hidden dims]
|
||||
// add_formula(&mut self, formula: Formula) [replaces same target+category]
|
||||
// remove_formula(&mut self, target, category)
|
||||
// category_names(&self) -> impl Iterator<Item = &str>
|
||||
|
||||
const MAX_CATEGORIES: usize = 12; // virtual categories don't count
|
||||
```
|
||||
|
||||
```rust
|
||||
// src/model/cell.rs
|
||||
#[derive(Clone, PartialEq, Eq, Hash)]
|
||||
pub struct CellKey(pub Vec<(String, String)>); // always sorted by category name
|
||||
// CellKey::new(coords) — sorts on construction, enforcing canonical form
|
||||
// CellKey::with(cat, item) -> Self — returns new key with coord added/replaced
|
||||
// CellKey::without(cat) -> Self — returns new key with coord removed
|
||||
// CellKey::get(cat) -> Option<&str>
|
||||
|
||||
#[derive(Clone, PartialEq)]
|
||||
pub enum CellValue {
|
||||
Number(f64),
|
||||
Text(String),
|
||||
}
|
||||
// CellValue::as_f64() -> Option<f64>
|
||||
|
||||
pub struct DataStore {
|
||||
cells: HashMap<InternedKey, CellValue>,
|
||||
pub symbols: SymbolTable,
|
||||
index: HashMap<(Symbol, Symbol), HashSet<InternedKey>>, // secondary index
|
||||
}
|
||||
// DataStore::set(&mut self, key: &CellKey, value: CellValue)
|
||||
// DataStore::get(&self, key: &CellKey) -> Option<&CellValue>
|
||||
// DataStore::matching_values(&self, partial: &[(String,String)]) -> Vec<CellValue>
|
||||
// DataStore::matching_cells(&self, partial) -> Vec<(CellKey, CellValue)>
|
||||
```
|
||||
|
||||
```rust
|
||||
// src/model/category.rs
|
||||
pub struct Category {
|
||||
pub id: CategoryId, // usize
|
||||
pub name: String,
|
||||
pub kind: CategoryKind,
|
||||
pub items: IndexMap<String, Item>, // ordered
|
||||
pub groups: IndexMap<String, Group>,
|
||||
next_item_id: ItemId, // private, auto-increment
|
||||
}
|
||||
// Category::add_item(&mut self, name) -> ItemId [deduplicates by name]
|
||||
// Category::ordered_item_names(&self) -> Vec<&str> [respects group order]
|
||||
|
||||
pub enum CategoryKind { Regular, VirtualIndex, VirtualDim, Label }
|
||||
```
|
||||
|
||||
### Formula System
|
||||
|
||||
```rust
|
||||
// src/formula/ast.rs
|
||||
pub enum Expr {
|
||||
Number(f64),
|
||||
Ref(String), // reference to an item name
|
||||
BinOp(BinOp, Box<Expr>, Box<Expr>),
|
||||
UnaryMinus(Box<Expr>),
|
||||
Agg(AggFunc, Box<Expr>, Option<Filter>),
|
||||
If(Box<Expr>, Box<Expr>, Box<Expr>),
|
||||
}
|
||||
pub enum BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }
|
||||
pub enum AggFunc { Sum, Avg, Min, Max, Count }
|
||||
pub struct Formula {
|
||||
pub raw: String, // "Profit = Revenue - Cost"
|
||||
pub target: String, // "Profit"
|
||||
pub target_category: String, // "Measure"
|
||||
pub expr: Expr,
|
||||
pub filter: Option<Filter>, // WHERE clause
|
||||
}
|
||||
|
||||
// src/formula/parser.rs
|
||||
pub fn parse_formula(raw: &str, target_category: &str) -> Result<Formula>
|
||||
```
|
||||
|
||||
Formula evaluation is in `model/types.rs` → `eval_formula()` / `eval_expr()`. Operates at full f64 precision. Display rounding in `format.rs` is view-only.
|
||||
|
||||
### View and Layout
|
||||
|
||||
```rust
|
||||
// src/view/axis.rs
|
||||
pub enum Axis { Row, Column, Page, None }
|
||||
|
||||
// src/view/types.rs
|
||||
pub struct View {
|
||||
pub name: String,
|
||||
pub category_axes: IndexMap<String, Axis>,
|
||||
pub page_selections: HashMap<String, String>,
|
||||
pub hidden_items: HashMap<String, HashSet<String>>,
|
||||
pub collapsed_groups: HashMap<String, HashSet<String>>,
|
||||
pub number_format: String, // e.g. ",.0" or ",.2f"
|
||||
pub prune_empty: bool,
|
||||
// scroll/selection state...
|
||||
}
|
||||
// View::set_axis(&mut self, cat, axis)
|
||||
// View::axis_of(&self, cat) -> Axis
|
||||
// View::cycle_axis(&mut self, cat) [Row→Column→Page→None→Row]
|
||||
// View::transpose(&mut self) [swap Row↔Column]
|
||||
// View::categories_on(&self, axis) -> Vec<&str>
|
||||
// View::on_category_added(&mut self, cat) [auto-assigns axis]
|
||||
|
||||
// src/view/layout.rs
|
||||
pub struct GridLayout { /* computed from Model + View */ }
|
||||
// GridLayout::new(model, view) -> Self
|
||||
// GridLayout::cell_key(row, col) -> Option<CellKey>
|
||||
// GridLayout::cell_value(row, col) -> Option<CellValue>
|
||||
// GridLayout::row_label(row) -> &str
|
||||
// GridLayout::col_label(col) -> &str
|
||||
// GridLayout::drill_records(row, col) -> Vec<(CellKey, CellValue)>
|
||||
// Records mode: auto-detected when _Index on Row + _Dim on Column
|
||||
```
|
||||
|
||||
### App State
|
||||
|
||||
```rust
|
||||
// src/ui/app.rs
|
||||
pub enum AppMode {
|
||||
Normal,
|
||||
Editing { minibuf: MinibufferConfig },
|
||||
FormulaEdit { minibuf: MinibufferConfig },
|
||||
FormulaPanel,
|
||||
CategoryPanel,
|
||||
ViewPanel,
|
||||
TileSelect,
|
||||
CategoryAdd { minibuf: MinibufferConfig },
|
||||
ItemAdd { minibuf: MinibufferConfig },
|
||||
ExportPrompt { minibuf: MinibufferConfig },
|
||||
CommandMode { minibuf: MinibufferConfig },
|
||||
ImportWizard,
|
||||
Help,
|
||||
Quit,
|
||||
}
|
||||
// Note: SearchMode is Normal + search_mode:bool flag, not a separate variant.
|
||||
|
||||
pub struct App {
|
||||
pub model: Model,
|
||||
pub mode: AppMode,
|
||||
pub file_path: Option<PathBuf>,
|
||||
pub dirty: bool,
|
||||
pub help_page: usize,
|
||||
pub transient_keymap: Option<Arc<Keymap>>, // for prefix keys
|
||||
// layout cache, drill_state, wizard, buffers, panel cursors, etc.
|
||||
}
|
||||
// App::handle_key(&mut self, KeyEvent) -> Result<()> [main input dispatch]
|
||||
// App::rebuild_layout(&mut self)
|
||||
// App::is_empty_model(&self) -> bool [true when only virtual categories exist]
|
||||
```
|
||||
|
||||
### Keymap System
|
||||
|
||||
```rust
|
||||
// src/command/keymap.rs
|
||||
pub enum KeyPattern { Key(KeyCode, KeyModifiers), AnyChar, Any }
|
||||
pub enum Binding {
|
||||
Cmd { name: &'static str, args: Vec<String> },
|
||||
Prefix(Arc<Keymap>), // Emacs-style sub-keymap
|
||||
Sequence(Vec<(&'static str, Vec<String>)>), // multi-command chain
|
||||
}
|
||||
pub enum ModeKey {
|
||||
Normal, Help, FormulaPanel, CategoryPanel, ViewPanel, TileSelect,
|
||||
Editing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode,
|
||||
SearchMode, ImportWizard,
|
||||
}
|
||||
|
||||
// Keymap::lookup(&self, key, mods) -> Option<&Binding>
|
||||
// Fallback chain: exact(key,mods) → Char with NONE mods → AnyChar → Any
|
||||
|
||||
// KeymapSet::default_keymaps() -> Self [builds all 14 mode keymaps]
|
||||
// KeymapSet::dispatch(&self, ctx, key, mods) -> Option<Vec<Box<dyn Effect>>>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Format (.improv)
|
||||
|
||||
Plain-text markdown-like. **Not JSON** (JSON is legacy, auto-detected by `{` prefix).
|
||||
|
||||
```
|
||||
# Model Name
|
||||
|
||||
## Category: Region
|
||||
- North
|
||||
- South
|
||||
- East [Coastal] ← item in group "Coastal"
|
||||
- West [Coastal]
|
||||
> Coastal ← group definition
|
||||
|
||||
## Category: Measure
|
||||
- Revenue
|
||||
- Cost
|
||||
- Profit
|
||||
|
||||
## Formulas
|
||||
- Profit = Revenue - Cost [Measure] ← [TargetCategory]
|
||||
|
||||
## Data
|
||||
Region=East, Measure=Revenue = 1200
|
||||
Region=East, Measure=Cost = 800
|
||||
Region=West, Measure=Revenue = "pending" ← text value in quotes
|
||||
|
||||
## View: Default (active)
|
||||
Region: row
|
||||
Measure: column
|
||||
Time: page, Q1 ← page axis with selected item
|
||||
hidden: Region/Internal
|
||||
collapsed: Time/2024
|
||||
format: ,.2f
|
||||
```
|
||||
|
||||
Gzip variant: `.improv.gz` (same content, gzipped). Persistence code: `persistence/mod.rs`.
|
||||
|
||||
---
|
||||
|
||||
## CLI
|
||||
|
||||
```
|
||||
improvise [model.improv] # open TUI (default)
|
||||
improvise import data.csv [--no-wizard] [-o out] # import CSV/JSON
|
||||
improvise cmd 'add-cat Region' -f model.improv # headless command(s)
|
||||
improvise script setup.txt -f model.improv # run script file
|
||||
```
|
||||
|
||||
Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`.
|
||||
|
||||
---
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| ratatui 0.29 | TUI framework |
|
||||
| crossterm 0.28 | Terminal backend |
|
||||
| clap 4.6 (derive) | CLI parsing |
|
||||
| serde + serde_json | Serialization |
|
||||
| indexmap 2 | Ordered maps (categories, views) |
|
||||
| anyhow | Error handling |
|
||||
| chrono 0.4 | Date parsing in import |
|
||||
| flate2 | Gzip for .improv.gz |
|
||||
| csv | CSV parsing |
|
||||
| enum_dispatch | CLI subcommand dispatch |
|
||||
| **dev:** proptest, tempfile | Property testing, temp dirs |
|
||||
|
||||
---
|
||||
|
||||
## File Inventory
|
||||
|
||||
Lines / tests / path — grouped by layer.
|
||||
|
||||
### Model layer
|
||||
```
|
||||
1692 / 66t model/types.rs Model struct, formula eval, CRUD, MAX_CATEGORIES=12
|
||||
621 / 28t model/cell.rs CellKey (canonical sort), CellValue, DataStore (interned)
|
||||
216 / 6t model/category.rs Category, Item, Group, CategoryKind
|
||||
79 / 3t model/symbol.rs Symbol interning (SymbolTable)
|
||||
6 / 0t model/mod.rs
|
||||
```
|
||||
|
||||
### Formula layer
|
||||
```
|
||||
461 / 8t formula/parser.rs Recursive descent parser → Formula AST
|
||||
77 / 0t formula/ast.rs Expr, BinOp, AggFunc, Formula, Filter (data only)
|
||||
5 / 0t formula/mod.rs
|
||||
```
|
||||
|
||||
### View layer
|
||||
```
|
||||
1013 / 23t view/layout.rs GridLayout (pure fn of Model+View), records mode, drill
|
||||
521 / 28t view/types.rs View config (axes, pages, hidden, collapsed, format)
|
||||
21 / 0t view/axis.rs Axis enum {Row, Column, Page, None}
|
||||
7 / 0t view/mod.rs
|
||||
```
|
||||
|
||||
### Command layer
|
||||
```
|
||||
3373 / 21t command/cmd.rs Cmd trait, CmdContext, CmdRegistry, 40+ commands
|
||||
1068 / 22t command/keymap.rs KeyPattern, Binding, Keymap, ModeKey, 14 mode keymaps
|
||||
236 / 19t command/parse.rs Script/command-line parser (prefix syntax)
|
||||
12 / 0t command/mod.rs
|
||||
```
|
||||
|
||||
### UI layer
|
||||
```
|
||||
942 / 0t ui/effect.rs Effect trait, 50+ effect types (all state mutations)
|
||||
914 / 30t ui/app.rs App state, AppMode (15 variants), handle_key, autosave
|
||||
1036 / 13t ui/grid.rs GridWidget (ratatui), col widths, rendering
|
||||
617 / 0t ui/help.rs 5-page help overlay, HELP_PAGE_COUNT=5
|
||||
347 / 0t ui/import_wizard_ui.rs Import wizard overlay rendering
|
||||
165 / 6t ui/cat_tree.rs Category tree flattener for panel
|
||||
113 / 0t ui/view_panel.rs View list panel
|
||||
107 / 0t ui/category_panel.rs Category tree panel
|
||||
95 / 0t ui/tile_bar.rs Tile bar (axis assignment tiles)
|
||||
87 / 0t ui/panel.rs Generic panel frame widget
|
||||
81 / 0t ui/formula_panel.rs Formula list panel
|
||||
67 / 0t ui/which_key.rs Prefix-key hint popup
|
||||
12 / 0t ui/mod.rs
|
||||
```
|
||||
|
||||
### Import layer
|
||||
```
|
||||
773 / 15t import/wizard.rs ImportPipeline + ImportWizard
|
||||
292 / 9t import/analyzer.rs Field kind detection (Category/Measure/Time/Skip)
|
||||
244 / 8t import/csv_parser.rs CSV parsing, multi-file merge
|
||||
3 / 0t import/mod.rs
|
||||
```
|
||||
|
||||
### Top-level
|
||||
```
|
||||
400 / 0t draw.rs TUI event loop (run_tui), frame composition
|
||||
391 / 0t main.rs CLI entry (clap): open, import, cmd, script
|
||||
228 / 29t format.rs Number display formatting (view-only rounding)
|
||||
806 / 22t persistence/mod.rs .improv save/load (markdown format + gzip + legacy JSON)
|
||||
```
|
||||
|
||||
### Context docs
|
||||
```
|
||||
context/SPEC.md Feature specification
|
||||
context/design-principles.md Architectural principles
|
||||
context/plan.md Show HN launch plan
|
||||
context/repo-map.md This file
|
||||
```
|
||||
|
||||
**Total: ~15,875 lines, 356 tests.**
|
||||
|
||||
---
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
### Coverage target
|
||||
|
||||
Aim for **~80% line and branch coverage** on logic code. This is a quality floor, not a
|
||||
ceiling — go higher where the code warrants it, but don't chase 100% on rendering
|
||||
widgets or write tests that just exercise trivial getters. Coverage should be run with
|
||||
`cargo llvm-cov` (available via `nix develop`).
|
||||
|
||||
### What to test and how
|
||||
|
||||
| Layer | Approach | Notes |
|
||||
|-------|----------|-------|
|
||||
| **Model** (types, cell, category, symbol) | Unit tests + **proptest** | The data model is the foundation. Property tests catch invariant violations that hand-picked cases miss (see CellKey sort invariant, axis consistency). |
|
||||
| **Formula** (parser, eval) | Unit tests per operator/construct | Cover each BinOp, AggFunc, IF, WHERE, unary minus, chained formulas, error cases (div-by-zero, missing ref). Ensure eval uses full f64 precision — never display-rounded values. |
|
||||
| **View** (types, layout) | Unit tests + **proptest** | Property tests for axis assignment invariants (each category on exactly one axis, transpose is involutive, etc.). Unit tests for layout computation, records mode detection, drill. |
|
||||
| **Command** (cmd, keymap, parse) | Unit tests | Test command execution by building a `CmdContext` and asserting on returned effects. Test keymap lookup fallback chain. Test script parser with edge cases (quoting, comments, dots). |
|
||||
| **Persistence** | Round-trip tests | `save → load → save` must be identical. Cover groups, formulas, views, hidden items, legacy JSON detection. |
|
||||
| **Format** | Unit tests | Boundary cases: comma placement at 3/4/7 digits, negative numbers, rounding half-away-from-zero (not banker's), zero, small fractions. |
|
||||
| **Import** (analyzer, csv, wizard) | Unit tests | Field classification heuristics, CSV quoting (RFC 4180), multi-file merge, date extraction. |
|
||||
| **UI rendering** (grid, panels, draw, help) | Generally skip | Ratatui widgets are hard to unit-test and change frequently. Test the *logic* they consume (layout, cat_tree, format) rather than the rendering itself. |
|
||||
| **Effects** | Test indirectly | Effects are thin `apply` methods. Test via integration: send a key through `App::handle_key` and assert on resulting app state. The complex ones (drill reconciliation, import) deserve targeted unit tests. |
|
||||
|
||||
### Property tests (proptest)
|
||||
|
||||
Use property tests for **invariants that must hold across all inputs**, not as a
|
||||
substitute for example-based tests. Good candidates:
|
||||
|
||||
- Structural invariants: CellKey always sorted, each category on exactly one axis,
|
||||
toggle-collapse is involutive, hide/show roundtrips.
|
||||
- Serialization roundtrips: save/load identity.
|
||||
- Determinism: `evaluate` returns the same value for the same inputs.
|
||||
|
||||
Keep proptest case counts reasonable. The defaults (256 cases) are fine for most
|
||||
properties. Don't crank them up to thousands — the test suite should complete in
|
||||
under 2 seconds. If a property needs more cases to feel confident, that's a sign
|
||||
the input space should be constrained with better strategies, not brute-forced.
|
||||
|
||||
### Bug-fix workflow
|
||||
|
||||
Per CLAUDE.md: **write a test that demonstrates the bug before fixing it.** Prefer
|
||||
a small unit test targeting the specific function over an integration test. The test
|
||||
should fail on the current code, then pass after the fix. Mark regression tests
|
||||
with a doc-comment explaining the bug (see `model/types.rs` `formula_tests` for
|
||||
examples).
|
||||
|
||||
### What not to test
|
||||
|
||||
- Trivial struct constructors and enum definitions (`ast.rs`, `axis.rs`).
|
||||
- Ratatui `Widget::render` implementations — these are pure drawing code.
|
||||
- Module re-export files (`mod.rs`).
|
||||
- One-line delegation methods.
|
||||
|
||||
---
|
||||
|
||||
## Patterns to Know
|
||||
|
||||
1. **Commands never mutate.** They receive `&CmdContext` (read-only) and return `Vec<Box<dyn Effect>>`.
|
||||
2. **CellKey is always sorted.** Use `CellKey::new()` — never construct the inner Vec directly.
|
||||
3. **`category_mut()` for adding items.** `Model` has no `add_item` method; get the category first: `m.category_mut("Region").unwrap().add_item("East")`.
|
||||
4. **Virtual categories** `_Index` and `_Dim` always exist. `is_empty_model()` checks whether any *non-virtual* categories exist.
|
||||
5. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full f64.
|
||||
6. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the current mode, then the corresponding `Keymap` is looked up. Normal + `search_mode=true` maps to `SearchMode`.
|
||||
7. **`effect_cmd!` macro** generates a command struct that just produces effects. Use for simple commands without complex logic.
|
||||
8. **`.improv` format is markdown-like**, not JSON. See `persistence/mod.rs`. JSON is legacy only.
|
||||
9. **`IndexMap`** is used for categories and views to preserve insertion order.
|
||||
10. **`MAX_CATEGORIES = 12`** applies only to `CategoryKind::Regular`. Virtual/Label categories are exempt.
|
||||
Reference in New Issue
Block a user