docs: add repository context files
Add new context files to assist with repository navigation and design consistency: - context/repo-map.md: A roadmap for the repository. - context/design-principles.md: Guidelines for maintaining repository consistency. Update CLAUDE.md to include instructions on using the new context files. Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
This commit is contained in:
@ -4,7 +4,8 @@
|
|||||||
- Option<...> or Result<...> are fine but should not be present in the majority of the code.
|
- Option<...> or Result<...> are fine but should not be present in the majority of the code.
|
||||||
- Similarly, code managing Box<...> or RC<...>, etc. for containers pointing to heap data should be split
|
- Similarly, code managing Box<...> or RC<...>, etc. for containers pointing to heap data should be split
|
||||||
from logic
|
from logic
|
||||||
|
- context/repo-map.md is your "road map" for the repository. use it to reduce exploration and keep it updated.
|
||||||
|
- context/design-principles.md is also important for keeping the repository consistent.
|
||||||
|
|
||||||
<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
|
<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
|
||||||
## Beads Issue Tracker
|
## Beads Issue Tracker
|
||||||
|
|||||||
228
context/design-principles.md
Normal file
228
context/design-principles.md
Normal file
@ -0,0 +1,228 @@
|
|||||||
|
# Improvise Design Principles
|
||||||
|
|
||||||
|
## 1. Functional-First Architecture
|
||||||
|
|
||||||
|
### Commands Are Pure, Effects Are Side-Effectful
|
||||||
|
|
||||||
|
Every user action flows through a two-phase pipeline:
|
||||||
|
|
||||||
|
1. **Command** (`Cmd` trait) — reads immutable context, returns a list of effects.
|
||||||
|
The `CmdContext` is a read-only snapshot: model, layout, mode, cursor position.
|
||||||
|
Commands never touch `&mut App`. All decision logic is pure.
|
||||||
|
|
||||||
|
2. **Effect** (`Effect` trait) — a small struct with an `apply(&self, app: &mut App)` method.
|
||||||
|
Each effect is one discrete, debuggable state change. The app applies them in order.
|
||||||
|
|
||||||
|
This separation means:
|
||||||
|
- Commands are testable without a terminal or an `App` instance.
|
||||||
|
- Effects can be logged, replayed, or composed.
|
||||||
|
- The only place `App` is mutated is inside `Effect::apply`.
|
||||||
|
|
||||||
|
### Prefer Transformations to Mutation
|
||||||
|
|
||||||
|
Where possible, build new values rather than mutating in place:
|
||||||
|
- `CellKey::with(cat, item)` returns a new key with an added/replaced coordinate.
|
||||||
|
- `CellKey::without(cat)` returns a new key with a coordinate removed.
|
||||||
|
- Viewport positioning is computed as a pure function (`viewport_effects`) that
|
||||||
|
returns a `Vec<Effect>`, not a method that pokes at scroll offsets directly.
|
||||||
|
|
||||||
|
### Compose Small Pieces
|
||||||
|
|
||||||
|
Commands compose via `Binding::Sequence` — a keymap entry can chain multiple
|
||||||
|
commands, each contributing effects independently. The `o` key (add row + begin
|
||||||
|
editing) is two commands composed at the binding level, not a monolithic handler.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Polymorphism Over Conditionals
|
||||||
|
|
||||||
|
### Dispatch Through Traits and Registries, Not Match Blocks
|
||||||
|
|
||||||
|
- **Commands**: 40+ types each implement `Cmd`. A `CmdRegistry` maps names to
|
||||||
|
constructor closures. Dispatching a key presses looks up the binding, resolves
|
||||||
|
the command name through the registry, and calls `execute`. No central
|
||||||
|
`match command_name { ... }` block.
|
||||||
|
|
||||||
|
- **Effects**: 50+ types each implement `Effect`. Collected into a `Vec<Box<dyn Effect>>`
|
||||||
|
and applied in order. No `match effect_kind { ... }`.
|
||||||
|
|
||||||
|
- **Keymaps**: Each mode has its own `Keymap` (a `HashMap<KeyPattern, Binding>`).
|
||||||
|
Mode dispatch is one table lookup, not a nested `match (mode, key)`.
|
||||||
|
|
||||||
|
### Use Enums to Make Invalid States Unrepresentable
|
||||||
|
|
||||||
|
- `BinOp` is an enum (`Add | Sub | Mul | ...`), not a string. Invalid operators
|
||||||
|
are caught at parse time, not silently ignored at eval time.
|
||||||
|
|
||||||
|
- `Axis` is `Row | Column | Page | None`. A category is on exactly one axis.
|
||||||
|
Cycling is a four-state rotation — no boolean flags, no "row_or_column" ambiguity.
|
||||||
|
|
||||||
|
- `Binding` is `Cmd | Prefix | Sequence`. The keymap lookup returns one of these
|
||||||
|
three shapes; dispatch pattern-matches exhaustively.
|
||||||
|
|
||||||
|
- `CategoryKind` is `Regular | VirtualIndex | VirtualDim | Label`. Business rules
|
||||||
|
(e.g., the 12-category limit counts only `Regular`) are enforced by matching
|
||||||
|
on the enum, not by checking name prefixes.
|
||||||
|
|
||||||
|
### When You Add a Variant, the Compiler Finds Every Call Site
|
||||||
|
|
||||||
|
Prefer exhaustive `match` over `if let` or `_ =>` wildcards. When a new `Axis`
|
||||||
|
variant or `AppMode` is added, non-exhaustive matches produce compile errors
|
||||||
|
that guide you to every place that needs updating.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Correctness by Construction
|
||||||
|
|
||||||
|
### Canonical Forms Prevent Equivalence Bugs
|
||||||
|
|
||||||
|
`CellKey::new()` sorts coordinates by category name. Two keys that name the same
|
||||||
|
intersection but in different order are identical after construction. Equality,
|
||||||
|
hashing, and storage all work correctly without callers needing to remember to
|
||||||
|
sort. Property tests verify this invariant.
|
||||||
|
|
||||||
|
### Smart Constructors Enforce Invariants
|
||||||
|
|
||||||
|
- `CellKey::new()` is the only way to build a key — it always sorts.
|
||||||
|
- `Category::add_item()` deduplicates by name and auto-assigns IDs via a private
|
||||||
|
counter. External code cannot fabricate an `ItemId`.
|
||||||
|
- `Model::add_category()` checks the 12-category limit before insertion.
|
||||||
|
- `Formula::new()` takes all required fields; there is no default/empty formula
|
||||||
|
to accidentally leave half-initialized.
|
||||||
|
|
||||||
|
### Type-Safe Identifiers
|
||||||
|
|
||||||
|
`CategoryId` and `ItemId` are typed aliases. While they are `usize` underneath,
|
||||||
|
using named types signals intent and prevents accidentally passing an item count
|
||||||
|
where an item ID is expected.
|
||||||
|
|
||||||
|
### Symbol Interning for Data Integrity
|
||||||
|
|
||||||
|
`DataStore` interns category and item names into `Symbol` values (small copyable
|
||||||
|
handles). This means:
|
||||||
|
- String comparison is integer comparison — fast and allocation-free.
|
||||||
|
- A secondary index maps `(Symbol, Symbol)` pairs to cell sets, enabling O(1)
|
||||||
|
lookups for aggregation queries.
|
||||||
|
- Symbols can only be created through the `SymbolTable`, so misspelled names
|
||||||
|
produce a distinct symbol rather than silently matching a wrong cell.
|
||||||
|
|
||||||
|
### Parse-Time Validation
|
||||||
|
|
||||||
|
Formulas are parsed into a typed AST (`Expr` enum) at entry time. If the syntax
|
||||||
|
is invalid, the user gets an error immediately. The evaluator only sees
|
||||||
|
well-formed trees — it does not need to handle malformed input.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Separation of Concerns
|
||||||
|
|
||||||
|
### Four Layers
|
||||||
|
|
||||||
|
| Layer | Directory | Responsibility |
|
||||||
|
|-------|-----------|----------------|
|
||||||
|
| **Model** | `src/model/` | Categories, items, groups, cell data, formulas. Pure data, no rendering. |
|
||||||
|
| **View** | `src/view/` | Axis assignments, page selection, hidden items, layout computation. Derived from model. |
|
||||||
|
| **Command / Effect** | `src/command/`, `src/ui/effect.rs` | Intent (commands) and state mutation (effects). Bridges user input to model changes. |
|
||||||
|
| **Rendering** | `src/draw.rs`, `src/ui/` | Terminal drawing. Reads model + view, writes pixels. No mutation. |
|
||||||
|
|
||||||
|
### Formulas Are Data, Not Code
|
||||||
|
|
||||||
|
A formula is a serializable struct: raw text, target name, category, AST, optional
|
||||||
|
filter. It is stored in the model alongside regular data. The evaluator walks the
|
||||||
|
AST at read time. Formulas never become closures or runtime-generated code.
|
||||||
|
|
||||||
|
### Display Rounding Is View-Only
|
||||||
|
|
||||||
|
Number formatting (`format_f64`) rounds for display. Formula evaluation always
|
||||||
|
operates on full `f64` precision. The rounding function is only called in
|
||||||
|
rendering paths — never in `eval_formula` or aggregation.
|
||||||
|
|
||||||
|
### Drill State Isolates Edits
|
||||||
|
|
||||||
|
When editing aggregated (drill-down) cells, a `DrillState` snapshot freezes the
|
||||||
|
current cell set. Pending edits accumulate in a staging map. On commit,
|
||||||
|
`ApplyAndClearDrill` writes them all atomically. On cancel, the snapshot is
|
||||||
|
discarded. No partial writes reach the model.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Guidelines for New Code
|
||||||
|
|
||||||
|
- **Add a command, not a special case.** If you need new behavior on a keypress,
|
||||||
|
implement `Cmd`, register it, and bind it in the keymap. Do not add an
|
||||||
|
`if key == 'x'` branch inside `handle_key`.
|
||||||
|
|
||||||
|
- **Return effects, do not mutate.** Your command's `execute` receives `&CmdContext`
|
||||||
|
(immutable). Produce a `Vec<Box<dyn Effect>>`. If you need a new kind of state
|
||||||
|
change, create a new `Effect` struct.
|
||||||
|
|
||||||
|
- **Use the type system.** If a value can only be one of N things, make it an enum.
|
||||||
|
If an invariant must hold, enforce it in the constructor. If a field is
|
||||||
|
optional, use `Option` — do not use sentinel values.
|
||||||
|
|
||||||
|
- **Test the logic, not the wiring.** Commands are pure functions of context;
|
||||||
|
test them by building a `CmdContext` and asserting on the returned effects.
|
||||||
|
You do not need a terminal.
|
||||||
|
|
||||||
|
- **Keep `Option`/`Result`/`Box` at the boundaries.** Core logic should work with
|
||||||
|
concrete values. Wrap in `Option` at the edges (parsing, lookup, I/O) and
|
||||||
|
unwrap early. Do not thread `Option` through deep call chains.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Testing
|
||||||
|
|
||||||
|
### Coverage and ambition
|
||||||
|
|
||||||
|
Aim for **~80% line and branch coverage** on logic code. This is a quality floor —
|
||||||
|
go higher where the code is tricky or load-bearing, but don't pad coverage by
|
||||||
|
testing trivial getters or chasing 100% on rendering widgets. The test suite
|
||||||
|
should remain fast (under 2 seconds). Slow tests erode the habit of running them.
|
||||||
|
|
||||||
|
### Demonstrate bugs before fixing them
|
||||||
|
|
||||||
|
Write a test that **fails on the current code** before writing the fix. Prefer a
|
||||||
|
small unit test targeting the broken function over an end-to-end test. After the
|
||||||
|
fix, the test stays as a regression guard. Document the bug in the test's
|
||||||
|
doc-comment (see `model/types.rs` → `formula_tests` for examples).
|
||||||
|
|
||||||
|
### Use property tests judiciously
|
||||||
|
|
||||||
|
Property tests (`proptest`) are for **invariants that must hold across all
|
||||||
|
inputs** — not a replacement for example-based tests. Good candidates:
|
||||||
|
|
||||||
|
- Structural invariants: CellKey is always sorted, each category lives on exactly
|
||||||
|
one axis, toggle-collapse is involutive, hide/show roundtrips.
|
||||||
|
- Serialization roundtrips: save/load identity.
|
||||||
|
- Determinism: `evaluate` returns the same result for the same inputs.
|
||||||
|
|
||||||
|
Keep case counts at the default (256). Don't crank them to thousands — if a
|
||||||
|
property needs more cases to feel safe, constrain the input space with better
|
||||||
|
strategies rather than brute-forcing. Property tests that take hundreds of
|
||||||
|
milliseconds each are a sign something is wrong.
|
||||||
|
|
||||||
|
### What to test
|
||||||
|
|
||||||
|
- **Model, formula, view**: the core logic. Unit tests for each operation and
|
||||||
|
edge case. Property tests for invariants. These are the highest-value tests.
|
||||||
|
- **Commands**: build a `CmdContext`, call `execute`, assert on the returned
|
||||||
|
effects. Pure functions — no terminal needed.
|
||||||
|
- **Persistence**: round-trip tests (`save → load → save` produces identical
|
||||||
|
output). Cover groups, formulas, views, hidden items, legacy JSON.
|
||||||
|
- **Format**: boundary cases for comma placement, rounding, negative numbers.
|
||||||
|
- **Import**: field classification heuristics, CSV quoting, multi-file merge.
|
||||||
|
|
||||||
|
### What not to test
|
||||||
|
|
||||||
|
- Ratatui `Widget::render` implementations — pure drawing code that changes
|
||||||
|
often. Test the data they consume (layout, cat_tree, format) instead.
|
||||||
|
- Trivial data definitions (`ast.rs`, `axis.rs`).
|
||||||
|
- Module re-export files.
|
||||||
|
|
||||||
|
### Test the property, not the implementation
|
||||||
|
|
||||||
|
A test like "calling `set_axis(cat, Row)` sets the internal map entry to `Row`"
|
||||||
|
is brittle — it mirrors the implementation and breaks if the storage changes.
|
||||||
|
Instead test the observable contract: "after `set_axis(cat, Row)`,
|
||||||
|
`axis_of(cat)` returns `Row` and `categories_on(Row)` includes `cat`." This
|
||||||
|
style survives refactoring and catches real bugs.
|
||||||
488
context/repo-map.md
Normal file
488
context/repo-map.md
Normal file
@ -0,0 +1,488 @@
|
|||||||
|
# Repository Map (LLM Reference)
|
||||||
|
|
||||||
|
Terminal pivot-table modeling app. Rust, Ratatui TUI, command/effect architecture.
|
||||||
|
Crate `improvise` v0.1.0, MIT, edition 2021.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How to Find Things
|
||||||
|
|
||||||
|
| I need to... | Look in |
|
||||||
|
|---------------------------------------|----------------------------------------------|
|
||||||
|
| Add a new keybinding | `command/keymap.rs` → `default_keymaps()` |
|
||||||
|
| Add a new user-facing command | `command/cmd.rs` → implement `Cmd`, register in `default_registry()` |
|
||||||
|
| Add a new state mutation | `ui/effect.rs` → implement `Effect` |
|
||||||
|
| Change formula evaluation | `model/types.rs` → `eval_formula()`, `eval_expr()` |
|
||||||
|
| Change how cells are stored/queried | `model/cell.rs` → `DataStore` |
|
||||||
|
| Change category/item behavior | `model/category.rs` → `Category` |
|
||||||
|
| Change view axis logic | `view/types.rs` → `View` |
|
||||||
|
| Change grid layout computation | `view/layout.rs` → `GridLayout` |
|
||||||
|
| Change .improv file format | `persistence/mod.rs` → `format_md()`, `parse_md()` |
|
||||||
|
| Change number display formatting | `format.rs` → `format_f64()` |
|
||||||
|
| Change CLI arguments | `main.rs` → clap structs |
|
||||||
|
| Change import wizard logic | `import/wizard.rs` → `ImportPipeline` |
|
||||||
|
| Change grid rendering | `ui/grid.rs` → `GridWidget` |
|
||||||
|
| Change TUI frame layout | `draw.rs` → `draw()` |
|
||||||
|
| Change app state / mode transitions | `ui/app.rs` → `App`, `AppMode` |
|
||||||
|
| Write a test for model logic | `model/types.rs` → `mod tests` / `mod formula_tests` |
|
||||||
|
| Write a test for a command | `command/cmd.rs` → `mod tests` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Types and Traits
|
||||||
|
|
||||||
|
### Command/Effect Pipeline (the central architecture pattern)
|
||||||
|
|
||||||
|
```
|
||||||
|
User keypress → Keymap lookup → Cmd::execute(&CmdContext) → Vec<Box<dyn Effect>> → Effect::apply(&mut App)
|
||||||
|
(immutable) (pure, read-only) (state mutations)
|
||||||
|
```
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/command/cmd.rs
|
||||||
|
pub trait Cmd: Debug + Send + Sync {
|
||||||
|
fn name(&self) -> &'static str;
|
||||||
|
fn execute(&self, ctx: &CmdContext) -> Vec<Box<dyn Effect>>;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct CmdContext<'a> {
|
||||||
|
pub model: &'a Model, // immutable
|
||||||
|
pub layout: &'a GridLayout, // immutable
|
||||||
|
pub registry: &'a CmdRegistry,
|
||||||
|
pub mode: &'a AppMode,
|
||||||
|
pub selected: (usize, usize), // (row, col) cursor
|
||||||
|
pub row_offset: usize,
|
||||||
|
pub col_offset: usize,
|
||||||
|
pub search_query: &'a str,
|
||||||
|
pub search_mode: bool,
|
||||||
|
pub yanked: &'a Option<CellValue>,
|
||||||
|
pub key_code: KeyCode, // the key that triggered this command
|
||||||
|
pub buffers: &'a HashMap<String, String>,
|
||||||
|
pub expanded_cats: &'a HashSet<String>,
|
||||||
|
// panel cursors, tile cursor, visible dimensions...
|
||||||
|
}
|
||||||
|
|
||||||
|
// src/ui/effect.rs
|
||||||
|
pub trait Effect: Debug {
|
||||||
|
fn apply(&self, app: &mut App);
|
||||||
|
fn changes_mode(&self) -> bool { false } // override if effect changes AppMode
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**To add a command**: implement `Cmd`, then in `default_registry()` call `r.register(...)` or use the `effect_cmd!` macro for simple cases. Bind it in `default_keymaps()`.
|
||||||
|
|
||||||
|
**To add an effect**: implement `Effect` in `effect.rs`, add a constructor function.
|
||||||
|
|
||||||
|
### Data Model
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/model/types.rs
|
||||||
|
pub struct Model {
|
||||||
|
pub name: String,
|
||||||
|
pub categories: IndexMap<String, Category>, // ordered
|
||||||
|
pub data: DataStore,
|
||||||
|
pub formulas: Vec<Formula>,
|
||||||
|
pub views: IndexMap<String, View>,
|
||||||
|
pub active_view: String,
|
||||||
|
pub measure_agg: HashMap<String, AggFunc>, // per-measure aggregation override
|
||||||
|
}
|
||||||
|
// Key methods:
|
||||||
|
// add_category(&mut self, name) -> Result<CategoryId> [max 12 regular]
|
||||||
|
// category(&self, name) -> Option<&Category>
|
||||||
|
// category_mut(&mut self, name) -> Option<&mut Category>
|
||||||
|
// set_cell(&mut self, key: CellKey, value: CellValue)
|
||||||
|
// evaluate(&self, key: &CellKey) -> Option<CellValue> [formulas + raw data]
|
||||||
|
// evaluate_aggregated(&self, key, none_cats) -> Option<CellValue> [sums over hidden dims]
|
||||||
|
// add_formula(&mut self, formula: Formula) [replaces same target+category]
|
||||||
|
// remove_formula(&mut self, target, category)
|
||||||
|
// category_names(&self) -> impl Iterator<Item = &str>
|
||||||
|
|
||||||
|
const MAX_CATEGORIES: usize = 12; // virtual categories don't count
|
||||||
|
```
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/model/cell.rs
|
||||||
|
#[derive(Clone, PartialEq, Eq, Hash)]
|
||||||
|
pub struct CellKey(pub Vec<(String, String)>); // always sorted by category name
|
||||||
|
// CellKey::new(coords) — sorts on construction, enforcing canonical form
|
||||||
|
// CellKey::with(cat, item) -> Self — returns new key with coord added/replaced
|
||||||
|
// CellKey::without(cat) -> Self — returns new key with coord removed
|
||||||
|
// CellKey::get(cat) -> Option<&str>
|
||||||
|
|
||||||
|
#[derive(Clone, PartialEq)]
|
||||||
|
pub enum CellValue {
|
||||||
|
Number(f64),
|
||||||
|
Text(String),
|
||||||
|
}
|
||||||
|
// CellValue::as_f64() -> Option<f64>
|
||||||
|
|
||||||
|
pub struct DataStore {
|
||||||
|
cells: HashMap<InternedKey, CellValue>,
|
||||||
|
pub symbols: SymbolTable,
|
||||||
|
index: HashMap<(Symbol, Symbol), HashSet<InternedKey>>, // secondary index
|
||||||
|
}
|
||||||
|
// DataStore::set(&mut self, key: &CellKey, value: CellValue)
|
||||||
|
// DataStore::get(&self, key: &CellKey) -> Option<&CellValue>
|
||||||
|
// DataStore::matching_values(&self, partial: &[(String,String)]) -> Vec<CellValue>
|
||||||
|
// DataStore::matching_cells(&self, partial) -> Vec<(CellKey, CellValue)>
|
||||||
|
```
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/model/category.rs
|
||||||
|
pub struct Category {
|
||||||
|
pub id: CategoryId, // usize
|
||||||
|
pub name: String,
|
||||||
|
pub kind: CategoryKind,
|
||||||
|
pub items: IndexMap<String, Item>, // ordered
|
||||||
|
pub groups: IndexMap<String, Group>,
|
||||||
|
next_item_id: ItemId, // private, auto-increment
|
||||||
|
}
|
||||||
|
// Category::add_item(&mut self, name) -> ItemId [deduplicates by name]
|
||||||
|
// Category::ordered_item_names(&self) -> Vec<&str> [respects group order]
|
||||||
|
|
||||||
|
pub enum CategoryKind { Regular, VirtualIndex, VirtualDim, Label }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Formula System
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/formula/ast.rs
|
||||||
|
pub enum Expr {
|
||||||
|
Number(f64),
|
||||||
|
Ref(String), // reference to an item name
|
||||||
|
BinOp(BinOp, Box<Expr>, Box<Expr>),
|
||||||
|
UnaryMinus(Box<Expr>),
|
||||||
|
Agg(AggFunc, Box<Expr>, Option<Filter>),
|
||||||
|
If(Box<Expr>, Box<Expr>, Box<Expr>),
|
||||||
|
}
|
||||||
|
pub enum BinOp { Add, Sub, Mul, Div, Pow, Eq, Ne, Lt, Gt, Le, Ge }
|
||||||
|
pub enum AggFunc { Sum, Avg, Min, Max, Count }
|
||||||
|
pub struct Formula {
|
||||||
|
pub raw: String, // "Profit = Revenue - Cost"
|
||||||
|
pub target: String, // "Profit"
|
||||||
|
pub target_category: String, // "Measure"
|
||||||
|
pub expr: Expr,
|
||||||
|
pub filter: Option<Filter>, // WHERE clause
|
||||||
|
}
|
||||||
|
|
||||||
|
// src/formula/parser.rs
|
||||||
|
pub fn parse_formula(raw: &str, target_category: &str) -> Result<Formula>
|
||||||
|
```
|
||||||
|
|
||||||
|
Formula evaluation is in `model/types.rs` → `eval_formula()` / `eval_expr()`. Operates at full f64 precision. Display rounding in `format.rs` is view-only.
|
||||||
|
|
||||||
|
### View and Layout
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/view/axis.rs
|
||||||
|
pub enum Axis { Row, Column, Page, None }
|
||||||
|
|
||||||
|
// src/view/types.rs
|
||||||
|
pub struct View {
|
||||||
|
pub name: String,
|
||||||
|
pub category_axes: IndexMap<String, Axis>,
|
||||||
|
pub page_selections: HashMap<String, String>,
|
||||||
|
pub hidden_items: HashMap<String, HashSet<String>>,
|
||||||
|
pub collapsed_groups: HashMap<String, HashSet<String>>,
|
||||||
|
pub number_format: String, // e.g. ",.0" or ",.2f"
|
||||||
|
pub prune_empty: bool,
|
||||||
|
// scroll/selection state...
|
||||||
|
}
|
||||||
|
// View::set_axis(&mut self, cat, axis)
|
||||||
|
// View::axis_of(&self, cat) -> Axis
|
||||||
|
// View::cycle_axis(&mut self, cat) [Row→Column→Page→None→Row]
|
||||||
|
// View::transpose(&mut self) [swap Row↔Column]
|
||||||
|
// View::categories_on(&self, axis) -> Vec<&str>
|
||||||
|
// View::on_category_added(&mut self, cat) [auto-assigns axis]
|
||||||
|
|
||||||
|
// src/view/layout.rs
|
||||||
|
pub struct GridLayout { /* computed from Model + View */ }
|
||||||
|
// GridLayout::new(model, view) -> Self
|
||||||
|
// GridLayout::cell_key(row, col) -> Option<CellKey>
|
||||||
|
// GridLayout::cell_value(row, col) -> Option<CellValue>
|
||||||
|
// GridLayout::row_label(row) -> &str
|
||||||
|
// GridLayout::col_label(col) -> &str
|
||||||
|
// GridLayout::drill_records(row, col) -> Vec<(CellKey, CellValue)>
|
||||||
|
// Records mode: auto-detected when _Index on Row + _Dim on Column
|
||||||
|
```
|
||||||
|
|
||||||
|
### App State
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/ui/app.rs
|
||||||
|
pub enum AppMode {
|
||||||
|
Normal,
|
||||||
|
Editing { minibuf: MinibufferConfig },
|
||||||
|
FormulaEdit { minibuf: MinibufferConfig },
|
||||||
|
FormulaPanel,
|
||||||
|
CategoryPanel,
|
||||||
|
ViewPanel,
|
||||||
|
TileSelect,
|
||||||
|
CategoryAdd { minibuf: MinibufferConfig },
|
||||||
|
ItemAdd { minibuf: MinibufferConfig },
|
||||||
|
ExportPrompt { minibuf: MinibufferConfig },
|
||||||
|
CommandMode { minibuf: MinibufferConfig },
|
||||||
|
ImportWizard,
|
||||||
|
Help,
|
||||||
|
Quit,
|
||||||
|
}
|
||||||
|
// Note: SearchMode is Normal + search_mode:bool flag, not a separate variant.
|
||||||
|
|
||||||
|
pub struct App {
|
||||||
|
pub model: Model,
|
||||||
|
pub mode: AppMode,
|
||||||
|
pub file_path: Option<PathBuf>,
|
||||||
|
pub dirty: bool,
|
||||||
|
pub help_page: usize,
|
||||||
|
pub transient_keymap: Option<Arc<Keymap>>, // for prefix keys
|
||||||
|
// layout cache, drill_state, wizard, buffers, panel cursors, etc.
|
||||||
|
}
|
||||||
|
// App::handle_key(&mut self, KeyEvent) -> Result<()> [main input dispatch]
|
||||||
|
// App::rebuild_layout(&mut self)
|
||||||
|
// App::is_empty_model(&self) -> bool [true when only virtual categories exist]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Keymap System
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// src/command/keymap.rs
|
||||||
|
pub enum KeyPattern { Key(KeyCode, KeyModifiers), AnyChar, Any }
|
||||||
|
pub enum Binding {
|
||||||
|
Cmd { name: &'static str, args: Vec<String> },
|
||||||
|
Prefix(Arc<Keymap>), // Emacs-style sub-keymap
|
||||||
|
Sequence(Vec<(&'static str, Vec<String>)>), // multi-command chain
|
||||||
|
}
|
||||||
|
pub enum ModeKey {
|
||||||
|
Normal, Help, FormulaPanel, CategoryPanel, ViewPanel, TileSelect,
|
||||||
|
Editing, FormulaEdit, CategoryAdd, ItemAdd, ExportPrompt, CommandMode,
|
||||||
|
SearchMode, ImportWizard,
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keymap::lookup(&self, key, mods) -> Option<&Binding>
|
||||||
|
// Fallback chain: exact(key,mods) → Char with NONE mods → AnyChar → Any
|
||||||
|
|
||||||
|
// KeymapSet::default_keymaps() -> Self [builds all 14 mode keymaps]
|
||||||
|
// KeymapSet::dispatch(&self, ctx, key, mods) -> Option<Vec<Box<dyn Effect>>>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Format (.improv)
|
||||||
|
|
||||||
|
Plain-text markdown-like. **Not JSON** (JSON is legacy, auto-detected by `{` prefix).
|
||||||
|
|
||||||
|
```
|
||||||
|
# Model Name
|
||||||
|
|
||||||
|
## Category: Region
|
||||||
|
- North
|
||||||
|
- South
|
||||||
|
- East [Coastal] ← item in group "Coastal"
|
||||||
|
- West [Coastal]
|
||||||
|
> Coastal ← group definition
|
||||||
|
|
||||||
|
## Category: Measure
|
||||||
|
- Revenue
|
||||||
|
- Cost
|
||||||
|
- Profit
|
||||||
|
|
||||||
|
## Formulas
|
||||||
|
- Profit = Revenue - Cost [Measure] ← [TargetCategory]
|
||||||
|
|
||||||
|
## Data
|
||||||
|
Region=East, Measure=Revenue = 1200
|
||||||
|
Region=East, Measure=Cost = 800
|
||||||
|
Region=West, Measure=Revenue = "pending" ← text value in quotes
|
||||||
|
|
||||||
|
## View: Default (active)
|
||||||
|
Region: row
|
||||||
|
Measure: column
|
||||||
|
Time: page, Q1 ← page axis with selected item
|
||||||
|
hidden: Region/Internal
|
||||||
|
collapsed: Time/2024
|
||||||
|
format: ,.2f
|
||||||
|
```
|
||||||
|
|
||||||
|
Gzip variant: `.improv.gz` (same content, gzipped). Persistence code: `persistence/mod.rs`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CLI
|
||||||
|
|
||||||
|
```
|
||||||
|
improvise [model.improv] # open TUI (default)
|
||||||
|
improvise import data.csv [--no-wizard] [-o out] # import CSV/JSON
|
||||||
|
improvise cmd 'add-cat Region' -f model.improv # headless command(s)
|
||||||
|
improvise script setup.txt -f model.improv # run script file
|
||||||
|
```
|
||||||
|
|
||||||
|
Import flags: `--category`, `--measure`, `--time`, `--skip`, `--extract`, `--axis`, `--formula`, `--name`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Dependencies
|
||||||
|
|
||||||
|
| Crate | Purpose |
|
||||||
|
|-------|---------|
|
||||||
|
| ratatui 0.29 | TUI framework |
|
||||||
|
| crossterm 0.28 | Terminal backend |
|
||||||
|
| clap 4.6 (derive) | CLI parsing |
|
||||||
|
| serde + serde_json | Serialization |
|
||||||
|
| indexmap 2 | Ordered maps (categories, views) |
|
||||||
|
| anyhow | Error handling |
|
||||||
|
| chrono 0.4 | Date parsing in import |
|
||||||
|
| flate2 | Gzip for .improv.gz |
|
||||||
|
| csv | CSV parsing |
|
||||||
|
| enum_dispatch | CLI subcommand dispatch |
|
||||||
|
| **dev:** proptest, tempfile | Property testing, temp dirs |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Inventory
|
||||||
|
|
||||||
|
Lines / tests / path — grouped by layer.
|
||||||
|
|
||||||
|
### Model layer
|
||||||
|
```
|
||||||
|
1692 / 66t model/types.rs Model struct, formula eval, CRUD, MAX_CATEGORIES=12
|
||||||
|
621 / 28t model/cell.rs CellKey (canonical sort), CellValue, DataStore (interned)
|
||||||
|
216 / 6t model/category.rs Category, Item, Group, CategoryKind
|
||||||
|
79 / 3t model/symbol.rs Symbol interning (SymbolTable)
|
||||||
|
6 / 0t model/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Formula layer
|
||||||
|
```
|
||||||
|
461 / 8t formula/parser.rs Recursive descent parser → Formula AST
|
||||||
|
77 / 0t formula/ast.rs Expr, BinOp, AggFunc, Formula, Filter (data only)
|
||||||
|
5 / 0t formula/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### View layer
|
||||||
|
```
|
||||||
|
1013 / 23t view/layout.rs GridLayout (pure fn of Model+View), records mode, drill
|
||||||
|
521 / 28t view/types.rs View config (axes, pages, hidden, collapsed, format)
|
||||||
|
21 / 0t view/axis.rs Axis enum {Row, Column, Page, None}
|
||||||
|
7 / 0t view/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Command layer
|
||||||
|
```
|
||||||
|
3373 / 21t command/cmd.rs Cmd trait, CmdContext, CmdRegistry, 40+ commands
|
||||||
|
1068 / 22t command/keymap.rs KeyPattern, Binding, Keymap, ModeKey, 14 mode keymaps
|
||||||
|
236 / 19t command/parse.rs Script/command-line parser (prefix syntax)
|
||||||
|
12 / 0t command/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### UI layer
|
||||||
|
```
|
||||||
|
942 / 0t ui/effect.rs Effect trait, 50+ effect types (all state mutations)
|
||||||
|
914 / 30t ui/app.rs App state, AppMode (15 variants), handle_key, autosave
|
||||||
|
1036 / 13t ui/grid.rs GridWidget (ratatui), col widths, rendering
|
||||||
|
617 / 0t ui/help.rs 5-page help overlay, HELP_PAGE_COUNT=5
|
||||||
|
347 / 0t ui/import_wizard_ui.rs Import wizard overlay rendering
|
||||||
|
165 / 6t ui/cat_tree.rs Category tree flattener for panel
|
||||||
|
113 / 0t ui/view_panel.rs View list panel
|
||||||
|
107 / 0t ui/category_panel.rs Category tree panel
|
||||||
|
95 / 0t ui/tile_bar.rs Tile bar (axis assignment tiles)
|
||||||
|
87 / 0t ui/panel.rs Generic panel frame widget
|
||||||
|
81 / 0t ui/formula_panel.rs Formula list panel
|
||||||
|
67 / 0t ui/which_key.rs Prefix-key hint popup
|
||||||
|
12 / 0t ui/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Import layer
|
||||||
|
```
|
||||||
|
773 / 15t import/wizard.rs ImportPipeline + ImportWizard
|
||||||
|
292 / 9t import/analyzer.rs Field kind detection (Category/Measure/Time/Skip)
|
||||||
|
244 / 8t import/csv_parser.rs CSV parsing, multi-file merge
|
||||||
|
3 / 0t import/mod.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Top-level
|
||||||
|
```
|
||||||
|
400 / 0t draw.rs TUI event loop (run_tui), frame composition
|
||||||
|
391 / 0t main.rs CLI entry (clap): open, import, cmd, script
|
||||||
|
228 / 29t format.rs Number display formatting (view-only rounding)
|
||||||
|
806 / 22t persistence/mod.rs .improv save/load (markdown format + gzip + legacy JSON)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Context docs
|
||||||
|
```
|
||||||
|
context/SPEC.md Feature specification
|
||||||
|
context/design-principles.md Architectural principles
|
||||||
|
context/plan.md Show HN launch plan
|
||||||
|
context/repo-map.md This file
|
||||||
|
```
|
||||||
|
|
||||||
|
**Total: ~15,875 lines, 356 tests.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Guidelines
|
||||||
|
|
||||||
|
### Coverage target
|
||||||
|
|
||||||
|
Aim for **~80% line and branch coverage** on logic code. This is a quality floor, not a
|
||||||
|
ceiling — go higher where the code warrants it, but don't chase 100% on rendering
|
||||||
|
widgets or write tests that just exercise trivial getters. Coverage should be run with
|
||||||
|
`cargo llvm-cov` (available via `nix develop`).
|
||||||
|
|
||||||
|
### What to test and how
|
||||||
|
|
||||||
|
| Layer | Approach | Notes |
|
||||||
|
|-------|----------|-------|
|
||||||
|
| **Model** (types, cell, category, symbol) | Unit tests + **proptest** | The data model is the foundation. Property tests catch invariant violations that hand-picked cases miss (see CellKey sort invariant, axis consistency). |
|
||||||
|
| **Formula** (parser, eval) | Unit tests per operator/construct | Cover each BinOp, AggFunc, IF, WHERE, unary minus, chained formulas, error cases (div-by-zero, missing ref). Ensure eval uses full f64 precision — never display-rounded values. |
|
||||||
|
| **View** (types, layout) | Unit tests + **proptest** | Property tests for axis assignment invariants (each category on exactly one axis, transpose is involutive, etc.). Unit tests for layout computation, records mode detection, drill. |
|
||||||
|
| **Command** (cmd, keymap, parse) | Unit tests | Test command execution by building a `CmdContext` and asserting on returned effects. Test keymap lookup fallback chain. Test script parser with edge cases (quoting, comments, dots). |
|
||||||
|
| **Persistence** | Round-trip tests | `save → load → save` must be identical. Cover groups, formulas, views, hidden items, legacy JSON detection. |
|
||||||
|
| **Format** | Unit tests | Boundary cases: comma placement at 3/4/7 digits, negative numbers, rounding half-away-from-zero (not banker's), zero, small fractions. |
|
||||||
|
| **Import** (analyzer, csv, wizard) | Unit tests | Field classification heuristics, CSV quoting (RFC 4180), multi-file merge, date extraction. |
|
||||||
|
| **UI rendering** (grid, panels, draw, help) | Generally skip | Ratatui widgets are hard to unit-test and change frequently. Test the *logic* they consume (layout, cat_tree, format) rather than the rendering itself. |
|
||||||
|
| **Effects** | Test indirectly | Effects are thin `apply` methods. Test via integration: send a key through `App::handle_key` and assert on resulting app state. The complex ones (drill reconciliation, import) deserve targeted unit tests. |
|
||||||
|
|
||||||
|
### Property tests (proptest)
|
||||||
|
|
||||||
|
Use property tests for **invariants that must hold across all inputs**, not as a
|
||||||
|
substitute for example-based tests. Good candidates:
|
||||||
|
|
||||||
|
- Structural invariants: CellKey always sorted, each category on exactly one axis,
|
||||||
|
toggle-collapse is involutive, hide/show roundtrips.
|
||||||
|
- Serialization roundtrips: save/load identity.
|
||||||
|
- Determinism: `evaluate` returns the same value for the same inputs.
|
||||||
|
|
||||||
|
Keep proptest case counts reasonable. The defaults (256 cases) are fine for most
|
||||||
|
properties. Don't crank them up to thousands — the test suite should complete in
|
||||||
|
under 2 seconds. If a property needs more cases to feel confident, that's a sign
|
||||||
|
the input space should be constrained with better strategies, not brute-forced.
|
||||||
|
|
||||||
|
### Bug-fix workflow
|
||||||
|
|
||||||
|
Per CLAUDE.md: **write a test that demonstrates the bug before fixing it.** Prefer
|
||||||
|
a small unit test targeting the specific function over an integration test. The test
|
||||||
|
should fail on the current code, then pass after the fix. Mark regression tests
|
||||||
|
with a doc-comment explaining the bug (see `model/types.rs` `formula_tests` for
|
||||||
|
examples).
|
||||||
|
|
||||||
|
### What not to test
|
||||||
|
|
||||||
|
- Trivial struct constructors and enum definitions (`ast.rs`, `axis.rs`).
|
||||||
|
- Ratatui `Widget::render` implementations — these are pure drawing code.
|
||||||
|
- Module re-export files (`mod.rs`).
|
||||||
|
- One-line delegation methods.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Patterns to Know
|
||||||
|
|
||||||
|
1. **Commands never mutate.** They receive `&CmdContext` (read-only) and return `Vec<Box<dyn Effect>>`.
|
||||||
|
2. **CellKey is always sorted.** Use `CellKey::new()` — never construct the inner Vec directly.
|
||||||
|
3. **`category_mut()` for adding items.** `Model` has no `add_item` method; get the category first: `m.category_mut("Region").unwrap().add_item("East")`.
|
||||||
|
4. **Virtual categories** `_Index` and `_Dim` always exist. `is_empty_model()` checks whether any *non-virtual* categories exist.
|
||||||
|
5. **Display rounding is view-only.** `format_f64` (half-away-from-zero) is only called in rendering. Formula eval uses full f64.
|
||||||
|
6. **Keybindings are per-mode.** `ModeKey::from_app_mode()` resolves the current mode, then the corresponding `Keymap` is looked up. Normal + `search_mode=true` maps to `SearchMode`.
|
||||||
|
7. **`effect_cmd!` macro** generates a command struct that just produces effects. Use for simple commands without complex logic.
|
||||||
|
8. **`.improv` format is markdown-like**, not JSON. See `persistence/mod.rs`. JSON is legacy only.
|
||||||
|
9. **`IndexMap`** is used for categories and views to preserve insertion order.
|
||||||
|
10. **`MAX_CATEGORIES = 12`** applies only to `CategoryKind::Regular`. Virtual/Label categories are exempt.
|
||||||
Reference in New Issue
Block a user