fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

chore: add launch plan

Browse Source
This commit is contained in:
Edward Langley
2026-04-08 21:12:00 -07:00
parent f4e4b75b20
commit 2dbedd36ce
2 changed files with 474 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files

287
context/SPEC.md
View File

@ -9,8 +9,7 @@ semantically labeled structure — separating data, computation, and views into
is a tool where formulas reference meaningful names, views can be rearranged instantly, and the same dataset is a tool where formulas reference meaningful names, views can be rearranged instantly, and the same dataset
can be explored from multiple perspectives simultaneously. can be explored from multiple perspectives simultaneously.
The application compiles to a single static binary (`x86_64-unknown-linux-musl`) and provides a rich TUI The application compiles to a single static binary and provides a rich TUI experience.
experience.
--- ---
@ -23,7 +22,7 @@ experience.
- Items within a category can be organized into **groups** forming a hierarchy. - Items within a category can be organized into **groups** forming a hierarchy.
- Example: Items "Jan", "Feb", "Mar" grouped under "Q1"; quarters grouped under "2025". - Example: Items "Jan", "Feb", "Mar" grouped under "Q1"; quarters grouped under "2025".
- Groups are collapsible/expandable for drill-down. - Groups are collapsible/expandable for drill-down.
- A model supports up to **12 categories**. - A model supports up to **12 categories** (virtual categories prefixed with `_` do not count).
### 1.2 Data Cells ### 1.2 Data Cells
- Each data cell is identified by the intersection of one item from each active category — not by grid coordinates. - Each data cell is identified by the intersection of one item from each active category — not by grid coordinates.
@ -32,7 +31,7 @@ experience.
### 1.3 Models ### 1.3 Models
- A **model** is the top-level container: it holds all categories, items, groups, data cells, formulas, and views. - A **model** is the top-level container: it holds all categories, items, groups, data cells, formulas, and views.
- Models are saved to and loaded from a single `.improv` file (JSON format). - Models are saved to and loaded from a single `.improv` file (plain-text markdown-like format; see §7.1).
--- ---
@ -46,23 +45,24 @@ experience.
- A formula applies uniformly across all intersections of the referenced categories. No copying or dragging. - A formula applies uniformly across all intersections of the referenced categories. No copying or dragging.
### 2.2 Formula Panel ### 2.2 Formula Panel
- Formulas are defined in a **dedicated formula panel**, separate from the data grid. - Formulas are defined in a **dedicated formula panel** (`F` key), separate from the data grid.
- All formulas are visible in one place for easy auditing. - All formulas are visible in one place for easy auditing.
- Formulas cannot be accidentally overwritten by data entry. - Formulas cannot be accidentally overwritten by data entry.
- Within the panel: `n`/`o` to create a new formula, `d` to delete.
### 2.3 Scoped Formulas (WHERE clause) ### 2.3 Scoped Formulas (WHERE clause)
- A formula can be scoped to a subset of items: - A formula can be scoped to a subset of items:
- Example: `Discount = 0.10 * Price WHERE Region = "West"` - Example: `Discount = 0.10 * Price WHERE Region = "West"`
### 2.4 Aggregation ### 2.4 Aggregation
- Built-in aggregation functions: `SUM`, `AVG`, `MIN`, `MAX`, `COUNT`. - Built-in aggregation functions: `Sum`, `Avg`, `Min`, `Max`, `Count`.
### 2.5 Formula Language ### 2.5 Formula Language
- Expression-based (not Turing-complete). - Expression-based (not Turing-complete).
- Operators: `+`, `-`, `*`, `/`, `^`, unary `-`. - Operators: `+`, `-`, `*`, `/`, `^`, unary `-`.
- Comparisons: `=`, `!=`, `<`, `>`, `<=`, `>=`. - Comparisons: `=`, `!=`, `<`, `>`, `<=`, `>=`.
- Conditionals: `IF(condition, then, else)`. - Conditionals: `IF(condition, then, else)`.
- `WHERE` clause for filtering: `SUM(Sales WHERE Region = "East")`. - `WHERE` clause for filtering: `Sum(Sales WHERE Region = "East")`.
- Parentheses for grouping. - Parentheses for grouping.
- Literal numbers and quoted strings. - Literal numbers and quoted strings.
@ -72,49 +72,80 @@ experience.
### 3.1 Views as First-Class Objects ### 3.1 Views as First-Class Objects
- A **view** is a named configuration specifying: - A **view** is a named configuration specifying:
- Which categories are assigned to **rows**, **columns**, and **pages** (filters/slicers). - Which categories are assigned to **rows**, **columns**, **pages** (filters/slicers), or **none** (hidden).
- Which items/groups are visible vs. hidden. - Which items/groups are visible vs. hidden per-view.
- Sort order (future). - Collapsed group state per-view.
- Number formatting. - Number formatting (per-view format string, e.g. `,.2f`).
- Multiple views can exist per model. Each is independent. - Multiple views can exist per model. Each is independent.
- Editing data in any view updates the underlying model; all other views reflect the change. - Editing data in any view updates the underlying model; all other views reflect the change.
### 3.2 Category Tiles ### 3.2 Category Tiles
- Each category is represented as a **tile** displayed in the tile bar. - Each category is represented as a **tile** displayed in the tile bar.
- The user can move tiles between row, column, and page axes to instantly pivot/rearrange the view. - Press `T` (or `Ctrl+Arrow`) to enter tile-select mode.
- Moving a tile triggers an instant recalculation and re-render of the grid. - `h`/`l` or `←`/`→` to select a tile.
- `Space`/`Enter` to cycle axis (Row → Column → Page).
- `r`/`c`/`p` to set axis directly to Row / Column / Page.
- Moving a tile triggers instant recalculation and re-render of the grid.
### 3.3 Page Axis (Slicing) ### 3.3 Page Axis (Slicing)
- Categories assigned to the page axis act as filters. - Categories assigned to the page axis act as filters.
- The user selects a single item from a paged category using `[` and `]`. - The user selects a single item from a paged category using `[` and `]`.
### 3.4 Collapsing and Expanding ### 3.4 Collapsing and Expanding
- Groups can be collapsed/expanded per-view (future: keyboard shortcut in grid). - Groups can be collapsed/expanded per-view with the `z` key.
### 3.5 Drill-Down and View History
- `>` drills into an aggregated cell, capturing a snapshot.
- `<` navigates back through view history.
### 3.6 Records Mode
- `R` toggles records mode: a long-format view activated when `_Index` is on Row and `_Dim` is on Column.
- `P` toggles "prune empty" to hide rows/columns with no data.
### 3.7 Transpose
- `t` swaps the Row and Column axes instantly.
--- ---
## 4. JSON Import Wizard ## 4. Import Wizard
### 4.1 Purpose ### 4.1 Purpose
- Users can import arbitrary JSON files to bootstrap a model. - Users can import CSV or JSON files to bootstrap a model.
- Multiple CSV files can be merged with an automatic "File" category.
### 4.2 Wizard Flow (interactive TUI) ### 4.2 Wizard Flow (interactive TUI)
**Step 1: Preview** — Structural summary of the JSON. **Step 1: Preview** — Structural summary of the data.
**Step 2: Select Array Path** — If the JSON is not a flat array, the user selects which key path contains the primary record array. **Step 2: Select Array Path** — (JSON only) If the JSON is not a flat array, the user selects which key path contains the primary record array.
**Step 3: Review Proposals** — Fields are analyzed and proposed as: **Step 3: Review Proposals** — Fields are analyzed and proposed as:
- Category (small number of distinct string values) - Category (small number of distinct string values)
- Measure (numeric) - Measure (numeric)
- Time Category (date-like strings) - Time Category (date-like strings, with optional date component extraction: Year, Month, Quarter)
- Label/Identifier (skip) - Skip (exclude from import)
**Step 4: Name the Model** — User names the model and confirms. **Step 4: Configure** — Set axis assignments, add formulas, name the model.
### 4.3 Headless Import ### 4.3 CLI Import
``` ```bash
improvise --cmd '{"op":"ImportJson","path":"data.json"}' # Interactive wizard
improvise import data.json
improvise import sales.csv expenses.csv # merge multiple CSVs
# Headless (skip wizard)
improvise import data.json --no-wizard -o model.improv
# With field overrides
improvise import data.csv \
--category Region \
--measure Revenue \
--time Date \
--extract Date:Month \
--axis Region:row \
--formula "Profit = Revenue - Cost" \
--name "Sales Model" \
-o output.improv
``` ```
--- ---
@ -124,7 +155,7 @@ improvise --cmd '{"op":"ImportJson","path":"data.json"}'
### 5.1 Layout ### 5.1 Layout
``` ```
+---------------------------------------------------------------+ +---------------------------------------------------------------+
| Improvise | Model: Sales 2025 [*] [F1 Help] [Ctrl+Q] | | improvise · Sales 2025 (model.improv) [+] ?:help :q quit|
+---------------------------------------------------------------+ +---------------------------------------------------------------+
| [Page: Region = East] | | [Page: Region = East] |
| | Q1 | Q2 | Q3 | Q4 | | | | Q1 | Q2 | Q3 | Q4 | |
@ -135,39 +166,79 @@ improvise --cmd '{"op":"ImportJson","path":"data.json"}'
|--------------+---------+---------+---------+---------+--------| |--------------+---------+---------+---------+---------+--------|
| Total | 4,100 | 4,670 | 3,750 | 5,800 | | | Total | 4,100 | 4,670 | 3,750 | 5,800 | |
+---------------------------------------------------------------+ +---------------------------------------------------------------+
| Tiles: [Time ↔] [Product ↕] [Region ☰] Ctrl+↑↓←→ tiles | | Tiles: [Time ↔] [Product ↕] [Region ☰] T to select |
+---------------------------------------------------------------+ +---------------------------------------------------------------+
| NORMAL | Default | Ctrl+F:formulas Ctrl+C:categories ... | | NORMAL | Default ?:help :q quit |
+---------------------------------------------------------------+ +---------------------------------------------------------------+
``` ```
### 5.2 Panels ### 5.2 Panels and Modes
- **Grid panel** (main): Scrollable table of the current view. - **Grid** (main): Scrollable table of the current view.
- **Tile bar**: Category tiles with axis symbols. `Ctrl+Arrow` enters tile-select mode. - **Tile bar**: Category tiles with axis symbols. `T` or `Ctrl+Arrow` enters tile-select mode.
- **Formula panel**: `Ctrl+F` — list and edit formulas. - **Formula panel**: `F` — list and edit formulas. `Ctrl+F` toggles visibility without focus.
- **Category panel**: `Ctrl+C` — manage categories and axis assignments. - **Category panel**: `C` — manage categories, items, and axis assignments. `Ctrl+C` toggles visibility.
- **View panel**: `Ctrl+V` — switch, create, delete views. - **View panel**: `V` — switch, create, delete views. `Ctrl+V` toggles visibility.
- **Status bar**: Mode, active view name, keyboard hints. - **Status bar**: Mode indicator, active view name, keyboard hints.
- **Help overlay**: `?` or `F1` — full key reference.
**Modes**: Normal, Insert (editing), Formula Edit, Formula Panel, Category Panel,
View Panel, Tile Select, Category Add, Item Add, Export Prompt, Command (`:` prefix),
Search, Import Wizard, Help, Quit.
### 5.3 Navigation and Editing ### 5.3 Navigation and Editing
| Key | Action | | Key | Action |
|-----|--------| |-----|--------|
| ↑↓←→ / hjkl | Move cursor | | ↑↓←→ / hjkl | Move cursor |
| Enter | Edit cell | | gg / G | Jump to first / last row |
| Esc | Cancel edit | | 0 / $ | Jump to first / last column |
| Tab | Focus next open panel | | Ctrl+D / Ctrl+U | Scroll 5 rows down / up |
| / | Search | | PageDown / PageUp | Page scroll |
| [ / ] | Page axis prev / next | | [ / ] | Page axis prev / next |
| Ctrl+Arrow | Tile select mode | | i / a | Enter insert mode (or drill into aggregated cell) |
| Enter/Space (tile) | Cycle axis (Row→Col→Page) | | Enter | Advance to next cell while editing |
| r / c / p (tile) | Set axis directly | | o | Add record row and begin editing |
| Ctrl+F | Toggle formula panel | | Esc | Cancel edit / return to Normal |
| Ctrl+C | Toggle category panel | | x | Clear cell |
| Ctrl+V | Toggle view panel | | yy | Yank (copy) cell value |
| p | Paste yanked value |
| / | Search grid |
| n | Next search match |
| N | New category quick-add (or previous search match context) |
| t | Transpose (swap rows ↔ columns) |
| z | Toggle group collapse under cursor |
| H | Hide current row item |
| > | Drill into aggregated cell |
| < | Navigate back (view history) |
| R | Toggle records mode |
| P | Toggle prune-empty rows/columns |
| T / Ctrl+Arrow | Enter tile-select mode |
| F | Toggle formula panel (focus) |
| C | Toggle category panel (focus) |
| V | Toggle view panel (focus) |
| Ctrl+F / Ctrl+C / Ctrl+V | Toggle panel visibility (no focus change) |
| Tab | Cycle focus to next open panel |
| : | Enter command mode |
| Ctrl+S | Save | | Ctrl+S | Save |
| Ctrl+E | Export CSV | | Ctrl+E | Export CSV prompt |
| F1 | Help | | ZZ | Save and quit |
| Ctrl+Q | Quit | | Ctrl+Q | Force quit |
| ? / F1 | Help overlay |
### 5.4 Command Mode (`:`)
| Command | Description |
|---------|-------------|
| `:q` | Quit (warns if unsaved) |
| `:q!` | Force quit |
| `:wq` / `ZZ` | Save and quit |
| `:w [path]` | Save (path optional) |
| `:export [path.csv]` | Export active view to CSV |
| `:import <path>` | Open import wizard |
| `:add-cat <name>` | Add a category |
| `:add-item <cat> <item>` | Add one item to a category |
| `:add-items <cat> a b c…` | Add multiple items at once |
| `:formula <cat> <Name=expr>` | Add a formula |
| `:add-view [name]` | Create a new view |
| `:show-item <cat> <item>` | Restore a hidden item |
--- ---
@ -178,63 +249,81 @@ All model mutations go through a typed command layer. This enables:
- Replay / audit log - Replay / audit log
- Testing without rendering - Testing without rendering
### 6.1 Command Format ### 6.1 CLI Subcommands
JSON object with an `op` field:
```json
{"op": "CommandName", ...args}
```
### 6.2 Available Commands
| op | Required fields | Description |
|----|-----------------|-------------|
| `AddCategory` | `name` | Add a category/dimension |
| `AddItem` | `category`, `item` | Add an item to a category |
| `AddItemInGroup` | `category`, `item`, `group` | Add an item in a named group |
| `SetCell` | `coords: [[cat,item],...]`, `number` or `text` | Set a cell value |
| `ClearCell` | `coords` | Clear a cell |
| `AddFormula` | `raw`, `target_category` | Add/replace a formula |
| `RemoveFormula` | `target` | Remove a formula by target name |
| `CreateView` | `name` | Create a new view |
| `DeleteView` | `name` | Delete a view |
| `SwitchView` | `name` | Switch the active view |
| `SetAxis` | `category`, `axis` (`"row"/"column"/"page"`) | Set category axis |
| `SetPageSelection` | `category`, `item` | Set page-axis filter |
| `ToggleGroup` | `category`, `group` | Toggle group collapse |
| `Save` | `path` | Save model to file |
| `Load` | `path` | Load model from file |
| `ExportCsv` | `path` | Export active view to CSV |
| `ImportJson` | `path`, `model_name?`, `array_path?` | Import JSON file |
### 6.3 Response Format
```json
{"ok": true, "message": "optional message"}
{"ok": false, "message": "error description"}
```
### 6.4 Invocation
```bash ```bash
# Single command # Open TUI (default)
improvise model.improv --cmd '{"op":"SetCell","coords":[["Region","East"],["Measure","Revenue"]],"number":1200}' improvise [model.improv]
# Script file (one JSON object per line, # comments allowed) # Single headless command(s)
improvise model.improv --script setup.jsonl improvise cmd 'set-cell Region/East Measure/Revenue 1200' -f model.improv
# Script file (one command per line, # or // comments)
improvise script setup.txt -f model.improv
# Import
improvise import data.json
``` ```
### 6.2 Script Syntax
Commands use a prefix syntax (one per line). Multiple commands can be separated by `.` on a single line. Lines starting with `#` or `//` are comments.
### 6.3 Available Commands
Commands are registered internally and accessible via `:` in the TUI or via the `cmd`/`script` CLI subcommands. Key operations include:
- Cell manipulation: `set-cell`, `clear-cell`, `yank`, `paste`
- Navigation: `move-selection`, `scroll-rows`, `page-scroll`, `jump-first-row`, `jump-last-row`
- View: `transpose`, `toggle-records-mode`, `toggle-prune-empty`, `drill-into-cell`, `view-back`, `page-next`, `page-prev`
- Categories: `add-cat`, `add-item`, `add-items`, `delete-category-at-cursor`, `cycle-axis-at-cursor`, `filter-to-item`, `hide-selected-row-item`, `show-item`
- Formulas: `formula`, `enter-formula-edit`, `delete-formula-at-cursor`, `commit-formula`
- Views: `switch-view-at-cursor`, `create-and-switch-view`, `delete-view-at-cursor`, `add-view`
- Tiles: `enter-tile-select`, `move-tile-cursor`, `cycle-axis-for-tile`, `set-axis-for-tile`
- File: `save`, `wq`, `export`, `import`
- Modes: `enter-mode`, `search`, `enter-edit-mode`, `enter-export-prompt`
--- ---
## 7. Persistence ## 7. Persistence
### 7.1 File Format ### 7.1 File Format
Native format: JSON-based `.improv` file containing all categories, items, groups, data cells, formulas, and view definitions. Native format: plain-text markdown-like `.improv` file. Structure:
Compressed variant: `.improv.gz` (gzip, same JSON payload). ```
# Model Name
## Category: Region
- North
- South
- East [Coastal]
- West [Coastal]
> Coastal
## Formulas
- Profit = Revenue - Cost [Measure]
## Data
Region=East, Measure=Revenue = 1200
Region=East, Measure=Cost = 800
Region=West, Measure=Revenue = "pending"
## View: Default (active)
Region: row
Measure: column
Time: page, Q1
hidden: Region/Internal
collapsed: Time/2024
format: ,.2f
```
Compressed variant: `.improv.gz` (gzip, same payload).
Legacy JSON format is auto-detected (by `{` prefix) for backward compatibility.
### 7.2 Export ### 7.2 Export
- `Ctrl+E` in TUI or `ExportCsv` command: exports active view to CSV. - `Ctrl+E` in TUI or `:export [path.csv]` command: exports active view to CSV.
- Respects current view axes and page filters.
### 7.3 Autosave ### 7.3 Autosave
- Periodic autosave (every 30 seconds when dirty) to `.model.improv.autosave`. - Periodic autosave (every 30 seconds when dirty) to `.{filename}.autosave`.
--- ---
@ -244,8 +333,9 @@ Compressed variant: `.improv.gz` (gzip, same JSON payload).
|---------|--------| |---------|--------|
| Language | Rust (stable) | | Language | Rust (stable) |
| TUI | [Ratatui](https://github.com/ratatui-org/ratatui) + Crossterm | | TUI | [Ratatui](https://github.com/ratatui-org/ratatui) + Crossterm |
| Serialization | `serde` + `serde_json` | | Serialization | `serde` + `serde_json` (for legacy compat); native format is plain text |
| Static binary | `x86_64-unknown-linux-musl` via `musl-gcc` | | CLI | `clap` with subcommands (`open`, `import`, `cmd`, `script`) |
| Build | Standard `cargo build --release` (LTO, stripped); optional musl target via Nix |
| Dev environment | Nix flake with `rust-overlay` | | Dev environment | Nix flake with `rust-overlay` |
| No runtime deps | Single binary, no database, no network | | No runtime deps | Single binary, no database, no network |
@ -266,17 +356,18 @@ Compressed variant: `.improv.gz` (gzip, same JSON payload).
```bash ```bash
# Build # Build
nix develop --command cargo build --release nix develop --command cargo build --release
file target/x86_64-unknown-linux-musl/release/improvise # → statically linked
# Import test # Import test (interactive wizard)
./improvise --cmd '{"op":"ImportJson","path":"sample.json"}' --cmd '{"op":"Save","path":"test.improv"}' ./improvise import sample.json
# Formula test # Import test (headless)
./improvise test.improv \ ./improvise import sample.json --no-wizard -o test.improv
--cmd '{"op":"AddFormula","raw":"Profit = Revenue - Cost","target_category":"Measure"}'
# Headless commands
./improvise cmd 'add-cat Region' 'add-item Region East' -f new.improv
# Headless script # Headless script
./improvise new.improv --script tests/setup.jsonl ./improvise script tests/setup.txt -f new.improv
# TUI # TUI
./improvise model.improv ./improvise model.improv

284
context/plan.md Normal file
View File

@ -0,0 +1,284 @@
# improvise — launch preparation plan
You are working in the `improvise` repository, a Rust terminal application that implements a Lotus Improv-style pivot-table modeling tool. Cells are keyed by sets of `(category, item)` coordinates rather than 2D grid addresses; views assign categories to row/column/page axes; formulas reference dimension names rather than cell addresses. The application is built with `ratatui` + `crossterm`, persists models in a markdown-based `.improv` format (with optional gzip), and supports both an interactive TUI and a headless command/script mode through a unified registry.
Your job is to prepare this repository for a public Show HN launch. The bar is "ready for strangers to install and try in 60 seconds," not "feature-complete." Do tasks in the order given. Do not start a later phase before earlier phases are complete. Do not add features, refactor warts, or restructure modules — those are explicitly out of scope and listed at the end.
The target audience is developers who use vim, who have opinions about data modeling, and who would understand a Lotus Improv reference. The pitch positions improvise as a pivot-table modeling tool, not "a spreadsheet in a terminal" — that framing competes with sc-im and VisiData and loses.
The development environment is Nix-based (the repo has a `flake.nix`). The user does not have Homebrew installed and will not install it. All tooling must be available through Nix, and any process that needs new tools must be codified as additions to `flake.nix` so the user can reach them via `nix develop` or `nix run`. Do not suggest `brew install` anywhere.
---
## Phase 1 — Repository hygiene
Goal: raise the quality floor of what a stranger sees when they land on the repo. Roughly 90 minutes of work.
### 1.1 Audit and remove or update `context/SPEC.md`
Read `context/SPEC.md` and compare it against the actual code. The spec is likely stale: it may describe a JSON import wizard when the real wizard handles CSV, and it may describe JSON persistence when the real format is markdown (look at `src/persistence/mod.rs` to confirm). It may also list commands that no longer exist in the registry.
If the spec contradicts the code in significant ways, delete it. If you want to preserve design intent, move salvageable conceptual content to `docs/design-notes.md` with a header that reads:
> **Note:** This document captures design intent and may not reflect the current implementation. The README and source code are authoritative.
Do not try to bring the spec back into sync with the code — that is not worth the effort for a personal project, and a stale spec is worse than no spec.
### 1.2 Fix `Cargo.toml` metadata
The current `[package]` section is missing fields that crates.io and `cargo dist` need. Update it to include:
- `description` — replace any generic placeholder with: `"Terminal pivot-table modeling in the spirit of Lotus Improv"`
- `repository` — ask the user for the GitHub URL if you don't already know it; otherwise leave a `TODO` comment and flag it
- `homepage` — same URL as `repository`
- `documentation` — same URL as `repository` for now
- `readme = "README.md"`
- `keywords = ["tui", "pivot", "spreadsheet", "data", "improv"]` (max 5 keywords, each ≤20 chars)
- `categories = ["command-line-utilities", "visualization"]`
- `license` — confirm this is set; if not, ask the user
Preserve all existing fields and dependency entries.
### 1.3 Verify publish-readiness
Run `cargo publish --dry-run` from inside `nix develop`. Fix any errors or warnings. Common issues: missing license file, files too large to publish, dependencies with incompatible versions. Do not actually publish — that happens in Phase 3.
### 1.4 Audit CSV quote handling
Look at `src/import/csv_parser.rs` (or wherever CSV parsing lives). Verify that it correctly handles RFC 4180 quoted fields: fields enclosed in double quotes, embedded commas inside quoted fields, and escaped quotes (`""` inside a quoted field).
If the parser is using the `csv` crate from `Cargo.toml`, this should be handled correctly by default — verify that the code is actually using it and not doing manual `split(',')` anywhere. If there are any places that do manual splitting or fragile quote handling, fix them by routing through the `csv` crate. Add a unit test that round-trips a CSV row with `"O'Reilly, Inc."` (embedded comma) and `"She said ""hi"""` (embedded escaped quotes).
If the parser is fundamentally broken in ways that can't be fixed quickly, do not block on this. Add a one-line note to the README under a "Known limitations" section: "CSV files with unusual quoting may not parse correctly; PRs welcome."
### 1.5 Create `examples/demo.improv` and `examples/demo.csv`
Create two synthetic example files with obviously-fake data. These exist so a new user can run `improvise examples/demo.improv` and immediately see a working pivot, or `improvise --import examples/demo.csv` and walk through the wizard.
**`examples/demo.csv`** should contain ~30-50 rows with columns like:
- `Date` (mix of dates across at least 2 quarters of one year)
- `Region` (3-4 values: North, South, East, West)
- `Product` (3-4 values: Widget, Gadget, Sprocket, Doohickey)
- `Customer` (5-8 fake company names: Acme Corp, Globex, Initech, Umbrella, Soylent, etc.)
- `Revenue` (round numbers, 100-10000)
- `Cost` (round numbers, less than Revenue)
**`examples/demo.improv`** should be the result of importing `demo.csv` through the tool, then manually saving. To create it: build the tool, run the import on `demo.csv`, optionally pivot it into an interesting default view (e.g., Region on rows, Date_Quarter on columns, Product on page axis), add a sample formula like `Profit = Revenue - Cost` if the formula system supports it, save as `examples/demo.improv`, and commit both files.
If you cannot determine the exact format syntax, look at existing test fixtures or run the tool's save path on a small in-memory model to generate one.
The data must be obviously synthetic. Do not copy data from any other file in the repo. Do not use any real-looking names, real amounts, or anything that looks like it might be from a real bank export.
---
## Phase 2 — README and demo artifacts
This is the main weekend's work. The README is 80% of the launch. The demo artifacts are the other 20%. Nothing in Phase 3 or 4 matters if Phase 2 is weak.
### 2.1 Add Nix tooling for asciinema and VHS
The user does not have Homebrew. Both asciinema (terminal session recording) and VHS (Charmbracelet's terminal-to-GIF tool, package name `vhs` in nixpkgs) need to be available through Nix.
Modify `flake.nix` to add `pkgs.asciinema` and `pkgs.vhs` to the dev shell's `nativeBuildInputs`. Both packages exist in nixpkgs unstable; verify before adding. If `vhs` requires additional runtime dependencies (it uses `ttyd` and `ffmpeg` internally), add those too — check the nixpkgs `vhs` derivation to see what's already bundled.
After modifying the flake, verify that `nix develop --command asciinema --version` and `nix develop --command vhs --version` both work.
While you're in the flake, also add:
- `pkgs.cargo-dist` for the release tooling in Phase 3 (if it's packaged in nixpkgs; if not, fall back to running it via `cargo install` inside the dev shell and note this in a comment)
- A `nix run` app or shell alias for the demo recording workflow (see 2.4)
### 2.2 Write the README
Replace any existing `README.md` with a new one structured as follows. Aim for under 250 lines total.
**Section order (do not deviate):**
1. **Title** — `# improvise`
2. **One-sentence pitch** — italicized, immediately under the title:
> *Terminal pivot-table modeling in the spirit of Lotus Improv — multidimensional cells, formulas over dimensions instead of cell addresses, and vim-style keybindings for reassigning axes on the fly.*
3. **Inline animated demo** — `![demo](docs/demo.gif)` (the GIF generated in step 2.5). This must be near the top so it shows in HN previews and the GitHub repo card.
4. **Why this exists** — exactly one paragraph. Explain the Improv data/view separation and why no terminal tool currently does it. Reference Lotus Improv (NeXT, 1991) by name. Briefly note that Excel pivot tables took the visual idea but not the formula model. Do not bash other tools — name sc-im and VisiData neutrally as adjacent-but-different.
5. **Quick start** — three code blocks:
```
nix develop
cargo build --release
./target/release/improvise examples/demo.improv
```
Followed by a one-line "or import your own CSV":
```
./target/release/improvise --import path/to/data.csv
```
6. **Key bindings to try first** — short list, not a complete reference. `T` for tile mode (reassign axes), `[` `]` for page axis cycling, `>` to drill into a cell, `<` to return, `F` for formula panel, `:w` to save, `:q` to quit, `?` or `F1` for full help.
7. **Installation** — three subsections:
- **From source with Nix** (preferred): `nix develop` then `cargo build --release`, then optionally `cargo install --path .`
- **From crates.io**: `cargo install improvise` (will be valid after Phase 3)
- **Prebuilt binaries**: link to GitHub releases page (will be populated after Phase 3)
Do not mention Homebrew. Do not mention `apt`, `dnf`, or other distro package managers.
8. **What's interesting about the codebase** — 10-15 lines, prose, no bullets. Cover: the multidimensional data model with categories instead of grid coordinates; the view layer as a pure function from `(Model, View)` to `GridLayout`; records mode as just another axis assignment; the command/effect architecture that lets the same registry serve both interactive dispatch and headless scripts; the markdown `.improv` persistence format that's human-readable and git-diffable.
9. **Expectations** — mandatory disclaimer paragraph, exact wording:
> improvise is a personal project I built for my own use. I'm sharing it because other people might find it useful, but I can't promise active maintenance or feature development. Issues and PRs are welcome but may not get a fast response. If you want to build on it, fork away.
10. **License** — one line.
Do not add: a table of contents, badges (build status, version, license badges, etc.), a contributing guide, a code of conduct, an "inspired by" gratitude section. These are noise for a personal project launch.
### 2.3 Create `docs/demo.tape` and generate `docs/demo.gif`
VHS is scripted: you write a `.tape` file describing keystrokes and timing, and VHS produces a GIF. This GIF goes inline in the README and is the single highest-leverage artifact in the launch — it's what shows up in HN preview cards and Google search results.
Create `docs/demo.tape` that scripts a ~20-second flow showing the pivot reassignment killer demo. Rough script structure (consult VHS docs for exact syntax):
```
Output docs/demo.gif
Set FontSize 14
Set Width 1000
Set Height 600
Set Theme "Dracula"
Type "improvise examples/demo.improv"
Enter
Sleep 1500ms
# Show initial pivot
Sleep 1s
# Enter tile mode and reassign an axis
Type "T"
Sleep 800ms
# (whatever keystrokes reassign Region from rows to columns)
Sleep 1500ms
# Show the new pivot
Sleep 1s
# Reassign again to demonstrate the speed
# (more keystrokes)
Sleep 1500ms
# Exit
Type ":q"
Enter
```
Generate the GIF by running `nix develop --command vhs docs/demo.tape`. Iterate on the script until the GIF is under 5MB, the timing is readable (not too fast, not too slow), and the key beats are clear: start in pivot view → press T → axis reassigns → press T again → axis reassigns again. The viewer should be able to understand "this tool re-pivots data with one keystroke" without any narration.
### 2.4 Record asciinema casts
Asciinema produces `.cast` files that the asciinema-player JS widget can replay in a browser, with selectable text and pixel-perfect terminal rendering. These will be embedded in the GitHub Pages landing page in Phase 4.
Before recording, set the terminal to exactly 100 columns by 30 rows: `stty cols 100 rows 30` (or resize the terminal window manually if `stty` doesn't take). This matters because the asciinema player renders at the recorded dimensions and a wrong size will look broken on the landing page.
Record four casts under `docs/casts/`:
- **`docs/casts/import.cast`** — start with `improvise` (no args, empty model), trigger CSV import for `examples/demo.csv`, walk through the wizard accepting defaults, end in pivot view.
- **`docs/casts/pivot.cast`** — start from `improvise examples/demo.improv`, demonstrate axis reassignment with `T`. This is the same flow as the README GIF but longer and more complete. Show 2-3 different pivots.
- **`docs/casts/drill.cast`** — from a pivot view, press `>` to drill into an aggregated cell, show the records view, demonstrate that you can edit a record, press `<` to return.
- **`docs/casts/formulas.cast`** — from a pivot view, open the formula panel with `F`, add `Profit = Revenue - Cost`, show it appearing across the pivot.
Record each with `nix develop --command asciinema rec -i 2 docs/casts/<name>.cast`. The `-i 2` flag caps idle gaps at 2 seconds, which prevents long pauses from making the playback feel dead. Each cast should be under 60 seconds.
If a take has flubs, delete it and re-record. Do not try to edit the JSON cast files manually.
Add a `nix run` app to the flake or a shell script in `scripts/record-demo.sh` that wraps the recording workflow with the right `stty` setup, so the user can re-record consistently in the future without remembering the exact incantation.
### 2.5 Verify all artifacts exist and are committed
Before moving to Phase 3, confirm:
- `README.md` exists with all 10 sections
- `docs/demo.gif` exists, is referenced from the README, and is under 5MB
- `docs/demo.tape` exists and regenerates the GIF when run through VHS
- `docs/casts/import.cast`, `pivot.cast`, `drill.cast`, `formulas.cast` all exist
- `examples/demo.csv` and `examples/demo.improv` exist and contain only synthetic data
- `flake.nix` includes `asciinema`, `vhs`, and `cargo-dist` (or a fallback for `cargo-dist`)
- `nix develop` succeeds and all the tools above are on PATH
---
## Phase 3 — Distribution
### 3.1 Configure `cargo dist`
`cargo dist` generates a GitHub Actions workflow that produces release tarballs and installer scripts when you push a version tag. Run `nix develop --command cargo dist init` and configure for these targets:
- `x86_64-unknown-linux-gnu`
- `aarch64-apple-darwin`
- `x86_64-apple-darwin`
Skip Windows. Skip musl unless `cargo dist init` strongly recommends it — the existing flake doesn't build for musl and adding that complication is out of scope.
Commit the generated `.github/workflows/release.yml` and the additions to `Cargo.toml`. Test the workflow by tagging `v0.1.0-rc1` and pushing the tag to a branch — verify the release builds successfully on GitHub Actions before doing a real release. Delete the rc tag afterward.
### 3.2 Publish to crates.io
After `cargo publish --dry-run` is clean (from step 1.3) and the user confirms they're ready, run `nix develop --command cargo publish`. Verify the crate appears at `https://crates.io/crates/improvise` and that `cargo install improvise` works on a clean machine (or in a fresh `nix shell --packages cargo`).
### 3.3 Tag the v0.1.0 release
Create a git tag `v0.1.0`, push it, and verify the `cargo dist` workflow produces release artifacts. Update the README's "Prebuilt binaries" link to point at the actual release.
---
## Phase 4 — Landing page (optional but recommended)
### 4.1 Create `docs/index.html`
Vanilla HTML, single file, under 200 lines. No framework, no build step. Dark background, monospace headings to match the terminal aesthetic. Embed the asciinema-player from jsdelivr CDN (`https://cdn.jsdelivr.net/npm/asciinema-player@3/dist/bundle/`) and reference the four `.cast` files from `docs/casts/`.
Section structure: title and one-line tagline → "Reassign axes on the fly" with the pivot cast → "Drill into aggregated cells" with the drill cast → "Formulas over dimensions" with the formulas cast → "Import a CSV" with the import cast → "Get it" with the install commands and a GitHub link.
Each cast should be embedded with `AsciinemaPlayer.create()` configured with `rows: 30, cols: 100, theme: 'monokai', autoPlay: false, loop: true`.
### 4.2 Enable GitHub Pages
In the GitHub repo settings, enable Pages with source set to the `main` branch and folder set to `/docs`. Verify the site is live at `https://<user>.github.io/improvise/` within a few minutes of pushing.
---
## Phase 5 — Buffer
The second weekend (or the second half of the first weekend) is reserved for whatever was missed in Phases 1-4. Do not use it to add features. Do not use it to fix the issues listed in "Out of scope" below. If everything is done early, stop.
---
## Out of scope — do not do these before launch
The following are real issues that have been identified but are explicitly not launch-blocking. Do not start any of them as part of this plan. They can be addressed after launch based on what real users actually trip over.
- **Do not** raise or restructure the `MAX_CATEGORIES = 12` limit, even though it's tight for some real use cases.
- **Do not** refactor the `ExecuteCommand` vim command match ladder, even though it duplicates registry logic.
- **Do not** fix the hardcoded `"Measure"` category name in `evaluate_aggregated`, even though it's the one place in the code that assumes a specific user-facing name.
- **Do not** change `Model::evaluate` from a linear formula walk to a HashMap lookup, even though it's O(N·M·F) in the render hot path.
- **Do not** add render-pass memoization to `matching_values` calls.
- **Do not** fix the `ApplyAndClearDrill` HashMap iteration ordering issue around coordinate rename + value edit interactions.
- **Do not** add new features: no YoY references, no parameterized formulas, no Datalog backend, no undo/redo, no charts, no plugin system.
- **Do not** write a documentation site beyond the single GitHub Pages landing page in Phase 4.
- **Do not** add Windows support.
- **Do not** rename types, restructure modules, or do any "while I'm in here" cleanups.
Each of these is worth doing eventually. None of them is worth doing before the launch post is up. The goal of this plan is to get the existing tool in front of strangers in a form they can install and try, not to make the tool perfect first.
---
## Definition of done
The plan is complete when, on a clean machine with only Nix installed:
1. `git clone <repo> && cd improvise && nix develop` succeeds.
2. `cargo build --release` succeeds.
3. `./target/release/improvise examples/demo.improv` opens an interesting pivot immediately.
4. The README renders correctly on GitHub with the inline GIF playing.
5. `cargo install improvise` from a fresh shell works and produces a runnable binary.
6. The GitHub releases page has prebuilt binaries for Linux x86_64 and macOS (Intel + Apple Silicon).
7. The GitHub Pages site at `https://<user>.github.io/improvise/` loads and the asciinema casts play.
8. Nothing in the "Out of scope" list has been touched.
When all eight conditions hold, stop. Report back to the user with a summary of what was done and any blockers encountered. Do not post to Hacker News yourself — that's the user's call and their timing.