Edward Langley
bf02472d89
Three findings from the review-only PR #13, which put 28 already-merged
commits in front of a second reader, plus two comments that had gone false.
worker.js — `self.onmessage` was `async` and awaited `handle()`, which awaits
`server.flush()`. An async listener does NOT serialize: the event loop may
start the next `message` while an earlier one is suspended at an await, so two
`on_message`/`flush` pairs interleave and a later frame's replies can be posted
before an earlier frame's. That breaks the one rule the file names first — a
`GridUpdate::Cells` that overtakes its labels is DROPPED by the client's fold,
not replayed (gotcha 33) — and the pre-start `pending` drain had the same race
twice over, since `server` was assigned before the loop that drained it, so a
message arriving mid-drain overtook the frames it was queued behind. Every
frame is now chained onto one tail promise, which also makes "arrived before
start() finished" an ordinary case of the same rule instead of a second path.
app.js — the handshake watchdog fires 8s after the handshake and writes "the
peer did not answer". A peer that failed LOUDLY has already said something more
precise (worker `error`, `messageerror`, socket `error`), and the watchdog then
overwrote it with the vaguer guess. Those three paths now disarm it:
`setShellFailure` clears the timer rather than merely pre-empting it. The
watchdog is for silence only, which is what it was always for.
README — the "In the browser" section still said the deployed page needs a
server and that the in-page worker "is not built yet". Both landed in the same
unreviewed batch (improvise-9x6, improvise-djm), so the README described the
published site as a client with nowhere to connect while index.html described
it correctly. Rewritten, and it now states the caveat the commits state: no
line of that JavaScript has ever run.
Two comments that had gone false with it, and for the same reason — a follow-up
commit updated one copy and not the others. `web/app.js`'s keyboard note said
key resolution was "text entry, and nothing else" and that upstream keys would
work "until improvise-bpq2 lands"; bpq2 landed, moved the grid, panel and tile
cursors to the client, and settled that the host must not know the keymap.
`Host::handle`'s `ClientFrame::Key` arm said the same thing; 43422d4 corrected
the worker's copy of that argument and missed this one.
NOT VERIFIED, and it cannot be here: there is no browser and no headless DOM in
this environment, so neither JavaScript change has been executed. Both are
argued rather than tested, which is why each carries its argument in the file.
`node --check` passes on both, the wasm bundles build, and the Rust gates are
unchanged: 1172 passing / 0 failed, clippy clean, fmt clean, actionlint clean,
wasm32 clean for all seven bundle crates.
Declined from the same review, with the reason on the thread: `Host::commands`
can apply a batch's earlier commands before rejecting a later one. The finding
is correct, the hole is real, and it is already stated on `ClientFrame::Commands`
and in `Host::commands` — closing it means the basis check moving from the
command to the batch, which is a design change with ADR 001 consequences and
not a patch to land beside three small ones.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
7.8 KiB
improvise
Terminal pivot-table modeling in the spirit of Lotus Improv — multidimensional cells, formulas over dimensions instead of cell addresses, and vim-style keybindings for reassigning axes on the fly.
Why this exists
Lotus Improv (NeXT, 1991) separated data from its presentation: cells were addressed by named dimensions, not grid coordinates, and rearranging a view didn't break formulas. That idea never made it into mainstream tools. Excel pivot tables borrowed the visual rearrangement but kept cell-address formulas. Terminal data tools like sc-im and VisiData do different things well — sc-im is a traditional spreadsheet, VisiData is a data explorer — but neither offers the dimension-keyed data model that made Improv interesting. improvise is a small attempt to bring that model to a modern terminal, with a formula language that references category and item names, views that can be rearranged with a single keystroke, and a plain-text file format you can diff in git.
See docs/design-notes.md for the original product vision and non-goals.
Quick start
nix build .
./result/bin/improvise examples/demo.improv
Or import your own CSV:
./result/bin/improvise import path/to/data.csv
The included examples/demo.improv was generated from examples/demo.csv:
improvise import examples/demo.csv \
--no-wizard \
--category Region --category Product --category Customer \
--measure Revenue --measure Cost \
--time Date --extract Date:Month \
--axis Region:row --axis Product:row \
--axis Date_Month:column --axis Measure:column \
--axis Customer:page --axis Date:none \
--formula "Profit = Revenue - Cost" \
--name "Acme Sales Demo" \
-o examples/demo.improv
Key bindings to try first
| Key | Action |
|---|---|
T |
Enter tile mode — reassign category axes |
[ / ] |
Cycle through page-axis items |
> |
Drill into an aggregated cell |
< |
Return from drill-down |
F |
Open the formula panel |
t |
Transpose rows and columns |
? / F1 |
Full key reference |
:w |
Save |
:q |
Quit |
Installation
With Nix (preferred)
nix build .
# or install into your profile:
nix profile install .
From crates.io
cargo install improvise
Prebuilt binaries
See the GitHub releases page for prebuilt binaries (Linux x86_64, macOS Intel and Apple Silicon).
The data model
Every cell lives at the intersection of named categories and items —
(Region=East, Measure=Revenue) — not at a grid address like B3. A model
can have up to 12 categories, each with an ordered list of items that can be
organized into collapsible groups (e.g. months grouped into quarters).
Formulas reference dimension names, not cell addresses:
Profit = Revenue - Cost
Tax = Revenue * 0.08
Margin = IF(Revenue > 0, Profit / Revenue, 0)
BigDeal = Sum(Revenue WHERE Region = "West")
The formula language supports + - * / ^, comparisons, IF, and
aggregation functions (Sum, Avg, Min, Max, Count) with optional
WHERE clauses. Formulas apply uniformly across all intersections — no
copying, no dragging, no $A$1 anchoring.
Views and axes
A view assigns each category to one of four axes: row, column,
page (slicer), or hidden. The grid layout is a pure function of
(Model, View) → GridLayout — transposing is just swapping row and column
assignments, and it happens in one keystroke (t). Page-axis categories act
as filters: [ and ] cycle through items.
Press T to enter tile mode, where each category appears as a tile in the
tile bar. Move between tiles with h/l, then press Space to cycle axes
or r/c/p to set one directly.
Records mode (R) flips to a long-format view by assigning the virtual
_Index and _Dim categories to row and column axes. Drill-down (>) on an
aggregated cell captures a snapshot; edits accumulate in a staging area and
commit atomically on exit.
File format
Models persist to a plain-text .improv format that reads like markdown and
diffs cleanly in git:
# Sales 2025
## Category: Region
- North
- South
- East [Coastal]
- West [Coastal]
> Coastal
## Category: Measure
- Revenue
- Cost
- Profit
## Formulas
- Profit = Revenue - Cost [Measure]
## Data
Region=East, Measure=Revenue = 1200
Region=East, Measure=Cost = 800
## View: Default (active)
Region: row
Measure: column
format: ,.2f
Gzip-compressed .improv.gz is also supported. Legacy JSON is auto-detected
for backward compatibility.
Import and scripting
The import wizard analyzes CSV columns and proposes each as a category (string-valued), measure (numeric), time dimension (date-like, with optional year/month/quarter extraction), or skip. Multiple CSVs merge automatically with a synthetic "File" category.
All model mutations go through a typed command registry, so the same operations that back the TUI work headless:
# single commands
improvise cmd 'add-cat Region' 'add-item Region East' -f model.improv
# script file (one command per line, # comments)
improvise script setup.txt -f model.improv
In the browser
The browser page is at https://fiddlerwoaroof.github.io/improvise/, built
and published from main by the pages-build / pages-deploy jobs in
.github/workflows/ci.yml. A deploy only happens after cargo test and the
wasm32 check have passed.
It needs no server. The page is still a thin client — it holds no model,
and every question about a cell, its text and whether it may be typed into, is
answered by a server and arrives already decided — but the server it talks to
is an in-page Web Worker hosting the whole App over the browser's private
storage (OPFS). It is seeded on first load from web/demo.improv, so a visitor
lands on a document, and edits persist across a reload without leaving the
machine. ?doc= and ?seed= name another document and the file it is first
seeded from.
To run the same page locally:
./web/build.sh # wasm-pack -> web/{pkg,worker-pkg}
python3 -m http.server 8080 --directory web
To open a document hosted by a separate process instead — which is how a
document open in the TUI is shared — start the websocket server and pass
?ws=:
cargo run -p improvise-ws-server -- examples/demo.improv --port 8787
then reload with ?ws=ws://127.0.0.1:8787. From an https origin a ws://
address is blocked as mixed content, so that pairing is most easily done with
the local page above.
Caveat worth stating plainly: no line of the browser JavaScript has ever
run. There is no browser and no headless DOM in the development environment,
so web/app.js, web/worker.js and the OPFS backend are compile-checked,
syntax-checked and read, never executed.
What's interesting about the architecture
All user actions flow through a two-phase command/effect pipeline. Commands
are pure functions: they receive an immutable CmdContext (model, layout,
cursor position, mode) and return a list of effects. Effects are the only
things that mutate app state, and each one is a small, debuggable struct.
This means commands are testable without a terminal, effects can be logged or
replayed, and the 40+ commands and 50+ effect types are all polymorphic —
dispatched through trait objects and a registry, not a central match block.
The keybinding system gives each of the 14 modes its own keymap, with
Emacs-style prefix keys for multi-stroke sequences.
Expectations
improvise is a personal project I built for my own use. I'm sharing it because other people might find it useful, but I can't promise active maintenance or feature development. Issues and PRs are welcome but may not get a fast response. If you want to build on it, fork away.
License
Apache-2.0