Commit Graph

52 Commits

Author SHA1 Message Date
Claude (numtower) 4d04b61b69 feat(numtower): Num, contagion, exact comparison, number language and sums (improvise-59nw.1)
WIP: the crate compiles and its unit tests pass; property and differential tests follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-10-08 21:53:37 -07:00
Claude (agent-adr007) 7ddc50ac62 Merge commit '62066e1e1c972218b099a36f3f10440eb93d430e' into adr007/plan
# Conflicts:
#	context/code/core.md
#	context/test-evidence/core.md
2026-09-25 10:05:02 -07:00
Edward Langley b155ce9f0e chore: update nix+cargo deps 2026-09-25 00:14:45 -07:00
Claude (agent-adr007) e56148fa64 feat(core): slugify and the gensym id allocator for ADR 007 (improvise-cq1q.2)
model::id adds slugify, which case folds, turns each run of spaces into one
`_`, strips everything outside a-z, 0-9, `-` and `_`, then drops leading
underscores. It also adds IdAllocator, which keeps one increment-only counter
per document plus every id seen this session; claim admits hand-written ids
verbatim. Full case folding comes from the unicase crate. Nothing calls it yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-24 22:02:32 -07:00
Edward Langley 6d5b4e328a improvise-km71: serde back in improvise-formula for FormulaId alone
tt22 removed the formula crate's serde dependency with the legacy JSON loader,
keeping serde only where a type crosses the wire. FormulaId crosses the wire, so
it is the third such type. No derive goes back on the formula AST.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-23 08:50:53 -07:00
Edward Langley ec548862ac Cargo.lock: improvise-formula no longer depends on serde 2026-09-23 07:50:30 -07:00
Edward Langley 487f0b6fef improvise-up8g: a headless run's extent is fixed, not read off the terminal
`run_headless_commands` read `crossterm::terminal::size()`, and the page size
decides where `page-scroll` lands, so one script wrote a different record in a
different window — and CI (no tty, 80x24 default) disagreed with a shell. Pin
it to 80x24 in `HEADLESS_EXTENT`. The test drives two sizes through LINES/
COLUMNS with the child detached by `setsid`, and asserts the same record.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-09-22 21:18:25 -07:00
Edward Langley 98bdad02dd chore: update cargo deps 2026-09-22 20:16:46 -07:00
Edward Langley 12714c7527 feat(browser): a boot failure is a state with a reason, not silence
improvise-1wft and the five beads it blocked were one failure: nothing
carried a boot failure's REASON from the worker to the page. The channel
is one rule -- a protocol frame crosses the port as a STRING, a shell
notice as an OBJECT -- so it needs no protocol change and the
discriminator is total.

A boot failure is a STATE, not a rejected promise. A document another
context holds the exclusive handle to now OPENS read-only: measured first
(harness probe, Chromium + Firefox), getFile() reads the bytes while the
handle is held, with the second-handle refusal asserted so the probe
cannot pass vacuously.

- improvise-1wft  the channel: Notice/NoticeKind, boot_notice, take_notice
- improvise-i0v9  read-only boot over vfs::ReadOnly + OpfsFs::read_unattached
- improvise-86nw  start() rejecting for any reason now posts BootFailed
- improvise-9cma  boot::document_name -- and a/b/c.improv was ACCEPTED as
                  c.improv, not rejected, which is worse than the report
- improvise-iobc  vfs::Origin: a seed error is a notice only when the
                  document really had to be created without it
- improvise-vep8  the watchdog arms BEFORE transport.ready; its message
                  stopped claiming the connection was accepted
- improvise-7gip  DedicatedWorkerGlobalScope (needs a web-sys feature)
- improvise-zre1  a persist that fails raises a notice; the trigger is the
                  dirty flag surviving, observed rather than predicted
- improvise-7gf8  the render breaker trips once instead of retrying forever
- improvise-ociy  non-OPEN sends are loud; a viewport announcement is not
- improvise-y01k  the three ws-server drops are logged
- improvise-r4nf  serde failures throw instead of reading as "nothing to send"

Every notice routes through shell.fail, so gotcha 48's disarm half is not
re-broken. Two comments in shell.spec.js stated the opposite of the new
arming order and were corrected rather than left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
2026-09-21 14:51:34 -07:00
Edward Langley 22360d46e5 Merge main into wave/a-grid-frame
Two conflicts, both expected.

**`invalidate_relabelled` (improvise-hkt6, WU-K).** Both sides kept: the
signature and the item-tuple comparison are this branch's — `old` is an
`AxisFrame` now, and a group header appearing or collapsing must not blank
cells it did not rename — and the totals blanking and its whole argument are
main's. `grid_updates` still diffs the totals against the *framed* mirror,
which is what makes a renamed column whose total did not change send one;
`a_relabelled_column_drops_its_total`,
`a_relabelled_row_leaves_the_totals_alone` and
`a_renamed_column_whose_total_did_not_change_still_gets_its_total_back` all
pass against the new frame.

**`lib.rs`'s composition test.** Main's deletion taken: 18189c1 moved
`client_showing`, `published_basis` and
`a_client_composes_a_write_from_the_id_it_was_given` into `socket.rs` when
it consolidated `set_cell_at` and `typed`.

Main's new `GridUpdate::RowLabels`/`ColLabels` call sites are converted, and
five doc comments naming the retired spellings with them — a fact stated in
several places is a grep, not an edit.

1327 passing, 0 failing. clippy/fmt clean, wasm gate clean, both bundles
build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 00:10:15 -07:00
Edward Langley e46429ecfc feat(fit): fit-measure — a fitted tree as an ordinary improvise formula
improvise-szrt / improvise-vx3. `improvise-gp` was a bridge with nothing
crossing it; this is the crossing.

`improvise_gp::fit::fit_measure` is the sequence between the two halves that
already existed: `dataset` (a View's records as a feature matrix) and
`fitted_formula` (a champion as formula text the document can resolve). It
lives in `improvise-gp` rather than in the command because none of it is
terminal-specific — a browser fitting the same measure must run exactly these
steps in exactly this order or it gets a different tree from the same seed.

`command/cmd/fit.rs` is the user-facing surface: `fit-measure <target>
[<measure-name>] [<seed>]`, returning `SetStatus` + `MarkDirty` + `AddFormula`.
After it there is nothing GP-specific in the document — the rule is a
`## Formulas` line, recomputed by the fixed-point evaluator, editable in the
formula panel, drillable, saved with no grammar change.

**Where the fit runs, and why.** In `execute`, on the command side. A fit is
decision logic — a pure function of (Model, View, target, seed, FitConfig)
that mutates nothing — so it belongs where decision logic belongs; putting an
expensive stochastic search inside `Effect::apply` would mix it into the one
place the app mutates. That buys purity, not asynchrony, so vx3's AC 4 is met
on its "bounded" arm: `FitRequest::default` is a smaller search than the CLI's,
**measured** at 6ms / 12ms / 105ms over 12 / 200 / 2,000 records in release.

**Three things the sequence had to get right**, each of which fails quietly:

* The fit is standardized and the formula must not be. Written down untouched,
  a champion predicts standard deviations — a well-formed formula evaluating to
  ±5 where the user's data is in the thousands, and nothing downstream can see
  it. `denormalize` folds the affine map back into the leaves first.
* The rule is stored as a *new* measure (`<target> (fit)`), never as the target
  itself: gotcha 7 makes a formula target and an explicit item one item, so the
  cells the fit was learned from would stop being visible.
* `Quality` is an enum. `cv_regress` pools nothing below two folds and scoring
  an empty pool gives R² exactly 0 — "nobody checked" reading as "this rule is
  worth nothing", which is the loudest thing a fit can say.

**A bug this found, fixed with its own failing test first.** `_Measure` is a
filtering category, and `Workbook::new`'s default view puts it on `Axis::Page`
— so `dataset::filter_coords` pinned it to one measure and the feature matrix
collapsed. On a document whose target is `_Measure`'s first item every feature
column came back blank; otherwise the table was empty and the error blamed the
data ("no visible record has a Revenue value"). `_Measure` is the coordinate
the columns are spread across, so paging it is not "which records am I looking
at", it is "throw away every column". Every existing test in that module built
its view with `View::new` + `on_category_added`, under which an underscore
category lands on `Axis::None` — the fixture was building the one world the
code worked in. The two new regression tests compare row *contents*, because
asserting the header and the row count passes with the bug present.

Evidence, and the faults injected to earn it. Dropping the `denormalize` call
fails `the_fitted_formula_predicts_in_the_documents_own_units` and nothing
else. Both `_Measure`-on-Page tests were watched failing before the one-line
fix. Every new test asserts against a real document — never a helper that
fabricates a `_Measure` item per feature name, which is what let improvise-8xhy
hide. 1216 passing, up 27 from 1189; suite still well under 2s.

Also fixes the refusal message, which a `rustfmt` pass over its multi-line
literal had left with runs of ten spaces mid-sentence — invisible in the source
and to every test that only asked whether a substring was present, and visible
to precisely the user it exists for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 22:45:23 -07:00
Edward Langley 301a74821a feat(protocol): the grid's frame on the wire, as AxisFrame
`GridCache::row_labels`/`col_labels` were `Vec<String>` built by
`GridLayout::row_label`, which slash-joins a multi-level tuple — the exact
rendering `two_row_categories_produce_cross_product_labels` asserts must not
appear. So the protocol carried a form of the labels the TUI itself
considered wrong, and `improvise-wasm-client/src/render.rs` drew precisely
that form: the browser showed `Food/Alice` where the TUI showed two header
columns, with no groups and no collapse indicators (gotcha 44).

Both fields become one `AxisFrame` per axis, carrying the five facts gotcha
44 enumerates: the categories (hence the header levels), the label *tuples*
and the inline group headers, the group a position sits in, and each group's
collapse state. `record_col` stays off — session-local, keys pending drill
edits, correct to keep off.

It is one type and not a `row_levels` beside `row_labels`, because two
answers to "what are the labels" is the disease (improvise-edp). The rules
that used to be written twice are methods on it and are now called from both
front ends: `shows_level` (repeat suppression, improvise-jjx's fix),
`group_of`, `body_from`, and `header_widths` — the row-header width rule,
which existed twice already, inline in `render_grid` and again in
`viewport::compute_row_header_width` (improvise-1125).

`GridWidget` loses its `&View`: nothing in it reads the view any more.

Evidence. `the_whole_frame_is_unchanged` — the exact-buffer characterization
over a grid with row groups, column groups, a page axis, multi-level headers
and a total row — passes byte for byte, so the TUI is unchanged. The new
`tests/grid_frame_agreement.rs` drives the widget into a ratatui buffer and
the browser client through a `Snapshot` of the *same* cache, and compares
what each draws; re-introducing the slash-join on either side was injected
and watched to fail before the test was kept.

`tests/web_demo_model.rs::the_web_demo_has_one_category_per_visible_axis` is
deleted: it guarded the gap this closes, and a consolidation that keeps its
compatibility test is unfinished.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 22:43:47 -07:00
Edward Langley bdce1636bb feat(worker-server): the whole App in a Web Worker, over postMessage + OPFS
improvise-9x6: the server half of the standalone static-web deployment
(epic improvise-tm6). `improvise-worker-server` hosts the authoritative
`App`, the command pipeline, the projection layer and VFS persistence behind
`self.onmessage`/`self.postMessage`.

It is small because everything it needs already existed: the frames are
`improvise_protocol::frame`'s (improvise-xks6), the session loop is
`Host` (previous commit), the basis check is `Sessions::apply`/`Checked`
(improvise-ondj), and persistence is `Storage` + `OpfsFs` (improvise-6mq,
i34). What is genuinely new is three things.

`vfs.rs` — how a document gets INTO a browser's empty filesystem. Loading is
unchanged: it is `persistence::load_from` on a `Storage`, the same call the
native binary makes. Seeding is the new problem, and it is a
`Storage::write`, so a fetched demo, an upload and a saved document take the
same parse path and a seed never clobbers an edit. `list` is the document
picker a browser has no OS file manager for.

`worker.rs` — the loop: one port, one client, the `Viewport` handshake
(dropped rather than closed, since there is no connection to close), and the
filter from `Addressed`. Every decision is here and every test points at it.

`bindings.rs` — `cfg(wasm32)`, decision-free, and the only part no test here
can reach: it awaits `OpfsFs::open`/`attach`/`refresh`/`commit_deletes` and
forwards strings.

Two gaps found and reported rather than papered over. No client on any
transport can ask its host to save — `effect::Save` is `ViewOnly`, so the
wire vocabulary derived from the model effects cannot name it — and a wire
`ModelCommand::SetCell` never sets `model_state.dirty`, because dirtiness is
raised by a separate `MarkDirty` effect the key path emits. So the worker
persists on its own behalf, keyed off the command variant rather than off a
flag that is never set; `touches_document` carries the argument.

`ClientFrame::Key` is still answered with nothing. A worker CAN call
`reduce_key` — it holds the `App` — but nothing in the projection stream
carries the host's own `ViewState`, so a half-wired arm diverges rather than
under-delivers. `a_key_resolved_on_the_host_moves_state_no_client_can_hear`
demonstrates it against a real `App` and `Sessions` (improvise-bpq2).

22 tests, all host-side: 16 in the crate and 6 in `tests/post_message.rs`,
which drives the worker with nothing but the JSON a `MessagePort` carries.
Bundle: 1,334,892 bytes raw, 496,965 gzipped — 485 KB against a 3 MB budget.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 19:08:19 -07:00
Edward Langley f07a4c2bde chore: one last round of updates
CI / Test and lint (host) (push) Has been cancelled
CI / wasm32 check (browser bundle crates) (push) Has been cancelled
CI / Knowledge base consistency (push) Has been cancelled
2026-09-19 17:52:43 -07:00
Edward Langley 4b643a6940 Merge branch 'review' 2026-09-19 17:52:00 -07:00
Edward Langley 74827f0415 chore: update Cargo.lock 2026-09-19 17:49:19 -07:00
Edward Langley c72832f542 feat(wasm-client,web): the DOM renderer for the browser grid (improvise-cr3)
The last blocker of improvise-1ey. Mostly assembly: `Client` already had the
fold, the key classifier and `set_cell`, `CellDisplay::id` already said what a
cell is (ADR 002), and `flatten_cat_tree` already owned the category panel's
flattening. What was missing was the three joints between them.

`render.rs` is the one that matters. The DOM half is JavaScript and JavaScript
here is untested, so every question with a *silent* wrong answer is answered
in Rust under test and crosses as decided data: `editable` is a fact about the
`CellId` variant (Record only — an Aggregate is a drill target and `Drill` is
not on the wire yet, and `None` is "no projected cell", not "aggregate"), the
selection is placed by subtracting the scroll offset because `GridCache` is
viewport-indexed while the cursor is not, and the category panel is
`flatten_cat_tree` rather than a second walk of the cache.

`socket.rs` frames what `improvise-ws-server` actually speaks, which is not
what `wire.rs` speaks: it has `Viewport` (the handshake — a snapshot built at
extent zero is an empty grid), it batches `Commands`, and it has no `Key`
variant. The two enums are duplicated because a native crate cannot be a wasm
dependency; `improvise-ws-server` is a dev-dependency so the compatibility is
checked through its own types, and a companion test proves the check is
discriminating on the variant names that would drift.

`web/` is the shell, and it is thin on purpose: four string calls in and out,
no protocol knowledge, no framework.

Gates: 1047 passing (was 1031), clippy and fmt clean, and
`cargo check -p improvise-wasm-client --target wasm32-unknown-unknown` clean.
Verified end to end headlessly — the wasm-pack artifact driven from node over
a real websocket to a real `improvise-ws-server`: 1,200 read, "98765" typed,
"98,765" projected back. The DOM painting itself is unverified; there is no
headless browser here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 17:34:39 -07:00
Edward Langley 2e7e757538 feat(core): RecordHash/AggregateHash and the record index in DataStore
ADR 002's surrogate, and the reverse index a server resolves it through.

The hash is specified rather than inherited: SipHash-2-4 under a constant
key over an explicit length-prefixed byte serialization of the sorted
(category, item) pairs. Not DefaultHasher/RandomState (randomly seeded per
process), not #[derive(Hash)] (no stability guarantee). 128 bits, argued in
the doc comment: a collision is a silent write to the wrong record and no
later check catches it, while the cost of the wider hash is a bounded 8
bytes per visible cell. The wire form is hex-as-a-string, because a JSON
number above 2^53 comes back changed in a JavaScript host.

RecordHash::of normalizes, so a cell key and its record hash alike -- the
_Measure-drop is a property of the constructor, not a rule callers follow.

The index unifies ADR 002's resolution with ADR 001's record_versions:
RecordHash -> RecordEntry { record, cells, version }, one lookup answering
both "which record do you mean?" and "is your basis stale?". A zero cell
count is a tombstone rather than a removal, so a delete-and-recreate cannot
hand a stale basis its own version number back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 17:09:46 -07:00
Edward Langley 114680744e Merge: improvise-ws-server — the websocket host (improvise-q08)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH

# Conflicts:
#	Cargo.lock
#	Cargo.toml
2026-09-19 16:45:25 -07:00
Edward Langley df746173bf feat(ws-server): websocket host for one document (improvise-q08)
A new crate `improvise-ws-server`: a testable session loop plus a thin
tokio/tungstenite shell around it.

The ticket predates the session machinery and describes building it by
hand; most of what it asks for now exists, so what this adds is the
transport and the loop (doctrine(check_the_ticket_premise)):

- `Host` is one authoritative `App`, one `Sessions`, and every client
  attached to them — the unit of authority is the *document*, not the
  connection (the ticket's premise) and not the server. A multi-document
  registry is a map of `Host`s and is deliberately not built yet; the
  seam is in `transport`, where `serve` takes the document it hosts.
- `handle(id, msg) -> Vec<Addressed>` makes every decision and is tested
  without a socket. `Applied::Rebased` sends every client a fresh
  `Snapshot`; `Applied::Diff.broadcast` is forwarded to the *other*
  clients and never back to the asker.
- The wire adds exactly one thing to the shared vocabulary: the extent a
  client can display, which no `Command` carries by construction and
  which doubles as the handshake, since the bootstrap `Snapshot` is
  built at it.

No auth, no TLS, loopback only: a stated MVP limit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 16:44:32 -07:00
Edward Langley 30c4bb6629 feat(wasm-client): the browser-side fold and key classifier (improvise-gsw)
A new crate `improvise-wasm-client`: a `ViewState` plus one widget cache
each, moved by folding the same `Command` stream the host folds, with a
wasm-bindgen surface over JSON.

The ticket's premise does not survive contact with the architecture
(`doctrine(check_the_ticket_premise)`). It asks `on_key_event` to
"resolve via the keymap and dispatch through reduce_view", but a binding
names an *intent* resolved against a `CmdContext` — a `Model`, a
`GridLayout`, an `App` — while a `Command` is the *resolved mutation*
(gotcha 38, improvise-gxi). A thin client holds none of that, so keys
travel upstream unresolved and `improvise-command` is a dev-dependency
rather than a dependency: the keymap could not be used here even if the
bundle could afford `App`, `Workbook`, persistence and the wizard.

The one exception is minibuffer text entry, whose target buffer is
`AppMode::minibuffer().buffer_key` — the same field the host's
`TextEntry` builder reads (gotcha 11), so the local rule is derived from
the same fact rather than copied. `tests/lockstep.rs` drives the
authoritative host and the client over the same keypress in all seven
text-entry modes and asserts the two `ViewState`s agree.

Rejected for local resolution, with reasons in `keys.rs`: cursor
movement (`GridCache` is viewport-indexed, so a client cannot clamp),
panel cursors (the flattening depends on `expanded_cats`), anything
modified (`AnyChar` matches under any modifier on the host), and
CommandMode's Backspace (the one `TextEntry::rebind`, which leaves the
mode).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 16:35:04 -07:00
Edward Langley 55ee619ae6 Merge: gp-tree formula emission, library preprocessing, seeded RNG (improvise-izk, 0yr, lzn)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 13:38:02 -07:00
Edward Langley 2a0b937447 feat(improvise-gp): emit a fitted gp-tree Tree as improvise formula text
improvise-izk. New crate `improvise-gp` — the seam between gp-tree and the
formula language. It cannot live in gp-tree (that inverts the dependency) and
it cannot live in improvise-formula (which must not know GP exists).

`formula_text(tree, spec, target)` turns Split{feat,thresh,lo,hi} into
`IF(<name> < <thresh>, <lo>, <hi>)` and a Leaf into its number;
`fitted_formula` hands back the parsed Formula, so the parser is the emitter's
own contract check.

Two things the emitter owes its caller:

* Names round-trip. Feature names come from user data — spaces, operators,
  pipes, keyword collisions, the one-hot encoder's own `col=value` labels —
  so every name goes through CL-style pipe quoting, and a keyword is quoted
  even though it would parse bare.
* Thresholds route identically. A threshold is emitted as the f32's *exact*
  decimal, not its shortest round-trip form. `0.7f32` is 0.699999988…, and
  "0.7" parses as an f64 strictly greater than it, so a row whose feature
  value equals the threshold would fall on the other side of the split in
  formula evaluation than it does in `gp_tree::eval`. There is a test for
  that boundary, and it fails against the naive rendering.

Tests: parse round-trip for leaf/nested/awkward-name/awkward-target cases, the
threshold-boundary regression, and the acceptance property — over random trees
and random feature vectors, `Model::evaluate` of the emitted formula equals
`gp_tree::eval`, exactly (f32→f64 is lossless, so no tolerance is needed).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 13:34:01 -07:00
Edward Langley 7c5a3f6332 chore: drop unused root deps (pest/pest_derive/dirs), ignore tags+graph artifacts
improvise-v9w: the root `improvise` crate's [dependencies] listed pest,
pest_derive and dirs, none of which appear anywhere under src/. pest and
pest_derive remain genuine dependencies of improvise-formula and
improvise-io (persistence/improv.pest); examples/gen-grammar.rs reaches
pest_meta through [dev-dependencies], which examples do get, so the
examples still build. Verified with cargo build --workspace, cargo build
--examples, cargo test --workspace and the wasm check. Cargo.lock loses
only dirs and its transitive deps.

improvise-wg1: add the GNU Global tag files (GTAGS/GRTAGS/GPATH/GSYMS) and
the root graphviz artifacts (/graph.dot, /graph.html, /graph.svg) to
.gitignore. Named exactly rather than globbed so no checked-in asset is
swallowed; `git ls-files | git check-ignore --stdin` prints nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EoW5BB87ZZdm96SvBTWawH
2026-09-19 13:31:38 -07:00
Ed L f18dcce821 chore: update locks 2026-09-03 17:56:39 -07:00
Edward Langley d25f3187d3 feat(protocol): extract improvise-protocol crate
The crate the browser epic hangs off: Command (the wire format),
ViewState, RenderCache/CellDisplay/Snapshot, and reduce_view. It depends
on improvise-core and serde and nothing else -- no IO, no command layer,
no terminal -- and improvise-command now depends on it.

ViewState, ViewCursor, ViewFrame, AppMode, Panel, MinibufferConfig,
MinibufColor, HELP_PAGE_COUNT and the key types move here. Everything
that moved is re-exported from ui::app and command::key, so no existing
path changed. The three tickets before this one are what made the move
mechanical: h0b evicted the unportable fields, a2x added the derives,
8uz took the cursor out of the shared View.

Command's client->server half is not invented. It is exactly the set of
EffectKind::ViewOnly effects whose target is the portable slice -- h0b
had already sorted the portable state from the host-local, so the
enumeration was done. Effects touching the wizard, staged drill edits,
storage or the layout cache are absent by construction. The other half
carries projections: already-formatted display values and their labels.

One thing had to be added rather than moved. reduce_view runs on a client
that holds no Workbook, and ViewState::cursors is keyed by view name, so
ViewState needs its own active_view. That makes two copies of "which view
is active"; App::switch_to_view and App::install_workbook are the only
writers, and the_two_active_view_names_never_diverge guards it. The 8uz
characterization test caught the desync the moment it appeared, because
it was calling workbook.switch_view directly.

Whether the two are *allowed* to diverge -- two clients on different
views -- is improvise-g0f, deliberately left open. It changes the shape
of improvise-cqq, and nothing here forecloses either answer.

reduce_view has 12 tests including a determinism check: folding the same
stream twice gives the same state, which is what stops the server's
mirror drifting from the client it mirrors. Every Command round-trips
through JSON. The crate is in the CI wasm job.

Closes improvise-cqi

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V69PGQfMJuGRmtSWpfVAqn
2026-09-01 02:03:45 -07:00
Edward Langley 07a29aa33b feat(io): OPFS-backed Storage for the browser, plus list/delete
Storage gains list and delete: a browser build has no OS file manager
standing behind it, so enumerating and removing documents has to be the
backend's job. Both are implemented for PhysicalFs and MemoryFs, and a
shared storage_contract test runs against both so they cannot drift.

OpfsFs (wasm32 only) implements the same *synchronous* trait. That works
because createSyncAccessHandle() yields a handle whose read, write,
truncate, flush and close are genuinely blocking -- available only inside
a dedicated Web Worker, which is where improvise-9x6 puts the app. So the
trait did not have to become async.

What is still async is pulled out into four methods the worker awaits
from its message loop: open(), attach(path), refresh() and
commit_deletes(). attach must precede any read or write of a document,
because acquiring a handle is async even though using it is not.

Two consequences worth knowing. delete() drops the name from the index
and queues a removeEntry, so it is visible to read and list immediately
and lands in OPFS when commit_deletes is awaited -- the alternative was
making the whole trait async for one operation. And handles are
exclusive per file entry, so attach closes any handle it replaces and
detach exists to release the lock.

Checked before writing it: Safari 16.4+ has synchronous access handle
methods (an early spec wrongly called them async, and every current
browser implements them sync). So the IndexedDB fallback the ticket
called for is not needed, per the user's "not unless recent Safari needs
it" -- and the async question it would have forced does not arise.

Compile- and clippy-clean for wasm32; there is no browser here, so
improvise-xf1 tracks verifying it against one. storage_contract is the
sequence it must satisfy.

Closes improvise-i34

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V69PGQfMJuGRmtSWpfVAqn
2026-08-31 23:11:08 -07:00
Edward Langley 850ea584d1 feat(command): make ViewState serializable
ViewState is the wire payload of the Snapshot a newly-connected client
receives (improvise-q08), so it and its field types now derive Serialize
and Deserialize: AppMode, MinibufferConfig, MinibufColor, ViewFrame,
KeyCode, KeyModifiers. serde joins improvise-command's dependencies.

The ticket did not anticipate the actual blocker. MinibufferConfig held
`buffer_key: &'static str`, and Deserialize cannot produce a borrowed
'static value -- so every text-entry AppMode variant that carries a
MinibufferConfig failed, five "lifetime may not live long enough" errors
pointing at the variants and none naming the field.

buffer_key and TextEntry::buffer are now owned Strings. Cow<'static, str>
would keep construction allocation-free but puts container plumbing in a
public field. A BufferKey enum would close the name set, but drags the
buffers map key type, the script parser's buffer args and the
hand-written "search" buffer with it. Mode construction is per-keypress;
seven small allocations are not worth designing around.

Note this does not reopen improvise-k8h's rejection of a Buffer enum:
that was the drift argument -- each side would still name a variant --
which is about who writes the name, not what type it has. buffer_of()
still reads it off the mode. k8h doesn't forbid an enum here; scope does.

The roundtrip test populates one non-default value per field family,
including a payload-carrying mode and an armed prefix; a roundtrip of
ViewState::default() would pass while proving nothing.

Closes improvise-a2x

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V69PGQfMJuGRmtSWpfVAqn
2026-08-31 21:05:37 -07:00
Edward Langley 69e77d6823 ci: add wasm32 to the toolchain and gate the bundle crates (improvise-ywd)
The audit could not start because the Nix toolchain shipped no wasm32 std and
there is no rustup here. Adding targets = ["wasm32-unknown-unknown"] to the
flake's rustToolchain was the whole blocker: all four bundle crates —
improvise-core, improvise-formula, improvise-io, improvise-command — then
check clean on the first try with no feature flags and no source changes.

The root improvise crate does not build for wasm because crossterm does not.
That is correct: it is the terminal binary, not part of the browser bundle,
so the CI job names the four bundle crates rather than checking the workspace.

Adds .github/workflows/ci.yml with that wasm job plus a host job running fmt,
clippy -D warnings and the test suite — there was no test or lint CI before,
only cargo-dist's release workflow.

A check gate cannot catch what merely panics at runtime, and the audit's real
finding was that two of the ticket's six named hazards are real while four are
not. Instant::now() panics on wasm yet compiles, so improvise-command now uses
web-time, which re-exports std::time off-wasm. std::fs likewise compiles and
then fails in a browser; StartImportWizard and LoadModel still bypass the
Storage trait, filed as improvise-fuj because the fix needs a decision on how
a wasm build injects its backend.

Verified by compilation on both targets and by the native suite. Nothing here
has been run on a wasm host.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V69PGQfMJuGRmtSWpfVAqn
2026-08-30 09:51:41 -07:00
Edward Langley 24ac1d6c11 chore: update cargo + flake locks 2026-08-13 17:34:48 -07:00
Edward Langley 0f03982be8 chore: arrange deps 2026-08-01 16:23:15 -07:00
Edward Langley 43404173b7 feat: add gp-tree 2026-08-01 10:12:26 -07:00
Edward Langley aa675f97c1 chore: update cargo+flake locks 2026-07-30 22:47:37 -07:00
Edward Langley a85e6ca28c refactor(workspace): extract improvise-command crate (improvise-3mm)
Command layer + semantic App state move below the TUI: command/ (cmd,
keymap, parse, key), ui/{app,effect,cat_tree}.rs, and the pure viewport
helpers now live in crates/improvise-command, which depends on core +
formula + io and provably not on ratatui/crossterm (cargo tree clean).
The root crate remains the TUI binary and re-exports improvise_command
under the old crate::command / crate::ui::{app,effect,cat_tree} paths,
so widgets, draw.rs, and main.rs are unchanged. App moves whole (per
the improvise-ltq design note) rather than splitting an AppState out —
its fields are all command-safe after the key/color decoupling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 12:11:43 -07:00
Edward Langley ecb524c947 chore(deps): update dependencies
Update Cargo.lock with new versions and additional dependencies.

This includes updates to bitflags, chrono, and hashbrown, along with the
addition of new packages such as approx, critical-section, and the futures
suite.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
2026-06-09 20:48:44 -07:00
Ed L 3324ceef69 chore: don't use IFD for Cargo.nix 2026-04-26 10:28:37 -07:00
dependabot[bot] 699d4d58dc chore(deps): bump rand from 0.8.5 to 0.8.6
Bumps [rand](https://github.com/rust-random/rand) from 0.8.5 to 0.8.6.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/0.8.6/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.8.5...0.8.6)

---
updated-dependencies:
- dependency-name: rand
  dependency-version: 0.8.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-22 22:17:23 +00:00
Edward Langley bd17aed169 refactor(io): scaffold empty improvise-io sub-crate
Lay groundwork for Phase 3 of the workspace split (improvise-8zh): a new
`crates/improvise-io/` sub-crate that will house the persistence and
import layers once the files move in the next commit.

Only scaffolding here:
- New `crates/improvise-io/Cargo.toml` declares deps on improvise-core,
  improvise-formula, and the external crates persistence+import already
  use: anyhow, chrono, csv, flate2, indexmap, pest, pest_derive, serde,
  serde_json. Dev-deps: pest_meta, proptest, tempfile.
- New `crates/improvise-io/src/lib.rs` re-exports the core modules under
  their conventional names (`format`, `model`, `view`, `workbook`,
  `formula`) so in the next commit the moved code's `crate::model::*`,
  `crate::view::*`, `crate::workbook::*`, `crate::format::*`, and
  `crate::formula::*` paths resolve unchanged.
- Root `Cargo.toml` adds the new crate to workspace members and the main
  crate's `[dependencies]`, ready to receive the move.

No source files change yet; cargo check --workspace still compiles as
before.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 23:05:17 -07:00
Edward Langley 0d75c7bd0b refactor(core): scaffold empty improvise-core sub-crate
Lay the groundwork for Phase B of improvise-36h: a new
`crates/improvise-core/` sub-crate that will house the pure-data core
(model, view, workbook, format) once the files move in the next commit.

Only scaffolding here:
- New `crates/improvise-core/Cargo.toml` mirroring improvise-formula's
  structure; declares deps on improvise-formula, anyhow, indexmap, serde.
- New `crates/improvise-core/src/lib.rs` with the single re-export
  `pub use improvise_formula as formula;` so that in the next commit the
  moved code's `crate::formula::*` paths resolve unchanged.
- Root `Cargo.toml` adds the new crate to the workspace members and the
  main crate's `[dependencies]`, ready to receive the move.

No source files change yet; cargo check --workspace still compiles as
before.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:21:58 -07:00
Edward Langley 5fbc73269f refactor(formula): trust grammar invariants in parser
Refactor the formula parser to assume grammar invariants, replacing
Result-based error handling in tree walkers with infallible functions and
.expect(GRAMMAR_INVARIANT) calls.

This simplification is based on the guarantee that the Pest grammar already
validates the input structure. To verify these invariants and ensure
correctness, extensive new unit tests and property-based tests using
proptest have been added.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (gemma-4-31B-it-UD-Q4_K_XL.gguf)
2026-04-15 04:32:14 -07:00
Edward Langley 3f69f88709 refactor!(formula): migrate parser to use pest
Replace the manual tokenizer and recursive descent parser with a PEG
grammar using the pest library.

This migration involves introducing a formal grammar in formula.pest and
updating the parser implementation to utilize the generated Pest parser
with a tree-walking approach to construct the AST.

The change introduces a stricter requirement for identifiers: multi-word
identifiers must now be enclosed in pipe quotes (e.g., |Total Revenue|) and
are no longer accepted as bare words.

Tests have been updated to reflect the new parsing logic, remove
tokenizer-specific tests, and verify the new pipe-quoting and escape
semantics.

BREAKING CHANGE: Multi-word identifiers now require pipe-quoting (e.g. |Total Revenue|) and
are no longer accepted as bare words.
Co-Authored-By: fiddlerwoaroof/git-smart-commit (gemma-4-31B-it-UD-Q4_K_XL.gguf)
2026-04-15 04:04:57 -07:00
Edward Langley f02d905aac refactor(formula): extract formula parser into separate crate
Extract the formula AST and parser into a dedicated `improvise-formula`
crate and convert the project into a Cargo workspace.

The root crate now re-exports `improvise-formula` as `crate::formula` to
maintain backward compatibility for internal callers. The repository map is
updated to reflect the new crate structure.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (gemma-4-31B-it-UD-Q4_K_XL.gguf)
2026-04-14 01:49:35 -07:00
Edward Langley 04058a48f9 chore: v0.1.0-rc2
Release / plan (push) Has been cancelled
Release / build-local-artifacts (${{ join(matrix.targets, ', ') }}) (push) Has been cancelled
Release / build-global-artifacts (push) Has been cancelled
Release / host (push) Has been cancelled
Release / announce (push) Has been cancelled
2026-04-11 00:47:36 -07:00
Edward Langley 5e90fbe0f7 chore(updates): last cargo updates 2026-04-11 00:45:41 -07:00
Edward Langley 1350e875c5 chore(update): dep updates for cargo 2026-04-11 00:42:20 -07:00
Edward Langley ee6739158e chore: update cargo files more
Release / plan (push) Has been cancelled
Release / build-local-artifacts (${{ join(matrix.targets, ', ') }}) (push) Has been cancelled
Release / build-global-artifacts (push) Has been cancelled
Release / host (push) Has been cancelled
Release / announce (push) Has been cancelled
2026-04-11 00:31:43 -07:00
Edward Langley e6c93a24d9 chore: some dep updates 2026-04-11 00:07:59 -07:00
Edward Langley cb24cce1f0 feat: use enum_dispatch for command dispatch and add Open command
Replace manual dynamic dispatch using Box <dyn Runnable> with enum_dispatch
for improved performance and cleaner code.

Add a new Open command to allow opening the TUI as the default behavior
when no subcommand is provided.

Co-Authored-By: fiddlerwoaroof/git-smart-commit (unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL)
2026-04-11 00:06:48 -07:00
Edward Langley 6647be30fa refactor: switch to clap with subcommands for CLI parsing
Replace hand-rolled arg parser with clap derive. Restructure as
subcommands: import, cmd, script. Import subcommand supports
--category, --measure, --time, --skip, --extract, --axis, --formula,
--name, --no-wizard, and --output flags for configurable imports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 21:43:16 -07:00
Edward Langley 23e26f0e06 Add CSV import functionality
- Use csv crate for robust CSV parsing (handles quoted fields, empty values, \r\n)
- Extend --import command to auto-detect format by file extension (.csv or .json)
- Reuse existing ImportPipeline and analyzer for field type detection
- Categories detected automatically (string fields), measures for numeric fields
- Updated help text and welcome screen to mention CSV support

All 201 tests pass.
2026-04-01 01:37:04 -07:00