From 7eae2a9352030d40c0a13b0dbb28768b9cb78b18 Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sat, 19 Sep 2026 16:46:23 -0700 Subject: [PATCH] chore(beads): q08 closed; composability list recorded --- .beads/issues.jsonl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 7a500bd4..4db5fd03 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -1,3 +1,4 @@ +{"_type":"issue","id":"improvise-bpq2","title":"Client-side key resolution: what must the projections carry?","description":"USER RULING (2026-09-19): keys are converted into commands IN THE BROWSER; the keymap is a client-only concept the server must not know. Follows from the standing principle -- the model is shared, the UI is not, and a keymap is UI. It also buys something real: two clients may have DIFFERENT bindings over the same document, which is impossible if the server owns the keymap.\n\nThis inverts improvise-gsw conclusion (which had keys travelling upstream unresolved) and closes the keypress-door ticket. The work is now to make client-side resolution POSSIBLE.\n\nTHE PROBLEM TO SOLVE. A binding names an INTENT resolved against a CmdContext -- Model, GridLayout, cursor, yanked cell, expanded cats, mode. A Command is the RESOLVED MUTATION, e.g. SetSelected { row, col }. A thin client holds ViewState plus the widget caches. So for each binding ask: can a client holding only that state compose the Command? Where the answer is no, the fix is to ENRICH THE PROJECTION, not to move resolution to the server.\n\nKNOWN GAPS, from the gsw agent own analysis of what it could not resolve:\n * CURSOR MOVEMENT -- GridCache is indexed by position WITHIN THE VIEWPORT, so a client cannot distinguish \"last visible row\" from \"last row\" and cannot clamp. It needs the total row/col count, or an explicit \"can I move further\" fact.\n * PANEL CURSOR MOVEMENT -- move-panel-cursor walks a flattening that depends on expanded_cats. Either the cache carries the flattened list, or the client re-derives it -- and re-deriving is how two ends drift.\n * MODIFIED CHARACTERS -- KeyPattern::AnyChar matches a Char under ANY modifiers on the host. A client mirroring that is fine now that the client owns the keymap outright; the old constraint (local resolution may only ever be a strict subset) DISSOLVES once there is no host keymap to be a subset of.\n\nMETHOD: enumerate the bindings, not the commands -- the keymap is the thing being made client-side, so walk it. doctrine(binding_dump_diff) already says most bindings have no direct test, so a green suite is not evidence here either.\n\nALSO IN SCOPE: move ClientMessage/ServerMessage into improvise-protocol. The vocabulary is there and the framing that says which a JSON blob is, is not; the ws-server will want the same envelopes and two crates should not define them.\n\nBlocks improvise-1ey.","status":"open","priority":1,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-09-19T23:42:49Z","created_by":"Edward Langley","updated_at":"2026-09-19T23:42:49Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"improvise-c4er","title":"[bug] There is no server door for a keypress; the asker gets no commands back","description":"Found by improvise-gsw (2026-09-19) and it is THE seam the browser MVP depends on. Blocks improvise-1ey.\n\nA browser client cannot resolve keys locally -- a binding names an INTENT resolved against a CmdContext while a Command is the RESOLVED MUTATION (see the improvise-gxi close reason and gotcha 38). So the client sends the raw keypress upstream as ClientMessage::Key { code, modifiers }.\n\nTHE GAP: Sessions::apply(app, id, cmd) takes a Command. ui::reduce::reduce_key(app, key) exists but returns only PROJECTIONS -- not the view-slice commands the ASKER must fold into its own ViewState. With keys travelling unresolved the host owes the asker back the commands its own keypress produced, and nothing returns them.\n\nNOTE this is NOT Applied::Diff.broadcast, which is what the OTHER clients fold (gotcha 36). The asker needs its own, and there is nowhere for them to come from.\n\nSUGGESTED SHAPE (from the gsw agent): Sessions::apply_key(app, id, key) -\u003e Applied, with the askers own view-slice commands in the result. Derive it rather than inventing -- doctrine(derive_the_enum_from_the_effects). The keypress is already KeyCode + KeyModifiers, which Command::SetTransientPrefix puts on the wire today; note improvise_protocol::KeyEvent itself does NOT derive Serialize/Deserialize, only its two components do, which is worth fixing at the same time.\n\nAlso from gsw: ClientMessage/ServerMessage envelopes currently live in the client crate because the VOCABULARY is in improvise-protocol but the FRAMING that says which a JSON blob is, is not. Moving them is a one-file lift and the ws-server will want the same ones -- do it here rather than letting two crates define them.","status":"closed","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-19T23:40:37Z","created_by":"Edward Langley","updated_at":"2026-09-19T23:42:47Z","closed_at":"2026-09-19T23:42:47Z","close_reason":"WRONG PREMISE -- CLOSED AND INVERTED by a user ruling (2026-09-19): keys are converted into commands IN THE BROWSER, and the keymap is a CLIENT-ONLY concept the server must not know at all.\n\nThis ticket asked for a server door for a keypress (Sessions::apply_key). There must not be one. The reasoning follows from the standing principle -- the model is shared, the UI is not -- and A KEYMAP IS UI. If the server knows the keymap, two clients cannot have different bindings, and that is a thing this architecture should permit: a browser client with browser-native keys and a TUI with vim keys, over the same document.\n\nSo the gap this ticket named does not exist. The asker never needs its own commands handed back, because the asker COMPOSED them and already knows what it sent. ClientMessage::Key should not exist; a client sends Commands only. ui::reduce::reduce_key stays, but it belongs to the native TUI -- which is a client that happens to run in-process -- not to the server client-facing surface.\n\nTHE REAL WORK THIS DISPLACES INTO: a client must hold enough PROJECTED STATE to resolve a binding into a Command. That is a requirement on what the projections carry, not a reason to move resolution serverward. Filed separately.\n\nThe one part of this ticket that survives independently: ClientMessage/ServerMessage envelopes belong in improvise-protocol rather than the client crate -- the vocabulary is there, the framing is not. Refiled with the projection work.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"improvise-acsn","title":"[bug] A document swap over the wire leaves the client's own cursors stale; nothing tells the host a rebase is owed","description":"Wave-4 review (2026-09-19), CONFIRMED by probe. Latent -- no host wires Sessions yet -- but newly reachable from the wire.\n\nModelCommand now lets a client send LoadModel / ImportJsonHeadless / AcceptWizardModel, the WholeWorkbook commands, for the first time. The SERVER mirrors self-heal: ClientSession::project reseeds on the document-generation bump, so the server is honest. But cursors live in the CLIENT own ViewState, and that is corrected only by a Snapshot -- whose only producer, Sessions::rebase, has NO NON-TEST CALLER in the tree (grep finds one hit, a test). Applied carries only { dirty, broadcast }, neither of which tells the host a rebase is owed.\n\nProbe (two sessions, MemoryFs, Model(LoadModel) where the incoming document active view is also named Default):\n generation 0 -\u003e 1, broadcast = None\n bystander mirror cursor before project = (3, 0)\n bystander mirror cursor after project = (0, 0)\nSo the server has it at (0,0) while the bystander client keeps cursors[Default] = (3,0) from the OUTGOING document -- the gotcha-24 cursor-vs-geometry mismatch that apply_switch own doc comment says this shape exists to prevent, now arriving through another client.\n\nFIX: give Applied a way to say the document was replaced -- carry the generation, or a rebase flag -- so a host CANNOT forget. Pre-existing in kind (a keypress-driven LoadModel had the same gap), which is why the fix belongs on Applied rather than on the command.","status":"closed","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-19T22:18:02Z","created_by":"Edward Langley","updated_at":"2026-09-19T22:24:25Z","closed_at":"2026-09-19T22:24:25Z","close_reason":"Applied went from a record to an enum: Applied::Diff { dirty, broadcast } | Applied::Rebased { snapshots }. The enum IS the mechanism -- a host that wants the Dirty must match, and matching names the case where a Snapshot is owed. Changing the shape made the compiler name all five existing read sites, which is the point. Rebased carries no Dirty and no broadcast, because each Snapshot already carries a whole view slice plus the seed reproducing the new document; a diff against the old one would describe nothing that still exists. Sessions::apply_model snapshots document_generation before and after apply_command and calls rebase itself, exactly as apply_switch adopts the accepted view itself -- rebase gains its first non-test caller.\n\nTHE TRIGGER IS THE GENERATION, NOT THE VARIANT: a LoadModel on an unreadable path leaves the document alone, and reseeding on the variant would cost a client its cursors for a FAILED load. Both arms are tested per doctrine(one_arm_of_a_symmetric_rule...): a_command_that_leaves_the_document_in_place_is_not_a_rebase and a_load_that_fails_is_not_a_rebase. Injections both directions: never-rebase fails the swap test (left 3, right 0); always-rebase fails four tests including the failed-load arm.\n\nFIXTURE TRAP WORTH KNOWING, recorded in the repo-map: the first version of the test PASSED, because the existing other_document() renames the active view Default-\u003eSheet, so apply_model broadcast a SwitchView, the client folded it, and the stale cursor was read under a key nothing had written -- right by accident. A new fixture keeps the view NAME and changes only the shape, so there is nothing to hide behind.\n\nRejected: Applied { dirty, broadcast, rebase: bool } -- the host that forgets to call rebase is the host that forgets to read the flag; carrying the generation -- same problem plus it asks the host to remember the comparison; a hook on the LoadModel effect -- Sessions is unreachable from an effect, which is why the generation bump exists at all.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"improvise-oi9b","title":"[bug] 24 of 28 model_effects mappings are unguarded; a wrong mapping passes the whole suite","description":"Wave-4 review (2026-09-19), CONFIRMED by injection. Test gap, not a live bug -- the mapping is correct today.\n\nui/reduce.rs::model_effects maps each ModelCommand back to its effect. Injecting wrong-but-still-Model mappings left cargo test --workspace GREEN:\n ModelCommand::HideItem -\u003e effect::ShowItem PASSES\n ModelCommand::CreateView -\u003e effect::DeleteView PASSES\n ModelCommand::AddItem -\u003e effect::RemoveItem PASSES\n ModelCommand::ViewBack -\u003e effect::ViewForward fails (caught incidentally by the project.rs lockstep test)\n\nWhy: every_model_command_names_an_effect_that_declares_a_model_change cannot distinguish two effects of the SAME EffectKind, and a_model_command_lands_where_its_keypress_lands drives only three variants (TransposeAxes, TogglePruneEmpty, AddCategory). That leaves 24 variants whose mapping nothing checks -- and every inverse pair (Hide/Show, Add/Remove item, Create/Delete view, SetCell/ClearCell) is exactly the case the surviving tests cannot separate.\n\nFAILURE SCENARIO: a rename or re-order during a future refactor silently re-points a wire command at its INVERSE. A client sending Model(HideItem) un-hides instead, and because the Dirty is identical every other client is told a consistent lie.\n\nFIX: extend a_model_command_lands_where_its_keypress_lands to a table over every_model_command(), or -- cheaper and total -- assert the variant name equals the effect type name. The mapping is name-for-name by construction, so that assertion covers all 28 at once. This is the same class the author already fixed once, when a one-category fixture could not tell TransposeAxes from CycleAxis.","status":"closed","priority":1,"issue_type":"bug","owner":"el-github@elangley.org","created_at":"2026-09-19T22:18:01Z","created_by":"Edward Langley","updated_at":"2026-09-19T22:24:24Z","closed_at":"2026-09-19T22:24:24Z","close_reason":"Guarded by name, covering all 28 at once. every_model_command_names_the_effect_of_the_same_name asserts each variant returns exactly one effect whose TYPE NAME equals the variant name -- read off the effect derived Debug, since Effect is a trait object and type_name is out of reach through it. The mapping is name-for-name by construction, so a variant whose effect must be named something else is a variant that stopped being derived: it should fail there and be argued about.\n\nThe gap was reproduced first: AddItem-\u003eRemoveItem, HideItem-\u003eShowItem and CreateView-\u003eDeleteView applied SIMULTANEOUSLY left cargo test --workspace at 918 passed, 0 failed -- three wire messages doing the exact inverse of what they say, suite fully green. Each injection was then re-run individually against the new test and each fails with the names printed.\n\nThe name READER is itself checked (the_effect_name_reader_reads_all_three_debug_shapes), because the delimiter after the identifier differs per Debug shape -- nothing, ( or space-brace -- and its own injection was verified to fail.\n\nRejected extending the behavioural test to a table over all 28: most variants have no single keypress to compare against, so you would invent one or drive the effect directly, at which point it stops being an independent yardstick. The behavioural test keeps its three genuinely key-driven cases and its two-category pivoted() fixture.","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -100,7 +101,7 @@ {"_type":"issue","id":"improvise-1ey","title":"Browser frontend MVP: end-to-end working demo","description":"Wire everything together into a working browser demo: start the ws-server, open the HTML shell in a browser, it loads the wasm client, connects to the websocket, receives the initial snapshot, renders the grid, accepts keystrokes, and round-trips commands to the server. This is the milestone where the architecture proves itself end-to-end. Scope: open an existing .improv file server-side, view it in the browser, type a number into a cell, see the server-side value update and the browser's projection arrive with the new value.","design":"HTML shell: minimal index.html with a \u003cdiv\u003e for the grid and a \u003cscript\u003e tag bootstrapping the wasm module. JS bootstrap: loads wasm, opens websocket connection, wires on_message from the socket into wasm.on_message, wires wasm.on_key_event output into the socket send. Reconnection: on socket close, show a status indicator; on reconnect, request a fresh snapshot. No auth, no TLS, localhost only. Test by pointing at bank-info.improv or any existing test file.","acceptance_criteria":"(1) ws-server serves a file from CLI arg, accepts a websocket connection. (2) Browser loads page, wasm initializes, grid renders. (3) Arrow keys move the cursor locally (view effect) with no round-trip. (4) Typing a number + Enter in a cell round-trips to the server and the new value appears. (5) Scrolling fetches cells for the new viewport. (6) Reload recovers cleanly from current server state. (7) Video or screenshot of the demo attached to the issue.","notes":"Final milestone of the browser epic. Depends on projection layer (improvise-cqq), ws-server (improvise-q08), wasm client (wasm-client id), DOM renderer (dom-renderer id). No new logic — just glue and testing.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:24:44Z","created_by":"spot","updated_at":"2026-07-30T23:06:32Z","dependencies":[{"issue_id":"improvise-1ey","depends_on_id":"improvise-cqq","type":"blocks","created_at":"2026-08-30T10:08:33Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-1ey","depends_on_id":"improvise-cr3","type":"blocks","created_at":"2026-04-14T00:25:43Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-1ey","depends_on_id":"improvise-q08","type":"blocks","created_at":"2026-04-14T00:25:43Z","created_by":"spot","metadata":"{}"}],"dependency_count":3,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"improvise-cr3","title":"DOM renderer for browser grid (reads ViewState + render cache)","description":"JS/TS (or Rust via web-sys) layer that subscribes to the wasm client's state and renders the grid to the DOM. MVP: rebuild a \u003ctable\u003e element on each state change, no virtual-DOM diffing. Reads ViewState for mode indicator, cursor highlight, minibuffer text; reads render cache for cell contents, labels, column widths. Captures browser keyboard events and routes them into the wasm client's on_key_event.","design":"Option A: pure JS/TS module that reads wasm-exposed state via JsValue and updates DOM imperatively. Simpler for MVP. Option B: Rust + web-sys in the wasm crate, rendering from inside wasm. More code sharing but bigger bundle. Start with Option A. Renderer: single \u003ctable\u003e for the grid body, \u003cthead\u003e for column labels, \u003ctbody\u003e with rows. On state change, re-render the affected sections. Cursor highlight is a CSS class on the selected \u003ctd\u003e. Mode indicator is a \u003cdiv\u003e above the table. Minibuffer is a \u003cdiv\u003e shown conditionally when mode is Editing/FormulaEdit/etc.","acceptance_criteria":"(1) Grid renders from a ViewState + RenderCache snapshot. (2) Cursor highlight updates on cursor move without full re-render (nice to have, not required for MVP). (3) Mode indicator reflects current AppMode. (4) Keyboard events on document are captured and routed to wasm on_key_event. (5) Works in Chrome and Firefox. Help overlay, panels, import wizard, tile bar — all deferred post-MVP.","notes":"Consumes viewmodels (not the render cache directly) per improvise-edp. Specifically GridViewModel for grid rendering. Shares GridViewModel type and compute_grid_viewmodel function with ratatui's grid widget (improvise-n10) — one derivation, two rendering backends. DOM-specific concerns (device pixel ratio, CSS class names) live in the RenderEnv the viewmodel is computed against, not in the viewmodel itself.\nPRIORITY RAISED P3-\u003eP2 (groomed 2026-08-30): this gates P2 work (improvise-1ey improvise-6jk improvise-djm ), so leaving it P3 made 'bd ready' unworkable in priority order — the P2 cannot be reached without it. Priority should be non-decreasing along dependency edges. Revert if the intent was that the downstream P2 is genuinely not wanted yet.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:24:41Z","created_by":"spot","updated_at":"2026-08-30T17:07:39Z","dependencies":[{"issue_id":"improvise-cr3","depends_on_id":"improvise-edp","type":"blocks","created_at":"2026-04-14T00:53:44Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cr3","depends_on_id":"improvise-gsw","type":"blocks","created_at":"2026-04-14T00:25:42Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} {"_type":"issue","id":"improvise-gsw","title":"improvise-wasm-client crate (keymap + reduce_view in wasm)","description":"Create the wasm crate that runs in the browser. Depends on improvise-protocol (Command, ViewState, reduce_view) and on the extracted improvise-command crate for the keymap. Exposes wasm-bindgen entry points: init, on_key_event, on_message (incoming command from server), current_view_state (for the DOM renderer to read). Resolves browser keyboard events locally via the keymap, dispatches the resulting command through reduce_view, serializes the command for sending upstream.","design":"crates/improvise-wasm-client/. wasm-bindgen + web-sys. Entry points: #[wasm_bindgen] fn init() -\u003e Handle (allocates ViewState + RenderCache); fn on_key_event(handle, browser_key_event) -\u003e Option\u003cString\u003e (resolves via keymap, dispatches via reduce_view, returns serialized Command to send upstream or None if no-op); fn on_message(handle, serialized_command) (deserializes incoming Command, applies via reduce_view, marks render dirty); fn render_state(handle) -\u003e JsValue (exposes ViewState + RenderCache for the DOM renderer). Key conversion: browser KeyboardEvent → improvise neutral Key enum via a From impl. The keymap is the same Keymap type used server-side, but with no ratatui dependency (from improvise-command after crate epic step 5).","acceptance_criteria":"(1) Crate compiles to wasm32-unknown-unknown. (2) Size budget: under 500 KB compressed. (3) on_key_event correctly resolves common keys (arrows, letters, Enter, Esc) via the keymap and dispatches through reduce_view. (4) on_message correctly deserializes and applies projection commands. (5) A small JS test harness can drive the crate end-to-end without a real websocket.","notes":"Transport-agnostic at the Rust layer: on_message accepts serialized commands regardless of source, and outgoing commands are returned to the JS bootstrap which routes them to a WebSocket (thin-client epic 6jk) or a worker MessageChannel (standalone epic tm6). The same wasm artifact is reused literally in both deployments — only the JS bootstrap differs. Depends on improvise-cqi (protocol crate) and crate-epic step 5 (improvise-3mm) so Keymap is importable without ratatui. Does not include the DOM renderer itself — that is improvise-cr3, which consumes the state this crate exposes.","status":"closed","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:24:37Z","created_by":"spot","updated_at":"2026-09-19T23:40:23Z","closed_at":"2026-09-19T23:40:23Z","close_reason":"crates/improvise-wasm-client landed complete, not a skeleton. 28 tests; cargo check -p improvise-wasm-client --target wasm32-unknown-unknown clean.\n\nTHE TICKET AC 3 IS A FALSE PREMISE and the design turns on it: a browser client resolves almost NO keys locally. A binding names an INTENT resolved against a CmdContext (Model, GridLayout, cursor, yanked cell) while a Command is the RESOLVED MUTATION -- the same reason improvise-gxi AC 4 was closed. So keys travel upstream UNRESOLVED as ClientMessage::Key { code, modifiers }.\n\nTHE ONE LOCAL PATH is minibuffer text entry, allowed for a STRUCTURAL reason rather than convenience: the target buffer is AppMode::minibuffer().buffer_key, the same field TextEntry::new reads (gotcha 11), so the two rules are DERIVED FROM ONE FACT rather than duplicated. Four things were considered and rejected with reasons in the module docs: cursor movement (GridCache is viewport-indexed, so a client cannot tell last-visible-row from last-row and cannot clamp), panel cursors (the flattening depends on expanded_cats), any modified character (KeyPattern::AnyChar matches under ANY modifier on the host, so mirroring the fallback would be a SUPERSET -- local resolution may only ever be a strict subset), and CommandMode Backspace (the single TextEntry::rebind, which leaves the mode on an empty buffer).\n\nimprovise-command is a DEV-dependency only, so the browser graph is formula -\u003e core -\u003e protocol -\u003e client: no App, Workbook, persistence or wizard. tests/lockstep.rs drives the real host and the client over the same keypress across all seven text-entry modes.\n\nAC 2 (under 500KB compressed) UNVERIFIED -- no wasm-pack/wasm-opt available; the dependency graph is the strongest evidence here and someone should measure a real release build. AC 5 (JS harness) not written, and belongs with improvise-cr3.","dependencies":[{"issue_id":"improvise-gsw","depends_on_id":"improvise-3mm","type":"blocks","created_at":"2026-04-14T00:25:41Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-gsw","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:41Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} -{"_type":"issue","id":"improvise-q08","title":"improvise-ws-server binary (tokio + tungstenite session wrapper)","description":"New binary crate that wraps today's App behind a websocket. Accepts connections, creates a ClientSession per connection, receives Command messages from clients, dispatches them into the App, drains the outbound projection queue, sends projection commands back down. One App instance per session for the MVP (no shared authoritative state across sessions yet).","design":"crates/improvise-ws-server/. Tokio runtime, tokio-tungstenite for websocket. On connect: create App + ClientSession, send initial Snapshot (full ViewState + render cache filled from current viewport). Main loop: recv Command from client → feed into reduce_full (today's App::handle_key path, but command-keyed instead of key-keyed) → drain projection queue → send Commands down. On disconnect: drop session. Single session per connection for MVP. No auth, no collab, localhost only. Wire format: serde_json to start (easy to debug), bincode/postcard later if size matters.","acceptance_criteria":"(1) Binary crate builds and accepts websocket connections on a configurable port. (2) Initial snapshot is sent on connect. (3) Command round-trip works: client Command in → model updated → projection Command out. (4) Disconnection cleanly drops session state. (5) Integration test: spawn server, connect a test client, send a SetCell, verify a CellsUpdated comes back.","notes":"Depends on improvise-protocol (wire types) and improvise-mae/vb4/projection-emission for the server-side logic. The server itself is a thin transport wrapper — all the real logic lives in the App and projection layer.","status":"open","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:14Z","created_by":"spot","updated_at":"2026-07-30T23:06:32Z","dependencies":[{"issue_id":"improvise-q08","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:39Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-q08","depends_on_id":"improvise-cqq","type":"blocks","created_at":"2026-04-14T00:25:40Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} +{"_type":"issue","id":"improvise-q08","title":"improvise-ws-server binary (tokio + tungstenite session wrapper)","description":"New binary crate that wraps today's App behind a websocket. Accepts connections, creates a ClientSession per connection, receives Command messages from clients, dispatches them into the App, drains the outbound projection queue, sends projection commands back down. One App instance per session for the MVP (no shared authoritative state across sessions yet).","design":"crates/improvise-ws-server/. Tokio runtime, tokio-tungstenite for websocket. On connect: create App + ClientSession, send initial Snapshot (full ViewState + render cache filled from current viewport). Main loop: recv Command from client → feed into reduce_full (today's App::handle_key path, but command-keyed instead of key-keyed) → drain projection queue → send Commands down. On disconnect: drop session. Single session per connection for MVP. No auth, no collab, localhost only. Wire format: serde_json to start (easy to debug), bincode/postcard later if size matters.","acceptance_criteria":"(1) Binary crate builds and accepts websocket connections on a configurable port. (2) Initial snapshot is sent on connect. (3) Command round-trip works: client Command in → model updated → projection Command out. (4) Disconnection cleanly drops session state. (5) Integration test: spawn server, connect a test client, send a SetCell, verify a CellsUpdated comes back.","notes":"Depends on improvise-protocol (wire types) and improvise-mae/vb4/projection-emission for the server-side logic. The server itself is a thin transport wrapper — all the real logic lives in the App and projection layer.","status":"closed","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:14Z","created_by":"spot","updated_at":"2026-09-19T23:46:08Z","closed_at":"2026-09-19T23:46:08Z","close_reason":"crates/improvise-ws-server landed whole; all five ACs met and tested. 14 tests (13 in host.rs + 1 real-socket integration). Host::handle is where every decision is made and every unit test points; transport.rs is framing, JSON and one handshake rule, with a doc comment saying that if the socket test ever becomes the place a behaviour is pinned, that behaviour belongs in host.rs.\n\nONE APP PER DOCUMENT, as ruled. Not per connection (the ticket premise, now false -- Sessions exists to hold many ClientSessions against one App, and per-connection Apps would make broadcast, apply_switch and apply_model dead code and give two tabs on one document two documents) and not per server. No singleton: Host is a value the transport is handed. The multi-document registry slots into transport.rs where host_loop is -- a HashMap\u003cDocumentId, Host\u003e plus a document id on Connect -- and nothing above it changes, because Sessions never owned the App.\n\nTHE ONLY ADDITION TO THE SHARED VOCABULARY IS ClientMsg::Viewport { rows, cols }, and it is derived rather than invented: the extent is the one session input no Command carries by construction. It is also the HANDSHAKE -- the bootstrap Snapshot seed carries the VISIBLE cells, so a snapshot built at extent zero is an empty grid needing an immediate second round trip. That rule lives in transport::connection alone, which keeps Host::connect total.\n\nNO KEYPRESS DOOR: the retraction arrived before one was built, so nothing had to be removed. ClientMsg carries Viewport and Command only.\n\nEvery non-obvious behaviour proven by injection, including both arms of the broadcast (relay to nobody FAILED; relay to everyone FAILED). The document-swap fixture keeps the incoming active view named Default per gotcha 39 fixture warning, so it cannot pass by accident.","dependencies":[{"issue_id":"improvise-q08","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:39Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-q08","depends_on_id":"improvise-cqq","type":"blocks","created_at":"2026-04-14T00:25:40Z","created_by":"spot","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} {"_type":"issue","id":"improvise-cqq","title":"Server-side projection emission layer + per-client viewport tracking","description":"After each command is applied on the server, walk the resulting effect list, identify model-touching effects, compute which connected clients have the affected cells in their viewport, and enqueue projection commands (CellsUpdated, ColumnLabelsChanged, etc.) for dispatch to those clients. Server must track per-client viewport state (visible row/col range) and mirror each client's ViewState for this to work.","design":"Server holds HashMap\u003cSessionId, ClientSession\u003e where ClientSession { view: ViewState, visible_region: (row_range, col_range) }. When a client dispatches a command, update its session's view state via the shared reduce_view. When the command affects Model, walk the effect list: for each ModelTouching effect, compute which cells changed, intersect with each client session's visible_region, emit CellsUpdated commands into that session's outbound queue. ScrollTo commands update visible_region and trigger a fresh CellsUpdated for the newly-visible region. Viewport intersection logic: a cell is visible if its row/col position (computed from layout) lies within the client's visible range.","acceptance_criteria":"(1) Server struct ClientSession tracks view state and visible region per connected client. (2) After effect application, a projection step computes Vec\u003c(SessionId, Command)\u003e to dispatch. (3) ScrollTo updates visible region and triggers a fresh render-cache fill. (4) Unit tests: dispatching a SetCell produces CellsUpdated only for clients whose viewport includes the cell. (5) Unit tests: ScrollTo produces a CellsUpdated covering the new visible region.","notes":"Depends on improvise-mae (effect classification) and improvise-protocol (shared types). Does not yet wire up the actual websocket transport — that's a separate issue. This is the logical projection layer.\n--- DECIDED 2026-09-19 (user) ---\nRenderCache is PER-WIDGET, and state synchronizes by EVENTS, not snapshots.\n\nThis is shape (C) from improvise-rkeq, with per-widget caches:\n * No flat grid-only wire type and no fat aggregate container. Each widget\n owns its own cache type (GridCache, CategoryTreeCache, FormulaCache,\n ViewCache, TileBarCache, WizardCache, HelpCache).\n * Each widget subscribes to and FOLDS its own stream of projection\n Commands. There is no aggregate RenderCache on the wire.\n * The ONE exception is the initial state transfer: a newly connected\n client gets a Snapshot to seed from, then follows the event stream.\n Snapshot stays bootstrap-only (wire_carries(snapshot,bootstrap_only)).\n\nConsequence for this ticket: read \"RenderCache\" as \"that widget's own cache\ntype\", and expect updates to arrive as folded projection commands rather\nthan as a rebuilt snapshot.\n\n--- IMPLEMENTATION LANDED 2026-09-19; left OPEN because it is blocked by improvise-g0f ---\nAll five ACs are met in crates/improvise-command/src/ui/project.rs (861 lines,\n9 tests). Not closed: bd reports this blocked by g0f (may two clients view\ndifferent views?) and gxi (ACs 4-5, itself blocked). g0f is a user decision and\nnow has teeth -- the per-client ClientSession makes divergence expressible.\n\nDesign, for whoever picks this up:\n* The producer decides what changed by DIFF, not by asking effects. It rebuilds\n each widget's desired state from the App and diffs against a per-client\n mirror. Making every effect report its touched cells would be a second\n description of the same mutation, free to drift from the first.\n* EffectKind is the GATE, not the answer: Dirty::of(kind) is an exhaustive\n match, so a new ModelChange variant fails the build. Over-reporting costs\n work; under-reporting is the bug.\n* Only the grid keeps a mirror -- every other cache replaces wholesale, so\n Dirty is that widget's whole decision. That also splits the two APIs:\n document-scoped projections come from reduce_full, the viewport-dependent\n grid from Sessions::project_all.\n* grid_updates emits RowLabels/ColLabels/ColWidths BEFORE Cells and diffs the\n cells against fold(mirror, label_updates) rather than against mirror, so\n emission is exact through reshapes and never needs Invalidated.\n* Viewport extent is host-reported (ClientSession::set_extent), not inferred:\n offsets already ride the command stream, but how many rows fit is a terminal\n fact no derived Command carries, and inventing a wire variant for it would\n break command_set_derived_not_invented.\n* Bootstrap shares the steady-state path: connect resets the mirror and\n projects Dirty::EVERYTHING, so a Snapshot seed is the diff-from-nothing.\nAC wording deviation: there is no ScrollTo in the derived Command set;\nSetRowOffset/SetColOffset do that job.\n15 tests, each proven to fire by injection; two injections that did NOT fire\nwere treated as test weaknesses and the tests rewritten rather than accepted.","status":"closed","priority":2,"issue_type":"feature","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:12Z","created_by":"spot","updated_at":"2026-09-19T23:29:48Z","closed_at":"2026-09-19T23:29:48Z","close_reason":"Implementation landed in wave 2 (crates/improvise-command/src/ui/project.rs), reviewed, three HIGH bugs fixed, merged to main. All five ACs met. Design recorded in this ticket and in gotchas 33/35/36/39 plus doctrine(lockstep_binds_every_session_not_the_asker). Blockers now all resolved: g0f decided by the user, mae and 8uz closed, gxi re-scoped and closed. Closing unblocks improvise-1ey.","dependencies":[{"issue_id":"improvise-cqq","depends_on_id":"improvise-8uz","type":"blocks","created_at":"2026-08-30T10:08:34Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-cqi","type":"blocks","created_at":"2026-04-14T00:25:39Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-g0f","type":"blocks","created_at":"2026-09-01T02:02:48Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-gxi","type":"blocks","created_at":"2026-04-14T00:53:45Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqq","depends_on_id":"improvise-mae","type":"blocks","created_at":"2026-04-14T00:25:38Z","created_by":"spot","metadata":"{}"}],"dependency_count":5,"dependent_count":4,"comment_count":0} {"_type":"issue","id":"improvise-cqi","title":"Extract improvise-protocol crate (Command, ViewState, reduce_view)","description":"Create a new crate that holds the types shared between the server and the wasm client: Command enum (the wire format), ViewState struct, render cache types (CellDisplay, visible region), and the reduce_view function. Depends on improvise-core so it can reference CellKey, CellValue, AppMode. Serde on everything that crosses the wire. This is the load-bearing crate for the browser epic.","design":"pub enum Command { /* user-initiated */ MoveCursor(Delta), EnterMode(AppMode), AppendMinibufferChar(char), ScrollTo { row, col }, SetCell { key: CellKey, value: CellValue }, AddCategory(String), ... /* server→client projections */ CellsUpdated { changes: Vec\u003c(CellKey, CellDisplay)\u003e }, ColumnLabelsChanged { labels: Vec\u003cString\u003e }, ColumnWidthsChanged { widths: Vec\u003cu16\u003e }, RowLabelsChanged { labels: Vec\u003cString\u003e }, ViewportInvalidated, ... }. pub fn reduce_view(vs: \u0026mut ViewState, cache: \u0026mut RenderCache, cmd: \u0026Command) — pattern-matches the command and applies view-slice effects + render-cache updates. Client imports this directly. Server imports Command + ViewState so it can serialize wire messages and maintain a mirror of each client's view state for projection computation.","acceptance_criteria":"(1) New crate crates/improvise-protocol/ exists. (2) Depends only on improvise-core, improvise-formula (if needed for anything), and serde. No ratatui, no crossterm. (3) Command, ViewState, RenderCache, CellDisplay all implement Serialize+Deserialize. (4) reduce_view has unit tests for view effect application and cache updates. (5) Crate builds for both host target and wasm32-unknown-unknown.","notes":"Depends on improvise-vb4 (ViewState must exist as a distinct type first) and on crate-epic step 2 (improvise-36h — improvise-core must be extracted so protocol can depend on it without pulling ratatui).\nNote (2026-08-29): acceptance criterion 5 (builds for wasm32-unknown-unknown) is not verifiable in the current dev environment — no wasm32 std in the Nix toolchain and no rustup. See improvise-ywd notes. The rest of this ticket is host-verifiable.\nCORRECTION (2026-08-29): the DESIGN section's justification for ViewState serialization is wrong, and misleading in a way that would invert the architecture.\n\nIt says: 'Server imports Command + ViewState so it can serialize wire messages and maintain a mirror of each client's view state.' The server does NOT serialize ViewState to maintain the mirror. Per improvise-cqq, it updates each session's view state 'via the shared reduce_view' — i.e. by applying the same commands the client applies. ViewState is a PROJECTION of the command stream, not a wire message. improvise-ltq (commands both directions) and improvise-m91 ('a command log is a deterministic replay') say the same.\n\nViewState: Serialize is still needed, but for exactly one reason: improvise-q08's bootstrap — 'On connect: create App + ClientSession, send initial Snapshot (full ViewState + render cache filled from current viewport)'. A client that never saw the earlier stream needs a value to fold onto. Snapshot-then-stream, not state-on-every-change.\n\nConsequences for whoever picks this up:\n- AC 3 stands, but do not build a server that ships ViewState on change. Steady-state traffic is Commands only.\n- RenderCache is local, rebuilt by applying projection commands; it needs Serialize only to ride in that same bootstrap Snapshot.\n- Because the only serialization point is bootstrap, host-only fields must NOT be in ViewState: a fresh client should not inherit a half-open import wizard, a pending prefix key, or staged drill edits.\n\nBLOCKER FOUND (2026-08-29): ViewState as it stands cannot live in this crate at all. crates/improvise-command/src/ui/app.rs:381 holds transient_keymap: Option\u003cArc\u003cKeymap\u003e\u003e. Keymap is in improvise-command, which will depend on improvise-protocol (improvise-gxi puts reduce_full there, taking \u0026Command) — a hard dependency cycle, independent of serde. wizard: Option\u003cImportWizard\u003e is a softer version (improvise-io is acyclic but outside AC 2's allowlist); drill_state and AppMode simply lack derives.\n\nProposed fix for the hard one, per the maintainer: flip the Arc. The transient keymap is always the sub-keymap of a Binding::Prefix at a specific (mode, key, mods), set during dispatch and consumed on the next keypress. ViewState can hold that address — Option\u003c(KeyCode, KeyModifiers)\u003e — and let KeymapSet, which already owns every keymap, resolve it. Both consumers can: App::handle_key has keymap_set and the mode; draw.rs only needs binding_hints() for the which-key popup. Nothing between set and consume changes the mode, since a Prefix binding emits SetTransientKeymap and nothing else.","status":"closed","priority":2,"issue_type":"feature","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-14T07:23:10Z","created_by":"spot","updated_at":"2026-09-01T09:03:56Z","started_at":"2026-09-01T08:55:39Z","closed_at":"2026-09-01T09:03:56Z","close_reason":"crates/improvise-protocol created. AC(1) crate exists; AC(2) depends only on improvise-core + serde, verified by cargo tree (no ratatui/crossterm/tokio) and by the CI wasm job; AC(3) Command/ViewState/RenderCache/CellDisplay all Serialize+Deserialize with a per-variant JSON round-trip test; AC(4) reduce_view has 12 tests covering view application, cache updates, out-of-range projections, and determinism over a repeated stream; AC(5) builds for host and wasm32, added to the CI wasm job.\n\nCommand's client-\u003eserver half was derived, not invented: it is the set of ViewOnly effects targeting the portable slice, which improvise-h0b had already separated from host-local state.\n\nOne addition beyond a pure move: ViewState::active_view, because reduce_view runs on a client with no Workbook and cursors is keyed by view name. Two writers only (App::switch_to_view, App::install_workbook), guarded by the_two_active_view_names_never_diverge. Whether the two copies may diverge is improvise-g0f, which now blocks improvise-cqq.\n\nFollowed the 2026-08-29 correction on the ticket: ViewState is a projection of the command stream, serialized only for the bootstrap Snapshot. Nothing here ships ViewState on change.","dependencies":[{"issue_id":"improvise-cqi","depends_on_id":"improvise-36h","type":"blocks","created_at":"2026-04-14T00:25:38Z","created_by":"spot","metadata":"{}"},{"issue_id":"improvise-cqi","depends_on_id":"improvise-a2x","type":"blocks","created_at":"2026-08-30T15:56:13Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-cqi","depends_on_id":"improvise-h0b","type":"blocks","created_at":"2026-08-29T20:48:44Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-cqi","depends_on_id":"improvise-jc8","type":"blocks","created_at":"2026-08-29T20:44:42Z","created_by":"Edward Langley","metadata":"{}"},{"issue_id":"improvise-cqi","depends_on_id":"improvise-vb4","type":"blocks","created_at":"2026-04-14T00:25:37Z","created_by":"spot","metadata":"{}"}],"dependency_count":5,"dependent_count":6,"comment_count":0} {"_type":"issue","id":"improvise-mae","title":"Tag existing effects as model / view / projection-emitting","description":"Add a classification to every Effect so the server can decide which effects trigger projection-command emission to connected clients. Pure view effects (cursor move, mode change) don't project. Model effects (SetCell, AddCategory, AddFormula) do project — they emit CellsUpdated or similar commands to clients whose viewport includes the change. Prerequisite for the server-side projection emission layer.","design":"Add a method to Effect like fn kind(\u0026self) -\u003e EffectKind where EffectKind is { ModelTouching, ViewOnly }. If Step 4 of crate epic (Effect→enum) has landed, this is a const match on variant. If not, each struct impls a kind() method. ModelTouching effects are further classified by what they change, so projection computation knows what kind of command to emit: { CellData, CategoryShape, ColumnLabels, RowLabels, FormulaResult, LayoutGeometry }.","acceptance_criteria":"(1) Every existing Effect has a kind classification. (2) A const/fn on Effect returns the kind without running apply. (3) Unit tests verify classification for each effect. (4) Audit notes list every effect and its kind.","notes":"Does not depend on Effect-as-enum refactor (crate epic step 4) but is much cleaner if that's landed. Depends on the AppState split so effects that cross slices are clearly cross-cutting.\nAUDIT (AC 4) — all 58 effects and their kinds, as of the closing commit:\n\nCellData (12): AddCategory, AddFormula, AddItem, AddItemInGroup, ApplyAndClearDrill, CleanEmptyRecords, ClearCell, RemoveCategory, RemoveFormula, RemoveItem, SetCell, SortData\n\nLayoutGeometry (14): CreateView, CycleAxis, DeleteView, HideItem, SetAxis, SetNumberFormat, SetPageSelection, ShowItem, SwitchView, ToggleGroup, TogglePruneEmpty, TransposeAxes, ViewBack, ViewForward\n\nWholeWorkbook (3): AcceptWizardModel, ImportJsonHeadless, LoadModel\n\nViewOnly (29): AbortChain, ChangeMode, CloseWizard, EnterEditAtCursor, ExportCsv, HelpPageNext, HelpPagePrev, HelpPageSet, MarkDirty, RebuildLayout, Save, SaveAs, SetBuffer, SetColOffset, SetDrillPendingEdit, SetPanelCursor, SetPanelOpen, SetRowOffset, SetSearchMode, SetSearchQuery, SetSelected, SetStatus, SetTileCatIdx, SetTransientKeymap, SetYanked, StartDrill, StartImportWizard, ToggleCatExpand, WizardApply\n\nNotes on the non-obvious ones:\n- SetSelected/SetRowOffset/SetColOffset are ViewOnly despite writing the shared, persisted View: cursor and scroll are per-client. Residual inconsistency filed as improvise-8uz.\n- Save/SaveAs/MarkDirty touch model_state (dirty, file_path) but change nothing renderable -\u003e ViewOnly.\n- ExportCsv reads the model and writes a file; no state change -\u003e ViewOnly.\n- RebuildLayout rebuilds a derived cache -\u003e ViewOnly.\n- SortData reorders the data store, which changes records-mode row identity -\u003e CellData.\n- ApplyAndClearDrill writes cells through free-function helpers -\u003e CellData.\n- SetTransientKeymap lives in command/keymap.rs, not effect.rs.","status":"closed","priority":2,"issue_type":"task","assignee":"Edward Langley","owner":"el-github@elangley.org","created_at":"2026-04-14T07:22:02Z","created_by":"spot","updated_at":"2026-08-29T20:03:28Z","started_at":"2026-08-29T19:57:52Z","closed_at":"2026-08-29T20:03:28Z","close_reason":"All 58 effects classified. EffectKind::{ViewOnly, Model(ModelChange)} with ModelChange::{CellData, LayoutGeometry, WholeWorkbook}. Full audit table in the notes (AC 4).\n\nkind() HAS NO DEFAULT IMPL. The design section offered a per-struct method; making it required instead means a new effect cannot forget to classify itself. That matters because the failure mode is silent — a mis-declared ViewOnly edit would simply never reach connected clients. It paid off immediately: the compiler found a 58th Effect impl, SetTransientKeymap, living in command/keymap.rs rather than effect.rs, which I would have missed classifying by eye.\n\nDERIVED THE CLASSIFICATION, THEN VERIFIED IT. I extracted what each apply() body touches and reasoned about it in Prolog rather than eyeballing 58 effects. Two things that came out of it:\n\n1. The naive signal is WRONG for 21 effects. The View is stored inside model_state.workbook, so 'touches model_state' does not mean 'is a model change'. SetAxis, SetPageSelection, ToggleGroup, HideItem, ShowItem, TransposeAxes, CycleAxis, SetNumberFormat, SwitchView, ViewBack, ViewForward, CreateView, DeleteView and TogglePruneEmpty are LayoutGeometry, not CellData; Save/SaveAs/MarkDirty touch model_state for bookkeeping only and are ViewOnly. A classification done by grepping for model_state would have had the projection layer broadcasting cell updates for cursor moves.\n\n2. My own extraction was wrong twice, and I caught it by disbelieving the output rather than by luck: multi-line method chains hid workbook.model.set_cell(), and free-function helpers hid ApplyAndClearDrill's writes, so SetCell, SortData, RemoveItem and others first came out ViewOnly — obviously false. Fixed the extraction (flatten chains) and re-derived, then read the remaining suspicious ones directly.\n\nTESTS VERIFY DECLARATIONS EMPIRICALLY, not by restating them (AC 3). Rather than a table asserting kind() == what kind() returns, the tests apply each effect to a probe App and fingerprint the shared state it claims not to touch: view_only_effects_leave_shared_state_alone (16 effects, model data and view config both unchanged), cell_data_effects_actually_change_model_data, layout_geometry_effects_actually_change_view_config. Verified they bite by mislabelling SetCell as ViewOnly — the exact silent-drop bug — which fails cell_data_effects_actually_change_model_data.\n\nJUDGEMENT CALL, recorded rather than buried: SetSelected/SetRowOffset/SetColOffset write the shared, persisted View but are per-client concerns — one client scrolling must not move another's viewport — so they are ViewOnly. They are the only ViewOnly effects that mutate shared workbook state, which the test has to special-case. Filed improvise-8uz to resolve that properly before improvise-cqq builds the projection layer on this.","dependencies":[{"issue_id":"improvise-mae","depends_on_id":"improvise-vb4","type":"blocks","created_at":"2026-04-14T00:25:36Z","created_by":"spot","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}