improvise/context/test-evidence/scale-bisect.md

Scale bisect: how big a document does improvise handle today (improvise-felo.10)

Measured on gh/main @ 8220a065, Apple M4 Max, 64 GB (sysctl: hw.memsize 68719476736), macOS, release build (lto, opt-level 3), native only (no wasm, no real OPFS, no terminal). Harness: benches/scale_bisect.rs (one configuration per process) driven by bench/scale_bisect.py, bench/sweep.py, bench/edge.py, bench/full.py. Reproduce: nix develop --command cargo build --release --bench scale_bisect, copy the binary from target/release/deps/scale_bisect-* to /tmp/sb, then python3 bench/edge.py.

Budgets (user ruling 2026-10-03): 8 ms is the headline (120 FPS, hot path or defer to background); 16 ms, 100 ms (noticeable) and 1 s (broken) are reference crossings.

Method, and what is real

Every number goes through the production path: App::new (open: formula recompute + GridLayout::with_frozen_records), App::dispatch_key(Down) (keypress/scroll: it calls rebuild_layout, then cmd_context, which runs compute_col_widths over the whole grid, then applies effects), project::grid_of (TUI frame: whole grid; browser: a 45x12 window), App::apply_effects with effect::SetCell, SetAxis/HideItem/ToggleGroup/TogglePruneEmpty (reshape), Host::connect and Host::handle with a caught_up ModelCommand::SetCell (browser path incl. projection), format_md/parse_md/save_to(MemoryFs). Document: C0..C{cats-1} with K items, N distinct random tuples ("records", capped at K^cats/2), two measures each (2N cells), _Measure on Column. Timings are the median of up to 3-8 in-process repeats; boundary rows are the median of 3 separate processes run sequentially (spread shown). Earlier exploratory sweeps ran 2-3 processes in parallel and agree with the sequential runs within ~5%.

View::prune_empty defaults to false (view/types.rs), so "prune off" is what a user gets until they toggle it; "prune on" is reported next to it. Rows below are the layout's row_count.

Dominant cost (reasoned, not profiled): the release profile strips symbols, so sample could not symbolicate. The reason is in the code and the timings agree with it: GridLayout::cell_key/row_item do data_items(&self.row_items).nth(row) (view/layout.rs ~253/292/807/870), O(row) per lookup, and compute_col_widths (every keypress, via cmd_context) and grid_of visit every cell, so both are O(rows^2). Evidence: with prune off, rows 729 -> 5.4 ms, 4096 -> 74.6 ms, 19683 -> 1248 ms (x6.7 rows -> x14 time -> x17 time). A 45x12 browser window costs the same as the whole-grid TUI frame (K=30: 2274 vs 2298 ms) because the widths pass is whole-grid whatever the viewport says, so viewport narrowing buys nothing today. GridLayout build itself is cheap (~90 ns/row, ADR 008).

Attribution: state+projection vs render/serialize (K=10, 1000 rows, prune off; 3 runs)

step	ms
keypress (state: rebuild + widths + effect)	9.5-9.7
grid_of whole grid (TUI projection)	11.7-12.0
GridWidget render into a 200x50 ratatui Buffer	0.12
grid_of 45x12 window (browser projection)	9.1-9.3
JSON encode of one SetCell's projected updates	0.001-0.005 (0.6-1.1 KB)
JSON encode of the connect Snapshot	0.02 (21 KB)

Render and serialize are <=0.5 ms in every row measured (GridWidget 0.08-0.53 ms up to 14,500 records; snapshot encode <=0.07 ms; records-mode SetCell updates 21 KB in 0.02 ms). The 8 ms is entirely state and projection. Not measured: the rest of draw() (panels, status; draw is private to the binary), terminal I/O, browser JSON decode and DOM paint.

Axis B: items per category K, 3 categories on Row, N = 100k (capped at K^3/2), prune OFF

K	rows	open	keypress	browser window	SetCell	maxrss MB
5	125	0.1	0.5	0.5	0.2	8
9	729	0.2	5.4 (5.3-5.5)	5.0	0.9	13
10	1,000	0.2	9.6 (9.5-9.7)	9.2	1.2	14
11	1,331	0.3	12.7	12.7	1.6	10
12	1,728	0.3	20.0	19.3	2.1	11
15	3,375	0.5	65	64	4.0	14
16	4,096	0.6	74.6 (74.3-75.2)	73.4	-	-
17	4,913	0.7	105 (105.0-105.5)	103	-	-
20	8,000	1.1	302	301	9.4	21
25	15,625	1.9	773	771	18.8	33
26	17,576	2.2	967-977	962	-	-
27	19,683	2.4	1,248 (1,244-1,251)	1,244	-	-
30	27,000	3.1	2,262	2,274	31	49
40	64,000	6.7	10,858	10,833	73	235
60	216,000	21.6	timeout (>60 s)	-	-	-
1000	10^9	refused (see below)

Hot-path keypress/scroll and browser window: largest inside 8 ms: K=9 (729 rows); inside 16 ms: K=11; first over 100 ms: K=17 (4,913 rows); first over 1 s: K=27 (19,683 rows). SetCell alone (no widths pass; layout rebuild only, linear in rows): inside 8 ms to K19 (K=15 4.0, K=20 9.4), 16 ms ~K=24, 100 ms ~K=45 (K=40 = 73 ms), 1 s extrapolated ~K=95 (extrapolated, not measured). Open stays <25 ms to K=60.

3 x 1,000 on Row cannot be opened at any N: the layout materializes the cross product (10^9 rows, ~150 B each = ~150 GB, ADR 008). The harness refuses above 30M rows (rows_cap) rather than swap the machine, so the 10^9 number is the ADR's estimate, not a measurement here.

Prune ON (same documents)

K	rows	open	keypress	browser window	SetCell
7	171	2.0	4.5 (4.4-4.6)	1.1	5.4
8	256	3.0	7.0 (7.0-7.1)	1.6	8.5
9	364	5.3	12.8	2.7	15.3
11	665	12.2	29.6	6.2	~28
13	1,098	29.0	71	14.5	~70
14	1,372	48	119	24	-
20	4,000	300	747	145	903
21	4,630	401	968-992	190	-
25	7,812	767	1,899	366	2,328
30	13,500	2,255	5,555	1,047	6,804

Prune on: keypress inside 8 ms to K=8, first over 100 ms K=14, 1 s at **K21-22** (K=22 measured 0.93-0.95 s, a different N cap; first clearly over: K=25). Pruning shrinks the grid, so the browser window is cheaper than prune off at the same K (K=20: 145 ms against 302 ms), but every keypress re-runs the prune (dispatch_key calls rebuild_layout), so keypress and SetCell are worse with prune on (K=10: 23 vs 9.9 ms). Prune is also quadratic (nth). Conclusion: prune on/off moves the boundary by about one K at 8 ms and makes 100 ms/1 s earlier (K=14/21 vs 17/27); it never rescues the 3x1000 case.

Axis A: records N at fixed K (K=30, 27,000 rows)

N	prune off keypress	prune on rows	prune on keypress	prune on window
100	451	100	861	0.3
1,000	527	1,000	1,059	6.4
5,000	1,013	5,000	2,178	140
13,500 (cap)	2,267	13,500	5,555	1,047

Prune off: ~450 ms of widths over 27k empty rows + ~1.3 us per stored cell; N never gets the keypress under 100 ms at K=30, and at K=1000 N is irrelevant because the layout cannot be built (rows = K^3). The N-axis has no bisect point in the 3x1000 regime; N only matters once K is small enough that rows fit, and there it is a second-order term. Open is flat (3.1-3.6 ms) in N with prune off.

Axis C: categories on Row (prune off)

K	cats	rows	open	keypress	note
10	2	100	0.1	0.2
10	3	1,000	0.2	9.9	over 8 ms
10	4	10,000	1.4	880	~1 s
10	5	100,000	14	timeout >100 s
3	6	729	0.3	27.9	5x worse than K=9 x 3 cats at the same rows
3	8	6,561	2.0	2,798
3	10-12	59k-531k	18-210	timeout	open alone passes 100 ms at 11 cats
2	10	1,024	0.6	172
2	12	4,096	2.2	3,190
2	14	16,384	9	63,100

The cap of 24 is unreachable: rows = product of items, so a view with c Row categories of K items dies by K^c ~ 10^4 rows (1 s) regardless of N, and with two items per category by 12-14 categories. Per-row cost also grows with category count (each row_item is a Vec of c items). Not measured: Row categories with prune on, other axes absorbing categories (the realistic mitigation: only 1-2 on Row).

Axis D: records mode over N records (with_frozen_records, the improvise-sk32 quadratic)

N	open	keypress	browser window	SetCell (apply)	host SetCell+project	maxrss MB
300	1.5	0.7	0.1	1.6	-	10
1,000	2.9	5.6 (5.5-5.7)	0.3	10	-	12
1,200	3.9	7.9 (7.7-8.0)	0.3	-	-	-
1,300	4.5	9.3	0.3	15.5	17	26
1,800	8.4	16.5	0.4	-	-	-
3,000	22.9	47	0.7	74	-	20
4,400-4,600	47-51	95-102	0.9	149	158	55
10,000	248	492	1.8	734	-	46
14,000-14,500	472-500	940-1,002	2.6-2.8	1,435	1,459	135-148
30,000	2,198	4,314	5.6	6,467	-	125
100,000	25,420	51,234	19.3	not reached (killed at 100 s)	-	not reached

Records mode, keypress/scroll: inside 8 ms to N~1,200; 16 ms N1,800; 100 ms N4,500 (first over: 4,600 = 102 ms); 1 s N~14,500. Open crosses 8 ms ~1,750, 16 ms ~2,300, 100 ms ~6,500, 1 s ~20,000. Time ~ N^1.95. SetCell is ~1.5x keypress (it rebuilds and re-groups). The browser window is cheap (<20 ms even at 100k) because records mode has two columns and the width walk is short, so the 51 s is rebuild_layout, not projection. Prune on/off does not change records mode (prune applies to pivoted axes; the sweep ran off). Records per KB: ~4 KB/record process-wide at 14k (maxrss includes the document generator, so read as an upper bound); the 100k memory was not obtained.

Axis E: save / load and per-command persistence

Mode none (all categories off-axis, so layout is trivial and only the store scales), 3 x 1,000, N records = 2N cells:

cells	format_md	text MB	parse_md	persist per accepted command (save_to MemoryFs)	maxrss MB
2,000	13.8	0.1	4.5	12.4	18
20,000	156	0.86	37	161	80
60,000	527	2.5	119	539	236
200,000 (the target)	1,974	8.4	401	1,984	663
600,000	6,470	25	1,320	6,460	1,894
2,000,000	23,120	84	4,668	23,088	5,611

Linear: ~10 us/cell to format, ~2 us/cell to parse. Crossings, format_md: 8 ms ~700 cells, 16 ms ~1.5k, 100 ms ~11k, 1 s ~100k cells. parse_md: 8 ms ~4k cells, 100 ms ~50k, 1 s ~450k. At the target (200k cells) every accepted command in the browser worker costs ~2 s of format alone before OPFS write time (not measured: MemoryFs only). Persist per command also appears in smaller documents inside the grid axes (K=10 grid: 5.5 ms; records N=4,400: 63 ms).

Other operations (3 runs each; "grouped" fixtures start with C0 groups collapsed, so rows are not K^3)

Reshape (prune off, K=10, 1000 rows): hide item 0.25 ms, set axis 0.19 ms, toggle prune 12.7 ms. K=26 grouped, 2,028 rows: hide 0.5, toggle group 1.8, set axis 1.8, toggle prune 727-750 ms. Prune on K=13 grouped, 321 rows: hide 9.4, toggle group 37, set axis 37, toggle prune 37-39 ms. Host::connect snapshot: K=10 6.5 ms, K=26 375 ms, records N=14,000 2.9 ms. Host SetCell+project: K=10 7.7 ms, K=26 ~400 ms, records N=4,400 158 ms (equal to apply + a whole-grid width walk).

Classification and feasibility of yielding

operation	class	today's first size over 8 ms	dominated by	chunked/yielding form
keypress / scroll	hot (<=8 ms)	1,000 rows (K=10) / N=1,300 records	rebuild_layout + whole-grid compute_col_widths (O(rows^2) nth), recomputed though nothing changed	not a slicing problem: cache widths/layout across keys, O(1) row lookup (ADR 008/009). Slicing would only hide it
browser window projection	hot	1,000 rows	same width walk, whole grid regardless of window	needs the same fix; then window cost is O(window)
SetCell apply + project	hot	K~19 (prune off), 1,300 records	layout rebuild O(rows) + widths + projection	incremental update of one row/total, not a rebuild
column totals	hot	inside grid_of (not separable here)	whole-grid sum in the projection	maintained total (ADR 009)
open / layout build	deferrable	40k rows (prune off); N~1,750 (records)	build is ~90 ns/row (sliceable by outer index); the prune/records grouping is the quadratic part	build is feasible in slices once the quadratic is removed; the quadratic itself is not sliceable
reshape (hide/collapse/axis)	deferrable	cheap prune-off (<2 ms to ~2k rows); 37 ms prune-on at 321 rows	layout rebuild + prune	one sort/remap at 100k occupied tuples was 2.3 ms in ADR 008's prototype, so a reshape can fit inline; prune toggle needs the quadratic fixed first
save (format_md)	deferrable	~700 cells	per-cell formatting, linear	the Data section is a flat per-cell line stream, so chunking by cell range with a yield every ~400 cells looks feasible, but needs a consistent snapshot (clone or journal); not profiled by function
load (parse_md)	deferrable	~4k cells	pest parse + tree walk, linear	tree walk is chunkable by line; the pest parse of one text is not, unless the Data section is parsed line by line (grammar is line oriented); not profiled by function
persistence per command	deferrable (debounce)	~700 cells	the whole-document format_md	coalesce/debounce or append-journal: formatting 2 s per command at target is unusable inline at any slicing

What I could not measure

• wasm: all numbers are native; the browser worker is typically slower. Real OPFS I/O, JSON decode and DOM paint, the full draw() frame, and Safari are not covered.
• Records mode at 100k: SetCell, host and RSS (killed after open and keypress, ~76 s).
• Per-function profile: symbols are stripped in [profile.release]; attribution is reasoned from the code and the timing exponents, not a flame graph.
• SetCell 100 ms/1 s for prune off past K=40 are extrapolations.
• Memory (maxrss) includes the generator's hash set and recs vector; it is an upper bound on the document. Layout cost ~150 B/row per ADR 008, consistent with K=40 (64k rows, 235 MB peak with prune off including the generator and a second layout during rebuild).
• Noise: boundary rows are 3 sequential processes, spread <=3% (e.g. K=17: 105.0-105.5 ms). Non-boundary rows are single runs, some taken with 2-3 concurrent processes on a 16+ core machine; treat them as +-5-10%.