fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge commit '50e1314d069a4f6e549cd7d9795b6491a89bf903' into adr9-rev-steward

Browse Source
This commit is contained in:
Claude (adr9-steward)
2026-10-05 14:43:31 -07:00
committed by Edward Langley
parent 7878decbaa 50e1314d06
commit 502373c025
164 changed files with 13048 additions and 7409 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/code-review/SKILL.md
+2 -1
View File
@@ -59,7 +59,8 @@ they mean, rather than testing for the variant
**Layering.** `forbidden/2` in `kb.pl` is the authority:
`improvise-core` no UI/IO, `improvise-io` no UI/commands, `improvise-command`
no ratatui/crossterm. A ratatui type in core is the usual violation — the fix
is a semantic enum in core that the render layer maps (see `AxisColor`).
is a semantic value the render layer maps; for appearance that is a `Face` in
`improvise-protocol`, resolved by `draw.rs::face_style` (ADR 004).
**`Option`/`Result`/`Box` at the boundaries.** Wrap at parse/lookup/IO, unwrap
early, keep core logic on concrete values. Heap-container plumbing should be
.agents/skills/pr-watch/SKILL.md
+61 -6
View File
@@ -5,13 +5,19 @@ description: Wait for a GitHub PR's checks to finish before merging or reporting
# Waiting on a PR
"Watching a PR" means tracking **both** CI checks **and** GitHub comments/reviews.
A PR is not ready to merge just because checks are green — review feedback must be
read and answered first.
## CI Checks
```
scripts/pr-watch <pr-number> [--expect <sha>] [--timeout <sec>] [--interval <sec>]
scripts/pr-watch <pr-number> [--expect <sha>] [--timeout <sec>] [--interval <sec>] [--reviews]
```
```
exit 0 every check COMPLETED, none failed, and MERGEABLE with a state of
CLEAN, UNSTABLE or HAS_HOOKS
exit 0 every check COMPLETED, none failed, MERGEABLE with a state of
CLEAN, UNSTABLE or HAS_HOOKS, and reviews fetched (if --reviews)
exit 1 a check failed (under any mergeability, BLOCKED included), or the
PR is not mergeable (CONFLICTING, DIRTY, BEHIND, DRAFT, or a state
it does not know)
@@ -21,21 +27,70 @@ exit 3 timed out with checks still running, mergeability still UNKNOWN,
exit 4 usage or environment error
exit 5 the PR is MERGED or CLOSED, at the head the watch pinned; it ends on
the first poll that sees it closed, which may be mid-wait
exit 6 checks green/mergeable, but pr-reviews failed (fetch error, not "no reviews")
```
**`--reviews`** (default): after checks complete, print all reviews and comments
via `scripts/pr-reviews`. Use `--no-reviews` to skip this. The reviews are shown
but do not change the exit code — answering them is your responsibility.
**Exit 0 now means mergeable, not only green.** GitHub computes mergeability
lazily and reports `UNKNOWN` for a while after the base moves. It also reports
`BLOCKED` for the moment before a required check that `needs:` another job has
registered. The watch waits out both, so you do not need to re-check
`mergeable` by hand after it. **Exit 3 is not a pass.** It may be a required
check that never ran. Why each
state counts the way it does: `context/code/tooling.md`, "pr-watch and
mergeability" (improvise-edpl).
check that never ran. Why each state counts the way it does:
`context/code/tooling.md`, "pr-watch and mergeability" (improvise-edpl).
**Pass `--expect <sha>` whenever you know which commit you are asking about.**
It is the difference between "this PR is green" and "the thing I pushed is
green", and a PR here was once merged at a head that predated its own fix.
## Reviews and Comments
Reviews are a **separate API surface** from the comment thread. Codex/ChatGPT
reviews arrive as *reviews*, not comments — checking only the comment thread
misses them entirely. `pr-watch --reviews` (the default) shows all three:
| Surface | What it is |
|---|---|
| `[REVIEW]` | The review's own body (pulls/{n}/reviews) |
| `[INLINE]` | A comment on a line of the diff (pulls/{n}/comments) |
| `[COMMENT]` | A comment on the PR thread (issues/{n}/comments) |
To fetch reviews directly: `./scripts/pr-reviews <n>` — **do not pipe it**. The
script's `$?` must be read directly; piping or chaining loses the exit code.
Redirect to a file if you need to process it.
### Review Obligations
Before treating a PR as done or merging:
1. **Read all review feedback** — reviews against an earlier head may still apply
2. **Answer substantive feedback with attribution** — every reply must include
the attribution line (see below)
3. **Re-check after each push** that new reviews target the current head SHA
4. **Don't wait indefinitely on Copilot/Codex** — reviews arrive as quota allows;
an absent review and an unrequested one look identical. The adversarial review
that gates a merge is dispatched by the orchestrator; external reviews are
read *in addition*, not instead.
### GitHub Attribution (Hard Rule)
**Every GitHub write must include an attribution line.** Posts go out under the
repository owner's account — an unattributed post reads as the human's words.
Open the body with: `_Posted by <Agent Stack> (<model>)_`
Examples:
- `_Posted by Claude (Opus 4.5)_`
- `_Posted by Cursor Agent (claude-sonnet-5)_`
A thread is not "answered" until the reply carries this attribution.
See AGENTS.md "GitHub Attribution" for the full rule and "Shipping a wave" for
Copilot request mechanics and how to answer declined findings.
## Do not write the loop yourself
Four signals look like "finished" and are not:
.agents/skills/wave/SKILL.md
+4 -1
View File
@@ -52,7 +52,10 @@ nobody corrects. The brief adds only what varies per wave:
suite, which it is not. The agent still measures its own.
3. **Its ground, and the ground that is not.** Name any orchestrator-only file
it is exceptionally allowed to edit.
4. **Its `BEADS_ACTOR` and its tickets.**
4. **Its `BEADS_ACTOR` and its tickets.** The actor also names the agent's
commits: it commits through `scripts/agent-git`, which authors each one as
`Claude (<BEADS_ACTOR>)` (`agent-rules.md`, "Discipline per ticket"). So
give each agent its own actor, and do not tell it to set `user.name`.
5. **The attribution line, commit trailers and PR footer**, spelled out.
They name the agent (Claude's line opens `_Posted by Claude (<model>)…_`)
and this session's URL, which is why the shared `agent-rules.md` cannot
.agents/skills/wave/agent-rules.md
+27
View File
@@ -216,6 +216,33 @@ out on purpose:
- **Claim first:** `export BEADS_ACTOR=<your actor>` in every shell, then
`bd update <id> --claim`. Exit 13 means another actor holds it. Pick another;
do not retry.
- **Author every commit as `Claude (<BEADS_ACTOR>)`.** The user: "agents
should override git author name, I think to include an agent identifier
(matching BEADS_ACTOR)" (2026-09-25; improvise-1xfb.3). Make every commit
through the wrapper, merge commits included:
`BEADS_ACTOR=<your actor> scripts/agent-git commit …`, and the same for
`merge`. The wrapper sets the author name for that one command, and it refuses (exit 2) when `BEADS_ACTOR` is unset or empty. It
reads `BEADS_ACTOR` from its own environment, so the prefix works in a shell
that keeps nothing between commands.
- **The author email, the committer and the `Co-Authored-By` trailer stay as
they are.** The email is the owner's configured one, so GitHub still links
the commit.
- **Never set `user.name` with `git config`.** Worktrees share
`.git/config`, so it renames every agent at once. Do not use
`git -c user.name=…` either: `user.name` names the committer as well as
the author. Why the wrapper uses `GIT_AUTHOR_NAME`:
`context/test-evidence/tooling.md`.
- A plain `git commit --amend` keeps the author it amends, so amend with
plain git or with the wrapper.
- **A cherry-pick keeps its source's author**, through the wrapper or not:
git ignores `GIT_AUTHOR_NAME` there. That is right, because rule 8 allows
only `cherry-pick -x` of commits that keep whoever wrote them.
- Before you push, check `git log --format='%an <%ae> | %cn' <pinned
SHA>..HEAD`. Every commit **you made** must name your actor. A
cherry-picked commit, or one from a PR you stack on, keeps its own author.
- **The format is a proposal.** `Claude (<BEADS_ACTOR>)` is the default
proposed on improvise-1xfb.3's PR. The user may change it there. The format
lives in `scripts/agent-git` and nowhere else.
- **`bd` needs `.envrc` allowed in your worktree.** Without it you get a
**version mismatch**, not a clean failure. Run `direnv allow` in the worktree,
or rely on the `[whitelist] prefix` in `~/.config/direnv/direnv.toml`.
.beads/issues.jsonl
+386 -247
View File
File diff suppressed because one or more lines are too long
.claude/hooks/session-start.sh
+46 -257
View File
@@ -1,273 +1,62 @@
#!/bin/bash
#!/usr/bin/env bash
# Prepare a Claude Code on the web container for work on this repository.
#
# Section 0 runs everywhere, cloud or not: it asserts that the `bd` on PATH
# is the one flake.nix pins. Everything after it is cloud-only setup.
# This hook delegates to the shared `scripts/agent-bootstrap.sh`, which works
# for any coding agent stack (Claude Code, Cursor, etc.). This file exists
# because Claude Code expects hooks at `.claude/hooks/session-start.sh`.
#
# Three things are slow or wrong by default in a fresh cloud session:
# For local checkouts, only the bd pin assertion runs.
# For cloud sessions (CLAUDE_CODE_REMOTE=true), full bootstrap runs.
#
# 1. `bd` is not installed, so the `bd prime` hook and every `bd` command
# in AGENTS.md silently do nothing.
# 2. `.beads/issues.jsonl` is a stale export. An agent that reads it picks
# work that is already done — this has happened. The authoritative
# tracker is the Dolt database on the git remote.
# 3. The Rust workspace is uncompiled, so the first `cargo test` pays a
# cold build before it reports anything.
#
# The tracker is re-synced on every session, not just when the database is
# missing: cloud containers reuse a cached workspace, so "already present"
# does not mean "still current", and a silently ageing tracker is the exact
# failure this hook exists to prevent. Installing bd and warming the build
# do check first, being genuinely one-time.
#
# Nothing here writes to a tracked file, stages anything, or creates a commit.
# See also: scripts/agent-bootstrap.sh (the stack-agnostic implementation)
set -uo pipefail
ROOT="${CLAUDE_PROJECT_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}"
cd "$ROOT" || exit 0
BOOTSTRAP="$ROOT/scripts/agent-bootstrap.sh"
log() { printf '[session-start] %s\n' "$*"; }
# ── 0. The bd on PATH must be the bd flake.nix pins ────────────────────────
# `flake.nix` pins beads so that one build of `bd` touches this repo. A git
# worktree does not load that pin — a fresh `.envrc` is unallowed, so a
# process there falls through to whatever `bd` the user's profile happens to
# carry. Today those agree; nothing enforces that they keep agreeing.
#
# The failure that matters is silent: writing .beads/issues.jsonl with a
# skewed build empties the dependency graph and reports success. Nobody
# notices until they ask what blocks what. So the mismatch is turned into a
# loud one here rather than trusted. Reading with a skewed bd is safe — it
# is the mutating commands that corrupt — so this blocks nothing, it shouts.
#
# The expected version is read from flake.nix, not written down twice: a
# hard-coded copy would go stale the first time the pin moves, and a stale
# assertion is worse than none.
beads_pin() {
sed -n 's|.*beads\.url[[:space:]]*=[[:space:]]*"github:[^/]*/[^/]*/v\{0,1\}\([^"]*\)".*|\1|p' \
flake.nix 2>/dev/null | head -1
}
bd_running_version() {
bd version 2>/dev/null | sed -n 's/^bd version \([0-9][0-9.]*\).*/\1/p' | head -1
}
bd_pin_banner() {
{
printf '\n[session-start] ===== BEADS VERSION SKEW =====\n'
printf '[session-start] %s\n' "$@"
printf '[session-start] =============================\n\n'
} >&2
}
# Prints the banner and returns 1 when the two disagree, or when either one
# cannot be read — an unreadable answer is a rotted assertion, not a pass.
assert_bd_pin() {
command -v bd >/dev/null 2>&1 || return 0 # nothing on PATH to assert about
pinned=$(beads_pin)
running=$(bd_running_version)
case "$pinned" in
[0-9]*.[0-9]*.[0-9]*) ;;
*)
bd_pin_banner "cannot read the beads pin out of flake.nix" \
"read: '${pinned:-<nothing>}'" \
"The pin moved to a ref this hook cannot parse (a branch or a" \
"commit, perhaps). Teach beads_pin() to read it, or the check" \
"silently protects nothing."
return 1
;;
esac
if [ -z "$running" ]; then
bd_pin_banner "cannot read the version of the bd on PATH" \
"bd: $(command -v bd)" \
"'bd version' did not answer in the expected shape. Until it does," \
"nothing here can tell whether that build matches the pin."
return 1
fi
[ "$running" = "$pinned" ] && return 0
bd_pin_banner "the bd on PATH is NOT the bd this repo pins" \
"pinned (flake.nix): $pinned" \
"running: $running ($(command -v bd))" \
"" \
"Reading is safe. Do NOT run a mutating bd command — create, update," \
"close, dep, dolt push. Version skew between builds empties the" \
"dependency graph in .beads/issues.jsonl and reports success." \
"" \
"Fix now: nix develop (in this directory), then re-run the" \
" command; or 'direnv allow' here." \
"Fix for good: worktrees are created fresh per dispatch, so allowing" \
" them one at a time never catches the next one. Put the" \
" repo root under [whitelist] prefix in" \
" ~/.config/direnv/direnv.toml."
return 1
}
pin_status=0
assert_bd_pin || pin_status=2
# Local checkouts have their own setup; this is for the web containers.
if [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then
exit "$pin_status"
fi
# Install a verified database at .beads/embeddeddolt.
#
# The staged copy lives on another filesystem, so it is brought alongside the
# live one FIRST. Every step after that is a same-filesystem rename, which
# either happens or does not — no half-finished copy. The previous database is
# removed only once the new one is in place, so no failure path leaves the repo
# without a tracker, and every move is checked: an unchecked one would let the
# new database land *inside* the old directory and still report success.
install_tracker() {
staged=$1
live=.beads/embeddeddolt
incoming=$live.incoming
previous=$live.previous
rm -rf "$incoming" "$previous" || return 1
cp -rf "$staged" "$incoming" || { rm -rf "$incoming"; return 1; }
if [ -d "$live" ] && ! mv "$live" "$previous"; then
rm -rf "$incoming"
return 1
fi
if mv "$incoming" "$live"; then
rm -rf "$previous"
return 0
fi
# Could not put the new one in place; restore what was there.
if [ -d "$previous" ] && ! mv "$previous" "$live"; then
log "ERROR: tracker left at $previous — move it back to $live by hand"
fi
rm -rf "$incoming"
return 1
}
# ── 1. bd on PATH ──────────────────────────────────────────────────────────
# The embedded Dolt engine needs CGO, so this builds from source (~3 min).
# The container image is cached after the hook, so it is paid once.
export PATH="$PATH:/root/go/bin:$HOME/go/bin"
# Pinned to a commit rather than `main`, and downloaded before it is run:
# piping a moving URL straight into a shell executes code nobody has seen,
# and leaves nothing to inspect afterwards. The installer stays at
# /tmp/bd-install.sh so what actually ran can be read.
#
# To take a newer beads, resolve the peeled commit of its tag:
# git ls-remote --tags https://github.com/steveyegge/beads | grep 'v1.3.0^{}'
BD_INSTALLER_REF=f45b249ce6b40ba62aecc03949e6371e8f7c79d8 # tag v1.3.0
BD_INSTALLER_URL="https://raw.githubusercontent.com/steveyegge/beads/$BD_INSTALLER_REF/scripts/install.sh"
if ! command -v bd >/dev/null 2>&1; then
log "installing bd (beads v1.3.0)…"
if ! curl -fsSL "$BD_INSTALLER_URL" -o /tmp/bd-install.sh 2>/tmp/bd-install.log; then
log "WARNING: could not download the bd installer; see /tmp/bd-install.log."
elif bash /tmp/bd-install.sh >>/tmp/bd-install.log 2>&1; then
log "bd installed"
else
log "WARNING: bd install failed; see /tmp/bd-install.log (the script that"
log " ran is at /tmp/bd-install.sh). Issue tracking unavailable."
fi
fi
# Forward the Claude env file variable to the generic name the bootstrap uses
if [ -n "${CLAUDE_ENV_FILE:-}" ]; then
# Single quotes are deliberate: $PATH and $HOME must be written literally so
# they expand when the session sources this, not now.
# shellcheck disable=SC2016
echo 'export PATH="$PATH:/root/go/bin:$HOME/go/bin"' >> "$CLAUDE_ENV_FILE"
fi
command -v bd >/dev/null 2>&1 && bd metrics off >/dev/null 2>&1
# bd may have arrived only just now, in which case §0 had nothing to check.
# Re-check, but only if §0 stayed quiet, so the banner is never printed twice.
if [ "$pin_status" -eq 0 ]; then
assert_bd_pin || pin_status=2
export AGENT_ENV_FILE="$CLAUDE_ENV_FILE"
fi
# ── 2. Sync the issue tracker from the Dolt remote ─────────────────────────
# `bd init` in-place rewrites .beads/config.yaml and commits, so the sync
# happens in a throwaway directory and only the database is moved back.
# That directory must be named after the database (see .beads/metadata.json),
# since bd derives the database name from it.
if command -v bd >/dev/null 2>&1; then
DB_NAME="$(jq -r '.dolt_database // "improvise"' .beads/metadata.json 2>/dev/null || echo improvise)"
ORIGIN="$(git remote get-url origin 2>/dev/null || true)"
case "$ORIGIN" in
git@github.com:*) ORIGIN="https://github.com/${ORIGIN#git@github.com:}" ;;
esac
ORIGIN="${ORIGIN%.git}"
# The mktemp is part of the condition on purpose. Unchecked, a failure
# leaves TMP empty, "$TMP/$DB_NAME" becomes an absolute path at the
# filesystem root, and the cleanup at the end degrades to `rm -rf ""`.
# Short-circuiting also means no temp directory is created when there is
# no origin to sync from.
if [ -n "$ORIGIN" ] && TMP=$(mktemp -d) && [ -d "$TMP" ]; then
log "syncing issue tracker from $ORIGIN …"
STAGE="$TMP/$DB_NAME"
mkdir -p "$STAGE"
# `bd init` exits non-zero when the remote is unreachable but STILL leaves
# a .beads/embeddeddolt behind — an empty database. Testing only for the
# directory would swap that over a good tracker, so check the status and
# confirm the result actually carries issues. An empty remote tracker is
# indistinguishable from a failed clone here; keeping what we already have
# is the safe way to be wrong.
sync_ok=0
if (
cd "$STAGE" && git init -q . &&
BD_NON_INTERACTIVE=1 bd init --non-interactive --skip-agents --skip-hooks -q \
--remote "git+$ORIGIN"
) >/tmp/bd-sync.log 2>&1 && [ -d "$STAGE/.beads/embeddeddolt" ]; then
listing="$( (cd "$STAGE" && bd list --all 2>/dev/null) || true )"
case "$listing" in
"" | "No issues found."*) ;;
*) sync_ok=1 ;;
esac
# For local checkouts, only verify the bd pin (no install, no sync)
if [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then
# Inline the pin-check only, since the full bootstrap is cloud-only
cd "$ROOT" || exit 0
beads_pin() {
sed -n 's|.*beads\.url[[:space:]]*=[[:space:]]*"github:[^/]*/[^/]*/v\{0,1\}\([^"]*\)".*|\1|p' \
flake.nix 2>/dev/null | head -1
}
bd_running_version() {
bd version 2>/dev/null | sed -n 's/^bd version \([0-9][0-9.]*\).*/\1/p' | head -1
}
if command -v bd >/dev/null 2>&1; then
pinned=$(beads_pin)
running=$(bd_running_version)
# Fail closed: if we can't parse either version, warn rather than silently pass
if [ -z "$pinned" ]; then
printf '[session-start] WARNING: cannot parse beads pin from flake.nix\n' >&2
printf '[session-start] The pin check is skipped, but mutating bd commands may be unsafe.\n' >&2
exit 2
fi
if [ "$sync_ok" -eq 1 ] && install_tracker "$STAGE/.beads/embeddeddolt"; then
log "issue tracker synced — 'bd ready' now reflects the remote, not issues.jsonl"
elif [ -d .beads/embeddeddolt ]; then
log "WARNING: tracker sync failed (see /tmp/bd-sync.log); kept the previous database."
log " It may be out of date — verify any bead against the code."
else
log "WARNING: tracker sync failed; see /tmp/bd-sync.log."
log " .beads/issues.jsonl is a STALE export — verify any bead against the code."
if [ -z "$running" ]; then
printf '[session-start] WARNING: cannot parse bd version from "bd version"\n' >&2
printf '[session-start] bd: %s\n' "$(command -v bd)" >&2
printf '[session-start] The pin check is skipped, but mutating bd commands may be unsafe.\n' >&2
exit 2
fi
if [ "$running" != "$pinned" ]; then
printf '\n[session-start] ===== BEADS VERSION SKEW =====\n' >&2
printf '[session-start] pinned (flake.nix): %s\n' "$pinned" >&2
printf '[session-start] running: %s (%s)\n' "$running" "$(command -v bd)" >&2
printf '[session-start] Reading is safe. Do NOT run mutating bd commands.\n' >&2
printf '[session-start] Fix: nix develop, or direnv allow.\n' >&2
printf '[session-start] =============================\n\n' >&2
exit 2
fi
rm -rf "$TMP"
elif [ -n "$ORIGIN" ]; then
log "WARNING: could not create a temporary directory; skipped the tracker sync."
log " .beads/issues.jsonl is a STALE export — verify any bead against the code."
fi
exit 0
fi
# ── 3. Warm the build so the first cargo test is not a cold compile ────────
if command -v cargo >/dev/null 2>&1; then
log "warming cargo build…"
if cargo build --workspace --tests >/tmp/cargo-warm.log 2>&1; then
log "workspace built"
else
log "WARNING: cargo build failed; see /tmp/cargo-warm.log"
fi
fi
# ── 4. Environment facts worth knowing up front ────────────────────────────
cat <<'NOTES'
[session-start] Environment notes (see context/cloud-agents.md):
• Use the preinstalled cargo toolchain for ordinary work. `nix develop`
also works, but only after scripts/nix-seed-flake-inputs.sh — GitHub
tarballs are not served here, so the locked inputs are seeded over git.
• `bd dolt push` is blocked outbound (403). Bead status changes stay local.
• Never run a bare `bd init` in this repo: it rewrites AGENTS.md, CLAUDE.md
and .claude/settings.json, and commits. Use --skip-agents --skip-hooks.
• `bd dolt push <remote>` ignores its argument and pushes to origin.
NOTES
# Non-zero only when §0 found version skew; the setup above still ran.
exit "$pin_status"
# Cloud session: run the full bootstrap
exec "$BOOTSTRAP"
.claude/skills/beads
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/beads
.cursor/environment.json
+5
View File
@@ -0,0 +1,5 @@
{
"name": "Improvise",
"install": "nix-daemon >/tmp/nix-daemon.log 2>&1 & ./scripts/agent-bootstrap.sh",
"start": "nix-daemon &"
}
.cursor/skills/beads
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/beads
.cursor/skills/beads-merge
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/beads-merge
.cursor/skills/code-review
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/code-review
.cursor/skills/kb
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/kb
.cursor/skills/pr-watch
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/pr-watch
.cursor/skills/wave
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/wave
.github/workflows/ci.yml
+27 -1
View File
@@ -146,13 +146,39 @@ jobs:
# dependency crate each run. The NixOS runners substitute from titan's
# harmonia first, and titan builds gh/main's `#improvise` hourly, so a PR that
# leaves Cargo.lock alone rebuilds only the workspace crates it changed.
#
# The repo-stats pipeline runs here because it needs this job's binary:
# `.#repo-stats-tools` is that build plus code-maat and its JRE, which stay
# out of the ci shell every other job enters.
# Its suite gates; the report on a push to main never fails the build, and
# its step fails only when improvise refuses the pipeline's input
# (context/code/tooling.md section 7). Full history is for the report.
nix-build:
name: Nix build (crate2nix)
name: Nix build (crate2nix) and the repo-stats pipeline
runs-on: [self-hosted, nixos, microvm]
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: nix build .#improvise
run: nix build --print-build-logs .#improvise
- name: The repo-stats pipeline computes what it claims to
run: nix shell .#repo-stats-tools --command ./scripts/checkers.sh --ci nix-build
- name: Churn, coupling and doc-growth report (report only)
if: github.event_name == 'push'
continue-on-error: true
run: |
rc=0
nix shell .#repo-stats-tools --command ./scripts/repo-stats --out repo-stats > repo-stats.txt || rc=$?
{ echo '```'; cat repo-stats.txt; echo '```'; } >> "$GITHUB_STEP_SUMMARY"
exit "$rc"
- name: Upload the repo-stats model and its CSVs
if: github.event_name == 'push'
uses: actions/upload-artifact@v7
with:
name: repo-stats
path: repo-stats
retention-days: 30
wasm:
name: wasm32 check (browser bundle crates)
.github/workflows/cloud-agent-smoke.yml
+70
View File
@@ -0,0 +1,70 @@
name: Cloud agent smoke
on:
pull_request:
paths:
- ".claude/**"
- ".cursor/**"
- "scripts/agent-bootstrap.sh"
- "scripts/nix-seed-flake-inputs.sh"
- ".github/workflows/cloud-agent-smoke.yml"
jobs:
claude:
name: Claude entry point + bead
runs-on: [self-hosted, nixos, microvm]
steps:
- uses: actions/checkout@v7
- name: Bootstrap through Claude session-start hook
shell: bash
env:
CLAUDE_CODE_REMOTE: "true"
run: |
if ! ./.claude/hooks/session-start.sh; then
echo "=== /tmp/nix-daemon.log ==="
cat /tmp/nix-daemon.log 2>/dev/null || true
echo "=== /tmp/nix-seed.log ==="
cat /tmp/nix-seed.log 2>/dev/null || true
echo "=== /tmp/bd-install.log ==="
cat /tmp/bd-install.log 2>/dev/null || true
exit 1
fi
- name: Create, read back, and close a smoke-test bead
shell: bash
run: |
set -euo pipefail
out="$(bd create "Cloud agent smoke: Claude entry point" --description="Ephemeral local-only bead used to verify the Claude cloud bootstrap entry point." --type=task --priority=4 --json)"
id="$(printf '%s' "$out" | jq -r '.id // .issue.id // empty')"
test -n "$id"
bd show "$id"
bd close "$id" --reason="Cloud-agent smoke test completed"
bd show "$id" | grep -qi closed
cursor:
name: Cursor install entry point + bead
runs-on: [self-hosted, nixos, microvm]
steps:
- uses: actions/checkout@v7
- name: Bootstrap through Cursor install command
shell: bash
run: |
nix-daemon >/tmp/nix-daemon.log 2>&1 &
if ! ./scripts/agent-bootstrap.sh; then
echo "=== /tmp/nix-daemon.log ==="
cat /tmp/nix-daemon.log 2>/dev/null || true
echo "=== /tmp/nix-seed.log ==="
cat /tmp/nix-seed.log 2>/dev/null || true
echo "=== /tmp/bd-install.log ==="
cat /tmp/bd-install.log 2>/dev/null || true
exit 1
fi
- name: Create, read back, and close a smoke-test bead
shell: bash
run: |
set -euo pipefail
out="$(bd create "Cloud agent smoke: Cursor entry point" --description="Ephemeral local-only bead used to verify the Cursor cloud bootstrap entry point." --type=task --priority=4 --json)"
id="$(printf '%s' "$out" | jq -r '.id // .issue.id // empty')"
test -n "$id"
bd show "$id"
bd close "$id" --reason="Cloud-agent smoke test completed"
bd show "$id" | grep -qi closed
.gitignore
+5
View File
@@ -61,6 +61,11 @@ GSYMS
__pycache__/
*.pyc
# The assembled site, written by web/assemble.sh (and the doc generator's
# default OUT_DIR). Build output: CI assembles it from source (improvise-1xfb.4).
/site/
# The release web capture, written to the repo root by
# scripts/build-web-release.sh (improvise-oujb).
/improvise-web.tar.gz
.opencode
AGENTS.md
+85 -2
View File
@@ -13,8 +13,10 @@
force-push. The rule, the message convention and the incidents are
`.agents/skills/wave/agent-rules.md`, rule 8.
- always start responses with bananaS!
- there is a prolog MCP available: load a session from `kb` at the root of the repository at the beginning of
every session, and query it as you work. **Reading only** — the MCP must never write `kb.pl`.
- **Knowledge base: use `scripts/kb`, not Prolog MCP.** The Prolog MCP is optional and not
available in all agent stacks (Cursor cloud agents do not have it). All agents must use
`scripts/kb add|replace|rm|query|check` for KB operations — see the `kb` skill at
`.agents/skills/kb/SKILL.md`. Reading `prolog-sessions/kb.pl` directly is fine for queries.
- save structured facts as you go — work done, interesting locations, important decisions — with
`scripts/kb add` (see the `kb` skill). Write a fact when you learn it; there is no reason to batch.
- query the knowledge base as needed and use Prolog reasoning when it makes sense.
@@ -25,6 +27,87 @@
This project uses **bd** (beads) for issue tracking. Run `bd prime` for full workflow context.
## Multi-Stack Agent Support
This repo supports multiple coding-agent stacks: Claude Code, Cursor (IDE and cloud), and
potentially others. All stacks share the same operational setup:
| Component | Location | Notes |
|---|---|---|
| Toolchain | `nix develop` / `direnv` | **Required.** Rust from flake, wasm target, beads pin, trealla |
| Skills | `.agents/skills/` | Canonical source; `.claude/skills/` and `.cursor/skills/` symlink here |
| Session bootstrap | `scripts/agent-bootstrap.sh` | Seeds nix inputs, installs bd, syncs tracker, warms cargo |
| Knowledge base | `scripts/kb` | **All stacks use this**, not Prolog MCP |
| Issue tracker | `bd` CLI | Synced from Dolt remote at session start |
### Toolchain Requirement
**Agents must work inside `nix develop` (or direnv) for this repo.** Bare system cargo that
fails `edition2024` or missing wasm target is a misconfigured session. The bootstrap seeds
nix flake inputs automatically (GitHub tarballs are blocked in cloud), so `nix develop`
should work after install completes.
Run commands inside the dev shell:
```bash
nix develop --command cargo test
nix develop --command cargo build --workspace
```
Or enter an interactive shell: `nix develop`
**Claude Code**: `.claude/hooks/session-start.sh` calls the bootstrap automatically.
**Cursor cloud**: `.cursor/environment.json` runs the bootstrap via `install`. Skills are
discovered from `.cursor/skills/` (symlinks to `.agents/skills/`).
**Local agents**: Run `scripts/agent-bootstrap.sh` manually, or use `direnv allow` for
automatic dev shell activation.
See `context/cloud-agents.md` for environment-specific notes (nix, network, etc.).
### GitHub Attribution (Hard Rule)
**Every GitHub write (PR, comment, review reply) must include an attribution line identifying
the AI agent.** Posts go out under the repository owner's account, and an unattributed post
reads as the human owner's words to everyone who sees it.
**Format:** Open the body with an italicized attribution line:
```
_Posted by <Agent Stack> (<model>)_
```
Examples:
- `_Posted by Claude (Opus 4.5)_`
- `_Posted by Cursor Agent (claude-sonnet-5)_`
- `_Posted by Claude Code_`
For PR bodies, the `🤖 Generated with …` footer that some tools add covers this. For
comments and review replies, add the line yourself — there is no automatic footer.
**This applies to:** PR creation, PR updates, issue comments, review comments, and review
replies. A thread is not "answered" until the reply carries this attribution.
### PR Tracking: Checks AND Reviews
For any PR an agent owns or is working on, **track both CI checks and GitHub comments/reviews**.
The `pr-watch` skill (`.agents/skills/pr-watch/SKILL.md`) is the agent-facing recipe.
**`scripts/pr-watch`** now prints reviews after checks are green (use `--no-reviews` to skip).
Reviews are a separate API surface from comments — Codex/ChatGPT reviews arrive as *reviews*,
not comments, and are easy to miss if you only read the comment thread.
1. **Run `scripts/pr-watch <n>`** — waits for checks, then prints all reviews/comments
2. **Read all review feedback** — reviews against an earlier head may still apply
3. **Answer substantive feedback with attribution** before merging (see format above)
4. **Re-check after each push** that new reviews target the current head SHA
5. **Don't wait indefinitely on Copilot/Codex** — reviews arrive as quota allows
To fetch reviews directly: `./scripts/pr-reviews <n>` — **do not pipe it** (exit code trap).
See the `pr-watch` skill for the full recipe, and "Shipping a wave" below for Copilot
request mechanics and how to answer declined findings.
## Quick Reference
```bash
Cargo.lock
Generated
+37 -29
View File
@@ -854,7 +854,7 @@ dependencies = [
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.20",
"thiserror 2.0.21",
"unicode-width",
]
@@ -886,6 +886,7 @@ dependencies = [
"serde",
"serde_json",
"siphasher",
"unicase",
]
[[package]]
@@ -1044,9 +1045,9 @@ checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.105"
version = "0.3.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e"
checksum = "7883d941dae510fb2d978fc3fe018c71c9e2892fd38854de3e8b92c2e5ad9cc5"
dependencies = [
"cfg-if",
"futures-util",
@@ -1061,7 +1062,7 @@ checksum = "bde5057d6143cc94e861d90f591b9303d6716c6b9602309150bd068853c10899"
dependencies = [
"hashbrown 0.16.1",
"portable-atomic",
"thiserror 2.0.20",
"thiserror 2.0.21",
]
[[package]]
@@ -1605,7 +1606,7 @@ dependencies = [
"palette",
"serde",
"strum",
"thiserror 2.0.20",
"thiserror 2.0.21",
"unicode-segmentation",
"unicode-truncate",
"unicode-width",
@@ -1892,9 +1893,9 @@ checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.16.1"
version = "1.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba467056f1b547ed52077911161fc86985becbc60e8e1857c8a144dab0def891"
checksum = "f9395f0f0eee849a9b707b2f06bb92a6a422090e2123bb2ef8e87a0e61892a8e"
[[package]]
name = "socket2"
@@ -2072,11 +2073,11 @@ dependencies = [
[[package]]
name = "thiserror"
version = "2.0.20"
version = "2.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f"
checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
dependencies = [
"thiserror-impl 2.0.20",
"thiserror-impl 2.0.21",
]
[[package]]
@@ -2092,9 +2093,9 @@ dependencies = [
[[package]]
name = "thiserror-impl"
version = "2.0.20"
version = "2.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
dependencies = [
"proc-macro2",
"quote",
@@ -2173,7 +2174,7 @@ dependencies = [
"log",
"rand 0.10.3",
"sha1",
"thiserror 2.0.20",
"thiserror 2.0.21",
]
[[package]]
@@ -2194,6 +2195,12 @@ version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94"
[[package]]
name = "unicase"
version = "2.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142"
[[package]]
name = "unicode-ident"
version = "1.0.26"
@@ -2282,9 +2289,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen"
version = "0.2.128"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf"
checksum = "9bb54f33acc68fd454578d9820b0bde1a1a3d17aa17bb7b6595806d02886d409"
dependencies = [
"cfg-if",
"once_cell",
@@ -2295,19 +2302,20 @@ dependencies = [
[[package]]
name = "wasm-bindgen-futures"
version = "0.4.78"
version = "0.4.79"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ef4c5d3d2cdf5c54f4231181768f5510842e350db025faf1f7163b1030ed928"
checksum = "3cbab34de2d982e9b48e18d216d04c4a6f641066ff19ffb699980f591ee3610e"
dependencies = [
"js-sys",
"tokio",
"wasm-bindgen",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.128"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed"
checksum = "2e29d0c35b16e224a7eeb5cd2d25e3e1968fbd65604117b44d3b789d00ee8535"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -2315,9 +2323,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.128"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a"
checksum = "6f501a8bc3719dba86ef8ae4728879c08001bea749eb1333ac5b91e040e2a6b7"
dependencies = [
"bumpalo",
"proc-macro2",
@@ -2328,18 +2336,18 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.128"
version = "0.2.129"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e"
checksum = "23f0c9c52aa7cd7d77769a4cfe2a9adb1b331f489a41d912ce14513d5ab995c6"
dependencies = [
"unicode-ident",
]
[[package]]
name = "web-sys"
version = "0.3.105"
version = "0.3.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fbddc4a036f00ec4f18c83445bd3115cb306a91da554919a099d9222fe4a7f8"
checksum = "88261b9deccee56594c11a3460c462c41f58d148598fe70ad77070126a68aba4"
dependencies = [
"js-sys",
"wasm-bindgen",
@@ -2525,18 +2533,18 @@ checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
[[package]]
name = "zerocopy"
version = "0.8.57"
version = "0.8.58"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d35102a9f36d089ccae9e4c6802bc118be4487b80aaffc0ab4e0cf5ce92d2873"
checksum = "c17e8fafad82b542ff3717217ecdc736231b59e387768c9630123b4ce4d2db44"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.57"
version = "0.8.58"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "146c01f5ab44258da43cf276c74a2763db2ff3969c9c652c3f2de07041d0b2bc"
checksum = "595f56e044df4f46a0c9a626f65c3d99eb8488f7e8a8baa12dd76326d9710bf2"
dependencies = [
"proc-macro2",
"quote",
Cargo.nix
+67 -38
View File
@@ -2657,7 +2657,7 @@ rec {
}
{
name = "thiserror";
packageId = "thiserror 2.0.20";
packageId = "thiserror 2.0.21";
}
{
name = "unicode-width";
@@ -2668,6 +2668,7 @@ rec {
{
name = "improvise-protocol";
packageId = "improvise-protocol";
features = [ "test-support" ];
}
{
name = "improvise-wasm-client";
@@ -2751,6 +2752,11 @@ rec {
}
];
devDependencies = [
{
name = "improvise-core";
packageId = "improvise-core";
features = [ "test-support" ];
}
{
name = "improvise-protocol";
packageId = "improvise-protocol";
@@ -2791,6 +2797,10 @@ rec {
name = "siphasher";
packageId = "siphasher";
}
{
name = "unicase";
packageId = "unicase";
}
];
devDependencies = [
{
@@ -2802,7 +2812,9 @@ rec {
packageId = "serde_json";
}
];
features = {
};
resolvedDefaultFeatures = [ "test-support" ];
};
"improvise-formula" = rec {
crateName = "improvise-formula";
@@ -3032,6 +3044,11 @@ rec {
name = "improvise-core";
packageId = "improvise-core";
}
{
name = "improvise-protocol";
packageId = "improvise-protocol";
features = [ "test-support" ];
}
{
name = "improvise-ws-server";
packageId = "improvise-ws-server";
@@ -3293,9 +3310,9 @@ rec {
};
"js-sys" = rec {
crateName = "js-sys";
version = "0.3.105";
version = "0.3.106";
edition = "2021";
sha256 = "17pr58p55pxbflkjl2wz6pyz854j80nlgavjq4mcwr583q6x4myf";
sha256 = "1icwmpjw54lb7vg5926k5y4y5jbiih0zxhwgjwnzn475v90xk0vq";
libName = "js_sys";
authors = [
"The wasm-bindgen Developers"
@@ -3347,7 +3364,7 @@ rec {
}
{
name = "thiserror";
packageId = "thiserror 2.0.20";
packageId = "thiserror 2.0.21";
usesDefaultFeatures = false;
}
];
@@ -4896,7 +4913,7 @@ rec {
}
{
name = "thiserror";
packageId = "thiserror 2.0.20";
packageId = "thiserror 2.0.21";
usesDefaultFeatures = false;
}
{
@@ -5877,9 +5894,9 @@ rec {
};
"smallvec" = rec {
crateName = "smallvec";
version = "1.16.1";
version = "1.16.2";
edition = "2018";
sha256 = "14gqvsqdli51r1bii3hfqv5vx1b9r0gic4br0x9fsixmy5b70ims";
sha256 = "13iai5hhwyp8z0pbn8r11q4j5956jaxhcbvvf2drm17f1q7myfgr";
authors = [
"The Servo Project Developers"
];
@@ -6413,18 +6430,18 @@ rec {
];
};
"thiserror 2.0.20" = rec {
"thiserror 2.0.21" = rec {
crateName = "thiserror";
version = "2.0.20";
version = "2.0.21";
edition = "2021";
sha256 = "0kxs6p295jffxhzaxpxv1dwaaf5iqlm6sx8h0djp6ancbxgj71pc";
sha256 = "17hq1lh5dyr3bkc7zzjrbrp4qgkvhc48kgq1n5fdxkindaw2rr89";
authors = [
"David Tolnay <dtolnay@gmail.com>"
];
dependencies = [
{
name = "thiserror-impl";
packageId = "thiserror-impl 2.0.20";
packageId = "thiserror-impl 2.0.21";
}
];
features = {
@@ -6458,11 +6475,11 @@ rec {
];
};
"thiserror-impl 2.0.20" = rec {
"thiserror-impl 2.0.21" = rec {
crateName = "thiserror-impl";
version = "2.0.20";
version = "2.0.21";
edition = "2021";
sha256 = "1bwjc94gi0xn5jz26h1a8bjj1wdkvvr6jifamyc4mp9n28zcs15w";
sha256 = "0945n8agp7kg6n6b35yyjb4g5xv2q22vrw15h6jj1nw76a99flgy";
procMacro = true;
libName = "thiserror_impl";
authors = [
@@ -6812,7 +6829,7 @@ rec {
}
{
name = "thiserror";
packageId = "thiserror 2.0.20";
packageId = "thiserror 2.0.21";
}
];
devDependencies = [
@@ -6874,6 +6891,17 @@ rec {
sha256 = "154smf048k84prsdgh09nkm2n0w0336v84jd4zikyn6v6jrqbspa";
};
"unicase" = rec {
crateName = "unicase";
version = "2.9.0";
edition = "2018";
sha256 = "0hh1wrfd7807mfph2q67jsxqgw8hm82xg2fb8ln8cvblkwxbri6v";
authors = [
"Sean McArthur <sean@seanmonstar.com>"
];
features = {
};
};
"unicode-ident" = rec {
crateName = "unicode-ident";
version = "1.0.26";
@@ -7114,9 +7142,9 @@ rec {
};
"wasm-bindgen" = rec {
crateName = "wasm-bindgen";
version = "0.2.128";
version = "0.2.129";
edition = "2021";
sha256 = "1gsi4ggm03dqr3j9bcc1sbxzmkw6ykm3cqx4g8xmw31v7niqgjxf";
sha256 = "02flhqld01jqb6vbfyx1gb8s78g1pnq2164daxad93y6mhrlzdcv";
libName = "wasm_bindgen";
authors = [
"The wasm-bindgen Developers"
@@ -7147,12 +7175,6 @@ rec {
rename = "rustversion-compat";
}
];
devDependencies = [
{
name = "once_cell";
packageId = "once_cell";
}
];
features = {
"default" = [ "std" ];
"enable-interning" = [ "std" ];
@@ -7165,9 +7187,9 @@ rec {
};
"wasm-bindgen-futures" = rec {
crateName = "wasm-bindgen-futures";
version = "0.4.78";
version = "0.4.79";
edition = "2021";
sha256 = "0a6r1q1v2qvi3ypmy0nva3il422iyml1f61i897wbxfdsb9wbx3f";
sha256 = "03k1wcg5j3wqk6vgy6gzcq868vsa9k81dlhqissfk0nrw96v7fiw";
libName = "wasm_bindgen_futures";
authors = [
"The wasm-bindgen Developers"
@@ -7178,6 +7200,13 @@ rec {
packageId = "js-sys";
usesDefaultFeatures = false;
}
{
name = "tokio";
packageId = "tokio";
usesDefaultFeatures = false;
target = { target, features }: (("emscripten" == target."os" or null) && (target."wasm_bindgen_unstable_tokio" or false));
features = [ "rt" "time" ];
}
{
name = "wasm-bindgen";
packageId = "wasm-bindgen";
@@ -7193,9 +7222,9 @@ rec {
};
"wasm-bindgen-macro" = rec {
crateName = "wasm-bindgen-macro";
version = "0.2.128";
version = "0.2.129";
edition = "2021";
sha256 = "1vdrjrb7yqh8p0r0yr93rlaq9iq05hy1wl9kbsjv7a61wc8xb456";
sha256 = "0dc5xq09sy1v9ns1fhb0cnyqz5p1wcjjvkdmxskj9qhnbg1x0a9f";
procMacro = true;
libName = "wasm_bindgen_macro";
authors = [
@@ -7217,9 +7246,9 @@ rec {
};
"wasm-bindgen-macro-support" = rec {
crateName = "wasm-bindgen-macro-support";
version = "0.2.128";
version = "0.2.129";
edition = "2021";
sha256 = "12p4jm9dr88h1j5sglycizvvxvr05pgmz7aa2v9g47h7y23lh7j1";
sha256 = "1dx6w90f14avmhri7ss9lyz03060g6475r4axy3bm7biqf5ill3g";
libName = "wasm_bindgen_macro_support";
authors = [
"The wasm-bindgen Developers"
@@ -7253,10 +7282,10 @@ rec {
};
"wasm-bindgen-shared" = rec {
crateName = "wasm-bindgen-shared";
version = "0.2.128";
version = "0.2.129";
edition = "2021";
links = "wasm_bindgen";
sha256 = "0ghh8hqx038h9c9zj1rlmzlv2k2shh917q756dn054hcipbir541";
sha256 = "1ilmp5d3sl8lrq9djhcs90gk66yvk8mgwk4sfrvpvkd75b2wkw13";
libName = "wasm_bindgen_shared";
authors = [
"The wasm-bindgen Developers"
@@ -7271,9 +7300,9 @@ rec {
};
"web-sys" = rec {
crateName = "web-sys";
version = "0.3.105";
version = "0.3.106";
edition = "2021";
sha256 = "1y57whpj5ncrl0cljmfsj5m31jqm66ylad68317yq03g0d5drgcz";
sha256 = "195bd1m14w3hsw5fg3sr938mh7y4cb260d0sq6a6brffxjfin9l8";
libName = "web_sys";
authors = [
"The wasm-bindgen Developers"
@@ -8440,9 +8469,9 @@ rec {
};
"zerocopy" = rec {
crateName = "zerocopy";
version = "0.8.57";
version = "0.8.58";
edition = "2021";
sha256 = "0wr85plmrkz0nh5grbqap23l9ghqq4mq1ip4x759q23dyflh4lfk";
sha256 = "0i6vsbj4qfqj62b8qxl7wdcin8rnqz6pw88p6zzl5dc2mnpqyzn1";
dependencies = [
{
name = "zerocopy-derive";
@@ -8472,9 +8501,9 @@ rec {
};
"zerocopy-derive" = rec {
crateName = "zerocopy-derive";
version = "0.8.57";
version = "0.8.58";
edition = "2021";
sha256 = "1g5js10p1q1d7wn6b74wjvrjznv34x5cfxpj7jj8s9a4mgsh2v0l";
sha256 = "1whbf7cjcqyp5nhvma78yy489swr7mfgc9m6r6h4ckyz8kh5cpsr";
procMacro = true;
libName = "zerocopy_derive";
dependencies = [
Cargo.toml
+5
View File
@@ -31,6 +31,11 @@ name = "reshape_observation"
harness = false
test = false
[[bench]]
name = "scale_bisect"
harness = false
test = false
[dependencies]
gp-tree = { path = "crates/gp-tree" }
improvise-core = { path = "crates/improvise-core" }
README.md
+11 -4
View File
@@ -40,9 +40,9 @@ improvise import examples/demo.csv \
--no-wizard \
--category Region --category Product --category Customer \
--measure Revenue --measure Cost \
--time Date --extract Date:Month \
--time Date --extract Date:Month --split-date Date \
--axis Region:row --axis Product:row \
--axis Date_Month:column --axis Measure:column \
--axis Date_Month:column --axis _Measure:column \
--axis Customer:page --axis Date:none \
--formula "Profit = Revenue - Cost" \
--name "Acme Sales Demo" \
@@ -102,7 +102,7 @@ rollback target — it is not what the published site is running.
Every cell lives at the intersection of named categories and items —
`(Region=East, Measure=Revenue)` — not at a grid address like `B3`. A model
can have up to 12 categories, each with an ordered list of items that can be
can have up to 24 categories, each with an ordered list of items that can be
organized into collapsible groups (e.g. months grouped into quarters).
Formulas reference dimension names, not cell addresses:
@@ -178,7 +178,14 @@ opening one fails with an error that says so. JSON *import* is unaffected.
The import wizard analyzes CSV columns and proposes each as a category
(string-valued), measure (numeric), time dimension (date-like, with optional
year/month/quarter extraction), or skip. Multiple CSVs merge automatically
with a synthetic "File" category.
with a synthetic "File" category, even when their headers differ. A record
simply has no value for a column its file lacks. A folder argument imports
every `*.csv` in it, in name order, even when it holds a single CSV. Each file
also gets its own view, filtered to that file and showing only its columns:
```sh
improvise import reports/ --no-wizard -o reports.improv
```
All model mutations go through a typed command registry, so the same
operations that back the TUI work headless:
bench/edge.py
+8
View File
@@ -0,0 +1,8 @@
import sys; sys.path.insert(0,'bench')
from scale_bisect import run
cfgs=[dict(k=k,prune=0) for k in (16,17,18,26,27)]+[dict(k=k,prune=1) for k in (7,11,12,13,14,21,22)]+[dict(k=1000,n=n,mode='records') for n in (1000,1200,1500,1800,2000,4600,14500)]
for c in cfgs:
c.setdefault('n',100000); c['ops']='open,key,window'
v=[run('/tmp/sb',c,timeout=200) for _ in range(3)]
f=lambda k:'/'.join(f"{r[k][1]:.1f}" if k in r else '-' for r in v)
print({k:c[k] for k in c if k!='ops'},'rows',v[0].get('rows'),'open',f('open'),'key',f('keypress_down'),'win',f('browser_grid_of_top'),'tuiframe',f('tui_frame_grid_of_whole'),flush=True)
bench/full.py
+10
View File
@@ -0,0 +1,10 @@
import sys, json; sys.path.insert(0,'bench')
from scale_bisect import run
cfgs=[dict(k=9,prune=0),dict(k=10,prune=0),dict(k=17,prune=0,groups=1),dict(k=26,prune=0,groups=1),
dict(k=8,prune=1),dict(k=13,prune=1,groups=1),
dict(k=1000,n=1300,mode='records'),dict(k=1000,n=4400,mode='records'),dict(k=1000,n=14000,mode='records')]
for c in cfgs:
c.setdefault('n',100000)
for rep in range(3):
r=run('/tmp/sb',c,timeout=300)
print(json.dumps({'cfg':c,'rep':rep,**r}),flush=True)
bench/scale_bisect.py
+38
View File
@@ -0,0 +1,38 @@
"""Drive benches/scale_bisect one configuration per process; parse its lines.
usage: scale_bisect.py BIN k=.. n=.. [cats=..] [prune=..] [mode=..] [ops=..] [--timeout S]
"""
import subprocess, sys, re, json
def run(binary, cfg, timeout=90):
args = [binary] + [f"{k}={v}" for k, v in cfg.items()]
try:
p = subprocess.run(args, capture_output=True, text=True, timeout=timeout)
out, status = p.stdout, "ok"
except subprocess.TimeoutExpired as e:
out, status = (e.stdout or b"").decode() if isinstance(e.stdout, bytes) else (e.stdout or ""), "timeout"
res = {"status": status}
for line in out.splitlines():
if line.startswith("REFUSED"):
res["status"] = "refused"
m = re.match(r"(\w+) min=([\d.]+) med=([\d.]+) max=([\d.]+)", line)
if m:
res[m[1]] = (float(m[2]), float(m[3]), float(m[4]))
for kv in re.findall(r"(\w+)=([\d.]+)", line):
if kv[0] in ("rows", "cols", "maxrss_mb", "improv_mb", "snapshot_kb", "rss_after_open_mb", "cells"):
res[kv[0]] = float(kv[1])
m = re.match(r"json_encode_setcell_updates ms=(.*)", line)
if m:
res["json_setcell"] = m[1]
return res
if __name__ == "__main__":
binary = sys.argv[1]
cfg, timeout = {}, 90
rest = sys.argv[2:]
for a in rest:
if a.startswith("--timeout"):
timeout = int(a.split("=")[1])
else:
k, v = a.split("=", 1)
cfg[k] = v
print(json.dumps(run(binary, cfg, timeout)))
bench/sweep.py
+10
View File
@@ -0,0 +1,10 @@
import sys; sys.path.insert(0,'bench')
from scale_bisect import run
import itertools
OPS=sys.argv[1];
cfgs=eval(sys.argv[2])
cols=sys.argv[3].split(',')
for c in cfgs:
c=dict(c); c['ops']=OPS
r=run('/tmp/sb',c,timeout=int(sys.argv[4]) if len(sys.argv)>4 else 90)
print(c, r['status'], 'rows',r.get('rows'), *[f"{k}={r[k][1]:.1f}" if k in r else f"{k}=-" for k in cols], f"rss={r.get('maxrss_mb')}", flush=True)
benches/grid_projection.rs
-6
View File
@@ -84,9 +84,6 @@ fn chain_app(regions: usize, products: usize) -> App {
v.set_axis("Region", Axis::Row);
v.set_axis("Product", Axis::Row);
v.set_axis("_Measure", Axis::Column);
for g in 0..regions.div_ceil(10) {
v.toggle_group_collapse("Region", &format!("G{g}"));
}
}
let mut n = 0.0f64;
for r in 0..regions {
@@ -127,9 +124,6 @@ fn grouped_app(days: usize, per_group: usize, products: usize, measures: usize)
v.set_axis("Day", Axis::Row);
v.set_axis("Product", Axis::Row);
v.set_axis("_Measure", Axis::Column);
for g in 0..days.div_ceil(per_group) {
v.toggle_group_collapse("Day", &format!("G{g}"));
}
}
let mut n = 0.0f64;
for d in 0..days {
benches/scale_bisect.rs
+347
View File
@@ -0,0 +1,347 @@
//! One configuration per process: build a sparse document, time the real paths.
//! `scale_bisect k=1000 n=100000 cats=3 prune=1 mode=grid`; see context/test-evidence/scale-bisect.md.
use std::collections::{HashMap, HashSet};
use std::io::Write;
use std::time::Instant;
use improvise::model::cell::{CellKey, CellValue};
use improvise::view::Axis;
use improvise::workbook::Workbook;
use improvise_command::ui::app::App;
use improvise_command::ui::effect::{self, ViewTarget};
use improvise_command::ui::host::Host;
use improvise_command::ui::project::{self, Viewport};
use improvise_command::ui::reduce::caught_up;
use improvise_io::persistence::{format_md, parse_md, save_to};
use improvise_io::storage::MemoryFs;
use improvise_protocol::{
ClientFrame, KeyCode, KeyEvent, KeyModifiers, ModelCommand, RecordHash, render::RecordCell,
};
fn lcg(s: &mut u64, n: usize) -> usize {
*s = s
.wrapping_mul(6_364_136_223_846_793_005)
.wrapping_add(1_442_695_040_888_963_407);
(*s >> 33) as usize % n
}
struct Cfg {
k: usize,
n: usize,
cats: usize,
prune: bool,
mode: String,
groups: bool,
ops: String,
}
fn arg(args: &HashMap<String, String>, k: &str, d: &str) -> String {
args.get(k).cloned().unwrap_or_else(|| d.to_string())
}
fn document(c: &Cfg) -> (Workbook, Vec<CellKey>) {
let mut wb = Workbook::new("bench");
for ci in 0..c.cats {
let cat = format!("C{ci}");
wb.add_category(cat.clone()).expect("under the limit");
for i in 0..c.k {
if c.groups && ci == 0 {
wb.add_item_in_group(&cat, format!("i{i}"), format!("G{}", i / 10))
.expect("added");
} else {
wb.add_item(&cat, format!("i{i}")).expect("added");
}
}
}
for m in ["Cost", "Rev"] {
wb.add_item("_Measure", m).expect("virtual");
}
let space = (c.k as f64).powi(c.cats as i32);
let wanted = c.n.min((space / 2.0).max(1.0) as usize);
let (mut s, mut seen, mut recs) = (0x9E37_79B9_7F4A_7C15u64, HashSet::new(), Vec::new());
while seen.len() < wanted {
let d: Vec<usize> = (0..c.cats).map(|_| lcg(&mut s, c.k)).collect();
if !seen.insert(d.clone()) {
continue;
}
let coords: Vec<(String, String)> = d
.iter()
.enumerate()
.map(|(ci, x)| (format!("C{ci}"), format!("i{x}")))
.collect();
recs.push(CellKey::new(coords.clone()));
for (m, v) in [("Cost", 3.0), ("Rev", 5.0)] {
let mut key = coords.clone();
key.push(("_Measure".to_string(), m.to_string()));
wb.model.set_cell(CellKey::new(key), CellValue::Number(v));
}
}
let view = wb.active_view_mut();
for ci in 0..c.cats {
let axis = if c.mode == "records" || c.mode == "none" {
Axis::None
} else {
Axis::Row
};
view.set_axis(&format!("C{ci}"), axis);
}
if c.mode == "records" {
view.set_axis("_Index", Axis::Row);
view.set_axis("_Dim", Axis::Column);
} else {
view.set_axis("_Measure", Axis::Column);
}
view.prune_empty = c.prune;
(wb, recs)
}
fn maxrss_mb() -> f64 {
let mut u: libc::rusage = unsafe { std::mem::zeroed() };
unsafe { libc::getrusage(libc::RUSAGE_SELF, &mut u) };
u.ru_maxrss as f64 / 1048576.0
}
fn rss_mb() -> f64 {
let out = std::process::Command::new("ps")
.args(["-o", "rss=", "-p", &std::process::id().to_string()])
.output()
.expect("ps");
String::from_utf8_lossy(&out.stdout)
.trim()
.parse::<f64>()
.unwrap_or(0.0)
/ 1024.0
}
/// Sample `f` up to `reps` times, stopping once `budget_ms` is spent (one sample minimum).
fn sample(name: &str, reps: usize, budget_ms: f64, mut f: impl FnMut(usize)) {
let mut v = Vec::new();
let start = Instant::now();
for r in 0..reps {
let t = Instant::now();
f(r);
v.push(t.elapsed().as_secs_f64() * 1000.0);
if start.elapsed().as_secs_f64() * 1000.0 > budget_ms {
break;
}
}
v.sort_by(|a, b| a.partial_cmp(b).expect("finite"));
println!(
"{name} min={:.3} med={:.3} max={:.3} reps={}",
v[0],
v[v.len() / 2],
v[v.len() - 1],
v.len()
);
std::io::stdout().flush().ok();
}
fn once<T>(name: &str, f: impl FnOnce() -> T) -> T {
let t = Instant::now();
let v = f();
println!(
"{name} min={0:.3} med={0:.3} max={0:.3} reps=1",
t.elapsed().as_secs_f64() * 1000.0
);
std::io::stdout().flush().ok();
v
}
fn key(code: KeyCode) -> KeyEvent {
KeyEvent::new(code, KeyModifiers::NONE)
}
fn main() {
let args: HashMap<String, String> = std::env::args()
.skip(1)
.filter_map(|a| {
a.split_once('=')
.map(|(k, v)| (k.to_string(), v.to_string()))
})
.collect();
let c = Cfg {
k: arg(&args, "k", "1000").parse().expect("k"),
n: arg(&args, "n", "100000").parse().expect("n"),
cats: arg(&args, "cats", "3").parse().expect("cats"),
prune: arg(&args, "prune", "1") == "1",
mode: arg(&args, "mode", "grid"),
groups: arg(&args, "groups", "0") == "1",
ops: arg(
&args,
"ops",
"open,key,frame,window,setcell,host,reshape,io",
),
};
let rows_cap: f64 = arg(&args, "rows_cap", "30000000").parse().expect("cap");
let product = (c.k as f64).powi(c.cats as i32);
if c.mode == "grid" && product > rows_cap {
println!(
"REFUSED product_rows={product:.3e} (> {rows_cap:.0}: layout would materialize it)"
);
return;
}
let want = |o: &str| c.ops.split(',').any(|x| x == o);
let (wb, recs) = document(&c);
println!(
"cells={} records={} base_rss_mb={:.0}",
wb.model.data.iter_cells().count(),
recs.len(),
rss_mb()
);
let mut app = once("open", || App::new(wb, None));
println!(
"rows={} cols={} rss_after_open_mb={:.0}",
app.layout.row_count(),
app.layout.col_count(),
rss_mb()
);
app.term_width = 200;
app.term_height = 50;
let name = app.view_name().to_string();
if want("key") {
sample("keypress_down", 8, 6000.0, |_| {
app.dispatch_key(key(KeyCode::Down));
});
}
if want("frame") {
let whole = |app: &App| Viewport {
row_offset: 0,
col_offset: 0,
rows: app.layout.row_count(),
cols: app.layout.col_count(),
};
sample("tui_frame_grid_of_whole", 3, 8000.0, |_| {
std::hint::black_box(project::grid_of(&app, whole(&app)));
});
let cache = project::grid_of(&app, whole(&app));
let area = ratatui::layout::Rect::new(0, 0, 200, 50);
sample("tui_render_gridwidget_buffer", 5, 4000.0, |_| {
let mut buf = ratatui::buffer::Buffer::empty(area);
let w = improvise::ui::grid::GridWidget::new(
&cache,
app.cursor(),
&app.view_state.active_view,
&app.layout,
&app.view_state.mode,
&app.view_state.search_query,
&app.view_state.buffers,
app.drill_state.as_ref(),
);
ratatui::widgets::Widget::render(w, area, &mut buf);
std::hint::black_box(buf);
});
}
if want("window") {
let mid = app.layout.row_count() / 2;
for (label, off) in [("top", 0), ("mid", mid)] {
let vp = Viewport {
row_offset: off,
col_offset: 0,
rows: 45,
cols: 12,
};
sample(&format!("browser_grid_of_{label}"), 5, 6000.0, |_| {
std::hint::black_box(project::grid_of(&app, vp));
});
}
}
if want("setcell") && !recs.is_empty() {
let existing = recs[recs.len() / 2].clone().with("_Measure", "Rev");
sample("setcell_existing", 5, 6000.0, |r| {
app.apply_effects(vec![Box::new(effect::SetCell(
existing.clone(),
CellValue::Number(10.0 + r as f64),
))]);
});
let mut s = 99u64;
sample("setcell_new_record", 5, 6000.0, |_| {
let mut coords: Vec<(String, String)> = (0..c.cats)
.map(|ci| (format!("C{ci}"), format!("i{}", lcg(&mut s, c.k))))
.collect();
coords.push(("_Measure".into(), "Rev".into()));
app.apply_effects(vec![Box::new(effect::SetCell(
CellKey::new(coords),
CellValue::Number(1.0),
))]);
});
}
if want("reshape") && c.mode == "grid" {
let v = || ViewTarget::Active;
sample("reshape_hide_item", 3, 6000.0, |r| {
app.apply_effects(vec![Box::new(effect::HideItem {
view: v(),
category: "C1".into(),
item: format!("i{r}"),
})]);
});
if c.groups {
sample("reshape_toggle_group", 3, 6000.0, |_| {
app.apply_effects(vec![Box::new(effect::ToggleGroup {
view: v(),
category: "C0".into(),
group: "G0".into(),
})]);
});
}
let last = format!("C{}", c.cats - 1);
sample("reshape_set_axis", 4, 6000.0, |r| {
let axis = if r % 2 == 0 { Axis::Page } else { Axis::Row };
app.apply_effects(vec![Box::new(effect::SetAxis {
view: v(),
category: last.clone(),
axis,
})]);
});
sample("reshape_toggle_prune", 2, 6000.0, |_| {
app.apply_effects(vec![Box::new(effect::TogglePruneEmpty(v()))]);
});
}
if want("io") {
let wbr = &app.model_state.workbook;
let text = once("format_md", || format_md(wbr));
println!("improv_mb={:.2}", text.len() as f64 / 1048576.0);
once("parse_md", || parse_md(&text).expect("parses"));
let fs = MemoryFs::new();
sample("persist_per_command_save_to", 3, 6000.0, |_| {
save_to(&fs, wbr, std::path::Path::new("d.improv")).expect("write");
});
}
if want("host") && !recs.is_empty() {
let mut host = Host::new(app);
let (id, snap) = once("host_connect_snapshot", || host.connect(45, 12));
sample("json_encode_snapshot", 3, 4000.0, |_| {
std::hint::black_box(serde_json::to_vec(&snap.msg).expect("json"));
});
println!(
"snapshot_kb={:.1}",
serde_json::to_vec(&snap.msg).expect("json").len() as f64 / 1024.0
);
let rec = recs[recs.len() / 2].clone();
let cell = RecordCell {
record: RecordHash::of(&rec),
measure: Some("Rev".into()),
};
let mut enc: Vec<(f64, usize)> = Vec::new();
sample("host_handle_setcell_and_project", 4, 8000.0, |r| {
let wire = caught_up(
host.app(),
ModelCommand::SetCell {
view: name.clone(),
cell: cell.clone(),
value: CellValue::Number(50.0 + r as f64),
},
);
let out = host.handle(id, ClientFrame::command(wire));
let t = Instant::now();
let bytes: usize = out
.iter()
.map(|a| serde_json::to_vec(&a.msg).expect("json").len())
.sum();
enc.push((t.elapsed().as_secs_f64() * 1000.0, bytes));
});
println!("json_encode_setcell_updates ms={:?}", enc);
}
println!("maxrss_mb={:.0}", maxrss_mb());
}
context/cloud-agents.md
+82 -13
View File
@@ -1,11 +1,30 @@
# Cloud Agents (LLM Reference)
Environment notes for agents running in Claude Code on the web. Local
checkouts are unaffected — the session hook exits early unless
Environment notes for agents running in cloud containers — Claude Code on the web,
Cursor cloud agents, and similar setups. Local checkouts are unaffected.
## Session Bootstrap
All agent stacks use the same bootstrap script: `scripts/agent-bootstrap.sh`. It:
1. **Seeds nix flake inputs** (GitHub tarballs are blocked; seeds over git instead)
2. Verifies the `bd` version matches the flake.nix pin
3. Installs `bd` if missing
4. Syncs the Dolt issue tracker from the git remote
5. Warms the cargo build **inside `nix develop`**
**After bootstrap, `nix develop` works.** This is the required toolchain — bare system
cargo that fails `edition2024` or missing wasm target is a misconfigured session.
**Claude Code**: `.claude/hooks/session-start.sh` calls the bootstrap automatically when
`CLAUDE_CODE_REMOTE=true`.
`.claude/hooks/session-start.sh` handles the setup automatically. This doc
is the "why", and the record of what bites when it doesn't run.
**Cursor cloud**: `.cursor/environment.json` runs the bootstrap via `install`. Skills are
in `.cursor/skills/` (symlinks to `.agents/skills/`).
**Other stacks**: Run `scripts/agent-bootstrap.sh` manually after checkout.
This doc is the "why", and the record of what bites when setup doesn't run.
---
@@ -60,7 +79,8 @@ same if you need to re-sync by hand; never run it in the working tree.
| Thing | Status | Use instead |
|---|---|---|
| `nix develop` **unseeded** | Fails: flake inputs are GitHub **tarballs**, which are not served here | Run `scripts/nix-seed-flake-inputs.sh` first — see below. Or just use the preinstalled `cargo`/`rustc`, the same stable toolchain the flake pins |
| `nix develop` **without bootstrap** | Fails: flake inputs are GitHub **tarballs**, which are blocked | Bootstrap seeds inputs automatically; if it failed, run `scripts/nix-seed-flake-inputs.sh` manually |
| Bare system cargo | Fails: `edition2024`, missing wasm target | **Always use `nix develop`** — this is the required toolchain |
| `bd dolt push` | Fails: 403 outbound | Nothing; bead status changes stay local. Say so rather than implying the tracker was updated |
| `git+ssh://` Dolt remote in `config.yaml` | No ssh client in the container | `git+https://<origin>`, which the hook passes explicitly |
| `git push`, GitHub API | **Work normally** | — |
@@ -70,13 +90,13 @@ starting by hand). Reads of public GitHub repos over the git protocol work —
it is specifically the `/archive/*.tar.gz` and `codeload` endpoints that are
refused.
### Making `nix develop` work
### How nix develop works after bootstrap
`scripts/nix-seed-flake-inputs.sh` makes the devShell work **without touching
`flake.nix` or `flake.lock`**. Verified end to end: `rustc 1.98.1`, `cargo`,
`crate2nix` in the shell, `git status` clean.
The bootstrap runs `scripts/nix-seed-flake-inputs.sh` automatically, which seeds
the flake inputs over git **without touching `flake.nix` or `flake.lock`**.
After bootstrap, `nix develop` should work. If it fails, check `/tmp/nix-seed.log`.
Why it is needed, and why the obvious fixes are not available:
Why the seeding is needed:
- Nix's `github:` fetcher wants a tarball. GitHub traffic **bypasses the
environment's domain allowlist entirely** and goes through a proxy that
@@ -105,9 +125,9 @@ around it — a mismatch fails loudly. Do not instead rewrite the lock's nodes
to `git` type: carrying the old `narHash` onto a different fetcher is exactly
the claim the lock exists to make.
It is not in the session-start hook because it is expensive (minutes, plus
several GB of store once the devShell builds) and most work does not need the
devShell. Run it when you do.
The bootstrap runs this automatically. It takes a few minutes and several GB of
store once the devShell builds. If bootstrap reported a seed failure, check
`/tmp/nix-seed.log` and re-run `scripts/nix-seed-flake-inputs.sh` manually.
Two alternatives if per-container seeding is unattractive: push the inputs to
a binary cache with `nix flake archive` and point sessions at it with
@@ -143,3 +163,52 @@ environment, where egress leaves through your own network boundary.
reorder or fold commits (`.agents/skills/wave/agent-rules.md`, rule 8).
- Keep `context/repo-map.md` and `context/design-principles.md` current —
they are the two docs agents are expected to maintain by hand.
---
## PR Tracking
For any PR you own, **track both CI checks and GitHub comments/reviews**:
- **`scripts/pr-watch <n>`**: Waits for checks, then prints all reviews/comments (default).
Reviews are a separate API surface — Codex/ChatGPT reviews arrive as *reviews* not comments.
- **Answer feedback with attribution** before marking work done (see below).
- **Re-check after push** that reviews target the current head SHA.
The `pr-watch` skill (`.agents/skills/pr-watch/SKILL.md`) is the agent-facing recipe.
See AGENTS.md "PR Tracking: Checks AND Reviews" and "Shipping a wave" for the full workflow.
---
## GitHub Attribution (Hard Rule)
**Every GitHub write (PR, comment, review reply) must include an attribution line.**
Posts go out under the repository owner's account — an unattributed post reads as the
human's words.
Open the body with: `_Posted by <Agent Stack> (<model>)_`
Examples:
- `_Posted by Claude (Opus 4.5)_`
- `_Posted by Cursor Agent (claude-sonnet-5)_`
A thread is not "answered" until the reply carries this attribution. See AGENTS.md
"GitHub Attribution" for the full rule.
---
## Knowledge Base
**Use `scripts/kb`, not Prolog MCP.** The Prolog MCP is not available in all agent stacks
(Cursor cloud agents do not have it). All agents must use `scripts/kb` for KB operations:
```bash
scripts/kb check # kb_consistent, exactly what CI runs
scripts/kb query '<goal>' # run a goal; exit 1 if it fails
scripts/kb add <delta.pl> # add clauses; refuses if a key already exists
scripts/kb replace <delta.pl> # add, overwriting existing keys
scripts/kb rm <keys.pl> # remove by key; refuses if a key is absent
```
Reading `prolog-sessions/kb.pl` directly for queries is fine. See the `kb` skill at
`.agents/skills/kb/SKILL.md` for the full interface.
context/code/cli.md
+169
View File
@@ -369,3 +369,172 @@ non-zero and writes nothing. It is checked before the build rather than read
off the build's error, so the listing is structured data and not a parsed
sentence. `separate` uses `ImportPipeline::suggested_duplicate_category`, and
there is deliberately no flag to name the category yet.
## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
`improvise import` takes any mix of files and folders. A folder stands for its
`*.csv` files, in name order. A folder always merges, even when it holds a
single CSV, so that file keeps its `File` item and view. A single file named
directly still imports alone (the user's ruling on the PR). The rules
and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
one sparse model, with a view per file". The CLI's part is the one question
`Storage` cannot answer: whether a path is a folder. `get_import_data` asks
`Path::is_dir` for each argument, hands the resulting `ImportPath`s to
`expand_import_paths` over `PhysicalFs`, and from there takes the path a list
of files always took. A folder with no CSV in it is refused before anything is
read. The message names the folder, and the run exits non-zero and writes
nothing.
A merge now reaches the build as an `ImportPipeline` carrying the merged files'
`sources`, rather than as a bare JSON value. Both `--no-wizard` and the TUI
wizard (`run_tui` → `App::start_import`) therefore build the per-file views.
The headless path matters most, because the code-maat pipeline
(improvise-1xfb.5.1) runs there. `a_headless_import_of_a_folder_merges_its_csv_files_and_repeats_byte_for_byte`
pins the views, the sparse Data lines and the byte-identical second run, with
the first run's output sitting in the folder it reads.
## 13 — a headless import refuses rather than saving less than was asked (improvise-8jk6.4, 8jk6.5, 8jk6.6)
The user treats the headless command and model layer as a product of its own,
and the scripted `import --no-wizard` is where it meets a pipeline. The
repo-stats pipeline (improvise-1xfb.5.1) found three ways that import saved
something other than what the flags asked for, and still exited 0. The
reasoning for each is in `context/code/formula-and-io.md`, "A CSV field is
text, and the field's role reads it" and "An import refuses what it cannot
keep". The CLI's contract:
- **`--category` keeps the CSV text.** A forced category's items are the
file's text, verbatim: `00565796` stays `00565796`, and `220944e2` stays
`220944e2`. Measures still read as numbers. A JSON number has no source
text, so it is named as JSON writes it (`1e3` is `1000.0`).
- **An unparseable `--formula` is a refusal.** It exits non-zero and writes
nothing. Every bad formula is named on stderr, each with its parse error.
This is §10's rule: the request was not met, so the exit code says so. The
wizard refuses the same text at the formula step, with the same wording.
- **An import that kept no record is a refusal.** It exits non-zero, writes
nothing, and says why: every record was skipped as not a JSON object, or
the source had none.
- **An import that skipped *some* records exits 0 by default.** It writes the
model and warns with a count. With `--strict` (the user's ruling on PR #321)
any skipped record is a refusal instead: exit 1, the same count named, and
nothing written. The argument is in `formula-and-io.md`.
Two refusals bail in `run_headless_import` before the build: the `--strict`
skip check (`src/main.rs:305`) and the `--on-duplicate refuse` collision
check. The rest come from `build_model_with_warnings` as an `Err`. All of
them reach `main` as exit 1 before `persistence::save`. So "writes nothing"
holds by statement order, not by where the refusal is raised. The headless
tests check it by asserting that the output path does not exist.
## 14 — the command language reads `|pipe quoting|`, and a surplus argument is refused (improvise-31oe.3)
`create-view |By area|` used to make a view named `|By` and drop `area|` without
a word. Two separate defects, fixed separately.
**The quoting.** `parse.rs` now has one lexer, `lex`, and everything else is
built on it: `parse_line_with`, `segments` and `parse_segment_interactive`. The
rules are:
- A `|…|` piece is read by `improvise_formula::quote::unescape_pipe_body`, the
one home of pipe quoting (gotcha 34). So `\|`, `\\` and `\n` mean exactly what
they mean in `.improv` and in formulas. The lexer only finds the closing pipe:
a backslash takes the next character with it, so `\|` does not close.
- A piece may sit **anywhere in a word**, and the pieces and bare text are
joined. So a coordinate quotes each half, `|Sales Region|/|East Coast|`, the
way a `.improv` data line quotes each name. The alternative, a pipe only at
the start of a word, would have left no way to spell a coordinate with a
space in it.
- `"…"` is **kept**, verbatim, with no escapes, and a `|` inside it stays a
`|`. It is not a workaround to retire. Formula text carries its own pipes:
`add-formula "Net = |Gross Rev| - Cost"` must reach `parse_formula` with the
pipes on. If the line had pipes only, every pipe in a formula would need
escaping. Its old leniency is kept too: a `"` that is never closed runs to
the end of the line. A `|` that is never closed is refused, naming the text
from the open pipe on, because a name is never meant to run to the end.
- The separator is a **bare** `.` word. A quoted `|.|` or `"."` is an argument,
and a `.` inside `|a . b|` does not split the line. The old `split_on_dot`
was a second scanner that knew double quotes and not pipes. With one lexer
there is no second scanner to disagree.
- A comment line (`#`, `//`) is not lexed, so an unmatched pipe in a comment is
not an error.
`segments` now returns `Result`, because an unterminated pipe is a fact about
the whole line. `main.rs` refuses the line before running any of it, as §7 does
for a parse error, and `ExecuteCommand` refuses it on the status line.
What is still not spellable: an item whose name contains `/`. The coordinate is
split on the first `/` **after** unquoting, so `|a/b|/x` is still read as
category `a` and item `b/x`. Closing that needs the lexer to hand structured
coordinates to the commands, which is a larger change than this ticket.
**The arity.** Every registration now declares `MaxArgs::AtMost(n)` or
`MaxArgs::Any` beside its prototype. `CmdRegistry::parse` and
`CmdRegistry::interactive` apply it after construction succeeds. So the rule is
written once and every command gets it, and a new registration cannot compile
without a declaration. The refusal names the command, the limit and every
surplus argument (`create-view: takes at most 1 argument(s), got 2; unexpected
"area"`). A headless run exits non-zero through the §7 parse gate, and the `:`
line refuses on the status line.
It runs **after** the command's own parse, deliberately. A command whose own
refusal states its shape keeps it: `add-formula`'s "quote the formula"
(improvise-z05h) is more use than an arity, and it is what `formula Product
Total = Widget + Gadget` still says.
`MaxArgs::Any` is a claim, and the list of commands making it is short:
`add-items`, `set-cell`, `set-format` (it joins its words), `clear-cell`,
`yank`, `paste`, `drill-into-cell` and `commit-cell-edit`. Each takes a list of
items or coordinates. `no_command_silently_drops_an_argument_it_was_given` names
that list, so widening another command to `Any` has to add it there.
## 15 — a command that names something the document lacks is refused (improvise-31oe.2)
`set-axis Regoin row`, `set-page Region Nowhere` and `set-axis Region sideways`
all exited 0 and said nothing. §10's rule applies: nothing changed, so the
request was not met.
- **An unreadable axis is an argument error.** `set-axis` now checks its axis
with `parse_axis` when it is parsed, so a script is refused before any of it
runs. `execute` still refuses an axis it cannot read rather than returning
nothing. That arm cannot be reached through the registry.
- **An unknown category, item, group or view is a refusal**, from `RequireKnown`
(below), because only the document knows what it holds. `effect::unknown_name`
asks the active view (`try_axis_of`) for a category. That is exactly the
condition under which `View::set_axis` was a silent no-op. It asks
`effective_item_names` for an item, so a formula target is a known `_Measure`
item. It asks both an item's own group and the category's `groups` for a
group. `add_item_in_group` registers no `Group`, and a parent group has no
items, so either one alone misses a real group. A view is asked of
`workbook.views`.
The siblings fixed with it: `hide-item` and `show-item` (hiding an unknown item
wrote it into `hidden_items`), `toggle-group`, and `switch-view` (the effect
ignores a view it lacks, and says nothing). `set-page` with an unknown item was
worse than a no-op: it wrote the name into `page_selections`.
The check runs at **apply** time, as an effect of its own. `execute` runs
against one context built before the line ran, and a `:` line builds every
segment up front, so a check in `execute` refused
`create-view Second . switch-view Second` and
`add-item Type Tools . set-page Type Tools` after their first halves had landed,
while `improvise cmd` (which rebuilds the context per segment) accepted them.
Each of these commands therefore returns `RequireKnown(needs)` followed by the
real effect. `RequireKnown` refuses and aborts the batch (gotcha 21), so the
real effect never applies. It is a `ViewOnly` effect that writes nothing and is
off the wire on purpose: the wire's own effects (`model_effects`, gotcha 38)
stay non-refusing, since a refusing effect there becomes a frame rejection
(`MutationRefused`). That is arguably right, but it changes the protocol, and
a browser composes these only from names it was sent. It is left for a ticket
of its own.
A refused name stops the rest of a `:` line, because the abort ends the one batch
the whole line runs as. Headless runs each segment as its own batch, so an abort
never crosses a `.` there: `:set-page Type Nowhere . add-item Type X` adds
nothing, while `improvise cmd` on the same line exits 1 and still adds `X`.
`load`/`Save` already behaved this way. The two front ends' chain semantics are
left for a ticket of their own.
The interactive cursor fallbacks of `clear-cell`, `yank`, `paste` and
`drill-into-cell` answer only for coordinates that were OMITTED
(`only_when_omitted`). Coordinates that were given and are malformed are
refused, never replaced by the cursor's cell.
context/code/clients.md
+8 -8
View File
@@ -831,7 +831,7 @@ The classes that remain:
column renumbers every index to its right and the advance was computed from
the pre-commit layout (gotcha 23), so `CommitAndAdvance` returns straight
after it and a client that advanced anyway would land one column off,
silently. `AddCategory` can also be refused by the 12-category limit and sets
silently. `AddCategory` can also be refused by the category limit and sets
`abort_effects`; the mode change therefore goes **first**, where an abort
cannot reach it (gotcha 21). **Since improvise-bd9o that exception is
unreachable**, and this bullet describes `commit` alone: the records decline
@@ -1652,7 +1652,7 @@ through the server's own types; with one definition there is nothing left for
them to compare, which is what improvise-xks6 was for.
**`rejection_message` is spelled here rather than on `Rejection`** for the
reason `AxisColor` is not a ratatui colour: the protocol carries the *fact*, and
reason a `Face` is not a ratatui colour: the protocol carries the *fact*, and
each front end words it. The wording tracks ADR 001's force table, because "try
again" and "this cannot be forced" are the two different things a user must be
able to tell apart — a `RecordGone` has nothing to write to, and offering force
@@ -1877,8 +1877,8 @@ Four smaller decisions that would be re-derived wrongly:
`improvise_protocol::read_buffer` so the `"search"` special case keeps its one
home (gotcha 11); `None` is "no text-entry mode is open", so the page asks
nothing about the mode. The colour stays semantic — `MinibufColor` crosses and
the CSS class is the page's last translation, exactly as `draw.rs::minibuf_fg`
is the TUI's (gotcha 26). It is drawn as a surface of its own rather than *in
the CSS class is the page's last translation. (The TUI no longer reads it: it
draws a face per text-entry mode, and improvise-nn4s.4 moves the page there.) It is drawn as a surface of its own rather than *in
place of* `#status` as the TUI's bottom bar does, because `#status` also
carries the transport's failures and a repaint must not erase one (gotcha 48).
Since improvise-n6t8 it shares `#bottom-dock` with the toast, which stacks
@@ -1920,7 +1920,7 @@ home for it.
The rows themselves are `improvise_protocol::help::pages()`, the same ones
`src/ui/help.rs` draws. `RenderHelpRow` carries a `kind` string
(`heading`, `keys`, `accent`, …) which is a *semantic* name the page maps to a
CSS class — the `AxisColor → axis_fg` shape again, and the reason the page can
CSS class — the `Face → draw.rs::face_style` shape again, and the reason the page can
style a heading without knowing what makes a line a heading. Where the content
comes from and what is derived from the keymap is `context/code/protocol.md` §8.
@@ -2353,7 +2353,7 @@ truncation. A browser has no column to fit, so the abbreviation buys nothing and
reads to a user as a rendering defect — it was spotted in PR #120's screenshots
as "Quarter Pag". `render.rs` now asks `plain_label`, which is the other
vocabulary already on `Axis`: the decision stays in the shared layer and only
the *choice* of vocabulary is local, exactly the `AxisColor → ratatui::Color`
the *choice* of vocabulary is local, exactly the `Face → ratatui::Style`
shape §5 blesses. `short_label` is untouched, because the TUI's width is
load-bearing.
@@ -2439,8 +2439,8 @@ So it is now a raised bar: fixed, centred over the grid, larger, bordered in the
mode's own colour and shadowed. **Every part of that is paint.** The content is
`RenderMinibuffer`, built in `render.rs` from `state.view.mode.minibuffer()` —
the same semantic value the TUI reads — and the colour is `MinibufColor`
resolved to a concrete colour exactly as `draw.rs::minibuf_fg` resolves it for
the terminal. Nothing here asks which mode is open, which buffer is being typed
resolved to a concrete colour by `style.css`. (The TUI now draws a face per
text-entry mode instead; improvise-nn4s.4 moves the page onto the same faces.) Nothing here asks which mode is open, which buffer is being typed
into, or what the prompt says; a second answer to any of those is the disease
design-principles §5 is about. The caret is drawn as a CSS `::after` rather than
appended to the text for the same reason: the text is the host's, and the page
context/code/command.md
+18 -2
View File
@@ -2996,8 +2996,10 @@ ride in `ViewState::transient_prefix`, so they must live where the wire types do
and the re-export keeps `crate::command::key::…` paths working.
`parse.rs` is the quasi-lisp prefix parser: `word arg1 arg2 …`, several commands per
line separated by `.`, coordinate pairs written `Category/Item`, and quoted strings
(`"Profit = Revenue - Cost"`).
line separated by `.`, coordinate pairs written `Category/Item`, verbatim quoted
strings (`"Profit = Revenue - Cost"`) and `|pipe-quoted names|` read by
`improvise_formula::quote`. One lexer serves every entry point, so the separator
and the quoting cannot disagree (improvise-31oe.3, `context/code/cli.md` §14).
### `cmd/core.rs`
@@ -3014,6 +3016,12 @@ writes any other field, so an accepted argument list is never overridden by
context (improvise-2bac, improvise-yf7p; the argument is `context/code/cli.md`
§8). `register_pure` is the case with no fallback: both arms are `parse`.
Every registration also declares its `MaxArgs`, and the registry refuses surplus
arguments on both arms after the command is built. `register_nullary` declares
`AtMost(0)` for itself, and an `effect_cmd!` declares its limit in the macro
call as `MAX_ARGS`. So the arity rule is written once and is total over the
registry (improvise-31oe.3, `context/code/cli.md` §14).
`write_buffer` is the effect that writes a named buffer — the mirror of
`read_buffer`, and the only place that knows "search" is not an entry in the buffers
map but its own `ViewState` field (improvise-gdl, gotcha 11). That knowledge used to
@@ -3372,6 +3380,14 @@ The parseable model-mutation commands are thin `Cmd` wrappers around effects,
constructible from string args, sharing the same execution path as
keymap-dispatched commands (gotcha 16).
A wrapper that names a category, item, group or view returns
`effect::RequireKnown(Needs)` followed by its real effect (improvise-31oe.2).
`RequireKnown` asks `unknown_name` against the live app when the batch applies
and refuses and aborts if the document lacks the name, so a `:` line sees what
its earlier segments made. The effects themselves stay silent about unknown
names, because they are also the wire's mutation path. Why the check is a
separate effect: `context/code/cli.md` §15.
### `cmd/wizard.rs` (improvise-2wm, improvise-907)
Import-wizard key handling. The wizard is a small step machine, and which key means
context/code/core.md
+335 -10
View File
@@ -258,9 +258,11 @@ user drilled in to see all the raw data.
On an axis with two grouped categories the runs **nest**: `North` in `Upper`
holds `Jan` in `Q1`, and after `Q1` ends `North/Mar` is still in `Upper`. So a
marker names its level. `GroupHeader { level, .. }` opens a run at `level` and
closes every deeper one; `GroupEnd { level }` closes `level` and every deeper
one. `expand_category` reads the level off its prefix, which holds one item per
marker names its level. `GroupHeader { level, depth, .. }` opens a run at
`(level, depth)` and closes every deeper one; `GroupEnd { level, depth }` closes
that run and every deeper one. "Deeper" is the ordering of the pair: the level
first, then the depth inside one category's hierarchy (§19, improvise-felo.6).
`expand_category` reads the level off its prefix, which holds one item per
outer category, so it is derived rather than passed. A row's group, for
`row_group_for`/`col_group_for` and so for `z`, is the **innermost** run open
at it. That is the user's ruling on the ticket: membership is per level, and
@@ -357,12 +359,14 @@ browser symptom and silently break drill.
`Axis` also owns four presentation vocabularies — `plain_label()`, `glyph()`
(which `Display` appends, giving "Row ↕"), `short_label()` (three columns for
the tile bar; `None` is "·", **not** a truncation of "None") and `initial()`
(one letter for the view panel; `None` is empty) — plus `color() -> AxisColor`.
`AxisColor` is a *semantic* palette because `improvise-core` must not name a
ratatui type; `draw.rs::axis_fg` does the one mapping, exactly as `minibuf_fg`
does for `MinibufColor` (improvise-rml). The tile bar and the category panel each
carried a copy of the colour table before, with nothing keeping them in
agreement.
(one letter for the view panel; `None` is empty). Its **colour is no longer
here**. It was `color() -> AxisColor` (improvise-rml), a semantic palette that
lived in core only because it was a method on a core type; ADR 004 clause 6
moved it to `improvise-protocol` as `Face::Axis(axis)`, because a face is UI and
this crate is the one that must have none. The TUI resolves it through
`draw.rs::face_style` (`context/code/tui.md`). The tile bar and the category
panel each carried a copy of the colour table before improvise-rml, with nothing
keeping them in agreement, and they still share one: the face.
### `on_category_added`: a virtual on an axis is a configuration, not a vacancy
@@ -518,12 +522,129 @@ explicitly would be redundant in the file format and confusing in the UI.
`Label` is a high-cardinality per-row field (description, id, note): stored
alongside the data so it shows up in record/drill views, but defaulting to
`Axis::None` and excluded from the `MAX_CATEGORIES = 12` pivot limit and from
`Axis::None` and excluded from the `MAX_CATEGORIES` pivot limit and from
auto Row/Column assignment.
`Category::add_item` deduplicates by name and assigns IDs from a private
counter, so external code cannot fabricate an `ItemId`.
### The limit is 24, and nothing is exponential in it (improvise-felo.1)
`MAX_CATEGORIES` counts **regular** categories only. It is `pub` in
`model/types.rs` and re-exported as `improvise_core::model::MAX_CATEGORIES`,
and it has one home: the import wizard's precondition reads it rather than
keeping a copy. It had a copy until felo.1, a private `const` of its own, and
raising the model's number alone would have left the wizard refusing a 13th
category the model accepts. Tests that fill to the cap derive it from the
constant; `add_category_max_limit` is the one place the number 24 is spelled.
**Why 12.** Nothing recorded says. The limit arrived in the initial commit
with no comment, ADR or ticket behind it, and nothing in `context/` or `kb.pl`
argues for it. The likeliest source is Lotus Improv, which this tool follows
by name (`context/plan.md`) and whose worksheets are remembered as capped at
12 categories. That is not verified here. It was never a cost bound. The user raised it on 2026-09-25: "12 categories is
feeling small, we should increase to 24". An E*Trade import hit exactly 12,
leaving no room for improvise-cgqt's disambiguating category.
**What scales with the number of categories**, read off the code:
- `CellKey` is a sorted `Vec` of pairs: building one is O(k log k), and
comparing or hashing one is O(k).
- The `DataStore` secondary index holds one `(category, item)` entry per
coordinate per cell. That is O(k) per insert and O(cells × k) in memory.
- Records mode has one column per regular category, so it is linear in k.
- `cross_product` is the product of the item counts of the categories **on
an axis**. That is exponential in what the user places on the axis, and the
cap never bounded it: 12 categories of 4 items on rows is already 16M rows.
`prune_empty` is what keeps it drawable.
- `improvise-gp` bounds its encoded width separately (`MAX_FEATURES = 64`),
so a wide fit is refused by name rather than slowed.
**Measured**, `cargo run --release -p improvise-core --example
category_width_bench`, on an Apple M4 Max. Each category has 4 items and each
record carries 2 measures. The pivot is one category on rows, `_Measure` on
columns and the rest aggregated away. Figures are the best of 5 (the store
build is the best of 3), and they held across two runs.
| categories | records | build store | pivot, every cell | records build | records draw |
|---|---|---|---|---|---|
| 12 | 2,500 | 26–30 ms | 1.6 ms | 79 ms | 1.6 ms |
| 24 | 2,500 | 80–83 ms | 3.0–3.7 ms | 82 ms | 3.7 ms |
| 12 | 5,000 | 54–57 ms | 3.2–3.9 ms | 302 ms | 3.1 ms |
| 24 | 5,000 | 163–172 ms | 6.3–8.0 ms | 321 ms | 7.0 ms |
Doubling the width doubles the per-cell work and triples loading the store,
which also builds and hashes CellKeys twice as long. The records-mode build is
the outlier, and width is not its cause: it grows 4× when the **records**
double, at either width. `RecordRow::group` finds each cell's row with a linear
scan (improvise-sk32).
#### Coupling found
Raising the constant was used to look for code that assumes few categories.
This inventory was taken at felo.1's merge of `gh/main` `4f38a35e`. Line
numbers are dated, so re-derive them before citing.
**Reads the constant (fixed in felo.1).** The wizard's precondition, and the
duplicate step's cap check that improvise-cgqt added (`separate_blocked`). A
fixture that fills to the cap either derives it or fills until
`add_category` refuses.
**Separate limits, left alone.** None of these is a category count:
- `MAX_ROW_HEADER_W = 24` (`improvise-protocol/src/render.rs`) is the widest
one header level is drawn, in cells. It equals the new cap by coincidence.
- `MAX_FEATURES = 64` (`improvise-gp/src/fit.rs`) is the encoded-width budget
of one fit.
- `CATEGORY_THRESHOLD = 20` (`import/analyzer.rs`) is the distinct-value count
at which a text field becomes a `Label`.
**Degrades as categories grow.** These are not fixed here; each is a place
where 24 is worse than 12. Most were already reachable at 12.
- **The TUI row header is the sum of its levels** (improvise-gtn7.5). In `src/ui/grid.rs`, around
`row_header_width`/`data_area_width` (and
`ui::viewport::compute_row_header_width`), each level takes 4 to 24 cells.
Categories on Row therefore cost at least 4 cells each, and the data area
saturates to zero width with no data drawn.
- **Column headers take one line per level, but the keypress path does not
count them** (improvise-gtn7.6). `App`'s `visible_rows` is `term_height − 8`, a fixed allowance
that ignores the column levels, the group row and the page bar. So with
several categories on Column, the cursor can scroll below the drawn grid.
- **The page bar is one unclipped line** (improvise-gtn7.7). It is
`GridWidget::render`'s `page_coords` join, drawn with a bare `set_string`
that is not clipped to the inner rectangle, so a long bar can overwrite the
block's right border. Default seating (`View::on_category_added`) sends every
regular category after the second to Page. A 24-category import therefore
puts 22 selectors on one line, clips most of them without a mark, and opens
on a single slice.
- **Side panels do not scroll** (improvise-gtn7.8). In `src/ui/panel.rs`, an item past
`item_height` is not drawn, and the cursor can move onto it unseen. The
category panel lists every category plus its expanded items.
- **The wizard's field list does not scroll** (improvise-8jk6.1). Its `ReviewProposals` arm in
`src/ui/import_wizard_ui.rs` breaks at the popup's bottom (the popup is at
most 30 rows high), unlike the steps that use `scrolling`. About 22 fields
fit, so the over-the-cap case cannot show every field it asks the user to
fix.
- **Loading the store is O(k log k) per cell and O(cells × k) in memory**
(improvise-felo.2). It
measured 3× from 12 to 24 (the table above).
- **A fit declines more columns.** `unaffordable_categories` drops the widest
category columns once the encoding passes `MAX_FEATURES`. A wide document
is fit on fewer of its categories, and the fit says so.
- **`cross_product` is exponential in the categories placed on one axis**, and
`prune_empty`'s `has_value` matrix is rows × columns. The cap never bounded
either. Virtualizing the grid is improvise-felo.3.
- **Header repeat suppression costs O(levels) per position and level**
(`render.rs::shows`). That is O(rows × levels²) per frame, which is small at
24.
The records-mode build's quadratic (improvise-sk32) is not category coupling;
it is named in the measurement paragraph above because this measurement found it.
The browser adapts where the TUI does not. `#tiles` and `#categories` scroll,
and `web/shell.js` charges each column level to the extent.
### Seating takes the kind, because a name cannot carry it (improvise-cj4u)
`View::on_category_added` takes a `CategoryKind` beside the name. It used to
@@ -1439,3 +1560,207 @@ editor stays open over the user's text (gotcha 21). `was` is not redundant
with a stable id: a concurrent *replace* keeps the id and changes the text,
which only `was` can see.
## 19. Group hierarchies: parents nest runs, and every group starts closed (improvise-felo.6)
`Group::parent` was parsed (`> Q1[2024]`), written, round-tripped and hashed,
and nothing read it: `Q1` was never drawn inside `2024`. It is now the
hierarchy.
### The user's rulings
- "I think hierarchies should start closed unless we somehow opt into a
specific one opened to a certain depth (view config maybe?)" — and the
reason: "in most cases, the closed by default start is preferable because it
makes the grid more dense."
- "Everything opens closed, including old files." There is no version gate: a
file written before this change opens with its groups closed, and its
`collapsed:` lines simply agree with the default.
- "I think we'll need a depth for expanded, something like expanded(2): ...".
- "expanded+collapsed in the same group is an error".
### The model
- **A group's chain** is the group and its ancestors, root first
(`Category::group_chain`). An item's chain is its group's
(`Category::item_chain`); an ungrouped item has none. A group's depth is its
number of ancestors.
- **A group covers every leaf beneath it.** `items_in_group` reads each item's
chain, so `2024` covers the months of `Q1-2024` and `Q2-2024`. That is the
whole change to evaluation: `evaluate_covering` already expanded a group
coordinate through `items_in_group` (§14), so a year's fold is the fold over
its months with no second path.
- **An item may sit directly in a parent group, beside its child groups.**
Chosen because nothing is simpler: the chain handles it (`Jun[H1]` after a
run of `Q2` inside `H1` ends the `Q2` run and stays in `H1`), and refusing it
would be a rule with nothing to protect. A date import never produces one.
- **A cycle is refused on load, naming its members** (`Refusal::HierarchyCycle`,
`persistence/parse.rs`). A cycle built in code cannot hang anything either:
`group_chain` stops where it would repeat.
- **There is no such thing as a dangling parent in this format.** A root group
is declared *only* by being named as a parent: the writer emits `>` lines
only for groups that have a parent, and nothing else can name `2024`. So
"a parent that names no group" and "a root" are the same line, and refusing
one refuses the other.
### Which groups are open
State lives on the `View`, in two places that cannot disagree about one group:
- `expand_depth: cat → N` — `expanded(N): Cat`, the category's top N levels
open.
- `group_folds: cat → group → GroupFold` — ONE map, so a group has at most one
fold. `GroupFold::Collapsed` is `collapsed: Cat/Group`;
`GroupFold::Expanded(N)` is `expanded(N): Cat/Group`, the group open and
N-1 levels beneath it. Two sets would have made "both" representable, which
is the state the user ruled an error
(`doctrine(an_invariant_is_made_unrepresentable_not_validated)`).
`View::is_group_collapsed(cat, chain)` answers, in order:
1. the group's own fold, if it has one — so a `collapsed:` group inside an
`expanded(N)` region stays collapsed, and an `expanded:` group below the
category's depth opens;
2. open, if an ancestor's `Expanded(n)` reaches it (the ancestor `k` levels up
reaches it when `k < n`), or the category's depth does (`depth < N`);
3. closed.
**`z` writes one fold, or none.** `toggle_group_collapse` flips the answer and
records the flip as `Collapsed` or `Expanded(1)` — unless the flip lands on
what the group would inherit from rule 2, in which case it removes the group's
fold. So `z z` leaves the view as it was, and no sequence of toggles writes a
file that names one group twice
(`no_sequence_of_toggles_writes_a_conflicting_file`). Closing a group that
carried `Expanded(3)` drops the 3: reopening it with `z` opens one level.
### The layout
`expand_category` walks each drawn item's chain. The **outermost** collapsed
group in the chain folds everything beneath it: the row's tuple is that group's
name, it folds once however non-contiguous its members are (§14), and the
headers above it are drawn. The run markers move from item to item exactly as
`remark` moves them after pruning — past the shared prefix of the two chains,
end the deeper runs innermost-out, then open what differs — so an unpruned axis
survives `prune_axis_items` marker for marker, and the enumeration in
`group_membership` (now with a `NESTED` fixture: a non-contiguous quarter, an
item directly in a half-year, an outer grouped category) asserts it.
A header carries its own `collapsed`, computed once in `expand_category`, so
the frame projection and the frame hash read it rather than asking the view
again with a chain they would have to rebuild.
### Persistence
`expanded(N): Cat`, `expanded(N): Cat/Group`, bare `expanded:` for N = 1, and
the existing `collapsed: Cat/Group` (`improv.pest`, `expanded_line`). N is at
least 1 by the grammar (`'1'..'9' ~ ASCII_DIGIT*`), so `expanded(0)` is a parse
error rather than a value to explain. Two further decisions:
- **The writer keeps what the view holds.** An old file's `collapsed:` lines
load as `Collapsed` folds and are written back verbatim, though they agree
with the default. They are the user's explicit closes, and they survive a
later `expanded(N): Cat` that would otherwise open those groups; dropping
them on save would change what that later line means. A `z` round trip
removes such a line, because `z` normalises (above).
- **A category depth is never written bare.** `axis_line` is tried before
`expanded_line`, so a category literally named `expanded` keeps its
`expanded: row`. The price is that `expanded: row` meaning "category `row`,
depth 1" would read as that axis line, so the writer always spells the
category form `expanded(N): Cat`. The group form has a `/` and is never
ambiguous.
A view naming one group with two different folds, or one category with two
depths, is refused on load, naming the view and the group
(`Refusal::FoldConflict`). The same line twice is not a conflict.
The grammar-walking generator emits cycles and conflicts with ordinary
probability (`> ||[||]` is a self-parent), so `generated_file_parses` accepts a
`Refusal` — a typed error, so it cannot accept a parse failure by accident.
### Known limits
- The TUI draws nested headers exactly as it draws any other: one line per
header, not indented by depth, and the column band shows each column's
innermost group only. Both are pre-existing presentation choices; the frame
carries `depth` for the day a front end wants to indent.
- Two `>` lines giving one group two different parents: the last wins, as it
did before this change. That makes the hierarchy a function of line order.
## 20. How an id is made — `model/id.rs` (ADR 007 clause 4, improvise-cq1q.2)
`slugify` and `IdAllocator` are the first piece of ADR 007. They are pure
functions of the display and the allocator's own history. Nothing calls them
yet: the creation sites move onto them in the re-key step (improvise-cq1q.4).
Each rule traces to a numbered answer in the ADR's "What the user decided".
**Case folding is full Unicode folding, and it comes from `unicase`.**
`str::to_lowercase` is not folding: `ß` lowercases to itself and is then
stripped, so `Straße` would slug to `strae` (answer 26). The same goes for `ſ`
(long s) and the `fi` ligature. `unicase` 2.9 is `no_std` and has no
dependencies, so it passes the wasm gate. Its `to_folded_case` applies the
C+F mappings of Unicode's CaseFolding.txt, and its own tests pin `Maße` →
`masse`. Most of what folding produces is stripped anyway: only an ASCII
letter survives step 3. Deriving that ASCII-producing subset of the table by
hand was rejected. It is a second copy of the Unicode data, and nothing would
say when it drifted.
**A "space" is U+0020.** Answer 18 says "replace space with `_`", and answer 21
says runs collapse. A tab or a no-break space is not a space here, so it is
stripped by step 3. Only a run of U+0020 becomes one `_`. An underscore the
display already had is never merged with the one a space run makes. So `a_ b`
slugs to `a__b`: the steps are applied as written, and no extra
underscore-collapsing step exists. A trailing run survives, so `East ` slugs
to `east_`. Only *leading* underscores are removed (step 4).
**The allocator records every id it admits and releases none.** That is the
whole of retirement (clause 4, "within a session a removed id is never handed
out again"). No release operation exists, so a removed id cannot be reissued
by construction; no check refuses it. The counter starts at 1, is shared by
every kind, and advances each time a suffix is tried, including a candidate
that turns out to be taken by a hand-written id (answers 28 and 29). So
`east-7` may follow `east` directly, and no order is enforced among suffixes.
**The walk past taken candidates is a loop, never recursion.** A
new-version file saved from a non-ASCII import holds `item-1` … `item-N`, and
the counter restarts at 1 on load, so one generation walks all N. The first
version recursed once per candidate, not in tail position, and aborted the
process on a stack overflow: in debug at N ≈ 11,000, in release at 50,000.
An abort, not a panic, so the TUI lost unsaved edits (Codex P2 on PR #318).
An empty slug always takes a suffix (`category-1`), even when the bare kind
name happens to be free, because answer 16(e) spells the result
`<kind>-<n>`.
**`claim` is the only door for a hand-written id.** It checks, in this order:
the alphabet (non-empty, `a`–`z`, `0`–`9`, `-`, `_`), the reserved leading
`_` (only `_index`, `_dim` and `_measure` may carry one, per answer 23, and
the match is exact: `_indexer` and `_measure-1` are Reserved, since "no other
id may start with `_`"), and
duplicates. It never renames. Refusing is the loader's job, and the loader
must name both lines of a duplicate; `ClaimError` carries only the id, so the
loader keeps the line numbers. The virtual views `_records` and `_drill` are
not yet in `VIRTUAL_IDS`. improvise-p003 item 1 asks for them, and they arrive
with the ADR text fix (improvise-cq1q.1) rather than ahead of it.
## 21. A display beside the key (ADR 007 clause 2, improvise-cq1q.3)
`Category`, `Item` and `View` each carry a private `display`. It starts equal to
the key and changes only through `set_display` / `set_item_display`, which move
no key. Until the re-key (improvise-cq1q.4), nothing in production calls those
setters, so `display == name` everywhere and behaviour does not change. The
split exists so that the re-key changes what goes *into* the key, and nothing a
person sees moves with it.
**The key travels beside the display, never in place of it.** `CategoryRow`,
`TileEntry` and `ViewSummary` each gain a `display` field. The key field
(`name`, `category`) stays exactly what it was, because both front ends compose
commands from it: `resolve.rs` builds `SetAxis`, `RemoveItem` and the rest out of
those strings. Carrying the display *in* the key field would make every
command name a key that does not exist once the two differ. That is the gap
improvise-cq1q.11 names, and cq1q.7 later turns these key fields into ids. The
active-view marker in both view panels still compares keys for the same reason.
**What this first slice covers:** the category panel's headers, the tile bar and
the view list, in both front ends. Item rows, the grid frame, the page bar,
records-mode labels, CSV export headers, status messages and the view title
still show keys. They follow in later slices of this step, because the grid's
label tuples are also cell coordinates (`cell_key`, `innermost_group`), so their
display has to be added at projection time rather than in place.
context/code/doc-site.md
+9
View File
@@ -181,6 +181,15 @@ The general form: when a check misses a defect, add the check that could have
seen it rather than only fixing the defect, and ask what *question* the
existing check was answering.
**`site/` is build output and is not tracked** (improvise-1xfb.4). It is
`web/assemble.sh`'s default `OUT_DIR`, and `site/docs` is this script's.
Every reader builds its own copy from source: CI's `build` job runs
`nix build .#site`, which assembles into the store; the harness assembles
into `harness/serve/`; `scripts/build-web-release.sh` assembles into a
temporary stage. For a while 27 generated pages under `site/docs` were
committed by accident. They went stale, still saying 12 categories after the
cap was raised, and nothing read them.
## 5. Why a node script and not a generator
The landing page is hand-written HTML with no build step and one pinned CDN
context/code/formula-and-io.md
+371 -12
View File
@@ -165,7 +165,7 @@ share, which is why the existing suite was silent — the same shape as the
`Category::kind` rather than the emitted text.
The consequence was a lost document, not a lost flag. `Label` sits outside
`MAX_CATEGORIES = 12` on purpose, and the import wizard's own precondition
`MAX_CATEGORIES` on purpose (12 then, 24 since improvise-felo.1), and the import wizard's own precondition
scores a label as zero, so it will happily build twelve categories plus a
label. On load every label came back `Regular`, the thirteenth was refused, and
`parse_md` failed: **any document where regular + label exceeded twelve was
@@ -218,6 +218,32 @@ times. The probe was removed rather than kept: its rate depends on proptest's
internal RNG, so as a permanent assertion it would fail on a dependency bump
for a reason unrelated to the format.
### Folds are view lines: `collapsed:` and `expanded(N):` (improvise-felo.6)
Groups start collapsed, so a view persists what it opens. `expanded(N): Cat`
opens a category's top N levels, `expanded(N): Cat/Group` opens that group and
N-1 levels beneath it, a bare `expanded:` is N = 1, and `collapsed: Cat/Group`
keeps its old meaning. The semantics, the precedence and the refusals are
`context/code/core.md` §19; what is particular to the format:
- **No version bump.** The line is additive, and the user ruled that old files
open closed like new ones, so `v2025-04-09` still reads every file it did.
- **N >= 1 is grammar, not validation**: `depth_number = @{ '1'..'9' ~
ASCII_DIGIT* }`. Spelled with a `Range` and `ASCII_DIGIT` rather than
`ASCII_NONZERO_DIGIT`, because the generator knows the first two and would
emit nothing for the third — `expanded(): ` — a file its own grammar refuses.
- **`axis_line` comes before `expanded_line`** in `view_entry`, so a category
named `expanded` keeps its axis line, and the writer never spells a category
depth bare (`expanded: row` would be that axis line).
- **Two refusals the grammar cannot express** — a cycle among `>` lines and one
group given two folds — are a typed `Refusal`, which `generated_file_parses`
accepts and nothing else does. The generator produces both at ordinary rates.
- **The generator reaches the new rule**, asserted by
`the_generator_writes_expanded_lines_with_and_without_a_depth` over 512
deterministic files. Unlike the kind-line probe above, it asserts that each
spelling occurs at all, not a rate, so a proptest bump fails it only if the
walk stopped reaching the rule.
### The tree walkers are infallible by construction
Every `build_*` in `parser.rs` operates on a `Pair` pest has already
@@ -587,6 +613,59 @@ different row order on screen, and a different detected date format, since
varied. The `HashSet` in `analyze_records` is there to deduplicate and **never
to iterate**.
### A date column imports as ONE hierarchical category (improvise-8jk6.3)
The user: "I think dates should usually be one category and we can use
formulas to crosstab." So the default `DateShape::Hierarchy` builds one
category named after the column, where the old import built up to three flat
ones beside it (`Date_Year`, `Date_Quarter`, `Date_Month`):
- **Its items are the finest chosen component**, spelled so they stay unique
across years: `2024-01`, `2024-Q1`, `2024`, which are
`extract_date_component`'s existing spellings. A month is never `Jan`.
- **The coarser chosen components are its groups**, coarsest first
(`DateComponent::COARSE_TO_FINE`): a month sits in its quarter, and the
quarter's `>` line names its year. A level left unchosen is skipped rather
than drawn empty, so Year + Month groups months straight into years.
- **It opens folded**, one row per year, with nothing to set, because every
group starts closed (`context/code/core.md` §19).
- **It costs one of the category slots**, not up to four, and
`proposed_category_count` counts it that way.
- **The raw dates are gone from that category.** The day is not a component, so
two records of one month land on one cell and the collision step asks, as it
does for any other collision. `Split` keeps the day.
- **A value the detected format cannot read keeps its raw text** as an
ungrouped item, with a warning naming the column and the row count, rather
than being dropped. The warning is separate from the split shape's, because
that one reports an absent coordinate and this one a coordinate that is there
and ungrouped. **Unless its spelling is one the hierarchy generates**
(improvise-d0n6): a raw `2024-01` or `2024` is the same name as the generated
month or year, so it would join the grouped item and the warning would be
false. The build **refuses**, naming the column and the spellings and pointing
at the split shape. That follows the collision rule below: where two records
would silently be one thing, the user is asked and nothing is defaulted. A
fallback to the split shape was rejected as a silent change of the user's
choice, and a renamed raw item as altering the data. The check is
`Plan::spelling_clash`.
- **Item order is first-seen, not chronological.** Order is a correctness
property here (gotcha 18), and a group need not be contiguous, so a CSV that
is not in date order still nests correctly. It is just drawn in the order
it was read.
`DateShape::Split` is the old behaviour, unchanged, and stays a choice. `s` on
the wizard's date step toggles the field under the cursor, and the header line
for each field says which it will get ("one category: Year > Quarter > Month"
or "split: Date, Date_Year, …"). It is worded once, in
`project.rs::date_shape_sentence`. Headless, `--split-date <field>` does the
same, and `--extract` keeps its meaning: which components to build.
**Crosstabs by formula are not available yet.** A formula can name a month
(`SUM(Amount WHERE Date = "2024-01")`, `Date IS "2024-01"`), but a `WHERE` or
`IS` naming a GROUP matches no record: both compare the coordinate to the name
exactly. `SUM(Amount WHERE Date = "2024")` is an empty sum, and `Date IS "2024"`
is 0 on every month of 2024. That was measured on this branch and reported for
filing, not built here.
### The ambiguous-date heuristic reads American-first, deliberately
`01/02/2025` parses under both `%m/%d/%Y` and `%d/%m/%Y` and nothing in the data
@@ -632,6 +711,256 @@ one proves nothing.
backend rather than only a file (improvise-fuj); the path-taking wrappers remain
for the native CLI.
### A CSV field is text, and the field's role reads it (improvise-8jk6.4)
The parser used to type every field as it read it: text that parsed as an
`i64` or a finite `f64` became a `Value::Number`. That is a lossy step, and it
ran before anyone knew what the column was for. A column the user then forced
to Category (`--category`, or the wizard's `c`) took its item names from the
number's display, not from the file. So commit `00565796` became item `565796`,
and `220944e2` was read as scientific notation and became `22094400.0`. The
repo-stats pipeline (improvise-1xfb.5.1) found it.
CSV has no types. Whether `00565796` is a number depends on whether its column
is a measure, and the parser cannot know that. So the parser now yields every
non-empty field as a `Value::String` of its text (blank is still `Null`), and
two readers apply the old numeric rule, `csv_parser::number_in`, only where a
number is wanted:
- **Analysis** proposes a kind from a typed copy of the records
(`Scalars::typed_record`), so a numeric CSV column is still proposed as a
Measure, exactly as before.
- **A measure** takes its value through `Scalars::number`.
A category or a label is named from the text itself, so it is verbatim.
`ImportPipeline` carries the reading as a private `scalars: Scalars`:
`Typed` for JSON and `Text` for CSV. JSON cannot share the text rule, because
a JSON string is text by the file's own say. `"02139"` in JSON is a zip code,
and reading it as a number would propose it as a Measure. A JSON number
forced to Category has no source text to keep, because serde_json does not
keep one. It is named by `Value`'s own display, which is what writing it back
as JSON would say: `42`, `1.5`, and `1000.0` for `1e3`.
**The parser returns `CsvRecords`, not `Vec<Value>`.** It derefs to
`[Value]`, so a caller can read it. It converts to a `Vec` only inside the
crate, and the only public way into a pipeline is `ImportPipeline::from_csv`.
This is not caution for its own sake. When the parser first changed, eleven
tests in four files failed. Every one handed parser output to
`ImportPipeline::new`, the JSON constructor, which would read every column as
text and propose `Revenue` as a Category. Production had the same shape at all
three CSV entry points: `main.rs`'s single file, `ImportPipeline::from_merge`
(the merge path, from PR #314), and `ImportJsonHeadless`. `from_merge` merged
in with `Self::new` and now calls `Self::from_csv`. Now that mistake does not
compile. Two holes remain, and
both take a deliberate step. `ImportPipeline::records` is still a public
`Vec<Value>`, and a test helper once copied it out of a CSV pipeline into a
JSON one. And `CsvRecords` derefs to a slice that `.to_vec()` will copy.
### An import refuses what it cannot keep (improvise-8jk6.5, improvise-8jk6.6)
Two failures used to end in an import that saved and reported success.
**A formula that does not parse.** `build_model_with_warnings` ran
`if let Ok(formula) = parse_formula(..)`, so a bad `--formula` vanished. The
headless run then saved and exited 0, and the wizard kept the bad text in its
list and dropped it the same way at build. Both now go through one
function, `import_formula`, whose error names the formula and carries pest's
parse error: `Formula error in 'Y = x +': …`. The prefix matches the TUI
formula minibuffer's `Formula error:`.
- `ImportPipeline::parsed_formulas` parses them all. If any fails, it refuses
and names **every** formula that failed, one per line (a message the user
needs is never truncated). The build calls it before creating anything, so a
refusal writes nothing.
- The wizard checks at `confirm_formula`, the one place the user can still fix
the text (gotcha 22). A formula that does not parse stays in the buffer with
the error as the step's message, and never enters the list.
**Nothing kept.** The one skip left after improvise-8jk6.2 is a record that
is not a JSON object. When every record is skipped, there are no fields, so
there are no proposals and no categories. The build did refuse that case, but
by accident, with "At least one category must be accepted". That tells the
user to change a field's kind when there are no fields. The ticket's own repro
(`[1, 2, "three"]`) already exited 1 on the lpzu base, so the headline did not
reproduce there. What did reproduce was the reason. `nothing_to_import` now
states the rule and runs before the category check: `Nothing to import: all 3
records were skipped as not JSON objects`, or `the source has no records`. The
wizard asks it first on `ReviewProposals`, where another array path can still
be chosen.
One case meets PR #314's "a header-only file is still a file". A folder whose
CSVs are **all** header-only has no records, only sources. Built from
`from_merge`'s seeded proposals alone, it would save `File` items, empty
columns and one empty records view per file. It is refused here, because it
kept no record. A header-only file *beside* files with rows is unaffected,
since some record was kept. The user confirmed this on PR #321: "Refuse (as
#321 does)".
**A partial skip stays a warning.** Some records kept and some skipped still
builds, exits 0, and warns with a count: `Skipped 1 of 3 records that were not
JSON objects`. The argument: a non-object in an array of records is not a
record that failed to import. It is structurally not a record, such as a
trailing `null` or a stray scalar in a hand-edited file. Every record that
*is* a record was kept, and the count is on stderr for a script to read.
Making it fatal would fail every import that carries one stray element.
Making it a distinct non-zero exit that still writes gives a contract where
`import … && next` stops even though the file was written.
The user ruled on PR #321: "Add a --strict flag". The default stays exit 0
with the counted warning, pinned by
`an_import_that_skips_some_records_warns_with_a_count_and_still_writes`.
`import --no-wizard --strict` turns any skipped record into a refusal:
`--strict: Skipped 1 of 3 records that were not JSON objects`, exit 1.
**A strict refusal writes nothing.** A refusal that still wrote the file would
be the half-met contract argued against above: `&&` stops while the output
sits there, and a script that checks only the file gets a model that `--strict`
existed to prevent. So the check runs in `run_headless_import` before the
build. It reads `ImportPipeline::skip_warning`, the same function the build's
warning comes from, so the two cannot disagree about what was skipped.
"Skipped" here means a record that is not an object, the only skip left after
8jk6.2. An unparsed date or a short CSV row is a warning about a record that
was kept, and `--strict` does not refuse it.
### A folder of CSVs is one sparse model, with a view per file (improvise-lpzu)
`improvise import <folder>` exists for improvise-1xfb.5.1. code-maat writes one
CSV per analysis into a directory, and the schemas differ: coupling has
`entity,coupled,degree,average-revs`, age has `entity,age-months`, and
abs-churn has `date,added,deleted,commits`. The user ruled (2026-09-25):
"Records are already sparse: the model can just leave null/empty values for
columns not in a given CSV file." So files with different headers merge into
**one** model. It is not refused, and it is not split into one model per schema.
**A folder argument means exactly its `*.csv` files.** `expand_import_paths`
takes the arguments in order and replaces each folder with its `*.csv`
entries. The extension check is case-insensitive, the listing is not recursive,
and the order is `Storage::list`'s, which is sorted by contract. Then the
existing path runs, with one difference: **a folder always merges**. Two or more
files merge through `merge_csvs_from` with a `File` category, and so does a
folder holding a single CSV. That gives the one file its `File` item and its
per-file view.
The single-CSV rule is **the user's ruling (2026-09-25)**, which replaced this
PR's first answer, "`import dir/` is exactly `import dir/*.csv`". Their words:
"yes, we might want to have some kind of update import eventually (don't plan
this yet, but we'd want file identity for this)". A folder names a collection
that may grow, so a document built from it should record which file each
record came from from the start.
**A single CSV named directly, not through a folder, still imports alone,
without `File`.** That is deliberate. Naming one file asks for that file, not
for a collection, and the shipped recipes (`examples/demo.improv` from
`examples/demo.csv`) and every existing single-file script would otherwise gain
a `File` category and a second view. File identity for update-import can be
asked for by naming a folder. The test is
`a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not`.
Apart from that, a folder still behaves as the glob would, and four cases follow
from that rather than being chosen one by one:
- **Non-CSV entries are skipped**, not refused, because a glob skips them. This
includes the output `.improv`, which a second run finds in the folder.
- **A folder with no CSV at all is refused**, naming the folder. A glob that
matches nothing is an error too, and importing nothing would write an empty
document.
- **Files and folders mix freely**, in argument order.
- **A file named twice is read twice.** For example, `dir/a.csv dir/` reads
`a.csv` twice, and its records collide with themselves. The headless path
refuses that like any other collision. Nothing deduplicates, because the
shell glob would not either.
The listing goes through `Storage::list` so the browser host can reuse it
(gotcha 2). `Storage` knows documents, not folders, so **whether an argument
is a folder is the caller's question**. The CLI answers it at the edge with
`Path::is_dir` and passes an `ImportPath::{File, Folder}`. An OPFS host would
answer it from its own UI.
**Sparse merge.** A record from a file without some column has no key for it,
so `Plan::place` gives it no coordinate there (improvise-8jk6.2 above), and no
measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already
classified each column over the values present (improvise-l1f5), so a column
only one file has is judged by that file's values alone. The category cap
counts the **union** of every file's categories, header-only files included
(see the next paragraph). Rows from different files cannot collide, because
`File` differs. One caveat on "judged by its values": a JSON `""` is not yet
dropped by the analyzer the way `null` is, so a JSON numeric column holding
`""` is proposed as a Label (improvise-8jk6.11). CSV is unaffected, because
its parser turns a blank field into `null`.
**A header-only file is still a file** (the Codex P2 on PR #314). code-maat
writes a header and no rows whenever nothing passes its threshold, so
`coupling.csv` is often exactly that. The pipeline is analyzed from *records*,
and such a file has none, so it had no `File` item, its columns never reached
the model, and its view filtered to an item that did not exist.
`ImportPipeline::from_merge` therefore **seeds from `Merged::sources`**:
- the `File` proposal's items are the sources' items, in file order, so every
file has its item whether it has rows or not;
- a source column that no record has a value in is proposed as a **Label**.
That is the analyzer's own verdict for an all-null column (improvise-l1f5),
and a column with no values is exactly that: nothing can be classified, so it
takes no pivot slot and does not count toward the category cap until the user
reclassifies it in the wizard or with `--category`/`--measure`.
This is also the file identity the user wants kept for a future update-import.
**A file with no measure column gets a records view** (improvise-8jk6.13). A
record with no measure makes no cell, so such a file contributes items and no
data. Its view had nothing to page `_Measure` to, and paged onto another file's
measure, which the same view hides; the cell there was a `Record`, editable,
under this file's `File`. Now `source_view` gives such a file the records
layout (`view::records_steps`, the one home of what a records view is) with
`File` still on Filter at the file's item. It draws zero rows, because there
are no cells. A header-only file whose columns are
all Labels is the same case. In records mode the columns are the whole model's
categories and measures, not just this file's: records mode has no per-view
column filter. With no rows that is cosmetic.
**Each file gets a view** (the user, 2026-09-25: "dir import should do this:
one view filtered by the file column and projected to the columns in that
file"). `merge_csvs_from` returns a `Merged` whose `sources` list each `File`
item with the columns its header named. Two files with one stem share an item,
so their columns are unioned. The pipeline carries `sources`, and the build
adds one view per source after the merged `Default` view, which stays the
initial view. Each per-file view:
- **Is named after the file's stem.** A name already taken, including the
virtual `_Records` and `_Drill`, gets ` 2`, ` 3` and so on. A view named
`_Drill` would never be written and would be dropped on load.
- **Puts `File` on `Axis::Filter`, fixed at that file's item.** Filter and
not Page, because the view *is* that file's projection. Paging to another
file would show that file's records through this file's columns and hidden
measures, which is wrong on every page but one. Filter constrains without
being navigable (gotcha 27). Until now it appeared only in the never-saved
`_Drill` view, but the `.improv` grammar has always round-tripped it.
`per_file_views_survive_a_round_trip_and_repeat_byte_for_byte` proves that on
a saved per-file view, and `a_saved_per_file_view_projects_only_its_file`
proves the shared `grid_in` projection, which both the TUI and the browser
draw from, carries it as a fixed coordinate with nothing to cycle.
- **Seats the file's own categories** (its columns, plus date parts derived
from them) through `View::on_category_added`, in model order: the rule a
single-file import uses. Every other category goes on `None`. The file's
records lack those, so nothing splits.
- **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files'
measures hidden** and the page selection on the file's first measure. A
formula target comes from no file, so it stays visible. A file with no measure
gets a records view instead (above).
- **Is never named like another view from the same import.** `Default.csv` and
`Default 2.csv` become `Default 2` and `Default 2 2`; without the same-batch
check both would be named `Default 2` and one view would silently replace the
other.
A per-file view is made only while `File` is an accepted category. A user who
skipped or reclassified it has no filter to build views from.
Two things this does not do. Page cycling still steps onto a hidden item and
draws an empty grid: `[`/`]` in a per-file view can reach another file's
measure (improvise-yd2t; the Default view already behaved this way). And a
CSV with its own `File` column has it overwritten by the merge's, which is the
existing behaviour of `merge_csvs`.
### The wizard's back path is a history stack
`advance` pushes the step it leaves; `previous` pops it. Keeping the path
@@ -643,8 +972,9 @@ Preconditions live in `validation_error()` and are checked **on the step where
the user can still fix them** — never deferred to `build_model` after the final
confirm. Two gates earn that: too many categories (improvise-mzv) and no
category at all (improvise-gfq); both are fixable with `c` at `ReviewProposals`,
and both used to refuse only at the last keystroke. `MAX_CATEGORIES` here mirrors
the model's private limit and counts regular categories only.
and both used to refuse only at the last keystroke. `MAX_CATEGORIES` here is the
model's own constant, which counts regular categories only. Until improvise-felo.1
it was a private copy (core.md §9).
`NameModel` is terminal for `advance`: leaving it is a **build**, not a step
change. `build()` parks the workbook in `built` on `Done`; a build failure leaves
@@ -657,10 +987,35 @@ and a blank string are **absences, not names**; stringifying them used to invent
an item literally called "null" (improvise-ych). Every other value keeps its
printed form.
A record missing a value for an accepted Category field has no place in the grid
and is dropped; `build_model_with_warnings` counts it against the *first*
category it is missing and reports the total, because without a warning that
loss is invisible in a thousand-row import (improvise-5j9).
**A record with no value for a category keeps its cell, without that
coordinate** (improvise-8jk6.2). Records are sparse: a `CellKey` names only the
coordinates it has. So a missing field, a `null` and a blank string all mean
"no coordinate here". None of them makes an item: not `"null"`, not `""`, and
no synthetic placeholder. The same rule covers labels, which used to become an
item named `""`. The user's words, 2026-09-25: "this limit in import should be
fixed, it dates to when this wasn't sparse." That limit was improvise-5j9's
rule: such a record "has no place in the grid" and was dropped, counted and
reported as `Skipped N of M records missing category values`. 5j9's warning
made the loss visible, and the loss is now gone, so the warning went with it.
The only record still skipped is one that is not a JSON object.
Where such a record shows up is decided by the axes, not by the import. The
test is `a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_the_grid`:
- **The category on Row or Column:** no position holds the record. Its other
coordinates may still draw a row, but the cell under the category's items is
empty, and the column total leaves it out.
- **The category on Page:** no selection shows it.
- **The category on None:** it is aggregated like any other record.
- **Records mode:** its row shows a blank cell in that category's column.
- **`.improv`:** its Data line omits the category, and it round-trips.
- **Drill:** a drill that does not constrain the category reaches it.
**No "(none)" position, by the user's decision (2026-09-25).** They were asked
whether Row, Column or Page should give such records a place of their own. Their
answer: "I think leaving it out for now is fine, maybe a UI element somewhere
that shows the number of records underneath the current view?" That indicator
is improvise-gtn7.10, and it is not an import concern.
### Colliding records are a question, not a default (improvise-cgqt)
@@ -677,7 +1032,9 @@ made it choose too. It chose `Refuse` and points at `:import`.
**Detection comes before the build and shares the build's placement.** Where a
record lands is `Plan::place`: its categories, then its date parts, then its
labels, then its measure values, or a reason it is dropped. `collisions()` and
labels, then its measure values. Only a record that is not an object is
dropped, since a missing value is no coordinate rather than a reason to drop
(improvise-8jk6.2). `collisions()` and
`build_model_with_warnings` both call it, so the question the wizard asks and
the thing the build does cannot disagree. Neither has a second walk of the
record. **A collision is a cell that more than one record writes**, meaning a
@@ -735,14 +1092,16 @@ is already a category. `suggested_duplicate_category` lives on the pipeline, so
the CLI's `separate` uses the same rule.
**At the category cap, Separate is shown and disabled.** The reported import
already had 12 regular categories, which is `MAX_CATEGORIES`. Option (b) adds a
had 12 regular categories, which was `MAX_CATEGORIES` when this was written.
Option (b) adds a
**regular** category, and the user declined a hidden virtual one. So at the cap
the option is drawn "(unavailable)" with its reason and remedy: go back and
Skip a column or make it a Measure, or choose Sum. The reason is decided in
`ImportWizard::duplicate_options`, so every front end words it the same way.
This is the conservative reading, and it is **an open question for the user**:
lifting the cap for this one category, or counting it outside the cap, is a
model-layer decision this ticket did not take.
This was the conservative reading, and it left an open question for the user.
**The user answered it on 2026-09-25 by raising the cap to 24**
(improvise-felo.1), so "keep separate" is available for the reported import
now. The option is still disabled for any import that fills the new cap.
**Headless paths refuse by default.** `improvise import --no-wizard` has
nobody to ask, so `--on-duplicate refuse|sum|separate` defaults to `refuse`.
context/code/protocol.md
+110 -7
View File
@@ -1311,8 +1311,8 @@ re-derived from `kind`, so the two ends cannot disagree about what "Measure
rather than re-exported, because this crate must not depend on `improvise-io`
(the wizard is an io type, gotcha 30) and a client that draws the wizard must
still be able to name the step it is on. `WizardFieldKind` is the semantic
half of a colour decision — the terminal mapping lives in `draw.rs`, exactly
as `AxisColor`/`axis_fg` does (gotcha 26).
half of a colour decision: it indexes the `wizard.field.*` faces (§10), and the
terminal mapping is `draw.rs::face_style`.
### `Projection` carries no multi-widget variant
@@ -1474,6 +1474,22 @@ sequence. The header's level also gives a client the group's category,
the dead field improvise-7xyx removed: `level` is read by `groups()`, and the
category name is not carried twice.
**Within one category runs nest by `depth` (improvise-felo.6).** A group's
parent (`> Q1[2024]`) puts the quarter's run inside the year's on the SAME
level, so both markers also carry `depth`, the group's number of ancestors.
A run's key is `(level, depth)`, compared in that order: a header at a key
closes every run with a greater key, and so does a `GroupEnd`. With `Date`
alone on an axis, `2024-03` after `Q1-2024` ends sits in `2024`, and after
`2024` ends in nothing. Depth rides beside `level` rather than being folded
into one flat index, because a flat index would need every category's tree
height to be known before the first marker, and `level` would stop naming
`cats[level]`. Both front ends draw a nested header exactly as they draw any
other: one inline row per `Group` entry, the innermost group in the column
band. `the_two_front_ends_draw_the_same_nested_hierarchy_on_rows` and
`..._on_columns` in `tests/grid_frame_agreement.rs` hold them together, and
`the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy` does the same for the
windowed frame against the layout.
`group_runs` is unchanged. It coalesces whatever `groups()` answers, by header
identity (improvise-h0iv), so the band still paints one group per position, now
the innermost. One band per level would be a separate feature.
@@ -1554,14 +1570,19 @@ runs. Reversing it is one comparison, and the test is renamed
## §3b. A frame's levels are checked as it arrives
A `FrameGroup.level` and a `GroupEnd { level }` name a category: they index
`cats`. `AxisFrame::groups` keeps one open run per level and sizes that stack
with `open.resize(g.level, None)`. So a level arriving over the wire decides how
much it allocates. A frame whose level is `4294967295` would ask for a
A `FrameGroup.level` and a `GroupEnd`'s `level` name a category: they index
`cats`. `AxisFrame::groups` used to keep one open run per level and size that
stack with `open.resize(g.level, None)`, so a level arriving over the wire
decided how much it allocated. A frame whose level is `4294967295` would ask for a
multi-gigabyte vector, and the client would be taken down instead of the frame
being declined. ChatGPT Codex found this on #250, the PR that gave levels to the
wire (improvise-e265).
Since improvise-felo.6 `groups()` keeps only the headers it has received,
ordered by `(level, depth)`, so no number on the wire sizes anything and the
new `depth` field needs no bound. The level check stays for its other reason:
`cats[level]` names the group's category, which a browser `z` reads.
The bound is enforced at deserialization, by `#[serde(try_from =
"UncheckedAxisFrame")]`. That is the one place a frame enters from outside. A
level that is not below `cats.len()` fails to parse, with an error that names the
@@ -2353,7 +2374,7 @@ deserializes now carries no English at all.
`Hint { keys, command, caption }` is the unit: `keys` come off
`Keymaps::default_keymaps()`, `command` is the semantic value a localized or
non-terminal front end keys off, and `caption` is the default English — exactly
the `AxisColor`/`draw.rs::axis_fg` shape, with the decision shared and only the
the shape faces now have (`Face` → `draw.rs::face_style`, §10), with the decision shared and only the
last translation local (`invariant(semantic_palette_at_the_core_boundary)`).
### What that bought, and what it cost
@@ -2441,3 +2462,85 @@ the mode, the same shape `col_groups` uses for "no groups". `web/app.js` paints
rows it does not author; `src/ui/help.rs` keeps its ratatui styles and lost
~310 lines of content that is now shared. Neither end restates what the keys
are.
## 10. `face.rs` — appearance named by role (ADR 004)
ADR 004 (`context/adr/004-faces-and-themes.md`) is the decision; this section
is where its first two migration steps landed and what they settled that the
ADR left open (improvise-nn4s.1, improvise-nn4s.2).
### What is here
- `Face`: one variant per role. Where a domain enum already exists the face is
**parameterised by it** — `Face::Axis(Axis)`, `Face::WizardField(WizardFieldKind)`,
`Face::ModePanel(Panel)`, `Face::ModeTextEntryOf(TextEntry)`,
`Face::MinibufferOf(TextEntry)` — so adding an arm to that enum adds a face by
construction (clause 2). `Face::name` spells each one as a theme file keys it
(`axis.row`, `mode.text-entry.command`, `minibuffer.category-add`), and those
names are a compatibility surface once a theme can be downloaded (clause 9).
- `Face::parent`: the inheritance, **fixed in the vocabulary rather than stated
by the theme**. A theme that omits a face must still get the face's parent's
values ("a theme written today keeps working when a face is added tomorrow",
clause 9), and that is only true if the chain does not depend on what the
theme says.
- `Colour` (`Default | Named(AnsiSlot) | Indexed(u8) | Rgb`), `Weight`, `Slant`,
and `FaceSpec`, whose six attributes are all `Option`: `None` inherits, and
`Some(Colour::Default)` states "the surface" and stops inheritance. The two
are different answers and the type keeps them apart.
- `Theme { faces }` and `Theme::resolve(Face) -> ResolvedFace`, which walks the
chain once and returns every attribute concrete. **No front end walks the
chain** (clause 3). `resolve` takes a face and nothing else.
- `Theme::shipped()`: today's terminal rendering, palette-relative throughout
(clause 4). `the_shipped_theme_states_no_absolute_colour` holds that line.
- `Face::of_mode(&AppMode)` and `Face::of_minibuffer(&AppMode)`, both
exhaustive over `AppMode`, so a new mode fails the build here as it used to
in `draw.rs::mode_style` (improvise-zx8t).
### The JSON shape
Face names are the keys and a colour is spelled for a person: `"default"`, a
slot name (`"green"`, `"bright-black"`), a bare number for an index (`235`),
`"#rrggbb"` for `Rgb`. A misspelt face or attribute is **refused**, not
ignored (`deny_unknown_fields`, and `Face`'s `Deserialize` rejects an unknown
name): a file the user edits by hand should say where it is wrong rather than
silently draw the default. The faces map is a `BTreeMap` keyed on `Face`'s
derived order, so a re-download after an edit is a reviewable diff (clause 9).
The protocol crate derives and the front ends encode, so `serde_json` stays a
dev-dependency here.
### Choices the ADR did not make, made here
- **`minibuffer.*` inherits `mode.text-entry` directly, as the ADR says, and
the shipped theme pays for it.** The mode indicator is black text on an
accent, and the minibuffer is accent text on `Indexed(235)`, bold. So under
this chain every `minibuffer.<entry>` in the shipped theme states all three of
its attributes and inherits none of them from its parent. A first version gave
them a `minibuffer` base face instead. ChatGPT Codex caught it on PR #319: a
published parent is compatibility behaviour, so a theme written to the
accepted ADR that restyles `mode.text-entry` would have silently missed every
prompt. `a_theme_that_restyles_text_entry_reaches_every_minibuffer` pins the
accepted chain. Changing it is the user's call and belongs in the ADR, not in
this file.
- **`mode.tile-select` hangs directly off `mode`.** The ADR's decomposition
covers the thirteen DarkGray modes and the text-entry chain, and does not
place `TileSelect`.
- **The mode faces still state `Named(Black)` ink.** Clause 5 wants the twelve
`fg(Black).bg(accent)` sites to become `reverse` over the accent, but that is
not a no-op on a `TestBackend` buffer, and clause 8 names only one exception.
So it waits for improvise-nn4s.6 and a ruling.
- **No slot tables yet.** `Theme` has no `slots`/`dark`/`default_scheme`: they
are the browser's substitute for a terminal palette (clause 4), and the first
consumer is improvise-nn4s.5. Adding them then is a field with a serde
default, not a format break.
- **Only the faces some call site draws with exist.** The ADR's full vocabulary
(`emphasis`, `error`, `selection`, `grid.*`, `panel.*`) arrives with the
call sites that use it, because a face with no caller is a published name
with no evidence behind it.
### The category-add exception
`minibuffer.category-add` resolves to green in the shipped theme, where the old
literal was yellow: the user ruled the split an accident (ADR 004 Q1), and it
is clause 8's one declared exception. `AppMode::category_add` now carries
`MinibufColor::Green` too, so the browser, which still reads the wire colour
until improvise-nn4s.4, agrees with the TUI rather than drifting from it.
context/code/tooling.md
+209
View File
@@ -1032,3 +1032,212 @@ changes. Two rivals were run:
- reading every failure as stale fails the offline case (exit 1, wanted 2);
- dropping `--locked` fails the stale case (exit 0, wanted 1). Without
`--locked`, cargo quietly rewrites the lock and reports success.
## 7. The repo-stats pipeline (improvise-1xfb.5.1)
`scripts/repo-stats` reports churn, change coupling and doc growth over a window
of history: by default the last 7 days of `gh/main`, falling back to `HEAD`,
without merges. It reports and never gates. The user asked for code-maat to do
the analyses it has, and for improvise to be the engine for the rest. So the
pipeline is also an end-to-end run of improvise's headless product: `import`,
`script` and `export`.
### The shape
1. `git log --numstat --no-renames --no-merges` in code-maat's `git2` format.
2. code-maat runs `coupling` (with `--min-revs`/`--min-shared-revs`), `abs-churn`,
`age` and `soc`. Each writes one CSV into the output directory, named after
the analysis. code-maat is pinned in `flake.nix` as a `fetchurl` of the
1.0.4 standalone jar, run by a JRE. It is in the dev shell and in the
`.#repo-stats-tools` package, but not in the `ci` shell: the JRE is about
0.6 GB on Linux, and every CI job enters that shell.
3. The script adds `churn.csv`, one row per (commit, file) with the date, the ISO
week, the path (`entity`, code-maat's own name for it), the area, the kind
(prose, code or other), and the lines added and deleted. It also adds two
flags to code-maat's `coupling.csv`: `crosses-crate` and `agent-rules`.
4. `improvise import <dir>` (improvise-lpzu) merges the directory's five CSVs
into **one** sparse model, `repo-stats.improv`. The merge's `File` category
names each source, and the import makes one view per file, named after its
stem and filtered to it. That is the user's ruling: "Records are already
sparse: the model can just leave null/empty values for columns not in a
given CSV file." A record keeps no coordinate in a category its file lacks.
5. `improvise script` runs the checked-in `scripts/repo-stats.improvise`. It
adds the prose-to-code formula, builds four views on top of the per-file
ones and `export`s each to `report/*.csv`. The four names (`By area`,
`By kind`, `Area by week`, `By day`) must never be a file's stem:
`create-view` of an existing name replaces that view silently (gap 17). The
suite's view count, 10, fails if one does. None of the report reads the
merged `Default` view, which can be empty for a typical code-maat pair
(improvise-8jk6.15).
6. The text report is those exports laid out as tables. The code-maat rankings,
doc sizes and the coupling flags are added to it. `--json` gives the same
data as JSON.
The `.improv` file is the artifact. It opens in the TUI, and CI uploads it with
the CSVs.
**A window in which no file changed builds no model.** Its commits are all
empty (`--allow-empty`), so there is no churn to import and nothing for
code-maat to analyse. The script writes `churn.csv` with its header alone, skips
code-maat and improvise, and the report says so. It exits 0. Running improvise
over an empty CSV was the other option. It was not taken, because it would test
only how improvise treats a CSV with no records, which is not what this pipeline
exists to exercise.
**`--out` must hold no CSV the script did not write.** `improvise import .`
merges every CSV in the directory, so a stale or foreign one would silently
join the model. The script refuses one with exit 2 and names it. It refuses
rather than deleting, because deleting files in a directory the caller named is
the wrong default. Its own five CSVs are overwritten on every run, so a rerun
into the same `--out` works.
**No collisions, by construction.** A `churn.csv` record is keyed by commit and
file, which `--no-renames` makes unique. code-maat's records are keyed by
entity, by entity pair or by date. `File` separates the sources. So the import
keeps its default, `--on-duplicate refuse`. A collision would be a bug in this
script, and the import refuses it.
**What improvise computes and what it does not.** Everything that is a sum or a
ratio of sums is a view:
- churn by area, with `DeletionRatio = deleted / added` as a `_Measure` formula;
- prose, code and other, with `ProseToCode = prose / code` as a formula on the
`kind` category;
- area by ISO week;
- lines added per day from `churn.csv` next to code-maat's `abs-churn`. The two
columns must agree, and the suite checks that they do.
Two kinds of output stay in Python. The ranked code-maat tables (coupling, soc
and age) are read from code-maat's own CSVs, because improvise cannot sort rows
(gap 11 below). Doc sizes are a property of files at one ref, not of the window.
### Definitions
- **Areas**: first match wins, in the order of `AREAS` in the script.
`generated` comes first, so `harness/opfs-contract/Cargo.lock` counts as
generated and not as harness. `.beads/` and `site/` are generated too.
`docs/*.md`, `README.md` and `RELEASING.md` are "other docs", and `.codex/`
is agent docs and skills. A Rust test is a `.rs` file under a `tests/` or
`benches/` directory, or one named `tests.rs` or `*_test(s).rs`. A
`#[cfg(test)] mod tests` inside a source file counts as Rust src, because no
path can tell the two apart.
- **Prose** is `context/code`, `context/test-evidence`, other `context/`, other
docs, `repo-map`, and the agent docs and skills. **Code** is Rust src, Rust tests,
scripts/CI, and web and harness. `kb.pl`, generated files and other files are
neither.
- **The deletion ratio** is deleted lines over added lines. It is 0 when nothing
was deleted, and blank when nothing was added.
- **A crate crossing**: both paths are in a crate (`crates/<name>/`, the root
crate's `src/`, `tests/`, `benches/`, `examples/` or `Cargo.toml`, or
`harness/opfs-contract/`), and the two crates differ. A pair with one side
outside every crate is not flagged.
- **Agent rules**: both paths are in the agent docs and skills area. Such a pair
means one rule stated in several places.
- **Gotchas** are the numbered items between `## Gotchas` and the next `---` or
`##`. The longest gotcha is measured in characters. Most gotchas are one long
line, so a line count would say little.
- **Test-evidence entries** are the `###` and `####` headings.
### What fails, and what never does
- **The numbers never fail anything.** No budget is checked. Budgets are a
separate decision under the compaction epic, improvise-1xfb.5.
- **improvise refusing the pipeline's input exits 3.** That covers a non-zero
exit from `import` or `script`, and any `warning:` on their stderr. The
warning rule is there because a skipped record is data loss that exits 0
(gaps 3 and 4). It is a product regression, not a statistic.
- **In CI**, the suite runs on every PR in `nix-build` and gates like any other
suite. The report runs only on a push to `main`. Its step has
`continue-on-error`, so an improvise failure there marks the step failed and
the job and the deploy go on. Whether that failure should block the deploy
is a question for the user on the PR. The report appends to
`$GITHUB_STEP_SUMMARY`, and the directory is uploaded as the `repo-stats`
artifact.
- The report runs in `nix-build` because that job already builds
`.#improvise`. Its steps run under `nix shell .#repo-stats-tools`, which joins
that same build with code-maat, so nothing is built twice. It is one package
rather than two installables because `check-ci-graph.py` reads a counted
step as `nix shell <one> --command`. The job's checkout fetches full history
(`fetch-depth: 0`) for the report.
### Gaps in the headless product, and the workarounds used
Each of these was found by running the pipeline, and each is a finding against
improvise's command and model layer. Where a workaround exists, the script
names it.
1. **A forced `--category` value is still parsed as a number.** The commit
`00565796` imported as item `565796`, and `220944e2` imported as
`22094400.0`. Workaround: `churn.csv` writes commits as `g<sha>`.
improvise-8jk6.4.
2. **`import --formula` drops a formula that fails to parse, and exits 0.**
`--formula 'X = ('` saves a model with no formula and says nothing.
`add-formula` in a script refuses it and exits 1, so the pipeline's own
category formula is added there. improvise-8jk6.5.
3. **A record that lacks any category's value is skipped.** A merge of CSVs with
different headers, which is the user's sparse model, imports **no records**.
Closed by improvise-lpzu, whose sparse import this pipeline now uses. Until
it landed, every CSV was padded with `-` for the categories it lacked. A
related hole is still open: a record with no measure value stores nothing
(improvise-8jk6.14).
4. **An import that skipped records exits 0**, even when it skipped all of
them. It saves an empty model and prints only a warning. improvise-8jk6.6.
5. **`set-axis` and `set-page` accept names that do not exist, and exit 0.** An
unknown category, an unknown axis name (`rwo`) and an unknown item are all
silent no-ops. improvise-31oe.2.
6. **The command language does not read `|pipe quoting|`.**
`create-view |By area|` made a view named `|By`, and the second word was
ignored. Commands that take a fixed number of arguments ignore extra ones
without saying so. Workaround: `"double quotes"`. improvise-31oe.3.
7. **`create-view` puts every category it is not told about on Page**, with its
first item selected. That silently filters the view. Every view has to set
every category's axis, one `set-axis` line each. improvise-31oe.4.
8. **A view cannot hide a category's column.** A per-file records view shows
every category and measure in the model. improvise-31oe.5.
9. **`export` of a records view writes the editor's `+cat` and `+` columns.**
improvise-8jk6.7.
10. **`export` writes display text.** It uses the view's number format, so the
import's default `,.2` gives `"17,512.00"`. Workaround: `set-format .0` on
every exported view, and `FORMAT .3` on the ratio formulas.
improvise-8jk6.8.
11. **Nothing sorts.** Items stay in first-seen order, so weeks come out
newest first. A top-N table cannot be produced, so code-maat's rankings are
read from its CSVs. improvise-31oe.6.
12. **`--extract` has no ISO week.** It has year, month and quarter. Workaround:
`churn.csv` carries a `week` column. improvise-8jk6.9.
13. **`import --formula` targets only `_Measure`.** A formula on another
category needs `add-formula <category> …` in a script. improvise-8jk6.10.
14. **The saved model opens on the import's `Default` view**, which puts `File`
on rows and `area` on columns, and not on the view the script left active.
That is improvise-azhb.
15. **No folder import.** Closed by improvise-lpzu (PR #314), which this
pipeline now uses. Until it landed, the script named the five files and
generated its own per-file views.
16. **There is no headless output but a file.** `export` writes a CSV, and the
tables are laid out in Python. improvise-31oe.7.
17. **`create-view` of an existing name replaces that view, and exits 0.**
`Workbook::create_view` inserts into an `IndexMap`, so it overwrites without
a word. Folder import makes a view per file stem, so a report view named
like a stem would replace that file's view.
improvise-31oe.8.
### The suite
`scripts/repo-stats-test.sh` builds a fixture repository whose every number is
known, runs the whole pipeline, code-maat and improvise included, and asserts
improvise's exported tables. It runs in `nix-build`, which puts improvise and
code-maat on `PATH`. Locally it uses improvise from `PATH`, or else
`target/debug/improvise`, and it fails with a message when there is neither.
The fallback does not check how fresh that binary is, so locally the suite can
test a `target/debug/improvise` older than the tree. Rebuild after changing
improvise.
It does not build improvise itself. `checkers.sh` scrubs the environment to an
allowlist, and on macOS that removes what the linker needs: a `cargo build`
inside the suite failed with `library not found for -liconv`. That is a bug in
`checkers.sh`: its allowlist drops the Nix linker variables, and the
orchestrator has filed it separately. Build first with
`cargo build --bin improvise`; `cargo test --workspace` also leaves the binary
there. The suite is the slowest in `checkers.sh`, at about 25 seconds, most of
it JVM start-up for code-maat's four analyses per run.
The evidence and the rivals are in `context/test-evidence/tooling.md`.
context/code/tui.md
+43 -38
View File
@@ -104,46 +104,49 @@ appear. Three rules:
- `SHIFT`/`CONTROL`/`ALT` survive; every other modifier bit (`SUPER`, `HYPER`,
`META`) is dropped.
### Semantic-to-terminal colour, and the one that is wrong
### `face_style` — the one place a face becomes a terminal style
This file holds the mapping design-principles §5 names as the shape everything
else should imitate: **the decision lives in the shared layer as a semantic
enum, and only the last translation is local.**
ADR 004 replaced the four per-family maps this section used to describe
(`axis_fg`, `minibuf_fg`, `field_kind_fg`, `mode_style`) with one function:
`face_style(Face) -> Style` resolves the face against the shipped theme
(`Theme::resolve`, in `improvise-protocol`) and translates the result. That is
design-principles §5's shape with nothing left over: **the decision is shared,
and only the last translation is local** — `Colour` → `ratatui::Color` and the
six attributes → `Modifier`. The chain is never walked here (ADR 004 clause 3).
What the vocabulary is and why is `context/code/protocol.md` §10.
- `axis_fg(AxisColor) -> Color` — `Axis::color` is the single place each axis's
accent is decided; this is the single place axis colouring is *rendered*
(improvise-rml, gotcha 26). Before it, the tile bar and the category panel
each carried their own copy of the colour table with nothing keeping them in
agreement.
- `minibuf_fg(MinibufColor) -> Color` — the same shape for the minibuffer.
- `src/ui/import_wizard_ui.rs::field_kind_fg(WizardFieldKind) -> Color` — the
same shape again. `improvise-protocol` must not name a ratatui `Color`.
Three things to know.
`mode_style(&AppMode) -> Style` stays here for the same reason (`Style` and
`Color` are ratatui types and `improvise-command`, where `AppMode` lives,
carries no rendering-framework dependency), and since `improvise-zx8t` it is
**exhaustive**: the thirteen variants that take the default chrome are spelled
out rather than swept up by a `_ =>` wildcard, so adding an `AppMode` fails the
build here until someone chooses its colour. The arithmetic is the check a
prose count does not get: 13 + `Editing`/`RecordsEditing` + `CommandMode` +
`TileSelect` = 17, which is `AppMode`'s variant count. It read *fourteen* until
2026-09-22, and nothing could have caught that — a count written in prose is
checked by no one.
- **`Colour::Default` sets nothing.** It maps to "no `fg`/`bg` on the `Style`",
so the terminal's own colour shows through. That is also why a face with no
stated colour produces `Style::default()` and not a `Reset`.
- **The slot names follow ANSI, not ratatui.** ANSI slot 7 is `White`, which
ratatui calls `Gray`; slot 8 is `BrightBlack`, ratatui's `DarkGray`; slot 15
is `BrightWhite`, ratatui's `White`. `terminal_colour` is the one place that
translation is written.
- **The shipped theme is resolved once**, into a `OnceLock`, because there is no
theme picker yet (ADR 004 "Deferred"). When one arrives the theme becomes the
TUI's own state rather than a static, and a theme is still never on `App`'s
`ViewState` (clause 7).
It is still not quite the same shape as `axis_fg` and its two siblings, and the
difference is deliberate rather than pending. Those three map a *semantic enum*
the shared layer owns; `mode_style` matches `AppMode` itself, because there is
no `ModeColor`. `improvise-zx8t` offered making one, and it was declined here
for two reasons: the colour is the only thing at stake, and the ticket says to
decide it together with `improvise-k099`, which asks the same question of
`hint_text`/`label` and is not in this unit. Exhaustiveness is what the
repo map's "the compiler names every one you must fill in" promise needed; a
semantic enum is a separate, larger answer and is still open.
**The mode indicator is still exhaustive over `AppMode`**, now in
`Face::of_mode` rather than here: adding a variant fails the build there
(improvise-zx8t's guard, moved with the table). The thirteen DarkGray variants
are no longer one arm: they are the five faces ADR 004 derives
(`mode.resting`, `mode.text-entry`, `mode.panel.*`, `mode.overlay`,
`mode.transient`), which all still inherit the same colours from `mode`.
**The guard is the compiler, not a test.** Nothing asserts the colours, and a
test enumerating the variants would not fail when a fifteenth is added — only
the missing match arm does. Verified by deleting the `AppMode::Help` arm:
`error[E0004]: non-exhaustive patterns: `&AppMode::Help` not covered`.
**No-op evidence.** `face_tests::the_shipped_theme_reproduces_every_migrated_literal`
holds the old literals as a table and asserts the shipped theme draws each one,
which is clause 8's property. The one deliberate difference, the category-add
prompt turning green, is asserted by
`the_category_add_prompt_draws_as_the_item_add_prompt_does` rather than being
left out of the table silently.
**Still literal**: every other `Color::` in `src/` (the selection sites, the
panels' accents, the grid's headers, help, the welcome popup). They move as
their files are touched (improvise-nn4s.3, improvise-nn4s.6). Do not touch
`src/ui/grid.rs` for this while improvise-felo.6 is in flight.
### The minibuffer shares the bottom row with the status line
@@ -580,9 +583,11 @@ to make room for a marker. `ImportWizard::scroll` clamps to `len - 1`, so the la
and the list can never scroll away. It is by item, not by line, because the
wizard state cannot know the panel width that line-level clamping needs. The
residual is honest. A **single** item taller than the whole list area, roughly
1,500 characters at 80 columns, is still clipped at the panel's bottom. No
wizard warning is that long today: a collision tuple is bounded by twelve
categories plus labels, but a label value is not bounded.
1,500 characters at 80 columns, is still clipped at the panel's bottom. A
collision tuple has at most `MAX_CATEGORIES` coordinates (24 since
improvise-felo.1, up from 12) plus labels. With short names that is a few
hundred characters. Neither the names nor a label value is bounded, though, so
reaching the residual got easier when the cap doubled.
**Every write is clipped to the panel.** The widget draws through `Panel`,
whose `put` writes only inside the panel's inner rectangle, and its height
context/design-principles.md
+5 -4
View File
@@ -69,7 +69,7 @@ same effect without the early return.
three shapes; dispatch pattern-matches exhaustively.
- `CategoryKind` is `Regular | VirtualIndex | VirtualDim | VirtualMeasure | Label`.
Business rules (e.g., the 12-category limit counts only `Regular`) are
Business rules (e.g., the 24-category limit counts only `Regular`) are
enforced by matching on the enum, not by checking name prefixes. Virtual
categories (`_Index`, `_Dim`, `_Measure`) always exist: `_Index` and `_Dim`
support drill-down/records mode; `_Measure` holds numeric data fields and
@@ -99,7 +99,8 @@ sort. Property tests verify this invariant.
- `CellKey::new()` is the only way to build a key — it always sorts.
- `Category::add_item()` deduplicates by name and auto-assigns IDs via a private
counter. External code cannot fabricate an `ItemId`.
- `Model::add_category()` checks the 12-category limit before insertion.
- `Model::add_category()` checks the category limit (`MAX_CATEGORIES`, 24)
before insertion.
- `Formula::new()` takes all required fields; there is no default/empty formula
to accidentally leave half-initialized.
@@ -268,8 +269,8 @@ why `doctrine(mirror_the_cmd_line_for_line)` is a rule about *resolvers* — the
narrow, unavoidable residue — and never a licence to restate a table.
**What legitimately stays per-framework.** Painting, event wiring, and the
mapping from a semantic value to a framework type: `AxisColor → ratatui::Color`
in `draw.rs::axis_fg`, `MinibufColor → minibuf_fg`. That mapping is the shape
mapping from a semantic value to a framework type: a `Face`, resolved by the
shared `Theme`, becomes a `ratatui::Style` in `draw.rs::face_style` (ADR 004). That mapping is the shape
to imitate — the *decision* is in the shared layer as a semantic enum, and only
the last translation is local. Anything larger than a lookup table is a
candidate for the store.
context/plan.md
+1 -1
View File
@@ -257,7 +257,7 @@ The second weekend (or the second half of the first weekend) is reserved for wha
The following are real issues that have been identified but are explicitly not launch-blocking. Do not start any of them as part of this plan. They can be addressed after launch based on what real users actually trip over.
- **Do not** raise or restructure the `MAX_CATEGORIES = 12` limit, even though it's tight for some real use cases.
- ~~**Do not** raise or restructure the `MAX_CATEGORIES = 12` limit, even though it's tight for some real use cases.~~ **Superseded** by the user's decision of 2026-09-25 ("12 categories is feeling small, we should increase to 24"). The limit is 24 (improvise-felo.1).
- **Do not** refactor the `ExecuteCommand` vim command match ladder, even though it duplicates registry logic.
- **Do not** fix the hardcoded `"Measure"` category name in `evaluate_aggregated`, even though it's the one place in the code that assumes a specific user-facing name.
- **Do not** change `Model::evaluate` from a linear formula walk to a HashMap lookup, even though it's O(N·M·F) in the render hot path.
context/repo-map.md
+55 -29
View File
File diff suppressed because one or more lines are too long
context/test-evidence/cli.md
+251
View File
@@ -407,3 +407,254 @@ named, and no output file. It has two siblings. `..._sums_duplicates_when_told_t
asserts `= 30` and no `Occurrence`. `..._keeps_duplicates_separate_when_told_to`
asserts the category, `= 10` and `= 20`, and no `= 30`. So the two flags cannot
both be summing.
### a_headless_import_of_a_folder_merges_its_csv_files_and_repeats_byte_for_byte
improvise-lpzu, at the process boundary. The folder holds code-maat's three
schemas (coupling, age and abs-churn) and a `README.txt`. At `e65d87fd`'s
parent the run failed with `Cannot read '…/maat': Is a directory (os error
21)`, because a single path went straight to `read_to_string`. The test
asserts:
- sparse Data lines for all three schemas, with no README;
- the four views, `Default` first and initial, then one per file;
- `File: filter, coupling`;
- a second run, with the first run's `maat.improv` now inside the folder,
writing the same bytes.
Rivals R2, R5, R6 and R7 each fail it.
### a_headless_import_takes_files_and_folders_together_and_refuses_an_empty_folder
improvise-lpzu. A file argument and a folder argument together are one merge,
and the extra file's records carry its own `File` item. Before the fix this
failed with `Multi-file import only supports CSV files`, because the folder was
not a `.csv` path. An empty folder exits non-zero, writes nothing, and names
itself: `no CSV files in folder …/empty`. Rivals R2, R4 and R5 each fail it.
### a_saved_per_file_view_projects_only_its_file
improvise-lpzu (`tests/folder_import_views.rs`). A folder import is saved,
reloaded and projected through `project::grid_in`, the one derivation the TUI
grid and the browser both draw from (gotcha 44). This is why no separate TUI
or browser test was written. For each file's view:
- the page bar carries `File` fixed at that file with nothing to cycle
(`items: None`, gotcha 27);
- the drawn cells are that file's alone: `2.00, 7.00` for age and
`5.00, 3.00` for coupling.
Rivals R2, R7 and R9 each fail it. Since the review of `4e68372c` the fixture
also holds a header-only `soc.csv`. Its view must carry `File` at `soc`, be a
records view, and draw zero rows. Rivals F7 and F8 fail that half.
### a_csv_column_forced_to_category_keeps_its_text_verbatim
improvise-8jk6.4. Before the fix, `commit,lines` with `00565796` and
`220944e2` forced to Category imported as items `565796` and `22094400.0`.
The CSV parser had typed both as numbers, and the category was named from the
number. On the base this failed with the items line `- |565796|, |22094400.0|`.
The test also asserts both Data lines, which proves the measure `lines` still
reads 3 and 5 through the same records.
- **R1**, a category named from the typed value, fails it.
- **R2**, a CSV measure read as a JSON value (`as_f64` on the text), fails it
on the Data lines.
### a_json_number_forced_to_category_is_named_as_json_writes_it
improvise-8jk6.4, the JSON half. This is not a bug demonstration: it passed
on the base, and it pins that the fix left JSON alone. JSON keeps no source
text for a number, so `42`, `1.5` and `1e3` name items `42`, `1.5` and
`1000.0`. `1e3` is the discriminating value, because its source text, its f64
display (`1000`) and its JSON display (`1000.0`) all differ. **R10**, naming a
number by its `f64` display, fails it on `1000.0`.
### an_import_formula_that_does_not_parse_refuses_names_it_and_writes_nothing
improvise-8jk6.5. Before the fix, `--formula 'Y = x +'` was dropped: the run
printed `Saved to …` and exited 0. That is the base failure (`a dropped
formula exited 0`). The test passes a good formula too (`Z = x`) and asserts
it is not named, so a refusal that lists every formula cannot pass.
`expected` comes from pest's error, which proves the parse error is carried
and not only the formula text. **R4**, a build that drops what does not
parse, fails it. Since improvise-8jk6.17 it passes two bad formulas, so
**R5**, naming only the first, fails it here too. At `efaa5050` only the io
unit test saw R5.
### an_import_that_keeps_no_record_refuses_and_says_why
improvise-8jk6.6. On the lpzu base `[1, 2, "three"]` already exited 1, so the
exit-code half was not the bug. The reason was: `At least one category must
be accepted`, which is true only because no fields exist, and which points
the user at a fix that cannot help. The base failure was on the message.
**R7**, removing `nothing_to_import` from the build, fails it the same way.
### an_import_that_skips_some_records_warns_with_a_count_and_still_writes
improvise-8jk6.6, the partial case. It passed on the base. It pins the
default the user kept on PR #321 ("Add a --strict flag"), so that changing
the default is a decision and not a drift. It asserts the warning's count (`1 of 3`), exit 0,
and a Data line from the record *after* the skipped one. That last check
shows the skip did not end the import early. **R9**, making any skip fatal,
fails it.
### a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not
improvise-lpzu, after the user's ruling on PR #314 ("we'd want file identity").
At `4e68372c`, a folder holding one CSV took the single-file path, because
`import dir/` was exactly `dir/*.csv`. That document had no `File` category and
no per-file view, and the test failed at the `## Category: File` assertion. The
other half pins the direction the ruling did *not* change: the same CSV named
directly imports without any `File`. The rival that merges every all-CSV
import, lone files included, fails that half ("a lone file keeps today's
shape"). So the test separates "a folder merges" from "everything merges".
### a_pipe_quoted_argument_is_one_argument_without_its_pipes
improvise-31oe.3. Red before the fix: the old tokenizer split on whitespace
only, so `create-view |By area|` built `CreateViewCmd(["|By", "area|"])`. It
asserts the whole built command, so a lexer that joined the words but kept the
pipes fails as well as one that split them. The second case has two quoted
arguments side by side, so a lexer that reads one pipe piece per line fails.
Rival: `bare_word` pushing `|` as a plain character. Fails here and in
`a_coordinate_may_pipe_quote_either_half`.
### a_pipe_quoted_argument_reads_the_escapes_improv_does
The three escapes `pipe_quote` writes, `\|`, `\\` and `\n`, each in its own
case. Red before the fix. Rival: `pipe_body` returning the raw body without
`unescape_pipe_body`. Fails here, at the first case, and nowhere else in the
crate, which is why the case exists. A lexer that treated `\|` as a closing
pipe would also fail the first case, because the pipe after `b` then opens a
piece that never closes.
### a_command_given_more_arguments_than_it_takes_is_refused
improvise-31oe.3's second half. Red before the fix: `create-view By area`
parsed and dropped `area`. It asserts the refusal opens with the command's name
and quotes the surplus word, and that the pipe-quoted spelling of the same line
is the control that parses. Rival: `MaxArgs::admit` answering `Ok` always.
Fails here, in `no_command_silently_drops_an_argument_it_was_given`, and in
`a_colon_line_naming_something_missing_refuses_on_the_status_line`.
### no_command_silently_drops_an_argument_it_was_given
The rule is one rule only if every registration obeys it, so this enumerates
every registered name against every candidate argument list, on both arms. For
each list a command accepts, it appends **four** surplus words. One would be
absorbed by an optional argument (`add-formula ["1"]` plus one more is a valid
two-argument call), and four is more than any optional span in the registry.
The last word must be named in the refusal. The exceptions are stated, not
inferred: the `Any` list, and three commands whose own shape refusal runs
first. Rivals, each run: declaring `add-item-in-group` as `Any` fails here,
naming the dropped list. Removing the check from the interactive arm alone
fails here and in the `:` line test. A test of the pure arm alone would have
missed that.
### a_command_naming_something_the_document_lacks_is_a_refusal
improvise-31oe.2 and its siblings: `set-axis`, `set-page`, `hide-item`,
`show-item`, `toggle-group` and `switch-view`, each naming a missing category,
item, group or view. Red before the fix at the first row, `set-axis Regoin
column`, with an empty status. Each row asserts the refusal flag, the missing
name in the message, and that the document's `.improv` text and the active view
are unchanged. The last two catch a refusal written after the write. The
control is `a_command_naming_what_the_document_has_is_not_refused`, whose group
row is the one that caught a group check reading only `Category::groups`:
`add_item_in_group` registers no `Group`. The checks are `effect::unknown_name`,
run by `RequireKnown` when the batch applies. Rivals, each run: the
`Needs::Item` arm answering `None` fails at `set-page ["Type", "Nowhere"]`. The
category test (`try_axis_of(..).is_none()`) never firing fails at the first
row. The `Needs::View` arm answering `None` fails at the `switch-view` row.
`RequireKnown` placed after the write, or refusing without `refuse_and_stop`'s
abort, fails the document-unchanged assertion.
### a_colon_line_naming_something_missing_refuses_on_the_status_line
The TUI path for both tickets, through `ExecuteCommand` and `App::apply_effects`,
not through the command alone. It asserts the refusal flag, the name on the
status line, and the mode back at Normal. Rivals: an unknown category accepted
fails at `:set-axis Regoin row`, and the arity check removed from the
interactive arm fails at `:create-view By area`.
### an_unknown_category_axis_or_item_exits_non_zero_and_names_it
The three repros from improvise-31oe.2, at the process boundary. Red against
the `gh/main` source with this test file: `set-axis Regoin row` exited 0 with an
empty stderr. The control, `set-page Region East`, must exit 0. Without it, a
fix that refused every `set-page` would pass.
### a_pipe_quoted_name_reaches_the_document_whole
improvise-31oe.3's repro at the process boundary: the saved document must carry
`## View: By area` and no `|By`. The second half, `create-view By area`, must
exit non-zero naming `"area"`. Red against the `gh/main` source: the document
carried the view `|By`.
### command::cmd::registry::tests::a_malformed_coordinate_is_refused_not_dropped
Codex's P2 on PR #316: `set-cell` and `parse_cell_key_from_args` (`clear-cell`,
`yank`, `paste`, `drill-into-cell`, `commit-cell-edit`) built coordinates with
`filter_map(split_once('/'))`, so `set-cell 1 Type/Food unexpected` dropped
`unexpected` without a word. The test names every caller, a half-empty
coordinate, and a control line that must still parse. Rival: the `filter_map`
restored in `parse_cell_key_from_args` fails at the first line,
`set-cell 1 Type/Food unexpected dropped "unexpected" without a word`.
### command::cmd::text_buffer::tests::colon_refuses_a_malformed_coordinate_and_leaves_the_cursor_cell_alone
The PR #316 review's blocker: on `:`, `parse_cell_key_from_args` failing sent
`clear-cell`, `yank`, `paste` and `drill-into-cell` into the cursor fallback,
which ignores its arguments. `:clear-cell Type/Clothing Month/Feb junk` cleared
the cursor's Food/Jan without a word. The test runs each line through the full
`:` path and asserts a refusal naming `"junk"`, both cells intact, nothing
yanked and no drill opened. Rival: `only_when_omitted` made unconditionally
`Ok` fails it at the first line.
### command::cmd::text_buffer::tests::colon_lets_a_later_segment_use_what_an_earlier_one_created
The review's second blocker: a `:` line is built before any segment runs, so the
name checks in `execute` refused `create-view Second . switch-view Second` and
`add-item Type Tools . set-page Type Tools` after the first half landed. The
checks now run when the effect applies. Rival: the checks back in `execute`
fails both lines with `No view named "Second"` and `Unknown item 'Tools'`.
### a_strict_import_that_skips_a_record_refuses_and_writes_nothing
improvise-8jk6.6, the user's ruling on PR #321. Written before `--strict`
existed, it failed on clap's unknown-argument error. It asserts the refusal
names the flag and the same count as the default warning, and that the output
path does not exist.
- **S1**, the flag parsed but ignored, fails it on the exit status.
- **S3**, refusing only after `persistence::save`, fails it on the output
path. That is the "a strict refusal still writes" contract this PR decided
against.
### a_strict_import_that_skips_nothing_still_writes
The control for the entry above. Without it, a `--strict` that refuses
**every** import passes that test. **S2** is that rival, and it fails here.
It failed before the flag existed too, but only on clap's error, which is not
the discriminating failure. The discriminating one is S2.
### a_csv_decimal_measure_imports_as_its_number
improvise-8jk6.17 (F3), from the adversarial review of #321. The fixture has
decimal CSV measures (`1000.50`, `-2.25`). With `number_in`'s `f64` branch
disabled (integers only), no headless test failed at `efaa5050`, and a decimal
measure was silently dropped. **F3** now fails this test and the next one.
This is coverage, not a bug: it passed when written.
### import_json_on_a_csv_reads_its_numeric_column_as_a_measure
improvise-8jk6.17 (F1). `improvise cmd 'import-json f.csv'` reaches
`ImportJsonHeadless`, a third CSV entry point besides `import`. **F1**
restores its CSV branch to the JSON constructor via
`ImportPipeline::new(Value::Array(recs.to_vec()))`. That still compiles,
because `CsvRecords` derefs to a slice. `Revenue` then becomes a category of
text items, and this test fails on its Data lines. At `efaa5050` only
`import_headless_refuses_records_that_collide` failed under F1, and only by
accident. The `## Category: Revenue` assertion names the symptom directly.
This is coverage: it passed when written.
context/test-evidence/clients.md
+16
View File
@@ -6605,3 +6605,19 @@ commits.
`target.view`), run across the whole workspace: fails **only** this test.
- A composer that hard-codes `"Default"` fails here as well. Neither view in
this test is `Default`, so the default cannot satisfy it by accident.
### a_download_commit_keeps_the_host_session_mirror_in_lockstep
Codex on PR #329: `LocalThenRequest` closed and cleared the browser's download
prompt, but the bytes-only request moved no corresponding state in the host's
session mirror. The test opens SavePrompt through the real browser key path,
types a nonempty filename, and commits it. It asserts both that the response is
still `DocumentBytes` and that the host mirror is Normal with an empty save
buffer. Another session then writes a cell; the requester folds that later
projection and must still equal its host mirror.
- **Rival** (the pre-fix download handlers, which returned bytes without
closing the mirrored prompt): fails at the host-mode assertion with
`SavePrompt` versus `Normal`, before the later projection.
- The nonempty filename proves Enter commits rather than taking the empty-name
refusal, and matching `DocumentBytes` preserves the bytes-download ruling.
context/test-evidence/command.md
+16 -3
View File
@@ -2479,8 +2479,8 @@ from `DocumentSnapshot` — it is no longer only the document.
**The fixture detail that matters**: a *duplicate* `AddCategory` is idempotent, so
the obvious way to reach the refusal proves nothing. This fills to
`MAX_CATEGORIES` (12) and overflows. And the predecessor is an `AddItem` rather
than a `SetCell`, because adding eleven categories reshapes the view enough that
`MAX_CATEGORIES` and overflows. And the predecessor is an `AddItem` rather
than a `SetCell`, because filling to the limit reshapes the view enough that
a record-addressed predecessor is refused `RecordGone` **first** — hiding the arm
under test behind an earlier one.
@@ -3505,7 +3505,7 @@ handshake. The stronger form fails under both injections.
improvise-75ld. `App::apply_effects` bumped `layout_version` and raised
`model_state.dirty` off the `EffectKind` **before** the `abort_effects` check, so
the refusing effect counted itself. Two arms, and the second is the reachable one:
a 13th category is refused (`MAX_CATEGORIES`) and aborts, while a *duplicate*
a category past `MAX_CATEGORIES` is refused and aborts, while a *duplicate*
`add-cat Region` returns `Ok` and does not abort at all — a pure no-op that still
refused every other client's next `Command::Model` with `Rejection::LayoutMoved`
and made the browser worker rewrite the whole OPFS document (gotcha 41).
@@ -7214,3 +7214,16 @@ cleared the cell cannot pass.
`the_shipped_demo_seeds_a_worker_whose_raw_cells_are_settable` wrote with
`view: "Default"` against `web/demo.improv`, whose view is `Plan`. It passed
only because a stored record ignored the view name. It now names `Plan`.
### the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy
improvise-felo.6. The windowed frame carries the runs open above the window;
those are now keyed by `(level, depth)`. Rival: `axis_frame` closing carried
runs by level alone. Only this test fails; the existing agreement test has one
group per level, where the two keyings coincide.
### z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again
`z` through a closed Year > Quarter > Month hierarchy, cursor at row 1. Rival:
`toggle_group_collapse` never removing a fold; the final assertion (no fold left
behind once the quarter is closed again) fails.
context/test-evidence/core.md
+163 -6
View File
@@ -710,11 +710,6 @@ Applying `next` four times is the identity from every starting point and no
navigable axis is a fixed point, so no variant is stranded or skipped. Together
with the entry above this pins the rotation without enumerating it twice.
### every_axis_gets_its_own_color
Distinct axes must be distinguishable at a glance; `color()` is the only reason
`AxisColor` exists, so a collision defeats it.
### a_routing_formula_over_a_group_sums_the_members_answers
improvise-4zl7, and the hand-computed half of its evidence. Every number in
@@ -1865,5 +1860,167 @@ were run against the whole `improvise-core` suite:
**What it cannot see is the allocation itself.** A visitor that clones the
stack at every row returns the same answers, so no assertion here can tell the
two apart without a counting allocator. The copy-free property is held by the
shape: the visitor receives a borrowed `&[Option<usize>]`, and no owned stack
shape: the visitor receives a borrowed slice of open runs (`&[OpenRun]` since
improvise-felo.6), and no owned stack
leaves the walk. It is not held by a test.
### a_parent_group_covers_every_leaf_beneath_its_child_groups
improvise-felo.6. `items_in_group` read only the items whose own `group` named
the group, so a year covered nothing: its months sit in quarters. Rival run
against `improvise-core`: the direct-only filter restored. Four tests fail —
this one, `a_hierarchy_opens_closed_as_one_fold_per_root` (the 2024 fold reads
empty), `collapsing_a_parent_folds_its_children_whatever_their_state` and
`hidden_items_leave_every_fold_above_them`. The fixture adds `2024-03` last, so
the year and its first quarter are non-contiguous in insertion order.
### a_year_quarter_month_hierarchy_nests_its_runs
improvise-felo.6: the whole marker sequence of a Year > Quarter > Month axis
opened to depth 2, compared exactly. Rival: every header inside a category
emitted at depth 0 (a flat hierarchy). This test fails, and so do
`pruning_keeps_every_surviving_rows_runs_at_every_level` (the `NESTED` fixture),
`the_nested_fixture_draws_runs_inside_one_category` and
`pruning_drops_an_empty_inner_group_and_keeps_its_parent`.
### a_hierarchy_opens_closed_as_one_fold_per_root
The user's ruling on improvise-felo.6: groups start closed. Two rivals fail it.
The old open default (`inherits_collapsed` always false) fails twelve tests
across `improvise-core` and `improvise-io`, this one among them. The INNERMOST
collapsed group folding instead of the outermost (`.rev()` on the fold search)
fails this one, `hidden_items_leave_every_fold_above_them` and
`the_group_under_the_cursor_is_the_innermost_at_every_depth`: under it a closed
year shows its quarters as rows.
### a_groups_depth_opens_it_and_n_minus_one_levels_beneath
`expanded(2): Time/2024` opens 2024 and its quarters, not its months. Rival: an
ancestor reaching `k <= n` levels instead of `k < n`. Only this test fails, which
is why it exists: the boundary is the whole semantics of N.
### a_toggle_overrides_what_a_group_inherits_either_way_and_toggling_back_clears_it
`z` writes `Collapsed` or `Expanded(1)`, or removes the fold where the flip
lands on what the group inherits. Rival: never remove. This test and
`command::cmd::grid::tests::z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again`
fail; every layout test passes, since a redundant fold draws the same grid. What
the rival breaks is the file (a line per toggle, forever) and `z z` being the
identity on the view.
### pruning_drops_an_empty_inner_group_and_keeps_its_parent
`prune_empty` at depth: an empty quarter goes, header and all, and its year's
header stays for the sibling quarter that has data. Rival: `prune_axis_items`
keying open runs by level alone. This test and the `NESTED` enumeration fail.
## `model/id.rs` (ADR 007 clause 4, improvise-cq1q.2)
These tests were written before `slugify` and `IdAllocator` had bodies. With
`todo!()` bodies all eleven failed. Each rival below was then injected into the
finished module, run against `cargo test -p improvise-core --lib model::id`, and
reverted. The reasoning is `context/code/core.md` §20.
### model::id::tests::slugify_case_folds_rather_than_lowercases
Rival: `display.to_lowercase()` in place of `unicase`'s full folding. Only
this test fails: `left: "strae"`, `right: "strasse"`. `ß` lowercases to itself
and step 3 strips it. `slugify_follows_the_adrs_examples` cannot see this rival,
because each of its displays lowercases and folds alike.
### model::id::tests::slugify_collapses_a_run_of_spaces_to_one_underscore
Rival: every space becomes `_`, with runs not collapsed. Only this test fails:
`"time___period"`. No other fixture has two spaces in a row.
### model::id::tests::slugify_keeps_an_underscore_the_display_already_had
Rival: `.replace("__", "_")` after the space step, which reads answer 21 as
"no doubled underscores". Only this test fails: `left: "a_b"`. The rule
collapses runs of *spaces*. An underscore that was in the display is kept.
### model::id::tests::the_suffix_is_the_next_counter_value_not_the_lowest_free_number
Rival: the lowest free `<base>-<n>` for each base, which revision 5 of the ADR
proposed and answer 28 replaced. Three tests fail: this one (`"west-1"` against
`"west-2"`), `an_empty_slug_takes_its_kind_and_the_shared_counter` (`"item-1"`)
and `a_slug_taken_by_another_kind_takes_a_suffix` (`"time_period-1"`). Only
interleaving two bases separates one shared counter from per-base numbering,
and this test is the one that does it on purpose.
### model::id::tests::a_claimed_id_keeps_its_spelling_and_generation_steps_past_it
Rival: `claim` validates the id but does not record it. This test and
`claim_refuses_a_duplicate_a_malformed_and_a_reserved_id` fail. The second
`East` is issued `east-1`, the file's own id, which is the collision answer 29
forbids: "doing otherwise will corrupt the data section".
### model::id::tests::generated_ids_are_distinct_well_formed_and_never_reserved
Rival: `generate` does not record the id it issues, so nothing is retired. Five
tests fail, this proptest among them (`reissued`), along with every
suffix-order test. The proptest is the one that states the property itself: no
id is issued twice in a session, whatever the displays.
### model::id::tests::two_allocators_fed_alike_generate_alike
This test builds two allocators independently, so they share no hash state,
rather than calling one twice. `seen` is a `HashSet`, but it is only ever
queried and never iterated, so its order cannot reach an id (gotcha 18). No
rival of the current shape breaks this test. It guards against a future
change that picks a candidate by walking `seen`.
### model::id::tests::a_long_claimed_run_of_suffixes_is_walked_without_recursion
This is the Codex P2 on PR #318 (review of `a2ddb06e`), reproduced by the adversarial review of
`689480bf`. `IdAllocator::suffixed` recursed once per taken candidate, and the
call was not in tail position. A new-version file saved from a non-ASCII import
claims `item-1` … `item-N`, and the counter restarts at 1 on load, so the
next empty-slug generation walks all N. Against the recursive version this test
(N = 20,000) aborted the whole test binary: `has overflowed its stack` /
`SIGABRT`. It is an abort, not a failed assertion, so no test could catch it.
The loop passes, and still advances the counter on every candidate it tries.
The second assertion, `category-20002`, pins that: a loop that restarted from
a local counter on each call would answer `category-1`.
### model::id::tests::claim_refuses_an_id_that_only_begins_like_a_virtual_one
Found by the adversarial review of `689480bf`. With `claim` rewritten to
exempt any id that begins with a virtual id (`VIRTUAL_IDS.iter().any(|v|
id.starts_with(v))`), all 11 earlier tests passed, because
`claim_refuses_a_duplicate_a_malformed_and_a_reserved_id` tries only `_draft`.
This test fails on that rival (`left: Ok(())` for `_indexer`). The match is
exact by the ADR's own wording: answer 23 says no other id may begin with `_`.
### model::id::tests::a_display_with_leading_spaces_never_slugs_into_the_reserved_space
Rival, from the same review: strip the leading `_` before the space step, so
that ` Draft` becomes `_draft`. This test fails (`left: "_draft"`), and so does
`generated_ids_are_distinct_well_formed_and_never_reserved`. Before this test,
the proptest was the only thing that saw the rival.
## Displays beside keys (ADR 007 clause 2, improvise-cq1q.3, first slice)
Each renderer test below builds a projection whose display differs from its key.
The test-only way to get one is `set_display`, since no production path makes
them differ yet. Every test failed before the change, with the key drawn where
the display should be: `left: "Region"` against `right: "Sales Area"` in the
projection and wasm tests, and a missing "Sales Area" or "Overview" in the TUI
buffers.
### ui::project::tests::the_category_panel_shows_a_display_and_keeps_the_key_commands_name
It finds the row by `name == "Region"` before asserting on the display. A rival
that writes the display into `name` fails the lookup, which covers the half that
keeps commands working. `_Measure` is checked to fall back to its own key.
### ui::view_panel::tests::a_view_row_draws_its_display_and_marks_the_active_view_by_its_key
Rival: the active marker compares `summary.display == active_view`. This test
fails on that rival (no `▶ Overview`), because the active view is a key
(`ViewState::active_view`) and its display differs from it.
### render::tests::the_panels_paint_displays_and_keep_keys_for_what_a_command_names
The browser half of the same three panels. It also asserts that the view row is
still `active` when found by key.
context/test-evidence/formula-and-io.md
+444 -12
View File
@@ -882,6 +882,11 @@ The control: a **lone** `\r` is itself a record terminator to this reader, so
it is not part of a CRLF pair, and it is why "two records, clean values" is a
real observation rather than a coincidence.
Since improvise-8jk6.4 the parser yields text, so the assertion is that
`Revenue` is the text `1`. A kept `\r` would make it `1\r`, which fails that
assertion, and which `number_in` would not read as a number where a measure
wanted one.
### an_empty_file_is_no_records_and_no_warnings
An empty file is no records and no complaint. The wizard's own preconditions
@@ -919,12 +924,17 @@ The consequence downstream is deliberate and worth stating: a column carrying
Label rather than a Measure. That is the honest answer — the column is not
numeric — and it is visible in the wizard, where a fabricated zero was not.
Since improvise-8jk6.4 the numeric rule is `csv_parser::number_in`, applied
through `typed` only where a number is wanted, and the parser itself yields
text. So this test asserts that `typed` leaves each token as text. The
`unwrap_or` fault would be injected into `number_in` now.
### an_ordinary_float_is_still_a_number
The control for the entry above. Guarding non-finiteness by deleting the float
branch entirely would also pass that test, so this one pins that `1000.50` is
still a `Number` and that `0` — the value the bug invented — is still reachable
when the file really says so.
when the file really says so. Since improvise-8jk6.4 both go through `typed`.
### a_repeated_header_is_renamed_rather_than_overwriting_its_twin
@@ -1002,7 +1012,7 @@ the error, rather than moving to a `Done` step with nothing to confirm.
### wizard_blocks_advance_when_over_category_limit
Bug improvise-mzv: with more than `MAX_CATEGORIES` (12) accepted Category
Bug improvise-mzv: with more than `MAX_CATEGORIES` (12 then, 24 since improvise-felo.1) accepted Category
proposals, the wizard let the user advance through every remaining step and
`build_model()` only failed at the final confirm. Advancing past
`ReviewProposals` must be blocked, with a message telling the user to mark
@@ -1026,6 +1036,14 @@ every blank column value in a CSV import grew a phantom "null" item. Null means
*absent*: the record must be skipped and counted, exactly as if the key were
missing.
**The skip is superseded by improvise-8jk6.2.** Null still means absent, and
still invents no item. But an absent coordinate no longer drops the record: it
keeps its cell without that coordinate. So the test now asserts both cells
(`region=East` and the bare `_Measure=revenue`) and no warning. The rival that
restores the skip (a present-but-blank category returns the record as
unplaceable) fails here, beside
`a_record_with_no_value_for_a_category_is_kept_without_that_coordinate`.
### build_model_literal_string_null_is_a_real_category_item
The other side of improvise-ych: the *string* `"null"` is an ordinary value — a
@@ -1038,7 +1056,9 @@ null" would pass the test above.
improvise-ych, third arm. An empty string is the JSON spelling of a blank cell —
the CSV parser already turns a blank field into `Value::Null`, so the two inputs
must agree. An item with an empty name is unnameable in the grid, so it is
treated as missing rather than created.
treated as missing rather than created. Since improvise-8jk6.2 "missing"
means the record keeps its cell without a `region` coordinate. It is no longer
skipped, and the same skip-restoring rival fails here.
### build_model_sums_records_that_share_every_coordinate
@@ -1107,12 +1127,19 @@ and scrolls them. It was red at compile time against the old `Option<String>`
signature, then red against a `Vec` that still held one elided sentence. The
rival **elided after three** fails it.
### build_model_warns_about_records_skipped_for_missing_category
### improvise-5j9's skip warning, retired by improvise-8jk6.2
improvise-5j9: skipping a record for a missing category value used to be silent,
so a 1000-row import could drop rows with no user-visible sign. The warning must
name the count, the total, and the offending field(s) — a bare "some records were
skipped" is not actionable.
so a 1000-row import could drop rows with no user-visible sign. The warning had
to name the count, the total, and the offending field(s); a bare "some records
were skipped" is not actionable. Its test,
`build_model_warns_about_records_skipped_for_missing_category`, was **deleted**
by improvise-8jk6.2. A record missing a category value is now kept without that
coordinate, so there is no skip left to report. Its successor is
`a_record_with_no_value_for_a_category_is_kept_without_that_coordinate`, which
asserts no warning at all. `build_model_record_with_missing_category_value_skipped`
became `build_model_record_with_missing_category_field_is_kept` and now asserts
the kept cell rather than only the East one.
### is_does_not_swallow_the_start_of_a_longer_word
@@ -1355,7 +1382,7 @@ Faults injected, both against the fixed tree:
### a_document_at_the_category_limit_plus_a_label_reloads
The P1 half, and the reason improvise-dy3j was not merely a lost flag. `Label`
is outside `MAX_CATEGORIES = 12` by design — `Model::add_label_category` has no
is outside `MAX_CATEGORIES` (12 then) by design — `Model::add_label_category` has no
limit check, and the import wizard's `proposed_category_count` scores a label
as zero — so a document with twelve categories and one label is legally
constructible and the wizard will build one from any CSV carrying a free-text
@@ -1364,7 +1391,7 @@ refused, and `parse_md` returned
`Err("Maximum of 12 categories reached")`: **the document was permanently
unloadable.** Reproduced at `d46ec59` with exactly that error.
The second assertion (`regular_category_names().len() == 12`) is what keeps the
The second assertion (`regular_category_names().len() == crate::model::MAX_CATEGORIES`) is what keeps the
first honest. A "fix" that simply raised or removed the limit would let the
document load while putting a high-cardinality label into the pivot count, and
this assertion fails on it.
@@ -1436,7 +1463,16 @@ The fix reports and does not relocate: the row still lands with the coordinates
it does have, exactly as before. Changing where such a row is placed is a
behaviour change the ticket did not ask for.
### import::wizard::tests::a_row_dropped_for_a_missing_category_is_not_charged_to_the_date_tally
### import::wizard::tests::a_row_kept_without_a_category_is_charged_to_the_date_tally
**Renamed from `a_row_dropped_for_a_missing_category_is_not_charged_to_the_date_tally`
by improvise-8jk6.2.** Its premise went with the skip. A row missing `Region`
now lands, so its unparseable `Date` *is* a derived column's loss, and it is
charged once to each of `Date_Month` and `Date_Year`. The test asserts the
landed cell (`Date=2025-02-05, _Measure=Amount = 200`, with no `Region` and no
derived part) and exactly those two warnings. The history below is kept
because the rule it established (charge only landed rows) still holds. Every
object row now lands.
improvise-yss5, found by ChatGPT Codex reviewing PR #150 at `af7fae0e` and
correct. The first fix incremented `unparsed_dates` while walking a record's
@@ -1566,8 +1602,11 @@ bounded to four advances, so the rival fails it rather than stalling the suite.
### keeping_duplicates_separate_is_refused_at_the_category_cap
Twelve categories plus a collision, which is the reported import. It asserts
option (b) is disabled with a reason naming 12 and Skip, that choosing it leaves
A full set of categories plus a collision, which is the shape of the reported
import. The fixture is `category_records(MAX_CATEGORIES)`, so it tracks the cap
(12 when the test was written, 24 since improvise-felo.1). It asserts option (b)
is disabled with a reason naming the cap (`reason.contains(&cap)`, where `cap`
is `MAX_CATEGORIES` spelled out) and Skip, that choosing it leaves
the step and the policy (`Refuse`) untouched, and that the message is the
reason. The rival **cap checked with `>`** fails it.
@@ -1614,3 +1653,396 @@ that name, which must be refused. It was green on arrival, so its evidence is
the reviewer's rival, **date-derived names dropped from the taken names**,
which fails it. That fault would have let `add_category` reuse the existing
category, and every cell key would then name `when_Year` twice.
### a_group_both_expanded_and_collapsed_is_refused_naming_view_and_group
The user's ruling on improvise-felo.6. Three spellings of one group given two
folds, each refused as `Refusal::FoldConflict` naming the view and the group,
plus a category given two depths. Rival: a second fold for a group silently
wins. Only this test fails. Its control is
`a_collapsed_group_inside_an_expanded_region_loads`: a DIFFERENT group
collapsed inside an expanded region, and the same line twice, both load.
### a_cycle_among_group_parents_is_refused_naming_it
`> Q1-2025[|2025|]` and `> |2025|[Q1-2025]`. Rival: the cycle check disabled;
only this test fails. The fixture's first version spelled `2024-01` bare, which
the grammar refuses (a bare name starts with a letter), so every test on it
passed or failed for the parse error, `a_malformed_depth_is_a_parse_error`
vacuously. It now carries a control that parses.
### an_axis_line_for_a_category_named_expanded_is_still_an_axis_line
`axis_line` is tried before `expanded_line`, so the writer must never spell a
category depth bare. Rival: the writer writing `expanded: Cat` for depth 1. This
test fails: the category `row` at depth 1 reads back as an axis line for a
category named `expanded`.
### an_old_file_with_only_collapsed_lines_loads_closed_and_keeps_its_lines
"Everything opens closed, including old files." Every group of the file opens
closed, the one its `collapsed:` line names and the three it does not, and the
line is written back. Rival: the old open default; this test fails with the
other eleven in that rival's list.
### a_date_column_imports_as_one_category_of_months_in_quarters_in_years
improvise-8jk6.3, the user's "dates should usually be one category". The
fixture has two days of one month, so that month is one summed cell, and
January of the next year, so a month must stay unique across years. Rivals, each
run against the `date_hierarchy` module, and each caught here:
- The split as the default shape.
- Levels taken finest first, so the year hangs under the month.
- Groups registered without their parents.
- The raw dates still added beside the months.
- A hierarchy counted as `1 + components` slots.
### a_level_left_unchosen_is_skipped_not_emptied
Year + Month nests months straight into years, and Quarter alone is a flat
category of quarters. Caught the finest-first rival and the raw-dates rival.
### an_imported_date_hierarchy_opens_folded_to_its_years
The import sets nothing: the hierarchy opens folded, one row per year, because
every group starts closed (improvise-felo.6). The 2024 row is the fold of its
months, 111. Caught three of the rivals above.
### an_unparsed_date_keeps_its_value_and_is_warned_about
A format that reads none of the values keeps each raw value as an ungrouped
item, and the build says so. Its first version passed on the base commit: the
shape was ignored there and the SPLIT's warning happened to contain the same
words. It now also asserts there is no `Date_Year`. Rival: the unparsed value
not counted; this test fails.
### a_headless_date_import_splits_when_told_to
`--split-date` is the wizard's `s`, headless. It failed first as an unknown
argument. Rival: the flag parsed and ignored; this test fails, and
`a_headless_date_import_is_one_hierarchical_category` does not, which is the
point of having both.
### every_fold_the_api_can_build_reads_back_from_the_file_it_writes
ChatGPT Codex, P2 on #317 at `e09ec752`: `GroupFold::Expanded(usize)` and
`View::set_group_fold` are public, so `Expanded(0)` could be built, written as
`expanded(0): …` and then refused by the grammar that wrote it. Confirmed by
reading: the writer formatted any `usize`, and `depth_number` starts at `1`.
The fix makes the state unrepresentable (`Expanded(NonZeroUsize)`), so the
regression guard is the type, and the old failing input no longer compiles.
This test covers every fold the API CAN build (`Collapsed` and depths 1 to 3,
beside a category depth), each saved and read back.
### a_record_with_no_value_for_a_category_is_kept_without_that_coordinate
improvise-8jk6.2, reported by the user from the wizard: "Skipped 6 of 342
records missing category values: Settlement Date (6)". `Plan::place` returned
`Placement::Missing` for a category with no value, so the build dropped the
whole record, measures and all. The fixture has one complete record, one with
`"Settlement Date": null` and one with `""`, for two different payees. At
`e65d87fd`'s parent the build stored only the first cell. Now the stored cells
are exactly the three records, the two blanks with no date coordinate, and
there is no warning. `Settlement Date` holds only its one real item, so neither
`"null"` nor `""` became one. Rival R1, a record with a present-but-blank
category treated as unplaceable, fails this test and the two improvise-ych
tests.
### files_with_different_headers_import_as_one_sparse_model
improvise-lpzu. code-maat's `age.csv` (`entity,age-months`) and `coupling.csv`
(`entity,coupled,degree,average-revs`) merged through `merge_csvs_from`. Before
the fix, the age records had no `coupled` key, so they took the same `Missing`
path and **every age record was dropped**. The build stored the four coupling
cells and nothing else. The test pins the proposals (each column's kind,
judged over its present values), all six stored cells sorted, no empty-named
item in any category, and no warning. Rival R2, a record whose file lacks a
category column skipped, fails it along with the two CLI folder tests, the
projection test and the per-file view test.
### a_label_column_a_record_lacks_is_no_coordinate_not_an_empty_item
improvise-lpzu, the label half. Labels were attached with
`scalar_name(..).unwrap_or_default()`, so an age record, whose file has no
`coupled`, stored `coupled=` as a coordinate, and `coupled` grew an item named
`""`. Before the fix the first sorted cell was
`File=age, _Measure=age-months, coupled=, entity=src/a.rs = 2`. Rival R3
(`unwrap_or_default` restored) fails it.
### a_folder_expands_to_its_csv_files_in_name_order_and_skips_the_rest
improvise-lpzu. The folder holds three CSVs written out of name order, one
with an upper-case `.CSV`, plus a `.txt`, an `.improv` (the output a second run
finds) and a CSV in *another* folder. Only the three are returned, sorted.
`MemoryFs` stores in a `HashMap`, so the order comes from `Storage::list`'s
contract and not from insertion. Rival R5 (the extension filter dropped) and
rival R6 (the listing passed through a `HashSet`) each fail it.
### a_folder_with_no_csv_files_is_refused_by_name
improvise-lpzu. Both an empty folder and a folder holding only non-CSV files
must refuse with `no CSV files in folder maat`. Importing nothing would write
an empty document. Rival R4 (the `ensure!` removed) fails it, and so does R5,
because the text-only folder then "has" files.
### files_and_folders_expand_in_argument_order
improvise-lpzu. `[z.csv, b/, a.csv]` must expand to
`[z.csv, b/one.csv, b/two.csv, a.csv]`. Argument order is kept, and a folder
is sorted within itself. A fixture listing only folders, or files in name
order, could not tell "argument order" from "sort everything". R6 fails it.
### two_files_with_one_stem_union_their_columns
improvise-8jk6.12 (3). `a/x.csv` (`entity,one`) and `b/x.csv` (`entity,two`) are
one `File` item, `x`, whose columns are `entity, one, two`. Rival F3 (the
reviewer's fault: clear the stem's columns before each file) leaves
`entity, two`, and the `x` view would then hide the measure `one` that half its
records carry.
### files_with_different_headers_merge_without_inventing_the_missing_columns
improvise-lpzu, at the parser. Each merged record carries exactly its own
file's keys plus `File`: no key, `null` or `0` for a column another file has.
Each `SourceFile` lists its file's columns in header order. The per-file views
are built from those lists.
### a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_the_grid
improvise-8jk6.2's downstream question, asked of the real layout rather than
argued. Bolt has no `Date`:
- With `Date` on Column or on Page, the grid draws only Acme's `10`.
- With `Date` on None, it draws both `10` and `20`.
- In records mode, Bolt's row reads `["", "Bolt", "", "20", ""]`, a blank
`Date` cell.
- The Data line is `Payee=Bolt, _Measure=Amount = 20`, and it round-trips.
- A drill on `Payee=Bolt` reaches the record.
This is a characterization of the model's existing sparse semantics. It found
no bug. Its purpose is to make the Row/Column answer ("invisible") a stated
fact the user can overrule, rather than an accident. R2 fails it.
### a_merge_gives_each_file_a_view_projected_to_its_columns
improvise-lpzu, the user's "one view filtered by the file column and
projected to the columns in that file". The test asserts:
- the view order (`Default`, then the files in read order), with `Default`
still active;
- `File` on Filter at the file's item;
- the file's own categories seated, and the other file's on None;
- `_Measure` on Page at the file's first measure, with the other file's
measures hidden;
- the drawn values.
The age view draws `[2, 7]`. The coupling view draws `[5, 3]`, because its
first measure in model order is `average-revs`: serde_json's map orders keys
alphabetically. Failed at `bb505f8f`'s parent, where no view existed. Rivals
R7 (File on None), R9 (every category seated) and R10 (nothing hidden) each
fail it.
### a_files_view_holds_only_that_files_records
improvise-lpzu. The code-maat fixture cannot show that the *filter* works,
because its files share no measure, so the measure page alone would isolate
them. Two quarterly files with the same `Region,Revenue` schema can: without
the filter, `q1` would draw `30` and `3`. Rival R7 fails it. R9 and R10 do
not, and are not meant to.
### a_files_view_name_skips_names_already_held
improvise-lpzu. Two stems collide with names already held: `Default`, the
merged view, and `_Drill`, a virtual view that is never written and is dropped
on load. The first becomes `Default 2` and the second `_Drill 2`, and each
still filters to its own item. Rival R8 (the virtual names not checked) fails
it: the file's view would have been saved as `_Drill` and lost.
### per_file_views_survive_a_round_trip_and_repeat_byte_for_byte
improvise-lpzu. Two builds from two independently written `MemoryFs` give
identical `.improv` text, and `format(parse(text)) == text`. The reloaded
document still has the three views in order, `File: filter` at the file's item,
and the same drawn values. This is the first persisted, non-virtual use of
`Axis::Filter` (gotcha 27). R7 and R9 fail it.
### a_csv_column_read_as_a_category_keeps_its_text_and_as_a_measure_its_number
improvise-8jk6.4, at the pipeline. It checks three things in one fixture,
one per reader of a CSV value:
- analysis still proposes the numeric-looking `commit` as a Measure (typed);
- forced to Category, its items are `00565796` and `220944e2` (text);
- `lines` is stored as the number 5 (typed).
The rivals, each run:
- **R1**, a category named from the typed value, fails it on the items.
- **R2**, a CSV measure read with `as_f64` on its text, fails it on the cell.
- **R3**, CSV analysed without typing, fails it on the proposal.
R2 and R3 also fail the five lpzu merge tests in this module and five
headless imports. That is how much of the suite leans on CSV measures reading
as numbers.
This test uses `from_csv` and so cannot compile on the base. The base failure
for 8jk6.4 is the headless test
`a_csv_column_forced_to_category_keeps_its_text_verbatim`.
### a_formula_that_does_not_parse_stops_the_build_and_is_named
improvise-8jk6.5. On the base the build returned `Ok`, keeping `Good` and
silently dropping `Bad`. Two formulas fail to parse here, so the test proves
every failure is named. **R5**, naming only the first, fails it on `Worse`,
and since improvise-8jk6.17 the headless
`an_import_formula_that_does_not_parse_refuses_names_it_and_writes_nothing`
sees R5 too (it passes two bad formulas). It also asserts `Good` is not named, so a
message that lists every formula cannot pass. **R4** (drop, as the base did)
fails it.
### confirming_a_formula_that_does_not_parse_keeps_it_out_and_the_text_in
improvise-8jk6.5, the wizard half. On the base `confirm_formula` pushed
`Bad = revenue +` into the list, which failed on `pipeline.formulas`. The test
asserts the text is still in the buffer and the editor still open. A fix that
refused but cleared the buffer would throw away what the user typed, the same
hazard improvise-v4hg fixed in the TUI formula minibuffer. **R6**, confirming
regardless, fails it. The headless suite cannot see R6.
### a_build_that_keeps_no_record_refuses_and_says_why
improvise-8jk6.6. On the base both halves got `At least one category must be
accepted`, a refusal right only by accident. It asserts the exact message for
the two cases `nothing_to_import` distinguishes: all skipped, and no records
at all.
- **R7**, the check removed from the build, fails it.
- **R9**, `any` swapped for `all`, also fails it, on the empty source: `all`
is vacuously true over no records.
### the_review_step_says_when_no_record_can_be_imported
improvise-8jk6.6, the wizard half (gotcha 22: the check lives on the step
where the user can still choose another array path). On the base the step
refused with `No categories: … Press c to cycle a field's kind`, which offers
a key that has no field to act on. **R8**, the step without the check, fails
it. The build test does not see R8.
### a_header_only_file_keeps_its_file_item_its_columns_and_its_view
The ChatGPT Codex P2 on PR #314 at `4e68372c`, confirmed by the adversarial
review. `age.csv` (one row), a header-only `coupling.csv` and `rev.csv` (one
row). At `005b7a76` the test failed at the first assertion: `File` held only
`age` and `rev`, so the `coupling` view filtered to an item that did not exist,
and `coupled`, `degree` and `average-revs` were not in the model. Now `File`
reads `age, coupling, rev` in file order, the three columns exist (as Labels),
and the `coupling` view is a zero-row records view filtered to `coupling`, while
the `age` view still draws `2`. Rival F5 (sources not seeded) and F6 (unseen
columns not proposed) each fail it, and so do F7 and F8 below.
### a_file_with_no_measure_gets_a_records_view_not_another_files_measure
improvise-8jk6.13, from the review of PR #314. `a.csv` (`entity,x`) and
`tags.csv` (`entity,tag`). At `005b7a76` the `tags` view had `_Measure` on Page
with no selection, so the layout paged onto `x`, a measure the same view hides.
The test failed there, with `row_count() == 1`: `a`'s record, drawn under
`File=tags`'s view. Now the view has no `_Measure` page, is a records layout,
and draws zero rows, while `a`'s view is still a pivot. Rival F7 (the old
paging) fails it, and F8 (the File selection lost when `records_steps` moves
`File` off Filter) fails it with the same `1`. F8 is the mistake the first
version of the fix made.
### a_files_view_seats_the_date_parts_of_its_own_date_column
improvise-8jk6.12 (1). The reviewer replaced the date-parts clause of
`source_view`'s `own` with `false` and every test passed, because no folder
fixture extracted a date part. Here `abs-churn.csv` has a `date` column,
extracted to `date_Month`, and `age.csv` does not. `date_Month` is seated in the
`abs-churn` view and on None in the `age` view. Rival F1 (the reviewer's fault)
puts it on None in both and fails the first assertion.
### two_files_whose_names_collide_in_one_batch_keep_a_view_each
improvise-8jk6.12 (2). `Default.csv` and `Default 2.csv`. The first takes
`Default 2`, so the second must take `Default 2 2`. Without the same-batch
check both are named `Default 2`, and `IndexMap::insert` silently replaces the
first, leaving two views where three are owed. The test asserts all three names
with their filters. Rival F2 (the reviewer's fault) fails it.
### a_whitespace_only_category_value_is_no_coordinate
improvise-8jk6.12 (4). A `region` of `" "` is kept as a record with no
`region` coordinate, and `region` holds only `East`. The trim in `scalar_name`
predates this PR (improvise-ych), but the sparse-record rule now rests on it:
without it the value becomes an item named `" "`. Rival F4 (the reviewer's
fault) fails it.
### a_raw_date_spelled_like_a_generated_item_is_refused
improvise-d0n6 (Codex P2 on PR #322). A `%Y-%m-%d` column imported Year > Month
with `2024-01-15` and a raw `2024-01`. Before the refusal the build succeeded,
put both records on the grouped item `2024-01` (summing them) and warned that
the raw value was "kept ungrouped", which was false. Fault injected: removing
the `spelling_clash` call makes this test fail with `unwrap_err()` on an `Ok`.
### a_raw_date_spelled_like_a_generated_group_is_refused
improvise-d0n6. The same with a raw `2024`, a generated group. Removing the
`spelling_clash` call fails it the same way. The control,
`a_raw_date_spelled_like_nothing_generated_is_kept_ungrouped`, passes either
way, so a blanket refusal of every unparsed value fails it.
### the_date_step_will_not_advance_over_a_spelling_clash
improvise-d0n6 (Copilot on PR #322, and the adversarial review's B1). The
spelling-clash refusal sat only in the build, so a wizard with a raw `2024` in
a `%Y-%m-%d` column passed ConfigureDates and failed on NameModel, three steps
from the `s` that fixes it. Before the check joined `validation_error` the
test failed with the step already `DefineFormulas`; removing that call again
reproduces it. It also toggles to the split shape and requires the step to
advance, so a check that blocks forever fails.
### split_date_naming_an_unknown_or_non_date_field_is_refused
improvise-tpm8 class. `--split-date Nope` and `--split-date Revenue` (a
measure) exited 0 and did nothing. Both now exit 1 naming the flag and the
field; removing the check in `run_headless_import` fails both rows.
### an_axis_naming_a_category_the_import_did_not_build_is_refused
improvise-8jk6.3 review. `--extract Date:Month --axis Date_Month:column` used
to build one `Date` category and drop the `--axis` for the vanished
`Date_Month`, exit 0. It now exits 1 naming the category and `--split-date`.
This also refuses any `--axis` naming an absent category, which was silently
ignored before this PR as well.
### a_folder_of_only_header_only_files_keeps_no_record_and_refuses
improvise-8jk6.6 meeting PR #314's rule that a header-only file is still a
file. Written after merging `main` at `c91cc055`, where the two rules first
met. The file seeds give the pipeline a `File` Category proposal even with no
records. So without `nothing_to_import` the build succeeds and saves `File`
items and empty views: the category check alone cannot refuse. The `setup`
assertion proves that seeded proposal exists, so the test is not passing only
because there was no category. **R7**, the check removed from the build, fails
it. The user confirmed the refusal on PR #321: "Refuse (as #321 does)".
### an_absent_axis_category_that_is_no_date_part_names_what_was_built
improvise-8jk6.3 review (B2). The README's own recipe passed
`--axis Measure:column`, a silent no-op because the category is `_Measure`; the
first refusal blamed dates for it. The test uses a non-date absent name and
asserts the message lists the built categories and carries no date hint.
Fault injected: making the hint unconditional fails it.
### the_readmes_import_recipe_runs
improvise-8jk6.3 review (B2). Runs the README's recipe as printed. Fault
injected: restoring `--axis Measure:column` in the README fails it with the
refusal.
### split_date_on_a_time_field_with_no_date_format_is_refused
improvise-8jk6.3 review. `--time Revenue --split-date Revenue` made the field a
time category with no date format and the split was a silent no-op. Without
the `date_format.is_some()` condition the import exits 0 and this fails.
context/test-evidence/layout-cost.md
+125
View File
@@ -0,0 +1,125 @@
# Layout and keypress cost (improvise-felo.13)
Measured on an **Apple M4 Max, 64 GB**, macOS, release build, with
`benches/scale_bisect.rs` (merged from `bench/scale-bisect`, `13918bb9`), one
configuration per process, 3 categories of K items on Row, N = 100,000 capped at
K^3/2 records, prune off unless stated. Baseline is this branch with only the
tests added; "after" is the branch head. `key` is `App::dispatch_key(Down)`;
"cold" is the first key after open or after an edit, which pays the one
whole-grid measurement; "warm" is every key after it.
## Before and after
| rows | before key | after key warm | after key cold | before browser window | after browser window | SetCell before / after |
|---|---|---|---|---|---|---|
| 729 (K=9) | 5.5 | 0.00 | 1.4 | 5.0 | 0.13 | 0.9 / 0.9 |
| 4,913 (K=17) | 108 | 0.00 | 6.0 | 105 | 0.12 | 5.8 / 5.9 |
| 5,832 (K=18) | | 0.00 | 7.7 | | 0.12 | 6.8 |
| 6,859 (K=19) | | 0.00 | 8.8 | | 0.12 | 8.6 |
| 19,683 (K=27) | 1,271 | 0.00 | 24.5 | 1,259 | 0.14 | 23.0 / 22.7 |
| 64,000 (K=40) | 10,771 | 0.00 | 84 | 10,746 | 0.20 | 79 / 80 |
Prune on: K=17 (2,456 rows) key 264 ms -> 0.00 warm, 3.6 cold; prune toggle
172 -> 8.8 ms; K=27 prune toggle 2,189 -> 41.5 ms. Records mode (3 x 1000, 14,500
records): key 1,004 ms -> 0.00 warm, 2.8 cold; **SetCell 1,511 ms -> 1,512 ms,
unchanged** (not this ticket's cost).
Boundaries, warm key and browser window: inside 8 ms at every size measured (to
216,000 rows, 303 ms cold). Cold key: inside 8 ms to K=18 (5,832 rows), over 8 ms at
K=19. Before: inside 8 ms to 729 rows, over 100 ms at 4,913, over 1 s at 19,683.
The remaining cold cost is linear (1.2-1.4 us per row) and is the whole-grid width
and total pass; the warm path does not repeat it.
## What the ticket's reading got right and what it did not predict
The O(rows^2) in `cell_key` was real and is one of three causes. Two more sat
under it. First, `compute_col_widths` ran on **every** keypress, through
`cmd_context`, and `dispatch_key` and `apply_effects` each rebuilt the layout
whether or not anything had moved. Second, once the index was O(1) the width pass
was still O(K^2) per cell: with `_Index`/`_Dim` on `Axis::None` the aggregation is
not `Exact`, so each cell went through `DataStore::matching_values`, which
narrows by the smallest index set (about N/K candidates) and probes the rest. A
partial as wide as the widest stored cell can match only one cell, so
`exact_positions` probes it directly. That second cause is why K=40's cold key
stayed at 8 s after the first two fixes.
### a_cursor_key_rebuilds_neither_the_layout_nor_the_widths
`ui::app::tests`. Two keys after a warm-up leave `layout_builds` and
`width_passes` where they were. Failed before the fix on the layout count (a
cursor key rebuilt it). Rival 1 (`refresh_layout` always rebuilding): fails on
`layout rebuilt on a cursor key`. The test seats the cursor at row 2, so "the
keys moved the cursor" is asserted and a key that does nothing cannot pass.
### an_edit_rebuilds_and_remeasures_so_the_widths_follow_the_value
The control for the test above, and the guard on invalidation. A `SetCell` of a
long text must rebuild the layout and remeasure, and the cached widths must equal
a fresh `compute_col_widths`. Rival 2 (`rebuild_layout` not dropping the cached
geometry): fails on `an edit must remeasure`. It also asserts the widths changed,
so a fixture where the edit does not widen a column cannot pass.
### switching_view_rebuilds_though_the_effect_is_view_only
`SwitchView` is `ViewOnly` (gotcha 3), so skipping the rebuild by `EffectKind`
alone would leave the old view's layout. The layout records the view it drew and
the drill snapshot it froze, and rebuilds when either differs.
### an_axis_lookup_does_work_independent_of_the_row_index
`view::layout::lookup_cost_tests`. Counts the items `data_items` yields during
`cell_key`, `row_tuple`, `row_label` and `column_key`; row 399 of 400 must cost no
more than row 0 plus two. Failed before the fix: `cell_key: row 399 cost 402 steps
against 3 at row 0`. Rival (`cell_key` going back to `data_items(..).nth(row)`):
`cost 400 steps against 1 at row 0`. Limit: the counter lives in `data_items`, so
a different linear walk written without it is not seen; the next test is what
holds the index to the walk's answers.
### the_position_index_answers_what_the_walk_answered
Compares `row_tuple`, `data_row_to_visual` and `row_folds_a_group` against the
walk over a grouped, collapsed and pruned axis, including two indices past the
end. Rival (`data_index_to_visual` reading `data_idx + 1`): fails at `visual 0`.
### a_full_width_partial_probes_one_cell_instead_of_narrowing
`model::cell::exact_path_tests`. A full-width partial must not reach set
narrowing, and an absent cell and an unseen item must match nothing. Rival
(`exact_positions` returning `None`): the narrowing count is 1, not 0. Rival
(`widest <= partial.len() + 1`): fails `a_wider_cell_turns_the_shortcut_off_until_it_is_removed`,
which stores a cell one coordinate wider and expects it matched.
`a_narrower_partial_still_narrows_and_matches_every_superset` is the control: the
shortcut is not a blanket exact lookup. `an_unsorted_full_width_partial_is_still_answered`
holds the canonical-order condition: a partial not in a stored key's order falls
back.
### a_window_over_a_held_summary_equals_the_one_walk_grid_body
`ui::viewport::tests`. The host keeps one `GridSummary` (widths, total formats,
totals) per layout and reads a window over it with `grid_window`; it must equal
what `grid_body` builds for the same window, over the whole grid, an interior
window, the last row and the empty window. Rival (`grid_window` reading column
`ci + 1`): fails at `window 0..2 x 0..10`. Widths stay the **whole-grid** maximum,
because the summary is the same walk `grid_body` did; nothing the user sees moves.
### a_frozen_records_snapshot_replaces_the_live_row_index
`view::layout::lookup_cost_tests`. Three records are frozen, two are then cleared,
and `with_frozen_records` over the snapshot must draw 3 rows (the control: with no
snapshot it draws 1). A drill held open while another client removes records is the
production analogue. Rival (deleting `layout.reindex()` after the snapshot swap):
fails `left: 1, right: 3`; the whole workspace passed it before this test.
### a_drill_snapshot_replaced_alone_rebuilds_the_layout
`ui::app::tests`. `StartDrill` is `ViewOnly`, so a batch of it alone has no `Model`
effect to mark the layout stale; the drill pointer in `LayoutBasis` is the only
thing that rebuilds it. The test starts one snapshot, then a second: the first
step alone passes even with a constant pointer (drill absent to present changes the
basis), which is why the second step is the test. Rival (pointer replaced by a
constant): fails `no rebuild for a second drill snapshot`.
### Observation: the TUI frame
`tui_frame_grid_of_whole` is still 47-53 ms at 19,683 rows after this change: the
draw path projects the whole grid, linear and out of scope for this ticket.
context/test-evidence/protocol.md
+48
View File
@@ -1259,3 +1259,51 @@ the Enter commit.
- **Rival** (the commit naming `state.view.active_view`, the view at
`Enter`), run across the whole workspace: fails **only** this test, with
`["Later"]` against `["Other"]`.
### resolution_takes_each_attribute_from_the_nearest_face_that_states_it
ADR 004 clause 3: inheritance resolves once, in shared code. The fixture states
a different attribute at each of three levels — `default` (foreground White,
italic), `mode` (foreground Black, background Blue), and
`mode.text-entry.save` (background `Colour::Default`, underline) — so the one
resolved face has to take its foreground from the middle, its slant from the
root and its background from itself. `Some(Colour::Default)` is the case that
matters: it is a stated value and must not fall through to `mode`'s Blue, which
the sibling `mode.text-entry.export` (stating nothing) does inherit.
**Injected fault**: fold the chain so the parent wins. This test fails, and so
does the TUI's `the_shipped_theme_reproduces_every_migrated_literal`, because
`mode.text-entry.edit` then draws its parent's DarkGray.
### every_axis_gets_its_own_accent_in_the_shipped_theme
Moved from `improvise-core`'s `every_axis_gets_its_own_color` when `AxisColor`
left core (ADR 004 clause 6). Distinct axes must be distinguishable at a
glance, and the axis faces exist for exactly that, so two axes resolving alike
defeats them. It now asks the shipped theme rather than an enum's arms, which
is the stronger question: two distinct `AxisColor` arms could always have been
mapped to one terminal colour.
### a_theme_that_restyles_text_entry_reaches_every_minibuffer
ADR 004 says `minibuffer.*` inherits `mode.text-entry` directly. PR #319's first
head gave the minibuffer faces a `minibuffer` base face instead, and ChatGPT
Codex flagged it (P2): face parents are compatibility behaviour once a theme
can be downloaded, so a theme that restyles `mode.text-entry` must reach every
prompt. The test overrides only `mode.text-entry` with italic, an attribute no
minibuffer face states in the shipped theme, and asserts every
`minibuffer.<entry>` resolves italic. A test that sets a colour instead would
be vacuous, because each minibuffer face states its own colours.
**Failed first**, against the `minibuffer` base parent: `Edit` resolved to
`Normal`. It passes under the accepted chain. The no-op oracle
`the_shipped_theme_reproduces_every_migrated_literal` passed both before and
after the change, which shows the change moved no pixel.
### a_quarters_end_leaves_its_year_open_on_the_same_level
improvise-felo.6: runs inside one category nest by `(level, depth)`. `Direct`
sits in `2024` after `Q1`'s `GroupEnd { level: 0, depth: 1 }`, and `Undated`
in nothing after `2024` ends. Rival: `groups()` closing by level alone, as it did
before depth existed. Only this test fails in `improvise-protocol`: every other
frame fixture puts one group per level.
context/test-evidence/scale-bisect.md
+218
View File
@@ -0,0 +1,218 @@
# Scale bisect: how big a document does improvise handle today (improvise-felo.10)
Measured on `gh/main` @ 8220a065, **Apple M4 Max, 64 GB** (`sysctl`: `hw.memsize 68719476736`), macOS,
release build (`lto`, `opt-level 3`), native only (no wasm, no real OPFS, no terminal).
Harness: `benches/scale_bisect.rs` (one configuration per process) driven by `bench/scale_bisect.py`,
`bench/sweep.py`, `bench/edge.py`, `bench/full.py`. Reproduce:
`nix develop --command cargo build --release --bench scale_bisect`, copy the binary from
`target/release/deps/scale_bisect-*` to `/tmp/sb`, then `python3 bench/edge.py`.
Budgets (user ruling 2026-10-03): **8 ms is the headline** (120 FPS, hot path or defer to background);
16 ms, 100 ms (noticeable) and 1 s (broken) are reference crossings.
## Method, and what is real
Every number goes through the production path: `App::new` (open: formula recompute +
`GridLayout::with_frozen_records`), `App::dispatch_key(Down)` (**keypress/scroll**: it calls
`rebuild_layout`, then `cmd_context`, which runs `compute_col_widths` over the whole grid, then applies effects),
`project::grid_of` (TUI frame: whole grid; browser: a 45x12 window), `App::apply_effects` with `effect::SetCell`,
`SetAxis`/`HideItem`/`ToggleGroup`/`TogglePruneEmpty` (reshape), `Host::connect` and `Host::handle` with a
`caught_up` `ModelCommand::SetCell` (browser path incl. projection), `format_md`/`parse_md`/`save_to(MemoryFs)`.
Document: C0..C{cats-1} with K items, N distinct random tuples ("records", capped at K^cats/2), two measures
each (2N cells), `_Measure` on Column. Timings are the **median** of up to 3-8 in-process repeats; boundary
rows are the median of **3 separate processes** run sequentially (spread shown). Earlier exploratory sweeps
ran 2-3 processes in parallel and agree with the sequential runs within ~5%.
**`View::prune_empty` defaults to false** (`view/types.rs`), so "prune off" is what a user gets until they
toggle it; "prune on" is reported next to it. Rows below are the layout's `row_count`.
**Dominant cost (reasoned, not profiled):** the release profile strips symbols, so `sample` could not
symbolicate. The reason is in the code and the timings agree with it: `GridLayout::cell_key`/`row_item` do
`data_items(&self.row_items).nth(row)` (`view/layout.rs` ~253/292/807/870), O(row) per lookup, and
`compute_col_widths` (every keypress, via `cmd_context`) and `grid_of` visit every cell, so both are
O(rows^2). Evidence: with prune off, rows 729 -> 5.4 ms, 4096 -> 74.6 ms, 19683 -> 1248 ms (x6.7 rows ->
x14 time -> x17 time). A 45x12 browser window costs **the same** as the whole-grid TUI frame (K=30:
2274 vs 2298 ms) because the widths pass is whole-grid whatever the viewport says, so viewport narrowing buys
nothing today. `GridLayout` build itself is cheap (~90 ns/row, ADR 008).
## Attribution: state+projection vs render/serialize (K=10, 1000 rows, prune off; 3 runs)
| step | ms |
|---|---|
| keypress (state: rebuild + widths + effect) | 9.5-9.7 |
| `grid_of` whole grid (TUI projection) | 11.7-12.0 |
| `GridWidget` render into a 200x50 ratatui `Buffer` | 0.12 |
| `grid_of` 45x12 window (browser projection) | 9.1-9.3 |
| JSON encode of one SetCell's projected updates | 0.001-0.005 (0.6-1.1 KB) |
| JSON encode of the connect Snapshot | 0.02 (21 KB) |
Render and serialize are **<=0.5 ms in every row measured** (GridWidget 0.08-0.53 ms up to 14,500 records;
snapshot encode <=0.07 ms; records-mode SetCell updates 21 KB in 0.02 ms). The 8 ms is entirely state and
projection. Not measured: the rest of `draw()` (panels, status; `draw` is private to the binary), terminal I/O,
browser JSON *decode* and DOM paint.
## Axis B: items per category K, 3 categories on Row, N = 100k (capped at K^3/2), prune OFF
| K | rows | open | keypress | browser window | SetCell | maxrss MB |
|---|---|---|---|---|---|---|
| 5 | 125 | 0.1 | 0.5 | 0.5 | 0.2 | 8 |
| 9 | 729 | 0.2 | **5.4** (5.3-5.5) | 5.0 | 0.9 | 13 |
| 10 | 1,000 | 0.2 | **9.6** (9.5-9.7) | 9.2 | 1.2 | 14 |
| 11 | 1,331 | 0.3 | 12.7 | 12.7 | 1.6 | 10 |
| 12 | 1,728 | 0.3 | 20.0 | 19.3 | 2.1 | 11 |
| 15 | 3,375 | 0.5 | 65 | 64 | 4.0 | 14 |
| 16 | 4,096 | 0.6 | 74.6 (74.3-75.2) | 73.4 | - | - |
| 17 | 4,913 | 0.7 | **105** (105.0-105.5) | 103 | - | - |
| 20 | 8,000 | 1.1 | 302 | 301 | 9.4 | 21 |
| 25 | 15,625 | 1.9 | 773 | 771 | 18.8 | 33 |
| 26 | 17,576 | 2.2 | 967-977 | 962 | - | - |
| 27 | 19,683 | 2.4 | **1,248** (1,244-1,251) | 1,244 | - | - |
| 30 | 27,000 | 3.1 | 2,262 | 2,274 | 31 | 49 |
| 40 | 64,000 | 6.7 | 10,858 | 10,833 | 73 | 235 |
| 60 | 216,000 | 21.6 | timeout (>60 s) | - | - | - |
| 1000 | 10^9 | **refused** (see below) | | | | |
Hot-path keypress/scroll and browser window: **largest inside 8 ms: K=9 (729 rows)**; inside 16 ms: K=11;
first over 100 ms: **K=17 (4,913 rows)**; first over 1 s: **K=27 (19,683 rows)**. SetCell alone
(no widths pass; layout rebuild only, ~linear in rows): inside 8 ms to K~19 (K=15 4.0, K=20 9.4), 16 ms ~K=24,
100 ms ~K=45 (K=40 = 73 ms), 1 s extrapolated ~K=95 (**extrapolated, not measured**). Open stays <25 ms to K=60.
**3 x 1,000 on Row cannot be opened at any N**: the layout materializes the cross product (10^9 rows, ~150 B
each = ~150 GB, ADR 008). The harness refuses above 30M rows (`rows_cap`) rather than swap the machine, so
the 10^9 number is the ADR's estimate, not a measurement here.
### Prune ON (same documents)
| K | rows | open | keypress | browser window | SetCell |
|---|---|---|---|---|---|
| 7 | 171 | 2.0 | 4.5 (4.4-4.6) | 1.1 | 5.4 |
| 8 | 256 | 3.0 | **7.0** (7.0-7.1) | 1.6 | 8.5 |
| 9 | 364 | 5.3 | 12.8 | 2.7 | 15.3 |
| 11 | 665 | 12.2 | 29.6 | 6.2 | ~28 |
| 13 | 1,098 | 29.0 | 71 | 14.5 | ~70 |
| 14 | 1,372 | 48 | **119** | 24 | - |
| 20 | 4,000 | 300 | 747 | 145 | 903 |
| 21 | 4,630 | 401 | 968-992 | 190 | - |
| 25 | 7,812 | 767 | 1,899 | 366 | 2,328 |
| 30 | 13,500 | 2,255 | 5,555 | 1,047 | 6,804 |
Prune on: keypress inside 8 ms to **K=8**, first over 100 ms **K=14**, ~1 s at **K~21-22** (K=22 measured
0.93-0.95 s, a different N cap; first clearly over: K=25). Pruning shrinks the grid, so the browser window is
cheaper than prune off at the same K (K=20: 145 ms against 302 ms), but **every keypress re-runs
the prune** (`dispatch_key` calls `rebuild_layout`), so keypress and SetCell are *worse* with prune on
(K=10: 23 vs 9.9 ms). Prune is also quadratic (`nth`). Conclusion: prune on/off moves the boundary by about
one K at 8 ms and makes 100 ms/1 s *earlier* (K=14/21 vs 17/27); it never rescues the 3x1000 case.
## Axis A: records N at fixed K (K=30, 27,000 rows)
| N | prune off keypress | prune on rows | prune on keypress | prune on window |
|---|---|---|---|---|
| 100 | 451 | 100 | 861 | 0.3 |
| 1,000 | 527 | 1,000 | 1,059 | 6.4 |
| 5,000 | 1,013 | 5,000 | 2,178 | 140 |
| 13,500 (cap) | 2,267 | 13,500 | 5,555 | 1,047 |
Prune off: ~450 ms of widths over 27k empty rows + ~1.3 us per stored cell; **N never gets the keypress
under 100 ms at K=30**, and at K=1000 N is irrelevant because the layout cannot be built (rows = K^3).
The N-axis has no bisect point in the 3x1000 regime; N only matters once K is small enough that rows fit, and
there it is a second-order term. Open is flat (3.1-3.6 ms) in N with prune off.
## Axis C: categories on Row (prune off)
| K | cats | rows | open | keypress | note |
|---|---|---|---|---|---|
| 10 | 2 | 100 | 0.1 | 0.2 | |
| 10 | 3 | 1,000 | 0.2 | 9.9 | over 8 ms |
| 10 | 4 | 10,000 | 1.4 | **880** | ~1 s |
| 10 | 5 | 100,000 | 14 | timeout >100 s | |
| 3 | 6 | 729 | 0.3 | 27.9 | 5x worse than K=9 x 3 cats at the same rows |
| 3 | 8 | 6,561 | 2.0 | 2,798 | |
| 3 | 10-12 | 59k-531k | 18-210 | timeout | open alone passes 100 ms at 11 cats |
| 2 | 10 | 1,024 | 0.6 | 172 | |
| 2 | 12 | 4,096 | 2.2 | 3,190 | |
| 2 | 14 | 16,384 | 9 | 63,100 | |
The cap of 24 is unreachable: rows = product of items, so a view with c Row categories of K items dies by
K^c ~ 10^4 rows (1 s) regardless of N, and with two items per category by 12-14 categories. Per-row cost
also grows with category count (each row_item is a Vec of c items). Not measured: Row categories with prune
on, other axes absorbing categories (the realistic mitigation: only 1-2 on Row).
## Axis D: records mode over N records (`with_frozen_records`, the improvise-sk32 quadratic)
| N | open | keypress | browser window | SetCell (apply) | host SetCell+project | maxrss MB |
|---|---|---|---|---|---|---|
| 300 | 1.5 | 0.7 | 0.1 | 1.6 | - | 10 |
| 1,000 | 2.9 | 5.6 (5.5-5.7) | 0.3 | 10 | - | 12 |
| 1,200 | 3.9 | **7.9** (7.7-8.0) | 0.3 | - | - | - |
| 1,300 | 4.5 | 9.3 | 0.3 | 15.5 | 17 | 26 |
| 1,800 | 8.4 | 16.5 | 0.4 | - | - | - |
| 3,000 | 22.9 | 47 | 0.7 | 74 | - | 20 |
| 4,400-4,600 | 47-51 | **95-102** | 0.9 | 149 | 158 | 55 |
| 10,000 | 248 | 492 | 1.8 | 734 | - | 46 |
| 14,000-14,500 | 472-500 | **940-1,002** | 2.6-2.8 | 1,435 | 1,459 | 135-148 |
| 30,000 | 2,198 | 4,314 | 5.6 | 6,467 | - | 125 |
| 100,000 | **25,420** | **51,234** | 19.3 | not reached (killed at 100 s) | - | not reached |
Records mode, keypress/scroll: inside 8 ms **to N~1,200**; 16 ms N~1,800; 100 ms N~4,500 (first over: 4,600 =
102 ms); 1 s N~14,500. Open crosses 8 ms ~1,750, 16 ms ~2,300, 100 ms ~6,500, 1 s ~20,000. Time ~ N^1.95.
SetCell is ~1.5x keypress (it rebuilds and re-groups). The *browser window* is cheap (<20 ms even at 100k)
because records mode has two columns and the width walk is short, so the 51 s is `rebuild_layout`, not
projection. Prune on/off does not change records mode (prune applies to pivoted axes; the sweep ran off).
Records per KB: ~4 KB/record process-wide at 14k (maxrss includes the document generator, so read as an
upper bound); the 100k memory was not obtained.
## Axis E: save / load and per-command persistence
Mode `none` (all categories off-axis, so layout is trivial and only the store scales), 3 x 1,000, N records
= 2N cells:
| cells | format_md | text MB | parse_md | persist per accepted command (`save_to` MemoryFs) | maxrss MB |
|---|---|---|---|---|---|
| 2,000 | 13.8 | 0.1 | 4.5 | 12.4 | 18 |
| 20,000 | 156 | 0.86 | 37 | 161 | 80 |
| 60,000 | 527 | 2.5 | 119 | 539 | 236 |
| **200,000 (the target)** | **1,974** | 8.4 | **401** | **1,984** | 663 |
| 600,000 | 6,470 | 25 | 1,320 | 6,460 | 1,894 |
| 2,000,000 | 23,120 | 84 | 4,668 | 23,088 | 5,611 |
Linear: ~10 us/cell to format, ~2 us/cell to parse. Crossings, format_md: 8 ms ~700 cells, 16 ms ~1.5k,
100 ms ~11k, 1 s ~100k cells. parse_md: 8 ms ~4k cells, 100 ms ~50k, 1 s ~450k. **At the target (200k cells)
every accepted command in the browser worker costs ~2 s of format alone** before OPFS write time (not
measured: MemoryFs only). Persist per command also appears in smaller documents inside the grid axes
(K=10 grid: 5.5 ms; records N=4,400: 63 ms).
## Other operations (3 runs each; "grouped" fixtures start with C0 groups collapsed, so rows are not K^3)
Reshape (prune off, K=10, 1000 rows): hide item 0.25 ms, set axis 0.19 ms, **toggle prune 12.7 ms**.
K=26 grouped, 2,028 rows: hide 0.5, toggle group 1.8, set axis 1.8, **toggle prune 727-750 ms**.
Prune on K=13 grouped, 321 rows: hide 9.4, toggle group 37, set axis 37, toggle prune 37-39 ms.
`Host::connect` snapshot: K=10 6.5 ms, K=26 375 ms, records N=14,000 2.9 ms. Host SetCell+project: K=10 7.7 ms,
K=26 ~400 ms, records N=4,400 158 ms (equal to apply + a whole-grid width walk).
## Classification and feasibility of yielding
| operation | class | today's first size over 8 ms | dominated by | chunked/yielding form |
|---|---|---|---|---|
| keypress / scroll | **hot (<=8 ms)** | 1,000 rows (K=10) / N=1,300 records | `rebuild_layout` + whole-grid `compute_col_widths` (O(rows^2) `nth`), recomputed though nothing changed | not a slicing problem: cache widths/layout across keys, O(1) row lookup (ADR 008/009). Slicing would only hide it |
| browser window projection | **hot** | 1,000 rows | same width walk, whole grid regardless of window | needs the same fix; then window cost is O(window) |
| SetCell apply + project | **hot** | K~19 (prune off), 1,300 records | layout rebuild O(rows) + widths + projection | incremental update of one row/total, not a rebuild |
| column totals | hot | inside `grid_of` (not separable here) | whole-grid sum in the projection | maintained total (ADR 009) |
| open / layout build | deferrable | 40k rows (prune off); N~1,750 (records) | build is ~90 ns/row (sliceable by outer index); the prune/records grouping is the quadratic part | build is feasible in slices once the quadratic is removed; the quadratic itself is not sliceable |
| reshape (hide/collapse/axis) | deferrable | cheap prune-off (<2 ms to ~2k rows); 37 ms prune-on at 321 rows | layout rebuild + prune | one sort/remap at 100k occupied tuples was 2.3 ms in ADR 008's prototype, so a reshape can fit inline; prune toggle needs the quadratic fixed first |
| save (format_md) | deferrable | ~700 cells | per-cell formatting, linear | the Data section is a flat per-cell line stream, so chunking by cell range with a yield every ~400 cells looks feasible, but needs a consistent snapshot (clone or journal); not profiled by function |
| load (parse_md) | deferrable | ~4k cells | pest parse + tree walk, linear | tree walk is chunkable by line; the pest parse of one text is not, unless the Data section is parsed line by line (grammar is line oriented); not profiled by function |
| persistence per command | deferrable (debounce) | ~700 cells | the whole-document `format_md` | coalesce/debounce or append-journal: formatting 2 s per command at target is unusable inline at any slicing |
## What I could not measure
* wasm: all numbers are native; the browser worker is typically slower. Real OPFS I/O, JSON *decode* and
DOM paint, the full `draw()` frame, and Safari are not covered.
* Records mode at 100k: SetCell, host and RSS (killed after open and keypress, ~76 s).
* Per-function profile: symbols are stripped in `[profile.release]`; attribution is reasoned from the
code and the timing exponents, not a flame graph.
* SetCell 100 ms/1 s for prune off past K=40 are extrapolations.
* Memory (`maxrss`) includes the generator's hash set and `recs` vector; it is an upper bound on the
document. Layout cost ~150 B/row per ADR 008, consistent with K=40 (64k rows, 235 MB peak with prune off
including the generator and a second layout during rebuild).
* Noise: boundary rows are 3 sequential processes, spread <=3% (e.g. K=17: 105.0-105.5 ms). Non-boundary
rows are single runs, some taken with 2-3 concurrent processes on a 16+ core machine; treat them as +-5-10%.
context/test-evidence/tooling.md
+143
View File
@@ -1772,3 +1772,146 @@ The output includes the user's inline answers on
**Scope sweep.** `scripts/pr-watch` passes its JSON through `printf '%s' | jq`,
never as an argument. `grep -rn 'argjson\|--arg '` over `scripts/`,
`.agents/` and `.claude/skills/` finds no other data argument.
### An agent authors its commits, and the committer is left alone — `scripts/agent-git-test.sh` (improvise-1xfb.3)
**What was missing.** Every agent commit was authored as the repository owner,
so history could not say which agent wrote what. `scripts/agent-git` runs one
git command with `GIT_AUTHOR_NAME="Claude ($BEADS_ACTOR)"` and refuses, exit 2,
when `BEADS_ACTOR` is unset or empty.
**Why not `git -c user.name=…`,** the mechanism the ticket and the brief both
offered first. `user.name` is the default for the committer as well as the
author, so it renames both. The ticket asks for the committer to be left
alone. Probed in a scratch repo: a `-c user.name` commit reads
`Claude (u) <o@x> | Claude (u)` as author and committer. A `GIT_AUTHOR_NAME`
commit reads `Claude (t) <o@x> | Owner`. The same probe showed that a merge
takes `GIT_AUTHOR_NAME` too, and that a plain `--amend` keeps the author it
amends.
**A cherry-pick keeps its source's author** (Codex P2 and the review on PR
#310). The first version of the rule listed `cherry-pick` among the wrapper's
subcommands, but git ignores `GIT_AUTHOR_NAME` there. That is the behaviour
this repo wants, since rule 8 allows only `cherry-pick -x` of commits that keep
their own author. So the contract names only `commit` and `merge`. The case "a
cherry-pick keeps its source's author" picks a commit authored `Claude
(agent-b2)` with the wrapper set to `agent-a1`, and asserts `agent-b2`. A pick
that failed would leave the owner-authored base at HEAD and fail the case too.
**Rivals** (run against the first eleven cases).
- **`exec git -c user.name="Claude ($BEADS_ACTOR)" "$@"`**: 10 passed,
1 failed, "the committer is untouched" (got `Claude (agent-a1)`).
- **The refusal removed** (`if false`): 7 passed, 4 failed. They are "an unset
BEADS_ACTOR is refused" (exit 1 from `set -u`, not 2), "the refusal names
BEADS_ACTOR", "an empty BEADS_ACTOR is refused" (exit 0: the commit landed
under the name `Claude ()`), and "a refusal commits nothing". So `set -u`
alone would not cover the empty case.
**Fixture.** The repo's identity comes from its own config, with
`GIT_CONFIG_GLOBAL=/dev/null` and `GIT_CONFIG_NOSYSTEM=1`, and every `GIT_*`
identity variable is unset first. So a caller's own identity cannot make the
author or committer assertions agree by accident.
### The repo-stats pipeline computes what it claims to — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
**What it pins.** The fixture repository has one commit before the window, a
side branch merged with `--no-ff`, a shrink in the next ISO week, and one file
in every area. So each number the report prints is known by construction. The
suite runs the real pipeline, code-maat and improvise included, and asserts
improvise's exported tables: each area's added, deleted and deletion ratio;
prose, code and the `ProseToCode` row; the week split; and the per-day totals.
It also checks the doc sizes and the coupling flags, and that an improvise that
refuses or warns makes the pipeline exit 3.
**Two fixture choices are load-bearing.**
- The repo map's `4.` item sits after the `---` that closes the Gotchas
section, with no `##` heading between them. In an earlier fixture a `##`
heading followed the `---`, so the `---` rule could be deleted and the suite
stayed green.
- `churn.csv` must equal code-maat's `abs-churn` on every day. That compares
two independent derivations of one number, one of them improvise's
aggregation. The fixture has a binary-free, rename-free history, so the two
agree exactly.
**Rivals**, each injected into the script or into `repo-stats.improvise` and run
through `checkers.sh repo-stats`:
- Rust src matched before Rust tests: the rust src and rust tests rows fail.
- `DeletionRatio = deleted / (added + deleted)`: six ratio rows fail.
- `ProseToCode = code / prose`: the `ProseToCode` row fails.
- `--no-merges` dropped: the commit count fails (6, wanted 5).
- The `---` end of the Gotchas section ignored: the gotcha count fails (4, wanted 3).
- A crate crossing as "the crates differ", so a crate against no crate counts:
the crate-crossing case fails.
- `kb.pl` counted as prose: the prose row and the ratio fail.
- The category padding removed: the import skipped every record, and both
pipeline runs failed with exit 3. That rival is retired: folder import
(improvise-lpzu) keeps sparse records, and the padding is gone. The suite
passes 41 of 41 without it.
- Months instead of ISO weeks: both week cases fail.
- A `warning:` from improvise ignored: the warning-stub case fails.
### A window of empty commits, and four faults the first fixture missed — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
**What broke.** ChatGPT Codex found it on PR #313, and the adversarial review
reproduced it. When every non-merge commit in the window is `--allow-empty`,
the log has commit headers and no files. `build_dir` took `churn.csv`'s header
from `rows[0]`, raised `IndexError` and exited 1. That exit is outside the
pipeline's failure split: it is neither a number nor improvise's exit 3.
**Fix.** `churn.csv` has a fixed header. When no file changed, the script
skips code-maat and improvise, writes the header alone, and exits 0.
**Red first.** The new case is a second fixture repository whose only
in-window commit is empty. Restoring the `rows[0]` header fails its run case
with the traceback.
**The review's four surviving faults, each now killed:**
- **`--no-renames` dropped.** The fixture now renames `scripts/foo.sh` to
`scripts/bar.sh`. Without the flag git reports one `{foo.sh => bar.sh}` entity
with no lines changed, not a delete and an add. Four cases fail: the scripts/CI
row, the code row, the ratio row and the coupling count.
- **`crate_of` forgetting the root crate.** `src/main.rs` now co-changes with
both crates. The crossing list loses three pairs.
- **A failing `script` call ignored.** A third stub exits 0 for `import` and
refuses `script`. The pipeline then reads exports that were never written and
exits 1, not 3.
- **`.beads/` and `site/` left out of generated.** Both are in the fixture now.
The generated row and the other row both fail.
**Also counted now:** `docs/guide.md` as other docs and `.codex/config.toml` as
agent docs. The coupling count is 16, not 15: `web/app.js` and
`scripts/foo.sh` change together in two commits, the 06-04 one and the rename.
### The pipeline on folder import — `scripts/repo-stats-test.sh` (improvise-1xfb.5.1)
Once #314 merged, `improvise import <dir>` replaced the file list, the `-`
padding and the generated per-file views. The suite needed no change of
expectations and still passed 41 of 41. That shows only that the numbers
agree. It does **not** show that the model is right: the suite could not tell
sparse from padded. The delta review re-added `pad_categories` and the suite
still passed 41 of 41, while the model then had a `|-|` item in 13 places and
the `churn` view seated `agent-rules` on rows and paged everything else. The
view case counted 10 views either way.
**What separates them now:**
- **"the sparse model has no padding item"**: no line of the model names `|-|`.
- **"the churn view seats …"**: four cases, one per seat. `date` is on rows, and
`agent-rules`, `coupled` and `crosses-crate`, which are coupling-only, are on
`none`. The churn view puts `entity` on columns, not `area` as the review
guessed. What discriminates is `date` on rows and the coupling-only
categories on `none`, which a padded CSV cannot give, because it claims all
nine categories.
**Rivals:**
- **Padding re-added** (the review's own injection): 5 cases fail. They are
the `|-|` case, with 161 lines naming it in the fixture model, and all four
seat cases.
- **A report view named like a file stem**: `By day` renamed to `churn` leaves
9 views, and the view-count case fails.
- **The foreign-CSV guard removed**: "a foreign CSV in --out is refused with
exit 2" fails with exit 0. The guard is the answer to the review's
non-blocking finding that `import .` picks up any CSV in `--out`. It refuses
instead of clearing. Deleting files in a directory the caller named is the
wrong default, and the script's own five CSVs are overwritten on every run,
so a rerun into the same `--out` is still accepted. The same fixture run
writes JSON and then text into one directory, which shows that.
context/test-evidence/tui.md
+48
View File
@@ -1429,3 +1429,51 @@ to the list's header row, which no item can cover. This test fills three rows
with a three-line item and a second item after it. It asserts the first item
is drawn whole and the header reads `1–1 of 2`. The rival **the header claims
every item shown** fails it.
### the_category_add_prompt_draws_as_the_item_add_prompt_does
ADR 004 clause 8's one declared exception, decided by the user (Q1 on PR #122:
the green/yellow split between item-add and category-add is an accident). It
draws both prompts through the real `draw_bottom_bar` into a `TestBackend` and
compares the style of the prompt's first cell, after checking that cell really
holds the prompt's `n`.
**Failed first**, at `gh/main` `b639266b` before any face existed: `left:
...yellow().bg(Indexed(235)).bold()`, `right: ...green()...` — the category-add
prompt was yellow. **Injected fault** after the migration: set the shipped
theme's `minibuffer.category-add` back to `Named(Yellow)`; this test fails and
nothing else in the TUI does, which is the point of naming the exception in a
test of its own rather than leaving it out of the no-op table.
### the_shipped_theme_reproduces_every_migrated_literal
ADR 004 clause 8's property: migrating a call site onto a face is a no-op. The
oracle is the literal tables as they stood before the migration — axis
accents, wizard field kinds, `mode_style`'s exhaustive match (kept verbatim as
`mode_bg_before_faces`) and the minibuffer's accent-on-`Indexed(235)`-bold —
and every one is compared with `face_style` of its face. Category-add is left
out on purpose (the entry above). It was first written against the old
functions themselves and passed, then the functions were deleted and their
tables moved into the test.
**Injected faults**, each failing here by the case name in the message:
`axis.column` changed from Blue to Cyan (fails at `Column`); `face_style` made
to add `BOLD` for `Weight::Normal` instead of `Bold`; `Theme::resolve` folded so
that the parent wins over the child (fails at `Editing`, which draws DarkGray
instead of its own Green); and `Face::of_mode(TileSelect)` returning
`mode.resting` (also caught by `the_status_line_is_drawn_in_the_mode_face`).
### the_two_front_ends_draw_the_same_nested_hierarchy_on_rows
improvise-felo.6: Year > Quarter > Month in one category, under a grouped outer
category, with a year open two levels, a quarter closed and a year folded. The
precondition pins the header sequence, so the agreement cannot pass on a frame
that drew no nested header. What makes the two ends agree is that neither
derives membership: both walk the same `FrameEntry` list. Rival: the browser's
`render.rs` skipping a `Group` entry whose `depth > 0`. This test fails on the
row count (the TUI draws the quarter line, the browser does not); the column
test does not, since the band shows only each column's innermost group, which
the rival leaves alone. Group names are short because the TUI
truncates a band label to its column's width, which the browser does not: the
first version named its years `Y2024` and the column test failed on `▶ Y2…`
against `▶ Y2025`, a truncation, not a disagreement.
crates/improvise-command/src/command/cmd/commit.rs
+18 -5
View File
@@ -204,10 +204,15 @@ mod tests {
let mut ctx = make_ctx(&m, &layout, &reg);
ctx.buffers = &bufs;
let effects = CommitExport.execute(&ctx);
assert_eq!(effects.len(), 2);
// 3 effects: SetBuffer (clear), ChangeMode, ExportCsv (improvise-2mqz).
assert_eq!(effects.len(), 3);
let dbg = effects_debug(&effects);
assert!(dbg.contains("ExportCsv"), "Expected ExportCsv, got: {dbg}");
assert!(dbg.contains("Normal"), "Expected Normal mode, got: {dbg}");
assert!(
dbg.contains("SetBuffer { name: \"export\", value: \"\" }"),
"Expected buffer clear, got: {dbg}"
);
assert!(
dbg.find("Normal").unwrap() < dbg.find("ExportCsv").unwrap(),
"the mode change must precede the write an abort can reach: {dbg}"
@@ -1270,9 +1275,13 @@ impl Cmd for CommitSave {
effect::change_mode(AppMode::Normal),
];
}
// The mode change comes **first**, or a failed `SaveAs` aborts with the
// minibuffer still drawn over its error (Copilot, PR #27).
// Clear before `SaveAs`, whose failure aborts; this command owns the clear
// because its LocalThenRequest cannot live in a sequence (improvise-2mqz).
vec![
Box::new(effect::SetBuffer {
name: "save".into(),
value: String::new(),
}),
effect::change_mode(AppMode::Normal),
Box::new(effect::SaveAs(std::path::PathBuf::from(path))),
]
@@ -1294,9 +1303,13 @@ impl Cmd for CommitExport {
effect::change_mode(AppMode::Normal),
];
}
// The mode change comes **first**, or a failed export aborts with the
// minibuffer still drawn over its error (gotcha 21).
// Clear before export, whose failure aborts; this command owns the clear
// because its LocalThenRequest cannot live in a sequence (improvise-2mqz).
vec![
Box::new(effect::SetBuffer {
name: "export".into(),
value: String::new(),
}),
effect::change_mode(AppMode::Normal),
Box::new(effect::ExportCsv(std::path::PathBuf::from(path))),
]
crates/improvise-command/src/command/cmd/core.rs
+61 -13
View File
@@ -250,12 +250,37 @@ pub trait AtCursor: Cmd {
fn cursor_mut(&mut self) -> &mut CursorState;
}
/// How many arguments a command takes. The registry refuses any beyond it, for
/// every command in one place — `context/code/cli.md` §14.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MaxArgs {
AtMost(usize),
Any,
}
impl MaxArgs {
fn admit(self, args: &[String]) -> Result<(), String> {
match self {
MaxArgs::AtMost(n) if args.len() > n => {
let extra: Vec<String> = args[n..].iter().map(|a| format!("{a:?}")).collect();
Err(format!(
"takes at most {n} argument(s), got {}; unexpected {}",
args.len(),
extra.join(" ")
))
}
MaxArgs::AtMost(_) | MaxArgs::Any => Ok(()),
}
}
}
type BoxParseFn = Box<dyn Fn(&[String]) -> Result<Box<dyn Cmd>, String>>;
type BoxInteractiveFn = Box<dyn Fn(&[String], &CmdContext) -> Result<Box<dyn Cmd>, String>>;
/// A registered command entry with both text and interactive constructors.
struct CmdEntry {
name: &'static str,
max: MaxArgs,
parse: BoxParseFn,
interactive: BoxInteractiveFn,
}
@@ -295,10 +320,11 @@ impl CmdRegistry {
pub fn register<C: Cmd + 'static>(
&mut self,
prototype: &C,
max: MaxArgs,
parse: fn(&[String]) -> Result<C, String>,
fallback: Option<FallbackFn<C>>,
) {
self.push(prototype.name(), parse, fallback, |_, _| {});
self.push(prototype.name(), max, parse, fallback, |_, _| {});
}
/// As [`register`](Self::register), then the interactive arm moves the
@@ -306,10 +332,11 @@ impl CmdRegistry {
pub fn register_at_cursor<C: AtCursor + 'static>(
&mut self,
prototype: &C,
max: MaxArgs,
parse: fn(&[String]) -> Result<C, String>,
fallback: Option<FallbackFn<C>>,
) {
self.push(prototype.name(), parse, fallback, |cmd, ctx| {
self.push(prototype.name(), max, parse, fallback, |cmd, ctx| {
*cmd.cursor_mut() = CursorState::from_ctx(ctx);
});
}
@@ -317,12 +344,14 @@ impl CmdRegistry {
fn push<C: Cmd + 'static>(
&mut self,
name: &'static str,
max: MaxArgs,
parse: fn(&[String]) -> Result<C, String>,
fallback: Option<FallbackFn<C>>,
at_cursor: fn(&mut C, &CmdContext),
) {
self.entries.push(CmdEntry {
name,
max,
parse: Box::new(move |args| parse(args).map(|c| Box::new(c) as Box<dyn Cmd>)),
interactive: Box::new(move |args, ctx| {
let mut built = match (parse(args), fallback) {
@@ -338,9 +367,10 @@ impl CmdRegistry {
/// Register a command that needs no interactive context: BOTH arms are
/// `parse`, so neither can demote the other — `context/code/command.md`.
pub fn register_pure(&mut self, prototype: &dyn Cmd, parse: ParseFn) {
pub fn register_pure(&mut self, prototype: &dyn Cmd, max: MaxArgs, parse: ParseFn) {
self.entries.push(CmdEntry {
name: prototype.name(),
max,
parse: Box::new(parse),
interactive: Box::new(move |args, _ctx| parse(args)),
});
@@ -352,6 +382,7 @@ impl CmdRegistry {
let name = f().name();
self.entries.push(CmdEntry {
name,
max: MaxArgs::AtMost(0),
parse: Box::new(move |_| Ok(f())),
interactive: Box::new(move |_, _| Ok(f())),
});
@@ -364,7 +395,9 @@ impl CmdRegistry {
let canonical = self.resolve(name);
for e in &self.entries {
if e.name == canonical {
return (e.parse)(args).map_err(|err| format!("{name}: {err}"));
return (e.parse)(args)
.and_then(|cmd| e.max.admit(args).map(|()| cmd))
.map_err(|err| format!("{name}: {err}"));
}
}
Err(format!("Unknown command: {name}"))
@@ -381,7 +414,7 @@ impl CmdRegistry {
let name = self.resolve(name);
for e in &self.entries {
if e.name == name {
return (e.interactive)(args, ctx);
return (e.interactive)(args, ctx).and_then(|cmd| e.max.admit(args).map(|()| cmd));
}
}
Err(format!("Unknown command: {name}"))
@@ -401,16 +434,31 @@ pub(super) fn require_args(args: &[String], n: usize) -> Result<(), String> {
}
}
/// Parse Cat/Item coordinate args into a CellKey.
pub(super) fn parse_cell_key_from_args(args: &[String]) -> crate::model::cell::CellKey {
let coords: Vec<(String, String)> = args
/// Parse Cat/Item coordinate args into a CellKey, refusing any token that is not
/// one rather than dropping it — `context/code/cli.md` §14.
pub(super) fn parse_cell_key_from_args(
args: &[String],
) -> Result<crate::model::cell::CellKey, String> {
let coords = args
.iter()
.filter_map(|s| {
let (cat, item) = s.split_once('/')?;
Some((cat.to_string(), item.to_string()))
.map(|s| match s.split_once('/') {
Some((cat, item)) if !cat.is_empty() && !item.is_empty() => {
Ok((cat.to_string(), item.to_string()))
}
_ => Err(format!("{s:?} is not a Category/Item coordinate")),
})
.collect();
crate::model::cell::CellKey::new(coords)
.collect::<Result<Vec<_>, _>>()?;
Ok(crate::model::cell::CellKey::new(coords))
}
/// A cursor fallback answers only for coordinates that were OMITTED; given and
/// wrong, they are refused, never the cursor's cell — `context/code/cli.md` §14.
pub(super) fn only_when_omitted(args: &[String]) -> Result<(), String> {
if args.is_empty() {
return Ok(());
}
parse_cell_key_from_args(args)?;
Err("takes Category/Item coordinates, or none to use the cursor".to_string())
}
/// Read the current value of a named buffer from context.
crates/improvise-command/src/command/cmd/effect_cmds.rs
+192 -48
View File
@@ -1,13 +1,13 @@
use crate::model::cell::CellValue;
use crate::ui::effect::{self, Effect, ViewTarget};
use crate::view::Axis;
use super::core::{Cmd, CmdContext, require_args};
use super::core::{Cmd, CmdContext, MaxArgs, parse_axis, parse_cell_key_from_args, require_args};
#[cfg(test)]
mod tests {
use super::*;
use crate::command::cmd::test_helpers::*;
use crate::view::Axis;
#[test]
fn add_category_cmd_produces_add_category_effect() {
@@ -59,7 +59,7 @@ mod tests {
let ctx = make_ctx(&m, &layout, &reg);
let cmd = SetAxisCmd(vec!["Type".to_string(), "col".to_string()]);
let effects = cmd.execute(&ctx);
assert_eq!(effects.len(), 1);
assert_eq!(effects.len(), 2, "the name check, then the write");
let mut app = crate::ui::app::App::new(two_cat_model(), None);
app.apply_effects(effects);
let view = app.model_state.workbook.active_view();
@@ -95,6 +95,104 @@ mod tests {
assert!(present.model_state.workbook.model.formulas().is_empty());
}
/// Run `name args` built by the pure arm against a fresh `App` on `wb`.
fn run_on(wb: &crate::workbook::Workbook, name: &str, args: &[&str]) -> crate::ui::app::App {
let args: Vec<String> = args.iter().map(|s| s.to_string()).collect();
let mut app = crate::ui::app::App::new(wb.clone(), None);
let cmd = crate::command::cmd::registry::default_registry()
.parse(name, &args)
.unwrap_or_else(|e| panic!("`{name} {args:?}` must build: {e}"));
let effects = {
let ctx = app.cmd_context(
crate::command::key::KeyCode::Null,
crate::command::key::KeyModifiers::NONE,
);
cmd.execute(&ctx)
};
app.apply_effects(effects);
app
}
/// A document where every sibling below has something real to name.
fn named_things() -> crate::workbook::Workbook {
let mut wb = two_cat_model();
wb.add_item_in_group("Type", "Tools", "Hardware").unwrap();
wb.create_view("Second");
wb
}
/// improvise-31oe.2 and its siblings: each names something missing, is
/// refused naming it, and changes nothing. Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_command_naming_something_the_document_lacks_is_a_refusal() {
let wb = named_things();
let cases: [(&str, &[&str], &str); 10] = [
("set-axis", &["Regoin", "column"], "Regoin"),
("set-page", &["Regoin", "Food"], "Regoin"),
("set-page", &["Type", "Nowhere"], "Nowhere"),
("hide-item", &["Regoin", "Food"], "Regoin"),
("hide-item", &["Type", "Nowhere"], "Nowhere"),
("show-item", &["Regoin", "Food"], "Regoin"),
("show-item", &["Type", "Nowhere"], "Nowhere"),
("toggle-group", &["Regoin", "Hardware"], "Regoin"),
("toggle-group", &["Type", "Nowhere"], "Nowhere"),
("switch-view", &["Nowhere"], "Nowhere"),
];
let untouched = crate::persistence::format_md(&wb);
for (name, args, missing) in cases {
let app = run_on(&wb, name, args);
let said = &app.view_state.status_msg;
assert!(
app.last_batch_refused(),
"`{name} {args:?}` was not refused: {said:?}"
);
assert!(
said.contains(missing),
"`{name} {args:?}` did not name {missing}: {said:?}"
);
assert_eq!(
crate::persistence::format_md(&app.model_state.workbook),
untouched,
"`{name} {args:?}` changed the document"
);
assert_eq!(
app.view_state.active_view, wb.active_view,
"`{name}` moved the view"
);
}
}
/// The controls: the same commands naming what IS there are not refused.
#[test]
fn a_command_naming_what_the_document_has_is_not_refused() {
let wb = named_things();
let cases: [(&str, &[&str]); 6] = [
("set-axis", &["Type", "column"]),
("set-page", &["Type", "Clothing"]),
("hide-item", &["Type", "Clothing"]),
("show-item", &["Type", "Clothing"]),
("toggle-group", &["Type", "Hardware"]),
("switch-view", &["Second"]),
];
for (name, args) in cases {
let app = run_on(&wb, name, args);
assert!(
!app.last_batch_refused(),
"`{name} {args:?}` was refused: {:?}",
app.view_state.status_msg
);
}
}
/// An axis keyword is an argument error, caught before anything runs.
#[test]
fn set_axis_with_an_unknown_axis_is_an_argument_error() {
let err = crate::command::parse_line("set-axis Type sideways")
.expect_err("sideways is not an axis");
assert!(err.starts_with("set-axis:"), "names the command: {err}");
assert!(err.contains("sideways"), "names the axis: {err}");
}
#[test]
fn write_cmd_without_args_saves() {
let m = two_cat_model();
@@ -129,7 +227,7 @@ mod tests {
// Thin Cmd wrappers around effects, on the same path as keymap dispatch.
macro_rules! effect_cmd {
($name:ident, $cmd_name:expr, $parse:expr, $exec:expr) => {
($name:ident, $cmd_name:expr, $max:expr, $parse:expr, $exec:expr) => {
#[derive(Debug)]
pub struct $name(pub Vec<String>);
impl Cmd for $name {
@@ -143,6 +241,7 @@ macro_rules! effect_cmd {
}
}
impl $name {
pub const MAX_ARGS: MaxArgs = $max;
pub fn parse(args: &[String]) -> Result<Box<dyn Cmd>, String> {
#[allow(clippy::redundant_closure_call)]
($parse)(args)?;
@@ -152,9 +251,16 @@ macro_rules! effect_cmd {
};
}
/// `effect`, preceded by the check that what it names exists, run at APPLY time:
/// a `:` line is built before any of it runs — `context/code/cli.md` §15.
fn unless_unknown(needs: effect::Needs, effect: impl Effect + 'static) -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::RequireKnown(needs)), Box::new(effect)]
}
effect_cmd!(
AddCategoryCmd,
"add-category",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::AddCategory(args[0].clone()))]
@@ -164,6 +270,7 @@ effect_cmd!(
effect_cmd!(
AddItemCmd,
"add-item",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::AddItem {
@@ -176,6 +283,7 @@ effect_cmd!(
effect_cmd!(
AddItemsCmd,
"add-items",
MaxArgs::Any,
|args: &[String]| {
if args.len() < 2 {
Err("add-items requires a category and at least one item".to_string())
@@ -200,6 +308,7 @@ effect_cmd!(
effect_cmd!(
AddItemInGroupCmd,
"add-item-in-group",
MaxArgs::AtMost(3),
|args: &[String]| require_args(args, 3),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::AddItemInGroup {
@@ -213,11 +322,12 @@ effect_cmd!(
effect_cmd!(
SetCellCmd,
"set-cell",
MaxArgs::Any,
|args: &[String]| {
if args.len() < 2 {
Err("set-cell requires a value and at least one Cat/Item coordinate".to_string())
} else {
Ok(())
parse_cell_key_from_args(&args[1..]).map(|_| ())
}
},
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
@@ -226,21 +336,17 @@ effect_cmd!(
} else {
CellValue::Text(args[0].clone())
};
let coords: Vec<(String, String)> = args[1..]
.iter()
.filter_map(|s| {
let (cat, item) = s.split_once('/')?;
Some((cat.to_string(), item.to_string()))
})
.collect();
let key = crate::model::cell::CellKey::new(coords);
vec![Box::new(effect::SetCell(key, value))]
match parse_cell_key_from_args(&args[1..]) {
Ok(key) => vec![Box::new(effect::SetCell(key, value))],
Err(why) => vec![effect::refuse(why)],
}
}
);
effect_cmd!(
AddFormulaCmd,
"add-formula",
MaxArgs::AtMost(2),
|args: &[String]| {
if args.is_empty() || args.len() > 2 {
// The arity alone does not tell anyone to quote — improvise-z05h.
@@ -269,6 +375,7 @@ effect_cmd!(
effect_cmd!(
ClearBufferCmd,
"clear-buffer",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![super::core::write_buffer(&args[0], String::new())]
@@ -278,6 +385,7 @@ effect_cmd!(
effect_cmd!(
RemoveFormulaCmd,
"remove-formula",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
let (category, target) = (&args[0], &args[1]);
@@ -298,6 +406,7 @@ effect_cmd!(
effect_cmd!(
CreateViewCmd,
"create-view",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::CreateView(args[0].clone()))]
@@ -307,6 +416,7 @@ effect_cmd!(
effect_cmd!(
DeleteViewCmd,
"delete-view",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::DeleteView(args[0].clone()))]
@@ -316,87 +426,111 @@ effect_cmd!(
effect_cmd!(
SwitchViewCmd,
"switch-view",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::SwitchView(args[0].clone()))]
unless_unknown(
effect::Needs::View(args[0].clone()),
effect::SwitchView(args[0].clone()),
)
}
);
effect_cmd!(
SetAxisCmd,
"set-axis",
|args: &[String]| require_args(args, 2),
MaxArgs::AtMost(2),
|args: &[String]| {
require_args(args, 2)?;
parse_axis(&args[1]).map(|_| ())
},
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
let axis = match args[1].to_lowercase().as_str() {
"row" => Axis::Row,
"column" | "col" => Axis::Column,
"page" => Axis::Page,
"none" => Axis::None,
_ => return vec![], // parse step already validated
};
vec![Box::new(effect::SetAxis {
view: ViewTarget::Active,
category: args[0].clone(),
axis,
})]
match parse_axis(&args[1]) {
Ok(axis) => unless_unknown(
effect::Needs::Category(args[0].clone()),
effect::SetAxis {
view: ViewTarget::Active,
category: args[0].clone(),
axis,
},
),
Err(why) => vec![effect::refuse(why)],
}
}
);
effect_cmd!(
SetPageCmd,
"set-page",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::SetPageSelection {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
})]
unless_unknown(
effect::Needs::Item(args[0].clone(), args[1].clone()),
effect::SetPageSelection {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
},
)
}
);
effect_cmd!(
ToggleGroupCmd,
"toggle-group",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::ToggleGroup {
view: ViewTarget::Active,
category: args[0].clone(),
group: args[1].clone(),
})]
unless_unknown(
effect::Needs::Group(args[0].clone(), args[1].clone()),
effect::ToggleGroup {
view: ViewTarget::Active,
category: args[0].clone(),
group: args[1].clone(),
},
)
}
);
effect_cmd!(
HideItemCmd,
"hide-item",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::HideItem {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
})]
unless_unknown(
effect::Needs::Item(args[0].clone(), args[1].clone()),
effect::HideItem {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
},
)
}
);
effect_cmd!(
ShowItemCmd,
"show-item",
MaxArgs::AtMost(2),
|args: &[String]| require_args(args, 2),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::ShowItem {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
})]
unless_unknown(
effect::Needs::Item(args[0].clone(), args[1].clone()),
effect::ShowItem {
view: ViewTarget::Active,
category: args[0].clone(),
item: args[1].clone(),
},
)
}
);
effect_cmd!(
SaveAsCmd,
"save-as",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::SaveAs(std::path::PathBuf::from(&args[0])))]
@@ -406,6 +540,7 @@ effect_cmd!(
effect_cmd!(
SetFormatCmd,
"set-format",
MaxArgs::Any,
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::SetNumberFormat(
@@ -418,6 +553,7 @@ effect_cmd!(
effect_cmd!(
ImportCmd,
"import",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::StartImportWizard(args[0].clone()))]
@@ -427,6 +563,7 @@ effect_cmd!(
effect_cmd!(
ExportCmd,
"export",
MaxArgs::AtMost(1),
|_args: &[String]| -> Result<(), String> { Ok(()) },
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
let path = args.first().map(|s| s.as_str()).unwrap_or("export.csv");
@@ -437,6 +574,7 @@ effect_cmd!(
effect_cmd!(
WriteCmd,
"w",
MaxArgs::AtMost(1),
|_args: &[String]| -> Result<(), String> { Ok(()) },
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
if args.is_empty() {
@@ -450,6 +588,7 @@ effect_cmd!(
effect_cmd!(
HelpCmd,
"help",
MaxArgs::AtMost(0),
|_args: &[String]| -> Result<(), String> { Ok(()) },
|_args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![
@@ -462,6 +601,7 @@ effect_cmd!(
effect_cmd!(
HelpPageNextCmd,
"help-page-next",
MaxArgs::AtMost(0),
|_args: &[String]| -> Result<(), String> { Ok(()) },
|_args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![effect::help_page_next()]
@@ -471,6 +611,7 @@ effect_cmd!(
effect_cmd!(
HelpPagePrevCmd,
"help-page-prev",
MaxArgs::AtMost(0),
|_args: &[String]| -> Result<(), String> { Ok(()) },
|_args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![effect::help_page_prev()]
@@ -480,6 +621,7 @@ effect_cmd!(
effect_cmd!(
LoadModelCmd,
"load",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::LoadModel(std::path::PathBuf::from(
@@ -491,6 +633,7 @@ effect_cmd!(
effect_cmd!(
ExportCsvCmd,
"export-csv",
MaxArgs::AtMost(1),
|args: &[String]| require_args(args, 1),
|args: &Vec<String>, _ctx: &CmdContext| -> Vec<Box<dyn Effect>> {
vec![Box::new(effect::ExportCsv(std::path::PathBuf::from(
@@ -502,6 +645,7 @@ effect_cmd!(
effect_cmd!(
ImportJsonCmd,
"import-json",
MaxArgs::AtMost(3),
|args: &[String]| {
if args.is_empty() {
Err("import-json requires a path".to_string())
crates/improvise-command/src/command/cmd/fit.rs
+3 -1
View File
@@ -4,7 +4,7 @@
use improvise_gp::fit::one_line;
use improvise_gp::{FitRequest, fit_measure, fitted_measure_name};
use crate::command::cmd::core::{Cmd, CmdContext};
use crate::command::cmd::core::{Cmd, CmdContext, MaxArgs};
use crate::ui::effect::{self, Effect};
/// A seed as the status line spells it (`0x…`, either case) or in decimal. The hex
@@ -27,6 +27,8 @@ pub struct FitMeasure {
}
impl FitMeasure {
pub const MAX_ARGS: MaxArgs = MaxArgs::AtMost(3);
/// A nameless instance for `register_pure`, which reads a command's name
/// off a prototype rather than off a string literal (gotcha 15).
pub fn prototype() -> Self {
crates/improvise-command/src/command/cmd/grid.rs
+71 -6
View File
@@ -177,6 +177,7 @@ mod tests {
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.add_item_in_group("Month", "Apr", "Q2").unwrap();
wb.add_item_in_group("Month", "May", "Q2").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb
}
@@ -213,7 +214,7 @@ mod tests {
!app.model_state
.workbook
.active_view()
.is_group_collapsed("Month", "Q2"),
.is_group_collapsed("Month", &["Q2"]),
"the fixture starts expanded, or the assertion below proves nothing"
);
@@ -225,11 +226,11 @@ mod tests {
);
let view = app.model_state.workbook.active_view();
assert!(
view.is_group_collapsed("Month", "Q2"),
view.is_group_collapsed("Month", &["Q2"]),
"the effect was returned but the view did not move"
);
assert!(
!view.is_group_collapsed("Month", "Q1"),
!view.is_group_collapsed("Month", &["Q1"]),
"the neighbour was left alone"
);
}
@@ -262,9 +263,9 @@ mod tests {
);
app.apply_effects(effects);
let view = app.model_state.workbook.active_view();
assert!(view.is_group_collapsed("Month", "Q2"));
assert!(view.is_group_collapsed("Month", &["Q2"]));
assert!(
!view.is_group_collapsed("Month", "Q1"),
!view.is_group_collapsed("Month", &["Q1"]),
"the neighbour stays"
);
}
@@ -286,6 +287,7 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.add_item("Type", "Tools").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Column);
@@ -328,6 +330,66 @@ mod tests {
);
}
/// improvise-felo.6: `z` walks DOWN a closed hierarchy — year, then quarter —
/// and back up, the cursor on the group it acted on each time.
#[test]
fn z_opens_a_closed_hierarchy_one_level_at_a_time_and_closes_it_again() {
use crate::ui::app::App;
let mut wb = crate::workbook::Workbook::new("T");
wb.add_category("Date").unwrap();
wb.add_category("Type").unwrap();
for (month, quarter) in [
("Jan", "Q1"),
("Feb", "Q1"),
("Apr", "Q2"),
("Jan25", "Q1-25"),
] {
wb.add_item_in_group("Date", month, quarter).unwrap();
}
let date = wb.model.category_mut("Date").unwrap();
for (quarter, year) in [("Q1", "Y2024"), ("Q2", "Y2024"), ("Q1-25", "Y2025")] {
date.add_group(crate::model::category::Group::new(quarter).with_parent(year));
}
wb.add_item("Type", "Food").unwrap();
let mut app = App::new(wb, None);
app.rebuild_layout();
assert_eq!(row_labels(&app), ["Y2024", "Y2025"], "closed by default");
app.cursor_mut().selected = (1, 0);
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Q1-25"],
"Y2025 opens to its quarter"
);
assert_eq!(app.cursor().selected, (1, 0));
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Jan25"],
"then the quarter to its month"
);
toggle_group_at_cursor(&mut app);
assert_eq!(
row_labels(&app),
["Y2024", "Q1-25"],
"z on the month closes its quarter"
);
app.cursor_mut().selected = (0, 0);
toggle_group_at_cursor(&mut app);
assert_eq!(row_labels(&app), ["Q1", "Q2", "Q1-25"]);
assert_eq!(
app.model_state
.workbook
.active_view()
.group_fold("Date", "Q1-25"),
None,
"closing back to the default leaves no fold behind"
);
}
/// Evidence: `context/test-evidence/command.md`.
#[test]
fn toggling_a_group_collapses_that_group_and_the_cursor_follows_it() {
@@ -393,6 +455,7 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("Month", Axis::Row);
@@ -704,7 +767,9 @@ impl ToggleGroupAtCursor {
};
let prefix = tuple.get(..cats.iter().position(|c| c == cat)?)?;
let mut view = ctx.view.clone();
view.toggle_group_collapse(cat, group);
let chain = ctx.model.group_chain(cat, group);
let chain: Vec<&str> = chain.iter().map(String::as_str).collect();
view.toggle_group_collapse(cat, &chain);
let after = GridLayout::with_frozen_records(ctx.model, &view, None);
match self.is_row {
true => after.row_run_start(cat, group, prefix),
crates/improvise-command/src/command/cmd/mod.rs
+2 -1
View File
@@ -155,6 +155,7 @@ pub(super) mod test_helpers {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("_Measure", "Revenue").unwrap();
wb.add_item("_Measure", "Cost").unwrap();
for (i, month) in ["Jan", "Feb", "Apr", "May"].into_iter().enumerate() {
@@ -168,7 +169,7 @@ pub(super) mod test_helpers {
}
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let mut app = crate::ui::app::App::new(wb, None);
app.rebuild_layout();
app.cursor_mut().selected = (2, 1);
crates/improvise-command/src/command/cmd/mode.rs
+2 -1
View File
@@ -197,8 +197,9 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let layout = make_layout(&wb);
let reg = make_registry();
let mut ctx = make_ctx(&wb, &layout, &reg);
crates/improvise-command/src/command/cmd/registry.rs
+223 -40
View File
@@ -81,52 +81,130 @@ pub fn default_registry() -> CmdRegistry {
let mut r = CmdRegistry::new();
// ── Model mutations (effect_cmd! wrappers) ───────────────────────────
r.register_pure(&AddCategoryCmd(vec![]), AddCategoryCmd::parse);
r.register_pure(&AddItemCmd(vec![]), AddItemCmd::parse);
r.register_pure(&AddItemsCmd(vec![]), AddItemsCmd::parse);
r.register_pure(&AddItemInGroupCmd(vec![]), AddItemInGroupCmd::parse);
r.register_pure(&SetCellCmd(vec![]), SetCellCmd::parse);
r.register_pure(&ClearBufferCmd(vec![]), ClearBufferCmd::parse);
r.register_pure(
&AddCategoryCmd(vec![]),
AddCategoryCmd::MAX_ARGS,
AddCategoryCmd::parse,
);
r.register_pure(&AddItemCmd(vec![]), AddItemCmd::MAX_ARGS, AddItemCmd::parse);
r.register_pure(
&AddItemsCmd(vec![]),
AddItemsCmd::MAX_ARGS,
AddItemsCmd::parse,
);
r.register_pure(
&AddItemInGroupCmd(vec![]),
AddItemInGroupCmd::MAX_ARGS,
AddItemInGroupCmd::parse,
);
r.register_pure(&SetCellCmd(vec![]), SetCellCmd::MAX_ARGS, SetCellCmd::parse);
r.register_pure(
&ClearBufferCmd(vec![]),
ClearBufferCmd::MAX_ARGS,
ClearBufferCmd::parse,
);
r.register(
&ClearCellCommand {
key: CellKey::new(vec![]),
},
MaxArgs::Any,
|args| {
if args.is_empty() {
return Err("clear-cell requires at least one Cat/Item coordinate".into());
}
Ok(ClearCellCommand {
key: parse_cell_key_from_args(args),
key: parse_cell_key_from_args(args)?,
})
},
Some(|_, ctx| {
Some(|args, ctx| {
only_when_omitted(args)?;
Ok(ClearCellCommand {
key: cursor_value_key(ctx)?,
})
}),
);
r.register_pure(&AddFormulaCmd(vec![]), AddFormulaCmd::parse);
r.register_pure(&FitMeasure::prototype(), FitMeasure::parse);
r.register_pure(&RemoveFormulaCmd(vec![]), RemoveFormulaCmd::parse);
r.register_pure(&CreateViewCmd(vec![]), CreateViewCmd::parse);
r.register_pure(&DeleteViewCmd(vec![]), DeleteViewCmd::parse);
r.register_pure(&SwitchViewCmd(vec![]), SwitchViewCmd::parse);
r.register_pure(&SetAxisCmd(vec![]), SetAxisCmd::parse);
r.register_pure(&SetPageCmd(vec![]), SetPageCmd::parse);
r.register_pure(&ToggleGroupCmd(vec![]), ToggleGroupCmd::parse);
r.register_pure(&HideItemCmd(vec![]), HideItemCmd::parse);
r.register_pure(&ShowItemCmd(vec![]), ShowItemCmd::parse);
r.register_pure(&SaveAsCmd(vec![]), SaveAsCmd::parse);
r.register_pure(&LoadModelCmd(vec![]), LoadModelCmd::parse);
r.register_pure(&ExportCsvCmd(vec![]), ExportCsvCmd::parse);
r.register_pure(&ImportJsonCmd(vec![]), ImportJsonCmd::parse);
r.register_pure(&SetFormatCmd(vec![]), SetFormatCmd::parse);
r.register_pure(&ImportCmd(vec![]), ImportCmd::parse);
r.register_pure(&ExportCmd(vec![]), ExportCmd::parse);
r.register_pure(&WriteCmd(vec![]), WriteCmd::parse);
r.register_pure(&HelpCmd(vec![]), HelpCmd::parse);
r.register_pure(&HelpPageNextCmd(vec![]), HelpPageNextCmd::parse);
r.register_pure(&HelpPagePrevCmd(vec![]), HelpPagePrevCmd::parse);
r.register_pure(
&AddFormulaCmd(vec![]),
AddFormulaCmd::MAX_ARGS,
AddFormulaCmd::parse,
);
r.register_pure(
&FitMeasure::prototype(),
FitMeasure::MAX_ARGS,
FitMeasure::parse,
);
r.register_pure(
&RemoveFormulaCmd(vec![]),
RemoveFormulaCmd::MAX_ARGS,
RemoveFormulaCmd::parse,
);
r.register_pure(
&CreateViewCmd(vec![]),
CreateViewCmd::MAX_ARGS,
CreateViewCmd::parse,
);
r.register_pure(
&DeleteViewCmd(vec![]),
DeleteViewCmd::MAX_ARGS,
DeleteViewCmd::parse,
);
r.register_pure(
&SwitchViewCmd(vec![]),
SwitchViewCmd::MAX_ARGS,
SwitchViewCmd::parse,
);
r.register_pure(&SetAxisCmd(vec![]), SetAxisCmd::MAX_ARGS, SetAxisCmd::parse);
r.register_pure(&SetPageCmd(vec![]), SetPageCmd::MAX_ARGS, SetPageCmd::parse);
r.register_pure(
&ToggleGroupCmd(vec![]),
ToggleGroupCmd::MAX_ARGS,
ToggleGroupCmd::parse,
);
r.register_pure(
&HideItemCmd(vec![]),
HideItemCmd::MAX_ARGS,
HideItemCmd::parse,
);
r.register_pure(
&ShowItemCmd(vec![]),
ShowItemCmd::MAX_ARGS,
ShowItemCmd::parse,
);
r.register_pure(&SaveAsCmd(vec![]), SaveAsCmd::MAX_ARGS, SaveAsCmd::parse);
r.register_pure(
&LoadModelCmd(vec![]),
LoadModelCmd::MAX_ARGS,
LoadModelCmd::parse,
);
r.register_pure(
&ExportCsvCmd(vec![]),
ExportCsvCmd::MAX_ARGS,
ExportCsvCmd::parse,
);
r.register_pure(
&ImportJsonCmd(vec![]),
ImportJsonCmd::MAX_ARGS,
ImportJsonCmd::parse,
);
r.register_pure(
&SetFormatCmd(vec![]),
SetFormatCmd::MAX_ARGS,
SetFormatCmd::parse,
);
r.register_pure(&ImportCmd(vec![]), ImportCmd::MAX_ARGS, ImportCmd::parse);
r.register_pure(&ExportCmd(vec![]), ExportCmd::MAX_ARGS, ExportCmd::parse);
r.register_pure(&WriteCmd(vec![]), WriteCmd::MAX_ARGS, WriteCmd::parse);
r.register_pure(&HelpCmd(vec![]), HelpCmd::MAX_ARGS, HelpCmd::parse);
r.register_pure(
&HelpPageNextCmd(vec![]),
HelpPageNextCmd::MAX_ARGS,
HelpPageNextCmd::parse,
);
r.register_pure(
&HelpPagePrevCmd(vec![]),
HelpPagePrevCmd::MAX_ARGS,
HelpPagePrevCmd::parse,
);
// ── Navigation (unified Move) ──────────────────────────────────────
r.register_at_cursor(
@@ -135,6 +213,7 @@ pub fn default_registry() -> CmdRegistry {
cursor: CursorState::default(),
cmd_name: "move-selection",
},
MaxArgs::AtMost(2),
|args| {
require_args(args, 2)?;
let dr = args[0].parse::<i32>().map_err(|e| e.to_string())?;
@@ -160,6 +239,7 @@ pub fn default_registry() -> CmdRegistry {
cursor: CursorState::default(),
cmd_name: $name,
},
MaxArgs::AtMost(0),
|_| {
Ok(Move {
kind: if $to_end {
@@ -185,6 +265,7 @@ pub fn default_registry() -> CmdRegistry {
cursor: CursorState::default(),
cmd_name: "scroll-rows",
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let n = args[0].parse::<i32>().map_err(|e| e.to_string())?;
@@ -202,6 +283,7 @@ pub fn default_registry() -> CmdRegistry {
cursor: CursorState::default(),
cmd_name: "page-scroll",
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let dir = args[0].parse::<i32>().map_err(|e| e.to_string())?;
@@ -217,6 +299,7 @@ pub fn default_registry() -> CmdRegistry {
&EnterAdvance {
cursor: CursorState::default(),
},
MaxArgs::AtMost(0),
|_| {
Ok(EnterAdvance {
cursor: CursorState {
@@ -239,15 +322,17 @@ pub fn default_registry() -> CmdRegistry {
&YankCell {
key: CellKey::new(vec![]),
},
MaxArgs::Any,
|args| {
if args.is_empty() {
return Err("yank requires at least one Cat/Item coordinate".into());
}
Ok(YankCell {
key: parse_cell_key_from_args(args),
key: parse_cell_key_from_args(args)?,
})
},
Some(|_, ctx| {
Some(|args, ctx| {
only_when_omitted(args)?;
Ok(YankCell {
key: cursor_key_to_read(ctx)?,
})
@@ -257,15 +342,17 @@ pub fn default_registry() -> CmdRegistry {
&PasteCell {
at: PasteAt::Spelled(CellKey::new(vec![])),
},
MaxArgs::Any,
|args| {
if args.is_empty() {
return Err("paste requires at least one Cat/Item coordinate".into());
}
Ok(PasteCell {
at: PasteAt::Spelled(parse_cell_key_from_args(args)),
at: PasteAt::Spelled(parse_cell_key_from_args(args)?),
})
},
Some(|_, ctx| {
Some(|args, ctx| {
only_when_omitted(args)?;
Ok(PasteCell {
at: PasteAt::Resolved(cursor_value_key(ctx)?),
})
@@ -288,6 +375,7 @@ pub fn default_registry() -> CmdRegistry {
&EditOrDrill {
edit_mode: AppMode::Normal,
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let edit_mode = parse_mode_name(&args[0])?;
@@ -298,6 +386,7 @@ pub fn default_registry() -> CmdRegistry {
&EnterEditAtCursorCmd {
target_mode: AppMode::Normal,
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let target_mode = parse_mode_name(&args[0])?;
@@ -309,15 +398,17 @@ pub fn default_registry() -> CmdRegistry {
&DrillIntoCell {
key: crate::model::cell::CellKey::new(vec![]),
},
MaxArgs::Any,
|args| {
if args.is_empty() {
return Err("drill-into-cell requires Cat/Item coordinates".into());
}
Ok(DrillIntoCell {
key: parse_cell_key_from_args(args),
key: parse_cell_key_from_args(args)?,
})
},
Some(|_, ctx| {
Some(|args, ctx| {
only_when_omitted(args)?;
Ok(DrillIntoCell {
key: cursor_key(ctx)?,
})
@@ -325,13 +416,13 @@ pub fn default_registry() -> CmdRegistry {
);
r.register_nullary(|| Box::new(ViewNavigate { forward: false }));
r.register_nullary(|| Box::new(ViewNavigate { forward: true }));
r.register_pure(&EnterMode(AppMode::Normal), |args| {
r.register_pure(&EnterMode(AppMode::Normal), MaxArgs::AtMost(1), |args| {
require_args(args, 1)?;
Ok(Box::new(EnterMode(parse_mode_name(&args[0])?)))
});
// ── Search ───────────────────────────────────────────────────────────
r.register_pure(&SearchNavigate(true), |args| {
r.register_pure(&SearchNavigate(true), MaxArgs::AtMost(1), |args| {
let forward = args.first().map(|s| s != "backward").unwrap_or(true);
Ok(Box::new(SearchNavigate(forward)))
});
@@ -344,6 +435,7 @@ pub fn default_registry() -> CmdRegistry {
panel: Panel::Formula,
target: None,
},
MaxArgs::AtMost(3),
|args| {
// Parse: toggle-panel-and-focus <panel> [open] [focused]
require_args(args, 1)?;
@@ -360,6 +452,7 @@ pub fn default_registry() -> CmdRegistry {
&TogglePanelVisibility {
panel: Panel::Formula,
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let panel = parse_panel(&args[0])?;
@@ -373,6 +466,7 @@ pub fn default_registry() -> CmdRegistry {
panel: Panel::Formula,
delta: 0,
},
MaxArgs::AtMost(2),
|args| {
require_args(args, 2)?;
let panel = parse_panel(&args[0])?;
@@ -396,7 +490,7 @@ pub fn default_registry() -> CmdRegistry {
r.register_nullary(|| Box::new(DeleteViewAtCursor));
// ── Tile select ──────────────────────────────────────────────────────
r.register_pure(&MoveTileCursor(0), |args| {
r.register_pure(&MoveTileCursor(0), MaxArgs::AtMost(1), |args| {
require_args(args, 1)?;
let delta = args[0].parse::<i32>().map_err(|e| e.to_string())?;
Ok(Box::new(MoveTileCursor(delta)))
@@ -406,6 +500,7 @@ pub fn default_registry() -> CmdRegistry {
&TileAxisOp {
axis: Some(crate::view::Axis::Row),
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let axis = parse_axis(&args[0])?;
@@ -423,6 +518,7 @@ pub fn default_registry() -> CmdRegistry {
&AppendChar {
buffer: String::new(),
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
Ok(Box::new(AppendChar {
@@ -434,6 +530,7 @@ pub fn default_registry() -> CmdRegistry {
&PopChar {
buffer: String::new(),
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
Ok(Box::new(PopChar {
@@ -454,13 +551,14 @@ pub fn default_registry() -> CmdRegistry {
edit_mode: AppMode::editing(),
opened: None,
},
MaxArgs::Any,
|args| {
if args.len() < 3 {
return Err("commit-cell-edit requires a mode, value, and coords".into());
}
let edit_mode = parse_mode_name(&args[0])?;
Ok(CommitAndAdvance {
key: parse_cell_key_from_args(&args[2..]),
key: parse_cell_key_from_args(&args[2..])?,
value: args[1].clone(),
advance: AdvanceDir::Down,
cursor: CursorState::default(),
@@ -479,6 +577,7 @@ pub fn default_registry() -> CmdRegistry {
edit_mode: AppMode::editing(),
opened: None,
},
MaxArgs::AtMost(1),
|_| Err("commit-and-advance-right requires context".into()),
Some(|args, ctx| commit_at_cursor(args, ctx, AdvanceDir::Right)),
);
@@ -486,6 +585,7 @@ pub fn default_registry() -> CmdRegistry {
&CommitBuffer {
kind: CommitKind::Category,
},
MaxArgs::AtMost(1),
|args| {
require_args(args, 1)?;
let kind = parse_commit_kind(&args[0], &AppMode::Normal)?;
@@ -534,6 +634,7 @@ pub(super) mod tests {
vec!["a"],
vec!["Region"],
vec!["Region", "East"],
vec!["Region", "row"],
vec!["Region", "East", "1"],
vec!["Profit = 1"],
vec![",.2f"],
@@ -544,6 +645,7 @@ pub(super) mod tests {
vec!["editing", "5", "Region/East"],
vec!["formula", "false", "false"],
vec!["Region/East"],
vec!["1", "Region/East"],
]
.into_iter()
.map(|v| v.into_iter().map(String::from).collect())
@@ -592,6 +694,87 @@ pub(super) mod tests {
);
}
/// Evidence: `context/test-evidence/cli.md`.
#[test]
fn no_command_silently_drops_an_argument_it_was_given() {
let r = default_registry();
let wb = two_cat_model();
let layout = make_layout(&wb);
let reg = make_registry();
let ctx = make_ctx(&wb, &layout, &reg);
let variadic = [
"add-items",
"set-cell",
"set-format",
"clear-cell",
"yank",
"paste",
"drill-into-cell",
"commit-cell-edit",
];
// Their own refusal states the shape, and it runs before the rule.
let own_shape = ["add-formula", "fit-measure", "commit-and-advance-right"];
let mut checked = 0;
for name in r.names().collect::<Vec<_>>() {
if variadic.contains(&name) {
continue;
}
for args in candidate_args() {
// Four more than any list accepted, so an optional argument
// cannot absorb them; the last one is surplus whatever the command.
let mut more = args.clone();
more.extend(["EXTRA1", "EXTRA2", "EXTRA3", "EXTRA4"].map(String::from));
let arms = [
r.parse(name, &args).map(|_| r.parse(name, &more)),
r.interactive(name, &args, &ctx)
.map(|_| r.interactive(name, &more, &ctx)),
];
for extended in arms.into_iter().flatten() {
checked += 1;
match extended {
Ok(_) => panic!("`{name} {more:?}` dropped arguments without a word"),
Err(e) if own_shape.contains(&name) || e.contains("\"EXTRA4\"") => {}
Err(e) => {
panic!("`{name}` refused an extra argument without naming it: {e}")
}
}
}
}
}
assert!(checked > 100, "only {checked} accepted lists were extended");
}
/// Codex on PR #316: a variadic coordinate list dropped a token that was no
/// coordinate. Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_malformed_coordinate_is_refused_not_dropped() {
let lines = [
"set-cell 1 Type/Food unexpected",
"clear-cell Type/Food unexpected",
"yank Type/Food unexpected",
"paste Type/Food unexpected",
"drill-into-cell Type/Food unexpected",
"commit-cell-edit editing 5 Type/Food unexpected",
];
for line in lines {
let err = crate::command::parse_line(line)
.err()
.unwrap_or_else(|| panic!("`{line}` dropped \"unexpected\" without a word"));
assert!(
err.contains("\"unexpected\""),
"`{line}` did not name it: {err}"
);
}
for half_empty in ["set-cell 1 Type/", "clear-cell /Food"] {
assert!(
crate::command::parse_line(half_empty).is_err(),
"`{half_empty}`"
);
}
let control = "set-cell 1 Type/Food Month/Jan . clear-cell Type/Food";
assert!(crate::command::parse_line(control).is_ok(), "the control");
}
/// A variant mix-up shows up as a **collision**, not a mismatch: the wrong entry
/// takes its sibling's name and the sibling vanishes. Uniqueness catches it.
#[test]
crates/improvise-command/src/command/cmd/text_buffer.rs
+147 -3
View File
@@ -216,6 +216,110 @@ mod tests {
app.apply_effects(effects);
}
fn cell(wb: &crate::workbook::Workbook, ty: &str, month: &str) -> Option<f64> {
use crate::model::cell::{CellKey, CellValue};
let key = CellKey::new(vec![
("Type".into(), ty.into()),
("Month".into(), month.into()),
]);
match wb.model.get_cell(&key) {
Some(CellValue::Number(n)) => Some(*n),
_ => None,
}
}
/// An app whose cursor (0,0) is on Food/Jan = 1, with Clothing/Feb = 2 beside it.
fn app_with_two_cells() -> crate::ui::app::App {
use crate::model::cell::{CellKey, CellValue};
let mut app = wizard_test_app();
for (ty, month, n) in [("Food", "Jan", 1.0), ("Clothing", "Feb", 2.0)] {
let key = CellKey::new(vec![
("Type".into(), ty.into()),
("Month".into(), month.into()),
]);
app.model_state
.workbook
.model
.set_cell(key, CellValue::Number(n));
}
app
}
/// PR #316 review: a malformed coordinate on `:` fell into the cursor fallback
/// and wrote the CURSOR's cell. Evidence: `context/test-evidence/cli.md`.
#[test]
fn colon_refuses_a_malformed_coordinate_and_leaves_the_cursor_cell_alone() {
let lines = [
"clear-cell Type/Clothing Month/Feb junk",
"yank Type/Clothing Month/Feb junk",
"paste Type/Clothing Month/Feb junk",
"drill-into-cell Type/Clothing Month/Feb junk",
];
for line in lines {
let mut app = app_with_two_cells();
app.view_state.yanked = Some(crate::model::cell::CellValue::Number(9.0));
let before = app.view_state.yanked.clone();
run_colon_command(&mut app, line);
assert!(app.last_batch_refused(), "`:{line}` was not refused");
assert!(
app.view_state.status_msg.contains("\"junk\""),
"`:{line}` did not name the token: {:?}",
app.view_state.status_msg
);
let wb = &app.model_state.workbook;
assert_eq!(
cell(wb, "Food", "Jan"),
Some(1.0),
"`:{line}` wrote the cursor cell"
);
assert_eq!(
cell(wb, "Clothing", "Feb"),
Some(2.0),
"`:{line}` wrote the named cell"
);
assert_eq!(app.view_state.yanked, before, "`:{line}` yanked");
assert!(app.drill_state.is_none(), "`:{line}` opened a drill");
}
}
/// The fallback still answers for coordinates that were OMITTED.
#[test]
fn colon_clear_cell_with_no_coordinates_still_uses_the_cursor() {
let mut app = app_with_two_cells();
run_colon_command(&mut app, "clear-cell");
let wb = &app.model_state.workbook;
assert_eq!(cell(wb, "Food", "Jan"), None);
assert_eq!(cell(wb, "Clothing", "Feb"), Some(2.0));
}
/// A later `:` segment sees what an earlier one made, as `improvise cmd` does:
/// the line is built before either half runs. Evidence: `context/test-evidence/cli.md`.
#[test]
fn colon_lets_a_later_segment_use_what_an_earlier_one_created() {
let mut app = wizard_test_app();
run_colon_command(&mut app, "create-view Second . switch-view Second");
assert!(!app.last_batch_refused(), "{:?}", app.view_state.status_msg);
assert_eq!(app.view_state.active_view, "Second");
let mut app = wizard_test_app();
run_colon_command(&mut app, "add-item Type Tools . set-page Type Tools");
assert!(!app.last_batch_refused(), "{:?}", app.view_state.status_msg);
}
/// The control: a name NOTHING creates is still refused, on `:` too.
#[test]
fn colon_still_refuses_a_name_nothing_creates() {
for line in [
"switch-view Nowhere",
"set-page Type Nowhere",
"hide-item Regoin Food",
] {
let mut app = wizard_test_app();
run_colon_command(&mut app, line);
assert!(app.last_batch_refused(), "`:{line}` was not refused");
}
}
/// improvise-toc: `StartImportWizard` swaps the mode only on success, so a
/// per-type answer snapped `:import` straight back out of the wizard.
#[test]
@@ -271,6 +375,43 @@ mod tests {
);
}
/// improvise-31oe.2 on the `:` line: the refusal is on the status line,
/// naming what is missing. Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_colon_line_naming_something_missing_refuses_on_the_status_line() {
for (line, missing) in [
("set-axis Regoin row", "Regoin"),
("set-axis Type sideways", "sideways"),
("set-page Type Nowhere", "Nowhere"),
("create-view By area", "\"area\""),
] {
let mut app = wizard_test_app();
run_colon_command(&mut app, line);
let said = &app.view_state.status_msg;
assert!(
app.last_batch_refused(),
"`:{line}` was not refused: {said:?}"
);
assert!(
said.contains(missing),
"`:{line}` did not name {missing}: {said:?}"
);
assert!(matches!(app.view_state.mode, AppMode::Normal), "`:{line}`");
}
}
/// improvise-31oe.3 on the `:` line.
#[test]
fn a_colon_line_reads_a_pipe_quoted_name() {
let mut app = wizard_test_app();
run_colon_command(&mut app, "create-view |By area|");
let views: Vec<&String> = app.model_state.workbook.views.keys().collect();
assert!(
app.model_state.workbook.views.contains_key("By area"),
"{views:?}"
);
}
/// A command that does not touch the mode still returns to Normal.
#[test]
fn colon_plain_command_returns_to_normal() {
@@ -372,9 +513,12 @@ impl Cmd for ExecuteCommand {
// Built against the live cursor, exactly as a script's is — improvise-6uva.
let built: Result<Vec<Box<dyn Cmd>>, String> = crate::command::parse::segments(&raw)
.into_iter()
.map(|seg| crate::command::parse::parse_segment_interactive(seg, ctx))
.collect::<Result<Vec<_>, _>>()
.and_then(|segments| {
segments
.into_iter()
.map(|seg| crate::command::parse::parse_segment_interactive(seg, ctx))
.collect::<Result<Vec<_>, _>>()
})
.map(|built| built.into_iter().flatten().collect());
match built {
Ok(cmds) => {
crates/improvise-command/src/command/cmd/wizard.rs
+1
View File
@@ -65,6 +65,7 @@ fn review_proposal_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
fn configure_date_keys(key: KeyCode) -> Vec<Box<dyn Effect>> {
match key {
KeyCode::Char(' ') => op(WizardOp::ToggleDateComponent),
KeyCode::Char('s') => op(WizardOp::ToggleDateShape),
KeyCode::Enter => op(WizardOp::Advance),
_ => cursor_keys(key),
}
crates/improvise-command/src/command/parse.rs
+208 -85
View File
@@ -1,5 +1,10 @@
//! Quasi-lisp prefix command parser: `word arg1 arg2 …`, `.` between commands,
//! `Category/Item` coordinate pairs, `"quoted strings"`.
//! `Category/Item` coordinate pairs, `"quoted strings"` and `|pipe-quoted names|`.
use std::iter::Peekable;
use std::str::CharIndices;
use improvise_formula::quote::unescape_pipe_body;
use super::cmd::{Cmd, CmdContext, CmdRegistry, default_registry};
@@ -11,39 +16,19 @@ pub fn parse_line(line: &str) -> Result<Vec<Box<dyn Cmd>>, String> {
/// Parse a line into commands using a given registry.
pub fn parse_line_with(registry: &CmdRegistry, line: &str) -> Result<Vec<Box<dyn Cmd>>, String> {
let line = line.trim();
if line.is_empty() || line.starts_with('#') || line.starts_with("//") {
return Ok(vec![]);
}
let mut commands = Vec::new();
for segment in split_on_dot(line) {
let segment = segment.trim();
if segment.is_empty() {
continue;
}
let tokens = tokenize(segment);
if tokens.is_empty() {
continue;
}
let word = &tokens[0];
let args = &tokens[1..];
commands.push(registry.parse(word, args)?);
}
Ok(commands)
groups(line.trim())?
.iter()
.map(|words| registry.parse(&words[0].text, &texts(&words[1..])))
.collect()
}
/// The command segments of a line, with comments and blank segments dropped.
pub fn segments(line: &str) -> Vec<&str> {
pub fn segments(line: &str) -> Result<Vec<&str>, String> {
let line = line.trim();
if line.is_empty() || line.starts_with('#') || line.starts_with("//") {
return Vec::new();
}
split_on_dot(line)
.into_iter()
.map(str::trim)
.filter(|s| !s.is_empty())
.collect()
Ok(groups(line)?
.iter()
.map(|words| &line[words[0].start..words[words.len() - 1].end])
.collect())
}
/// Build one segment's command against a LIVE context — `context/code/cli.md`.
@@ -51,7 +36,7 @@ pub fn parse_segment_interactive(
segment: &str,
ctx: &CmdContext,
) -> Result<Option<Box<dyn Cmd>>, String> {
let tokens = tokenize(segment.trim());
let tokens = texts(&lex(segment.trim())?);
let Some(word) = tokens.first() else {
return Ok(None);
};
@@ -67,65 +52,99 @@ pub fn parse_segment_interactive(
})
}
/// Split a line on ` . ` separators (dot must be a standalone word,
/// surrounded by whitespace or at line boundaries). Respects quoted strings.
fn split_on_dot(line: &str) -> Vec<&str> {
let mut segments = Vec::new();
let mut start = 0;
let mut in_quote = false;
let bytes = line.as_bytes();
for (i, c) in line.char_indices() {
match c {
'"' => in_quote = !in_quote,
'.' if !in_quote => {
let before_ws = i == 0 || bytes[i - 1].is_ascii_whitespace();
let after_ws = i + 1 >= bytes.len() || bytes[i + 1].is_ascii_whitespace();
if before_ws && after_ws {
segments.push(&line[start..i]);
start = i + 1;
}
}
_ => {}
}
}
segments.push(&line[start..]);
segments
/// One word of a line and the byte span it was read from. A `separator` is a
/// bare `.`; a quoted one is an argument like any other.
struct Word {
text: String,
start: usize,
end: usize,
separator: bool,
}
/// Tokenize a command segment into words, handling quoted strings.
fn tokenize(input: &str) -> Vec<String> {
let mut tokens = Vec::new();
let mut chars = input.chars().peekable();
fn texts(words: &[Word]) -> Vec<String> {
words.iter().map(|w| w.text.clone()).collect()
}
while let Some(&c) = chars.peek() {
/// The words of a trimmed line, split at separators, blank commands and
/// comment lines dropped.
fn groups(line: &str) -> Result<Vec<Vec<Word>>, String> {
if line.starts_with('#') || line.starts_with("//") {
return Ok(Vec::new());
}
let mut groups: Vec<Vec<Word>> = vec![Vec::new()];
for word in lex(line)? {
match word.separator {
true => groups.push(Vec::new()),
false => groups.last_mut().expect("never empty").push(word),
}
}
groups.retain(|g| !g.is_empty());
Ok(groups)
}
/// Split `line` into words. The quoting rules: `context/code/cli.md` §14.
fn lex(line: &str) -> Result<Vec<Word>, String> {
let mut words = Vec::new();
let mut chars = line.char_indices().peekable();
while let Some(&(start, c)) = chars.peek() {
if c.is_whitespace() {
chars.next();
continue;
}
if c == '"' {
chars.next(); // consume opening quote
let mut s = String::new();
for ch in chars.by_ref() {
if ch == '"' {
break;
}
s.push(ch);
let (text, bare) = match c {
'"' => (double_quoted(&mut chars), false),
_ => bare_word(&mut chars, line)?,
};
let end = chars.peek().map_or(line.len(), |&(i, _)| i);
words.push(Word {
separator: bare && text == ".",
text,
start,
end,
});
}
Ok(words)
}
/// `"…"` read verbatim: no escapes, and a `|` inside stays a `|`.
fn double_quoted(chars: &mut Peekable<CharIndices>) -> String {
chars.next();
chars
.by_ref()
.map(|(_, c)| c)
.take_while(|&c| c != '"')
.collect()
}
/// Text up to whitespace, each `|…|` piece unquoted in place; `bare` is false
/// once a piece was quoted, so a quoted `|.|` is an argument and not a separator.
fn bare_word(chars: &mut Peekable<CharIndices>, line: &str) -> Result<(String, bool), String> {
let mut text = String::new();
let mut bare = true;
while let Some((at, c)) = chars.next_if(|&(_, c)| !c.is_whitespace()) {
match c {
'|' => {
text.push_str(&pipe_body(chars, line, at)?);
bare = false;
}
tokens.push(s);
} else {
let mut s = String::new();
while let Some(&ch) = chars.peek() {
if ch.is_whitespace() {
break;
}
s.push(ch);
chars.next();
}
tokens.push(s);
_ => text.push(c),
}
}
tokens
Ok((text, bare))
}
/// The body of the `|…|` opened at `open`, escapes applied by the one shared
/// reader (gotcha 34).
fn pipe_body(chars: &mut Peekable<CharIndices>, line: &str, open: usize) -> Result<String, String> {
let mut body = String::new();
while let Some((_, c)) = chars.next() {
match c {
'|' => return Ok(unescape_pipe_body(&body)),
'\\' => body.extend(std::iter::once(c).chain(chars.next().map(|(_, e)| e))),
_ => body.push(c),
}
}
Err(format!("unterminated |…| in `{}`", &line[open..]))
}
#[cfg(test)]
@@ -302,11 +321,14 @@ mod tests {
#[test]
fn segments_drops_comments_and_blanks() {
assert_eq!(segments("# a comment"), Vec::<&str>::new());
assert_eq!(segments("// also a comment"), Vec::<&str>::new());
assert_eq!(segments(" "), Vec::<&str>::new());
assert_eq!(segments("a . . b"), vec!["a", "b"]);
assert_eq!(segments("move-selection 2 0"), vec!["move-selection 2 0"]);
assert_eq!(segments("# a comment"), Ok(Vec::<&str>::new()));
assert_eq!(segments("// also a comment"), Ok(Vec::<&str>::new()));
assert_eq!(segments(" "), Ok(Vec::<&str>::new()));
assert_eq!(segments("a . . b"), Ok(vec!["a", "b"]));
assert_eq!(
segments("move-selection 2 0"),
Ok(vec!["move-selection 2 0"])
);
}
/// Evidence: `context/test-evidence/cli.md`.
@@ -347,4 +369,105 @@ mod tests {
.expect_err("an unknown word is refused");
assert!(err.contains("no-such-command"), "got: {err}");
}
// ── pipe quoting and arity (improvise-31oe.3) ─────────────────────────
/// The one command `line` builds, as its `Debug` shows it.
fn built(line: &str) -> String {
let cmds = parse_line(line).unwrap_or_else(|e| panic!("`{line}` must parse: {e}"));
assert_eq!(cmds.len(), 1, "`{line}` is one command");
format!("{:?}", cmds[0])
}
/// Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_pipe_quoted_argument_is_one_argument_without_its_pipes() {
assert_eq!(
built("create-view |By area|"),
r#"CreateViewCmd(["By area"])"#
);
assert_eq!(
built("set-page |Sales Region| |East Coast|"),
r#"SetPageCmd(["Sales Region", "East Coast"])"#
);
}
/// Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_pipe_quoted_argument_reads_the_escapes_improv_does() {
assert_eq!(built(r"create-view |a\|b|"), r#"CreateViewCmd(["a|b"])"#);
assert_eq!(
built(r"create-view |back\\slash|"),
r#"CreateViewCmd(["back\\slash"])"#
);
assert_eq!(
built(r"create-view |two\nlines|"),
r#"CreateViewCmd(["two\nlines"])"#
);
}
/// A coordinate quotes each half, as a `.improv` data line does.
#[test]
fn a_coordinate_may_pipe_quote_either_half() {
assert_eq!(
built("set-cell 5 |Sales Region|/|East Coast| Measure/Revenue"),
r#"SetCellCmd(["5", "Sales Region/East Coast", "Measure/Revenue"])"#
);
}
#[test]
fn a_dot_inside_pipes_does_not_split_the_line() {
assert_eq!(built("create-view |a . b|"), r#"CreateViewCmd(["a . b"])"#);
assert_eq!(
segments("create-view |a . b|"),
Ok(vec!["create-view |a . b|"])
);
}
/// Double quotes stay verbatim: formula text carries its own pipes.
#[test]
fn a_double_quoted_argument_keeps_the_pipes_inside_it() {
assert_eq!(
built(r#"add-formula Measure "Net = |Gross Rev| - Cost""#),
r#"AddFormulaCmd(["Measure", "Net = |Gross Rev| - Cost"])"#
);
}
/// `context/code/cli.md` §14: only a BARE `.` separates commands. A quoted one
/// is an argument, so `|.|` and `"."` name a thing called `.`.
#[test]
fn a_quoted_dot_is_an_argument_not_a_separator() {
for line in ["create-view |.|", "create-view \".\""] {
let groups = groups(line).expect("lexes");
assert_eq!(groups.len(), 1, "`{line}` split into commands");
assert_eq!(texts(&groups[0]), ["create-view", "."], "`{line}`");
}
assert_eq!(groups("create-view A . create-view B").unwrap().len(), 2);
}
#[test]
fn an_unterminated_pipe_is_refused() {
let err = parse_line("create-view |By area").expect_err("an open pipe is refused");
assert!(
err.contains("|By area"),
"the refusal names the text: {err}"
);
}
/// Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_command_given_more_arguments_than_it_takes_is_refused() {
let err = parse_line("create-view By area").expect_err("two words for one argument");
assert!(err.starts_with("create-view:"), "names the command: {err}");
assert!(err.contains("\"area\""), "names the extra argument: {err}");
let err = parse_line("set-axis Type row sideways").expect_err("a third argument");
assert!(
err.contains("\"sideways\""),
"names the extra argument: {err}"
);
assert!(
parse_line("create-view |By area|").is_ok(),
"the control parses"
);
}
}
crates/improvise-command/src/ui/app.rs
+216 -31
View File
@@ -24,7 +24,7 @@ use crate::persistence;
use crate::ui::effect::{EffectKind, ModelChange, ViewTarget};
use crate::ui::project::Dirty;
use crate::ui::viewport::{
compute_col_widths, compute_row_header_width, compute_visible_cols, grid_width,
GridSummary, compute_row_header_width, compute_visible_cols, grid_summary, grid_width,
};
use crate::view::{GridLayout, View};
use crate::workbook::Workbook;
@@ -50,17 +50,17 @@ fn view_shape(model: &crate::model::types::Model, view: &View) -> u64 {
.filter(|item| !hidden.is_some_and(|h| h.contains(item)))
.collect();
drawn.hash(&mut h);
// Off the drawn items, not `Category::groups`: `add_item_in_group` does not
// populate that, and a group with nothing drawn under it is not shape.
// Off the drawn items' chains, not `Category::groups`: a group with nothing
// drawn under it is not shape, and a fold at any depth is.
let mut collapsed: Vec<&str> = model
.category(cat)
.into_iter()
.flat_map(|c| drawn.iter().filter_map(|item| c.items.get(item)))
.filter_map(|item| item.group.as_deref())
.filter(|group| {
view.collapsed_groups
.get(cat)
.is_some_and(|c| c.contains(*group))
.flat_map(|c| drawn.iter().map(move |item| c.item_chain(item)))
.flat_map(|chain| {
(0..chain.len())
.filter(|&d| view.is_group_collapsed(cat, &chain[..=d]))
.map(|d| chain[d])
.collect::<Vec<_>>()
})
.collect();
collapsed.sort_unstable();
@@ -77,15 +77,14 @@ fn view_shape(model: &crate::model::types::Model, view: &View) -> u64 {
fn view_frame(model: &crate::model::types::Model, view: &View) -> u64 {
// `with_frozen_records`, not `new`: `new` does NOT apply `prune_empty`, and a
// pruned row is exactly the kind of reframing this observation exists for.
frame_of(
&crate::view::layout::GridLayout::with_frozen_records(model, view, None),
view,
)
frame_of(&crate::view::layout::GridLayout::with_frozen_records(
model, view, None,
))
}
/// The same hash off a layout already built. The batch's opening basis is free
/// this way — `context/code/command.md` (improvise-mcq4).
fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
fn frame_of(layout: &crate::view::layout::GridLayout) -> u64 {
use std::hash::{Hash, Hasher};
let mut h = std::collections::hash_map::DefaultHasher::new();
layout.row_cats.hash(&mut h);
@@ -95,14 +94,17 @@ fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
for entry in entries {
match entry {
crate::view::layout::AxisEntry::GroupHeader {
depth,
cat_name,
group_name,
collapsed,
..
} => {
0u8.hash(&mut h);
depth.hash(&mut h);
cat_name.hash(&mut h);
group_name.hash(&mut h);
view.is_group_collapsed(cat_name, group_name).hash(&mut h);
collapsed.hash(&mut h);
}
crate::view::layout::AxisEntry::DataItem { tuple, collapsed } => {
1u8.hash(&mut h);
@@ -111,9 +113,10 @@ fn frame_of(layout: &crate::view::layout::GridLayout, view: &View) -> u64 {
}
// Drawn as nothing, but it ends a group's run (improvise-gd83),
// so where it sits is what a row's group attribution depends on.
crate::view::layout::AxisEntry::GroupEnd { level } => {
crate::view::layout::AxisEntry::GroupEnd { level, depth } => {
2u8.hash(&mut h);
level.hash(&mut h);
depth.hash(&mut h);
}
}
}
@@ -204,7 +207,16 @@ pub(crate) fn document_config(workbook: &Workbook) -> u64 {
axis.hash(&mut h);
}
sorted_pairs(&view.hidden_items).hash(&mut h);
sorted_pairs(&view.collapsed_groups).hash(&mut h);
let mut depths: Vec<_> = view.expand_depth.iter().collect();
depths.sort_unstable();
depths.hash(&mut h);
let mut folds: Vec<_> = view
.group_folds
.iter()
.flat_map(|(cat, groups)| groups.iter().map(move |(g, fold)| (cat, g, fold)))
.collect();
folds.sort_unstable();
folds.hash(&mut h);
let mut pages: Vec<(&String, &String)> = view.page_selections.iter().collect();
pages.sort_unstable();
pages.hash(&mut h);
@@ -422,6 +434,20 @@ impl Default for ModelState {
}
}
/// What a layout was built from other than the model.
#[derive(Debug, Default, PartialEq, Eq)]
struct LayoutBasis {
view: String,
/// The address of the frozen records, never dereferenced.
drill: Option<usize>,
}
/// What the keypress path measures of a built layout.
pub(crate) struct GridGeometry {
pub summary: GridSummary,
pub row_header_width: u16,
}
/// Top-level application state: `model_state`, `view_state`, and a residue of
/// fields tagged below. The five residue tags are in `context/code/command.md`.
pub struct App {
@@ -441,8 +467,21 @@ pub struct App {
pub term_width: u16,
pub term_height: u16,
/// **Derived cache**: a pure function of the slices, rebuilt by
/// `rebuild_layout()`. Owned by `App` because the rebuild is the host's job.
/// `rebuild_layout()`; a writer that bypasses effects must call it (felo.13).
pub layout: GridLayout,
/// How many times `rebuild_layout` has run: what a keypress that changes
/// neither the model nor the view must leave alone (improvise-felo.13).
layout_builds: usize,
/// How many times the grid's column widths were measured, for the same reason.
width_passes: std::cell::Cell<usize>,
/// The widths of the layout above, measured on first use and dropped with it.
geometry: std::cell::OnceCell<GridGeometry>,
/// What the layout above was built from besides the model: the view it drew
/// and the drill snapshot it froze. `layout_current` compares it.
layout_basis: LayoutBasis,
/// The model changed since the layout was built. Set by the one writer that
/// does not rebuild at once.
layout_stale: bool,
/// **Transient**: set during `apply_effects` to skip the rest of the batch,
/// reset at the start of every call. Reach it through `AbortChain`.
pub abort_effects: bool,
@@ -541,6 +580,11 @@ impl App {
term_width: 80,
term_height: 24,
layout,
layout_builds: 0,
width_passes: std::cell::Cell::new(0),
geometry: std::cell::OnceCell::new(),
layout_basis: LayoutBasis::default(),
layout_stale: false,
abort_effects: false,
batch_aborted: false,
batch_refused: false,
@@ -554,6 +598,7 @@ impl App {
keymap_set: KeymapSet::default_keymaps(),
storage,
};
app.layout_basis = app.current_layout_basis();
app.settle_normal_mode();
app
}
@@ -598,6 +643,7 @@ impl App {
self.view_state.view_back_stack.clear();
self.view_state.view_forward_stack.clear();
self.model_state.workbook = workbook;
self.layout_stale = true;
// Every client's mirror is scoped to the outgoing document too, and an
// effect cannot reach the session list. The generation is how it hears.
self.document_generation += 1;
@@ -862,7 +908,7 @@ impl App {
fn opening_view_frame(&self) -> u64 {
let (_, view) = self.resolve_view(self.view_state.active_view.as_str());
match self.drill_state.is_none() {
true => frame_of(&self.layout, view),
true => frame_of(&self.layout),
// A drill froze the record list, so the held layout is not what
// `view_frame` would build and the two must not be compared.
false => view_frame(&self.model_state.workbook.model, view),
@@ -888,6 +934,42 @@ impl App {
let frozen = self.drill_state.as_ref().map(|s| Rc::clone(&s.records));
self.layout =
GridLayout::with_frozen_records(&self.model_state.workbook.model, view, frozen);
self.layout_builds += 1;
self.layout_stale = false;
self.layout_basis = self.current_layout_basis();
self.geometry = std::cell::OnceCell::new();
}
/// Rebuild only if the model, the view drawn or the drill snapshot moved; a
/// cursor key moves none of them (layout-cost.md).
fn refresh_layout(&mut self) {
if self.layout_stale || self.layout_basis != self.current_layout_basis() {
self.rebuild_layout();
}
}
fn current_layout_basis(&self) -> LayoutBasis {
LayoutBasis {
view: self.view_name().to_string(),
drill: self
.drill_state
.as_ref()
.map(|d| Rc::as_ptr(&d.records) as usize),
}
}
/// The measurements a keypress needs of the grid: column widths and the row
/// header. Counted, so a test can see whether a keypress paid for them.
pub(crate) fn grid_geometry(&self) -> &GridGeometry {
self.geometry.get_or_init(|| {
self.width_passes.set(self.width_passes.get() + 1);
let model = &self.model_state.workbook.model;
let plan = FormatPlan::of(model, self.view());
GridGeometry {
summary: grid_summary(model, &self.layout, &plan),
row_header_width: compute_row_header_width(&self.layout),
}
})
}
/// Editable text for the cell under the cursor: staged drill edit, else the
@@ -968,12 +1050,10 @@ impl App {
display_value: self.display_value_at_cursor(),
visible_rows: (self.term_height as usize).saturating_sub(8),
visible_cols: {
let model = &self.model_state.workbook.model;
let col_widths = compute_col_widths(model, layout, &FormatPlan::of(model, view));
let row_header_width = compute_row_header_width(layout);
let geometry = self.grid_geometry();
compute_visible_cols(
&col_widths,
row_header_width,
&geometry.summary.col_widths,
geometry.row_header_width,
self.grid_width(),
cursor.col_offset,
)
@@ -1104,7 +1184,8 @@ impl App {
// is read only behind the epoch gate — `context/code/command.md`.
let reshaped = self.last_reshaped_views.clone();
self.last_reframed_views.extend(reshaped);
self.rebuild_layout();
self.layout_stale |= touches_model;
self.refresh_layout();
// After the rebuild, or a batch that crosses into or out of records mode
// settles against the grid it started on — `context/code/command.md`.
self.settle_normal_mode();
@@ -1184,7 +1265,7 @@ impl App {
/// Resolve one keypress through the keymap and apply what it produces. The
/// projection half is factored out; `ui::reduce` is the named door (improvise-gxi).
pub fn dispatch_key(&mut self, key: KeyEvent) -> Dirty {
self.rebuild_layout();
self.refresh_layout();
// An armed prefix key takes priority. Resolve it back to its
// sub-keymap through the keymap set (improvise-jc8).
@@ -1224,7 +1305,12 @@ impl App {
}
pub fn start_import_wizard(&mut self, json: serde_json::Value) {
self.wizard = Some(ImportWizard::new(json));
self.start_import(crate::import::wizard::ImportPipeline::new(json));
}
/// Open the wizard over a pipeline the host built, sources and all.
pub fn start_import(&mut self, pipeline: crate::import::wizard::ImportPipeline) {
self.wizard = Some(ImportWizard::from_pipeline(pipeline));
self.view_state.mode = AppMode::ImportWizard;
}
@@ -1577,6 +1663,32 @@ mod tests {
app
}
/// improvise-felo.13: `StartDrill` is `ViewOnly`, so a batch of it alone moves
/// the layout with no `Model` effect to mark it stale. Evidence: layout-cost.md.
#[test]
fn a_drill_snapshot_replaced_alone_rebuilds_the_layout() {
let mut app = records_app_for_drill();
assert_eq!(app.layout.row_count(), 3);
let rows: Vec<_> = app.layout.records.as_ref().unwrap().to_vec();
let start = |app: &mut App, n: usize| {
let snapshot = rows.iter().take(n).cloned().collect();
app.apply_effects(vec![Box::new(crate::ui::effect::StartDrill(snapshot))]);
};
start(&mut app, 1);
assert_eq!(
app.layout.row_count(),
1,
"the layout draws the frozen snapshot"
);
let builds = app.layout_builds;
start(&mut app, 2);
assert!(
app.layout_builds > builds,
"no rebuild for a second drill snapshot"
);
assert_eq!(app.layout.row_count(), 2);
}
/// The opening basis may be read off `App::layout` only when no drill froze
/// it. Evidence: `context/test-evidence/command.md`.
#[test]
@@ -1881,6 +1993,71 @@ mod tests {
App::new(wb, None)
}
/// improvise-felo.13: context/test-evidence/layout-cost.md.
#[test]
fn a_cursor_key_rebuilds_neither_the_layout_nor_the_widths() {
let mut app = two_col_model();
let down = || KeyEvent::new(KeyCode::Down, KeyModifiers::NONE);
app.dispatch_key(down());
let (builds, widths) = (app.layout_builds, app.width_passes.get());
app.dispatch_key(down());
app.dispatch_key(down());
assert_eq!(app.cursor().selected.0, 2, "the keys moved the cursor");
assert_eq!(app.layout_builds, builds, "layout rebuilt on a cursor key");
assert_eq!(app.width_passes.get(), widths, "widths remeasured");
}
/// improvise-felo.13: the control for the test above, and the guard on
/// invalidation: an edit must rebuild, and the widths must follow the value.
#[test]
fn an_edit_rebuilds_and_remeasures_so_the_widths_follow_the_value() {
use crate::ui::effect::SetCell;
let mut app = two_col_model();
let down = || KeyEvent::new(KeyCode::Down, KeyModifiers::NONE);
app.dispatch_key(down());
let narrow = app.grid_geometry().summary.col_widths.clone();
let (builds, widths) = (app.layout_builds, app.width_passes.get());
let key = CellKey::new(vec![("Row".into(), "A".into()), ("Col".into(), "X".into())]);
let long = CellValue::Text("a value far wider than a column".into());
app.apply_effects(vec![Box::new(SetCell(key, long))]);
assert!(
app.layout_builds > builds,
"an edit must rebuild the layout"
);
app.dispatch_key(down());
assert!(app.width_passes.get() > widths, "an edit must remeasure");
let model = &app.model_state.workbook.model;
let fresh = crate::ui::viewport::compute_col_widths(
model,
&app.layout,
&FormatPlan::of(model, app.view()),
);
assert_eq!(
app.grid_geometry().summary.col_widths,
fresh,
"stale widths"
);
assert_ne!(
fresh, narrow,
"the edit widened a column, or this proves nothing"
);
}
/// improvise-felo.13: a `ViewOnly` effect that changes which view is drawn
/// must still rebuild, so skipping the rebuild is not keyed on `EffectKind` alone.
#[test]
fn switching_view_rebuilds_though_the_effect_is_view_only() {
let mut app = two_col_model();
app.model_state.workbook.create_view("Second");
app.rebuild_layout();
let builds = app.layout_builds;
app.apply_effects(vec![Box::new(crate::ui::effect::SwitchView(
"Second".into(),
))]);
assert_eq!(app.view_state.active_view.as_str(), "Second");
assert!(app.layout_builds > builds, "the layout still draws Default");
}
/// improvise-htl6. Evidence: `context/test-evidence/command.md`.
#[test]
fn the_view_history_walks_both_ways_from_the_keyboard() {
@@ -2117,6 +2294,7 @@ mod tests {
.unwrap()
.add_item(format!("R{i}"));
}
app.rebuild_layout();
app.term_height = 28; // ~20 visible rows → delta = 15
app.cursor_mut().selected = (0, 0);
app.handle_key(KeyEvent::new(KeyCode::PageDown, KeyModifiers::NONE))
@@ -2141,6 +2319,7 @@ mod tests {
.unwrap()
.add_item(format!("R{i}"));
}
app.rebuild_layout();
app.term_height = 28;
app.cursor_mut().selected = (20, 0);
app.handle_key(KeyEvent::new(KeyCode::PageUp, KeyModifiers::NONE))
@@ -2160,6 +2339,7 @@ mod tests {
.unwrap()
.add_item(format!("R{i}"));
}
app.rebuild_layout();
app.term_height = 13; // ~5 visible rows
app.cursor_mut().selected = (0, 0);
// G jumps to last row (row 12)
@@ -2187,6 +2367,7 @@ mod tests {
.unwrap()
.add_item(format!("R{i}"));
}
app.rebuild_layout();
app.term_height = 13; // ~5 visible rows
app.cursor_mut().selected = (0, 0);
// Ctrl+d scrolls by 5 rows
@@ -3281,7 +3462,9 @@ mod tests {
/// when any segment refused, which is what its exit code reads.
fn run_line(app: &mut App, line: &str) -> bool {
let mut refused = false;
for segment in crate::command::parse::segments(line) {
let segments = crate::command::parse::segments(line)
.unwrap_or_else(|e| panic!("`{line}` must lex: {e}"));
for segment in segments {
let effects = {
let ctx = app.cmd_context(KeyCode::Null, KeyModifiers::NONE);
crate::command::parse::parse_segment_interactive(segment, &ctx)
@@ -3314,7 +3497,7 @@ mod tests {
/// Eleven regular categories: one short of the limit.
fn one_category_short_of_the_limit() -> App {
let mut wb = Workbook::new("T");
for n in 0..11 {
for n in 0..crate::model::MAX_CATEGORIES - 1 {
wb.add_category(format!("C{n}")).unwrap();
}
App::new(wb, None)
@@ -3425,8 +3608,9 @@ mod tests {
for (month, quarter) in [("Jan", "Q1"), ("Feb", "Q1"), ("Apr", "Q2"), ("May", "Q2")] {
wb.add_item_in_group("Month", month, quarter).unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
app
@@ -3464,12 +3648,13 @@ mod tests {
fn every_refusal_site_refuses_its_failure_and_only_its_failure() {
type Setup = fn() -> App;
type Drive = fn(&mut App) -> bool;
let limit = format!("Maximum of {} categories", crate::model::MAX_CATEGORIES);
let cases: Vec<(&str, Setup, Drive, &str, Setup, Drive)> = vec![
(
"add-category past the limit",
one_category_short_of_the_limit,
|app| run_line(app, "add-category A . add-category B"),
"Maximum of 12 categories",
&limit,
one_category_short_of_the_limit,
|app| run_line(app, "add-category A"),
),
crates/improvise-command/src/ui/cat_tree.rs
+1
View File
@@ -137,6 +137,7 @@ mod tests {
fn cat_name_works_for_both_variants() {
let header = CatTreeEntry::Category {
name: "Region".into(),
display: "Region".into(),
item_count: 2,
expanded: false,
axis: crate::view::Axis::None,
crates/improvise-command/src/ui/effect.rs
+95 -39
View File
@@ -550,8 +550,14 @@ pub struct ToggleGroup {
}
impl Effect for ToggleGroup {
fn apply(&self, app: &mut App) {
let chain = app
.model_state
.workbook
.model
.group_chain(&self.category, &self.group);
let chain: Vec<&str> = chain.iter().map(String::as_str).collect();
on_view(app, &self.view, |v| {
v.toggle_group_collapse(&self.category, &self.group);
v.toggle_group_collapse(&self.category, &chain);
});
}
fn kind(&self) -> EffectKind {
@@ -1047,6 +1053,7 @@ pub enum WizardOp {
ToggleProposal,
CycleProposalKind,
ToggleDateComponent,
ToggleDateShape,
StartFormulaEdit,
DeleteFormula,
ConfirmFormula,
@@ -1072,6 +1079,7 @@ impl WizardOp {
Self::ToggleProposal => wizard.toggle_proposal(),
Self::CycleProposalKind => wizard.cycle_proposal_kind(),
Self::ToggleDateComponent => wizard.toggle_date_component(),
Self::ToggleDateShape => wizard.toggle_date_shape(),
Self::StartFormulaEdit => wizard.start_formula_edit(),
Self::DeleteFormula => wizard.delete_formula(),
Self::ConfirmFormula => wizard.confirm_formula(),
@@ -1239,7 +1247,7 @@ pub struct ImportJsonHeadless {
}
impl Effect for ImportJsonHeadless {
fn apply(&self, app: &mut App) {
use crate::import::analyzer::{FieldKind, analyze_records};
use crate::import::analyzer::FieldKind;
use crate::import::wizard::{DuplicatePolicy, ImportPipeline};
let is_csv = self
@@ -1247,10 +1255,10 @@ impl Effect for ImportJsonHeadless {
.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("csv"));
// Parse the file exactly once: `records` feeds analysis, `raw` is reused
// as the pipeline's value (improvise-oaq). CSV warnings go to the status.
// Parse the file exactly once, into the pipeline that reads it
// (improvise-oaq, improvise-8jk6.4). CSV warnings go to the status.
let mut warnings: Vec<String> = vec![];
let (records, raw) = if is_csv {
let mut pipeline = if is_csv {
let bytes = match app.storage.read(&self.path) {
Ok(b) => b,
Err(e) => {
@@ -1261,8 +1269,7 @@ impl Effect for ImportJsonHeadless {
match crate::import::csv_parser::parse_csv_bytes_with_warnings(&bytes) {
Ok((recs, w)) => {
warnings = w;
let raw = serde_json::Value::Array(recs.clone());
(recs, raw)
ImportPipeline::from_csv(recs)
}
Err(e) => {
refuse_and_stop(app, format!("CSV error: {e}"));
@@ -1285,37 +1292,28 @@ impl Effect for ImportJsonHeadless {
}
};
match json_import_records(&value, self.array_path.as_deref()) {
Ok(recs) => (recs, value),
Ok(recs) => {
let mut pipeline = ImportPipeline::new(value);
pipeline.selected_path = self.array_path.clone().unwrap_or_default();
pipeline.records = recs;
pipeline.analyze();
pipeline
}
Err(msg) => {
refuse_and_stop(app, msg);
return;
}
}
};
let proposals = analyze_records(&records);
let pipeline = ImportPipeline {
raw,
array_paths: vec![],
selected_path: self.array_path.as_deref().unwrap_or("").to_string(),
records,
proposals: proposals
.into_iter()
.map(|mut p| {
p.accepted = p.kind != FieldKind::Label;
p
})
.collect(),
model_name: self
.model_name
.as_deref()
.unwrap_or("Imported Model")
.to_string(),
formulas: vec![],
// Headless: nobody to ask, so a collision refuses (improvise-cgqt).
on_duplicate: DuplicatePolicy::Refuse,
};
pipeline.model_name = self
.model_name
.clone()
.unwrap_or_else(|| "Imported Model".to_string());
// Headless: nobody to ask, so a collision refuses (improvise-cgqt).
pipeline.on_duplicate = DuplicatePolicy::Refuse;
for p in &mut pipeline.proposals {
p.accepted = p.kind != FieldKind::Label;
}
match pipeline.build_model_with_warnings() {
Ok((new_workbook, build_warnings)) => {
@@ -1372,6 +1370,63 @@ impl Effect for SetPanelCursor {
// ── Chain control ────────────────────────────────────────────────────────────
/// What a command's effects name that the document must have.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Needs {
Category(String),
Item(String, String),
Group(String, String),
View(String),
}
/// Why the document cannot satisfy `needs` in the active view, if it cannot.
fn unknown_name(app: &App, needs: &Needs) -> Option<String> {
let workbook = &app.model_state.workbook;
let (category, rest) = match needs {
Needs::View(name) => {
return (!workbook.views.contains_key(name))
.then(|| format!("No view named \"{name}\""));
}
Needs::Category(c) => (c, None),
Needs::Item(c, i) => (c, Some((i, false))),
Needs::Group(c, g) => (c, Some((g, true))),
};
let view = workbook.views.get(app.view_name())?;
if view.try_axis_of(category).is_none() {
return Some(format!("Unknown category '{category}'"));
}
let (name, is_group) = rest?;
if !is_group {
// `effective_item_names`, so a formula target is a known `_Measure` item.
let known = workbook
.model
.effective_item_names(category)
.iter()
.any(|i| i == name);
return (!known).then(|| format!("Unknown item '{name}' in '{category}'"));
}
// An item's own group registers no `Group`; a parent group has no items.
let known = workbook.model.category(category).is_some_and(|c| {
!c.items_in_group(name).is_empty() || c.groups.iter().any(|g| g.name == *name)
});
(!known).then(|| format!("Unknown group '{name}' in '{category}'"))
}
/// Refuse and stop unless the document has what the next effects name, asked at
/// apply time so a `:` line sees its earlier segments (`cli.md` §15).
#[derive(Debug)]
pub struct RequireKnown(pub Needs);
impl Effect for RequireKnown {
fn apply(&self, app: &mut App) {
if let Some(why) = unknown_name(app, &self.0) {
refuse_and_stop(app, why);
}
}
fn kind(&self) -> EffectKind {
EffectKind::ViewOnly
}
}
/// Skip the remaining effects in this `apply_effects` batch, for when a
/// sequence's premise no longer holds. The flag resets per dispatch.
#[derive(Debug)]
@@ -1926,8 +1981,8 @@ mod tests {
#[test]
fn add_category_past_the_limit_reports_the_failure() {
let mut app = test_app();
// test_app() holds 2 regular categories; fill up to the limit of 12.
for i in 0..10 {
// test_app() holds 2 regular categories; fill up to the limit.
for i in 0..crate::model::MAX_CATEGORIES - 2 {
AddCategory(format!("Cat{i}")).apply(&mut app);
}
app.view_state.status_msg = "Added category \"Overflow\"".to_string();
@@ -1938,7 +1993,7 @@ mod tests {
.model
.category("Overflow")
.is_none(),
"the 13th category must not be added"
"the category past the limit must not be added"
);
assert!(
app.view_state.status_msg.contains("Maximum"),
@@ -3875,10 +3930,11 @@ mod tests {
}
.apply(&mut app);
assert!(
app.model_state
!app.model_state
.workbook
.active_view()
.is_group_collapsed("Type", "MyGroup")
.is_group_collapsed("Type", &["MyGroup"]),
"a group starts collapsed, so the first toggle opens it"
);
ToggleGroup {
view: ViewTarget::Active,
@@ -3887,10 +3943,10 @@ mod tests {
}
.apply(&mut app);
assert!(
!app.model_state
app.model_state
.workbook
.active_view()
.is_group_collapsed("Type", "MyGroup")
.is_group_collapsed("Type", &["MyGroup"])
);
}
crates/improvise-command/src/ui/host.rs
+138 -9
View File
@@ -5,7 +5,7 @@ use crate::ui::app::App;
use crate::ui::effect;
use crate::ui::project::{Applied, Broadcast, Dirty, SessionId, Sessions};
use crate::ui::reduce::{self, Checked, CheckedBatch};
use improvise_protocol::{Command, Rejection};
use improvise_protocol::{AppMode, Command, Rejection};
use improvise_protocol::{ClientFrame, ServerFrame};
@@ -87,6 +87,8 @@ impl Host {
// resolution diverges (improvise-bpq2, gotcha 47).
ClientFrame::Key { .. } => Vec::new(),
ClientFrame::Save => self.save(id),
ClientFrame::DownloadDocument { filename } => self.download_document(id, filename),
ClientFrame::DownloadCsv { filename } => self.download_csv(id, filename),
}
}
@@ -113,6 +115,67 @@ impl Host {
}
}
/// Serialize the document to `.improv` format and return the bytes for download
/// after closing the prompt in the session mirror (improvise-2mqz).
fn download_document(&mut self, id: SessionId, filename: String) -> Vec<Addressed> {
let Some(mut answer) = self.close_download_prompt(id, "save") else {
return Vec::new();
};
let bytes =
improvise_io::persistence::format_md(&self.app.model_state.workbook).into_bytes();
answer.push(Addressed {
to: id,
msg: ServerFrame::DocumentBytes { filename, bytes },
});
answer
}
/// Render the requesting session's view as downloadable CSV bytes.
/// Close its prompt mirror first; reject only if rendering fails (improvise-2mqz).
fn download_csv(&mut self, id: SessionId, filename: String) -> Vec<Addressed> {
let Some(view_name) = self
.sessions
.get(id)
.map(|session| session.view().active_view.clone())
else {
return Vec::new();
};
let Some(mut answer) = self.close_download_prompt(id, "export") else {
return Vec::new();
};
match improvise_io::persistence::render_csv(&self.app.model_state.workbook, &view_name) {
Ok(bytes) => answer.push(Addressed {
to: id,
msg: ServerFrame::CsvBytes { filename, bytes },
}),
Err(_) => answer.push(Addressed {
to: id,
msg: ServerFrame::Rejected(Rejection::ExportFailed),
}),
}
answer
}
/// The browser folded these before sending the download request; mirror
/// that fold so a later projection cannot restore the prompt.
fn close_download_prompt(
&mut self,
id: SessionId,
buffer: &'static str,
) -> Option<Vec<Addressed>> {
self.sessions.get(id)?;
Some(self.commands(
id,
&[
Command::SetBuffer {
name: buffer.into(),
value: String::new(),
},
Command::ChangeMode(AppMode::Normal),
],
))
}
/// Fan out whatever moved, for a host that changed the document outside
/// `handle` — a worker's own persist is the caller (improvise-z754).
pub fn project_all(&mut self) -> Vec<Addressed> {
@@ -435,9 +498,12 @@ mod tests {
}
}
}
// Neither answer carries state: one applied nothing, and what
// the other applied arrived as projections before it.
ServerFrame::Rejected(_) | ServerFrame::Accepted => {}
// These carry no foldable state: downloads are consumed by the
// browser, while other results arrived as projections or did nothing.
ServerFrame::Rejected(_)
| ServerFrame::Accepted
| ServerFrame::DocumentBytes { .. }
| ServerFrame::CsvBytes { .. } => {}
}
}
grid
@@ -1177,7 +1243,7 @@ mod tests {
let mut host = host();
// Fill to MAX_CATEGORIES, so the frame's second command is refused by
// `Model::add_category` — a DUPLICATE add is idempotent and vacuous.
for n in 0..11 {
for n in 0..crate::model::MAX_CATEGORIES - 1 {
host.app_mut()
.model_state
.workbook
@@ -1547,11 +1613,12 @@ mod tests {
wb.model.set_cell(month_cell(month, measure), value);
}
}
wb.active_view_mut().set_expand_depth("Month", 1);
let profit = crate::formula::parse_formula("Profit = Revenue - Cost", "_Measure");
wb.model.add_formula(profit.unwrap());
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().toggle_group_collapse("Month", "Q2");
wb.active_view_mut().toggle_group_collapse("Month", &["Q2"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
assert!(
@@ -1733,13 +1800,14 @@ mod tests {
for measure in ["Revenue", "Cost"] {
wb.add_item_in_group("_Measure", measure, "Money").unwrap();
}
wb.active_view_mut().set_expand_depth("_Measure", 1);
wb.add_item("_Measure", "Units").unwrap();
let jan = month_cell("Jan", "Revenue");
wb.model.set_cell(jan, CellValue::Number(10.0));
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut()
.toggle_group_collapse("_Measure", "Money");
.toggle_group_collapse("_Measure", &["Money"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
let target = month_cell("Feb", "Units");
@@ -1779,12 +1847,13 @@ mod tests {
wb.model.set_cell(key, CellValue::Number(10.0));
}
}
wb.active_view_mut().set_expand_depth("_Measure", 1);
wb.add_item("_Measure", "Units").unwrap();
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
if collapsed {
wb.active_view_mut()
.toggle_group_collapse("_Measure", "Money");
.toggle_group_collapse("_Measure", &["Money"]);
}
let mut app = App::new(wb, None);
app.rebuild_layout();
@@ -1808,6 +1877,7 @@ mod tests {
for measure in ["Revenue", "Cost"] {
wb.add_item_in_group("_Measure", measure, "Money").unwrap();
}
wb.active_view_mut().set_expand_depth("_Measure", 1);
let measures = wb.model.category_mut("_Measure").unwrap();
measures.add_group(crate::model::category::Group::new("Money").with_parent("Totals"));
wb.model
@@ -2709,6 +2779,64 @@ mod tests {
);
}
/// improvise-2mqz: view selection belongs to a session, so an export must
/// not silently use the document's default view or another client's view.
#[test]
fn a_csv_download_exports_the_requesting_sessions_active_view() {
let mut host = host();
let second = host.app.model_state.workbook.create_view("Second");
second.set_axis("Region", Axis::Row);
second.set_axis("_Measure", Axis::Column);
second.hide_item("Region", "South");
let (requester, _) = host.connect(10, 10);
let (bystander, _) = host.connect(10, 10);
host.handle(
requester,
ClientFrame::command(Command::SwitchView("Second".into())),
);
assert_eq!(
host.sessions().get(requester).unwrap().view().active_view,
"Second",
"setup: the requester must be on its session-local view"
);
assert_eq!(
host.sessions().get(bystander).unwrap().view().active_view,
"Default",
"setup: the second session must remain on the default view"
);
assert_eq!(
host.app().model_state.workbook.active_view,
"Default",
"setup: the document's active view must differ from the requester"
);
let answer = host.handle(
requester,
ClientFrame::DownloadCsv {
filename: "second.csv".into(),
},
);
let expected =
improvise_io::persistence::render_csv(&host.app().model_state.workbook, "Second")
.unwrap();
let default =
improvise_io::persistence::render_csv(&host.app().model_state.workbook, "Default")
.unwrap();
assert_ne!(expected, default, "setup: the two exports must differ");
assert_eq!(
to(&answer, requester),
vec![ServerFrame::CsvBytes {
filename: "second.csv".into(),
bytes: expected,
}]
);
assert!(
to(&answer, bystander).is_empty(),
"the download is owed only to its requester: {answer:?}"
);
}
// ── disconnect ──────────────────────────────────────────────────────
/// AC 4: a dropped client stops being projected to, and a late frame
@@ -2979,7 +3107,8 @@ mod tests {
let fresh = || {
let mut wb = workbook();
wb.create_view("Scratch");
(0..12).for_each(|n| drop(wb.add_category(format!("Filler{n}"))));
(0..crate::model::MAX_CATEGORIES)
.for_each(|n| drop(wb.add_category(format!("Filler{n}"))));
let mut app = App::new(wb, None);
app.storage = Box::new(super::import_fixture_storage());
app.rebuild_layout();
crates/improvise-command/src/ui/project.rs
+211 -33
View File
@@ -11,7 +11,7 @@ use improvise_core::model::record::{AggregateHash, RecordHash};
use improvise_core::view::layout::{AxisEntry, GridLayout, RecordCol};
use improvise_core::view::{Axis, View};
use improvise_core::workbook::Workbook;
use improvise_io::import::analyzer::{DateComponent, FieldKind};
use improvise_io::import::analyzer::{DateComponent, DateShape, FieldKind};
use improvise_io::import::wizard::{DuplicatePolicy, ImportWizard, WizardStep};
use improvise_protocol::{
AxisFrame, Basis, BasisCache, CategoryRow, CategoryTreeCache, CategoryTreeUpdate, CellDisplay,
@@ -25,7 +25,7 @@ use improvise_protocol::{
use crate::ui::app::App;
use crate::ui::effect::{EffectKind, ModelChange};
use crate::ui::reduce::{Checked, apply_command};
use crate::ui::viewport::{GridBody, grid_body};
use crate::ui::viewport::{GridBody, grid_body, grid_window};
/// Which widgets a batch of effects could have changed. A gate, not a description:
/// over-reporting costs work, under-reporting is a correctness bug (gotcha 35).
@@ -142,13 +142,23 @@ pub fn grid_in(app: &App, view_name: &str, viewport: Viewport) -> GridCache {
let plan = FormatPlan::of(model, view);
let rows = viewport.rows_in(layout.row_count());
let cols = viewport.cols_in(layout.col_count());
// ONE walk: the widths, the totals and this window's values all come off the
// same evaluation — `context/code/command.md` (improvise-jmjz).
let body = grid_body(model, layout, &plan, rows.clone(), cols.clone());
// One walk gives widths, totals and window (improvise-jmjz); the host's layout
// keeps the first two until rebuilt, so its window costs its own cells.
let hosted = elsewhere.is_none() && derived.is_none();
let body = match hosted {
true => grid_window(
model,
layout,
&app.grid_geometry().summary,
rows.clone(),
cols.clone(),
),
false => grid_body(model, layout, &plan, rows.clone(), cols.clone()),
};
GridCache {
row_frame: axis_frame(&layout.row_cats, &layout.row_items, view, rows.clone()),
col_frame: axis_frame(&layout.col_cats, &layout.col_items, view, cols.clone()),
row_frame: axis_frame(&layout.row_cats, &layout.row_items, rows.clone()),
col_frame: axis_frame(&layout.col_cats, &layout.col_items, cols.clone()),
col_widths: cols
.clone()
.map(|c| body.col_widths.get(c).copied().unwrap_or(0))
@@ -212,12 +222,7 @@ pub(crate) fn rederived_for(
/// One axis's frame, windowed to the data indices `window` names. Three edge rules,
/// all about group headers — see `context/code/command.md`.
fn axis_frame(
cats: &[String],
items: &[AxisEntry],
view: &View,
window: std::ops::Range<usize>,
) -> AxisFrame {
fn axis_frame(cats: &[String], items: &[AxisEntry], window: std::ops::Range<usize>) -> AxisFrame {
let total = items
.iter()
.filter(|e| matches!(e, AxisEntry::DataItem { .. }))
@@ -225,40 +230,48 @@ fn axis_frame(
let mut data_idx = 0usize;
let mut entries: Vec<FrameEntry> = Vec::new();
let mut pending: Vec<FrameEntry> = Vec::new();
// Before the window opens, only the runs still open matter, one per level:
// O(levels), however many groups a deep scroll passed (improvise-e265).
let mut carried: Vec<Option<FrameEntry>> = Vec::new();
// Before the window opens, only the runs still open matter, keyed `(level,
// depth)`: O(depth), however many groups a deep scroll passed (e265, felo.6).
let mut carried: Vec<((usize, usize), FrameEntry)> = Vec::new();
let close = |carried: &mut Vec<((usize, usize), FrameEntry)>, key: (usize, usize)| {
carried.truncate(carried.partition_point(|(k, _)| *k < key))
};
for entry in items {
match entry {
AxisEntry::GroupHeader {
level,
cat_name,
depth,
group_name,
collapsed,
..
} => {
let group = FrameEntry::Group(FrameGroup {
level: *level,
depth: *depth,
name: group_name.clone(),
collapsed: view.is_group_collapsed(cat_name, group_name),
collapsed: *collapsed,
});
match entries.is_empty() {
true => {
carried.truncate(*level);
carried.resize(*level, None);
carried.push(Some(group));
close(&mut carried, (*level, *depth));
carried.push(((*level, *depth), group));
}
false => pending.push(group),
}
}
AxisEntry::GroupEnd { level } => match entries.is_empty() {
true => carried.truncate(*level),
false => pending.push(FrameEntry::GroupEnd { level: *level }),
AxisEntry::GroupEnd { level, depth } => match entries.is_empty() {
true => close(&mut carried, (*level, *depth)),
false => pending.push(FrameEntry::GroupEnd {
level: *level,
depth: *depth,
}),
},
AxisEntry::DataItem { tuple, collapsed } => {
let idx = data_idx;
data_idx += 1;
if window.contains(&idx) {
if entries.is_empty() {
entries.extend(carried.drain(..).flatten());
entries.extend(carried.drain(..).map(|(_, group)| group));
}
entries.append(&mut pending);
entries.push(FrameEntry::Item {
@@ -271,7 +284,7 @@ fn axis_frame(
}
if window.end >= total {
if entries.is_empty() {
entries.extend(carried.drain(..).flatten());
entries.extend(carried.drain(..).map(|(_, group)| group));
}
entries.append(&mut pending);
}
@@ -423,6 +436,11 @@ fn non_item(category: &str, item: &str) -> CellId {
}
}
/// What a person sees for a category key; the key itself when no category holds it.
fn category_display<'a>(model: &'a Model, name: &'a str) -> &'a str {
model.category(name).map_or(name, |c| c.display())
}
/// The categories the category panel draws: **every** category, virtual ones
/// included, with their items and their axis (improvise-vjms, gotcha 7).
pub fn category_rows(model: &Model, view: &View) -> Vec<CategoryRow> {
@@ -431,6 +449,7 @@ pub fn category_rows(model: &Model, view: &View) -> Vec<CategoryRow> {
.into_iter()
.map(|name| CategoryRow {
name: name.to_string(),
display: category_display(model, name).to_string(),
items: model.effective_item_names(name),
// `try_axis_of`, not `axis_of`: a view that has never seen a category
// has no entry and `axis_of` panics. Unseen shows as `None`.
@@ -460,6 +479,7 @@ pub fn view_summaries(app: &App) -> Vec<ViewSummary> {
.iter()
.map(|(name, view)| ViewSummary {
name: name.clone(),
display: view.display().to_string(),
axes: view
.category_axes
.iter()
@@ -481,6 +501,7 @@ pub fn tiles_of(model: &Model, view: &View) -> Vec<TileEntry> {
.into_iter()
.map(|cat| TileEntry {
category: cat.to_string(),
display: category_display(model, cat).to_string(),
axis: view.try_axis_of(cat).unwrap_or(Axis::None),
kind: model
.category(cat)
@@ -641,6 +662,7 @@ pub fn wizard_cache(w: &ImportWizard) -> WizardCache {
((*label).to_string(), p.date_components.contains(component))
})
.collect(),
shape: date_shape_sentence(p),
})
.collect(),
collisions: w
@@ -679,6 +701,27 @@ pub fn wizard_cache(w: &ImportWizard) -> WizardCache {
}
}
/// What a date field's chosen components become, worded for every front end.
fn date_shape_sentence(p: &improvise_io::import::analyzer::FieldProposal) -> String {
let chosen = |order: &[DateComponent]| -> Vec<&str> {
order
.iter()
.filter(|c| p.date_components.contains(c))
.filter_map(|c| DATE_COMPONENTS.iter().find(|(d, _)| d == c))
.map(|(_, label)| *label)
.collect()
};
match (p.date_shape, chosen(&DateComponent::COARSE_TO_FINE)) {
(_, none) if none.is_empty() => "one category of the raw dates".to_string(),
(DateShape::Hierarchy, levels) => format!("one category: {}", levels.join(" > ")),
(DateShape::Split, _) => {
let parts = chosen(&DATE_COMPONENTS.map(|(c, _)| c));
let names: Vec<String> = parts.iter().map(|l| format!("{}_{l}", p.field)).collect();
format!("split: {}, {}", p.field, names.join(", "))
}
}
}
/// The date components the wizard offers, **in the order it offers them**:
/// `ImportWizard::date_config_at_cursor` counts these same rows.
const DATE_COMPONENTS: [(DateComponent, &str); 3] = [
@@ -1458,6 +1501,7 @@ mod tests {
wb.add_item("Region", "North").unwrap();
wb.add_item("Region", "South").unwrap();
wb.add_item_in_group("Region", "East", "Late").unwrap();
wb.active_view_mut().set_expand_depth("Region", 1);
wb.add_item("_Measure", "Revenue").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
@@ -1469,7 +1513,8 @@ mod tests {
wb.model.set_cell(key, CellValue::Number(1.0));
}
if collapsed {
wb.active_view_mut().toggle_group_collapse("Region", "Late");
wb.active_view_mut()
.toggle_group_collapse("Region", &["Late"]);
}
let mut app = App::new(wb, None);
app.rebuild_layout();
@@ -1573,6 +1618,7 @@ mod tests {
wb.add_item("Region", "North").unwrap();
wb.add_item_in_group("Region", "Inner", "Hidden").unwrap();
wb.add_item_in_group("Region", "East", "Late").unwrap();
wb.active_view_mut().set_expand_depth("Region", 1);
wb.add_item("_Measure", "Revenue").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
@@ -1584,7 +1630,7 @@ mod tests {
wb.model.set_cell(key, CellValue::Number(1.0));
}
wb.active_view_mut()
.toggle_group_collapse("Region", "Hidden");
.toggle_group_collapse("Region", &["Hidden"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
app
@@ -1626,9 +1672,11 @@ mod tests {
wb.add_category("Region").unwrap();
wb.add_category("Month").unwrap();
wb.add_item_in_group("Region", "North", "Upper").unwrap();
wb.active_view_mut().set_expand_depth("Region", 1);
wb.add_item("Region", "South").unwrap();
wb.add_item_in_group("Month", "Jan", "Q1").unwrap();
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Month", "Mar").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
@@ -1674,6 +1722,52 @@ mod tests {
);
}
/// improvise-felo.6: the same agreement for runs nested INSIDE one category,
/// at every window — the carried stack is keyed by depth as well as level.
#[test]
fn the_frame_agrees_with_the_layout_inside_one_categorys_hierarchy() {
let mut wb = Workbook::new("T");
wb.add_category("Region").unwrap();
wb.add_category("Date").unwrap();
wb.add_item_in_group("Region", "North", "Upper").unwrap();
wb.add_item("Region", "South").unwrap();
for (month, group) in [("Jan", "Q1"), ("Feb", "Q1"), ("Jun", "H1"), ("Jul", "Q3")] {
wb.add_item_in_group("Date", month, group).unwrap();
}
wb.add_item("Date", "Undated").unwrap();
let date = wb.model.category_mut("Date").unwrap();
date.add_group(crate::model::category::Group::new("Q1").with_parent("H1"));
date.add_group(crate::model::category::Group::new("Q3").with_parent("H2"));
wb.add_item("_Measure", "Revenue").unwrap();
let view = wb.active_view_mut();
view.set_axis("Region", Axis::Row);
view.set_axis("Date", Axis::Row);
view.set_axis("_Measure", Axis::Column);
view.set_expand_depth("Region", 1);
view.set_expand_depth("Date", 2);
let mut app = App::new(wb, None);
app.rebuild_layout();
let rows = app.layout.row_count();
assert_eq!(rows, 10, "two regions by five dates");
for offset in 0..rows {
let frame = grid_of(&app, rows_from(offset)).row_frame;
let from_frame: Vec<Option<(String, String)>> = frame
.groups()
.into_iter()
.map(|g| g.map(|g| (frame.cats[g.level].clone(), g.name.clone())))
.collect();
let from_layout: Vec<Option<(String, String)>> = (offset..rows)
.map(|r| app.layout.row_group_for(r))
.collect();
assert_eq!(from_frame, from_layout, "window from row {offset}");
}
assert_eq!(
app.layout.row_group_for(2),
Some(("Date".into(), "H1".into())),
"precondition: North/Jun sits in H1 once Q1 ends, not in Q1 nor Upper"
);
}
/// A window opening inside `North/Q1` carries the open run at EVERY level, or
/// `North/Mar` loses `Upper` once `Q1` ends. context/test-evidence/command.md.
#[test]
@@ -1693,7 +1787,7 @@ mod tests {
fn z_on_a_row_past_an_inner_run_folds_the_outer_group() {
use crate::command::key::{KeyCode, KeyEvent, KeyModifiers};
let z = || KeyEvent::new(KeyCode::Char('z'), KeyModifiers::NONE);
let collapsed = |app: &App, cat, group| app.view().is_group_collapsed(cat, group);
let collapsed = |app: &App, cat, group| app.view().is_group_collapsed(cat, &[group]);
let mut app = app_with_nested_groups();
app.cursor_mut().selected = (0, 0);
@@ -1739,6 +1833,46 @@ mod tests {
app
}
/// One category and the active view each shown under a display that differs from their key.
fn app_with_displays_unlike_their_keys() -> App {
let mut app = app_with_data();
let wb = &mut app.model_state.workbook;
wb.model
.category_mut("Region")
.unwrap()
.set_display("Sales Area");
wb.active_view_mut().set_display("Overview");
app
}
#[test]
fn the_category_panel_shows_a_display_and_keeps_the_key_commands_name() {
let app = app_with_displays_unlike_their_keys();
let rows = category_rows(&app.model_state.workbook.model, app.view());
let region = rows.iter().find(|r| r.name == "Region").unwrap();
assert_eq!(region.display, "Sales Area");
let measure = rows.iter().find(|r| r.name == "_Measure").unwrap();
assert_eq!(measure.display, "_Measure");
}
#[test]
fn a_tile_shows_a_display_and_keeps_the_key_commands_name() {
let app = app_with_displays_unlike_their_keys();
let tiles = tiles_of(&app.model_state.workbook.model, app.view());
let region = tiles.iter().find(|t| t.category == "Region").unwrap();
assert_eq!(region.display, "Sales Area");
}
#[test]
fn the_view_list_shows_a_display_and_keeps_the_key_commands_name() {
let app = app_with_displays_unlike_their_keys();
let active = app.model_state.workbook.active_view.clone();
let summaries = view_summaries(&app);
let summary = summaries.iter().find(|s| s.name == active).unwrap();
assert_eq!(summary.display, "Overview");
assert_ne!(summary.name, "Overview");
}
/// A pivot-shaped default view plus a records-shaped second one, so a
/// switch between them crosses the boundary `settled_mode` watches.
fn app_with_a_records_shaped_second_view() -> App {
@@ -2895,6 +3029,7 @@ mod tests {
.unwrap();
wb.add_item_in_group("Product", "Gadget", "Hardware")
.unwrap();
wb.active_view_mut().set_expand_depth("Product", 1);
wb.add_item("Product", "Service").unwrap();
wb.add_item("_Measure", "Revenue").unwrap();
wb.add_item("_Measure", "Cost").unwrap();
@@ -3698,7 +3833,7 @@ mod tests {
item: "Gizmo".into(),
group: "Hardware".into(),
},
// The 12-category limit; the fixture already holds two.
// The category limit; the fixture already holds two.
ModelCommand::AddCategory("Overflow".into()),
];
@@ -4658,6 +4793,7 @@ mod cell_identity {
wb.add_item("Region", "North").unwrap();
wb.add_item_in_group("Region", "East", "Coastal").unwrap();
wb.add_item_in_group("Region", "West", "Coastal").unwrap();
wb.active_view_mut().set_expand_depth("Region", 1);
wb.add_item("_Measure", "Revenue").unwrap();
wb.active_view_mut().set_axis("Region", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
@@ -4671,7 +4807,7 @@ mod cell_identity {
);
}
wb.active_view_mut()
.toggle_group_collapse("Region", "Coastal");
.toggle_group_collapse("Region", &["Coastal"]);
let mut app = App::new(wb, None);
app.rebuild_layout();
assert_eq!(app.layout.row_label(1), "Coastal");
@@ -5642,9 +5778,11 @@ mod cell_identity {
let mut wb = Workbook::new("T");
wb.add_category("Month").unwrap();
wb.add_item_in_group("Month", "Apr", "Q2").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
for month in ["Jan", "Feb", "Mar"] {
wb.add_item_in_group("Month", month, "Q1").unwrap();
}
wb.active_view_mut().set_expand_depth("Month", 1);
for (month, revenue, cost) in [("Jan", 10.0, 5.0), ("Feb", 30.0, 3.0), ("Mar", 50.0, 50.0)]
{
for (measure, n) in [("Revenue", revenue), ("Cost", cost)] {
@@ -5663,7 +5801,7 @@ mod cell_identity {
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().hide_item("Month", "Mar");
wb.active_view_mut().toggle_group_collapse("Month", "Q1");
wb.active_view_mut().toggle_group_collapse("Month", &["Q1"]);
let app = App::new(wb, None);
let grid = grid_of(&app, WHOLE);
@@ -6334,13 +6472,14 @@ mod column_totals {
wb.model.set_cell(at, CellValue::Number(v));
}
}
wb.active_view_mut().set_expand_depth("Month", 1);
let max = crate::formula::AggFunc::Max;
wb.model.measure_agg.insert("Revenue".into(), max);
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().set_axis("Region", Axis::None);
if collapsed {
wb.active_view_mut().toggle_group_collapse("Month", "Q1");
wb.active_view_mut().toggle_group_collapse("Month", &["Q1"]);
}
wb
};
@@ -6405,3 +6544,42 @@ mod column_totals {
);
}
}
/// improvise-8jk6.3: the wizard's date step says what each date field becomes,
/// and `s` switches it between one category and the split.
#[cfg(test)]
mod date_shape {
use super::wizard_cache;
use crate::import::wizard::{ImportWizard, WizardStep};
use serde_json::json;
#[test]
fn the_date_step_says_what_a_date_field_becomes_and_s_switches_it() {
let mut w = ImportWizard::new(json!([
{"Region": "East", "When": "2024-01-15", "Revenue": 1.0},
{"Region": "West", "When": "2024-04-02", "Revenue": 2.0},
]));
w.advance();
assert_eq!(w.step, WizardStep::ConfigureDates, "setup");
let shape = |w: &ImportWizard| wizard_cache(w).date_fields[0].shape.clone();
assert_eq!(shape(&w), "one category of the raw dates");
w.toggle_date_component();
w.move_cursor(1);
w.toggle_date_component();
w.move_cursor(1);
w.toggle_date_component();
assert_eq!(shape(&w), "one category: Year > Quarter > Month");
w.toggle_date_shape();
assert_eq!(
shape(&w),
"split: When, When_Year, When_Month, When_Quarter"
);
let built = w.pipeline.build_model().unwrap();
assert!(built.model.category("When_Quarter").is_some());
w.toggle_date_shape();
let built = w.pipeline.build_model().unwrap();
assert!(built.model.category("When_Quarter").is_none());
let when = built.model.category("When").unwrap();
assert_eq!(when.item_chain("2024-04"), vec!["2024", "2024-Q2"]);
}
}
crates/improvise-command/src/ui/reduce.rs
+3 -3
View File
@@ -1052,9 +1052,9 @@ pub(crate) mod tests {
view.hide_item("Region", "West");
view.hide_item("Region", "South");
view.hide_item("Month", "Mar");
view.toggle_group_collapse("Region", "Coastal");
view.toggle_group_collapse("Region", "Inland");
view.toggle_group_collapse("Month", "Q1");
view.toggle_group_collapse("Region", &["Coastal"]);
view.toggle_group_collapse("Region", &["Inland"]);
view.toggle_group_collapse("Month", &["Q1"]);
app.rebuild_layout();
app
}
crates/improvise-command/src/ui/reduce/vocabulary_is_complete.rs
+7 -11
View File
@@ -16,8 +16,6 @@ enum Verdict {
Frame(&'static str),
/// Deliberately off the wire, and the argument.
Off(&'static str),
/// Missing, and the ticket that owns closing it.
Gap(&'static str),
}
/// Checked in all directions: an entry must name a real effect, must not be on
@@ -29,14 +27,8 @@ const NOT_ON_THE_WIRE: &[(&str, Verdict)] = &[
// Its text; the refusal flag is one host batch's outcome (context/code/cli.md §10).
("Refuse", Verdict::Command("SetStatus")),
("Save", Verdict::Frame("Save")),
(
"SaveAs",
Verdict::Gap("improvise-2mqz: writes the host's Storage and has no frame"),
),
(
"ExportCsv",
Verdict::Gap("improvise-2mqz: writes the host's Storage and has no frame"),
),
("SaveAs", Verdict::Frame("DownloadDocument")),
("ExportCsv", Verdict::Frame("DownloadCsv")),
(
"ApplyAndClearDrill",
Verdict::Off("operand is one session's staged drill edits (gotcha 38, improvise-3x3u)"),
@@ -71,6 +63,10 @@ const NOT_ON_THE_WIRE: &[(&str, Verdict)] = &[
"refreshes the host's layout cache mid-batch; a client holds none (gotcha 23)",
),
),
(
"RequireKnown",
Verdict::Off("a command-side name check that writes no state (gotcha 21, cli.md §15)"),
),
(
"AbortChain",
Verdict::Off("control flow inside one apply_effects batch; it writes no state (gotcha 21)"),
@@ -134,7 +130,7 @@ fn the_verdict_list_names_only_real_effects_and_real_variants() {
frames.iter().any(|f| f == variant),
"{name}'s verdict names ClientFrame::{variant}, which does not exist"
),
Verdict::Off(why) | Verdict::Gap(why) => {
Verdict::Off(why) => {
assert!(!why.is_empty(), "{name}'s verdict gives no argument")
}
}
crates/improvise-command/src/ui/viewport.rs
+76 -2
View File
@@ -101,6 +101,50 @@ pub fn grid_body(
}
}
/// What of a grid build no window changes: the widths, each column's total
/// format and the totals. Measured over every cell, so a host keeps one per layout.
pub struct GridSummary {
pub col_widths: Vec<u16>,
pub col_formats: Vec<NumberFormat>,
pub totals: Vec<Option<CellValue>>,
}
/// The whole-grid walk, answering only what a window cannot.
pub fn grid_summary(model: &Model, layout: &GridLayout, plan: &FormatPlan) -> GridSummary {
let body = grid_body(model, layout, plan, 0..0, 0..0);
GridSummary {
col_widths: body.col_widths,
col_formats: body.col_formats,
totals: body.totals,
}
}
/// [`grid_body`] for a window, given the `summary` of the same layout: evaluates
/// only the window's cells. Equal to `grid_body` where `summary` is current.
pub fn grid_window(
model: &Model,
layout: &GridLayout,
summary: &GridSummary,
rows: std::ops::Range<usize>,
cols: std::ops::Range<usize>,
) -> GridBody {
let eval = layout.evaluator(model);
let records = layout.is_records_mode();
let window = rows
.map(|ri| {
cols.clone()
.map(|ci| cell_value(layout, &eval, records, ri, ci).1)
.collect()
})
.collect();
GridBody {
col_widths: summary.col_widths.clone(),
col_formats: summary.col_formats.clone(),
totals: summary.totals.clone(),
window,
}
}
/// The one evaluation. Records mode reads its wide row; pivot mode aggregates.
fn cell_value(
layout: &GridLayout,
@@ -146,7 +190,7 @@ fn widen(widths: &mut [u16], col: usize, width: usize) {
/// Adaptive column widths alone, for the keypress path, which needs no cells.
/// It is [`grid_body`] with an empty window so there is one width rule.
pub fn compute_col_widths(model: &Model, layout: &GridLayout, plan: &FormatPlan) -> Vec<u16> {
grid_body(model, layout, plan, 0..0, 0..0).col_widths
grid_summary(model, layout, plan).col_widths
}
/// The total row-header width for a layout. The rule is the protocol's; this is the
@@ -214,6 +258,7 @@ mod tests {
wb.add_item_in_group("Month", "Apr", "Q2").unwrap();
wb.add_item_in_group("Month", "Jan", "Q1").unwrap();
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("_Measure", "Revenue").unwrap();
for (month, revenue) in [("Jan", 10.0), ("Feb", 30.0)] {
let key = CellKey::new(vec![
@@ -231,7 +276,7 @@ mod tests {
}
wb.active_view_mut().set_axis("Month", Axis::Row);
wb.active_view_mut().set_axis("_Measure", Axis::Column);
wb.active_view_mut().toggle_group_collapse("Month", "Q1");
wb.active_view_mut().toggle_group_collapse("Month", &["Q1"]);
wb.model.recompute_formulas(&wb.active_view().none_cats());
wb
}
@@ -271,4 +316,33 @@ mod tests {
"F0..F8 once for the Q1 row, once for the totals"
);
}
/// improvise-felo.13: a window read off a held summary is the window `grid_body`
/// builds. Evidence: `context/test-evidence/layout-cost.md`.
#[test]
fn a_window_over_a_held_summary_equals_the_one_walk_grid_body() {
let wb = folded_chain();
let view = wb.active_view();
let layout = GridLayout::new(&wb.model, view);
let plan = FormatPlan::of(&wb.model, view);
let summary = grid_summary(&wb.model, &layout, &plan);
let (rows, cols) = (layout.row_count(), layout.col_count());
assert!(
rows >= 2 && cols >= 3,
"a window strictly inside is possible"
);
for (r, c) in [
(0..rows, 0..cols),
(1..2, 1..3),
(rows - 1..rows, 0..1),
(0..0, 0..0),
] {
let walked = grid_body(&wb.model, &layout, &plan, r.clone(), c.clone());
let held = grid_window(&wb.model, &layout, &summary, r.clone(), c.clone());
assert_eq!(held.window, walked.window, "window {r:?} x {c:?}");
assert_eq!(held.col_widths, walked.col_widths);
assert_eq!(held.totals, walked.totals);
assert_eq!(held.col_formats, walked.col_formats);
}
}
}
crates/improvise-core/Cargo.toml
+2
View File
@@ -16,6 +16,8 @@ serde = { version = "1", features = ["derive"] }
# (ADR 002). Already in the tree as a transitive dependency; no_std and
# wasm-clean.
siphasher = "1"
# Unicode full case folding for slugify (ADR 007 answer 26). no_std, no dependencies.
unicase = "2.9"
[features]
# Exposes the fold-evaluation counter to other crates' tests (improvise-0jw5).
crates/improvise-core/examples/category_width_bench.rs
+128
View File
@@ -0,0 +1,128 @@
//! What a wider document costs: the same records at 12 and at 24 regular
//! categories. improvise-felo.1; the argument is `context/code/core.md` §9.
use std::time::Instant;
use improvise_core::model::cell::{CellKey, CellValue};
use improvise_core::view::{Axis, GridLayout, ViewStep, records_steps};
use improvise_core::workbook::Workbook;
const MEASURE: &str = "_Measure";
const ITEMS: u64 = 4;
/// `records` records over `width` categories of `ITEMS` items, two measures
/// each; coordinates from a fixed LCG so both widths see comparable data.
fn document(width: usize, records: usize) -> Workbook {
let mut wb = Workbook::new("bench");
for c in 0..width {
wb.add_category(format!("C{c}")).expect("under the limit");
}
let mut state: u64 = 0x9E37_79B9_7F4A_7C15;
let mut next = move || {
state = state
.wrapping_mul(6_364_136_223_846_793_005)
.wrapping_add(1_442_695_040_888_963_407);
(state >> 33) % ITEMS
};
for _ in 0..records {
let coords: Vec<(String, String)> = (0..width)
.map(|c| (format!("C{c}"), format!("i{}", next())))
.collect();
for (name, value) in [("Cost", 3.0), ("Rev", 5.0)] {
let mut key = coords.clone();
key.push((MEASURE.to_string(), name.to_string()));
wb.model
.set_cell(CellKey::new(key), CellValue::Number(value));
}
}
for c in 0..width {
let cat = wb.model.category_mut(&format!("C{c}")).expect("added");
for i in 0..ITEMS {
cat.add_item(format!("i{i}"));
}
}
for name in ["Cost", "Rev"] {
wb.model
.category_mut(MEASURE)
.expect("virtual")
.add_item(name);
}
wb
}
/// Best of `runs`, in milliseconds.
fn best(runs: usize, mut f: impl FnMut()) -> f64 {
(0..runs)
.map(|_| {
let t = Instant::now();
f();
t.elapsed().as_secs_f64() * 1000.0
})
.fold(f64::INFINITY, f64::min)
}
fn pivot(wb: &mut Workbook, width: usize) -> f64 {
let view = wb.active_view_mut();
for c in 0..width {
view.set_axis(
&format!("C{c}"),
if c == 0 { Axis::Row } else { Axis::None },
);
}
view.set_axis(MEASURE, Axis::Column);
let (model, view) = (&wb.model, wb.active_view());
best(5, || {
let layout = GridLayout::with_frozen_records(model, view, None);
let eval = layout.evaluator(model);
for r in 0..layout.row_count() {
for c in 0..layout.col_count() {
let key = layout.cell_key(r, c).expect("in bounds");
std::hint::black_box(eval.cell(&key));
}
}
})
}
/// (build, draw): the records layout, then every cell's display text.
fn records(wb: &mut Workbook) -> (f64, f64) {
let steps = records_steps(&wb.model);
let view = wb.active_view_mut();
for step in steps {
if let ViewStep::Axis { category, axis } = step {
view.set_axis(&category, axis);
}
}
let (model, view) = (&wb.model, wb.active_view());
let build = best(5, || {
std::hint::black_box(GridLayout::with_frozen_records(model, view, None));
});
let layout = GridLayout::with_frozen_records(model, view, None);
let draw = best(5, || {
for r in 0..layout.row_count() {
for c in 0..layout.col_count() {
std::hint::black_box(layout.records_display(r, c));
}
}
});
(build, draw)
}
fn main() {
println!(
"{:>6} {:>8} {:>7} {:>9} {:>9} {:>14} {:>13}",
"width", "records", "cells", "store ms", "pivot ms", "rec build ms", "rec draw ms"
);
for records_n in [2_500, 5_000] {
for width in [12, 24] {
let mut built = None;
let store = best(3, || built = Some(document(width, records_n)));
let mut wb = built.expect("ran");
let cells = wb.model.data.iter_cells().count();
let p = pivot(&mut wb, width);
let (rb, rd) = records(&mut wb);
println!(
"{width:>6} {records_n:>8} {cells:>7} {store:>9.1} {p:>9.1} {rb:>14.1} {rd:>13.1}"
);
}
}
}
crates/improvise-core/src/model/category.rs
+154 -5
View File
@@ -8,19 +8,27 @@ pub type ItemId = usize;
pub struct Item {
pub id: ItemId,
pub name: String,
/// What a person sees; the key until a rename (ADR 007 clause 2).
display: String,
/// Parent group name, if any
pub group: Option<String>,
}
impl Item {
pub fn new(id: ItemId, name: impl Into<String>) -> Self {
let name = name.into();
Self {
id,
name: name.into(),
display: name.clone(),
name,
group: None,
}
}
pub fn display(&self) -> &str {
&self.display
}
pub fn with_group(mut self, group: impl Into<String>) -> Self {
self.group = Some(group.into());
self
@@ -129,6 +137,8 @@ impl CategoryKind {
pub struct Category {
pub id: CategoryId,
pub name: String,
/// What a person sees; the key until a rename (ADR 007 clause 2).
display: String,
/// Items in insertion order
pub items: IndexMap<String, Item>,
/// Named groups (hierarchy nodes)
@@ -141,9 +151,11 @@ pub struct Category {
impl Category {
pub fn new(id: CategoryId, name: impl Into<String>) -> Self {
let name = name.into();
Self {
id,
name: name.into(),
display: name.clone(),
name,
items: IndexMap::new(),
groups: Vec::new(),
next_item_id: 0,
@@ -151,6 +163,23 @@ impl Category {
}
}
pub fn display(&self) -> &str {
&self.display
}
/// Changes what a person sees and nothing else: the key does not move.
pub fn set_display(&mut self, display: impl Into<String>) {
self.display = display.into();
}
/// Changes an item's display; false when the category has no such item.
pub fn set_item_display(&mut self, item: &str, display: impl Into<String>) -> bool {
self.items
.get_mut(item)
.map(|i| i.display = display.into())
.is_some()
}
pub fn with_kind(mut self, kind: CategoryKind) -> Self {
self.kind = kind;
self
@@ -199,15 +228,61 @@ impl Category {
self.items.keys().map(|s| s.as_str()).collect()
}
/// The items whose own `group` names `group`, in insertion order. Read off the
/// items, never `groups`: `add_item_in_group` registers no `Group` (core.md §14).
/// The items `group` covers — its own and every child group's, through any
/// depth — in insertion order. Read off the items, not `groups` (core.md §14).
pub fn items_in_group(&self, group: &str) -> Vec<&str> {
self.items
.values()
.filter(|i| i.group.as_deref() == Some(group))
.filter(|i| self.item_chain(&i.name).contains(&group))
.map(|i| i.name.as_str())
.collect()
}
/// The parent a `>` line gave `group`; `None` for a root.
pub fn group_parent(&self, group: &str) -> Option<&str> {
self.groups
.iter()
.find(|g| g.name == group)
.and_then(|g| g.parent.as_deref())
}
/// `group` and its ancestors, ROOT FIRST. A cycle built in code stops the walk
/// where it would repeat; a file cannot carry one (`hierarchy_cycle`).
pub fn group_chain<'a>(&'a self, group: &'a str) -> Vec<&'a str> {
let mut chain = vec![group];
while let Some(parent) = chain.last().and_then(|g| self.group_parent(g)) {
if chain.contains(&parent) {
break;
}
chain.push(parent);
}
chain.reverse();
chain
}
/// The groups an item sits in, root first; empty for an ungrouped item.
pub fn item_chain(&self, item: &str) -> Vec<&str> {
self.items
.get(item)
.and_then(|i| i.group.as_deref())
.map(|g| self.group_chain(g))
.unwrap_or_default()
}
/// A cycle among the declared parents, its members in walk order: the one
/// shape a hierarchy can take that is not a forest (core.md §19).
pub fn hierarchy_cycle(&self) -> Option<Vec<&str>> {
self.groups.iter().find_map(|start| {
let mut path: Vec<&str> = vec![start.name.as_str()];
while let Some(parent) = path.last().and_then(|g| self.group_parent(g)) {
if let Some(at) = path.iter().position(|g| *g == parent) {
return Some(path[at..].to_vec());
}
path.push(parent);
}
None
})
}
}
#[cfg(test)]
@@ -218,6 +293,20 @@ mod tests {
Category::new(0, "Region")
}
#[test]
fn a_display_starts_as_the_key_and_changing_it_moves_no_key() {
let mut c = cat();
c.add_item("East");
assert_eq!(c.display(), "Region");
assert_eq!(c.items["East"].display(), "East");
c.set_display("Sales Area");
assert!(c.set_item_display("East", "Eastern"));
assert_eq!((c.name.as_str(), c.display()), ("Region", "Sales Area"));
assert_eq!(c.items["East"].display(), "Eastern");
assert_eq!(c.items["East"].name, "East");
assert!(!c.set_item_display("West", "Western"), "no such item");
}
/// improvise-2lh: call sites ask `of_name` rather than compare literals,
/// so both directions of the mapping must agree.
#[test]
@@ -297,6 +386,66 @@ mod tests {
assert_eq!(c.groups.len(), 1);
}
/// Year > Quarter > Month, with `2024-03` added LAST so Q1-2024 and 2024 are
/// non-contiguous in insertion order.
fn dates() -> Category {
let mut c = Category::new(0, "Date");
c.add_item_in_group("2024-01", "Q1-2024");
c.add_item_in_group("2024-04", "Q2-2024");
c.add_item_in_group("2025-01", "Q1-2025");
c.add_item("Undated");
c.add_item_in_group("2024-03", "Q1-2024");
c.add_group(Group::new("Q1-2024").with_parent("2024"));
c.add_group(Group::new("Q2-2024").with_parent("2024"));
c.add_group(Group::new("Q1-2025").with_parent("2025"));
c
}
#[test]
fn a_parent_group_covers_every_leaf_beneath_its_child_groups() {
let c = dates();
assert_eq!(
c.items_in_group("2024"),
vec!["2024-01", "2024-04", "2024-03"]
);
assert_eq!(c.items_in_group("Q1-2024"), vec!["2024-01", "2024-03"]);
assert_eq!(c.items_in_group("2025"), vec!["2025-01"]);
assert!(c.items_in_group("Undated").is_empty());
}
#[test]
fn an_items_chain_runs_root_first() {
let c = dates();
assert_eq!(c.item_chain("2024-03"), vec!["2024", "Q1-2024"]);
assert!(c.item_chain("Undated").is_empty());
assert_eq!(c.group_chain("2024"), vec!["2024"]);
assert_eq!(c.group_chain("Q2-2024"), vec!["2024", "Q2-2024"]);
}
#[test]
fn a_cycle_among_parents_is_found_and_named() {
let mut c = dates();
assert_eq!(c.hierarchy_cycle(), None);
c.add_group(Group::new("2024").with_parent("Q2-2024"));
let cycle = c.hierarchy_cycle().expect("2024 -> Q2-2024 -> 2024");
assert!(
cycle.contains(&"2024") && cycle.contains(&"Q2-2024"),
"{cycle:?}"
);
assert!(!cycle.contains(&"Q1-2025"), "{cycle:?}");
assert!(
c.group_chain("Q1-2024").len() <= 3,
"a walk through a cycle stops"
);
}
#[test]
fn a_self_parent_is_a_cycle_of_one() {
let mut c = cat();
c.add_group(Group::new("G").with_parent("G"));
assert_eq!(c.hierarchy_cycle(), Some(vec!["G"]));
}
#[test]
fn item_index_reflects_insertion_order() {
let mut c = cat();
crates/improvise-core/src/model/cell.rs
+141
View File
@@ -194,6 +194,12 @@ impl RecordEntry {
}
}
#[cfg(test)]
thread_local! {
/// How many lookups fell through to set narrowing, for the exact-path test.
static NARROWED: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
#[derive(Clone, Default)]
pub struct DataStore {
/// Primary storage: interned keys, insertion-ordered so records mode can
@@ -206,6 +212,9 @@ pub struct DataStore {
/// Reverse index for the client-facing surrogate (ADR 002), plus each
/// record's write counter. Why it lives here: core.md §8.
records: HashMap<RecordHash, RecordEntry>,
/// How many stored cells have each number of coordinates; its largest key
/// says when a partial can match only one exact cell.
arities: std::collections::BTreeMap<usize, usize>,
}
/// Resolved through the symbol table and sorted wherever the storage is a
@@ -217,6 +226,7 @@ impl std::fmt::Debug for DataStore {
symbols,
index,
records,
arities: _,
} = self;
let cells: Vec<_> = cells
.iter()
@@ -291,7 +301,11 @@ impl DataStore {
for pair in &ikey.0 {
self.index.entry(*pair).or_default().insert(ikey.clone());
}
let width = ikey.0.len();
let is_new_cell = self.cells.insert(ikey, value).is_none();
if is_new_cell {
*self.arities.entry(width).or_default() += 1;
}
let entry = self.records.entry(hash).or_insert(RecordEntry {
record,
cells: 0,
@@ -327,6 +341,12 @@ impl DataStore {
return;
};
if self.cells.shift_remove(&ikey).is_some() {
if let Some(n) = self.arities.get_mut(&ikey.0.len()) {
*n -= 1;
if *n == 0 {
self.arities.remove(&ikey.0.len());
}
}
for pair in &ikey.0 {
if let Some(set) = self.index.get_mut(pair) {
set.remove(&ikey);
@@ -378,11 +398,28 @@ impl DataStore {
/// Positions in `cells` of every cell matching `partial`, **in insertion
/// order**. The index narrows; the primary map orders (§8).
fn matching_positions(&self, partial: &[(String, String)]) -> Vec<usize> {
if let Some(exact) = self.exact_positions(partial) {
return exact;
}
self.intern_partial(partial)
.map(|pairs| self.positions_of(&pairs))
.unwrap_or_default()
}
/// The answer without narrowing, when `partial` is as wide as the widest cell
/// stored: only an equal cell can match (layout-cost.md). `None` otherwise.
fn exact_positions(&self, partial: &[(String, String)]) -> Option<Vec<usize>> {
let widest = *self.arities.keys().next_back()?;
// Strictly sorted by category, the order a stored key is in, so one probe is the lookup.
let canonical = partial.windows(2).all(|w| w[0].0 < w[1].0);
(!partial.is_empty() && widest <= partial.len() && canonical).then(|| {
let found = self
.intern_partial(partial)
.and_then(|pairs| self.cells.get_index_of(&InternedKey(pairs)));
found.into_iter().collect()
})
}
/// `partial` as symbols, or `None` when a name was never interned, which
/// no stored cell can then match.
fn intern_partial(&self, partial: &[(String, String)]) -> Option<Vec<(Symbol, Symbol)>> {
@@ -412,6 +449,8 @@ impl DataStore {
}
sets.sort_by_key(|s| s.len());
#[cfg(test)]
NARROWED.with(|n| n.set(n.get() + 1));
let mut positions: Vec<usize> = match sets.split_first() {
Some((first, rest)) => first
@@ -1356,3 +1395,105 @@ mod record_index {
}
}
}
#[cfg(test)]
mod exact_path_tests {
use super::{CellKey, CellValue, DataStore, NARROWED};
fn key(a: &str, b: &str, c: &str) -> CellKey {
CellKey::new(vec![
("A".into(), a.into()),
("B".into(), b.into()),
("C".into(), c.into()),
])
}
fn store() -> DataStore {
let mut d = DataStore::new();
for i in 0..6 {
for j in 0..6 {
d.set(
key(&format!("a{i}"), &format!("b{j}"), "c"),
CellValue::Number((i * 6 + j) as f64),
);
}
}
d
}
fn narrowings(f: impl FnOnce()) -> usize {
NARROWED.with(|n| n.set(0));
f();
NARROWED.with(|n| n.get())
}
/// improvise-felo.13: context/test-evidence/layout-cost.md.
#[test]
fn a_full_width_partial_probes_one_cell_instead_of_narrowing() {
let d = store();
let full = key("a2", "b3", "c").0;
let mut got = Vec::new();
assert_eq!(
narrowings(|| got = d.matching_values(&full).into_iter().cloned().collect()),
0
);
assert_eq!(got, vec![CellValue::Number(15.0)]);
let missing = key("a2", "b9", "c").0;
assert!(
d.matching_values(&missing).is_empty(),
"an absent cell matches nothing"
);
let unknown = key("a2", "b3", "zzz").0;
assert!(
d.matching_values(&unknown).is_empty(),
"an unseen item matches nothing"
);
}
/// The control: a narrower partial still sums its supersets, so the shortcut
/// is not a blanket exact lookup.
#[test]
fn a_narrower_partial_still_narrows_and_matches_every_superset() {
let d = store();
let narrow = vec![
("A".to_string(), "a2".to_string()),
("C".to_string(), "c".to_string()),
];
let mut n = 0;
assert_eq!(narrowings(|| n = d.matching_values(&narrow).len()), 1);
assert_eq!(n, 6);
}
/// A wider cell appearing, or leaving, moves the line the shortcut is allowed at.
#[test]
fn a_wider_cell_turns_the_shortcut_off_until_it_is_removed() {
let mut d = store();
let wide = CellKey::new(vec![
("A".into(), "a2".into()),
("B".into(), "b3".into()),
("C".into(), "c".into()),
("D".into(), "d".into()),
]);
d.set(wide.clone(), CellValue::Number(100.0));
let full = key("a2", "b3", "c").0;
assert_eq!(
d.matching_values(&full).len(),
2,
"the exact cell and the wider cell under it"
);
d.remove(&wide);
assert_eq!(
narrowings(|| assert_eq!(d.matching_values(&full).len(), 1)),
0
);
}
/// An unsorted partial is not a stored key's order, so it must not be probed.
#[test]
fn an_unsorted_full_width_partial_is_still_answered() {
let d = store();
let mut full = key("a2", "b3", "c").0;
full.reverse();
assert_eq!(d.matching_values(&full).len(), 1);
}
}
crates/improvise-core/src/model/id.rs
+290
View File
@@ -0,0 +1,290 @@
//! How an id is made: `slugify`, and the document's `IdAllocator` (ADR 007 clause 4).
//! Reasoning: `context/code/core.md` §20.
use std::collections::HashSet;
/// What a generated id names; spells the `<kind>` of an empty slug's `<kind>-<n>`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum IdKind {
Category,
Item,
Formula,
View,
}
impl IdKind {
pub fn name(self) -> &'static str {
match self {
IdKind::Category => "category",
IdKind::Item => "item",
IdKind::Formula => "formula",
IdKind::View => "view",
}
}
}
/// The only ids that may begin with `_` (answer 23).
pub const VIRTUAL_IDS: [&str; 3] = ["_index", "_dim", "_measure"];
/// A display name's slug; empty when nothing survives (answers 16, 18, 21, 23, 26).
pub fn slugify(display: &str) -> String {
let folded = unicase::UniCase::new(display).to_folded_case();
let kept: String = underscore_space_runs(&folded)
.chars()
.filter(|&c| in_id_alphabet(c))
.collect();
kept.trim_start_matches('_').to_string()
}
fn underscore_space_runs(s: &str) -> String {
let (out, _) = s
.chars()
.fold((String::new(), false), |(mut out, prev), c| {
let space = c == ' ';
if !(space && prev) {
out.push(if space { '_' } else { c });
}
(out, space)
});
out
}
fn in_id_alphabet(c: char) -> bool {
matches!(c, 'a'..='z' | '0'..='9' | '-' | '_')
}
/// Why a hand-written id cannot enter the document.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ClaimError {
Duplicate(String),
Malformed(String),
Reserved(String),
}
/// One document's ids: every id it has held this session, and the `gensym` counter.
#[derive(Debug, Clone)]
pub struct IdAllocator {
next: u64,
seen: HashSet<String>,
}
impl Default for IdAllocator {
fn default() -> Self {
Self {
next: 1,
seen: HashSet::new(),
}
}
}
impl IdAllocator {
pub fn new() -> Self {
Self::default()
}
/// Admit a hand-written id verbatim: it is canonical and never renamed (answer 29).
pub fn claim(&mut self, id: &str) -> Result<(), ClaimError> {
let refusal = if id.is_empty() || !id.chars().all(in_id_alphabet) {
Some(ClaimError::Malformed(id.to_string()))
} else if id.starts_with('_') && !VIRTUAL_IDS.contains(&id) {
Some(ClaimError::Reserved(id.to_string()))
} else if self.seen.contains(id) {
Some(ClaimError::Duplicate(id.to_string()))
} else {
None
};
match refusal {
Some(e) => Err(e),
None => {
self.seen.insert(id.to_string());
Ok(())
}
}
}
/// A fresh id for `display`, suffixed from the counter only when the slug is taken.
pub fn generate(&mut self, kind: IdKind, display: &str) -> String {
let slug = slugify(display);
let id = if slug.is_empty() {
self.suffixed(kind.name())
} else if self.seen.contains(&slug) {
self.suffixed(&slug)
} else {
slug
};
self.seen.insert(id.clone());
id
}
fn suffixed(&mut self, base: &str) -> String {
loop {
let candidate = format!("{base}-{}", self.next);
self.next += 1;
if !self.seen.contains(&candidate) {
break candidate;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use proptest::prelude::*;
#[test]
fn slugify_follows_the_adrs_examples() {
let cases = [
("East", "east"),
("Time Period", "time_period"),
("Coastal_East", "coastal_east"),
("Q1-2024", "q1-2024"),
("Region(code)", "regioncode"),
("Café", "caf"),
("_Draft", "draft"),
("Ü", ""),
];
for (display, slug) in cases {
assert_eq!(slugify(display), slug, "slugify({display:?})");
}
}
#[test]
fn slugify_case_folds_rather_than_lowercases() {
assert_eq!(slugify("Straße"), "strasse");
assert_eq!(slugify("ſtate"), "state");
assert_eq!(slugify("file"), "file");
}
#[test]
fn slugify_collapses_a_run_of_spaces_to_one_underscore() {
assert_eq!(slugify("Time Period"), "time_period");
assert_eq!(slugify("East "), "east_");
}
#[test]
fn slugify_keeps_an_underscore_the_display_already_had() {
assert_eq!(slugify("a_ b"), "a__b");
assert_eq!(slugify("a__b"), "a__b");
}
#[test]
fn an_empty_slug_takes_its_kind_and_the_shared_counter() {
let mut ids = IdAllocator::new();
assert_eq!(ids.generate(IdKind::Category, "Ü"), "category-1");
assert_eq!(ids.generate(IdKind::Item, "Ö"), "item-2");
assert_eq!(ids.generate(IdKind::View, ""), "view-3");
}
#[test]
fn a_slug_taken_by_another_kind_takes_a_suffix() {
let mut ids = IdAllocator::new();
assert_eq!(ids.generate(IdKind::Category, "East"), "east");
assert_eq!(ids.generate(IdKind::Item, "East"), "east-1");
assert_eq!(ids.generate(IdKind::View, "Time Period"), "time_period");
assert_eq!(
ids.generate(IdKind::Formula, "Time_Period"),
"time_period-2"
);
}
#[test]
fn the_suffix_is_the_next_counter_value_not_the_lowest_free_number() {
let mut ids = IdAllocator::new();
assert_eq!(ids.generate(IdKind::Item, "East"), "east");
assert_eq!(ids.generate(IdKind::Item, "West"), "west");
assert_eq!(ids.generate(IdKind::Item, "East"), "east-1");
assert_eq!(ids.generate(IdKind::Item, "West"), "west-2");
assert_eq!(ids.generate(IdKind::Item, "EAST"), "east-3");
}
#[test]
fn a_claimed_id_keeps_its_spelling_and_generation_steps_past_it() {
let mut ids = IdAllocator::new();
assert_eq!(ids.claim("east-1"), Ok(()));
assert_eq!(ids.generate(IdKind::Item, "East"), "east");
assert_eq!(ids.generate(IdKind::Item, "East"), "east-2");
}
#[test]
fn claim_refuses_a_duplicate_a_malformed_and_a_reserved_id() {
let mut ids = IdAllocator::new();
assert_eq!(ids.claim("east"), Ok(()));
assert_eq!(ids.claim("east"), Err(ClaimError::Duplicate("east".into())));
assert_eq!(ids.claim("East"), Err(ClaimError::Malformed("East".into())));
assert_eq!(ids.claim(""), Err(ClaimError::Malformed(String::new())));
assert_eq!(
ids.claim("_draft"),
Err(ClaimError::Reserved("_draft".into()))
);
assert_eq!(ids.claim("_measure"), Ok(()));
let generated = ids.generate(IdKind::Category, "West");
assert_eq!(
ids.claim(&generated),
Err(ClaimError::Duplicate("west".into()))
);
}
#[test]
fn claim_refuses_an_id_that_only_begins_like_a_virtual_one() {
let mut ids = IdAllocator::new();
for lookalike in ["_indexer", "_measure-1", "_dim_2", "_measures"] {
assert_eq!(
ids.claim(lookalike),
Err(ClaimError::Reserved(lookalike.into()))
);
}
}
#[test]
fn a_display_with_leading_spaces_never_slugs_into_the_reserved_space() {
assert_eq!(slugify(" Draft"), "draft");
assert_eq!(slugify(" _ Draft"), "draft");
}
#[test]
fn a_long_claimed_run_of_suffixes_is_walked_without_recursion() {
let mut ids = IdAllocator::new();
for n in 1..=20_000 {
assert_eq!(ids.claim(&format!("item-{n}")), Ok(()));
}
assert_eq!(ids.generate(IdKind::Item, "Ü"), "item-20001");
assert_eq!(ids.generate(IdKind::Category, ""), "category-20002");
}
#[test]
fn two_allocators_fed_alike_generate_alike() {
let displays = [
"East",
"east",
"Ü",
"Time Period",
"Time_Period",
"",
"EAST",
];
let run = || {
let mut ids = IdAllocator::new();
displays
.iter()
.map(|d| ids.generate(IdKind::Item, d))
.collect::<Vec<_>>()
};
assert_eq!(run(), run());
}
proptest! {
#[test]
fn generated_ids_are_distinct_well_formed_and_never_reserved(
displays in proptest::collection::vec(".{0,6}", 0..24)
) {
let mut ids = IdAllocator::new();
let mut out = HashSet::new();
for d in &displays {
let id = ids.generate(IdKind::Item, d);
prop_assert!(!id.is_empty() && !id.starts_with('_'), "{id:?}");
prop_assert!(id.chars().all(|c| matches!(c, 'a'..='z' | '0'..='9' | '-' | '_')));
prop_assert!(out.insert(id.clone()), "reissued {id:?}");
}
}
}
}
crates/improvise-core/src/model/mod.rs
+2 -1
View File
@@ -1,7 +1,8 @@
pub mod category;
pub mod cell;
pub mod id;
pub mod record;
pub mod symbol;
pub mod types;
pub use types::{Aggregation, Model};
pub use types::{Aggregation, MAX_CATEGORIES, Model};
crates/improvise-core/src/model/types.rs
+28 -9
View File
@@ -7,7 +7,9 @@ use super::category::{Category, CategoryId, CategoryKind, MEASURE_CATEGORY};
use super::cell::{CellKey, CellValue, DataStore};
use crate::formula::{AggFunc, Expr, Filter, Formula, FormulaId};
const MAX_CATEGORIES: usize = 12;
/// Regular categories a model may hold; labels and virtuals do not count.
/// Why 24 and what it costs: context/code/core.md §9 (improvise-felo.1).
pub const MAX_CATEGORIES: usize = 24;
/// Shared by `=`/`!=` **and** the division-by-zero guard, in both evaluators.
/// They must agree; why, and why 1e-10: context/code/core.md §4.
@@ -523,6 +525,15 @@ impl Model {
self.categories.get(name)
}
/// `group` and its ancestors in `cat`, root first and owned: the chain a
/// `View` folds by (core.md §19). Just `[group]` where `cat` is unknown.
pub fn group_chain(&self, cat: &str, group: &str) -> Vec<String> {
match self.category(cat) {
Some(c) => c.group_chain(group).into_iter().map(String::from).collect(),
None => vec![group.to_string()],
}
}
pub fn set_cell(&mut self, key: CellKey, value: CellValue) {
self.data.set(key, value);
}
@@ -1386,7 +1397,8 @@ mod model_tests {
assert_eq!(id, m.categories["C0"].id);
assert_eq!(m.categories.len(), before, "nothing was created");
// The limit still refuses a category that WOULD be created.
assert!(m.add_category("C12").is_err(), "the limit still holds");
let fresh = format!("C{}", super::MAX_CATEGORIES);
assert!(m.add_category(fresh).is_err(), "the limit still holds");
}
/// improvise-s08t — see context/test-evidence/core.md.
@@ -1550,13 +1562,20 @@ mod model_tests {
assert_eq!(m.category_names().len(), 4);
}
/// The user's number (improvise-felo.1), spelled once here; every other
/// fill-to-the-cap test derives it from `MAX_CATEGORIES`.
#[test]
fn add_category_max_limit() {
let mut m = Model::new("Test");
for i in 0..12 {
m.add_category(format!("Cat{i}")).unwrap();
for i in 0..24 {
m.add_category(format!("Cat{i}"))
.unwrap_or_else(|e| panic!("category {} of 24 refused: {e}", i + 1));
}
assert!(m.add_category("TooMany").is_err());
let err = m.add_category("TooMany").expect_err("the 25th is refused");
assert!(
err.to_string().contains("24"),
"the refusal names the real limit: {err}"
);
}
#[test]
@@ -3152,11 +3171,11 @@ mod five_category {
// 4 regular + 3 virtual.
assert_eq!(m.category_names().len(), 7);
let mut m2 = build_model();
for i in 0..8 {
for i in 0..super::MAX_CATEGORIES - 4 {
m2.add_category(format!("Extra{i}")).unwrap();
}
// 12 regular + 3 virtuals = 15.
assert_eq!(m2.category_names().len(), 15);
// The limit's worth of regular categories, plus 3 virtuals.
assert_eq!(m2.category_names().len(), super::MAX_CATEGORIES + 3);
assert!(m2.add_category("OneMore").is_err());
}
}
@@ -3225,7 +3244,7 @@ mod prop_tests {
if let Some(c) = m.category_mut("_Measure") { c.add_item("Revenue"); }
let key = CellKey::new(vec![("_Measure".into(), "Revenue".into())]);
m.set_cell(key.clone(), CellValue::Number(val));
let _ = m.add_category(&new_cat); // may hit the 12-cat limit
let _ = m.add_category(&new_cat); // may be refused; either way the cell survives
prop_assert_eq!(m.evaluate(&key), Some(CellValue::Number(val)));
}
crates/improvise-core/src/view/axis.rs
+1 -35
View File
@@ -1,6 +1,6 @@
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Axis {
Row,
@@ -12,17 +12,6 @@ pub enum Axis {
None,
}
/// Accent color for an axis. Semantic palette only; `draw.rs::axis_fg` maps
/// each variant to a terminal color, so `Axis` names no ratatui type.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AxisColor {
Row,
Column,
Page,
Filter,
Inactive,
}
impl Axis {
/// The next axis in the Row → Column → Page → None → Row rotation.
/// `Filter` is outside it and cycling one is a no-op (improvise-dz8).
@@ -48,18 +37,6 @@ impl Axis {
!matches!(self, Axis::Filter)
}
/// Accent color. The single source for axis coloring: the tile bar and the
/// category panel each carried a copy before (improvise-rml).
pub fn color(self) -> AxisColor {
match self {
Axis::Row => AxisColor::Row,
Axis::Column => AxisColor::Column,
Axis::Page => AxisColor::Page,
Axis::Filter => AxisColor::Filter,
Axis::None => AxisColor::Inactive,
}
}
/// Name with no decoration: "Row", "Col", "Page", "None".
pub fn plain_label(self) -> &'static str {
match self {
@@ -176,15 +153,4 @@ mod tests {
assert!(!Axis::Column.filters_data());
assert!(!Axis::None.filters_data());
}
/// A collision defeats the only reason `AxisColor` exists.
#[test]
fn every_axis_gets_its_own_color() {
let colors = ALL.map(Axis::color);
for (i, a) in colors.iter().enumerate() {
for b in &colors[i + 1..] {
assert_ne!(a, b, "two axes share an accent color");
}
}
}
}
crates/improvise-core/src/view/layout.rs
+715 -145
View File
File diff suppressed because it is too large Load Diff
crates/improvise-core/src/view/mod.rs
+2 -2
View File
@@ -2,12 +2,12 @@ pub mod axis;
pub mod layout;
pub mod types;
pub use axis::{Axis, AxisColor};
pub use axis::Axis;
pub use layout::{
AxisEntry, GridEval, GridLayout, NEW_CATEGORY_COL, NEW_MEASURE_COL, RecordCol, RecordRow,
ViewStep, drill_constraint, drill_snapshot, drill_steps, record_row, records_spine,
records_steps, synthetic_record_info,
};
pub use types::{
DEFAULT_VIEW, DRILL_VIEW, RECORDS_VIEW, VIRTUAL_VIEW_NAMES, View, unused_view_name,
DEFAULT_VIEW, DRILL_VIEW, GroupFold, RECORDS_VIEW, VIRTUAL_VIEW_NAMES, View, unused_view_name,
};
crates/improvise-core/src/view/types.rs
+201 -39
View File
@@ -1,5 +1,6 @@
use indexmap::IndexMap;
use std::collections::{HashMap, HashSet};
use std::num::NonZeroUsize;
use super::axis::Axis;
use crate::model::category::CategoryKind;
@@ -27,17 +28,32 @@ pub fn unused_view_name<'a>(existing: impl IntoIterator<Item = &'a str>) -> Stri
.expect("a name is free below usize::MAX, there being fewer views")
}
/// One group's own fold, overriding what it inherits (core.md §19).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum GroupFold {
/// `collapsed: Cat/Group` — closed, whatever reaches it from above.
Collapsed,
/// `expanded(N): Cat/Group` — open, with N-1 levels beneath it. Non-zero by
/// type, since `expanded(0)` is no spelling the grammar reads back.
Expanded(NonZeroUsize),
}
#[derive(Clone)]
pub struct View {
pub name: String,
/// What a person sees; the key until a rename (ADR 007 clause 2).
display: String,
/// Axis assignment for each category
pub category_axes: IndexMap<String, Axis>,
/// For page axis: selected item per category
pub page_selections: HashMap<String, String>,
/// Hidden items per category
pub hidden_items: HashMap<String, HashSet<String>>,
/// Collapsed groups per category
pub collapsed_groups: HashMap<String, HashSet<String>>,
/// How deep each category's hierarchy opens: groups shallower than this start
/// open, the rest closed. Absent is 0 — everything closed (core.md §19).
pub expand_depth: HashMap<String, usize>,
/// Per-group answers that override the depth default, in either direction.
pub group_folds: HashMap<String, HashMap<String, GroupFold>>,
/// Number format string (e.g. ",.0f" for comma-separated integer)
pub number_format: String,
/// When true, empty rows/columns are pruned from the display.
@@ -51,10 +67,12 @@ impl std::fmt::Debug for View {
use std::collections::{BTreeMap, BTreeSet};
let View {
name,
display,
category_axes,
page_selections,
hidden_items,
collapsed_groups,
expand_depth,
group_folds,
number_format,
prune_empty,
} = self;
@@ -64,12 +82,19 @@ impl std::fmt::Debug for View {
.collect()
};
let pages: BTreeMap<_, _> = page_selections.iter().collect();
let depths: BTreeMap<_, _> = expand_depth.iter().collect();
let folds: BTreeMap<_, BTreeMap<_, _>> = group_folds
.iter()
.map(|(k, v)| (k, v.iter().collect()))
.collect();
f.debug_struct("View")
.field("name", name)
.field("display", display)
.field("category_axes", category_axes)
.field("page_selections", &pages)
.field("hidden_items", &sorted(hidden_items))
.field("collapsed_groups", &sorted(collapsed_groups))
.field("expand_depth", &depths)
.field("group_folds", &folds)
.field("number_format", number_format)
.field("prune_empty", prune_empty)
.finish()
@@ -82,13 +107,25 @@ impl View {
VIRTUAL_VIEW_NAMES.contains(&self.name.as_str())
}
pub fn display(&self) -> &str {
&self.display
}
/// Changes what a person sees and nothing else: the key does not move.
pub fn set_display(&mut self, display: impl Into<String>) {
self.display = display.into();
}
pub fn new(name: impl Into<String>) -> Self {
let name = name.into();
Self {
name: name.into(),
display: name.clone(),
name,
category_axes: IndexMap::new(),
page_selections: HashMap::new(),
hidden_items: HashMap::new(),
collapsed_groups: HashMap::new(),
expand_depth: HashMap::new(),
group_folds: HashMap::new(),
// Decimals by default: a ratio is the case a format cannot be
// guessed for, and `.0` draws 2/3 as `1` (improvise-e0ex).
number_format: ",.2".to_string(),
@@ -135,7 +172,8 @@ impl View {
self.category_axes.shift_remove(cat_name);
self.page_selections.remove(cat_name);
self.hidden_items.remove(cat_name);
self.collapsed_groups.remove(cat_name);
self.expand_depth.remove(cat_name);
self.group_folds.remove(cat_name);
}
/// Forget an item this view names. A hidden or paged-to item that outlives
@@ -222,23 +260,78 @@ impl View {
self.page_selections.get(cat_name).map(|s| s.as_str())
}
pub fn toggle_group_collapse(&mut self, cat_name: &str, group_name: &str) {
let set = self
.collapsed_groups
.entry(cat_name.to_string())
.or_default();
if set.contains(group_name) {
set.remove(group_name);
} else {
set.insert(group_name.to_string());
/// How many of `cat_name`'s top levels open (`expanded(N): Cat`); 0 — all
/// closed — if unset.
pub fn expand_depth(&self, cat_name: &str) -> usize {
self.expand_depth.get(cat_name).copied().unwrap_or(0)
}
/// Open `cat_name`'s top `depth` levels; 0 forgets the setting.
pub fn set_expand_depth(&mut self, cat_name: &str, depth: usize) {
match depth {
0 => self.expand_depth.remove(cat_name),
_ => self.expand_depth.insert(cat_name.to_string(), depth),
};
}
/// Whether the group ending `chain` (root first, from `Category::group_chain`)
/// is collapsed: its own fold wins, else what it inherits (core.md §19).
pub fn is_group_collapsed(&self, cat_name: &str, chain: &[&str]) -> bool {
match chain.last().and_then(|g| self.group_fold(cat_name, g)) {
Some(GroupFold::Collapsed) => true,
Some(GroupFold::Expanded(_)) => false,
None => self.inherits_collapsed(cat_name, chain),
}
}
pub fn is_group_collapsed(&self, cat_name: &str, group_name: &str) -> bool {
self.collapsed_groups
.get(cat_name)
.map(|s| s.contains(group_name))
.unwrap_or(false)
/// The answer without the group's own fold: open when the category's depth or
/// an ancestor's `expanded(N)` reaches it, closed otherwise.
fn inherits_collapsed(&self, cat_name: &str, chain: &[&str]) -> bool {
let depth = chain.len().saturating_sub(1);
let by_ancestor = chain[..depth].iter().enumerate().any(|(at, ancestor)| {
matches!(self.group_fold(cat_name, ancestor),
Some(GroupFold::Expanded(n)) if depth - at < n.get())
});
!by_ancestor && depth >= self.expand_depth(cat_name)
}
/// The fold this view holds for one group, if any.
pub fn group_fold(&self, cat_name: &str, group_name: &str) -> Option<GroupFold> {
self.group_folds.get(cat_name)?.get(group_name).copied()
}
/// Record a fold as given — what a file's `collapsed:`/`expanded:` says.
pub fn set_group_fold(&mut self, cat_name: &str, group_name: &str, fold: GroupFold) {
self.group_folds
.entry(cat_name.to_string())
.or_default()
.insert(group_name.to_string(), fold);
}
/// Flip the group ending `chain`: `z`. It writes `Collapsed` or `Expanded(1)`
/// — or nothing, where the flip lands on what the group inherits (§19).
pub fn toggle_group_collapse(&mut self, cat_name: &str, chain: &[&str]) {
let Some(group) = chain.last() else {
return;
};
let collapse = !self.is_group_collapsed(cat_name, chain);
match collapse == self.inherits_collapsed(cat_name, chain) {
true => {
if let Some(folds) = self.group_folds.get_mut(cat_name) {
folds.remove(*group);
if folds.is_empty() {
self.group_folds.remove(cat_name);
}
}
}
false => {
let fold = match collapse {
true => GroupFold::Collapsed,
false => GroupFold::Expanded(NonZeroUsize::MIN),
};
self.set_group_fold(cat_name, group, fold);
}
}
}
pub fn hide_item(&mut self, cat_name: &str, item_name: &str) {
@@ -292,9 +385,18 @@ impl View {
#[cfg(test)]
mod tests {
use super::{View, unused_view_name};
use super::{GroupFold, View, unused_view_name};
use crate::model::category::CategoryKind;
use crate::view::Axis;
use std::num::NonZeroUsize;
#[test]
fn a_view_display_starts_as_the_key_and_changing_it_moves_no_key() {
let mut v = View::new("Default");
assert_eq!(v.display(), "Default");
v.set_display("Overview");
assert_eq!((v.name.as_str(), v.display()), ("Default", "Overview"));
}
/// improvise-vw47: the count is the starting guess, never the answer.
/// What a taken name costs: `context/test-evidence/core.md`.
@@ -563,28 +665,87 @@ mod tests {
assert_eq!(v.page_selection("Region"), None);
}
/// The user's ruling on improvise-felo.6: every group starts closed.
#[test]
fn toggle_group_collapse_toggles_twice() {
fn a_group_starts_collapsed_at_every_depth() {
let v = View::new("Test");
assert!(v.is_group_collapsed("Time", &["2024"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1-2024"]));
}
/// `expanded(1): Time` opens the roots and nothing below them.
#[test]
fn a_category_depth_opens_its_top_levels() {
let mut v = View::new("Test");
assert!(!v.is_group_collapsed("Time", "Q1"));
v.toggle_group_collapse("Time", "Q1");
assert!(v.is_group_collapsed("Time", "Q1"));
v.toggle_group_collapse("Time", "Q1");
assert!(!v.is_group_collapsed("Time", "Q1"));
v.set_expand_depth("Time", 1);
assert!(!v.is_group_collapsed("Time", &["2024"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1-2024"]));
assert!(v.is_group_collapsed("Other", &["G"]), "per category");
v.set_expand_depth("Time", 0);
assert!(v.expand_depth.is_empty(), "0 is the absence, not a setting");
}
/// `expanded(2): Time/2024` opens 2024 and one level beneath it, no deeper,
/// and nothing of the sibling year.
#[test]
fn a_groups_depth_opens_it_and_n_minus_one_levels_beneath() {
let mut v = View::new("Test");
v.set_group_fold(
"Time",
"2024",
GroupFold::Expanded(NonZeroUsize::new(2).unwrap()),
);
assert!(!v.is_group_collapsed("Time", &["2024"]));
assert!(!v.is_group_collapsed("Time", &["2024", "Q1"]));
assert!(v.is_group_collapsed("Time", &["2024", "Q1", "Jan"]));
assert!(v.is_group_collapsed("Time", &["2025", "Q1"]));
}
/// A group's own `collapsed:` wins over an `expanded(N)` that reaches it.
#[test]
fn a_collapsed_group_inside_an_expanded_region_stays_collapsed() {
let mut v = View::new("Test");
v.set_expand_depth("Time", 3);
v.set_group_fold("Time", "Q1", GroupFold::Collapsed);
assert!(v.is_group_collapsed("Time", &["2024", "Q1"]));
assert!(!v.is_group_collapsed("Time", &["2024", "Q2"]));
}
/// `z` writes `Expanded(1)` or `Collapsed`, in BOTH directions, and
/// toggling back leaves no fold behind.
#[test]
fn a_toggle_overrides_what_a_group_inherits_either_way_and_toggling_back_clears_it() {
let mut v = View::new("Test");
v.set_expand_depth("Time", 1);
v.toggle_group_collapse("Time", &["2024", "Q1-2024"]);
assert_eq!(
v.group_fold("Time", "Q1-2024"),
Some(GroupFold::Expanded(NonZeroUsize::MIN))
);
v.toggle_group_collapse("Time", &["2024"]);
assert_eq!(v.group_fold("Time", "2024"), Some(GroupFold::Collapsed));
assert!(
!v.is_group_collapsed("Time", &["2025"]),
"a sibling is untouched"
);
v.toggle_group_collapse("Time", &["2024", "Q1-2024"]);
v.toggle_group_collapse("Time", &["2024"]);
assert!(v.group_folds.is_empty(), "{:?}", v.group_folds);
}
#[test]
fn is_group_collapsed_isolated_across_categories() {
let mut v = View::new("Test");
v.toggle_group_collapse("Cat1", "G1");
assert!(!v.is_group_collapsed("Cat2", "G1"));
v.toggle_group_collapse("Cat1", &["G1"]);
assert!(!v.is_group_collapsed("Cat1", &["G1"]));
assert!(v.is_group_collapsed("Cat2", &["G1"]));
}
#[test]
fn is_group_collapsed_isolated_across_groups() {
let mut v = View::new("Test");
v.toggle_group_collapse("Cat1", "G1");
assert!(!v.is_group_collapsed("Cat1", "G2"));
v.toggle_group_collapse("Cat1", &["G1"]);
assert!(v.is_group_collapsed("Cat1", &["G2"]));
}
/// Arm 2 of improvise-qx1z: the other three lines of `on_category_removed`
@@ -593,12 +754,13 @@ mod tests {
fn on_category_removed_forgets_a_collapsed_group() {
let mut v = View::new("V");
v.on_category_added("Month", CategoryKind::of_name("Month"));
v.toggle_group_collapse("Month", "Q1");
v.set_expand_depth("Month", 1);
v.toggle_group_collapse("Month", &["Q1"]);
v.on_category_removed("Month");
v.on_category_added("Month", CategoryKind::of_name("Month"));
assert!(
!v.is_group_collapsed("Month", "Q1"),
"a re-added category came back with its group collapsed"
v.group_folds.is_empty() && v.expand_depth.is_empty(),
"a re-added category came back with its old fold state"
);
}
@@ -808,10 +970,10 @@ mod prop_tests {
group in "[A-Za-z][a-z]{1,7}",
) {
let mut v = View::new("T");
let initial = v.is_group_collapsed(&cat, &group);
v.toggle_group_collapse(&cat, &group);
v.toggle_group_collapse(&cat, &group);
prop_assert_eq!(v.is_group_collapsed(&cat, &group), initial);
let initial = v.is_group_collapsed(&cat, &[group.as_str()]);
v.toggle_group_collapse(&cat, &[group.as_str()]);
v.toggle_group_collapse(&cat, &[group.as_str()]);
prop_assert_eq!(v.is_group_collapsed(&cat, &[group.as_str()]), initial);
}
}
}
crates/improvise-io/src/import/analyzer.rs
+27
View File
@@ -22,6 +22,26 @@ pub enum DateComponent {
Quarter,
}
impl DateComponent {
/// The components as a hierarchy nests them, coarsest first.
pub const COARSE_TO_FINE: [DateComponent; 3] = [
DateComponent::Year,
DateComponent::Quarter,
DateComponent::Month,
];
}
/// How a date column's components become categories (improvise-8jk6.3).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum DateShape {
/// ONE category named after the column: its items are the finest chosen
/// component, grouped by the coarser ones. The user's default.
#[default]
Hierarchy,
/// The column's raw values, plus one flat category per chosen component.
Split,
}
#[derive(Debug, Clone)]
pub struct FieldProposal {
pub field: String,
@@ -32,6 +52,8 @@ pub struct FieldProposal {
pub date_format: Option<String>,
/// Which date components to extract as new categories.
pub date_components: Vec<DateComponent>,
/// Whether those components nest in one category or split into several.
pub date_shape: DateShape,
}
impl FieldProposal {
@@ -132,6 +154,7 @@ pub fn analyze_records(records: &[Value]) -> Vec<FieldProposal> {
accepted: true,
date_format: None,
date_components: vec![],
date_shape: DateShape::default(),
};
}
@@ -159,6 +182,7 @@ pub fn analyze_records(records: &[Value]) -> Vec<FieldProposal> {
accepted: true,
date_format,
date_components: vec![],
date_shape: DateShape::default(),
};
}
@@ -170,6 +194,7 @@ pub fn analyze_records(records: &[Value]) -> Vec<FieldProposal> {
accepted: true,
date_format: None,
date_components: vec![],
date_shape: DateShape::default(),
};
}
@@ -180,6 +205,7 @@ pub fn analyze_records(records: &[Value]) -> Vec<FieldProposal> {
accepted: true,
date_format: None,
date_components: vec![],
date_shape: DateShape::default(),
};
}
@@ -191,6 +217,7 @@ pub fn analyze_records(records: &[Value]) -> Vec<FieldProposal> {
accepted: true,
date_format: None,
date_components: vec![],
date_shape: DateShape::default(),
}
})
.collect()
crates/improvise-io/src/import/csv_parser.rs
+288 -51
View File
@@ -1,18 +1,38 @@
use std::collections::HashSet;
use std::path::Path;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use anyhow::{Context, Result, ensure};
use csv::ReaderBuilder;
use serde_json::Value;
use crate::storage::{PhysicalFs, Storage};
/// Parsed CSV records. Every value is its source text, so they reach an import
/// only through `ImportPipeline::from_csv` (improvise-8jk6.4).
#[derive(Debug, Clone, Default, PartialEq)]
pub struct CsvRecords(Vec<Value>);
impl std::ops::Deref for CsvRecords {
type Target = [Value];
fn deref(&self) -> &[Value] {
&self.0
}
}
impl CsvRecords {
pub(crate) fn into_values(self) -> Vec<Value> {
self.0
}
}
pub fn csv_path_p(path: &Path) -> bool {
path.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("csv"))
}
/// Parse a CSV file into a `serde_json::Value` array. Short-row warnings go to
/// stderr; [`parse_csv_with_warnings`] captures them instead.
pub fn parse_csv(path: &Path) -> Result<Vec<Value>> {
/// Parse a CSV file into records. Short-row warnings go to stderr;
/// [`parse_csv_with_warnings`] captures them instead.
pub fn parse_csv(path: &Path) -> Result<CsvRecords> {
let (records, warnings) = parse_csv_with_warnings(path)?;
for warning in &warnings {
eprintln!("warning: {warning}");
@@ -22,7 +42,7 @@ pub fn parse_csv(path: &Path) -> Result<Vec<Value>> {
/// Parse a CSV file, returning records plus warnings. A short row keeps every
/// header column, missing ones `Null`, each reported with a row count.
pub fn parse_csv_with_warnings(path: &Path) -> Result<(Vec<Value>, Vec<String>)> {
pub fn parse_csv_with_warnings(path: &Path) -> Result<(CsvRecords, Vec<String>)> {
let bytes = std::fs::read(path)
.with_context(|| format!("Failed to open CSV file: {}", path.display()))?;
parse_csv_bytes_with_warnings(&bytes)
@@ -30,7 +50,13 @@ pub fn parse_csv_with_warnings(path: &Path) -> Result<(Vec<Value>, Vec<String>)>
/// As [`parse_csv_with_warnings`] but over bytes, so import can come from any
/// [`Storage`](crate::storage::Storage) backend, not only a file (improvise-fuj).
pub fn parse_csv_bytes_with_warnings(bytes: &[u8]) -> Result<(Vec<Value>, Vec<String>)> {
pub fn parse_csv_bytes_with_warnings(bytes: &[u8]) -> Result<(CsvRecords, Vec<String>)> {
let (_, records, warnings) = parse_csv_bytes(bytes)?;
Ok((CsvRecords(records), warnings))
}
/// The header (after renaming), the records and the warnings.
fn parse_csv_bytes(bytes: &[u8]) -> Result<(Vec<String>, Vec<Value>, Vec<String>)> {
let mut reader = ReaderBuilder::new()
.has_headers(true)
.flexible(true)
@@ -110,7 +136,7 @@ pub fn parse_csv_bytes_with_warnings(bytes: &[u8]) -> Result<(Vec<Value>, Vec<St
}),
);
Ok((records, warnings))
Ok((headers, records, warnings))
}
/// A header name keys the record object, so two equal names were one column and
@@ -144,8 +170,42 @@ fn disambiguate_headers(raw: Vec<String>) -> (Vec<String>, Vec<String>) {
/// Parse multiple CSV files and merge into a single JSON array.
/// Each record gets a "File" field set to the filename stem (e.g., "sales" from "sales.csv").
pub fn merge_csvs(paths: &[impl AsRef<Path>]) -> Result<Vec<Value>> {
let mut all_records = Vec::new();
pub fn merge_csvs(paths: &[impl AsRef<Path>]) -> Result<CsvRecords> {
let merged = merge_csvs_from(&PhysicalFs, paths)?;
for warning in &merged.warnings {
eprintln!("warning: {warning}");
}
Ok(merged.records)
}
/// The category every merged record names its source file by.
pub const FILE_CATEGORY: &str = "File";
/// One merged file: its `File` item and the columns its header named.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SourceFile {
pub item: String,
pub columns: Vec<String>,
}
/// Several CSV files as one record set, with each file's own columns kept.
#[derive(Debug, Clone, PartialEq)]
pub struct Merged {
pub records: CsvRecords,
/// One per distinct `File` item, in the order the files were read.
pub sources: Vec<SourceFile>,
/// Each file's parse warnings, prefixed with its path.
pub warnings: Vec<String>,
}
/// As [`merge_csvs`] over any [`Storage`]. Files may have different headers,
/// and two files with one stem share its item and its columns (improvise-lpzu).
pub fn merge_csvs_from(fs: &dyn Storage, paths: &[impl AsRef<Path>]) -> Result<Merged> {
let mut merged = Merged {
records: CsvRecords::default(),
sources: Vec::new(),
warnings: Vec::new(),
};
for path in paths {
let path = path.as_ref();
let stem = path
@@ -153,37 +213,104 @@ pub fn merge_csvs(paths: &[impl AsRef<Path>]) -> Result<Vec<Value>> {
.and_then(|s| s.to_str())
.unwrap_or("unknown")
.to_string();
let records = parse_csv(path)?;
let bytes = fs.read(path)?;
let (headers, records, warnings) = parse_csv_bytes(&bytes)
.with_context(|| format!("Failed to parse CSV file: {}", path.display()))?;
let at = merged.source_for(&stem);
let columns = &mut merged.sources[at].columns;
for header in headers {
if header != FILE_CATEGORY && !columns.contains(&header) {
columns.push(header);
}
}
merged
.warnings
.extend(warnings.iter().map(|w| format!("{}: {w}", path.display())));
for mut record in records {
if let Value::Object(ref mut map) = record {
map.insert("File".to_string(), Value::String(stem.clone()));
map.insert(FILE_CATEGORY.to_string(), Value::String(stem.clone()));
}
all_records.push(record);
merged.records.0.push(record);
}
}
Ok(all_records)
Ok(merged)
}
impl Merged {
fn source_for(&mut self, item: &str) -> usize {
match self.sources.iter().position(|s| s.item == item) {
Some(at) => at,
None => {
self.sources.push(SourceFile {
item: item.to_string(),
columns: vec![],
});
self.sources.len() - 1
}
}
}
}
/// One `improvise import` argument. Whether a path names a folder is the
/// host's question, so the caller classifies and [`expand_import_paths`] lists.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ImportPath {
File(PathBuf),
Folder(PathBuf),
}
/// The files an import reads: each argument in turn, a folder standing for its
/// `*.csv` documents in name order. See `context/code/formula-and-io.md`.
pub fn expand_import_paths(fs: &dyn Storage, args: &[ImportPath]) -> Result<Vec<PathBuf>> {
let mut paths = Vec::new();
for arg in args {
match arg {
ImportPath::File(path) => paths.push(path.clone()),
ImportPath::Folder(dir) => paths.extend(csv_documents_in(fs, dir)?),
}
}
Ok(paths)
}
/// A folder's `*.csv` documents, sorted by `Storage::list`'s contract. A folder
/// with none is refused by name rather than imported as nothing.
fn csv_documents_in(fs: &dyn Storage, dir: &Path) -> Result<Vec<PathBuf>> {
let csvs: Vec<PathBuf> = fs
.list(dir)?
.into_iter()
.filter(|p| csv_path_p(p))
.collect();
ensure!(!csvs.is_empty(), "no CSV files in folder {}", dir.display());
Ok(csvs)
}
/// A field is its source text: which ones are numbers is decided by the role
/// the import gives the column (improvise-8jk6.4, `context/code/formula-and-io.md`).
fn parse_csv_field(field: &str) -> Value {
if field.is_empty() {
return Value::Null;
match field.is_empty() {
true => Value::Null,
false => Value::String(field.to_string()),
}
}
// Try to parse as number (integer or float)
if let Ok(num) = field.parse::<i64>() {
return Value::Number(serde_json::Number::from(num));
/// The number a CSV field's text reads as, where a number is wanted.
pub fn number_in(text: &str) -> Option<serde_json::Number> {
if let Ok(num) = text.parse::<i64>() {
return Some(serde_json::Number::from(num));
}
// NaN and the infinities have no JSON number. A fabricated 0 is an invented
// measurement (improvise-jc95), so the token falls through and stays text.
if let Ok(num) = field.parse::<f64>()
&& let Some(number) = serde_json::Number::from_f64(num)
{
return Value::Number(number);
}
// measurement (improvise-jc95), so the token stays text.
text.parse::<f64>()
.ok()
.and_then(serde_json::Number::from_f64)
}
// Otherwise treat as string
Value::String(field.to_string())
/// A CSV value as a typed JSON value: numeric text becomes its number.
pub fn typed(value: &Value) -> Value {
match value.as_str().and_then(number_in) {
Some(number) => Value::Number(number),
None => value.clone(),
}
}
#[cfg(test)]
@@ -208,8 +335,9 @@ mod tests {
assert_eq!(records.len(), 2);
assert_eq!(records[0]["Region"], Value::String("East".to_string()));
assert_eq!(records[0]["Product"], Value::String("Shirts".to_string()));
assert_eq!(records[0]["Revenue"], Value::String("1000".to_string()));
assert_eq!(
records[0]["Revenue"],
typed(&records[0]["Revenue"]),
Value::Number(serde_json::Number::from(1000))
);
}
@@ -221,9 +349,10 @@ mod tests {
let records = parse_csv(&path).unwrap();
assert_eq!(records.len(), 2);
assert!(records[0]["Revenue"].is_f64());
assert_eq!(records[0]["Revenue"], Value::String("1000.50".to_string()));
assert!(typed(&records[0]["Revenue"]).is_f64());
assert_eq!(
records[0]["Revenue"],
typed(&records[0]["Revenue"]),
Value::Number(serde_json::Number::from_f64(1000.50).unwrap())
);
}
@@ -261,10 +390,10 @@ mod tests {
assert_eq!(records.len(), 2);
assert_eq!(records[0]["Name"], Value::String("Widget".to_string()));
assert_eq!(
records[0]["Count"],
typed(&records[0]["Count"]),
Value::Number(serde_json::Number::from(5))
);
assert!(records[0]["Price"].is_f64());
assert!(typed(&records[0]["Price"]).is_f64());
assert_eq!(records[0]["Active"], Value::String("true".to_string()));
}
@@ -284,10 +413,7 @@ mod tests {
assert_eq!(records[3]["File"], Value::String("expenses".to_string()));
// Original fields preserved
assert_eq!(records[0]["Region"], Value::String("East".to_string()));
assert_eq!(
records[2]["Revenue"],
Value::Number(serde_json::Number::from(50))
);
assert_eq!(records[2]["Revenue"], Value::String("50".to_string()));
}
#[test]
@@ -302,6 +428,118 @@ mod tests {
assert_eq!(records[0]["Name"], Value::String("A".to_string()));
}
// ── Folder arguments (improvise-lpzu) ─────────────────────────────
use crate::storage::MemoryFs;
fn folder(files: &[(&str, &str)]) -> MemoryFs {
let fs = MemoryFs::new();
for (path, body) in files {
fs.write(Path::new(path), body.as_bytes()).unwrap();
}
fs
}
fn names(paths: &[PathBuf]) -> Vec<String> {
paths.iter().map(|p| p.display().to_string()).collect()
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_folder_expands_to_its_csv_files_in_name_order_and_skips_the_rest() {
let fs = folder(&[
("maat/coupling.csv", "entity\n"),
("maat/notes.txt", "not data"),
("maat/age.CSV", "entity\n"),
("maat/out.improv", "v2025-04-09\n"),
("maat/abs-churn.csv", "date\n"),
("elsewhere/other.csv", "x\n"),
]);
let paths = expand_import_paths(&fs, &[ImportPath::Folder("maat".into())]).unwrap();
assert_eq!(
names(&paths),
["maat/abs-churn.csv", "maat/age.CSV", "maat/coupling.csv"]
);
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_folder_with_no_csv_files_is_refused_by_name() {
let only_text = folder(&[("maat/notes.txt", "x"), ("maat/out.improv", "y")]);
let empty = MemoryFs::new();
for fs in [&only_text, &empty] {
let err = expand_import_paths(fs, &[ImportPath::Folder("maat".into())])
.expect_err("a folder with nothing to import is refused");
assert_eq!(err.to_string(), "no CSV files in folder maat");
}
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn files_and_folders_expand_in_argument_order() {
let fs = folder(&[
("b/two.csv", "x\n"),
("b/one.csv", "x\n"),
("a.csv", "x\n"),
("z.csv", "x\n"),
]);
let args = [
ImportPath::File("z.csv".into()),
ImportPath::Folder("b".into()),
ImportPath::File("a.csv".into()),
];
let paths = expand_import_paths(&fs, &args).unwrap();
assert_eq!(names(&paths), ["z.csv", "b/one.csv", "b/two.csv", "a.csv"]);
}
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn files_with_different_headers_merge_without_inventing_the_missing_columns() {
let fs = folder(&[
(
"m/coupling.csv",
"entity,coupled,degree\nsrc/a.rs,src/b.rs,80\n",
),
("m/age.csv", "entity,age-months\nsrc/a.rs,2\n"),
]);
let merged = merge_csvs_from(&fs, &["m/coupling.csv", "m/age.csv"]).unwrap();
assert!(merged.warnings.is_empty(), "{:?}", merged.warnings);
let source = |item: &str, columns: &[&str]| SourceFile {
item: item.to_string(),
columns: columns.iter().map(|c| c.to_string()).collect(),
};
assert_eq!(
merged.sources,
[
source("coupling", &["entity", "coupled", "degree"]),
source("age", &["entity", "age-months"]),
]
);
assert_eq!(
*merged.records,
[
serde_json::json!({"entity": "src/a.rs", "coupled": "src/b.rs", "degree": "80", "File": "coupling"}),
serde_json::json!({"entity": "src/a.rs", "age-months": "2", "File": "age"}),
]
);
}
/// improvise-8jk6.12 (3): two files with one stem are one item, and its
/// columns are both files'. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn two_files_with_one_stem_union_their_columns() {
let fs = folder(&[
("a/x.csv", "entity,one\ne,1\n"),
("b/x.csv", "entity,two\ne,2\n"),
]);
let merged = merge_csvs_from(&fs, &["a/x.csv", "b/x.csv"]).unwrap();
let expected = SourceFile {
item: "x".to_string(),
columns: ["entity", "one", "two"].map(String::from).to_vec(),
};
assert_eq!(merged.sources, [expected]);
}
// ── RFC 4180 edge cases ───────────────────────────────────────────
#[test]
@@ -442,10 +680,7 @@ mod tests {
assert!(warnings.is_empty());
assert_eq!(records.len(), 2);
assert_eq!(records[0]["Region"], Value::String("East".to_string()));
assert_eq!(
records[0]["Revenue"],
Value::Number(serde_json::Number::from(1))
);
assert_eq!(records[0]["Revenue"], Value::String("1".to_string()));
assert_eq!(records[1]["Region"], Value::String("West".to_string()));
let (control, _) = parse_csv_bytes_with_warnings(b"Region,Revenue\nEa\rst,1\n").unwrap();
@@ -500,7 +735,8 @@ mod tests {
token.parse::<f64>().is_ok_and(|n| !n.is_finite()),
"precondition: {token} parses as a non-finite f64"
);
assert_eq!(parse_csv_field(token), Value::String(token.to_string()));
let text = Value::String(token.to_string());
assert_eq!(typed(&text), text);
}
}
@@ -509,10 +745,10 @@ mod tests {
#[test]
fn an_ordinary_float_is_still_a_number() {
assert_eq!(
parse_csv_field("1000.50"),
typed(&Value::String("1000.50".into())),
Value::Number(serde_json::Number::from_f64(1000.50).unwrap())
);
assert_eq!(parse_csv_field("0"), Value::Number(0.into()));
assert_eq!(typed(&Value::String("0".into())), Value::Number(0.into()));
}
/// improvise-5hfh: two equal header names are one JSON key, so the later
@@ -521,9 +757,9 @@ mod tests {
fn a_repeated_header_is_renamed_rather_than_overwriting_its_twin() {
let (records, warnings) = parse_csv_bytes_with_warnings(b"A,B,A\n1,2,3\n").unwrap();
let map = records[0].as_object().unwrap();
assert_eq!(map["A"], Value::Number(1.into()));
assert_eq!(map["B"], Value::Number(2.into()));
assert_eq!(map["A_2"], Value::Number(3.into()));
assert_eq!(map["A"], Value::String("1".into()));
assert_eq!(map["B"], Value::String("2".into()));
assert_eq!(map["A_2"], Value::String("3".into()));
assert_eq!(warnings.len(), 1);
assert!(warnings[0].contains("'A'") && warnings[0].contains("'A_2'"));
assert!(warnings[0].contains("column 3"), "{}", warnings[0]);
@@ -536,8 +772,8 @@ mod tests {
let (records, _) = parse_csv_bytes_with_warnings(b"A,A_2,A\n1,2,3\n").unwrap();
let map = records[0].as_object().unwrap();
assert_eq!(map.len(), 3, "{map:?}");
assert_eq!(map["A_2"], Value::Number(2.into()));
assert_eq!(map["A_3"], Value::Number(3.into()));
assert_eq!(map["A_2"], Value::String("2".into()));
assert_eq!(map["A_3"], Value::String("3".into()));
}
#[test]
@@ -572,8 +808,9 @@ mod tests {
assert_eq!(records.len(), 2);
assert_eq!(records[0]["Date"], Value::String("03/31/2026".to_string()));
assert_eq!(records[0]["Amount"], Value::String("-50.00".to_string()));
assert_eq!(
records[0]["Amount"],
typed(&records[0]["Amount"]),
Value::Number(serde_json::Number::from_f64(-50.00).unwrap())
);
assert_eq!(records[0]["Flag"], Value::String("*".to_string()));
@@ -583,7 +820,7 @@ mod tests {
Value::String("VENMO PAYMENT 260331".to_string())
);
assert_eq!(
records[1]["Amount"],
typed(&records[1]["Amount"]),
Value::Number(serde_json::Number::from_f64(-240.00).unwrap())
);
}
crates/improvise-io/src/import/wizard.rs
+1256 -164
View File
File diff suppressed because it is too large Load Diff
crates/improvise-io/src/persistence/export.rs
+5 -4
View File
@@ -22,9 +22,9 @@ pub fn export_csv_to(
storage.write(path, &render_csv(workbook, view_name)?)
}
/// Render a view as CSV bytes. Separated from writing so the bytes can go to
/// any [`Storage`] backend, not just a file (improvise-fuj).
fn render_csv(workbook: &Workbook, view_name: &str) -> Result<Vec<u8>> {
/// Render a view as CSV bytes for any [`Storage`] backend or a download frame
/// (improvise-fuj, improvise-2mqz).
pub fn render_csv(workbook: &Workbook, view_name: &str) -> Result<Vec<u8>> {
let view = workbook
.views
.get(view_name)
@@ -214,6 +214,7 @@ mod tests {
wb.add_category("Type").unwrap();
wb.add_item_in_group("Month", "Jan", "Q1").unwrap();
wb.add_item_in_group("Month", "Feb", "Q1").unwrap();
wb.active_view_mut().set_expand_depth("Month", 1);
wb.add_item("Type", "Food").unwrap();
for (month, value) in [("Jan", 3.0), ("Feb", 4.0)] {
wb.model.set_cell(
@@ -226,7 +227,7 @@ mod tests {
v.set_axis("Month", Axis::Row);
v.set_axis("Type", Axis::Column);
v.prune_empty = true;
v.toggle_group_collapse("Month", "Q1");
v.toggle_group_collapse("Month", &["Q1"]);
}
let csv = String::from_utf8(super::render_csv(&wb, "Default").unwrap()).unwrap();
let rows: Vec<&str> = csv.lines().skip(1).collect();
crates/improvise-io/src/persistence/format.rs
+34 -15
View File
@@ -2,7 +2,7 @@
use crate::model::category::CategoryKind;
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
use crate::view::{Axis, GroupFold};
use crate::workbook::Workbook;
// The one home of the pipe-quoting rule (improvise-i2m5); a name written here
@@ -119,19 +119,37 @@ pub fn format_md(workbook: &Workbook) -> String {
if view.prune_empty {
w!(out, "prune: true");
}
for (prefix, map) in [
("hidden", &view.hidden_items),
("collapsed", &view.collapsed_groups),
] {
let mut pairs: Vec<_> = map
.iter()
.flat_map(|(cat, items)| {
items.iter().map(move |item| (cat.as_str(), item.as_str()))
})
.collect();
pairs.sort();
for (cat, item) in pairs {
w!(out, "{prefix}: {}/{}", quote_name(cat), quote_name(item));
let mut hidden: Vec<_> = view
.hidden_items
.iter()
.flat_map(|(cat, items)| items.iter().map(move |item| (cat.as_str(), item.as_str())))
.collect();
hidden.sort();
for (cat, item) in hidden {
w!(out, "hidden: {}/{}", quote_name(cat), quote_name(item));
}
let mut depths: Vec<_> = view.expand_depth.iter().collect();
depths.sort();
// Never bare: `expanded: row` would read back as an axis line.
for (cat, depth) in depths {
w!(out, "expanded({depth}): {}", quote_name(cat));
}
let mut folds: Vec<_> = view
.group_folds
.iter()
.flat_map(|(cat, groups)| {
groups
.iter()
.map(move |(g, fold)| (cat.as_str(), g.as_str(), *fold))
})
.collect();
folds.sort();
for (cat, group, fold) in folds {
let target = format!("{}/{}", quote_name(cat), quote_name(group));
match fold {
GroupFold::Collapsed => w!(out, "collapsed: {target}"),
GroupFold::Expanded(n) if n.get() == 1 => w!(out, "expanded: {target}"),
GroupFold::Expanded(n) => w!(out, "expanded({n}): {target}"),
}
}
}
@@ -370,7 +388,8 @@ mod tests {
#[test]
fn format_md_collapsed_group() {
let mut m = two_cat_model();
m.active_view_mut().toggle_group_collapse("Type", "MyGroup");
m.active_view_mut()
.set_group_fold("Type", "MyGroup", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
assert!(text.contains("collapsed: Type/MyGroup"));
}
crates/improvise-io/src/persistence/improv.pest
+9 -1
View File
@@ -120,7 +120,9 @@ view_section = {
view_entry*
}
view_entry = _{ format_line | prune_line | hidden_line | collapsed_line | axis_line }
// `axis_line` is tried before `expanded_line`, so a category named `expanded`
// keeps its `expanded: row`; the writer never spells a category depth bare.
view_entry = _{ format_line | prune_line | hidden_line | collapsed_line | axis_line | expanded_line }
axis_line = {
name ~ ": " ~ axis_kind ~ (", " ~ name)? ~ NEWLINE ~ blank_lines
@@ -135,6 +137,12 @@ prune_line = { "prune: " ~ bool_kw ~ NEWLINE ~ blank_lines }
bool_kw = @{ "true" | "false" }
hidden_line = { "hidden: " ~ name ~ "/" ~ name ~ NEWLINE ~ blank_lines }
collapsed_line = { "collapsed: " ~ name ~ "/" ~ name ~ NEWLINE ~ blank_lines }
// Groups start collapsed. `expanded(N): Cat` opens the category's top N levels;
// `expanded(N): Cat/Group` opens the group and N-1 levels beneath it. A bare
// `expanded:` is N = 1, and N is at least 1. context/code/formula-and-io.md.
expanded_line = { "expanded" ~ fold_depth? ~ ": " ~ name ~ ("/" ~ name)? ~ NEWLINE ~ blank_lines }
fold_depth = { "(" ~ depth_number ~ ")" }
depth_number = @{ '1'..'9' ~ ASCII_DIGIT* }
// ── Names ────────────────────────────────────────────────────────────────────
//
crates/improvise-io/src/persistence/mod.rs
+1 -1
View File
@@ -36,7 +36,7 @@ mod test_helpers {
}
}
pub use export::{export_csv, export_csv_to};
pub use export::{export_csv, export_csv_to, render_csv};
pub use format::format_md;
pub use parse::parse_md;
crates/improvise-io/src/persistence/parse.rs
+161 -13
View File
@@ -9,10 +9,10 @@ use pest_derive::Parser;
use crate::formula::{AggFunc, parse_formula};
use crate::model::category::{CategoryKind, Group};
use crate::model::cell::{CellKey, CellValue};
use crate::view::Axis;
use crate::view::{Axis, GroupFold};
use crate::workbook::Workbook;
use crate::formula::quote::unescape_pipe_body;
use crate::formula::quote::{quote_name, unescape_pipe_body};
#[derive(Parser)]
#[grammar = "persistence/improv.pest"]
@@ -40,9 +40,74 @@ struct PView {
format: String,
prune_empty: bool,
hidden: Vec<(String, String)>,
collapsed: Vec<(String, String)>,
folds: Vec<PFold>,
}
/// One `collapsed:` or `expanded(N):` line. `group` is `None` for the category
/// form, which only `expanded` has.
struct PFold {
category: String,
group: Option<String>,
fold: GroupFold,
}
/// A file the grammar accepts and a model cannot hold (improvise-felo.6).
/// Typed, so the generator's property can tell it from a parse failure.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Refusal {
/// `> A[B]` and `> B[A]`: a hierarchy must be a forest.
HierarchyCycle {
category: String,
groups: Vec<String>,
},
/// One view gives one group (or one category's depth) two different folds.
FoldConflict {
view: String,
category: String,
group: Option<String>,
},
}
impl std::fmt::Display for Refusal {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Refusal::HierarchyCycle { category, groups } => write!(
f,
"category {}: the group parents form a cycle through {}",
quote_name(category),
groups
.iter()
.map(|g| quote_name(g))
.collect::<Vec<_>>()
.join(" > ")
),
Refusal::FoldConflict {
view,
category,
group: Some(group),
} => write!(
f,
"view {}: group {}/{} is both expanded and collapsed, or expanded to two depths",
quote_name(view),
quote_name(category),
quote_name(group)
),
Refusal::FoldConflict {
view,
category,
group: None,
} => write!(
f,
"view {}: category {} is expanded to two depths",
quote_name(view),
quote_name(category)
),
}
}
}
impl std::error::Error for Refusal {}
/// Everything pass 1 collects, before any of it is resolved against the model.
#[derive(Default)]
struct ParsedFile {
@@ -332,7 +397,31 @@ fn walk_axis_line(entry: Pair<'_, Rule>, pv: &mut PView) -> Result<()> {
Ok(())
}
/// `## View: Name` plus its axis, format, hidden and collapsed lines.
/// `expanded(N): Cat` or `expanded(N): Cat/Group`; a bare `expanded:` is N = 1.
fn walk_expanded_line(entry: Pair<'_, Rule>) -> Result<PFold> {
let mut depth = std::num::NonZeroUsize::MIN;
let mut names = Vec::new();
for part in entry.into_inner() {
match part.as_rule() {
Rule::fold_depth => {
let digits = first_str(part)?;
depth = digits
.parse()
.with_context(|| format!("expanded({digits}): depth out of range"))?;
}
_ => names.push(extract_name(part)?),
}
}
let mut names = names.into_iter();
let category = names.next().context("expanded_line: no category")?;
Ok(PFold {
category,
group: names.next(),
fold: GroupFold::Expanded(depth),
})
}
/// `## View: Name` plus its axis, format, hidden, collapsed and expanded lines.
fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
let mut inner = pair.into_inner();
let mut pv = PView {
@@ -342,7 +431,7 @@ fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
format: String::new(),
prune_empty: false,
hidden: Vec::new(),
collapsed: Vec::new(),
folds: Vec::new(),
};
for entry in inner {
match entry.as_rule() {
@@ -350,7 +439,15 @@ fn walk_view_section(pair: Pair<'_, Rule>) -> Result<PView> {
Rule::format_line => pv.format = first_str(entry)?,
Rule::prune_line => pv.prune_empty = first_str(entry)? == "true",
Rule::hidden_line => pv.hidden.push(extract_name_pair(entry)?),
Rule::collapsed_line => pv.collapsed.push(extract_name_pair(entry)?),
Rule::collapsed_line => {
let (category, group) = extract_name_pair(entry)?;
pv.folds.push(PFold {
category,
group: Some(group),
fold: GroupFold::Collapsed,
});
}
Rule::expanded_line => pv.folds.push(walk_expanded_line(entry)?),
other => bail!("unhandled rule in view_section: {other:?}"),
}
}
@@ -411,6 +508,13 @@ fn build_category(wb: &mut Workbook, pc: &PCategory) -> Result<()> {
None => cat.add_group(Group::new(group_name).with_parent(parent)),
}
}
if let Some(cycle) = cat.hierarchy_cycle() {
return Err(Refusal::HierarchyCycle {
category: pc.name.clone(),
groups: cycle.into_iter().map(String::from).collect(),
}
.into());
}
Ok(())
}
@@ -436,8 +540,29 @@ fn build_view(wb: &mut Workbook, pv: &PView) -> Result<()> {
for (cat, item) in &pv.hidden {
view.hide_item(cat, item);
}
for (cat, grp) in &pv.collapsed {
view.toggle_group_collapse(cat, grp);
for PFold {
category,
group,
fold,
} in &pv.folds
{
let conflict = || Refusal::FoldConflict {
view: pv.name.clone(),
category: category.clone(),
group: group.clone(),
};
match (group, fold) {
(Some(g), _) => match view.group_fold(category, g) {
Some(held) if held != *fold => return Err(conflict().into()),
_ => view.set_group_fold(category, g, *fold),
},
(None, GroupFold::Expanded(n)) => match view.expand_depth(category) {
0 => view.set_expand_depth(category, n.get()),
held if held != n.get() => return Err(conflict().into()),
_ => {}
},
(None, GroupFold::Collapsed) => bail!("a category-wide collapse has no spelling"),
}
}
Ok(())
}
@@ -534,6 +659,7 @@ mod grammar_coverage {
Rule::axis_line | Rule::axis_kind => "walk_axis_line",
Rule::format_line | Rule::prune_line | Rule::bool_kw => "walk_view_section",
Rule::hidden_line | Rule::collapsed_line => "walk_view_section",
Rule::expanded_line | Rule::fold_depth | Rule::depth_number => "walk_expanded_line",
Rule::view_entry => "silent: hoisted into view_section",
Rule::name => "silent: hoisted into its parent",
Rule::pipe_quoted | Rule::pipe_inner | Rule::bare_name => "extract_name",
@@ -1059,12 +1185,14 @@ mod parser_edge_cases {
fn collapsed_group_with_slash_in_name() {
let mut m = Workbook::new("Test");
m.add_category("Type").unwrap();
m.active_view_mut().toggle_group_collapse("Type", "Q1/Q2");
m.active_view_mut()
.set_group_fold("Type", "Q1/Q2", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
let loaded = parse_md(&text).unwrap();
assert!(
loaded.active_view().is_group_collapsed("Type", "Q1/Q2"),
loaded.active_view().group_fold("Type", "Q1/Q2")
== Some(crate::view::GroupFold::Collapsed),
"Collapsed group with '/' in name was corrupted.\n{text}"
);
}
@@ -1604,6 +1732,24 @@ mod generator {
})
}
/// improvise-felo.6: the round-trip properties see `expanded(N):` lines only
/// if the walk emits them, depths included.
#[test]
fn the_generator_writes_expanded_lines_with_and_without_a_depth() {
use proptest::strategy::ValueTree;
use proptest::test_runner::TestRunner;
let mut runner = TestRunner::deterministic();
let strategy = improv_file();
let files: Vec<String> = (0..512)
.map(|_| strategy.new_tree(&mut runner).unwrap().current())
.collect();
let lines = || files.iter().flat_map(|f| f.lines());
assert!(lines().any(|l| l.starts_with("expanded(")), "a depth");
assert!(lines().any(|l| l.starts_with("expanded: ")), "a bare line");
assert!(lines().any(|l| l.starts_with("collapsed: ")), "a collapse");
}
/// improvise-8shu. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn the_grammar_is_parsed_once_however_many_files_the_strategy_yields() {
@@ -1625,7 +1771,7 @@ mod generator {
#[cfg(test)]
mod grammar_prop_tests {
use super::{generator, parse_md};
use super::{Refusal, generator, parse_md};
use crate::persistence::format_md;
use proptest::prelude::*;
@@ -1633,11 +1779,13 @@ mod grammar_prop_tests {
proptest! {
#![proptest_config(ProptestConfig::with_cases(256))]
/// parse(generate()) — every generated file parses without error.
/// parse(generate()) — every generated file parses, or is refused for a
/// named reason the grammar cannot express (a `Refusal`, improvise-felo.6).
#[test]
fn generated_file_parses(file in generator::improv_file()) {
let result = parse_md(&file);
prop_assert!(result.is_ok(),
let refused = result.as_ref().err().is_some_and(|e| e.downcast_ref::<Refusal>().is_some());
prop_assert!(result.is_ok() || refused,
"Generated file failed to parse:\n{}\nError: {}",
file, result.unwrap_err());
}
crates/improvise-io/src/persistence/roundtrip_tests.rs
+224 -7
View File
@@ -267,10 +267,198 @@ fn parse_md_round_trips_text_cell_values() {
#[test]
fn parse_md_round_trips_collapsed_group() {
let mut m = two_cat_model();
m.active_view_mut().toggle_group_collapse("Type", "MyGroup");
m.active_view_mut()
.set_group_fold("Type", "MyGroup", crate::view::GroupFold::Collapsed);
let text = format_md(&m);
let loaded = parse_md(&text).unwrap();
assert!(loaded.active_view().is_group_collapsed("Type", "MyGroup"));
assert!(
loaded.active_view().group_fold("Type", "MyGroup")
== Some(crate::view::GroupFold::Collapsed)
);
}
// ── Group hierarchies and folds (improvise-felo.6) ─────────────────────
/// Year > Quarter > Month as a file spells it: `> Quarter[Year]` lines.
const DATES: &str = "v2025-04-09\n# T\nInitial View: Default\n\n## View: Default\nDate: row\n\
_Measure: column\n@FOLDS@\n## Category: Date\n\
- |2024-01|[Q1-2024]\n- |2024-04|[Q2-2024]\n- |2025-01|[Q1-2025]\n\
> Q1-2024[|2024|]\n> Q2-2024[|2024|]\n> Q1-2025[|2025|]\n\n\
## Category: _Measure\n- Revenue\n";
fn dates_with(folds: &str) -> String {
DATES.replace("@FOLDS@", folds)
}
/// Every spelling of a fold, together in one view, survives save and load —
/// and the writer's own text is stable.
#[test]
fn expanded_and_collapsed_lines_round_trip_together() {
use crate::view::GroupFold;
let text = dates_with(
"expanded(2): Date\ncollapsed: Date/Q1-2024\nexpanded: Date/Q2-2024\n\
expanded(3): Date/|2025|",
);
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
assert_eq!(v.expand_depth("Date"), 2);
assert_eq!(v.group_fold("Date", "Q1-2024"), Some(GroupFold::Collapsed));
assert_eq!(
v.group_fold("Date", "Q2-2024"),
Some(GroupFold::Expanded(std::num::NonZeroUsize::MIN))
);
assert_eq!(
v.group_fold("Date", "2025"),
Some(GroupFold::Expanded(std::num::NonZeroUsize::new(3).unwrap()))
);
let written = format_md(&loaded);
for line in [
"expanded(2): Date",
"collapsed: Date/Q1-2024",
"expanded: Date/Q2-2024",
"expanded(3): Date/|2025|",
] {
assert!(written.lines().any(|l| l == line), "{line} in:\n{written}");
}
assert_eq!(format_md(&parse_md(&written).unwrap()), written);
}
/// A file from before improvise-felo.6 carries only `collapsed:` lines. Every
/// group opens closed, the ones it names AND the ones it does not.
#[test]
fn an_old_file_with_only_collapsed_lines_loads_closed_and_keeps_its_lines() {
let text = dates_with("collapsed: Date/Q1-2024");
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
let cat = loaded.model.category("Date").unwrap();
for group in ["2024", "Q1-2024", "Q2-2024", "2025"] {
assert!(
v.is_group_collapsed("Date", &cat.group_chain(group)),
"{group} opens closed"
);
}
assert!(
format_md(&loaded)
.lines()
.any(|l| l == "collapsed: Date/Q1-2024"),
"the writer keeps the line the file had, though it agrees with the default"
);
}
/// The user's ruling: expanded and collapsed on ONE group is an error, named.
#[test]
fn a_group_both_expanded_and_collapsed_is_refused_naming_view_and_group() {
use super::parse::Refusal;
for folds in [
"collapsed: Date/Q1-2024\nexpanded: Date/Q1-2024",
"expanded(2): Date/Q1-2024\ncollapsed: Date/Q1-2024",
"expanded: Date/Q1-2024\nexpanded(2): Date/Q1-2024",
] {
let err = parse_md(&dates_with(folds)).unwrap_err();
assert_eq!(
err.downcast_ref::<Refusal>(),
Some(&Refusal::FoldConflict {
view: "Default".into(),
category: "Date".into(),
group: Some("Q1-2024".into()),
}),
"{folds}: {err}"
);
assert!(err.to_string().contains("Default"), "{err}");
assert!(err.to_string().contains("Date/Q1-2024"), "{err}");
}
let err = parse_md(&dates_with("expanded(2): Date\nexpanded(3): Date")).unwrap_err();
assert!(err.downcast_ref::<Refusal>().is_some(), "{err}");
}
/// Not the error: a DIFFERENT group collapsed inside an expanded region, and
/// the same line twice.
#[test]
fn a_collapsed_group_inside_an_expanded_region_loads() {
let text = dates_with(
"expanded(2): Date\nexpanded(2): Date/|2024|\ncollapsed: Date/Q1-2024\n\
collapsed: Date/Q1-2024",
);
let loaded = parse_md(&text).unwrap();
let v = loaded.active_view();
assert!(v.is_group_collapsed("Date", &["2024", "Q1-2024"]));
assert!(!v.is_group_collapsed("Date", &["2024", "Q2-2024"]));
}
/// A cycle among `>` lines is refused on load, naming its members.
#[test]
fn a_cycle_among_group_parents_is_refused_naming_it() {
use super::parse::Refusal;
let text = DATES
.replace("@FOLDS@", "")
.replace("> Q1-2025[|2025|]", "> Q1-2025[|2025|]\n> |2025|[Q1-2025]");
let err = parse_md(&text).unwrap_err();
match err.downcast_ref::<Refusal>() {
Some(Refusal::HierarchyCycle { category, groups }) => {
assert_eq!(category, "Date");
let mut groups = groups.clone();
groups.sort();
assert_eq!(groups, vec!["2025", "Q1-2025"]);
}
other => panic!("not a cycle refusal: {other:?} ({err})"),
}
}
/// `expanded(0)` opens nothing and is not a spelling; `(x)` is not a depth.
#[test]
fn a_malformed_depth_is_a_parse_error() {
assert!(
parse_md(&dates_with("expanded(10): Date")).is_ok(),
"the control"
);
for bad in ["expanded(0): Date", "expanded(x): Date", "expanded(): Date"] {
assert!(parse_md(&dates_with(bad)).is_err(), "{bad}");
}
}
/// A category named `expanded` keeps its axis line, and a category whose NAME is
/// an axis keyword keeps its depth: the writer never spells a depth bare.
#[test]
fn an_axis_line_for_a_category_named_expanded_is_still_an_axis_line() {
let mut wb = Workbook::new("T");
wb.add_category("expanded").unwrap();
wb.add_category("row").unwrap();
wb.active_view_mut().set_axis("expanded", Axis::Column);
wb.active_view_mut().set_expand_depth("row", 1);
let loaded = parse_md(&format_md(&wb)).unwrap();
assert_eq!(loaded.active_view().axis_of("expanded"), Axis::Column);
assert_eq!(loaded.active_view().expand_depth("row"), 1);
}
/// `z` writes through ONE map, so no sequence of toggles can save a file that
/// names a group both ways. Every toggle order over three levels, twice over.
#[test]
fn no_sequence_of_toggles_writes_a_conflicting_file() {
let base = parse_md(&dates_with("expanded(1): Date")).unwrap();
let chains: [&[&str]; 4] = [
&["2024"],
&["2024", "Q1-2024"],
&["2024", "Q2-2024"],
&["2025", "Q1-2025"],
];
for order in 0..(4usize.pow(4)) {
let mut wb = base.clone();
let mut code = order;
for _ in 0..4 {
let chain = chains[code % 4];
code /= 4;
wb.active_view_mut().toggle_group_collapse("Date", chain);
let text = format_md(&wb);
let back = parse_md(&text).unwrap_or_else(|e| panic!("{e}\n{text}"));
for chain in chains {
assert_eq!(
back.active_view().is_group_collapsed("Date", chain),
wb.active_view().is_group_collapsed("Date", chain),
"{chain:?} after order {order}:\n{text}"
);
}
}
}
}
// ── parse_md: number format roundtrip ───────────────────────────────
@@ -336,7 +524,8 @@ fn full_roundtrip_preserves_all_features() {
m.active_view_mut().set_axis("Month", Axis::Page);
m.active_view_mut().set_page_selection("Month", "Jan");
m.active_view_mut().hide_item("Type", "Gas");
m.active_view_mut().toggle_group_collapse("Type", "G1");
m.active_view_mut()
.set_group_fold("Type", "G1", crate::view::GroupFold::Collapsed);
m.active_view_mut().number_format = ",.0".to_string();
let text = format_md(&m);
@@ -361,7 +550,7 @@ fn full_roundtrip_preserves_all_features() {
assert_eq!(v.axis_of("Month"), Axis::Page);
assert_eq!(v.page_selection("Month"), Some("Jan"));
assert!(v.is_hidden("Type", "Gas"));
assert!(v.is_group_collapsed("Type", "G1"));
assert!(v.group_fold("Type", "G1") == Some(crate::view::GroupFold::Collapsed));
assert_eq!(v.number_format, ",.0");
}
@@ -842,7 +1031,7 @@ fn a_csv_header_with_an_embedded_newline_survives_a_save() {
let csv = "\"Region\nName\",Amount\nEast,1\nWest,2\n";
let (records, _) =
crate::import::csv_parser::parse_csv_bytes_with_warnings(csv.as_bytes()).unwrap();
let wb = crate::import::wizard::ImportPipeline::new(serde_json::Value::Array(records))
let wb = crate::import::wizard::ImportPipeline::from_csv(records)
.build_model()
.unwrap();
assert!(
@@ -1187,7 +1376,7 @@ fn a_label_category_survives_the_round_trip_as_a_label() {
#[test]
fn a_document_at_the_category_limit_plus_a_label_reloads() {
let mut wb = Workbook::new("T");
for i in 0..12 {
for i in 0..crate::model::MAX_CATEGORIES {
wb.add_category(format!("C{i}")).unwrap();
}
wb.add_label_category("Note").unwrap();
@@ -1197,7 +1386,10 @@ fn a_document_at_the_category_limit_plus_a_label_reloads() {
loaded.model.category("Note").unwrap().kind,
CategoryKind::Label
);
assert_eq!(loaded.model.regular_category_names().len(), 12);
assert_eq!(
loaded.model.regular_category_names().len(),
crate::model::MAX_CATEGORIES
);
}
/// A file written before the `kind:` line existed reads its categories as
@@ -1234,3 +1426,28 @@ fn a_loaded_label_category_is_not_on_a_pivot_axis() {
);
}
}
/// ChatGPT Codex on #317: a fold built through the public API must survive its
/// own save. `Expanded(0)` is now unconstructible; every depth that is survives.
#[test]
fn every_fold_the_api_can_build_reads_back_from_the_file_it_writes() {
use crate::view::GroupFold;
use std::num::NonZeroUsize;
let base = parse_md(&dates_with("")).unwrap();
for n in 1..=3 {
let depth = NonZeroUsize::new(n).unwrap();
for fold in [GroupFold::Collapsed, GroupFold::Expanded(depth)] {
let mut wb = base.clone();
wb.active_view_mut().set_group_fold("Date", "2024", fold);
wb.active_view_mut().set_expand_depth("Date", n);
let text = format_md(&wb);
let back = parse_md(&text).unwrap_or_else(|e| panic!("{e}\n{text}"));
assert_eq!(
back.active_view().group_fold("Date", "2024"),
Some(fold),
"{text}"
);
assert_eq!(back.active_view().expand_depth("Date"), n, "{text}");
}
}
}
crates/improvise-protocol/src/basis.rs
+6 -2
View File
@@ -64,6 +64,9 @@ pub enum Rejection {
/// A [`ClientFrame::Save`](crate::ClientFrame::Save) whose write failed.
/// No message: [`Rejection`] is `Copy` and an I/O error's text is not.
SaveFailed,
/// A [`ClientFrame::DownloadCsv`](crate::ClientFrame::DownloadCsv) failed
/// to render the view (improvise-2mqz). Typically the view was not found.
ExportFailed,
}
/// The basis a client holds, folded from the stream like any other widget
@@ -131,7 +134,7 @@ impl Rejection {
| Self::ChainBroken(_)
| Self::MutationRefused
| Self::ReadOnly => true,
Self::NoDocumentPath | Self::SaveFailed => false,
Self::NoDocumentPath | Self::SaveFailed | Self::ExportFailed => false,
}
}
@@ -160,7 +163,8 @@ impl Rejection {
Self::MutationRefused => Self::ReadOnly,
Self::ReadOnly => Self::NoDocumentPath,
Self::NoDocumentPath => Self::SaveFailed,
Self::SaveFailed => Self::DocumentReplaced,
Self::SaveFailed => Self::ExportFailed,
Self::ExportFailed => Self::DocumentReplaced,
}
}
}
crates/improvise-protocol/src/client.rs
+3
View File
@@ -514,6 +514,7 @@ mod tests {
live.fold(&Command::Project(Projection::CategoryTree(
CategoryTreeUpdate::Categories(vec![CategoryRow {
name: "Region".into(),
display: "Region".into(),
items: vec!["North".into(), "South".into()],
axis: Axis::Row,
}]),
@@ -523,11 +524,13 @@ mod tests {
])));
live.fold(&Command::Project(Projection::TileBar(vec![TileEntry {
category: "Region".into(),
display: "Region".into(),
kind: crate::CategoryKind::Regular,
axis: Axis::Row,
}])));
live.fold(&Command::Project(Projection::ViewList(vec![ViewSummary {
name: "Default".into(),
display: "Default".into(),
axes: vec![("Region".into(), Axis::Row)],
}])));
live.fold(&Command::SetSelected { row: 1, col: 1 });
crates/improvise-protocol/src/face.rs
+682
View File
@@ -0,0 +1,682 @@
//! Faces: appearance named by role, and a theme that binds roles to values
//! (ADR 004). The reasoning is `context/code/protocol.md` §10.
use std::collections::BTreeMap;
use std::fmt;
use serde::de::{self, Deserializer, Visitor};
use serde::{Deserialize, Serialize, Serializer};
use crate::render::WizardFieldKind;
use crate::{AppMode, Panel};
use improvise_core::view::Axis;
/// One of the sixteen palette slots: "whatever THIS display calls that slot".
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum AnsiSlot {
Black,
Red,
Green,
Yellow,
Blue,
Magenta,
Cyan,
White,
BrightBlack,
BrightRed,
BrightGreen,
BrightYellow,
BrightBlue,
BrightMagenta,
BrightCyan,
BrightWhite,
}
impl AnsiSlot {
pub const ALL: [AnsiSlot; 16] = [
AnsiSlot::Black,
AnsiSlot::Red,
AnsiSlot::Green,
AnsiSlot::Yellow,
AnsiSlot::Blue,
AnsiSlot::Magenta,
AnsiSlot::Cyan,
AnsiSlot::White,
AnsiSlot::BrightBlack,
AnsiSlot::BrightRed,
AnsiSlot::BrightGreen,
AnsiSlot::BrightYellow,
AnsiSlot::BrightBlue,
AnsiSlot::BrightMagenta,
AnsiSlot::BrightCyan,
AnsiSlot::BrightWhite,
];
/// The slot's number in the 256-colour palette, 0–15.
pub fn index(self) -> u8 {
self as u8
}
/// The spelling a theme file uses.
pub fn name(self) -> &'static str {
match self {
AnsiSlot::Black => "black",
AnsiSlot::Red => "red",
AnsiSlot::Green => "green",
AnsiSlot::Yellow => "yellow",
AnsiSlot::Blue => "blue",
AnsiSlot::Magenta => "magenta",
AnsiSlot::Cyan => "cyan",
AnsiSlot::White => "white",
AnsiSlot::BrightBlack => "bright-black",
AnsiSlot::BrightRed => "bright-red",
AnsiSlot::BrightGreen => "bright-green",
AnsiSlot::BrightYellow => "bright-yellow",
AnsiSlot::BrightBlue => "bright-blue",
AnsiSlot::BrightMagenta => "bright-magenta",
AnsiSlot::BrightCyan => "bright-cyan",
AnsiSlot::BrightWhite => "bright-white",
}
}
pub fn from_name(name: &str) -> Option<Self> {
Self::ALL.into_iter().find(|slot| slot.name() == name)
}
}
/// A colour attribute. `Default`, `Named` and `Indexed` are palette-relative;
/// `Rgb` is absolute, and the shipped theme never uses it (ADR 004 clause 4).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Colour {
Default,
Named(AnsiSlot),
Indexed(u8),
Rgb(u8, u8, u8),
}
impl Colour {
/// True for the one arm that opts a viewer out of their own palette.
pub fn is_absolute(self) -> bool {
match self {
Colour::Rgb(..) => true,
Colour::Default | Colour::Named(_) | Colour::Indexed(_) => false,
}
}
}
impl Serialize for Colour {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
match *self {
Colour::Default => s.serialize_str("default"),
Colour::Named(slot) => s.serialize_str(slot.name()),
Colour::Indexed(n) => s.serialize_u8(n),
Colour::Rgb(r, g, b) => s.serialize_str(&format!("#{r:02x}{g:02x}{b:02x}")),
}
}
}
struct ColourVisitor;
impl Visitor<'_> for ColourVisitor {
type Value = Colour;
fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.write_str("\"default\", a slot name such as \"green\", 0-255, or \"#rrggbb\"")
}
fn visit_u64<E: de::Error>(self, n: u64) -> Result<Colour, E> {
u8::try_from(n)
.map(Colour::Indexed)
.map_err(|_| E::custom(format!("colour index {n} is outside 0-255")))
}
fn visit_str<E: de::Error>(self, s: &str) -> Result<Colour, E> {
parse_colour(s).ok_or_else(|| E::invalid_value(de::Unexpected::Str(s), &self))
}
}
fn parse_colour(s: &str) -> Option<Colour> {
match s.strip_prefix('#') {
Some(hex) if hex.len() == 6 && hex.bytes().all(|b| b.is_ascii_hexdigit()) => {
let byte = |i: usize| u8::from_str_radix(&hex[i..i + 2], 16).ok();
Some(Colour::Rgb(byte(0)?, byte(2)?, byte(4)?))
}
Some(_) => None,
None if s == "default" => Some(Colour::Default),
None => AnsiSlot::from_name(s).map(Colour::Named),
}
}
impl<'de> Deserialize<'de> for Colour {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
d.deserialize_any(ColourVisitor)
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Weight {
Normal,
Bold,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Slant {
Normal,
Italic,
}
/// The text-entry modes, one per `MinibufferConfig`-carrying `AppMode` arm
/// except that cell editing is one role over two layouts.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum TextEntry {
Edit,
Formula,
Command,
CategoryAdd,
ItemAdd,
Export,
Save,
}
impl TextEntry {
pub const ALL: [TextEntry; 7] = [
TextEntry::Edit,
TextEntry::Formula,
TextEntry::Command,
TextEntry::CategoryAdd,
TextEntry::ItemAdd,
TextEntry::Export,
TextEntry::Save,
];
/// The text-entry role of `mode`, or `None` for a mode that takes no text.
pub fn of(mode: &AppMode) -> Option<Self> {
match mode {
AppMode::Editing { .. } | AppMode::RecordsEditing { .. } => Some(TextEntry::Edit),
AppMode::FormulaEdit { .. } => Some(TextEntry::Formula),
AppMode::CommandMode { .. } => Some(TextEntry::Command),
AppMode::CategoryAdd { .. } => Some(TextEntry::CategoryAdd),
AppMode::ItemAdd { .. } => Some(TextEntry::ItemAdd),
AppMode::ExportPrompt { .. } => Some(TextEntry::Export),
AppMode::SavePrompt { .. } => Some(TextEntry::Save),
AppMode::Normal
| AppMode::RecordsNormal
| AppMode::FormulaPanel
| AppMode::CategoryPanel
| AppMode::ViewPanel
| AppMode::TileSelect
| AppMode::ImportWizard
| AppMode::Help
| AppMode::Quit => None,
}
}
fn name(self) -> &'static str {
match self {
TextEntry::Edit => "edit",
TextEntry::Formula => "formula",
TextEntry::Command => "command",
TextEntry::CategoryAdd => "category-add",
TextEntry::ItemAdd => "item-add",
TextEntry::Export => "export",
TextEntry::Save => "save",
}
}
}
const AXES: [Axis; 5] = [
Axis::Row,
Axis::Column,
Axis::Page,
Axis::Filter,
Axis::None,
];
const FIELD_KINDS: [WizardFieldKind; 4] = [
WizardFieldKind::Category,
WizardFieldKind::Measure,
WizardFieldKind::TimeCategory,
WizardFieldKind::Label,
];
fn axis_name(axis: Axis) -> &'static str {
match axis {
Axis::Row => "row",
Axis::Column => "column",
Axis::Page => "page",
Axis::Filter => "filter",
Axis::None => "none",
}
}
fn field_kind_name(kind: WizardFieldKind) -> &'static str {
match kind {
WizardFieldKind::Category => "category",
WizardFieldKind::Measure => "measure",
WizardFieldKind::TimeCategory => "time",
WizardFieldKind::Label => "label",
}
}
/// A role something is drawn in. Its `name` is a compatibility surface once a
/// theme file carries it (ADR 004 clause 9), so never rename one.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum Face {
Default,
Dim,
Axis(Axis),
WizardField(WizardFieldKind),
Mode,
ModeResting,
ModeTextEntry,
ModeTextEntryOf(TextEntry),
ModePanel(Panel),
ModeOverlay,
ModeTransient,
ModeTileSelect,
MinibufferOf(TextEntry),
}
impl Face {
/// Every face, parents before children.
pub fn all() -> Vec<Face> {
let mut out = vec![Face::Default, Face::Dim];
out.extend(AXES.map(Face::Axis));
out.extend(FIELD_KINDS.map(Face::WizardField));
out.extend([Face::Mode, Face::ModeResting, Face::ModeTextEntry]);
out.extend(TextEntry::ALL.map(Face::ModeTextEntryOf));
out.extend(Panel::ALL.map(Face::ModePanel));
out.extend([Face::ModeOverlay, Face::ModeTransient, Face::ModeTileSelect]);
out.extend(TextEntry::ALL.map(Face::MinibufferOf));
out
}
/// The face this one inherits every attribute it does not state from.
pub fn parent(self) -> Option<Face> {
match self {
Face::Default => None,
Face::Dim | Face::Mode => Some(Face::Default),
Face::Axis(Axis::None) | Face::WizardField(WizardFieldKind::Label) => Some(Face::Dim),
Face::Axis(_) | Face::WizardField(_) => Some(Face::Default),
Face::ModeResting
| Face::ModeTextEntry
| Face::ModePanel(_)
| Face::ModeOverlay
| Face::ModeTransient
| Face::ModeTileSelect => Some(Face::Mode),
Face::ModeTextEntryOf(_) | Face::MinibufferOf(_) => Some(Face::ModeTextEntry),
}
}
/// The spelling a theme file keys this face by.
pub fn name(self) -> String {
match self {
Face::Default => "default".into(),
Face::Dim => "dim".into(),
Face::Axis(axis) => format!("axis.{}", axis_name(axis)),
Face::WizardField(kind) => format!("wizard.field.{}", field_kind_name(kind)),
Face::Mode => "mode".into(),
Face::ModeResting => "mode.resting".into(),
Face::ModeTextEntry => "mode.text-entry".into(),
Face::ModeTextEntryOf(entry) => format!("mode.text-entry.{}", entry.name()),
Face::ModePanel(panel) => format!("mode.panel.{}", panel.name()),
Face::ModeOverlay => "mode.overlay".into(),
Face::ModeTransient => "mode.transient".into(),
Face::ModeTileSelect => "mode.tile-select".into(),
Face::MinibufferOf(entry) => format!("minibuffer.{}", entry.name()),
}
}
pub fn from_name(name: &str) -> Option<Face> {
Face::all().into_iter().find(|face| face.name() == name)
}
/// The face the mode indicator is drawn in.
pub fn of_mode(mode: &AppMode) -> Face {
match mode {
AppMode::Normal | AppMode::RecordsNormal => Face::ModeResting,
AppMode::FormulaPanel => Face::ModePanel(Panel::Formula),
AppMode::CategoryPanel => Face::ModePanel(Panel::Category),
AppMode::ViewPanel => Face::ModePanel(Panel::View),
AppMode::ImportWizard | AppMode::Help => Face::ModeOverlay,
AppMode::Quit => Face::ModeTransient,
AppMode::TileSelect => Face::ModeTileSelect,
AppMode::Editing { .. }
| AppMode::RecordsEditing { .. }
| AppMode::FormulaEdit { .. }
| AppMode::CommandMode { .. }
| AppMode::CategoryAdd { .. }
| AppMode::ItemAdd { .. }
| AppMode::ExportPrompt { .. }
| AppMode::SavePrompt { .. } => TextEntry::of(mode)
.map(Face::ModeTextEntryOf)
.unwrap_or(Face::ModeTextEntry),
}
}
/// The face a text-entry mode's minibuffer is drawn in.
pub fn of_minibuffer(mode: &AppMode) -> Option<Face> {
TextEntry::of(mode).map(Face::MinibufferOf)
}
}
impl Serialize for Face {
fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&self.name())
}
}
impl<'de> Deserialize<'de> for Face {
fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
let name = String::deserialize(d)?;
Face::from_name(&name).ok_or_else(|| de::Error::custom(format!("unknown face {name:?}")))
}
}
/// What a theme states about one face. `None` inherits from the parent;
/// `Some(Colour::Default)` states "the surface" and does not inherit.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct FaceSpec {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub foreground: Option<Colour>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub background: Option<Colour>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub weight: Option<Weight>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub slant: Option<Slant>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub underline: Option<bool>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reverse: Option<bool>,
}
impl FaceSpec {
fn or(self, parent: FaceSpec) -> FaceSpec {
FaceSpec {
foreground: self.foreground.or(parent.foreground),
background: self.background.or(parent.background),
weight: self.weight.or(parent.weight),
slant: self.slant.or(parent.slant),
underline: self.underline.or(parent.underline),
reverse: self.reverse.or(parent.reverse),
}
}
fn fg(colour: Colour) -> FaceSpec {
FaceSpec {
foreground: Some(colour),
..FaceSpec::default()
}
}
fn bg(colour: Colour) -> FaceSpec {
FaceSpec {
background: Some(colour),
..FaceSpec::default()
}
}
}
/// A face with every attribute concrete: what a front end translates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResolvedFace {
pub foreground: Colour,
pub background: Colour,
pub weight: Weight,
pub slant: Slant,
pub underline: bool,
pub reverse: bool,
}
/// Faces bound to values. Client-local: never a `Command`, never in
/// `ViewState`, never projected, never in `.improv` (ADR 004 clause 7).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Theme {
pub faces: BTreeMap<Face, FaceSpec>,
}
impl Theme {
/// The theme improvise ships: today's terminal rendering, palette-relative
/// throughout, so the user's terminal theme still applies.
pub fn shipped() -> Theme {
use AnsiSlot::*;
let named = Colour::Named;
let minibuffer = |accent| FaceSpec {
foreground: Some(named(accent)),
background: Some(Colour::Indexed(235)),
weight: Some(Weight::Bold),
..FaceSpec::default()
};
let faces = [
(Face::Dim, FaceSpec::fg(named(BrightBlack))),
(Face::Axis(Axis::Row), FaceSpec::fg(named(Green))),
(Face::Axis(Axis::Column), FaceSpec::fg(named(Blue))),
(Face::Axis(Axis::Page), FaceSpec::fg(named(Magenta))),
(Face::Axis(Axis::Filter), FaceSpec::fg(named(Yellow))),
(
Face::WizardField(WizardFieldKind::Category),
FaceSpec::fg(named(Green)),
),
(
Face::WizardField(WizardFieldKind::Measure),
FaceSpec::fg(named(Cyan)),
),
(
Face::WizardField(WizardFieldKind::TimeCategory),
FaceSpec::fg(named(Magenta)),
),
(
Face::Mode,
FaceSpec {
foreground: Some(named(Black)),
background: Some(named(BrightBlack)),
..FaceSpec::default()
},
),
(
Face::ModeTextEntryOf(TextEntry::Edit),
FaceSpec::bg(named(Green)),
),
(
Face::ModeTextEntryOf(TextEntry::Command),
FaceSpec::bg(named(Yellow)),
),
(Face::ModeTileSelect, FaceSpec::bg(named(Magenta))),
(Face::MinibufferOf(TextEntry::Edit), minibuffer(Green)),
(Face::MinibufferOf(TextEntry::ItemAdd), minibuffer(Green)),
(
Face::MinibufferOf(TextEntry::CategoryAdd),
minibuffer(Green),
),
(Face::MinibufferOf(TextEntry::Formula), minibuffer(Cyan)),
(Face::MinibufferOf(TextEntry::Command), minibuffer(Yellow)),
(Face::MinibufferOf(TextEntry::Export), minibuffer(Yellow)),
(Face::MinibufferOf(TextEntry::Save), minibuffer(Yellow)),
];
Theme {
faces: faces.into_iter().collect(),
}
}
/// Walk `face`'s inheritance chain once, so no front end ever does.
pub fn resolve(&self, face: Face) -> ResolvedFace {
let stated = std::iter::successors(Some(face), |f| f.parent())
.filter_map(|f| self.faces.get(&f).copied())
.fold(FaceSpec::default(), FaceSpec::or);
ResolvedFace {
foreground: stated.foreground.unwrap_or(Colour::Default),
background: stated.background.unwrap_or(Colour::Default),
weight: stated.weight.unwrap_or(Weight::Normal),
slant: stated.slant.unwrap_or(Slant::Normal),
underline: stated.underline.unwrap_or(false),
reverse: stated.reverse.unwrap_or(false),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn every_face_name_is_distinct_and_reads_back_as_that_face() {
let all = Face::all();
for face in &all {
assert_eq!(Face::from_name(&face.name()), Some(*face), "{face:?}");
}
let mut names: Vec<String> = all.iter().map(|f| f.name()).collect();
names.sort();
names.dedup();
assert_eq!(names.len(), all.len(), "two faces share a name");
}
#[test]
fn every_parent_is_a_face_listed_before_its_child() {
let all = Face::all();
for (i, face) in all.iter().enumerate() {
if let Some(parent) = face.parent() {
let at = all.iter().position(|f| *f == parent);
assert!(at.is_some_and(|p| p < i), "{face:?}'s parent {parent:?}");
}
}
}
/// ADR 004 clause 4: the shipped theme keeps the user's terminal palette.
#[test]
fn the_shipped_theme_states_no_absolute_colour() {
let theme = Theme::shipped();
for face in Face::all() {
let resolved = theme.resolve(face);
assert!(!resolved.foreground.is_absolute(), "{}", face.name());
assert!(!resolved.background.is_absolute(), "{}", face.name());
}
}
/// ADR 004: `minibuffer.*` inherits `mode.text-entry` directly, so a theme
/// that restyles text entry reaches every prompt. `context/test-evidence/protocol.md`.
#[test]
fn a_theme_that_restyles_text_entry_reaches_every_minibuffer() {
let mut theme = Theme::shipped();
let italic = FaceSpec {
slant: Some(Slant::Italic),
..FaceSpec::default()
};
theme.faces.insert(Face::ModeTextEntry, italic);
for entry in TextEntry::ALL {
let got = theme.resolve(Face::MinibufferOf(entry));
assert_eq!(got.slant, Slant::Italic, "{entry:?}");
}
}
/// A collision defeats the reason the axis faces exist (improvise-rml).
#[test]
fn every_axis_gets_its_own_accent_in_the_shipped_theme() {
let theme = Theme::shipped();
let accents = AXES.map(|axis| theme.resolve(Face::Axis(axis)));
for (i, a) in accents.iter().enumerate() {
for b in &accents[i + 1..] {
assert_ne!(a, b, "two axes share an accent");
}
}
}
/// A child overrides what it states and inherits what it does not, from
/// the nearest ancestor that states it — `context/test-evidence/protocol.md`.
#[test]
fn resolution_takes_each_attribute_from_the_nearest_face_that_states_it() {
let named = Colour::Named;
let theme = Theme {
faces: [
(
Face::Default,
FaceSpec {
foreground: Some(named(AnsiSlot::White)),
slant: Some(Slant::Italic),
..FaceSpec::default()
},
),
(
Face::Mode,
FaceSpec {
foreground: Some(named(AnsiSlot::Black)),
background: Some(named(AnsiSlot::Blue)),
..FaceSpec::default()
},
),
(
Face::ModeTextEntryOf(TextEntry::Save),
FaceSpec {
background: Some(Colour::Default),
underline: Some(true),
..FaceSpec::default()
},
),
]
.into_iter()
.collect(),
};
let got = theme.resolve(Face::ModeTextEntryOf(TextEntry::Save));
assert_eq!(
got,
ResolvedFace {
foreground: named(AnsiSlot::Black),
background: Colour::Default,
weight: Weight::Normal,
slant: Slant::Italic,
underline: true,
reverse: false,
}
);
let sibling = theme.resolve(Face::ModeTextEntryOf(TextEntry::Export));
assert_eq!(sibling.background, named(AnsiSlot::Blue));
}
#[test]
fn a_theme_survives_its_own_json() {
let theme = Theme::shipped();
let text = serde_json::to_string_pretty(&theme).unwrap();
assert!(text.contains("\"minibuffer.category-add\""), "{text}");
assert_eq!(serde_json::from_str::<Theme>(&text).unwrap(), theme);
}
#[test]
fn a_colour_is_spelled_for_a_person_editing_the_file() {
let cases = [
(Colour::Default, "\"default\""),
(Colour::Named(AnsiSlot::BrightBlack), "\"bright-black\""),
(Colour::Indexed(235), "235"),
(Colour::Rgb(0x3f, 0x7a, 0x4d), "\"#3f7a4d\""),
];
for (colour, json) in cases {
assert_eq!(serde_json::to_string(&colour).unwrap(), json);
assert_eq!(serde_json::from_str::<Colour>(json).unwrap(), colour);
}
for bad in ["\"grene\"", "\"#3f7a4\"", "256", "\"#gggggg\""] {
assert!(serde_json::from_str::<Colour>(bad).is_err(), "{bad}");
}
}
#[test]
fn a_misspelt_face_or_attribute_is_refused_rather_than_ignored() {
let face = r#"{"faces":{"axis.rwo":{"foreground":"green"}}}"#;
assert!(serde_json::from_str::<Theme>(face).is_err());
let attr = r#"{"faces":{"axis.row":{"forground":"green"}}}"#;
assert!(serde_json::from_str::<Theme>(attr).is_err());
}
#[test]
fn every_text_entry_mode_has_a_text_entry_role_and_no_other_mode_does() {
for mode in AppMode::all() {
assert_eq!(
TextEntry::of(&mode).is_some(),
mode.minibuffer().is_some(),
"{mode:?}"
);
}
}
}
crates/improvise-protocol/src/frame.rs
+26
View File
@@ -23,6 +23,12 @@ pub enum ClientFrame {
code: KeyCode,
modifiers: KeyModifiers,
},
/// Ask for document bytes named by SavePrompt; the browser downloads them
/// via Blob + `<a download>` (improvise-2mqz).
DownloadDocument { filename: String },
/// Ask for active-view CSV bytes named by ExportPrompt; the browser downloads
/// them via Blob + `<a download>` (improvise-2mqz).
DownloadCsv { filename: String },
}
impl ClientFrame {
@@ -55,6 +61,12 @@ pub enum ServerFrame {
/// The asker's mutating frame landed whole. Carries no state: everything
/// it moved arrived as projections first — `context/code/protocol.md` §4.
Accepted,
/// Downloadable `.improv` bytes addressed to the asker alone; the filename
/// is echoed from its request (improvise-2mqz).
DocumentBytes { filename: String, bytes: Vec<u8> },
/// Downloadable active-view CSV bytes addressed to the asker alone; the
/// filename is echoed from its request (improvise-2mqz).
CsvBytes { filename: String, bytes: Vec<u8> },
}
impl ServerFrame {
@@ -83,6 +95,12 @@ mod tests {
)),
ClientFrame::key(KeyEvent::new(KeyCode::Esc, KeyModifiers::NONE)),
ClientFrame::Save,
ClientFrame::DownloadDocument {
filename: "doc.improv".into(),
},
ClientFrame::DownloadCsv {
filename: "export.csv".into(),
},
] {
let json = serde_json::to_string(&frame).expect("serializes");
assert_eq!(
@@ -105,6 +123,14 @@ mod tests {
))]),
ServerFrame::Rejected(Rejection::LayoutMoved),
ServerFrame::Accepted,
ServerFrame::DocumentBytes {
filename: "test.improv".into(),
bytes: b"v2025-04-09\n# Test\n".to_vec(),
},
ServerFrame::CsvBytes {
filename: "export.csv".into(),
bytes: b"col1,col2\na,b\n".to_vec(),
},
] {
let json = serde_json::to_string(&frame).expect("serializes");
assert_eq!(
crates/improvise-protocol/src/keymap.rs
+4 -2
View File
@@ -927,11 +927,13 @@ impl Keymaps {
),
);
// LocalThenRequest cannot live in a sequence, so these commands own
// their buffer clears and are rebound directly (improvise-2mqz).
set.insert(
ModeKey::ExportPrompt,
Arc::new(
TextEntry::new(&AppMode::export_prompt(), "normal", global.clone())
.commit(KeyCode::Enter, "commit-export", vec![])
.rebind(KeyCode::Enter, "commit-export")
.finish(),
),
);
@@ -940,7 +942,7 @@ impl Keymaps {
ModeKey::SavePrompt,
Arc::new(
TextEntry::new(&AppMode::save_prompt(), "normal", global.clone())
.commit(KeyCode::Enter, "commit-save", vec![])
.rebind(KeyCode::Enter, "commit-save")
.finish(),
),
);
crates/improvise-protocol/src/lib.rs
+4 -2
View File
@@ -6,6 +6,7 @@ use std::collections::HashMap;
pub mod basis;
pub mod client;
pub mod face;
pub mod frame;
pub mod help;
pub mod key;
@@ -23,6 +24,7 @@ mod vocabulary_is_composed;
pub use basis::{Basis, BasisCache, Rejection};
pub use client::{ClientState, Sent};
pub use face::{Face, Theme};
pub use frame::{ClientFrame, ServerFrame};
pub use help::{HelpLine, Hint, hint_line, hints, mode_label};
pub use key::{KeyCode, KeyEvent, KeyModifiers};
@@ -103,7 +105,7 @@ pub struct MinibufferConfig {
/// One of the three side panels. Paired with [`AppMode::panel`], which names
/// the panel a mode is focused in.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
pub enum Panel {
Formula,
Category,
@@ -394,7 +396,7 @@ impl AppMode {
minibuf: MinibufferConfig {
buffer_key: "category".into(),
prompt: "new category: ".into(),
color: MinibufColor::Yellow,
color: MinibufColor::Green,
},
}
}
crates/improvise-protocol/src/reduce.rs
+2
View File
@@ -556,10 +556,12 @@ mod tests {
Command::Project(Projection::Formulas(vec!["Profit = Revenue - Cost".into()])),
Command::Project(Projection::ViewList(vec![ViewSummary {
name: "Default".into(),
display: "Default".into(),
axes: vec![("Region".into(), Axis::Row)],
}])),
Command::Project(Projection::TileBar(vec![TileEntry {
category: "Region".into(),
display: "Region".into(),
kind: crate::CategoryKind::Regular,
axis: Axis::Page,
}])),
crates/improvise-protocol/src/render.rs
+87 -18
View File
@@ -214,6 +214,9 @@ pub struct FrameGroup {
/// Which header level opened it — an index into [`AxisFrame::cats`]. Runs
/// nest, so membership is per level (improvise-e265).
pub level: usize,
/// The group's depth in its category's hierarchy, 0 for a root. Runs nest
/// by `(level, depth)`: `context/code/protocol.md` §3a (improvise-felo.6).
pub depth: usize,
pub name: String,
pub collapsed: bool,
}
@@ -223,10 +226,11 @@ pub struct FrameGroup {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum FrameEntry {
Group(FrameGroup),
/// The run at `level`, and every run inside it, ends here — `AxisEntry::GroupEnd`
/// on the wire (improvise-gd83, e265). Nothing is drawn for it.
/// The run at `(level, depth)`, and every run inside it, ends here —
/// `AxisEntry::GroupEnd` on the wire (improvise-gd83, e265). Nothing is drawn.
GroupEnd {
level: usize,
depth: usize,
},
/// One label per header level, outermost first. Data indices — the
/// `row`/`col` of every other field on [`GridCache`] — count only these.
@@ -279,7 +283,7 @@ impl TryFrom<UncheckedAxisFrame> for AxisFrame {
.iter()
.filter_map(|e| match e {
FrameEntry::Group(g) => Some(g.level),
FrameEntry::GroupEnd { level } => Some(*level),
FrameEntry::GroupEnd { level, .. } => Some(*level),
FrameEntry::Item { .. } => None,
})
.find(|level| *level >= raw.cats.len());
@@ -418,20 +422,22 @@ impl AxisFrame {
.collect()
}
/// The innermost group each data item sits in, in order — one pass. A
/// group at L closes deeper runs; so does a `GroupEnd` at L (improvise-e265).
/// The innermost group each data item sits in, in order — one pass. A group
/// at `(level, depth)` closes deeper runs; so does a `GroupEnd` there (§3a).
pub fn groups(&self) -> Vec<Option<&FrameGroup>> {
let mut open: Vec<Option<&FrameGroup>> = Vec::new();
let mut open: Vec<&FrameGroup> = Vec::new();
let close = |open: &mut Vec<&FrameGroup>, key: (usize, usize)| {
open.truncate(open.partition_point(|g| (g.level, g.depth) < key))
};
let mut out = Vec::new();
for entry in &self.entries {
match entry {
FrameEntry::Group(g) => {
open.truncate(g.level);
open.resize(g.level, None);
open.push(Some(g));
close(&mut open, (g.level, g.depth));
open.push(g);
}
FrameEntry::GroupEnd { level } => open.truncate(*level),
FrameEntry::Item { .. } => out.push(open.iter().rev().find_map(|g| *g)),
FrameEntry::GroupEnd { level, depth } => close(&mut open, (*level, *depth)),
FrameEntry::Item { .. } => out.push(open.last().copied()),
}
}
out
@@ -753,6 +759,8 @@ impl WidgetCache for GridCache {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct CategoryRow {
pub name: String,
/// What the panel shows; `name` stays what a command names (ADR 007).
pub display: String,
pub items: Vec<String>,
pub axis: Axis,
}
@@ -851,6 +859,8 @@ impl WidgetCache for FormulaCache {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ViewSummary {
pub name: String,
/// What the view panel shows; `name` stays what a command names.
pub display: String,
/// `(category, axis)` for the categories this view displays.
pub axes: Vec<(String, Axis)>,
}
@@ -875,6 +885,8 @@ impl WidgetCache for ViewListCache {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct TileEntry {
pub category: String,
/// What the tile shows; `category` stays what a command names.
pub display: String,
pub axis: Axis,
pub kind: CategoryKind,
}
@@ -927,7 +939,7 @@ pub enum WizardStepKind {
/// What kind of thing a proposed field is. The semantic half of a colour
/// decision; the terminal mapping lives in `draw.rs` (gotcha 26).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
pub enum WizardFieldKind {
Category,
Measure,
@@ -955,6 +967,9 @@ pub struct WizardDateField {
/// what is drawn, so the substitution is made once, here.
pub format: String,
pub components: Vec<(String, bool)>,
/// What the chosen components become, as a sentence: one hierarchical
/// category, or a split (improvise-8jk6.3). Worded once, in the projection.
pub shape: String,
}
/// Several source records sharing every coordinate (improvise-cgqt). Data,
@@ -1066,6 +1081,7 @@ mod tests {
fn tile(category: &str, axis: Axis, kind: CategoryKind) -> TileEntry {
TileEntry {
category: category.to_string(),
display: category.to_string(),
axis,
kind,
}
@@ -1150,6 +1166,7 @@ mod tests {
fn cat(name: &str, items: &[&str], axis: Axis) -> CategoryRow {
CategoryRow {
name: name.into(),
display: name.into(),
items: items.iter().map(|s| (*s).to_string()).collect(),
axis,
}
@@ -1349,6 +1366,7 @@ mod tests {
FrameEntry::item(vec!["A".into(), "y".into()]),
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Late".into(),
collapsed: true,
}),
@@ -1362,6 +1380,7 @@ mod tests {
fn repeated_group_frame() -> AxisFrame {
let q1 = || FrameGroup {
level: 1,
depth: 0,
name: "Q1".into(),
collapsed: false,
};
@@ -1388,12 +1407,13 @@ mod tests {
entries: vec![
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Q1".into(),
collapsed: false,
}),
FrameEntry::item(vec!["Jan".into()]),
FrameEntry::item(vec!["Feb".into()]),
FrameEntry::GroupEnd { level: 0 },
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Sep".into()]),
],
}
@@ -1406,6 +1426,7 @@ mod tests {
let group = |level: usize, name: &str| {
FrameEntry::Group(FrameGroup {
level,
depth: 0,
name: name.into(),
collapsed: true,
})
@@ -1416,7 +1437,7 @@ mod tests {
entries: vec![
group(0, "Upper"),
group(1, "Q1"),
FrameEntry::GroupEnd { level: 1 },
FrameEntry::GroupEnd { level: 1, depth: 0 },
],
};
let json = serde_json::to_string(&frame).unwrap();
@@ -1424,10 +1445,13 @@ mod tests {
assert_eq!(back, frame);
for (from, to) in [
(
r#""level":1,"name":"Q1""#,
r#""level":4294967295,"name":"Q1""#,
r#""level":1,"depth":0,"name":"Q1""#,
r#""level":4294967295,"depth":0,"name":"Q1""#,
),
(
r#"{"GroupEnd":{"level":1,"depth":0}}"#,
r#"{"GroupEnd":{"level":2,"depth":0}}"#,
),
(r#"{"GroupEnd":{"level":1}}"#, r#"{"GroupEnd":{"level":2}}"#),
] {
let bad = json.replacen(from, to, 1);
assert_ne!(bad, json, "the fault {to} was injected");
@@ -1440,6 +1464,45 @@ mod tests {
}
}
/// improvise-felo.6: runs nest by `(level, depth)` inside one category, so
/// the end of a quarter leaves its year open. See the evidence file.
#[test]
fn a_quarters_end_leaves_its_year_open_on_the_same_level() {
let group = |depth: usize, name: &str| {
FrameEntry::Group(FrameGroup {
level: 0,
depth,
name: name.into(),
collapsed: false,
})
};
let frame = AxisFrame {
cats: vec!["Date".into()],
first: 0,
entries: vec![
group(0, "2024"),
group(1, "Q1"),
FrameEntry::item(vec!["Jan".into()]),
FrameEntry::GroupEnd { level: 0, depth: 1 },
FrameEntry::item(vec!["Direct".into()]),
group(1, "Q2"),
FrameEntry::item(vec!["Apr".into()]),
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Undated".into()]),
],
};
let names: Vec<Option<&str>> = frame
.groups()
.into_iter()
.map(|g| g.map(|g| g.name.as_str()))
.collect();
assert_eq!(
names,
vec![Some("Q1"), Some("2024"), Some("Q2"), None],
"Direct sits in 2024 after Q1 ends; Undated in nothing"
);
}
/// improvise-gd83's property, asserted in the crate that owns the type.
/// See `context/test-evidence/protocol.md`.
#[test]
@@ -1467,10 +1530,11 @@ mod tests {
entries: vec![
FrameEntry::Group(FrameGroup {
level: 0,
depth: 0,
name: "Q1".into(),
collapsed: true,
}),
FrameEntry::GroupEnd { level: 0 },
FrameEntry::GroupEnd { level: 0, depth: 0 },
FrameEntry::item(vec!["Sep".into()]),
],
};
@@ -1748,12 +1812,14 @@ mod tests {
let views = ViewListCache::folded(&[vec![ViewSummary {
name: "Default".into(),
display: "Default".into(),
axes: vec![("Region".into(), Axis::Row)],
}]]);
assert_eq!(views.views[0].name, "Default");
let tiles = TileBarCache::folded(&[vec![TileEntry {
category: "Region".into(),
display: "Region".into(),
axis: Axis::Page,
kind: crate::CategoryKind::Regular,
}]]);
@@ -2148,10 +2214,12 @@ mod tests {
Projection::Formulas(vec!["Profit = Revenue - Cost".into()]),
Projection::ViewList(vec![ViewSummary {
name: "Default".into(),
display: "Default".into(),
axes: vec![("Region".into(), Axis::None)],
}]),
Projection::TileBar(vec![TileEntry {
category: "Region".into(),
display: "Region".into(),
axis: Axis::Page,
kind: crate::CategoryKind::Regular,
}]),
@@ -2189,6 +2257,7 @@ mod tests {
Projection::Formulas(vec!["Profit = Revenue - Cost".into()]),
Projection::TileBar(vec![TileEntry {
category: "Region".into(),
display: "Region".into(),
axis: Axis::Page,
kind: crate::CategoryKind::Regular,
}]),
crates/improvise-protocol/src/resolve.rs
+116 -35
View File
@@ -27,6 +27,12 @@ pub enum KeyOutcome {
/// Resolved here as a **request**: the caller sends the frame and folds
/// nothing. Why it is not a `Command`: `context/code/clients.md`.
Request(Request),
/// Fold commands locally, then send a request: mode changes here and
/// download bytes come from the server (improvise-2mqz).
LocalThenRequest {
cmds: Vec<Command>,
request: Request,
},
/// Bound to a command this client cannot compose. Forwarded to the host,
/// which today answers nothing — `context/code/clients.md`.
Upstream(KeyEvent),
@@ -37,10 +43,16 @@ pub enum KeyOutcome {
/// What a key may ask its **host** to do, as opposed to what it may compose.
/// Narrow on purpose — the argument is in `context/code/clients.md`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Request {
/// Write the document back where the host keeps it (improvise-368h).
Save,
/// Get the document as bytes for download (improvise-2mqz). The filename
/// is what the user typed in the SavePrompt minibuffer.
DownloadDocument { filename: String },
/// Get the active view as CSV bytes for download (improvise-2mqz). The filename
/// is what the user typed in the ExportPrompt minibuffer.
DownloadCsv { filename: String },
}
impl Request {
@@ -49,6 +61,8 @@ impl Request {
pub fn frame(self) -> ClientFrame {
match self {
Request::Save => ClientFrame::Save,
Request::DownloadDocument { filename } => ClientFrame::DownloadDocument { filename },
Request::DownloadCsv { filename } => ClientFrame::DownloadCsv { filename },
}
}
}
@@ -92,6 +106,9 @@ pub fn resolve(state: &ClientState, event: KeyEvent) -> KeyOutcome {
Binding::Cmd { name, args } => match step(state, event, name, args) {
Step::Local(commands) | Step::Declined(commands) => KeyOutcome::of(commands),
Step::Request(request) => KeyOutcome::Request(request),
Step::LocalThenRequest { cmds, request } => {
KeyOutcome::LocalThenRequest { cmds, request }
}
Step::Upstream => KeyOutcome::Upstream(event),
},
// All or nothing, each step resolved against the *unchanged* state.
@@ -131,7 +148,7 @@ fn chord(state: &ClientState, event: KeyEvent, binding: Option<&Binding>) -> Key
Step::Local(commands) | Step::Declined(commands) => Some(commands),
// DISCARDED, which is why the guard refuses both classes rather
// than counting them answered — `context/code/clients.md`.
Step::Request(_) | Step::Upstream => None,
Step::Request(_) | Step::LocalThenRequest { .. } | Step::Upstream => None,
},
Some(Binding::Sequence(steps)) => sequence(state, event, steps),
};
@@ -294,7 +311,7 @@ fn sequence(
Step::Declined(said) => return Some(said),
// A request is a frame of its own and a sequence is one frame, so
// there is nowhere to put it. No binding does — clients.md.
Step::Request(_) | Step::Upstream => return None,
Step::Request(_) | Step::LocalThenRequest { .. } | Step::Upstream => return None,
}
}
Some(commands)
@@ -306,22 +323,31 @@ enum Step {
Local(Vec<Command>),
Declined(Vec<Command>),
Request(Request),
/// Fold commands locally, THEN send a request. For commit-save/commit-export:
/// the mode change folds here, the download request goes to the server.
LocalThenRequest {
cmds: Vec<Command>,
request: Request,
},
Upstream,
}
/// One name, answered. Each list is checked before the next, so no name is in
/// two of them and [`composed`] has no arm for any earlier one.
fn step(state: &ClientState, event: KeyEvent, name: &str, args: &[String]) -> Step {
match requested(name) {
Some(request) => Step::Request(request),
None => match declined(name, args).or_else(|| refused_here(state, name)) {
Some(said) => Step::Declined(vec![Command::SetStatus(said)]),
None => match committed(state, name, args) {
Some(answer) => answer,
None => match composed(state, event, name, args) {
Some(commands) => Step::Local(commands),
None => Step::Upstream,
},
if let Some(request) = requested(name) {
return Step::Request(request);
}
if let Some(step) = download_request(state, name) {
return step;
}
match declined(name, args).or_else(|| refused_here(state, name)) {
Some(said) => Step::Declined(vec![Command::SetStatus(said)]),
None => match committed(state, name, args) {
Some(answer) => answer,
None => match composed(state, event, name, args) {
Some(commands) => Step::Local(commands),
None => Step::Upstream,
},
},
}
@@ -365,6 +391,36 @@ fn requested(name: &str) -> Option<Request> {
}
}
/// Download requests that need to read a buffer and fold a mode change locally
/// before sending the request (improvise-2mqz).
fn download_request(state: &ClientState, name: &str) -> Option<Step> {
let (buffer_name, make_request): (&str, fn(String) -> Request) = match name {
"commit-save" => ("save", |f| Request::DownloadDocument { filename: f }),
"commit-export" => ("export", |f| Request::DownloadCsv { filename: f }),
_ => return None,
};
let filename = read_buffer(&state.view, buffer_name).trim().to_string();
if filename.is_empty() {
return Some(Step::Declined(vec![Command::SetStatus(format!(
"No filename given — not {}",
if name == "commit-save" {
"saved"
} else {
"exported"
}
))]));
}
// This owns the clear and mode change because LocalThenRequest cannot be
// part of a Binding::Sequence (improvise-2mqz).
Some(Step::LocalThenRequest {
cmds: vec![
write_buffer(buffer_name, String::new()),
Command::ChangeMode(AppMode::Normal),
],
request: make_request(filename),
})
}
/// The commands this client cannot compose and answers **out loud**; the one
/// home for the wording. Why silence was worse: clients.md (improvise-rmvx).
fn declined(name: &str, args: &[String]) -> Option<String> {
@@ -681,8 +737,8 @@ fn write_buffer(name: &str, value: String) -> Command {
/// to decline by name. A dead room is worse than a locked door: clients.md.
fn stranding(mode: &AppMode) -> Option<&'static str> {
match mode {
AppMode::SavePrompt { .. } => Some("saving to a path"),
AppMode::ExportPrompt { .. } => Some("exporting"),
// SavePrompt and ExportPrompt were stranded before improvise-2mqz added
// DownloadDocument and DownloadCsv frames. Now they have wire forms.
AppMode::CommandMode { .. } => Some("the command line"),
_ => None,
}
@@ -1892,11 +1948,13 @@ mod tests {
crate::CategoryTreeUpdate::Categories(vec![
crate::CategoryRow {
name: "Region".into(),
display: "Region".into(),
items: vec!["North".into(), "South".into()],
axis: Axis::Row,
},
crate::CategoryRow {
name: "Product".into(),
display: "Product".into(),
items: vec!["Shirts".into()],
axis: Axis::Column,
},
@@ -1975,11 +2033,13 @@ mod tests {
state.tiles.tiles = vec![
crate::TileEntry {
category: "Region".into(),
display: "Region".into(),
axis: Axis::Row,
kind: crate::CategoryKind::Regular,
},
crate::TileEntry {
category: "Product".into(),
display: "Product".into(),
axis: Axis::Column,
kind: crate::CategoryKind::Regular,
},
@@ -2334,10 +2394,12 @@ mod tests {
state.fold(&Command::Project(Projection::ViewList(vec![
crate::ViewSummary {
name: "Default".into(),
display: "Default".into(),
axes: vec![],
},
crate::ViewSummary {
name: "Plan".into(),
display: "Plan".into(),
axes: vec![],
},
])));
@@ -2617,6 +2679,7 @@ mod tests {
.into_iter()
.map(|name| crate::CategoryRow {
name: name.into(),
display: name.into(),
items: vec!["North".into(), "South".into()],
axis: Axis::Row,
})
@@ -2635,6 +2698,7 @@ mod tests {
.into_iter()
.map(|name| crate::ViewSummary {
name: name.into(),
display: name.into(),
axes: Vec::new(),
})
.collect();
@@ -2642,6 +2706,7 @@ mod tests {
.into_iter()
.map(|category| crate::TileEntry {
category: category.into(),
display: category.into(),
axis: Axis::Row,
kind: crate::CategoryKind::Regular,
})
@@ -2661,6 +2726,15 @@ mod tests {
.insert("category".into(), "Product".into());
state.view.buffers.insert("item".into(), "Shirts".into());
state.view.buffers.insert("edit".into(), "42".into());
// commit-save and commit-export need their buffers filled (improvise-2mqz).
state
.view
.buffers
.insert("save".into(), "data.improv".into());
state
.view
.buffers
.insert("export".into(), "data.csv".into());
state
}
@@ -2678,7 +2752,7 @@ mod tests {
fn heard(state: &mut ClientState, outcome: KeyOutcome) -> Heard {
let commands = match outcome {
KeyOutcome::Upstream(_) | KeyOutcome::Ignored => return Heard::Silent,
KeyOutcome::Request(_) => return Heard::Acts,
KeyOutcome::Request(_) | KeyOutcome::LocalThenRequest { .. } => return Heard::Acts,
KeyOutcome::Local(commands) => commands,
};
let quiet = |view: &ViewState| ViewState {
@@ -2826,8 +2900,6 @@ mod tests {
// Reached only from a mode `declined` refuses at the door, so a
// browser is never standing in front of these.
"COMMAND: Enter execute-command Silent",
"EXPORT: Enter commit-export Silent",
"SAVE AS: Enter commit-save Silent",
// Honestly unavailable, and saying so in `declined`'s words.
"NORMAL: > drill-into-cell Declines",
"NORMAL: : enter-mode Declines",
@@ -3482,37 +3554,45 @@ mod tests {
);
}
/// The refusal above is only half the property: the bug guarded is
/// *entering a room with no exit*. See `context/test-evidence/clients.md`.
/// A client must not enter a room with no exit. SavePrompt and ExportPrompt
/// now have download frames; see `context/test-evidence/clients.md`.
#[test]
fn a_prompt_this_client_cannot_commit_is_never_entered() {
let state = in_mode(AppMode::Normal);
for (key, mode, subject) in [
let mode = "command-mode";
let KeyOutcome::Local(said) = resolve(&state, ev(KeyCode::Char(':'))) else {
panic!("{mode} must be refused *out loud*, not in silence");
};
let [Command::SetStatus(msg)] = &said[..] else {
panic!("{mode}: expected one status, got {said:?}");
};
assert!(
msg.contains("command line"),
"{mode}: the refusal must name its own subject, got {msg}"
);
assert!(
!said.iter().any(|c| matches!(c, Command::ChangeMode(_))),
"{mode} must not be entered by a client that cannot commit it"
);
// SavePrompt and ExportPrompt ARE now enterable (improvise-2mqz).
for (key, mode_name) in [
(
KeyEvent::new(KeyCode::Char('w'), KeyModifiers::CONTROL),
"save-prompt",
"saving",
),
(
KeyEvent::new(KeyCode::Char('e'), KeyModifiers::CONTROL),
"export-prompt",
"exporting",
),
(ev(KeyCode::Char(':')), "command-mode", "command line"),
] {
let KeyOutcome::Local(said) = resolve(&state, key) else {
panic!("{mode} must be refused *out loud*, not in silence");
};
let [Command::SetStatus(msg)] = &said[..] else {
panic!("{mode}: expected one status, got {said:?}");
let outcome = resolve(&state, key);
let KeyOutcome::Local(cmds) = outcome else {
panic!("{mode_name} should resolve locally now that it has a wire form");
};
assert!(
msg.contains(subject),
"{mode}: the refusal must name its own subject, got {msg}"
);
assert!(
!said.iter().any(|c| matches!(c, Command::ChangeMode(_))),
"{mode} must not be entered by a client that cannot commit it"
cmds.iter().any(|c| matches!(c, Command::ChangeMode(_))),
"{mode_name} should be entered now that commit-save/commit-export have wire forms"
);
}
@@ -3747,6 +3827,7 @@ mod tests {
let mut tiles = in_mode(AppMode::TileSelect);
tiles.tiles.tiles = vec![crate::TileEntry {
category: "Region".into(),
display: "Region".into(),
axis: improvise_core::view::Axis::Row,
kind: crate::CategoryKind::Regular,
}];
crates/improvise-protocol/src/viewmodel.rs
+5 -1
View File
@@ -11,9 +11,10 @@ use crate::render::{CategoryTreeCache, ViewSummary};
/// so the entry list *is* what the panel draws.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CatTreeEntry {
/// Category header row: name, item count, expanded?
/// Category header row: name (the key), what it shows, item count, expanded?
Category {
name: String,
display: String,
item_count: usize,
expanded: bool,
axis: Axis,
@@ -49,6 +50,7 @@ pub fn flatten_cat_tree(
let is_expanded = expanded.contains(&row.name);
entries.push(CatTreeEntry::Category {
name: row.name.clone(),
display: row.display.clone(),
item_count: row.items.len(),
expanded: is_expanded,
axis: row.axis,
@@ -94,6 +96,7 @@ mod tests {
.iter()
.map(|(name, items)| CategoryRow {
name: (*name).to_string(),
display: (*name).to_string(),
items: items.iter().map(|i| i.to_string()).collect(),
axis: Axis::None,
})
@@ -112,6 +115,7 @@ mod tests {
tree,
vec![CatTreeEntry::Category {
name: "Region".into(),
display: "Region".into(),
item_count: 2,
expanded: false,
axis: Axis::None,

Some files were not shown because too many files have changed in this diff Show More