From f1011d7f6faac9ba4ecd109fbaceec5f7d16bb12 Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sun, 20 Sep 2026 13:37:20 -0700 Subject: [PATCH 1/7] feat(docs): publish the context/ prose as a site folded into Pages The landing page is the site root and the app is /app/; the prose now sits at /docs/, assembled by the same web/assemble.sh the Pages job and the browser harness both run, so what ships and what is tested are one tree. Published: design-notes, design-principles, repo-map, the three ADRs, the seven context/code/ arguments, and the 501 test-evidence entries as their own clearly-labelled section. Not published: context/plan.md, which is an agent prompt about a launch rather than a description of the program, and context/cloud-agents.md, which is addressed to nobody who visits a website. Neither is referenced by a published document outside a code fence, so leaving them out dangles nothing. The reasoning is context/code/doc-site.md, which is itself published. Two things the corpus needed that a generator would not have given it. The ~190 improvise-xxxx citations resolve to a generated ticket index built from the already-public .beads/issues.jsonl rather than to an invented URL that would 404; an id the export does not yet carry stays plain text. And a code span that is wholly a repo path links to GitHub at the assembled ref, but only when the file exists on disk, so a stale citation degrades to text. The generator link-checks its own output and exits non-zero on a break, and assemble.sh runs under set -eu, so a dead cross-reference fails the assembly rather than shipping. It earned that on its first two runs: it caught a fence-close regex comparing a three-character marker to a one-character string, which swallowed every document from its first code block onward, and a list parser that ended the gotcha list at the first blank line, emitting 1 of the 48 anchors the jump strip links to. Proved by injection afterwards -- renaming the anchors reports 48 missing anchors, pointing the ticket links at an ungenerated page reports 763 missing pages. No framework, no node_modules for the site, no build step beyond one zero-dependency node script, matching the landing page's posture. improvise-djoo Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 6 + context/code/doc-site.md | 154 ++++++++++ docs/index.html | 4 +- scripts/build-doc-site.mjs | 585 +++++++++++++++++++++++++++++++++++++ web/assemble.sh | 5 + 5 files changed, 753 insertions(+), 1 deletion(-) create mode 100644 context/code/doc-site.md create mode 100644 scripts/build-doc-site.mjs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0015f78a..126eccdd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -228,6 +228,12 @@ jobs: - name: Build the wasm bundle run: ./web/build.sh + # assemble.sh generates the prose site with node. The runner image ships + # one, but pinning it here is what makes a local run evidence about this. + - uses: actions/setup-node@v4 + with: + node-version: "24" + # `web/assemble.sh`, not an inline copy of it: the browser harness job # below assembles the same tree, and a harness that runs against a # different arrangement of files is testing something nobody deploys. diff --git a/context/code/doc-site.md b/context/code/doc-site.md new file mode 100644 index 00000000..4e900781 --- /dev/null +++ b/context/code/doc-site.md @@ -0,0 +1,154 @@ +# The documentation site — the arguments + +`scripts/build-doc-site.mjs` turns the prose in `context/` into the static site +served at `/docs/` on GitHub Pages. This is the reasoning behind what it +publishes, what it deliberately does not, and why it is a hundred-odd lines of +node rather than a static site generator (improvise-djoo). + +--- + +## 1. What is published, and what is not + +The manifest is the `SECTIONS` array at the top of the script, and **a document +is published only by being named there**. That is the point: a glob over +`context/**` would publish the next working file somebody drops in, and the +decision about what a stranger should read is not one to make by accident. + +Published, in four sections: + +| Section | Contents | Why | +|---|---|---| +| Start here | `docs/design-notes.md`, `context/design-principles.md`, `context/repo-map.md` | What the project is for, the doctrine, and the single best explanation of how it is built. | +| Decision records | `context/adr/*.md` | An ADR is written for an outside reader by construction — it states the options that were rejected, which is the part that is worth reading and the part that is nowhere else. | +| The arguments, by area | `context/code/*.md` | This is the reasoning that used to live in code comments and was moved out under the two-line cap (design-principles §6). It is the most valuable prose in the repository and it is addressed to exactly the reader a documentation site has. | +| Test evidence | `context/test-evidence/*.md` | 501 entries, one per test whose narrative is longer than its code. | + +Not published: + +- **`context/plan.md`** is a *prompt*: it opens "You are working in the + `improvise` repository… Your job is to prepare this repository for a public + Show HN launch", and it instructs a reader to audit a `context/SPEC.md` that + no longer exists. It is an instruction to an agent about a launch, not a + description of the program, and publishing a launch plan on the site the + launch points at inverts the two. +- **`context/cloud-agents.md`** is operational notes for agents running in a + cloud session — tracker hydration, `bd` pitfalls, what the sandbox blocks. + It is true and it is useful and it is addressed to nobody who visits a + website. + +Neither is referenced by any published document except as a row in +`repo-map.md`'s file inventory, which is inside a code fence, so leaving them +out creates no dangling link. That was checked rather than assumed. + +**Test evidence is the one judgement call worth stating.** The argument for +leaving it out is that 501 entries about individual tests are noise to a reader +who wanted to know what a pivot table is. The argument for keeping it is +stronger: every one of the seven `context/code/*.md` documents points at its +evidence file by name, so omitting it would break the published prose in +exactly the way this site is otherwise careful not to; and the evidence file is +the most distinctive artefact the repository has — a written record of the +injected fault behind each load-bearing test is not a thing most projects can +show. It is therefore published as its **own section**, last, described as a +contributor reference rather than a tour, so nobody wanders into it expecting +a narrative. + +## 2. Ticket ids resolve to a page, never to a URL that does not exist + +The prose cites `improvise-xxxx` roughly 190 times. On a public site those are +meaningless strings, and the obvious repair — link them to an issue tracker — +is unavailable: beads keeps its issues in a Dolt database on a git ref, and +there is no public issue URL to point at. Inventing one would produce 190 +links that 404, which is worse than plain text because a link is a promise. + +So the ids resolve **inward**. `tickets.html` is generated from +`.beads/issues.jsonl` — already committed, already public — and carries id, +status and title, nothing else; descriptions are internal working notes and are +not the site's business. Each citation in the prose becomes a link to that +page's anchor, with the title in a `title=` attribute so a hover answers the +question without a navigation. + +**`issues.jsonl` is a passive export and lags the database**, which is not a +defect to work around but a fact to render honestly: an id the export does not +carry is emitted as plain text with a dotted underline and no link, and the +ticket page lists it as "not exported". The site therefore never links to an +anchor it did not generate, whatever state the export is in. At the time of +writing 8 of 178 linkified ids are in that state. A generator that instead +linked every id it saw would produce a page that is right today and broken the +first time somebody cites a fresh ticket. + +## 3. Source paths link only where the file exists + +A code span whose entire content is a repo path becomes a link — to the +generated page when the path is one of the published documents, to GitHub +otherwise. Two constraints make that safe. The whole span must be the path, so +a sentence that merely mentions a directory is untouched; and **the path must +exist on disk at generation time**, so a citation that has gone stale degrades +to plain text rather than to a 404. The GitHub ref is the cache-bust token +`web/assemble.sh` already receives, used only when it is a 40-hex sha and +falling back to `main` otherwise — a local `dev` build must not emit +`blob/dev/…`. + +## 4. The link check is the test, and it is part of the build + +The generator collects every anchor it emits and every internal href it writes, +resolves them against each other, and **exits non-zero** if any href names a +page it did not generate or a fragment that page does not carry. +`web/assemble.sh` runs under `set -eu`, so a broken cross-reference fails the +assembly rather than shipping. + +This is not ceremony. It caught two real defects on its first two runs: a +fence-close regex that compared a three-character marker against a +one-character string, which swallowed every document from its first code block +onward; and a list parser that ended the gotcha list at the first blank line, +so 47 of the 48 gotcha anchors were never emitted while the "jump to gotcha N" +strip linked to all of them. Both were invisible in a rendered page that +otherwise looked fine. + +It is proved by injection rather than asserted: renaming the emitted anchor ids +makes it report 48 missing anchors, and pointing the ticket links at a page +name that is never generated makes it report 763 missing pages — two different +diagnoses, because "the page is absent" and "the page is there but the anchor +is not" call for different repairs. + +What it does **not** cover is links that leave the `docs/` subtree — the ones +to `../index.html` and `../app/`. Those are checked against the assembled +`site/` tree by hand at the moment, and the honest statement is that a rename +of the app directory would not be caught here. + +## 5. Why a node script and not a generator + +The landing page is hand-written HTML with no build step and one pinned CDN +script; the same posture applies here. A static site generator would add a +dependency tree, a lockfile and a toolchain to a repository whose site is +currently four files, in exchange for features this corpus does not use — the +docs contain no front matter, no shortcodes and, as it happens, **not a single +markdown link**. Every cross-reference in them is prose naming a path or an +id, which is precisely why §2 and §3 are the interesting parts and Markdown +rendering is the boring part. + +So the renderer is a deliberately small subset covering exactly what the corpus +contains: ATX headings, fenced code, pipe tables, blockquotes, ordered and +unordered lists with indented continuations, thematic breaks, and inline code, +strong, emphasis and links. Anything it does not recognise becomes paragraph +text, so **no input line is ever dropped** — the failure mode is an ugly +paragraph, never a missing one. + +Two details in it are load-bearing and easy to undo. **Code spans are replaced +by placeholders before emphasis is matched**, because this corpus routinely +opens `**` on one side of a code span and closes it on the other; matching +emphasis per segment silently drops that bold, which is a fidelity loss with +no visible symptom. And **strong is matched before emphasis, with a body that +may contain a single `*` pair**, because `[^*]+` cannot match +`**a *b* c**` and the leftover `*` then pairs across the bold — which is how a +single sentence in gotcha 33 came out as three italic paragraphs. + +## 6. Navigation + +`repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and +splitting it per gotcha was rejected: the gotchas cross-reference each other +constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader +following those wants a scroll, not 48 page loads. It stays one page, with an +id on every list item and a compact strip of 48 numbered links under the +heading. The ids come from the list item's **own number**, not from a counter, +so an inserted gotcha renumbers the anchors exactly as it renumbers the prose +that cites them. diff --git a/docs/index.html b/docs/index.html index 090bea45..ebcfc879 100644 --- a/docs/index.html +++ b/docs/index.html @@ -134,6 +134,7 @@ ▶ Run improvise in your browser + Documentation Source on GitHub @@ -191,7 +192,8 @@ nix profile install github:fiddlerwoaroof/improvise diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs new file mode 100644 index 00000000..6cde83c1 --- /dev/null +++ b/scripts/build-doc-site.mjs @@ -0,0 +1,585 @@ +#!/usr/bin/env node +// Generate the documentation site from context/ into OUT_DIR (default site/docs). +// The argument and the rationale live in context/code/doc-site.md. +import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync } from "node:fs"; +import { dirname, join, relative } from "node:path"; +import { fileURLToPath } from "node:url"; + +const root = join(dirname(fileURLToPath(import.meta.url)), ".."); +const out = process.argv[2] || join(root, "site", "docs"); +const token = process.argv[3] || ""; +const ref = /^[0-9a-f]{40}$/.test(token) ? token : "main"; +const REPO = "https://github.com/fiddlerwoaroof/improvise"; + +// --------------------------------------------------------------------------- +// The manifest. A doc is published only by being named here — see +// context/code/doc-site.md for what was left out and why. +const SECTIONS = [ + { + id: "start", + title: "Start here", + blurb: + "What the project is for, and the principles the code is held to.", + docs: [ + ["docs/design-notes.md", "design-notes", "Design notes"], + ["context/design-principles.md", "design-principles", "Design principles"], + ["context/repo-map.md", "repo-map", "Repository map"], + ], + }, + { + id: "adr", + title: "Decision records", + blurb: "Decisions with consequences, including the options that were rejected.", + docs: [ + ["context/adr/001-concurrent-edits.md", "adr/001-concurrent-edits", "ADR 001 — Concurrent edits"], + ["context/adr/002-naming-a-cell.md", "adr/002-naming-a-cell", "ADR 002 — Naming a cell"], + ["context/adr/003-live-backed-models.md", "adr/003-live-backed-models", "ADR 003 — Live-backed models"], + ], + }, + { + id: "code", + title: "The arguments, by area", + blurb: + "The reasoning that used to live in code comments. Written for a reader with the source open.", + docs: [ + ["context/code/core.md", "code/core", "improvise-core"], + ["context/code/formula-and-io.md", "code/formula-and-io", "improvise-formula and improvise-io"], + ["context/code/protocol.md", "code/protocol", "improvise-protocol"], + ["context/code/command.md", "code/command", "improvise-command"], + ["context/code/clients.md", "code/clients", "The clients"], + ["context/code/tui.md", "code/tui", "The TUI"], + ["context/code/gp.md", "code/gp", "gp-tree and improvise-gp"], + ], + }, + { + id: "evidence", + title: "Test evidence", + blurb: + "For each test whose narrative is longer than its code: what broke, what fault was injected, and why the test can fail. Contributor reference, not a tour.", + docs: [ + ["context/test-evidence/core.md", "test-evidence/core", "improvise-core"], + ["context/test-evidence/formula-and-io.md", "test-evidence/formula-and-io", "improvise-formula and improvise-io"], + ["context/test-evidence/protocol.md", "test-evidence/protocol", "improvise-protocol"], + ["context/test-evidence/command.md", "test-evidence/command", "improvise-command"], + ["context/test-evidence/clients.md", "test-evidence/clients", "The clients"], + ["context/test-evidence/tui.md", "test-evidence/tui", "The TUI"], + ["context/test-evidence/gp.md", "test-evidence/gp", "gp-tree and improvise-gp"], + ], + }, + { + id: "meta", + title: "About this site", + blurb: "What is published here, what is not, and why.", + docs: [["context/code/doc-site.md", "doc-site", "About this site"]], + }, +]; + +const PAGE_BY_SOURCE = new Map(); +for (const s of SECTIONS) for (const [src, slug] of s.docs) PAGE_BY_SOURCE.set(src, `${slug}.html`); + +// --------------------------------------------------------------------------- +// The ticket index. `.beads/issues.jsonl` is a passive export and lags the Dolt +// database, so an id it does not carry is rendered as plain text rather than a +// link that 404s. +function readTickets() { + const path = join(root, ".beads", "issues.jsonl"); + const map = new Map(); + if (!existsSync(path)) return map; + for (const line of readFileSync(path, "utf8").split("\n")) { + if (!line.trim()) continue; + let o; + try { + o = JSON.parse(line); + } catch { + continue; + } + if (o && o._type === "issue" && typeof o.id === "string" && typeof o.title === "string") { + map.set(o.id, { title: o.title, status: o.status || "unknown", kind: o.issue_type || o.type || "" }); + } + } + return map; +} +const TICKETS = readTickets(); + +// Tokens that look like an id but name a crate or a package. Anchored, because +// an unanchored alternative is a prefix match that silently excludes a real id. +const NON_TICKET = new Set( + "core formula io protocol command gp ws worker wasm harness registry side".split(" "), +); + +// --------------------------------------------------------------------------- +// Markdown. A deliberately small subset, covering exactly what the corpus uses; +// anything unrecognised becomes paragraph text, so no input line is ever lost. +const esc = (s) => s.replace(/&/g, "&").replace(//g, ">"); + +function slugify(s) { + return ( + s + .toLowerCase() + .replace(/<[^>]+>/g, "") + .replace(/&[a-z]+;/g, "") + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, "") || "section" + ); +} + +// A code span whose whole content is a repo path becomes a link: to the +// generated page when the path is a published doc, to GitHub otherwise. The +// path must exist on disk, so a stale citation stays plain text. +function pathHref(p, depth) { + const clean = p.replace(/^\.\//, ""); + if (!/^[A-Za-z0-9_.\/-]+$/.test(clean) || !/[\/.]/.test(clean)) return null; + if (clean.includes("..") || clean.startsWith("/")) return null; + if (PAGE_BY_SOURCE.has(clean)) return "../".repeat(depth) + PAGE_BY_SOURCE.get(clean); + if (!existsSync(join(root, clean))) return null; + return `${REPO}/blob/${ref}/${clean}`; +} + +function inline(text, ctx) { + // Code spans first: a backtick run opens a span only if a run of the same + // length closes it, so an unmatched ``` in prose stays literal. + const parts = []; + let i = 0; + while (i < text.length) { + if (text[i] === "`") { + let n = 0; + while (text[i + n] === "`") n++; + const fence = "`".repeat(n); + const end = text.indexOf(fence, i + n); + const after = end >= 0 ? end + n : -1; + if (end >= 0 && (after >= text.length || text[after] !== "`")) { + parts.push({ code: text.slice(i + n, end) }); + i = after; + continue; + } + } + const next = text.indexOf("`", i + 1); + parts.push({ text: text.slice(i, next < 0 ? text.length : next) }); + i = next < 0 ? text.length : next; + } + + // Code spans stand aside as placeholders while emphasis is matched, because + // the corpus routinely opens `**` on one side of a code span and closes it on + // the other; matching per segment silently drops that bold. + const held = []; + let s = parts + .map((p) => { + if (p.code === undefined) return esc(p.text); + const body = `${esc(p.code)}`; + const href = pathHref(p.code.trim(), ctx.depth); + held.push(href ? `${body}` : body); + return `\u0000${held.length - 1}\u0000`; + }) + .join(""); + + s = s.replace(/\[([^\]]+)\]\(([^)\s]+)\)/g, (_m, label, href) => `${label}`); + // Strong first, and its body may contain a single `*` pair: `[^*]+` cannot + // match `**a *b* c**`, and the leftover `*` then pairs across the bold. + s = s.replace(/\*\*((?:[^*]|\*(?!\*))+)\*\*/g, "$1"); + s = s.replace(/\*([^*]+)\*/g, "$1"); + s = s.replace(/\bimprovise-([a-z0-9]{3,8})\b/g, (m, tail) => { + if (NON_TICKET.has(tail)) return m; + ctx.cited.add(m); + if (!TICKETS.has(m)) { + ctx.unresolved.add(m); + return `${m}`; + } + const t = TICKETS.get(m); + const href = `${"../".repeat(ctx.depth)}tickets.html#${m}`; + return `${m}`; + }); + return s.replace(/\u0000(\d+)\u0000/g, (_m, n) => held[Number(n)]); +} + +// Block parse. Returns { html, toc }. +function renderMarkdown(src, ctx) { + const lines = src.split("\n"); + const toc = []; + const seen = new Map(); + const html = blocks(lines); + return { html: html.join("\n"), toc }; + + function heading(level, text) { + const rendered = inline(text, ctx); + let id = slugify(text); + const n = (seen.get(id) || 0) + 1; + seen.set(id, n); + if (n > 1) id = `${id}-${n}`; + ctx.anchors.add(id); + if (level <= 3) toc.push({ level, id, text: rendered }); + const mark = level > 1 ? `#` : ""; + return `${mark}${rendered}`; + } + + function blocks(ls) { + const outHtml = []; + let i = 0; + let para = []; + const flush = () => { + if (para.length) outHtml.push(`

${inline(para.join(" "), ctx)}

`); + para = []; + }; + while (i < ls.length) { + const line = ls[i]; + const fence = line.match(/^(\s*)(`{3,}|~{3,})\s*([A-Za-z0-9_-]*)\s*$/); + if (fence) { + flush(); + const close = new RegExp(`^\\s*${fence[2][0] === "~" ? "~" : "\`"}{3,}\\s*$`); + const body = []; + i++; + while (i < ls.length && !close.test(ls[i])) { + body.push(ls[i]); + i++; + } + i++; + outHtml.push(`
${esc(body.join("\n"))}
`); + continue; + } + const h = line.match(/^(#{1,6})\s+(.*)$/); + if (h) { + flush(); + outHtml.push(heading(h[1].length, h[2].trim())); + i++; + continue; + } + if (/^\s*(-{3,}|\*{3,}|_{3,})\s*$/.test(line) && !para.length) { + flush(); + outHtml.push("
"); + i++; + continue; + } + if (/^\s*\|/.test(line) && i + 1 < ls.length && /^\s*\|[\s:|-]+\|?\s*$/.test(ls[i + 1])) { + flush(); + const rows = []; + while (i < ls.length && /^\s*\|/.test(ls[i])) { + rows.push(ls[i]); + i++; + } + outHtml.push(table(rows)); + continue; + } + if (/^\s*>/.test(line)) { + flush(); + const body = []; + while (i < ls.length && /^\s*>/.test(ls[i])) { + body.push(ls[i].replace(/^\s*>\s?/, "")); + i++; + } + outHtml.push(`
${blocks(body).join("\n")}
`); + continue; + } + const li = line.match(/^(\s*)([-*+]|\d{1,3}\.)\s+(.*)$/); + if (li) { + flush(); + const [listHtml, next] = list(ls, i, li[1].length); + outHtml.push(listHtml); + i = next; + continue; + } + if (!line.trim()) { + flush(); + i++; + continue; + } + para.push(line.trim()); + i++; + } + flush(); + return outHtml; + } + + // An item's continuation lines are dedented by that item's own marker width, + // so a nested list keeps its relative indentation and reparses as a list. + function list(ls, start, indent) { + const ordered = /^\s*\d{1,3}\./.test(ls[start]); + const items = []; + let i = start; + let first = null; + while (i < ls.length) { + const m = ls[i].match(/^(\s*)([-*+]|(\d{1,3})\.)\s+(.*)$/); + if (m && m[1].length === indent) { + if (ordered && first === null) first = Number(m[3]); + items.push({ num: m[3] ? Number(m[3]) : null, cw: m[0].length - m[4].length, body: [m[4]] }); + i++; + continue; + } + if (!items.length) break; + const cur = items[items.length - 1]; + if (!ls[i].trim()) { + let j = i; + while (j < ls.length && !ls[j].trim()) j++; + if (j >= ls.length) break; + // A blank line ends the list only if what follows is neither the next + // item nor a continuation of the open one. + const isSibling = /^(\s*)([-*+]|\d{1,3}\.)\s+/.test(ls[j]) && ls[j].search(/\S/) === indent; + if (isSibling) { + i = j; + continue; + } + if (ls[j].search(/\S/) > indent) { + cur.body.push(""); + i = j; + continue; + } + break; + } + const col = ls[i].search(/\S/); + if (col <= indent) break; + cur.body.push(ls[i].slice(Math.min(col, cur.cw))); + i++; + } + const tag = ordered ? "ol" : "ul"; + const attr = ordered && first !== null && first !== 1 ? ` start="${first}"` : ""; + const rendered = items + .map((it) => { + const inner = blocks(it.body); + const body = inner.length === 1 && inner[0].startsWith("

") + ? inner[0].slice(3, -4) + : inner.join("\n"); + const id = ctx.listItemId && it.num !== null ? ctx.listItemId(it.num) : null; + if (id) ctx.anchors.add(id); + return `${body}`; + }) + .join("\n"); + return [`<${tag}${attr}>\n${rendered}\n`, i]; + } + + function table(rows) { + const cells = (r) => + r + .trim() + .replace(/^\|/, "") + .replace(/\|$/, "") + .split("|") + .map((c) => inline(c.trim(), ctx)); + const head = cells(rows[0]); + const body = rows.slice(2).map(cells); + return ( + `${head.map((c) => ``).join("")}` + + body.map((r) => `${r.map((c) => ``).join("")}`).join("") + + "
${c}
${c}
" + ); + } +} + +// --------------------------------------------------------------------------- +const CSS = `:root{--bg:#14161a;--panel:#1b1e24;--fg:#d7dae0;--dim:#8b93a1;--accent:#8ec07c;--rule:#2b2f37;--mono:ui-monospace,"SFMono-Regular","Menlo","Consolas",monospace} +*{box-sizing:border-box} +body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.65 system-ui,-apple-system,"Segoe UI",sans-serif} +a{color:var(--accent)} +.shell{display:flex;align-items:flex-start;max-width:1400px;margin:0 auto} +nav.side{position:sticky;top:0;flex:0 0 17rem;max-height:100vh;overflow-y:auto;padding:1.5rem 1rem 3rem;border-right:1px solid var(--rule);font-size:.9rem} +nav.side .brand{font-family:var(--mono);font-size:1.15rem;color:var(--accent);text-decoration:none;font-weight:600} +nav.side h3{font-size:.72rem;letter-spacing:.09em;text-transform:uppercase;color:var(--dim);margin:1.6rem 0 .4rem} +nav.side ul{list-style:none;margin:0;padding:0} +nav.side li{margin:.15rem 0} +nav.side a{display:block;padding:.2rem .45rem;border-radius:4px;color:var(--fg);text-decoration:none} +nav.side a:hover{background:var(--panel)} +nav.side a.here{background:var(--panel);color:var(--accent);font-weight:600} +main{flex:1 1 auto;min-width:0;max-width:60rem;padding:2.5rem 2rem 6rem} +h1,h2,h3,h4,h5{font-family:var(--mono);font-weight:600;letter-spacing:-.01em;line-height:1.3} +h1{font-size:2rem;margin:0 0 1.2rem;color:var(--accent)} +h2{font-size:1.25rem;margin:2.6rem 0 .6rem;padding-top:1.2rem;border-top:1px solid var(--rule)} +h3{font-size:1.02rem;margin:1.8rem 0 .4rem;color:#c8cdd6} +h4,h5{font-size:.95rem;margin:1.3rem 0 .3rem;color:var(--dim)} +.anchor{float:left;margin-left:-1.1em;padding-right:.35em;color:var(--rule);text-decoration:none;opacity:0} +h2:hover .anchor,h3:hover .anchor,h4:hover .anchor,h5:hover .anchor{opacity:1} +p{max-width:74ch} +li{max-width:74ch} +code{font-family:var(--mono);font-size:.88em;background:var(--panel);padding:.1em .3em;border-radius:3px;overflow-wrap:anywhere} +a code{background:#22262e} +pre{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:1rem;overflow-x:auto} +pre code{background:none;padding:0;font-size:.84rem;line-height:1.5;overflow-wrap:normal} +blockquote{margin:1rem 0;padding:.5rem .9rem;border-left:3px solid var(--accent);background:var(--panel)} +blockquote p{margin:.4rem 0} +table{border-collapse:collapse;margin:1rem 0;font-size:.92rem;display:block;overflow-x:auto} +th,td{border:1px solid var(--rule);padding:.4rem .6rem;text-align:left;vertical-align:top} +th{background:var(--panel)} +hr{border:0;border-top:1px solid var(--rule);margin:2rem 0} +.tid{font-family:var(--mono);font-size:.86em;white-space:nowrap} +.tid-unknown{color:var(--dim);border-bottom:1px dotted var(--rule)} +.toc{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.8rem 1.1rem;margin:0 0 2rem;font-size:.92rem} +.toc summary{cursor:pointer;color:var(--dim);font-family:var(--mono)} +.toc ul{list-style:none;margin:.6rem 0 0;padding:0;columns:2;column-gap:2rem} +.toc li{break-inside:avoid;margin:.12rem 0} +.toc li.l3{padding-left:1rem;font-size:.92em} +.toc a{text-decoration:none} +.toc a:hover{text-decoration:underline} +.jump{margin:0 0 2rem;font-size:.85rem;font-family:var(--mono)} +.jump a{display:inline-block;min-width:2.1rem;text-align:center;padding:.12rem .3rem;margin:.1rem;border:1px solid var(--rule);border-radius:4px;text-decoration:none} +.jump a:hover{border-color:var(--accent)} +.lede{color:var(--dim);max-width:70ch} +.src{font-size:.85rem;color:var(--dim);font-family:var(--mono);margin:-.6rem 0 1.8rem} +.cards{display:grid;gap:1rem;grid-template-columns:repeat(auto-fill,minmax(19rem,1fr));margin:1rem 0 2rem} +.card{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.9rem 1.1rem} +.card a{font-family:var(--mono);text-decoration:none} +.card p{color:var(--dim);font-size:.88rem;margin:.3rem 0 0} +.tickets{width:100%;display:table} +.tickets td:first-child{white-space:nowrap;font-family:var(--mono);font-size:.86rem} +.st{font-size:.78rem;color:var(--dim);text-transform:uppercase;letter-spacing:.05em} +@media(max-width:900px){.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}}`; + +function nav(currentSlug, depth) { + const up = "../".repeat(depth); + const link = (href, label, here) => + `

  • ${label}
    • `; + const parts = [ + `"); + return parts.join(""); +} + +function page({ slug, title, body, depth }) { + const up = "../".repeat(depth); + return ` + + + + +${esc(title)} — improvise docs + + + +
    +${nav(slug, depth)} +
    +${body} +
    +
    + + +`; +} + +function write(rel, content) { + const p = join(out, rel); + mkdirSync(dirname(p), { recursive: true }); + writeFileSync(p, content); + return p; +} + +// --------------------------------------------------------------------------- +rmSync(out, { recursive: true, force: true }); +mkdirSync(out, { recursive: true }); +write("style.css", CSS); + +const cited = new Set(); +const unresolved = new Set(); +const written = []; + +for (const section of SECTIONS) { + for (const [src, slug, title] of section.docs) { + const depth = slug.split("/").length - 1; + const ctx = { depth, cited, unresolved, anchors: new Set() }; + if (slug === "repo-map") ctx.listItemId = (n) => (n >= 1 && n <= 200 ? `gotcha-${n}` : null); + const md = readFileSync(join(root, src), "utf8"); + const { html, toc } = renderMarkdown(md, ctx); + + const tocHtml = toc.length + ? `
    On this page
      ${toc + .filter((t) => t.level <= 3) + .map((t) => `
    • ${t.text}
      • `) + .join("")}
    ` + : ""; + + const jump = + slug === "repo-map" + ? `

    Gotchas: ${Array.from({ length: 48 }, (_, k) => `${k + 1}`).join("")}

    ` + : ""; + + const srcLine = `

    Source: ${src}

    `; + written.push({ + rel: `${slug}.html`, + html: page({ + slug, + title, + depth, + body: `${srcLine}${tocHtml}${jump}${html}`, + }), + anchors: ctx.anchors, + }); + } +} + +// Ticket index. +{ + const ctx = { depth: 0, cited, unresolved, anchors: new Set() }; + const ids = [...cited].sort(); + const rows = ids + .map((id) => { + const t = TICKETS.get(id); + ctx.anchors.add(id); + if (!t) + return `${id}not exportedCited in the prose; the committed tracker export does not carry it.`; + return `${id}${esc(t.status)}${esc(t.title)}`; + }) + .join(""); + const body = `

    Ticket index

    +

    The prose on this site cites issue ids like improvise-9c8c. They are this project's own tracker ids, kept in-repo with beads; there is no public issue URL to link to, so this page is what they resolve to. Titles come from .beads/issues.jsonl, a passive export that lags the database — an id it does not carry is listed here without a title and is not linked from the prose.

    +

    ${ids.length} ids are cited, of which ${ids.length - unresolved.size} resolve.

    +${rows}
    IdStatusTitle
    `; + written.push({ rel: "tickets.html", html: page({ slug: "tickets", title: "Ticket index", depth: 0, body }), anchors: ctx.anchors }); +} + +// Landing page for the docs section. +{ + const cards = SECTIONS.map( + (s) => + `

    #${s.title}

    ${s.blurb}

    ${s.docs + .map(([, slug, title]) => `
    ${title}
    `) + .join("")}
    `, + ).join(""); + const body = `

    improvise — documentation

    +

    These are the working documents of the improvise repository, published as written. They are addressed to someone reading or changing the code: the arguments behind the architecture, the decisions that were made and the ones that were rejected, and the record of what each load-bearing test is guarding against.

    +

    Nothing here is a tutorial. For what the program does, see the project home or run the browser demo.

    +${cards} +

    #Conventions

    +

    Ids like improvise-9c8c are tracker issues and link to the ticket index. Paths like crates/improvise-core/src/model/types.rs link to the source on GitHub at ${ref}.

    `; + written.push({ rel: "index.html", html: page({ slug: "", title: "Documentation", depth: 0, body }), anchors: new Set(["conventions", ...SECTIONS.map((s) => s.id)]) }); +} + +for (const w of written) write(w.rel, w.html); + +// --------------------------------------------------------------------------- +// Link check over the generated output. Every internal href must resolve to a +// generated file, and every fragment to an id that page really carries. +const anchorsByPage = new Map(written.map((w) => [w.rel, w.anchors])); +const files = new Set(written.map((w) => w.rel).concat(["style.css"])); +const broken = []; +let checked = 0; +for (const w of written) { + const base = dirname(w.rel); + for (const m of w.html.matchAll(/(?:href|src)="([^"]+)"/g)) { + const raw = m[1]; + if (/^(https?:|mailto:|data:)/.test(raw)) continue; + checked++; + const [pathPart, frag] = raw.split("#"); + if (!pathPart) { + if (frag && !w.anchors.has(frag)) broken.push(`${w.rel} -> ${raw} (no such anchor on this page)`); + continue; + } + const resolved = relative(".", join(base === "." ? "" : base, pathPart)); + if (resolved.startsWith("..")) continue; // leaves the docs tree: checked below + const target = resolved.endsWith("/") ? `${resolved}index.html` : resolved; + if (!files.has(target)) broken.push(`${w.rel} -> ${raw} (no such page: ${target})`); + else if (frag && !anchorsByPage.get(target)?.has(frag)) + broken.push(`${w.rel} -> ${raw} (no such anchor: ${frag})`); + } +} + +console.log(`doc-site: ${written.length} pages -> ${out}`); +console.log(`doc-site: ${cited.size} ticket ids cited, ${cited.size - unresolved.size} resolve, ${unresolved.size} not in the export`); +console.log(`doc-site: ${checked} internal links checked, ${broken.length} broken`); +if (broken.length) { + for (const b of broken) console.error(`doc-site: broken link: ${b}`); + process.exit(1); +} diff --git a/web/assemble.sh b/web/assemble.sh index e5b90cbf..796fca50 100755 --- a/web/assemble.sh +++ b/web/assemble.sh @@ -42,6 +42,11 @@ mkdir -p "$out" cp -f "$root/docs/index.html" "$out/index.html" cp -rf "$root/docs/casts" "$out/casts" +# The prose site under `docs/`, generated from context/ by the same script the +# harness runs. It link-checks its own output and exits non-zero on a break, so +# a dead cross-reference fails the assembly rather than shipping. +node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token" + app="$out/app" mkdir -p "$app" cp -f "$root/web/index.html" "$root/web/app.js" "$root/web/style.css" "$root/web/worker.js" "$app/" From 04d8c1ff9a05b69c261fed52b180650d1fff7a22 Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sun, 20 Sep 2026 13:40:00 -0700 Subject: [PATCH 2/7] fix(docs): a source link names a directory as a tree, and `.` is not a path Two ways the path linker could emit a URL that resolves by luck or not at all, both found by reading the generated links back rather than by the link check, which only sees the site's own pages. A code span naming a directory -- `crates/gp-tree/`, `context/test-evidence`, and about a dozen more -- was linked under /blob/. GitHub redirects that to /tree/ today, so it worked, but the correctness of ~30 links rested on a redirect nobody here controls. The kind is now read off the filesystem at generation time, which is the same source the existence check already uses, and the trailing slash is dropped. And a code span of punctuation alone passed the path guard: `.` is matched by both the character-class test and the "contains a slash or a dot" test, exists on disk, and is a directory, so it became a link to the repository root from the middle of a sentence. A path now has to contain at least one alphanumeric. Verified with a 40-hex token: 195 source links, zero of them `blob/main`, no blob link ending in a slash, no tree link naming a file, and no link to `.`. improvise-djoo Co-Authored-By: Claude Opus 5 (1M context) --- scripts/build-doc-site.mjs | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index 6cde83c1..f197480e 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -1,7 +1,7 @@ #!/usr/bin/env node // Generate the documentation site from context/ into OUT_DIR (default site/docs). // The argument and the rationale live in context/code/doc-site.md. -import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync } from "node:fs"; +import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync, statSync } from "node:fs"; import { dirname, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; @@ -129,10 +129,16 @@ function slugify(s) { function pathHref(p, depth) { const clean = p.replace(/^\.\//, ""); if (!/^[A-Za-z0-9_.\/-]+$/.test(clean) || !/[\/.]/.test(clean)) return null; + // A span of punctuation alone (`.`, `-`, `/`) is prose, not a path. + if (!/[A-Za-z0-9]/.test(clean)) return null; if (clean.includes("..") || clean.startsWith("/")) return null; if (PAGE_BY_SOURCE.has(clean)) return "../".repeat(depth) + PAGE_BY_SOURCE.get(clean); - if (!existsSync(join(root, clean))) return null; - return `${REPO}/blob/${ref}/${clean}`; + const abs = join(root, clean); + if (!existsSync(abs)) return null; + // GitHub serves a directory under /tree/, not /blob/; relying on its + // redirect is a 404 waiting on a change of behaviour we do not control. + const kind = statSync(abs).isDirectory() ? "tree" : "blob"; + return `${REPO}/${kind}/${ref}/${clean.replace(/\/$/, "")}`; } function inline(text, ctx) { From 1a0f04856ea18fe0712d78b9fb1851b2a0aef245 Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sun, 20 Sep 2026 13:56:56 -0700 Subject: [PATCH 3/7] fix(docs): the review's four rendering defects, and the check that could see one Review of PR #62 found four live defects and four smaller ones. The most valuable is the first, because of what it says about the rest. A TOC entry wrapping a heading whose rendered text already contained a link emitted .... The parser closes the outer anchor early, so the entry became an empty unclickable stub and clicking "src/draw.rs" in the TOC navigated to GitHub. 19 occurrences over 10 pages. Both hrefs resolved perfectly, so the link check, the whole-site link check and a screenshot pass all missed it -- it answers "does every href resolve" and this is not that question. So the fix is two things: the TOC keeps a tag-free variant of each heading, AND a new check asserts no appears inside another . Proved by reverting the first half: 66 nested anchors reported, assembly exits 1. Clicking that TOC entry now lands on #src-draw-rs. esc() escaped & < > but not the quote it was interpolated into title="..." with. Five shipped bead titles contain one, and Chromium parsed the resulting tag as having three junk attributes. Now escapes " and ' as well. Today's corpus being friendly is not a reason to interpolate tracker text unescaped. Seven pages overflowed at 390px by up to 92%. The reviewer attributed it to the TOC; the TOC was real but not sufficient, and the first fix measured clean by getBoundingClientRect while the page still overflowed, because the heading text overflows its box without widening it. Only per-element scrollWidth names the culprit: hundreds of snake_case test names as headings. overflow-wrap on the headings and on .toc a, plus dropping display:table from .tickets, which had been cancelling the mobile overflow-x guard. 0 of 23 pages overflow at 390 and 1280 now, measured. table() split rows on every pipe before inline parsing, so `|Time Period|` -- this repo's own quoting syntax, in a corpus whose subject is that syntax -- was shredded into extra columns. A row that still parses, so no column count could see it. Now walked, respecting code spans and \|. Proved with a fixture: 9 tds with the fix, 12 with the old splitter reinstated. The four smaller ones: the link check counted 48 links it never verified behind a comment promising they were "checked below" when there was no below -- those are the links leaving the docs tree, and they are now resolved against the filesystem, which is why assemble.sh runs the generator LAST, once its siblings exist. The gotcha jump bar's hardcoded 48 is derived from the emitted anchors, because it failed closed on deletion and silently on addition, and the list only grows. context/code/doc-site.md cited improvise-xxxx and so broke the audit the surrounding paragraph describes; the repo-wide audit is now 189 ids, all resolving. And the Conventions paragraph's own example path now goes through inline(), so the sentence demonstrates the linking instead of asserting it. The ticket index keeps all 180 rows, open bugs and titles included, by the user's ruling: the data is already public in .beads/issues.jsonl, the page only makes it legible, and filtering it while the lede links the raw JSONL would be theatre. Gates: 1378 rust tests, 26 node --test, 16 playwright (3 skipped, webkit OPFS), all three check-comment-runs modes, 3460 site links / 0 broken, 0 nested anchors, 189 bead ids all resolving. improvise-djoo Co-Authored-By: Claude Opus 5 (1M context) --- context/code/doc-site.md | 88 +++++++++++++++++++++++---- scripts/build-doc-site.mjs | 118 ++++++++++++++++++++++++++++++------- web/assemble.sh | 12 ++-- 3 files changed, 180 insertions(+), 38 deletions(-) diff --git a/context/code/doc-site.md b/context/code/doc-site.md index 4e900781..7e61a559 100644 --- a/context/code/doc-site.md +++ b/context/code/doc-site.md @@ -54,7 +54,10 @@ a narrative. ## 2. Ticket ids resolve to a page, never to a URL that does not exist -The prose cites `improvise-xxxx` roughly 190 times. On a public site those are +The prose cites bead ids roughly 190 times — this paragraph deliberately spells +none, because a document that gives one as an example makes itself a source of +audit noise, and a placeholder is worse than a real id: it is ID-shaped and +resolves to nothing. On a public site those citations are meaningless strings, and the obvious repair — link them to an issue tracker — is unavailable: beads keeps its issues in a Dolt database on a git ref, and there is no public issue URL to point at. Inventing one would produce 190 @@ -71,10 +74,26 @@ question without a navigation. defect to work around but a fact to render honestly: an id the export does not carry is emitted as plain text with a dotted underline and no link, and the ticket page lists it as "not exported". The site therefore never links to an -anchor it did not generate, whatever state the export is in. At the time of -writing 8 of 178 linkified ids are in that state. A generator that instead -linked every id it saw would produce a page that is right today and broken the -first time somebody cites a fresh ticket. +anchor it did not generate, whatever state the export is in. A generator that +instead linked every id it saw would produce a page that is right today and +broken the first time somebody cites a fresh ticket. + +**The page publishes every cited id, open bugs and their titles included, and +that is a decision rather than a default** (user ruling). The data is already +public in `.beads/issues.jsonl` in a repository anyone can clone; the page only +makes it legible. An honest open-bug list is a normal thing for an open-source +project to have, and filtering one while the page's own lede links the raw +JSONL would be theatre. Note the second-order reason to keep it unfiltered: +a citation whose row was suppressed would be a link into a page that does not +answer it, which is the failure this whole section exists to avoid. + +**Titles are tracker data and are escaped as attribute values.** Five shipped +bead titles contain a double quote; interpolated into `title="…"` by an `esc` +that handled only `&`, `<` and `>`, they broke out of the attribute and the +browser parsed the remains as three junk attributes. `esc` now escapes `"` and +`'` as well. Today's corpus being friendly is not a reason to interpolate +untrusted-shaped text unescaped — the tracker is written by whoever files a +bug. ## 3. Source paths link only where the file exists @@ -110,10 +129,29 @@ name that is never generated makes it report 763 missing pages — two different diagnoses, because "the page is absent" and "the page is there but the anchor is not" call for different repairs. -What it does **not** cover is links that leave the `docs/` subtree — the ones -to `../index.html` and `../app/`. Those are checked against the assembled -`site/` tree by hand at the moment, and the honest statement is that a rename -of the app directory would not be caught here. +Links that **leave** the `docs/` subtree — `../index.html`, `../app/` — name +files this script does not generate, so they are resolved against the +filesystem instead. That is the whole reason `assemble.sh` runs the generator +**last**: its siblings have to exist before they can be checked. An earlier +version skipped those 48 links while still counting them as checked, behind a +comment promising they were "checked below" when there was no below. + +**And the link check is not sufficient, which is the more important lesson.** +It answers one question — does every href resolve — and a defect outside that +question is invisible to it however carefully it is run. The worked example is +a nested ``: a TOC entry wrapping a heading whose rendered text already +contained a link produced `…`, and +the HTML parser closes the outer anchor early, so the entry became an empty +unclickable stub and clicking it navigated to GitHub. Both hrefs resolved +perfectly. It shipped past the link check, past the whole-site link check, and +past a screenshot pass, and was caught by a reviewer *clicking a link*. The +repair is therefore two things, not one: the TOC keeps a tag-free variant of +each heading, **and** a separate check asserts no `` appears inside another +`` — proved by reverting the first half and watching the second report 66. + +The general form: when a check misses a defect, add the check that could have +seen it rather than only fixing the defect, and ask what *question* the +existing check was answering. ## 5. Why a node script and not a generator @@ -142,13 +180,37 @@ may contain a single `*` pair**, because `[^*]+` cannot match `**a *b* c**` and the leftover `*` then pairs across the bold — which is how a single sentence in gotcha 33 came out as three italic paragraphs. +A third, added later and for the same reason: **a table cell boundary is an +unescaped pipe outside a code span**, so the row is walked rather than +`split("|")`. This corpus documents a file format whose own quoting syntax is +`|Time Period|`, and a sentence containing one inside a table cell would have +been shredded into extra columns — a row that still parses, so no column count +could detect it. It was latent when fixed, which is the point: the trigger is +one ordinary sentence away in a document whose subject is pipe quoting. + ## 6. Navigation `repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and splitting it per gotcha was rejected: the gotchas cross-reference each other constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader following those wants a scroll, not 48 page loads. It stays one page, with an -id on every list item and a compact strip of 48 numbered links under the -heading. The ids come from the list item's **own number**, not from a counter, -so an inserted gotcha renumbers the anchors exactly as it renumbers the prose -that cites them. +id on every list item and a compact strip of numbered links under the heading. +The ids come from the list item's **own number**, not from a counter, so an +inserted gotcha renumbers the anchors exactly as it renumbers the prose that +cites them. + +The strip's contents are **derived from the anchors actually emitted**, never +from a count. A hardcoded 48 failed in only one direction: deleting a gotcha +left a link to a missing anchor, which the link check catches, while *adding* +one produced an unlinked gotcha and a clean build. The list only ever grows, so +the hardcoded form was silent in precisely the direction it would be used — +the same shape as an exclusion list that over-matches, failing in the direction +that looks like success. + +**Long unbreakable headings need `overflow-wrap`.** The test-evidence pages are +hundreds of `snake_case` test names as headings; at phone width seven of them +overflowed the viewport by up to 92%. The diagnosis is worth keeping because +the first guess was wrong: `getBoundingClientRect` reported every element +inside the viewport, because the text overflows its box without widening it. +Only `scrollWidth` per element names the culprit. Measure the property that +actually overflowed rather than the one that usually does. diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index f197480e..e384c15d 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -110,7 +110,18 @@ const NON_TICKET = new Set( // --------------------------------------------------------------------------- // Markdown. A deliberately small subset, covering exactly what the corpus uses; // anything unrecognised becomes paragraph text, so no input line is ever lost. -const esc = (s) => s.replace(/&/g, "&").replace(//g, ">"); +const esc = (s) => + s + .replace(/&/g, "&") + .replace(//g, ">") + .replace(/"/g, """) + .replace(/'/g, "'"); + +// A TOC entry must not nest an anchor inside the one wrapping it: a heading +// whose text is a path or a ticket id already carries an , and the parser +// closes the outer one early, leaving an empty unclickable stub. +const stripTags = (html) => html.replace(/<[^>]+>/g, ""); function slugify(s) { return ( @@ -212,7 +223,7 @@ function renderMarkdown(src, ctx) { seen.set(id, n); if (n > 1) id = `${id}-${n}`; ctx.anchors.add(id); - if (level <= 3) toc.push({ level, id, text: rendered }); + if (level <= 3) toc.push({ level, id, text: stripTags(rendered) }); const mark = level > 1 ? `#` : ""; return `${mark}${rendered}`; } @@ -350,14 +361,44 @@ function renderMarkdown(src, ctx) { return [`<${tag}${attr}>\n${rendered}\n`, i]; } + // A cell boundary is an UNESCAPED pipe outside a code span. Splitting the raw + // row on every `|` shreds `\|` and this repo's own `|Time Period|` syntax into + // extra columns, which no column count can detect because the row still parses. + function splitRow(r) { + const out = []; + let cur = ""; + let tick = 0; + for (let i = 0; i < r.length; i++) { + const c = r[i]; + if (c === "\\" && r[i + 1] === "|" && !tick) { + cur += "|"; + i++; + continue; + } + if (c === "`") { + let n = 0; + while (r[i + n] === "`") n++; + if (!tick) tick = n; + else if (tick === n) tick = 0; + cur += "`".repeat(n); + i += n - 1; + continue; + } + if (c === "|" && !tick) { + out.push(cur); + cur = ""; + continue; + } + cur += c; + } + out.push(cur); + if (out.length && !out[0].trim()) out.shift(); + if (out.length && !out[out.length - 1].trim()) out.pop(); + return out; + } + function table(rows) { - const cells = (r) => - r - .trim() - .replace(/^\|/, "") - .replace(/\|$/, "") - .split("|") - .map((c) => inline(c.trim(), ctx)); + const cells = (r) => splitRow(r.trim()).map((c) => inline(c.trim(), ctx)); const head = cells(rows[0]); const body = rows.slice(2).map(cells); return ( @@ -383,7 +424,7 @@ nav.side a{display:block;padding:.2rem .45rem;border-radius:4px;color:var(--fg); nav.side a:hover{background:var(--panel)} nav.side a.here{background:var(--panel);color:var(--accent);font-weight:600} main{flex:1 1 auto;min-width:0;max-width:60rem;padding:2.5rem 2rem 6rem} -h1,h2,h3,h4,h5{font-family:var(--mono);font-weight:600;letter-spacing:-.01em;line-height:1.3} +h1,h2,h3,h4,h5{font-family:var(--mono);font-weight:600;letter-spacing:-.01em;line-height:1.3;overflow-wrap:anywhere} h1{font-size:2rem;margin:0 0 1.2rem;color:var(--accent)} h2{font-size:1.25rem;margin:2.6rem 0 .6rem;padding-top:1.2rem;border-top:1px solid var(--rule)} h3{font-size:1.02rem;margin:1.8rem 0 .4rem;color:#c8cdd6} @@ -407,9 +448,9 @@ hr{border:0;border-top:1px solid var(--rule);margin:2rem 0} .toc{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.8rem 1.1rem;margin:0 0 2rem;font-size:.92rem} .toc summary{cursor:pointer;color:var(--dim);font-family:var(--mono)} .toc ul{list-style:none;margin:.6rem 0 0;padding:0;columns:2;column-gap:2rem} -.toc li{break-inside:avoid;margin:.12rem 0} +.toc li{break-inside:avoid;margin:.12rem 0;overflow-wrap:anywhere} .toc li.l3{padding-left:1rem;font-size:.92em} -.toc a{text-decoration:none} +.toc a{text-decoration:none;overflow-wrap:anywhere} .toc a:hover{text-decoration:underline} .jump{margin:0 0 2rem;font-size:.85rem;font-family:var(--mono)} .jump a{display:inline-block;min-width:2.1rem;text-align:center;padding:.12rem .3rem;margin:.1rem;border:1px solid var(--rule);border-radius:4px;text-decoration:none} @@ -420,8 +461,9 @@ hr{border:0;border-top:1px solid var(--rule);margin:2rem 0} .card{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.9rem 1.1rem} .card a{font-family:var(--mono);text-decoration:none} .card p{color:var(--dim);font-size:.88rem;margin:.3rem 0 0} -.tickets{width:100%;display:table} +.tickets{width:100%} .tickets td:first-child{white-space:nowrap;font-family:var(--mono);font-size:.86rem} +.tickets td:last-child{overflow-wrap:anywhere} .st{font-size:.78rem;color:var(--dim);text-transform:uppercase;letter-spacing:.05em} @media(max-width:900px){.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}}`; @@ -497,10 +539,17 @@ for (const section of SECTIONS) { .join("")}` : ""; - const jump = - slug === "repo-map" - ? `

    Gotchas: ${Array.from({ length: 48 }, (_, k) => `${k + 1}`).join("")}

    ` - : ""; + // Derived from the anchors actually emitted. A hardcoded count fails closed + // on a deleted gotcha (the link check catches it) and SILENTLY on an added + // one, which is the direction the list moves in. + const gotchas = [...ctx.anchors] + .map((a) => /^gotcha-(\d+)$/.exec(a)) + .filter(Boolean) + .map((m) => Number(m[1])) + .sort((a, b) => a - b); + const jump = gotchas.length + ? `

    Gotchas: ${gotchas.map((n) => `${n}`).join("")}

    ` + : ""; const srcLine = `

    Source: ${src}

    `; written.push({ @@ -538,6 +587,15 @@ for (const section of SECTIONS) { // Landing page for the docs section. { + // The two conventions go through `inline` rather than being hand-written, so + // the sentence describing the linking demonstrates it instead of asserting it. + const conventions = inline( + "Ids like `improvise-9c8c` are tracker issues and link to the ticket index below. " + + "Paths like `crates/improvise-core/src/model/types.rs` link to the source on GitHub at `" + + ref + + "`.", + { depth: 0, cited, unresolved, anchors: new Set() }, + ); const cards = SECTIONS.map( (s) => `

    #${s.title}

    ${s.blurb}

    ${s.docs @@ -549,13 +607,22 @@ for (const section of SECTIONS) {

    Nothing here is a tutorial. For what the program does, see the project home or run the browser demo.

    ${cards}

    #Conventions

    -

    Ids like improvise-9c8c are tracker issues and link to the ticket index. Paths like crates/improvise-core/src/model/types.rs link to the source on GitHub at ${ref}.

    `; +

    ${conventions}

    `; written.push({ rel: "index.html", html: page({ slug: "", title: "Documentation", depth: 0, body }), anchors: new Set(["conventions", ...SECTIONS.map((s) => s.id)]) }); } for (const w of written) write(w.rel, w.html); // --------------------------------------------------------------------------- +// An nested in an is closed early by the parser, so the outer link +// becomes an empty unclickable stub and the inner one wins. The link check +// cannot see it: both hrefs resolve. It asks a different question. +const nested = []; +for (const w of written) { + for (const m of w.html.matchAll(/]*>((?:(?!<\/a>)[\s\S])*?) inside near ${JSON.stringify(m[1].slice(0, 60))}`); +} + // Link check over the generated output. Every internal href must resolve to a // generated file, and every fragment to an id that page really carries. const anchorsByPage = new Map(written.map((w) => [w.rel, w.anchors])); @@ -574,7 +641,16 @@ for (const w of written) { continue; } const resolved = relative(".", join(base === "." ? "" : base, pathPart)); - if (resolved.startsWith("..")) continue; // leaves the docs tree: checked below + // A link out of the docs tree (../index.html, ../app/) names a file this + // script does not generate, so it is checked on disk instead — which is why + // assemble.sh builds the docs LAST, once its siblings exist. + if (resolved.startsWith("..")) { + const onDisk = join(out, base === "." ? "" : base, pathPart); + const withIndex = raw.endsWith("/") ? join(onDisk, "index.html") : onDisk; + if (!existsSync(withIndex) && !existsSync(onDisk)) + broken.push(`${w.rel} -> ${raw} (no such file outside the docs tree: ${withIndex})`); + continue; + } const target = resolved.endsWith("/") ? `${resolved}index.html` : resolved; if (!files.has(target)) broken.push(`${w.rel} -> ${raw} (no such page: ${target})`); else if (frag && !anchorsByPage.get(target)?.has(frag)) @@ -585,7 +661,9 @@ for (const w of written) { console.log(`doc-site: ${written.length} pages -> ${out}`); console.log(`doc-site: ${cited.size} ticket ids cited, ${cited.size - unresolved.size} resolve, ${unresolved.size} not in the export`); console.log(`doc-site: ${checked} internal links checked, ${broken.length} broken`); -if (broken.length) { +console.log(`doc-site: ${nested.length} nested anchors`); +for (const n of nested) console.error(`doc-site: nested anchor: ${n}`); +if (broken.length || nested.length) { for (const b of broken) console.error(`doc-site: broken link: ${b}`); process.exit(1); } diff --git a/web/assemble.sh b/web/assemble.sh index 73d74c6b..1fe1cd20 100755 --- a/web/assemble.sh +++ b/web/assemble.sh @@ -42,11 +42,6 @@ mkdir -p "$out" cp -f "$root/docs/index.html" "$out/index.html" cp -rf "$root/docs/casts" "$out/casts" -# The prose site under `docs/`, generated from context/ by the same script the -# harness runs. It link-checks its own output and exits non-zero on a break, so -# a dead cross-reference fails the assembly rather than shipping. -node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token" - app="$out/app" mkdir -p "$app" # shell.js and serial.js are imported by app.js and worker.js: leaving either @@ -103,3 +98,10 @@ for entry in app.js worker.js; do } done done + +# The prose site at `docs/`, generated from context/ by the same script the +# harness serves. It runs LAST because its link check verifies the links that +# leave its own tree (`../index.html`, `../app/`) against the filesystem, which +# means their targets must already be here. It exits non-zero on a break, so a +# dead cross-reference fails the assembly rather than shipping. +node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token" From 41f694f53590001d10b118a9f9c9e8401925567c Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sun, 20 Sep 2026 14:04:19 -0700 Subject: [PATCH 4/7] fix(docs): collapse the phone nav, and stop the header comment lying Round 2 of review. Three notes and the mobile-nav call. THE NAV, and the correction that prompted it. I had called the 25-link sidebar wall pre-existing and declined to fix it. That was wrong: the entire CSS is new in this PR, so the wall ships for the first time here and it is mine. Every phone page opened on ~1500px of nav before any content. The mechanism is a checkbox and not
    , and the reason is measured rather than assumed.
    was the obvious control, and it collapses correctly on a phone -- but the sidebar layout has to force it OPEN, and a closed
    hides its content through ::details-content, which `display` cannot override. Chromium showed the desktop sidebar collapsed to one link with its toggle hidden: unusable. So the hiding is ours, in a media query, where overriding it is just cascade. Measured at both widths: 390px shows 1 link and the toggle, with the h1 at 189px instead of ~1500; 1280px shows all 26 links with no toggle and the h1 at 40px, unchanged; and the phone toggle really expands 1 -> 26 on click. Note the first probe was wrong -- getBoundingClientRect reports a height for content-visibility-hidden children, so it read 26 visible in both states and would have passed the broken version. checkVisibility() is what separates them. The same argument applies to the table of contents, which is equally new here: repo-map opened on 1397px of TOC on a phone. Capped at 45vh so it scrolls; 1397 -> 924, desktop identical at 652. THE HEADER COMMENT. The generator stopped running standalone when the link check began resolving ../index.html and ../app/ on disk, and its header still advertised a default OUT_DIR that now exits 1. That is gotcha 48's third paragraph exactly -- a fact stated in two places and updated in one, with the copy that was updated being doc-site.md and the copy someone about to run the script actually reads being the header. It now says so, keeps the strictness (the hole it replaced was counting those links as checked while skipping them), and records the other thing the reorder changed: the DIRECTION of a failure, from an obviously-broken tree with no app/ to a complete-looking one with a partial docs/. And one documented subset: a \| inside a code span in a table cell keeps its backslash where GFM would unescape it. Splitting is suppressed inside code spans, so the escape is never needed there and never consumed. No corpus instance; recorded as a decision rather than left to be rediscovered. Gates: 1378 rust, 26 node --test, 16 playwright (3 skipped, webkit OPFS), all three check-comment-runs modes, 3460 site links / 0 broken, 0 nested anchors, 0 of 23 pages overflow at 390 and 1280. improvise-djoo Co-Authored-By: Claude Opus 5 (1M context) --- context/code/doc-site.md | 6 ++++++ scripts/build-doc-site.mjs | 23 +++++++++++++++++++++-- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/context/code/doc-site.md b/context/code/doc-site.md index 7e61a559..370e6942 100644 --- a/context/code/doc-site.md +++ b/context/code/doc-site.md @@ -188,6 +188,12 @@ been shredded into extra columns — a row that still parses, so no column count could detect it. It was latent when fixed, which is the point: the trigger is one ordinary sentence away in a document whose subject is pipe quoting. +One deliberate divergence from GFM inside that rule: a `\|` **inside a code +span** in a table cell keeps its backslash, where GFM would unescape it to a +bare pipe. Splitting is suppressed inside code spans, so the escape is never +needed there and never consumed. The corpus contains no instance; it is +recorded as a decision rather than left to be rediscovered as a bug. + ## 6. Navigation `repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs index e384c15d..f35128b8 100644 --- a/scripts/build-doc-site.mjs +++ b/scripts/build-doc-site.mjs @@ -1,6 +1,16 @@ #!/usr/bin/env node // Generate the documentation site from context/ into OUT_DIR (default site/docs). // The argument and the rationale live in context/code/doc-site.md. +// +// NOT standalone: the link check resolves ../index.html and ../app/ on disk, so +// OUT_DIR's parent must already be an assembled site. Run web/assemble.sh, which +// invokes this last for that reason. Bare `node scripts/build-doc-site.mjs out` +// exits 1 on 48 outward links, and that strictness is the point — the hole it +// replaced was counting those links as checked while skipping them. +// +// The reorder also moved WHERE a failure shows: a dead link now leaves a site/ +// that looks complete with a partial docs/, where before it left one with no +// app/. Contained only because assemble.sh is `set -eu` and CI gates on it. import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync, statSync } from "node:fs"; import { dirname, join, relative } from "node:path"; import { fileURLToPath } from "node:url"; @@ -417,6 +427,9 @@ a{color:var(--accent)} .shell{display:flex;align-items:flex-start;max-width:1400px;margin:0 auto} nav.side{position:sticky;top:0;flex:0 0 17rem;max-height:100vh;overflow-y:auto;padding:1.5rem 1rem 3rem;border-right:1px solid var(--rule);font-size:.9rem} nav.side .brand{font-family:var(--mono);font-size:1.15rem;color:var(--accent);text-decoration:none;font-weight:600} +nav.side .navsummary{display:none;cursor:pointer;margin:.9rem 0 .2rem;padding:.4rem .55rem;border:1px solid var(--rule);border-radius:4px;font-family:var(--mono);font-size:.85rem;color:var(--dim)} +nav.side .navsummary::before{content:"\\2630 "} +nav.side .navtoggle:checked~.navsummary{border-color:var(--accent);color:var(--fg)} nav.side h3{font-size:.72rem;letter-spacing:.09em;text-transform:uppercase;color:var(--dim);margin:1.6rem 0 .4rem} nav.side ul{list-style:none;margin:0;padding:0} nav.side li{margin:.15rem 0} @@ -465,14 +478,20 @@ hr{border:0;border-top:1px solid var(--rule);margin:2rem 0} .tickets td:first-child{white-space:nowrap;font-family:var(--mono);font-size:.86rem} .tickets td:last-child{overflow-wrap:anywhere} .st{font-size:.78rem;color:var(--dim);text-transform:uppercase;letter-spacing:.05em} -@media(max-width:900px){.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}}`; +@media(max-width:900px){.toc ul{max-height:45vh;overflow-y:auto}nav.side .navsummary{display:block}nav.side .navlinks{display:none}nav.side .navtoggle:checked~.navlinks{display:block}.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}}`; function nav(currentSlug, depth) { const up = "../".repeat(depth); const link = (href, label, here) => `
  • ${label}
    • `; + // A checkbox and not
    : the disclosure must be forced OPEN in the + // sidebar layout, and a closed
    hides its content through + // ::details-content, which `display` cannot override (measured in Chromium). const parts = [ `"); + parts.push("
    "); return parts.join(""); } From c0b786cf0d65b4342b41cd29553719be30c2f3cc Mon Sep 17 00:00:00 2001 From: Edward Langley Date: Sun, 20 Sep 2026 14:09:37 -0700 Subject: [PATCH 5/7] fix(docs): the phone nav toggle needs a keyboard, not a mouse A regression the previous commit introduced, found in review. The checkbox driving the collapsed nav carried `hidden`, which is `display:none`, which removes it from the tab order AND from the accessibility tree. A