fiddlerwoaroof/improvise
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'gh/main' into fix/web-panel-clicks

Browse Source
This commit is contained in:
Edward Langley
2026-09-20 14:24:18 -07:00
parent ac6f17288c 8e6738af72
commit ad160b8b54
5 changed files with 1007 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/ci.yml
+6
View File
@@ -68,6 +68,12 @@ jobs:
"wasm-pack-${WASM_PACK_VERSION}-x86_64-unknown-linux-musl/wasm-pack"
wasm-pack --version
# Both assemblies below generate the prose site with node. The runner
# image ships one, but pinning it is what makes a local run evidence.
- uses: actions/setup-node@v4
with:
node-version: "24"
# The same script a developer runs locally, so there is one answer to
# "how is the bundle built" rather than two that can drift. It builds all
# three wasm bundles (client, worker, opfs-contract) and assembles
context/code/doc-site.md
+303
View File
@@ -0,0 +1,303 @@
# The documentation site — the arguments
`scripts/build-doc-site.mjs` turns the prose in `context/` into the static site
served at `/docs/` on GitHub Pages. This is the reasoning behind what it
publishes, what it deliberately does not, and why it is a hundred-odd lines of
node rather than a static site generator (improvise-djoo).
---
## 1. What is published, and what is not
The manifest is the `SECTIONS` array at the top of the script, and **a document
is published only by being named there**. That is the point: a glob over
`context/**` would publish the next working file somebody drops in, and the
decision about what a stranger should read is not one to make by accident.
Published, in four sections:
| Section | Contents | Why |
|---|---|---|
| Start here | `docs/design-notes.md`, `context/design-principles.md`, `context/repo-map.md` | What the project is for, the doctrine, and the single best explanation of how it is built. |
| Decision records | `context/adr/*.md` | An ADR is written for an outside reader by construction — it states the options that were rejected, which is the part that is worth reading and the part that is nowhere else. |
| The arguments, by area | `context/code/*.md` | This is the reasoning that used to live in code comments and was moved out under the two-line cap (design-principles §6). It is the most valuable prose in the repository and it is addressed to exactly the reader a documentation site has. |
| Test evidence | `context/test-evidence/*.md` | 501 entries, one per test whose narrative is longer than its code. |
Not published:
- **`context/plan.md`** is a *prompt*: it opens "You are working in the
`improvise` repository… Your job is to prepare this repository for a public
Show HN launch", and it instructs a reader to audit a `context/SPEC.md` that
no longer exists. It is an instruction to an agent about a launch, not a
description of the program, and publishing a launch plan on the site the
launch points at inverts the two.
- **`context/cloud-agents.md`** is operational notes for agents running in a
cloud session — tracker hydration, `bd` pitfalls, what the sandbox blocks.
It is true and it is useful and it is addressed to nobody who visits a
website.
Neither is referenced by any published document except as a row in
`repo-map.md`'s file inventory, which is inside a code fence, so leaving them
out creates no dangling link. That was checked rather than assumed.
**Test evidence is the one judgement call worth stating.** The argument for
leaving it out is that 501 entries about individual tests are noise to a reader
who wanted to know what a pivot table is. The argument for keeping it is
stronger: every one of the seven `context/code/*.md` documents points at its
evidence file by name, so omitting it would break the published prose in
exactly the way this site is otherwise careful not to; and the evidence file is
the most distinctive artefact the repository has — a written record of the
injected fault behind each load-bearing test is not a thing most projects can
show. It is therefore published as its **own section**, last, described as a
contributor reference rather than a tour, so nobody wanders into it expecting
a narrative.
## 2. Ticket ids resolve to a page, never to a URL that does not exist
The prose cites bead ids roughly 190 times — this paragraph deliberately spells
none, because a document that gives one as an example makes itself a source of
audit noise, and a placeholder is worse than a real id: it is ID-shaped and
resolves to nothing. On a public site those citations are
meaningless strings, and the obvious repair — link them to an issue tracker —
is unavailable: beads keeps its issues in a Dolt database on a git ref, and
there is no public issue URL to point at. Inventing one would produce 190
links that 404, which is worse than plain text because a link is a promise.
So the ids resolve **inward**. `tickets.html` is generated from
`.beads/issues.jsonl` — already committed, already public — and carries id,
status and title, nothing else; descriptions are internal working notes and are
not the site's business. Each citation in the prose becomes a link to that
page's anchor, with the title in a `title=` attribute so a hover answers the
question without a navigation.
**`issues.jsonl` is a passive export and lags the database**, which is not a
defect to work around but a fact to render honestly: an id the export does not
carry is emitted as plain text with a dotted underline and no link, and the
ticket page lists it as "not exported". The site therefore never links to an
anchor it did not generate, whatever state the export is in. A generator that
instead linked every id it saw would produce a page that is right today and
broken the first time somebody cites a fresh ticket.
**The page publishes every cited id, open bugs and their titles included, and
that is a decision rather than a default** (user ruling). The data is already
public in `.beads/issues.jsonl` in a repository anyone can clone; the page only
makes it legible. An honest open-bug list is a normal thing for an open-source
project to have, and filtering one while the page's own lede links the raw
JSONL would be theatre. Note the second-order reason to keep it unfiltered:
a citation whose row was suppressed would be a link into a page that does not
answer it, which is the failure this whole section exists to avoid.
**Titles are tracker data and are escaped as attribute values.** Five shipped
bead titles contain a double quote; interpolated into `title="…"` by an `esc`
that handled only `&`, `<` and `>`, they broke out of the attribute and the
browser parsed the remains as three junk attributes. `esc` now escapes `"` and
`'` as well. Today's corpus being friendly is not a reason to interpolate
untrusted-shaped text unescaped — the tracker is written by whoever files a
bug.
## 3. Source paths link only where the file exists
A code span whose entire content is a repo path becomes a link — to the
generated page when the path is one of the published documents, to GitHub
otherwise. Two constraints make that safe. The whole span must be the path, so
a sentence that merely mentions a directory is untouched; and **the path must
exist on disk at generation time**, so a citation that has gone stale degrades
to plain text rather than to a 404. The GitHub ref is the cache-bust token
`web/assemble.sh` already receives, used only when it is a 40-hex sha and
falling back to `main` otherwise — a local `dev` build must not emit
`blob/dev/…`.
## 4. The link check is the test, and it is part of the build
The generator collects every anchor it emits and every internal href it writes,
resolves them against each other, and **exits non-zero** if any href names a
page it did not generate or a fragment that page does not carry.
`web/assemble.sh` runs under `set -eu`, so a broken cross-reference fails the
assembly rather than shipping.
This is not ceremony. It caught two real defects on its first two runs: a
fence-close regex that compared a three-character marker against a
one-character string, which swallowed every document from its first code block
onward; and a list parser that ended the gotcha list at the first blank line,
so 47 of the 48 gotcha anchors were never emitted while the "jump to gotcha N"
strip linked to all of them. Both were invisible in a rendered page that
otherwise looked fine.
It is proved by injection rather than asserted: renaming the emitted anchor ids
makes it report 48 missing anchors, and pointing the ticket links at a page
name that is never generated makes it report 763 missing pages — two different
diagnoses, because "the page is absent" and "the page is there but the anchor
is not" call for different repairs.
Links that **leave** the `docs/` subtree — `../index.html`, `../app/` — name
files this script does not generate, so they are resolved against the
filesystem instead. That is the whole reason `assemble.sh` runs the generator
**last**: its siblings have to exist before they can be checked. An earlier
version skipped those 48 links while still counting them as checked, behind a
comment promising they were "checked below" when there was no below.
**And the link check is not sufficient, which is the more important lesson.**
It answers one question — does every href resolve — and a defect outside that
question is invisible to it however carefully it is run. The worked example is
a nested `<a>`: a TOC entry wrapping a heading whose rendered text already
contained a link produced `<a href="#anchor"><a href="github…">…</a></a>`, and
the HTML parser closes the outer anchor early, so the entry became an empty
unclickable stub and clicking it navigated to GitHub. Both hrefs resolved
perfectly. It shipped past the link check, past the whole-site link check, and
past a screenshot pass, and was caught by a reviewer *clicking a link*. The
repair is therefore two things, not one: the TOC keeps a tag-free variant of
each heading, **and** a separate check asserts no `<a>` appears inside another
`<a>` — proved by reverting the first half and watching the second report 66.
The general form: when a check misses a defect, add the check that could have
seen it rather than only fixing the defect, and ask what *question* the
existing check was answering.
## 5. Why a node script and not a generator
The landing page is hand-written HTML with no build step and one pinned CDN
script; the same posture applies here. A static site generator would add a
dependency tree, a lockfile and a toolchain to a repository whose site is
currently four files, in exchange for features this corpus does not use — the
docs contain no front matter, no shortcodes and, as it happens, **not a single
markdown link**. Every cross-reference in them is prose naming a path or an
id, which is precisely why §2 and §3 are the interesting parts and Markdown
rendering is the boring part.
So the renderer is a deliberately small subset covering exactly what the corpus
contains: ATX headings, fenced code, pipe tables, blockquotes, ordered and
unordered lists with indented continuations, thematic breaks, and inline code,
strong, emphasis and links. Anything it does not recognise becomes paragraph
text, so **no input line is ever dropped** — the failure mode is an ugly
paragraph, never a missing one.
Two details in it are load-bearing and easy to undo. **Code spans are replaced
by placeholders before emphasis is matched**, because this corpus routinely
opens `**` on one side of a code span and closes it on the other; matching
emphasis per segment silently drops that bold, which is a fidelity loss with
no visible symptom. And **strong is matched before emphasis, with a body that
may contain a single `*` pair**, because `[^*]+` cannot match
`**a *b* c**` and the leftover `*` then pairs across the bold — which is how a
single sentence in gotcha 33 came out as three italic paragraphs.
A third, added later and for the same reason: **a table cell boundary is an
unescaped pipe outside a code span**, so the row is walked rather than
`split("|")`. This corpus documents a file format whose own quoting syntax is
`|Time Period|`, and a sentence containing one inside a table cell would have
been shredded into extra columns — a row that still parses, so no column count
could detect it. It was latent when fixed, which is the point: the trigger is
one ordinary sentence away in a document whose subject is pipe quoting.
One deliberate divergence from GFM inside that rule: a `\|` **inside a code
span** in a table cell keeps its backslash, where GFM would unescape it to a
bare pipe. Splitting is suppressed inside code spans, so the escape is never
needed there and never consumed. The corpus contains no instance; it is
recorded as a decision rather than left to be rediscovered as a bug.
## 6. Navigation
`repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and
splitting it per gotcha was rejected: the gotchas cross-reference each other
constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader
following those wants a scroll, not 48 page loads. It stays one page, with an
id on every list item and a compact strip of numbered links under the heading.
The ids come from the list item's **own number**, not from a counter, so an
inserted gotcha renumbers the anchors exactly as it renumbers the prose that
cites them.
The strip's contents are **derived from the anchors actually emitted**, never
from a count. A hardcoded 48 failed in only one direction: deleting a gotcha
left a link to a missing anchor, which the link check catches, while *adding*
one produced an unlinked gotcha and a clean build. The list only ever grows, so
the hardcoded form was silent in precisely the direction it would be used —
the same shape as an exclusion list that over-matches, failing in the direction
that looks like success.
**Long unbreakable headings need `overflow-wrap`.** The test-evidence pages are
hundreds of `snake_case` test names as headings; at phone width seven of them
overflowed the viewport by up to 92%. The diagnosis is worth keeping because
the first guess was wrong: `getBoundingClientRect` reported every element
inside the viewport, because the text overflows its box without widening it.
Only `scrollWidth` per element names the culprit. Measure the property that
actually overflowed rather than the one that usually does.
## 7. The phone nav is a checkbox, and that is not arbitrary
At phone width the sidebar's 25 links stacked above the content, so every page
opened on roughly 1500px of nav. The wall was not inherited: **all of this CSS
is new**, so it shipped for the first time with the site, which is the test for
whether a defect is pre-existing and the one most easily got wrong in the
direction of declining work you introduced.
`<details>` is the obvious control and it is **wrong here**, for a reason no
spec-level reading predicts. The sidebar layout has to force the disclosure
*open* at wide widths, and a closed `<details>` hides its content through
`::details-content` — which `display` cannot override. Measured in Chromium,
the desktop sidebar came out collapsed to a single link with its toggle also
hidden: unusable, and in the half nobody would think to re-check after a
phone-only change. A hidden checkbox plus a `<label>` puts the hiding in *our*
media query, so forcing it open is ordinary cascade. Anyone reading the markup
will see a checkbox where `<details>` obviously belongs; this paragraph is why
it must not be tidied into one.
**And the first probe of the fix could not see the difference.** Counting nav
links whose `getBoundingClientRect().height > 0` reported **26 visible in both
the collapsed and the expanded state**, because Chrome reports a height for
`content-visibility`-hidden children — so it would have passed the broken
`<details>` build. `checkVisibility({contentVisibilityAuto: true,
visibilityProperty: true})` discriminates.
That is the same instrument failing twice: §6's overflow fix also measured
clean by `getBoundingClientRect` while the page still scrolled. **A measurement
that cannot distinguish the two states is not a measurement** — the repo's
signature defect, a test that passes while the thing it names is broken,
arriving through the instrument rather than through the assertion. Before
trusting a browser measurement, injure the thing being measured and check the
number moves.
**And the fix's own probe had the same shape one level up.** Counting *visible*
nav links answers "does operating the toggle reveal the links" — correct and
complete for a mouse — and is silent on whether any non-mouse route exists. The
first version of the checkbox carried `hidden`, which is `display:none`, which
removes it from the tab order *and* from the accessibility tree: the label is
not focusable on its own, so at ≤900px the navigation could be clicked and
reached no other way, and the accessibility snapshot contained neither the
checkbox nor any of the 25 links. A visibility count cannot see that, however
sharp the visibility instrument. The toggle is therefore visually hidden but
focusable at narrow widths and `display:none` at wide ones, and the check is
tab order, `.focus()` and the accessibility snapshot — measured at 390, 900,
901 and 1280, because the regression lives on one side of a media query.
So the rule from §6 has a second half: **a measurement of the wrong property is
not a measurement either, however precise it is.** `checkVisibility` fixed the
visibility instrument; it does not turn a visibility count into an operability
check. Ask what the defect would *do* to a user, then pick the instrument that
observes that.
**A correction, and the risk the original argument missed.** The commit that
added `content: "\2630 " / ""` said Firefox ignores the alt-text form and falls
back to reading the content, so the change was "an improvement in two engines
and a no-op in the third". **That mechanism is wrong** — Firefox shipped the
syntax in 133, and all three engines parse it (measured: Chrome 141, Firefox
142, WebKit 26 each compute the declaration intact). The conclusion held; the
reason given for it did not, and the reason was the load-bearing part, because
it was what made the change look like a one-way door. It is one because every
engine shipped to supports the syntax — not because the unsupported path is
harmless. That commit message is superseded by this paragraph (design-principles
§6: an argument must not live only in a commit message).
**The unsupported path is not harmless, and that is the risk neither the change
nor its first review named.** `/ alt` in an engine that does not know it is a
*parse error for the whole `content` declaration*, so the glyph would not lose
its alt text — it would **vanish**. A silent visual regression, the opposite of
the assumed failure mode. The glyph is therefore verified painted in all three
engines by pixel diff against `::before { content: none }`, with the text
starting 20px further right in each.
That check also produced this file's fourth instance of the wrong-property
trap, and the sharpest, because it produced a confident **false positive**: the
first probe measured the label's *width* with and without `::before` and
reported the glyph missing in all three engines. The label is `display: block`,
so its width comes from the container and cannot see inline content at all.
Right question, wrong property, and this time the wrong instrument accused
working code rather than clearing broken code.
docs/index.html
+3 -1
View File
@@ -134,6 +134,7 @@
<a class="cta primary" href="./app/"
>▶ Run improvise in your browser</a
>
<a class="cta" href="./docs/">Documentation</a>
<a class="cta" href="https://github.com/fiddlerwoaroof/improvise"
>Source on GitHub</a
>
@@ -191,7 +192,8 @@ nix profile install github:fiddlerwoaroof/improvise
<footer>
Apache-2.0 ·
<a href="https://github.com/fiddlerwoaroof/improvise">GitHub</a> ·
<a href="./app/">Live browser demo</a>
<a href="./app/">Live browser demo</a> ·
<a href="./docs/">Documentation</a>
</footer>
</main>
scripts/build-doc-site.mjs
+688
View File
@@ -0,0 +1,688 @@
#!/usr/bin/env node
// Generate the documentation site from context/ into OUT_DIR (default site/docs).
// The argument and the rationale live in context/code/doc-site.md.
//
// NOT standalone: the link check resolves ../index.html and ../app/ on disk, so
// OUT_DIR's parent must already be an assembled site. Run web/assemble.sh, which
// invokes this last for that reason. Bare `node scripts/build-doc-site.mjs out`
// exits 1 on 48 outward links, and that strictness is the point — the hole it
// replaced was counting those links as checked while skipping them.
//
// The reorder also moved WHERE a failure shows: a dead link now leaves a site/
// that looks complete with a partial docs/, where before it left one with no
// app/. Contained only because assemble.sh is `set -eu` and CI gates on it.
import { readFileSync, writeFileSync, mkdirSync, rmSync, existsSync, statSync } from "node:fs";
import { dirname, join, relative } from "node:path";
import { fileURLToPath } from "node:url";
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
const out = process.argv[2] || join(root, "site", "docs");
const token = process.argv[3] || "";
const ref = /^[0-9a-f]{40}$/.test(token) ? token : "main";
const REPO = "https://github.com/fiddlerwoaroof/improvise";
// ---------------------------------------------------------------------------
// The manifest. A doc is published only by being named here — see
// context/code/doc-site.md for what was left out and why.
const SECTIONS = [
{
id: "start",
title: "Start here",
blurb:
"What the project is for, and the principles the code is held to.",
docs: [
["docs/design-notes.md", "design-notes", "Design notes"],
["context/design-principles.md", "design-principles", "Design principles"],
["context/repo-map.md", "repo-map", "Repository map"],
],
},
{
id: "adr",
title: "Decision records",
blurb: "Decisions with consequences, including the options that were rejected.",
docs: [
["context/adr/001-concurrent-edits.md", "adr/001-concurrent-edits", "ADR 001 — Concurrent edits"],
["context/adr/002-naming-a-cell.md", "adr/002-naming-a-cell", "ADR 002 — Naming a cell"],
["context/adr/003-live-backed-models.md", "adr/003-live-backed-models", "ADR 003 — Live-backed models"],
],
},
{
id: "code",
title: "The arguments, by area",
blurb:
"The reasoning that used to live in code comments. Written for a reader with the source open.",
docs: [
["context/code/core.md", "code/core", "improvise-core"],
["context/code/formula-and-io.md", "code/formula-and-io", "improvise-formula and improvise-io"],
["context/code/protocol.md", "code/protocol", "improvise-protocol"],
["context/code/command.md", "code/command", "improvise-command"],
["context/code/clients.md", "code/clients", "The clients"],
["context/code/tui.md", "code/tui", "The TUI"],
["context/code/gp.md", "code/gp", "gp-tree and improvise-gp"],
],
},
{
id: "evidence",
title: "Test evidence",
blurb:
"For each test whose narrative is longer than its code: what broke, what fault was injected, and why the test can fail. Contributor reference, not a tour.",
docs: [
["context/test-evidence/core.md", "test-evidence/core", "improvise-core"],
["context/test-evidence/formula-and-io.md", "test-evidence/formula-and-io", "improvise-formula and improvise-io"],
["context/test-evidence/protocol.md", "test-evidence/protocol", "improvise-protocol"],
["context/test-evidence/command.md", "test-evidence/command", "improvise-command"],
["context/test-evidence/clients.md", "test-evidence/clients", "The clients"],
["context/test-evidence/tui.md", "test-evidence/tui", "The TUI"],
["context/test-evidence/gp.md", "test-evidence/gp", "gp-tree and improvise-gp"],
],
},
{
id: "meta",
title: "About this site",
blurb: "What is published here, what is not, and why.",
docs: [["context/code/doc-site.md", "doc-site", "About this site"]],
},
];
const PAGE_BY_SOURCE = new Map();
for (const s of SECTIONS) for (const [src, slug] of s.docs) PAGE_BY_SOURCE.set(src, `${slug}.html`);
// ---------------------------------------------------------------------------
// The ticket index. `.beads/issues.jsonl` is a passive export and lags the Dolt
// database, so an id it does not carry is rendered as plain text rather than a
// link that 404s.
function readTickets() {
const path = join(root, ".beads", "issues.jsonl");
const map = new Map();
if (!existsSync(path)) return map;
for (const line of readFileSync(path, "utf8").split("\n")) {
if (!line.trim()) continue;
let o;
try {
o = JSON.parse(line);
} catch {
continue;
}
if (o && o._type === "issue" && typeof o.id === "string" && typeof o.title === "string") {
map.set(o.id, { title: o.title, status: o.status || "unknown", kind: o.issue_type || o.type || "" });
}
}
return map;
}
const TICKETS = readTickets();
// Tokens that look like an id but name a crate or a package. Anchored, because
// an unanchored alternative is a prefix match that silently excludes a real id.
const NON_TICKET = new Set(
"core formula io protocol command gp ws worker wasm harness registry side".split(" "),
);
// ---------------------------------------------------------------------------
// Markdown. A deliberately small subset, covering exactly what the corpus uses;
// anything unrecognised becomes paragraph text, so no input line is ever lost.
const esc = (s) =>
s
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
// A TOC entry must not nest an anchor inside the one wrapping it: a heading
// whose text is a path or a ticket id already carries an <a>, and the parser
// closes the outer one early, leaving an empty unclickable stub.
const stripTags = (html) => html.replace(/<[^>]+>/g, "");
function slugify(s) {
return (
s
.toLowerCase()
.replace(/<[^>]+>/g, "")
.replace(/&[a-z]+;/g, "")
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "") || "section"
);
}
// A code span whose whole content is a repo path becomes a link: to the
// generated page when the path is a published doc, to GitHub otherwise. The
// path must exist on disk, so a stale citation stays plain text.
function pathHref(p, depth) {
const clean = p.replace(/^\.\//, "");
if (!/^[A-Za-z0-9_.\/-]+$/.test(clean) || !/[\/.]/.test(clean)) return null;
// A span of punctuation alone (`.`, `-`, `/`) is prose, not a path.
if (!/[A-Za-z0-9]/.test(clean)) return null;
if (clean.includes("..") || clean.startsWith("/")) return null;
if (PAGE_BY_SOURCE.has(clean)) return "../".repeat(depth) + PAGE_BY_SOURCE.get(clean);
const abs = join(root, clean);
if (!existsSync(abs)) return null;
// GitHub serves a directory under /tree/, not /blob/; relying on its
// redirect is a 404 waiting on a change of behaviour we do not control.
const kind = statSync(abs).isDirectory() ? "tree" : "blob";
return `${REPO}/${kind}/${ref}/${clean.replace(/\/$/, "")}`;
}
function inline(text, ctx) {
// Code spans first: a backtick run opens a span only if a run of the same
// length closes it, so an unmatched ``` in prose stays literal.
const parts = [];
let i = 0;
while (i < text.length) {
if (text[i] === "`") {
let n = 0;
while (text[i + n] === "`") n++;
const fence = "`".repeat(n);
const end = text.indexOf(fence, i + n);
const after = end >= 0 ? end + n : -1;
if (end >= 0 && (after >= text.length || text[after] !== "`")) {
parts.push({ code: text.slice(i + n, end) });
i = after;
continue;
}
}
const next = text.indexOf("`", i + 1);
parts.push({ text: text.slice(i, next < 0 ? text.length : next) });
i = next < 0 ? text.length : next;
}
// Code spans stand aside as placeholders while emphasis is matched, because
// the corpus routinely opens `**` on one side of a code span and closes it on
// the other; matching per segment silently drops that bold.
const held = [];
let s = parts
.map((p) => {
if (p.code === undefined) return esc(p.text);
const body = `<code>${esc(p.code)}</code>`;
const href = pathHref(p.code.trim(), ctx.depth);
held.push(href ? `<a href="${href}">${body}</a>` : body);
return `\u0000${held.length - 1}\u0000`;
})
.join("");
s = s.replace(/\[([^\]]+)\]\(([^)\s]+)\)/g, (_m, label, href) => `<a href="${href}">${label}</a>`);
// Strong first, and its body may contain a single `*` pair: `[^*]+` cannot
// match `**a *b* c**`, and the leftover `*` then pairs across the bold.
s = s.replace(/\*\*((?:[^*]|\*(?!\*))+)\*\*/g, "<strong>$1</strong>");
s = s.replace(/\*([^*]+)\*/g, "<em>$1</em>");
s = s.replace(/\bimprovise-([a-z0-9]{3,8})\b/g, (m, tail) => {
if (NON_TICKET.has(tail)) return m;
ctx.cited.add(m);
if (!TICKETS.has(m)) {
ctx.unresolved.add(m);
return `<span class="tid tid-unknown" title="not in the committed tracker export">${m}</span>`;
}
const t = TICKETS.get(m);
const href = `${"../".repeat(ctx.depth)}tickets.html#${m}`;
return `<a class="tid" href="${href}" title="${esc(t.title)}">${m}</a>`;
});
return s.replace(/\u0000(\d+)\u0000/g, (_m, n) => held[Number(n)]);
}
// Block parse. Returns { html, toc }.
function renderMarkdown(src, ctx) {
const lines = src.split("\n");
const toc = [];
const seen = new Map();
const html = blocks(lines);
return { html: html.join("\n"), toc };
function heading(level, text) {
const rendered = inline(text, ctx);
let id = slugify(text);
const n = (seen.get(id) || 0) + 1;
seen.set(id, n);
if (n > 1) id = `${id}-${n}`;
ctx.anchors.add(id);
if (level <= 3) toc.push({ level, id, text: stripTags(rendered) });
const mark = level > 1 ? `<a class="anchor" href="#${id}">#</a>` : "";
return `<h${level} id="${id}">${mark}${rendered}</h${level}>`;
}
function blocks(ls) {
const outHtml = [];
let i = 0;
let para = [];
const flush = () => {
if (para.length) outHtml.push(`<p>${inline(para.join(" "), ctx)}</p>`);
para = [];
};
while (i < ls.length) {
const line = ls[i];
const fence = line.match(/^(\s*)(`{3,}|~{3,})\s*([A-Za-z0-9_-]*)\s*$/);
if (fence) {
flush();
const close = new RegExp(`^\\s*${fence[2][0] === "~" ? "~" : "\`"}{3,}\\s*$`);
const body = [];
i++;
while (i < ls.length && !close.test(ls[i])) {
body.push(ls[i]);
i++;
}
i++;
outHtml.push(`<pre><code>${esc(body.join("\n"))}</code></pre>`);
continue;
}
const h = line.match(/^(#{1,6})\s+(.*)$/);
if (h) {
flush();
outHtml.push(heading(h[1].length, h[2].trim()));
i++;
continue;
}
if (/^\s*(-{3,}|\*{3,}|_{3,})\s*$/.test(line) && !para.length) {
flush();
outHtml.push("<hr />");
i++;
continue;
}
if (/^\s*\|/.test(line) && i + 1 < ls.length && /^\s*\|[\s:|-]+\|?\s*$/.test(ls[i + 1])) {
flush();
const rows = [];
while (i < ls.length && /^\s*\|/.test(ls[i])) {
rows.push(ls[i]);
i++;
}
outHtml.push(table(rows));
continue;
}
if (/^\s*>/.test(line)) {
flush();
const body = [];
while (i < ls.length && /^\s*>/.test(ls[i])) {
body.push(ls[i].replace(/^\s*>\s?/, ""));
i++;
}
outHtml.push(`<blockquote>${blocks(body).join("\n")}</blockquote>`);
continue;
}
const li = line.match(/^(\s*)([-*+]|\d{1,3}\.)\s+(.*)$/);
if (li) {
flush();
const [listHtml, next] = list(ls, i, li[1].length);
outHtml.push(listHtml);
i = next;
continue;
}
if (!line.trim()) {
flush();
i++;
continue;
}
para.push(line.trim());
i++;
}
flush();
return outHtml;
}
// An item's continuation lines are dedented by that item's own marker width,
// so a nested list keeps its relative indentation and reparses as a list.
function list(ls, start, indent) {
const ordered = /^\s*\d{1,3}\./.test(ls[start]);
const items = [];
let i = start;
let first = null;
while (i < ls.length) {
const m = ls[i].match(/^(\s*)([-*+]|(\d{1,3})\.)\s+(.*)$/);
if (m && m[1].length === indent) {
if (ordered && first === null) first = Number(m[3]);
items.push({ num: m[3] ? Number(m[3]) : null, cw: m[0].length - m[4].length, body: [m[4]] });
i++;
continue;
}
if (!items.length) break;
const cur = items[items.length - 1];
if (!ls[i].trim()) {
let j = i;
while (j < ls.length && !ls[j].trim()) j++;
if (j >= ls.length) break;
// A blank line ends the list only if what follows is neither the next
// item nor a continuation of the open one.
const isSibling = /^(\s*)([-*+]|\d{1,3}\.)\s+/.test(ls[j]) && ls[j].search(/\S/) === indent;
if (isSibling) {
i = j;
continue;
}
if (ls[j].search(/\S/) > indent) {
cur.body.push("");
i = j;
continue;
}
break;
}
const col = ls[i].search(/\S/);
if (col <= indent) break;
cur.body.push(ls[i].slice(Math.min(col, cur.cw)));
i++;
}
const tag = ordered ? "ol" : "ul";
const attr = ordered && first !== null && first !== 1 ? ` start="${first}"` : "";
const rendered = items
.map((it) => {
const inner = blocks(it.body);
const body = inner.length === 1 && inner[0].startsWith("<p>")
? inner[0].slice(3, -4)
: inner.join("\n");
const id = ctx.listItemId && it.num !== null ? ctx.listItemId(it.num) : null;
if (id) ctx.anchors.add(id);
return `<li${id ? ` id="${id}"` : ""}>${body}</li>`;
})
.join("\n");
return [`<${tag}${attr}>\n${rendered}\n</${tag}>`, i];
}
// A cell boundary is an UNESCAPED pipe outside a code span. Splitting the raw
// row on every `|` shreds `\|` and this repo's own `|Time Period|` syntax into
// extra columns, which no column count can detect because the row still parses.
function splitRow(r) {
const out = [];
let cur = "";
let tick = 0;
for (let i = 0; i < r.length; i++) {
const c = r[i];
if (c === "\\" && r[i + 1] === "|" && !tick) {
cur += "|";
i++;
continue;
}
if (c === "`") {
let n = 0;
while (r[i + n] === "`") n++;
if (!tick) tick = n;
else if (tick === n) tick = 0;
cur += "`".repeat(n);
i += n - 1;
continue;
}
if (c === "|" && !tick) {
out.push(cur);
cur = "";
continue;
}
cur += c;
}
out.push(cur);
if (out.length && !out[0].trim()) out.shift();
if (out.length && !out[out.length - 1].trim()) out.pop();
return out;
}
function table(rows) {
const cells = (r) => splitRow(r.trim()).map((c) => inline(c.trim(), ctx));
const head = cells(rows[0]);
const body = rows.slice(2).map(cells);
return (
`<table><thead><tr>${head.map((c) => `<th>${c}</th>`).join("")}</tr></thead><tbody>` +
body.map((r) => `<tr>${r.map((c) => `<td>${c}</td>`).join("")}</tr>`).join("") +
"</tbody></table>"
);
}
}
// ---------------------------------------------------------------------------
const CSS = `:root{--bg:#14161a;--panel:#1b1e24;--fg:#d7dae0;--dim:#8b93a1;--accent:#8ec07c;--rule:#2b2f37;--mono:ui-monospace,"SFMono-Regular","Menlo","Consolas",monospace}
*{box-sizing:border-box}
body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.65 system-ui,-apple-system,"Segoe UI",sans-serif}
a{color:var(--accent)}
.shell{display:flex;align-items:flex-start;max-width:1400px;margin:0 auto}
nav.side{position:sticky;top:0;flex:0 0 17rem;max-height:100vh;overflow-y:auto;padding:1.5rem 1rem 3rem;border-right:1px solid var(--rule);font-size:.9rem}
nav.side .brand{font-family:var(--mono);font-size:1.15rem;color:var(--accent);text-decoration:none;font-weight:600}
nav.side .navtoggle{display:none}
nav.side .navsummary{display:none;cursor:pointer;margin:.9rem 0 .2rem;padding:.4rem .55rem;border:1px solid var(--rule);border-radius:4px;font-family:var(--mono);font-size:.85rem;color:var(--dim)}
nav.side .navsummary::before{content:"\\2630 " / ""}
nav.side .navtoggle:checked~.navsummary{border-color:var(--accent);color:var(--fg)}
nav.side h3{font-size:.72rem;letter-spacing:.09em;text-transform:uppercase;color:var(--dim);margin:1.6rem 0 .4rem}
nav.side ul{list-style:none;margin:0;padding:0}
nav.side li{margin:.15rem 0}
nav.side a{display:block;padding:.2rem .45rem;border-radius:4px;color:var(--fg);text-decoration:none}
nav.side a:hover{background:var(--panel)}
nav.side a.here{background:var(--panel);color:var(--accent);font-weight:600}
main{flex:1 1 auto;min-width:0;max-width:60rem;padding:2.5rem 2rem 6rem}
h1,h2,h3,h4,h5{font-family:var(--mono);font-weight:600;letter-spacing:-.01em;line-height:1.3;overflow-wrap:anywhere}
h1{font-size:2rem;margin:0 0 1.2rem;color:var(--accent)}
h2{font-size:1.25rem;margin:2.6rem 0 .6rem;padding-top:1.2rem;border-top:1px solid var(--rule)}
h3{font-size:1.02rem;margin:1.8rem 0 .4rem;color:#c8cdd6}
h4,h5{font-size:.95rem;margin:1.3rem 0 .3rem;color:var(--dim)}
.anchor{float:left;margin-left:-1.1em;padding-right:.35em;color:var(--rule);text-decoration:none;opacity:0}
h2:hover .anchor,h3:hover .anchor,h4:hover .anchor,h5:hover .anchor{opacity:1}
p{max-width:74ch}
li{max-width:74ch}
code{font-family:var(--mono);font-size:.88em;background:var(--panel);padding:.1em .3em;border-radius:3px;overflow-wrap:anywhere}
a code{background:#22262e}
pre{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:1rem;overflow-x:auto}
pre code{background:none;padding:0;font-size:.84rem;line-height:1.5;overflow-wrap:normal}
blockquote{margin:1rem 0;padding:.5rem .9rem;border-left:3px solid var(--accent);background:var(--panel)}
blockquote p{margin:.4rem 0}
table{border-collapse:collapse;margin:1rem 0;font-size:.92rem;display:block;overflow-x:auto}
th,td{border:1px solid var(--rule);padding:.4rem .6rem;text-align:left;vertical-align:top}
th{background:var(--panel)}
hr{border:0;border-top:1px solid var(--rule);margin:2rem 0}
.tid{font-family:var(--mono);font-size:.86em;white-space:nowrap}
.tid-unknown{color:var(--dim);border-bottom:1px dotted var(--rule)}
.toc{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.8rem 1.1rem;margin:0 0 2rem;font-size:.92rem}
.toc summary{cursor:pointer;color:var(--dim);font-family:var(--mono)}
.toc ul{list-style:none;margin:.6rem 0 0;padding:0;columns:2;column-gap:2rem}
.toc li{break-inside:avoid;margin:.12rem 0;overflow-wrap:anywhere}
.toc li.l3{padding-left:1rem;font-size:.92em}
.toc a{text-decoration:none;overflow-wrap:anywhere}
.toc a:hover{text-decoration:underline}
.jump{margin:0 0 2rem;font-size:.85rem;font-family:var(--mono)}
.jump a{display:inline-block;min-width:2.1rem;text-align:center;padding:.12rem .3rem;margin:.1rem;border:1px solid var(--rule);border-radius:4px;text-decoration:none}
.jump a:hover{border-color:var(--accent)}
.lede{color:var(--dim);max-width:70ch}
.src{font-size:.85rem;color:var(--dim);font-family:var(--mono);margin:-.6rem 0 1.8rem}
.cards{display:grid;gap:1rem;grid-template-columns:repeat(auto-fill,minmax(19rem,1fr));margin:1rem 0 2rem}
.card{background:var(--panel);border:1px solid var(--rule);border-radius:6px;padding:.9rem 1.1rem}
.card a{font-family:var(--mono);text-decoration:none}
.card p{color:var(--dim);font-size:.88rem;margin:.3rem 0 0}
.tickets{width:100%}
.tickets td:first-child{white-space:nowrap;font-family:var(--mono);font-size:.86rem}
.tickets td:last-child{overflow-wrap:anywhere}
.st{font-size:.78rem;color:var(--dim);text-transform:uppercase;letter-spacing:.05em}
@media(max-width:900px){.toc ul{max-height:45vh;overflow-y:auto}nav.side .navtoggle{display:block;position:absolute;width:1px;height:1px;margin:-1px;clip-path:inset(50%);overflow:hidden}nav.side .navtoggle:focus-visible~.navsummary{outline:2px solid var(--accent);outline-offset:2px}nav.side .navsummary{display:block}nav.side .navlinks{display:none}nav.side .navtoggle:checked~.navlinks{display:block}.shell{display:block}nav.side{position:static;flex:none;max-height:none;border-right:0;border-bottom:1px solid var(--rule)}main{padding:1.5rem 1rem 4rem}.toc ul{columns:1}}`;
function nav(currentSlug, depth) {
const up = "../".repeat(depth);
const link = (href, label, here) =>
`<li><a href="${href}"${here ? ' class="here"' : ""}>${label}</a></li>`;
// Do NOT "simplify" this to <details>: the sidebar must force it OPEN, which
// `display` cannot do, and the desktop nav collapses. doc-site.md §7.
const parts = [
`<nav class="side"><a class="brand" href="${up}index.html">improvise docs</a>`,
`<input type="checkbox" id="navtoggle" class="navtoggle" />`,
`<label class="navsummary" for="navtoggle">Pages</label>`,
`<div class="navlinks">`,
`<ul>${link(up + "../index.html", "← Project home")}${link(up + "../app/", "Browser demo")}${link(REPO, "Source on GitHub")}</ul>`,
];
for (const s of SECTIONS) {
parts.push(`<h3>${s.title}</h3><ul>`);
for (const [, slug, title] of s.docs)
parts.push(link(`${up}${slug}.html`, title, slug === currentSlug));
parts.push("</ul>");
}
parts.push(`<h3>Reference</h3><ul>${link(up + "tickets.html", "Ticket index", currentSlug === "tickets")}</ul>`);
parts.push("</div></nav>");
return parts.join("");
}
function page({ slug, title, body, depth }) {
const up = "../".repeat(depth);
return `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>${esc(title)} — improvise docs</title>
<link rel="stylesheet" href="${up}style.css" />
</head>
<body>
<div class="shell">
${nav(slug, depth)}
<main>
${body}
</main>
</div>
</body>
</html>
`;
}
function write(rel, content) {
const p = join(out, rel);
mkdirSync(dirname(p), { recursive: true });
writeFileSync(p, content);
return p;
}
// ---------------------------------------------------------------------------
rmSync(out, { recursive: true, force: true });
mkdirSync(out, { recursive: true });
write("style.css", CSS);
const cited = new Set();
const unresolved = new Set();
const written = [];
for (const section of SECTIONS) {
for (const [src, slug, title] of section.docs) {
const depth = slug.split("/").length - 1;
const ctx = { depth, cited, unresolved, anchors: new Set() };
if (slug === "repo-map") ctx.listItemId = (n) => (n >= 1 && n <= 200 ? `gotcha-${n}` : null);
const md = readFileSync(join(root, src), "utf8");
const { html, toc } = renderMarkdown(md, ctx);
const tocHtml = toc.length
? `<details class="toc" open><summary>On this page</summary><ul>${toc
.filter((t) => t.level <= 3)
.map((t) => `<li class="l${t.level}"><a href="#${t.id}">${t.text}</a></li>`)
.join("")}</ul></details>`
: "";
// Derived from the anchors actually emitted. A hardcoded count fails closed
// on a deleted gotcha (the link check catches it) and SILENTLY on an added
// one, which is the direction the list moves in.
const gotchas = [...ctx.anchors]
.map((a) => /^gotcha-(\d+)$/.exec(a))
.filter(Boolean)
.map((m) => Number(m[1]))
.sort((a, b) => a - b);
const jump = gotchas.length
? `<p class="jump">Gotchas: ${gotchas.map((n) => `<a href="#gotcha-${n}">${n}</a>`).join("")}</p>`
: "";
const srcLine = `<p class="src">Source: <a href="${REPO}/blob/${ref}/${src}">${src}</a></p>`;
written.push({
rel: `${slug}.html`,
html: page({
slug,
title,
depth,
body: `${srcLine}${tocHtml}${jump}${html}`,
}),
anchors: ctx.anchors,
});
}
}
// Ticket index.
{
const ctx = { depth: 0, cited, unresolved, anchors: new Set() };
const ids = [...cited].sort();
const rows = ids
.map((id) => {
const t = TICKETS.get(id);
ctx.anchors.add(id);
if (!t)
return `<tr id="${id}"><td>${id}</td><td class="st">not exported</td><td>Cited in the prose; the committed tracker export does not carry it.</td></tr>`;
return `<tr id="${id}"><td>${id}</td><td class="st">${esc(t.status)}</td><td>${esc(t.title)}</td></tr>`;
})
.join("");
const body = `<h1>Ticket index</h1>
<p class="lede">The prose on this site cites issue ids like <code>improvise-9c8c</code>. They are this project's own tracker ids, kept in-repo with <a href="https://github.com/gastownhall/beads">beads</a>; there is no public issue URL to link to, so this page is what they resolve to. Titles come from <a href="${REPO}/blob/${ref}/.beads/issues.jsonl">.beads/issues.jsonl</a>, a passive export that lags the database — an id it does not carry is listed here without a title and is not linked from the prose.</p>
<p class="lede">${ids.length} ids are cited, of which ${ids.length - unresolved.size} resolve.</p>
<table class="tickets"><thead><tr><th>Id</th><th>Status</th><th>Title</th></tr></thead><tbody>${rows}</tbody></table>`;
written.push({ rel: "tickets.html", html: page({ slug: "tickets", title: "Ticket index", depth: 0, body }), anchors: ctx.anchors });
}
// Landing page for the docs section.
{
// The two conventions go through `inline` rather than being hand-written, so
// the sentence describing the linking demonstrates it instead of asserting it.
const conventions = inline(
"Ids like `improvise-9c8c` are tracker issues and link to the ticket index below. " +
"Paths like `crates/improvise-core/src/model/types.rs` link to the source on GitHub at `" +
ref +
"`.",
{ depth: 0, cited, unresolved, anchors: new Set() },
);
const cards = SECTIONS.map(
(s) =>
`<h2 id="${s.id}"><a class="anchor" href="#${s.id}">#</a>${s.title}</h2><p class="lede">${s.blurb}</p><div class="cards">${s.docs
.map(([, slug, title]) => `<div class="card"><a href="${slug}.html">${title}</a></div>`)
.join("")}</div>`,
).join("");
const body = `<h1>improvise — documentation</h1>
<p class="lede">These are the working documents of the improvise repository, published as written. They are addressed to someone reading or changing the code: the arguments behind the architecture, the decisions that were made and the ones that were rejected, and the record of what each load-bearing test is guarding against.</p>
<p class="lede">Nothing here is a tutorial. For what the program does, see the <a href="../index.html">project home</a> or run the <a href="../app/">browser demo</a>.</p>
${cards}
<h2 id="conventions"><a class="anchor" href="#conventions">#</a>Conventions</h2>
<p>${conventions}</p>`;
written.push({ rel: "index.html", html: page({ slug: "", title: "Documentation", depth: 0, body }), anchors: new Set(["conventions", ...SECTIONS.map((s) => s.id)]) });
}
for (const w of written) write(w.rel, w.html);
// ---------------------------------------------------------------------------
// An <a> nested in an <a> is closed early by the parser, so the outer link
// becomes an empty unclickable stub and the inner one wins. The link check
// cannot see it: both hrefs resolve. It asks a different question.
const nested = [];
for (const w of written) {
for (const m of w.html.matchAll(/<a\b[^>]*>((?:(?!<\/a>)[\s\S])*?)<a\b/g))
nested.push(`${w.rel}: <a> inside <a> near ${JSON.stringify(m[1].slice(0, 60))}`);
}
// Link check over the generated output. Every internal href must resolve to a
// generated file, and every fragment to an id that page really carries.
const anchorsByPage = new Map(written.map((w) => [w.rel, w.anchors]));
const files = new Set(written.map((w) => w.rel).concat(["style.css"]));
const broken = [];
let checked = 0;
for (const w of written) {
const base = dirname(w.rel);
for (const m of w.html.matchAll(/(?:href|src)="([^"]+)"/g)) {
const raw = m[1];
if (/^(https?:|mailto:|data:)/.test(raw)) continue;
checked++;
const [pathPart, frag] = raw.split("#");
if (!pathPart) {
if (frag && !w.anchors.has(frag)) broken.push(`${w.rel} -> ${raw} (no such anchor on this page)`);
continue;
}
const resolved = relative(".", join(base === "." ? "" : base, pathPart));
// A link out of the docs tree (../index.html, ../app/) names a file this
// script does not generate, so it is checked on disk instead — which is why
// assemble.sh builds the docs LAST, once its siblings exist.
if (resolved.startsWith("..")) {
const onDisk = join(out, base === "." ? "" : base, pathPart);
const withIndex = raw.endsWith("/") ? join(onDisk, "index.html") : onDisk;
if (!existsSync(withIndex) && !existsSync(onDisk))
broken.push(`${w.rel} -> ${raw} (no such file outside the docs tree: ${withIndex})`);
continue;
}
const target = resolved.endsWith("/") ? `${resolved}index.html` : resolved;
if (!files.has(target)) broken.push(`${w.rel} -> ${raw} (no such page: ${target})`);
else if (frag && !anchorsByPage.get(target)?.has(frag))
broken.push(`${w.rel} -> ${raw} (no such anchor: ${frag})`);
}
}
console.log(`doc-site: ${written.length} pages -> ${out}`);
console.log(`doc-site: ${cited.size} ticket ids cited, ${cited.size - unresolved.size} resolve, ${unresolved.size} not in the export`);
console.log(`doc-site: ${checked} internal links checked, ${broken.length} broken`);
console.log(`doc-site: ${nested.length} nested anchors`);
for (const n of nested) console.error(`doc-site: nested anchor: ${n}`);
if (broken.length || nested.length) {
for (const b of broken) console.error(`doc-site: broken link: ${b}`);
process.exit(1);
}
web/assemble.sh
+7
View File
@@ -98,3 +98,10 @@ for entry in app.js worker.js; do
}
done
done
# The prose site at `docs/`, generated from context/ by the same script the
# harness serves. It runs LAST because its link check verifies the links that
# leave its own tree (`../index.html`, `../app/`) against the filesystem, which
# means their targets must already be here. It exits non-zero on a break, so a
# dead cross-reference fails the assembly rather than shipping.
node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token"