diff --git a/context/code/doc-site.md b/context/code/doc-site.md
index 4e900781..7e61a559 100644
--- a/context/code/doc-site.md
+++ b/context/code/doc-site.md
@@ -54,7 +54,10 @@ a narrative.
## 2. Ticket ids resolve to a page, never to a URL that does not exist
-The prose cites `improvise-xxxx` roughly 190 times. On a public site those are
+The prose cites bead ids roughly 190 times — this paragraph deliberately spells
+none, because a document that gives one as an example makes itself a source of
+audit noise, and a placeholder is worse than a real id: it is ID-shaped and
+resolves to nothing. On a public site those citations are
meaningless strings, and the obvious repair — link them to an issue tracker —
is unavailable: beads keeps its issues in a Dolt database on a git ref, and
there is no public issue URL to point at. Inventing one would produce 190
@@ -71,10 +74,26 @@ question without a navigation.
defect to work around but a fact to render honestly: an id the export does not
carry is emitted as plain text with a dotted underline and no link, and the
ticket page lists it as "not exported". The site therefore never links to an
-anchor it did not generate, whatever state the export is in. At the time of
-writing 8 of 178 linkified ids are in that state. A generator that instead
-linked every id it saw would produce a page that is right today and broken the
-first time somebody cites a fresh ticket.
+anchor it did not generate, whatever state the export is in. A generator that
+instead linked every id it saw would produce a page that is right today and
+broken the first time somebody cites a fresh ticket.
+
+**The page publishes every cited id, open bugs and their titles included, and
+that is a decision rather than a default** (user ruling). The data is already
+public in `.beads/issues.jsonl` in a repository anyone can clone; the page only
+makes it legible. An honest open-bug list is a normal thing for an open-source
+project to have, and filtering one while the page's own lede links the raw
+JSONL would be theatre. Note the second-order reason to keep it unfiltered:
+a citation whose row was suppressed would be a link into a page that does not
+answer it, which is the failure this whole section exists to avoid.
+
+**Titles are tracker data and are escaped as attribute values.** Five shipped
+bead titles contain a double quote; interpolated into `title="…"` by an `esc`
+that handled only `&`, `<` and `>`, they broke out of the attribute and the
+browser parsed the remains as three junk attributes. `esc` now escapes `"` and
+`'` as well. Today's corpus being friendly is not a reason to interpolate
+untrusted-shaped text unescaped — the tracker is written by whoever files a
+bug.
## 3. Source paths link only where the file exists
@@ -110,10 +129,29 @@ name that is never generated makes it report 763 missing pages — two different
diagnoses, because "the page is absent" and "the page is there but the anchor
is not" call for different repairs.
-What it does **not** cover is links that leave the `docs/` subtree — the ones
-to `../index.html` and `../app/`. Those are checked against the assembled
-`site/` tree by hand at the moment, and the honest statement is that a rename
-of the app directory would not be caught here.
+Links that **leave** the `docs/` subtree — `../index.html`, `../app/` — name
+files this script does not generate, so they are resolved against the
+filesystem instead. That is the whole reason `assemble.sh` runs the generator
+**last**: its siblings have to exist before they can be checked. An earlier
+version skipped those 48 links while still counting them as checked, behind a
+comment promising they were "checked below" when there was no below.
+
+**And the link check is not sufficient, which is the more important lesson.**
+It answers one question — does every href resolve — and a defect outside that
+question is invisible to it however carefully it is run. The worked example is
+a nested ``: a TOC entry wrapping a heading whose rendered text already
+contained a link produced `…`, and
+the HTML parser closes the outer anchor early, so the entry became an empty
+unclickable stub and clicking it navigated to GitHub. Both hrefs resolved
+perfectly. It shipped past the link check, past the whole-site link check, and
+past a screenshot pass, and was caught by a reviewer *clicking a link*. The
+repair is therefore two things, not one: the TOC keeps a tag-free variant of
+each heading, **and** a separate check asserts no `` appears inside another
+`` — proved by reverting the first half and watching the second report 66.
+
+The general form: when a check misses a defect, add the check that could have
+seen it rather than only fixing the defect, and ask what *question* the
+existing check was answering.
## 5. Why a node script and not a generator
@@ -142,13 +180,37 @@ may contain a single `*` pair**, because `[^*]+` cannot match
`**a *b* c**` and the leftover `*` then pairs across the bold — which is how a
single sentence in gotcha 33 came out as three italic paragraphs.
+A third, added later and for the same reason: **a table cell boundary is an
+unescaped pipe outside a code span**, so the row is walked rather than
+`split("|")`. This corpus documents a file format whose own quoting syntax is
+`|Time Period|`, and a sentence containing one inside a table cell would have
+been shredded into extra columns — a row that still parses, so no column count
+could detect it. It was latent when fixed, which is the point: the trigger is
+one ordinary sentence away in a document whose subject is pipe quoting.
+
## 6. Navigation
`repo-map.md` is one long document whose gotcha list is numbered 1 to 48, and
splitting it per gotcha was rejected: the gotchas cross-reference each other
constantly ("see gotcha 35", "for the reason gotcha 39 watches…"), and a reader
following those wants a scroll, not 48 page loads. It stays one page, with an
-id on every list item and a compact strip of 48 numbered links under the
-heading. The ids come from the list item's **own number**, not from a counter,
-so an inserted gotcha renumbers the anchors exactly as it renumbers the prose
-that cites them.
+id on every list item and a compact strip of numbered links under the heading.
+The ids come from the list item's **own number**, not from a counter, so an
+inserted gotcha renumbers the anchors exactly as it renumbers the prose that
+cites them.
+
+The strip's contents are **derived from the anchors actually emitted**, never
+from a count. A hardcoded 48 failed in only one direction: deleting a gotcha
+left a link to a missing anchor, which the link check catches, while *adding*
+one produced an unlinked gotcha and a clean build. The list only ever grows, so
+the hardcoded form was silent in precisely the direction it would be used —
+the same shape as an exclusion list that over-matches, failing in the direction
+that looks like success.
+
+**Long unbreakable headings need `overflow-wrap`.** The test-evidence pages are
+hundreds of `snake_case` test names as headings; at phone width seven of them
+overflowed the viewport by up to 92%. The diagnosis is worth keeping because
+the first guess was wrong: `getBoundingClientRect` reported every element
+inside the viewport, because the text overflows its box without widening it.
+Only `scrollWidth` per element names the culprit. Measure the property that
+actually overflowed rather than the one that usually does.
diff --git a/scripts/build-doc-site.mjs b/scripts/build-doc-site.mjs
index f197480e..e384c15d 100644
--- a/scripts/build-doc-site.mjs
+++ b/scripts/build-doc-site.mjs
@@ -110,7 +110,18 @@ const NON_TICKET = new Set(
// ---------------------------------------------------------------------------
// Markdown. A deliberately small subset, covering exactly what the corpus uses;
// anything unrecognised becomes paragraph text, so no input line is ever lost.
-const esc = (s) => s.replace(/&/g, "&").replace(//g, ">");
+const esc = (s) =>
+ s
+ .replace(/&/g, "&")
+ .replace(//g, ">")
+ .replace(/"/g, """)
+ .replace(/'/g, "'");
+
+// A TOC entry must not nest an anchor inside the one wrapping it: a heading
+// whose text is a path or a ticket id already carries an , and the parser
+// closes the outer one early, leaving an empty unclickable stub.
+const stripTags = (html) => html.replace(/<[^>]+>/g, "");
function slugify(s) {
return (
@@ -212,7 +223,7 @@ function renderMarkdown(src, ctx) {
seen.set(id, n);
if (n > 1) id = `${id}-${n}`;
ctx.anchors.add(id);
- if (level <= 3) toc.push({ level, id, text: rendered });
+ if (level <= 3) toc.push({ level, id, text: stripTags(rendered) });
const mark = level > 1 ? `#` : "";
return `
Gotchas: ${Array.from({ length: 48 }, (_, k) => `${k + 1}`).join("")}
` - : ""; + // Derived from the anchors actually emitted. A hardcoded count fails closed + // on a deleted gotcha (the link check catches it) and SILENTLY on an added + // one, which is the direction the list moves in. + const gotchas = [...ctx.anchors] + .map((a) => /^gotcha-(\d+)$/.exec(a)) + .filter(Boolean) + .map((m) => Number(m[1])) + .sort((a, b) => a - b); + const jump = gotchas.length + ? `Gotchas: ${gotchas.map((n) => `${n}`).join("")}
` + : ""; const srcLine = `Source: ${src}
`; written.push({ @@ -538,6 +587,15 @@ for (const section of SECTIONS) { // Landing page for the docs section. { + // The two conventions go through `inline` rather than being hand-written, so + // the sentence describing the linking demonstrates it instead of asserting it. + const conventions = inline( + "Ids like `improvise-9c8c` are tracker issues and link to the ticket index below. " + + "Paths like `crates/improvise-core/src/model/types.rs` link to the source on GitHub at `" + + ref + + "`.", + { depth: 0, cited, unresolved, anchors: new Set() }, + ); const cards = SECTIONS.map( (s) => `${s.blurb}
Nothing here is a tutorial. For what the program does, see the project home or run the browser demo.
${cards}Ids like improvise-9c8c are tracker issues and link to the ticket index. Paths like crates/improvise-core/src/model/types.rs link to the source on GitHub at ${ref}.
${conventions}
`; written.push({ rel: "index.html", html: page({ slug: "", title: "Documentation", depth: 0, body }), anchors: new Set(["conventions", ...SECTIONS.map((s) => s.id)]) }); } for (const w of written) write(w.rel, w.html); // --------------------------------------------------------------------------- +// An nested in an is closed early by the parser, so the outer link +// becomes an empty unclickable stub and the inner one wins. The link check +// cannot see it: both hrefs resolve. It asks a different question. +const nested = []; +for (const w of written) { + for (const m of w.html.matchAll(/]*>((?:(?!<\/a>)[\s\S])*?) inside near ${JSON.stringify(m[1].slice(0, 60))}`); +} + // Link check over the generated output. Every internal href must resolve to a // generated file, and every fragment to an id that page really carries. const anchorsByPage = new Map(written.map((w) => [w.rel, w.anchors])); @@ -574,7 +641,16 @@ for (const w of written) { continue; } const resolved = relative(".", join(base === "." ? "" : base, pathPart)); - if (resolved.startsWith("..")) continue; // leaves the docs tree: checked below + // A link out of the docs tree (../index.html, ../app/) names a file this + // script does not generate, so it is checked on disk instead — which is why + // assemble.sh builds the docs LAST, once its siblings exist. + if (resolved.startsWith("..")) { + const onDisk = join(out, base === "." ? "" : base, pathPart); + const withIndex = raw.endsWith("/") ? join(onDisk, "index.html") : onDisk; + if (!existsSync(withIndex) && !existsSync(onDisk)) + broken.push(`${w.rel} -> ${raw} (no such file outside the docs tree: ${withIndex})`); + continue; + } const target = resolved.endsWith("/") ? `${resolved}index.html` : resolved; if (!files.has(target)) broken.push(`${w.rel} -> ${raw} (no such page: ${target})`); else if (frag && !anchorsByPage.get(target)?.has(frag)) @@ -585,7 +661,9 @@ for (const w of written) { console.log(`doc-site: ${written.length} pages -> ${out}`); console.log(`doc-site: ${cited.size} ticket ids cited, ${cited.size - unresolved.size} resolve, ${unresolved.size} not in the export`); console.log(`doc-site: ${checked} internal links checked, ${broken.length} broken`); -if (broken.length) { +console.log(`doc-site: ${nested.length} nested anchors`); +for (const n of nested) console.error(`doc-site: nested anchor: ${n}`); +if (broken.length || nested.length) { for (const b of broken) console.error(`doc-site: broken link: ${b}`); process.exit(1); } diff --git a/web/assemble.sh b/web/assemble.sh index 73d74c6b..1fe1cd20 100755 --- a/web/assemble.sh +++ b/web/assemble.sh @@ -42,11 +42,6 @@ mkdir -p "$out" cp -f "$root/docs/index.html" "$out/index.html" cp -rf "$root/docs/casts" "$out/casts" -# The prose site under `docs/`, generated from context/ by the same script the -# harness runs. It link-checks its own output and exits non-zero on a break, so -# a dead cross-reference fails the assembly rather than shipping. -node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token" - app="$out/app" mkdir -p "$app" # shell.js and serial.js are imported by app.js and worker.js: leaving either @@ -103,3 +98,10 @@ for entry in app.js worker.js; do } done done + +# The prose site at `docs/`, generated from context/ by the same script the +# harness serves. It runs LAST because its link check verifies the links that +# leave its own tree (`../index.html`, `../app/`) against the filesystem, which +# means their targets must already be here. It exits non-zero on a break, so a +# dead cross-reference fails the assembly rather than shipping. +node "$root/scripts/build-doc-site.mjs" "$out/docs" "$token"