Files

T

Edward Langley 3405fc4def kb: retire the Prolog MCP for a tool that costs the clause, not the predicate

loadProgram REPLACES a whole predicate, so adding one clause meant re-sending
every existing clause of it through an agent's context window. Measured on the
wave-3 batch: ~113KB read and ~113KB written to deliver ~25KB of new text. That
sets the behaviour -- writing becomes a thing you batch and do rarely -- and the
batch is where a short payload silently drops the rest of a predicate, which has
cost this repo 22 decisions and 19 doctrines once already.

scripts/kb reads clauses as DATA and never consults them, so the cost of adding
a clause is the clause. Canonical form is read_term in, portray_clause out --
measured, not designed: all 495 existing facts round-trip byte-identically and
the form is a fixed point in one pass. Rules move to one line, which is why the
reformat lands here rather than earlier: the MCP would re-expand them on its
next save and the two would thrash.

trealla moves into the devShell. Reaching it through `nix shell` per invocation
took the tool's first real use past a two-minute timeout for a write that takes
0.42s with tpl on PATH.

Four guards, each added because injecting its fault showed the tool accepting
what it should refuse. None was found by reading:

- an unparseable delta dropped trealla into its REPL, which HANGS with a
  terminal and EXITS 0 without one -- a refusal that reads as acceptance
- `-g main,halt` exits 0 on a throw AND on a failure, because a goal that fails
  never reaches the halt beside it; halt(2) alone exits 2, so the default is the
  hazard. The goal now guarantees a halt on every path (0/2/1)
- `42.` parses, is not callable, and landed as a fact whose head is a number;
  kb_consistent has no opinion about stray terms
- fmt discarded the merge's exit code and would have committed a failed
  canonicalization

Plus the two that were designed in: the result is read back and required to be
the clauses intended (variant equality -- == compares variable identity and
reports every rule as different), and kb_consistent runs on the CANDIDATE, so a
bad delta leaves the knowledge base untouched rather than needing a revert.

scripts/kb-test.sh proves all nine by injection. context/code/kb-tool.md carries
the argument, including what it does not reach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH

2026-09-20 15:54:39 -07:00

5.4 KiB

Raw Blame History

Maintaining the knowledge base

prolog-sessions/kb.pl is executable knowledge, checked by CI. This is the argument for how it is written, and why the Prolog MCP stopped being the way.

Why the MCP was retired

loadProgram replaces a whole predicate. Adding one clause therefore means re-sending every existing clause of that predicate — through an agent's context window, since the tool takes the program as a string. The wave-3 batch measured that cost: about 113 KB read and 113 KB written to deliver roughly 25 KB of new text, for 53 clauses.

That is not merely expensive. It sets the behaviour: writing becomes something you batch up and do rarely, when what a knowledge base wants is a fact written down at the moment it is learned. The batch is also where the danger lives — a short payload silently drops the rest of the predicate, which has cost this repository 22 decisions and 19 doctrines once already.

Three further frictions, each of which bit during that batch:

saveSession re-adds the MCP's own host_rpc shims on every write. AGENTS.md enumerates seven; there are eight — js_eval_json is missing from that list, so a grep built from it is one case short.
saveSession reserializes the whole file, so every write risks the formatting as well as the content.
assertz is not an escape hatch. A consulted predicate is static in this engine and assertz/retract raise permission_error(modify, static_procedure). A delta of directives was written and verified in a scratch engine before this was discovered; it can never run against a loaded session.

What replaced it

scripts/kb, a shell entry point over scripts/kb-merge.pl.

Clauses are read as DATA, never consulted. read_term/3 in a loop yields a list of terms; merging is list manipulation; portray_clause/1 writes them back. Nothing is ever asserted, so the static-procedure wall is not reached and the cost of adding a clause is the clause.

Canonical form is read_term in, portray_clause out. This was not designed, it was measured: all 495 facts in the existing file — including every escaped quote — round-trip byte-identically, and the form is a fixed point in one pass. The only change on first use is that rules move from the MCP's two-line layout to one line, which is why the reformat lands with this tool rather than before it: the MCP would re-expand them on its next save and the two would thrash.

trealla comes from the dev shell. Reaching it through nix shell per invocation cost more than the work: the tool's first real use timed out past two minutes on a write that takes 0.42 seconds with tpl on PATH.

The guards, and why each exists

Every one of these was added because injecting its fault showed the tool accepting something it should refuse. None was found by reading.

The round-trip check. After writing, the file is read back and required to be the clauses intended. A serializer bug is the wipe-the-knowledge-base failure with a new owner, and this is the only check that sees it. Note it compares variant equality via numbervars over a copy — reading a term back yields fresh variables, so == compares variable identity and reports every rule as a difference.

kb_consistent runs on the CANDIDATE. The file is replaced only after the check passes, so a delta that breaks the knowledge base leaves it untouched rather than requiring a revert.

Parse errors are caught explicitly. An uncaught syntax error drops trealla into its REPL, which hangs with a terminal and exits 0 without one — a refusal that reads as acceptance. scripts/kb also closes stdin on every call as a backstop.

Every run reaches a halt/1. tpl exits 0 unless a halt is reached: measured on 3.10.3, -g "fail" and -g "throw(boom)" both exit 0, and so do fail,halt and throw(boom),halt, because a goal that fails or throws never reaches the halt beside it. halt(2) alone exits 2, so halt works — the default is the hazard. The goal used is therefore

(catch(main,E,(write(user_error,E),nl(user_error),halt(2))) -> halt(0) ; halt(1))

which gives 0, 2 and 1 for success, throw and failure. Without it, if ! merge ... reads a crashed merge as a clean one and replaces the knowledge base.

Terms must be callable. 42. parses, is not a clause, and reached the knowledge base as a fact whose head is a number. kb_consistent has no opinion about stray terms, so nothing objected.

Duplicate keys are refused by default. A key is the functor plus every argument but the last, so doctrine(Name,_) keys on the name and decision(Id,Key,_) on the pair. replace is the door a correction goes through, and it overwrites in place — a count that grew would be two clauses under one name.

What no test here reaches

The key rule is looser than functional/1 for crate/3, which keys on the first argument alone; kb_consistent is the backstop for that case rather than the merge.

A clause name cited inside another clause's string is unchecked — dropping or renaming a clause silently dangles every clause referring to it (improvise-2f97).

A shebang would let kb-merge.pl run directly and collapse the two files into one. It does not work on trealla 3.10.3: #! is a syntax error that aborts the consult, so main/0 is never defined, and tpl wants -- args after the file, which a shebang cannot inject. Worth re-testing if trealla gains script support.

5.4 KiB Raw Blame History