fix(protocol): the live host's answer is never refused as a left epoch (improvise-z2vr)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
2026-10-10 18:22:14 -07:00
parent cae6f26ee8
commit bf02a4b589
4 changed files with 118 additions and 18 deletions
@@ -119,6 +119,14 @@ dead host's replies are refused. A duplicate answer also prunes every queued ent
 and moves `seen` onto the live epoch, so a late notice from a dead host costs one extra round and cannot
 livelock `sync`.

+**The host that answers a request is live, whatever the client believed.** `left` is a guess, because
+epochs of different hosts are unordered: a late reply from a host the client never joined moves it onto
+that host, and writes off the live one. Refusing the live host's answers as "left" then strands the client
+(`sync` returned `Exhausted`, found by the property test). So `apply_answer`, which `sync` and `poll` use
+for the reply to a request they just sent, takes an epoch off `left` before it applies. `apply` is for a
+reply that arrives on its own, and keeps the guard. A stale reply can still move the client for one round;
+the next request it makes is answered by the live host and brings it back.
+
 `poll` hands a pulled notice only to the shape it asked about. A notice carries only that shape's keys,
 and applying it to another tracked shape would advance that shape past keys it never saw.

@@ -172,3 +172,20 @@ host, and the old reply then arrives. Rival: the `left` check removed.
 every restart and swap. Rival: the `left` check removed fails it. The older-generation guard on its own is
 not separable from `left` in the histories the property reaches, so it is held by
 `a_late_window_reply_from_an_older_generation_does_not_regress` alone.
+
+## The left-epoch livelock
+
+### a_late_reply_from_a_host_the_client_never_joined_cannot_strand_it
+
+The property test failed with `Exhausted` on a shrunk history of two restarts and four writes. Cause: the
+client was on host 3, never having joined host 2; a late host-2 reply was applied (an unknown epoch is
+accepted), which wrote host 3 into `left`, and every later answer from the live host 3 was then refused as
+a duplicate. The test replays that history by hand: three hosts, the host-2 reply lost in flight, the client
+settling on host 3, the late reply applied, the connection lost, and a sync that must settle on host 3's
+grid. Rival: `apply_answer` treated like `apply` (the `left.retain` removed); this test and the random
+property fail. Rival: the left-epoch guard removed fails `a_reply_held_across_a_host_restart_cannot_pull_the_client_back`
+and the property, as before.
+
+The property's invariant changed with it. "The watermark never returns to an epoch it left" was false by
+design once an unknown-epoch reply can move the client, so the property now checks the narrower, true
+rule: a replayed reply from an epoch the client was in and has left must be a duplicate.