Skip to content

handoff: the OK line says how old the snapshot is; a stale page end to end (discussion #413, ADR-0187) - #434

Merged
StepenkoAnatoli merged 4 commits into
mainfrom
stale-snapshot-handoff-2
Oct 21, 2026
Merged

StepenkoAnatoli merged 4 commits into
mainfrom
stale-snapshot-handoff-2

Conversation

@StepenkoAnatoli

Copy link
Copy Markdown
Owner

Discussion #413: a builder can receive a snapshot that is intact but stale, and handoff OK said nothing about how old it was. This PR does three things:

  • It documents what the handoff proves and what it does not (Part A).
  • It makes the OK line give the snapshot's age (C1, ADR-0187, which lifts the freeze for this one item).
  • It pins the whole stale-page path in an offline end-to-end test (Part B).

Commits (one task each)

  1. docs: handoff OK proves the corpus is intact, not that its pages are current.
    • README builder section and the "Try it" pointer.
    • freshness-recheck SKILL.md step 2: --refresh-days 0 against --force, and what --only limits.
    • CHANGELOG.
  2. handoff: the OK line says how old the snapshot is (ADR-0187).
    • verifyHandoff returns fetched: { from, to, oldestDays }, or null.
    • The OK line ends ; captures fetched <from>..<to> (oldest N days), or ; no capture date on record.
    • Each row is dated the way stale-evidence dates it: by the capture its Raw cell names, and by its Retrieved cell when it names none (freshnessDay, now exported).
    • Only current rows count: superseded rows are left out, and an unparseable day is left out.
    • The age is at least 0.
    • The age is never a finding. The exit code and the doctor line are unchanged.
    • fetched does not reach packages: the artifact copies only ok, entries and findings.
    • ADR-0187, with ARCHITECTURE, CONTEXT, README, CHANGELOG and 4 tests in the same commit.
  3. test: a page that changes after collection, end to end.
    • Setup: a local page server in its own process, http-keyless, and a temp config and kit home for each machine.
    • Collector: collect, cite, PASS, brief.
    • Then the page changes.
    • Builder: handoff stays OK and dated, and preflight PASSes.
    • Collector: --refresh-days 0 collects 0.
    • --force adds E-02, and the gate FAILs with cites-superseded-row and brief-stale.
    • Moving the citation brings PASS back, and the old row and its capture are kept.
  4. docs: --force also runs every search again, closed ones included (a fix to commit 1's wording, found in review).

Verification

  • Rebuilt on the rename. This change was first built on cd11b49, before the Fetchproof rename (rename: the kit is Fetchproof, with a clean break (ADR-0186) #433), and rebuilt here on the renamed main: paths fetchproof/, variables FETCHPROOF_*, temp prefixes fp-. Its ADR is 0187 because the rename took 0186. Apart from the README count, every line it adds or removes is the first build's mapped through the rename's spelling (checked by diffing the two). The red-first runs and the review below were made on the first build.
  • C1 was written red first: handoff 34/3 and cli 149/2 before, then 37/0 and 151/0 after.
  • Part B's red check: with evidence-supersession removed from the registry, the test fails on the preflight exit.
  • Each commit passed the commit gate of its own tree, node fetchproof/bin/gate.mjs --gate commit, run by hand on the staged index with FETCHPROOF_TEST_TIMEOUT=180000. Its suite rule ran the full suite on that index (ADR-0120, ADR-0183): allow in 3679, 3835, 4005 and 4004 s for commits 1 to 4. Gates 3 and 4 ran at the same time. Gate 4 ran in a scratch worktree whose HEAD tree was commit 3's, so it saw commit 4's exact diff. The hook installed on the operator's PC is still the pre-rename kit's. It does not recognise a fetchproof/ checkout and skips the suite rule (ADR-0186, Consequences), so the gate was run by hand first. Each commit then went through the installed hook. No --no-verify.
  • The README test count is 2113 (main 2104, +8 in commit 2, +1 in commit 3).
  • A fresh-context breaker review found no blocker. Findings F1-F8 were fixed in commits 2-4.
  • Nothing touches the network beyond 127.0.0.1. There are no Firecrawl credits, no real ~/.agents, no global git config and no keys.

To flag

  • The line prints on every role. It was briefed as "on a builder", but the approved plan's C1 had no role condition, and a collector checking before a push should see what the builder will see. This is a rejected alternative in ADR-0187.
  • An intermittent red in the commit gate, not caused by this change. browser-transport > LIVE: a page whose resource never arrives… failed in 2 of the first build's 6 gate runs on the operator's PC. It is a guard bug: on Windows the guard waited with no limit for a killed Chromium's exit, so it overran its 70 s. It is fixed separately on branch fix/stalled-flake (ADR-0188), in its own PR. Both PRs add rows to the ADR index, ARCHITECTURE and CHANGELOG, and change the README test count. Whichever merges second rebases those, and the count is 2117 with both in.
  • The Research-Kit: primary-source research for AI coding agents #413 reply is drafted but not posted; it waits for the operator's OK.

🤖 Generated with Claude Code

StepenkoAnatoli and others added 4 commits October 21, 2026 15:41
…current

what changed
README.md, "Hand the research to a builder": a paragraph saying that
`handoff OK` means the corpus arrived intact and consistent, not that the
pages still say the same thing; a builder cannot fetch, the fetch dates
are the Retrieved column of research/EVIDENCE.md, a builder who suspects
a change reports it, the collector re-checks it (freshness-recheck), and
re-collecting a changed page fails the gate with cites-superseded-row
until the citation moves. README.md, "Try it in five minutes": one
closing line pointing there. fetchproof/skills/freshness-recheck/
SKILL.md, step 2: `--refresh-days 0` does not re-collect a capture made
today; a same-day re-check is `--force` with `--dry-run` first, which
re-fetches every page the plan names, and `--only` narrows only the
searches. CHANGELOG Unreleased entry.

why
Discussion #413 (bomtobe-hgkim, 2026-10-17): a handoff should let the next
agent tell a current citation from a consistent but stale snapshot. An
offline spike on cd11b49 (local server, http-keyless, temp configs)
showed a builder's handoff.mjs and preflight still pass after the source
page changes, by design - the handoff checks integrity, and a builder
cannot fetch - and that nothing told the reader so. Plan items A1, A2,
A3, A5 (docs; fits the feature freeze, ADR-0117). The age on the handoff
line (C1) and its ADR are the next commit.

what it touched
README.md, fetchproof/skills/freshness-recheck/SKILL.md, CHANGELOG.md.
No declared code path, no ADR owed (wording), no CONTEXT.md term.

what you verified
verified: in a scratch project (temp FETCHPROOF_CONFIG and
FETCHPROOF_HOME, a local HTTP server, http-keyless), two plan pages
collected today; `research.mjs --refresh-days 0 --dry-run` -> both
"cached ... fresh capture from 2026-10-21", collected 0; `--force
--dry-run` -> both "would collect (refreshed: --force)"; `--force --only
zzz --dry-run` -> refused, "--only matched none of the plan's 0 queries".
verified: `research.mjs --help` says "--refresh-days <n> re-collect a
capture older than n days" and "--only <text> run only the plan queries
containing this text"; the skill's wording follows it.
expected: the anchor #hand-the-research-to-a-builder is GitHub's slug of
the existing "## Hand the research to a builder" heading.
verified: full suite in the commit gate on this commit's own index
(ADR-0183), run by hand with fetchproof/bin/gate.mjs --gate commit:
allow in 3679 s; the installed hook is the pre-rename kit's and
skips the suite rule (ADR-0186, Consequences).

what you got wrong and fixed
mistake: the plan said to recommend `--force` "with `--only` to limit it"
for a same-day re-check.
where: discussion-413 plan, item A3 (not in the repository).
impact: none delivered; the skill would have told a collector that
--only limits the pages re-fetched, and it does not.
cause: I read --only as a filter on the whole plan; lib/research-run.mjs
applies it to settings.queries only, never to plan.urls.
fix: the skill says --force re-fetches every page the plan names and
--only narrows only the searches.
verified: the --force dry-runs above, with and without --only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
what changed
lib/handoff.mjs: verifyHandoff(root, { corpus, now }) returns `fetched:
{ from, to, oldestDays }` (fetchedRange) - the first and last day the
current evidence rows' captures were fetched and the oldest one's age
in whole days, never below 0 - or null when no current row carries a
date that parses. Current rows are every row but those supersededRows
says a later capture of the same URL replaced (ADR-0026 keeps them as
history). Each row is dated by freshnessDay, the rule stale-evidence
uses: the capture the Raw cell names, the Retrieved cell when it names
none. It never changes `ok` and adds no finding. lib/checks.mjs:
freshnessDay is exported. bin/handoff.mjs: the OK line ends `; captures
fetched <from>..<to> (oldest N days)` or `; no capture date on record`,
on every role; --help says the handoff checks integrity, not currency.
ADR-0187 records the decision and lifts the freeze (ADR-0117) for this
one item.

why
Discussion #413: a builder got `handoff OK` and PASS for a corpus whose
source page had changed. That is correct - the handoff checks the corpus
arrived intact, and a builder cannot fetch - but nothing the builder
read said how old the snapshot was; the only age signal was
stale-evidence, after 180 days. The owner chose (2026-10-20) to show the
age now rather than defer it behind a trigger (plan item C1). Rejected,
in the ADR: docs only; a Retrieved column in the brief (C2); the git
commit in the brief (C3); a handoff threshold; dating by the editable
cell (G8); a second dating rule for blank-Raw rows; counting superseded
history; a builder-only line.

what it touched
fetchproof/lib/handoff.mjs, fetchproof/lib/checks.mjs,
fetchproof/bin/handoff.mjs; fetchproof/test/handoff.test.mjs (seven
tests: the range and age; capture, not cell; null with no rows;
superseded history left out; a blank-Raw row dated by its cell; an
unparseable day left out; a future-dated capture is 0 days old),
fetchproof/test/cli.test.mjs (the "counts the rows" test pins the full
line; a new builder-line test); docs/adr/0187-*.md and its index row;
docs/ARCHITECTURE.md rows for checks.mjs and handoff.mjs and the bin
handoff.mjs entry; CONTEXT.md "handoff check" (integrity, not currency;
`fetched`); README builder section (the line's dates); CHANGELOG;
fetchproof/README.md test count 2104 -> 2112.

what you verified
verified: red first - with the first three handoff tests and the cli
tests before the code, `selftest.mjs handoff` 34 passed / 3 failed and
`selftest.mjs cli.test` 149 passed / 2 failed (both OK-line tests).
verified: after the code, handoff 37/0 and cli.test 151/0 (687 s).
verified: the review's three code findings red first - superseded
(from 2026-09-13, 37 days), unparseable (from 2026-00-13, null days),
future-dated (-1 days) failed; the blank-Raw pin passed (it pins the
documented rule); after the fix `selftest.mjs handoff` 41 passed / 0.
verified: `selftest.mjs architecture-map repo-hygiene bundle` 45/0
(the docs this commit edits).
verified: no import cycle - checks.mjs and its imports import none of
handoff, doctor, gate or artifact*; those already loaded checks.mjs.
verified: lib/artifact.mjs copies only ok, entries and failing findings
from the handoff report into a package, so `fetched` (which depends on
the day) never reaches a package or its digest; CI workflows use
handoff's exit code only; the panel shows its output as printed.
verified: a fresh-context review (first-pass:breaker) of the diff:
no blocker; its findings on this commit are fixed below.
verified: the runs above were made on this change's first build, on
cd11b49 before the rename (ADR-0186); this commit is that change rebuilt
on the rename: its lines are the first build's mapped through the
rename's spelling, with this ADR numbered 0187 (the rename took 0186),
and only the README test count differs.
verified: full suite in the commit gate on this commit's own index
(ADR-0183), run by hand with fetchproof/bin/gate.mjs --gate commit:
allow in 3835 s; the installed hook is the pre-rename kit's and
skips the suite rule (ADR-0186, Consequences).
expected: doctor's handoff line is unchanged (not edited).

what you got wrong and fixed
mistake: the age counted superseded history rows.
where: lib/handoff.mjs fetchedRange; ADR-0187 decision 4.
impact: caught in review before commit: after the collector re-checked
a page, the line kept the first fetch's age forever (the old row is
kept), so it could not tell a re-checked snapshot from a stale one -
the point of #413.
cause: I scoped to "every row the line counts" without following a
re-collect to its end.
fix: rows supersededRows names are left out; ADR, map, CONTEXT, index
and CHANGELOG say "current rows".
verified: the superseded test, red then green.

mistake: the CHANGELOG, the map row and the ADR index said the cell is
read only when there is no dated capture, and that a cell edit cannot
make the corpus look younger.
where: CHANGELOG.md, docs/ARCHITECTURE.md handoff.mjs row,
docs/adr/README.md 0187 row (decision 3 itself was precise).
impact: an overstatement, caught in review: a row with a blank Raw cell
is dated by its cell even when a ledgered capture of its URL exists, so
one edit can.
cause: I described the rule by its usual case.
fix: all say the cell is read when the Raw cell names no capture; a test
pins the rule; borrowing the URL's earliest capture is recorded as
rejected (a second rule beside stale-evidence's).
verified: the blank-Raw test.

mistake: an unparseable ISO-shaped day printed "(oldest null days)", and
a capture dated after the asking machine's clock "(oldest -1 days)".
where: lib/handoff.mjs fetchedRange.
impact: caught in review; a garbled OK line, nothing else.
cause: I trusted freshnessDay's shape and today's clock.
fix: days that do not parse are left out; the age is at least 0.
verified: both tests, red then green.

mistake: the README said each capture's own date is in the Retrieved
column.
where: README.md, builder section.
impact: wording; the column is the editable cell, the line reads the
capture's date.
cause: loose wording.
fix: the README says the line reads the date each capture recorded at
its fetch and the column gives each row's date.
verified: re-read against lib/checks.mjs freshnessDate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
what changed
fetchproof/test/handoff.test.mjs: one offline end-to-end test. A local
page server, in its own process, serves "10 requests per minute"; the
real bins collect it over http-keyless on a collector (its own temp
FETCHPROOF_CONFIG and FETCHPROOF_HOME), an unknown closes on E-01,
preflight PASSes and the brief is drafted. The page then says 2. On a
builder (its own temp config), handoff exits 0 with the C1 date range
on its OK line (`captures fetched <day>..<day> (oldest 0 days)`, or 1
day past UTC midnight) and preflight PASSes. On the collector,
`--refresh-days 0` collects 0; `--force` adds E-02 with the new text;
preflight FAILs with cites-superseded-row (E-01 to E-02) and
brief-stale; once the citation moves to E-02 it PASSes, and E-01 and
its capture are kept. A run that crosses UTC midnight before the
same-day step ends UNSUP, not red. The server exits when its stdin
closes. fetchproof/README.md: test count 2112 -> 2113.

why
Discussion #413, plan Part B: pin what the kit does with a consistent
but stale snapshot, so a change to the handoff, the gate's supersession
check or the refresh rule shows up as a red test.

what you verified
verified: `selftest.mjs handoff` 42 passed / 0 (Part B 17 s on its
own, while another suite ran; each spawn has a 60 s timeout).
verified: red check - with evidence-supersession dropped from the check
registry in lib/checks.mjs (restored after), the test FAILs on the
preflight exit (`0 !== 1`): the gate passes over the stale citation.
verified: no network beyond 127.0.0.1, no Firecrawl, no real ~/.agents:
the bins run with temp configs and kit homes and with FETCHPROOF_* and
proxy variables stripped; the plan has no queries.
verified: on Windows a parent killed outright leaves no server behind,
with or without the stdin exit (libuv's kill-on-close job).
expected: on POSIX (CI's ubuntu jobs) the stdin exit is what stops an
orphan when the suite is killed from outside; not run there.
verified: the runs above were made on this change's first build, on
cd11b49 before the rename (ADR-0186); this commit is that change rebuilt
on the rename: its lines are the first build's mapped through the
rename's spelling, and only the README test count differs.
verified: full suite in the commit gate on this commit's own index
(ADR-0183), run by hand with fetchproof/bin/gate.mjs --gate commit:
allow in 4005 s, at the same time as commit 4's gate (two suites at
once); the installed hook is the pre-rename kit's and skips the
suite rule (ADR-0186, Consequences).

what you got wrong
mistake: the first draft would go red, not UNSUP, if a run crossed UTC
midnight at the `--refresh-days 0` step, and left its server running if
the suite was killed from outside.
where: fetchproof/test/handoff.test.mjs, the new test.
impact: caught in review before commit; a rare false red in the commit
gate, and an orphaned server holding the gate's scratch folder on POSIX.
cause: I tolerated midnight only on the handoff line, and relied on
`finally` alone to stop the server.
fix: the same-day step ends UNSUP when today() is no longer the
collection day; the server exits when its stdin closes.
verified: the test passes; the orphan check above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
what changed
fetchproof/skills/freshness-recheck/SKILL.md step 2 and the CHANGELOG
docs entry: `--force` re-fetches every page the plan names AND runs
every search again, those for closed unknowns included, keeping up to
`perQuery` pages each within the run's scrape budget; the dry run lists
what it would spend.

why
A fresh-context review of this branch (2026-10-20) found the first
commit's wording understated what `--force` spends: lib/research-run.mjs
skips a search for a closed unknown only without --force
(`force ? [] : closedUnknowns(why)`), so --force pays for every search
and the pages it keeps, not only the plan's pages.

what you verified
verified: read lib/research-run.mjs (the closed-unknown skip, perQuery,
the scrape budget) against the new wording.
verified: full suite in the commit gate on this commit's tree
(ADR-0183), run by hand with fetchproof/bin/gate.mjs --gate commit in a
scratch worktree whose HEAD tree was commit 3's, so the gate saw this
commit's diff: allow in 4004 s, at the same time as commit 3's gate;
the installed hook is the pre-rename kit's and skips the suite rule
(ADR-0186, Consequences).

what you got wrong
mistake: the skill and CHANGELOG said `--force` re-fetches every page
the plan names, and stopped there.
where: fetchproof/skills/freshness-recheck/SKILL.md step 2;
CHANGELOG.md docs entry (commit a6408d1 on this branch).
impact: not delivered (the branch is not merged); a collector could
have under-estimated a same-day re-check on a metered route - the dry
run would still have shown it.
cause: I checked --force against the plan's URLs only.
fix: both say it also runs every search again.
verified: re-read against lib/research-run.mjs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StepenkoAnatoli
StepenkoAnatoli merged commit 30b77c9 into main Oct 21, 2026
7 checks passed
@StepenkoAnatoli
StepenkoAnatoli deleted the stale-snapshot-handoff-2 branch October 24, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant