Skip to content

Reconciliation: a pass that did not succeed and lists no record is read, after a second look (#709, record 0190) - #716

Merged
don-petry merged 4 commits into
mainfrom
adapters/cancelled-passes-read-709
Nov 11, 2026
Merged

don-petry merged 4 commits into
mainfrom
adapters/cancelled-passes-read-709

Conversation

@don-petry

@don-petry don-petry commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Refs #709 ("Factory alerts"): the reconciliation_unreadable alert, raised by every pass since 2026-11-10T03:06Z. It does not close the issue, which is the Controller's own.

What was wrong

A cancelled or failed poller run uploads no written. adapters/passes.written took "lists none" on such a run as "may be yet to", so it was never counted read: every later pass asked of it again, and each asking spent one of the 50 a pass reads (READS). The poller cancels its waiting run whenever another is queued, so the 30-hour window held 50 cancelled passes and 1 failed, against 180 that succeeded. From 51 unread on, every pass found more than 50 to read and was short, with nothing unread that could be read.

What changes

A pass that did not succeed and lists no written Looked at once more in the same pass, after one wait of 5 seconds for them all (WAIT, as decision 0036's in adapters/records.py). Listing none again, it is read with nothing recorded, so no later pass asks of it
Each look Asks for the run's list of artifacts uncached (Cache-Control: no-cache), the first and the second. GitHub lets the answer be held for 60 seconds (decision 0036, trials/P-63.json), so a second look 5 seconds on could otherwise be the first's answer
What the second look finds A record is read as any record is. A list or a record that cannot be read, or one that is no written/1, leaves the pass unread and what was read short
A pass that succeeded and lists none Unchanged: short at its one look
A record that holds null No written/1: the pass is not read, not looked at twice, and what was read is short, whatever the pass concluded. Before, it passed for a pass that lists none
A completed run with no conclusion Unchanged: not looked at twice, not read, asked of again by a later pass. The second look is for a conclusion that says the pass did not succeed
A run not completed Unchanged: not looked at, read once it has ended
A write such a pass made Matched to no record, so write_unmatched is raised. Reconciliation fails closed where it always did. Should GitHub's list lag past both looks, a recorded write alarms as unmatched; it is never passed in silence
The bound most bounds the first looks as before; the second looks are at most as many again, one request each unless a record has appeared, inside the step's 6 minutes

The record is docs/decisions/0190-a-pass-that-did-not-succeed-and-lists-no-record-is-read-after-a-second-look.md, 0190 by the Factory Builder's number (main holds 0184 to 0189). It amends one row of 0112, for passes only. The branch carried it as 0184, a placeholder, through 4adf2de; 98b592d renames it and c29dcc2 merges main, and nothing else has changed since the readers' heads.

Not here, each a follow-up for its own ruling:

  • adapters/applied.effects has the same shape for a Run that failed and lists no applied (read again at every pass, counted toward its 50). The ruling was on the passes, and the record says so.
  • The entry (controller/__main__.py) does not hand its injected sleep down to passes.written, as it does not to records.runs on main. A test holds that the default is time.sleep.
  • 0112's own Boundaries row still says a pass that failed and lists none "is read again". 0190 amends the Decision row; the checks accept it.

The step's budget: over the last 40 successful poller runs "Read and decide" took 134 s on average and 197 s at most, of its 360 s. This adds one wait of 5 s and at most 50 list requests to a pass, and passes get shorter once the backlog of 51 is read.

It ships in the Release after v0.1.4. At its first pass it clears the backlog of 51 in two passes: 50 first looks, one wait, 49 or 50 second looks.

Checks

  • Four boundary tests in tests/test_adapters.py: test_a_pass_that_did_not_succeed_and_lists_none_is_read_after_a_second_look, test_more_passes_that_list_none_than_a_pass_reads_are_read_whole_by_the_next, test_a_pass_still_running_is_not_looked_at_and_may_yet_record, and test_a_write_by_a_pass_that_failed_and_recorded_nothing_is_unmatched_not_short, which runs the read through controller.reconcile. Each was watched failing first; the assertions the review added are each held by a mutant that puts the old code back.
  • Twenty-two new mutants and seven retargeted across the two commits, each named in the record's Boundaries: twenty-one in adapters/passes.py, one in adapters/artifacts.py. Two old ones are gone with the rule they held ('a pass that lists none counts as read', 'a pass past its retention is read again').
  • make checks is clean at c29dcc2, and tests.test_adapters, tests.test_reconcile and tests.test_controller pass (231 tests). Every mutant of adapters/passes.py (73) and of adapters/artifacts.py (28) was applied at the desk at c29dcc2 against the tests it names, and each was killed. The desk's unfiltered run of 4adf2de, restarted after a reboot ended the first at 4,430, finished after the merge: 6,439 of 6,439 mutants killed; an earlier try could not finish its baseline inside 600 s on a loaded machine. The merge group runs every shard.

Protected paths

adapters/, tests/ and tools/ are the verify Lens's inputs, and tools/mutants.py and docs/decisions/ are reported paths: pre-approved by the Maintainer 2026-11-08, ruled by the Factory Builder (the ruling of 2026-11-10 on #709).

Reviews

Reader Head Findings Disposition
Argon (gemini-4-argon-high) afa64c0 0, four parts read clean
Muse (muse-wonder-full-1.0) afa64c0 0 clean
Claude review afa64c0 3 major, 2 minor, 1 info Fixed in 4adf2de: both looks could be answered from GitHub's 60-second cache; no test held where the wait falls; no test held that every pass listing none is looked at again; a null record passed for "lists none"; per-pass dating and a later second look after one unreadable were unheld. Held, not threaded: the default sleep (follow-up above)
Claude security pass afa64c0 0 blocking or high; 3 low, 4 info No fail-open found. Fixed: a completed run with no conclusion was looked at twice. By design, in 0190: a list that lags past both looks leaves a standing false write_unmatched. Measured above: the step's budget. Pre-existing and not changed here: a written/1 is not bound to its run and attempt; deleting a failed run's unreadable written clears the short read
Astra (gpt-6-astra) 4adf2de 0 clean, the one read after the fixes

🤖 Generated with Claude Code

…ad, after a second look (#709, record 0184)

A cancelled or failed poller run uploads no `written`, and was never counted
read: every later pass asked of it again, and each asking spent one of the 50
a pass reads. With 51 such runs in the 30-hour window, every pass since
2026-11-10T03:06Z was short and raised `reconciliation_unreadable`.

Such a pass is now looked at once more in the same pass, after one wait of
5 seconds for them all. Listing none again, it is read with nothing recorded.
A write it made is matched to no record, so `write_unmatched` is raised:
reconciliation fails closed where it always did. A pass that succeeded and
lists none is still short; a run not completed is still not looked at.

Co-Authored-By: Claude <noreply@anthropic.com>
@don-petry don-petry added the factory:hands-off A person's mark: the Factory writes nothing on this pull request (kernel/terms.py LABEL_HANDS_OFF) label Nov 10, 2026
@don-petry
don-petry requested a review from a team as a code owner November 10, 2026 17:28
@don-petry don-petry added the factory:hands-off A person's mark: the Factory writes nothing on this pull request (kernel/terms.py LABEL_HANDS_OFF) label Nov 10, 2026
@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-10T17:28:08.602552Z
We couldn’t complete this review. Comment @codex review to try again.
afa64c0 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

don-petry and others added 3 commits November 10, 2026 12:16
…hat holds null is short (#709, record 0184)

The review of the second look found what the tests did not hold:

- Both looks went through a list GitHub lets be held for 60 seconds, so a
  second look 5 seconds on could be the first's answer. Each look now sends
  Cache-Control: no-cache, as the records port does (decision 0036).
- A `written` file holding `null` passed for a pass that lists none. It is
  no `written/1`: the pass is not read, not looked at twice, and what was
  read is short.
- A completed run with no conclusion is not looked at twice and not read,
  as before the change: the second look is for a conclusion that says the
  pass did not succeed.
- The tests now hold where the wait falls, that every pass that listed none
  is looked at again, that each record is dated as its own pass, that a
  second look that cannot be read does not end the ones after it, and that
  the wait is time.sleep unless another is handed.

Ten new mutants, three retargeted; record 0184 has the rows.

Co-Authored-By: Claude <noreply@anthropic.com>
… record 0190)

The Factory Builder's number, given at enqueue: main holds 0184 to 0189.

Co-Authored-By: Claude <noreply@anthropic.com>
… a pass (#709, record 0190)

Co-Authored-By: Claude <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt Review Status

✅  Review completed

Status Commit Started (UTC) Finished (UTC)
✅  #e9762 Reviewed your PR c29dcc2 Nov 10, 2026 · 23:29 23:33

@codeant-ai

codeant-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@don-petry don-petry changed the title Reconciliation: a pass that did not succeed and lists no record is read, after a second look (#709, record 0184) Reconciliation: a pass that did not succeed and lists no record is read, after a second look (#709, record 0190) Nov 10, 2026
@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Nov 10, 2026
@sonarqubecloud

Copy link
Copy Markdown

@codeant-ai

codeant-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown

CodeAnt  CodeAnt Nitpicks

1 code suggestion

1. controller.main's injected sleep is not forwarded to written, so eligible failed-pass retries still block for five real seconds when callers provide a no-op or controlled sleep.

Api mismatch · adapters/passes.py:203

@codeant-ai

codeant-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown

CodeAnt  CodeAnt PR Risk

🟢  Low Risk

  • The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: c29dcc2be58c

@don-petry
don-petry added this pull request to the merge queue Nov 10, 2026
Merged via the queue into main with commit 7a01a21 Nov 11, 2026
31 checks passed
@don-petry
don-petry deleted the adapters/cancelled-passes-read-709 branch November 11, 2026 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory:hands-off A person's mark: the Factory writes nothing on this pull request (kernel/terms.py LABEL_HANDS_OFF) size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant