Repository navigation
fix(task-orphan-check): a task a pool worker has accepted is in flight, not an orphan - #4198
qingyun-wu wants to merge 1 commit into
Conversation
…t, not an orphan The route handler hands a task to a worker by writing deliveries/<worker>/<id>.accepted; the boot pass read only results/ markers, so an accepted task past 300 s was archived and re-queued, and the owner got two answers. classify.py gains a delivered verdict (accepted or claimed in any inbox): left in tasks/, never archived, counted but not listed in the recovery DM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 90%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
|
|
Reviewed at head Bug confirmed on main (REVIEW.md criterion 1). Main's So the double-answer incident is reproducible from the code, not only from the report. The finding — the safety property rests on a caller that does not exist yet. SKILL.md says the task is left in The crash-recovery machinery is written and has no importer on main. So today a stale The failure mode inverts rather than closing: before, a delivered task was archived and re-queued and the owner got two answers, which is noisy but visible. After, a task held by a dead worker is never archived, never re-queued and never listed in the recovery DM — silent, and invisible precisely in the pass built to find stranded work. I do not think that blocks this PR; the trade is deliberate and the loud failure was the worse one. But Worth saying plainly: Stand: Echo Act IV Pro |
qingyun-wu
left a comment
There was a problem hiding this comment.
Changes requested: account for the pending delivery state through the existing delivery owner. The accepted-marker fix is real: HEAD runs 60 tests successfully; replacing only classify.py with current main causes the three new delivered assertions to fail.
An adjacent production transition still reproduces the duplicate-execution problem:
accepted marker -> classify: delivered
pool_delivery.release(the same marker) -> pending .txt sentinel
same old task -> classify: orphan
That pending sentinel still assigns the task to its worker. Releasing acceptance for retry does not return authority to the core’s orphan recovery. The current private suffix check loses that fact and can archive/requeue the task while the worker retakes it. Use the shared owner’s delivery/held contract with the appropriate serialization or injected optional-capability seam; cover pending, accepted, release, and no-assignment controls.
I read the full diff, classifier/archive instructions, production release behavior, and the existing reviewer’s dead-worker note. The stale-worker visibility concern is separate from this reproduced assignment loss. Added-path checks pass. Worst case remains the owner receiving duplicate executions/replies after a normal worker release.
Reviewed head 04deefdb13905fc8eabca11805470545630087be. Current GitHub mergeability: MERGEABLE; failed/cancelled checks: none. Required CI, CLA, two current maintainer approvals, and any stated parent merge order remain separate gates. No merge, contributor-branch change, or live service restart performed.
COMMENT verdict because the authenticated account also authored this PR.
| return "" | ||
|
|
||
|
|
||
| DELIVERY_SUFFIXES = (".accepted", ".claimed") |
There was a problem hiding this comment.
[P1] A pending sentinel is still a worker assignment. With the same old task, the real pool_delivery.release() renames .accepted to .txt; this helper then returns no holder and classify_task() changes delivered to orphan. Recovery can archive/requeue work the worker is about to retake. Delegate to the delivery owner’s held/assignment contract, including pending and release transitions, instead of copying two suffixes.
keweichen
left a comment
There was a problem hiding this comment.
REQUEST_CHANGES — exact head 04deefdb13905fc8eabca11805470545630087be. The accepted/claimed-marker defect is real, but this head still archives pool-owned work in an adjacent state and fails open when delivery ownership cannot be read. It is not ready to merge.
HEAD^ equals merge-base 2da48ab76cd902a892fcf0c62ca867f892d63518; this is one topic commit, not a stack. I could not inspect the cited host log, but independently reproduced the parent defect by running this head’s six worker-delivery tests against the parent classifier:
Ran 6 tests — failures=3
accepted_by_a_worker / claimed_by_a_worker / delivered_is_counted
The exact-head classifier suite runs all 60 tests successfully; test definitions moved 54 → 60 and -v showed all six new names collected.
Blocker 1 — a pending sentinel is already an assignment. src/pool_delivery.py:5-15 says deliveries/<worker>/<id>.txt existing is the assignment; find() checks pending, accepted, and legacy-claimed names at src/pool_delivery.py:123-131, with transition arbitration defined at lines 134-145. The private matcher added at skills/task-orphan-check/scripts/classify.py:199-213 checks only .accepted and .claimed and takes no such ownership seam.
A temp-workspace repro using production pool_delivery.residue() and accept() produced:
pending marker: pool_residue=pending head=orphan
accept-during-lookup: verdict=orphan accepted_exists=True pending_exists=False
accept-after-classify: cached_verdict=orphan payload_exists=False pool_residue=stale-sentinel
The last line applies the documented ORPHAN archive from skills/task-orphan-check/SKILL.md:175. A recipient can therefore lose its payload, or—if it already read the payload—finish while recovery requeues the same work, reproducing duplicate side effects. Move this decision to a shared/injected delivery-owner predicate covering pending, accepted, and legacy-claimed states plus recipient grammar; do not merely append .txt to the local suffix tuple, because sequential probes still race with accept/release. Add pending and transition controls.
Blocker 2 — an unreadable delivery tree becomes destructive ORPHAN. root.iterdir() at classify.py:209 can raise, and the broad handler at classify.py:332-337 mislabels every such failure as an unreadable task file and returns orphan. With a readable 10-second-old task and accepted marker, forcing only the deliveries listing to raise gave:
fresh-readable-task + unreadable-deliveries: verdict=orphan; unreadable task file (delivery tree unreadable); treated as orphan
Ownership lookup needs a third unknown result that leaves the task untouched and reports the uncertainty. Absence and inability to check are not equivalent in an auto-archiving startup pass.
Live-path gate: skills/startup/SKILL.md:37-50 invokes this before the watcher, so a real post-restart pool round trip is required. The supplied evidence covers the incident and harnesses, while the existing thread explicitly says the fixed path was not exercised live. After fixing the two cases above, show a real worker-owned task across a core restart producing exactly one reply, with no orphan recovery/requeue.
Worst case as written is a silently archived voice/phone ask or duplicate external side effect/conflicting reply. The mitigation is an authoritative fail-closed ownership query: held and unknown are never archived, and stale holds are surfaced without requeueing.
Focused checks run: classifier 60/60, pool-delivery 60/60, import-trigger 13/13, git diff --check clean, and merge-base diff review checks PASS. No host-specific paths were added; the path scanner’s /Users/reviewer/probe positive control failed with exit 1 as expected. The supplied hosted checks and CLA are green, but the implementation fixes and real restart witness above are required before approval.
Reviewed by Kewei's Codex automation.
|
Closed at the owner's direction (PR triage, 2026-09-12 03:45Z: "close the ones that are dated"). Superseded: #4206 landed the module move, and the tested multi-worker behavior is being consolidated into one PR on top of |
What broke
/task-orphan-check(boot, step 1 of/startup) classifies a live task byresults/markers and age. On a pool host the route handler delivers a task to a worker by writingdeliveries/<worker>/<id>.acceptedand the worker'sfinisharchives the task file when it answers — but the boot pass never reads that marker. So an accepted task older than 300 s was ORPHAN, archived into the recovery DM, and the core re-queued and answered it.Observed 2026-09-11 (this host,
logs/pool-route-handler.log): three owner asks delivered to worker39041ce6…at 12:19–12:31 PT,.acceptedwritten; the core booted at 12:38, archived them at 12:42, re-queued at 12:48; the worker's answers went out 12:49–12:51 and the core's second answers followed. The owner got two replies per ask, with conflicting advice on one.Fix
classify.pygains adeliveredverdict:deliveries/*/<id>.acceptedor.claimedpresent (grammar ispool_delivery's, matched by path so the skill imports with no pool skill installed) → left intasks/, never archived, counted in the summary, not listed in the DM. Checked after the completion marker (a worker that finished still readsdone) and before the import branch.SKILL.mdstep 2 / 4 / 5 say the same.Evidence
The other three new tests pass at both ends by design: the no-marker control (still
orphan), another task's marker (stillorphan), and a result file present (stilldone).bash scripts/review-checks.sh --diff <cumulative>: PASS (hardcoded-paths + root-artifacts + prose-cap clean).Worst case for existing installs
A host with no
deliveries/dir is unchanged (worker_deliveryreturns '' before any listdir). A stale.acceptedleft by a dead worker keeps its task intasks/instead of archiving it — the watcher's sweep re-emits it, which is the pre-existing behaviour for anything the pass leaves; the reaper (reclaim_claimed) owns that case, not this pass.Signed: Sutando core (qingyun-001).
🤖 Generated with Claude Code