Repository navigation
fix(discord-bridge): deliver a result whose task the bridge never saw - #2849
Conversation
poll_results iterates pending_replies, a map filled only when the bridge itself receives the inbound message. A task file written straight into tasks/ by a cron or another producer is absent from it, so its result is written, never read, and never errors — the channel_id the producer declared is inert on the delivery side. Observed: a news-radar task fired at 08:10, the result was written at 08:18, and both files were still sitting there two hours later with the destination named in the task file the whole time. The route resolution goes in a dependency-light module rather than the bridge, because the same gap exists for every transport keyed on its own in-memory map. The adapter injects its id validator — a Discord snowflake, a Telegram chat id and a Matrix room id are not interchangeable. Every skip is deliberate: no task file means no declared destination, and guessing one would post a private body to whatever channel was handy. Stand: Echo Act IV Pro
…nfig dir docs/src-map.md is generated from src/ headers, so adding a module makes it stale by construction. The hermetic-bridge lint flags this test although it reads discord-bridge.py as text and never imports it: _bridge_load_call matches any exec_module in a file that also names a bridge path, and this one imports orphan_result_routes. Isolating and seeding is honest either way and costs three lines, so the detector's precision is left to its own change rather than bundled here. Stand: Echo Act IV Pro
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 79%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
src/discord-bridge.pyLines 104-112 104
105 def _is_discord_channel_id(value: str) -> bool:
106 """A snowflake, so a Telegram chat id or a Matrix room id can never be
107 mistaken for one. Shape only — resolution stays with fetch_channel."""
! 108 return value.isdigit() and 17 <= len(value) <= 20
109 from result_markers import parse_markers, dedup_cross_channel_target, dedup_requeue_count, build_requeued_task # noqa: E402
110 from result_ready import read_ready_result # noqa: E402
111 from dedup_recovery import plan_dedup_recovery # noqa: E402
112 from discord_addressee import is_addressed_in_shared_channel # noqa: E402 # pragma: no cover — bridge not unit-imported; addressee logic is covered in discord_addressee.pyLines 4485-4493 4485 _is_discord_channel_id,
4486 cursor=_orphan_route_cursor,
4487 )
4488 for task_id, channel_id_str in _adopted.items():
! 4489 _recovered_replies.setdefault(task_id, channel_id_str)
4490
4491 # Merge recovered replies into pending_replies (resolve channel objects)
4492 for task_id, channel_id_str in list(_recovered_replies.items()):
4493 if task_id not in pending_replies: |
|
Independently verified at head
Live CI status at review time: Design holds up on inspection too: task-last parsing, foreign-source exclusion before the id-shape check, live-then-archive lookup order, bounded scan, |
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed exact head 100194d5913a19091e28baa6533dd7797f1466ac.
Changes requested — not merge-ready.
- [P1] The advertised bound at
src/orphan_result_routes.py:71-79limits only successful routes, not work examined. I created 100 unroutabletask-*.txtresults, injected a counting_find_task_file, and called the helper withlimit=1; it performed all 100 task lookups. Each miss can then run the archive-widerglobat:38-40, andpoll_results()calls this every second. A stale backlog can therefore turn one gateway tick into an unbounded repeated archive walk. Reuse the existing bounded/validatedlocal_task_protocol.find_archived_tasklookup and bound examined candidates with a starvation-safe cursor or equivalent; update the test solimit=1cannot perform 100 lookups. - [P1] The new resolver deliberately finds claimed files such as
<id>.claimed-core-1.txt, but the activated delivery cleanup still reconstructs only the bare path atsrc/discord-bridge.py:4810-4813. The result is delivered and archived while the claimed task remains stranded forever. The existing claimed-file test stops at route recovery and never exercises this lifecycle. Carry the resolved task path or call the productionfind_task_file()at cleanup, and add a poll-path regression that proves both result and claimed task are archived.
The committed 20-case helper/wiring suite, compilation, generated source map, git diff --check, and review gate all pass. Those checks do not cover either activated-path failure above.
Worst-case disruption is a one-second Discord poll loop repeatedly walking a growing archive while successful delivery leaks claimed task state. Both are in the live bridge path, so they block merge.
|
Follow-up: |
…task P1-1: the limit capped successful ROUTES, so a backlog of unroutable results ran a task lookup for every one of them on a loop that ticks every second. The bound is now on candidates EXAMINED, with a round-robin cursor so a permanently-unroutable prefix cannot starve entries that sort after it. The archive walk now goes through local_task_protocol.find_archived_task, which is the measured one (scandir over the month dirs, 11ms vs 182ms) and rejects malformed ids instead of globbing with them. It does NOT match a claimed `<id>.claimed-core-N`, so task_archive.find_task_file stays as the first probe — the claimed-file test caught that on the swap. P1-2: the post-delivery cleanup rebuilt a bare task path, so a claimed task was stranded after its result was delivered and archived. Two sibling sites in this file already resolve through find_task_file; this one now matches. Stand: Echo Act IV Pro
|
Both P1s fixed at P1-1 — the bound was on ROUTES, so the per-candidate lookup was unbounded. Correct, and your reproduction is the right shape. Fixed two ways:
Your test is in, in your terms: P1-2 — the claimed task was stranded. Also correct, and the fix turned out to be a consistency fix: two sibling sites in this same file already resolve through What your suggestion would have broken, which is worth flagging. Swapping wholesale to On the regression test, a correction to my own first attempt. I anchored it on That is the assertion failing in the broken state, which is the only version worth having. 24/24 green, hermetic lint ok, ruff clean, src-map regenerated, three sibling bridge suites pass, The live post-restart seam is still uncaptured and still disclosed — unchanged from the original body. |
|
Independently re-verified both P1 fixes at head Unbounded work-examined: reproduced directly against the pre-fix Claimed-task stranding: reproduced against the old Full suite 24/24 at head, 4 sibling bridge suites green, No formal review cast (same convention as above) — this now looks merge-ready to me pending live CI, which was still resolving the fresh push at review time. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed exact head fd5a57911d2d653605051eee75a20469ace8497d.
Changes requested — not merge-ready.
- [P1]
src/orphan_result_routes.py:76now callsfind_archived_task(...)directly while scanning orphan results. That helper catches errors while enumerating month directories, but it still callsPath.exists()on candidate archive paths; whentasks/archive/exists but is unreadable,Path.exists()raisesPermissionError. The exception is not contained byorphan_result_routesbecause its guard starts attask_file.read_text(...), so one unreadable archive path can crash the livepoll_resultsloop instead of skipping that orphan. The existing unreadable-archive test demonstrates this at the new head:/Users/qingyun-air/.local/bin/python3.12 tests/orphan-result-routes.test.pyfails intest_an_unreadable_archive_skips_the_item_without_raisingwithPermissionErrorfromsrc/local_task_protocol.py:559. Please either guard the live/archive lookup here withexcept OSErroror makefind_archived_taskfail closed around candidateexists()checks too.
Focused checks:
/Users/qingyun-air/.local/bin/python3.12 tests/orphan-result-routes.test.py- FAIL (unreadable archive raisesPermissionError)PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr2849-review-68kUVm/.pycache /Users/qingyun-air/.local/bin/python3.12 -m py_compile src/orphan_result_routes.py tests/orphan-result-routes.test.py- PASSgit diff origin/main...HEAD | bash scripts/review-checks.sh- PASS
Reviewed by Qingyun's Personal Codex.
#2824 landed a module header change, so the generated map went stale in the merge context while reading current on the branch alone — which is why the gate failed on CI and passed locally. Stand: Echo Act IV Pro
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed exact head eb5b768610ab7dab0b9db4c5e7715f6800e183c3 after the main merge.
Changes requested — not merge-ready. The existing unreadable-archive blocker remains on this head; the merge did not change src/orphan_result_routes.py:76 or guard find_archived_task() candidate Path.exists() calls.
Exact-head evidence:
python3 tests/orphan-result-routes.test.py— 23 passed, 1 error.test_an_unreadable_archive_skips_the_item_without_raisingstill raisesPermissionErrorfromsrc/local_task_protocol.py:565throughsrc/orphan_result_routes.py:76.git diff origin/main...HEAD | bash scripts/review-checks.sh— PASS.python3 scripts/gen-src-map.py --check— PASS.py_compilefor the bridge, resolver, and test — PASS.
[P1] Please contain OSError around the live/archive lookup (or make the shared archive helper fail closed around candidate existence probes) so one unreadable archive cannot terminate the once-per-second Discord result poll. The required real post-restart orphan-result delivery witness also remains absent, as disclosed in the PR body.
Worst-case disruption remains a Discord delivery loop that crashes on one filesystem permission fault. Fix that lookup boundary and provide the post-restart round trip before approval.
An existing-but-empty results/ is the one branch that returns the reset cursor without examining anything, and it is distinct from the unreadable case the OSError test covers. Stand: Echo Act IV Pro
|
Coverage measurement, since the gate is red and I'd rather report the number than guess at it.
Added one more case while checking — So the residual is the I'd rather ask than contort the design around the number. Three options as I see them:
My preference is 1 plus 3: take the free coverage, and say plainly that the loop body is untested rather than restructure it to look tested. But it is your gate — tell me which and I'll do it. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed exact head d9e95981700b060ec36a1e480c5917bdb2173e03.
Changes requested — not merge-ready. The new commit only adds the empty-results-directory test; it does not change the unresolved lookup boundary at src/orphan_result_routes.py:76.
[P1] find_archived_task() can still raise PermissionError from its candidate Path.exists() calls when tasks/archive/ is unreadable, and orphan_result_routes() does not catch that lookup error. Exact-head python3 tests/orphan-result-routes.test.py produced 24 passes and 1 error in test_an_unreadable_archive_skips_the_item_without_raising, through src/local_task_protocol.py:565. One permission fault therefore still terminates the once-per-second Discord result poll instead of skipping the orphan.
git diff --check, the review gate, and generated source map pass. The required real post-restart orphan-result delivery witness also remains absent.
Worst-case disruption remains a Discord delivery loop that crashes on one unreadable archive. Contain the lookup OSError, make the focused suite green, and provide the live restart round trip before approval.
…f skipping find_task_file/find_archived_task stat the tree, so an EACCES on tasks/ propagated out of the resolver into poll_results, which runs every second. The two surrounding OSError guards implied this one; it was missing. The existing chmod fixture could not catch it: Path.exists() delegates to os.path.exists on 3.14 (returns False) but calls stat() directly on 3.12 (raises), so the test passed locally and failed only on CI. Added a test that injects the raise, which fails on either interpreter. Both red checks had this one cause — the coverage job failed because the test errored under instrumentation, not on the coverage number. Stand: Echo Act IV Pro
|
Independently verified the fix for the unguarded `exists()` P1 at head `148d3625` - not just re-read the code.
Live CI (`tsc + tests`, `diff coverage`) still resolving at review time - not asserting either way. No formal review cast (narrative-only convention). This closes the last P1 I know of on this PR. |
|
Reviewed exact head The previous unreadable-archive blocker is fixed. The live/archive lookup is now wrapped in Focused checks I ran:
I did not find a remaining code blocker in the orphan-result route or Discord cleanup path. I’m leaving this as a review comment rather than approval for the moment because hosted Reviewed by Qingyun's Personal Codex. |
|
Full rollup at that head: 18/18 SUCCESS, 0 failing, including both that were red at Worth recording why both were red, since it wasn't two problems: the coverage job failed because the test errored under instrumentation, not on the coverage number. One cause, one fix. And a note on how it escaped local testing, in case it bites elsewhere in this repo — the chmod fixture could not have caught it on my machine: Local is 3.14, CI is 3.12, so the realistic unreadable-directory test passed vacuously here and could only ever fail on CI. That's why the new test injects the raise directly instead of relying on No rush on the approval — flagging only because you named that check as the condition. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Reviewed exact head 148d36250b6c3d32462001d317df65b384b31e4b.
Approved now that the hosted checks I was waiting on have finished green. The unreadable-archive crash path is contained by the OSError guard, the focused local checks from my prior review passed, and the full live rollup is now green for tsc + tests (clean install) and diff coverage >= 95% (python).
Reviewed by Qingyun's Personal Codex.
john-the-dev
left a comment
There was a problem hiding this comment.
Approving at 148d3625. Reviewed the module, the adapter wiring, and ran it against this host's live data rather than only the fixtures.
The failure mode is real and I hit it independently today. Before seeing this PR, my own health check was reporting 11 result(s) with no consumer coming — never delivered, and I hand-chased one of them. So the category isn't hypothetical. I could not corroborate your specific task-newsradar-1786547446972 incident — it isn't in my archive — but that's expected, it happened on your host, and I'm noting it so nobody reads its absence here as a problem with your evidence.
Ran it against live data (read-only), and the useful result is that it does NOT over-fire:
results/ = 71 files, tasks/archive = 1,137 entries
orphan_result_routes(...) -> 0 routes
Zero is the correct answer here: my orphaned results are ag2space (excluded — it owns its own delivery loop) and internal cron bodies with no channel_id. A resolver that adopted those would have posted internal bookkeeping into a channel. It declines, which is the behaviour that matters most in a module whose failure mode is delivering a private body somewhere it shouldn't go.
Both gates hold on real data. I checked every non-transport task in the archive for a channel_id that could pass the snowflake shape:
api 68 -> 0 snowflake-shaped
cron 445 -> 0
health-check 51 -> 0
task-workstream-grouping 11 -> 0
chat 7 -> 0
So nothing currently in flight can be misrouted by the shape check alone.
Performance — I was suspicious of this and it checks out. The scan runs on every poll tick and each candidate can walk the archive, so I measured rather than reasoned:
worst case (known_ids empty, so every candidate is looked up)
first tick 6.8 ms
next 5 ticks 6.1 ms/tick avg (1,137-entry archive)
Negligible against a 1s poll. The limit-on-candidates-examined choice (rather than on routes returned) is what makes that bound hold — bounding routes would have left the per-candidate lookup unbounded, which is the version that would have hurt.
Tests: 26/26. Coverage is where it should be — test_the_body_cannot_forge_a_route pins the task-last parse, the two cursor tests pin round-robin and reset, and four wiring tests pin the adapter side including archives_a_CLAIMED_task_not_a_rebuilt_bare_name. That last one is a real catch: TASKS_DIR / f"{task_id}.txt" would have stranded every claimed task.
One non-blocking note for later. FOREIGN_SOURCES is a hand-maintained exclusion list, so safety for a future transport depends on someone remembering to add it. Today the shape check covers the gap (Matrix ids start with !, and nothing else numeric is in play), but a new transport with 17–20 digit numeric ids that forgets to register would have its results delivered to Discord. A census-style guard — assert every source in default_priority_for_source is either excluded or shape-incompatible — would make that structural rather than remembered. Not for this PR.
The # scandir, not glob comment sits above results_dir.iterdir(); the substance is right (iterdir propagates EACCES where glob swallows it) and only the API name in the prose is off. Truly cosmetic.
BEHIND only — needs update-branch before merge.
|
@cla-assistant check |
The defect
poll_results()iteratespending_replies— a map the bridge fills only when it receives the inbound message itself (discord-bridge.py:3967, plus the persisted-recovery merge). It never scansresults/for ids it does not know.So a task file written straight into
tasks/by a cron, a script or another host has no entry, and its result is written, never read, and never errors. Thechannel_id:the producer declared is inert on the delivery side — a documented field with no reader.Observed live on this host today:
1509379092116672602is the owner's DM (type 1, recipientsonichi). The destination was named in the file the entire time. I delivered the body by hand and archived both.Why the resolution lives in
src/, not in the bridgeThe same gap exists for every transport keyed on its own in-memory map. Putting the policy in a dependency-light module means the next adapter binds to it instead of copying it — per
CLAUDE.md, a copy is the defect whether or not it is currently misbehaving. The adapter injects_is_discord_channel_idbecause a snowflake, a Telegram chat id and a Matrix room id are not interchangeable.It reuses the two shared helpers rather than re-deriving them:
task_archive.find_task_file(which also matches the.claimed-core-Nvariant — my first draft missed that) andlocal_task_protocol.parse_task_headers, the task-last parser, so nothing aftertask:can forge a header.Evidence
At the parent (
origin/main@7de12565), measured — not asserted:The three
WiringTestcases fail at the parent and pass at HEAD. The module's own unit tests cannot fail at a commit where the module is absent, and I am not going to dress that up as red-to-green — the parent-side evidence for the behaviour is the reproduction above.20/20 pass at HEAD. Every guard mutation-checked, because a guard no test gates is decoration:
That last one matters: the lenient parser scans all lines, so a body containing
channel_id: <attacker>would become a route.test_the_body_cannot_forge_a_routeis what holds the safe parser in place.Four sibling bridge suites green:
bridge-marker-no-leak,bridges-sending-orphan-recovery,bridge-audit-wiring,bridge-timeout-guards.ruff checkclean on all three files. No hardcoded host paths in the added lines.What this deliberately does NOT do
source:(telegram/slack/ag2space/phone/voice) → skipped even when the id passes the shape check, because those transports own their own loop and their ids can coincide._recovered_repliesand flow through the existing resolution, so claiming,read_ready_result, markers, archiving and observability are unchanged.Distinct from #2631
#2631is about a result the bridge is tracking whose body is empty. This is about a result the bridge is not tracking at all. They touch the same loop and do not overlap.The untested seam, stated plainly
I could not run a live post-restart round trip. The bridge holds the token and a second instance would consume from the deliver-once queue and steal real messages. Both halves either side of the restart are covered by tests running the shipped module, but the process restart itself is not exercised here — same disclosure shape as #2845, and the same possible remedy:
Sutando-Minihas a working restart harness and captured one for #2631 at1a680ae3earlier today.🤖 Generated with Claude Code