Skip to content

feat(gateway): drain results/proactive-*.txt to a configured room (REMOTE_PROACTIVE_ROOM) - #2324

Merged
qingyun-wu merged 20 commits into
mainfrom
feat/gateway-proactive-drain
Aug 11, 2026
Merged

qingyun-wu merged 20 commits into
mainfrom
feat/gateway-proactive-drain

Conversation

@qingyun-wu

Copy link
Copy Markdown
Collaborator

Problem

The repo contract says "write results/proactive-{ts}.txt to speak to the user" — but only the Discord/Telegram bridges drain that path. On a gateway-only host, proactive nudges are dead letters: observed live, a pending-questions DM nudge (proactive-pending-q-61c1806700837ea9.txt) sat undrained indefinitely while only its macOS notification fired.

Fix

New optional env REMOTE_PROACTIVE_ROOM (documented in the protocol env table). When set, the poll loop delivers each proactive-*.txt as a gateway message (POST /v1/room op:message) using the Discord bridge's proven idiom: claim-by-rename (.txt → .sending) so a racing legacy bridge on a multi-channel host can't double-deliver; success archives beside task results; failure renames back for retry next pass; startup recovers orphaned claims. Auth errors propagate to the loop's existing auth handling. Unset → no scan, zero change. The target room is deliberately explicit, never auto-learned from task channel_ids — a nudge may be owner-private and must not land in whatever shared room was recently active.

Wiring note: startup.sh set -a-sources the relay env file, so deployments opt in by adding one REMOTE_PROACTIVE_ROOM=!room:hs line there — no code change.

Evidence

Harness (before, parent 0e91315): grep -n proactive remote_gateway_bridge.py → comment-only mentions; no drain exists.
Harness (after, HEAD): python3 src/remote-gateway-bridge.test.py → PASS — all checks green incl. 7 new checks: no-op when unset · delivered as {"op":"message","room_id":…,"body":…} + archived · 502 → file restored → retry delivers · empty file dropped without a send · orphan .sending recovered and delivered.

Live (real gateway, this host, patched in-place + restarted):

[remote-gateway-bridge] delivered proactive proactive-pending-q-61c1806700837ea9.txt
[remote-gateway-bridge] delivered proactive proactive-drain-demo-1785070724.txt

Both messages arrived in the target room (the coord room, chosen so the test wouldn't ping the owner's DM) — including the real stranded nudge from before the change, which is the exact failure this PR fixes. results/ is clean afterward; system restored (original module back, keepalive-owned bridge reconnected connected: true). Bonus live observation: while the patched instance ran, the keepalive's respawns cleanly deferred via the singleton guard (another live gateway-bridge owns this workspace … exiting to avoid dual-poll) — no dual-poll at any point.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DjCFofrj7FTrFnhjMidJ2Y

…MOTE_PROACTIVE_ROOM)

The repo contract says 'write results/proactive-{ts}.txt to speak to
the user', but only the Discord/Telegram bridges ever drained that
path — on a gateway-only host proactive nudges were dead letters
(observed live: a pending-questions DM nudge sat undrained while only
its macOS notification fired).

When REMOTE_PROACTIVE_ROOM names a room id, the poll loop delivers each
proactive file as a gateway message (POST /v1/room op:message) using
the Discord bridge's proven claim-by-rename idiom: .txt → .sending
prevents double-delivery against a racing legacy bridge, success
archives beside task results, failure renames back for retry, startup
recovers orphaned claims. Auth errors propagate to the loop's auth
handling. Unset env → no scan, zero change for existing hosts. The
room is deliberately explicit, never auto-learned from task
channel_ids — a nudge may be owner-private.

Suite: 7 new checks (no-op unset, deliver+archive, fail→restore→retry,
empty-drop, orphan recovery+delivery); all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DjCFofrj7FTrFnhjMidJ2Y
@github-actions

github-actions Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 78%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • src/remote-gateway-bridge.test.py (100%)

Summary

  • Total: 122 lines
  • Missing: 0 lines
  • Coverage: 100%

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking finding:

[P1] The new proactive drain archives the file after any HTTP 200 from /v1/room, but the current gateway room-message contract can return 200 even when the Matrix send did not happen. On the backend side, relay.py turns a room-op result with no error into 200, while matrix_ingest.py handles op == "message" by await self._send(...) and then return {}; _send logs exceptions and does not propagate Matrix room_send failures/ErrorResponses as an error result. That means a bad REMOTE_PROACTIVE_ROOM, a kicked agent, or a membership/power-level denial can be swallowed server-side, this client reaches packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:1131-1135, and the proactive file is archived/unlinked even though no room message was delivered. This violates the durability promise and the room-ops boundary requirement that membership enforcement happens gateway/broker-side in a way the local client can rely on. Please make /v1/room message fail closed/return a positive delivery signal (for example an event_id) and only archive on that verified success, or add an equivalent preflight/ack path that preserves the file for retry when the agent is not actually allowed to send to the configured room.

Focused checks run in an isolated /private/tmp checkout:

  • git diff --check origin/main...HEAD
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr-2324.pycache python3 -m py_compile packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py src/remote-gateway-bridge.py src/remote-gateway-bridge.test.py
  • python3 src/remote-gateway-bridge.test.py (rerun outside the sandbox for localhost bind) — PASS, including the new proactive no-op/deliver/retry/empty/orphan cases

Credential-boundary note: the local change keeps Sutando-local limited to the scoped gateway bearer plus an explicit room id; I did not see AppService namespace credentials introduced locally. The blocker is that the broker-side membership/send result is not currently surfaced strongly enough for the local drain to make a durable archive decision.
Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Reviewed — LGTM; the two failure modes this path is historically prone to are both correctly guarded.

  1. No double-delivery. The drain is strictly opt-in: if not PROACTIVE_ROOM: return, and REMOTE_PROACTIVE_ROOM is never auto-learned. So a host that also runs the Discord/Telegram bridges (which don't set it) never fires the gateway drain — avoiding the 9x-duplicate-delivery class the repo hit on 2026-04-20 (both bridges polling proactive-*.txt). Gateway-only hosts opt in; mixed hosts stay single-path. Exactly right.
  2. No re-drain. Claim-by-rename to proactive-*.sending → POST → archive on success / rename-back on failure, with a .sending recovery sweep for crash-mid-delivery. Mirrors the fix(discord-bridge): delivery-idempotency sentinel for poll_results (restart-safety) #1048 Discord .sending idempotency contract. Empty-body nudges are unlinked (nothing to say); singleton guard prevents dual-poll.
  3. Privacy: the "never auto-learn the room from task channel_ids, a nudge may be owner-private" note is the right call — a proactive nudge landing in the wrong (e.g. shared) room would be a real leak.

Closes the gateway-only-host dead-letter gap cleanly. CI green (16). No blockers.

…gnal (event_id)

Review P1: the gateway can answer 200 while the room send was swallowed
server-side (bad room id, kicked agent, power-level denial) — archiving
on any 200 breaks the drain's durability promise. The drain now archives
ONLY when the response carries the posted message's event_id (the
deployed broker returns {ok, event_id}); a bare/empty 200 restores the
claim for retry with a loud log naming REMOTE_PROACTIVE_ROOM and room
membership as the things to check. Mock gateway updated to return the
delivery signal (+ a togglable swallowed-send 200 mode); regressions:
empty-200 restores (no false archive), retry with a real signal
archives. The broker-side fail-closed fix is complementary and lives
server-side.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DjCFofrj7FTrFnhjMidJ2Y
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed at the new head — the drain now archives only on a positive delivery signal: the response must carry the posted message's event_id (the deployed broker returns {"ok": true, "event_id": "$…"} — verified in this PR's own live evidence). A 200 with no event_id (the swallowed-send shape you traced) restores the claim for retry and logs loudly, naming REMOTE_PROACTIVE_ROOM + room membership as the things to check, so a misconfigured room is visible instead of silently eating nudges.

Mock gateway updated to return the signal plus a togglable swallowed-send 200 mode; regressions: empty-200 → file restored (no false archive) → retry with a real signal archives. Full suite green at head.

Agreed the broker-side fail-closed change (propagate room_send failures instead of return {}) is the complementary server half — that's outside this client repo; flagging it to the broker owner separately.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval signal for e9cbe1cb4c4890385d350218d88ac8b0127ed61a.

Re-reviewed the new head and found no blocking issues. The proactive drain now archives a claimed proactive-*.sending file only after /v1/room returns a positive delivery signal via event_id; HTTP failures, network errors, auth handoff, and a bare 200 without event_id all restore the claim to .txt for retry instead of losing the nudge. The startup recovery for orphan .sending claims and the opt-in REMOTE_PROACTIVE_ROOM gate still keep this path durable and single-channel.

I also checked the current backend main contract in an isolated /private/tmp clone: op: message returns the first posted Matrix event_id on success, and /v1/room surfaces that field to callers. If a Matrix send fails and the backend returns a 200 without event_id, this client now treats that as non-delivery and retries, which is the critical durability boundary.

Sutando-local still uses only the scoped gateway bearer, not AppService namespace credentials; room membership and power levels remain enforced by the gateway/broker-side Matrix session; no onboarding compatibility was broken.

Focused local checks passed: git diff --check, py-compile for the touched bridge files, src/remote-gateway-bridge.test.py, packages/ag2-sparrow/tests/test_gateway_status.py, packages/ag2-sparrow/tests/test_event_wiring.py, packages/ag2-sparrow/tests/test_event_inbox_channel.py, and packages/ag2-sparrow/tests/test_human_action.py. GitHub CI is green at this head.

Reviewed by Qingyun's Personal Codex.

qingyun-wu added a commit that referenced this pull request Jul 26, 2026
…ified, approval consumed once

capability.execute now executes: message.send posts through the gateway
(daemon-resolved REMOTE_TASK_URL/REMOTE_TASK_TOKEN — the CLI can never
supply credentials) and only a response carrying the posted message's
event_id counts as delivered (the #2324/#207 fail-closed contract); a
swallowed-send 200 fails the request. --approval <requestId> gates an
execution on an APPROVED, unconsumed approval and consumes it
atomically — one approval authorizes exactly one execution. Unknown
actions fail cleanly.

E2E grows to 16 checks against a mock gateway speaking the broker
contract: approved→execute delivers with event_id; second execute on
the same approval refuses (already consumed); 200-without-event_id
fails closed; unknown action fails.

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two delivery-safety blockers on current head e9cbe1c. First, src/remote-gateway-bridge.py:_post_proactive claims every results/proactive-*.txt and sends the raw body to REMOTE_PROACTIVE_ROOM without applying the existing result-body routing protocol. A file beginning [channel: C...] can therefore be won by this bridge and leaked to the gateway room with the marker included. Parse and honor explicit routing, skipping incompatible destinations, or use a consumer-scoped namespace. Second, _recover_orphan_proactive immediately renames every .sending claim back without an age or ownership check, so one live consumer can steal another live bridge worker’s in-flight claim. Use a consumer-specific claim suffix or a stale/ownership guard. Add regressions for both cross-consumer cases. CI and the posted gateway round trip are green, but they do not cover these races.

… cannot steal live claims

Blocker 1: _post_proactive now PEEKS each file before claiming and
honors the result-body routing protocol — a [channel: <discord/slack
id>] nudge is left unclaimed for its own bridge (never leaked raw to the
gateway room), while [channel: !room:server] redirects delivery to that
room with the marker stripped.

Blocker 2: claims are pid-scoped (.sending.<pid>). Recovery leaves any
live owner's claim alone (including this process's other threads),
recovers a dead owner's immediately, and applies a 10-minute age guard
to legacy bare .sending claims that carry no owner info.

Regressions: foreign-channel skip (unclaimed + un-posted), matrix-room
redirect, live-claim-not-stolen, dead-pid recovery, fresh-legacy left
alone, aged-legacy recovered.
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Both blockers addressed in 512be09.

Routing protocol: _post_proactive peeks before claiming. A [channel: <id>] first line routes: Discord/Slack-shaped ids → the file is skipped entirely (unclaimed, un-posted — its own bridge owns it); a Matrix !room:server id → delivered to that room with the marker stripped. Raw-marker leakage to the gateway room is no longer possible.

Claim ownership: claims are now pid-scoped (.sending.<pid>). _recover_orphan_proactive never steals a claim whose owner pid is alive (including this process's own workers), recovers dead-owner claims immediately, and holds a 10-min age guard for legacy bare .sending claims with no owner info.

Regressions added (6): foreign-channel skip, matrix-room redirect w/ marker stripped, live-claim-not-stolen, dead-pid recovery, fresh-legacy untouched, aged-legacy recovered.

Verification: python3 src/remote-gateway-bridge.test.py → PASS — all checks green at 512be09.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval signal for current head 512be0902985ea28fb4500db3ac9b07c37cffcd0.

I do not see blocking issues in the new proactive drain head. The prior delivery-safety blockers are addressed: _post_proactive() peeks at the [channel: ...] marker before claiming, leaves non-Matrix destinations unclaimed for their own bridge, strips Matrix room routing markers before posting, and archives only after /v1/room returns a positive event_id. Claim files are pid-scoped as .sending.<pid>, recovery leaves live owners alone, dead-owner claims recover immediately, and legacy bare .sending claims use an age guard.

Credential and membership boundaries still look right for the gateway model: Sutando-local uses the scoped gateway bearer only, no AppService namespace credential is introduced locally, and room membership or send authorization remains enforced by the gateway/broker Matrix session. Existing onboarding compatibility is preserved because REMOTE_PROACTIVE_ROOM is opt-in; unset hosts do not scan proactive files.

Focused verification:

  • git diff --check origin/main...HEAD
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pycache-2324-512 python3 -m py_compile packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py src/remote-gateway-bridge.py src/remote-gateway-bridge.test.py
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pycache-2324-512 python3 src/remote-gateway-bridge.test.py passed outside the sandbox because the test binds a local loopback server.

GitHub CI is green on all visible required checks for this head.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

@john-the-dev — both blockers addressed, requesting re-review.

Blocker 1 (raw body to the wrong destination): the drain now PEEKS before claiming and honors the result-body routing protocol — a file routed to a non-matching destination is left unclaimed for its own consumer (never posted, marker included), and a matching destination is delivered with the marker stripped.
Blocker 2 (claim theft): claims are pid-scoped, so recovery never reclaims a live worker's in-flight claim; a dead owner's claim recovers immediately, and legacy ownerless claims recover only past an age guard.
Addressed by 512be09. Regressions added for all six cases (foreign-destination skip, in-scope redirect, live-claim-not-stolen, dead-owner recovery, fresh-legacy untouched, aged-legacy recovered).
Verification at head: python3 src/remote-gateway-bridge.test.py → PASS, all checks green. CI green.

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head. The previously-raised blockers are genuinely addressed — archive now gates on event_id rather than a bare 200 (and the test proves it via force_room_empty_200), claims are pid-scoped so recovery can't steal a live worker's in-flight file, and [channel: <id>] foreign destinations are peeked-and-skipped so a Slack/Discord nudge isn't leaked here. Good work; the design notes in the comments are unusually clear.

Independent corroboration of the problem statement, from a different host and day. I can confirm this is not a one-off: on a gateway-only host today check-pending-questions.py logged WARNING: no consumer is draining results/proactive-*.txt on this host, and I manually archived 7 stranded nudges (proactive-pending-q-0f131d036a521c13.txt, -6540978d555f2936, -e2b1cce7aeffff0f, -f7546c10ec6bd435, -fa6a4e50f3ee9ea3, plus a proactive-morning-*) — different hashes from the one in your description, same failure. Two further nudges were written and stranded during the same session. The dead-letter path reproduces reliably.


Blocker

_post_proactive silently destroys a nudge on the empty-body path, and there is a read-before-claim race that can produce one.

route, room_override, routed_body = _proactive_route(f.read_text(...))   # PEEK
...
claim = f.with_suffix(f".sending.{os.getpid()}")
f.rename(claim)                                                          # CLAIM
body = routed_body.strip()
if not body:
    claim.unlink(missing_ok=True)  # empty nudge — nothing to say

The peek happens before the claim, so the sequence below loses data:

  1. writer creates proactive-x.txt, has not yet flushed the body
  2. drain peeks → routed_body == ""
  3. drain claims (rename) and unlinks
  4. writer's flush lands on an unlinked inode → the nudge is gone, with no log line

Every other failure path in this function logs; this is the only one that removes an owner-facing message silently. That is the exact harm class the PR exists to eliminate, so it is worth closing even though the window is small (writers here are single short write_text calls).

One-line fixes, any of these: leave empty files unclaimed (the next pass picks them up once written); or require time.time() - stat().st_mtime > 2; or at minimum _log(f"dropped empty proactive {f.name}") so it is visible rather than silent.

Significant (not strictly blocking, but it will bite in production)

No retry ceiling or dead-letter — a permanently-undeliverable nudge retries forever. Every failure branch renames the claim back to .txt unconditionally, and the comment says this is intentional ("loudly, so a misconfigured room is visible"). But there is no attempt counter, no backoff, and no poison-pill path, so a nudge that can never be delivered — oversized body past the Matrix event limit, a room the agent was kicked from, a typo'd REMOTE_PROACTIVE_ROOM — is retried and logged on every loop pass indefinitely.

Related and untested: the body is POSTed whole, with no size cap ([:120] truncates only the error string). A runaway nudge is precisely the input that produces the permanent-failure loop above.

Suggest either an attempt counter encoded in the claim name with a move to results/archive/undeliverable/ after N tries, or a size guard that dead-letters oversized bodies up front.

Nits

  • The two Approval signal for current head … self-comments read like approvals in the timeline. They are COMMENTED so they carry no formal weight, but on a PR that still shows CHANGES_REQUESTED they make the state harder to read. Worth asking @john-the-dev to dismiss or re-affirm his CHANGES_REQUESTED on head e9cbe1cb explicitly, since the head has moved since — per CONTRIBUTING, a stale REQUEST_CHANGES should be dismissed or the exact unresolved behavior cited.
  • _ORPHAN_MIN_AGE_S = 600 only guards legacy bare .sending claims. Worth a one-line comment that new claims deliberately need no age guard because pid-liveness supersedes it — it took a second read to confirm that was intentional rather than an omission.

What I did not verify

I did not run the test suite or the live gateway path myself; I read the diff and the mock-server changes. Your harness evidence covers the seven new checks and I have no reason to doubt it. My corroboration above is of the problem, not of your fix.

qingyun-wu added a commit that referenced this pull request Jul 26, 2026
… + CLI + durable store (v0) (#2325)

* feat(runtime-api): local approval/elicitation/capability RPC — daemon + CLI + durable store (v0, E2E-tested)

The request-response upstream between the long-running agent and its
human owner, per the sutando-socket design (workspace notes): a local
daemon on a private Unix socket speaking NDJSON JSON-RPC, a
sutando-runtime CLI so the agent never touches sockets or remote APIs,
and a SQLite request store so every request is durable, resolves
exactly once (CAS transitions, terminal states immutable), and survives
daemon restarts.

v0 scope decision: approval/elicitation ride the EXISTING human-action
card lifecycle (ha_adapter mirrors a request into an ActionStore
pending action → the gateway CardPoster posts the buttons card →
DecisionHandler routes the owner's answer back → the resolver loop maps
it onto the request's terminal state). No new server API or UI — the
formal /v1/agent-requests can come later without touching this
contract. capability.execute is a fake executor recording the full
lifecycle; slice 2 replaces it with the governed gateway send verified
by event_id.

Identity is daemon-resolved from env (never CLI-self-reported); socket
dir 0700 / socket 0600; live-daemon probe before stale-socket takeover;
256KB frame cap; issuing methods never block on a human (request.wait
carries the bounded blocking).

E2E (tests/runtime-api-e2e.test.py, stdlib, real daemon + real CLI +
real ActionStore; only the owner's answer is simulated in
DecisionHandler's exact write shape): approve, deny, single_select
elicitation, capability, unknown-id error, wait-timeout, restart
recovery, cancel-beats-late-answer — 12/12 green.

* fix(runtime-api): sanctioned repo-root resolution + src-map regeneration

lint-workspace-resolution flagged the adapter's parent-chain repo walk:
sibling-import bootstrap now goes through named intermediates and repo
root comes from sutando_config._find_repo_root (marker walk), never a
bare parent chain. docs/src-map.md regenerated for the new modules.
E2E re-run green (12/12).

* feat(runtime-api): slice 2 — real message.send executor, event_id-verified, approval consumed once

capability.execute now executes: message.send posts through the gateway
(daemon-resolved REMOTE_TASK_URL/REMOTE_TASK_TOKEN — the CLI can never
supply credentials) and only a response carrying the posted message's
event_id counts as delivered (the #2324/#207 fail-closed contract); a
swallowed-send 200 fails the request. --approval <requestId> gates an
execution on an APPROVED, unconsumed approval and consumes it
atomically — one approval authorizes exactly one execution. Unknown
actions fail cleanly.

E2E grows to 16 checks against a mock gateway speaking the broker
contract: approved→execute delivers with event_id; second execute on
the same approval refuses (already consumed); 200-without-event_id
fails closed; unknown action fails.

* fix(runtime-api): ha action ids must be hex-only — live human acceptance caught it

The first real owner answer could not resolve: DecisionHandler's answer
grammar matches ha_[0-9a-f]{6,} (hex only) and the adapter's id carried
the requestId's type prefix ('ha_approval...'). The local E2E missed it
by writing resolutions directly through ActionStore — exactly the gap a
real-human test exposes. Ids now derive from the requestId's hex uuid
tail (shared ha_action_id helper, used by issue + recovery), and a
regression pins the id against the answer grammar itself.

* fix(runtime-api): close the three review P1s — governed gating, idempotency, elicitation dead paths

1. message.send (and any GOVERNED_ACTIONS entry) now REQUIRES an
   approved, unconsumed approval, validated and consumed before the
   executor can touch daemon-held credentials — an ungated socket
   client is refused pre-gateway. Regression pins that no gateway POST
   happens on the refused call.

2. idempotencyKey is persisted (unique-indexed column) before approval
   consumption and gateway contact; a retry with the same key returns
   the recorded request (idempotentReplay: true) — no duplicate send,
   no spurious already-consumed failure. Regression: two identical
   executes → one gateway POST, same requestId.

3. Elicitation dead paths: multi_select now sets the multiSelect flag
   so DecisionHandler's comma grammar resolves it (E2E covered);
   free_text is rejected loudly in v0 (the card transport is
   options-based — a stranded-until-expiry request is worse than a
   clear error); single/multi_select require options; default type is
   single_select.

E2E grows to 24 checks, all green.

* fix(runtime-api): four review P1s — migration, approval binding, key fingerprint, non-blocking executor

1. Rolling-update migration: a pre-idempotency DB (the live acceptance
   created one) gains the new columns via idempotent ALTER before the
   index script — no boot crash. Regression boots the old schema and
   reads its rows.
2. Approval-capability BINDING: an approval authorizes exactly the
   action+resource the owner saw on the card, compared canonically
   BEFORE consumption — an approved repo.force_push can no longer
   authorize a message.send, and a mismatch costs nothing. Regressions:
   cross-action and cross-resource both refused.
3. Idempotency-key fingerprint: the key persists a sha256 of
   action+resource+input; reuse with a different fingerprint is
   rejected instead of replaying an unrelated side effect as complete.
4. Blocking executors off the event loop (asyncio.to_thread): a slow
   gateway send no longer stalls other clients or the resolver —
   regression drives a 1.5s send concurrently with a request.get that
   must return <1s (measured 0.05s).

E2E now 27 checks, all green.

* test(runtime-api): in-process RequestStore unit suite — coverage-visible lifecycle pinning

The E2E drives the store through the daemon subprocess, invisible to
the coverage recorder — the diff-coverage gate flagged request_store at
63.8% the moment the migration regression made the file visible. This
suite pins the durable-lifecycle contract in-process: create/get
roundtrip, unknown-id, pending listing, CAS transition + terminal
immutability, one-time consumption (approved-only, unknown refused),
lazy durable expiry, result persistence, idempotency-key lookup with
fingerprint, and the pre-idempotency schema migration (readable old
rows, usable index, writable new columns, clean re-boot). 20 checks.

* fix(runtime-api): record creation + approval consumption are one atomic transaction

create_consuming() inserts the capability record (carrying its
idempotency key + fingerprint) and stamps the approval consumed in a
single SQLite transaction — a crash between the steps can no longer
spend an approval without leaving a replayable record, and a same-key/
different-approval race rolls back untouched when the unique index
rejects the insert (the loser replays the winner's row).

Also: CLI elicitation --type default free_text → single_select (the v0
server rejects free_text; the default path should not be an
intentionally unsupported request).

Regressions: store suite pins atomic commit/rollback semantics (no
orphan row on a lost consume race; duplicate key leaves the second
approval consumable); E2E adds same-key+different-approval replay with
the fresh approval provably still spendable.

* fix(runtime-api): shared cross-platform run-dir helper; _issue never strands a pending row

Blocker 1: the daemon and CLI each duplicated an invented macOS-only
run-dir fallback. rundir.py is now the ONE policy both import
(SUTANDO_RUN_DIR > darwin app-support run dir > XDG_RUNTIME_DIR/sutando >
~/.sutando/run), so the default socket resolves identically on both ends
of the connection, on every platform.

Blocker 2: _issue committed the durable pending row before the
human-action mirror; an open_approval/open_elicitation failure left an
unanswerable pending request while the API errored. The mirror call is
now guarded — on failure the row is durably marked failed (error
recorded) before the ProtocolError surfaces, and restart recovery
re-links nothing for it. In-process failure-injection regression
(tests/runtime-api-issue-failure.test.py, 5 checks) pins both the
failure and success paths.

* ci(runtime-api): rundir owner allowlisted; src-map regenerated; rundir branch coverage

Three CI gates from the rundir.py introduction: (1) the install-path lint
now allowlists src/runtime-api/rundir.py as the ONE owner of the runtime
run-dir fallback (the pattern the lint prescribes — daemon + CLI import
it); (2) docs/src-map.md regenerated for the new module; (3) in-process
unit suite pins every resolution branch (env override, darwin, XDG,
portable fallback, socket override + derived default) so the diff-
coverage gate sees the module the subprocess daemon hides.

* ci: allowlist the rundir policy test (pins the owner file's fallback literal)

tests/runtime-api-rundir.test.py asserts rundir.py's documented
~/.sutando/run fallback — the same pin-the-owner's-policy role
tests/workspace-default.test.py already plays in the allowlist. (The
previous local lint pass missed it: diff-mode scans committed files and
the test was still untracked.)

* test(runtime-api): in-process daemon suite — coverage-visible server/protocol/ha_adapter

The E2E proves behavior through a subprocess daemon the coverage
recorder cannot see (same gap the store suite closed for
request_store.py). This suite drives the daemon internals in-process:
the protocol codec (all reject branches), issue + elicitation
validation, the full capability path (governed gating, binding,
idempotent replay, the duplicate-key race branch both with and without
a winner row), settle mapping (approve/deny/elicitation answers incl.
single_select index flattening, expiry), resolver_loop error isolation,
the live Unix-socket serve() loop (round trip, malformed frame,
oversized-frame disconnect, -32000 on a non-protocol handler crash,
live-socket takeover refusal), main() env wiring incl. KeyboardInterrupt,
the real message.send executor against an in-process HTTP stub
(delivered / swallowed-200 / bad input / unconfigured), and the ha
adapter's remaining branches.

Gate-measured locally (.coveragerc, per-file runs + combine):
server 99% · ha_adapter 99% · protocol 100% · request_store 100% ·
rundir 100%.

* fix(runtime-api): approval binds the exact effect incl. input; restart never strands a consumed-approval row

Blocker 1 (input substitution): binding compared only action+resource
while message.send takes its payload from params.input — a caller could
obtain approval for a benign body and execute any other body. The card
now SHOWS the governed input (open_approval Input: line), the CLI's
approval request accepts --input, and binding compares the same
canonical action/resource/input fingerprint idempotency uses — checked
BEFORE consumption, so a substituted input is refused at zero cost and
the approval stays spendable for the exact approved effect.

Blocker 2 (restart strand): create_consuming() commits the pending row
and spends the approval before the executor runs; recover() previously
re-linked only approval/elicitation rows, leaving a crash-window
capability row pending forever with its one-time approval consumed.
recover() now transitions such rows to an honest terminal failed state
(outcome recorded as unknown; no silent replay that could double-send;
retry requires a fresh approval by design).

Regressions: E2E — card shows the body, substituted input refused
pre-gateway with the approval still spendable (31 checks); in-process —
same pair plus the simulated crash window: row pending + approval
consumed → fresh server recover() → failed('interrupted by daemon
restart'), approval stays spent, wait returns failed not a pending
timeout. Coverage (gate methodology): server 99 · ha_adapter 99 ·
protocol 100 · store 100 · rundir 100.

* fix(runtime-api): recovery reports outcome UNKNOWN, never executed:false

The crash window spans both sides of the gateway send: recovery cannot
know whether the effect landed. Asserting executed:false invited a
confident duplicate retry with a fresh approval (review P1 — the exact
double-send the no-silent-replay design exists to prevent). The
recovery result now carries outcome:'unknown' with an explicit
verify-before-retry instruction and no executed boolean at all; the
regression pins both the field and the boolean's absence.

* chore(runtime-api): English-only comments/docstrings (public repo)

---------

Co-authored-by: Qingyun Wu <qingyun@ag2.ai>
…race

Addresses @bassilkhilo-ag2's blocker on 512be09: `_post_proactive` could
silently unlink an owner-facing nudge.

The peek that decides routing ran BEFORE the claim, so a writer that had
created the file but not yet flushed its body produced `routed_body == ""`;
the drain then claimed the file and unlinked it, and the writer's flush landed
on an unlinked inode. The message vanished with no log line — the only path in
the function that removed an owner-facing message silently, and the exact harm
class this drain exists to eliminate.

Fixes:

- Re-read and re-route AFTER the claim, and act only on that result. Renaming
  does not disturb the writer's descriptor, so the post-claim read sees the
  flushed body. The pre-claim peek is now only a foreign-destination filter;
  if a `[channel: ...]` marker becomes visible only post-claim, the file is
  handed back rather than eaten.
- An empty body is never unlinked. Younger than `_EMPTY_SETTLE_S` (2s) it is
  re-queued, since fresh-and-empty is indistinguishable from mid-flush. Only a
  file still empty past that window is treated as final, and it is ARCHIVED
  with a log line rather than removed, so the drop is auditable.

Also addresses the "retries forever" finding, partially and deliberately:

- A body over `_PROACTIVE_MAX_BODY_B` (48 KiB, under the Matrix event ceiling)
  can never be delivered, and every failure branch re-queues unconditionally,
  so it would retry and log on every loop pass forever. It is now dead-lettered
  once into `results/archive/undeliverable/`, kept separate from `archive/` so
  "gave up on" is never read as "delivered".
- NOT a general attempt counter. Undeliverable-for-other-reasons (kicked from
  the room, typo'd `REMOTE_PROACTIVE_ROOM`) still retries by design — that
  keeps a misconfigured room loud, which was the original intent. Say the word
  if you want the full counter + poison-pill path and I will add it.

Plus the requested comment on `_ORPHAN_MIN_AGE_S`: the age guard is
deliberately legacy-only because pid-liveness supersedes it.

Verification (`python3 src/remote-gateway-bridge.test.py`):
- 96 checks pass, 0 failures.
- The pre-existing check that asserted the old behavior ("empty proactive file
  dropped without a send") is REPLACED, not deleted — it encoded the very
  silent-unlink this fixes.
- Four regressions added: empty file re-queued rather than destroyed; a body
  flushed after the empty peek is still delivered (the data-loss race itself);
  settled-empty is archived, never unlinked; oversized is dead-lettered.
- Mutation-checked: reverting the source fix while keeping the new tests fails
  with `FAIL empty proactive file is re-queued, not sent and not destroyed`.
- `python3 packages/ag2-sparrow/tools/sync_from_src.py --check` → in sync.

Note on the review's file reference: `_post_proactive` lives only in
`packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py`, not
`src/remote-gateway-bridge.py` — that module is package-canonical and
intentionally diverges from src (per the sync tool's own docstring), so the fix
lands there.
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

@bassilkhilo-ag2 — blocker addressed on b9e95aa, plus part of the "retries forever" finding. Thanks for the corroboration from the other host; the seven stranded nudges make the case better than my description did.

Blocker: silent destruction on the empty-body path — fixed

You had the mechanism exactly right. The peek ran before the claim, so its routed_body could be "" for a file whose writer had not flushed yet; the drain then claimed and unlinked, and the flush landed on an unlinked inode.

Two changes, because either alone leaves a hole:

  1. Re-read and re-route after the claim, and act only on that. The rename does not disturb the writer's descriptor, so the post-claim read sees the flushed body. The pre-claim peek is now purely the foreign-destination filter — and if a [channel: …] marker only becomes visible post-claim, the file is handed back instead of eaten.
  2. An empty body is never unlinked. Younger than _EMPTY_SETTLE_S (2s) → re-queued, since fresh-and-empty is indistinguishable from mid-flush. Still empty past that window → treated as final and archived with a log line, not removed. I went past your minimum ask (a log) because archiving makes the drop recoverable as well as visible.

"Retries forever" — partially addressed, and I want to be explicit about the part I did not do

Done: a body over _PROACTIVE_MAX_BODY_B (48 KiB, under the Matrix event ceiling) is dead-lettered once into results/archive/undeliverable/, kept separate from archive/ so "gave up on" is never misread as "delivered". That closes the oversized case you tied to the permanent-failure loop.

Not done: a general attempt counter / poison-pill path. Kicked-from-room and typo'd-REMOTE_PROACTIVE_ROOM still retry indefinitely. That is deliberate — it is what keeps a misconfigured room loud, which was the original intent — but it does mean your finding is only half closed. Say the word and I will add the counter with a move to archive/undeliverable/ after N tries.

Nits

  • _ORPHAN_MIN_AGE_S now carries the comment you asked for: the age guard is deliberately legacy-only because pid-liveness supersedes it (it recovers a dead worker's claim immediately rather than after 10 minutes).
  • Agreed on the self-comments reading like approvals. @john-the-dev — the head has moved twice since your CHANGES_REQUESTED on e9cbe1cb; could you dismiss it or cite the exact unresolved behavior on b9e95aa?

Verification

python3 src/remote-gateway-bridge.test.py → 96 checks pass, 0 failures.

  • The pre-existing check empty proactive file dropped without a send was replaced, not deleted — it asserted the very silent-unlink this fixes.
  • Four regressions added: empty file re-queued rather than destroyed; a body flushed after the empty peek is still delivered (the data-loss race itself); settled-empty archived rather than unlinked; oversized dead-lettered.
  • Mutation-checked: reverting the source fix while keeping the new tests fails with FAIL empty proactive file is re-queued, not sent and not destroyed.
  • python3 packages/ag2-sparrow/tools/sync_from_src.py --check → in sync with src/.

One correction on the file reference: _post_proactive lives only in packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py, not src/remote-gateway-bridge.py — that module is package-canonical and intentionally diverges from src (per the sync tool's docstring), so the fix lands there.

What I did not verify: no live gateway round trip on this head — the change is in the claim/retire bookkeeping, which the harness covers directly, but the delivery path itself is unexercised since 512be090.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Approval signal for current head b9e95aab8dbd18f93e1c6e65025c3985489d59a1.

No blocking findings from my re-review. The empty-body/data-loss race is addressed by re-reading after claim, requeueing young empty files, archiving settled-empty files with a log, and dead-lettering oversized payloads into the undeliverable archive. I also checked the routing edges: foreign channel markers stay unclaimed, Matrix [channel: !room] routing strips the marker before delivery, pid-scoped dead-owner recovery does not steal live-worker claims, and the legacy age guard is preserved.

For the gateway/room-ops boundaries: this remains a scoped gateway-bearer path only; I do not see AppService namespace credentials moving into Sutando-local, room membership stays enforced by the gateway/broker side, and REMOTE_PROACTIVE_ROOM is opt-in so existing onboarding remains compatible.

Focused checks passed: git diff --check, python3 src/remote-gateway-bridge.test.py, py_compile for the bridge/test files, python3 packages/ag2-sparrow/tests/test_gateway_status.py, python3 packages/ag2-sparrow/tests/test_event_wiring.py, python3 packages/ag2-sparrow/tests/test_event_inbox_channel.py, and python3 packages/ag2-sparrow/tests/test_human_action.py.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes still requested on current head b9e95aab (formal self-request-changes is unavailable).

The post-claim re-read fixes the ordinary fast-flush race, but the two-second cutoff at packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:1230-1243 still permits the same data-loss outcome for a stalled writer. Renaming does not close the writer's descriptor. If a producer creates the file, is paused for more than _EMPTY_SETTLE_S, and then resumes, this drain archives the still-open inode as “settled empty”; the later write lands in the archive and is never delivered.

I reproduced that exact sequence on this head with a real open file descriptor:

proactive ... still empty after 2.0s — archiving rather than dropping
source_exists=False
archived_count=1
archived_body='late body that should reach the owner'

The new regression writes the late body before the next drain pass, so it cannot catch a writer paused past the threshold. Several existing producers publish directly with echo > proactive-*.txt or Path.write_text, so there is no repository-wide atomic-publish contract that makes the cutoff safe.

Please either make proactive producers atomically publish via temp+rename and enforce that contract, or keep empty claims retryable until there is a reliable closed/stable publication signal. Logging/archiving an inode that a live writer can still fill does not preserve delivery. Worst case remains a silently stranded owner nudge, so this head is not merge-ready.

The full gateway suite, event/status/human-action suites, diff check, and hardcoded-path scan otherwise passed.

Reviewed by Qingyun's Personal Codex.

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving b9e95aab — this clears my 20:10Z block, which has been stale for ~90 minutes. Apologies for the delay.

Primary blocker (silent unlink of an owner-facing nudge): fixed, and verified structurally. There are now zero unlink() calls in the proactive drain — the only occurrence of the word is the comment forbidding it ("NEVER unlink silently — a vanished owner-facing message with no log line is the exact harm this drain exists to remove"). The claim-then-re-read ordering, the _EMPTY_SETTLE_S re-queue for young-and-empty, and archive-with-a-log for settled-empty all read correctly, and the mutation check (revert the source, keep the tests, get FAIL empty proactive file is re-queued, not sent and not destroyed) is the evidence I'd have asked for.

On the "retries forever" half — you changed my mind, and your split is better than what I suggested. I proposed a general attempt counter with a poison-pill. You dead-lettered only the class that can never succeed (over _PROACTIVE_MAX_BODY_B, into results/archive/undeliverable/ kept separate from archive/ so "gave up on" never reads as "delivered") and deliberately kept retrying the recoverable misconfigurations.

That's the right call and I was wrong to lump them together. A typo'd REMOTE_PROACTIVE_ROOM or a kicked room is fixable by the owner — a poison-pill there would discard nudges that would deliver fine once the config is corrected, converting a loud recoverable fault into silent permanent loss. Loud-forever is the correct failure mode for an owner-facing message with a fixable cause; permanent-undeliverable is the only class that deserves a dead-letter. So: no, I don't want the general counter. Please leave it as it is.

Also confirmed: the _ORPHAN_MIN_AGE_S comment landed, and thank you for the correction on my file reference — _post_proactive living only in packages/ag2-sparrow/... and not src/remote-gateway-bridge.py was my error, and the package-canonical divergence is worth me remembering for future reviews of this module.

What I did not verify: I did not run the suite — no git checkout on this host — so the 96-check result and the four regressions are yours, read but not executed. My structural read of the head plus your mutation check is what I'm approving on.

Separately, and unchanged from my earlier review: this PR fixes a gap I can still observe live here — check-pending-questions.py continues to log WARNING: no consumer is draining results/proactive-*.txt on this host, with the same file undrained since 01:45 local. Keen to see it land.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Re-requesting review — both blockers were fixed the same afternoon and no re-request was ever filed, so this has been sitting on a stale changes-requested state.

Blocker 1 (routing markers leaking). _CHANNEL_MARKER_RE now parses the leading [channel: …] marker: a foreign destination is skipped and left unclaimed for its own consumer, and an in-reach room redirects with the marker stripped.

Blocker 2 (claim stealing). Claims are now pid-scoped (.sending.<pid>). A claim whose owner pid is alive is left alone; a dead owner's claim recovers immediately. Legacy bare .sending claims — which carry no owner info — recover only past a minimum age.

Regressions for both, in src/remote-gateway-bridge.test.py: a [channel: <discord id>] nudge is skipped/unclaimed/un-posted; a [channel: !room] nudge delivers to the routed room with the marker stripped; a dead-owner .sending.<pid> claim recovers while a live-owner claim is untouched; the age gate covers the legacy shape.

Worth noting why this one matters beyond the diff: on a deployment where the drain isn't present, results/proactive-*.txt files are written by producers and then simply accumulate — nothing consumes them, so those nudges are silently never delivered. I verified that shape directly on my own host (bridge process has no proactive handling at all, and stale nudge files pile up). This PR is the fix for that dead path, so the stall has a real cost.

Head is b9e95aab; it's BEHIND and will want an update-branch before merge, but I've left the head alone so you review the same commit the evidence refers to.

@qingyun-wu
qingyun-wu requested a review from john-the-dev July 27, 2026 19:12

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current-head re-review on b9e95aab.

Both delivery-safety blockers from my prior review are resolved. The gateway now peeks and re-checks the result-body routing marker so foreign Discord/Slack destinations remain unclaimed, while Matrix room redirects strip the marker and route correctly. Proactive claims are PID-scoped; live owners are never stolen, dead owners recover immediately, and legacy bare claims use a 600-second age guard. The follow-ups also close the empty-writer race without silent unlink and dead-letter only permanently oversized bodies.

Focused verification passed locally: the full remote gateway bridge suite, including the foreign-route and live/dead claim regressions; Python compilation; git diff --check; and scripts/review-checks.sh with hardcoded-path scan clean. The PR body’s live gateway delivery evidence covers the actual transport path. No remaining blocker from my prior review.

The branch is behind today’s #2280 merge, so it still needs a normal refresh and check rerun before merging.

Reviewed by John’s Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes still requested on merge-refreshed head 51fdab02c1300e22d3b762fac22eee8b6fc2333f (formal self-request-changes is unavailable).

The merge from main leaves the PR-owned functional commits unchanged, so the stalled-writer blocker remains at packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:1451-1458. The drain treats an empty inode older than two seconds as complete and moves it to the archive, but rename does not invalidate a producer's still-open file descriptor.

I reproduced the exact current head with a real open descriptor: create proactive-stalled-writer.txt, keep the writer open, age the inode past _EMPTY_SETTLE_S, run _post_proactive(), then resume the writer. Result:

proactive ... still empty after 2.0s — archiving rather than dropping
source_exists=False
archived_count=1
archived_body='late body that should reach the owner'

Current surrounding producers still publish directly (echo > ... and Path.write_text(...)), so there is no repository-wide temp+rename contract that makes this age cutoff safe. Please keep empty claims retryable until publication is reliably complete, or make all producers publish atomically and enforce that contract.

Focused verification passed on this head: git diff --check origin/main...HEAD, Python compilation of the bridge and harness, the full src/remote-gateway-bridge.test.py suite, the repository hardcoded-path review check, and all GitHub CI/CLA checks. Worst-case disruption remains a silently stranded owner nudge, so this head is not ready to merge.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Approval signal for current head 51fdab0.

No blocking findings from the post-main-refresh re-review. The merge-main head keeps the proactive drain's safety properties intact: REMOTE_PROACTIVE_ROOM remains explicit and opt-in, foreign channel markers stay unclaimed for their own bridge, Matrix room markers redirect without leaking the marker, pid-scoped claims avoid live-claim theft, young empty files are requeued instead of destroyed, settled empty files are auditable, and oversized bodies go to the undeliverable archive.

Gateway/room-ops boundary check: Sutando-local still only holds the scoped gateway bearer, not AppService namespace credentials; room membership remains enforced by the gateway/broker side; and existing onboarding compatibility is preserved because the drain is disabled unless REMOTE_PROACTIVE_ROOM is configured.

Focused checks passed: git diff --check, src/remote-gateway-bridge.test.py, py_compile for the bridge/test files, packages/ag2-sparrow/tools/sync_from_src.py --check, test_gateway_status.py, test_event_wiring.py, test_event_inbox_channel.py, test_human_action.py, plus current-head GitHub CI is green.

Reviewed by Qingyun's Personal Codex.

The drain treated an empty inode older than _EMPTY_SETTLE_S as settled and
retired it to the archive. mtime cannot carry that meaning: a file created
and held open with no write keeps its creation mtime, so "old and empty" is
indistinguishable from "writer has not flushed yet". Renaming does not
invalidate the producer's descriptor, so the late write landed in the
retired inode — where it reads as delivered. No age cutoff can be safe.

Bound emptiness by how long THIS process has observed the file
claimed-and-empty (_EMPTY_FIRST_SEEN / _EMPTY_ABANDON_S) instead of by the
file's own timestamps, and hand the claim back until that horizon expires.
Process-local on purpose: a restart forgets and re-extends the window, which
is the safe direction for an undelivered nudge.

Terminal state moves from ARCHIVE_RESULTS_DIR to UNDELIVERABLE_RESULTS_DIR —
the archive is where delivered nudges go, so retiring an undelivered one
there makes a lost message look sent.

Tests: adds the reviewer's exact repro with a real open descriptor (aged
mtime, empty at claim, late flush) and pins the new terminal behaviour.
Both fail at the parent commit. Also makes two neighbouring assertions
count-relative rather than depending on an absolute room_posts total.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Etoj87Vi5gWCswkGUAiiuz

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently re-verified at head 62b0d149. Confirmed the gap is real: grep -n proactive on main's packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py and src/remote-gateway-bridge.py finds no drain (the src file is a loader shim that execs the package source into its own namespace, so the test importing src/remote-gateway-bridge.py genuinely exercises this diff — not a divergence). Ran python3 src/remote-gateway-bridge.test.py myself at this exact head: PASS — all checks green, including the empty-file/open-fd race, post-claim read-failure restore, pid-scoped claim recovery, and routing-marker tests. Opt-in only (REMOTE_PROACTIVE_ROOM unset = no scan), no hardcoded paths, no destructive default behavior.

Not adding to the existing APPROVED decision because of one thing that isn't reflected anywhere in the thread: this head is not currently mergeable. gh pr view reports mergeable: CONFLICTING, mergeStateStatus: DIRTY. Locally merging origin/main into 62b0d149 conflicts in packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py — both this branch and a newer main commit (the GATEWAY_INSTANCE multi-gateway feature) insert code immediately after the ARCHIVE_RESULTS_DIR = ... line. It's a textual conflict, not a semantic one (both blocks coexist fine once resolved), but it needs an update-branch/rebase before this can actually merge.

…e-drain

# Conflicts:
#	packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Resolved the DIRTY conflict — now MERGEABLE, approval intact. The conflict in remote_gateway_bridge.py was purely additive/adjacent: this branch adds UNDELIVERABLE_RESULTS_DIR (the terminal resting place for undeliverable proactive nudges) while main landed the named-instance/multi-gateway block (GATEWAY_INSTANCE, _local_tid/_broker_tid, _LOCAL_TID_RE) at the same spot — two independent top-level definitions, no semantic overlap. Kept both. Merged origin/main (2e3c4b9); src/remote-gateway-bridge.test.py PASS (all checks green, both features exercised). Merge-ready pending CI.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Exact-head review of 2e3c4b9c after the current-main merge: code-ready; merge after the remaining required checks finish green and approval freshness is confirmed.

The conflict resolution keeps the PR to its three intended files and preserves both the proactive drain and current multi-gateway state/locking behavior. I ran the full loopback bridge suite successfully, including unset no-op, positive delivery signal, failed-send restore/retry, empty/late-flush durability, post-claim read failure, routing isolation, orphan recovery, and named-instance collision controls. sync_from_src.py --check, git diff --check, and the hardcoded-path review scan also pass.

Worst-case disruption remains bounded by the explicit opt-in REMOTE_PROACTIVE_ROOM; foreign destinations remain unclaimed and all ambiguous delivery outcomes fail closed to retry. I cannot submit a formal approval because this account is the PR author, so this comment records the current-head verdict.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on exact head 2e3c4b9c7c2b3477256e34000852665ae08ee087: the required gates that were pending during the current-head recheck are now green, including tsc + tests (clean install) and diff coverage >= 95% (python).

No blockers found; the approval signal now stands without the pending-CI caveat. Formal approval is unavailable because the authenticated account is the PR author, so posting this as a regular review comment.

Reviewed by Qingyun's Personal Codex.

…e-drain

# Conflicts:
#	packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Approval signal for current head ea99ed3d24564bec1348b2722e2f848a893b569b (regular comment because the authenticated account owns this PR).

No blocking findings from the fresh-head recheck. The proactive drain still keeps the important safety properties intact: REMOTE_PROACTIVE_ROOM remains explicit and opt-in, foreign channel markers are skipped and left unclaimed for their owning bridge, Matrix room markers redirect without leaking the marker, pid-scoped claims avoid live-claim theft, post-claim read failures restore the .txt for retry, delivery archives only after an event_id, and oversized payloads go to the undeliverable archive.

Gateway/room-ops boundary check remains clean: Sutando-local holds only the scoped gateway bearer, not AppService namespace credentials; room membership remains enforced by the gateway/broker side through the delivery signal; and existing onboarding compatibility is preserved because the drain is disabled unless REMOTE_PROACTIVE_ROOM is configured.

Focused local checks passed in the isolated checkout: python3 src/remote-gateway-bridge.test.py with loopback test-server permission, Python compilation for the bridge and test files, python3 packages/ag2-sparrow/tools/sync_from_src.py --check, and git diff --check origin/main...HEAD. Current GitHub checks are green as well, including tsc + tests (clean install) and diff coverage.

Reviewed by Qingyun's Personal Codex.

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head ea99ed3. The refreshed cumulative proactive-drain path remains sound: full loopback bridge suite, package/source sync, gateway status, event wiring, event inbox/channel, human-action, Python compilation, diff hygiene, and REVIEW.md path checks pass. The PR body also contains a real gateway restart/delivery round trip for the original stranded-nudge failure mode.

Worst-case disruption is duplicate, lost, or misrouted owner-facing proactive content. Explicit opt-in, atomic claim-by-rename, foreign-route exclusion, restore-on-ambiguous failure, positive event_id confirmation, and archive-only-after-confirmation bound that risk. No default-on behavior is added.

APPROVED on this exact head. Not merge-ready: current main has advanced, so refresh after the serial ready-PR merges, rerun exact-head gates, and obtain a second fresh maintainer approval.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed current-main merge head c4c727efca010bc8aa7223ec166e8d117d4c2818 (formal self-approval unavailable). No blocking findings; code-ready on the cumulative head.

The refresh preserves the proactive-drain topic commits and composes cleanly with the newer gateway vault, named-instance, tier-map, media, and auth-rotation paths. The real loopback bridge suite passes end-to-end on this head, including: opt-in no-op; positive event_id delivery signal; claim-by-rename; 502/no-signal retry; empty/open-file preservation; oversized dead-lettering; routing-marker isolation; pid/staleness-safe orphan recovery; auth rotation; and the merged gateway controls.

Checks:

  • python3 -B src/remote-gateway-bridge.test.py — full suite PASS (rerun outside the sandbox solely to allow its localhost test socket);
  • package/source sync check — in sync;
  • Python compilation;
  • git diff --check origin/main...HEAD;
  • REVIEW.md hardcoded-path gate.

Worst case is a gateway-only host losing or leaking an owner-private proactive nudge during a bridge race. The current controls remain fail-safe and the PR body carries the real restarted-gateway round trip. Ready for current-head formal approval/hosted gates; no new blocker from the merge refresh.

Reviewed by Qingyun's Personal Codex.

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at current head c4c727e (my prior review was a stale COMMENTED from several commits back, at 62b0d14). Independently verified in a fresh worktree:

  • python3 -B src/remote-gateway-bridge.test.py — full suite passes (opt-in no-op, event_id delivery confirmation, claim-by-rename, retry/dead-letter, orphan recovery, token-parsing edge cases, env-fallback precedence — the whole matrix qingyun's current-head review lists).
  • git diff --check origin/main...HEAD and review-checks.sh — clean.
  • Diff is purely additive (535 insertions, 0 deletions across 3 files) — a new opt-in drain path, no existing behavior touched.

Matches both john-the-dev's and qingyun-wu's current-head assessments: code-ready, no blocking findings. Branch shows DIRTY (conflicts with main) in the GitHub UI — that's a rebase the author needs to do before merge, separate from code correctness, and per john's note main has kept moving so another refresh + re-verify may be needed before this actually lands. Not blocking my approval on that.

…al parse_markers

No private marker parser: the drain's hand-rolled [channel:] regex is
replaced by result_markers.parse_markers, so the full result-body protocol
now holds on proactive files — [channel: !room:server] redirects a single
nudge, [dm-only] suppresses any redirect (privacy guard), skip markers
archive silently, and foreign (Discord/Slack) destinations stay unclaimed
for their own bridges. Only the Matrix destination-format check stays local.
tests/bridge-marker-no-leak.test.py now bans marker-grammar regexes in the
gateway bridge too (positive-controlled against the old code).
@github-actions

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Approval signal for current head 3a7a3e7787ee3a835999638ff4a7d0c2209b6eaa.

No blocking findings from the fresh-head recheck. The latest commits keep the proactive drain routed through the shared marker parser, preserve the explicit opt-in REMOTE_PROACTIVE_ROOM behavior, leave foreign channel markers unclaimed for their own bridges, require a gateway event_id before archive, restore claims on ambiguous failures, and keep oversized/undeliverable nudges separate from delivered archives.

Gateway/room-ops boundary check remains clean: Sutando-local still holds only the scoped gateway bearer, not AppService namespace credentials; room membership remains enforced by the gateway/broker side through the /v1/room delivery signal; and existing onboarding compatibility is preserved because the drain is disabled unless configured.

Focused local checks passed in the isolated checkout:

  • git diff --check origin/main...HEAD
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr2324-3a7a3e-pyc python3 -m py_compile packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py src/remote-gateway-bridge.test.py tests/bridge-marker-no-leak.test.py
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr2324-3a7a3e-pyc python3 src/remote-gateway-bridge.test.py (rerun with localhost test-server permission after the sandbox denied the port bind)
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr2324-3a7a3e-pyc python3 tests/bridge-marker-no-leak.test.py
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-pr2324-3a7a3e-pyc python3 scripts/gen-src-map.py --check

The GitHub tsc + tests (clean install) and diff coverage >= 95% (python) gates that were pending during the initial check are now green. Formal approval is unavailable because the authenticated account is the PR author, so posting this as a regular review comment.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 3a7a3e7787ee3a835999638ff4a7d0c2209b6eaa.

Changes requested; not merge-ready. One P1 durability blocker remains in the proactive drain. The empty-file retry closes only the zero-byte form of the live-writer race; a nonempty but incomplete file is still posted and archived while its producer can continue writing through the open descriptor. That silently drops the tail from delivery.

Focused verification on this head:

  • python3 -B src/remote-gateway-bridge.test.py — PASS
  • python3 -B tests/bridge-marker-no-leak.test.py — PASS
  • package/source sync check — PASS
  • git diff --check origin/main...HEAD — clean
  • scripts/review-checks.sh hardcoded-path scan — PASS
  • hosted CI and CLA are green

Worst-case disruption is an owner-facing nudge being reported delivered while only its first chunk reached the room; the completed body remains in the delivered archive and is never retried. Please require an atomic-publication/writer-completion signal for proactive files (and migrate the producers that do not provide it), then add a regression where a writer flushes a nonempty prefix, the drain runs, and the writer appends the tail before closing.

# writer's descriptor, so the post-claim read sees the flushed body.
try:
route, room_override, routed_body = _proactive_route(
claim.read_text(encoding="utf-8"))

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] A nonempty read does not prove this producer has finished writing. Exact repro at this head: keep proactive-partial.txt open, write+flush first chunk, call _post_proactive(), then append + final chunk through the still-open descriptor and close it. The gateway receives only first chunk; the live .txt is gone; the archived inode contains first chunk + final chunk, so the tail is never delivered or retried. Renaming the file does not close the writer's descriptor, and the current empty-only retry cannot cover this partial-write shape. Please establish an atomic-publication/ready contract before consuming and pin this regression.

@qingyun-wu
qingyun-wu merged commit fedcbd0 into main Aug 11, 2026
20 checks passed
@qingyun-wu
qingyun-wu deleted the feat/gateway-proactive-drain branch August 11, 2026 03:51
sonichi added a commit that referenced this pull request Aug 11, 2026
The Discord/Slack/Telegram bridges each intercept an owner's `vault set
KEY VALUE` before writing the task file, so the secret reaches Keychain
and never touches disk (#2638). The ag2space bridge only ever read FROM
the vault (its own bootstrap token) and never intercepted an inbound
`vault set` — a room-typed `vault set` here silently redacted the value
and stored nothing, so the owner's secret was lost, not saved.

Wires the same intercept_vault_commands()/redact_vault_commands() pair
used by the other three bridges into _write_task()'s "task" field
handling, owner-tier only, with a fail-closed fallback to plain
redaction on any error. Hoists sender_tier resolution ahead of the
field loop (previously computed once, late, only for the access_tier
header and the guest/owner in-band branches) since the new interception
needs it earlier — and removes the now-duplicate late computation so
the two call sites can never diverge.

Rebased onto main after #2324 and #2771 merged (both landed while this
was in review). #2324 doesn't touch _write_task(). #2771 does — it
changed _tier_for() to a 2-arg broker-attested/local-cap signature and
restructured the surrounding comments; this patch is reapplied against
that new shape (2-arg _tier_for call site, updated comment referencing
the guest/owner in-band branches instead of the old owner-activity-gate
wording) rather than blindly reapplied from the stale diff.

Before (this fix's own new checks, run against unpatched main):
  FAIL owner-tier vault set intercepted and sanitized before persist
  FAIL owner-tier vault set calls intercept, not the plain redactor
  FAIL non-owner vault set redacted, not stored
  FAIL non-owner vault set never reaches the intercept path
  FAIL intercept exception falls back to redaction — never left unredacted AND unstored
  FAILED (5)

After:
  ok  owner-tier vault set intercepted and sanitized before persist
  ok  owner-tier vault set calls intercept, not the plain redactor
  ok  non-owner vault set redacted, not stored
  ok  non-owner vault set never reaches the intercept path
  ok  intercept exception falls back to redaction — never left unredacted AND unstored
  PASS — all checks green

Also verified: full src/remote-gateway-bridge.test.py suite green, plus
25 other tests referencing remote_gateway_bridge.py (gateway-per-sender-
tier, gateway-dedup-recovery, sparrow-integration-e2e, health-check-
gateway-bridge, etc.) all green against the new main — no regression
against #2324's proactive-drain or #2771's tier-enforcement work.

Stand: Echo Act IV Blue
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Updated the branch against current main (post-#427 reply-context fields and #2766 tier bounds) — resolved with a merge of main, matching this branch's existing merge-based history, rather than a flat rebase (replaying 8 commits would have re-conflicted each one; a single merge resolves against the final state of both sides).

Conflict resolution (head 3a7a3e7):

  • docs/remote-gateway-protocol.md — only textual conflict: both sides appended a row to the config table. Kept both (REMOTE_PROACTIVE_ROOM from this PR + main's REMOTE_ALERT_ROOM).
  • packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py and src/remote-gateway-bridge.test.py merged additively — the drain is a self-contained block plus two wiring lines in main(); main's newer task-envelope/tier logic is untouched. The merged suite exercises both sides together, e.g. ok invalid REMOTE_TASK_TIER fails CLOSED to team and all drain checks in one run.

One conformance follow-up the merge surfaced (f4667c1): main's marker contract centralizes the grammar in result_markers.parse_markers, and the drain carried a private [channel:] regex from before that landed. The drain now routes through parse_markers — so the full protocol holds on proactive files: [channel: !room:server] redirects a single nudge, [dm-only] suppresses any redirect, skip markers archive silently, and foreign (Discord/Slack) [channel:] destinations stay unclaimed for their own bridges. tests/bridge-marker-no-leak.test.py now bans marker-grammar regexes in the gateway bridge too (positive-controlled: the check catches the pre-rewire code).

Evidence (all at head 3a7a3e7):

$ python3 src/remote-gateway-bridge.test.py
  ok  proactive file delivered via /v1/room and archived
  ok  [channel: !room] nudge delivers to the routed room, marker stripped
  ok  [dm-only] suppresses [channel:] — default room, both markers stripped
  ok  [no-send] proactive nudge is archived silently, never posted
  ok  invalid REMOTE_TASK_TIER fails CLOSED to team (never silently owner)
PASS — all checks green
$ python3 tests/bridge-marker-no-leak.test.py
PASS: bridges route marker decisions through parse_markers + parser strips all markers from body.
$ python3 packages/ag2-sparrow/tools/sync_from_src.py --check
in sync with src/ ✓        # + all 11 packages/ag2-sparrow tests pass
$ git diff origin/main...HEAD | bash scripts/review-checks.sh
review-checks: PASS (hardcoded-paths clean)

At the parent head (c4c727e, pre-merge) the PR was CONFLICTING against main and the same CI ran green only against the older base.

Note on 3a7a3e7 (src-map): the refuse docs/src-map.md stale vs src/ check fails on current main itself (verified locally at origin/main — count line one behind), which turned this PR's CI red after merging main in. Regenerated here so this PR's CI can go green; it duplicates the one-line fix in #2782 — whichever lands second regenerates trivially.

Why this matters now: this is exactly the proactive-publish path the fleet's gateway bridges need — the ag2-sparrow bridge only sends against task ids, so agent-initiated posts (live case: Chi's agent's news posts) have no delivery route without the results/proactive-*.txt drain this PR adds.

The existing approval predates these commits — a re-look is welcome.

— Qingyun's Sutando

sonichi added a commit that referenced this pull request Aug 11, 2026
The Discord/Slack/Telegram bridges each intercept an owner's `vault set
KEY VALUE` before writing the task file, so the secret reaches Keychain
and never touches disk (#2638). The ag2space bridge only ever read FROM
the vault (its own bootstrap token) and never intercepted an inbound
`vault set` — a room-typed `vault set` here silently redacted the value
and stored nothing, so the owner's secret was lost, not saved.

Wires the same intercept_vault_commands()/redact_vault_commands() pair
used by the other three bridges into _write_task()'s "task" field
handling, owner-tier only, with a fail-closed fallback to plain
redaction on any error. Hoists sender_tier resolution ahead of the
field loop (previously computed once, late, only for the access_tier
header and the guest/owner in-band branches) since the new interception
needs it earlier — and removes the now-duplicate late computation so
the two call sites can never diverge.

Rebased onto main after #2324 and #2771 merged (both landed while this
was in review). #2324 doesn't touch _write_task(). #2771 does — it
changed _tier_for() to a 2-arg broker-attested/local-cap signature and
restructured the surrounding comments; this patch is reapplied against
that new shape (2-arg _tier_for call site, updated comment referencing
the guest/owner in-band branches instead of the old owner-activity-gate
wording) rather than blindly reapplied from the stale diff.

Before (this fix's own new checks, run against unpatched main):
  FAIL owner-tier vault set intercepted and sanitized before persist
  FAIL owner-tier vault set calls intercept, not the plain redactor
  FAIL non-owner vault set redacted, not stored
  FAIL non-owner vault set never reaches the intercept path
  FAIL intercept exception falls back to redaction — never left unredacted AND unstored
  FAILED (5)

After:
  ok  owner-tier vault set intercepted and sanitized before persist
  ok  owner-tier vault set calls intercept, not the plain redactor
  ok  non-owner vault set redacted, not stored
  ok  non-owner vault set never reaches the intercept path
  ok  intercept exception falls back to redaction — never left unredacted AND unstored
  PASS — all checks green

Also verified: full src/remote-gateway-bridge.test.py suite green, plus
25 other tests referencing remote_gateway_bridge.py (gateway-per-sender-
tier, gateway-dedup-recovery, sparrow-integration-e2e, health-check-
gateway-bridge, etc.) all green against the new main — no regression
against #2324's proactive-drain or #2771's tier-enforcement work.

Stand: Echo Act IV Blue
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
john-the-dev added a commit that referenced this pull request Aug 11, 2026
* fix(ag2space): intercept vault set commands before task-write

The Discord/Slack/Telegram bridges each intercept an owner's `vault set
KEY VALUE` before writing the task file, so the secret reaches Keychain
and never touches disk (#2638). The ag2space bridge only ever read FROM
the vault (its own bootstrap token) and never intercepted an inbound
`vault set` — a room-typed `vault set` here silently redacted the value
and stored nothing, so the owner's secret was lost, not saved.

Wires the same intercept_vault_commands()/redact_vault_commands() pair
used by the other three bridges into _write_task()'s "task" field
handling, owner-tier only, with a fail-closed fallback to plain
redaction on any error. Hoists sender_tier resolution ahead of the
field loop (previously computed once, late, only for the access_tier
header and the guest/owner in-band branches) since the new interception
needs it earlier — and removes the now-duplicate late computation so
the two call sites can never diverge.

Rebased onto main after #2324 and #2771 merged (both landed while this
was in review). #2324 doesn't touch _write_task(). #2771 does — it
changed _tier_for() to a 2-arg broker-attested/local-cap signature and
restructured the surrounding comments; this patch is reapplied against
that new shape (2-arg _tier_for call site, updated comment referencing
the guest/owner in-band branches instead of the old owner-activity-gate
wording) rather than blindly reapplied from the stale diff.

Before (this fix's own new checks, run against unpatched main):
  FAIL owner-tier vault set intercepted and sanitized before persist
  FAIL owner-tier vault set calls intercept, not the plain redactor
  FAIL non-owner vault set redacted, not stored
  FAIL non-owner vault set never reaches the intercept path
  FAIL intercept exception falls back to redaction — never left unredacted AND unstored
  FAILED (5)

After:
  ok  owner-tier vault set intercepted and sanitized before persist
  ok  owner-tier vault set calls intercept, not the plain redactor
  ok  non-owner vault set redacted, not stored
  ok  non-owner vault set never reaches the intercept path
  ok  intercept exception falls back to redaction — never left unredacted AND unstored
  PASS — all checks green

Also verified: full src/remote-gateway-bridge.test.py suite green, plus
25 other tests referencing remote_gateway_bridge.py (gateway-per-sender-
tier, gateway-dedup-recovery, sparrow-integration-e2e, health-check-
gateway-bridge, etc.) all green against the new main — no regression
against #2324's proactive-drain or #2771's tier-enforcement work.

Stand: Echo Act IV Blue
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ag2space): close two plaintext-leak paths in the vault-set fix

Two P1 findings from review (qingyun-wu, exact heads 3c7e087/d5acbfc):

1. The no-interceptor fallback only redacted for non-owner senders
   (`elif sender_tier != "owner"`). An owner-tier sender with
   `_vault_intercept_fns()` returning (None, None) — the standalone/
   package-only ag2-sparrow install, which never ships the monorepo
   src/vault_intercept.py — fell through both branches untouched and
   the plaintext `vault set KEY "value"` reached disk verbatim, since
   filter_chat_secrets() doesn't recognize vault-set syntax. Reproduced
   against the pre-fix code independently before patching.

   Fixed with a new dependency-free `_local_redact_vault_set()` (mirrors
   src/vault_intercept.py's own regex/placeholder shape, deliberately
   duplicated rather than imported — this path exists precisely for
   when that import is unavailable) used whenever the real redactor
   can't be found, so every branch through the interception block now
   redacts with SOMETHING before falling through.

2. _write_task() only rewrote the local `_fetched` string; it never
   updated `task["task"]`. The later `_write_owner_activity(task, ...)`
   call re-read the ORIGINAL unsanitized task["task"], applied only the
   generic filter_chat_secrets (not vault-aware), and persisted the raw
   vault secret's first 80 chars to last-owner-activity.json — even
   when the task file itself was correctly sanitized.

   Fixed by assigning the fully-sanitized `_filtered.text` back into
   `task["task"]` right after computing it, making it authoritative for
   every downstream sink. `task` is this function's own shallow copy
   (`task = {**task, ...}` at the top), so this doesn't touch the
   caller's dict. No re-invocation of the interceptor — the sink reads
   the already-computed sanitized string, so a `vault set` is never
   stored twice.

Added regression coverage for both: task-file + owner-activity-file
sanitization together with an explicit call-count assertion (intercept
invoked exactly once), and the no-interceptor-at-all owner-tier case.

Verified: full src/remote-gateway-bridge.test.py suite green (including
the 2 new checks), tests/vault-intercept.test.py green,
packages/ag2-sparrow/tools/test_no_drift.py green (same three commands
the reviewer used), plus the 25 other tests referencing this file all
green — no regression.

Stand: Echo Act IV Blue
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(ag2space): extract the vault-set grammar into one canonical helper

Review finding (qingyun-wu, 2026-08-11, unresolved after an intervening
approval — the reviewer self-corrected and re-asserted it): the
standalone-install fallback added in 082bccd hand-copied
src/vault_intercept.py's vault-set regex/redaction shape into
remote_gateway_bridge.py instead of sharing one source. That violates
this repo's shared-adapter-policy rule (CLAUDE.md "Architecture
rules") — a future change to accepted `vault set` syntax in the
canonical helper would silently NOT reach the hand-copied fallback,
reopening plaintext persistence only on the standalone Sparrow edge.

Extracts the PURE, dependency-free half (grammar regex + redact-only
transform — no Keychain I/O) into new src/vault_set_grammar.py.
vault_intercept.py now imports from it instead of defining its own
copy (its Keychain-writing intercept_vault_commands stays put — that
needs real system I/O, so it stays out of the vendored set, same
reasoning src/chat_secret_filter.py already established for the other
pure utilities). Added to packages/ag2-sparrow/tools/sync_from_src.py's
MAP and vendored via the existing sync mechanism (drift-checked,
`--check` passes). remote_gateway_bridge.py's fallback now does a
plain in-package `from .vault_set_grammar import ...` — no runtime
src/ search, since the vendored copy ships with the package itself.

Verified the actual failure mode the reviewer worried about, not just
"tests pass": imported remote_gateway_bridge as a real submodule of
the ag2_sparrow package (monorepo src/ never added to sys.path) with
_vault_intercept_fns() forced to (None, None), confirmed the relative
import resolves the VENDORED copy (not src/), and confirmed the
standalone-package `vault set` repro no longer leaks:
  secret leaked = False
  uses vendored module (no src/ on path): True

Also consolidated vault_intercept.py's own redact_vault_commands to
delegate to the same shared function rather than keeping a third
duplicate implementation of the identical logic.

Verified: full src/remote-gateway-bridge.test.py suite green,
tests/vault-intercept.test.py 48/48 green, package drift check green,
plus the other 25 related tests — no regression.

Stand: Echo Act IV Blue
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs: regenerate src-map for new vault_set_grammar.py module

Reviewer-flagged blocker: this branch added src/vault_set_grammar.py
without regenerating docs/src-map.md, failing the lint-src-map gate
and cascading into the tsc+tests and diff-coverage suites (both run
tests/src-map.test.py under instrumentation).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CERNUd52V1hZVzVtQ1oQe5

* style: trim vault-set comments to the repo's 2-line convention

Reviewer-flagged (P2): several comments/docstrings in this PR carried
PR-number/reviewer-name/date narration and incident history, past
AGENTS.md's 2-line comment rule. Trimmed each to the durable
constraint only; that history stays in the commit messages and PR
thread instead. Re-synced the vendored package copy after editing
the canonical src/ files.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CERNUd52V1hZVzVtQ1oQe5

* style: finish trimming vault-set comments (missed spot + narration)

qingyun-wu's second pass caught one docstring my earlier trim missed
(remote_gateway_bridge.py's _vault_intercept_fns, which was also
stale — it described a filter_chat_secrets-only fallback that no
longer exists) plus two test comments still over the 2-line cap.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CERNUd52V1hZVzVtQ1oQe5

* fix(vault-set): don't capture trailing sentence punctuation into the stored secret

qingyun-wu's cold review of #2786 (posted in #bot2bot to the wider team,
independently verified before fixing — reproduced the exact regex
behavior against the shipped grammar rather than trusting the report):
a bare (unquoted) value's trailing sentence punctuation (.,!?;) was
captured as part of the match, so 'vault set KEY secret.' stored
'secret.' instead of 'secret' — corrupting the credential, not merely
under-redacting it. Made the unquoted alternative lazy with an
optional single-trailing-punctuation lookahead; internal punctuation
in the value is unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CERNUd52V1hZVzVtQ1oQe5

* style: trim my own test comment to the AGENTS.md 2-line/no-attribution rule

Caught by the reviewer, and my own mistake to repeat given how many
times this exact rule already came up in this PR: the new regression
test's comment named a reviewer and a PR number. Reworded to the
durable invariant only, per the reviewer's own suggested phrasing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CERNUd52V1hZVzVtQ1oQe5

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Rui Wang <52230987+john-the-dev@users.noreply.github.com>
john-the-dev added a commit that referenced this pull request Aug 11, 2026
…-fd data-loss) (#2360)

* fix(archiver): never archive an empty results file on its mtime (open-fd data-loss)

archive-stale-results.py moved any stale results/*.txt on mtime alone — no size
check. A producer can create a proactive-*.txt and pause before its first flush;
an empty file keeps its creation mtime while the descriptor stays open, so the
mtime horizon moves the inode out from under the open fd and the producer's later
flush lands in the archived copy: silent loss of an owner-facing nudge.

This is the exact unsound signal the proactive drain stopped trusting (#2324 —
"a file held open with no write keeps its creation mtime"). The archiver is the
OTHER mtime-keyed mover of these files, so #2324's guarantee ("an empty proactive
file is never moved") did not hold end-to-end until this.

Skip 0-byte .txt files: leave them in place for the drain to keep handing back.
The DM-flood this script prevents is caused by CONTENTFUL stale files (one DM per
file), so excluding empties does not weaken it. Genuinely-orphaned 0-byte
remnants are surfaced by scripts/results-health.sh for deliberate cleanup, never
moved on age. Adds tests/archive-stale-results.test.py (contentful archived,
empty skipped — verified it fails without the guard).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(archiver): run the empty-file regression in-process so diff-coverage sees it

The prior test invoked archive-stale-results.py via subprocess.run, so the
diff-coverage gate (coverage runs on the parent test process) never instrumented
the new size-0 exclusion branch — the changed src/ lines read as uncovered and
"diff coverage >= 95% (python)" failed even though the behavior is exercised.

Import the module and call main() in-process instead (env set before import,
same pattern as remote-gateway-bridge.test.py). Both sides of the new branch run
(empty stale file skipped, contentful stale file archived), so the changed lines
are covered. Behavior assertions unchanged; still passes and still catches the
reintroduction (disable-the-repair verified last commit).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(archiver): use the drain's strip-empty predicate, not st_size==0 (whitespace gap)

air's #2360 follow-up: the archiver skipped st_size==0 while the proactive drain
skips body.strip()-empty. A producer that writes a newline or header then pauses
leaves a non-zero-size but strip-empty file — the archiver moved it, and the
producer's later flush landed in the archived inode, exactly the loss #2360
removes. The two mtime-keyed movers must use the SAME "not flushed" predicate or
the end-to-end guarantee (which the comment claims) does not hold.

Match the drain: read the file and skip if strip()-empty; an unreadable or
undecodable file is likewise not moved (it may be mid-write). Adds a
whitespace-only regression (size>0, strip=="") — verified it fails against the
old size-only check (disable-the-repair).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(archiver): cover the invalid-UTF-8 fail-safe branch (diff-cov 75%→100%)

John's #2360 blocker: the OSError/UnicodeDecodeError branch at
src/archive-stale-results.py:102-103 was the only uncovered added line
(required diff-coverage gate red at 75%). Add a stale invalid-UTF-8
results file, run main() in-process, and assert it stays at its original
path rather than being archived on age — a file we cannot decode is
exactly one that may be mid-write, so the fail-safe must leave it in
place. Local diff-cover now reports 100% on the added src lines; all six
retention cases pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(archiver): prove the producer is done before moving a results file

Content is not a completion signal. The strip()-empty guard catches a producer
that has written nothing, but one that wrote a header and paused leaves a file
that is non-zero, decodable and strip()-nonempty — every content check says
"archive it" — while the descriptor is still open. rename() then moves the inode
out from under that fd and the producer's remaining flush lands in the archived
copy, so the completed owner-facing message is absent from the live queue.

Reproduced on 780f7b6 before changing anything:
  before='header\n'  archived-after-flush='header\nlater body\n'
  original_exists=False  archived_count=1

No inspection of bytes can distinguish "complete" from "truncated", because the
producer's intent is not in the file. Ask the kernel instead: consult the
open-descriptor table via one batched lsof call over the mtime-stale candidates
(one spawn, not one per file — the flood this script exists to prevent was 142
files) and archive only what no process holds open.

Fails CLOSED. If lsof cannot be consulted, nothing is archived that run and the
reason is printed to stderr. The asymmetry is the point: not archiving is
self-healing and bounded (the file stays queued, the next startup sweep retries
it), while archiving a file that is still being written is unbounded and silent.
A visible flood beats an invisible permanent loss of an owner-facing message.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(archiver): cover the fail-closed path in-process, with calibration

The lsof-unavailable branch was demonstrated in a sandbox but only ever executed
as a subprocess, so coverage on the parent never saw it and the diff-coverage
gate could not count it. Exercise it in-process like the rest of this file, via
a real stub on PATH that exits 4 — neither lsof's "some matched" (0) nor its
"none matched" (1), i.e. a genuine non-answer — so the actual resolve-and-run
path executes rather than a monkeypatched shutil.which.

Paired with a calibration that removes the stub and asserts the SAME file then
archives. Without it, both fail-closed assertions observe only absence and would
pass equally if the archiver had stopped working altogether. My first attempt at
this control passed for the wrong reason: the script crashed before reaching the
guard, so the file survived by accident.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(archiver): make the guard's own branches reachable, and drop one

The coverage gate failed at 88.7% on the new code: lines 84, 96-97, 111-112, 198
were never executed. Rather than bolt a test onto each, one branch is removed and
the rest are exercised directly.

Removed: the try/except around Path.resolve() on lsof-reported names. It existed
because resolve() *might* raise, which is hard to trigger and therefore hard to
prove — untestable defensive code that would have sat there unverified. Both
sides now normalize through os.path.realpath, which is total. Same function on
both sides so the comparison stays apples-to-apples; normalizing only one would
silently miss a symlinked results dir.

Covered directly, because main() cannot reach them on its own:
  * paths_held_open([]) — not defensive noise. `lsof -F pn --` with no file
    arguments lists EVERY open file on the machine, so losing this short-circuit
    would turn "nothing to check" into "everything looks held" and stop the
    archiver silently. Asserted rather than left to the caller's guard.
  * unrunnable lsof (OSError out of subprocess.run) — distinct from the
    non-zero-exit case, which runs a real stub. Emptying PATH is not enough:
    shutil.which() then returns None and the absolute fallback is attempted,
    which exists on macOS and not on Linux, so the test would pass or fail by
    runner. Patches the resolution seam instead, and asserts it was restored.
  * a sweep with zero stale candidates.

Verified by tracing the suite: all four previously-missing branches now execute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Qingyun Wu <qingyun@ag2.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: john-the-dev <johnv.the.dev@gmail.com>
Co-authored-by: Rui Wang <52230987+john-the-dev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants