Repository navigation
feat(sparrow): REMOTE_TASK_ON_TASK wakes the agent once per newly queued task - #5317
Conversation
…ued task Sparrow writes each relay task to TASKS_DIR/<task-id>.txt and stops there, so every outside agent writes its own folder-polling hook to notice it. With REMOTE_TASK_ON_TASK set, Sparrow runs that command once per task file it newly publishes: argv split with shlex (no shell), the file path appended, SPARROW_TASK_ID / SPARROW_TASK_FILE in the environment, and every relay credential removed from it. It is started detached and never waited on, killed or retried; a start failure or non-zero exit is logged once and the task file is unaffected. Unset keeps today's behavior. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 91%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.pyLines 3311-3321 3311 _log(f"on-task command failed to start for {tid}: {type(exc).__name__}: {exc}")
3312 return
3313
3314 def _reap() -> None:
! 3315 code = proc.wait()
! 3316 if code != 0:
! 3317 _log(f"on-task command exited {code} for {tid}")
3318
3319 threading.Thread(target=_reap, name=f"on-task-{tid}", daemon=True).start()
3320 packages/ag2-sparrow/tests/test_on_task_wake.pyLines 99-109 99 real = m.subprocess.Popen
100
101
102 def spy(*a, **k):
! 103 if "SPARROW_TASK_ID" in (k.get("env") or {}):
! 104 spawned.append(a)
! 105 return real(*a, **k)
106
107 m.subprocess.Popen = spy # type: ignore[assignment]
108 try:
109 tid, _ = m._write_task(_task())Lines 138-146 138 assert len(files) == 1, f"{name}: {files}"
139 if name == "nonzero exit":
140 deadline = time.time() + 10
141 while time.time() < deadline and not any("exited 3" in ln for ln in logs):
! 142 time.sleep(0.05)
143 if name != "slow":
144 assert any("on-task command" in ln for ln in logs), f"{name}: {logs}"
145 print(f"PASS test_failing_or_slow_command ({name})") |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@cla-assistant check |
sonichi
left a comment
There was a problem hiding this comment.
Reviewed at head e61ef7d6. Traced the claims directly against the diff rather than the body alone.
Fire-once-on-fresh-publish, confirmed against the real control flow, not just the prose. _write_task has three paths that return before _fire_on_task is ever reached: _task_pending(tid) (still queued/assigned/claimed), task_archived or _delivered_copy_exists(tid) (redelivery of already-handled work), and only the genuine fresh-publish path falls through to the write, _write_owner_activity, then _fire_on_task. So a redelivery or restart hitting either early return never re-fires — matches test_redelivery_and_restart_do_not_refire.
Credential stripping is belt-and-suspenders, not just name-matching. _on_task_env() drops REMOTE_*/AG2_* keys containing TOKEN, and separately drops any variable whose value contains one of the actual secret strings (TOKEN, _RAW, HS_MEDIA_TOKEN) — so a credential stored under a differently-named variable is still caught. test_token_never_reaches_the_command checks both the absent-key and absent-value-substring cases, and separately confirms an unrelated setting is not swept up, so this isn't over-redacting either.
Non-blocking and fail-safe is real, not asserted. Popen(..., start_new_session=True), stdio to /dev/null, a daemon reaper thread, broad exception catch around the spawn itself with one log line — never raises into the pull loop. test_failing_or_slow_command_never_blocks_or_loses_the_task backs this with a < 3s elapsed assertion across missing-binary, nonzero-exit, unparseable-command and slow-command cases, each also confirming the task file itself still lands durably regardless.
CODEOWNERS: this touches /packages/ag2-sparrow/ (owned solely by @qingyun-wu per the /packages/ CODEOWNERS line — not @sonichi) plus .github/, docs/, tests/ under the repo-wide//tests/ lines. Authored under the shared sonichi login, so my own review can't count as the gate-clearing approval here regardless — qingyun-wu's is the one this needs, and for the /packages/ path specifically, hers is the only valid code-owner approval, full stop.
The one real gap, and it matches this repo's own standard: the PR body is honest about it — "Not done: a round trip against the real AG2 Space relay... the live Sutando core was not restarted." This is the gateway bridge's task-write path, squarely a live delivery-loop change, and the mock end-to-end (real redelivery + restart against a local HTTP relay) is good but isn't the live witness this class of change needs per CONTRIBUTING.md and this session's own precedent on #5278/#5287/#5304 etc. Approve on that witness landing, same as the rest of today's live-path PRs.
Nothing else to push back on — the design (one optional setting, defaults to today's no-op behavior, argv via shlex with no shell) is minimal and well-scoped.
|
Correction to my own comment above: on reflection (Chi asked directly whether a core restart is actually needed here, and that's the right question), demanding a live Sutando core restart is the wrong ask for this specific change. Checked directly: nothing in this repo sets The already-present end-to-end test is the right shape of evidence for what's actually new here: a real local HTTP server implementing the long-poll protocol, a real process restart against persisted dirs with real redelivery, and a real Net: I'm dropping the "needs a core restart witness" ask. Nothing else in my review above changes. |
liususan091219
left a comment
There was a problem hiding this comment.
Reviewed head e61ef7d6babca03382315d9b3f76244f2fca7567. I checked the code itself, not just the PR body. No change-request condition fires. The verdict is COMMENT only because the PR has no formal approval at this head yet (sonichi's review is COMMENTED and uses the shared login). As qingyun-wu is the sole CODEOWNER of /packages/, that approval has to come from a maintainer other than the author.
Claims checked against packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:
- No shell:
_fire_on_taskbuildsshlex.split(ON_TASK_CMD) + [str(path)]and callssubprocess.Popen(argv, ...). It never usesshell=True. Task content is never placed in argv. The command gets only the task-file path, plusSPARROW_TASK_ID/SPARROW_TASK_FILEin its environment. - No argument injection: the path is
TASKS_DIR / f"{tid}.txt".tidcomes from_local_tidafter_valid_tid([A-Za-z0-9._-]{1,64}), and it always starts withtask-. The leading part of the path is the operator's task dir, so a relay-supplied id cannot add a flag. - Never blocks:
Popen(start_new_session=True)with all three stdio streams on DEVNULL. A daemon thread waits on the child. Any exception at spawn time is logged once and returns. - Credentials scrubbed:
_on_task_envremovesREMOTE_*/AG2_*names that containTOKEN. It also removes any variable whose value containsTOKEN,_RAWorHS_MEDIA_TOKEN.TOKENis a module global that is reassigned on rotation (line ~2278), so a rotated secret is still caught by value. Those three are the only relay secrets the module holds. The token read from the ag2space channel.envor the vault never entersos.environ. - Fires once per task: the call is the last line of
_write_task, after_publish_staged. The_task_pendingpath (queued/assigned/claimed) and the archived/delivered path both return before it. The task file is the dedupe state, so a restart or a redelivery does not fire again. The skipped-wake-on-crash trade-off is stated in the README. - Vendoring rule:
remote_gateway_bridge.pyis package-canonical.src/remote-gateway-bridge.pyis a 342-line shim that executes the package source, andsync_from_src.pydeliberately leaves this module out of MAP. That means no private copy exists to drift.tools/sync_from_src.py --checkreports "in sync with src/".
Tests run here: packages/ag2-sparrow/tests/test_on_task_wake.py ALL PASS, tests/ag2-sparrow-on-task-wake.test.py ALL PASS, test_write_task_atomic.py ALL PASS. The tests call the real _write_task and record the real child process. They do not grep the source.
Revert controls (each restored afterwards):
- Fire call removed:
test_fires_once...fails withAssertionError: []. - Value scrub removed:
test_token_never_reaches_the_commandfails. - Reaper made synchronous: the slow-command case fails with "the pull loop waited 12.2s".
- Name scrub removed: still ALL PASS (nit 1).
CI: at 056c3b53 one check failed: diff coverage >= 95% (python), with remote_gateway_bridge.py at 33.3% and lines 3294–3319 uncovered. The package test ran outside the coverage job. 426e4633 added the runpy wrapper tests/ag2-sparrow-on-task-wake.test.py to fix it. At e61ef7d6, all 32 check runs succeed (2 skipped), and license/cla = success with 1 context. I found only one failure on this PR's commits, not two.
Named, not blocking: there is no live round trip against the real AG2 Space relay with the setting on. The body says so. The mock-relay end-to-end through main() is good evidence for the code path.
Nits (none blocking):
- Nothing pins the name-based scrub on its own. In the test, every token variable also holds the secret value, so the value scrub covers them. An assert that
REMOTE_TASK_TOKEN_FILE(a path, not a secret) is absent would pin it. - The README says "absolute path". It is absolute only if
AGENT_CONNECT_TASK_DIRis.str(dest.resolve())would make that true. - "Never sees the token" covers the environment only. The child runs as the same user and can read the token file. Worth one README clause so nobody reads this as a sandbox.
- A command that never exits leaves one reaper thread per task for the life of the process.
— Sutando (susans-MacBook-Pro)
sonichi
left a comment
There was a problem hiding this comment.
Approving at e61ef7d6, on Chi's explicit word.
I recommend APPROVE #5317, authored by qingyun-wu. What it does (from the diff, not the title): adds an optional setting to the ag2-sparrow task bridge that runs a command once each time it writes a new task file, so an outside agent doesn't need its own folder-polling loop. Never blocks the main pull loop, runs without a shell, and strips every token-bearing env var by both name and actual secret value before the child sees it.
Files: packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py (the logic), two test files, README.md + docs/remote-gateway-protocol.md, and .github/workflows/ci.yml (wires the new test in).
Breaking change: none declared, none found — purely additive, off by default. Confirmed nothing in this repo currently sets the new variable, so merging changes zero behavior for any existing deployment, including this core's own.
Live test: no round trip against the real AG2 Space relay with the setting on — the PR names this gap itself. The existing mock end-to-end (real local HTTP relay, real process restart, real subprocess spawn) is strong substitute evidence, but neither review exercised the actual production relay. Not a blocker given the feature is opt-in and inert everywhere today.
Gate values at this head: CI 32/32 green including license/cla, no CHANGES_REQUESTED anywhere, two independent prior reviews (mine and Susan's) both clean. She's the sole CODEOWNERS owner of packages/ag2-sparrow/ and the PR's own author — the singleton-owner-blocked-on-own-PR case this repo's CODEOWNERS names, whose recovery is a bypass approval/merge by sonichi or qingyun-wu. This approval is that.
0.4.0 ships REMOTE_TASK_ON_TASK (#5317): an optional command Sparrow runs once per newly queued task, so an outside agent needs no folder-watching hook. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
What changed, and why
ag2-sparrow writes each relay task to
TASKS_DIR/<task-id>.txtand stops there. Nothing wakes the agent, so every outside agent writes its own folder-polling hook. This adds the wake step to Sparrow itself.New optional setting
REMOTE_TASK_ON_TASK(env, like the otherREMOTE_TASK_*settings; the package has no CLI flags, so none is added):shlex.spliton the value, no shell; the absolute task-file path is appended as the last argument. The environment addsSPARROW_TASK_IDandSPARROW_TASK_FILE.REMOTE_*/AG2_*variable whose name containsTOKENis dropped (REMOTE_TASK_TOKEN,AG2_REMOTE_TOKEN,REMOTE_TASK_TOKEN_FILE,REMOTE_MEDIA_HS_TOKEN), as is any variable whose value contains the relay secret, the raw onboarding string, or the HS media token.Popen(start_new_session=True)with stdio on/dev/null; a daemon thread reaps it and logs one line on a non-zero exit. A start failure (missing binary, unparseable string) is logged once. Sparrow never waits for, kills or retries the command. The task file, the ack and the in-flight ledger are not affected._write_task, on the fresh-publish path only. Every idempotent early return (still queued /.assigned-*/.claimed-*/ archived / delivered) returns before it, so a redelivery or a restart does not fire it again. Chosen trade-off, stated in the README: if Sparrow dies between publishing the file and spawning the command, that one wake is skipped (no state-dir marker, no replay); the file is still queued for the agent's next run._fire_on_taskreturns at once. Behavior is unchanged.No version bump:
__version__changes only in publish PRs (git log ofag2_sparrow/__init__.py: #2302, #3074).Where to look first
packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:_on_task_env,_fire_on_task, and the call at the end of_write_task.packages/ag2-sparrow/tests/test_on_task_wake.py(wired intoci.ymlnext totest_write_task_atomic.py), plustests/ag2-sparrow-on-task-wake.test.py, a runpy wrapper so the diff-coverage gate measures it. Same pattern astests/ag2-sparrow-human-action.test.py.packages/ag2-sparrow/README.md: new "Waking your agent" section.User behavior this proves
An agent paired with Sparrow sets
REMOTE_TASK_ON_TASK="my-agent run-task"and gets onemy-agent run-task <task-file>per new task, with no folder-watching hook of its own.Feature/documentation consistency
packages/ag2-sparrow/README.md: "Waking your agent" section with an example.docs/remote-gateway-protocol.md(canonical for the remote gateway, perdocs/catalog.json): aREMOTE_TASK_ON_TASKrow in the config table.last_verifiedbumped to 2026-10-10.Before / after evidence
New unit test at the parent commit
0f920a148(origin/main), with the new test file copied in:At HEAD:
End-to-end through
main()against a local mock relay. The realpython -m ag2_sparrow.remote_gateway_bridgelong-polls a local HTTP server that serves one task on the first poll. Run 1 is a fresh start. Run 2 restarts Sparrow on the same dirs, and the relay redelivers the same task id. The command is a recorder script. Isolated env:HOME/dirs in a temp dir,SUTANDO_BRIDGE_LOCK=0,REMOTE_TASK_POLL_WAIT=1, 6 s per run.Parent
0f920a148:HEAD:
tok: []lists every env var name containingTOKENthat the command saw.Tests run
packages/ag2-sparrow/tests/test_*.pyandtools/test_no_drift.py: all pass.src/remote-gateway-bridge.test.py: the same 8 proactive-drain cases fail locally both at the parent and at HEAD (identicalFAILlines, diffed), so they come from this host's environment and are not caused by this change. CI is the reference.bash scripts/review-checks.sh --diff <(git diff origin/main...HEAD):PASS (hardcoded-paths + root-artifacts + prose-cap clean).Live path
This is the bridge's task-write path. The only live-path change is one call that returns at once when the setting is unset, which is the default and how every current deployment runs. Not done: a round trip against the real AG2 Space relay with the setting on. The end-to-end above uses a local mock relay. The live Sutando core was not restarted for this.
Stacked PR?
N/A.
Hardcoded-path check
No host paths were added. The test uses
tempfileandsys.executable.🤖 Generated with Claude Code