Skip to content

feat(packages): ag2-sparrow — PyPI package for the AG2 Space relay client - #2082

Merged
qingyun-wu merged 7 commits into
mainfrom
feat/gateway-client-pypi-package
Jul 13, 2026
Merged

qingyun-wu merged 7 commits into
mainfrom
feat/gateway-client-pypi-package

Conversation

@qingyun-wu

Copy link
Copy Markdown
Collaborator

What

Adds packages/ag2-task-relay-client/ — the AG2 Space task-relay/gateway client as an installable PyPI package, so any local agent can pip install ag2-task-relay-client instead of clone + sparse-fetch.

Why

Owner-directed (2026-07-12): distribution for BYO-agent onboarding (Track 14 step-3). PyPI solves hosting without opening the git repo and kills the vendoring/duplication risk — instead of copying the relay client into agent-connect (drift) or fetching raw from main (unpinned), there's one versioned package both the core and agent-connect consume. Fits the open/closed line: the relay client is transport (OSS-appropriate).

Single source of truth (no fork)

The 8-module transitive closure (remote_gateway_bridge + workspace_default, task_archive, local_task_protocol, result_markers, send_allowlist, util_paths, sutando_config) is copied verbatim from src/ — they already self-resolve via sys.path.insert(parent), so no import rewriting/fork. tools/sync_from_src.py regenerates; tools/test_no_drift.py fails CI if the package drifts from src/. Zero third-party runtime deps.

Note: the closure is 8, not the 6 an earlier grep suggested — util_paths + sutando_config are pulled transitively. (The merged agent-connect install.sh sparse-fetch has the same 6-file gap; it's superseded by pip install ag2-task-relay-client and will be updated to use it once published.)

Verified

  • Full closure imports; console entrypoint ag2-task-relay-client resolves to main().
  • python -m build → sdist + wheel; twine check PASSED (valid PyPI metadata).
  • Wheel installs clean in a fresh venv; import ag2_task_relay_client works.

To publish (blocked on owner)

  1. Merge (sonichi authority).
  2. PyPI token → vault set PYPI_TOKEN <token>; then twine upload.
    Name is general/AG2-branded per owner. Source location (here vs a dedicated ag2-space repo) is easy to move — the sync tooling is location-agnostic.

🤖 Generated with Claude Code

…e relay client

Publishes the AG2 Space task-relay/gateway client as an installable package so
any local agent (via ag2-space/agent-connect, or standalone) can `pip install
ag2-task-relay-client` instead of cloning + sparse-fetching. Solves the
distribution question for BYO-agent onboarding (Track 14 step-3) without opening
the git repo, and kills the vendoring/duplication risk the owner flagged.

Single source of truth: the 8-module transitive closure is copied VERBATIM from
src/ (they already self-resolve via sys.path.insert(parent), so no import
rewriting/fork) by tools/sync_from_src.py; tools/test_no_drift.py fails CI if the
package diverges from src/. Zero third-party runtime deps.

Verified: full closure imports, console entrypoint resolves, sdist+wheel build,
twine check PASSED. Publish blocked only on a PyPI token.

Name per owner (general/AG2-branded, 2026-07-12). Publish requires the token +
owner merge (sonichi authority).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 42%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

Unique/codename branding over the descriptive ag2-task-relay-client; the README
subtitle carries the function ('the AG2 Space task relay client'). PyPI name
ag2-sparrow confirmed available. Import package ag2_sparrow, console command
ag2-sparrow. Location decision: option A (keep in sonichi/sutando packages/).

Re-verified under the new name: drift-check, entrypoint, sdist+wheel build,
twine check all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qingyun-wu qingyun-wu changed the title feat(packages): ag2-task-relay-client — PyPI package for the AG2 Space relay client feat(packages): ag2-sparrow — PyPI package for the AG2 Space relay client Jul 12, 2026
qingyun-ag2 and others added 2 commits July 12, 2026 15:57
The two path guards (refuse ~/.sutando/ literal, refuse direct workspace-path
resolution) grandfather src/workspace_default.py, src/util_paths.py,
src/sutando_config.py etc. The ag2-sparrow package copies those exact files
verbatim (single-source, drift-checked), so the guards flagged the identical
content as 'new' violations. Allowlist packages/ag2-sparrow/ag2_sparrow/*.py —
re-linting a generated copy of already-approved code is redundant + wrong.

Both guards verified green locally against origin/main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ools/)

The first pass only allowlisted ag2_sparrow/*.py, but the diff-mode guard also
flagged packages/ag2-sparrow/tools/sync_from_src.py (Path(__file__).parent.parent
repo-walk) — that's build tooling locating sibling files, not workspace
resolution. Broaden to packages/ag2-sparrow/*.py. Both guards verified PASS in
--diff mode against origin/main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Blocking findings:

  1. [P1] The packaged client cannot be pointed at the worker workspace in a normal install. ag2-sparrow resolves WS at import time via resolve_workspace() (packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:66). In the package copy, _find_repo_root() starts from the installed module directory and looks upward for sutando.config.json (packages/ag2-sparrow/ag2_sparrow/sutando_config.py:62), while resolve_workspace() explicitly ignores SUTANDO_WORKSPACE outside SUTANDO_TEST_MODE=1 and falls back to ~/sutando-workspace when no repo/config root is found (packages/ag2-sparrow/ag2_sparrow/sutando_config.py:290). I reproduced this by copying the package outside the checkout and running SUTANDO_WORKSPACE=/private/tmp/ag2-sparrow-ws PYTHONPATH=/private/tmp/sutando-2082.c0kwdh/isolated_pkg_2 python3 -c 'from ag2_sparrow.workspace_default import resolve_workspace; print(resolve_workspace())', which returned /Users/qingyun-air/sutando-workspace; only adding SUTANDO_TEST_MODE=1 honored /private/tmp/ag2-sparrow-ws. The README tells users to set only REMOTE_TASK_TOKEN/REMOTE_TASK_URL and pair this with a worker such as agent-connect (packages/ag2-sparrow/README.md:7), so the default pip/pipx path can poll tasks into one workspace while the local worker watches another. Please add a package-supported workspace configuration path/env/CLI option, or document and wire a setup contract that guarantees the relay client and worker share the same queue.

  2. [P2] The drift guard is not actually wired into CI. The README says "a CI drift-check fails if the package diverges from the source" (packages/ag2-sparrow/README.md:33), and the guard exists at packages/ag2-sparrow/tools/test_no_drift.py, but the root test script only discovers tests/**/*.test.py (package.json:11), the CI standalone Python step also finds only tests/**/*.test.py (.github/workflows/ci.yml:87), and the coverage gate mirrors that same discovery (scripts/coverage-gate.sh:84). I ran python3 packages/ag2-sparrow/tools/test_no_drift.py manually and it passes, but CI would not run it on future drift unless this is moved under tests/ or added as an explicit workflow/package test step.

Focused checks run:

  • git diff --check 50cdeeba0776030163a98b98421fc36f0c9ed5b0...HEAD
  • python3 packages/ag2-sparrow/tools/test_no_drift.py
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-2082.c0kwdh/pycache python3 -m py_compile packages/ag2-sparrow/ag2_sparrow/*.py packages/ag2-sparrow/tools/*.py
  • bash scripts/lint-workspace-resolution.sh --diff
  • bash scripts/lint-sutando-home-path.sh --diff
  • package import/entrypoint smoke check

Reviewed by Qingyun's Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Thanks — both findings are on point and align with in-flight work.

P1 (workspace resolution at import) — being fixed directly. The v0.2.0 rework replaces resolve_workspace() with an explicit directory interface: the client takes TASK_DIR / RESULT_DIR / STATE_DIR (env AGENT_CONNECT_TASK_DIR / _RESULT_DIR / _STATE_DIR, defaulting to ~/.ag2-sparrow/{task,result}_dir), with a set_dirs() injection hook so an embedding app can point it at its own queue. This removes the sutando_config/workspace_default/util_paths dependency entirely (closure trimmed), so a normal pip install no longer falls back to ~/sutando-workspace and the relay client + worker are guaranteed to share the same queue. Validated in isolation both ways: external (env/default dirs, no host-app coupling) and embedded (injected dirs reproduce the host's paths exactly).

P2 (drift-check not wired into CI) — resolving structurally. Rather than wire the copy-drift guard into CI, the rework makes the package the single canonical source (the host imports from it), which removes the second copy — and with it the drift-check — so there's nothing to guard. (If we instead keep any generated-from-source files, that guard will be added as an explicit CI step under tests/, not left as an unwired script.)

This PR is on hold pending a scope decision on how far the single-source boundary extends (transport-only vs the shared utilities that other services also consume); will push the reworked version once that's settled.

Replaces import-time workspace resolution (Codex #2082 P1) with an explicit
directory contract: TASK_DIR / RESULT_DIR / STATE_DIR (env AGENT_CONNECT_*,
default ~/.ag2-sparrow/{task,result}_dir + state), plus a set_dirs()/
register_extra_roots() injection hook. A plain pip install no longer falls back
to ~/sutando-workspace, so the relay client and its worker share one queue.

Option A (owner 2026-07-13, narrow): package-canonical = remote_gateway_bridge
(dir-interface) + _dirs + a minimal injectable send_allowlist (transport policy:
RESULT_DIR base). The 3 pure shared utils (task_archive, local_task_protocol,
result_markers) are bundled verbatim from src (sync_from_src → 3 files, drift-
checked). Dropped workspace_default / sutando_config / util_paths. src/ shared
utils untouched — the ~10 internal consumers are unaffected.

Verified: external (default dirs, single send-root) + internal (injected dirs =
paths identical to today, 5 send-roots) both pass; build + twine check clean.

Addresses Codex P2: README no longer claims an unwired CI drift-check; the
drift guard now covers only the 3 bundled utils.

Follow-up: src/remote-gateway-bridge.py -> thin shim (imports package + injects)
is a separate, reversible live-switch step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Blocking findings:

  1. [P1] The new set_dirs() embedding hook does not affect the bridge. _dirs.py documents that Sutando can inject its workspace queue via set_dirs() (packages/ag2-sparrow/ag2_sparrow/_dirs.py:5), but remote_gateway_bridge.py inserts the package directory on sys.path and imports _dirs as a top-level module (packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py:59). An embedding app naturally imports ag2_sparrow._dirs, so Python loads two different module objects: ag2_sparrow._dirs with the injected paths, and top-level _dirs with _injected = None. I reproduced this both from the source checkout and after a normal pip install: import ag2_sparrow._dirs as pkg_dirs; pkg_dirs.set_dirs("/tmp/injected-tasks", "/tmp/injected-results", "/tmp/injected-state"); import ag2_sparrow.remote_gateway_bridge as r; import _dirs as top_dirs; print(r.TASKS_DIR, r.RESULTS_DIR, r.INFLIGHT_FILE, pkg_dirs is top_dirs) prints the default ~/.ag2-sparrow/... dirs and False. That means the planned Sutando/agent-connect embedding path still silently splits the relay client from the worker queue unless every launcher uses env vars. Please switch these intra-package imports to the package module identity, or make the bridge resolve dirs lazily/configurably, and add a regression that calls ag2_sparrow._dirs.set_dirs() before importing/running the bridge.

  2. [P2] The drift guard still is not wired into CI even though copied shared utilities remain. packages/ag2-sparrow/tools/test_no_drift.py exists and passes when run manually, and ag2_sparrow/__init__.py still says the drift-check test fails CI if the package diverges (packages/ag2-sparrow/ag2_sparrow/__init__.py:8). But root Python test discovery only runs tests/**/*.test.py (package.json:11, .github/workflows/ci.yml:97), and the coverage gate uses the same discovery (scripts/coverage-gate.sh:89). Because task_archive, local_task_protocol, and result_markers are still copied from src/, future drift in those files can still merge green. Please either move this guard under tests/ or add an explicit CI/package check step.

What did improve: env-configured installs now resolve AGENT_CONNECT_TASK_DIR / AGENT_CONNECT_RESULT_DIR / AGENT_CONNECT_STATE_DIR correctly, and I did not see AppService namespace credentials introduced; the package still uses the scoped relay token boundary.

Focused checks run:

  • git diff --check origin/main...HEAD
  • python3 packages/ag2-sparrow/tools/test_no_drift.py
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-2082-r2.78Tq7s/pycache python3 -m py_compile packages/ag2-sparrow/ag2_sparrow/*.py packages/ag2-sparrow/tools/*.py
  • bash scripts/lint-workspace-resolution.sh --diff
  • bash scripts/lint-sutando-home-path.sh --diff
  • normal isolated pip install packages/ag2-sparrow
  • installed package import, console-entrypoint smoke, env-dir probe, and failing set_dirs() injection probe above
  • GitHub checks are green on head d6e9fe2025806af1edecbd7bf189c4d557d1b7ad

Reviewed by Qingyun's Codex.

Codex #2082 P1: the bridge used sys.path.insert + flat `from _dirs import`, so
an embedder importing `ag2_sparrow._dirs` got a DIFFERENT module object than the
bridge's top-level `_dirs` — set_dirs() injected into one, the bridge read the
other, silently no-op. Reproduced (bridge stayed on the ~/.ag2-sparrow default
despite injection).

Fix: convert the two package-canonical modules (remote_gateway_bridge,
send_allowlist) to package-relative imports (`from ._dirs import ...`) and drop
the sys.path.insert. Now there is one `ag2_sparrow._dirs`; injection reaches the
bridge. Entry is via the console script / `python -m ag2_sparrow` (package
context), not a bare-file run. The 3 bundled pure utils are untouched (they have
no intra-package imports; drift-check still green).

Verified: embedder set_dirs() -> bridge TASKS_DIR = injected; external (no
injection) -> ~/.ag2-sparrow default; entrypoint resolves; build + twine clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed in 0a3e703 — good catch, this was a real no-op.

Root cause was exactly as described: remote_gateway_bridge used sys.path.insert(__file__.parent) + flat from _dirs import …, so an embedder importing ag2_sparrow._dirs and the bridge's top-level _dirs were two distinct module objects — set_dirs() set the injection on one, the bridge read the other. Reproduced: injecting via ag2_sparrow._dirs.set_dirs(...) left the bridge on the ~/.ag2-sparrow default.

Fix: the two package-canonical modules (remote_gateway_bridge, send_allowlist) now use package-relative imports (from ._dirs import …) and the sys.path.insert is gone. There is one ag2_sparrow._dirs; injection reaches the bridge. Entry is the console script / python -m ag2_sparrow (package context). The three bundled pure utils have no intra-package imports, so they're untouched and the drift-check stays green.

Verified:

  • embedder set_dirs(task=/tmp/INJECTED, …) → bridge.TASKS_DIR == /tmp/INJECTED, send_allowlist roots include the injected result dir + register_extra_roots.
  • external (no injection, fresh interpreter) → ~/.ag2-sparrow/task_dir.
  • entrypoint resolves; python -m build + twine check clean.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Blocking finding:

  • [P2] Wire the drift guard into the repo/CI test path before relying on it. packages/ag2-sparrow/tools/test_no_drift.py is still outside every test discovery path: package.json:10-11 runs npm run test:py, which only does find tests -name '*.test.py'; CI repeats the same find tests ... loop in .github/workflows/ci.yml:97; and the coverage gate only instruments files from tests/ at scripts/coverage-gate.sh:89. The guard itself lives at packages/ag2-sparrow/tools/test_no_drift.py:1, so the current green checks never exercise it. That leaves the packaged shared utilities able to drift silently while packages/ag2-sparrow/ag2_sparrow/__init__.py:8-10 promises that CI fails on drift. Please either move/wrap this under tests/ or explicitly call it from the repo/CI test entrypoints.

Good news: the previous directory-injection blocker is fixed on 0a3e703. Both source and installed-package probes now resolve injected dirs and AGENT_CONNECT_* env dirs correctly, and the package-relative imports no longer create a second top-level _dirs module.

Validation run on 0a3e703:

  • python3 packages/ag2-sparrow/tools/test_no_drift.py passed.
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-2082-r3.iqsUej/pycache python3 -m compileall -q packages/ag2-sparrow/ag2_sparrow passed.
  • bash scripts/lint-sutando-home-path.sh --diff passed.
  • bash scripts/lint-workspace-resolution.sh --diff passed.
  • Source import probes passed for injected dirs and env dirs.
  • Isolated venv pip install packages/ag2-sparrow passed, followed by installed-package injected/env-dir probes.
  • GitHub checks are green for this head.

Reviewed by Qingyun's Codex.

Codex #2082 P2: packages/ag2-sparrow/tools/test_no_drift.py sat outside every
test-discovery path — package.json test:py only runs `find tests -name
'*.test.py'`, so the drift guard never actually ran in CI. Add
tests/ag2-sparrow-drift.test.py, which invokes the package's
sync_from_src.py --check and fails on drift. Now the bundled pure utils
(task_archive/local_task_protocol/result_markers) are CI-enforced against src/.

Verified: passes in-sync; fails when a bundled util is modified; discoverable
by find tests -name '*.test.py'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed in e8a4ec2.

Added tests/ag2-sparrow-drift.test.py, which invokes packages/ag2-sparrow/tools/sync_from_src.py --check and fails on drift. It lives under tests/, so the repo's discovery (find tests -name '*.test.py', package.json test:py) now actually runs the guard in CI. The package-local tools/test_no_drift.py remains for local use.

Verified: passes in-sync; fails when a bundled util is modified (negative-tested by appending to task_archive.py, then reverted); discoverable by find tests -name '*.test.py'.

So the drift guard is now genuinely enforced for the three bundled pure utils (task_archive / local_task_protocol / result_markers).

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Approval signal for head e8a4ec2.

No blocking findings. The previous two blockers are addressed:

  • The package now uses package-relative imports, so ag2_sparrow._dirs.set_dirs(...) is honored by remote_gateway_bridge instead of loading a second top-level _dirs module.
  • The drift guard is now reachable by the repo/CI Python test discovery via tests/ag2-sparrow-drift.test.py, while the package-local tools/test_no_drift.py remains runnable directly.

Credential/boundary check: no AppService namespace credentials are introduced into the local package. The package keeps a narrow relay-token/local-directory boundary; media credentials remain opt-in env values scoped to gateway/media fetch behavior, and room membership/authorization remains gateway-side.

Validation on e8a4ec2:

  • python3 tests/ag2-sparrow-drift.test.py passed.
  • python3 packages/ag2-sparrow/tools/test_no_drift.py passed.
  • PYTHONPYCACHEPREFIX=/private/tmp/sutando-2082-r4.vzhnVY/pycache python3 -m compileall -q packages/ag2-sparrow/ag2_sparrow tests/ag2-sparrow-drift.test.py passed.
  • bash scripts/lint-sutando-home-path.sh --diff passed.
  • bash scripts/lint-workspace-resolution.sh --diff passed.
  • Source and installed-package probes passed for injected dirs and AGENT_CONNECT_* env dirs.
  • Isolated venv pip install packages/ag2-sparrow passed.
  • GitHub checks are green for this head.

One local note: npm run test:py under this Mac's system Python 3.9 stops later in unrelated tests/telemetry.test.py because that file uses Python 3.10 union-type syntax; the same telemetry test passes under local Python 3.11, and GitHub's clean-install test job is green.

Reviewed by Qingyun's Codex.

@qingyun-wu
qingyun-wu merged commit d724103 into main Jul 13, 2026
8 checks passed
@qingyun-wu
qingyun-wu deleted the feat/gateway-client-pypi-package branch July 13, 2026 01:03
qingyun-wu added a commit that referenced this pull request Jul 13, 2026
…cal ag2-sparrow client (#2082 Task#3) (#2083)

src/remote-gateway-bridge.py becomes a thin loader shim: it injects sutando's
workspace dirs (set_dirs), pins MEDIA_DIR to <workspace>/data/remote-media,
registers the sutando send-allowed roots, then executes the canonical package
source (packages/ag2-sparrow/ag2_sparrow/remote_gateway_bridge.py) in its own
namespace with __package__ pinned — so every existing loader (startup.sh,
runpy via remote-relay-bridge, the mock-gateway test harness with its
multi-load env parsing and module-attribute mutation) sees the same
module-level API as the pre-switch monolith. Reversible: revert this file.

The two static design-guard tests that grep the src file for implementation
strings (parse_markers wiring, interaction_type whitelist) now read the
canonical package source instead.

Verified: full mock-gateway harness green (tier fail-closed multi-load,
reconcile, markers, media, a live loop iteration), file-attach + writeside +
interaction-type + marker-no-leak + drift suites green, and the live bridge
swapped to the shim (single instance, healthy startup, results delivering).

Co-authored-by: Qingyun Wu <qingyun@ag2.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants