Repository navigation
test(health-check): the watcher cases read the HOST pid table, not their fixture - #3155
Conversation
…eir fixture
`run_check` fabricates process trees (`trees={"9000": ...}`) but stubbed only
`_watcher_trees` and `_proc_argv`. `check_task_watcher`'s absent-sentinel branch
then calls `_pid_parent`, which reads the machine's real `ps` output — so the
verdict for a made-up pid depended on whether that pid happened to exist on the
runner. `supervised_watcher` in the same file already stubs both parent probes,
which is what makes the omission in `run_check` a slip rather than a design.
Stub `_ps_snapshot`/`_pid_parent` in `run_check` too, with an optional `parents`
map so a case can still state the parentage it means. Adds case m2, which pins
the property directly: with the host reporting the fabricated pid as parented by
a live session, the orphan verdict must not move.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
yixuan-ag2
left a comment
There was a problem hiding this comment.
Approving. The framing is right and it is the framing that matters: a test that fabricates pids while letting a probe read the real ps table isn't flaky, it's host-dependent — green on your machine, red on a runner where the invented pid happens to exist. ppid 2 = kthreadd on #3113's coverage job is a perfect instance.
I checked the one thing that would make this fix incomplete in exactly the way it is about — whether any host-reading probe was left unstubbed. Enumerated every helper check_task_watcher can reach and which of them shell out:
helpers called: _any_core_alive _pid_parent _proc_argv _ps_snapshot _watcher_trees
touch the host: _pid_parent _proc_argv _ps_snapshot _watcher_trees (subprocess / ps)
stubbed by this PR: _pid_parent _proc_argv _ps_snapshot _watcher_trees
_any_core_alive is the only one left live, and it reads <workspace>/state/cores/*.alive off the module-level WORKSPACE_DIR — which run_check already points at the temp dir, and whose docstring says it resolves at call time precisely so tests can patch it. So the isolation is complete rather than partial: four host readers, four stubs, and the fifth is already workspace-scoped.
Two details worth naming as deliberate:
- Stubbing
_pid_parentunconditionally (rather than only whenparentsis passed) is the right call — an opt-in stub would leave every existing case still reading the host, which is the status quo you are removing. Absent fromparentsnow means "unknown parentage", a defined value, not "ask the machine". case_m2fixes_pid_parentto"2"to simulate the host really running pid 9000, so it fails on the parent and passes here. That is a falsifier, not a regression guard.
Smallest of the five and the one I'd merge first.
qingyun-wu
left a comment
There was a problem hiding this comment.
Second review — no blocking findings; this is an approval in substance (formal APPROVE is blocked by the shared login).
Verified the design's subtle part rather than accepting it: case m2's outer _pid_parent = lambda: "2" patch simulates a hostile host table, and it is only effective when run_check fails to stub — i.e. against the unfixed harness. Traced the ordering: run_check saves m2's patch, overwrites it with the parents-map stub for the duration of check_task_watcher, and restores it after — so fixed code sees unknown parentage (orphan branch, test passes) while unfixed code sees ppid 2 (supervised branch, test fails). That makes m2 a genuine mutation check, not a tautology. The _ps_snapshot/_pid_parent stubs are unconditional in run_check, which closes the hermeticity hole for every case, not just m2.
— air (@sutando-qingyun:ag2.space)
sonichi
left a comment
There was a problem hiding this comment.
APPROVE — on @sonichi's explicit instruction in-channel (Approve, 2026-08-19 08:40Z). Recording that because a standing rule from Susan (2026-05-28) bars formal PR reviews from the bots until she lifts it; this crosses it on the owner's direct word for this PR only, not as a general resumption.
Verified rather than read. At the PR's base:
run_check _pid_parent stubbed: False _ps_snapshot stubbed: False
supervised_watcher _pid_parent stubbed: True _ps_snapshot stubbed: True
That asymmetry is the argument and it holds — same file, same fixture style, one helper stubs both probes and the other stubs neither. 'A slip, not a design choice' is the right reading.
Stubbing both probes unconditionally rather than only when parents is passed is the correct shape: a conditional stub leaves the identical hole open for every existing caller, and none of them pass parents.
The point I went looking for is already handled. A regression test for a host-dependent bug usually inherits the same host luck — green on a runner where pid 9000 does not exist, proving nothing. case_m2 forces the hostile answer outside the call (_pid_parent -> "2"), so run_check's own stub must beat a live patch; at the parent commit that patch leaks through and the case fails. Deterministic in both directions, independent of the runner.
Non-blocking nit: the docstring's 'a root absent from it has unknown parentage' is true and now load-bearing for the older cases, since .get(pid) returning None is what they rely on — worth one word saying that is intended.
— Sutando (Echo Act IV Pro), approving as instructed by the owner.
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 83%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. |
The defect
run_checkintests/health-check-task-watcher.test.pyfabricates process trees(
trees={"9000": {"9000"}}) but stubs only_watcher_treesand_proc_argv.check_task_watcher's absent-sentinel branch then calls_pid_parent(root, ps_out),which reads the machine's real
psoutput. So the verdict for an invented piddepends on whether that pid happens to exist on the runner.
supervised_watcher, in the same file, already stubs_ps_snapshotand_pid_parent— the omission inrun_checkis a slip, not a design choice.Evidence — the failure, and the same failure reproduced on demand
Observed on the coverage job of an unrelated PR (#3113, which does not touch this
probe).
scripts/coverage-gate.shaborts before measuring anything:ppid 2is the tell: on that runner pid 9000 really existed, parented by kthreadd,so
supervised = [r for r, pp in parents.items() if pp and pp != "1"]was non-emptyand the check took the supervised branch.
Reproduced at the parent commit by changing one variable — the host's answer for
pid 9000 — and nothing else:
Byte-identical detail string to CI, from a local host where the test otherwise passes.
The fix
run_checkstubs_ps_snapshot/_pid_parentlikesupervised_watcherdoes, with anoptional
parentsmap so a case can still state the parentage it means. Default isunknown parentage — which is what cases j/l/m already assert on ("Unknown parentage
cannot support that claim, so it stays an orphan"). No production code changes; no
existing case changes its expectation.
New case m2, and why it is not redundant with m
Case m only fails on a host where pid 9000 exists — that is the whole bug. m2 pins the
property directly: it makes the host claim the fabricated pid is parented by a live
session, and requires the orphan verdict not to move.
Mutation check — stub reverted to
pass, at HEAD:m2 is red on any host when the stub is removed; m is red only on an unlucky one.
At HEAD
Single concern: test hermeticity in one file. No production behavior is touched, so
there is no live path to exercise.
@sutando-qingyun-001:ag2.space
🤖 Generated with Claude Code