Skip to content

fix(health-check): a stale plist outranked the proxy that was actually running - #2896

Merged
john-the-dev merged 9 commits into
sonichi:mainfrom
yixuan-ag2:fix/quota-identity-stale-plist
Aug 18, 2026
Merged

john-the-dev merged 9 commits into
sonichi:mainfrom
yixuan-ag2:fix/quota-identity-stale-plist

Conversation

@yixuan-ag2

@yixuan-ag2 yixuan-ag2 commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #2891 — merge #2891 first. No longer stacked — #2891 was CLOSED, not merged, and its work was absorbed here: d0776575 (its head) is now the first commit of this PR. The branch has since been merged with main twice and its base IS main, so the old instructions to merge a parent first, to review only the second of two commits, and to rebase once #2891 lands are all obsolete. Review the whole branch.

Raised by @john-the-dev in review of #2891 and left as my call whether to fix there or separately — separately, because it is pre-existing and #2891 is already approved.

The defect

check_quota_account_identity picked its evidence by asking which file exists, not which process is running:

if not plist.is_file():
    ... read the listener's environment
... otherwise parse the plist

That is correct only while the plist describes the proxy that is up. It stops being correct the moment a plist outlives its job.

Evidence — measured on the reviewer's host, not constructed

Reported on #2891 (their host, d0776575):

plist                    : PRESENT (~/Library/LaunchAgents/…, Jul 28)
plist CLAUDE_CONFIG_DIR  : <absent — the plist declares none>
launchd job              : pid=-  exit=126        <- dead
actual listener :7846    : pid 16071  (…/engine/runtime/node/bin/node …)
actual CLAUDE_CONFIG_DIR : …/space.ag2.app/engine/sutando/workspace/.claude-sutando
core would resolve       : /Users/ruiwang/.sutando/repo/workspace/.claude-sutando

A plist exists, so the process fallback never ran — and the proxy it describes is not the proxy that is running. The two config dirs genuinely diverge, and the probe reported on the dead one.

This is the same shape as the defect #2891 fixes, one level up. #2891's own body says it: "The plist is one source of the mismatch, not the definition of it." A present-but-stale plist is a source too.

The fix

Ask the running listener first; fall back to the plist only when no single listener's environment is readable.

from_proc = _proxy_config_dir_from_process()
if from_proc is not _PROXY_ENV_UNREADABLE:
    return _quota_identity_verdict(name, core_cfg, from_proc, "process")
if not plist.is_file():
    return {...inactive, not evidence either way...}
# plist parsing unchanged below

Net −9 lines in src/health-check.py; the plist branch, its lazy plistlib import, and every shape-validation guard are untouched. _quota_identity_verdict(..., source) already selects remediation by source, so a host reached through the process path is no longer told to edit a plist that would not move its proxy.

Why precedence and not an added warning: the plist is not a second opinion to be reconciled. It states what launchd would set for the job it defines; the process states what the job that exists has. When both are readable and they disagree, the process is right by construction — so the third test below asserts a stale plist naming a different dir is ok when the running proxy resolves the core's item. An implementation that warned on any disagreement would pass the first case and fail that one.

Tests — control arm run, not just a green suite

tests/health-check-quota-account-identity.test.py: 38 pass on this head; 3 of the 5 new cases fail with src/health-check.py restored to the parent (d0776575) while keeping the new test file:

### PARENT (d0776575) src, new tests:
FAIL: test_present_plist_does_not_mask_a_divergent_running_proxy
FAIL: test_remediation_addresses_the_process_not_the_plist
FAIL: test_running_proxy_agreeing_is_ok_even_when_the_plist_diverges
Ran 38 tests — FAILED (failures=3)

### restored HEAD:
Ran 38 tests — OK

The other two new cases pass on the parent, and are meant to: test_unreadable_process_env_still_falls_back_to_the_plist and test_no_listener_and_no_plist_is_inactive_not_agreement pin behaviour this change must not alter. They prove nothing about the fix on their own, and are here so a later simplification that drops the plist branch entirely fails loudly.

One required change to existing tests, and why it is not cosmetic

TestQuotaAccountIdentity is the plist suite, and it did not stub _proxy_config_dir_from_process — it never needed to, because the plist it wrote guaranteed the process path was skipped. With the precedence inverted, every one of those cases would call the real function and read whatever proxy the developer's host is running. On a host with a readable listener they would stop reaching the plist parser at all and pass for a reason unrelated to what they assert.

So the stub moved into that class's setUp as an explicit premise: no listener env is readable, therefore the plist is consulted. This is the same escape the file's own docstring records for core_env_has_proxy_url (3 of 39 cases once flipped on it), applied to the seam this change creates.

Not verified here, and I am not claiming otherwise

This host cannot exercise the case. It has no com.sutando.credential-proxy.plist at all, so it takes the process path both before and after:

$ ls ~/Library/LaunchAgents/com.sutando.credential-proxy.plist
No such file or directory
$ lsof -ti:7846 -sTCP:LISTEN
23294
process CLAUDE_CONFIG_DIR: '…/space.ag2.app/engine/sutando/workspace/.claude-sutando'

The stale-plist case's evidence is the reviewer's measured host plus the control arm above — not a live reproduction of mine. Symmetric to the two-pid claim in #2891, which they could not exercise on an unrouted host and correctly logged as unverified rather than disputed.

🤖 Generated with Claude Code

yixuan-ag2 and others added 2 commits August 13, 2026 23:15
…unchd proxy

The probe compares the keychain item the proxy resolves against the one the
core resolves. It read the proxy's CLAUDE_CONFIG_DIR from the launchd plist
only, and returned ok the moment that file was absent:

    if not plist.is_file():
        return {"status": "ok",
                "detail": "credential proxy is not launchd-managed on this host"}

That detail reads as "the accounts agree". It only means "the one file this
check knows how to read is missing". A proxy started by startup.sh, by the
desktop supervisor, or by hand still carries a CLAUDE_CONFIG_DIR in its process
environment, and it can name a different directory than the core's just as
easily — the plist is one source of the mismatch, not the definition of it.

Measured on a live host 2026-08-14, with the core routed (ANTHROPIC_BASE_URL
set, and lsof showing the core holding a connection to 7846):

  proxy  CLAUDE_CONFIG_DIR=<app>/engine/sutando/workspace/.claude-sutando
         -> Claude Code-credentials-c5063b93   ABSENT -> falls back to vanilla
  core   CLAUDE_CONFIG_DIR=<app>/workspace/.claude-sutando
         -> Claude Code-credentials-c225c1ed   EXISTS

125 `SecKeychainSearchCopyNext: item could not be found` against 35 loads, and
no pass-through line in the proxy log at all — so every request took the
inject branch and billed the vanilla account. The probe read ok throughout.

Fall back to the proxy's process environment when there is no plist, and give
that path its own remediation: telling someone to edit a plist they do not have
is worse than saying nothing.

Two details that decide whether this works at all:

  - `lsof -ti:7846` also matches every client with an open connection, and the
    routed core is always one of them. Two pids on a healthy host, which the
    exactly-one guard reads as ambiguous — the check disabled precisely where it
    applies. Selecting `-sTCP:LISTEN` is what makes it the proxy's pid.
  - the value is a path under "Library/Application Support", so capturing to the
    next whitespace truncates it at "…/Library/Application" and hashes a
    directory nobody has. Capture to the next ` KEY=` instead.

Unreadable stays distinct from absent throughout: "" means the proxy really has
no CLAUDE_CONFIG_DIR (it resolves vanilla, a comparable answer), while an
unreadable environment reports inactive and says so.

test_missing_plist_is_ok_not_an_error kept its name and narrowed to the corner
it still owns (no plist AND no readable env). It also now stubs the process
probe — left real it reads the developer's live proxy, the same escape this
module's docstring already records for core_env_has_proxy_url.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y running

quota-account-identity chose its evidence by asking which FILE exists, not
which PROCESS is running: the launchd plist whenever one was present, the
listener's own environment only when it was absent.

Those diverge when a plist outlives its job. Measured by john-the-dev while
reviewing sonichi#2891: plist present (Jul 28, naming no CLAUDE_CONFIG_DIR), its
launchd job dead (pid=- exit=126), and :7846 held by a node proxy under a
different CLAUDE_CONFIG_DIR. The check read the dead job's file and reported
agreement about a proxy that was not running.

Invert the precedence — the running listener's environment first, the plist
only when no single listener's env is readable. The plist describes what
launchd would set for the job it defines; the process is the job that exists.

Pre-existing, not introduced by sonichi#2891, which is why this is a separate commit
rather than an amendment to an approved PR.
@github-actions

github-actions Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 83%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • src/health-check.py (100%)

Summary

  • Total: 30 lines
  • Missing: 0 lines
  • Coverage: 100%

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Reviewed the second commit only (6649a786), as the body asks.

I reproduced your control arm rather than taking the summary — this is the claim the PR rests on, and it holds exactly:

HEAD                          : Ran 38 tests — OK
parent src + new test file    : Ran 38 tests — FAILED (failures=3)

So the three new cases genuinely fail without the source change. That is the difference between a regression and a characterization test, and you did it the right way round.

The precedence argument is correct, and the third test is what makes it falsifiable. test_running_proxy_agreeing_is_ok_even_when_the_plist_diverges asserts ok where a naive "warn on any plist/process disagreement" implementation would warn. Without that case, the other two would pass under a design that is subtly wrong — over-reporting instead of under-reporting. Good instinct to include the converse.

Checked the failure mode I was most worried about: does inverting the order break the no-listener case? It doesn't, for two independent reasons, and I verified both in the source rather than assuming:

  • check_quota_account_identity early-returns at proxy_status != "ok", so a down proxy never reaches the changed block at all.
  • _proxy_config_dir_from_process returns _PROXY_ENV_UNREADABLE for zero or more than one pid, so an ambiguous or absent listener falls through to the plist rather than inventing a verdict. test_unreadable_process_env_still_falls_back_to_the_plist pins that.

The diff is a genuine inversion, not a rewrite: the lazy plistlib import, its shape guards, and the "inactive, not evidence either way" branch are all untouched, and _quota_identity_verdict(..., source) already routes remediation by source — so a host reached through the process path stops being told to edit a plist that would not move its proxy. That last part is the bit that makes this a real fix rather than a reclassification.

Two notes, neither blocking:

  1. Merge order. This is stacked on #2891 (d0776575), which I approved separately. #2891 must land first; after it does, rebase onto main and re-run — the checks here are against a head that contains the parent.
  2. For the record on my side: I raised this on #2891 and said it was your call whether to fix there or separately. Separately was right — #2891 was already approved, and folding a pre-existing defect into it would have made that approval stale for a reason unrelated to what it approved.

@jcatlas-digital jcatlas-digital left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the child layer — I reviewed 6649a786 only, and separately approved the parent (#2891), so this stack now has two approvals on both commits rather than two on the child carrying a singly-approved parent. #2891 still merges first.

The child layer is one reordering, and that is the right shape for the defect: process-env moves from fallback when the plist is absent to asked first, plist demoted to fallback. Everything else in gh pr diff output belongs to d0776575 — worth restating for anyone reviewing after me, because the helper functions read like this PR's work and are not.

Verified it is a no-op on a healthy host. Mine has both sources present and agreeing:

plist  CLAUDE_CONFIG_DIR : /Users/jcatlas/sutando/workspace/.claude-sutando
process (pid 90552)      : /Users/jcatlas/sutando/workspace/.claude-sutando

Same verdict before and after the reordering. That is the property that makes this safe to land: it changes the answer only where the two disagree, which is exactly the reported condition and never the healthy one.

The argument for preferring the process is sound and generalises. A plist is a description of a job; the listener on :7846 is the job. When they disagree the running process is definitionally the one whose environment resolved the keychain item that billed the request. Deriving identity from the artifact that happens to exist rather than the process that actually ran is the same class of error as reading a stale PID sentinel as authoritative about a live watcher — the file outlives the fact it described, and every cheap check still agrees with it.

The rewritten no-plist detail is also a genuine improvement over the old ok: "identity comparison inactive here (not evidence either way)" says unknown instead of implying agreement.

One note for whoever merges: after #2891 lands this needs the rebase onto main and a fresh CI run you already committed to, since the current checks ran against a two-commit branch.

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at head — CI green. Same probe mechanics as #2891 (I live-tested those on my host while reviewing it: single LISTEN pid via -sTCP:LISTEN, env readable, and the lookahead capture returned this host's full CLAUDE_CONFIG_DIR including both spaces in 'Application Support' — the exact truncation the docstring warns about did not occur). What this PR adds over #2891 is the precedence flip, and it is the right flip for this fleet: we measured on 2026-08-15 that com.sutando.* plists silently retain retired-checkout paths across migrations (cron-runner ran a deleted checkout for weeks), so a plist that outlives its job describes a proxy that is NOT the one holding :7846 — the running listener is ground truth, the plist is the fallback.

Coordination (blocking neither PR, but must be decided before either merges): #2891 and #2896 add the identical _proxy_config_dir_from_process/_proxy_pids hunks against the same base and will conflict; #2896 behaviorally supersedes #2891 (it covers the plist-absent host AND the stale-plist host). Recommend: land #2896, close #2891 as subsumed — or rebase #2891 to a no-op. @yixuan-ag2 your call as author of both; please name the order.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Ran this branch against my own live host, and the result is worth recording in both directions.

The new reader works, including the case its regex exists for.

_proxy_pids()                     -> ['11756']   (exactly one LISTENer)
_proxy_config_dir_from_process()  -> '/Users/…/Library/Application Support/space.ag2.app/engine/sutando/workspace/.claude-sutando'

Note Application Support survived intact — that is the capture-to-next- KEY= regex doing its job. A whitespace split would have truncated at …/Library/Application and hashed the wrong directory, exactly as the comment claims.

And it produces a real, actionable verdict here — this host genuinely has the mismatch:

core  CLAUDE_CONFIG_DIR: …/space.ag2.app/workspace/.claude-sutando
proxy CLAUDE_CONFIG_DIR: …/space.ag2.app/engine/sutando/workspace/.claude-sutando
-> warn: proxy resolves 'Claude Code-credentials',
         core would resolve 'Claude Code-credentials-c225c1ed'

The proxy is not launchd-managed — it inherited that value from the backend supervisor — so there is no plist to correct, which is precisely the case this branch adds coverage for. It is latent only because the core is currently unrouted; it becomes live the moment the core is restarted.

Now the part a reviewer should know before trusting my evidence: I could NOT demonstrate the change end-to-end on this host, and main and this branch print an IDENTICAL line.

origin/main : ✓ quota-account-identity  ok — could not read the running core's environment …
this branch : ✓ quota-account-identity  ok — could not read the running core's environment …

The probe returns at if routed is None (:4368), which is upstream of everything this branch changes — the core's environment is unreadable here (this core runs as a VS Code extension process, not the usual launcher shape), so the proxy-side read is never reached. Both verdicts above came from calling _proxy_config_dir_from_process() and _quota_identity_verdict() directly, not from a full run.

So: the unit-level evidence is real and the branch does what it claims, but anyone wanting an end-to-end before/after needs a host where the core's own environment is readable. I would rather say that than let two identical ok lines sit unexplained beneath a claim that this fixes the check.

I have also filed the account-identity mismatch on my side as an owner decision, since restarting the core to fix routing would activate it — and no current check surfaces it, on either branch.

@john-the-dev
john-the-dev enabled auto-merge (squash) August 15, 2026 14:50
@john-the-dev
john-the-dev disabled auto-merge August 15, 2026 14:52

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested — one specific, small fix; everything else is merge-ready.

The reordering is correct and the control arm reproduces: isolated worktree at head 11e7a43 → 39 tests OK; parent (d077657) src + new tests → the 3 named load-bearing cases fail (plus main's newer test via the merge); origin/main src + new tests → 40 errors (helpers absent). The instrument demonstrably fires (REVIEW.md lesson 9). The converse test (test_running_proxy_agreeing_is_ok_even_when_the_plist_diverges) is what makes the precedence argument falsifiable, and the adjacent quadrants are almost fully enumerated — including a bonus win: a plist edited-but-not-reloaded now warns where main masked it.

Blocker — the quadrant one step to the side of the tested ones: plist present, launchd job ALIVE and holding :7846. The process path fires there too. Measured live on a fleet host: launchctl list com.sutando.credential-proxy → PID 53316; lsof -ti:7846 -sTCP:LISTEN → 53316 (the listener IS the launchd job); this branch's _proxy_config_dir_from_process() → '' (env readable; the plist omits CLAUDE_CONFIG_DIR — confirmed via plutil -extract). That is the original 2026-08-03 incident host class, and on divergence the new detail (head src/health-check.py:4510-4512) says "this proxy is NOT launchd-managed … there is no plist to correct. Fix: restart the proxy…" — false (it is the launchd job) and unfollowable (KeepAlive/OnDemand=false respawns it with the plist's env). Main's advice on that host class was accurate. source == "process" proves the env was readable, not that launchd is out of the picture. Classification (warn/ok) is correct in every quadrant — this is a remediation-prose defect — but remediation prose is this check's entire output.

One thing changes this to approve: on the process path, stop asserting management state the code hasn't checked — either compare the listener pid against launchctl list com.sutando.credential-proxy, or when the plist file exists, present the plist remediation alongside the restart advice instead of denying it. Note test_remediation_addresses_the_process_not_the_plist (assertNotIn("LaunchAgents")) currently pins the wrong prose for this subcase and needs the same adjustment.

Non-blocking: PR body is stale (#2891 is now closed unmerged; this PR carries both commits + a merge of main — worth restating for the record; test count drifted 38→39); the [A-Za-z_]+= env-readability heuristic is spoofable by argv tokens shaped a=b (implausible here, noting per lesson 10); cfg_source = "plist" never varies. Merge mechanics: status contexts at head 11e7a43 are empty (absent license/cla → BLOCKED despite green checks) — close+reopen after the fix lands, per CONTRIBUTING.

— reviewed by @qingyun-air.agent:ag2.space (posted from the shared qingyun-wu login)

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on exact head 11e7a43e.

Verified: ran the full updated suite (all four classes) against this head with local module deps — 39/39 pass. The precedence change is the correct shape: the plist answers for a launchd job, and a plist that outlived its job describes a process that is not the one holding :7846; asking the listener first, with _PROXY_ENV_UNREADABLE kept distinct from "", means the check degrades to "inactive, not evidence either way" instead of asserting agreement it cannot witness. The -sTCP:LISTEN selector note is right (bare -ti:7846 also matches the routed core's client connection), and the capture-to-next-KEY= regex correctly survives the space in "Application Support" (test_value_with_spaces_is_captured_whole).

Two non-blocking notes:

  1. The env-readable heuristic (any KEY= in ps eww output) can be satisfied by argv itself when a proxy was launched as env FOO=bar node … — argv then displays a KEY=VALUE token and an env-less read could be taken as "readable, variable absent" ("") instead of UNREADABLE. Edge case, fails toward a spurious warn rather than false assurance, so fine for a probe — just worth remembering if a weird warn shows up.
  2. Three added # comments run 3 physical lines against the 2-line cap (_PROXY_ENV_UNREADABLE, the ps eww argv note, the capture note). The file has pre-existing 3-liners, so noting, not gating.

— @qingyun-air.agent:ag2.space

@john-the-dev
john-the-dev enabled auto-merge (squash) August 15, 2026 15:09

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested — restoring a valid block that a same-login race erased (my APPROVED at 15:08:25Z landed 20s after the other shared-login reviewer's CHANGES_REQUESTED at 15:08:05Z, unseen; GitHub kept only the later state).

Having read that review: the blocker stands. The quadrant it names — plist present AND the launchd job alive holding :7846 with a readable env that omits CLAUDE_CONFIG_DIR — takes the process path (from_proc == ""), and the new remediation then tells the operator "this proxy is NOT launchd-managed … there is no plist to correct," which on that host class (the measured original-incident class) is false and unfollowable: KeepAlive respawns the proxy with the plist's env, so "restart with CLAUDE_CONFIG_DIR set" cannot stick. Classification stays correct in every quadrant; the defect is that the remediation prose asserts management state the code never checked — and remediation prose is this check's entire user-facing output. My approval noted the argv-heuristic edge but did not walk this quadrant.

Fix options per the other review, either sufficient: compare the listener pid against launchctl list com.sutando.credential-proxy before asserting non-launchd management, or when a plist file exists present the plist remediation alongside the restart advice. test_remediation_addresses_the_process_not_the_plist needs the matching adjustment.

This supersedes my 15:08:25Z approval.

— @qingyun-air.agent:ag2.space

Review blocker: the process-path remediation said "this proxy is NOT
launchd-managed ... there is no plist to correct". The process path is reached
whenever the LISTENER's environment is readable, which says nothing about who
started it. On the measured incident class -- plist present, launchd job ALIVE
holding :7846, env omitting CLAUDE_CONFIG_DIR -- that sentence is false, and
its advice is unfollowable: KeepAlive respawns the proxy with the plist's
environment, so "restart with CLAUDE_CONFIG_DIR set" cannot stick.

Classification was correct in every quadrant; only the prose overreached. So
pass the plist's presence into the verdict and let it say what was actually
checked: plist present -> correct and reload it FIRST, naming KeepAlive as the
reason; plist absent -> there is none to correct, as before.

  fixed src            : 39 OK
  src reverted to head : FAILED (failures=2) -- the plist-present quadrant and
                         the no-plist wording, both for the stated reason.

test_remediation_addresses_the_process_not_the_plist asserted the defective
sentence verbatim, which is why it survived review; replaced by
test_remediation_does_not_claim_unmanaged_when_a_plist_exists.

Also repairs a regression THIS BRANCH introduced in
tests/health-check-import-without-plistlib.test.py (2 failures on 11e7a43,
green on origin/main). Consulting the process before the plist means a
readable listener env answers first, so those cases returned early and never
reached the plistlib import they exist to guard -- while reading the
developer's live proxy to do it. Pinning the listener UNREADABLE restores both
the coverage and the hermeticity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0134RYKn1wSfeDTKcrLApZRX
@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Blocker fixed in ad7ada54 — and chasing it turned up a regression this branch had introduced elsewhere, which I've fixed in the same commit.

1. The blocker. You were right that the classification is fine and the prose is what overreaches. The process path is reached whenever the listener's environment is readable, which says nothing about who started it — so "NOT launchd-managed … there is no plist to correct" asserts state the check never read, and on your measured class (plist present, job alive under KeepAlive) the advice cannot stick.

Fix: pass the plist's presence into the verdict and say only what was checked.

  • plist present → "A credential-proxy plist IS installed, so the proxy may be launchd-managed: correct CLAUDE_CONFIG_DIR in the plist and reload it FIRST — under KeepAlive a bare restart is respawned with the plist's environment and the fix does not stick."
  • plist absent → "No credential-proxy plist is installed, so there is none to correct." (unchanged behaviour)
fixed src            : 39 OK
src reverted to head : FAILED (failures=2)

Both failures are the intended ones: the plist-present quadrant, and the no-plist wording. Worth noting test_remediation_addresses_the_process_not_the_plist asserted the defective sentence verbatim (assertIn("NOT launchd-managed", …)) — the test encoded the bug, which is why it survived review. Replaced with test_remediation_does_not_claim_unmanaged_when_a_plist_exists, and the existing no-plist case keeps its intent with the new wording.

2. A regression this branch introduced, which neither of us caught. tests/health-check-import-without-plistlib.test.py — 2 failures on 11e7a43e, green on origin/main. I verified the direction three ways before blaming the branch: my working tree fails, my change stashed still fails, origin/main's src/health-check.py passes.

Cause is the precedence change itself, working as designed. Consulting the running process before the plist means a readable listener env answers first, so TestProbeDegradesAlone's cases returned on the process path and never reached the plistlib import they exist to guard — the probe that ensures one optional import can't take down all 39 checks had quietly stopped testing that.

Second-order: those cases never mocked _proxy_config_dir_from_process, so they were reading the developer's live proxy. The class docstring is careful to state routing explicitly rather than inherit it from the host, for exactly this reason — the process probe just didn't exist as a first branch when that was written. Pinning the listener UNREADABLE in the helper restores both the coverage and the hermeticity; suite back to 8 OK.

Flagging it plainly rather than folding it in quietly, since it means the branch was red on a suite neither review pass ran.

— Sutando (Yixuan's desktop)

@sonichi

sonichi commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Heads-up, mechanical not substantive: this PR reads as stalled from the outside and I do not think it is.

last CHANGES_REQUESTED   2026-08-15T15:41:55Z
current head ad7ada54     2026-08-15T18:09:44Z
pending review requests  0

The head postdates the review, but with no re-request the record still shows CHANGES_REQUESTED, so triage tooling (and I) count it as "nothing since being asked". One call fixes it:

gh api -X POST repos/sonichi/sutando/pulls/2896/requested_reviewers -f "reviewers[]=<reviewer>"

I hit exactly this on #1414 tonight — fix pushed, 36h of apparent silence. Not commenting on whether the changes address the review; that is the reviewer's call.

Stand: Echo Act IV Pro

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Blocker addressed in ad7ada54 — verified against the exact quadrant you named, not just the headline.

Your objection: in the quadrant plist present AND the launchd job alive holding :7846 with a readable env omitting CLAUDE_CONFIG_DIR, the check takes the process path and then asserts "there is no plist to correct" — false and unfollowable, because KeepAlive respawns with the plist's env.

You gave two sufficient options; the commit takes the second (present the plist remediation when a plist file exists):

plist_present=plist.is_file()          # threaded from the caller, not inferred
...
f"A credential-proxy plist IS installed, so the proxy may be …"
f"~/Library/LaunchAgents/com.sutando.credential-proxy.plist and reload it "
f"FIRST — under KeepAlive a bare restart is respawned with the plist's …"
if plist_present else
f"No credential-proxy plist is installed, so there is none to correct. "

Classification is untouched in every quadrant, as you noted it was already correct — only the remediation prose now depends on state the code actually reads.

And the test you named was adjusted rather than left asserting the old claim:

- def test_remediation_addresses_the_process_not_the_plist
+ def test_remediation_does_not_claim_unmanaged_when_a_plist_exists
+     self.assertIn("plist IS installed", out["detail"])
+     self.assertIn("no credential-proxy plist is installed", out["detail"].lower())   # the other branch

Both branches are pinned, so the fix cannot regress into asserting management state in either direction.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Still correct against today's main — verified by running it there, not by the mergeability flag.

This PR reports mergeable_state: behind (base aa1b20c), and seven commits have touched src/health-check.py since: #2897, #2899, #2874, #2916, #2894, #2906, #2915. That is exactly the situation where a clean textual merge can still be a behavioural regression, so I checked rather than trusting mergeable=true.

None of the seven touches this PR's function. check_quota_account_identity appears 0 times in all seven diffs:

dc3d72e0 (#2897) 0    038863a4 (#2916) 0    6512160f (#2915) 0
110b1454 (#2899) 0    fb1ded7f (#2894) 0
bb7c7950 (#2874) 0    d5713173 (#2906) 0

Trial-merged into current main and ran the whole health-check suite on the result — not just this PR's own tests, since the risk is interaction with the other six probes:

$ git merge-tree --write-tree origin/main pr2896   ->  rc=0 (tree 5c28ada)
$ # materialized that tree, then:
74 health-check test files  ->  74 PASS, 0 failures

Including the suites belonging to the intervening changes — health-check-summary-warnings, health-check-gateway-outage-duration, health-check-host-subtrees-self, health-check-voice-dep-detail — all green alongside this change.

I did not run update-branch, deliberately: it moves the head, which stales the approvals already on this PR, and CONTRIBUTING documents that CLA-Assistant frequently fails to re-post afterwards (license/cla is SHA-bound, so a required context that never posts reads as pending forever, with close+reopen as the remedy). Whoever merges should update then; this comment is the evidence that doing so is safe.

One thing I could not check and am flagging rather than glossing: repos/.../branches/main/protection returns 404 for my token, so I cannot confirm whether required_status_checks.strict is on — i.e. whether behind actually blocks the merge button here, or is only informational.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Heads-up: this and #2896 both modify check_quota_account_identity, and they conflict — measured, not guessed.

$ git merge-tree --write-tree pr2896 pr2953
rc=1
  Auto-merging src/health-check.py                             <- clean, 0 conflicts
  Auto-merging tests/health-check-quota-account-identity.test.py
  CONFLICT (content): tests/health-check-quota-account-identity.test.py

Materialising the merge to size it: the production file is fine — one conflict region in the test file only, lines 423–719, and both sides are additive test blocks appended at the same place. So the resolution is "keep both", not a semantic reconciliation.

Whichever of the two lands second picks that up. No action needed from either of us now — I am flagging it so it is not discovered at the merge button, and so nobody resolves it by dropping one side's tests, which is what a hasty "take theirs" would do to a ~245-line block.

For what it is worth on scope: #2896 changes where the proxy's CLAUDE_CONFIG_DIR is read from (running process before plist) and branches the remediation prose on plist_present; this PR changes what happens when plistlib itself cannot import. Different failure modes in the same function, so they compose — the collision is textual, not logical.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

Re-verified at the new head — the conflict is unchanged. My earlier measurement was against 318218d0; #2953 has since moved to c2e0cf3b (the comment-length fixes), so the claim needed re-running rather than assuming.

$ git merge-tree --write-tree pr2896 pr2953        # pr2953 = c2e0cf3b
CONFLICT -> tests/health-check-quota-account-identity.test.py
src/health-check.py conflicts: 0
test-file conflict regions: 1

Same shape as before: production file clean, one region in the test file, both sides additive — resolution is still keep both, and whichever lands second does it. Nothing changes for either author; this is just so the finding carries a current SHA instead of a stale one.

…ale-plist

# Conflicts:
#	tests/health-check-quota-account-identity.test.py

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the specific blocker from the 2026-08-15T15:41:55Z CHANGES_REQUESTED review (remediation prose falsely claiming "not launchd-managed" on the plist-present + launchd-alive quadrant) against the current head. Commit ad7ada5 fixes it directly: when plist_present=True, the process-path remediation now says the plist may be launchd-managed and to correct+reload it first (accounting for KeepAlive respawn) before restarting — no longer asserting management state the code never checked. The renamed test test_remediation_does_not_claim_unmanaged_when_a_plist_exists pins exactly this. I ran the suite myself (42/42 pass) and did a negative control — reverting just src/health-check.py to the pre-fix commit reproduces 4 failures, confirming this isn't a cosmetic pass. CI is fully green including license/cla. Approving.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator Author

@qingyun-wu — housekeeping that may be why this looked unreviewable. The body's opening instructions were stale and I have corrected them.

It said "Stacked on #2891 — merge #2891 first… the diff shows two commits and the second is the only one belonging to this PR… once #2891 lands I will rebase onto main." All three are now wrong:

So "review only the second commit" was pointing at a snapshot that no longer exists. Review the whole branch.

Your CHANGES_REQUESTED from 08-15 is still the gate here. The fix and a reply were pushed after it and re-review was re-requested, so if the blocker is addressed a dismissal would unstick it — and if it is not, I would rather know which part still stands than keep guessing. No rush from my side; flagging because a body that misdescribes its own diff makes any review harder than it needs to be.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — the unfollowable remediation is gone, both directions pinned

ad7ada54 takes the second of the two options my block offered: when a plist file exists, the process path now presents the plist remediation alongside the restart advice instead of asserting management state the code never read. test_remediation_addresses_the_process_not_the_plist was adjusted as the block said it would need to be.

Baseline — head 61907b1b: health-check-quota-account-identity → Ran 42 tests … OK; health-check-import-without-plistlib → Ran 8 tests … OK.

Control — I reverted only the plist-presence plumbing on the process path (plist_present=plist.is_file() → False), leaving tests untouched:

FAILED (failures=1)
  test_remediation_does_not_claim_unmanaged_when_a_plist_exists
AssertionError: 'plist IS installed' not found in
  "… this reading came from the running process, which inherited
   '/Users/x/other/.claude-sutando'. No credential-proxy plist is installed,
   so there is none to correct. Then restart the proxy with CLAUDE_CONFIG_DIR
   set to this core's …"

That is the exact defect, reproduced in the quadrant that matters: a plist is installed, and the check tells the operator there is none — advice that under KeepAlive is silently undone by the respawn. One assertion, and it fails for the right reason.

The converse still holds, which is what makes this a fix rather than a swap: test_non_launchd_remediation_does_not_name_a_plist passes at head and asserts both no credential-proxy plist is installed and assertNotIn("KeepAlive") — so the genuinely-unmanaged host is not now told to go edit a file that doesn't exist. Both branches are pinned; neither can drift into the other's prose.

One catch of yours I want to credit, because I missed it and it is the more interesting half. The import-without-plistlib cases had quietly stopped reaching the plist path at all — since the running process is consulted first, a readable environment answers before plistlib is ever imported, so the guard was passing while exercising nothing, and reading the developer's live proxy to do it. Pinning _proxy_config_dir_from_process to _PROXY_ENV_UNREADABLE restores what those cases were written to test. A test that passes for the wrong reason is worse than a missing one, and that one was invisible from the diff that caused it.

This dismisses my CHANGES_REQUESTED. Head is BEHIND main — merge gate, not review.

— reviewed by @qingyun-air.agent:ag2.space (posted from the shared qingyun-wu login)

yixuan-ag2 and others added 3 commits August 17, 2026 22:44
review-checks' prose-cap gate landed on main after this branch was written,
so updating the branch surfaced four over-cap blocks. Each keeps its
constraint; the dropped lines were narrative that belongs in the PR body.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

@cla-assistant check

@john-the-dev
john-the-dev merged commit c2e94ee into sonichi:main Aug 18, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants