Skip to content

fix(start-cli): don't lose the proxy-listener race — bounded wait when the proxy is expected - #2800

Merged
john-the-dev merged 3 commits into
mainfrom
fix/proxy-listener-race
Aug 15, 2026
Merged

john-the-dev merged 3 commits into
mainfrom
fix/proxy-listener-race

Conversation

@qingyun-wu

Copy link
Copy Markdown
Collaborator

The bug (field-verified, pids + timestamps)

start-cli.sh wires ANTHROPIC_BASE_URL only when lsof -nP -iTCP:7846 -sTCP:LISTEN finds a listener — evaluated once at core launch. On boots where the credential proxy starts alongside/after the core, the guard loses the race and the core runs unrouted for its entire life: no credential-proxy protection, no quota telemetry, quota-state.json never written.

Measured on the Air:

event UTC
core launched (start-cli.sh autowire check ran, no listener yet) 16:09:53Z
credential-proxy listener bound on :7846 16:09:58Z

5 seconds late → the one-shot check had already decided "no proxy" and the core stayed unrouted.

The guard's original purpose stands: never point the core at a dead port (the #1086/#1291 failure class). So "always wire" is not acceptable — the fix must distinguish "proxy will never exist here" from "proxy is expected but hasn't bound yet".

The fix

  • "Expected" signal: launchctl print "gui/$(id -u)/com.sutando.credential-proxy" succeeds ⇔ the launchd job is loaded. This is the exact loaded-job check startup.sh (proxy install section) and restart.sh (_PROXY_SERVICE) already use, so all three scripts now agree on what "this host runs a supervised proxy" means. It's cheap (one launchctl call), and it's causally the right signal: a loaded KeepAlive job is the promise that a listener will appear — a pid/pgrep check would miss the window where launchd hasn't spawned the worker yet.
  • Expected + no listener: poll the same LISTEN check every 0.5 s up to ~10 s (20 iterations — the fixed-poll shape this repo already uses: restart.sh 20×0.25 s, startup.sh 10×0.5 s, start-cli's own launch-verify 25×0.2 s; no new env var — the manifest/config rules reserve ad-hoc env knobs, and none of the existing conventions carries a script timeout). Listener appears → wire and proceed. Budget exhausted → proceed unwired with a one-line warning naming the consequence (no proxy protection, no quota telemetry this session) — still never a dead port.
  • Not expected (no launchd job): behavior byte-identical to today — single check, no wait, proxy-less hosts don't stall.
  • Caller-set ANTHROPIC_BASE_URL: honored exactly as today; the whole block (including the wait) is skipped.

Before / after evidence

Before — extended suite vs. the parent commit (e926005)'s start-cli.sh (race cases fail; note 1 lsof probes = the one-shot evaluation):

  ok  live proxy listener forwards ANTHROPIC_BASE_URL=http://localhost:7846
  ok  dead proxy port omits ANTHROPIC_BASE_URL (never point the core at a dead port)
  ok  proxy-less host does not wait (fast path, 0s elapsed)
  ok  caller-set ANTHROPIC_BASE_URL wins over the listener autowire
  ok  caller preset is not duplicated/overridden with the local proxy URL
  ok  core-session marker still present in CORE_ENV_ARGS
  FAIL expected + late listener → bounded wait catches it and wires the proxy
  FAIL the wait actually re-polled (0s, 1 lsof probes)
  ok  expected + never-listener → budget exhausts, core env stays unwired (no dead port)
  FAIL budget exhaustion warns with the consequence (no proxy protection / telemetry)
  ok  unwired launch still proceeds (core env intact after budget exhaustion)
  ok  caller preset still wins when the proxy is expected but down
  ok  caller preset skips the wait (fast path, 0s elapsed)

FAIL — 3 failed, 10 passed

After — same suite at HEAD (all four pre-existing cases plus the new ones):

  ok  live proxy listener forwards ANTHROPIC_BASE_URL=http://localhost:7846
  ok  dead proxy port omits ANTHROPIC_BASE_URL (never point the core at a dead port)
  ok  proxy-less host does not wait (fast path, 0s elapsed)
  ok  caller-set ANTHROPIC_BASE_URL wins over the listener autowire
  ok  caller preset is not duplicated/overridden with the local proxy URL
  ok  core-session marker still present in CORE_ENV_ARGS
  ok  expected + late listener → bounded wait catches it and wires the proxy
  ok  the wait actually re-polled (2s, 5 lsof probes)
  ok  expected + never-listener → budget exhausts, core env stays unwired (no dead port)
  ok  budget exhaustion warns with the consequence (no proxy protection / telemetry)
  ok  unwired launch still proceeds (core env intact after budget exhaustion)
  ok  caller preset still wins when the proxy is expected but down
  ok  caller preset skips the wait (fast path, 0s elapsed)

PASS — 13 checks green

Live-host round trip (real launchctl + lsof, no stubs; the --print-core-env probe exits before tmux so it's non-destructive on a busy host — a full core restart was deliberately not run to avoid killing the live core mid-task):

--- host state:
launchd job loaded
node    33867 qingyunwu   26u  IPv4 ...      0t0  TCP 127.0.0.1:7846 (LISTEN)
--- probe (real host, no stubs):
SUTANDO_CORE_SESSION=1
ANTHROPIC_BASE_URL=http://localhost:7846
bash src/agent/claude/cli/start-cli.sh --print-core-env  0.10s user 0.13s system 88% cpu 0.260 total

Healthy path adds zero latency (0.26 s total, listener found on the first check).

Full pre-existing start-cli surface stays green at HEAD:

--- tests/start-cli-claude-config-dir.test.sh      exit=0 | FAILED: 0
--- tests/start-cli-compat-shim.test.sh            exit=0 | PASS: shell-setup compat shim forwards to its canonical target
--- tests/start-cli-visible.test.sh                exit=0 | all tests passed — start-cli --visible contract
--- tests/start-cli-chrome-seed-bundled-python.test.py  exit=0 | OK
--- tests/start-cli-dangerous-mode-seed.test.py    exit=0 | OK
--- tests/start-cli-model-pin.test.py              exit=0 | start-cli.sh ignores the model pin and clears it.
--- tests/start-cli-restart-verify.test.py         exit=0 | start-cli.sh --restart verifies liveness and never false-succeeds.

Test hermeticity notes

launchctl is now PATH-stubbed in every case (the real binary lives at /bin, which stays on PATH), so host launchd state can't leak into any assert — including the four pre-existing cases, which now explicitly model a proxy-less host. The late-listener case uses a stateful lsof stub (listener "appears" on the 4th probe) so the assert proves the poll actually re-checks; the never-listener case shims sleep so the full 20-iteration budget runs instantly (the pin is iteration-boundedness, not wall-clock); the two fast-path timing asserts run with real sleep so a leaked wait would trip their ≤5 s bound against the 10 s budget.

🤖 Generated with Claude Code

…n the proxy is expected

The credential-proxy autowire evaluated its listener check exactly once at
launch. On boots where the launchd-supervised proxy binds a few seconds
after the core, the guard loses the race and the core runs unrouted for
its entire life: no credential-proxy protection, no quota telemetry,
quota-state.json never written.

Keep the dead-port guarantee, but make it race-tolerant: when the
com.sutando.credential-proxy launchd job is loaded (the same loaded-job
check startup.sh and restart.sh already use), poll the LISTEN check every
0.5s up to ~10s. Listener appears → wire as before. Budget exhausted →
proceed unwired with a one-line warning naming the consequence. Hosts
without the job (proxy-less installs) and caller-set ANTHROPIC_BASE_URL
launches skip the wait entirely — their behavior is byte-identical.

tests/start-cli-proxy-routing.test.sh grows four cases pinning the new
policy (expected+late wires, expected+never warns+proceeds unwired,
not-expected stays on the fast path, caller preset skips the wait), with
launchctl now PATH-stubbed alongside lsof so host launchd state cannot
leak into any case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ No Python changes — nothing to gate (bar: 95% on changed lines).

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — reviewed at head 3f83f72. Correct, minimal, and I verified the negative control myself.

Ran both directions:

  • At HEAD: 13/13 shell checks green.
  • Negative control (new test vs parent commit): exactly 3 fail, and they're the right three — 'expected + late listener → bounded wait catches it and wires the proxy', 'the wait actually re-polled (0s, 1 lsof probes)' (this is the assertion that pins the old code to a single one-shot probe), and 'budget exhaustion warns with the consequence'. So the suite catches the racy old behavior and passes only with the fix — non-vacuous.

The fix is right: gate the wait on the proxy being EXPECTED (launchctl loaded-job check, shared with startup.sh/restart.sh — good that all three launch scripts now agree on one definition), bounded ~10s poll (20×0.5s), wire on listener appearance, else proceed unwired with a warning that names the consequence ('no proxy protection, no quota telemetry'). Non-proxy hosts keep the one-shot fast path; a caller-set ANTHROPIC_BASE_URL is untouched; no new env knobs. The test stubs model the field shape well (exp-late: listener on the 4th probe; exp-never: sleep no-op so the full 20-poll budget runs instantly — budget semantics pinned, not wall-clock).

Personal note, reviewed on merits regardless: this is the exact race that's had THIS session unrouted on LIGHT quota all day (proxy up on :7846 but start-cli never exported the base URL because the one-shot lsof lost the cold-boot race). Good fix.

No blockers. tsc+tests was still pending at review time — let it settle green before merge (the shell suite that matters here is green). Two head-timed approvals needed; merge owner-gated.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Blocking review for exact head 3f83f72fa863ec062a5bbb0c5e19d11070f0cf28 (GitHub will not let this account request changes on its own PR).

Changes requested; the bounded-wait behavior itself checks out, but the added comments do not meet the repository comment contract.

[P2 blocking repository contract] tests/start-cli-proxy-routing.test.sh adds several over-cap/history comments: the new block at lines 9-17 names a dated field report and narrates the launch-order story, lines 27-30 are a four-line explanation of the stub layout, and lines 62-64 are a three-line case comment. AGENTS.md limits code comments to at most two lines, no narration or incident history. Please trim the added test commentary down to the constraints the assertions cannot state. The production comments at src/agent/claude/cli/start-cli.sh:92-100 are already in the right shape.

Validation passed in the isolated /private/tmp/sutando-heartbeat-review-BpQjuu worktree:

  • bash tests/start-cli-proxy-routing.test.sh — 13 checks
  • bash -n src/agent/claude/cli/start-cli.sh tests/start-cli-proxy-routing.test.sh
  • git diff --check origin/main...HEAD
  • git diff origin/main...HEAD | bash scripts/review-checks.sh

The functional guard remains sound: caller-provided ANTHROPIC_BASE_URL still wins, proxy-less hosts stay on the fast path, an expected late listener is caught by the bounded poll, and an expected never-listener proceeds unwired with a warning rather than pointing the core at a dead port.

Reviewed by Qingyun's Personal Codex.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Comment-contract blocker addressed at the new head: all three flagged blocks trimmed to ≤2 lines stating only what the assertions cannot (the policy summary, the stub-isolation constraint, and the leaked-poll bound rationale) — no dated field reports, no narration. Suite re-run: 13/13 green. Production comments unchanged per the review.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Collaborator Author

Reviewed exact head 06d6edd9f117b73a36532cb2217b0f662393a69b (formal approval unavailable because this account authored the PR).

No blocking findings. The new commit only trims the three previously flagged test-comment blocks to the repository’s two-line/no-history contract; production behavior is unchanged. I reran tests/start-cli-proxy-routing.test.sh (13/13), shell syntax, diff hygiene, the hardcoded-path scan, and both path-resolution diff lints; all passed.

The prior functional assessment still holds: caller configuration wins, proxy-less hosts do not wait, expected late listeners are caught by a bounded poll, and a never-listening proxy remains unwired rather than pointing the core at a dead port. Hosted static/smoke/coverage checks are green; clean-install is still pending. Ready to merge once the remaining required check passes and the normal approval gate is satisfied.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Re-review for exact head 06d6edd9f117b73a36532cb2217b0f662393a69b.

I do not see blocking issues in this revision. The earlier comment-policy problems are fixed, and the proxy routing behavior remains sound: caller-provided ANTHROPIC_BASE_URL wins without waiting, proxy-less hosts stay on the fast path, a loaded proxy job gets a bounded listener wait, and a never-binding expected proxy proceeds unwired with a clear warning rather than pointing the core at a dead port.

Tests run locally:

  • bash tests/start-cli-proxy-routing.test.sh
  • bash -n src/agent/claude/cli/start-cli.sh tests/start-cli-proxy-routing.test.sh
  • git diff --check origin/main...HEAD
  • git diff origin/main...HEAD | bash scripts/review-checks.sh

Reviewed by Qingyun's Personal Codex.

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approve at head 06d6edd — my prior approval (3f83f72) was staled only by a comment-only push. Verified the delta 3f83f72..06d6edd: solely tests/start-cli-proxy-routing.test.sh (+6/-24), and every changed line is a comment (the three test-comment blocks shrunk to two lines) — zero functional change. My earlier verification stands: 13/13 at head, negative control fails exactly 3 at the parent incl. the 'exactly 1 lsof probe' assertion. Still the correct proxy-listener-race fix (the one that's had this session unrouted on LIGHT quota). No blockers.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Field verification from the machine that originally reported the race (measured by air, the reporting host's agent; relayed with credit):

  • The gate fires there: launchctl list shows com.sutando.credential-proxy loaded (pid 25484, parent launchd) → PROXY_EXPECTED=1 → the bounded wait executes on that host.
  • The budget covers the observed gap: the measured race was core at 16:09:53Z vs proxy listener at 16:09:58Z (5s); the wait budget is 20×0.5s ≈ 10s. 5s < 10s — the actually-observed race is caught.

So both conditions for the fix to matter on the affected host are empirically true, not assumed. Air also endorsed the test design distinction (sleep-shim pinning budget semantics in exp-never vs real sleep proving skip-the-wait in the caller-preset case).

🤖 Generated with Claude Code

@qingyun-wu
qingyun-wu requested a review from sonichi August 11, 2026 17:48
@sonichi

sonichi commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Cold review at head 06d6edd9f. No blockers. Comment-only — see the note at the end on why I don't cast a formal approval.

Ran the discrimination both ways by pointing this PR's test file at each revision of start-cli.sh:

PR's test + origin/main's start-cli.sh    FAIL — 3 failed, 10 passed
    FAIL expected + late listener → bounded wait catches it and wires the proxy
    FAIL the wait actually re-polled (1s, 1 lsof probes)
    FAIL budget exhaustion warns with the consequence
PR's test + this PR's start-cli.sh        PASS — 13 checks green

The middle line is the body's own claim, verified literally: on main the suite measures 1 lsof probe, which is the one-shot race. Good that the test asserts the probe count rather than just the outcome — an outcome-only assertion would pass on a machine where the listener happened to be up already.

The gating is the part I'd have worried about, and it's right. The bounded wait runs only when launchctl print gui/$(id -u)/com.sutando.credential-proxy succeeds, so a host without the proxy job never pays the 10s. On anything without launchd that check fails and PROXY_EXPECTED stays empty — degrades to the old fast path rather than hanging. And the export still happens only after proxy_listener_up returns true, so "never points at a dead port" holds.

The elif warning is the most valuable line in the diff. Previously an unrouted core was silent; now budget exhaustion says so and names the consequence (no proxy protection, no quota telemetry). That converts an invisible failure into a visible one, which is the same defect shape this PR is fixing one layer up.

One observation, not a request: if the launchd job is loaded but deliberately stopped, every core launch now pays ~10s and then warns. That's the correct trade — a warned 10s beats a silent unrouted session — but it is a new startup cost on that specific configuration, worth knowing before someone reports "the core got slower."

On approval: I was asked in the Sutando Triage room for a formal review on this and #2803. I've reviewed both with both-direction evidence (#2803 comment is already posted), but I don't cast formal APPROVE/REQUEST_CHANGES — the gh credential on this host is the owner's identity, so an approval from me would land under his name and count toward the two-approval gate as if he had reviewed it. Comment-only keeps the evidence and leaves the approval to a human.

@bassilkhilo-ag2

Copy link
Copy Markdown
Collaborator

Verified independently on pull/2800/head: ran the new test suite as-is (13/13 pass), then swapped in origin/main's start-cli.sh as a negative control against the same test file — reproduced the exact claimed failure signature (3 failed: the late-listener wait, the re-poll-count assert, the budget-exhaustion warning; 10 passed). The diff is small and matches the PR body precisely: a PROXY_EXPECTED gate off the same loaded-launchd-job check startup.sh/restart.sh already use, a bounded 20×0.5s poll only when expected, otherwise byte-identical fast path.

Worth connecting to #2806 for whoever reviews this: that PR replaced a health-check diagnostic message's explanation of "why did this core end up unrouted" (supervisor-launched core skips start-cli.sh → new theory: lsof one-shot check lost the race against the proxy's bind). This PR is the actual behavioral fix for that second mechanism — a supervised proxy binding a few seconds after a cold-boot core no longer gets checked exactly once and permanently missed. #2806 was diagnosing a symptom this PR now removes for the case where start-cli.sh genuinely is the launch path (john-the-dev's finding on #2806 — a raw tmux invocation bypassing start-cli.sh entirely — is a separate, still-unaddressed cause and out of scope here).

No blockers from me.

qingyun-wu added a commit that referenced this pull request Aug 12, 2026
…cause

Review of be371b0: the new text asserts one cause and is false on a host with
the other. On the reviewer's host the core was launched straight from tmux --

    64653 <- 56595 : bin/zsh -c source .../snapshot-zsh-1
    56595 <- 56594 : claude --name sutando-core --remote-control Sutando ...
    56594 <- 1     : tmux -S /tmp/sutando-tmux.sock new-session -d ...

start-cli.sh is not in that chain at all, so there was no lsof guard and no
race: the launcher simply never ran. Text that says it "exports it only when
the proxy port already has a listener at launch" points that reader at a
guard that never executed. That is the same harm this PR opened by
describing, moved to a different host.

Two causes, and the probe cannot tell them apart -- it observes an unrouted
core, not how it got there:

  (a) core started outside the launcher entirely (reviewer's host)
  (b) core started BY the launcher, before the proxy bound (this host, #2800)

Both messages, the function docstring, and the test docstring now name the
condition and cover both, keeping the actionable part (check
ANTHROPIC_BASE_URL on the running core). The function docstring still carried
the pre-#2417 startup.sh rationale that this PR corrected everywhere else.

Two new tests gate it -- assertIn on the launcher path, on "outside the
launcher" and on "before the proxy bound", assertNotIn on startup.sh -- one
per message, since the same reader reads both on the same host.

Evidence
--------
    python3 tests/health-check-quota-telemetry.test.py
    Ran 45 tests   OK   rc=0

Control -- the new tests fail on BOTH prior texts, with the tests held at this
head and only src/health-check.py swapped:

    git show be371b0:src/health-check.py   -> FAILED (failures=2)
      'outside the launcher' not found in '...exports it only when the proxy
      port already has a listener at the moment the core starts...'
    git show origin/main:src/health-check.py -> FAILED (failures=2)
      'src/agent/claude/cli/start-cli.sh' not found in '...set by
      src/startup.sh, which a supervisor-launched core never runs...'
    restored                                 -> Ran 45 tests OK

    python3 tests/python39-union-annotations.test.py   -> all passed
    /usr/bin/python3 (3.9.6) exec_module src/health-check.py -> IMPORT OK
    git diff | bash scripts/review-checks.sh
    review-checks: PASS (hardcoded-paths clean)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonichi

sonichi commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Verified the premise the "expected" signal rests on, and it holds exactly: src/restart.sh:57-58 builds gui/$(id -u)/com.sutando.credential-proxy and gates on launchctl print, and src/startup.sh:732 uses the identical form. Same check, as claimed.

Datapoint from a warm host (Chis-Mac-mini): job loaded = yes, listener on :7846 = up. So the new path engages here and exits on the first probe — the fix costs one extra lsof on a healthy host, no added latency. Your field case was the cold-boot side; this is the other one.

The test I'd single out is case 5, and it's the reason this is falsifiable. The stateful lsof stub that fails three probes then succeeds reproduces the exact race, and asserting probe-count >= 4 proves the loop ran rather than that the outcome happened to be right. A fix that accidentally wired on the first probe would pass an outcome-only assert and fail this one. Case 2's <=5s timing assert is the matching pin on the fast path — together they make "bounded wait, only when expected" checkable instead of asserted.

One thing worth being deliberate about: the budget asymmetry. startup.sh:738 already uses this idiom immediately after installing the proxy — for _ in $(seq 1 10); do lsof -i :7846 ... ; sleep 0.5; done, a 10-poll/~5s budget. This PR uses 20 polls/~10s. Both are defensible (start-cli races a proxy it didn't just install, so a longer budget makes sense), but the two now differ by 2x for the same wait-for-:7846 question. Worth either aligning them or leaving a note saying the asymmetry is intentional, so the next person doesn't "fix" one to match the other.

A concern I checked and am withdrawing, so you know it was considered: I first read startup.sh:732's extra [ "$_PROXY_PLIST_NODE" = "${SUTANDO_NODE:-}" ] as an is-this-proxy-ours condition your check omits — which would have meant a foreign-node job triggering a 10s wait plus a scary warning. Reading the surrounding block, that comparison is a reinstall/drift gate (loaded AND runtime current → skip reinstall; otherwise reinstall), not an ownership test. A loaded job means "proxy belongs on this host" in both files, so launchctl print alone is the correct expected-signal here.

CI 16/16 SUCCESS at head. No formal approval from me — bot approvals are owner-gated on this repo.

Stand: Echo Act IV Mini

john-the-dev added a commit that referenced this pull request Aug 12, 2026
…nrouted core (#2806)

* fix(health-check): quota-telemetry names the wrong mechanism for an unrouted core

Both quota-telemetry messages tell the reader to blame src/startup.sh and "a
supervisor-launched core never runs it". Measured on a live host, that is not
why the core is unrouted.

#2417 moved the routing decision into the core launcher. On a host where the
core is unrouted, src/agent/claude/cli/start-cli.sh HAS run — it is the core
process's own parent — and it exported nothing because its guard is a one-shot
`lsof` on the proxy port evaluated at launch:

    16:09:53Z  start-cli.sh      (guard evaluates: no listener yet)
    16:09:58Z  proxy listener binds

Five seconds. Both are supervised, so the ordering is routine rather than
exceptional, and the core stays unrouted for its whole session.

This matters more than a stale sentence normally would: the probe fires on
every health run, so it is the most-read explanation of this symptom, and it
points at a file that is no longer the mechanism. Following it leads to
"the supervisor bypasses startup.sh", which is a different (and absent) bug.

Both messages now name the launcher and the launch-order condition. The test's
docstring carried the same wrong rationale and is corrected with them — it is
the explanation a future reader trusts when deciding whether the probe is right.

Evidence
--------
    python3 tests/health-check-quota-telemetry.test.py
    Ran 43 tests   OK   rc=0

Control — the suite really does gate this text (it is not vacuously passing):
deliberately corrupting the surviving assertion string gives

    rc=1, 7 failure lines

and restoring it returns rc=0.

    bash scripts/review-checks.sh --diff <diff>
    review-checks: PASS (hardcoded-paths clean)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3i6aj3KAec3QKDY922J1E

* fix(health-check): the message names the unrouted CONDITION, not one cause

Review of be371b0: the new text asserts one cause and is false on a host with
the other. On the reviewer's host the core was launched straight from tmux --

    64653 <- 56595 : bin/zsh -c source .../snapshot-zsh-1
    56595 <- 56594 : claude --name sutando-core --remote-control Sutando ...
    56594 <- 1     : tmux -S /tmp/sutando-tmux.sock new-session -d ...

start-cli.sh is not in that chain at all, so there was no lsof guard and no
race: the launcher simply never ran. Text that says it "exports it only when
the proxy port already has a listener at launch" points that reader at a
guard that never executed. That is the same harm this PR opened by
describing, moved to a different host.

Two causes, and the probe cannot tell them apart -- it observes an unrouted
core, not how it got there:

  (a) core started outside the launcher entirely (reviewer's host)
  (b) core started BY the launcher, before the proxy bound (this host, #2800)

Both messages, the function docstring, and the test docstring now name the
condition and cover both, keeping the actionable part (check
ANTHROPIC_BASE_URL on the running core). The function docstring still carried
the pre-#2417 startup.sh rationale that this PR corrected everywhere else.

Two new tests gate it -- assertIn on the launcher path, on "outside the
launcher" and on "before the proxy bound", assertNotIn on startup.sh -- one
per message, since the same reader reads both on the same host.

Evidence
--------
    python3 tests/health-check-quota-telemetry.test.py
    Ran 45 tests   OK   rc=0

Control -- the new tests fail on BOTH prior texts, with the tests held at this
head and only src/health-check.py swapped:

    git show be371b0:src/health-check.py   -> FAILED (failures=2)
      'outside the launcher' not found in '...exports it only when the proxy
      port already has a listener at the moment the core starts...'
    git show origin/main:src/health-check.py -> FAILED (failures=2)
      'src/agent/claude/cli/start-cli.sh' not found in '...set by
      src/startup.sh, which a supervisor-launched core never runs...'
    restored                                 -> Ran 45 tests OK

    python3 tests/python39-union-annotations.test.py   -> all passed
    /usr/bin/python3 (3.9.6) exec_module src/health-check.py -> IMPORT OK
    git diff | bash scripts/review-checks.sh
    review-checks: PASS (hardcoded-paths clean)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(health-check): trim added prose to the two-line comment contract

The added narrative in the quota-telemetry docstrings and the test module
header carried incident history and mechanism explanation that the
repository's comment contract keeps in the PR body. Only the durable
constraint stays in code.

No executable line changes: the diff after this commit differs from the
previous head only in comment and docstring text.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Rui Wang <52230987+john-the-dev@users.noreply.github.com>
@john-the-dev

Copy link
Copy Markdown
Collaborator

Flagging a non-obvious merge blocker — mergeable reads MERGEABLE and a check-runs sweep shows everything green, so this one hides.

license/cla is a required status context on main, and it is a commit status, not a check-run. On this PR's head it has never reported at all:

06d6edd9 (head)   combined=pending   statuses: (none)

A required context that never reports cannot go green, so this cannot merge until cla-assistant runs against the current head. On this repo it does not reliably re-post after a head moves — a fresh @cla-assistant check comment usually does it, and close-and-reopen is the documented fallback in CONTRIBUTING.md. I have not done either here since it is your PR and a close/reopen pings every reviewer.

Context: I swept all 93 open non-draft PRs against the status API and six are in this state, in two classes — three where the bot never ran on the head (this one, #1858, #2160), and three where a commit is authored under an email mapping to a different GitHub account that has not signed (#2158, #2270, #2276). On my own #2158 the fix was re-authoring a single merge commit — identical tree and parents, authorship only — which took it from pending to success.

Nothing here is about your code.

@john-the-dev

Copy link
Copy Markdown
Collaborator

Following up with a tested remedy, since what I suggested earlier turns out not to work.

I had both of my own PRs in this same state and used them to check the two documented options:

remedy result
@cla-assistant check comment does not work — posted on #1858 and #2158, no status appeared
push to the branch works — fires the cla-recheck-on-push pull_request_target workflow
close + reopen works — verified on #1858 just now, license/cla went ABSENT → success

The comment being useless was not obvious: on #2158 the status did go green right after I commented, but github-actions[bot] had posted its own recheck 7 seconds earlier, triggered by my force-push. The push did it; my comment was a coincidence sitting next to the cause. On #1858 there was no push, only the comment, and nothing happened — which is what isolated it.

So for this PR: close and reopen it, or push anything to the branch. Close+reopen is the non-destructive one — it leaves the head commit alone. It does notify reviewers, which is why I have not done it on your PR.

Worth noting CONTRIBUTING.md currently offers the comment first; on the evidence above it should not. I will raise that separately rather than bury it here.

@john-the-dev

Copy link
Copy Markdown
Collaborator

Correcting the last line of my previous comment — I named the wrong file, and I should have read before experimenting.

I wrote that CONTRIBUTING.md "currently offers the comment first" and that it should not. That is wrong. CONTRIBUTING.md already says the opposite, at line 207:

If it is absent, close and reopen the PR. Reopening fires pull_request.reopened, which CLA-Assistant does act on.

And it goes considerably further than my experiment did — it distinguishes license/cla FAILING from ABSENT, gives the exact gh api .../commits/{sha}/status invocation, explains that the status is SHA-bound so any later push drops it again, notes that cla-recheck-on-push.yml is the thing that comments the trigger and that "that repair is not reliable", and backs it with webhook delivery-log evidence — twelve synchronize deliveries on 2026-08-11, all HTTP 200, nine written and three never written. It even names the failure mode precisely: the write can fail after CLA-Assistant has acknowledged the event, which is why close+reopen works as a retry rather than as a push-specific fix.

So the correct attribution: the comment-first advice is in CLAUDE.md line 78 ("After update-branch, CLA-Assistant may not auto-rerun — try @cla-assistant check comment or close+reopen if stuck"), not in CONTRIBUTING. That is the line worth changing, and it matters more than I first thought because CLAUDE.md is auto-loaded into every agent session while CONTRIBUTING.md is not — so the weaker guidance is the one that actually gets acted on.

My measurements stand and are consistent with the documented model; what does not stand is my claim about which document was wrong. Re-deriving something the repo had already written down, and then mis-citing it, is a worse outcome than just reading it first.

@yixuan-ag2 yixuan-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 06d6edd9. Approving — second distinct approver. The bug is real, the field evidence (16:09:53Z core vs 16:09:58Z listener) is the right kind, and refusing "always wire" because it would re-open #1086/#1291 is the correct call rather than the convenient one.

Reusing the launchctl print loaded-job check so start-cli.sh, startup.sh and restart.sh finally agree on what "this host runs a supervised proxy" means is the best part of the diff. And the reasoning for choosing it over a pid/pgrep probe — a loaded KeepAlive job is the promise that a listener will appear — is causally right, not just cheaper.

⚠ A host class the "expected" signal cannot see, measured here

The gate is launchctl print "gui/$(id -u)/com.sutando.credential-proxy". On this machine:

launchctl print gui/501/com.sutando.credential-proxy  -> FAILS (no loaded job)
~/Library/LaunchAgents/com.sutando.credential-proxy.plist -> ABSENT
lsof -nP -iTCP:7846 -sTCP:LISTEN                      -> node, pid 23294
ps -o ppid= -p 23294                                  -> 1     (started Thu Aug 13 16:05:04)

A real proxy is listening, and launchctl print says "not expected". This host's proxy is started by startup.sh / the desktop supervisor, not launchd — reparented to init, no plist anywhere. Under this PR it takes the not-expected branch: single check, no wait, byte-identical to today. So on exactly this shape — proxy present but unsupervised — the race you are fixing is still lost.

I am not asking you to fix it here, and I want to be precise about why rather than hand-wave "out of scope": your PR is strictly better on every host, the not-expected branch is deliberately unchanged so proxy-less hosts don't stall, and widening the signal is a different decision with its own failure mode (waiting 10s on a host that will never have a proxy). Shipping the launchd case first is right.

But it should be named in the body, because the current text reads as though the race is closed, and on this host it is not.

The precedent, because this is the second time the same blind spot has surfaced this week

health-check.py's quota-account-identity had the identical shape: it consulted the launchd plist, and returned ok — reading as the accounts agree — on every host whose proxy was started any other way. I fixed that in #2891 by reading the running listener's own environment (lsof -ti:7846 -sTCP:LISTEN → ps eww -p <pid>), and in #2896 by preferring that process over a plist that may describe a dead job.

The same instrument would work here: if a listener is already bound, you do not need the launchd job to tell you a proxy is expected — you can see it. That does not cover the true race (nothing bound yet), which is what the launchd signal is for, but it does mean the two signals are complementary rather than alternatives:

  • listener already bound → wire now, no wait needed
  • no listener + loaded launchd job → your bounded wait
  • no listener + no job → today's fast path

Worth one sentence in the body either way, since the next person to hit this will search for the previous time it was hit.

One thing I checked because the shape invites it

The 20 × 0.5 s poll and the "no new env var" reasoning are both consistent with the existing conventions you cite (restart.sh 20×0.25, startup.sh 10×0.5, start-cli's own 25×0.2). Declining to add a knob because the manifest/config rules reserve ad-hoc env vars is the right reading of that rule — a timeout is not configuration a user should be tuning, and every one of those three call sites hardcodes its own.

Not verified

I did not run the suite or reproduce the race — no second host, and the launch timing you measured is not reproducible on demand here. My evidence is the classification of this host, which is what it takes to show the gap exists; yours is what shows the fix works.

APPROVE.

@john-the-dev
john-the-dev enabled auto-merge (squash) August 15, 2026 04:08
@github-actions

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Reviewed exact head 8fdc38e4be5573631c0928a5c09f20dd10e5c09e.

The bounded listener wait is functionally sound: it only waits when the launchd job is loaded, never forwards a dead local endpoint, and preserves a caller-provided base URL. The focused 13-case suite, shell syntax, diff hygiene, coverage, and completed hosted checks pass.

[P2] The new source and test prose violates the repository's explicit code-comment contract in AGENTS.md:29: comments must be at most two lines, state only non-obvious constraints, and contain no incident/issue narration. Examples include src/agent/claude/cli/start-cli.sh:115-118 (four lines) and tests/start-cli-proxy-routing.test.sh:68-70 / :80-82 (three lines each); the added implementation comments also narrate the cold-boot incident. Please collapse every newly added contiguous comment block to at most two lines and leave the field history in the PR body.

Reviewed by Qingyun's Personal Codex.

@john-the-dev
john-the-dev merged commit 0e2b117 into main Aug 15, 2026
17 checks passed
@john-the-dev
john-the-dev deleted the fix/proxy-listener-race branch August 15, 2026 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants