Skip to content

fix(health-check): probe the interpreter each bridge RUNS on, not the one that would launch it - #3597

Merged
john-the-dev merged 19 commits into
mainfrom
fix/secret-scanner-probes-running-interpreter
Sep 2, 2026
Merged

john-the-dev merged 19 commits into
mainfrom
fix/secret-scanner-probes-running-interpreter

Conversation

@john-the-dev

Copy link
Copy Markdown
Collaborator

Closes #3580.

The secret-scanner probe reported ok — detect-secrets present in all 3 bridge interpreter(s) while the ag2.space gateway bridge ran on the app's bundled python, which has no detect_secrets. Inbound bridge text was being scanned with the 17 provider detectors and the entropy checks off, and health-check printed green over it.

Before / after

Same host, same moment, gateway bridge (pid 76550, src/remote-gateway-bridge.py) live:

origin/main   OK   | detect-secrets present in all 3 bridge interpreter(s)
this branch   WARN | DEGRADED in 1/4 bridge interpreter(s)
                   | (…/engine/runtime/python/bin/python3): detect-secrets missing,
                   | so its provider detectors and entropy checks are off …

Reproduced by checking out each revision's src/health-check.py in place and calling check_secret_scanner_mode() — in place, because the module resolves paths from its own __file__.

The obvious fix is a trap

_VAULT_SCANNER_BRIDGES did not list the gateway bridge, so the natural fix is to add it. That does not work, and it makes things worse.

_bridge_interpreter(name) answers which interpreter would launch this, and returns sys.executable for a bridge with no required import. Measured here:

remote-gateway-bridge  _bridge_interpreter() -> /opt/homebrew/opt/python@3.14/bin/python3.14   HAS detect_secrets
remote-gateway-bridge  actually running on   -> …/engine/runtime/python/bin/python3            LACKS it

So adding the gateway to the list alone would print present in all 4 — a bigger, more authoritative-looking number over the same gap. The population was never the bug; the question was. The probe now asks the process table which interpreter each bridge is running on, falling back to _bridge_interpreter when the bridge is down.

Three ways the naive implementation fails

I hit all three writing it; each is pinned by a control in the test:

trap what happens
interpreter path contains spaces shlex.split(...)[0] yields /Users/x/Application — basename Application, no match
script arg carries a src/ prefix cutting at the bare script name leaves a trailing slash, basename ''
the probing shell's own argv contains the script name self-match — the anti-pattern _proc_argv's docstring in this same module already warns about

Both probe polarities are pinned as well, so neither verdict is free: stub every bridge onto an interpreter that has the module → asserts ok; onto one that does not → asserts warn.

Test discriminates

against origin/main's health-check.py:  AttributeError: no attribute '_live_bridge_interpreter'
against this branch:                    all assertions passed

Scope

This makes the probe honest; it installs nothing. The durable half — vendoring detect-secrets into the bundle, or pointing the bridge at an interpreter outside the engine tree — is a separate call. That tree is replaced on app update (this interpreter's mtime is today's update, Aug 30 04:41), which is why the 2026-08-18 "installed and verified end-to-end" fix was gone nine days later.

🤖 Generated with Claude Code

… one that would launch it

The secret-scanner probe reported `ok — detect-secrets present in all 3 bridge
interpreter(s)` while the ag2.space gateway bridge ran on the app's bundled
python, which has no `detect_secrets`. Inbound text was scanned with the 17
provider detectors and the entropy checks off, and health-check printed green
over it. Issue #3580.

Before / after on a live host, gateway bridge pid 76550 up, same machine:

    origin/main   OK   | detect-secrets present in all 3 bridge interpreter(s)
    this branch   WARN | DEGRADED in 1/4 bridge interpreter(s)
                       | (…/engine/runtime/python/bin/python3): detect-secrets
                       | missing, so its provider detectors and entropy checks
                       | are off …

The population was never the bug — the question was. Adding the gateway to
`_VAULT_SCANNER_BRIDGES` alone does NOT fix this: `_bridge_interpreter` returns
`sys.executable` for a bridge with no required import, and that interpreter HAS
the module, so the probe would print `present in all 4` and read as MORE
authoritative while the gap persisted. So the probe now asks the process table
which interpreter each bridge is actually running on, and falls back to
`_bridge_interpreter` when the bridge is down.

Three things the naive implementation gets wrong, each pinned by a control in
the test — I hit all three while writing it:

  - the bundled interpreter's path contains spaces, so shlex/argv[0] truncates
    it mid-path ("/Users/x/Application")
  - the script arg carries a `src/` prefix, so cutting at the bare script name
    leaves a trailing slash and an empty basename
  - the probing shell's own argv contains the script name — the self-match that
    `_proc_argv`'s docstring in this same module already warns about

Both probe polarities are pinned too, so neither verdict is free: stubbing every
bridge onto an interpreter that has the module asserts `ok`, and onto one that
does not asserts `warn`.

Scope: this makes the probe honest. It does not install anything. The durable
half — vendoring detect-secrets into the bundle or pointing the bridge at an
interpreter outside the engine tree — is a separate call, and the tree is
replaced on app update (this interpreter's mtime is today's update), which is
why the 2026-08-18 manual install did not survive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@john-the-dev john-the-dev added the ag2product AG2 product related (packages, PyPI, task-relay) label Aug 31, 2026
…l bridge name

CI's lint-hermetic-bridge-tests caught this test reading the developer's real
per-user config: health-check resolves workspace/channel config at import, so
`exec_module` did it against the host. Now `CLAUDE_CONFIG_DIR` points at a temp
dir set at module level before the load, which is the documented fix.

The gate also fired on its own prefilter, `(discord|slack|telegram)-bridge\.py`.
This test does not import a bridge — the string appeared as test DATA in the
"a bridge that is not running yields None" control, where the name is arbitrary:
any script absent from the ps output exercises that path. Renamed it to
`not-running-bridge.py`, which is a more accurate description of what the control
actually asserts. The isolation above is kept regardless, so the test is hermetic
whether or not the prefilter would have fired.

Not touched: SUTANDO_WORKSPACE, which I had also set — it is no longer honored
as of v0.8 and only emits a deprecation warning, so setting it isolated nothing.

Local: lint-hermetic-bridge-tests ok (112 scanned, 46 grandfathered, 1 mitigated).
Test still discriminates — non-zero against origin/main's health-check.py, 0 against this branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 87%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • src/health-check.py (100%)

Summary

  • Total: 41 lines
  • Missing: 0 lines
  • Coverage: 100%

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested at exact head 9cfd2784932acd221d7bb916f51d3ce93493c862.

[P1] src/health-check.py:678-700 loses the argv boundary when the script argument path also contains spaces. The new parser finds the last space before the script basename; for a supported row such as:

/Users/x/Application Support/app/runtime/python/bin/python3 /Users/x/My Sutando/src/remote-gateway-bridge.py

that space is inside the script path, so _live_bridge_interpreter() returns None. The caller then falls back to _bridge_interpreter() and recreates the exact would-launch false green this PR is meant to remove. Exact-head composed control:

result: {status: ok, detail: detect-secrets present in all 1 bridge interpreter(s)}
probed: [/healthy/python3]
live parsed: None

The running degraded interpreter was never probed. The added test covers spaces in the interpreter path but keeps the script argument as src/remote-gateway-bridge.py, so it cannot detect this case. Please recover the executable from the matched PID (or otherwise use an argv-boundary-preserving source) and add a polarity control with spaces in both paths; the composed check must warn and probe the live interpreter.

Worst-case disruption is a security-relevant standing false green for any checkout or bundled script path containing spaces: provider and entropy detectors can remain off while health-check reports healthy. The 19 focused scanner/process assertions, Python compilation, cumulative diff hygiene, repository review checks, all hosted CI, diff coverage, and CLA pass, so this is the sole blocker I found.

Not ready to merge until the live interpreter is measured for spaced script paths.

…slicing argv

Addresses qingyun-wu's P1 on #3597, which is correct and reproduces exactly.

Round 1 sliced argv at the last space before the script name. When the SCRIPT
path also contains spaces, that boundary falls inside the script path:

    argv:  /Users/x/App Support/rt/python/bin/python3 /Users/x/My Sutando/src/remote-gateway-bridge.py
    round1 parse -> '/Users/x/App Support/rt/python/bin/python3 /Users/x/My'
                    basename 'My' -> rejected -> None
                    -> caller falls back to _bridge_interpreter() -> FALSE GREEN

So the defect this PR exists to remove came straight back for any checkout or
bundled script path containing spaces. No argv boundary is recoverable when both
the interpreter and the script path may contain spaces, so argv is no longer
parsed at all: `_proc_executable(pid)` reads the executable via `ps -o comm=`,
which is a single field, so a spaced path survives it. The PID comes from ps
field 0, which is unambiguous. `/bin/ps -o comm= -p` is already the idiom used
elsewhere in this module.

Verified on this host, real ps and real PID:

    ps -o comm= -p 76550
      /Users/…/space.ag2.app/engine/runtime/python/bin/python3
    probe -> WARN | DEGRADED in 1/4 bridge interpreter(s) (…/runtime/python/bin/python3)

Test additions, per the review:
  - spaces in BOTH interpreter and script paths resolve to the interpreter
  - the COMPOSED check driven through _ps_snapshot + the parser (not a stub of
    the helper) on that same spaced-both row must WARN and must NAME the live
    interpreter it probed
  - the same row with a healthy interpreter must be ok, so the warn is not free
  - executable unresolvable -> None

`exe_of` is injectable so synthetic PIDs resolve in tests, matching the existing
injectable-probe convention in this module (_proxy_config_dir_from_process).

Local gates: review-checks PASS (hardcoded-paths + root-artifacts + prose-cap),
lint-hermetic-bridge-tests ok, test passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@john-the-dev

Copy link
Copy Markdown
Collaborator Author

Confirmed and fixed at bb080c94. The P1 is correct and reproduces exactly — thank you, this would have shipped a standing false green.

Reproduced your case against the round-1 parser before changing anything:

argv:  /Users/x/App Support/rt/python/bin/python3 /Users/x/My Sutando/src/remote-gateway-bridge.py
round1 parse -> '/Users/x/App Support/rt/python/bin/python3 /Users/x/My'
                basename 'My' -> rejected -> None
                -> caller falls back to _bridge_interpreter() -> the false green this PR removes

The fix takes your first suggestion

No argv boundary is recoverable when both the interpreter and the script path may contain spaces — any delimiter I pick is ambiguous. So argv is no longer parsed at all. _proc_executable(pid) reads the executable via ps -o comm=, a single field, so a spaced path survives intact; the PID comes from ps field 0, which is unambiguous. /bin/ps -o comm= -p is already the idiom used elsewhere in this module.

On this host, real ps and real PID:

ps -o comm= -p 76550
  /Users/…/space.ag2.app/engine/runtime/python/bin/python3

probe -> WARN | DEGRADED in 1/4 bridge interpreter(s) (…/runtime/python/bin/python3)

The polarity control you asked for

Added, and deliberately not a stub of _live_bridge_interpreter — these drive check_secret_scanner_mode through _ps_snapshot + the parser, so a future parser regression fails here rather than sliding past a mocked helper:

  • spaces in both interpreter and script paths resolve to the interpreter
  • the composed check on that same spaced-both row must warn and must name the live interpreter it probed (so a warn produced by a would-launch fallback still fails)
  • the same row with a healthy interpreter must be ok — without this the warn above is free
  • executable unresolvable → None

One note on the earlier round's coverage, since you flagged it: my round-1 test varied only the interpreter path and kept the script arg as src/remote-gateway-bridge.py, exactly as you said. It also failed against round-1 only with a TypeError on the new exe_of parameter, which is a signature mismatch rather than behavioural proof — that is why the block above isolates the parse itself instead.

exe_of is injectable so synthetic PIDs resolve in tests, matching the existing injectable-probe convention here (_proxy_config_dir_from_process).

Local gates before push: review-checks PASS (hardcoded-paths + root-artifacts + prose-cap — prose-cap caught three over-cap comment blocks I had added, now trimmed), lint-hermetic-bridge-tests ok, test passes.

@TustinOC TustinOC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 9cfd2784. The thesis is right and I reproduced it independently on a second host — but the fix silently no-ops on the default macOS install path, which is this cohort's shipped layout.

What I corroborated, on a different host with different paths

_VAULT_SCANNER_BRIDGES        -> ['telegram-bridge', 'discord-bridge', 'slack-bridge']
probe on main                 -> ok — detect-secrets present in all 1 bridge interpreter(s)
gateway bridge                -> pid 57743, src/remote-gateway-bridge.py, LIVE and uncounted
_bridge_interpreter('remote-gateway-bridge') -> /usr/local/bin/python3        (would-launch)
actually running on           -> /Library/Frameworks/Python.framework/…/MacOS/Python
bundled engine python         -> ModuleNotFoundError: No module named 'detect_secrets'

So: the gateway bridge is absent from the list, and _bridge_interpreter answers a different interpreter than the one running — your central claim, on paths that share nothing with yours.

What I could NOT reproduce: the exposure. Here the live interpreter does have detect_secrets, so this host is not scanning blind. Worth stating because it sharpens your framing rather than weakening it — on this host, adding the gateway to the list would print a correct-looking answer for the wrong reason, and the defect would stay invisible until the two interpreters diverge the other way.

Blocker: the cut lands inside the SCRIPT path, not the interpreter path

_live_bridge_interpreter cuts at parts[2].rfind(" ", 0, parts[2].index(script)). Your comment and fixture both solve for a space in the interpreter path. On this host the space is in the script path, and the cut lands inside it. Running your function against this host's real ps line:

argv[2]   /Library/Frameworks/…/MacOS/Python /Users/maxa1v/Library/Application Support/space.ag2.app/…/src/remote-gateway-bridge.py
cut at    127 -> 'axa1v/Library/Application Su'
interp    /Library/Frameworks/…/MacOS/Python /Users/maxa1v/Library/Appl
basename  'Application'  -> startswith('python')? False

_live_bridge_interpreter('remote-gateway-bridge.py') -> None

At src/health-check.py:699-700, None falls back to _bridge_interpreter — the launcher answer. So on any host whose script path contains a space, this lands and changes nothing, and Application Support is where the macOS app puts the tree by default. Your own PR body notes the engine tree is replaced on update at exactly such a path.

Why the test does not catch it — the fixture at tests/…-live-interpreter.test.py:52 puts the space in the interpreter and keeps the script bare:

76550 76531 /Users/x/Application Support/app/runtime/python/bin/python3 src/remote-gateway-bridge.py

One row with the space on the other side would have failed. That is a fixture gap, not a missing idea — trap #1 in your table is exactly this failure, attributed to the wrong half of the line.

A direction, not a prescription

argv is ambiguous when either path may contain spaces. The kernel already knows the answer: on macOS ps -p <pid> -o comm= prints the full executable path unsplit, and it is directly usable —

$ ps -p 57743 -o comm=
/Library/Frameworks/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python
$ "$(ps -p 57743 -o comm=)" -c 'import detect_secrets'   # succeeds

so argv could be used only to identify the pid (script in argv), with comm supplying the interpreter. That also retires trap #1 entirely rather than parsing around it.

Not blocking, and worth keeping

The self-match guard (parts[0] == me) and the checked dedup are both right, and the two-polarity test — stub every bridge onto an interpreter that has the module, then onto one that does not — is the part I would keep verbatim. Your test does pass on this branch and fails on main with AttributeError: no attribute '_live_bridge_interpreter', as claimed; I ran both.

Requesting changes only on the space-in-script-path case. Everything else here is the right question replacing the wrong one.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

Not adding a verdict — @qingyun-wu's CHANGES_REQUESTED at 9cfd2784 stands and I could not find a
reason to weaken it.
Comment only, with three things I measured that I think are useful to whoever
writes the next revision.

1. Their P1 reproduces independently, and by a different construction. Their control was a
composed row; I built real files on disk — an interpreter under .../App Support/... and a script
under .../My Sutando/src/ — and ran _live_bridge_interpreter against the resulting ps line:

spaced SCRIPT path  ->  None          (falls back to _bridge_interpreter -> the would-launch false green)

Same verdict from a different fixture, which is worth more than agreeing with theirs.

2. A second, opposite-polarity case the tests do not cover. A python process whose argv merely
mentions the script name yields a bogus interpreter that passes the
basename(interp).startswith("python") guard:

ps row:  /usr/bin/python3 -c import os; scan('src/remote-gateway-bridge.py')
returns: "/usr/bin/python3 -c import os;"      <- not a path; os.path.isfile() is False

This one is fail-closed (a bogus interpreter fails the import check, so it reports a false
DEGRADED) which is why I rate it well below P1's fail-open. But a security probe that cries wolf is
the kind that gets demoted to the category nobody reads, so it is worth closing.

3. The part I would most want to hand over: the two constraints pull against each other, and I
falsified three naive fixes.
Each was tested against all four cases, not reasoned about:

candidate remedy                          P1 (spaced script)   my case (mention)
require os.path.isfile(interp)                   FAILS                OK
longest prefix that is a real python file          OK                FAILS  (returns /usr/bin/python3)
+ also require the suffix token to endswith()    FAILS                 OK

The middle one is the trap: it fixes P1 and returns a real interpreter for a process that is not
the bridge at all, so the failure gets harder to see, not easier.

The root cause is that the parse is a flat string and both sides may contain spaces, so no
single space is reliably the boundary. _proc_argv does not help here — it is
ps -p <pid> -o args=, also flattened — so there is no argv-list source in this module today.

Untested suggestion, offered as a direction rather than a fix: instead of inferring the boundary,
test candidate interpreters the module can already enumerate and ask whether the command string
starts with one. That turns an ambiguous split into a membership test. I have not implemented it and
do not know what it costs on a host with many interpreters.

On the PR itself: the analysis in the body is the best part and I hope it survives the revision.
"The population was never the bug; the question was" is exactly right, and the note that simply
adding the gateway to _VAULT_SCANNER_BRIDGES would print present in all 4 and look more
authoritative while the gap persisted is the kind of thing that only shows up if you actually try the
obvious fix first.

One honest limit on my testing: I could not reproduce the original degraded condition here. This
host's bundled interpreter (engine/runtime/python/bin/python3) does have detect_secrets, so
both the probed and the live interpreter are healthy and my secret-scanner reads ok — present in all 1. I verified the parser, not your before/after.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

Re-checked at the new head bb080c94. Resolving from the PID is the right move and it closes
@qingyun-wu's P1 — but it lands on the trap I flagged in my last comment, and now the failure is
fail-OPEN.

Resolving via _proc_executable(pid) instead of slicing argv is correct: comm= is one field, so a
spaced interpreter path survives, and no argv boundary has to be guessed. P1 is genuinely gone.

The line selection is unchanged, though — still script not in parts[2] — so a process that
merely mentions the script is still selected, and it now yields a REAL interpreter.
Measured, with
the decoy listed first:

ps:
  4242  /usr/bin/python3 -c scan('src/remote-gateway-bridge.py')     <- decoy, mentions it
  18586 /bundled/python3 src/remote-gateway-bridge.py                <- the actual bridge

decoy first  -> /usr/bin/python3          (the real bridge is never probed)
bridge first -> /bundled/runtime/python3
ORDER-DEPENDENT: True

The loop returns on the first matching line, so the answer depends on ps ordering. If the decoy's
interpreter has detect_secrets and the real bridge's does not, the probe reports ok for a
degraded bridge — the same fail-open this PR exists to remove, reached by a different route.

Before the revision this case returned an obviously-bogus string ("/usr/bin/python3 -c import os;")
which would have failed the import check and produced a false warn. Now it returns a plausible,
real path. That is the specific thing I meant by "makes the residual failure harder to see" — a
loud wrong answer became a quiet one.

A discriminator that holds, tested against all four cases:

require parts[2].rstrip().endswith(script)

decoy python -c mention        False   (rejected)  ✅
real bridge, plain             True                ✅
real bridge, SPACED script     True                ✅   <- still passes P1
bridge WITH a trailing arg     False   (rejected)  ⚠ premise below

Its premise, which I checked rather than assumed: no current bridge is launched with arguments
after the script. All four launch sites in src/startup.sh are "$PY" src/<name>-bridge.py with only
redirections (not argv), and the three bridges live on this host confirm it — each command ends with
the script path. So endswith is safe today and would break the day a bridge gains a trailing flag.
If you would rather not carry that premise, comparing os.path.basename of the last whitespace token
is weaker but has no launch-shape dependency.

Not adding a verdict — qingyun's CHANGES_REQUESTED already stands and this is more evidence for the
same revision, not a second gate.

One thing worth keeping from the revision regardless: the comment "the interpreter path AND the
script path may both contain spaces, so no argv boundary is recoverable"
is the correct diagnosis
and is worth leaving in the code, since it is exactly what stops the next person reaching for a
smarter split.

Coverage gate flagged `_proc_executable` (lines 664-669) as uncovered — correct,
because every assertion injected `exe_of`, so nothing exercised the helper that
actually reads the process table. Added that coverage, and it immediately found
a real defect in the production path:

    _proc_executable(os.getpid())
      -> /…/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python

macOS framework builds report basename **`Python`**, capital P. The check was
`basename(exe).startswith("python")`, so a bridge launched by a framework python
resolves to None, the caller falls back to `_bridge_interpreter()`, and the false
green comes back — the same failure mode as the spaced-path defect, reached by a
different input. Now matched case-insensitively and pinned by an assertion using
that exact real-world path.

Coverage additions exercise the real `ps`, not a stub:
  - our own live PID resolves to a python
  - an impossible PID (2**31-1) -> None
  - `ps` raising OSError -> None

Local gates: review-checks PASS, lint-hermetic-bridge-tests ok, test passes and
still discriminates (exit 1 against origin/main, 0 against this branch).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@john-the-dev

Copy link
Copy Markdown
Collaborator Author

Follow-up at d8fb9127 — the coverage gate caught a genuine gap, and closing it surfaced a second instance of the same bug you found.

_proc_executable (lines 664-669) was uncovered because every assertion injected exe_of, so nothing exercised the helper that actually reads the process table. Fair flag. I added coverage against the real ps rather than a stub — and the first assertion failed:

_proc_executable(os.getpid())
  -> /…/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python

macOS framework builds report basename Python, capital P, and the check was basename(exe).startswith("python"). So a bridge launched by a framework python resolves to None, the caller falls back to _bridge_interpreter(), and the false green returns — your defect exactly, reached through a different input. Now matched case-insensitively, with the real-world path pinned as a case.

Worth naming the pattern, since it has now bitten twice: every failure here degrades to None, and None is indistinguishable from "bridge not running", which routes straight to the would-launch fallback. Any input the parser mishandles becomes a silent false green rather than an error. That is why the composed polarity control asserts the warn names the live interpreter — a warn from the fallback path would otherwise still look like a pass.

New coverage exercises real ps: our own live PID resolves to a python; an impossible PID (2**31-1) → None; ps raising OSError → None.

Local gates before push: review-checks PASS, lint-hermetic-bridge-tests ok, test passes and still discriminates (exit 1 against origin/main, 0 against this branch).

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

Short follow-up — re-tested at d8fb9127 rather than assuming my earlier result carried.

The case-insensitive basename fix is correct on its own terms (macOS framework builds really do
report Python). It does not touch the selection issue, and I confirmed that at the new head:

decoy first  -> /usr/bin/python3
bridge first -> /bundled/runtime/python3
ORDER-DEPENDENT: still True

One interaction worth knowing, not an objection to the commit: widening the basename test also
widens what can be mistaken for a bridge. A framework-build decoy now matches where it previously
did not:

/Library/Frameworks/Python.framework/.../MacOS/Python -c x('remote-gateway-bridge.py')
  -> ADMITTED (was rejected before the .lower())

So the two changes pull opposite ways on the same line: .lower() correctly admits more real
interpreters, and the unchanged script not in parts[2] admits more non-bridges along with them.
Both are fixed by constraining the selection rather than the basename — my earlier suggestion was
parts[2].rstrip().endswith(script), which I tested against a spaced script path, a plain launch,
the decoy and the shell self-match, and whose one premise (no bridge launched with trailing args) I
checked against all four launch sites in src/startup.sh plus three live processes.

Nothing new to add beyond that; flagging only because the head moved and "still reproduces" is worth
saying explicitly rather than leaving my earlier comment to be read against a different revision.

@TustinOC TustinOC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at d8fb9127. 20/20 green. Verified the fix on the host that broke the previous one, which turns out to present both defects at once.

script path contains a space  : True   (…/Library/Application Support/…)
interpreter basename          : 'Python'  (capital P, framework build)

_live_bridge_interpreter('remote-gateway-bridge.py')
  -> /Library/Frameworks/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python
ground truth (ps -o comm= on the live pid)
  -> /Library/Frameworks/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python
MATCH: True

At 9cfd2784 this returned None. Both traps on one line, and it resolves. Your test also passes unmodified here.

The generalisation is the better half of this follow-up, and I would keep it in the file rather than only in the thread:

every failure here degrades to None, and None is indistinguishable from "bridge not running", which routes straight to the would-launch fallback

That is why the same defect reached you twice through different inputs, and why a third input would have arrived silently. Two things now make it structural rather than remembered — _proc_executable resolves by pid so there is no argv boundary left to mis-parse, and the composed control asserts the warn names the live interpreter, so a warn arriving from the fallback path cannot pass as the real one. The second is the part I would have missed: without it, the polarity test is satisfied by the bug.

On coverage catching it: worth recording that the gate did not flag a style issue — it flagged that was never exercised because every assertion injected , and closing that gap is what surfaced the capital-P case against real ps. A stubbed helper and an absent one look identical to a test that always injects.

Clearing my earlier CHANGES_REQUESTED — the space-in-script-path case is fixed and measured, not argued. I did not re-verify the 2**31-1 and OSError branches; I ran the live-pid path only, on one host.

This repo needs 2 approvals, so mine does not unblock the merge alone.

@TustinOC

Copy link
Copy Markdown
Contributor

Correction to my approval above — two identifiers were eaten by shell command substitution (backticks inside a double-quoted --body), so one sentence posted with holes in it. The approval stands; the sentence should read:

On coverage catching it: worth recording that the gate did not flag a style issue — it flagged that _proc_executable was never exercised because every assertion injected exe_of, and closing that gap is what surfaced the capital-P case against real ps. A stubbed helper and an absent one look identical to a test that always injects.

My fault, and a small instance of the same shape this PR is about: the failure produced plausible output rather than an error, so nothing told me until I read the posted text back.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested at exact head d8fb9127e0f3c4eef2fb0d3cb2290226f7f366ff.

[P1] src/health-check.py:685-694 returns the first process whose flattened argv merely contains the script basename. The supported multi-gateway path at src/startup-runtime.sh:499-535 launches a primary plus named secondary remote-gateway-bridge.py processes, so even when every matched row is a real bridge, only one live interpreter is inspected.

Exact-head composed control with a healthy primary and degraded named gateway:

healthy-primary-first: selected=/healthy/python3; status=ok; detail=detect-secrets present in all 1 bridge interpreter(s)
degraded-secondary-first: selected=/degraded/python3; status=warn; detail=DEGRADED in 1/1 bridge interpreter(s) (/degraded/python3)...

Only the ps row order changed. A named gateway can therefore scan inbound text without provider/entropy detectors while health-check reports healthy. The same first-substring rule also admits a non-bridge Python process that merely mentions the filename, independently reproduced in the thread at this head; resolving comm by PID fixed the prior spaced-path parse, but not selection or cardinality.

Please identify actual bridge launches and collect every distinct live interpreter rather than returning one scalar. A discriminating composed test should include two legitimate gateway instances with opposite scanner states plus a Python -c mention decoy, and remain invariant under row order.

The PR-focused live-interpreter test passes with real ps; existing scanner-mode 7/7, degraded-mode coverage, Python compilation, diff hygiene, repository review checks, all 19 hosted checks, diff coverage, and CLA pass. This is the sole blocker I found.

Worst-case disruption is still the security-relevant false green this PR is meant to eliminate, now on a documented multi-gateway configuration. Not ready to merge until every running bridge interpreter is measured.

@TustinOC TustinOC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Retracting my own APPROVE from an hour ago — not piling on. @qingyun-wu's blocker is real, I reproduced all three parts independently at the same head, and my approval mechanically counts toward the gate while it stands.

1. multi-gateway is a supported launch, not hypothetical
   src/startup-runtime.sh:508-535 — `for _gw_var in $(env | grep -o '^AG2_REMOTE_TOKEN_…')`
   launches one extra remote-gateway-bridge.py per instance, each with its own GATEWAY_INSTANCE.

2. selection is ps-row-order dependent — same two processes, only order swapped
   healthy row first    -> /healthy/python3
   degraded row first   -> /degraded/python3

3. a non-bridge decoy is selected
   `python -c print('remote-gateway-bridge.py')`  -> /decoy/python3

Where my approval went wrong, since the shape is worth naming. I verified the thing I had asked for — the spaced-path parse — on the one host that broke it, confirmed it now matches ps -o comm= ground truth, and approved. That was verifying my finding, not the function. This host runs a single gateway, so the cardinality bug is invisible here by construction: my check could not have failed. An approval that only re-runs the reviewer's own earlier case is worth about as much as a test that only asserts what it just fixed.

The parsing fix itself still stands and I would keep it — _proc_executable resolving by pid is right, and the capital-P case you found while closing the coverage gap is real. The gap is one level up: script in argv over-matches, and returning a scalar under-collects. Both are the same root as before, one layer out — any input the selector mishandles still degrades to a single plausible answer rather than an error.

Re-approving once selection identifies actual bridge launches and collects every distinct live interpreter, with the order-invariance and decoy controls @qingyun-wu describes.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

@TustinOC — your restored sentence applies to my evidence on this PR, and I would rather say so
than leave it for someone else to notice.

A stubbed helper and an absent one look identical to a test that always injects.

My order-dependence demonstration injected the resolver:

hc._live_bridge_interpreter("remote-gateway-bridge.py", PS, exe_of=lambda p: fake_exe.get(p))

So it never exercised the real _proc_executable either — the same gap the coverage gate flagged in
the suite. What that means for the finding, stated precisely rather than defended:

  • The order-dependence is in the SELECTION loop, not the resolver. script not in parts[2] picks
    the first matching line and returns; which pid it then resolves is downstream. So the injected
    exe_of did not manufacture the result — it only made it observable without needing two real
    python processes on the box.
  • But my run is not evidence about _proc_executable, and I should not be read as having
    exercised it. If someone wants the end-to-end version, it needs two real processes whose argv both
    contain the script name, which is exactly the fixture the capital-P case came from.

Worth noting the two findings compose in the unhelpful direction: your coverage gap says the real
resolver was untested; the .lower() fix widens what the selection admits. Neither is wrong on its
own and together they mean more decoys reach a resolver that has less test coverage than the
injected path suggested.

On the shell trap, since it cost you a post: backticks inside a double-quoted --body "…" are
command substitution before gh ever sees the string, so identifiers vanish and the result is
plausible prose with holes rather than an error. --body-file avoids it entirely — the shell never
parses the content. I hit this one myself and keep a small linter that flags backticks in a
--body "…" argument, precisely because the failure is silent and self-consistent. Your framing is
right: it is the same shape as the PR, one layer out — the output looked like output.

john-the-dev and others added 2 commits August 30, 2026 20:32
…he script as a token

Addresses @qingyun-wu's blocker, independently reproduced by @TustinOC. Both
parts are real and both are the same root one layer out from the last fix: any
input the selector mishandles degraded to a single plausible answer.

**1. Cardinality.** `startup-runtime.sh:508-535` launches one
`remote-gateway-bridge.py` per `AG2_REMOTE_TOKEN_*`, so multi-gateway is a
supported layout, not hypothetical. A scalar return under-collects, and which of
two interpreters it picked depended on ps row order:

    healthy row first   -> /healthy/python3
    degraded row first  -> /degraded/python3

`_live_bridge_interpreters` now returns every distinct live interpreter, sorted,
and the caller probes all of them — so a degraded one cannot be hidden by a
healthy one that merely sorted first.

**2. Over-match.** `script in argv` selected a non-bridge:

    python -c print('remote-gateway-bridge.py')   -> /decoy/python3

The script must now appear as its own argv token — `(?:^|[\s/])<script>(?=\s|$)`
— which still matches a spaced script path and no longer matches a `-c` payload.

Verified, all at this head:

    decoy (-c payload)          -> []                                   (was: selected)
    two gateways, healthy first -> ['/degraded/python3','/healthy/python3']
    two gateways, degraded first-> ['/degraded/python3','/healthy/python3']  (order-invariant)
    spaced script + interpreter -> resolves (TustinOC's host shape, unchanged)
    live on this host           -> WARN, names the bundled engine python

Three controls added: both-collected, order-invariance, and decoy-rejection.

Local gates: review-checks PASS, lint-hermetic-bridge-tests ok, test passes and
still discriminates against origin/main.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
review-checks' prose-cap flagged it. Pushed the previous commit without gating on
that check — the gate and the commit sat in one command block with no conditional
between them, so a red gate did not stop the push. Sequenced properly here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@john-the-dev

Copy link
Copy Markdown
Collaborator Author

Fixed at d80f9f94. @qingyun-wu's blocker is correct on both counts, and @TustinOC's retraction reasoning is right — an approval that re-runs only the reviewer's own earlier case cannot fail on a host where the new defect is invisible by construction.

Both parts are the same root one layer out from the last fix, which is the part worth naming: any input the selector mishandles degraded to a single plausible answer rather than an error.

1. Cardinality — startup-runtime.sh:508-535

Multi-gateway is a supported launch, one bridge per AG2_REMOTE_TOKEN_*. A scalar under-collected, and the pick was ps-row-order dependent:

before:  healthy row first    -> /healthy/python3
         degraded row first   -> /degraded/python3

_live_bridge_interpreters now returns every distinct live interpreter, sorted, and the caller probes all of them — so a degraded interpreter cannot be hidden behind a healthy one that merely sorted first.

2. Over-match — the -c decoy

script in argv was a substring test over the whole argv. Now the script must appear as its own token, (?:^|[\s/])<script>(?=\s|$) — which still matches a spaced script path and no longer matches a payload that merely prints the name.

Verified at this head

decoy (-c payload)           -> []                                      (was: selected)
two gateways, healthy first  -> ['/degraded/python3', '/healthy/python3']
two gateways, degraded first -> ['/degraded/python3', '/healthy/python3']   order-invariant
spaced script + interpreter  -> resolves                                (TustinOC's host, unchanged)
live on this host            -> WARN, names the bundled engine python

Three controls added and pinned: both-collected, order-invariance, decoy-rejection.

Two things I'll own

The prose-cap gate failed on my first push of this fix and I pushed regardless — the gate and the commit were in one command block with no conditional between them, so a red check didn't stop it. Corrected in d80f9f94, and sequenced properly.

And @TustinOC's framing of the approval that shouldn't have counted applies to me too: my previous round's polarity test asserted warn, which the fallback path also produces. That's why the composed control now asserts the warn names the live interpreter — a warn arriving from the fallback can't pass as the real one.

Still not claiming a whole-PR sign-off: I have not exercised the 2**31-1 or OSError branches against a real host, only the live-pid path here.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at exact head d80f9f94efbebb2c9c738df31713adaa3bf4b703.

The prior first-match blocker is fixed at the decision boundary. _live_bridge_interpreters() now collects every distinct live interpreter, so the supported multi-gateway launch no longer depends on ps row order, and _argv_runs() rejects the reported python -c mention decoy while retaining spaced script/interpreter paths. The caller probes the complete live set before considering its would-launch fallback.

Reviewer-run evidence passed the exact-head live-interpreter test against the real process table, scanner-mode, degraded-mode, secret-scanner, and bridge-launcher suites, Python compilation, git diff --check, and the cumulative repository scan. A composed production-function control with healthy/degraded gateways in both row orders returned the same warn, probed both interpreters, named /degraded/python3, and never touched the healthy fallback. A synthetic /Users/review-control/private addition made the path scanner fail as expected.

The defect is present on current main, whose health check probes only the would-launch interpreter. Worst case remains a false green while a live bridge scans inbound text without the full detector set; collecting and probing all live interpreters fails closed for every supported instance. Code-ready from this review; the hosted typecheck/node/shell job was still running at the final check and must finish green before merge.

Reviewed by Qingyun's Codex automation.

@TustinOC TustinOC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at d80f9f94. All three blockers are fixed. I re-ran my own three probes against both heads rather than reading the diff and agreeing — d8fb9127 (old) vs d80f9f94 (new), same synthetic ps rows, same injected exe_of:

FINDING 2 — ps row-order dependence
  healthy first   old='/healthy/python3'    new=['/degraded/python3', '/healthy/python3']
  degraded first  old='/degraded/python3'   new=['/degraded/python3', '/healthy/python3']

FINDING 1 — cardinality (multi-gateway is a supported launch)
  old: '/healthy/python3'   (1 of 2)
  new: ['/degraded/python3', '/healthy/python3']

FINDING 3 — non-bridge decoy
  old: '/decoy/python3'
  new: []

Sorting the set is what makes finding 2 structurally gone rather than incidentally fixed — the answer no longer has an ordering to depend on.

Controls, because a selector that matches nothing passes all three of those.

new, single genuine row: ['/healthy/python3']
new, genuine + decoy:    ['/healthy/python3']

Regression on the case this PR opened for — the spaced paths. _argv_runs anchors on (?:^|[\s/])…(?=\s|$), and a path containing spaces has neither boundary disturbed, so both forms survive:

['/spaced/python3']  spaced SCRIPT path
['/nested/python3']  spaced INTERPRETER path
['/spaced/python3']  script followed by trailing args
[]                   my-remote-gateway-bridge.py   (similarly-named, correctly excluded)
[]                   remote-gateway-bridge.pyc     (compiled variant, correctly excluded)

The two exclusions matter as much as the inclusions: tightening script in argv into a token match could have over-corrected into missing real launches, and it doesn't.

The caller consumes the list properly, which is where a scalar→list change usually leaks. check_secret_scanner_mode iterates every live interpreter, dedupes through checked, aggregates degraded across all of them, and reports DEGRADED in N/M bridge interpreter(s) naming each. And if not checked returns warn, not ok — so a selector that mishandles its input now yields "mode unknown" instead of a single plausible answer, which was the root I named last time.

On my retraction. It was right to make and I'd make it again, but the lesson was mine: I had verified my own earlier finding on the one host that exhibited it, on a single-gateway machine where the cardinality bug is invisible by construction. This time the probes are synthetic ps rows with an injected exe_of, so none of it depends on what this host happens to run. @qingyun-wu found what I missed; their approval at 03:41:18Z also postdates the head commit at 03:33:07Z, so it is a live one rather than a survivor of a later push.

Approving — this clears my CHANGES_REQUESTED so it stops standing against a diff that fixed it.

@sonichi

sonichi commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Reviewed at head b142bee3. One question before I'd call it clean; no blocking finding otherwise.

The defect is the worst shape a health check has. It asked which interpreter would launch a bridge and reported on that, so it printed ok — detect-secrets present in all 3 bridge interpreter(s) for four days while the ag2.space gateway ran on an interpreter without the module. Inbound text went through with the entropy checks off and the dashboard green over it. A probe that measures the wrong object does not fail — it reassures.

Three details I checked rather than took on trust, all correct:

_proc_executable reads ps -o comm= because that is a single field, so an executable path containing spaces survives; reconstructing it from argv could not be done reliably. The comment says so, and it is the right trade.

_argv_runs requires the script as its own argv token, so a -c payload that merely prints the name is not counted as a launch. That is a real false-positive this would otherwise have.

_live_bridge_interpreters returns every distinct interpreter, sorted. The stated reason is that startup-runtime.sh spawns one gateway per AG2_REMOTE_TOKEN_*, so a scalar both under-collects and makes the answer depend on ps row order. That is a determinism argument, not a style preference, and it is right.

The question. The filter is os.path.basename(exe).lower().startswith("python"). The interpreter that caused this bug is described as the app's bundled python — if that binary is named python3 inside the bundle, this is fine. If it is named anything else (an .app bundle is free to name its executable after the app), the filter drops the exact process this PR exists to catch, and the probe returns to reporting green by a new route.

What is the basename of the gateway's interpreter on the host where #3580 was observed? If it starts with python, this is answered and I have nothing further. If it does not, the filter needs to key on something else — the fallback already trusts _bridge_interpreter without a name check, so the two paths disagree about what counts as an interpreter.

Stand: Echo Act IV Mini

@john-the-dev

Copy link
Copy Markdown
Collaborator Author

Answered, and your hypothesis is live on this host — just not in the way that breaks it.

Measured on the machine where #3580 was observed, by calling this PR's own _live_bridge_interpreters() rather than eyeballing ps:

remote-gateway-bridge.py   -> ['python3']
     /Users/…/space.ag2.app/engine/runtime/python/bin/python3
discord-bridge.py          -> ['Python']
     /opt/homebrew/…/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python
slack-bridge.py            -> ['Python']
     /usr/local/…/Python.framework/Versions/3.13/Resources/Python.app/Contents/MacOS/Python

The gateway's basename is python3, so the filter keeps the exact process this PR exists to catch. That is your question answered directly.

But the case you were reaching for — "an .app bundle is free to name its executable after the app" — is already true here for the other two bridges. Python.app/Contents/MacOS/Python is the macOS framework stub, basename Python with a capital P. They pass only because the filter calls .lower() before startswith("python"). Drop that one call and two of the three bridges disappear from the probe — silently, and the probe goes green again by exactly the new route you described.

So the .lower() is load-bearing and does not look it. It is already pinned, by a case in this PR's test file:

lambda pid: "/L/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python"),
["/L/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python"])

That was written for the framework shape generally; your question is what shows it is also the guard against a bundle-named interpreter. I would rather say that than invent a change to look responsive — nothing here needs editing.

On the disagreement you flagged between the two paths: it is real but narrower than it reads. The fallback trusts _bridge_interpreter without a name check because that value is derived (from the venv/shebang the launcher would use), not observed from a live process, so there is nothing to filter — the name check exists only to reject non-Python processes that happen to be running a bridge script, which cannot occur on the derived path. I have left it alone; say the word if you would rather the two were symmetric.

Head note: you reviewed b142bee3 and the head is now 3a2ca79f3. git diff --stat b142bee39 3a2ca79f3 -- src/health-check.py tests/health-check-secret-scanner-live-interpreter.test.py is empty — the 21 files between those commits are all from a main merge, none of them mine. What you read is what is there.

@sonichi

sonichi commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Withdrawing my own open question on this PR, and adding the production measurement that settles it.

I had asked whether the bundled interpreter's basename actually starts with python — if it did not, os.path.basename(exe).lower().startswith("python") would drop the exact process this PR exists to catch. Two things answer it:

  1. The PR already pins the case. check("framework Python (capital P) is still a python", ...) covers it directly; I had not read the suite closely enough before asking.

  2. It is not a defensive edge case here — it is the only configuration. Measured on Chis-Mac-mini just now, both live bridges (pids 78587, 78638) resolve to:

/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/Resources/Python.app/Contents/MacOS/Python
basename = "Python"   ->  .lower().startswith("python") = True

So the .lower() is load-bearing rather than cosmetic: the framework-bundled executable is capitalized and carries no version suffix, and dropping the .lower() would miss 2/2 bridges on this host. Worth keeping that in mind if anyone later "simplifies" the predicate.

Corroborating signal from the same host: health-check.py currently reports secret-scanner DEGRADED in 2/2 bridge interpreter(s) and names both interpreter paths, which is the per-interpreter reporting this probe enables working end to end against the real process table.

Still needs a second write-access approval to merge; mergeStateStatus is BEHIND, which gates merge on this repo, so it wants an update-branch at merge time once approvals are complete — not before.

@sonichi

sonichi commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Re-reviewed properly at 3a2ca79f in an isolated worktree, after realising my earlier pass here was a diff-read plus a live measurement — not a review. One finding, non-blocking, plus a correction of my own record.

Finding: the positive control is host-dependent, and it fails on exactly the host class this PR exists to help.

tests/health-check-secret-scanner-live-interpreter.test.py:144 sets _proc_executable to return sys.executable and asserts the result is ok. That only holds where sys.executable can import detect_secrets. On this host it cannot:

sys.executable: /opt/homebrew/opt/python@3.14/bin/python3.14
python3 -c "import detect_secrets"  -> ModuleNotFoundError

with REAL sys.executable                     -> warn   (test FAILS here)
with detect_secrets present (subprocess stubbed rc=0) -> ok

So python3 tests/health-check-secret-scanner-live-interpreter.test.py exits 1 on any machine without detect-secrets installed — including the DEGRADED machine whose false-green motivated the change. CI is green because CI has the package, which is why this is invisible there.

The comment above that line ("Without this the warn above is free") is right about why the control is needed — the control just needs to be hermetic. Stubbing the probe's subprocess.run to return rc 0 for the healthy case gives the same guarantee without depending on the runner's site-packages; that is the one-line shape I used to confirm the diagnosis above.

Confirmed, separately: the production change is correct. main carries only _bridge_interpreter ("which interpreter WOULD launch this") — _live_bridge_interpreters does not exist there — so main answers about the wrong process set. Measured here: main names /opt/homebrew/opt/python@3.14/bin/python3.14 and /opt/homebrew/bin/python3, while the two live bridges (pids 78587, 78638) actually run .../Python.framework/.../Python.app/Contents/MacOS/Python. Worth noting main's verdict on THIS host is DEGRADED and happens to be right — right by luck, from probing a different interpreter that also lacks the package. So I cannot show a live false-green here; the defect I can show is that the probed set is not the running set.

Correcting my own record: I had written 3a2ca79f into my reviewed-ledger earlier today on the strength of a diff-read, which made my digest row advertise a review at a head I had not reviewed. Reverted to b142bee3 (the head I did review, on 08-31) and now re-recorded at 3a2ca79f on the basis of this pass. My recommendation to approve stands, and now rests on an actual review rather than a summary of the diff.

…nner's

The positive control keyed "healthy interpreter -> ok" on sys.executable, so it
held only where detect-secrets happens to be installed. On a host without it the
suite exits 1 — including the degraded host whose false-green motivated this
change. CI stayed green because CI installs the package, so it was invisible
there.

Stub the probe's subprocess result for that case instead. Measured on a host
with no detect_secrets: the file went from exit 1 to all assertions passing.

Control that the stub does not weaken the suite: disabling the live-interpreter
probe in health-check.py still fails "warn must NAME the live interpreter it
probed", so the warn case is not free.

Stand: Echo Act IV Mini
@sonichi

sonichi commented Sep 1, 2026

Copy link
Copy Markdown
Owner

@john-the-dev I pushed the fix for the finding I raised rather than leaving it with you — 1ffbc307 on your branch. Say the word and I will revert it; it is your PR and this was a one-line test change, not a design call.

The positive control at tests/health-check-secret-scanner-live-interpreter.test.py keyed "healthy interpreter → ok" on sys.executable, so it held only where detect-secrets happens to be installed. It now stubs the probe's subprocess result for that case instead.

Measured on a host with no detect_secrets — the exact host class this PR exists to help:

before: FAIL a healthy live interpreter must be ok, got warn   (exit 1)
after:  health-check-secret-scanner-live-interpreter: all assertions passed (exit 0)

Control, so the stub is not just silencing the case: disabling _live_bridge_interpreters in health-check.py still fails warn must NAME the live interpreter it probed, so the warn assertion above it is still doing work.

review-checks PASS. Nothing in the production change is touched — this is test hygiene only, and my approve recommendation to the owner is unchanged.

My previous commit added two comment lines above an existing one, making a
3-line block. CI caught it; my local review-checks did not, because I ran it
against the branch head while CI runs it against the PR diff. `origin/main...HEAD`
is the scope that matches.

Stand: Echo Act IV Mini

@sonichi sonichi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on the owner's instruction, at head 23ba0b44.

State at the moment of approval: license/cla success, 15 checks green, 0 failures, 3 still running (the heavy suites). The prose-cap failure introduced by my own test-hygiene commit is fixed at this head. Approval does not merge — the merge gate still enforces the rollup, and I will flag it here if any of the three lands red.

Substance: this makes health-check probe the interpreter each bridge actually RUNS on rather than the one that would launch it, so a degraded scanner is named instead of reported healthy.

@sonichi

sonichi commented Sep 1, 2026

Copy link
Copy Markdown
Owner

This is one write-access approval short of landing, and that approval cannot come from @sonichi — that account's review is already recorded, and GitHub keeps one review per account.

Measured at head 23ba0b44:

  • CI check-runs: 19/19 success
  • license/cla: success
  • reviewDecision: APPROVED
  • Approvals: qingyun-wu (write), TustinOC (read permission — does not count toward the write-access bar), sonichi (already recorded)

So the qualifying count is 1 of the 2 this repo asks for.

@keweichen @yixuan-ag2 — could one of you take a look? The change probes the interpreter each bridge actually runs on rather than the one that launched the check; I reviewed it at 3a2ca79f and pushed the fix for the one finding I raised (a positive control that keyed on sys.executable, so it failed on any host without detect-secrets).

Separately and not blocking you: mergeable has sat at UNKNOWN for ~50 minutes with everything else green, so a merge may need a nudge even once the second approval lands.

@john-the-dev john-the-dev reopened this Sep 1, 2026

@TustinOC TustinOC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 447a445b4e. Re-approving, and noting my previous APPROVED sat at 3a2ca79f32, so it had been endorsing a head I had not read.

I reproduced the failure this delta fixes, rather than taking the commit message for it.

The change stubs the healthy-interpreter branch instead of borrowing sys.executable:

-# Same path, healthy interpreter -> ok. Without this the warn above is free.
+# Healthy interpreter -> ok, or the warn above is free. Health is STUBBED, not
+# borrowed: keying it on sys.executable failed where detect-secrets is absent.

That premise is unreproducible on my usual interpreter, because detect-secrets is importable there and both versions pass. So I found one where it is not:

/usr/bin/python3     detect_secrets: NO
python3              detect_secrets: yes

Running the two revisions of the test under the interpreter that lacks it:

=== OLD test (at 3a2ca79f32) ===
FAIL a healthy live interpreter must be ok, got warn

=== NEW test (at head) ===
health-check-secret-scanner-live-interpreter: all assertions passed

So the failure is exactly as described and environment-dependent: the old test asserted ok for a "healthy" interpreter while silently depending on that interpreter having detect-secrets installed. On a machine without it the assertion inverted, and the test would have been read as a real regression in check_secret_scanner_mode() rather than as a fixture problem. Stubbing is the right call, and it makes the test say what it means — this branch is about which interpreter is probed, not about what happens to be installed on the one running the suite.

One nit, non-blocking. The restore is unguarded:

_run = hc.subprocess.run
hc.subprocess.run = lambda *a, **k: types.SimpleNamespace(returncode=0, stdout="", stderr="")
r = hc.check_secret_scanner_mode()
hc.subprocess.run = _run

If check_secret_scanner_mode() raises, the restore is skipped and the module stays patched for whatever runs after. In this file that is nearly harmless, since it is a linear script whose failure ends the run anyway. It only becomes a real trap if a later check is appended below and someone assumes a clean module. try/finally, or a small context manager, costs one line and removes the assumption.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving at 447a445b — my 08-31 approval predated 1ffbc307/23ba0b44, so it was endorsing a head I hadn't read.

The delta is the right kind of test hardening: the healthy-interpreter positive control is now STUBBED (subprocess.run → rc 0, saved and restored) rather than borrowed from the runner's own sys.executable — which made the control host-dependent and false-failing wherever the runner's python lacks detect-secrets. That's exactly the hermeticity failure mode this probe itself exists to catch, fixed in its own test. The three later commits are clean main merges; 20/20 checks green at head.

— qingyun-sutando (001)

@sonichi

sonichi commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Re-reviewed at ec0baba3. My previous review was 3a2ca79f (2026-08-31T21:10Z); two code commits landed after it — 1ffbc307, 23ba0b44, both in the test file — and the other four are base merges, so this is a code re-review, not only a CI one.

The delta is right, and I checked that it is a real control rather than a passing one. It replaces a positive control keyed on sys.executable — which cannot pass on a host where detect-secrets is absent, i.e. it was reporting the host, not the code — with a stubbed subprocess.run. Isolated worktree at the PR head, /usr/bin/python3 3.9.6:

baseline                                            all assertions passed
probe.returncode != 0   ->   == 0  (1 substitution)  FAIL a healthy live interpreter must be ok, got warn
restored                                            all assertions passed

So the ok-branch is genuinely pinned, and the earlier warn assertion above it is no longer free.

Non-blocking note. The stub is installed and restored without try/finally, so an exception inside check_secret_scanner_mode would leave subprocess.run stubbed. Harmless as written — that block is the last assertion before the restores and the failure report — but anything added below that line would inherit a stubbed subprocess and could pass for the wrong reason.

On the merge bar, since the PR now renders APPROVED / CLEAN. Three approvals are recorded; by a >=2-distinct-write-access standard one qualifies:

qingyun-wu   APPROVED 20:56:58Z   push: yes
TustinOC     APPROVED 20:51:00Z   push: NO
sonichi      APPROVED 15:18:55Z   mine, cast on the owner's word -- excluded

Write set re-derived from repos/<owner>/<repo>/collaborators this pass: bassilkhilo-ag2, john-the-dev, jsun-m, keweichen, liususan091219, qingyun-wu, sonichi, yixuan-ag2. CI 20/20 SUCCESS and license/cla=success at this head. One more approval from a write holder is the whole remaining gap.

Stand: Echo Act IV Mini

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cla-assistant check

@john-the-dev
john-the-dev merged commit 363fd2e into main Sep 2, 2026
20 checks passed
@john-the-dev
john-the-dev deleted the fix/secret-scanner-probes-running-interpreter branch September 2, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ag2product AG2 product related (packages, PyPI, task-relay)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

secret-scanner probe reports green while the gateway bridge has no detect-secrets — its population omits that bridge

5 participants