Skip to content

fix(*): a job handed off on Windows outlives the host's exit - #1115

Merged
LivXue merged 10 commits into
mainfrom
fix/jobs_outlive_host
Nov 1, 2026
Merged

LivXue merged 10 commits into
mainfrom
fix/jobs_outlive_host

Conversation

@silverLXT

@silverLXT silverLXT commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A job the on-call agent hands off on Windows dies with the Raven host
mid-run. Allowing breakaway on the shared ACP job object is not a fix:
Git for Windows' MSYS2 runtime adds CREATE_BREAKAWAY_FROM_JOB to every
native program it starts when its job allows breakaway
(winsup/cygwin/spawn.cc), so an ordinary Bash tool command would outlive
the host too and the leak #951 closed would reopen.

This revision keeps the job object closed to breakaway. The host serves
a hand-off channel over a named pipe: the pipe's address and a random
key reach the agent in RAVEN_ACP_HANDOFF_PIPE and
RAVEN_ACP_HANDOFF_KEY, and the agent asks the host, which is not in the
job, to start the relay. The relay starts the launcher outside the job,
so the handed-off job outlives the host while nothing in the agent's
tree can leave the job.

The channel

  • raven/acp_client/windows.py starts one hand-off server per host
    process on the first ACP launch. It is a thread of the host, not a
    child of the agent, so what it starts is outside every job object the
    host owns. The pipe uses a 32-byte authentication key and an explicit
    DACL that grants full control to the launching user (its owner),
    Administrators, and SYSTEM; another local user cannot connect even
    before the key is checked.
  • launch() adds the pipe's name and key to the agent's environment.
  • oncall_flow/windows_backend.py asks the host to start the relay. A
    request carries the relay argv, the job directory and the agent's
    environment, so the job body keeps the environment it had before.
  • The request and the answer are length-prefixed JSON over the
    authenticated pipe (send_bytes/recv_bytes), not pickle; the host
    caps a request at 1 MiB and requires env to be an object. An empty
    request environment therefore stays empty instead of inheriting the
    host's.
  • With no channel (a host older than this change, or a backend run
    outside an ACP agent), the relay starts inside the job and its log
    says the host's hand-off channel was unavailable; this job was started inside a Windows job object and may end when that job closes.
  • If the request may have gone out but the answer did not come back,
    submit raises instead of falling back: the host may have started the
    job, and a second relay would run it twice. Before the request goes
    out, a deadline falls back to starting the relay locally instead.

Review follow-ups

The reviews found failure paths the first revision did not cover; the
branch carries the follow-ups from two review rounds:

  • c02041cb2 keeps the channel alive across a bad client. The accept
    loop now only accepts on a raw PipeListener; each connection gets
    its own thread for the key handshake and the request (bounded by a
    64-slot semaphore), and the request wait is bounded by poll. A
    client that aborts or stalls a handshake ends or blocks its own
    thread, not the only accept thread; the host retries a transient open
    failure and replaces a server whose accept loop died.
  • 1c7ef9713 frames the request and the answer as JSON instead of
    pickle, caps the request size, and requires env to be an object.
  • e881459ba bounds the client's exchange with a deadline and keeps
    the old local subprocess.run(timeout=60) bound: before the request
    goes out the caller falls back, after it may have gone out the caller
    raises. It also stops the local fallback from inheriting
    RAVEN_ACP_HANDOFF_*, writes the fallback note after the relay
    process returns (the second round tightens this to a successful
    start), and makes the Git Bash probe prefer Git's install over a WSL
    bash on PATH.
  • 44a42b02b adds unit coverage for the hand-off server's failure
    branches; without it the PR diff coverage gate failed at 72.32%
    against the 90% threshold.

Second review round

A second review found the pipe's default DACL, a blocking client wait,
and smaller defects in the failure paths; two commits close them:

  • 21ce1e1f9 creates every pipe instance with an explicit DACL for
    SYSTEM, Administrators, and the owner instead of the named-pipe
    default that also granted Everyone and Anonymous read access. Another
    local user could otherwise hold the 64 handshake slots without the
    key and force every later hand-off onto the in-job fallback. The
    accept loop now backs off and gives up after five consecutive errors
    so a dead listener can be rebuilt, and close() wakes a blocked
    accept instead of leaving it serving one more request. The
    unreachable recv_bytes None branch is gone: the stdlib raises
    OSError when a request exceeds the cap.
  • ce2a7b98c runs the relay start through asyncio.to_thread, so a
    wedged host cannot freeze the on-call agent's event loop; the client
    deadline is now 100 seconds, longer than the host's 30-second request
    plus 60-second relay budget. It resolves the job directory to an
    absolute path before the request, since the host resolves a relative
    cwd against its own working directory, and writes the unhosted note
    only after a successful relay. The channel-free environment is built
    in one helper, test results are published atomically and read only
    when parseable, and CONTEXT.md scopes the host-started claim to the
    hosted path.

A remaining design point is not changed here: the channel is injected
into every locally launched ACP agent, not only the on-call backend, so
any agent that holds the key can ask the host to start a process
outside the job. The risk section below treats that as the same
capability the agent already has as the user, not a new boundary.
tools/ops.py also still treats the new "may have started the job"
error as a refusal when it reports "Nothing is running"; that caller
needs a distinct indeterminate state, which is not part of this PR.

Type

  • Fix

Verification

Windows 11, CPython 3.12.13:

pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q -n 0
79 passed

pytest tests/test_acp_client_launch.py tests/test_acp_client_teardown.py tests/test_subagent_acp.py -k close_without_killpg -q -n 0
2 passed, 378 deselected

pytest tests/test_node_client.py -q -n 0
57 passed, 37 skipped

pytest tests/integration/test_acp_stdio_smoke.py tests/integration/test_subagent_connect_windows_real_process.py -m integration -q -n 0
13 passed

ruff check and ruff format --check over the changed files are clean.
ty check on the two changed source files is clean. lint-imports
keeps all ten contracts. The new head's checks all passed (2 skipped:
docs deploy and the claude job), including the Windows ACP job (79
passed), the four Ubuntu unit shards, the coverage gates (diff
coverage 92.14% against the 90% threshold), and the commit-message
check.

The new tests cover:

  • a Git Bash grandchild under the agent's job reports
    IsProcessInJob true, and does not survive the job object closing;
  • a handed-off job still writes its result after the host closes the
    agent's job object;
  • with no channel, the job starts inside the job and the log says it
    ends when that job closes, and closing the job ends it mid-run;
  • a real client that aborts the handshake and a real client that
    connects and sends nothing both leave the server able to serve the
    next request (the accept loop survives);
  • the hand-off server's pipe name, key, request and answer, its failure
    answer, its request-size/object/env validation, and a failed
    handshake that must not stop the server;
  • the producer and consumer spell the two environment variables the
    same way;
  • a lost answer after the request is not retried locally, and a host
    that never answers is bounded by the deadline: before the request
    goes out it falls back, after it raises;
  • the local fallback starts without RAVEN_ACP_HANDOFF_* and writes
    its note only after the relay started;
  • a transient hand-off open failure is retried, and a dead server is
    replaced;
  • the hand-off pipe's DACL grants no access to Everyone or Anonymous,
    and an accept loop that keeps failing gives up for a rebuild;
  • a closed server accepts nothing more, and an oversized request takes
    the stdlib's length error;
  • the host request environment carries neither hand-off variable, and
    the client deadline outlasts the host's request plus relay budget;
  • a relative remote_dir becomes an absolute job directory, and submit
    runs the relay off the event loop;
  • the fallback note is not written for a relay that exits non-zero,
    and a result file that is still being written is not read early;
  • the server's close, accept-loop error/slot/thread-start branches, the
    inline key handshake, and the request's poll/validation errors (the
    diff coverage gate).

Mutation checks, each watching its test fail:

  • before the change, the Git Bash test fails: the grandchild is outside
    the job and writes after the job closes;
  • before the accept-loop fix, the real abort test fails: the accept
    thread is gone and the next client hangs; the real stalled-handshake
    test fails the same way;
  • before the JSON change, the server tests fail on the pickled
    recv/send (the fakes assert they are never called);
  • before the deadline, both never-answer tests fail with the calling
    thread still alive after the join;
  • before the filter and note reorder, the fallback tests fail on the
    inherited key and on a log written for a relay that never started;
  • with the _HANDOFF_TRIED latch, the retry and dead-server tests fail;
  • with the named-pipe default DACL, the DACL test fails on the
    Everyone ACE; without the accept failure limit, it counts 50
    attempts instead of 5; without the close wake, the closed-server
    test leaves the accept thread alive;
  • with the note written unconditionally, the non-zero relay test finds
    job.log; with the relative path left as is, the absolute-path test
    fails; with a synchronous relay start, the event-loop test sees the
    main thread; with the 60-second deadline, the ordering test fails;
    with the old existence-only read, the still-writing test raises
    JSONDecodeError;
  • without the coverage tests, the CI diff coverage gate fails at
    72.32%.

Known failures on this machine, reproduced on the unmodified head and
not caused by this change: tests/test_subagent_acp.py has the same 19
failing ids at the base and at the previous head (the diff of the id
sets is empty); tests/test_cli_agents_commands.py has the same four
filesystem failures at the base; tests/test_acp_mirasim_server.py
stops at test 47 at the base and at the previous head. The new head's
four Ubuntu unit shards and Windows ACP job passed; the Windows ACP
job runs the two hand-off test files.

Risk

  • Security impact considered: the pipe is created with an explicit
    DACL for SYSTEM, Administrators, and the owner, so another local user
    cannot connect before the key is checked. The channel can start a
    process outside the agent's job, but only when a process in the
    agent's tree asks it to, with the key from its environment. This is
    narrower than a breakaway permission, which Git for Windows spends
    automatically on every native child. An agent already runs as the
    user; kill-on-close was cleanup, not a boundary. The request/answer
    framing is JSON over the authenticated pipe, so a key holder cannot
    execute code in the host by sending a pickle; the payload is limited
    to 1 MiB.
  • Backward compatibility considered: POSIX is untouched. On
    Windows, a host without the channel gets the old behaviour plus one
    log line; the channel and its JSON framing are new in this branch, so
    there is no older wire format to keep.
  • Rollback path is clear: revert the follow-up commits from both
    review rounds and the branch returns to the host-relay design; revert
    the whole branch and jobs go back to ending with the host.

Related Issues

#951, #1016

@silverLXT
silverLXT requested a review from LivXue October 26, 2026 11:13

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The breakaway permission is opt-in at process creation, and the detached on-call relay is the only changed caller that requests it. The fallback preserves the prior launch behavior when breakaway is refused, while the new Windows test exercises the host job-object close path that exposed the bug.

Reviewed the repository rules and domain map, the full diff, the ACP launch and on-call submit callers, relevant history, backward compatibility and nested-job behavior, test strength, and the architecture boundary. I found no weakened tests or plain regression.

Verification: uv run --frozen --python 3.12 --extra dev pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q completed with 32 passed and 5 Windows-only skips on this Linux host. The native Windows assertions were therefore reviewed but not independently executed here. Focused Ruff checks also passed.

#951 keeps each ACP agent in a kill-on-close job object so nothing it
started is left behind when the host exits. A job the on-call agent hands
off is meant to be left behind: the windows backend starts it through a
relay that exits at once, so no parent chain leads to it (#1016). But job
object membership is inherited, not traced through parents, so the job was
still in the agent's job object and died the moment the host exited, mid-run.

The job object now allows a process to leave it (JOB_OBJECT_LIMIT_BREAKAWAY_OK),
and only the relay asks to (CREATE_BREAKAWAY_FROM_JOB); everything else an
agent starts, the agent included, still ends with the job. Where leaving is
refused, the relay starts the job as before and its log says the job ends if
the host exits; the log says the same when the job is still in a job object
after asking to leave, which a launcher with a job object of its own between
the host and the relay (a virtual environment's python.exe) can cause
without an error.

Tests: an agent started through raven.acp_client.windows.launch submits a
job, the job object is closed: the agent ends, the job writes its result
(fails on the old relay). With leaving not allowed the job still starts and
its log carries the note.
@silverLXT
silverLXT force-pushed the fix/jobs_outlive_host branch from 949d209 to 33e38e4 Compare October 26, 2026 11:25
@silverLXT silverLXT changed the title fix(acp_client,oncall): a job handed off on Windows outlives the host's exit fix(*): a job handed off on Windows outlives the host's exit Oct 26, 2026
@silverLXT

Copy link
Copy Markdown
Contributor Author

Commit message and title fixed for the commit messages and pull request title checks: the scope was acp_client,oncall, which is not one scope from the list. It is now fix(*), as in #1016. The code is unchanged: 949d209d -> 33e38e4c (amended, git diff 949d209d 33e38e4c is empty). Both checks pass locally with scripts/check_pr_title.py and scripts/check_commit_messages.py.

The failing unit (py3.12 / ubuntu-latest / 1/4) was tests/test_node_stderr_replay.py ("did not happen in time" by 0.02 s, then an agent left under the stand-in daemon). It does not touch the two changed files, and the change only acts on Windows.

@gloryfromca thanks for the review: agreed on all three points. Breakaway is opt-in at creation, the relay is the only caller asking for it, and the fallback keeps the prior launch. The new Windows test starts its agent through raven.acp_client.windows.launch, the path the host uses, rather than assigning a running process to a job object. I made it so after finding that the latter can leave the agent's real interpreter outside the job when a venv python.exe sits in between.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The amended revision has the same tree as the previously reviewed head (ffd543c09d5506e1e55f50bb965166dce1cbbbb7); only the commit and PR scope changed to the valid cross-scope (*). I verified the old-to-new diff is empty and the current title and commit message pass the repository check scripts.

This revision check covered the tree/diff, repository metadata rules, current comments and threads, and focused tests. The prior caller, history, backward-compatibility, test-strength, and architecture assessment remains applicable because the code tree is identical.

Verification: uv run --frozen --python 3.12 --extra dev pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q completed with 32 passed and 5 Windows-only skips on this Linux host.

@0xKT

0xKT commented Oct 26, 2026

Copy link
Copy Markdown
Member

Not a blocker (disclosure for the merger, from my acceptance pass on head 33e38e4 merged onto main 4718421. I reviewed it by reading; the Windows test record is this PR's CI, which includes the windows-latest matrix.)

Severity of this one finding, not a verdict on the pull request. Any blocking findings from this pass are review threads on the changed files; GitHub renders those collapsed, as a file name with no text.

A deliberate loosening of #951's guarantee, worth accepting knowingly. JobObject now sets JOB_OBJECT_LIMIT_BREAKAWAY_OK (raven/acp_client/windows.py:104). The only caller is launch() (:147), so the setting applies to every ACP agent Raven starts on Windows. As a result, any process in an agent's tree that is created with CREATE_BREAKAWAY_FROM_JOB now outlives the host, not only the on-call relay's job.

I do not think this changes what an agent can do:

  • An agent already runs as the user and could leave the job by other means: WMI Win32_Process.Create, the Task Scheduler, or a service.
  • Kill-on-close was clean-up, not a boundary.
  • The default still holds. Without SILENT_BREAKAWAY_OK, nothing leaves unless it asks, so an agent's ordinary children still end with the job.

What changes is the clean-up promise. A third-party agent, or a tool it runs, that happens to pass CREATE_BREAKAWAY_FROM_JOB now leaks past the host's exit.

Two smaller points:

  • The relay's "still in a job object" note fires whenever the relay sits in any job. The check is IsProcessInJob(h, None, ...) (windows_backend.py, relay), so it is true whenever any job object contains the relay. Examples are a terminal, IDE or CI runner that keeps its own job, or a venv python.exe. In those setups the job log can say "this job ends if the host exits" for a job that has in fact left the host's job object. The wording hedges ("if it is the host's"), so this is only noise in a log a person reads after a failure. Checking against the host's job, if the relay can be handed a handle to it, would make the note exact.
  • No CHANGELOG entry, for a user-visible fix: Windows on-call jobs no longer die with the host.

The relay's check after starting a job called OpenProcess, IsProcessInJob
and CloseHandle through ctypes.windll with no declared types. ctypes takes
an undeclared return value as a 32-bit int, so on 64-bit Windows the process
handle could be cut short and the wrong handle passed on. It only fed the
"still in a job object" note, never the job's start, but it should not rely
on handle values being small. The three functions are now declared as
raven.acp_client.windows declares them: pointer-wide handles, int results.
@silverLXT

Copy link
Copy Markdown
Contributor Author

New commit 33ebea48: the relay's after-start check called OpenProcess, IsProcessInJob and CloseHandle through ctypes.windll with no declared types. An undeclared return is taken as a 32-bit int, so on 64-bit Windows the process handle could be cut short and the wrong handle passed on. It only fed the "still in a job object" note, never the job's start, but it should not rely on handle values being small. The three functions are now declared as raven.acp_client.windows declares them: pointer-wide handles, int results.

Checked: tests/test_acp_windows.py and tests/test_agents_oncall_flow_windows_backend.py, 37 passed (Windows 11). Through raven.acp_client.windows.launch, closing the job object still leaves the job running and ends the agent, and the note is still written.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The new delta correctly declares pointer-width-safe Win32 handle signatures for the relay's diagnostic check. It does not alter process creation, breakaway, or fallback behavior, and I found no new defect.

For this revision I reviewed the ctypes delta, its surrounding relay control flow, the repository rules, compatibility with the existing Windows bindings, test strength, current comments, CI, and review threads. The earlier full-diff, caller, history, backward-compatibility, and architecture review remains applicable to the unchanged remainder.

Verification: the focused pytest command completed with 32 passed and 5 Windows-only skips on Linux; focused Ruff, relay compilation, and git diff --check passed. The current Windows CI job was still pending when reviewed.

@LivXue

LivXue commented Oct 26, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — PR #1115 @ 33ebea48a23983084cb44e97e5f8219209c60200

Scope: full base-to-head diff (4 files: raven/acp_client/windows.py, agents/raven-oncall/plugins/oncall-flow/oncall_flow/windows_backend.py, and two test files), the callers (JobObject, windows.launch, WindowsExecutor.submit, the relay), and existing review threads. Reviewed at this exact head; base-to-head only.

Result: no Windows compatibility defect found.

What I checked:

  • JOB_OBJECT_LIMIT_BREAKAWAY_OK (0x0800) is added next to KILL_ON_JOB_CLOSE on the host's job object; the on-call relay starts the handed-off launcher with CREATE_BREAKAWAY_FROM_JOB and falls back to launching inside the job on OSError, writing a line to job.log. Both constants exist on CPython/Windows; neither path is reachable on POSIX.
  • The diagnostic OpenProcess/IsProcessInJob/CloseHandle calls declare pointer-wide handle types (the fix in 33ebea4), so a 64-bit handle is not truncated. log is opened binary ("ab"), matching the bytes writes.
  • The containment tradeoff is real and intended: BREAKAWAY_OK applies to every process in the agent's tree, so any descendant that asks can now outlive the kill-on-close job, not only the on-call relay. The existing taskkill /T tree-kill test still passes, so the ordinary termination path is unchanged.

Verification (native Windows 11, isolated detached worktree D:\tmp\rv-1115 pinned to this head, own venv, HOME/USERPROFILE pointed at a scratch dir):

PYTHONUTF8=1 .venv/Scripts/python.exe -m pytest \
  tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q
→ 37 passed in 24.64s

The run includes the two new Windows-only tests — a job surviving the host closing the agent's job object, and a job still starting (with the log note) when breakaway is refused — plus the pre-existing tree-kill test.

Non-blocking observation: the relay catches bare OSError around the first Popen. A launch failure unrelated to breakaway (e.g. a missing interpreter) also writes could not leave the agent's job object; this job ends if the host exits before the retry raises; narrowing the catch to WinError 5 (ERROR_ACCESS_DENIED) would keep that diagnostic honest. Cosmetic, not correctness.

Limitations: verified on this one Windows 11 / CPython 3.12 host only; outer-job and nested-job arrangements were not reproduced locally (the PR body documents them as measured).

@LivXue

LivXue commented Oct 27, 2026

Copy link
Copy Markdown
Member

Blocking: 1 finding needs correction before this revision can merge; it is stated below rather than inline, because no changed file is in my review scope.

Scope: my review covers the raven/agent/** portion of a PR -- 0 of the 4 changed files here. All four files, their callers (raven/acp_client/client.py launch path, WindowsExecutor.submit, the relay) and the existing comments were read as context; the findings below are disclosed observations on files outside that scope, which is why none is filed inline.

This supersedes the "no Windows compatibility defect found" result of the Windows compatibility comment posted from this account earlier on this head (6421127503): that run did not exercise a process tree under Git Bash, which is where the finding below lives.

Confirmed findings:

  • Blocker - keep the host's job object closed to breakaway, or confine the permission to the hand-off. JobObject now sets JOB_OBJECT_LIMIT_BREAKAWAY_OK for every ACP connection on Windows (raven/acp_client/windows.py:104; the sole caller is launch()). @0xKT has already disclosed that this lets any process passing CREATE_BREAKAWAY_FROM_JOB outlive the host, and weighed it on the premise that an agent's ordinary children never ask. One common runtime always asks: the Cygwin/MSYS2 runtime behind Git for Windows' bash.exe adds CREATE_BREAKAWAY_FROM_JOB to every process it spawns whenever its own job allows breakaway (git-for-windows/msys2-runtime@81d9bd3d1c36, winsup/cygwin/spawn.cc:444-473: an IsProcessInJob / QueryInformationJobObject check that ORs the flag in when LimitFlags has JOB_OBJECT_LIMIT_BREAKAWAY_OK or JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK; upstream Cygwin reaches the same result through enforce_breakaway_from_job in setup_user_rlimits). On main the host's job refuses breakaway, so those children end with the host; on this head they leave it. Claude Code runs its Bash tool through Git Bash on Windows, so every command it starts there (a dev server, a watcher, a long test run) now survives the host's exit -- the leak fix(acp_client): end Windows agents when their host exits #951 closed. How I know: code-reading of that source and this diff; I have not run it on Windows, and the Windows runs reported on this PR (37 passed) do not include a process tree under Git Bash. A two-minute check: through raven.acp_client.windows.launch, start C:\Program Files\Git\usr\bin\bash.exe -c "'<python>' '<script that sleeps 6 s, then writes a file>'; true", close the job after 2 s, and see whether the file appears; with windows._BREAKAWAY_OK = 0 (main's flags) it should not. Remedy direction, not a patch: keep the shared host job closed to breakaway and give the hand-off a route that does not depend on the agent's job allowing it (the comment in WindowsExecutor.submit already notes that processes a system service started ran on), or confine BREAKAWAY_OK to the connections that declare they hand work off, and record the cost in CONTEXT.md.

  • Nonblocking - update the two places that still promise the whole tree ends. CONTEXT.md:3238-3243 (the Windows paragraph of the ACP launch entry) says closing the job's only handle ends "its processes and descendants", and the module docstring at raven/acp_client/client.py:9-11 says "the whole tree ends even if the host dies before teardown". This PR updates the docstring of raven/acp_client/windows.py but not these two, so the domain map now states a guarantee the code no longer gives. Code-reading.

  • Nonblocking - run the new Windows tests in CI. Neither tests/test_agents_oncall_flow_windows_backend.py nor tests/test_acp_windows.py is in the Windows ACP stdio job (.github/workflows/ci.yml:384-460, which lists its files explicitly), and the Ubuntu unit jobs skip the five Windows-only tests. So the regression test for this fix, the tree-kill test from fix(*): run on-call campaign jobs on a Windows host #1016, and fix(acp_client): end Windows agents when their host exits #951's tests/integration/test_acp_windows_real_process.py have only run on personal machines. That is easy to misread: one disclosure on this PR describes the Windows test record as this PR's CI. Adding these files to that job would make it so.

Verification:

  • Read git diff 9efb8799..33ebea48 in full; checked the 33e38e4c..33ebea48 delta (typed OpenProcess / IsProcessInJob / CloseHandle) against the declarations in raven/acp_client/windows.py:76-88: correct.
  • pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q -n 0 on a Linux checkout of 33e38e4 (the tests are unchanged at this head): 32 passed, 5 skipped.
  • Grepped .github/workflows/*.yml for both test files: no match.
  • The msys2-runtime behaviour is from its source at the commit cited above; not run on Windows.

Already filed, so not repeated here: the bare except OSError around the first Popen (the Windows compatibility comment above), the "still in a job object" note firing inside any job and the missing CHANGELOG entry (@0xKT), and the undeclared ctypes types (fixed in 33ebea4).

@gloryfromca

Copy link
Copy Markdown
Member

Blocking: Git Bash descendants can escape the shared ACP job and survive host exit.

The new evidence changes my previous clean stance. I independently confirmed in the cited Git for Windows MSYS2 source that, when the current job advertises JOB_OBJECT_LIMIT_BREAKAWAY_OK, the runtime automatically adds CREATE_BREAKAWAY_FROM_JOB to spawned native processes. This makes the containment regression reachable through ordinary Bash tool commands, not only through a deliberately breakaway-aware program, and recreates the leak that #951 intended to close.

I am not re-grading @LivXue's finding; this corrects a factual premise of my earlier assessment. The focused tests still complete with 32 passed and 5 Windows-only skips on Linux, but they do not cover a Git Bash descendant tree.

Comment thread raven/acp_client/windows.py Resolved
A handed-off job has to outlive the host, but the shared ACP job object
cannot allow breakaway: Git for Windows' MSYS2 runtime adds
CREATE_BREAKAWAY_FROM_JOB to every native child when the job permits it,
so an ordinary Bash tool command would leak past the host again.

The job stays kill-on-close with no breakaway. The host serves a named
pipe, whose address and key reach the agent in its environment, and
starts the hand-off relay outside the job on the agent's request. With
no channel the relay starts inside the job and the job's log says it
ends with the host; a request whose answer is lost is not retried
locally, so one job cannot run twice.

Tests: a Git Bash descendant stays in the agent's job; a handed-off job
outlives the closed job object; the fallback starts inside the job and
says so; the fallback dies with the job; the channel protocol, its
failures and the no-duplicate path; the producer and consumer variable
names.
@LivXue
LivXue force-pushed the fix/jobs_outlive_host branch from 780f331 to a3df989 Compare October 31, 2026 16:44

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the newly added handoff-server tests must pass in the Ubuntu unit suite.

The host-side handoff architecture fixes the Git Bash containment blocker: the shared job again forbids breakaway, the relay is launched outside it, the documented contract is reconciled, and native Windows coverage is added to CI. One newly introduced portability failure remains, marked inline.

This satisfies the late-round blocker bar: this revision introduces the failure, every Linux execution of the affected tests reaches it, and there is no working route to a green required unit suite without correcting it.

I reviewed the current base-to-head diff after its rebase, the handoff protocol and lifecycle, callers, failure and duplicate-start behavior, repository rules and domain terms, backward compatibility, test strength, history/comments, and the architecture boundary. Focused pytest completed with 2 failed, 42 passed, and 7 Windows-only skipped; focused Ruff and git diff --check passed.

Comment thread raven/acp_client/windows.py Resolved
`_HandoffServer._serve` passed `subprocess.CREATE_NO_WINDOW` as a keyword
value, which does not exist on POSIX, so the two unit tests that mock
`subprocess.run` raised AttributeError before reaching the mock. The flag
is now read with getattr(..., 0), and the tests delete the attribute so
the POSIX shape is exercised on every host.
@LivXue

LivXue commented Oct 31, 2026

Copy link
Copy Markdown
Member

The shared-job breakaway finding is fixed at a3df98946 (head
d02c8e226719345a19ec05bb0b9cc04834f07373): the job is
KILL_ON_JOB_CLOSE only, the host starts the relay outside it over a
named-pipe channel, and
test_a_git_bash_descendant_stays_in_the_agents_job pins the descendant
case (it fails on the old head). The follow-up you filed on the Linux flag
is fixed in d02c8e2 and that thread is resolved. Details are in the inline
reply.

@LivXue

LivXue commented Oct 31, 2026

Copy link
Copy Markdown
Member

Both blocking points are addressed at head
d02c8e226719345a19ec05bb0b9cc04834f07373.

  • The shared ACP job is back to JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE only.
    Nothing in the agent's tree can leave it;
    test_a_git_bash_descendant_stays_in_the_agents_job fails on the old head
    and passes here.
  • The relay is started by the host, which is not in the job, over a named
    pipe whose address and key reach the agent in the environment. The
    handed-off job outlives the host. With no channel the relay starts inside
    the job and the log says it ends with the host; a lost answer is not
    retried locally, so the job cannot run twice.
  • CONTEXT.md now says the job has no breakaway and describes the channel;
    client.py's whole-tree claim is true again; CHANGELOG.md has the entry.
    The bare OSError branch and the misleading in-any-job note are gone.
  • tests/test_acp_windows.py and
    tests/test_agents_oncall_flow_windows_backend.py are a step in the
    windows-acp CI job. The POSIX flag follow-up is fixed in d02c8e2; the
    new unit tests run in the Ubuntu shards.

Windows 11 / CPython 3.12: 51 passed over the two files. The two inline
threads are resolved.

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The portability blocker is fixed: CREATE_NO_WINDOW now safely falls back to 0, and the two affected tests force the POSIX attribute shape. The focused suite reports 44 passed and 7 Windows-only skips; Ruff and git diff --check pass.

This round covered the repository rules, the complete delta from a3df98946e94, the affected handoff method and tests, compatibility with POSIX test collection, test-strength preservation, and the prior host-side handoff architecture and history. The delta does not alter callers or the architecture reviewed in the preceding round. I found no new defect.

@LivXue

LivXue commented Oct 31, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — PR #1115 @ d02c8e226719345a19ec05bb0b9cc04834f07373

Scope: the PR's own diff at this head is b05267b2..d02c8e2267 (7 files, +645/−24): a host-side hand-off server in raven/acp_client/windows.py, its pipe name and key injected into the agent's environment at launch, the on-call flow backend sending the relay start to the host instead of spawning it inside the kill-on-close job (with an in-job fallback plus a log note), explicit kernel32 ctypes declarations, CI/docs, and tests.

Finding (low–medium, reproduced on Windows): the new hand-off subprocess calls decode relay output with the ANSI code page, not the UTF-8 Raven sets for its process tree.

  • Where: raven/acp_client/windows.py:186-195 (_HandoffServer._serve, subprocess.run(..., text=True, ...)), and the fallback agents/raven-oncall/plugins/oncall-flow/oncall_flow/windows_backend.py:155-166 (_start_relay).
  • Scenario/mechanism: raven/cli/entry.py:48 sets PYTHONIOENCODING=utf-8, so the relay Python child writes UTF-8 to the pipes; the host's subprocess.run(text=True) decodes them with locale.getpreferredencoding(False), which on a Chinese Windows host is cp936. A non-ASCII message (e.g. a Chinese path in a traceback) then fails to decode in the reader thread.
  • Evidence (native Windows 11, cp936, utf8_mode=0): a child writing the UTF-8 bytes of 错误:找不到指定路径 to stderr, read by subprocess.run(..., text=True, capture_output=True), prints UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 4 from the reader thread and returns stderr=None (return code 0). In _serve that answer reaches the agent as "stderr": null; _start_relay then raises JobBackendError("could not start the job ...") with its reason clause empty — the diagnostic this channel exists to deliver.
  • Remedy direction: pass encoding="utf-8", errors="replace" to both new subprocess.run calls (and the same for the other new text=True reads), matching the UTF-8 the Raven entry point already sets for its tree.

Verification (native Windows 11; detached worktrees D:\tmp\rv-1115b at head and D:\tmp\rv-base-1115b at base b05267b2; offline venv; scratch HOME/USERPROFILE; PYTHONUTF8=1; PYTHONPATH pinned):

  • uv run --no-sync pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q -n 0 --tb=line
    • head: 51 passed (58.1 s)
    • base: 35 passed (25.4 s)
    • the new Windows job/hand-off tests pass, including test_a_job_outlives_the_host_closing_the_agents_job_object and the git-bash job-containment cases; the suite warns only about >3 s idle sleeps in the job-lifetime tests.
  • The decode probe above was run with the host's Miniconda Python 3.13.12 (cp936); the repo venv (3.12.13) has the same preferred encoding.

Coverage gaps: no live end-to-end run of host + ACP agent + pipe + detached job was driven outside the test stubs; the pipe's cross-process behaviour under a real host exit was not exercised; no other locale was tested.

LivXue and others added 3 commits November 1, 2026 12:21
The one accept thread ran the key handshake inline and returned on the
EOFError/BrokenPipeError a client raises by disconnecting mid-handshake;
a client that connected and sent nothing blocked it for as long as it
lived. The cached server kept advertising the dead pipe, so later submits
connected to a stranded instance and hung forever instead of falling back.

Accept on a raw PipeListener, run each connection's handshake and request
in its own thread (bounded by a slot semaphore), and bound the request wait
with poll(). The host recreates a server whose accept loop has died and
retries a transient open failure on the next launch.

Regression tests cover a real aborted handshake, a real stalled handshake,
the retry, and the dead-server replacement.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The host decoded the agent's request with Connection.recv(), which
unpickles it before any schema check; a process holding the pipe key
could therefore run code inside the host instead of only asking it to
start a relay. Send the request and the answer as length-prefixed JSON
over send_bytes/recv_bytes, cap the request size, and require the env to
be an object so an empty request can no longer inherit the host's
environment.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…fallback

_handoff_answer ran Client's blocking handshake, send, and recv on the
agent's event-loop thread with no deadline. When the host or its accept
loop was wedged, ops_submit never returned and never reached the local
fallback; the old subprocess.run(timeout=60) had bounded that wait. Run
the exchange on a daemon thread and wait at most _HANDOFF_DEADLINE_S:
before the request goes out the caller falls back; after it may have gone
out the caller raises the "may have started" error.

The local fallback now starts the relay with the same filtered environment
the host path sends, so the hand-off key no longer reaches the job body or
its log, and the fallback note is written only after the relay started.
The note no longer claims the job is in the agent's job object: _in_a_job
only answers "any job". The Git Bash probe prefers Git's install over a
WSL bash on PATH, and the relay's no-op log flush is gone.

Co-Authored-By: Claude Code <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

I reviewed the full base-to-head diff and the three-commit delta from d02c8e226719. The raw named-pipe handshake keeps bad or stalled clients off the accept loop, JSON framing removes pickle decoding and bounds requests, the client deadline preserves the no-duplicate-start rule across races, and both host and fallback paths keep the channel key out of the relay environment. Missing/invalid channels still retain the established local fallback.

Coverage this round included the repository rules and CONTEXT.md, the diff, affected callers, commit history, backward compatibility, architecture constraints, and whether tests were weakened; the new tests strengthen the failure and race cases. Focused suites pass on Python 3.12 and 3.13 with 52 passed and 9 Windows-only skips each; Ruff and git diff --check pass. The PR's dedicated Windows job-object/on-call hand-off CI step also passed at this head. My previously opened thread remains resolved.

The PR diff coverage gate counts the changed lines in windows.py, and the
Ubuntu shards skip the Windows-only abort/stall tests, so close(), the
accept loop's error/no-slot/thread-start branches, the inline handshake,
and the request's poll/validation errors were uncovered (72.32% against
the 90% threshold). Drive each branch with the fake listener/connection
the other hand-off tests already use; the Windows-only real-pipe tests
keep covering the live path.

Co-Authored-By: Claude Code <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The delta from e881459ba59a is test-only and additive. It covers the hand-off server's remaining accept, slot, authentication, close, timeout, and schema-rejection branches without weakening existing assertions or changing production behavior.

I checked the repository rules, the base-to-head diff and new commit, the production paths these tests exercise, compatibility and architecture implications, and the existing resolved review record. The focused Python 3.13 suite reports 61 passed and 9 Windows-only skips; Ruff and git diff --check pass. My previously opened thread remains resolved.

@LivXue

LivXue commented Nov 1, 2026

Copy link
Copy Markdown
Member

Windows compatibility re-review — PR #1115 @ 44a42b02be3304613e13c1b74814090dc72ae319

Scope: the delta since the previously reviewed head d02c8e22 is 4 commits / 5 files (+681/−90): the hand-off server now uses PipeListener with explicit deliver_challenge/answer_challenge, JSON framing instead of pickle, a per-connection handshake thread bounded to 64 slots, a 30 s request timeout and 1 MiB cap, an accept loop that survives a bad client, and _handoff() retrying a failed start or replacing a dead accept loop; the backend runs the exchange on a daemon thread with a 60 s deadline and phase tracking, sends no channel key on the fallback, and writes its fallback note only after the relay started.

Finding (low–medium, reproduced on Windows — carried over from the previous review of this PR): the hand-off subprocess calls still decode relay output with the ANSI code page, not the UTF-8 Raven sets for its tree.

  • Where: raven/acp_client/windows.py:264 (_HandoffServer._serve, subprocess.run(..., text=True, ...)) and agents/raven-oncall/plugins/oncall-flow/oncall_flow/windows_backend.py:291 (_start_relay).
  • Mechanism/evidence (native Windows 11, cp936, utf8_mode=0): raven/cli/entry.py:48 sets PYTHONIOENCODING=utf-8, so the relay child writes UTF-8; text=True decodes with locale.getpreferredencoding(False). A child writing the UTF-8 bytes of 错误:找不到指定路径 to stderr makes the reader thread raise UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 4 and subprocess.run returns stderr=None (exit code 0), so the hand-off answer carries "stderr": null and the JobBackendError loses the real reason.
  • Remedy: pass encoding="utf-8", errors="replace" to both new subprocess.run calls, matching the UTF-8 the Raven entry point sets for its tree.

Windows angle of the rest of the delta: the new auth exchange uses the standard multiprocessing.connection helpers that exist only on Windows and is instantiated only under the sys.platform == "win32" gate; the JSON framing removes the pickle surface from a pipe another same-session process could reach; the bounded slots and request timeout keep a stalled client from holding the accept loop. No new path, argv, or filesystem surface was added.

Verification (native Windows 11; detached worktrees D:\tmp\rv-1115c at head and D:\tmp\rv-base-1115c at base b05267b2; offline venv; scratch HOME/USERPROFILE; PYTHONUTF8=1; PYTHONPATH pinned):

  • uv run --no-sync pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q -n 0 --tb=line
    • head: 70 passed (51.0 s)
    • base: 35 passed (21.9 s)
    • zero failures at both; the 35 added tests (bad client, wrong key, oversized/invalid request, missing channel, host-exit paths) pass on Windows. The suite warns only about >3 s idle sleeps in the job-lifetime tests.

Coverage gaps: no live host + ACP agent + pipe + detached job run outside the test stubs; the persistent-accept-error path was exercised only as far as the tests drive it; the decoding finding above is reproduced by probe, not by an end-to-end relay.

LivXue and others added 2 commits November 1, 2026 19:12
Create every hand-off pipe instance with an explicit DACL for SYSTEM,
Administrators, and the owner, instead of the named-pipe default that
also grants Everyone and Anonymous read access. Any local user could
otherwise hold the 64 handshake slots without the key and force every
later hand-off onto the in-job fallback.

The accept loop now backs off and gives up after five consecutive
errors so a dead listener can be rebuilt, and close() wakes a blocked
accept instead of leaving it serving one more request. Drop the
unreachable recv_bytes None branch: the stdlib raises OSError when a
request exceeds the cap.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Run the relay start through asyncio.to_thread so a wedged host cannot
freeze the on-call agent's loop, and make the client deadline (100 s)
outlast the host's request (30 s) plus relay (60 s) budget. Resolve
the job directory to an absolute path before sending it, since the
host resolves a relative cwd against its own working directory.

Write the unhosted note only after the relay has actually started and
report that the job may end with its job object, which need not be
kill-on-close. Build the channel-free environment in one helper so a
future variable cannot reach the relay through only one of the two
paths. Make the test body publish result.json atomically and wait for
a parseable result, which removes the observed local flake. Scope the
CONTEXT.md hand-off claim to the hosted path and document the fallback.

Co-Authored-By: Claude Code <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The delta from 44a42b02be33 is sound. The explicit pipe DACL removes the default Everyone/Anonymous read access while retaining SYSTEM, Administrators, and owner access; close now wakes the blocked accept, repeated accept failures terminate for replacement, and the oversized-message path matches the stdlib's actual exception. The client work moves off the event loop, its deadline exceeds the host request-plus-relay budget, relative job paths are resolved before crossing process cwd boundaries, and both hosted and fallback relay environments consistently drop the channel capability.

I covered the repository rules and updated CONTEXT.md, the base-to-head diff and two-commit history, affected callers, backward-compatible fallback behavior, test-strength changes, and the established no-breakaway/host-side hand-off architecture. Focused tests report 68 passed and 11 Windows-only skips; Ruff and git diff --check pass. The dedicated Windows job-object/on-call hand-off CI step also passed at this head, including the native DACL and shutdown tests. My previously opened thread remains resolved.

@LivXue

LivXue commented Nov 1, 2026

Copy link
Copy Markdown
Member

Windows compatibility re-review — PR #1115 @ ce2a7b98c31bb4e9a00193a47a302ce5e2e66ddf

Scope: the delta since the previously reviewed head 44a42b02 is 2 commits / 5 files (+405/−44): the hand-off pipe is created with an explicit DACL (D:P(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;OW) — SYSTEM, Administrators, and the launching user) instead of the default named-pipe descriptor; the accept loop gives up after five consecutive failures with a small backoff so _handoff() can rebuild it; close() wakes a blocked accept; the relay timeout and the client deadline are named and ordered (100 s client budget outlasting the host's 30 s poll + 60 s relay); submit runs the relay start through asyncio.to_thread so the agent's event loop is no longer blocked for up to the deadline; the job directory is made absolute before the host sees it; the fallback note is written only after a relay that actually started; and one helper builds the channel-free environment for both paths.

Finding (low–medium, reproduced on Windows — carried over from the two earlier reviews of this PR): the two hand-off subprocess calls still decode relay output with the ANSI code page, not the UTF-8 Raven sets for its tree.

  • Where: raven/acp_client/windows.py:383 (_HandoffServer._serve, text=True) and agents/raven-oncall/plugins/oncall-flow/oncall_flow/windows_backend.py:304 (_start_relay).
  • Mechanism/evidence (native Windows 11, cp936, utf8_mode=0): raven/cli/entry.py:48 sets PYTHONIOENCODING=utf-8, so the relay child writes UTF-8; text=True decodes with locale.getpreferredencoding(False). A child writing the UTF-8 bytes of 错误:找不到指定路径 to stderr makes the reader thread raise UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 4 and subprocess.run returns stderr=None (exit code 0), so the hand-off answer carries "stderr": null and the JobBackendError loses the reason. Two attempts at this PR have fixed the surrounding channel; this decode line has not changed.
  • Remedy: pass encoding="utf-8", errors="replace" to both calls.

Windows angle of the rest: the SDDL DACL is correct in shape (P protected, owner-rights SID OW is the host user) and addresses the real default-DACL exposure; the PipeListener subclass's _new_handle(self, first=False) matches CPython's call shape on 3.12; the bounded accept loop removes the busy-spin risk; and moving the relay start off the event loop removes the up-to-100 s stall that the fallback could impose on submit.

Verification (native Windows 11; detached worktrees D:\tmp\rv-1115d at head and D:\tmp\rv-base-1115d at base b05267b2; offline venv; scratch HOME/USERPROFILE; PYTHONUTF8=1; PYTHONPATH pinned):

  • uv run --no-sync pytest tests/test_acp_windows.py tests/test_agents_oncall_flow_windows_backend.py -q -n 0 --tb=line
    • head: 79 passed (54.5 s)
    • base: 35 passed (24.3 s)
    • zero failures at both; the 44 added tests (DACL, accept-loop give-up and rebuild, slow-host deadline, relative-cwd resolution, fallback note ordering) pass on Windows. The suite warns only about >3 s idle sleeps in the job-lifetime tests.

Coverage gaps: the DACL was exercised through the tests and native bindings, not from a second local user account; no live host + ACP agent + pipe + detached job run was driven; the decoding finding is probe-reproduced, not end-to-end.

@LivXue LivXue left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at ce2a7b9. The two-commit delta and the full base-to-head diff are sound; CI is green (22 success, 2 skipped), the Windows ACP job ran the hand-off tests (79 passed), and the diff coverage gate passed at 92.14%.

@LivXue
LivXue merged commit c813725 into main Nov 1, 2026
30 of 31 checks passed
@LivXue
LivXue deleted the fix/jobs_outlive_host branch November 1, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants