Skip to content

fix(cli): a killed CLI no longer leaves its child processes running - #6431

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/cli-orphan-children
Oct 26, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
fix/cli-orphan-children

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

What

A CLI that dies outright (SIGKILL, out of memory, a crash) no longer leaves its child processes running. Every process it started through runCancellableProcess, and the Chrome a render launches, now ends with it.

Why

Repro on Linux, on main:

  1. hyperframes render a 20 s project, and kill -9 the CLI after 10 s. All five chrome-headless-shell processes keep running, re-parented to init, in their own process group.
  2. hyperframes preview on a large project, and kill -9 the server while it lints. The lint worker keeps running until it finishes: two of them ran for 12 minutes at about 80% CPU on a 9.6 MB composition.

A clean exit was already fine: the CLI's signal handlers stop the setup worker's process tree, and closeManagedBrowser stops Chrome. Nothing covered a process that cannot run a handler.

Two different gaps:

  • runCancellableProcess starts children in their own process group on macOS and Linux, so terminateProcessTree can stop a whole tree on cancel. That same choice takes them out of anything that would stop them when the CLI dies. A child cannot watch its own way out either: the lint worker parses synchronously, so no listener in it runs until the lint ends.
  • The engine launched Chrome over a websocket. The CLI's own launcher (cli/src/browser/launch.ts) already uses pipe: true, where Chrome ends when the other end of its pipe closes, and that is why check already passes the existing "killed mid busy-loop" test.

How

  • utils/childReaper.ts: the first runCancellableProcess call starts one small reaper process, detached, whose stdin is a pipe from the CLI. Each spawn sends + <pid> g|p, each close sends - <pid>. When that pipe closes, because the CLI exited or died, the reaper kills whatever is still held: the whole process group for a detached child, the pid for an attached one (taskkill /T /F on Windows). It is the stdin-close pattern of browser/windowsChromeWatchdog.ts, held once per CLI process instead of once per Chrome.
  • A setup worker that itself spawns children gets its own reaper, so a killed worker takes its children too.
  • The engine's shared launch options (sigintLaunchOptions, renamed ownedLaunchOptions since it now carries both) add pipe: true, for every engine Chrome launch: render, the beginframe fallback, the GPU probe and HDR capture. Nothing connects to the engine's Chrome by websocket (no wsEndpoint or connect use in engine, producer or cli).

Options considered for the non-node children and the busy worker:

Option Covers a busy worker Covers non-node children Cost
Each child watches a pipe from its parent no (a sync parse never sees it) no (ffprobe, unzip, ldd cannot) none
Stop detaching yes, for a whole-group kill by the shell only yes loses tree cancel on cancel and timeout, and a hard kill of one pid still reaches no child
One reaper per CLI process (this PR) yes yes one idle node process while children run

What changes in output

Nothing for a run that ends normally. After a hard kill, no chrome-headless-shell, lint worker or probe is left running. The orphan sweep at the next CLI start (orphanCleanup.ts) still runs; it now finds nothing to clean in these cases.

Test plan

  • Unit tests added/updated

  • Manual testing performed

  • Documentation updated (if applicable): not needed, no flag or output changes

  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

  • childReaper.test.ts: a fixture CLI starts a child through runCancellableProcess that blocks its event loop like a synchronous lint and starts a grandchild. The test SIGKILLs the fixture and waits for both to be gone. Without the reaper it fails ("Condition not met within 5000 ms"). Passed 3 runs in a row.

  • chromeOrphans.process.test.ts: "leaves no Chrome process behind when the CLI is killed mid-render" renders a 30 s film, SIGKILLs the CLI while Chrome is up, and waits for every Chrome process to be gone.

  • Re-checked on a Linux box: the new tests and the engine browserManager suite pass 3 runs in a row. Removing the hold in runCancellableProcess fails childReaper.test.ts, and removing pipe: true fails the mid-render test, each alone.

Does pipe: true cost render time?

Same 6 s, 30 fps, 1920x1080 GSAP composition (180 frames, beginframe capture), rendered with the built CLI on a Linux box, main and this branch interleaved, three runs each. The box was shared and busy (load 6 to 10 on 8 cores), so read the spread, not single runs.

main this branch
Render wall, 3 runs (s) 10.17, 10.81, 9.17 (median 10.17) 9.64, 9.00, 8.97 (median 9.00)
Pipeline total, 3 runs (ms) 8183, 9752, 9046 (median 9046) 7977, 8044, 8628 (median 8044)
Capture p50 per frame (ms) 76, 94, 77 (median 77) 82, 80, 83 (median 82)
Capture mean per frame (ms) 37, 46, 38 (median 38) 36, 36, 40 (median 36)

Wall time and pipeline total are not higher. Capture p50 reads 5 ms higher at the median, inside the spread main shows on its own runs (76 to 94 ms), and the mean per frame is not higher. I did not measure a difference outside that noise; a quiet-box rerun would tighten it.

@github-actions

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 165 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (3)

  • seqplainkeys-none-center-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-px-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits · head 302e2aa9

Scope
  • Re-pinned from 9eb4a39c to 302e2aa9. The only added change is a two-line fixture comment.
  • Reaper: node pipes are CLOEXEC, so no child inherits the write end. The hold is released on close, so a normal run kills nothing. The -e script is CJS, and it works under both node and bun: tested directly with a detached sleep and a SIGKILLed parent, and the sleeper died in both runtimes. orphanCleanup matches only the Chrome marker, so it can't kill the reaper.
  • pipe: true: nothing in engine, producer, cli, lambda or cloud-run reads wsEndpoint or calls connect, and the old sigintLaunchOptions name has no other users.
  • Run locally: childReaper.test.ts passes 3/3, and it fails with cancellableProcess.ts reverted to main. chromeOrphans.process.test.ts passes 3/3, and the mid-render case fails with pipe: true removed.
  • CI on this head is all green, Windows render included.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 26, 2026
@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 26, 2026
@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 5148326 Oct 26, 2026
124 checks passed
@miguel-heygen
miguel-heygen deleted the fix/cli-orphan-children branch October 26, 2026 18:30
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Oct 26, 2026
Brings in 10 upstream commits (ad4d3fa..15367a9): heygen-com#6425, heygen-com#6427, heygen-com#6429,
heygen-com#6431, heygen-com#6436, heygen-com#6432, heygen-com#6453, heygen-com#6457, heygen-com#6450, heygen-com#6451.

One conflict, packages/core/src/compiler/compositionScoping.ts (heygen-com#6436). This
fork passes authored composition code to Function() as a JSON string literal,
so upstream's markers around the raw source would sit outside the text its
page-side scan reads and the fix would silently not apply: the wrapper's own
__timelines use would make every composition look able to register a timeline.
The markers go inside the literal instead, begin marker on the first source
line so error line numbers do not shift. The pageBuild goldens change only by
those two marker comments. A new engine test runs upstream's scan on this
fork's real wrapper output and fails without the markers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Oct 26, 2026
…d PR watermark to heygen-com#6459

Commit watermark 8f618c5; PR watermark heygen-com#6459 only, because heygen-com#6460 onward and
issue heygen-com#6465 were not reviewed this round (listed in DECISIONS). Issue watermark
stays heygen-com#6443. Records the heygen-com#6436 Function() wrapper port, the chromeOrphans
failure that is not caused by heygen-com#6431, and the environment-only test failures.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant