Skip to content

fix(engine): a PNG-sequence frame no longer captures DOM still pending in virtual rAF - #6599

Open
santhiprakash wants to merge 5 commits into
heygen-com:mainfrom
santhiprakash:fix/5888-capture-path-raf-drain
Open

santhiprakash wants to merge 5 commits into
heygen-com:mainfrom
santhiprakash:fix/5888-capture-path-raf-drain

Conversation

@santhiprakash

Copy link
Copy Markdown
Contributor

What

Keeps seekToTime's single virtual-rAF flush and moves the bounded drain into the capture path. After __hf.seek returns, prepareFrameForCapture runs a new per-seek barrier, settlePostSeekDomUpdates: the shim's drainAnimationFrames runs still-queued author rAF work at the frozen virtual time, then the barrier decodes newly active images, waits document.fonts, and commits two native paints before the screenshot when the compositor free-runs.

Why

Fixes #5888. On a PNG-sequence render, each worker's first frames could capture the DOM while an update deferred through author requestAnimationFrame turns was still pending — reproduced upstream at 18/48 frames with no text on 0.8.193. This is the resubmission invited in #6007, in the requested shape: the drain lives where the post-seek settle runs, and it stops instead of multiplying perpetual loops.

Related work

Fixes #5888. Resubmits the closed #6007 in the shape its review requested — drain in the capture path rather than inside seekToTime, a registration-lineage guard for loops, and a timing measurement on a render longer than 48 frames. The settle barrier and the shim change are split into separate commits.

How

  • packages/producer/src/services/fileServer.ts (virtual-time shim): seekToTime still calls only flushAnimationFrame(). Registrations now carry seek lineage: hf.seek arms a boundary via beginSeek() before renderSeek, which is where timeline onUpdate hooks queue the deferred work a seek must settle; seekToTime rolls the boundary itself for direct callers such as same-origin child frames. A self- or ancestor re-registration inherits that ancestor's lineage, so a registration cycle stays distinguishable from spawned deferred work.
  • __HF_VIRTUAL_TIME__.drainAnimationFrames runs queued entries to quiescence except: lineage registered before the first seek (a page-load self-rescheduling loop keeps exactly one turn per seek), callbacks that already ran in the previous seek (established loops), and callbacks past a 16-execution lifetime drain allowance (a perpetual driver spawned mid-seek stays bounded). Turn (64) and execution (2048) caps bound queues that never converge. function tick(){ rotation++; requestAnimationFrame(tick) } ends at 24 runs over 24 frames.
  • packages/engine/src/services/frameCapture.ts: settlePostSeekDomUpdates runs in prepareFrameForCapture after __hf.seek and LUT waits, before compositing. Date/performance.now are virtualized in-page, so the shared 1500 ms wall-clock budget is armed through the shim's stored originalSetTimeout (HF_POST_SEEK_SETTLE_MS; 0 disables the settle). The two-paint commit uses the stored originalRequestAnimationFrame and only runs for free-running compositors — BeginFrame sessions skip it since the capture's own BeginFrame is the commit — and only when the settle observed pending work.

Test plan

  • Unit tests added/updated — packages/producer/src/services/fileServer-rafDrain.test.ts (13 bun:test cases: deferred chains settle at the current seek, first-seek registrations drain, page-load perpetual loops keep one turn per seek, spawned loops stay bounded, non-converging queues cap, stats and cancellation) and packages/engine/src/services/frameCapture.postSeekSettle.test.ts (6 vitest cases: drains work queued after the seek, decodes images once across seeks, two-paint commit only on observed work and a free-running compositor, no-ops without the shim).
  • Manual testing performed — the issue's exact composition rendered on this branch (Linux aarch64, Chromium 156, PRODUCER_FORCE_SCREENSHOT=true, --format png-sequence --fps 24 --workers 2 --experimental-fast-capture=false):
    • 48-frame fixture: 18/48 fully transparent frames to 1/48. Frames 1-9 and 25-33 (each worker's first nine) are recovered. The remaining frame is inherent to the requested loop guard: at the t=0 seek the timeline is already rendered at 0, so onUpdate does not fire and the only pending rAF work is the chain queued at page load — a self-re-registering lineage indistinguishable from a perpetual loop. Running it to completion is the 64-turns-per-frame behavior the earlier version was closed for, so the drain correctly leaves it on its one-turn-per-seek pace.
    • 96-frame fixture, same shape (longer than the issue's 48): 18/96 blank to 1/96; capture phase 3.4 s to 5.1 s (+18 ms/frame). The same composition without the deferred reveal: 3.4 s to 4.7 s (+14 ms/frame) — the two-paint commit dominates whenever a seek's flush ran rAF work, which GSAP's ticker always does.
  • Documentation updated (if applicable) — no user-facing surface changed.
  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

Sabotage check: all 13 shim tests fail against upstream's fileServer.ts (the drain API does not exist there); the 6 settle tests exercise a function absent before this change.

scripts/package-size-budgets.json bumps only @hyperframes/cli unpacked for upstream drift (24,067,526 bytes measured vs the 23,722,874 budget — over by more than 1% before this change). The packages this PR touches still fit: @hyperframes/engine 451,480 packed vs 456,397, @hyperframes/producer 820,762 vs 828,932.

AI-assisted. I wrote and verified this change.

@santhiprakash

Copy link
Copy Markdown
Contributor Author

Verification numbers on this branch (Linux aarch64, Chromium 156, PRODUCER_FORCE_SCREENSHOT=true, --format png-sequence --fps 24 --workers 2 --experimental-fast-capture=false, the issue's exact composition):

  • Frame counts, 48-frame fixture: 18/48 fully transparent frames with the settle disabled (frames 1-9 and 25-33, matching the 0.8.193 reproduction) → 1/48 with it enabled.
  • Frame counts, 96-frame fixture: 18/96 → 1/96.
  • Timing, 96 frames: capture phase 3.4 s → 5.1 s (+18 ms/frame). Same composition without the deferred reveal: 3.4 s → 4.7 s (+14 ms/frame) — the two-paint commit dominates whenever the seek's flush ran rAF work (GSAP's ticker always does). Whole render 6.7 s → 8.6 s on this box.
  • Perpetual-loop contract (unit): function tick(){ rotation++; requestAnimationFrame(tick) } ends at 24 runs over 24 frames; a page-load a↔b ping-pong at 25 (one discovered edge); a loop first registered mid-seek bounded at 40 by the drain allowance.

One caveat worth stating plainly: the single remaining blank frame is the t=0 frame. renderSeek re-seeks to a time the timeline already rendered, so onUpdate never fires and the only pending rAF work is the chain registered at page load — self-re-registering lineage indistinguishable from a perpetual loop under the requested guard. Settling it early is exactly the 64-turns-per-frame behavior the previous version was closed for, so the drain leaves it at one turn per seek. If a different trade-off is preferred there, it is a one-line isDeferred change — but every option that runs that chain early also runs page-load loops extra turns.

@santhiprakash
santhiprakash force-pushed the fix/5888-capture-path-raf-drain branch from 8a28f42 to 37ee072 Compare October 27, 2026 23:48

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 37ee0721eb4f0c24275de121e1e0be8c1b8917ba across all five changed files, including the actual generated shim, bridge seek order, capture barrier, same-origin child caller, and package-budget delta. Two changes are needed:

  • [P2] Fresh finite work is blocked when its callback was used in the previous seek. packages/producer/src/services/fileServer.ts (isDeferred, lines 361–365) tests callback identity rather than whether this registration belongs to the old loop. With one reusable reveal function, each renderSeek resets a five-turn reveal and requests it anew. The first seek settles all five turns; the second flush runs one, then the drain leaves four pending. A fresh function per seek passes. I reproduced this using the exact shim + bridge + settle bodies in Chrome 156: the first screenshot is green, the second stays red. Track loop status/allowances per registration lineage so independent invocations can settle, and add a regression with the same finite callback on consecutive seeks. Keep the perpetual-loop controls.
  • [P2] Retain bounded lineage, rather than every historical entry. packages/producer/src/services/fileServer.ts:461: every rescheduling entry points to the entire previous activeEntry through parent, but lookup stops after 64 hops. A page-load self-loop leaves one queued callback retaining 1,001 entry ancestors after 1,000 seeks. Separately, after 1,000 completed two-turn finite invocations the queue is empty but drainRunCounts still strongly retains all 1,000 callbacks. Long renders therefore accumulate history that the guard no longer needs. Bound/compact parent history and use collectible or reclaimed accounting; add a many-seek retention control. This is demonstrated growth, not a measured out-of-memory failure.

Verification: the prior 13 passing shim cases are reused with identical generated execution AST (comments removed); the queued rerun was cancelled before execution due runner backlog. Five independent capture-barrier checks pass (free-running/BeginFrame/idle, real deadline, absent shim). The reused-callback failure has raw Chrome screenshots and pixel checks, with a passing fresh-callback VM control. No API-absent mutant is counted as behavioral discrimination. I did not rerender the original issue composition, run full PNG/encode parity, or independently measure the reported +14/+18 ms per-frame cost. The documented remaining t=0 blank frame and performance tradeoff remain limits, not evidence of complete parity. Same-origin children are sought directly, but the added barrier evaluates the top document only.

Current-head checks: required checks pass; the skills-manifest check is skipped. No other peer review was present at the publication guard.

Need: the capture-side placement addresses a real readiness gap. Size: localized changes; the unrelated CLI budget increase should stay justified by measured upstream drift. Reuse/Simpler/Structure: preserve the single seek flush and real timer, while separating fresh work from loop identity and bounding retained state.

Verdict: request changes.
Reasoning: repeated finite work still reaches capture unfinished, and scheduling bookkeeping grows with render length.

— Magi

Deferral and the drain-run allowance were keyed on callback identity, so a
finite chain re-requested with the same function on the next seek was left
pending, and per-callback maps plus unbounded parent chains grew retained
state with render length. Entries now carry their registration lineage
({epoch, drainRuns}, shared by every link of one invocation) plus a bounded,
callback-deduped ancestor record list for cycle detection: re-requested
finite work opens a fresh lineage and settles, established loops keep their
one-turn-per-seek pace, and a completed invocation's bookkeeping is
collectible.
@santhiprakash

Copy link
Copy Markdown
Contributor Author

Both P2s are addressed on 73a45b1b.

[P2] Fresh finite work blocked when its callback was used in the previous seek. Root cause: isDeferred keyed on the callback function (prevSeekRanCallbacks), so a reveal re-requested at the next seek inherited the previous invocation's loop status. Deferral and the rerun allowance now live on a per-registration lineage object: every entry carries {epoch, drainRuns} shared by all links of one invocation, and a re-requested registration opens a fresh lineage (registrationContext / isDeferred in packages/producer/src/services/fileServer.ts). The previous-seek callback set is gone entirely.

Sabotage on 37ee0721 (matching your Chrome result): the new case "settles the same finite callback re-requested on consecutive seeks" gives perSeek [5,1,2], visible [true,false,false] — first seek green, the rest red. On 73a45b1b it gives [5,5,5] / [true,true,true]. A sibling case covers a page-load driver re-spawning the same callback each seek (previously [5,0,1] via the repeated-edge epoch inheritance, which is also removed). All 13 prior cases pass unchanged, including the 24-turn perpetual-loop contract and the mid-seek rerun-allowance bound.

[P2] Bounded lineage, collectible accounting. entry.parent chains are gone: cycle detection reads a bounded, callback-deduped ancestor window capped at RAF_LINEAGE_MAX_ANCESTORS (64) records per queued entry — a page-load self-loop keeps a single record after any number of seeks instead of 1,001 entry ancestors. drainRunCounts is replaced by lineage.drainRuns, which dies with its lineage. New case "releases a completed invocation's bookkeeping across many seeks": 200 completed two-turn invocations, a WeakRef per callback, forced GC — 37ee0721 retains all 200; 73a45b1b retains ≤8 (conservative-scanner noise on random indices).

Verification: bun test packages/producer/src/services/fileServer-rafDrain.test.ts — 16/16 pass; bunx vitest run packages/engine/src/services/frameCapture.postSeekSettle.test.ts — 6/6 pass; oxlint, oxfmt --check, and the comment ratchet are clean.

@santhiprakash
santhiprakash force-pushed the fix/5888-capture-path-raf-drain branch from 4b5ba84 to 269f9eb Compare October 28, 2026 20:33

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 269f9eb2b7e310f1a9a9e633fb0855c7263ccd3d.

The earlier callback-history and retention findings are resolved by per-registration lineage and collectible bookkeeping. I traced the virtual clock, capture caller and post-seek settling path in two independent passes.

The exact extracted shim passes 16 authored Bun VM tests, including repeated finite callbacks and retention controls. The exact extracted settling function passes 6 authored tests. The extraction keeps the function bodies unchanged and changes only test imports. This is not a packaged runtime build, Chrome capture or PNG-render acceptance.

Nit: frameCapture.ts:3339 reports imagesDecoded = decodes.length even if the deadline wins before completion. Count completed decodes inside their handlers so the diagnostic reflects completed work; keep the bounded wait.

Need: yes · Size: ok · Reuse: existing frame clock and settling budget · Simpler: bounded registration state · Structure: fine
Risk: medium - virtual frame callback lifecycle and post-seek settling.

Verdict: APPROVE
Reasoning: The scoped controls and independent source pass resolve the prior findings; the remaining counter accuracy issue is non-blocking.

Reasoning: The scoped shim and settling controls pass, and earlier callback-retention findings are resolved.

@santhiprakash
santhiprakash force-pushed the fix/5888-capture-path-raf-drain branch from 269f9eb to 157c10e Compare November 5, 2026 14:49
@santhiprakash
santhiprakash force-pushed the fix/5888-capture-path-raf-drain branch from 157c10e to f8c8f7b Compare November 8, 2026 10:31
- Problem: the virtual-rAF drain and seek-settle fix adds engine and
  producer code that is bundled into producer's four dist bundles
  (index.js, public-server.js, distributed.js, runtime.iife.js) and the
  rolled-up service declarations, pushing the unpacked tarball to
  3178884 bytes — over the 3130000 budget by 1.56%, past the 1% gate.
- Fix: raise @hyperframes/producer unpacked budget 3130000 -> 3180000,
  the repo-sanctioned path (verify-packed-manifests error text: raise
  the budget only with the reason in the PR). Reason: size growth is the
  fix's own code multiplied across the dist bundles; no source maps,
  tests, or unused assets entered the tarball.
- Verification: fresh bun run build at f8c8f7b reproduces CI's number
  exactly (3178884, ci.yml Build job 128704040260); npm pack after the
  bump: 3.2 MB unpacked <= 3211800 allowance (headroom 32916).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

render: PNG sequence can capture pending virtual-rAF DOM updates after seek (0.8.145)

3 participants