Skip to content

test(engine): blur gates fail on CI when they cannot run, and prove blur drew - #7302

Merged
vanceingalls merged 5 commits into
mainfrom
test/blur-gates-fail-closed
Nov 6, 2026
Merged

vanceingalls merged 5 commits into
mainfrom
test/blur-gates-fail-closed

Conversation

@vanceingalls

Copy link
Copy Markdown
Collaborator

What

The blur test gates could go quiet in four ways. This closes them.

  1. A missing browser used to skip, so CI could stay green with nothing running. requireBlurBrowser (in launch(), __fixtures__/motionBlur/blurPage.ts) now throws on CI (process.env.CI) when Chrome will not start, the built runtime is missing, or drawElementImage is absent, for all ten engine blur browser files at once (including the three differential files, through startChrome). Off CI the old skip behaviour stays, so local runs without Chrome still work. The producer files also fail closed when ffmpeg is missing on CI.
  2. The preview liveness test stopped after 3 distinct frames, so a freeze later in playback passed. held-card now keeps sampling through its slide-out and needs a frame that is not in the hold set, in a shot wholly inside 1.6-2.0 s (__fixtures__/motionBlur/slideOutWatch.ts, with a pure table test). A frozen preview fails with "froze mid-film"; a wall-clock cutoff mutation in step() makes it fail, and index.ts is byte-identical afterwards.
  3. The "blur really drew" guard covered only held-card. Every inside-the-trail case now asserts blur-on differs from the sharp frame at the same time, reusing frames the cases already capture (no new page loads). Engine side the floor is INVARIANTS.outsideMaxMae (1.0), not insideMaxMae: at the fixtures' moving times blur-on vs sharp measures only 9.3-69 worst-tile MAE (mostly about 13.7); the 39-88 range I had quoted was the producer's whole-render maximum. Blur-on mutated to blur-off fails all 24 inside cases; a half-strength blur would still pass. The producer guard uses insideMaxMae for all seven moving fixtures.
  4. stateful-callback was not in the producer's multi-worker render. It is now. The producer differential ran about 83 s on CI against the Flake gate's 120 s per-run cap, so it is split into five files (helpers extracted to render/__test_utils__/blurRender.ts): blurDifferential (callbacks), .once, .stateful, .movers, .offsets. They join runsAlone and run with --no-file-parallelism (otherwise several 4-7-Chrome renders would run at once on a 4-core runner); integration-tests.json and the pinned runner hash in scripts/test-reachability.json are updated. Seven fixtures are covered: the old six plus stateful-callback; no block-start guard for the new fixtures.

Tests and timing

Worst local time per file with 14 CPU burners (3 runs each; CI measured about 1.65x slower than that against a 120 s per-run cap): producer blurDifferential 21.7 s, .once 18.0, .stateful 19.5, .movers 30.6, .offsets 27.7; engine liveness 16.3, differential.inside 21.3, differential.filter 19.9. The five blur producer files run serially in 96.5 s (30 tests). Eight consecutive liveness runs under burners passed. The Flake gate will rerun 7 files; I estimate 35-40 minutes against its 60-minute cap, not yet measured on CI.

Notes

  • The brief's own check for item 1 (CI=1 PRODUCER_HEADLESS_SHELL_PATH=/nonexistent) already failed before this change because resolveHeadlessShellPath throws at import. The new branches were verified separately: Chrome missing (empty HOME), runtime build missing, and drawElementImage forced absent each fail on CI and skip off CI.
  • A CI lane without a working Chrome would now fail instead of skip. Test (runtime) and the Windows lanes already have one.
  • Residual liveness false-fail: a single screenshot wider than 0.2 s covering the end of the window (measured spacing on a healthy preview is about 0.05 s of clock).

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 136 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqab-none-center-r0-root-z100: tracking 0.03, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.03, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.03, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 1 nit

Both producer lanes are red on test:classification: none of the five split files imports a runner directly. Nothing else blocks. slideOutWatch's stop rule holds up: the clock caps at the film end, so a frozen preview reaches advanced >= endS with 0 new frames and fails, and the end frame is excluded by the strict < endS check. Fail-closed is safe on Windows: on this head both engine-cli lanes ran every blur browser file with nothing skipped (same as main), and Test (runtime) ran held-card through the slide-out (1.8 s).

Scope

Read all 14 files. I checked all ten engine blur files: each goes through launch() → requireBlurBrowser. The extracted helper differs from the old file only in the intended hunks. Windows evidence: run 42180886994 jobs 127770426622/127770426638, and on main run 42170043922. Flake gate was still running when I posted.

— Genesis

vanceingalls and others added 5 commits November 6, 2026 00:15
… run

On CI a missing Chrome, a missing core runtime build or a Chrome without drawElementImage
fails the suite's beforeAll instead of skipping every case. Off CI the suites still skip.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…held-card

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… split across five suites

The render differential's suite moves into __test_utils__/blurRender.ts and five files run it
over one or two fixtures each, so none nears the flake gate's 120 s cap. Every fixture that
moves must now blur more than insideMaxMae, not only held-card, and CI fails instead of
skipping when ffmpeg is missing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…classification reads it

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@vanceingalls
vanceingalls force-pushed the test/blur-gates-fail-closed branch from d5f7a61 to 7d559a0 Compare November 6, 2026 08:15

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE — 2 resolved, 0 open, 0 new · d5f7a61 → 7d559a0 (rebase is otherwise identical). All CI is green, and the integration lane ran all 30 blur cases with none skipped.

— Genesis

@vanceingalls
vanceingalls added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit 0782409 Nov 6, 2026
91 checks passed
@vanceingalls
vanceingalls deleted the test/blur-gates-fail-closed branch November 6, 2026 10:08
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…view

Adopts heygen-com#7299 (accented Latin letters draw in a bundled family at its
bundled weight), heygen-com#7301 (installing Chrome on a busy Windows PC), heygen-com#7262
(cloud render logs a line a minute while a job holds its status), heygen-com#7265
(video starts a second early and catches up hidden) and heygen-com#7305
(check_report names the finding that stopped the check). Player packed
budget takes upstream's +1655 from heygen-com#7265 since the player bundles core's
media-steer. Defers heygen-com#7302; heygen-com#7261 and heygen-com#7263 do not apply. Watermark
advances to 9e2510b.

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: WaterrrForever <miao.yang@heygen.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant