Skip to content

test(studio): a script-hidden backdrop stays hidden over the A-roll while the preview plays - #5475

Merged
miguel-heygen merged 1 commit into
mainfrom
test/studio-aroll-backdrop-fixture
Oct 11, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
test/studio-aroll-backdrop-fixture

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

What changes for a user

Nothing directly: this adds a check. The 0.8.148 fix (#5458) for a talking-head video that went black while Studio preview played is now guarded by a real project played in Studio's real preview, not only by unit tests. If the preview ever again forces a script-hidden backdrop visible, CI goes red.

Before (the #5458 visibility change reverted)

Before, playing: the dark backdrop covers the video, only the caption shows

After (main)

After, playing: the video shows under the caption

After, playing inside the backdrop's own window: the backdrop shows, so the check cannot pass by the backdrop never appearing

What the check does

packages/studio/tests/e2e/fixtures/aroll-backdrop is a 10 s, 9:16 film: a looping A-roll video (a copy of the repo's own colour-bar design-panel-qa/assets/test.mp4, 28 KB), an untimed full-frame #plate above it that the film's own GSAP script shows only from 6 s to 8 s, a caption on top, and two sub-compositions with GSAP timelines. In Studio preview the runtime stamps #plate with full-length timing so the timeline can list it, which is the shape that broke.

aroll-backdrop.mjs plays it in Studio and reads the preview's pixels and the element on top at the frame's centre: the video must show paused at the start, while playing at about 1 s, and paused after the backdrop's window; the backdrop must show while playing inside 6-8 s. Every wait is on a condition (Play enabled, the video has a frame, the playhead inside a range). Studio can re-fit the preview just after a pause, so a screenshot taken across that reflow is retaken (up to 5 times).

It runs in the existing studio-timeline-viewport job, in the same step as the trim-snap check (ci.yml), which also runs on packages/core/** changes.

Checked

Not covered here

  • Run on Linux headless Chrome for Testing only; not yet on the CI runner's Chrome (this PR's CI run is that check).
  • It does not fail if a future Studio stops stamping the backdrop at all; it records that in its output.

…hile the preview plays

A 9:16 fixture with a looping A-roll video, an untimed full-frame backdrop above
it that the film's own GSAP script shows only from 6 s to 8 s, captions on top
and two sub-compositions. The check plays it in Studio's preview and reads the
preview's pixels and the element on top at the frame's centre: the video must
show at rest and while playing outside the backdrop's window, and the backdrop
must show inside it. It fails with #5458's visibility skip reverted.
@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 2061 (base branch 2061), smooth 1640 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 46e930b3c10db8aae09024cd1adffb59361f8e20 against effective merge-base 943b6512b43b106c79839a977f3bfa4aa115a11d (the observed API base tip was e1925d4317fcd13c0e5799cd4767a3e2dd4c9238). Full scope: seven files, +369/−1; one modified workflow, five added text files and one added MP4. No supported introduced blocker found after the first-pass and independent adversarial source audits.

Audited

  • All changed text, the fixture's complete asset references, and the real Studio preview/player/compiler/runtime caller chain. The MP4 is byte-identical to the existing tracked design-panel-qa/assets/test.mp4 (28,273 bytes; SHA-256 4662cef1ee4423640d4db8b8880ea889d6e0af6e4466d88f5ee15f2dc6d18030). That establishes provenance, not independently decoded pixels.
  • The untimed root-child #plate reaches Studio runtime stamping. The scripted visibility window is [6,8); the runtime seeks the timeline before applying timed visibility during playback. An offline source-extracted control removing only the runtime-stamped visibility skip produces the intended outside-window failure; it does not revert unrelated #5458 behavior.
  • packages/studio/tests/e2e/aroll-backdrop.mjs:119–154 checks actual iframe screenshots and composition hit-testing: A-roll before the plate during playback, a positive dark/topmost plate witness inside its window, and A-roll in paused samples before and after. A plate that never appears cannot satisfy the positive witness.
  • .github/workflows/ci.yml:178–183,849–866,885–886,1005–1037 routes this change to the existing Studio job, builds the relevant packages, copies the fixture, runs this check against the server built from the job's checked-out PR tree, and propagates its failure through the accumulated exit status. The normal PR checkout can be the integration merge tree; this is source wiring, not a claim of a bare-head CI run.

Non-blocking coverage limits
The walk pauses before the post-window screenshot and does not exercise an explicit seek. It records rather than asserts the runtime-stamped marker, and the fifth screenshot is accepted even if the rounded frame bounds still differ. A still-playing post-window sample, seek sample, and stability-exhaustion failure would strengthen coverage; none establishes an introduced failure in this test-only delta. CI currently keeps this check's JSON in logs without configuring its optional PNG output.

Verification / trusting
The offline, source-extracted guard/oracle controls passed with simulated DOM, clocks, media and pixel values. I did not run Puppeteer, decode media, build Studio, query/rerun CI, or verify the author's repeated native-pass/fail claims. No submitted GitHub reviews or inline findings were present in the refreshed history; the edit-accuracy bot comment is not a review approval. Approval is on the audited code merits, not those execution claims.

— tai

Verdict: APPROVE
Reasoning: The test reaches the production preview path and has both negative and positive visual oracles for the targeted backdrop regression, with source-confirmed CI failure propagation. The remaining gaps are coverage hardening, not demonstrated blockers.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 11, 2026
@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 11, 2026
Merged via the queue into main with commit 83b6419 Oct 11, 2026
152 checks passed
@miguel-heygen
miguel-heygen deleted the test/studio-aroll-backdrop-fixture branch October 11, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants