Skip to content

fix(core): a muted video's picture no longer waits on its own audio track at a cut - #7412

Merged
miguel-heygen merged 3 commits into
mainfrom
rsafari/parity
Nov 8, 2026
Merged

miguel-heygen merged 3 commits into
mainfrom
rsafari/parity

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

What

A video that carries the muted attribute has its own audio track switched off in the runtime's sync tick, so its picture no longer waits on that track at a cut.

Why

In Safari a muted video is clocked by its audio track. When that track starts late after the cut, the picture stalls and then catches up by rate steering, so the first frames after some cuts arrive 100-250 ms late, and a cue's first frames can be skipped entirely. The sound itself comes from separate audio elements and is not affected.

Related work

Follows the pre-roll revert and the start-latency learning already on main. No issue.

How

dropSilentAudioTrack runs per clip in syncRuntimeMedia: for a <video> whose defaultMuted is true it sets every audioTracks[i].enabled to false. Engines without audioTracks (Chromium) skip it. Videos without muted are untouched; the runtime already keeps defaultMuted videos muted (init.ts mute paths), so nothing audible changes.

Size

A lone fix: nine lines of runtime and its tests; nothing else in flight touches this path, so there is no PR to fold it into.

Test plan

Measured on the iOS 26.5 Simulator by capturing the screen and the sound the app produces, 10-take fixture, 44 scored cues per run, three runs per arm interleaved (load 14-15 at start). Picture minus playhead, + = picture late:

run 1 run 2 run 3
main (0.9.23) median, within 40 ms +106, 32% +124, 5% +69, 27%
this PR median, within 40 ms -34, 84% +17, 70% 0, 70%
flashes seen of 44 (main / PR) 41 / 44 44 / 44 42 / 44

Sound against the playhead stays +18..+24 ms on both.

  • Unit tests added/updated: three cases in media.test.ts; fails without the fix: two of the three fail on main (the track stays enabled), the control passes on both.
  • Manual testing performed
  • Documentation updated (if applicable)
  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

Small on purpose: a lone urgent playback fix that nothing open can carry.

…rack at a cut

A muted video takes its clock from its audio track in Safari; a late start of that track stalls the picture after the cut. The runtime now switches the track off.
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 99 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

Unstable (1)

  • seqfastkeys-none-px-r0-root-z100: tracking 0.07, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes at 994e2743. CI is red because of this change.

Blocker: typecheck fails. Both the Typecheck and the Test: runtime contract jobs in the CI workflow fail on the new loop:

src/runtime/media.ts(481,9): error TS2532: Object is possibly 'undefined'.
src/runtime/media.ts(481,29): error TS2532: Object is possibly 'undefined'.

Under noUncheckedIndexedAccess, tracks![i] is T | undefined. Iterating the tracks avoids both the index and the non-null assertions:

for (const track of Array.from(tracks ?? [])) if (track.enabled) track.enabled = false;

What I checked and found sound:

  • Fails without the fix: I ran media.test.ts with and without the dropSilentAudioTrack(el) call. Without it, "is switched off for a video the author muted" and "is switched off when the browser lists it after the first tick" fail (expected true to be false), and the control "stays on for a video that can sound" passes. With it, all 191 tests in the file pass.

  • Video that starts muted and is unmuted later: the helper only ever disables tracks and never turns them back on. That is safe today because no path keeps the same element alive across an unmute:

    • The Studio mute toggle and "has audio" toggle remove the muted attribute through a batch commit, which reloads the preview by default (skipReload only applies to style-only ops).
    • A scene swap keeps a media element only when its authoredShape (all attributes) is unchanged, so removing muted rebuilds the element with fresh, enabled tracks.
    • The runtime, player and slideshow mute paths (el.muted = x || el.defaultMuted, init.ts 4317/6605/6635) never unmute a defaultMuted video, so a disabled track loses nothing audible.
    • Web Audio decodes from the source, not the element's track, and Studio scrub targets the music <audio>.

    It still holds only by those invariants. If a future path removes muted in place without a rebuild, that video would stay silent in Safari. Re-enabling the tracks when defaultMuted is false would close that, but it's optional and not a blocker.

Once the typecheck is fixed and CI is green on the new head, I expect to approve.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at e74ae6d2. The typecheck blocker from my earlier review is fixed.

  • Fix: the loop is now for (const track of Array.from(tracks ?? [])) track.enabled = false;. The core tsc --noEmit and the runtime tsconfig typecheck both pass locally, and the CI workflow (Typecheck and runtime contract) is green on this head.
  • Tests: all 192 tests in media.test.ts pass, including the new "stays on for a video that is muted by script and unmuted later" case. I already confirmed the "fails without the fix" claim on the previous head: two of the new cases fail without dropSilentAudioTrack(el), and the control passes.
  • Muted, then unmuted later: this is still safe for the reasons in my earlier review. An attribute unmute rebuilds the element or reloads the preview, and a script unmute leaves defaultMuted false, so the helper never touches that video.
  • CI: CI, regression, CodeQL, Player perf, Player Safari audio, Player Safari perf, preview-regression and PR captures are green. In Windows render verification, the engine-cli-2 lane hit its 35-minute timeout inside the @hyperframes/cli unit tests. Those run in Node and never reach syncRuntimeMedia, and Chromium has no audioTracks, so this change is a no-op there. The core Windows lane passed. I've rerun the failed lane, and that result is still pending.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 8, 2026
Merged via the queue into main with commit fa2e635 Nov 8, 2026
152 of 154 checks passed
@miguel-heygen
miguel-heygen deleted the rsafari/parity branch November 8, 2026 19:07
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 9, 2026
Port the adopted upstream fixes from f5bde95 to the current upstream tip:
heygen-com#7375 multipart cloud upload, heygen-com#7382 minParallelChunks, heygen-com#7349 HLG header SEI,
heygen-com#7369 skip unmounted compositions in lint/check, heygen-com#7381 filesystem-path media
in the cloud archive, heygen-com#7383 Safari cut-point pre-roll revert, heygen-com#7373 clock
steering between audio time updates, heygen-com#7384 injector test isolation, heygen-com#7378
preview seek retention, heygen-com#7390 virtual timer schedule per worker, heygen-com#7394 CDN
script failure naming, heygen-com#7395 menu row radius, heygen-com#7119 decode timeout caller
test, heygen-com#7397 authored tween order, heygen-com#7402 capture page clip loading, heygen-com#7412 muted
video audio track at cuts, heygen-com#6150 template root variable test, heygen-com#7442 held tail
seek once.

Deliberate local differences: heygen-com#7067's audio-start hold is not carried into
clock.ts, mountedSources comes from heygen-com#6827 as a helper only, cloud/_gen/types.ts
omits the image-request types the fork does not generate, and the lint
symlink test skips through symlinkOrSkip where the account cannot create links.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant