Repository navigation
fix(core): a muted video's picture no longer waits on its own audio track at a cut - #7412
Conversation
…rack at a cut A muted video takes its clock from its audio track in Safari; a late start of that track stalls the picture after the cut. The runtime now switches the track off.
Edit accuracy: accurate 267 (base branch 267), smooth 99 of thoseThe gate passes. Quarantined, measured but not gated (8)
Unstable (1)
|
There was a problem hiding this comment.
Requesting changes at 994e2743. CI is red because of this change.
Blocker: typecheck fails. Both the Typecheck and the Test: runtime contract jobs in the CI workflow fail on the new loop:
src/runtime/media.ts(481,9): error TS2532: Object is possibly 'undefined'.
src/runtime/media.ts(481,29): error TS2532: Object is possibly 'undefined'.
Under noUncheckedIndexedAccess, tracks![i] is T | undefined. Iterating the tracks avoids both the index and the non-null assertions:
for (const track of Array.from(tracks ?? [])) if (track.enabled) track.enabled = false;What I checked and found sound:
-
Fails without the fix: I ran
media.test.tswith and without thedropSilentAudioTrack(el)call. Without it, "is switched off for a video the author muted" and "is switched off when the browser lists it after the first tick" fail (expected true to be false), and the control "stays on for a video that can sound" passes. With it, all 191 tests in the file pass. -
Video that starts muted and is unmuted later: the helper only ever disables tracks and never turns them back on. That is safe today because no path keeps the same element alive across an unmute:
- The Studio mute toggle and "has audio" toggle remove the
mutedattribute through a batch commit, which reloads the preview by default (skipReloadonly applies to style-only ops). - A scene swap keeps a media element only when its
authoredShape(all attributes) is unchanged, so removingmutedrebuilds the element with fresh, enabled tracks. - The runtime, player and slideshow mute paths (
el.muted = x || el.defaultMuted,init.ts4317/6605/6635) never unmute adefaultMutedvideo, so a disabled track loses nothing audible. - Web Audio decodes from the source, not the element's track, and Studio scrub targets the music
<audio>.
It still holds only by those invariants. If a future path removes
mutedin place without a rebuild, that video would stay silent in Safari. Re-enabling the tracks whendefaultMutedis false would close that, but it's optional and not a blocker. - The Studio mute toggle and "has audio" toggle remove the
Once the typecheck is fixed and CI is green on the new head, I expect to approve.
There was a problem hiding this comment.
Approving at e74ae6d2. The typecheck blocker from my earlier review is fixed.
- Fix: the loop is now
for (const track of Array.from(tracks ?? [])) track.enabled = false;. The coretsc --noEmitand the runtime tsconfig typecheck both pass locally, and the CI workflow (Typecheck and runtime contract) is green on this head. - Tests: all 192 tests in
media.test.tspass, including the new "stays on for a video that is muted by script and unmuted later" case. I already confirmed the "fails without the fix" claim on the previous head: two of the new cases fail withoutdropSilentAudioTrack(el), and the control passes. - Muted, then unmuted later: this is still safe for the reasons in my earlier review. An attribute unmute rebuilds the element or reloads the preview, and a script unmute leaves
defaultMutedfalse, so the helper never touches that video. - CI: CI, regression, CodeQL, Player perf, Player Safari audio, Player Safari perf, preview-regression and PR captures are green. In Windows render verification, the
engine-cli-2lane hit its 35-minute timeout inside the@hyperframes/cliunit tests. Those run in Node and never reachsyncRuntimeMedia, and Chromium has noaudioTracks, so this change is a no-op there. The core Windows lane passed. I've rerun the failed lane, and that result is still pending.
Port the adopted upstream fixes from f5bde95 to the current upstream tip: heygen-com#7375 multipart cloud upload, heygen-com#7382 minParallelChunks, heygen-com#7349 HLG header SEI, heygen-com#7369 skip unmounted compositions in lint/check, heygen-com#7381 filesystem-path media in the cloud archive, heygen-com#7383 Safari cut-point pre-roll revert, heygen-com#7373 clock steering between audio time updates, heygen-com#7384 injector test isolation, heygen-com#7378 preview seek retention, heygen-com#7390 virtual timer schedule per worker, heygen-com#7394 CDN script failure naming, heygen-com#7395 menu row radius, heygen-com#7119 decode timeout caller test, heygen-com#7397 authored tween order, heygen-com#7402 capture page clip loading, heygen-com#7412 muted video audio track at cuts, heygen-com#6150 template root variable test, heygen-com#7442 held tail seek once. Deliberate local differences: heygen-com#7067's audio-start hold is not carried into clock.ts, mountedSources comes from heygen-com#6827 as a helper only, cloud/_gen/types.ts omits the image-request types the fork does not generate, and the lint symlink test skips through symlinkOrSkip where the account cannot create links. Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
What
A video that carries the
mutedattribute has its own audio track switched off in the runtime's sync tick, so its picture no longer waits on that track at a cut.Why
In Safari a muted video is clocked by its audio track. When that track starts late after the cut, the picture stalls and then catches up by rate steering, so the first frames after some cuts arrive 100-250 ms late, and a cue's first frames can be skipped entirely. The sound itself comes from separate audio elements and is not affected.
Related work
Follows the pre-roll revert and the start-latency learning already on main. No issue.
How
dropSilentAudioTrackruns per clip insyncRuntimeMedia: for a<video>whosedefaultMutedis true it sets everyaudioTracks[i].enabledto false. Engines withoutaudioTracks(Chromium) skip it. Videos withoutmutedare untouched; the runtime already keepsdefaultMutedvideos muted (init.tsmute paths), so nothing audible changes.Size
A lone fix: nine lines of runtime and its tests; nothing else in flight touches this path, so there is no PR to fold it into.
Test plan
Measured on the iOS 26.5 Simulator by capturing the screen and the sound the app produces, 10-take fixture, 44 scored cues per run, three runs per arm interleaved (load 14-15 at start). Picture minus playhead, + = picture late:
Sound against the playhead stays +18..+24 ms on both.
media.test.ts; fails without the fix: two of the three fail on main (the track stays enabled), the control passes on both.Small on purpose: a lone urgent playback fix that nothing open can carry.