PlayerMessages positioned inside a server-side-inserted ad period are never delivered

[timrijckaert]

Version

Media3 main branch

More version details

1.10.1 (latest stable at the time of writing)

Devices that reproduce the issue

Any.
Reproduced deterministically in a Robolectric unit test and on physical devices (Android 14) playing a live DASH stream with AWS MediaTailor server-side ad insertion via ServerSideAdInsertionMediaSource.

Devices that do not reproduce the issue

None

Reproducible in the demo app?

Not tested

Reproduction steps

Use case: client-side ad tracking for a server-side-stitched stream (e.g. AWS MediaTailor).
The stitcher tells the client at which stream positions its tracking beacons (impression, quartiles, complete) are due; createMessage().setPosition(...) is the natural API to be notified at those positions.
Every position that falls inside an ad period is silently ignored.

Prepare a ServerSideAdInsertionMediaSource wrapping a live, dynamic, multi-period timeline.
Mark one mid-window period as a single whole-period server-side ad group
Send a PlayerMessage with a position inside that ad period.
Play from the window start to past the ad period.
The message is never delivered.
A control message positioned inside a content period of the same stream is delivered normally.

import androidx.media3.common.AdPlaybackState
import androidx.media3.common.Player
import androidx.media3.common.Timeline
import androidx.media3.exoplayer.ExoPlayer
import androidx.media3.exoplayer.source.ads.ServerSideAdInsertionMediaSource
import androidx.media3.exoplayer.source.ads.ServerSideAdInsertionUtil
import androidx.media3.test.utils.FakeMediaSource
import androidx.media3.test.utils.FakeTimeline
import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition
import androidx.media3.test.utils.TestExoPlayerBuilder
import androidx.media3.test.utils.robolectric.TestPlayerRunHelper
import androidx.test.core.app.ApplicationProvider
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.google.common.collect.ImmutableMap
import org.junit.After
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import java.util.concurrent.atomic.AtomicBoolean

@RunWith(AndroidJUnit4::class)
@Config(sdk = [33])
class Media3SsaiPlayerMessageReproTest {

    private lateinit var player: ExoPlayer

    @After
    fun tearDown() {
        player.release()
    }

    @Test
    fun `a player message inside a server-side inserted ad period is delivered when playback passes it`() {
        // window: ├─ p0 content (0..20s) ─┼─ p1 ad (20..40s) ─┼─ p2 content (40..60s) ─┤
        // message at window position 30s, inside the server-side ad period.
        prepareSsai()
        val delivered = AtomicBoolean(false)

        player
            .createMessage { _, _ -> delivered.set(true) }
            .setPosition(30_000)
            .setDeleteAfterDelivery(true)
            .send()
        TestPlayerRunHelper.play(player).untilPosition(0, 50_000)

        // FAILS
        assertTrue("message at 30s (inside the server-side-inserted ad period) was never delivered", delivered.get())
    }

    @Test
    fun `control - a player message in a content period is delivered when playback passes it`() {
        // Identical setup; the message sits at 10s, inside content period p0. PASSES.
        prepareSsai()
        val delivered = AtomicBoolean(false)

        player
            .createMessage { _, _ -> delivered.set(true) }
            .setPosition(10_000)
            .setDeleteAfterDelivery(true)
            .send()
        TestPlayerRunHelper.play(player).untilPosition(0, 15_000)

        assertTrue("message at 10s (inside a content period) was never delivered", delivered.get())
    }

    private fun prepareSsai() {
        player = TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()).build()
        val timeline = liveTimeline()
        val content = FakeMediaSource(timeline)
        val ssai = ServerSideAdInsertionMediaSource(content) { false }
        ssai.setAdPlaybackStates(adPlaybackStates(timeline, adPeriodIndex = 1), timeline)
        player.setMediaSource(ssai)
        player.prepare()
        TestPlayerRunHelper.advance(player).untilState(Player.STATE_READY)
    }

    /** FakeTimeline period uids are Pair(windowUid, periodIndex): stable across refreshes. */
    private fun liveTimeline(): Timeline =
        FakeTimeline(
            TimelineWindowDefinition.Builder()
                .setUid("window")
                .setLive(true)
                .setDynamic(true)
                .setSeekable(true)
                .setPeriodCount(3)
                .setDurationUs(60_000_000)
                .setWindowStartTimeUs(1_720_000_000_000_000)
                .setWindowPositionInFirstPeriodUs(0)
                .setDefaultPositionUs(0)
                .build(),
        )

    /** One state per period, keyed by period uid (`ServerSideAdInsertionTimeline` requires full coverage). */
    private fun adPlaybackStates(timeline: Timeline, adPeriodIndex: Int): ImmutableMap<Any, AdPlaybackState> {
        val builder = ImmutableMap.builder<Any, AdPlaybackState>()
        for (periodIndex in 0 until timeline.periodCount) {
            builder.put(
                timeline.getUidOfPeriod(periodIndex),
                if (periodIndex == adPeriodIndex) {
                    ServerSideAdInsertionUtil.addAdGroupToAdPlaybackState(
                        AdPlaybackState(ADS_ID),
                        /* fromPositionUs = */ 0,
                        /* contentResumeOffsetUs = */ 20_000_000,
                        /* adDurationsUs... = */ 20_000_000,
                    )
                } else {
                    AdPlaybackState(ADS_ID)
                },
            )
        }
        return builder.build()
    }

    private companion object {
        const val ADS_ID = "repro-ads-id"
    }
}

Expected result

The message is delivered when playback passes its position.
Unlike client-side ad insertion, content time does not freeze during a server-side-inserted ad the stitched ad occupies real stream time and the playhead sweeps the message position.

Actual result

The message stays pending forever.

Media

Not applicable

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[marcbaechinger]

Thanks for your report.

This is a known restriction. My understanding is that without having a way to set the adGroupIndex and the adIndexInAdGroup when creating the message this can't really work because it's unclear to the player in what period the position should be applied.

From you code sample:

player
            .createMessage { _, _ -> delivered.set(true) }
            .setPosition(30_000)
            .setDeleteAfterDelivery(true)
            .send()

Internal bug ref: b/445665316

[timrijckaert]

Thanks for the confirmation and the internal reference.

For context: our use case is client-side ad tracking for a server-side-inserted stream (AWS MediaTailor).
The stitcher tells us at which stream positions tracking beacons are due, and most of those positions fall inside ad periods.

As a workaround we now poll getContentPosition() at a fixed interval, combined with onPositionDiscontinuity and onTimelineChanged for seeks and window updates.
That works, but gives up the frame accuracy PlayerMessage offers.

Is there a recommended way today to get frame-accurate position callbacks that also works inside server-side-inserted ad periods, or is polling the intended approach until messages can target ad positions (b/445665316)?

[marcbaechinger]

I think the recommended way would be to have timed metadata in the media that then can be fired at a given position. This would leverage an approach that works on any player out of the box and app developers wouldn't need to write app logic for it. It would just integrate into the streaming media ecosystem on any platform naturally. That's a standard way of the stitcher is telling us that likely works on any player as it is based on metadata standards.

If that's not the case I agree that the ExoPlayer proprietary PlayerMessage would be the best way to support that with ExoPlayer. As you report, it doesn't work for ad periods though I'm afraid. We can put this on the list for implementation (see enhancement tag). I'm not sure though if and when we get around looking into this I'm afraid.