Skip to content

Fix analyzer clip framerate #52

Description

@Abdullah-Builds

Summary

render_timeline_clip() in artemis/utils/video.py renders every analyzer clip at ~25fps instead of the requested fps (default 15), silently duplicating frames. Since the whole point of this function is to keep a frame index mappable back to a real recording timestamp, the mismatch breaks that guarantee on every analyzer clip the agent renders.

Root cause

Each input segment's ffmpeg filter chain sets an explicit fps={fps}, but the final encode command never sets an explicit output frame rate (-r). Without one, ffmpeg has no framerate to encode the muxed output with and falls back to a default of 25fps, inserting duplicate frames (via PTS) to preserve the real wall-clock duration:

frame=   77 fps=0.0 q=28.0 Lsize=  4KiB time=00:00:03.00 bitrate=10.6kbits/s dup=31 drop=0

A 3.0s window that should produce ~45 frames at 15fps instead produces 77 (~40% duplicate frames). Duration is correct; encoded framerate is not.

Impact

UnifiedMobileController.render_timeline_clip (artemis/controllers/unified_controller.py:408) is the only caller and runs on the production path — every analyzer clip the agent renders during a task goes through this. Any downstream logic assuming frame_index / fps ≈ recording_time (which is exactly what this function's docstring promises: "so start_time + frame_offset still names the true recording time") is working against a clip whose actual framerate silently diverges from fps.

Reproduction

This is caught by an existing, currently-failing test on main:

tests/unit/test_unified_controller_video.py::test_analyzer_clip_keeps_timeline_time_across_restart_gap
FAILED - assert 43 <= len(frames) <= 47
  AssertionError: assert 77 <= 47

Reproduces the same way outside pytest by calling render_timeline_clip() directly and reading frame count with cv2.VideoCapture.

Fix

Pass -r {fps} to the output stage of the ffmpeg command so the encoder gets an explicit framerate instead of guessing:

             "-filter_complex", ";".join(filter_parts),
             "-map", "[outv]",
+            "-r", str(fps),
             "-c:v", "libx264",

Verified: frame count drops from 77 → 46 (within the test's expected 43–47 range); the previously-failing test now passes, and the full tests/unit/test_unified_controller_video.py file (22 tests) passes. ruff format --check, ruff check, and pyright are clean on the changed file.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions