Skip to content

CI: replace the emulator screenshot pipeline with JVM screenshot tests #125

Closed

Description

@dzid26

What the current pipeline costs

  • Commit share. 63 of the 279 commit subjects so far are about screenshots, capture, sheets or attachments, and they dominate the last two days on main.
  • Duplication. Four workflows boot an emulator: android.yml (tag and preview jobs), pr-screenshots.yml and demo-smoke.yml. That is eight copies of the android-emulator-runner capture-plus-retry block. The six required screenshot names are hard-coded in five places (android.yml, pr-screenshots.yml, demo-smoke.yml and both publish scripts).
  • Fragility. .github/scripts/capture-screenshots.sh is 505 lines of bash that drive uiautomator by visible text, so it breaks on UI changes. Recent Scanning list: RSSI for all cars, stable order, spinner, auto-stop #100 commits are fixes of exactly this kind ("Gate the scan shot on the cars-list title", "Return to the list before the light Settings step"). The daily Demo smoke on main failed on 2026-10-07, and PR screenshots are failing on Scanning list: RSSI for all cars, stable order, spinner, auto-stop #100.
  • Time. Every preview build waits about 15 minutes for the capture, which is why main CI runs can't be cancelled.
  • Little signal. The screenshots assert nothing about behaviour. They are posted as a new PR comment on every review round: 8 of the 13 comments on Replace the history CSV with an append-only raw ChargeState log #94 are screenshot-bot comments.
  • A misleading comment. demo-smoke.yml and pr-screenshots.yml say "The windowed default never renders the app on CI runners". But the explicit emulator-options they set are the action's defaults; what actually changed was the retry.

Proposal

  • Render each screen from fixed BleUiState and history fixtures with Roborazzi (on Robolectric) or Paparazzi, inside testDebugUnitTest, as light and dark golden images. No emulator and no fake-BLE timing are involved, and the same fixtures can drive Compose UI tests, which covers the open master-plan item "Compose UI smoke tests + screenshot tests in CI".
  • Publish screenshots for the README, website and release notes from that JVM run. Keep at most one emulator smoke test (install, launch, demo car connects) as a reusable workflow.
  • Post one sticky comment per PR and update it in place, instead of a new comment per round — or post only when the golden images change.

Activity

  1. dzid26 commented on Oct 15, 2026

    @dzid26
    CollaboratorAuthor

    Progress: #194 replaced the PR emulator screenshots with Paparazzi card diffs, followed by #205, #206, #203 and #221 (highlighted diff image). The demo-smoke emulator capture remains by design (#220 added the notification shot). Still open here: whatever emulator workflows are left to retire.


    Generated by Claude Code

  2. dzid26 commented on Oct 16, 2026

    @dzid26
    CollaboratorAuthor

    Status against main: most of this is done, so the issue stays open for the leftovers.

    Done:

    Left:

    • The emulator-runner block still appears four times (two in android.yml for the smoke test and its retry, two in demo-smoke.yml). Extract a reusable workflow or composite action.
    • The "windowed default never renders the app on CI runners" comment in demo-smoke.yml is still there; the issue noted it is misleading (the options are the action's defaults, the retry is what changed).
    • capture-screenshots.sh is still the 500-line uiautomator script for those smoke runs.

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions