Skip to content

ci(web): run web unit tests in parallel shards and only on web changes - #8158

Open
mmabrouk wants to merge 1 commit into
mainfrom
ci/web-unit-tests-split
Open

mmabrouk wants to merge 1 commit into
mainfrom
ci/web-unit-tests-split

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Nov 6, 2026

Copy link
Copy Markdown
Member

Problem

run-web-unit-tests took about 32 minutes on every PR, and every PR waited for it.

  • It ran all 19 web packages one after another on one small runner. mobile alone took 13.6 minutes.
  • It also ran on PRs that change no web code. fix(deps): bump filelock to 4.2.1 in services and sdks/python #8139 changed only two uv.lock files and still ran 33 minutes of web tests.
  • It ran the old /w app tests (web/oss vitest, 3.5 minutes, plus 7 Playwright specs that needed a Chromium install).

Measured from 108 finished runs since 2026-11-04. Before #8116 (all packages at once), 71 of 81 runs failed, mostly on timeouts. After #8116 (one package at a time), 4 of 27 runs failed: 3 were real bugs in the PR (#8130, #8133, #8116), and 1 looked flaky (#8134). So the job finds real bugs. The issue is the time.

Change

  • A new detect-web-changes job checks the changed files with dorny/paths-filter. The web shards run only when something under web/ or this workflow changed.
  • The tests run in 5 parallel jobs: mobile in 3 vitest shards, entity-ui alone, and the other packages together. Inside each job, packages still run one at a time, so the CPU starvation from before fix(frontend): web unit tests fail at random from CPU contention #8116 does not come back.
  • web/oss and web/ee tests no longer run here, so the job no longer installs Chromium.
  • One run-web-unit-tests job collects the JUnit files, publishes one "Application Web Unit Test Results" check, and fails when any shard fails. It passes when the shards were skipped because no web code changed. Agents and rules can keep waiting on the same check name.

run-tests.ts is unchanged, so the local --layer unit command still runs everything.

Before and after

Before After
Web PR, wall time 32 min median measured on this PR's runs (see comment)
Non-web PR 32 min skipped

…on-web PRs

The web unit job ran every package one after another for about 32 minutes,
and it ran on API, SDK and dependency PRs that change no web code.

- Detect web changes with dorny/paths-filter; skip the web shards otherwise.
- Run mobile in 3 vitest shards, entity-ui alone, and the other packages
  together, each on its own runner.
- Stop running the old /w app tests (web/oss vitest and Playwright unit specs),
  so the job no longer installs Chromium.
- Keep one run-web-unit-tests check that publishes all results and fails when
  any shard fails.

Entire-Checkpoint: 01M6R6E7GAXZ021VBPJVGYVE2V
@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 5 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 8 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 8d75125 and dc5c059.

Files selected for processing: 1

  • .github/workflows/12-check-unit-tests.yml
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@mmabrouk

mmabrouk commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

🤖 The AI agent says: Measured on this PR's first run (run 42202999493). All 5 shards passed. The published check reports 9,100 tests passing.

Before (27 runs after #8116) After (this PR)
Web PR, wall time 32 min median, 33.5 min p90 8.6 min for all shards, 10.8 min until the final check (about 2 min of that was a queue wait for a runner)
Slowest shard n/a other packages, 7.7 min of tests
mobile shards 13.6 min in one run 2.9 to 4.2 min each
Non-web PR 33 min (for example #8139) skipped
Failure rate 4 of 27 (3 real bugs, 1 flaky) 0 of 1 so far; needs more runs to judge

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant