Skip to content

fix: balance habit mutation shards - #799

Merged
thomasluizon merged 1 commit into
mainfrom
thomasluizon/ticket-1572-mutation-shard
Oct 19, 2026
Merged

thomasluizon merged 1 commit into
mainfrom
thomasluizon/ticket-1572-mutation-shard

Conversation

@thomasluizon

Copy link
Copy Markdown
Owner

The combined habit/freeze mutation job can exhaust its 45 minute limit when Habit.cs changes. Split it into three Stryker character-span shards in .github/scripts/mutation-matrix.mjs, with selection and coverage checks in .github/scripts/mutation-matrix.test.mjs.

The first two shards include nested prefixes and exclude the preceding prefix; the final shard includes the whole file and excludes the second prefix. Because Stryker requires full span containment, mutants crossing a boundary move to the following shard. The final shard retains StreakFreeze.cs and covers Habit.cs growth beyond the current file length. The workflow, thresholds and 45 minute timeout remain unchanged.

Closes thomasluizon/orbit-tickets#1572

Measurement and supported interface

Inspected the successful rerun of orbit-api#793, including its job log and mutation-report-pull_request-domain-habit-freeze artifact. Reproduce with:

gh run list --repo thomasluizon/orbit-api --workflow mutation.yml --limit 20
gh run view 39312601676 --repo thomasluizon/orbit-api --log
gh run download 39312601676 --repo thomasluizon/orbit-api --name mutation-report-pull_request-domain-habit-freeze --dir /tmp/orbit-1572-report

The mutation phase ran from 20:34:43 to 21:04:58, about 30 minutes after roughly 8 minutes of setup, compilation and coverage. Habit.cs supplied all 423 executed mutants (333 killed, 59 survived, 31 timed out), plus 7 without coverage. StreakFreeze.cs supplied no executed mutants: 30 were removed by the since filter and 3 failed compilation. A file-only split would leave the slow work together.

Using the report's actual files, source, mutants, location.start, location.end and status values, checked every Habit.cs and StreakFreeze.cs mutant against the emitted matrix's inclusion and exclusion patterns. All 546 report mutants have exactly one owner, including ignored and compile-error entries. The 430 compiling, non-ignored Habit.cs mutants divide as follows:

Shard Mutants including no coverage Timeouts No coverage
domain-habit-create-log 140 12 2
domain-habit-schedule 133 9 4
domain-habit-update-freeze 157 10 1

Character boundaries 15181 and 25215 fall before GetRemainingCompletions and Update in the measured source. These balance mutant counts and observed timeouts; the report does not provide individual mutant durations. New shard runtimes remain for CI to measure.

Ran dotnet tool restore and dotnet stryker --help for pinned 4.16.0. The installed NuGet manifest identifies source commit f9109e24c615a7030a3b33e5532c665c974e4ec5. Cross-checked character spans, full containment and exclusion precedence against that commit's FilePattern.cs, FilePatternMutantFilter.cs and FilePatternTests.cs. Range values are source character positions, not mutant IDs or line numbers.

Assumptions

  • Preserve project-wide domain selection rather than introduce file-only selection: a change to any domain source or test selects all four domain shards, retaining coverage for shared dependencies.
  • Keep StreakFreeze.cs with the final Habit.cs shard rather than add a standalone job: its 33 report mutants are small relative to Habit.cs, and it contributed no executed mutants in the measured rerun.

Test evidence

  • Before changing implementation or tests, node --test .github/scripts/mutation-matrix.test.mjs passed all 19 existing tests with the slow combined shard still present.
  • Strengthened shard-selection and nightly-scope expectations, then ran the same command before changing the selector: 15 of 24 tests failed because the old combined shard was returned. This includes a change to Habit.cs selects every domain shard and nightly and manual runs keep all shards and their mutation scopes without input.
  • After the split and an additional boundary/file-growth check, the same command passed all 25 tests. Tests drive the selector subprocess through --stdin, including each scoped domain file, domain tests, configuration, shared changes, renames and deletions.
  • dotnet build Orbit.slnx: exit 0, zero errors. Generated OpenAPI files were unchanged.
  • dotnet test tests/Orbit.Domain.Tests --no-build --no-restore: exit 0, 767 passed.
  • dotnet test tests/Orbit.Application.Tests --no-build --no-restore: exit 0, 5219 passed.
  • dotnet test tests/Orbit.Analyzers.Tests --no-build --no-restore: exit 0, 40 passed.
  • dotnet test tests/Orbit.Infrastructure.Tests --no-build --no-restore: exit 0, 3770 passed. All four .NET test projects passed.
  • Mutation execution is left to CI, as ordered; no server or browser was started.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes Reviewed the mutation-shard split and its selector tests, including workflow argument handling and pinned Stryker span semantics.

  • Balanced Habit shards: Replaces the combined Habit/StreakFreeze shard with three disjoint Habit character-span scopes, retaining StreakFreeze in the final shard and covering boundary-crossing mutants and future file growth.
  • Selection coverage: Preserves project-wide domain selection and adds scope, boundary, and file-growth assertions. All 25 selector tests passed locally.
  • Runtime validation: The workflow, thresholds, and timeout remain unchanged. Actual new-shard runtimes remain for CI to measure; full mutation execution was not rerun locally.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@sonarqubecloud

Copy link
Copy Markdown

@thomasluizon
thomasluizon merged commit 4b1c9d7 into main Oct 19, 2026
25 checks passed
@thomasluizon
thomasluizon deleted the thomasluizon/ticket-1572-mutation-shard branch October 19, 2026 21:52
thomasluizon added a commit that referenced this pull request Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant