Skip to content

A mutant shard's baseline runs what its own mutants name, so it no longer outgrows the 600 s a run is given #396

Closed

Description

@don-petry

Part of the epic "Epic: slice 1, first approval. A Release on next reviews a pull request to this repo, and the machine user's approval stands" (#182).

Why. Before breaking anything, each mutant shard runs a baseline: one run of the whole suite, to show the tests pass on the unbroken code. That run has the same 600-second ceiling as every other run (TIMEOUT in tools/mutants.py). The whole suite now nears 600 seconds on a CI runner and grows with every fixture.

On 2026-10-28 the baseline ran out of time before any mutant was tested, three times:

A baseline that is only too slow reads as one that failed: "the tests fail before anything is broken".

What is wanted (the Factory Builder, 2026-10-28)

  • A shard's baseline runs what that shard's own mutants name, not the whole suite, so it stays proportional to the shard. A filtered run's does the same.
  • A ceiling is kept, derived from what the shard runs or set in configuration with its reason.
  • A baseline that truly fails, with a test red before anything is broken, still stops the shard as today.
  • A baseline that is only too slow says so, in a message distinct from a red test.
  • A test and a mutant hold the per-shard baseline.
  • If the shard workflow changes, check_workflows holds its exact shape and CodeQL stays clean.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions