Skip to content

Scheduled mutation run: 20 surviving mutants in schema.rs conditional/variant checks #1009

Open
Open
Scheduled mutation run: 20 surviving mutants in schema.rs conditional/variant checks#1009

Description

@avrabe
avrabe
opened on Sep 29, 2026

The scheduled full mutation run on main (run 36549638304, commit 3554f46) failed shard 10 of 16 with 20 surviving mutants. The other failed shard, 13, died in its tool-install step, and 14 shards were cancelled, so this is a lower bound on the whole crate.

Function Missed
schema.rs conditions_equivalent 7
schema.rs get_field_value_for_variant 6
schema.rs yaml_value_to_cow, requirements_overlap, check_conditional_consistency 1 each
sexpr.rs lex 1

Examples: get_field_value_for_variant survives deleting the "status" and "description" match arms and replacing + with - or *. So no unit test checks which field the variant/conditional-rule machinery reads. conditions_equivalent backs the conditional-rule consistency check.

Full list:

rivet-core/src/schema.rs:702:9: delete match arm rivet_yaml::Value::Null in yaml_value_to_cow
rivet-core/src/schema.rs:758:35: replace + with - in get_field_value_for_variant
rivet-core/src/schema.rs:758:35: replace + with * in get_field_value_for_variant
rivet-core/src/schema.rs:765:9: delete match arm "status" in get_field_value_for_variant
rivet-core/src/schema.rs:766:9: delete match arm "description" in get_field_value_for_variant
rivet-core/src/schema.rs:767:9: delete match arm "title" in get_field_value_for_variant
rivet-core/src/schema.rs:768:9: delete match arm "id" in get_field_value_for_variant
rivet-core/src/schema.rs:964:21: replace + with * in check_conditional_consistency
rivet-core/src/schema.rs:997:9: delete match arm (Condition::Matches{field:f1, pattern:p1,}, Condition::Matches{field:f2, pattern:p2,},) in conditions_equivalent
rivet-core/src/schema.rs:1007:9: delete match arm (Condition::Exists{field:f1}, Condition::Exists{field:f2}) in conditions_equivalent
rivet-core/src/schema.rs:996:23: replace && with || in conditions_equivalent
rivet-core/src/schema.rs:1006:23: replace && with || in conditions_equivalent
rivet-core/src/schema.rs:1006:17: replace == with != in conditions_equivalent
rivet-core/src/schema.rs:1006:29: replace == with != in conditions_equivalent
rivet-core/src/schema.rs:1007:82: replace == with != in conditions_equivalent
rivet-core/src/schema.rs:1029:9: delete match arm (Requirement::RequiredLinks{link_types:l1}, Requirement::RequiredLinks{link_types:l2},) in requirements_overlap
rivet-core/src/schema.rs:1092:80: replace == with != in Schema::merge
rivet-core/src/schema.rs:1198:20: delete ! in Schema::validate_consistency
rivet-core/src/schema.rs:1206:20: delete ! in Schema::validate_consistency
rivet-core/src/sexpr.rs:212:20: replace match guard { // Distinguish sign from symbol: +/- is numeric only if followed by digit. let is_sign = matches!(bytes[i], b'+' | b'-'); !is_sign || (i + 1 < bytes.len() && bytes[i + 1].is_ascii_digit()) } with true in lex

Remaining work: add in-module unit tests (the mutation gate runs --lib, so tests under tests/ don't count), rerun shards 10 and 13, and look at why 14 shards were cancelled.

🤖 Generated with Claude Code

Activity

  1. avrabe
    added a commit that references this issue on Sep 30, 2026

    test(schema, sexpr): kill the scheduled run's surviving mutants (REQ-…

    21242f8

  2. avrabe commented on Sep 30, 2026

    @avrabe
    avrabe
    on Sep 30, 2026 – with Claude
    ContributorAuthor

    Triage from the scheduled issue-fixing routine (first pass — filed 2026-09-29, no prior triage comment).

    Body has a concrete artefact (20 surviving mutants with file:line and mutation shape) plus a Remaining work line — "add in-module unit tests (the mutation gate runs --lib, so tests under tests/ don't count), rerun shards 10 and 13, and look at why 14 shards were cancelled." Enough to act on, but no ## Acceptance Criteria block the routine can verify against before opening a PR, so it's deferred until one is pinned.

    The surviving mutants split naturally into two groups that deserve separate acceptance-slice treatment: the semantic ones in schema.rs (13 of 20, in conditions_equivalent / get_field_value_for_variant / check_conditional_consistency / requirements_overlap / merge / validate_consistency) and the CI-shard ones (shard 13 tool-install death, 14 shards cancelled).

    Sketch matching the shape used across #468 / #693 / other quality-loop threads (feel free to revise / re-scope):

    ## Acceptance Criteria — semantic mutants (13 of 20, in-tree)

- [ ] `#[cfg(test)] mod tests` blocks in `rivet-core/src/schema.rs` add in-module `--lib` unit tests that kill every listed mutant in `conditions_equivalent`, `get_field_value_for_variant`, `check_conditional_consistency`, `requirements_overlap`, `Schema::merge`, and `Schema::validate_consistency`. Each mutant killed by a test whose failure message names the mutation shape (e.g. `deleting the "status" match arm in get_field_value_for_variant`).
- [ ] `#[cfg(test)] mod tests` in `rivet-core/src/sexpr.rs` kills the surviving `lex` mutant at `sexpr.rs:212:20` (sign-vs-symbol match-guard replacement with `true`).
- [ ] A re-run of the mutation gate on the changed crate produces **0 surviving mutants in the functions listed above**. The evidence is a link to the re-run (`cargo mutants -p rivet-core -f schema.rs -f sexpr.rs` or the CI shard equivalent), pasted into the PR body.
- [ ] Coverage of the new tests is asserted via `cargo tarpaulin` / `llvm-cov` at ≥ the current file's coverage — no regression.
- [ ] `cargo test -p rivet-core --lib` green; `cargo fmt --check` / `cargo clippy --all-targets -- -D warnings` clean; `rivet validate` PASS on the rivet repo.
- [ ] Commit trailers: `Verifies: REQ-<N>` for the schema conditional/variant/merge invariant (whichever REQ pins the semantic contract these functions implement) + `Refs: #1009`.

## Acceptance Criteria — CI shard health (separate slice, file follow-up if wanted)

- [ ] Root-caused: why did shard 13 die in its tool-install step and 14 of 16 shards get cancelled on run 36549638304? Fix landed either in the shard workflow (retry/pinning) or documented as a known-flake with a matching mitigation.
- [ ] A re-run of the full mutation matrix on `main` completes all 16 shards without cancellation or tool-install failure.
- [ ] Explicitly out of scope for THIS issue: the property-harness proposal in #809 (that's the different pattern — mutation is asking "did anything move that shouldn't", the harness is asking "did the intended edit land correctly"). Adjacent, not the same slice.

    Process side-rail: https://pulseengine.eu/blog/ returned HTTP 503 on this run (checked twice, both blog/ and blog/index.xml). The binding workflow guidance couldn't be consulted this pass, so no PR would land regardless — comment is the AC ask only. If the blog is up next run and speaks to per-issue AC shape, the sketch above will be revised to match.

    Not implemented this run.

    Generated by Claude Code

  3. avrabe commented on Oct 1, 2026

    @avrabe
    avrabe
    on Oct 1, 2026
    ContributorAuthor

    v0.40.0 (tag v0.40.0 on 5a6ef70, PR #1013, REQ-386) kills all 20 survivors listed here. In-module tests cover variant field lookup, condition equivalence, conditional-rule overlap and the s-expression lexer, and a clean local run caught every listed mutant. The truncated-input test also kills 11 more survivors in lex.

    Keeping this open. One acceptance clause isn't met: a clean rerun of the full sharded scheduled job reporting its surviving set. The first scheduled run on the release commit (run 36845145314) didn't produce one:

    • Shard 10, the shard that held these survivors, was cancelled at its 45-minute limit.
    • Shard 13 lost its runner.
    • Shards 00, 01, 04, 06, 08 and 14 were also cancelled.
    • 9 of 16 shards passed.

    Remaining: a scheduled run in which every shard completes. That is either a larger per-shard budget or more shards. It's the same budget problem the PR-diff job hit on #1019, where 90 mutants exceeded 45 minutes.

    🤖 Generated with Claude Code

    https://claude.ai/code/session_015HMQUV3u86jN2hmCtXNTc9

  4. avrabe commented on Oct 16, 2026

    @avrabe
    avrabe
    on Oct 16, 2026
    ContributorAuthor

    v0.41.0 status — remaining: #1013 (v0.40.0) killed all 20 listed mutants plus 11 in lex, verified by clean local cargo-mutants runs. Still missing is the clause recorded on 2026-10-01: a scheduled run where every shard completes and reports schema.rs with no missed mutants. That depends on #1031 — the 64-shard layout (#1083) has not run yet. Closes with the first complete nightly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions