mutate: string content was being offered as constants to perturb - #3256
Merged
Conversation
Two defects, same family, both measured with the tool against master's binary. `masked()` is deliberately language-agnostic -- it runs on a Python oracle, a Verilog header and a YAML workflow. But `#` opens a comment two rules down, so `r#"` was read as `r` followed by a comment to end of line, and the raw string's CONTENTS came back marked as code. A probe holding `pub const FIXTURE: &str = r#"\n threshold = 12345\n other = 6789\n"#;` was reported as "4 literal(s)", offering both fixture numbers as constants. `test_module_lines` ended a `#[cfg(test)]` module at a line that is exactly `}`, and competitors.rs has one inside the raw string `const TWO` (opens 681, the brace at 692). Everything after it was reported as production: 36 offered sites sat below that file's own `#[cfg(test)]` at 567. Perturbing fixture text and reading the red as "the checker noticed" is the tautology `drop_test_module_sites` exists to prevent, through another door. Measured, master's binary vs this one, same eleven files: competitors.rs offered 114 -> 45 skipped 14 -> 77 in-module 36 -> 0 fpga.rs offered 376 -> 376 skipped 637 -> 617 nine others unchanged Four mutants, all previously green, now red: the rule never firing, the hash count ignored, the close searched from the wrong offset, and an unterminated raw string stopping at the quote instead of end of file. Two branches were REMOVED after mutation showed them unexercisable and a probe showed why: a `!prev_is_word` guard and a `br#"` prefix branch, both in `masked()`, where the ordinary-string rule reaches those bytes first and masks the same span. The same guard IS load-bearing in `raw_string_opens`, which has no such rule -- without it `"cannot occur"` reads as an unclosed raw-string opener. Only measurement tells the two apart. Not fixed, reported in the issue: types_dup.rs uses an ORDINARY string continued with a backslash whose fixture has a column-0 brace at 1115, leaving 29 test-only sites offered. A quote-parity rule was written and reverted after measurement -- it took fpga.rs from 376 offered / 637 skipped to 993 / 0. Refs #3255 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gHashTag
enabled auto-merge (squash)
September 5, 2026 08:23
Contributor
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
…ring in the next Records the raw-string pass. The bug is small; three things around it are not. A `!prev_is_word` guard was written into two functions. In `masked()` mutation removed it and everything stayed green -- the ordinary-string rule reaches those bytes first -- so it was deleted. In `raw_string_opens`, a per-line scanner with no such rule ahead of it, the identical check carries the whole case. The same code one function apart is dead in one place and structural in the other, and only measurement separates them. A plausible fix for the ordinary multi-line string -- an odd number of unescaped quotes opens one -- was written and reverted after measurement took fpga.rs from 376 offered / 637 skipped to 993 / 0. And the baseline has to come from the other binary: I recorded gates.rs at 213 from a partially-modified build, then read 221 and chased a regression that did not exist. Master reads 221 too. Refs #3255 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # .claude/skills/ci-gates/SKILL.md
Contributor
PR DashboardGenerated at: 2026-09-05 08:32:43 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-09-05 08:56:32 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This was referenced Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects in the same family. Both measured with the tool itself, against a
binary built from master's own
mutate.rsandgates.rs, so the before columnis measured rather than remembered.
1.
masked()did not know Rust raw stringsIt is deliberately language-agnostic — it runs on a Python oracle, a Verilog
header and a YAML workflow "without knowing which is which". But
#opens acomment two rules down, so
r#"read asrfollowed by a comment to end ofline, and the string's contents came back marked as code:
Now 2 literal(s):
7and42.2. A column-0
}inside a fixture ended the test moduletest_module_linescloses a#[cfg(test)]module at a line that is exactly}.competitors.rshas one at line 692, inside the raw stringconst TWOwhich opens at 681. Everything after was reported as production — 36 offered
sites below that file's own
#[cfg(test)]at line 567.Perturbing a test's own arithmetic fails that test, and the red is then read as
"the checker noticed": the tautology
drop_test_module_sitesexists toprevent, reached through another door.
Measured
competitors.rsfpga.rsMutation
Four mutants, all previously green, now red: the rule never firing · the hash
count ignored · the close searched from the wrong offset · an unterminated raw
string stopping at the quote instead of end of file.
Two branches were removed after mutation showed them unexercisable and a
probe showed why — a
!prev_is_wordguard and abr#"prefix branch, both inmasked(), where the ordinary-string rule reaches those bytes first and masksthe same span. Run on
let _ = xr"junk 55 junk";andbr#"bytes 4242 here"#the tool reports identical mutants either way.
The same guard is load-bearing in
raw_string_opens, which has noordinary-string rule: without it
"cannot occur"reads as an unclosedraw-string opener and swallows every line after it,
#[cfg(test)]included.Only measurement told the two apart.
Not fixed, and why
types_dup.rsholdsconst THREE_SPELLINGS: &str = "\— an ordinarystring continued with a backslash — whose fixture has a column-0
}at line1115. 29 test-only sites are still offered there.
A parity rule ("an odd number of unescaped quotes opens a string") was written
and reverted after measurement: one unbalanced quote puts the scanner into
string mode permanently, so
#[cfg(test)]is never seen again. Across elevenfiles it took
fpga.rsfrom 376 offered / 637 skipped to 993 / 0,prcheck.rs43 → 122, and madecompetitors.rsworse (36 → 43 sites inside itsown test module). Left for a real scanner rather than guessed at.
Prior art
cargo-mutantsparses withsynand applies edits textually. The documentedfailure mode of naive textual mutators is exactly this — a literal
"1 + 2"mutated to
"1 - 2", failing a test for the wrong reason — and the statedminimum is a tokenizer-based skip mask. This tool has one; it was not
Rust-aware.
cargo test -p tri --bin tri: 777 passed, 0 failed.Refs #3255