Skip to content

verify(tooling): Stryker mutation tooling + upstream 9246095 pilot (batch 1) - #55

Closed
axisrow wants to merge 2 commits into
mainfrom
verify/tooling
Closed

axisrow wants to merge 2 commits into
mainfrom
verify/tooling

Conversation

@axisrow

@axisrow axisrow commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Verify batch 1: mutation tooling + pilot (upstream 9246095)

First batch of the macOS verification wave for upstream v2.2.31/v2.2.32
bugfixes (red → green → mutation, per the /fix TDD pipeline).

Pilot: 9246095 — accept only release versions from the npm registry

Step Result
Red (pre-fix 7f31041) upstream-update-checker.test.ts fails 3/4: hostile (2.3.0 & calc) and incomplete (2.3) registry versions give update-available; runGlobalPackageInstall resolves on a hostile version
Green on fixed tree (bun test) 10/10 pass
Green on fixed tree (bunx vitest run) 10/10 pass
Mutation (bunx stryker run --mutate src/utils/update-checker.ts) every mutant on the lines this fix added (RELEASE_VERSION regex, both guards) killed; survivors found by the first run (anchor-drop, \d+→\d, build-group, if (true)) killed by strengthened assertions

The test is mock-free by construction (injectable latestVersionFetcher,
PATH-stubbed npm), so it runs under both runners; the stub never touches
the network.

Tooling

  • @stryker-mutator/core + @stryker-mutator/vitest-runner (devDeps),
    stryker.config.json (perTest coverage, concurrency 4, empty mutate
    guard — a forgotten --mutate fails loudly), vitest.stryker.config.ts.
  • The Stryker vitest profile excludes the test files that mock Node
    builtins via vi.spyOn(<builtin>, fn): they run green under bun test
    (the CI runner, 3481 pass / 1 known macOS-only LFS fail) but cannot run
    under real vitest ("Module namespace is not configurable in ESM").
    The profile itself runs green: 148 files, 2455 tests. Files covered only
    by excluded tests are triaged in PR tables as bun-covered.

🤖 Generated with Claude Code

axisrow and others added 2 commits October 12, 2026 00:44
Mutation-testing harness for the upstream-fix verification wave (/fix
pipeline, step 4): @stryker-mutator/core + @stryker-mutator/vitest-runner,
a scoped config (perTest coverage, concurrency 4, empty `mutate` guard —
a forgotten --mutate fails loudly instead of mutating anything), and a
vitest profile that excludes the test files which mock Node builtins via
`vi.spyOn(<builtin>, fn)` — those run green under `bun test` (the CI
runner) but cannot run under real vitest ("Module namespace is not
configurable in ESM"). Everything excluded here is bun-covered; the
profile itself runs green: 148 files, 2455 tests.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Verification of sirmalloc/ccstatusline@9246095 (accept only release
versions from the npm registry) on macOS, red→green→mutation:

- Red: on pre-fix 7f31041 the test fails 3/4 — a hostile ('2.3.0 & calc')
  or incomplete ('2.3') registry version yields 'update-available', and
  runGlobalPackageInstall resolves for a hostile version.
- Green: 10/10 under `bun test` and `bunx vitest run` on the fixed tree.
  Mock-free by construction (injectable fetcher, PATH-stubbed npm), so it
  runs under both runners; the stubbed npm never touches the network.
- Mutation: Stryker on src/utils/update-checker.ts — every mutant on the
  lines this fix added (RELEASE_VERSION regex, both guards) is killed;
  'latest is 2.3.0' and multi-digit/build-suffix cases were added to kill
  the anchor-drop, \d+→\d and build-group survivors. The rest of the file
  is covered by src/utils/__tests__/update-checker.test.ts, which runs
  under bun only (excluded from the Stryker profile).

Co-Authored-By: Claude Code <noreply@anthropic.com>

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review of the inherited branch (from the owning session), full diff vs main 049cad5 verified locally on macOS.

Branch state: merge-base = current main 049cad5 (upstream v2.2.32 merge) — no rebase/update needed; GitHub reports MERGEABLE; CI green on head 5004690 (Build / Lint & Type Check / Test).

Claims verified locally:

  • src/utils/__tests__/upstream-update-checker.test.ts: 10/10 pass under bun test; also green under the vitest profile. Assertions match what they claim: hostile (2.3.0 & calc, latest is 2.3.0) and incomplete (2.3) registry versions → registry-failure with the exact error message; prerelease (2.3.0-beta.1) and plain releases (incl. build metadata 2.3.0+b7) pass the guard; runGlobalPackageInstall rejects a hostile version before spawning (a dropped guard would run the PATH stub npm and the rejects assertion would fail — the test kills that mutant by construction), and resolves a release through the stub.
  • Stryker wiring: bunx stryker run --mutate src/utils/update-checker.ts --dryRunOnly succeeds (local v10, perTest coverage, initial run 38 tests in 6s).
  • Empty mutate: [] guard: bunx stryker run --dryRunOnly fails loudly with ConfigError: No tests were executed — as documented.
  • vitest.stryker.config.ts profile: 149 files / 2465 tests locally; only 2 failures observed, both timing-flaky and pre-existing on macOS: daemon/__tests__/idle-stop.test.ts and utils/__tests__/custom-command-process.test.ts — pass in isolation, don't use builtin spies (so correctly not in the exclusion list), and are outside the pilot's mutation scope (perTest coverage skips them when mutating update-checker.ts). Not a blocker.

Findings requiring changes: none. Non-blocking notes: .gitignore entry reports also matches bare files named reports anywhere (could be reports/); the two stubNpmOnPath() temp dirs are never removed (harmless, OS tmpdir); PR body counts (148 files / 2455 tests) predate the new test file joining the profile (149 / 2465). None worth a follow-up commit.

@axisrow axisrow left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review complete (from the owning session): full diff vs main 049cad5 verified locally — tests assert what they claim (10/10 bun test, green in the vitest profile), Stryker wiring works (dry run + empty-mutate guard both verified), CI green, branch already based on current main. No findings requiring changes. Full notes: see my comment-review on the PR.

@axisrow

axisrow commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Closing: all content is already in main — the 9246095 test landed via #57 (1cccb73) and the Stryker tooling via #56/#57; rebasing dropped both commits as already upstream, leaving a zero-diff branch. Nothing left to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant