Skip to content

test(ci): prove the merge-base ratchet still fails a real coverage drop — DO NOT MERGE - #275

Closed
rubenvdlinde wants to merge 4 commits into
developmentfrom
test/coverage-ratchet-drop
Closed

rubenvdlinde wants to merge 4 commits into
developmentfrom
test/coverage-ratchet-drop

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Second half of the proof for ConductionNL/.github#181. The companion PR #274 carried no code change and Guard coverage baseline passed with "coverage unchanged against the merge base" (head 1020/1422 = 71.73%, merge base 1020/1422 = 71.73%).

This PR deletes tests/unit/Service/CharacterServiceTest.php — a real 21 KB test file. The expected result is that Guard coverage baseline FAILS with a measured drop against the merge base.

That pairing is the point: a ratchet that cannot fail is worse than the race it replaced, so passing an unchanged PR proves nothing on its own until the same mechanism is also shown to go red on a genuine regression.

Do not merge.

.coverage-baseline was read as a floor by the phpunit guard and as an exact
target by the push-side staleness check. Together they demand equality with a
checked-in constant, which against a moving base branch is not satisfiable:
closing "stale" means committing the value the tree will measure after the PR
lands. Measured on openregister — committed 58.93, development advanced
16030->16038 tests, merge result measured 58.88, guard reported a 0.05% drop.

coverage-guard.php gains --against=<clover.xml>, naming a report measured at
the merge base. When present it is the only floor; the committed constant is
reported but not enforced. Both numbers then come from one driver in one job,
so the xdebug/pcov statement-counting difference cancels rather than being
baked in, and the merge base cannot go stale.

Ratios are compared as exact integer cross-products, not rounded percentages:
at two decimals a one-statement regression read as "unchanged" and exited 0.
An empty or zero-statement report is now a hard error rather than 0%, which as
the merge-base side would set the floor to zero and pass every drop.

Verified on real CI clover artifacts: a genuine 1.44% drop fails, an unchanged
tree passes, and adding untested code fails while adding tested code passes.
@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Proof complete — closing unmerged as intended.

Run 31081653450 against the merged quality.yml@main, designated leg job 92552932171:

Run PHPUnit tests           success   <- the suite still passes
Measure merge-base coverage success   <- base measured
Guard coverage baseline     FAILURE   <- the ratchet is what failed
Coverage current:       71.66%  (1019/1422 statements)
Coverage merge base:    71.73%  (1020/1422 statements)
Committed .coverage-baseline: 67.68% — recorded only, NOT enforced on this run.
FAIL: coverage dropped by 0.07% against the merge base.
      merge base 1020/1422 -> head 1019/1422 statements.

Note what the old design would have done with this same PR: 71.66% is comfortably above the committed floor of 67.68%, so the constant-based guard would have reported success. The drop is only visible against a measured merge base. The ratchet did not just survive the change — it caught something the previous one could not.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/larpingapp @ a62667a

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-specs ✅
test-l10n ❌
composer ✅ ✅ 101/101
npm ✅ ✅ 537/537
PHPUnit ❌
Newman ❌
Playwright ✅
Hydra gates ❌

Quality workflow — 2026-08-06 08:02 UTC

Download the full PDF report from the workflow artifacts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant