Skip to content

CI collects coverage but enforces no threshold: _mstest-coverage.yml never converts to Cobertura or compares a floor #561

Description

@drmoisan

Summary

.github/workflows/_mstest-coverage.yml collects code coverage but never gates on it. The job runs
vstest.console.exe ... /EnableCodeCoverage and uploads the resulting .trx and .coverage
artifacts. It never converts the .coverage output to Cobertura and never compares any figure
against a floor.

Impact

A coverage regression cannot fail CI on main. The 80% Cobertura line-coverage gate reconciled by
#494 is enforced only by local tooling (Assert-CoberturaLineCoverageThreshold in
scripts/vscode/Invoke-MSTestWithCoverage.Helpers.ps1), which developers invoke through the VS Code
task. It is not one of the five required status checks in the main ruleset.

This is the same class of defect the build-ci-coverage-gate-fidelity epic set out to remove: a
gate that appears to measure something and does not. The epic corrected the arithmetic the gate
computes without wiring that gate into CI.

Evidence

  • .github/workflows/_mstest-coverage.yml lines 54-96: coverage is collected and uploaded; no
    threshold step exists.
  • main branch ruleset required contexts: actionlint / actionlint,
    format-check / Verify formatting, build-analyzers / Build with analyzers and code style enforcement,
    build-nullable / Build with nullable warnings treated as errors,
    mstest-coverage / Run MSTest suite with coverage. None asserts a coverage percentage.

Suggested resolution

Add a step to _mstest-coverage.yml that converts the .coverage artifact to Cobertura and invokes
the same threshold assertion the local tooling uses, so the CI gate and the developer gate enforce
one number. Resolve #TBD-threshold-contradiction first, so the number being enforced is settled.

Found during the build-ci-coverage-gate-fidelity epic fan-in review.

Activity

  1. drmoisan commented on Sep 28, 2026

    @drmoisan
    OwnerAuthor

    Verified fixed on main at 177b6d7 (2026-09-28). PR #897 (item #869, commit 9f7aa9c) made .github/workflows/_mstest-coverage.yml:87-96 run Invoke-MSTestWithCoverage.ps1 -SearchRoot .. That run asserts the 80% line and 75% branch floors (Invoke-MSTestWithCoverage.ps1:386-387) and fails the required mstest-coverage check below either one. Closing as fixed by #897.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions