Skip to content

test(e2e): put the skip reasons where a run can see them - #1552

Merged
rubenvdlinde merged 1 commit into
developmentfrom
fix/skip-reasons-in-the-report
Aug 24, 2026
Merged

rubenvdlinde merged 1 commit into
developmentfrom
fix/skip-reasons-in-the-report

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Both specs here are deliberate and correctly documented. Neither is a defect
and neither is re-enabled. What changes is that their reasons now reach the
run's report instead of living only in a source comment.

spec skips reason in the report
regression/or-cutover-smoke.spec.ts 1 none
spec-coverage/configuration-import-export-ui.spec.ts 4 none

The rationales were already thorough — the cutover file explains its assertions
moved to the Newman collection (folders 02/11/12), and the import/export file
carries a full explanation plus the #1187 re-enable checklist.

But Playwright records neither:

test.describe.skip(title, fn)   // cannot carry a reason at all
test.skip(title, fn)            // records no description either

So a run reported five skipped tests with no explanation whatsoever —
indistinguishable from a spec somebody disabled and forgot.

That is not hypothetical. pipelinq had three describe.skip files whose only
explanation was a // TODO(#392) comment, and they sat dark for months
precisely because nothing that reads a run could see them
(ConductionNL/pipelinq#1414 revived 26 tests from them).

Both now use test.skip(true, reason) — same tests disabled, reason in the
artifact.

Measured

On a real run, with the skip-discipline gate reading the produced report:

before after
V3 skips with no reason recorded 4 0
V1 spec files executing zero tests 1 1 — unchanged, correctly

That V1 line is the point: a reason does not hide a dead file, it only
explains it.
configuration-import-export-ui still contributes no executed
tests and the gate still says so, exactly as it should.

Surfaced by ConductionNL/.github#559.

Both of these specs are DELIBERATE and correctly documented. Neither is a
defect and neither is re-enabled here. What changes is that their reasons now
reach the run's report instead of living only in a source comment.

    regression/or-cutover-smoke.spec.ts             1 skip, no reason recorded
    spec-coverage/configuration-import-export-ui.ts 4 skips, no reason recorded

The rationales were already there and thorough — the cutover file explains its
assertions moved to the Newman collection (folders 02/11/12), and the
import/export file carries a full explanation plus the #1187 re-enable
checklist. But Playwright records NEITHER:

    test.describe.skip(title, fn)   cannot carry a reason at all
    test.skip(title, fn)            records no description either

So a run reported five skipped tests with no explanation whatsoever —
indistinguishable from a spec somebody disabled and forgot. That is not
hypothetical: pipelinq had three `describe.skip` files whose only explanation
was a `// TODO(#392)` comment, and they sat dark for months precisely because
nothing that reads a run could see them.

Both now use `test.skip(true, reason)`, which disables exactly the same tests
and puts the reason in the artifact.

Measured on a real run, with the e2e skip-discipline gate reading the report:

    before   V3 skips with no reason recorded : 4
    after    V3 skips with no reason recorded : 0
             V1 spec files executing ZERO tests : 1   <- unchanged, correctly

That V1 line is the point: a reason does not hide a dead file, it only explains
it. configuration-import-export-ui still contributes no executed tests and the
gate still says so.

See ConductionNL/.github#559.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/integriq @ 19982e4

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
build ✅
check-specs ✅
test-l10n ✅
format ✅
check-schema-l10n ✅
composer ✅ ✅ 145/145
npm ✅ ✅ 534/534
app:check-code ⏭️
info.xml ✅
REUSE ❌
PHPUnit ✅
Newman ✅
Playwright ✅
Hydra gates ✅

Quality workflow — 2026-08-24 09:56 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit d057780 into development Aug 24, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant