Repository navigation
test(e2e): put the skip reasons where a run can see them - #1552
Merged
Merged
Conversation
Both of these specs are DELIBERATE and correctly documented. Neither is a
defect and neither is re-enabled here. What changes is that their reasons now
reach the run's report instead of living only in a source comment.
regression/or-cutover-smoke.spec.ts 1 skip, no reason recorded
spec-coverage/configuration-import-export-ui.ts 4 skips, no reason recorded
The rationales were already there and thorough — the cutover file explains its
assertions moved to the Newman collection (folders 02/11/12), and the
import/export file carries a full explanation plus the #1187 re-enable
checklist. But Playwright records NEITHER:
test.describe.skip(title, fn) cannot carry a reason at all
test.skip(title, fn) records no description either
So a run reported five skipped tests with no explanation whatsoever —
indistinguishable from a spec somebody disabled and forgot. That is not
hypothetical: pipelinq had three `describe.skip` files whose only explanation
was a `// TODO(#392)` comment, and they sat dark for months precisely because
nothing that reads a run could see them.
Both now use `test.skip(true, reason)`, which disables exactly the same tests
and puts the reason in the artifact.
Measured on a real run, with the e2e skip-discipline gate reading the report:
before V3 skips with no reason recorded : 4
after V3 skips with no reason recorded : 0
V1 spec files executing ZERO tests : 1 <- unchanged, correctly
That V1 line is the point: a reason does not hide a dead file, it only explains
it. configuration-import-export-ui still contributes no executed tests and the
gate still says so.
See ConductionNL/.github#559.
Contributor
Quality Report — ConductionNL/integriq @
|
| Check | PHP | Vue | Security | License | Tests |
|---|---|---|---|---|---|
| lint | ✅ | ||||
| phpcs | ✅ | ||||
| phpmd | ✅ | ||||
| psalm | ✅ | ||||
| phpstan | ✅ | ||||
| phpmetrics | ✅ | ||||
| eslint | ✅ | ||||
| stylelint | ✅ | ||||
| build | ✅ | ||||
| build | ✅ | ||||
| check-specs | ✅ | ||||
| test-l10n | ✅ | ||||
| format | ✅ | ||||
| check-schema-l10n | ✅ | ||||
| composer | ✅ | ✅ 145/145 | |||
| npm | ✅ | ✅ 534/534 | |||
| app:check-code | ⏭️ | ||||
| info.xml | ✅ | ||||
| REUSE | ❌ | ||||
| PHPUnit | ✅ | ||||
| Newman | ✅ | ||||
| Playwright | ✅ | ||||
| Hydra gates | ✅ |
Quality workflow — 2026-08-24 09:56 UTC
Download the full PDF report from the workflow artifacts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both specs here are deliberate and correctly documented. Neither is a defect
and neither is re-enabled. What changes is that their reasons now reach the
run's report instead of living only in a source comment.
regression/or-cutover-smoke.spec.tsspec-coverage/configuration-import-export-ui.spec.tsThe rationales were already thorough — the cutover file explains its assertions
moved to the Newman collection (folders 02/11/12), and the import/export file
carries a full explanation plus the #1187 re-enable checklist.
But Playwright records neither:
So a run reported five skipped tests with no explanation whatsoever —
indistinguishable from a spec somebody disabled and forgot.
That is not hypothetical. pipelinq had three
describe.skipfiles whose onlyexplanation was a
// TODO(#392)comment, and they sat dark for monthsprecisely because nothing that reads a run could see them
(ConductionNL/pipelinq#1414 revived 26 tests from them).
Both now use
test.skip(true, reason)— same tests disabled, reason in theartifact.
Measured
On a real run, with the skip-discipline gate reading the produced report:
That V1 line is the point: a reason does not hide a dead file, it only
explains it.
configuration-import-export-uistill contributes no executedtests and the gate still says so, exactly as it should.
Surfaced by ConductionNL/.github#559.