Skip to content

fix(ci): green PHPUnit on Nextcloud 32-34 and Playwright on development - #1200

Merged
rjzondervan merged 3 commits into
developmentfrom
fix/ci-matrix-and-setup-race
Oct 7, 2026
Merged

rjzondervan merged 3 commits into
developmentfrom
fix/ci-matrix-and-setup-race

Conversation

@rjzondervan

Copy link
Copy Markdown
Member

Gets PHPUnit and Playwright green on development. Neither failure was a product bug: the tests and two CI setup scripts were wrong.

Verified with a manual Code Quality run on this branch, which gets the full matrix and E2E (a PR into development gets neither): run 37600893679. All 8 PHPUnit legs (Nextcloud stable32–35 × PHP 8.3/8.4) and the Playwright E2E job pass. That run's only red, "Changed paths" and the Quality Report gate after it, is a bug in the shared quality.yml, described at the end.

PHPUnit

On development every push run failed on stable32–34; pull requests only run stable35, so it never showed on a PR.

  • stable33/34 (4 errors): FederatedSharesMigrationTest and SiemConnectorColumnsMigrationTest built their table double from ISchemaWrapper::getTable()'s native return type. Up to Nextcloud 34 there is none, so they mocked \stdClass, which has no hasColumn(). They now use a plain recording table there, as ConsolidatedSchemaMigrationTest already does. The migrations themselves were fine: on those versions getTable() returns a Doctrine Table, which has hasColumn().
  • stable32 (81 errors, 12 failures): all in tests/Unit/Federation. Those tests use OCM APIs from Nextcloud 33. Keepiq supports 32–35, and on 32 federation is off by design (FederationRootService::isSupported() is false). The OCM-dependent test classes now skip on 32 through FederationFixtures::skipWithoutFederation(), which uses the same check. With the API present all 145 federation tests run; with it simulated missing, 116 skip and 29 pass.

Playwright

The same three tests failed, retry included, on every E2E run since at least 4f942f7d:

  • l10n-browser-catalogue: the t() probe took the first catalogue entry whose value differs from its key. Plural entries (_singular_::_plural_) hold an array of forms, so they always qualified, and t() cannot resolve them. It now takes a plain string entry.
  • member-overview-offboarding: it expected "1 team folders."; the summary is pluralised now and says "1 team folder.".
  • vault-unlock (locked vault makes no API calls): it walks all 17 manifest routes while locked. Measured from the CI trace, each route costs about 4.6 s (a full page load), so it ran out of the default timeout at route 15. The timeout now scales with the number of routes.
    • With the timeout fixed, the test reached its assertion for the first time since device approval came to the lock screen (fix(vault): show "Approve from another device" on the lock screen; device approval e2e; archive (#787) #1024), and found two calls the lock screen itself makes: device-approvals/status (whether to offer "Approve from another device") and, with device approval on, recovery/enrolment (whether to offer the administrator path). Both return only flags about the session user, {enabled} and {enrolled, policy}, no vault material. They are now on the test's allowlist, each with its reason.

Run locally against a dev instance: the three specs pass (10 passed, 1 skipped), and the development versions of the l10n and offboarding specs fail exactly as in CI.

Setup race in standalone and two-instances

Each failed once on 4934a642 and passed on the other run of the same commit: the first API call right after occ app:enable keepiq got a transient 404. Both setup scripts now wait up to about a minute for the API first. A persistent 404 still fails the run.

Not in this PR

  • Shared workflow bug: in ConductionNL/.github's quality.yml, the step "Does E2E run on this trigger?" aborts on a manual run on a branch without an open PR: printf … | grep -xE 'beta|main' | head -1 exits 1 under set -o pipefail and bash -e. That turns the Quality Report red on such runs. It belongs in that repository.
  • Pre-existing debt: the browser-extension Firefox load check and the CLI image tag on pull requests.

AI disclosure: implemented with Claude Code; reviewed by the author.

🤖 Generated with Claude Code

rjzondervan and others added 3 commits October 7, 2026 10:55
…enable

The push runs on development test Nextcloud stable32-34; pull requests
only run stable35, so these failures never showed on a PR.

- FederatedSharesMigrationTest and SiemConnectorColumnsMigrationTest
  mocked stdClass when ISchemaWrapper::getTable() declares no return type
  (Nextcloud 34 and older), and stdClass has no hasColumn(). They now use
  a plain recording table there, as ConsolidatedSchemaMigrationTest does.
  The migrations were fine: getTable() returns a Doctrine Table there.
- The federation tests that use the OCM APIs of Nextcloud 33 skip on 32,
  where FederationRootService::isSupported() is false and federation is
  off by design. The skip uses the same check.
- The standalone and two-instances setups hit a transient 404 on the
  first API call right after app:enable, once each on 4934a64 (and green
  on the other run of the same commit). They now wait for the API first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The same three failed, retry included, on every E2E run since at least
4f942f7:

- l10n-browser-catalogue picked the first entry whose value differs from
  its key as the t() probe. Plural entries (`_singular_::_plural_`) hold
  an array of forms, so they always qualified, and t() cannot resolve
  them. The probe now takes a plain string entry.
- member-overview-offboarding expected "1 team folders."; the summary is
  pluralised now and says "1 team folder.".
- vault-unlock walks every manifest route while locked. Each goto is a
  full page load, about 4.6 s a route in CI (measured from the trace), so
  the 17 routes ran out of the default timeout at route 15. The timeout
  now scales with the number of routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With its timeout fixed, the locked-vault leak test reaches its assertion
for the first time since device approval came to the lock screen (#1024),
and finds two requests the lock screen itself makes:

- device-approvals/status: checkSuite() asks, whenever online, whether to
  offer "Approve from another device". Answers {enabled}.
- recovery/enrolment: with device approval on, DeviceApprovalRequest
  reads the user's own enrolment to decide whether to offer the
  administrator path. Answers {enrolled, policy}.

Neither carries vault material. Run locally with Playwright against an
instance with device approval enabled: all three specs fixed on this
branch pass (10 passed, 1 skipped), and the development versions of the
l10n and offboarding specs fail exactly as in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 6b110b0

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ✅
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-07 10:11 UTC

Download the full PDF report from the workflow artifacts.

@rjzondervan
rjzondervan merged commit 8b7fd8b into development Oct 7, 2026
99 of 102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant