Skip to content

fix(quality): PHPUnit/phpstan/psalm green and six gates closed at source - #180

Merged
rubenvdlinde merged 8 commits into
developmentfrom
fix/quality-sweep-2026-08-11
Aug 11, 2026
Merged

rubenvdlinde merged 8 commits into
developmentfrom
fix/quality-sweep-2026-08-11

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Quality sweep on hermiq, which was missing from the fleet quality table entirely.

Honest full-scope baseline (run 31490144919, workflow_dispatch --full)

measure before
Hydra Gates 14 failing, 743 findings
PHPUnit all six matrix cells red
phpstan ❌ 3 errors
psalm ❌ 2 errors
E2E (Playwright) ❌
phpcs / phpmd ✅

⚠️ The push-scope figure of "16 gates / ~1,184 findings" is not comparable: gate-16
(spec-coverage, 473) and gate-52 (custom-widget-ratchet) are diff-scoped and drop out
of a full-repo run by design.

What this PR does

PHPUnit, phpstan and psalm were ONE problem wearing three hats

All three were red for the same two missing test stubs. Fixing them takes PHPUnit
from Tests: 1500, Errors: 12 to OK (1500 tests, 4693 assertions), phpstan to
[OK] No errors and psalm to No errors found!. No test was skipped, deleted or
weakened. Details in the commit message — the short version is four defects:

  1. tests/Stubs/Talk/Model/Participant.php stood in for a class spreed does not
    have
    (OCA\Talk\Participant lives at lib/Participant.php; there is no
    OCA\Talk\Model\Participant in spreed at all), so five tests could never pass.
  2. IFlowNodeLogActions had no stub while its sibling IFlowNode did — and
    HermiqAgentNode implements it, so the absence is a class-header fatal.
  3. The ToolRegistryFacade stub had not followed OpenRegister's describeTools().
  4. AgentsControllerTest::testToolsReturnsFacadeCatalogue still stubbed listTools()
    after the controller moved to describeTools(), so the stub never applied.

The six cells were not failing identically. Three carried Errors: 5, Failures: 1
and three Errors: 12. The split tracks occ app:enable openregister failing, which
the shared workflow swallows as a ::warning:: — so an environment failure resurfaced
as application failures. Filed separately; see the commit message for the 6/6 correlation.

Six mechanical gates closed at source

gate-13 modal-isolation 1→0 · gate-40 form-label-association 4→0 ·
gate-24 integration-parity 1→0 · gate-51 schema-property-titles 9→0 ·
gate-54 relation-dialect 1→0 · gate-30 public-monitoring 1→0

Nothing was baselined, excluded or thresholded. Where a gate's own remedy would have
been a regression it was not applied: gate-30 asked for #[PublicPage] on
/api/metrics, which ADR-006 makes admin-only on purpose — the posture is unchanged,
only the wording that declares it.

gate-7 (no-admin-idor) was a FALSE POSITIVE, fixed in .github

All three findings are genuinely guarded. _GUARD_HELPER_NAME_RE required the auth
token to be the final CamelCase segment, so hermiq's verb-object predicates
(canUserAccessAgent) were invisible to it. Fixed fleet-wide in
ConductionNL/.github#353 — hermiq 3→0, every other app in the fleet unchanged.

Verified the gate was not blind first: a planted #[NoAdminRequired] IDOR in the
tracked file lib/Controller/AgentVersionController.php took the count 3→4
(gate-7 enumerates via git ls-files, so an untracked plant is never scanned).
Plant removed. gate-13 and gate-40 were proven live the same way.

All six matrix cells were red, and — contrary to the baseline reading — NOT
identically: three carried `Errors: 5, Failures: 1` and three carried
`Errors: 12` (full-scope run 31490144919). The split is not a version rule.
It tracks `occ app:enable openregister` failing, which the shared workflow
swallows as `::warning::Failed to enable openregister, continuing...`, so a
missing dependency resurfaced as application test failures:

  93774658402 8.4/stable33 enable ok     -> Errors: 5, Failures: 1
  93774658407 8.3/stable33 enable ok     -> Errors: 5, Failures: 1
  93774658415 8.4/stable32 enable ok     -> Errors: 5, Failures: 1
  93774658413 8.3/stable31 enable FAILED -> Errors: 12
  93774658418 8.3/stable32 enable FAILED -> Errors: 12
  93774658420 8.4/stable31 enable FAILED -> Errors: 12

on stable31 deterministically (OpenRegister declares min-version="32"), and on
8.3/stable32 through a transient `curl error 60 ... self-signed certificate`
during composer install that left Twig missing.

Four separate defects, all fixed at source; no test skipped, deleted or
weakened:

1. `tests/Stubs/Talk/Model/Participant.php` stood in for a class spreed does
   not have. spreed declares `OCA\Talk\Participant` in `lib/Participant.php`;
   there is no `OCA\Talk\Model\Participant` anywhere in it. Five
   DeliveryServiceTest cases errored with `UnknownTypeException: Class or
   interface "OCA\Talk\Participant" does not exist` in ALL SIX cells — the
   stub could never have satisfied them. Moved to `tests/Stubs/Talk/
   Participant.php` under `namespace OCA\Talk`, and the two stub files that
   imported the invented namespace updated with it.

2. `tests/Stubs/Service/Flow/IFlowNodeLogActions.php` was missing while its
   sibling `IFlowNode` was stubbed. `HermiqAgentNode` IMPLEMENTS it, so the
   absence is a class-header fatal, not a lazy-resolution miss — six
   HermiqAgentNodeFailureTest cases died on it in every cell where a real
   OpenRegister did not enable.

3. The `ToolRegistryFacade` stub mirrored a two-method contract OpenRegister
   has since outgrown. `AgentsController::tools()` calls `describeTools()`;
   the stub had `listTools()` + `invokeTool()` only, so the mock had no such
   method.

4. `AgentsControllerTest::testToolsReturnsFacadeCatalogue` still stubbed
   `listTools()` after the controller moved to `describeTools()`. The stub
   therefore never applied, the real call returned `[]`, and the test failed
   on assertion in exactly the cells where OpenRegister DID enable — which is
   why the same test appears as an Error in three cells and a Failure in the
   other three. Its descriptor also now matches `describeTools()`'s documented
   shape (app/tool/group/right) rather than the raw LLM-function shape.

Verified: `php vendor/bin/phpunit -c phpunit.xml` in nextcloud:34.0.0-apache
(PHP 8.4.22) goes from `Tests: 1500, Errors: 12` to
`OK (1500 tests, 4693 assertions)`.
gate-13 modal-isolation (1 -> 0)
  AgentTemplateRowActions.vue wrote both its export and its publish modal
  inline. Extracted to src/modals/AgentTemplateExportModal.vue and
  src/modals/AgentTemplatePublishModal.vue. The publish modal now owns its own
  form state and credential fetch; the parent keeps the success notice and the
  list refresh, because both outlive the modal.

gate-40 form-label-association (4 -> 0)
  Four <textarea> elements had no accessible name. Each is now WRAPPED in a
  <label> — the canonical HTML association, no for/id pair needed. Deliberately
  not aria-label: that names the field for a screen reader while leaving a
  speech-input user with nothing visible to say, so it would trade one barrier
  for another. Two of the four relied on a placeholder, which is not a label —
  it disappears as soon as the field has content.

gate-24 integration-parity (1 -> 0)
  The two halves of the `hermiq-agent` leaf registration described different
  leaves: the server half said icon `RobotOutline`, the JS half and
  src/manifest.json say `Creation`. Which icon a consumer drew depended on
  which half it read. `RobotOutline` is the retired spelling — src/icons.js
  keeps it registered only because an agent's OWN icon is free-form and older
  saved agents still carry it. Server half corrected.

gate-51 schema-property-titles (9 -> 0)
  Nine properties on ModelPolicy.defaultModel, GuardrailPolicy.toolPolicy and
  AgentFlowRun carried a title but no description, so the form rendered a label
  with no help text. Descriptions added.

gate-54 relation-dialect (1 -> 0)
  `CourseRecommendation.recommendations.items.courseId` is a foreign key into
  Scholiq's Course register, which OpenRegister cannot express as a $ref — it
  resolves $ref within ONE register set. That is exactly what
  `x-external-register` exists for (the same shape docudesk uses for Procest's
  Zaak), so it is declared rather than the description being reworded until the
  gate's regex stopped matching.

gate-30 public-monitoring (1 -> 0)
  MetricsController::index said "Admin auth per ADR-006"; gate-30's metrics
  carve-out recognises "admin-only", the wording openregister's own
  GenericMetricsController uses. Same posture, different words, so the gate
  read the posture as UNDECLARED and asked for #[PublicPage] — which would have
  published the exposition to anonymous callers to satisfy a gate. The posture
  is unchanged; only the way it is stated.

Both gates whose remedy could have been faked were proven live first by
planting a true positive and watching the count rise, then removing it:
gate-13 and gate-40 each went to FAIL with exactly one planted finding.
hermiq's matrix included `stable31`. OpenRegister declares `min-version="32"`
and is a REQUIRED additional app here — hermiq's storage IS OpenRegister and it
implements OpenRegister interfaces — so on Nextcloud 31 the install step hits

  App "Open Register" cannot be installed because it is not compatible with
  this version of the server.

and the shared workflow swallows it as `::warning::Failed to enable
openregister, continuing...`. The run then proceeds WITHOUT the dependency, and
the absence resurfaces as application failures: both stable31 cells of run
31490144919 reported 12 PHPUnit errors where the three healthy cells reported
5 — seven phantom errors that looked exactly like app debt.

So the cell was not testing hermiq on Nextcloud 31; it was testing hermiq
without OpenRegister. Removed, and `appinfo/info.xml` now declares
`min-version="32"` instead of the unsupportable `30`, so the app states the
floor it actually has.

Measured across the fleet: six repos (app-versions, hermiq, hrmq, petstore,
planix, nextcloud-app-template) run a stable31 cell while requiring
openregister. The swallowed `app:enable` failure itself is a defect in the
shared workflow and is filed separately against ConductionNL/.github — this
commit fixes only hermiq's half, which is wrong independently of it.
…uirements

gate-46 spec-anchor-existence: 201 unresolved findings from 52 distinct
targets -> 25 from 5 targets. Every fixed tag now points at
openspec/specs/<capability>/spec.md#requirement-..., the canonical home,
instead of a change directory.

Two defect shapes were behind the 176:

  * an invented `#task-N-M` convention that markdown never produced --
    `openspec/changes/cost-guardrails/tasks.md#task-4-1` and friends. The
    changes are archived and the gate resolves the archive, but the
    headings there are `### Task 4: BudgetController + routes -- ...`, so
    no `#task-4-1` anchor ever existed.
  * a delta-spec requirement heading that was reworded (or a `#req-NNN-`
    prefix that was never in the heading) before promotion, e.g.
    `#requirement-the-agent-responds-only-when-addressed-in-a-room-it-did-not-create`
    against the canonical `... in a group room`.

No requirement text was written, reworded or added -- every tag was moved
onto a requirement that already existed. Nothing was excluded, baselined
or deleted to make the gate fall.

25 findings are LEFT DANGLING on purpose, because the capability they
name has no canonical requirement to point at:

  * 20  session-context-performance / agent-engine-port conversation-title
        deferral (ConversationTitleWriter, ConversationTitleJob and their
        tests). The change does not exist in this repo at all -- not
        active, not archived -- and no spec anywhere carries a
        conversation-title requirement.
  *  4  openspec/specs/flow-engine/spec.md -- that spec belongs to
        OpenRegister (ADR-065); hermiq has no openspec/specs/flow-engine/.
  *  1  src/formFields/EmailField.vue -> openspec/changes/scaffold-v2/...
        -- a leftover scaffold demo; no scaffold-v2 capability exists here.

Also fixes two pre-existing phpcs errors in AgentsController (inline
comment capitalisation, inline IF) encountered while verifying.

phpunit OK (1500 tests, 4693 assertions); phpstan [OK] No errors;
phpcs 0 errors.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/hermiq @ 59d8031

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-specs ✅
check-manifest ✅
test-l10n ❌
composer ✅ ✅ 117/117
npm ✅ ✅ 746/746
PHPUnit ✅
Newman ✅
Playwright ❌
Hydra gates ❌

Quality workflow — 2026-08-11 13:19 UTC

Download the full PDF report from the workflow artifacts.

…quirements

The modals extracted for gate-13 inherited their parent's @SPEC tags, which
pointed into change directories — one of them at an anchor that no longer
resolves (gate-46), and their new methods carried no @SPEC at all (gate-16, 6
findings). Both now point at `openspec/specs/` requirements that already exist:
publish at "MUST let a template owner publish it to a new tagged GitHub
repository", the credential picker at "MUST never hold or log the GitHub token",
and the form-completeness check at "MUST validate repo coordinates before any
GitHub call". No requirement was written to point at.
@rubenvdlinde
rubenvdlinde requested a review from Rem-Dam as a code owner August 11, 2026 13:24
Both were caused by this branch and are fixed here rather than left for CI to
carry.

l10n — the three visible <label> texts added for gate-40 ("Package contents",
"Skill package", "Template package") were new source strings with no entry in
l10n/en.json, so `test:l10n` and the l10n-coverage check both went red.
Extracted with the repo's own `node tests/l10n/check-l10n.js --write`, and
translated on the Dutch side too: a string that lands only in en.json renders
untranslated for every Dutch user, and nothing downstream would have said so.

phpmd — retargeting the @SPEC anchors also cleared two pre-existing phpcs
errors in AgentsController, and the fix for the inline-IF one replaced a
ternary with an if/else, which phpmd then rejected as ElseExpression. phpmd had
been green before, so this was a fresh red traded for the phpcs one. Rewritten
with neither an `else` nor an inline ternary, so both linters are satisfied at
once, with a comment saying why the shape looks the way it does.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/hermiq @ 349fa9f

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ❌
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-specs ✅
check-manifest ✅
test-l10n ❌
composer ✅ ✅ 117/117
npm ✅ ✅ 746/746
PHPUnit ✅
Newman ✅
Playwright ❌
Hydra gates ❌

Quality workflow — 2026-08-11 13:33 UTC

Download the full PDF report from the workflow artifacts.

`git add -A` in a checkout that already had untracked files staged three
things this sweep never touched and never reviewed:

  --app-root                        a stray file created by a bad CLI call
  g40_before.txt                    scratch gate output
  tests/e2e/flow-execution.spec.ts  285 lines of somebody else's e2e work

All three are removed from the branch with `git rm --cached`, so they stay in
the working tree exactly as their author left them and can be committed
deliberately by whoever wrote them. Nothing is deleted from disk.

The e2e spec is real work — a regression test that presses Run in the browser
and watches objects appear, written because the twelve builder specs can all
pass while no flow on the instance is capable of running. It belongs in a
commit whose message says so, reviewed by someone who knows it, not smuggled
into a quality sweep.

⚠️ This checkout has a SECOND WRITER: `src/api/flows.js` and
`src/store/flowEditor.js` carry uncommitted changes whose mtimes moved DURING
this session (15:35), converting `runFlow()` to a synchronous call. They are
deliberately left unstaged and untouched.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/hermiq @ 2a13c9d

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-specs ✅
check-manifest ✅
test-l10n ✅
composer ✅ ✅ 117/117
npm ✅ ✅ 746/746
PHPUnit ✅
Newman ✅
Playwright ❌
Hydra gates ❌

Quality workflow — 2026-08-11 13:40 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/hermiq @ c8c06dd

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-specs ✅
check-manifest ✅
test-l10n ✅
composer ✅ ✅ 117/117
npm ✅ ✅ 746/746
PHPUnit ✅
Newman ✅
Playwright ❌
Hydra gates ❌

Quality workflow — 2026-08-11 13:50 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit 8252413 into development Aug 11, 2026
33 of 36 checks passed
@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Post-merge record: the one red cell was pre-existing and is unchanged by this PR

Recording this on the PR itself so the merge is auditable without re-deriving it.

E2E (Playwright) was red before this branch and is red after it, identically.
Same test, same assertion, same retry, same totals:

1 failed / 3 skipped / 33 passed
[chromium] tests/e2e/spec-coverage/hydra-console-agent-leaves.spec.ts:197
  the triage flow is seeded once, declares its trigger, and contains only permitted step types
  Error: node "finding" has an unexpected type "undefined"

Nothing in this PR touches tests/e2e/spec-coverage/, lib/Repair/SeedHydraTriageFlow.php
or the flow seed.

Everything else this PR set out to move, moved, verified on development @ 468f5f13
(runs 31502914249 push / 31502920541 dev→beta):

before after
PHPUnit all six cells red (Errors: 12 / Errors: 5 + 1 failure) four cells, all green (stable31 dropped — OpenRegister declares min-version="32")
phpstan ❌ 3 errors ✅
psalm ❌ 2 errors ✅
phpcs / phpmd / eslint / stylelint / build / l10n / Newman mixed ✅ all
E2E ❌ ❌ unchanged, pre-existing

Gates: 7 3→0 · 13 1→0 · 24 1→0 · 30 1→0 · 40 4→0 · 51 9→0 · 54 1→0 ·
46 201→25.

The E2E red is now diagnosed, and it is not a test bug. The seeded Hydra Triage
flow is authored in the pre-inversion shape (behaviour on edges[]) and OpenRegister
inverted that contract on 2026-08-04 (8a7d05440, or-flow-action-nodes / ADR-065),
so the document fails both of the builder's guards and cannot be lowered to a
Definition at all. It is deliberately not fixed here: the flow engine exists only
on OpenRegister development
— origin/main and origin/beta carry zero files
under lib/Service/Flow/ — and this repo's CI pins ref: development, so migrating
the seed would go green against an unreleased branch while breaking every install on
beta/main. Full measurement, the graph dual, and the disposition are in #184.

⚠️ Also on record: tests/Unit/Repair/SeedHydraTriageFlowTest.php is green while
asserting the shape the engine refuses
(testNoNodeCarriesExecutableConfig() asserts
a node must NOT carry a type; the engine throws when a node has none). It has to go
red for the eventual fix to be right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant