Skip to content

hydra-gates: five false-positive modes (~130 of 276 fleet findings), one with dangerous remediation advice — plus the runner is unsafe to run concurrently #158

Closed

Description

@rubenvdlinde

Re-running the gates against the full tree across 21 fleet repos (excluding openregister) produced 276 failing gates. Triaging the authorisation gates line by line — reading every call path — turned up five distinct false-positive modes. Together they account for ~130 of the 276, and one of them ships remediation advice that would introduce a vulnerability if followed.

Measurement: gate script ConductionNL/.github@ad00986 (main, includes #147/#148/#149), ajv installed under hydra-gates/scripts/lib, each repo at its current origin/development, no --scope-to-diff. Each run was executed in its own mount namespace with a private tmpfs /tmp, because the runner writes its detail logs to hardcoded /tmp/hydra-gate-<name>.log paths and reads the counts back out of them — two concurrent runs silently corrupt each other's numbers (see item 6).


1. gate-38 skip-link — fails in 21 of 21 repos

Uniformity across independent inputs points at the instrument, and it is.

The gate greps a root component for a literal <NcContent> / <NcAppContent>. The fleet has migrated to <CnAppRoot> (nc-vue), and CnAppRoot renders <NcContent> internally — src/components/CnAppRoot/CnAppRoot.vue:60. Every manifest-driven app root therefore fails a check it satisfies.

Second cause in the same gate: it scopes templates/settings/*.php. Those are Nextcloud settings-framework templates; the framework owns the page shell, so such a file can never contain a skip-link and can never pass.

Fix: accept <CnAppRoot> (and any component that transitively renders NcContent) as a shell; drop templates/settings/*.php from scope.

2. gate-9 semantic-auth, rule public-page-annotation-with-auth-body — 36 of 39 findings, and the advice is dangerous

The gate tells you: "remove #[PublicPage] or remove body auth check". Every one of the 36 is a webhook or public-portal endpoint that correctly bypasses NC session auth and authenticates the caller itself. Following either half of that advice breaks the endpoint or removes its only authentication.

Verified call paths:

  • openconnector PaymentsController::webhook — HMAC signature verify with constant-time compare + timestamp tolerance, 401 before any state change. Its docblock already says "the signature check IS the auth body for this route".
  • portaliq ContributionController::inbox (and 9 siblings) — resolves a portal subject(), 401 when absent; per-row subject + tenant + trust-boundary filtering.
  • Same shape: openconnector DSO / IwmoIjw / Notificaties / NotifyNl / OpenFormulieren / Peppol / StufZkn inbound, procest CaseFederationController ×3, hermiq EgressAuthorize + McpRun, launchpad PublicShareController ×2, doriath ApplicationController::create + ApplicationTokenController::exchange, scholiq PaymentTransactionController::callback, shillinq PortalPaymentInitiationController::initiate, softwarecatalog ×2, petstore ×1.

Fix: the rule must distinguish "public but self-authenticating" (signature verify, bearer/portal token, capability token → correct) from "#[PublicPage] while relying on an IUserSession session check" (→ the real mismatch). Only 3 of the 39 findings are the no-admin-required-annotation-with-admin-body shape the gate was actually built for, and those were real — fixed in openbuild#127.

3. gate-7 no-admin-idor — 14 of 15 findings false, even after #149

#149 taught the gate to follow delegation; two blind spots remain.

(a) A guard that returns bool/null instead of throwing or returning 401/403. hermiq ×7: AgentVersionController::index/diff → loadAccessibleAgent($id, $userId) → canUserAccessAgent() (private/owner/invited); AgentsController::index applies canUserAccessAgent() per row inside the result loop. _GUARD_HELPER_NAME_RE requires the name to end in Admin|Access|Permission|Permitted|Owner|Allowed|Authorised — canUserAccessAgent ends in Agent, loadAccessibleAgent starts with load; and _HELPER_GUARD_BODY_RE wants a throw or a 401/403/404, which a bool-returning predicate never has.

(b) Endpoints with no object identifier at all. nldesign CatalogController::tokenSets (public catalogue, no args), nldesign ContrastController::evaluate (pure colour math on caller-supplied input), procest AssistantController::availability (boolean feature probe), hermiq AgentsController::create/stats/tools. There is no object to be insecure-direct-referenced; the gate should not ask for a per-object guard where no object is addressed.

doriath CxpRelayController::put/get are a capability-URL relay (the 24-char ISecureRandom pairing id is the bearer credential, payloads are sealed) and SettingsController::getPolicy is a documented read-open-to-all-authenticated. Both false.

The one real finding was docudesk AnonymizationController::updateRelation — genuinely unguarded through four service hops to an unscoped primary-key write. Fixed in docudesk#382. So gate-7's true-positive rate here is 1 in 15 — high enough to be worth keeping, low enough that the noise will train people to skip it.

4. gate-14 route-reachability — 22 of 30 findings are AppHost delegation

openbuild (7), shillinq (7), doriath (5), planix (3) are reported as having unrouted dashboard#page, dashboard#catchAll, settings#index/create/load, preferences#getPreference/setPreference. All are supplied by OCA\OpenRegister\AppHost\Routes::standard(). The gate parses only the literal array in appinfo/routes.php, so it cannot see routes contributed by the shared builder.

Tell: docudesk passed this gate only because it happens to carry a literal $canonicalRoutes fallback array for the openregister-absent case. Passing should not depend on that.

Fix: when appinfo/routes.php calls Routes::standard(), fold in Routes::canonicalRoutes() before deciding.

The other 8 findings are real and worth keeping — see item 5.

5. gate-6 orphan-auth — 2 of 5 false

  • launchpad LiveTileController::validateSource — a routed controller method (liveTile#validateSource, POST, appinfo/routes.php:557). Its caller is the NC router, not PHP code.
  • pipelinq LogBerichtenboxAdapter::checkMailbox — implements BerichtenboxAdapterInterface::checkMailbox; callers go through the interface.

Fix: exempt routed controller methods and declared-interface implementations.

The other three are real decidesk#60-shaped orphans — implemented, unit-tested, never wired into any production path — and are being filed against their repos: procest BewijsstukService::assertMutable, pipelinq PortalTenantService::isSelfSignupAllowed, pipelinq ZgwCoexistenceValidator::validateWritePath.

6. The runner is not safe to run twice at once

run-hydra-gates.sh writes ~50 detail logs to hardcoded /tmp/hydra-gate-<name>.log and derives each verdict from wc -l on that shared file. Two concurrent runs — trivially easy across repos, or a stray backgrounded run nobody noticed — produce numbers that are plausible but wrong, in both directions.

This bit me during this exercise: a first measurement pass reported softwarecatalog with 7 gate-7 IDOR findings. Running check_no_admin_idor.py directly against the same tree returned zero — the 7 belonged to hermiq, running in parallel. procest moved 22 → 21, softwarecatalog 18 → 16, zaakafhandelapp 8 → 9 once the runs were isolated. Only gate-6 was immune, because it is the one gate already using mktemp.

Fix: derive a per-run log directory once (mktemp -d, or honour TMPDIR) and hang every gate's log off it — the pattern gate-6 already uses. Cheap, and it removes a whole class of "measured, but wrong" from every future fleet sweep.


Suggested priority

  1. item 6 — until it lands, no fleet-wide number from this runner is trustworthy.
  2. item 2 — the remediation text is actively harmful.
  3. items 1, 4 — pure noise, ~43 findings, and they are what makes people stop reading gate output.
  4. items 3, 5 — narrow the guard vocabulary and the scope.

Nothing was suppressed, baselined or excluded anywhere in this exercise.

Activity

  1. rubenvdlinde commented on Aug 5, 2026

    @rubenvdlinde
    ContributorAuthor

    Four more false-positive modes — this time in the two largest finding classes: gate-46 (2,630) and gate-40 (1,365)

    Same method as the five modes above, applied to the two gates that dominate the fleet by finding count rather than by failing-gate count. 2,210 of 3,995 findings (55%) are false, and both gates' true positives are currently buried under them.

    Measurement

    Gate script ConductionNL/.github@ad00986 — verified unchanged for gates 40 and 46 at the time of writing (git log ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.sh is empty). Each of 25 repos exported clean with git archive origin/development (origin/main for tilburg-woo-ui) into a throwaway tree, so no shared checkout's feature branch is in the numbers. Full tree, no --scope-to-diff.

    Isolation, per item 6 above: the gate bodies for 40 and 46 were extracted verbatim out of the runner and driven against private log paths, so nothing touched /tmp/hydra-gate-*.log while other agents were running. Validated against a real isolated run before being trusted — unshare --mount --map-root-user sh -c 'mount -t tmpfs tmpfs /tmp && run-hydra-gates.sh' on openbuild reports gate-40: 162, gate-46: 313, and the extracted harness reports 162 / 313 on the same tree. Same numbers, so the harness is the gate.


    6. gate-46 spec-anchor-existence — 1,479 of 2,630 findings (56%) are requirement-ID anchors that the resolver cannot see

    The fleet writes requirement and scenario identifiers into the heading and cites the identifier in the tag:

    @spec openspec/specs/cmmn-adaptive-case/spec.md#REQ-CMMN-007
    ### Requirement: REQ-CMMN-007 — REST Surface

    The heading exists. The requirement exists. The tag names it unambiguously. has_anchor() returns false anyway, in two distinct shapes.

    6a. ### Requirement: <ID> — <title> cited as #<ID> — 351 findings

    This is an asymmetry inside gate-46's own rules. The heading branch accepts a --bounded prefix match:

    if any(h.startswith(f + '-') for h in hslugs for f in frags if f):
        return True

    …but the :-tail branch, three lines later, only accepts exact equality:

    if ':' in head:
        tail = head.split(':', 1)[1]
        if slugify(tail) in frags or gh_slugify(tail) in frags:
            return True

    Walk Requirement: REQ-CMMN-007 — REST Surface against #REQ-CMMN-007:

    branch value matches req-cmmn-007?
    slugify(head) requirement-req-cmmn-007-rest-surface no
    startswith(frag + '-') starts requirement-, not req-cmmn-007- no
    slugify(tail) req-cmmn-007-rest-surface prefix, but the branch tests equality

    The tail is exactly the case the prefix rule was written for, and it is the one place the prefix rule is not applied.

    Fix: apply the same startswith(f + '-') test to the tail slugs. Not a weakening — the heading must still exist, and must still begin with the cited identifier.

    Distribution: openbuild 179, zaakafhandelapp 78, procest 67, pipelinq 23, openconnector 3, openregister 1.

    6b. ### Requirement: <title> (<ID>) cited as #<ID> — 944 findings

    ### Requirement: Degrade gracefully without the Contacts app (REQ-004)
    ### Requirement: A monthly TimedJob SHALL accrue holiday leave onto every active employee's LeaveBalance (REQ-ACCR-001)
    ### Requirement: Concept resolution API (SKOS-004)

    Cited as #REQ-004, #REQ-ACCR-001, #SKOS-004. No rule covers a trailing parenthesised identifier, so every one is reported.

    Distribution: hrmq 603, zaakafhandelapp 272, docudesk 45, openregister 21, opencatalogi 3.

    A further 184 are the same idea in a heading shape neither sub-rule reaches, most of them pipelinq's #### Scenario REQ-BIE-004-02: Worker picks up pending runs within 60 seconds cited as #REQ-BIE-004-02 — identifier in the head rather than the tail.

    Fix for both: accept a fragment that appears as a -/(/whitespace-bounded token anywhere in the heading. The requirement still has to exist, and the heading still has to name it.

    Why "fix the gate", not "retag 1,479 call sites"

    Two reasons, and the second is the stronger one.

    1. Precedent, in this gate, for this exact judgement. gh_slugify() was added on 2026-08-05 for openconnector's 137 findings, "none of them real", with the reasoning "tags are written against the anchor GitHub actually serves … without this the gate reports a correct, working anchor as unresolved." Requirement-ID citation is the same call: has_anchor already blesses three non-GitHub conventions — positional checkbox ordinals for #task-N, leading-token #5.2, and the requirement--prefix-dropping tail rule. A trailing or leading requirement ID is that same class.

    2. The honest caveat, stated plainly: #REQ-CMMN-007 is not a working GitHub link. Clicking it lands at the top of the file. So there is a real, separate argument for retagging to the full slug — but that argument applies to 1,479 sites across 8 repos and is a fleet-wide convention decision, not something to settle inside a burn-down PR. Whichever way it goes, the gate should not be reporting 56% noise in the interim: at this ratio it trains people to skip the gate, and the 1,151 findings underneath it are where the actual dangling anchors are.

    6c. - [~] partial-progress checkboxes are invisible to the task-item rule — 28 findings, and it silently shifts the positional rule

    portaliq's openspec/changes/supplier-portal/tasks.md:

    - [~] **T02**: Wire the eHerkenning→JWT edge. — PARTIAL (first slice) …

    Both task rules hard-code the three states [ ], [x], [X]:

    if re.match(r'^-\s*\[[ xX]\]', line):        # positional counter
    t = re.match(r'^\s*-\s*\[[ xX]\]\s*(?:\*\*)?([A-Za-z0-9][A-Za-z0-9.\-]*)', line)  # id

    So #T02 never resolves — 28 findings in portaliq alone. The second-order effect is worse than the findings: [~] items are also skipped by item_count, so the positional #task-N convention silently resolves to the wrong item in any tasks.md that mixes states. That is a green-but-wrong path, not a red one — a #task-7 can resolve happily against item 8.

    Fix: widen both character classes to [^\]] (or at minimum [ xX~-]), and count every checkbox item regardless of state.


    7. gate-40 form-label-association — 731 of 1,365 findings (54%) are correctly-labelled controls

    7a. NcCheckboxRadioSwitch has no label prop — its label is the default slot — 461 findings

    The gate asks every one of its five components for a label / input-label prop:

    def has_nc_label_prop(attrs):
        return bool(re.search(r'(^|\s)(:?label|input-label|:input-label|inputLabel)\s*=', attrs))

    NcTextField has label (dist/components/NcTextField/NcTextField.vue.d.ts declares label, labelOutside, placeholder) — correct there. NcCheckboxRadioSwitch does not have one at all. Its published API puts the label in the default slot, and it wires the association itself. From the shipped @nextcloud/vue build (dist/chunks/NcCheckboxRadioSwitch-*.mjs):

    // the <input>
    "aria-labelledby": !$options.isButtonType && !$props.ariaLabel ? $setup.labelId : null,
    // the element that id points at — whose body is the default slot
    createVNode(_component_NcCheckboxContent, { id: !$options.isButtonType ? `${$props.id}-label` : void 0, … })
    …
    renderSlot(_ctx.$slots, "default", {}, void 0, true)

    So this, which the gate reports:

    <NcCheckboxRadioSwitch v-model="showOnlyWithChanges" @update:modelValue="applyFilters">
        {{ t('openregister', 'Show only entries with changes') }}
    </NcCheckboxRadioSwitch>

    …renders an <input aria-labelledby="…-label"> pointing at that text. It is programmatically labelled, by the component's own design. Asking for a label= prop asks for an API that does not exist, and there is no way to satisfy the gate without aria-label-duplicating the visible text — which is worse markup than what is there.

    Real subset, kept: 67 self-closing <NcCheckboxRadioSwitch … /> and 10 with an empty default slot, all with no aria-label. Those genuinely have nothing to point aria-labelledby at.

    Fix: for NcCheckboxRadioSwitch, require aria-label or non-empty default-slot content; drop the label-prop test.

    Distribution: openregister 147, nextcloud-vue 63, procest 34, softwarecatalog 33, pipelinq 31, zaakafhandelapp 24, opencatalogi 24, openbuild 23, hermiq 21, docudesk 13, doriath 11, decidesk 6, others 31.

    7b. Implicit <label> wrapping — 270 findings

    has_for_match() only implements explicit association (<label for=id> ↔ id=). The HTML spec's other form — the control nested inside the label — is not checked, and the fleet uses it heavily:

    <!-- doriath/src/views/HealthReportView.vue:34 -->
    <label v-if="breachGateOn" class="health-report__optin" data-testid="breach-optin">
        <input v-model="breachOptIn" type="checkbox" @change="onBreachToggle">
        {{ t('doriath', 'Check passwords against the Have I Been Pwned breach corpus') }}
    </label>
    
    <!-- shillinq/src/views/invoice/AdminInvoiceList.vue:34 -->
    <label>
        {{ t('shillinq', 'To') }}
        <input v-model="filters.toDate" type="date" @change="reload">
    </label>

    Both are conformant and both satisfy SC 1.3.1 / 4.1.2. Reported anyway.

    Fix: treat an <input> / <textarea> whose offset falls inside a <label>…</label> range as labelled.

    Distribution: openbuild 83, nextcloud-vue 59, doriath 47, shillinq 28, pipelinq 21, procest 18, openconnector 7, openregister 6, scholiq 4, decidesk 3, others 7.


    The remaining 1,785 look real

    gate-46, 1,151 remaining: 678 anchors with no matching heading or body identifier anywhere in the target, 265 tags whose target file does not exist, 40 whose file moved (archived / renamed — the archive index missed them), and 168 citing an identifier that appears only in prose or in a plain bullet, where there is genuinely no anchor to land on. Concentrated in portaliq (128), shillinq (124), doriath (109), procest (107), hermiq (92), openregister (89), pipelinq (164).

    gate-40, 634 remaining: 276 bare <input> with no label of any kind, 155 NcTextField with no label prop, 102 <textarea>, 67 self-closed NcCheckboxRadioSwitch, 23 NcInputField, 10 empty-slot, 1 NcRichContenteditable. These are real WCAG 2.2 AA defects against a fleet that states AA as a requirement. Concentrated in procest (197), pipelinq (82), openregister (81), nextcloud-vue (72), openbuild (56).

    Nothing here was silenced, baselined, excluded or weakened in any repo.

  2. rubenvdlinde commented on Aug 5, 2026

    @rubenvdlinde
    ContributorAuthor

    Independent re-measurement of gates 6/7/8/9 across 19 repos — and a stronger way to take the measurement

    Re-derived the authorisation gates on 2026-08-05 against the full tree of the 19 non-openregister repos this issue covers, and read the call path for every finding. Two things to add: a contamination-proof measurement method, and three more gate-7 FP modes with call paths.

    A measurement method with no shared state at all

    This issue recommends a private mount namespace (unshare --mount + tmpfs /tmp) because the runner writes detail logs to hardcoded paths and derives verdicts from wc -l on them. That works, but there is a stronger option for the authorisation gates specifically: invoke the Python helpers directly and read stdout.

    python3 hydra-gates/scripts/lib/check_orphan_auth.py    $(git ls-files lib/Service lib/Controller | grep '\.php$')
    python3 hydra-gates/scripts/lib/check_no_admin_idor.py  $(git ls-files lib/Controller | grep '\.php$')
    python3 hydra-gates/scripts/lib/check_semantic_auth.py  $(git ls-files lib/Controller | grep '\.php$')
    

    No temp file is created, so there is nothing for a concurrent run to clobber and nothing for a tmpfs to have to isolate. Gate-8 has no helper, so it still needs the shell — a copy with _uar_log="$(mktemp)" substituted for the hardcoded path reproduces it exactly.

    Scale of the shared-state problem, counted: grep -c '_log=/tmp/hydra-gate' run-hydra-gates.sh = 61. Exactly one gate uses mktemp — gate-6 orphan-auth, fixed under hydra#110. So 61 of 62 gate logs are still a fixed path in a shared directory, and every one of them derives its PASS/FAIL from wc -l on that file. This is not a corner case: it is the default for the whole runner.

    Results (19 repos, full tree, no diff scoping)

    gate findings real
    6 orphan-auth 4 1 (procest assertMutable)
    7 no-admin-idor 9 0
    8 unsafe-auth-resolver 0 0
    9 semantic-auth 36 0

    Gate-9's 36 reproduce this issue's item 2 exactly — same repos, same counts, all public-page-annotation-with-auth-body, all webhook/portal self-authenticating endpoints. Confirmed independently.

    Gate-6's 4: procest assertMutable (real — fixed in ConductionNL/dossiq#737), pipelinq checkMailbox (plugin seam, known FP), plus pipelinq isSelfSignupAllowed and validateWritePath — see the note at the end.

    Three more gate-7 false-positive modes, with call paths

    All 9 gate-7 findings are false. They fall into three modes, only one of which is already documented here.

    Mode A — a stateless endpoint with no direct object reference at all is flagged because it does not read a session identity. 4 of 9.

    The Pattern-3 exemption requires zero parameters + no request reads + a session-derived identity ($this->userId / getUID()). The third condition is a positive signal that a stateless endpoint has no reason to emit. An endpoint that returns a feature flag or a public catalogue reads no session identity precisely because it does not need one — and is then flagged for it, despite there being no object id an attacker could substitute. IDOR is structurally impossible here; the exemption fails closed on the safest possible shape.

    • nldesign CatalogController::tokenSets — zero params; returns TokenSetService::getPublicCatalogue(), a deliberately reduced 5-field projection whose docblock names the admin-only fields it excludes.
    • nldesign ContrastController::evaluate — pure computation over caller-supplied colours; touches no stored object.
    • doriath SettingsController::getPolicy — zero params; the org password-policy floor, which the spec (org-password-policies §1.3) says every authenticated user may read.
    • procest AssistantController::availability — zero params; returns one boolean, {available: bool}.

    Fix: treat "no parameters AND no request reads" as sufficient on its own. There is no direct object reference to substitute, so the session-identity condition adds no safety — it only rejects the endpoints that are safest.

    Mode B — capability-token (bearer-secret) authorisation is not in the whitelist. 2 of 9.

    doriath CxpRelayController::put / ::get. The pairingId is the credential: 24 chars of ISecureRandom::CHAR_ALPHANUMERIC (~2^143), validated ctype_alnum and length-capped, slot allow-listed, payload opaque sealed bytes the relay never inspects, one-shot (cache->remove on read), TTL-bounded. This is the same "public but self-authenticating" family as gate-9 item 2 in this issue, and deserves the same treatment.

    Mode C — bool-returning / load* predicate guards applied per row. 3 of 9. Already known; recording the current call paths for the record.

    • hermiq AgentVersionController::index and ::diff — both open with loadAccessibleAgent($id, $userId) and return 404 when it yields null.
    • hermiq AgentsController::index — ObjectService org-scoped multitenancy for the page, then canUserAccessAgent($agent, $userId) applied per row inside the list loop.

    Note for anyone reconciling counts: an earlier triage recorded 7 hermiq gate-7 findings; the current tree yields 3. Given item 6 of this issue, a count taken without isolation is not reconcilable after the fact — which is the argument for the direct-invocation method above.

    Not fixed, and why

    pipelinq isSelfSignupAllowed and ZgwCoexistenceValidator::validateWritePath are left as-is deliberately. Both guard features that do not exist: there is no portal self-signup endpoint anywhere in the repo, and ZrcClient::createZaak() — the "createZaak flow" validateWritePath's docblock claims to protect — has zero callers itself. More decisively, the stufEndpoint schema was relocated to procest, so activeStufWriters() can only ever return [] and validateWritePath() is structurally incapable of throwing at HEAD. Wiring it would install a guard that cannot fire — a green gate protecting nothing, which is the defect this programme exists to remove. Both were already triaged to LEAVE in the archived 2026-07-16-orphan-auth-remediation change, and that call still looks right.

  3. rubenvdlinde commented on Aug 5, 2026

    @rubenvdlinde
    ContributorAuthor

    Correction to my comment above — the gate build I measured on is NOT the one I named, and CI sees MORE than I reported

    Two errors in the measurement note. The FP modes and their counts stand; the provenance sentence does not, and the difference matters to anyone acting on the numbers.

    1. ad00986 does not contain gh_slugify. I measured on a build that does.

    I wrote "Gate script ConductionNL/.github@ad00986 — verified unchanged for gates 40 and 46 (git log ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.sh is empty)."

    The command really was empty. It was empty because git log A..B -- <path> applies history simplification and drops a change that arrives through a merge. HEAD in my checkout was a merge of origin/main, gh_slugify came in on that merge, and the simplified log hid it. --full-history shows it immediately:

    $ git log --oneline ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.sh
    (empty)
    
    $ git log --oneline --full-history ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.sh
    1f1b852 fix(gate-46): resolve anchors the way GitHub actually publishes them
    
    $ git show ad00986:hydra-gates/scripts/run-hydra-gates.sh | grep -c gh_slugify
    0
    $ git show origin/main:hydra-gates/scripts/run-hydra-gates.sh | grep -c gh_slugify
    3
    

    Worth writing down on its own: a path-filtered git log range is not a safe way to prove a file did not change across a merge. It answers a different question than the one it looks like it answers, and it fails silently in the direction of "nothing changed" — the same shape as the dead gates this issue is about. git diff A B -- <path> would have been the honest check and takes the same keystrokes.

    2. No released tag contains the fix — so every repo's CI is on the pre-fix build

    gh_slugify is on main only. Checked every tag:

    tag date gh_slugify
    v1.0.0 2026-08-03 0
    v1.0.1 2026-08-03 0
    v1.1.0 2026-08-04 0
    v1.2.0 2026-08-05 0
    v1.3.0 2026-08-05 0
    origin/main — 3

    This compounds #159. There the problem was that the fleet pins v1.0.1, which predates the fixes; here repinning to the newest tag would not help either, because the fix has never been cut into a tag. A consumer-ref sweep alone does not deliver this one — it needs a release first.

    Verified end-to-end rather than inferred: running the v1.0.1 and v1.3.0 suites against the same docudesk branch, each in its own mount namespace with a private tmpfs /tmp, produces byte-identical verdicts — same six gates, same counts, including gate-46: FAIL — 1.

    3. What that does to my numbers

    My 2,630 was measured with gh_slugify. CI, on every repo, runs without it. Re-measured both ways on the same origin/development trees:

    repo as I reported (with fix) what CI sees (no fix) delta
    openconnector 42 109 +67
    procest 174 180 +6
    zaakafhandelapp 350 350 0
    hrmq 642 642 0
    openbuild 313 313 0
    pipelinq 331 331 0

    So the fleet total in my table is understated by at least ~73 against what CI actually reports, concentrated almost entirely in openconnector. Two cross-checks that the "no fix" column is the right one: this issue's own opening note gives procest as 180, and the openconnector figure recovers the "137 findings, none of them real" observation that motivated gh_slugify in the first place.

    (openconnector reads 42/109 rather than the 46 in my table because its origin/development moved between the two runs — another agent landed work. Not a measurement artefact, just a moving target.)

    4. What still stands

    The four FP modes and their counts are unaffected, because I measured with gh_slugify — my 1,479 is the residue that survives it. Concretely:

    • 6a (Requirement: REQ-X — Title ← #REQ-X, 351) is the prefix-vs-equality asymmetry in the :-tail branch. gh_slugify does not touch it.
    • 6b (Requirement: Title (REQ-X) ← #REQ-X, 944) is a trailing parenthesised id. gh_slugify does not touch it either.
    • 6c (- [~] checkboxes, 28) and the two gate-40 modes are unrelated to slugging.

    So #165 landing on main is a real fix for a real mode — and it leaves the 1,479 above exactly where they were. Both still need doing, and the release needs cutting before either reaches CI.

  4. rubenvdlinde commented on Aug 5, 2026

    @rubenvdlinde
    ContributorAuthor

    gate-8 unsafe-auth-resolver has the same blind spot as gate-7, one layer down: it never reads the CALLER

    Adding to this issue rather than opening a duplicate, because it is item 1's defect wearing a different gate number.

    Measured on hrmq (whole tree, .github@f7eaf2a, private mount namespace + private tmpfs): 13 gate-8 findings, and all 13 are false positives. Every one is rule=throwable-caught-returns-null in an authorize* method:

    lib/Controller/AvgDsrController.php:358    authorizeEmployee
    lib/Controller/CompController.php:143      authorizeAdjustment
    lib/Controller/DocumentController.php:169  authorizeContract
    lib/Controller/DocumentController.php:205  authorizePayslip
    lib/Controller/ExpenseController.php:138   authorizeExpense
    lib/Controller/InterviewController.php:153 authorizeInterview
    lib/Controller/LeaveController.php:142     authorizeTransaction
    lib/Controller/LoonbeslagController.php:332 authorizeLoonbeslag
    lib/Controller/OfferController.php:161     authorizeApplication
    lib/Controller/PayrollController.php:554   authorizeAdjustment
    lib/Controller/PayrollController.php:641   authorizeAssessment
    lib/Controller/PayrollController.php:676   authorizeRun
    lib/Controller/RosterController.php:119    authorizeRoster
    

    Why they are all false

    The gate flags the catch (\Throwable) { return null; } and stops there. It never reads what the caller does with the null — which is the only thing that decides whether the shape is fail-open or fail-closed.

    Here, every caller is fail-closed:

    $transaction = $this->authorizeTransaction($transactionId);
    if ($transaction === null) {
        return new JSONResponse(['error' => 'Transactie niet gevonden.'], Http::STATUS_NOT_FOUND);
    }

    And the null is deliberate. From the method's own docblock:

    Returns null when the transaction does not exist OR the caller's RBAC denies it (both collapse to the same 404 so existence is never leaked).

    That is the multi-tenant tenancy guard already described in item 1 of this issue for gate-7: refusing with "not found" so a 403 cannot be used to enumerate other tenants' object ids. gate-8 flags exactly the code that got this right.

    It is also the opposite of the decidesk#45 defect gate-8 was built for, which was fail-open in the caller:

    $auth = $this->getAuthorizationService();   // returns null on Throwable
    if ($auth !== null) {                        // <- check SKIPPED when null
        $auth->requireChair($user);
    }

    The resolver body is identical in both. Only the caller differs, and the caller is what the gate does not look at.

    Suggested fix

    Make the finding conditional on caller-side handling, which is the same evidence-based move #149 made for gate-7's Pattern 4a — parse the collaborator rather than pattern-match the name:

    • Resolve call sites of the flagged method within the same class (these are all private, so that is the complete set).
    • Clear the finding when every call site guards on the falsy branch — if ($x === null) { return <4xx>; }, ?? throw, if (!$x) return, or an early-return equivalent.
    • Keep it when any call site takes the truthy branch to perform the check (if ($x !== null) { $x->require...() }) — the real fail-open.
    • Where no call site is found, keep the finding: an authorisation resolver with no caller is gate-6's orphan problem and should stay loud.

    That reduces gate-8 on hrmq from 13 to 0 without weakening it: the decidesk#45 shape still trips, because its call site is the truthy branch.

    Scale note

    This matters for the fleet totals in this issue. gate-8 was previously recorded as "0 real" — that is consistent with this, and now has a mechanism rather than a tally. Any repo using OpenRegister's RBAC through a private authorize* helper will show the same pattern, because collapsing deny and not-found into one 404 is the documented multi-tenant idiom (see also #160 item 1).

    Measured with the runner in a private mount namespace and a private tmpfs throughout — the hardcoded /tmp/hydra-gate-*.log paths are shared state, and another agent was running the gates unisolated on the same host during this work.

  5. rubenvdlinde commented on Aug 5, 2026

    @rubenvdlinde
    ContributorAuthor

    Fixed in #175 — plus two defect shapes this issue did not name

    Item 6 (the shared /tmp paths) and item 2 (gate-9's dangerous advice) are closed. Items 1, 3, 4, 5 are not — see the bottom.

    Baseline for everything below is v1.4.0 (46cc1c8), not the numbers in the opening comment: those predate gh_slugify (#165), which lives in v1.4.0 and in no earlier tag. Measured across 21 fleet repos at origin/development.


    Item 6 — the shared /tmp paths. Confirmed, and it is worse than "numbers move"

    You suspected the collision. Here it is, two concurrent runs on different repos at v1.4.0:

    petstore:     [gate-46] FAIL — 26 unresolved @spec target(s)
                  — see /tmp/hydra-gate-spec-anchor-existence.log   ← 0 lines
    app-versions: [gate-46] PASS
    
    petstore:     [gate-40] FAIL — 1     ┐ one file,
    app-versions: [gate-40] FAIL — 7     ┘ 7 lines
    

    petstore's 26 findings were truncated away by app-versions between the write and the wc -l. The ordering is the only thing that stopped this being a falsely-GREEN gate rather than a report pointing at an empty file. Had app-versions truncated a moment earlier, petstore would have printed PASS over 26 real findings.

    61 paths now hang off one private directory per invocation, printed on the first line, TMPDIR honoured. If the directory cannot be created the run refuses (exit 97) rather than falling back to the shared path — an unverifiable scope must not look like a clean one.

    The same defect was in the package's own test suite, which is how it stayed invisible: test_gate_route_auth.sh reported 7 failures under run-helper-suites.sh and 0 standalone. Its _run() even carried a comment acknowledging that a concurrent run "would truncate" the logs, with a snapshot taken after the run as the mitigation — which cannot help, because the truncation happens during it. Fixed there, in test_check_manifest.sh, and in the harness itself.

    Same test on #175: two private directories holding exactly what their own runs reported (26/0 and 0/1). Three concurrent harness runs: 0 failures each.


    Item 2 — gate-9. 34 of 45 cleared, and the advice is rewritten

    Your reading was right in every particular. Returning 401/403 is not, on its own, evidence of a session dependency — it is what a correctly self-authenticating public endpoint does when the credential in the request is missing or revoked. The rule now splits:

    • public-page-annotation-with-session-auth-body — the body tests the session (requireAdmin(), IUserSession) under #[PublicPage]. This is the decidesk defect the rule was built for. Still fires. A session test fires even alongside a token read, because self-authentication excuses an unsourced denial, never a session dependency.
    • public-page-annotation-with-unsourced-denial — 401/403 with nothing resolving a request-borne credential.

    45 → 11 fleet-wide. The advice no longer contains "remove #[PublicPage]" or "remove body auth check" in any finding; RemediationTextIsSafe asserts that on the string itself, because the advice is part of the gate — a precise gate that tells you to open the endpoint is still a vulnerability generator.

    Fixtures are your call shapes: openconnector's HMAC webhook, portaliq's portal subject() inbox, openregister's bearer share token.


    Two shapes this issue did not name

    gate-40's remediation was an accessibility regression. 463 of its 1,211 findings were <NcCheckboxRadioSwitch> elements naming themselves from their default slot. The only way to clear them was aria-label — which overrides the visible label. Two burn-down PRs (opencatalogi#808, docudesk#385) are stuck on exactly that. 1,211 → 517.

    gate-46 was 54% false. 1,995 → 918. The largest single cause: a requirement id in trailing parentheses — ### Requirement: Payment Provider Adapter Interface (REQ-PAY-001) rejecting #REQ-PAY-001. Lifted id tokens match by equality only, so #REQ and #REQ-PAY still do not resolve.

    And a third vacuous-PASS shape, which is the one I would read first: shillinq's development run finished in 22 seconds, all green. Not a shallow checkout — on a push to a mainline branch origin/development IS HEAD, so the diff is empty by construction and every gate passes over nothing. Verified at c64e9fe: 52 gates PASS scoped, 18 FAIL unscoped. Now exit 99.

    ⚠️ That last one will turn currently-green fleet CI red. Those runs are falsely green, but it is worth knowing before the roll.


    Not fixed here

    item status
    1 — gate-38 skip-link / CnAppRoot (21 of 21) open
    3 — gate-7 blind spots (bool guards, no-object endpoints) partial — the tenancy blind spot from #160 is fixed (32 → 26 fleet, openregister 18 → 12). The bool-returning predicate and the no-object-identifier shapes are untouched.
    4 — gate-14 AppHost delegation (22 of 30) open
    5 — gate-6 routed methods / interface impls (2 of 5) open

    Every relaxation in #175 ships with the true-positive case it must not swallow, in four new auto-discovered suites; each relaxed predicate was mutation-checked in both directions (always-true and always-false both fail the suite). Nothing was suppressed or baselined.

    Tag v1.5.0 is ready to cut and has not been rolled — the new refusals mean it must not arrive as a patch.

  6. rubenvdlinde commented on Aug 6, 2026

    @rubenvdlinde
    ContributorAuthor

    v1.5.0 is cut and ready to roll — plus three more defect shapes this issue did not name

    Following #175, PR #176 landed four more. v1.5.0 = 82fa1c5, containing both. Not rolled — see the warning at the bottom.

    Since the last comment

    A third SCOPE WAS EMPTY shape, and this one pointed the wrong way. bin/hydra-gates decided "was the scope empty?" with grep -q "0 changed file(s)" — a substring match. 10 changed file(s) contains it, and so do 20, 30, 100. Runs whose header said ten files changed, with every gate running, also printed the epilogue. Since that string had been adopted fleet-wide as the tell for a vacuous run, the bug manufactured doubt about runs that were fine while saying nothing new about runs that were not. Both statements now derive from one emitted integer, SCOPE-FILE-COUNT. (⚠️ the epilogue's prose also appears in comments in both files — grep the output, or SCOPE-FILE-COUNT: 0.)

    gate-19 accepted a switched-off test as coverage. decidesk carries four tests with empty bodies and a hardcoded test.skip(true, …), each tagged @e2e, each counted, together asserting nothing. Now dead: .skip/.fixme/xit, describe.skip, an unconditional test.skip(true), and an empty body. Still live: test.skip(browserName === 'firefox', …) — the discriminator is the argument, not the call.

    gate-47 classified on the whole file. A PR whose hunks were CSS tokens and a chevron column was told to add a CSRF test; a comment-only PR was told to co-change tests. Neither used the opt-out — the tell. Classifies on git diff -U0 hunks now, with a comment line counting only for an annotation (@NoAdminRequired is the declaration in docblock form).

    gate-62's lib/ scan ignored diff scoping while its manifest half honoured it — the worst of both, since the gate only runs on a manifest change and then judged untouched code. One pipelinq violation blocked every manifest-touching PR there permanently.

    Version-normalised counts

    None of these were taken from CI, because the fleet runs two gate versions. Each number is a specific version of the gate's own code run over the same 22 trees:

    gate v1.3.0 (14 repos) v1.4.0 (2 repos) v1.5.0
    46 spec-anchor-existence 2,228 1,995 918
    40 form-label-association 1,211 1,211 517
    9 semantic-auth 45 45 11
    7 no-admin-idor 32 32 26

    Status of this issue's six items

    item status
    1 — gate-38 skip-link / CnAppRoot (21 of 21) open
    2 — gate-9 advice ✅ 45 → 11, advice rewritten and asserted on the string
    3 — gate-7 blind spots partial — the tenancy blind spot from #160 is fixed (32 → 26). bool-returning predicates and no-object endpoints untouched
    4 — gate-14 AppHost delegation (22 of 30) open
    5 — gate-6 routed methods / interface impls open
    6 — shared /tmp paths ✅ demonstrated, fixed, and the package's own suites fixed with it

    ⚠️ Before rolling

    v1.5.0 will turn currently-green runs red, deliberately: a --scope-to-diff run whose base is HEAD now exits 99 instead of passing 52 gates over nothing. Every push-to-mainline scoped run in the fleet has that shape. Those runs are measuring nothing — but this is a roll to plan, not a rubber stamp.

    Six new test suites, all auto-discovered. Every relaxed predicate mutation-checked in both directions. Nothing suppressed or baselined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions