Repository navigation
Description
Re-running the gates against the full tree across 21 fleet repos (excluding openregister) produced 276 failing gates. Triaging the authorisation gates line by line — reading every call path — turned up five distinct false-positive modes. Together they account for ~130 of the 276, and one of them ships remediation advice that would introduce a vulnerability if followed.
Measurement: gate script ConductionNL/.github@ad00986 (main, includes #147/#148/#149), ajv installed under hydra-gates/scripts/lib, each repo at its current origin/development, no --scope-to-diff. Each run was executed in its own mount namespace with a private tmpfs /tmp, because the runner writes its detail logs to hardcoded /tmp/hydra-gate-<name>.log paths and reads the counts back out of them — two concurrent runs silently corrupt each other's numbers (see item 6).
1. gate-38 skip-link — fails in 21 of 21 repos
Uniformity across independent inputs points at the instrument, and it is.
The gate greps a root component for a literal <NcContent> / <NcAppContent>. The fleet has migrated to <CnAppRoot> (nc-vue), and CnAppRoot renders <NcContent> internally — src/components/CnAppRoot/CnAppRoot.vue:60. Every manifest-driven app root therefore fails a check it satisfies.
Second cause in the same gate: it scopes templates/settings/*.php. Those are Nextcloud settings-framework templates; the framework owns the page shell, so such a file can never contain a skip-link and can never pass.
Fix: accept <CnAppRoot> (and any component that transitively renders NcContent) as a shell; drop templates/settings/*.php from scope.
2. gate-9 semantic-auth, rule public-page-annotation-with-auth-body — 36 of 39 findings, and the advice is dangerous
The gate tells you: "remove #[PublicPage] or remove body auth check". Every one of the 36 is a webhook or public-portal endpoint that correctly bypasses NC session auth and authenticates the caller itself. Following either half of that advice breaks the endpoint or removes its only authentication.
Verified call paths:
openconnector PaymentsController::webhook— HMAC signature verify with constant-time compare + timestamp tolerance, 401 before any state change. Its docblock already says "the signature check IS the auth body for this route".portaliq ContributionController::inbox(and 9 siblings) — resolves a portalsubject(), 401 when absent; per-row subject + tenant + trust-boundary filtering.- Same shape:
openconnectorDSO / IwmoIjw / Notificaties / NotifyNl / OpenFormulieren / Peppol / StufZkn inbound,procest CaseFederationController×3,hermiq EgressAuthorize+McpRun,launchpad PublicShareController×2,doriath ApplicationController::create+ApplicationTokenController::exchange,scholiq PaymentTransactionController::callback,shillinq PortalPaymentInitiationController::initiate,softwarecatalog×2,petstore×1.
Fix: the rule must distinguish "public but self-authenticating" (signature verify, bearer/portal token, capability token → correct) from "#[PublicPage] while relying on an IUserSession session check" (→ the real mismatch). Only 3 of the 39 findings are the no-admin-required-annotation-with-admin-body shape the gate was actually built for, and those were real — fixed in openbuild#127.
3. gate-7 no-admin-idor — 14 of 15 findings false, even after #149
#149 taught the gate to follow delegation; two blind spots remain.
(a) A guard that returns bool/null instead of throwing or returning 401/403. hermiq ×7: AgentVersionController::index/diff → loadAccessibleAgent($id, $userId) → canUserAccessAgent() (private/owner/invited); AgentsController::index applies canUserAccessAgent() per row inside the result loop. _GUARD_HELPER_NAME_RE requires the name to end in Admin|Access|Permission|Permitted|Owner|Allowed|Authorised — canUserAccessAgent ends in Agent, loadAccessibleAgent starts with load; and _HELPER_GUARD_BODY_RE wants a throw or a 401/403/404, which a bool-returning predicate never has.
(b) Endpoints with no object identifier at all. nldesign CatalogController::tokenSets (public catalogue, no args), nldesign ContrastController::evaluate (pure colour math on caller-supplied input), procest AssistantController::availability (boolean feature probe), hermiq AgentsController::create/stats/tools. There is no object to be insecure-direct-referenced; the gate should not ask for a per-object guard where no object is addressed.
doriath CxpRelayController::put/get are a capability-URL relay (the 24-char ISecureRandom pairing id is the bearer credential, payloads are sealed) and SettingsController::getPolicy is a documented read-open-to-all-authenticated. Both false.
The one real finding was docudesk AnonymizationController::updateRelation — genuinely unguarded through four service hops to an unscoped primary-key write. Fixed in docudesk#382. So gate-7's true-positive rate here is 1 in 15 — high enough to be worth keeping, low enough that the noise will train people to skip it.
4. gate-14 route-reachability — 22 of 30 findings are AppHost delegation
openbuild (7), shillinq (7), doriath (5), planix (3) are reported as having unrouted dashboard#page, dashboard#catchAll, settings#index/create/load, preferences#getPreference/setPreference. All are supplied by OCA\OpenRegister\AppHost\Routes::standard(). The gate parses only the literal array in appinfo/routes.php, so it cannot see routes contributed by the shared builder.
Tell: docudesk passed this gate only because it happens to carry a literal $canonicalRoutes fallback array for the openregister-absent case. Passing should not depend on that.
Fix: when appinfo/routes.php calls Routes::standard(), fold in Routes::canonicalRoutes() before deciding.
The other 8 findings are real and worth keeping — see item 5.
5. gate-6 orphan-auth — 2 of 5 false
launchpad LiveTileController::validateSource— a routed controller method (liveTile#validateSource, POST,appinfo/routes.php:557). Its caller is the NC router, not PHP code.pipelinq LogBerichtenboxAdapter::checkMailbox— implementsBerichtenboxAdapterInterface::checkMailbox; callers go through the interface.
Fix: exempt routed controller methods and declared-interface implementations.
The other three are real decidesk#60-shaped orphans — implemented, unit-tested, never wired into any production path — and are being filed against their repos: procest BewijsstukService::assertMutable, pipelinq PortalTenantService::isSelfSignupAllowed, pipelinq ZgwCoexistenceValidator::validateWritePath.
6. The runner is not safe to run twice at once
run-hydra-gates.sh writes ~50 detail logs to hardcoded /tmp/hydra-gate-<name>.log and derives each verdict from wc -l on that shared file. Two concurrent runs — trivially easy across repos, or a stray backgrounded run nobody noticed — produce numbers that are plausible but wrong, in both directions.
This bit me during this exercise: a first measurement pass reported softwarecatalog with 7 gate-7 IDOR findings. Running check_no_admin_idor.py directly against the same tree returned zero — the 7 belonged to hermiq, running in parallel. procest moved 22 → 21, softwarecatalog 18 → 16, zaakafhandelapp 8 → 9 once the runs were isolated. Only gate-6 was immune, because it is the one gate already using mktemp.
Fix: derive a per-run log directory once (mktemp -d, or honour TMPDIR) and hang every gate's log off it — the pattern gate-6 already uses. Cheap, and it removes a whole class of "measured, but wrong" from every future fleet sweep.
Suggested priority
- item 6 — until it lands, no fleet-wide number from this runner is trustworthy.
- item 2 — the remediation text is actively harmful.
- items 1, 4 — pure noise, ~43 findings, and they are what makes people stop reading gate output.
- items 3, 5 — narrow the guard vocabulary and the scope.
Nothing was suppressed, baselined or excluded anywhere in this exercise.
Activity
rubenvdlinde commentedon Aug 5, 2026
Four more false-positive modes — this time in the two largest finding classes: gate-46 (2,630) and gate-40 (1,365)
Same method as the five modes above, applied to the two gates that dominate the fleet by finding count rather than by failing-gate count. 2,210 of 3,995 findings (55%) are false, and both gates' true positives are currently buried under them.
Measurement
Gate script
ConductionNL/.github@ad00986— verified unchanged for gates 40 and 46 at the time of writing (git log ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.shis empty). Each of 25 repos exported clean withgit archive origin/development(origin/mainfortilburg-woo-ui) into a throwaway tree, so no shared checkout's feature branch is in the numbers. Full tree, no--scope-to-diff.Isolation, per item 6 above: the gate bodies for 40 and 46 were extracted verbatim out of the runner and driven against private log paths, so nothing touched
/tmp/hydra-gate-*.logwhile other agents were running. Validated against a real isolated run before being trusted —unshare --mount --map-root-user sh -c 'mount -t tmpfs tmpfs /tmp && run-hydra-gates.sh'on openbuild reportsgate-40: 162,gate-46: 313, and the extracted harness reports 162 / 313 on the same tree. Same numbers, so the harness is the gate.6. gate-46
spec-anchor-existence— 1,479 of 2,630 findings (56%) are requirement-ID anchors that the resolver cannot seeThe fleet writes requirement and scenario identifiers into the heading and cites the identifier in the tag:
@spec openspec/specs/cmmn-adaptive-case/spec.md#REQ-CMMN-007### Requirement: REQ-CMMN-007 — REST SurfaceThe heading exists. The requirement exists. The tag names it unambiguously.
has_anchor()returns false anyway, in two distinct shapes.6a.
### Requirement: <ID> — <title>cited as#<ID>— 351 findingsThis is an asymmetry inside gate-46's own rules. The heading branch accepts a
--bounded prefix match:…but the
:-tail branch, three lines later, only accepts exact equality:Walk
Requirement: REQ-CMMN-007 — REST Surfaceagainst#REQ-CMMN-007:req-cmmn-007?slugify(head)requirement-req-cmmn-007-rest-surfacestartswith(frag + '-')requirement-, notreq-cmmn-007-slugify(tail)req-cmmn-007-rest-surfaceThe tail is exactly the case the prefix rule was written for, and it is the one place the prefix rule is not applied.
Fix: apply the same
startswith(f + '-')test to the tail slugs. Not a weakening — the heading must still exist, and must still begin with the cited identifier.Distribution: openbuild 179, zaakafhandelapp 78, procest 67, pipelinq 23, openconnector 3, openregister 1.
6b.
### Requirement: <title> (<ID>)cited as#<ID>— 944 findingsCited as
#REQ-004,#REQ-ACCR-001,#SKOS-004. No rule covers a trailing parenthesised identifier, so every one is reported.Distribution: hrmq 603, zaakafhandelapp 272, docudesk 45, openregister 21, opencatalogi 3.
A further 184 are the same idea in a heading shape neither sub-rule reaches, most of them pipelinq's
#### Scenario REQ-BIE-004-02: Worker picks up pending runs within 60 secondscited as#REQ-BIE-004-02— identifier in the head rather than the tail.Fix for both: accept a fragment that appears as a
-/(/whitespace-bounded token anywhere in the heading. The requirement still has to exist, and the heading still has to name it.Why "fix the gate", not "retag 1,479 call sites"
Two reasons, and the second is the stronger one.
Precedent, in this gate, for this exact judgement.
gh_slugify()was added on 2026-08-05 for openconnector's 137 findings, "none of them real", with the reasoning "tags are written against the anchor GitHub actually serves … without this the gate reports a correct, working anchor as unresolved." Requirement-ID citation is the same call:has_anchoralready blesses three non-GitHub conventions — positional checkbox ordinals for#task-N, leading-token#5.2, and therequirement--prefix-dropping tail rule. A trailing or leading requirement ID is that same class.The honest caveat, stated plainly:
#REQ-CMMN-007is not a working GitHub link. Clicking it lands at the top of the file. So there is a real, separate argument for retagging to the full slug — but that argument applies to 1,479 sites across 8 repos and is a fleet-wide convention decision, not something to settle inside a burn-down PR. Whichever way it goes, the gate should not be reporting 56% noise in the interim: at this ratio it trains people to skip the gate, and the 1,151 findings underneath it are where the actual dangling anchors are.6c.
- [~]partial-progress checkboxes are invisible to the task-item rule — 28 findings, and it silently shifts the positional ruleportaliq's
openspec/changes/supplier-portal/tasks.md:Both task rules hard-code the three states
[ ],[x],[X]:So
#T02never resolves — 28 findings in portaliq alone. The second-order effect is worse than the findings:[~]items are also skipped byitem_count, so the positional#task-Nconvention silently resolves to the wrong item in any tasks.md that mixes states. That is a green-but-wrong path, not a red one — a#task-7can resolve happily against item 8.Fix: widen both character classes to
[^\]](or at minimum[ xX~-]), and count every checkbox item regardless of state.7. gate-40
form-label-association— 731 of 1,365 findings (54%) are correctly-labelled controls7a.
NcCheckboxRadioSwitchhas nolabelprop — its label is the default slot — 461 findingsThe gate asks every one of its five components for a
label/input-labelprop:NcTextFieldhaslabel(dist/components/NcTextField/NcTextField.vue.d.tsdeclareslabel,labelOutside,placeholder) — correct there.NcCheckboxRadioSwitchdoes not have one at all. Its published API puts the label in the default slot, and it wires the association itself. From the shipped@nextcloud/vuebuild (dist/chunks/NcCheckboxRadioSwitch-*.mjs):So this, which the gate reports:
…renders an
<input aria-labelledby="…-label">pointing at that text. It is programmatically labelled, by the component's own design. Asking for alabel=prop asks for an API that does not exist, and there is no way to satisfy the gate withoutaria-label-duplicating the visible text — which is worse markup than what is there.Real subset, kept: 67 self-closing
<NcCheckboxRadioSwitch … />and 10 with an empty default slot, all with noaria-label. Those genuinely have nothing to pointaria-labelledbyat.Fix: for
NcCheckboxRadioSwitch, requirearia-labelor non-empty default-slot content; drop thelabel-prop test.Distribution: openregister 147, nextcloud-vue 63, procest 34, softwarecatalog 33, pipelinq 31, zaakafhandelapp 24, opencatalogi 24, openbuild 23, hermiq 21, docudesk 13, doriath 11, decidesk 6, others 31.
7b. Implicit
<label>wrapping — 270 findingshas_for_match()only implements explicit association (<label for=id>↔id=). The HTML spec's other form — the control nested inside the label — is not checked, and the fleet uses it heavily:Both are conformant and both satisfy SC 1.3.1 / 4.1.2. Reported anyway.
Fix: treat an
<input>/<textarea>whose offset falls inside a<label>…</label>range as labelled.Distribution: openbuild 83, nextcloud-vue 59, doriath 47, shillinq 28, pipelinq 21, procest 18, openconnector 7, openregister 6, scholiq 4, decidesk 3, others 7.
The remaining 1,785 look real
gate-46, 1,151 remaining: 678 anchors with no matching heading or body identifier anywhere in the target, 265 tags whose target file does not exist, 40 whose file moved (archived / renamed — the archive index missed them), and 168 citing an identifier that appears only in prose or in a plain bullet, where there is genuinely no anchor to land on. Concentrated in portaliq (128), shillinq (124), doriath (109), procest (107), hermiq (92), openregister (89), pipelinq (164).
gate-40, 634 remaining: 276 bare
<input>with no label of any kind, 155NcTextFieldwith nolabelprop, 102<textarea>, 67 self-closedNcCheckboxRadioSwitch, 23NcInputField, 10 empty-slot, 1NcRichContenteditable. These are real WCAG 2.2 AA defects against a fleet that states AA as a requirement. Concentrated in procest (197), pipelinq (82), openregister (81), nextcloud-vue (72), openbuild (56).Nothing here was silenced, baselined, excluded or weakened in any repo.
rubenvdlinde commentedon Aug 5, 2026
Independent re-measurement of gates 6/7/8/9 across 19 repos — and a stronger way to take the measurement
Re-derived the authorisation gates on 2026-08-05 against the full tree of the 19 non-openregister repos this issue covers, and read the call path for every finding. Two things to add: a contamination-proof measurement method, and three more gate-7 FP modes with call paths.
A measurement method with no shared state at all
This issue recommends a private mount namespace (
unshare --mount+tmpfs /tmp) because the runner writes detail logs to hardcoded paths and derives verdicts fromwc -lon them. That works, but there is a stronger option for the authorisation gates specifically: invoke the Python helpers directly and read stdout.No temp file is created, so there is nothing for a concurrent run to clobber and nothing for a
tmpfsto have to isolate. Gate-8 has no helper, so it still needs the shell — a copy with_uar_log="$(mktemp)"substituted for the hardcoded path reproduces it exactly.Scale of the shared-state problem, counted:
grep -c '_log=/tmp/hydra-gate' run-hydra-gates.sh= 61. Exactly one gate usesmktemp— gate-6orphan-auth, fixed under hydra#110. So 61 of 62 gate logs are still a fixed path in a shared directory, and every one of them derives its PASS/FAIL fromwc -lon that file. This is not a corner case: it is the default for the whole runner.Results (19 repos, full tree, no diff scoping)
assertMutable)Gate-9's 36 reproduce this issue's item 2 exactly — same repos, same counts, all
public-page-annotation-with-auth-body, all webhook/portal self-authenticating endpoints. Confirmed independently.Gate-6's 4:
procest assertMutable(real — fixed in ConductionNL/dossiq#737),pipelinq checkMailbox(plugin seam, known FP), pluspipelinq isSelfSignupAllowedandvalidateWritePath— see the note at the end.Three more gate-7 false-positive modes, with call paths
All 9 gate-7 findings are false. They fall into three modes, only one of which is already documented here.
Mode A — a stateless endpoint with no direct object reference at all is flagged because it does not read a session identity. 4 of 9.
The Pattern-3 exemption requires zero parameters + no request reads + a session-derived identity (
$this->userId/getUID()). The third condition is a positive signal that a stateless endpoint has no reason to emit. An endpoint that returns a feature flag or a public catalogue reads no session identity precisely because it does not need one — and is then flagged for it, despite there being no object id an attacker could substitute. IDOR is structurally impossible here; the exemption fails closed on the safest possible shape.nldesign CatalogController::tokenSets— zero params; returnsTokenSetService::getPublicCatalogue(), a deliberately reduced 5-field projection whose docblock names the admin-only fields it excludes.nldesign ContrastController::evaluate— pure computation over caller-supplied colours; touches no stored object.doriath SettingsController::getPolicy— zero params; the org password-policy floor, which the spec (org-password-policies§1.3) says every authenticated user may read.procest AssistantController::availability— zero params; returns one boolean,{available: bool}.Fix: treat "no parameters AND no request reads" as sufficient on its own. There is no direct object reference to substitute, so the session-identity condition adds no safety — it only rejects the endpoints that are safest.
Mode B — capability-token (bearer-secret) authorisation is not in the whitelist. 2 of 9.
doriath CxpRelayController::put/::get. ThepairingIdis the credential: 24 chars ofISecureRandom::CHAR_ALPHANUMERIC(~2^143), validatedctype_alnumand length-capped, slot allow-listed, payload opaque sealed bytes the relay never inspects, one-shot (cache->removeon read), TTL-bounded. This is the same "public but self-authenticating" family as gate-9 item 2 in this issue, and deserves the same treatment.Mode C —
bool-returning /load*predicate guards applied per row. 3 of 9. Already known; recording the current call paths for the record.hermiq AgentVersionController::indexand::diff— both open withloadAccessibleAgent($id, $userId)and return 404 when it yields null.hermiq AgentsController::index—ObjectServiceorg-scoped multitenancy for the page, thencanUserAccessAgent($agent, $userId)applied per row inside the list loop.Note for anyone reconciling counts: an earlier triage recorded 7 hermiq gate-7 findings; the current tree yields 3. Given item 6 of this issue, a count taken without isolation is not reconcilable after the fact — which is the argument for the direct-invocation method above.
Not fixed, and why
pipelinq isSelfSignupAllowedandZgwCoexistenceValidator::validateWritePathare left as-is deliberately. Both guard features that do not exist: there is no portal self-signup endpoint anywhere in the repo, andZrcClient::createZaak()— the "createZaak flow"validateWritePath's docblock claims to protect — has zero callers itself. More decisively, thestufEndpointschema was relocated to procest, soactiveStufWriters()can only ever return[]andvalidateWritePath()is structurally incapable of throwing at HEAD. Wiring it would install a guard that cannot fire — a green gate protecting nothing, which is the defect this programme exists to remove. Both were already triaged to LEAVE in the archived2026-07-16-orphan-auth-remediationchange, and that call still looks right.rubenvdlinde commentedon Aug 5, 2026
Correction to my comment above — the gate build I measured on is NOT the one I named, and CI sees MORE than I reported
Two errors in the measurement note. The FP modes and their counts stand; the provenance sentence does not, and the difference matters to anyone acting on the numbers.
1.
ad00986does not containgh_slugify. I measured on a build that does.I wrote "Gate script
ConductionNL/.github@ad00986— verified unchanged for gates 40 and 46 (git log ad00986..HEAD -- hydra-gates/scripts/run-hydra-gates.shis empty)."The command really was empty. It was empty because
git log A..B -- <path>applies history simplification and drops a change that arrives through a merge. HEAD in my checkout was a merge oforigin/main,gh_slugifycame in on that merge, and the simplified log hid it.--full-historyshows it immediately:Worth writing down on its own: a path-filtered
git logrange is not a safe way to prove a file did not change across a merge. It answers a different question than the one it looks like it answers, and it fails silently in the direction of "nothing changed" — the same shape as the dead gates this issue is about.git diff A B -- <path>would have been the honest check and takes the same keystrokes.2. No released tag contains the fix — so every repo's CI is on the pre-fix build
gh_slugifyis onmainonly. Checked every tag:gh_slugifyorigin/mainThis compounds #159. There the problem was that the fleet pins
v1.0.1, which predates the fixes; here repinning to the newest tag would not help either, because the fix has never been cut into a tag. A consumer-ref sweep alone does not deliver this one — it needs a release first.Verified end-to-end rather than inferred: running the v1.0.1 and v1.3.0 suites against the same docudesk branch, each in its own mount namespace with a private
tmpfs /tmp, produces byte-identical verdicts — same six gates, same counts, includinggate-46: FAIL — 1.3. What that does to my numbers
My 2,630 was measured with
gh_slugify. CI, on every repo, runs without it. Re-measured both ways on the sameorigin/developmenttrees:So the fleet total in my table is understated by at least ~73 against what CI actually reports, concentrated almost entirely in openconnector. Two cross-checks that the "no fix" column is the right one: this issue's own opening note gives procest as 180, and the openconnector figure recovers the "137 findings, none of them real" observation that motivated
gh_slugifyin the first place.(openconnector reads 42/109 rather than the 46 in my table because its
origin/developmentmoved between the two runs — another agent landed work. Not a measurement artefact, just a moving target.)4. What still stands
The four FP modes and their counts are unaffected, because I measured with
gh_slugify— my 1,479 is the residue that survives it. Concretely:Requirement: REQ-X — Title←#REQ-X, 351) is the prefix-vs-equality asymmetry in the:-tail branch.gh_slugifydoes not touch it.Requirement: Title (REQ-X)←#REQ-X, 944) is a trailing parenthesised id.gh_slugifydoes not touch it either.- [~]checkboxes, 28) and the two gate-40 modes are unrelated to slugging.So #165 landing on main is a real fix for a real mode — and it leaves the 1,479 above exactly where they were. Both still need doing, and the release needs cutting before either reaches CI.
rubenvdlinde commentedon Aug 5, 2026
gate-8
unsafe-auth-resolverhas the same blind spot as gate-7, one layer down: it never reads the CALLERAdding to this issue rather than opening a duplicate, because it is item 1's defect wearing a different gate number.
Measured on hrmq (whole tree,
.github@f7eaf2a, private mount namespace + private tmpfs): 13 gate-8 findings, and all 13 are false positives. Every one isrule=throwable-caught-returns-nullin anauthorize*method:Why they are all false
The gate flags the
catch (\Throwable) { return null; }and stops there. It never reads what the caller does with the null — which is the only thing that decides whether the shape is fail-open or fail-closed.Here, every caller is fail-closed:
And the null is deliberate. From the method's own docblock:
That is the multi-tenant tenancy guard already described in item 1 of this issue for gate-7: refusing with "not found" so a 403 cannot be used to enumerate other tenants' object ids. gate-8 flags exactly the code that got this right.
It is also the opposite of the decidesk#45 defect gate-8 was built for, which was fail-open in the caller:
The resolver body is identical in both. Only the caller differs, and the caller is what the gate does not look at.
Suggested fix
Make the finding conditional on caller-side handling, which is the same evidence-based move #149 made for gate-7's Pattern 4a — parse the collaborator rather than pattern-match the name:
private, so that is the complete set).if ($x === null) { return <4xx>; },?? throw,if (!$x) return, or an early-return equivalent.if ($x !== null) { $x->require...() }) — the real fail-open.That reduces gate-8 on hrmq from 13 to 0 without weakening it: the decidesk#45 shape still trips, because its call site is the truthy branch.
Scale note
This matters for the fleet totals in this issue. gate-8 was previously recorded as "0 real" — that is consistent with this, and now has a mechanism rather than a tally. Any repo using OpenRegister's RBAC through a private
authorize*helper will show the same pattern, because collapsing deny and not-found into one 404 is the documented multi-tenant idiom (see also #160 item 1).Measured with the runner in a private mount namespace and a private tmpfs throughout — the hardcoded
/tmp/hydra-gate-*.logpaths are shared state, and another agent was running the gates unisolated on the same host during this work.rubenvdlinde commentedon Aug 5, 2026
Fixed in #175 — plus two defect shapes this issue did not name
Item 6 (the shared
/tmppaths) and item 2 (gate-9's dangerous advice) are closed. Items 1, 3, 4, 5 are not — see the bottom.Baseline for everything below is v1.4.0 (
46cc1c8), not the numbers in the opening comment: those predategh_slugify(#165), which lives in v1.4.0 and in no earlier tag. Measured across 21 fleet repos atorigin/development.Item 6 — the shared
/tmppaths. Confirmed, and it is worse than "numbers move"You suspected the collision. Here it is, two concurrent runs on different repos at v1.4.0:
petstore's 26 findings were truncated away by app-versions between the write and the
wc -l. The ordering is the only thing that stopped this being a falsely-GREEN gate rather than a report pointing at an empty file. Had app-versions truncated a moment earlier, petstore would have printedPASSover 26 real findings.61 paths now hang off one private directory per invocation, printed on the first line,
TMPDIRhonoured. If the directory cannot be created the run refuses (exit 97) rather than falling back to the shared path — an unverifiable scope must not look like a clean one.The same defect was in the package's own test suite, which is how it stayed invisible:
test_gate_route_auth.shreported 7 failures underrun-helper-suites.shand 0 standalone. Its_run()even carried a comment acknowledging that a concurrent run "would truncate" the logs, with a snapshot taken after the run as the mitigation — which cannot help, because the truncation happens during it. Fixed there, intest_check_manifest.sh, and in the harness itself.Same test on #175: two private directories holding exactly what their own runs reported (26/0 and 0/1). Three concurrent harness runs: 0 failures each.
Item 2 — gate-9. 34 of 45 cleared, and the advice is rewritten
Your reading was right in every particular. Returning 401/403 is not, on its own, evidence of a session dependency — it is what a correctly self-authenticating public endpoint does when the credential in the request is missing or revoked. The rule now splits:
public-page-annotation-with-session-auth-body— the body tests the session (requireAdmin(),IUserSession) under#[PublicPage]. This is the decidesk defect the rule was built for. Still fires. A session test fires even alongside a token read, because self-authentication excuses an unsourced denial, never a session dependency.public-page-annotation-with-unsourced-denial— 401/403 with nothing resolving a request-borne credential.45 → 11 fleet-wide. The advice no longer contains "remove
#[PublicPage]" or "remove body auth check" in any finding;RemediationTextIsSafeasserts that on the string itself, because the advice is part of the gate — a precise gate that tells you to open the endpoint is still a vulnerability generator.Fixtures are your call shapes: openconnector's HMAC webhook, portaliq's portal
subject()inbox, openregister's bearer share token.Two shapes this issue did not name
gate-40's remediation was an accessibility regression. 463 of its 1,211 findings were
<NcCheckboxRadioSwitch>elements naming themselves from their default slot. The only way to clear them wasaria-label— which overrides the visible label. Two burn-down PRs (opencatalogi#808, docudesk#385) are stuck on exactly that. 1,211 → 517.gate-46 was 54% false. 1,995 → 918. The largest single cause: a requirement id in trailing parentheses —
### Requirement: Payment Provider Adapter Interface (REQ-PAY-001)rejecting#REQ-PAY-001. Lifted id tokens match by equality only, so#REQand#REQ-PAYstill do not resolve.And a third vacuous-PASS shape, which is the one I would read first: shillinq's
developmentrun finished in 22 seconds, all green. Not a shallow checkout — on a push to a mainline branchorigin/developmentISHEAD, so the diff is empty by construction and every gate passes over nothing. Verified atc64e9fe: 52 gates PASS scoped, 18 FAIL unscoped. Now exit 99.Not fixed here
skip-link/CnAppRoot(21 of 21)bool-returning predicate and the no-object-identifier shapes are untouched.Every relaxation in #175 ships with the true-positive case it must not swallow, in four new auto-discovered suites; each relaxed predicate was mutation-checked in both directions (always-true and always-false both fail the suite). Nothing was suppressed or baselined.
Tag v1.5.0 is ready to cut and has not been rolled — the new refusals mean it must not arrive as a patch.
rubenvdlinde commentedon Aug 6, 2026
v1.5.0 is cut and ready to roll — plus three more defect shapes this issue did not name
Following #175, PR #176 landed four more.
v1.5.0=82fa1c5, containing both. Not rolled — see the warning at the bottom.Since the last comment
A third⚠️ the epilogue's prose also appears in comments in both files — grep the output, or
SCOPE WAS EMPTYshape, and this one pointed the wrong way.bin/hydra-gatesdecided "was the scope empty?" withgrep -q "0 changed file(s)"— a substring match.10 changed file(s)contains it, and so do 20, 30, 100. Runs whose header said ten files changed, with every gate running, also printed the epilogue. Since that string had been adopted fleet-wide as the tell for a vacuous run, the bug manufactured doubt about runs that were fine while saying nothing new about runs that were not. Both statements now derive from one emitted integer,SCOPE-FILE-COUNT. (SCOPE-FILE-COUNT: 0.)gate-19 accepted a switched-off test as coverage. decidesk carries four tests with empty bodies and a hardcoded
test.skip(true, …), each tagged@e2e, each counted, together asserting nothing. Now dead:.skip/.fixme/xit,describe.skip, an unconditionaltest.skip(true), and an empty body. Still live:test.skip(browserName === 'firefox', …)— the discriminator is the argument, not the call.gate-47 classified on the whole file. A PR whose hunks were CSS tokens and a chevron column was told to add a CSRF test; a comment-only PR was told to co-change tests. Neither used the opt-out — the tell. Classifies on
git diff -U0hunks now, with a comment line counting only for an annotation (@NoAdminRequiredis the declaration in docblock form).gate-62's
lib/scan ignored diff scoping while its manifest half honoured it — the worst of both, since the gate only runs on a manifest change and then judged untouched code. One pipelinq violation blocked every manifest-touching PR there permanently.Version-normalised counts
None of these were taken from CI, because the fleet runs two gate versions. Each number is a specific version of the gate's own code run over the same 22 trees:
Status of this issue's six items
skip-link/CnAppRoot(21 of 21)bool-returning predicates and no-object endpoints untouched/tmppathsv1.5.0will turn currently-green runs red, deliberately: a--scope-to-diffrun whose base isHEADnow exits 99 instead of passing 52 gates over nothing. Every push-to-mainline scoped run in the fleet has that shape. Those runs are measuring nothing — but this is a roll to plan, not a rubber stamp.Six new test suites, all auto-discovered. Every relaxed predicate mutation-checked in both directions. Nothing suppressed or baselined.