Repository navigation
Description
gate-44 and gate-52 print PASS over an unopened scope — #361's shape in two more gates
Found while proving gates 44–64 can fail, using repo-shaped git fixtures driven through the real bin/hydra-gates, with every plant committed and a prediction written before each arm.
gate-44 (autocomplete-attr) — run-hydra-gates.sh:5945 falls through to _pass 44 on an empty file set. Same tree, only the scope input changed:
| scope | gate-44 |
|---|---|
| unscoped | FAIL — 1 |
| diff-scoped, empty diff | PASS |
Its immediate neighbour gate-45 reads the same enumerator and correctly reports na on the same input — so the correct behaviour is already implemented one gate away.
gate-52 (custom-widget-ratchet) has the same fall-through, and additionally reproduces #352 exactly: base=0 head=3 delta=+3 when the merge base predates the app's own registry script.
Why this is worse than a hidden number
This is the distinction drawn in #361: a NOT APPLICABLE is excluded from the verdict, but a PASS counts toward "N of N applicable gates ran". It does not hide a finding — it falsifies the coverage line, which is the one line this programme relies on to tell an honest green from a vacuous one. Every COVERAGE: N of 64 quoted while these two gates were in scope is overstated by up to two.
The fix that shipped for gate-16 in #364 applies unchanged: always run the checker; evaluate WIRING before SCOPE. A crash must never be reported as an empty scope, and an empty scope must never be reported as a pass.
Crash behaviour, measured separately
The package's own crash suite passes but reaches only gates 56–64. Running the missing half with a python3 shim exiting 1: gates 44, 45, 46, 49, 50, 51, 52, 54, 55 all reported SKIPPED (wiring) — none PASS, none FAIL. So the wiring half is sound in both gates; it is specifically the empty-scope path that is wrong.
Related, from the same sweep
- gate-60's entire Tier B half is unreachable code. All 10 values in
CONCEPT_LABELSare Tier A keys, whilesemantic-icons.jsonships 137 Tier B concepts, so thewarns.append(… Tier B — SHOULD)branch can never execute. A green gate-60 says nothing about 137 of 147 concepts. - gate-49 is silenced by a file-header
@throws \Throwable.METHOD_RE's optional non-greedy docblock group binds the file header, so the first method in the file is exempted. Positive-controlled: plain plantFAIL; header@throwsadded ⇒PASS; a second unhandled method stillFAIL. Same "tag position decides" family as gate-19: a file-level @e2e tag is counted as live coverage without checking any test — _ref_is_live returns True when no declaration owns it #343. - gates 60 and 62 disagree about whether a
type:"caption"menu node is a menu entry — 60 exempts it, 62 fails it twice. Undeclared divergence. - Exemption lists are now the load-bearing defect three times over: gate-60 (
CONCEPT_LABELS), gate-57 (WRITE_VERB_PREFIXES— 1 of 4 orphaned write methods seen), gate-50 (accepted app-id shapes — PASS over two unguarded reads of the exact opencatalogi#86 defect, written with a variable app id).
Activity
rubenvdlinde commentedon Aug 12, 2026
Expanded: this is 13 gates, not 2 — measured independently in a second sweep
A parallel proof of gates 22–43, run through the real wrapper (never the checkers directly) on a committed repo-shaped fixture, found the same defect in eleven more gates.
Arm: a docs-only diff over a planted tree, with all twelve planted defects still on disk.
PASSover an unopened scopeNOT APPLICABLEBanner read
23 of 23 applicable gates ran.Together with gate-44 and gate-52 from the 44–64 sweep, 13 of 64 gates can print
PASSover a scope they never opened, and — unlikeNOT APPLICABLE, which is excluded from the tally — every one of those PASSes is counted as an applicable gate that ran. So aCOVERAGE: N of 64line can be overstated by up to 13, and the overstatement is invisible in exactly the runs where the diff is narrow.The fix already exists in-tree, for four gates
test_gate_empty_scope_never_passes.shalready enforces this property — for gates 19, 25, 62 and 63 only. Extending that test to the full set is the smallest correct fix, and it makes the acceptance suite the enforcement point rather than a convention.Gates 26, 31 and 32 are the model for the runtime behaviour: each emits its own gate-level
nawith a reason, on the same input where the eleven above pass.gate-40's inline justification is refuted by the same run that reads it
gate-40 carries a note explaining why it passes rather than reporting
na. Both of its claims are false, and both are refuted by the run in which it makes them:nareason.test-hydra-gates-bin.sh:465requires it" — that test matches the applicability table's wording, which a gate-levelnapasses unharmed.The stated blocker does not exist. This is the third instance in this programme of an exemption's stated reason being the load-bearing defect rather than the detection logic — after the accessibility selectors excused "for older Nextcloud" that were dead on the oldest supported server, and gate-7's blanket
@PublicPageexemption.A second, separate defect: three gates make FALSE CLAIMS ABOUT THE REPOSITORY when their checker crashes
Crashed-checker arm (a
python3shim exiting 1). Gates 34, 37, 38, 39, 40, 41, 42, 43 and 44–55 all reported an honestSKIPPED (wiring). But:@licensedeclaration" — when both files in the fixture dogate-28's own header comment was written to prevent exactly this class of false claim; it guards against a missing helper but not a crashing one. A crashed checker that invents a finding is worse than one that invents a pass: it sends someone to fix a defect that does not exist, and when they find nothing they learn to distrust the gate.
Also confirmed live, one file and one run: #363 in both directions
gate-25's fixed 20-line auth lookback simultaneously:
adminOnlyPurge— it reads the previous method's#[PublicPage]), andfarAttribute, whose attribute sits 24 lines up).The silent half is the dangerous one.
Corrections to earlier claims in this repo's issues
#309landed. Proven with the component placed outsidesrc/views/so only the manifest half could classify it. The prose defect (#358) is still live: a//comment saying you still owe a baseline yieldsPASS — all have a visual proof. It also FAILs a page that a runtime-generated spec genuinely covers.nextcloud-vuecheckout and inspects 27 real descriptors (positive-controlled — that green sentence reads identically over zero). What is blocked is distribution: the npm package omitsscripts/, and 14 of 18 apps ship no wrapper.git check-ignore— deliberately, with an inline warning against a future edit that would reintroduce it (check-ignore -qexits 0 when a negation matches).<noun>Url— the\b-anchored noun list cannot matchavatarUrl/thumbnailUrl/photoUrl(4 of 7 in a controlled probe). Live fleet exposure is zero, positive-controlled: 5<img>tags fleet-wide.tabindex="5"inside a comment — the last raw grep in the 34–43 family.NOT APPLICABLEthere means accessibility is unverified, not clean.mainand returned "absent" for all eighteen —mainis stale (379 vs 15,054 bytes); caught by a byte-count control.App.vueexempted by shell, 10/10*Root.vueexempted, 40/40 templates classified fragment. They work; they have never had anything to look at.rubenvdlinde commentedon Aug 12, 2026
Final count: 16 gates, from three independent sweeps covering all 64
The 1–21 sweep completes the picture. Every gate in the package has now been driven through the real wrapper against committed, repo-shaped planted / clean / near-miss fixtures, with a prediction written before each arm.
Gates that print
PASSover a scope they never opened:16 of 64.
NOT APPLICABLEis excluded from the verdict; each of these PASSes is counted as an applicable gate that ran. SoCOVERAGE: N of 64can be overstated by up to 16 — and it is overstated precisely when the diff is narrow, which is the common case on a push run.test_gate_empty_scope_never_passes.shalready enforces the property, for gates 12, 13, 19, 25, 33, 62 and 63 only — 7 of 64. Extending it across the package is the smallest correct fix, and it puts enforcement in the acceptance suite rather than in convention. Gates 26, 31 and 32 model the correct runtime behaviour: a gate-levelnawith a reason, on the same input where the sixteen pass.A second scope defect, in a different mechanism: gate-17
run-hydra-gates.sh:238never absolutisesAPP_DIRbeforecd, and gate-17 is the only gate that hands${APP_DIR}to its checker afterwards. On a relative app-dir path it scans zero files →# count=0→ PASS. Same tree, absolute path → FAIL — 1.bin/hydra-gates:155absolutises, so CI is safe — but the documented human invocation is not.gate-7: both mechanisms reproduced independently, both predicted in advance
Byte-identical IDOR plant, one file, one run:
no user → 401preamble#[PublicPage]— an unauthenticated write to an arbitrary$idSources:
check_no_admin_idor.py:388(STATUS_UNAUTHORIZEDinside_GUARD_BODY_RE) and:1143— note, not:1141as reported earlier — the unconditional#[PublicPage]skip.Both live in the exemption path, not the detection path. The regex that finds IDORs is sound; the skip list is what blinds it. And the
#[PublicPage]exemption carries no reason at all, so there is nothing to test — which makes it the strongest case yet for the rule this programme has hit three times: an exemption's reason is a testable claim, and an exemption without one cannot be audited.Corrections to earlier claims in this thread
57bcb2bit fires on a plantedfindObjects(and is silent on the commented-out and wrong-receiver near-misses. A silent gate-20 today is evidence.--full" does not generalise into 1–21. Exactly one (gate-16) is diff-dependent, and fix(gate-16): an EMPTY SCOPE is not a PASS (.github#361) #364 made it honest (NOT APPLICABLE), not dead. fix(gate-16): an EMPTY SCOPE is not a PASS (.github#361) #364 and fix(gate-61): an empty scope has two causes and only one of them is a diff (.github#347) #371 both re-verified as merged and effective.^={7}$. It is also the only gate in 1–21 that enumerates withfindrather thangit ls-files, so an untracked plant is seen.private function exit(), a legal PHP semi-reserved method declaration, is flagged. Call sites and$exitCodeare correctly silent.@e2e tag::beta" flips FAIL → PASS. Its honest and counterfeit arms emit byte-identical verdicts.expect.confrow — the runner names its log viamktemp "…hydra-gate-orphan-auth.XXXXXX.log", so the basename is random and the driver's fixed-basename contract cannot address it.test_gate_crashed_checker_is_not_a_finding.sh(38 assertions, all passing) stalls for minutes on its gate-17 arm — two backgrounded runs returned exit 0 having printed three lines. Require the trailingALL PASSbefore believing it.Coverage delivered
Acceptance coverage went 13 → 30 of 64 gates;
55 passed / 0 failed, ratchet intact,UNCOVERED.mdshrunk. Two honest limits: gate-4 rots by construction (it depends on live Packagist advisories), and every "silent on near-miss" result is only as strong as the near-misses the checkers' own docstrings record.rubenvdlinde commentedon Aug 12, 2026
Correction: eighteen gates, not sixteen — and this list is still a floor
#378's author re-measured while implementing the fix and found two further instances:FAIL — 1, docs-only diffPASS.Complete set: 14, 17, 18, 20, 21, 22, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 52.
The signature worth reusing when hunting the rest: a findings counter standing in for an inspection counter. Where those two are conflated, an unopened scope and a clean scope produce the same number, and the gate cannot tell them apart. Fixed at the fall-through via one shared helper rather than eighteen separate edits.
gate-40's inline note claiming it could not report
nais deleted — both of its claims were false, and both were refuted by the run that read them.The root cause was one conflation, and it is the same one behind #347 and #361
File scope and the delta base were a single input. They are now two named ones:
SCOPE-MODE: full|diffcontrols the state gates.BASE_REFno longer carries a hardcodedorigin/development.This matters more than it sounds. The five genuine delta gates — 16, 29, 47, 48, 61 — key on whether a base resolved, not on file scope. Had they keyed on file scope, the whole-tree default would have silently retired all five fleet-wide. As built, a PR keeps every delta gate and gains whole-tree coverage.
Corrections to the delta list as I stated it
PASS.Full classification of all 64: 51 STATE · 6 RATCHET · 5 DELTA.
The clamp does not generalise — and one of the two flagged sites was real
_skip_empty_scopeanswers "nothing was opened." The clamp answers "the count could not be read." Those are opposite verdicts and must not share a helper.FAIL — 1 parity violation(s)from a crash — an app-owned wrapper with no marker contract. Its verdict deliberately staysFAIL; guessingwiringthere would open a green hole. It simply stops claiming a number nobody measured._helper_finishedbranch sits directly above, so a crash cannot reach it.The other five clamp sites were not audited and belong with
#379.Two regressions the package's own suites caught, and one of them indicts a test
Both are worth recording because they are the same defect class we are fixing, occurring inside the fix:
#364's first draft.#347invariant fired correctly — but its trigger was a proxy: it keyed on--fullwhen the property it defends is about whether a diff was computed. That is the same conflation this PR exists to remove, living inside the test written to catch it. Repairing the trigger exposed a real gap: gate-61's advisory backlog sweep ran only on the no-base path, which is no longer the common one.Not verified
Nothing was run against a real fleet app.
quality.ymlis untouched — the reversal reaches all 18 apps with no workflow edit, so the PR's own CI is the first real proof. Gates 23 and 24 shell out to scripts absent from this package. And gates 25/26 become much stricter under#376/#377, while their header comments still say "ADDED in this PR" — those are now false, and they belong to the checker owner.Merge main into fix/gates-2-21-35-36-noise — resolved in favour of bo…
fix(gates 15, 23, 47, 48, 57): a gate must not report a verdict about…
fix(gates 3, 14): a comment is not a route, and a TODO is not a use (#…