Skip to content

gate-46 reads only @spec: 1,964 @e2e anchors fleet-wide are checked by nothing, 194 of them dangling #726

Closed

Description

@rubenvdlinde

284 unchecked anchors is how six rotted for months in a repo that runs 49 checks. The rest is detail.

The gap

Gate 46, spec-anchor-existence, resolves @spec targets and nothing else. Its pattern, at
hydra-gates/scripts/lib/check_spec_anchors.py:102, is:

r'@spec\s+(openspec/[^\s`\'"]+)',

@e2e tags use the identical target grammar and carry the identical traceability meaning. No gate
reads them. There are 1,964 of them across the 21 fleet apps, and none has ever been resolved by CI.

What that hides, measured

Every @e2e target in each app was rewritten to @spec in a probe file outside the repo and run
through the real check_spec_anchors.py with cwd set to that app, so these are the gate's own
semantics, not a reimplementation.

app @e2e anchors unresolved
procest (dossiq) 286 46
zaakafhandelapp 84 43
pipelinq 298 36
decidesk 370 29
scholiq 202 12
openregister 168 10
hermiq 76 6
doriath 16 6
larpingapp 145 4
docudesk 160 2
nldesign, openbuild, shillinq, opencatalogi 159 0
launchpad, portaliq, openconnector, softwarecatalog, hrmq, app-versions, planix 0 0
total 1,964 194

For contrast, the same helper over the same apps' @spec tags in its real scope: 55,720 anchors, 1
unresolved.
The gate works. It just does not look at a fifth of the anchors.

Why this is worth a gate rather than a sweep

Two independent sessions in dossiq found the same six dangling @e2e anchors on the same day, by
hand, hours apart (ConductionNL/dossiq#2067 and #2116). Neither found them via CI, because CI does not
look. They named four capabilities (workflow-board, bezwaar-management, case-map,
subsidy-intake) that have never existed in any of the three homes a spec can occupy, and git log --all finds no commit that ever created one.

A worked example of the failure mode, from this week. dossiq#2057 renumbered two scenarios from
DASH-V1-006d/e to 006f/g to clear a name collision. Two @e2e anchors in
kanban-board-keyboard-status-transition.spec.ts still cited 006d and 006e. Every check passed.
The tests kept running and kept reporting green, while citing scenarios that no longer existed. Under
@spec that is a gate-46 failure the same afternoon.

Another, from openregister: tests/e2e/spec-coverage/mdm-survivorship-override.spec.ts cites

openspec/changes/mdm-survivorship-override/specs/<capability>/spec.md#

A literal <capability> placeholder from a template, never filled in, never noticed.

Most of the 194 are not missing files. They are "anchor not found": the spec resolves and the
#fragment names a heading nobody wrote. That is precisely the opencatalogi#85 shape this gate was
built for, surviving in the half of the corpus the gate does not read.

Suggested fix

Read @e2e with the same resolver and the same rules as @spec. The target grammar is already
identical, so this is the pattern plus a label in the finding message, not new resolution logic.

Ship warning-only first, and here is why the usual argument is weaker than normal

The house rule is that a new gate ships as a warning, and the reason is well earned: gates resolve at
@main and reach 21 repos the minute they merge.

This one is close to the line. Only 11 of 21 apps have any finding at all, and 7 have no @e2e
anchors whatsoever, so half the fleet cannot go red. But 194 findings concentrated in six repos is
still somebody's morning, and four of those repos would see 29 to 46 at once.

My view: warning-only, with a fleet-wide deadline rather than an open one. Blocking on day one
turns a reporting gap into a work-stoppage in exactly the six repos that carry the most e2e coverage,
which is a poor way to reward having written it. Warning-only makes the 194 visible and countable, and
the count is small enough to burn down deliberately. It is also the number I would use to decide when
to flip it: this is not inherited debt of the 1,184-finding kind, it is 194 items with a known shape.

One caveat on the numbers, stated plainly: they were taken from local working copies, several of which
sit on feature branches rather than development. The shape is right and the order of magnitude is
right; the exact per-repo figure will move a little on a clean checkout.

Reproducing

# from an app root, with hydra-gates checked out at $G
{ echo '<?php'; echo '/**'
  git ls-files lib src tests appinfo scripts \
    | grep -E '\.(php|js|ts|vue|cjs|mjs)$' \
    | xargs -r grep -ho '@e2e[[:space:]]\+openspec/[^[:space:]`"'"'"')]*' \
    | sed 's/@e2e/ * @spec/'
  echo '*/'; } > /tmp/probe.php
python3 "$G/scripts/lib/check_spec_anchors.py" /tmp/e2e.log /tmp/probe.php
wc -l < /tmp/e2e.log

The probe file lives outside the repo on purpose: the helper reads tags from the path given and
resolves openspec/ against cwd, so nothing is written into the tree being measured.

Activity

  1. added a commit that references this issue on Sep 9, 2026
  2. rubenvdlinde commented on Sep 9, 2026

    @rubenvdlinde
    ContributorAuthor

    Fixed in #733, shipped warning only, and the numbers in this issue are the reason.

    Four repos would go red on day one with no PR having introduced anything: dossiq 46, zaakafhandelapp 43, pipelinq 36, decidesk 29. That is 154 of the 194, and it lands on exactly the four repos with the most end to end coverage, which is the wrong incentive to publish on a Tuesday. Two gates were once merged blocking and failed 14 of 21 repos on inherited debt; this one would repeat it with a smaller blast radius and the same shape.

    So gate-46 keeps blocking on @spec, exactly as today, and reports @e2e as a warning that prints on every run. The gate has one verdict line either way:

    • any unresolved @spec finding, FAIL, as now, with the @e2e count stated in the same reason
    • unresolved @e2e findings only, WARNING
    • neither, PASS

    HYDRA_GATE_SPEC_ANCHOR_E2E_BLOCKING=1 flips one repo at a time once it is worked down. Gate 46 is added to _ADVISORY_ALLOWED_GATES in test_gate_discarded_counts_and_empty_deltas.sh with this issue named, so the demotion is bounded and a second one cannot land silently.

    The probe technique in this issue is what the PR leans on and it is worth naming. Rewriting every @e2e target to @spec in files outside the repo and running the real check_spec_anchors.py with cwd set to the app gives the gate's own semantics rather than a reimplementation of them, and it is mutation checkable: a planted bad anchor must appear, and the count must go to zero when the anchors are fixed. It also settles the two questions the fix depends on. Every @e2e line carrying an openspec/ path already matches the gate's tag regex, and no such tag lives outside tests/, which is already in scope. The pattern really is the only thing standing between the gate and those anchors.

    One thing this does not fix, filed separately as #732: a citation that resolves is not a citation that agrees. One @e2e anchor in the measured set resolves cleanly to a scenario saying the opposite of what its test asserts. Reading @e2e finds the 194 that point at nothing. It cannot find that one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions