From 183a3b28c433535590072bae2ac4b83e00ee410d Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Thu, 6 Aug 2026 08:01:50 +0200 Subject: [PATCH 1/2] =?UTF-8?q?fix(gate-28):=20an=20EMPTY=20DIFF=20SCOPE?= =?UTF-8?q?=20is=20not=20a=20structural=20gap=20=E2=80=94=20#172=20swung?= =?UTF-8?q?=20one=20state=20too=20far?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #172 was right that gate-28 must not report PASS having opened zero files. But there are TWO ways to read zero files and it gave them the same word: (a) lib/**/*.php files ARE in this diff and none carries a licence tag — a genuine gap in the repository, correctly `structural`; (b) no lib/**/*.php file is in this diff AT ALL — diff-scoping working exactly as ADR-020 designed it, and no statement about the repo. (b) is the ordinary case. `_in_scope` filters to the PR's diff, so every workflow-only and frontend-only PR reads zero lib PHP files. Calling that `structural` counts it against coverage, and with hydra-gates-require-full-coverage on by default the run fails with exit 98 for a licence problem the repository does not have. MEASURED on the fleet's own unpinning PRs, each a single-file workflow diff: hrmq#74 CLEAN baseline -> RED on nothing but gate-28 app-versions#129 gate 28 its only gate that did not run opencatalogi#813 " nextcloud-app-template#132 " launchpad#60 " hrmq's 168 lib PHP files all carry their tags. The gate had simply not been handed any of them to read, and then reported that as the repository's fault. This is the "a gate that is legitimately not applicable must NOT fail the run" rule broken — the same shape as #173, one gate lower down. It surfaced now because the fleet just unpinned (#177): at v1.3.0 gate-28 reported PASS on an empty read, so the false GREEN hid what is now a false RED. Neither was a verdict. The `structural` branch is preserved for case (a) and now names the count of files that WERE in scope, so the two states no longer read alike. TESTS — scripts/lib/test_gate_license_triangle_scope.sh, four control pairs, discovered automatically by tests/run-helper-suites.sh: wf-only diff, tagged repo -> NOT APPLICABLE (b, the false red) untagged PHP in the diff -> structural (a, must stay red) tagged PHP in the diff -> PASS (the gate compared something) no lib/ at all -> NOT APPLICABLE (#172's original case) Verified in BOTH directions rather than assumed: against the fixed runner 5/5 green; against the runner at 954a78a the first two FAIL and the last three still pass, so the suite discriminates and the fix is targeted rather than a mute for #172. The did-not-run assertion took two attempts to measure anything — v1 matched only a summary line this suite never triggers, v2 matched a roster the runner also prints for NOT-APPLICABLE gates. Both are recorded in the file, because a dead assertion inside a suite meant to tell two states apart is the exact defect the suite exists to catch. --- .../lib/test_gate_license_triangle_scope.sh | 226 ++++++++++++++++++ hydra-gates/scripts/run-hydra-gates.sh | 36 ++- 2 files changed, 260 insertions(+), 2 deletions(-) create mode 100755 hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh diff --git a/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh b/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh new file mode 100755 index 000000000..85d7cd71d --- /dev/null +++ b/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh @@ -0,0 +1,226 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: EUPL-1.2 +# +# test_gate_license_triangle_scope.sh — control-pair suite for gate-28's +# THREE-WAY distinction between "diff-scoped out", "a real gap", and "clean". +# +# WHY THIS EXISTS +# --------------- +# ConductionNL/.github#172 fixed a real defect: gate-28 reported PASS on repos +# where it had opened zero files, so a green said nothing. The fix made an +# empty read report `structural` instead — a claim that the REPOSITORY is +# missing licence declarations. +# +# But there are two ways to read zero files, and #172 gave them the same word: +# +# (a) lib/**/*.php files exist and ARE in this diff, and none carries a tag +# — a genuine gap in the repository, correctly `structural`; +# (b) no lib/**/*.php file is in this diff AT ALL — diff-scoping working +# exactly as ADR-020 designed it, and NOT a statement about the repo. +# +# (b) is the ordinary case: every workflow-only and frontend-only PR hits it. +# With `hydra-gates-require-full-coverage` on by default, calling it +# `structural` fails the run with exit 98 for a licence problem that does not +# exist. Measured on the fleet's own unpinning PRs — each a single-file +# workflow diff — hrmq#74 went from a CLEAN baseline to red on nothing but +# this, and app-versions#129, opencatalogi#813 and nextcloud-app-template#132 +# named gate 28 as their only gate that did not run. hrmq's 168 lib PHP files +# all carry their tags; the gate had simply not been given any of them to read. +# +# THE RISK THIS SUITE GUARDS +# -------------------------- +# The obvious "fix" — make an empty read NOT APPLICABLE — walks straight back +# into #172, because case (a) also reads zero files. A gate that stops failing +# is worse than the false positive it replaced. So every assertion here is one +# half of a control pair: for each state that must NOT fail the coverage +# verdict, there is a neighbouring fixture where the same code path MUST. +# +# wf-only diff, tagged repo -> NOT APPLICABLE (b: diff-scoped out) +# untagged PHP in the diff -> structural (a: the real gap, still red) +# tagged PHP in the diff -> PASS (the gate actually compared) +# no lib/ at all -> NOT APPLICABLE (#172's original case) +# +# Run: bash scripts/lib/test_gate_license_triangle_scope.sh (exit 0 = green) +set -uo pipefail + +PKG_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")/../.." && pwd)" +RUNNER="${PKG_ROOT}/scripts/run-hydra-gates.sh" + +_fail_n=0 +_pass_n=0 +_ok() { _pass_n=$((_pass_n + 1)); printf 'PASS — %s\n' "$1"; } +_bad() { _fail_n=$((_fail_n + 1)); printf 'FAIL — %s\n' "$1"; } + +_TMP="$(mktemp -d "${TMPDIR:-/tmp}/gate28-scope.XXXXXXXX")" +trap 'rm -rf "${_TMP}"' EXIT + +_LICENSED_PHP=' — a committed base tree. Echoes the base SHA. +_REPO="" +_BASE="" +_mkrepo() { + local _name="$1" _withlib="$2" + _REPO="${_TMP}/${_name}" + mkdir -p "${_REPO}/appinfo" "${_REPO}/.github/workflows" + ( + cd "${_REPO}" || exit 1 + git init -q . + git symbolic-ref HEAD refs/heads/development + git config user.email "ci@example.invalid" + git config user.name "gate28 test" + printf '\nfixture1.0.0\n' > appinfo/info.xml + printf '{\n "name": "conduction/fixture",\n "license": "EUPL-1.2"\n}\n' > composer.json + printf 'name: ci\non: push\njobs:\n a:\n runs-on: ubuntu-latest\n steps:\n - run: echo base\n' > .github/workflows/ci.yml + if [ "${_withlib}" = "yes" ]; then + mkdir -p lib + printf '%s' "${_LICENSED_PHP}" > lib/Tagged.php + fi + git add -A + git commit -qm "base" + ) >/dev/null 2>&1 + _BASE="$(git -C "${_REPO}" rev-parse HEAD)" +} + +# _run — capture the run. A run that aborts before the COVERAGE line is not a +# result, however many PASS lines precede it. +_OUT="" +_run() { + local _logdir + _logdir="$(mktemp -d "${TMPDIR:-/tmp}/hydra-gate-test.XXXXXXXX")" + _OUT="$(HYDRA_GATE_LOG_DIR="${_logdir}" bash "${RUNNER}" \ + --app-dir "${_REPO}" --base "${_BASE}" --scope-to-diff 2>&1 || true)" + rm -rf "${_logdir}" + if ! printf '%s' "${_OUT}" | grep -q '^\[hydra-gates\] COVERAGE:'; then + _bad "run in ${_REPO} ABORTED before the summary — not a result" + printf '%s\n' "${_OUT}" | tail -15 | sed 's/^/ /' + return 1 + fi + return 0 +} + +# _expect +_expect() { + local _want="$1" _desc="$2" _line + _line="$(printf '%s' "${_OUT}" | grep -E '^\[gate-28\] ' | head -1)" + if [ -z "${_line}" ]; then + _bad "${_desc} — gate-28 emitted NO line at all (a silent gate is the #147 shape)" + return + fi + case "${_want}" in + PASS) + printf '%s' "${_line}" | grep -qE ': PASS' \ + && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; + NOTAPPLICABLE) + printf '%s' "${_line}" | grep -qE ': NOT APPLICABLE' \ + && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; + structural) + printf '%s' "${_line}" | grep -qE ': SKIPPED \(structural\)' \ + && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; + esac +} + +# --- 1. (b) THE FALSE RED. Repo HAS licensed lib PHP; the diff touches only a +# workflow file. Nothing was withheld — there was nothing to read. +_mkrepo scope-wf-only yes +printf 'name: ci\non: push\njobs:\n a:\n runs-on: ubuntu-latest\n steps:\n - run: echo changed\n' > "${_REPO}/.github/workflows/ci.yml" +git -C "${_REPO}" add -A >/dev/null 2>&1 +git -C "${_REPO}" commit -qm "workflow-only change" >/dev/null 2>&1 +if _run; then + _expect NOTAPPLICABLE "workflow-only diff in a fully-tagged repo: diff-scoped out, not a gap" + # And it must NOT be counted against coverage, or the taxonomy is cosmetic. + # + # THIS ASSERTION TOOK TWO TRIES TO MEASURE ANYTHING, and both failures are + # worth naming because they are the same mistake in opposite directions: + # + # v1 matched only `GATES THAT DID NOT RUN: 28`, a summary line the + # runner emits only under --require-full-coverage. This suite does + # not pass that flag, so the pattern matched nothing and the + # assertion passed against the FIXED and the UNFIXED runner alike — + # a dead assertion inside the suite whose entire purpose is to tell + # those two apart. + # v2 also matched the indented `[hydra-gates] gate-28 ` roster + # — but the runner prints that roster for NOT-APPLICABLE gates too. + # So it fired on the correct behaviour and the assertion failed + # against the FIXED runner. + # + # The roster line is ambiguous on its own; only its HEADING disambiguates. + # So read the did-not-run block specifically: from its heading up to the + # next `[hydra-gates] ` line that starts a new section. + _didnotrun_block="$(printf '%s\n' "${_OUT}" \ + | sed -n '/^\[hydra-gates\] GATES THAT DID NOT RUN/,/^\[hydra-gates\] [A-Z].*[a-z]/p')" + if printf '%s' "${_didnotrun_block}" | grep -qE '(^|[[:space:]])gate-28([[:space:]]|$)|DID NOT RUN:.*\b28\b'; then + _bad "gate-28 was named in GATES THAT DID NOT RUN despite being NOT APPLICABLE — it will still fail the run under --require-full-coverage" + else + _ok "gate-28 excluded from the did-not-run tally" + fi +fi + +# --- 2. (a) THE CONTROL. Untagged lib PHP IS in the diff. This is a real gap +# and MUST stay red, or this whole change is a mute for #172. +_mkrepo scope-untagged yes +printf '%s' "${_UNLICENSED_PHP}" > "${_REPO}/lib/Untagged.php" +rm -f "${_REPO}/lib/Tagged.php" +git -C "${_REPO}" add -A >/dev/null 2>&1 +git -C "${_REPO}" commit -qm "add untagged lib PHP" >/dev/null 2>&1 +if _run; then + _expect structural "untagged lib PHP IN scope: still a structural gap (#172 preserved)" +fi + +# --- 3. THE CLEAN PATH. Tagged lib PHP in the diff — the gate compared something. +_mkrepo scope-tagged yes +printf '%s' "${_LICENSED_PHP}" > "${_REPO}/lib/Second.php" +git -C "${_REPO}" add -A >/dev/null 2>&1 +git -C "${_REPO}" commit -qm "add tagged lib PHP" >/dev/null 2>&1 +if _run; then + _expect PASS "tagged lib PHP in scope: gate-28 actually compared and passed" +fi + +# --- 4. #172's ORIGINAL CASE. No lib/ at all — must remain NOT APPLICABLE and +# must never be PASS, which is the bug #172 was opened for. +_mkrepo scope-no-lib no +printf 'name: ci\non: push\njobs:\n a:\n runs-on: ubuntu-latest\n steps:\n - run: echo changed\n' > "${_REPO}/.github/workflows/ci.yml" +git -C "${_REPO}" add -A >/dev/null 2>&1 +git -C "${_REPO}" commit -qm "workflow change, no lib/" >/dev/null 2>&1 +if _run; then + _expect NOTAPPLICABLE "no lib/ at all: NOT APPLICABLE, never PASS (#172)" +fi + +echo +echo "== summary ==" +echo " passed: ${_pass_n}" +echo " failed: ${_fail_n}" +if [ "${_pass_n}" -eq 0 ]; then + echo "FAIL — zero assertions ran; an empty suite is not a green suite" + exit 1 +fi +[ "${_fail_n}" -eq 0 ] || exit 1 +echo "ALL gate-28 scope control pairs PASSED" +exit 0 diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index e4dfc2bb9..93e108018 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -2346,6 +2346,27 @@ fi # distinguishes them deliberately. Collapsing them into one `na` here would # undo #172. _lt_ran=1 +# Was the DIFF SCOPE empty, as distinct from "files were in scope and none of +# them carried a tag"? #172 collapsed those two, and the collapse is a false +# RED on the most ordinary PR there is. +# +# `_in_scope` filters to this PR's diff (ADR-020). So a PR that touches only +# .github/workflows/, or only src/, has ZERO lib/**/*.php in scope — not +# because the repository is missing licence tags, but because diff-scoping is +# working exactly as designed. #172 reported that as `structural`, which is a +# claim about the REPOSITORY, and with hydra-gates-require-full-coverage on by +# default it fails the run with exit 98. +# +# MEASURED on the fleet's own unpinning PRs, each a single-file workflow diff: +# hrmq#74 went from a CLEAN baseline to red on nothing but this, and +# app-versions#129, opencatalogi#813, nextcloud-app-template#132 and +# launchpad#60 all named gate 28 as their only gate that did not run. +# +# That is the "a gate that is legitimately not applicable must NOT fail the +# run" rule, broken — the same shape as #173, one gate lower down. #172 was +# right that an empty read must not report PASS; it just swung past +# NOT APPLICABLE on the way to structural. +_lt_scope_empty=0 _lt_helper="${SCRIPT_DIR}/lib/check_license_triangle.py" if [ -n "${_composer_lic}" ] && [ -d lib ]; then _lt_files=() @@ -2355,7 +2376,12 @@ if [ -n "${_composer_lic}" ] && [ -d lib ]; then _lt_files+=("${_php}") done < <(_enum_tracked '\.php$' lib) if [ "${#_lt_files[@]}" -eq 0 ]; then - : # falls through to the `structural` branch below — nothing compared. + # Distinguish the two ways to arrive here. If the repo HAS lib PHP but + # none of it is in this diff, the gate is diff-scoped out and that is + # NOT APPLICABLE. If the repo has no lib PHP at all, it is still not + # this gate's subject matter. Either way nothing was withheld: there + # was nothing to read. + _lt_scope_empty=1 elif [ ! -f "${_lt_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). This is the one state # #172's chain would misread: with no helper, `_lt_checked` stays 0 and @@ -2394,8 +2420,14 @@ elif [ ! -f composer.json ]; then _skip 28 "license-triangle" na "lib/ exists but this repo has no composer.json, so there is no \`license\` declaration for per-file @license tags to be compared against. This gate compares two declarations and only one exists here." elif [ -z "${_composer_lic}" ]; then _skip 28 "license-triangle" structural "lib/ and composer.json both exist, but composer.json declares no \`license\` field, so there is nothing to compare per-file @license tags against. Per-file/composer license agreement is UNVERIFIED by this run. Fixable here: add \`\"license\": \"EUPL-1.2\"\` to composer.json." +elif [ "${_lt_scope_empty}" -eq 1 ]; then + # DIFF-SCOPED OUT — not a gap. No lib/**/*.php file is in this PR's scope, + # so there was nothing for this gate to read. Reporting `structural` here + # makes every workflow-only and frontend-only PR red for a licence problem + # the repository does not have. + _skip 28 "license-triangle" na "no lib/**/*.php file is in this diff's scope, so there was nothing to compare composer.json's license=${_composer_lic} against. Diff-scoped out per ADR-020 — this says nothing about the repository's licence tags, only that this PR did not touch the code they live in. The gate becomes applicable again the moment a PR changes a file under lib/." else - _skip 28 "license-triangle" structural "lib/ exists and composer.json declares license=${_composer_lic}, but 0 in-scope lib/**/*.php file carried an @license or SPDX-License-Identifier declaration, so NOTHING was compared. Per-file/composer license agreement is UNVERIFIED by this run. (Presence of the tag is gate-1's job, not this gate's.)" + _skip 28 "license-triangle" structural "lib/ exists and composer.json declares license=${_composer_lic}, and ${#_lt_files[@]} lib/**/*.php file(s) WERE in scope, but none carried an @license or SPDX-License-Identifier declaration, so NOTHING was compared. Per-file/composer license agreement is UNVERIFIED by this run. (Presence of the tag is gate-1's job, not this gate's.)" fi # --------------------------------------------------------------------------- From 4dc8ada3d869bb556a5f094e581fea98f86c3cb3 Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Thu, 6 Aug 2026 08:13:47 +0200 Subject: [PATCH 2/2] =?UTF-8?q?fix(test):=20explicit=20if/else=20in=20=5Fe?= =?UTF-8?q?xpect=20=E2=80=94=20`A=20&&=20=5Fok=20||=20=5Fbad`=20can=20coun?= =?UTF-8?q?t=20one=20assertion=20twice?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ShellCheck SC2015 on the new suite, and it is not a style note here. In `grep -q ... && _ok "$d" || _bad "$d"`, a non-zero return from _ok makes _bad run as well, so the SAME assertion is counted once as a pass and once as a failure. In a harness whose entire job is to count assertions correctly, that is instrument corruption — the class this suite exists to catch, one level up. Also adds a `*)` arm: an unknown expectation string now fails loudly instead of matching no case and silently asserting nothing. Re-verified in both directions after the change: fixed runner 5 passed / 0 failed, runner at 954a78a 3 passed / 2 failed, entry-point invariants 39/0. --- .../lib/test_gate_license_triangle_scope.sh | 26 ++++++++++++------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh b/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh index 85d7cd71d..27adbe31c 100755 --- a/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh +++ b/hydra-gates/scripts/lib/test_gate_license_triangle_scope.sh @@ -134,17 +134,25 @@ _expect() { _bad "${_desc} — gate-28 emitted NO line at all (a silent gate is the #147 shape)" return fi + # `A && _ok || _bad` would be wrong here, and not merely untidy (SC2015): + # if _ok ever returned non-zero, _bad would ALSO run and the same + # assertion would be counted twice, once each way. In a harness whose + # entire job is to count assertions correctly, that is instrument + # corruption. Explicit if/else. + local _pattern case "${_want}" in - PASS) - printf '%s' "${_line}" | grep -qE ': PASS' \ - && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; - NOTAPPLICABLE) - printf '%s' "${_line}" | grep -qE ': NOT APPLICABLE' \ - && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; - structural) - printf '%s' "${_line}" | grep -qE ': SKIPPED \(structural\)' \ - && _ok "${_desc}" || _bad "${_desc} — got: ${_line}" ;; + PASS) _pattern=': PASS' ;; + NOTAPPLICABLE) _pattern=': NOT APPLICABLE' ;; + structural) _pattern=': SKIPPED \(structural\)' ;; + *) + _bad "${_desc} — test bug: unknown expectation '${_want}'" + return ;; esac + if printf '%s' "${_line}" | grep -qE "${_pattern}"; then + _ok "${_desc}" + else + _bad "${_desc} — got: ${_line}" + fi } # --- 1. (b) THE FALSE RED. Repo HAS licensed lib PHP; the diff touches only a