The handoff quiesce timeout hides its own cause, which is already in hand #3002

🤖 Generated by the Agentic Engineer

Evidence

When pause_flux_policy_handoff() in scripts/refresh-flux-ghcr-auth.sh exhausts its retry budget
it emits exactly one line:

The Flux image-verification policy owner did not quiesce before the image-verification policy handoff.

That names the step that gave up. It does not name what the step was waiting on, and in the one
observed occurrence the real cause was four resources upstream and in an unrelated subsystem.

Measured on CI run 31110692734
(2026-08-06), which evicted #3001 from the merge queue and whose 🩹 Heal Prod job failed
identically while re-deploying main:

Every fact needed to explain the failure was already in the two Kustomization status conditions
the function had just read into flux_policy_handoff_state_file — the message simply discards
them.

Affected audience and impact

Whoever is on the other end of a failed prod deploy — maintainer or agent. The step burned ~4
minutes and reported a subsystem (image verification, GHCR credentials) that was working correctly,
while the actual blocker was an un-stepped CNI rollout. Reconstructing the chain took a live
four-resource walk against prod.

This is not specific to the Cilium rollout in #2963. Any condition that keeps infrastructure
reconciling — a stuck health check, an unready dependency, a slow HelmRelease — produces the same
misdirecting message, so the cost recurs for causes not yet seen. It is also the failure mode the
AGENTS.md merge-queue rule most depends on being legible: it requires root-causing a kick-out
before re-queuing, and this message actively points away from the root cause.

Expected behaviour

When the handoff quiesce wait times out, the error names why the owner was not quiescent: the
blocking condition's type/reason/message, and — when the reason is DependencyNotReady — the
named dependency and its own blocking condition.

Acceptance criteria

• On timeout the step reports the infrastructure Kustomization's non-quiescent condition
  (reason + message), not only that quiescence was not reached.
• When that reason is DependencyNotReady, the named dependency's own blocking condition is
  read and reported in the same failure.
• The existing single-line summary is preserved as the headline so log-scrapers and the
  ::error:: annotation keep working; the detail is additive.
• RED/GREEN proof in scripts/tests/refresh-flux-ghcr-auth/: drive fake_kubectl to return a
  non-quiescent infrastructure carrying DependencyNotReady on
  infrastructure-controllers, and assert the emitted failure names that dependency. The test
  must fail against the current message and pass after the change.
• An ablation showing the added read cannot itself fail the step: if the dependency lookup
  errors or returns nothing, the original error is still emitted and the exit status is
  unchanged.

Notes on approach

read_flux_policy_fences() already fetches both the parent and child Kustomization into state
files, so the child's conditions are in hand with no extra API call. Only the DependencyNotReady
branch needs one additional read, and it must be strictly best-effort — this runs on the production
credential path, and a diagnostic must never be able to change the outcome it is describing.

Size

Small. One error path plus its tests; no behaviour change on the success path.

Discovered while root-causing #2963.