Refresh root Flux GHCR auth before reconciliation #2624

🤖 Generated by the Codex Daily AI Engineer instance.

Description

The production deploy can push a valid Platform artifact but then deadlock before Flux applies it: flux-system/ksail-registry-credentials is KSail-managed and can retain a revoked GHCR pull credential even after k8s/bases/bootstrap/secret.enc.yaml is rotated. The workflow runs workload reconcile before cluster update, and KSail currently ignores credential-only drift, so the root OCIRepository returns GHCR DENIED and every dependent Kustomization stalls.

Steps to Reproduce

1. Rotate stringData.ghcr_dockerconfigjson in the Git/SOPS bootstrap Secret while the live root Flux Secret still has the previous value.
2. Run the production deploy.
3. Observe that artifact push/signing succeeds.
4. Observe workload reconcile fail on OCIRepository/flux-system before cluster update can run.

Expected Behavior

The deploy proves the authoritative Git/SOPS pull credential can read the production artifact and refreshes the root Flux Secret before reconciliation. It reasserts the same value after any KSail-owned cluster update.

Actual Behavior

Merge/heal run 29265027478 timed out with the root source returning GHCR DENIED; apps remained blocked. Credential-only rotation was not applied to the KSail-managed Secret.

Environment

• Production Talos/Hetzner cluster
• KSail 7.167.0
• Flux OCI source flux-system/flux-system

Acceptance Criteria

• Decrypt only ghcr_dockerconfigjson into a mode-restricted temporary file; never expose plaintext or derived base64 in argv/logs.
• Fail closed unless the credential can pull ghcr.io/devantler-tech/platform/manifests:latest.
• Merge-patch only .data[".dockerconfigjson"] before workload reconcile.
• Reassert it after cluster update, including partial update failures after a successful initial preflight.
• Cover normal deploy and DR-rebuild paths with secret-free regression tests.
• Document the Git/SOPS → root Flux → OpenBao → tenant fan-out and emergency recovery.

Additional Context

Child of #2613. KSail credential-only refresh remains tracked in devantler-tech/ksail#6107; the broader Talos node pull-credential convergence remains in the parent and is not closed by this slice.