Skip to content

Publish prod manifests through an immutable staging reference so latest never names unsigned bytes #2849

Description

@devantler

🤖 Generated by the Agentic Engineer

Evidence

ghcr.io/devantler-tech/platform/manifests:latest is moved by ksail workload push, and signing does
not begin until three steps later. Measured on main (2026-07-28), step positions in
.github/actions/deploy-prod/action.yml:

# Step
9 📦 Push manifests to GHCR — latest now points at the new bytes
12 🖋️ Sign OCI manifest with cosign
14 🪪 Attest SBOM
15 🪪 Attest build provenance
18 🔁 Trigger Flux reconciliation

There is no staging reference: the mutable tag is the publication target, so the artifact is reachable
under the name production tracks for the whole span between steps 9 and 15.

Audience and problem

The cluster's root Flux source polls that tag on a 1-minute interval and delivers the entire platform —
every controller, tenant binding and policy. Today spec.verify on that source is null, so the window is
latent. It becomes load-bearing the moment root verification is switched on (#2627's other child): Flux
can resolve bytes whose signature or attestations do not exist yet.

Hypothesis

Publishing to an immutable reference, completing the evidence against it, and only then repointing
latest closes the window without changing what production ultimately runs.

First step is measurement, not a guess

Open question to answer first: whether ksail workload push can target a reference other than
latest. If it can, this is a deploy-path change. If it cannot, the capability belongs in KSail and this
issue depends on that work. Nothing else here should be designed before that is settled.

Acceptance criteria

  • latest is only repointed once the signature and both attestations exist for that exact digest.
  • A failed push, sign, or attest leaves the previously verified artifact active under latest.
  • The DR rebuild path publishes through the same transaction (it writes the same tag).
  • A static test pins the transaction, in the shape of scripts/validate-publication-order (test(ci): pin the prod publication ordering so evidence precedes release #2848).

Rough size: M — the transaction itself is small; the KSail capability question is what decides whether
this is one repo or two.

Part of #2627

Activity

  1. moved this to 📥 Backlog in 🌊 Project Boardon Jul 28, 2026
  2. coderabbitai commented on Jul 28, 2026

    @coderabbitai
    Contributor
    🔗 Related PRs

    #2319 - feat(security): cosign-verify the infrastructure Flux OCI artifact [merged]
    #2625 - fix(deploy): synchronize production GHCR pull auth [merged]
    #2633 - fix(deploy): verify incoming image on Talos nodes [merged]
    #2635 - fix(deploy): reboot stale nodes so containerd adopts refreshed GHCR auth [merged]
    #2793 - feat(cilium): activate homogeneous device selection [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  3. self-assigned this
    on Aug 2, 2026
  4. moved this from 📥 Backlog to 🏃🏻‍♂️ In Progress in 🌊 Project Boardon Aug 2, 2026
  5. moved this from 🏃🏻‍♂️ In Progress to 👀 In Review in 🌊 Project Boardon Aug 2, 2026
  6. moved this from 👀 In Review to 🚀 Ready to Merge in 🌊 Project Boardon Aug 2, 2026
  7. moved this from 🚀 Ready to Merge to ✅ Done in 🌊 Project Boardon Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Fields

Priority

None yet

Start date

None yet

Target date

None yet

Effort

None yet

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions