Reduce reachable base-image CVEs: pin/refresh operator-managed images flagged by Kubescape #2149

Activity

1. devantler

   added

   security

   on Jun 18, 2026
2. github-project-automation

   added this to 🌊 Project Boardon Jun 18, 2026
3. devantler commented on Jun 21, 2026

   

   devantler

   on Jun 21, 2026

   ContributorAuthor

   More actions

   🤖 Generated by the Daily AI Assistant

   Triage + status against live main — 2 of 4 acceptance criteria already met; remaining work is the CNPG base-image pin (maintainer-gated) + the inherent Kubescape re-survey

   Verified each item against main before commenting so this issue reflects reality rather than its original (pre-#2161) snapshot.

   ✅ Item 1 — coroot images pinned (DONE via #2161)

   k8s/bases/infrastructure/coroot/coroot.yaml now pins every CE component (coroot / node-agent / cluster-agent) explicitly under image.name — the COMPONENT IMAGE PINS block at the top of the CR even references this issue. No :latest remains in observability, and a dedicated Renovate customManager (name: ghcr.io/coroot/<component>:<tag> → docker datasource) keeps them bumped. Acceptance criterion "No :latest image tags in observability" is satisfied.

   ✅ Item 2 — dex + flagger-loadtester images are Renovate-tracked (no config gap)

   Confirmed in .github/renovate.json:

   • dex (chart: dex 0.24.1) — matched by the security-critical controllers packageRule (/^dex$/, automerge: true). Chart bumps carry the dex image (the chart's appVersion), so its image flows automatically.
   • flagger-loadtester (chart: loadtester 0.37.0) — picked up by the built-in flux manager (k8s/**.yaml) with minor/patch automerge on.

   So both images are already covered via their chart version. The residual reachable-criticals on them are upstream base-image age inside the published charts — Renovate bumps the chart when upstream ships a refresh (after the 7-day minimumReleaseAge); it can't independently re-base an image baked into someone else's chart. Acceptance criterion "Renovate confirmed to cover dex / flagger-loadtester images" is satisfied.

   On "is flagger-loadtester even needed on prod?" — yes, keep it. Per loadtester-helm-release.yaml it's the webhook target Flagger calls during canaries to run acceptance/smoke tests and generate load (used by the onboarded umami / homepage / opencost canaries; see docs/progressive-delivery.md). Removing it would break canary acceptance. Its CVE count is high only because it's a stale upstream test image — bounded by upstream chart refreshes, not a local gap.

   ⏳ Item 3 — CNPG Postgres base image (the real remaining work; maintainer-gated)

   Verified: neither k8s/bases/apps/umami/postgres-cluster.yaml nor k8s/providers/hetzner/infrastructure/coroot/postgres-cluster.yaml (nor wedding-db) sets spec.imageName — all three run the operator-default image (18.3-system-trixie). The cloudnative-pg operator chart (0.28.3) is Renovate-tracked via the flux manager, so option (a) "keep the operator current" is already in place — but the default image only refreshes when the operator bumps its default, which is why the 5 fixable gnutls/openssl criticals linger.

   Recommended stance (documenting the cadence — the last acceptance criterion):

   • Default cadence = operator-driven. The operator-default image is the steady-state source; Renovate keeps the operator (and thus its default Postgres base) current. This is the right default for most patches.
   • Fast-patch escape hatch when a fixable reachable critical is flagged (this case): pin the base explicitly so it can move ahead of the operator. Prefer a CNPG ClusterImageCatalog/ImageCatalog (spec.imageCatalogRef) over a per-Cluster spec.imageName — one central, Renovate-trackable catalog entry covers all three clusters (coroot-db, umami-db, wedding-db) instead of three scattered pins, and a Renovate customManager (mirroring the coroot one) can then keep the catalog tag current.

   I have not made this change autonomously: it's a production-database image move (a wrong/incompatible system-trixie tag would stall the rolling update on three HA clusters), and the exact patched tag plus the "net reduction in reachable-critical count" can only be confirmed against the live cluster's Kubescape survey — both your calls. Happy to open the ClusterImageCatalog + Renovate-customManager PR once you confirm the target patched 18.x-system-trixie tag (or say "use latest" and I'll wire the catalog so Renovate tracks it).

   Suggested scope going forward

   Criteria 1 & 2 are met. What's left is genuinely Item 3 + the live re-survey: (1) the CNPG base-image catalog/pin decision above, and (2) re-running the Kubescape reachable-critical count for observability + the DB namespaces to confirm the net reduction (the coroot pins from #2161 should already show up there). Once the CNPG piece lands, this can close.

   (No code change in this pass — this is a verification/triage + recommendation; the one remaining change is a prod-DB decision reserved for you.)
4. devantler commented on Jun 30, 2026

   

   devantler

   on Jun 30, 2026

   ContributorAuthor

   More actions

   🤖 Generated by the Daily AI Assistant

   Status (verified against main this run) — the our-side levers are landed; the residual is upstream.

   AC	State
   No :latest in observability	✅ Done. coroot.yaml pins all three CE components: coroot:1.22.2, coroot-node-agent:1.33.3, coroot-cluster-agent:1.7.1, with a dedicated Renovate customManager (the image.name ImageSpec field the built-in kubernetes manager can't see).
   CNPG Postgres base pinned/tracked	✅ Done. coroot-db / umami-db / backstage-db all pin imageName: ghcr.io/cloudnative-pg/postgresql:18.4-system-trixie (off the operator-default 18.3 that carried the 5 fixable gnutls/openssl criticals), with a Renovate customManager tracking the 18.x-system-trixie line. wedding-db is tenant-owned (pins in the external wedding-app artifact, intentionally out of scope here).
   Renovate covers dex / flagger-loadtester	✅ Confirmed. Both are HelmRelease chart pins (dex 0.24.1, flagger-loadtester 0.37.0) tracked by the built-in flux manager. flagger-loadtester is needed on prod — it's the webhook target the umami / homepage / opencost canaries call for acceptance + load generation, so it stays (not removable).

   Residual reachable-CVEs are upstream base-image age, not a config gap we can close: flagger-loadtester, dex, velero, flux-system/kustomize-controller, openbao/vault-snapshot, cert-manager/origin-ca-issuer all ride chart-bundled images that are already version-pinned and Renovate-tracked — the only lever is keeping the charts current, which Renovate does. There is no remaining declarative pin/refresh we own.

   Recommendation: the actionable scope (AC#1–#3) is complete; this can be narrowed to AC#4 (the ongoing reachable-critical count metric) or closed, with the upstream-image residual folded into the normal Renovate cadence under #2043. Happy to close it if you agree.
5. devantler commented on Jul 3, 2026

   

   devantler

   on Jul 3, 2026

   ContributorAuthor

   More actions

   🤖 Generated by the Daily AI Assistant

   Status re-scope — AC#1–AC#3 are already implemented on main; only the outcome check (AC#4) remains. Verified live this run:

   • AC#1 (no :latest in observability) ✅ — the Coroot CR (k8s/bases/infrastructure/coroot/coroot.yaml) pins all three CE components explicitly: coroot:1.22.2, coroot-node-agent:1.33.3, coroot-cluster-agent:1.7.1 (via image.name, which short-circuits the operator's float-to-highest-semver auto-resolution). A dedicated renovate.json regex customManager tracks each name: ghcr.io/coroot/<component>:<tag> on the docker datasource, so they now get normal bump PRs.
   • AC#3 (CNPG Postgres base cadence) ✅ — implemented rather than just documented: a renovate.json customManager pins spec.imageName on the platform-owned CNPG Clusters (coroot-db, umami-db, backstage-db) to ghcr.io/cloudnative-pg/postgresql:18.x-system-trixie and tracks the 18.4→18.5 line, because the GA operator default (18.3-system-trixie) is a frozen tag carrying the fixable openssl/gnutls CVEs. (Tenant-owned wedding-db pins its own imageName in the wedding-app artifact by design.)
   • AC#2 (dex / flagger-loadtester Renovate coverage) ✅ — both are Flux HelmReleases, so Renovate's built-in flux manager tracks their chart versions; renovate.json additionally scopes the dex path via a fileMatch. flagger-loadtester is a required Flagger canary-analysis component (the umami/homepage/opencost canaries drive load through it), so "remove it" does not apply — keeping it Renovate-current is the right stance.

   Remaining: AC#4 only — confirm a net reduction in Kubescape reachable-critical counts for the listed namespaces. This needs a fresh live Kubescape scan on a cluster day (the pins take effect as images roll on the next Renovate bump / operator reconcile). Leaving open until that scan verifies the drop; everything else is landed.
6. devantler commented on Jul 4, 2026

   

   devantler

   on Jul 4, 2026

   ContributorAuthor

   More actions

   🤖 Generated by the Daily AI Assistant

   Re-scoped under the Kubescape 100% epic #2447. Heads-up: a live investigation found kubevuln is producing no CVE data at all (197/197 image manifests have zero grype matches — the relevancy ScanCP path aborts on partial ApplicationProfiles), so this issue's premise ("reduce reachable CVEs flagged by Kubescape") currently has nothing to act on. It is gated on #2450, which restores CVE scanning. Once data flows, this resumes.
7. github-project-automation

   moved this to 👀 In Review in 🌊 Project Boardon Jul 9, 2026
8. devantler

   closed this as completedin #2542on Jul 9, 2026
9. github-project-automation

   moved this from 👀 In Review to ✅ Done in 🌊 Project Boardon Jul 9, 2026
10. botantler-1 commented on Jul 9, 2026

    

    botantler-1Bot

    on Jul 9, 2026

    Contributor

    More actions

    🎉 This issue has been resolved in version 1.109.1 🎉

    The release is available on GitHub release

    Your semantic-release bot 📦🚀
11. botantler-1

    added

    released

    on Jul 9, 2026