Skip to content

No control stops an installed chart granting itself cluster-admin-equivalent RBAC #3486

Closed
2 of 2 issues completed
Closed

Description

@devantler

🤖 Generated by the Agentic Engineer

Evidence

Nothing currently stops a Helm chart from granting itself cluster-admin-equivalent permissions
through a custom role.

Two layers were checked and neither covers it:

1. The CI authorization rules cannot see chart content. reject-new-cluster-admin-bindings in
scripts/tests/production-authorization-rules.yaml matches on
object.roleRef.name != 'cluster-admin', so it rejects bindings to the built-in role by name. A
chart emitting its own ClusterRole with wildcard rules, or with bind/escalate on
rbac.authorization.k8s.io, and binding it to its own controller, satisfies that condition and
passes. More fundamentally, a HelmRelease's chart is rendered in-cluster by Flux, not during
the CI render, so its RBAC objects are never in the tree these rules evaluate at all — this is not a
matching bug that a better CEL expression fixes.

2. No admission-time policy covers it either. Enumerated live against prod: 26 Kyverno
ClusterPolicy objects, and none constrains RBAC. The three that mention roles at all
(restrict-github-team-management, restrict-tenant-secret-stores) are about GitHub teams and
tenant secret stores. There is no disallow-cluster-admin, no wildcard-verb restriction, and no
escalation guard.

So the only control on what an installed chart may grant itself is the pinned chart digest plus
PR review of any bump.

Why the digest pin is not sufficient on its own

The pin is genuinely strong against a moved tag — digests are immutable, so a compromised upstream
cannot change what a pinned digest resolves to. What it does not do is make a digest bump
self-describing: the reviewer of a bump sees a hash, not the RBAC the new chart version emits.
Reviewing that today means manually rendering the chart and reading its templates, which is exactly
the kind of check that is skipped when a bump looks routine.

This matters more since isolated-chart scoping exists, because an isolated chart's Flux objects are
deliberately outside the fingerprinted authorization surface — that isolation is correct and
reviewed, but it means the layer above has to be the one that constrains what the chart may grant.

Impact

Defence in depth, not an open door — reaching it requires a merged digest bump to a chart that grants
itself the permissions. Recording it honestly: this is a gap in a layer that exists precisely so a
single reviewed mistake is not sufficient, not a live exploit path.

Expected behaviour

Admission-time enforcement rejects a role granting cluster-admin-equivalent permissions to a
workload's own controller, so the guarantee holds regardless of what a chart renders, and independent
of whether CI can see it.

Acceptance criteria

  • An admission policy rejects ClusterRole/Role objects granting wildcard apiGroups,
    resources and verbs, and ones granting bind, escalate, or impersonate on
    rbac.authorization.k8s.io.
  • The measured legitimate exceptions keep working — flux-operator, longhorn-support-bundle
    and velero-server are the three the CI rules already name as reviewed cluster-admin holders.
    Enumerate current holders before enforcing; a policy that wedges Flux is worse than the gap.
  • Ships Audit first and graduates to Enforce on evidence, per the fix-vs-except ladder — this
    is exactly the class where an untested enforce policy can stop production reconciling.
  • Consider whether this belongs in devantler-tech/kyverno-policies rather than the platform: the
    guarantee is product-neutral, but the exception set is platform-specific, so the policy may be
    shared while its exceptions stay local. Second consumer required before extracting, per the
    sharing rule.

Size

Medium. The policy is small; establishing the current legitimate holders and staging Audit → Enforce
safely is the real work.

Found by a Codex P2 on platform#3476. That PR is not blocked on this: it neither introduces the gap
nor widens what a chart can render — it is the review that surfaced the question.

Activity

  1. moved this to 📥 Backlog in 🌊 Project Boardon Aug 31, 2026
  2. self-assigned this
    on Sep 6, 2026
  3. added a commit that references this issue on Sep 6, 2026
  4. devantler commented on Sep 6, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    The Audit-first implementation merged in #3630 at 6a4ca89ae29845bb6082e2d263239a2a90694c44; merge-group CI completed successfully. The policy is Ready in Audit mode and its narrow reporting read grant is present. All 280 policy fixtures pass with the deployed engine version.

    Natural report publication is not yet verified: a complete post-deployment report census has no result for this policy. A later complete event census confirms natural background evaluation of all expected privileged findings, narrowing the remaining gap to report publication or aggregation. This rules out a missing watcher for those evaluated resources. Effective read permissions and API discovery were checked, but the publication cause remains unproven. No permission widening or speculative restart was performed. Raw production evidence remains private.

    This issue stays open. The next acceptance step is to establish complete background report coverage, evaluate the legitimate findings, then prepare an evidence-based Enforce change. The deployed Audit policy does not reject privileged grants. CI/runtime version drift found during verification is separately addressed by #3631 / #3632.

  5. devantler commented on Sep 13, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Report coverage is now complete for every role Kyverno can evaluate

    The open gap from the last update was report publication. It is resolved. The reports controller was stuck and has been fixed by #3779, merged 2026-09-13. Reports for audit-privileged-rbac now publish from background evaluation. All reads below are read-only production reads taken 2026-09-13 around 22:30Z.

    Kind Live With a report result Without one
    ClusterRole 253 243 10
    Role 63 43 20

    Every missing object is explained by Kyverno's own global resource filters, not by the policy. Kyverno never evaluates these objects for any policy:

    • 10 ClusterRoles: Kyverno's own controller roles, filtered by exact name.
    • 20 Roles: 15 in kube-system, 4 in kyverno, and 1 in kube-public. All three namespaces are filtered wholesale.

    With those removed, coverage is exact: 243 of 243 ClusterRoles and 43 of 43 Roles.

    Result counts. 272 pass and 14 warn:

    • 12 warns are cluster-scoped ClusterRoles.
    • 1 warn is a Role in longhorn-system.
    • 1 warn is a Role in velero.

    Every warn carries the policy's wildcard-administration, bind/escalate or impersonation message. The object-level list stays in private operator notes.

    What this means for Enforce. Enforcement can only ever cover the evaluable set above. A privileged grant created in kube-system, kube-public or kyverno passes regardless of this policy's mode, because Kyverno's own filters skip it. Whether to narrow those filters is a separate decision, and the Enforce change should say so explicitly rather than imply complete coverage.

    Next acceptance step: classify the 14 warn findings as legitimate or not, then prepare the evidence-based Enforce change with scoped exceptions for the legitimate ones.

  6. removed their assignment
    on Sep 14, 2026
  7. moved this from 📥 Backlog to 🏃🏻‍♂️ In Progress in 🌊 Project Boardon Sep 14, 2026
  8. self-assigned this
    on Sep 14, 2026
  9. added a commit that references this issue on Sep 14, 2026
  10. moved this from 🏃🏻‍♂️ In Progress to 👀 In Review in 🌊 Project Boardon Sep 14, 2026
  11. moved this from 👀 In Review to 🚀 Ready to Merge in 🌊 Project Boardon Sep 14, 2026
  12. moved this from 🚀 Ready to Merge to 📊 Verifying in 🌊 Project Boardon Sep 14, 2026
  13. devantler commented on Sep 14, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Enforce is live, but 14 old warn results have not cleared

    Read-only production reads, 2026-09-14 at 05:39Z. Counts only.

    • The Enforce change is deployed. The policy's single rule runs with failureAction: Enforce and 7 exclusion blocks.
    • The background scan is running. It rescanned at 05:27–05:29Z and refreshed all 272 pass results: 41 namespaced and 231 cluster-scoped.
    • 14 warn results did not refresh. 2 are namespaced and 12 cluster-scoped. All still carry timestamps from 04:15–04:16Z, which is before the Enforce change was applied at 04:16:36Z. None of the 14 reports holding them contains any newer result.
    • There are no fail results.

    What this most likely means. The 14 are the identities the Enforce change excluded. Once a resource is excluded, the scan skips it and writes nothing for it, so the result from the old Audit policy stays in its report. These are stale leftovers, not live warnings.

    Why this stays open for now. The acceptance check was "the excluded identities carry no result and there are 0 fail". The second half holds, but the first does not while those stale results remain. The next step is to confirm how Kyverno cleans up results for newly excluded resources, and whether they clear on their own (for example on the next change to those resources) or need a report refresh. Closing now would leave 14 misleading warnings in the reports.

  14. devantler commented on Sep 14, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Decision: the control is delivered and verified, so I'm closing this

    Read-only production reads, 2026-09-14 at 07:55Z. Counts only.

    • The control works. audit-privileged-rbac runs in Enforce and there are 0 fail results.
    • Every covered role is re-evaluated. The hourly background scan refreshed all 272 pass results at 07:42–07:43Z: 231 cluster-scoped and 41 namespaced.
    • The 14 old warn results are leftovers, not live findings. They match the policy's excluded identities one for one. They are still timestamped 04:15–04:16Z, before Enforce was applied, and survived at least two later scans. Kyverno stops evaluating a resource once it is excluded, so it never removes that resource's old result.

    Why close rather than keep waiting. The last update asked whether the 14 clear on their own. They do not: waiting three and a half hours through two scans changed nothing, so waiting longer would only park this issue. The security outcome here, that no installed chart can grant itself cluster-admin-equivalent RBAC without a reviewed exclusion, is live. Removing the misleading report entries is a separate, smaller report-hygiene change. It needs a cluster-side refresh that this read-only verification could not perform, and it is tracked in #3800, linked as a sub-issue.

  15. moved this from 📊 Verifying to ✅ Done in 🌊 Project Boardon Sep 14, 2026
  16. devantler commented on Sep 14, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Correction to the closing note: the stale-result cleanup is now tracked in #2573, the older issue for the same Kyverno behaviour. #3800 was closed as its duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Fields

No fields configured for Security.

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions