Skip to content

Let tenant FQDN egress resolve through the platform DNS floor instead of tenant kube-system rules #5823

Open

Description

@devantler

🤖 Generated by the Agentic Engineer

Evidence

  • The tenant network guard (restrict-tenant-network-policies) refuses any tenant rule whose endpoint selector names another namespace. That includes the kube-system DNS service.
  • The generated allow-dns floor allows DNS traffic on port 53. It carries no rules.dns section, so Cilium's DNS proxy never sees a tenant's lookups.
  • Cilium only enforces a toFQDNs egress rule when the pod's DNS traffic passes through that proxy. So a tenant toFQDNs rule needs its own DNS rule toward kube-system, and the guard refuses that rule for any new or changed policy.
  • Both live tenants rely on this shape today, in their public sources: devantler-tech/wedding-app deploy/networkpolicy.yaml, and devantler-tech/ascoachingogvaner deploy/networkpolicy.yaml and deploy/external-dns-networkpolicy.yaml. They also name kube-system, cnpg-system, observability and their own namespace by namespace label.
  • These policies are admitted today only because Kyverno's legacy engine allows an UPDATE to an object that already violated the rule. The ValidatingPolicy migration in platform#5759 has no equivalent. At 08:40 UTC on 2026-11-03, its merge-group deployment left both tenants' Flux reconciliations failing admission. The merge group's recovery restored them at about 09:19 UTC (see the comments on platform#5759).

Audience and problem

This affects tenant application owners and the platform migration (#4995). A tenant cannot write an admissible policy that restricts outbound traffic to named external hosts. That is the egress shape the guard exists to encourage. Meanwhile, the existing tenants' policies are non-compliant and admitted only by a legacy-engine allowance that the migration removes.

Hypothesis

If the platform's generated floor provides the DNS-proxy rule, and the tenants drop the rules the floor already covers, then every tenant policy will be compliant on its own merits. The guard can then move to ValidatingPolicy without relying on any existing-violation allowance. The rules the floor already covers are DNS on port 53 and the CloudNativePG operator's ingress.

Success signal

  • The guard admits a new tenant policy that uses toFQDNs and resolves names through the platform floor. A positive admission fixture covers this.
  • A dry-run of both tenants' committed policies under the tenant identity passes the guard with no existing-violation allowance, and both reconciliations stay Ready. Measure this at the next deployment of the tenant-guard migration.
  • Guardrail: a tenant still cannot name another namespace, a reserved identity or another cluster. The existing negative fixtures stay green.

Smallest useful change

  1. Platform: give the generated allow-dns floor a DNS-proxy rule, either a match-all rules.dns or an opt-in tenant-scoped variant. Then FQDN policies work without tenant-authored kube-system selectors. Record the visibility trade-off in the change.
  2. Tenants: drop the DNS and CloudNativePG-operator rules the floor already provides, and replace own-namespace namespace-label selectors with tenant labels.
  3. Decide, in a platform-owned rule, how the observability ingress to the wedding database is granted. A tenant cannot express it under the guard.

Acceptance criteria

  • The generated floor carries a DNS-proxy rule, with admission and render fixtures.
  • Both tenant repositories' policies pass the guard as a new CREATE under the tenant identity.
  • The observability-to-database path is provided by a platform-owned rule or explicitly retired, with the reason recorded.
  • platform#5759 (or its successor) deploys without tenant reconciliation failures.

Rough size: medium. It spans three repositories, each with a small diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    None yet

    Effort

    None yet

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions