Skip to content

feat(sandboxpolicy): report the applied generation and a true tier in node status - #1082

Closed
AlonLiwsky wants to merge 86 commits into
feat/sandbox-policy-exec-enforcementfrom
feat/sandbox-policy-node-status
Closed

AlonLiwsky wants to merge 86 commits into
feat/sandbox-policy-exec-enforcementfrom
feat/sandbox-policy-node-status

Conversation

@AlonLiwsky

@AlonLiwsky AlonLiwsky commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Overview

Part of the per-node status work for sandbox policies. Two fixes to what each node reports in SandboxPolicyNodeStatus:

Applied generation. The report now fills appliedGeneration: the oldest policy generation still installed on the node, counting only bindings whose protection is confirmed. After a rejected edit it stays on the last-good generation, so observedGeneration: 5, appliedGeneration: 4 says plainly that 4 still enforces. Before, that was only in the free-text ActiveRevision message.

A true tier. Every section enforces through BPF LSM (exec via bprm_check_security, privilege via lsm/*), but only the privilege section consulted the BPF LSM probe. On a node without BPF LSM (default AKS images), an exec policy reported Ready=True, tier: bpf-lsm while nothing ran. Now the probe runs once at startup for all sections; on such a node every report says tier: none, every enabled policy is Ready=False with reason BPFLSMInactive, and appliedGeneration is absent because nothing enforces. A refused privilege section (PrivilegeNotConfigured) reports tier: none only when no last-good binding still enforces.

Pod cgroup bindings count. Privilege pod bindings are driven by the integration service, so the controller never saw them: when the container binding failed while a pod binding enforced, the report said no last-good enforcement exists and omitted appliedGeneration. The privilege backend now reports its bound pod plans (BoundPodPlans), and the report counts them like container bindings. File/network pod bindings from #1073/#1075 will need to report the same way.

Not in this PR: kprobe-override and tripwire tiers (no implementation exists yet), and filling degraded[] (nothing in this branch weakens a single rule; the first case, unresolved classes, lives in #1073 / #1075).

Additional Information

Stacked on #1068. That branch lives on my fork, so this PR targets feat/sandbox-policy-exec-enforcement and also shows the #1067/#1068 commits until those merge. The new commits are the last four:

  • feat(sandboxpolicy): report the applied generation and a true tier per node
  • fix(sandboxpolicy): keep tier and applied generation true in edge cases
  • chore: pin storage to the node status API change
  • fix(sandboxpolicy): count pod cgroup bindings in node status

Temporary storage pin. go.mod replaces kubescape/storage with the fork commit of kubescape/storage#485, which adds the appliedGeneration field and the none tier. I'll swap it for the upstream commit once #485 merges.

How to Test

go test ./pkg/sandboxpolicy/...

Each case in the table below has a unit test (status_test.go, sections_test.go, controller_test.go):

Node / policy state tier appliedGeneration Ready
BPF LSM on, gen 4 rejected, last-good gen 3 bpf-lsm 3 False · NotReady
Bindings at gen 3 and gen 2 bpf-lsm 2 True
BPF LSM off, policy enabled none absent False · BPFLSMInactive
BPF LSM off, mode off none absent True
No privilege image, last-good exec binding bpf-lsm 1 False · PrivilegeNotConfigured
Container binding refused, pod binding at gen 2 bpf-lsm 2 False (last-good kept)

Related issues/PRs

🤖 Generated with Claude Code

…ent library

pkg/ebpf/gadgets/common is a small library for BPF LSM enforcement gadgets:
the bindings map (cgroup -> policy slot), policy rank resolution, verdict
base fields and counters, the arming rule, and the Go binding controller of
one node-wide instance. It is taken from the enforce_exec simple engine of
the pinned Inspektor Gadget fork.

pkg/ebpf/gadgets/enforce_privilege enforces the SandboxPolicy privilege
section with BPF LSM, no seccomp: namespaces, mounts, pivot_root/chroot,
capabilities, ptrace, modules, bpf, perf, keys, io_uring, userfaultfd,
mknod, setuid/setgid and capset. Its compiler merges several policies into
one slot per combination.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…cy controller

- CompiledPolicy carries the privilege section; the compiler accepts it
- SectionBackend splits each plan per section; plans without a privilege
  section go to the exec backend unchanged
- PrivilegeBackend owns one enforce_privilege instance per node: live
  bind/unbind; a new policy combination replaces the instance with overlap
- slots are created when a matching pod is scheduled to the node
- sandboxPolicyPrivilegeImage enables the section; empty rejects it

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an
  allow for one flag cannot carry another flag past a default deny
- compat ptrace read-mode access is not judged
- failed arming marks are counted; a deep tree arms instead of matching a
  wrong namespace
- transient arming failures are retried and reported
- a binding whose rollback failed stays tracked for Unbind
- reject NUL in mountFsType and duplicate slot generation entries
- the gadget build loop stops on the first failure

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an
  allow for one flag cannot carry another flag past a default deny
- compat ptrace read-mode access is not judged
- failed arming marks are counted; a deep tree arms instead of matching a
  wrong namespace
- transient arming failures are retried and reported
- a binding whose rollback failed stays tracked for Unbind
- reject NUL in mountFsType and duplicate slot generation entries
- the gadget build loop stops on the first failure

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The binding cgroup path comes from the container collection, which reads
/proc/<pid>/cgroup in the cgroup namespace of PID 1. node-agent runs in its
own cgroup namespace, where the same file shows a /../.. path, so every
lease check failed with "process cgroup changed" and no container bound.
Check now reads the file in PID 1's cgroup namespace when it differs.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…efix

Gadget eBPF parameters are named operator.oci.ebpf.<key>. The engine set
operator.ebpf.host-mntns-id, which IG ignored, so host_mntns_id stayed 0 and
the program used the global init task's mount namespace. In kind the
runtime runs in the node container's mount namespace, so runc init for
kubectl exec was armed and its setns(CLONE_NEWNS) was denied: every exec
into a bound pod failed.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- keep an unready candidate that failed to stop for a stop retry
- prepare slots for ephemeral containers too
- list the node's pods for slot preparation only after a Pending-pod event
  or every 30 s, not on every refresh

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file;
  the SandboxPolicy translation moves to the controller connection (#1068)
- neutral names in the gadget, BPF, gadget.yaml and README
- a guard test fails on imports outside Inspektor Gadget and the gadget
  folders, and on host-product names or outside includes
- an opt-in test runs the gadget with Inspektor Gadget alone

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file;
  the SandboxPolicy translation moves to the controller connection (#1068)
- neutral names in the gadget, BPF, gadget.yaml and README
- a guard test fails on imports outside Inspektor Gadget and the gadget
  folders, and on host-product names or outside includes
- an opt-in test runs the gadget with Inspektor Gadget alone

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…n the controller layer

PrivilegeDocument (pkg/sandboxpolicy/privilege.go) replaces the gadget former
FromAPI, so the gadget no longer imports the CRD types.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…active

Without "bpf" in the host's active LSM list, BPF LSM programs attach but
never run (for example default AKS Ubuntu and Azure Linux images), so a
policy could report Ready with no enforcement. The service now reads
<host>/sys/kernel/security/lsm at startup; if bpf is missing it creates no
privilege backend, privilege bindings are refused, and every policy with a
privilege section reports Ready=False with reason BPFLSMInactive.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot

When an armed task forks and the child's task storage mark cannot be
allocated, the child escaped every rule while the instance stayed ready.
Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM
(a clone into another cgroup too), and counts a tracking fault per slot
in arming_faults. A lost mark at exec, which cannot be refused, is
counted the same way. Audit-only slots and unbound cgroups allow the
fork. A new thread of an armed group leader is covered through the
leader and is never refused. Moving a container to another slot arms
its running tasks again. The binding controller reads the count with
SlotFaults. A test-only parameter injects the failure for the new
kernel test.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot

When an armed task forks and the child's task storage mark cannot be
allocated, the child escaped every rule while the instance stayed ready.
Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM
(a clone into another cgroup too), and counts a tracking fault per slot
in arming_faults. A lost mark at exec, which cannot be refused, is
counted the same way. Audit-only slots and unbound cgroups allow the
fork. A new thread of an armed group leader is covered through the
leader and is never refused. Moving a container to another slot arms
its running tasks again. The binding controller reads the count with
SlotFaults. A test-only parameter injects the failure for the new
kernel test.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
AlonLiwsky and others added 22 commits November 5, 2026 11:26
…ntainer

A parent cgroup bound before its containers start (a pod cgroup) also
covered the OCI createContainer and startContainer hooks: runc init forks
them inside the container mount namespace, so they were armed and judged
as workload. A strict policy then stopped the container from starting (for
example kind's own createContainer hook, a bind mount).

A task is no longer armed at exec when its parent is unarmed, outside the
runtime mount namespace, and in the same mount namespace and cgroup as the
task: it was forked by a runtime helper inside the container. The workload
and exec commands are execed by runc init itself, whose parent is in the
runtime mount namespace, so they stay armed, and their children inherit
the mark as before. An unknown runtime namespace trusts no parent.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>

# Conflicts:
#	pkg/ebpf/gadgets/enforce_privilege/README.md
… of Linux 6.15

Linux 6.15 added `bool kernel` to the bpf hook and dropped the attr
argument of perf_event_open, so the previous-verdict check refused every
6.15+ kernel (and the old perf_event_open program would not load there).
Both programs now read `ret` at the position of the running kernel: a
CO-RE type match against security_list_options picks the branch at load.
The BTF check returns a result per hook (LSMHookArgMismatches) and
accepts both signatures for these two hooks. The other 21 hooks have the
same argument count from 6.1 to 6.18 (checked in lsm_hook_defs.h).

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
setns into a namespace checks CAP_SYS_ADMIN twice (the namespace user
namespace and the caller's). Audit lets the first check pass, so one
audited setns gives two identical verdicts; enforce stops at the first.
A per-syscall dedupe is not reliable in an LSM hook (there is no syscall
instance id, and a time window would merge real calls), so the README
documents it and a kernel test pins it down. The test fixture now records
the fields of each verdict.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…e-gadget

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The #1054 base moved to storage v0.0.388: PrivilegeTarget has no path,
fromImage or fsType fields any more, and Rule.Origin is a pointer. The
translator follows: it rejects actor and pathClass as before, now also
rejects except (the gadget has no exceptions, and they were ignored
before), and maps a missing origin and an empty one to no origin.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…e-gadget

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>

# Conflicts:
#	pkg/sandboxpolicy/integration/service.go
#	pkg/sandboxpolicy/integration/service_test.go
The walk repeated until a pass saw no new task id, so a container that
keeps starting short-lived tasks could fail the bind with "tasks still
changing". A pass now counts only the tasks it had to arm: a task that
already carries the mark (a child forked after its parent was armed) or
that exited is not progress. The rescan for children forked before their
parent was armed stays.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Review findings on the pod binding paths:
- An edited pod plan whose slot is not ready keeps the last-good pod
  binding; the pod moves to the new slot once it exists. Only a pod that
  is not given any more (an authoritative removal) is unbound.
- A candidate that does not fit drops only the speculative prepared
  slots. Active pod bindings are never dropped to make room; when the
  candidate still does not fit, the new container binding is refused.
- Incomplete input (an invalid policy or class, a policy that does not
  compile, too many policies) gives the pod an empty plan, which keeps
  its current binding, as the container path keeps its last good plan.
  Containers with different policies, no privilege policy, or a policy
  that needs a ContainerProfile still give no pod plan.
Each case has a regression test.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…origin

Rule origins and the new section default origin are pointers in the
policy document. The compiler gives an explicit "" its own origin id;
id 0 still means no origin. Compiled.DefaultOrigins lists, per slot, the
default origins of the documents that agree with the merged default,
for events of the default.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
A section can set the mode of its default apart from the policy mode.
The compiler ignored it, so an audit policy with an enforce default only
audited its default deny. The merged default and the audit-deny raise
now use each document's defaultMode, else its policy mode.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…e gadget

The translator now keeps origin presence (a missing origin stays nil, an
explicit "" stays "") for rules and the section default, and passes the
section defaultMode, which it dropped before.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…r node

Node status now carries appliedGeneration, the oldest generation still
installed, so a rejected revision no longer hides which one enforces.

The BPF LSM probe ran only for the privilege section, but exec also
enforces through BPF LSM, so an exec policy reported Ready and tier
bpf-lsm on nodes where nothing runs. The probe result now applies to
every section and such nodes report tier none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Without BPF LSM, installed bindings enforce nothing, so the report no
longer names an applied generation. A refused privilege section no
longer reports tier none while last-good exec bindings still enforce.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Temporary replace to the fork commit of kubescape/storage#485, until it
merges into the sandbox policy API branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 20d9a5d3-4001-4dd4-b539-6da9c077a2e0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Privilege pod bindings are driven by the integration service, so the
controller never saw them. When the container binding failed while a pod
binding enforced, the report said no last-good enforcement exists and
omitted appliedGeneration. The privilege backend now reports its bound
pod plans, and the report counts them like container bindings. A pod
whose plan changes only in generation records the new plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.027 0.000 -100.0%
Peak CPU (cores) 0.029 0.000 -100.0%
Peak CPU p95 (cores) 0.029 0.000 -100.0%
Avg Memory (MiB) 210.690 0.000 -100.0%
Peak Memory (MiB) 214.559 0.000 -100.0%
Dedup Effectiveness

No data available.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.111 0.106 -4.2%
Peak CPU (cores) 0.114 0.115 +0.7%
Peak CPU p95 (cores) 0.114 0.113 -0.7%
Avg Memory (MiB) 277.129 337.606 +21.8%
Peak Memory (MiB) 291.520 347.254 +19.1%
Dedup Effectiveness

No data available.

@AlonLiwsky

Copy link
Copy Markdown
Collaborator Author

Folded into #1068 to shorten the stack: the privilege-controller branch now carries these commits unchanged, and review continues there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant