Repository navigation
feat(sandboxpolicy): enforce the privilege section with a BPF LSM gadget #1067
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AlonLiwsky
wants to merge
35
commits into
kubescape:feat/sandbox-policy-exec-enforcement
Choose a base branch
from
AlonLiwsky:feat/privilege-gadget
base: feat/sandbox-policy-exec-enforcement
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,724
−2
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
044cc37
feat(enforce_privilege): BPF LSM privilege gadget and common enforcem…
AlonLiwsky de8c006
fix(enforce_privilege): address review comments
AlonLiwsky 89fc294
docs(enforce_privilege): document the known limits
AlonLiwsky 931392e
docs(enforce_privilege): correct the no-audit limit
AlonLiwsky ca6857c
fix(enforcement): report cgroup listing errors other than a removed c…
AlonLiwsky d06212f
refactor(enforce_privilege): make the gadget and common library portable
AlonLiwsky 5d5a845
docs(enforce_privilege): mountFsType does not cover bind mounts
AlonLiwsky 630299d
test(enforcement): only ../common/ includes may leave the gadget folder
AlonLiwsky 3bd49a5
test(enforce_privilege): the kernel test takes an absolute cgroup v2 …
AlonLiwsky 864119c
test(enforcement): include guard also catches spaced, absolute and ne…
AlonLiwsky 359b53a
fix(enforce_privilege): refuse a fork whose child cannot be armed in …
AlonLiwsky 0c07186
test(merge): shared merge contract and test vectors
AlonLiwsky 38743dd
fix(enforcement): a same-slot retry arms again after a failed arming
AlonLiwsky 0a469cf
fix(enforce_privilege): reject clone and clone3 rules without flags
AlonLiwsky b7e6336
test(enforcement): a slot move retry arms again after a failed rollback
AlonLiwsky b307ca1
fix(enforcement): arm the group leader of a thread that cannot be opened
AlonLiwsky f6d9b1d
fix(enforce_privilege): a slot enforces only when one of its entries …
AlonLiwsky 2f51eeb
docs(enforce_privilege): compat mknod and leaving the bound subtree
AlonLiwsky 95c2f23
fix(enforcement): arm a thread whose group leader has exited
AlonLiwsky b09cddf
test(enforce_privilege): another BPF LSM deny holds under the gadget
AlonLiwsky 0e012a2
fix(enforce_privilege): return an earlier BPF verdict unchanged
AlonLiwsky c92c5bd
Merge feat/sandbox-policy-exec-enforcement (#1054) into feat/privileg…
AlonLiwsky ccd5202
fix(enforcement): do not arm OCI hooks that runc init forks in the co…
AlonLiwsky ec17414
fix(enforce_privilege): handle the bpf and perf_event_open signatures…
AlonLiwsky 8e0a168
docs(enforce_privilege): a verdict is one kernel check, not one syscall
AlonLiwsky 071f543
Merge feat/sandbox-policy-exec-enforcement (#1054) into feat/privileg…
AlonLiwsky 7ba4643
Merge feat/sandbox-policy-exec-enforcement (#1054) into feat/privileg…
AlonLiwsky 9a53ab3
fix(enforcement): arming converges while a container keeps forking
AlonLiwsky b4fbf87
feat(enforce_privilege): keep an explicit empty origin apart from no …
AlonLiwsky c05ec59
fix(enforce_privilege): the default follows the section's defaultMode
AlonLiwsky d2200c1
fix(enforcement): exact binding scope, fail closed on a missing slot,…
AlonLiwsky 8af40f7
refactor(enforce_privilege): export StrictSlots for the owner's healt…
AlonLiwsky 9388578
fix(enforcement): deny a task outside a Strict exact binding's scope
AlonLiwsky 9f9dfde
test(enforce_privilege): a Strict exact parent denies its descendants…
AlonLiwsky 16ecd63
fix(enforcement): the nearest bound parent decides; refuse a Strict b…
AlonLiwsky File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Common enforcement library | ||
|
|
||
| Shared pieces of the BPF LSM enforcement gadgets (`enforce_privilege` today). | ||
| They are taken from the enforce_exec gadget of the pinned Inspektor Gadget fork | ||
| and keep the first verdict fields, the mode encoding, the `counters` slots, the | ||
| identity type and the `bindings` key and value size. To move enforce_exec onto | ||
| the library, these still differ and need a decision: the `bindings` value is a | ||
| policy slot here (one instance per node) but 0 or a runtime mount namespace | ||
| inode in enforce_exec (one instance per container, restricted bindings); | ||
| enforce_exec reports allowed execs with | ||
| the policy mode (this library reports only denies), counts every allowed | ||
| decision, has restricted bindings and kernel-fault checks in its controller, | ||
| and keeps its path rules in an array that `policy.h` does not describe. | ||
|
|
||
| | File | Content | | ||
| |---|---| | ||
| | `bindings.h` | `bindings` map: cgroup id → the container's policy slot and flags. Exact scope by default; `BINDING_SUBTREE` covers descendant cgroups, `BINDING_STRICT` carries the slot's class (a fault denies), `BINDING_RUNTIME_HELPERS` admits the arming exemption | | ||
| | `policy.h` | policy modes (0 off, 1 audit, 2 enforce), rank resolution (explicit deny > allow > default, one mode per rule), shared policy value | | ||
| | `verdict.h` | first verdict fields (`generation`, `cgroup`, `deny`, `mode`), per-CPU `counters` (denied, submitted, dropped, allowed, section error, rate limited), 100 reports per CPU per second | | ||
| | `arming.h` | arming rule: judge only tasks that executed outside the runtime mount namespace, and their children; under `BINDING_RUNTIME_HELPERS` only, not those forked there by an unarmed runtime helper (OCI hooks). `arming_faults`: faults per slot; `arming_witness`: node faults and exempted execs. Include `bindings.h` first | | ||
| | `merge/` | the merge contract for several policies on one container, and its shared test vectors | | ||
| | `enforcement/` | Go: the binding controller of one node-wide instance (policy written and frozen before any binding, live `Bind`/`Unbind` of a container to a slot, readiness, arming of tasks already running, `SlotFaults`, `ArmingWitness`) | | ||
|
|
||
| Section-specific parts (the policy key layout, hooks, compiler) stay in each | ||
| gadget. | ||
|
|
||
| ## Build | ||
|
|
||
| `ig image build` mounts only the gadget folder, so `#include "../common/..."` | ||
| does not resolve by default. A gadget Makefile sets `IG_SOURCE_PATH` to the | ||
| repository root; the builder then mounts the whole repository. This reuses an | ||
| Inspektor Gadget variable meant for in-tree gadgets of the Inspektor Gadget | ||
| repository. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,249 @@ | ||
| // SPDX-License-Identifier: GPL-2.0 | ||
| // Copyright 2026 The Kubescape Authors | ||
|
|
||
| // Arming rule shared by the BPF LSM enforcement gadgets. | ||
| // | ||
| // A cgroup-scoped enforcer also sees the container runtime's own setup inside | ||
| // the container cgroup (runc init creates namespaces, mounts, pivots root and, | ||
| // for exec, joins namespaces). Seccomp never sees that work because runc | ||
| // applies the filter as its last step, right before it execs the workload. | ||
| // | ||
| // This header gives the same boundary to BPF: a task is "armed" when it | ||
| // executes a program outside the host mount namespace, and a child inherits | ||
| // the mark at fork. Only armed tasks are judged. Process names are never | ||
| // trusted. Tasks that already ran before the gadget loaded are armed from user | ||
| // space by writing the task storage map with a pidfd key. | ||
| // | ||
| // runc init for "exec" execs inside the container cgroup but still in the host | ||
| // mount namespace, so it is not armed. The command it execs after joining the | ||
| // container mount namespace is armed. | ||
| // | ||
| // OCI createContainer and startContainer hooks are runtime setup too: runc init | ||
| // forks them inside the container mount namespace and cgroup, before the | ||
| // workload starts. Only a binding with BINDING_RUNTIME_HELPERS exempts them: | ||
| // the owner sets it on a parent cgroup bound before its containers start, never | ||
| // on a container binding. Under that flag a task is not armed at exec when its | ||
| // parent is unarmed, outside the host mount namespace, and in the same mount | ||
| // namespace and cgroup as the task (a runtime helper inside the container). | ||
| // The workload and an "exec" command are execed by runc init itself, whose | ||
| // parent (the runtime or its shim) is in the host mount namespace. Such a | ||
| // helper, and a hook, are trusted not to leave a process behind: its children | ||
| // stay unarmed. Every exempted exec is counted. A Strict container binding has | ||
| // no exemption, as in enforce_exec. | ||
|
|
||
| #ifndef __ENFORCEMENT_ARMING_H | ||
| #define __ENFORCEMENT_ARMING_H | ||
|
|
||
| #include <vmlinux.h> | ||
| #include <bpf/bpf_helpers.h> | ||
| #include <bpf/bpf_core_read.h> | ||
|
|
||
| // The binding flags come from bindings.h, which a gadget includes first | ||
| // (after it sets BINDINGS_CGROUP_DEPTH). | ||
| #ifndef BINDING_RUNTIME_HELPERS | ||
| #error "include bindings.h before arming.h" | ||
| #endif | ||
|
|
||
| // Inode number of the mount namespace the container runtime runs in. 0 means | ||
| // the mount namespace of the global init task. Set it when the runtime does not | ||
| // run in init's mount namespace (for example a kind node). | ||
| const volatile __u64 host_mntns_id = 0; | ||
|
|
||
| struct { | ||
| __uint(type, BPF_MAP_TYPE_TASK_STORAGE); | ||
| __uint(map_flags, BPF_F_NO_PREALLOC); | ||
| __type(key, int); | ||
| __type(value, __u32); | ||
| } armed_tasks SEC(".maps"); | ||
|
|
||
| // A thread counts as armed when it or its thread group leader is armed. User | ||
| // space can arm only the leader on kernels without PIDFD_THREAD (< 6.9). | ||
| static __always_inline bool arming_task_armed(struct task_struct *task) | ||
| { | ||
| __u32 *a = bpf_task_storage_get(&armed_tasks, task, 0, 0); | ||
| struct task_struct *leader; | ||
|
|
||
| if (a && *a) | ||
| return true; | ||
| leader = task->group_leader; | ||
| if (!leader || leader == task) | ||
| return false; | ||
| a = bpf_task_storage_get(&armed_tasks, leader, 0, 0); | ||
| return a && *a; | ||
| } | ||
|
|
||
| // Fault injection for tests only: fail every child mark at fork. Owners never | ||
| // set it in production. | ||
| const volatile bool arming_fail_fork_marks = false; | ||
|
|
||
| // Faults per policy slot, for the owner's health check: a mark that failed for | ||
| // a task in a Strict binding, or a Strict binding whose slot has no policy. The | ||
| // array is preallocated, so counting never fails. A new instance starts at zero | ||
| // and arms the running tasks again. | ||
| #ifndef ARMING_FAULT_SLOTS | ||
| #define ARMING_FAULT_SLOTS 64 | ||
| #endif | ||
|
|
||
| struct { | ||
| __uint(type, BPF_MAP_TYPE_ARRAY); | ||
| __uint(max_entries, ARMING_FAULT_SLOTS); | ||
| __type(key, __u32); | ||
| __type(value, __u64); | ||
| } arming_faults SEC(".maps"); | ||
|
|
||
| static __always_inline void arming_slot_fault(__u32 slot) | ||
| { | ||
| __u64 *value = bpf_map_lookup_elem(&arming_faults, &slot); | ||
|
|
||
| if (value) | ||
| __sync_fetch_and_add(value, 1); | ||
| } | ||
|
|
||
| // Node-wide witness counters of the arming rule, for the owner's health check. | ||
| enum arming_witness { | ||
| // A child mark failed outside a Strict binding. The child can be in | ||
| // another cgroup (clone3 with CLONE_INTO_CGROUP), so the owner treats a | ||
| // nonzero count as a fault of every Strict binding. | ||
| ARMING_WITNESS_NODE_FAULT = 0, | ||
| // Execs that BINDING_RUNTIME_HELPERS left unarmed. | ||
| ARMING_WITNESS_EXEMPT_EXEC = 1, | ||
| ARMING_WITNESS_MAX = 2, | ||
| }; | ||
|
|
||
| struct { | ||
| __uint(type, BPF_MAP_TYPE_ARRAY); | ||
| __uint(max_entries, ARMING_WITNESS_MAX); | ||
| __type(key, __u32); | ||
| __type(value, __u64); | ||
| } arming_witness SEC(".maps"); | ||
|
|
||
| static __always_inline void arming_witness_count(__u32 key) | ||
| { | ||
| __u64 *value = bpf_map_lookup_elem(&arming_witness, &key); | ||
|
|
||
| if (value) | ||
| __sync_fetch_and_add(value, 1); | ||
| } | ||
|
|
||
| // arming_mark returns false when the task storage could not be allocated (memory | ||
| // pressure): the task then stays unarmed, and the caller handles the failure. | ||
| static __always_inline bool arming_mark(struct task_struct *task) | ||
| { | ||
| __u32 *a = bpf_task_storage_get(&armed_tasks, task, 0, | ||
| BPF_LOCAL_STORAGE_GET_F_CREATE); | ||
| if (!a) | ||
| return false; | ||
| *a = 1; | ||
| return true; | ||
| } | ||
|
AlonLiwsky marked this conversation as resolved.
Resolved
|
||
|
|
||
| #define ARMING_MAX_ANCESTORS 64 | ||
|
|
||
| // Cached mount namespace of the global init task (0 = not known yet). | ||
| __u64 arming_init_mntns = 0; | ||
|
|
||
| // arming_host_mntns returns host_mntns_id, or the mount namespace of the global | ||
| // init task. The real_parent chain of every task ends at the global init task | ||
| // (container processes are reparented to the runtime shim, which descends from | ||
| // it). | ||
| static __always_inline __u64 arming_host_mntns(struct task_struct *task) | ||
| { | ||
| struct task_struct *t = task; | ||
|
|
||
| if (host_mntns_id) | ||
| return host_mntns_id; | ||
| if (arming_init_mntns) | ||
| return arming_init_mntns; | ||
|
|
||
| for (int i = 0; i < ARMING_MAX_ANCESTORS; i++) { | ||
| struct task_struct *parent = BPF_CORE_READ(t, real_parent); | ||
|
|
||
| if (!parent || parent == t) { | ||
| // Reached the top: cache the init task's namespace. | ||
| arming_init_mntns = | ||
| BPF_CORE_READ(t, nsproxy, mnt_ns, ns.inum); | ||
| return arming_init_mntns; | ||
| } | ||
| t = parent; | ||
| } | ||
| // Tree deeper than the walk: no namespace inode is 0, so the task is | ||
| // treated as outside the runtime namespace (armed), and nothing is cached. | ||
| return 0; | ||
| } | ||
|
|
||
| // arming_runtime_child reports whether the task was forked by a runtime | ||
| // helper inside the container (an OCI hook, or a hook's child): its parent is | ||
| // unarmed, outside the host mount namespace, and in the same mount namespace | ||
| // and cgroup as the task. A parent with no mount namespace (exiting) does not | ||
| // match, so the task is armed. The parent pointer is a direct BTF access (not | ||
| // CO-RE): task storage needs a trusted task pointer. | ||
| static __always_inline bool arming_runtime_child(struct task_struct *task, | ||
| __u64 mntns, __u64 host) | ||
| { | ||
| struct task_struct *parent = task->real_parent; | ||
| __u64 parent_mntns; | ||
|
|
||
| if (!parent || arming_task_armed(parent)) | ||
| return false; | ||
| parent_mntns = BPF_CORE_READ(parent, nsproxy, mnt_ns, ns.inum); | ||
| if (!parent_mntns || parent_mntns == host || parent_mntns != mntns) | ||
| return false; | ||
| return BPF_CORE_READ(parent, cgroups, dfl_cgrp, kn, id) == | ||
| BPF_CORE_READ(task, cgroups, dfl_cgrp, kn, id); | ||
| } | ||
|
|
||
| // arming_on_exec runs after a successful exec (bprm_committed_creds), with the | ||
| // flags of the binding that covers the task (0 when unbound). | ||
| // arming_on_exec returns false when a needed mark failed. | ||
| static __always_inline bool arming_on_exec(__u32 binding_flags) | ||
| { | ||
| struct task_struct *task = bpf_get_current_task_btf(); | ||
| __u64 mntns = BPF_CORE_READ(task, nsproxy, mnt_ns, ns.inum); | ||
| __u64 host; | ||
|
|
||
| if (arming_task_armed(task)) | ||
| return true; | ||
| host = arming_host_mntns(task); | ||
| if (mntns == host) | ||
| return true; | ||
| // An unknown host namespace trusts no parent. | ||
| if ((binding_flags & BINDING_RUNTIME_HELPERS) && host && | ||
| arming_runtime_child(task, mntns, host)) { | ||
| arming_witness_count(ARMING_WITNESS_EXEMPT_EXEC); | ||
| return true; | ||
| } | ||
| return arming_mark(task); | ||
| } | ||
|
|
||
| #define ARMING_CLONE_THREAD 0x00010000ULL | ||
|
|
||
| // arming_on_fork runs in task_alloc: the child inherits the parent's mark. | ||
| // It returns false only when the parent is armed and the child's mark failed. | ||
| // An unarmed parent (for example a runtime helper) is out of scope, not a | ||
| // failure. A new thread whose group leader is armed is covered through the | ||
| // leader (see arming_task_armed), so its own mark is best effort. | ||
| static __always_inline bool arming_on_fork(struct task_struct *child, | ||
| __u64 clone_flags) | ||
| { | ||
| struct task_struct *parent = bpf_get_current_task_btf(); | ||
| struct task_struct *leader; | ||
| __u32 *a; | ||
|
|
||
| if (!arming_task_armed(parent)) | ||
| return true; | ||
| if (clone_flags & ARMING_CLONE_THREAD) { | ||
| leader = parent->group_leader; | ||
| a = leader ? bpf_task_storage_get(&armed_tasks, leader, 0, 0) : | ||
| NULL; | ||
| if (a && *a) { | ||
| if (!arming_fail_fork_marks) | ||
| arming_mark(child); | ||
| return true; | ||
| } | ||
| } | ||
| if (arming_fail_fork_marks) | ||
| return false; | ||
| return arming_mark(child); | ||
| } | ||
|
|
||
| #endif /* __ENFORCEMENT_ARMING_H */ | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.