[milantracy]

In a pod, all processes of the pod run in the cgroup of the sandbox. runsc configures that cgroup from the sandbox spec: the spec of the infrastructure (pause) container in a CRI pod, which has no swap limit. Kubernetes sets the swap limit on each application container instead (memory.swap.max=0 under the default NoSwap behavior on cgroup v2). runsc applied these limits only to the per-container compat cgroups, which have no processes. On a node with swap, a gVisor pod thus swapped without limit after it reached the memory limit of the pod cgroup, where a runc pod is OOM-killed. The swap support for pre-created cgroups (b393e6f) reads the pause container spec, so it does not apply here.

Set memory.swap.max of the sandbox cgroup to the sum of the swap limits of the application containers, but not more than the swap limit of the sandbox spec, when a container starts, changes its resources, or is destroyed. If a container does not limit swap, only the sandbox spec limits it. This applies to CRI pods and to sandboxes without a root container. With the systemd driver, set the MemorySwapMax unit property. Hosts without swap accounting are skipped. Rootless permission errors are tolerated, as for the other cgroup settings, and a failure while a container is deleted is only logged.

COMPATIBILITY:NEW-APP-CONFIG: Kubernetes nodes with swap and the default NoSwap swap behavior (also LimitedSwap), as reported in #15756 on kind with both the systemd and cgroupfs drivers. gVisor pods ignored the swap limits.

Fix #15756
Assisted-by: Claude Opus 5.5

[heyvaldemar]

Tested on the reproducer from #15756, on arm64, against the same tree without this change.

Build: the diff at 8c867f3 applied to the go branch at 1cea747 (master 7a675ac), with runsc and its sidecars built with go build for linux/arm64. The baseline is the same tree without the diff. Node: kind v0.33.0, kindest/node:v1.37.0, kernel 7.0.12, 1 GiB of swap. The pod is the hog from the issue with its 128Mi limit; the loop is capped at 512 MiB so the baseline does not exhaust the node's swap.

Node	Build	memory.swap.max of the cgroup runsc runs in	Last MiB printed	Ended
systemd, NoSwap	baseline	max	512	Completed
systemd, NoSwap	this PR	0	128	OOMKilled
systemd, LimitedSwap	baseline	max	512	Completed
systemd, LimitedSwap	this PR	8658944	144	OOMKilled
cgroupfs, NoSwap	baseline	max	512	Completed
cgroupfs, NoSwap	this PR	0	128	OOMKilled

Under LimitedSwap, 8658944 is the kubelet's budget for the container's 64Mi request on this node (64Mi / 8124000Ki of memory x 1 GiB of swap, rounded down to a page), now on the sandbox cgroup. In every run that cgroup held the pod's 8 or 9 processes, and the container's own cgroup stayed empty.

A note on the build, for anyone who tries it the same way: on the go branch, pkg/sentry/platform/systrap/stubfilter_precompiled.go does not match arm64, and a go build sentry panics at start with cannot render precompiled seccomp program "systrap_sysmsg_thread": called with inconsistent vars. I regenerated it on arm64 with precompile_gen.go from master, identically for both builds.

[heyvaldemar]

Re-tested at c2c3f32 on the reproducer from #15756, the same way as in my earlier comment: the diff applied to the go branch at c24795a (master 2552c67), with runsc and its sidecars built with go build for linux/arm64 and stubfilter_precompiled.go regenerated for arm64, against the same tree without this change. Rootless was not tested: every run uses runsc as root under containerd, so the rootless path changed in this version was not exercised.

Node: kind v0.33.0, kindest/node:v1.37.0, kernel 7.0.12, 1 GiB of swap. hog is the pod from the issue (64Mi request, 128Mi limit, loop capped at 512 MiB). hog + idle adds a container with a 32Mi request and a 64Mi limit.

Node	Pod	Build	memory.swap.max of the cgroup runsc runs in	Last MiB printed	hog ended
systemd, NoSwap	hog	baseline	max	512	Completed
systemd, NoSwap	hog	this PR	0	128	OOMKilled
systemd, LimitedSwap	hog	baseline	max	512	Completed
systemd, LimitedSwap	hog	this PR	8658944	144	OOMKilled
systemd, LimitedSwap	hog + idle	baseline	max	512	Completed
systemd, LimitedSwap	hog + idle	this PR	12992512	192	OOMKilled
cgroupfs, NoSwap	hog	baseline	max	512	Completed
cgroupfs, NoSwap	hog	this PR	0	128	OOMKilled
cgroupfs, LimitedSwap	hog + idle	baseline	max	512	Completed
cgroupfs, LimitedSwap	hog + idle	this PR	12992512	192	OOMKilled

12992512 is the sum of the kubelet's budgets for the two containers (8661801 + 4330900 bytes), rounded down to a page. With the systemd driver, the MemorySwapMax property of the sandbox scope is 0, 8661801 and 12992701 (infinity in the baseline). In every run that cgroup held all of the pod's processes, and the containers' own cgroups stayed empty.

After a container goes away: with the loop in hog capped at 64 MiB instead, so that it exits on its own while idle keeps running, crictl rm of the exited hog lowered memory.swap.max of that cgroup from 12992512 to 4329472, the budget of idle alone, on both drivers. MemorySwapMax went from 12992701 to 4330900. In the baseline it stayed max.