Upgrade Inspektor Gadget to v0.57.0 via kubescape fork and preserve exec/path semantics #1050

Overview

Upgrade Inspektor Gadget to the latest release, v0.57.0 as of 2026-10-26, through our kubescape/inspektor-gadget fork. Keep the fork replacement and pin a merged fork commit after integrating upstream; do not bypass the fork by switching directly to the upstream Go module.

Current node-agent main (1051d7b7) separately pins the runtime to fork commit 06b0d12baca0 in go.mod and the bundled upstream gadgets to v0.48.1 in Makefile. Updating only the Go dependency will not upgrade trace_exec bytecode.

Problem and verified improvement

The old exec gadget can lose the pathname and arguments when syscall-entry user-memory reads fault, even though exec itself succeeds. This creates false positives for rules interpreting missing arguments as a bare shell, and prevents command-based rules from seeing the command.

Upstream fixed successful-exec capture in PR #5600, first released in v0.54.0: arguments are read from the new process's mm->arg_start/arg_end at sched_process_exec.

An A/B test on an isolated Ubuntu 24.04 amd64 host, kernel 6.8.0-142-generic, glibc 2.39 and musl 1.2.4, produced:

The cold-page cases retained valid file mappings and dropped present PTEs with MADV_DONTNEED; an actually unmapped pathname was a separate negative control and correctly failed exec with EFAULT. Warm, popen, and invalid-pointer controls were also included. All four runs passed: 1,080 attempts, including 480 successful v0.57.0 execs with complete expected argv.

This used the production ExecTracer and existing node-agent argument decoder in a harness, with runtime fork commit 056ddc9e059f held fixed and only the exec gadget swapped. It was not a full daemon/rules/application replay or full SDK-upgrade test. Published v0.57.0 image digest: sha256:9182278eb6a11b6e2d59a748674b646a8512b715d3563ec3afdf340a2f55d435. The current upstream main and v0.57.0 have identical exec BPF source.

Compatibility caveats

Confirmed behavior and repository contracts

1. args[0] becomes actual argv[0], rather than the requested pathname. For glibc system, the new event is ["sh", "-c", "--", ":"], while the legacy contract puts "/bin/sh" first. GetExecPathFromEvent currently treats args[0] as the executable path. Audit CEL/rules, alert adapters, profiles and process-tree command lines. Preserve the outward contract or explicitly migrate its consumers; keep requested pathname, resolved executable path and argv distinct. Include custom/spoofed argv[0], symlinks and relative paths in tests.

2. Scripts and capture limits change too. Successful script execs now expose committed interpreter/script argv, rather than just the original exec-call arguments. Successful capture is bounded by a 5,120-byte block, replacing the old per-argument/count limits; args_count is set to a byte-count upper bound on this path. Test shebangs, long individual arguments, many/empty arguments, truncation, execveat, failed execs and non-leader-thread execs. These are described in #5600; post-exec recovery does not recover failed execs.

3. Event schemas and derived fields need updating. Newer exec gadgets add TTY device-number and inode-time fields. TestExecFields rejects unexpected fields, so update the expectations and validate consumers against the rebuilt bundle. Verify /dev/pts/0 detection using tty_major/tty_minor; this upgrade also addresses the gadget-version prerequisite in hasTty is false on /dev/pts/0: bump trace_exec gadget to v0.55.0+ #975. Keep missing/partial capture distinct from a genuinely argument-less shell.

4. Preserve fork changes and the vendored trace_open patch. Forward-integrate upstream into the active fork without losing its uprobe reattach/cache/streaming, non-Kubernetes namespace filtering, overlayfs inode, bounded-buffer/startup-budget, and mapped-library discovery/retry/lifecycle fixes (merged fork PRs). The vendored open gadget is built locally under the upstream image name and a pinned builder digest; replacing it with an upstream pull would drop the local path-resolution behavior.

Integration checks still required

5. Coordinate runtime, images and build tooling. Update the fork Go pin, hardcoded tracer image references, Makefile versions, CLI/builder pins and CI, and rebuild the complete tracers.tar and custom gadgets. Upstream changed gadget build internals (#5469) and releases OCI/spec/object resources earlier (#5462, #5461, #5466); audit fork/custom-operator assumptions and load/unload behavior. The gadget-only A/B run does not prove SDK compatibility.

6. Audit the Go/API and image-verification upgrades. Reconcile v0.57.0's module graph and replacements, including Docker client migration (#5414); dependency-module replace directives do not propagate to the consuming module. Check newer WASM layer handling, cosign v3 bundle signatures (#5308) and SBOM OCI referrers (#5495) with our actual fork-built, bundled/offline and mirrored image paths. The old runtime skipped the newer WASM layer in the smoke test and used node-agent's own exec operator; that is not validation of the full new operator/signature stack.

7. Check discovery assumptions if adopting the corresponding upstream paths. Automatic runtime-socket discovery now uses direct kubelet /configz and nodes/configz RBAC (#5343); verify applicability, connectivity and fallback. Upstream's reduced-capability mode cannot use fanotify without CAP_SYS_ADMIN (#5637); an upgrade alone is not evidence that node-agent can drop capabilities while still discovering new containers.

Acceptance criteria

• Integrate the chosen upstream release into kubescape/inspektor-gadget, preserve fork fixes, and pin node-agent to a merged fork commit; pass scripts/check-inspektor-gadget-pin.sh.
• Update runtime/image/builder/CLI pins together; retain the patched open gadget and rebuild all bundled/custom gadgets.
• Resolve and test executable-path versus argv compatibility across rule, alert, profile and process-tree consumers.
• Add regression coverage for the libc/cold-page cases and exec edge cases above; verify the TTY case in hasTty is false on /dev/pts/0: bump trace_exec gadget to v0.55.0+ #975 and updated field schemas.
• Run build/unit/component checks and full-agent validation on supported kernels and native amd64/arm64, including verifier loading, discovery, restart/reattach, and bundled-image verification. Preserve explicit behavior for unavailable/truncated arguments.

Alternatives and related work

A narrow backport of upstream PR #5600 into the fork is a possible interim fix, but it has the same exec-semantics caveats and would not replace the broader upgrade. Related: #975 (hasTty on /dev/pts/0) and #429 (image-based gadget integration).