SbomManager logs "container not found in shared data" at ERROR when a short-lived container exits early #848

Description

node-agent logs SbomManager - container not found in shared data at ERROR level for something that is not really an error: a container that started and then exited very quickly, before node-agent finished collecting the data it needs to scan it.

When that happens there is nothing left to scan (the container is already gone), so skipping the SBOM is the correct outcome. The problem is only that it is logged at ERROR. On a cluster that runs many short-lived pods (for example frequent jobs), every container that exits early produces an error line, so this turns into a steady stream of ERROR logs and false alerts for a harmless, expected race.

The line is here (current main, unchanged at the v0.3.142 release tag), pkg/sbommanager/v1/sbom_manager.go:250:

sharedData, err := s.waitForSharedContainerData(notif.Container.Runtime.ContainerID)
if err != nil {
    logger.L().Error("SbomManager - container not found in shared data",
        helpers.String("container ID", notif.Container.Runtime.ContainerID))
    return
}

waitForSharedContainerData retries with a backoff and then gives up; on failure the SBOM is skipped and nothing is requeued. Other "nothing to do" cases in the same file are logged at Debug, which is why this one stands out.

One caveat I want to be upfront about: the same code path is also hit if a still-running container's data never arrives (a genuinely stuck pipeline), which would be a real, silently dropped SBOM. So I am not sure simply lowering this to Debug is right. It may be better to tell the two cases apart: "container already gone" (harmless, low level) versus "data missing for a container that is still running" (keep it visible). I am reporting the symptom and leaving the remedy to you.

Environment

• OS: Talos Linux
• node-agent: v0.3.142 (quay.io/kubescape/node-agent:v0.3.142)

Steps To Reproduce

1. Run node-agent on a cluster that frequently creates very short-lived containers (for example fast-exiting jobs).
2. Tail node-agent logs at the default level.
3. Watch for recurring SbomManager - container not found in shared data ERROR lines whose container IDs belong to containers that exited almost immediately.

Expected behavior

A container that exits before its data is ready should not be logged as an ERROR, or it should be told apart from a genuine data-missing failure for a still-running container so that only the latter is an ERROR.

Actual Behavior

The harmless race is logged at ERROR, adding noise at the default log level. SBOM generation otherwise works for normal-lifetime containers.

Additional context

Observed on Talos with v0.3.142. SBOMs are produced normally for longer-lived workloads, which is what leads me to read this line as a harmless race rather than a real failure.