[reyreavman]

Summary

An os-pm package whose files pm recorded under /lib64, /usr/lib64 or /usr/local/lib64 is no longer marked werf:shipped=false in the image that installed it, nor in an image that imports its libraries. On base-images v3.1.16 (builder/distroless, pm 0.1.12 — the first pm writing /var/lib/pm/files.json) a packages: os-pm install of curl marked libc, gnu-gcc-lib, libstdc++ and libunistring as not shipped in the donor itself, and a scratch image importing curl with its libraries additionally marked openssl, brotli and c-ares although their .so files were imported.

What

• A file pm recorded at /usr/lib64/x while the image symlinks /usr/lib64 to /usr/lib is evidence at /usr/lib/x, with the digest the image holds there; no occurrence names a path the image does not hold.
• Symlink chains between directories and a .. inside a symlink target resolve as the kernel walks them; a chain that never settles leaves the package without files, as before.
• Only the leaf keeps being a symlink when it is one: /usr/lib/libcurl.so.4 stays symlink evidence.
• The SBOM artifact format version is bumped, so an unchanged image whose SBOM an older werf generated gets a new SBOM on the next build instead of the cached one.
• VERIFIED: on builder/distroless v3.1.16 with curl==8.12.1 jq==1.8.1, the donor's 19 os-pm components all carry evidence and none is marked; the importing image leaves curl, libc, openssl, brotli, c-ares, nghttp2, libidn2, libpsl, zstd and libunistring unmarked and marks jq, gnu-gcc-lib, libstdc++, baselayout, busybox, tzdata, rsync, findutils and gnu-glibc-locale; a rebuild reuses the SBOM unchanged.
• VERIFIED: on the v3.0.2 base, whose pm writes index.json but no files.json, werf still warns and marks no os-pm package in either image.
• Python, JavaScript and Lua filesets do not change: the resolution runs only on the paths pm recorded.
• New e2e specs SBOM os-pm shipped scope (labels e2e, sbom, ospm, shipped-scope, simple) with fixtures inject/ospm_partial_import and inject/ospm_partial_import_no_filelist.

Why

pm 0.1.12 records every file under the path its package archive spells (recordPath(header.Name) in its untar), and the archives of libc, openssl, brotli, c-ares, gnu-gcc-lib, libstdc++ and libunistring spell lib64/usr/lib64, which baselayout lays down as symlinks to /usr/lib. The layer index werf builds knows the file only at the resolved path, collect looked the recorded path up literally, and a package with every file under such a directory ended with an empty occurrence list — which the rule reads as "looked for, found nothing" and marks. Left alone, every os-pm library of these base images is a false "not shipped" in the SBOM of every consumer, the signal the shipped-scope work exists to produce.

Resolving on the pm side (record the on-disk path) would also work, but every image already built with pm 0.1.12 carries the archive paths, and the index already holds the symlinks needed to follow them.

[reyreavman]

Verification

• task test:e2e paths=./test/e2e/sbom labelFilter=ospm parallel=1 on a Linux host with Docker and a local registry (Docker backend, no Buildah): 3/3 specs pass, 104 s. An exploratory build of the same fixture before the fix showed the marks the Summary describes, which is where the expected sets in the spec come from.
• Base image identified as builder/distroless of base-images v3.1.16 by its werf labels (werf-project-repo-commit=1b459c04, PACKAGES_VERSION=v3.1.16); pm --version inside it prints 0.1.12, and the spec asserts that.
• Mutation: resolveDirSymlinks returning its input unchanged → unit "finds a file the package manager recorded under a directory the image symlinks elsewhere" fails on /usr/lib/libc.so.6; e2e spec 1 fails "gnu-gcc-lib is installed, so its files are in the image and must be its evidence", spec 2 fails "a file of libc came along with curl, so it ships and must not be marked"; spec 3 (old base) unaffected.
• Mutation: maxDirSymlinkHops = 1 → unit fails "a directory symlink pointing at another directory symlink is followed to the end".
• Mutation: cleaning the symlink target with path.Join before walking it (the first version of the fix) → unit fails "a .. in a target steps out of what the symlink before it resolved to".
• Mutation: dropping the /usr/lib import from the fixture → spec 2 fails on libc.
• Not run: the e2e on GitHub Actions — the runners are currently failing at /run/user/998 before any job starts, independently of this PR.
• Not run: a hardlink recorded by pm as a second regular file — none of the 19 packages this install produces contains a hardlink (find / -xdev -type f -links +1 in the image is empty), so only pm's own unit tests cover that case.

Review focus

• resolveDirSymlinks in pkg/sbom/shippedscope/os_pm_fileset.go: a component walk with .., empty and . components handled inline and a hop budget instead of path.Clean; the unit test's dotdot, gcc and loop entries are the cases it exists for.
• The e2e pins the full set of 19 packages and the library file names curl links; a refresh of the base-image pin changes both and the spec is updated with it.
• Side observation, not changed here: the donor SBOM holds the base image's own packages twice, pkg:generic/baselayout@1.0?containerfactoryversion=v3.1.16 and …=latest (the base carries /var/lib/pm/container-factory-version = latest).

Follow-up

• base-images / pm (deckhouse/container-base/pm): consider recording the on-disk path instead of the archive path in files.json, so readers need not follow directory symlinks; this PR works either way.