Skip to content

fix(sbom): avoid marking editable Python packages as absent - #474

Merged
reyreavman merged 2 commits into
mainfrom
fix/sbom/editable-python-evidence
Oct 26, 2026
Merged

reyreavman merged 2 commits into
mainfrom
fix/sbom/editable-python-evidence

Conversation

@reyreavman

Copy link
Copy Markdown
Collaborator

Summary

Editable Python packages are no longer marked werf:shipped=false merely because an importing image lacks their installation hooks. Previously, an editable donor's RECORD could list only a .pth file and metadata; importing the actual source without those files then produced a false absence verdict.

What

  • Python file evidence is incomplete when direct_url.json reports dir_info.editable=true; malformed origin metadata, unreadable symlinked origins and RECORD entries naming __editable__* or .egg-link scaffolding also preserve uncertainty.
  • Valid non-editable origins, an omitted editable flag and ordinary wheel installs retain their existing file matching. A generic .pth name alone does not classify an install as editable.
  • Incomplete file evidence writes werf:files=unknown on the component. Copying only metadata or hooks does not become complete evidence of the external source tree.
  • Unknown file evidence survives serialization, same-PURL merges and imports, even beside another origin's checked-empty list or old occurrences; those digests may remain as provenance but do not determine the verdict.
  • A later complete install does not erase inherited uncertainty for the same component: it cannot prove that another origin's source tree was absent.
  • SBOM artifact format 16 invalidates previous generated SBOM caches without changing image-stage cache keys. Existing external donor SBOMs without identifiable editable provenance cannot be repaired retroactively by this change.
  • No component is deleted, no Python hooks or source code are executed for evidence discovery, and no new configuration or runtime dependency is added. Other package ecosystems and Python bytecode handling are unchanged.

Why

PEP 660 installations intentionally expose source outside site-packages. A RECORD may enumerate every installed hook while omitting the actual source payload, so its file list cannot prove that payload absent. Simply omitting the editable fileset is insufficient: evidence from another copy or imported SBOM of the same package could restore false certainty. Persisting uncertainty follows the existing binary-evidence model and keeps that distinction across merges and caches.

Treat editable installation records as incomplete payload evidence. Preserve uncertainty across SBOM merges and imports instead of inferring absence from missing hooks, and regenerate artifacts written before that distinction existed.

Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
Signed-off-by: Radmir Khurum <radmir.khurum@flant.com>
@reyreavman

Copy link
Copy Markdown
Collaborator Author

Verification

  • The initial image → filesets → evidence → JSON → import regression was red in all three cases: only source copied, only a path hook copied, and no package content copied. It is green after the fix.
  • Real Linux VM e2e: task test:e2e paths=./test/e2e/sbom labelFilter=editable parallel=1 -- -no-color -timeout=15m passed. The spec checks four images, real pip install -e versus an ordinary local wheel, two import levels, PURLs/licenses, unknown versus absent evidence and a second cached build.
  • The first e2e run exposed byte-identical editable/ordinary fixture payloads: the existing content matcher legitimately retained the ordinary control. Only its fixture payload was made unique; no assertion was weakened. The rerun passed.
  • Mutation: removing direct-url capture, ignoring incomplete empty RECORDs, ignoring the persisted unknown verdict, dropping inherited uncertainty, disabling the scaffold fallback, ignoring symlinked origin metadata, or reverting cache version 16 to 15 each failed assertions. Restored source passed both full affected unit suites.
  • Full local format/build/lint/unit checks passed. Two independent review passes covered production/unit semantics and end-to-end fixtures; blocking findings are resolved.
  • The VM integration attempt was refused by the wrapper because another executor had acquired the shared slot. That run was not interrupted or overwritten. Integration and full regression coverage remain assigned to the existing PR CI jobs; no passing CI result is claimed yet.

Review focus

  • werf:files=unknown is intentionally persistent across same-package origins. Check the unknown/none/property-order cases rather than interpreting a retained occurrence as complete evidence.
  • This covers PEP 660/610 metadata and recognizable scaffolding, not arbitrary editable installers that omit all such evidence or retroactive interpretation of old external donor SBOMs.

Follow-up

  • BLOCKER: confirm the required CI jobs, including integration and e2e, before considering the draft ready. Do not merge automatically.

@reyreavman
reyreavman marked this pull request as ready for review October 26, 2026 09:16
@reyreavman
reyreavman merged commit e0b4b66 into main Oct 26, 2026
8 checks passed
@reyreavman
reyreavman deleted the fix/sbom/editable-python-evidence branch October 26, 2026 09:16
reyreavman pushed a commit that referenced this pull request Oct 26, 2026
🤖 I have created a release *beep* *boop*
---


##
[3.12.0-dk.3](v3.12.0-dk.2...v3.12.0-dk.3)
(2026-10-26)


### Features

* **build, stapel:** sign files matched by a glob pattern
([#471](#471))
([ffda24e](ffda24e))
* **sbom:** inventory installed Python, JavaScript and Lua packages
([#472](#472))
([7fde219](7fde219))


### Bug Fixes

* **build, stapel, import:** stop imports failing with sbom enabled
([#470](#470))
([478b751](478b751))
* **sbom:** avoid marking editable Python packages as absent
([#474](#474))
([e0b4b66](e0b4b66))
* **sbom:** retain legacy Python bytecode after source removal
([#476](#476))
([36debcb](36debcb))
* **sbom:** stop marking os-pm packages under /lib64 as not shipped
([#475](#475))
([cd0a029](cd0a029))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant