Skip to content

fix(ci): catalog publishing skips runs whose only change is vector noise - #5480

Merged
miguel-heygen merged 1 commit into
mainfrom
fix/catalog-publish-noop
Oct 12, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
fix/catalog-publish-noop

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What

"Publish catalog" on main stops failing with HTTP 422 "No commits between main and bot/catalog-publish", and stops opening pull requests whose only change is float noise in local-vectors.bin. It has failed on 9 of the last 12 main pushes, including the v0.8.147 release merge.

Why it failed

Regenerating the catalog rewrites local-vectors.bin with tiny float differences. sameVectors already treats those as equal, but publish() only cleared when zero files changed, so a noise-only run was treated as a real publication. That broke in two ways:

  1. Branch missing. Branches are deleted on merge, so after a publish PR merges, the next noise-only run commits the noise and opens a PR. That is how chore(catalog): publish generated catalog #5435 came to change only local-vectors.bin (+0/-0).
  2. Branch is plain main. When that PR is closed, the branch is reset to main. On the next noise-only run, keptSnapshotBase sees no generated difference and opens a PR from a branch with no commits of its own, and GitHub answers 422.

Signature in the failing logs: Catalog publication: 1 files, 1 signed commit batches., followed by the 422.

How

publish() now decides "nothing to publish" by comparing the regenerated catalog with main through the existing sameGenerated check (vector noise counts as equal), instead of counting changed files. A noise-only run takes the same path as a no-change run: close any open publish PR and move the branch to main. Real changes, including real vector changes, publish as before.

Test

Three new cases in scripts/catalog-publication.test.mjs:

  • Noise over a standing branch already in main: the only write is the branch reset.
  • Noise after the branch was deleted: no writes at all.
  • Vectors that point elsewhere than main's: the run still publishes.

Results:

  • 24/24 pass, 3 runs in a row.
  • With main's publish-catalog.mjs, both noise cases fail (22 pass, 2 fail).
  • With the check forced to always clear, the "still publishes" case fails (with 6 existing publish tests).
  • oxfmt --check, oxlint and fallow audit --base origin/main are clean.

Known and left alone: on a noise-only run, the log still prints "Catalog publication: 1 files" before "No unpublished catalog changes", and --dry-run stops before the new check.

Follow-up worth doing separately: the "nothing to publish" decision now uses sameGenerated's tree diff, while the publish list still comes from catalogChanges. CI's fresh checkout keeps the two in agreement. Deriving both from one diff would leave a single owner.

@somanshreddy somanshreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 77e81d1c: no blockers, approving.

What I checked

  • The failure is real. The latest failing catalog-publish.yml run on main (30359f80) logs Catalog publication: 1 files, 1 signed commit batches. and then 422 "No commits between main and bot/catalog-publish". 9 of the last 12 runs on main failed.
  • sameGenerated(root, base) loses no real change compared with batches.length === 0. catalogTree is read-tree HEAD plus git add -A -- GENERATED_CATALOG_PATHS, so it covers the same untracked files (gitignore respected) and deletions that catalogChanges gets from diff plus ls-files --others --exclude-standard. The check returns true only when every difference is an M on a vectors file and sameVectors agrees. New files, deletions, metadata changes, and vectors that point elsewhere all still publish.
  • Clearing on noise is safe. sameGenerated(base) only holds when main's committed catalog already equals what regenerating gives, apart from noise. Any open publish PR is obsolete then, exactly as in the old zero-files case. A pending real publication can't be dropped: regenerating from main's source reproduces it, so the check is false.
  • Tests: scripts/catalog-publication.test.mjs passes 24/24.
  • Mutants:
Mutant Result
Revert to batches.length === 0 Both noise tests fail
sameGenerated ignores sameVectors (any vector change counts as noise) "point elsewhere" and "still publishes" fail
batches.length === 0 || sameGenerated(...) Passes, as expected: an empty diff already makes sameGenerated true, so the new check covers the old one

Non-blocking: This repeats the follow-up you already named: the clear decision and the publish list now come from two different diffs. A single owner would stop them drifting outside a fresh CI checkout.

— Somu

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 12, 2026
Merged via the queue into main with commit 5f9db98 Oct 12, 2026
60 checks passed
@miguel-heygen
miguel-heygen deleted the fix/catalog-publish-noop branch October 12, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants