Repository navigation
Forward-roll baseline is not fenced to an ancestor of the change under test #4447
Description
The released-artifact forward roll qualifies durable state by writing it with a published build and reading it with the build under test. Which published build it picks is resolved at run time, and nothing establishes that build as an ancestor of the change being tested.
resolveRegistryNightlyPredecessor (scripts/release-cli-publication.mjs:221) reads dist-tags.nightly from the registry and validates only that the version metadata's identity matches the tag it came from:
const version = packageMetadata?.['dist-tags']?.nightly;
parseProductNightlyVersion(version);
// ...
if (versionMetadata.name !== PACKAGE_NAME || versionMetadata.version !== version) {
throw new Error('Registry Nightly identity does not match its dist-tag');
}There is no check that the commit this Nightly was published from is reachable from the checkout under test.
How it goes wrong
A branch based on main@X enters CI. While that run is in flight — or simply because the branch has not been rebased — main advances to Y and publishes nightly(Y). The forward roll then writes state with nightly(Y) and reads it with a workspace whose code predates it. The roll runs backwards.
Two outcomes, and the quieter one is worse:
- False failure.
nightly(Y)writes a record the older workspace cannot decode. The job fails for a reason that has nothing to do with the change under review. - Silent no-op.
nightly(Y)already contains the very migration the change is meant to exercise, so both sides of the roll speak the same vocabulary. The job passes without having crossed a version boundary at all. Nothing distinguishes this from a real pass.
The second is the reason to fence it: a guard that quietly stops guarding reports the same green as one that works.
Reachability
Ordinary path — no adversary and no unusual timing required. Any pull request whose branch point predates the current Nightly is exposed, which over a day of merges is most of them.
Material already available for a fence
- npm version metadata carries
gitHeadfor packages published from a repository. - The CI checkout already uses
fetch-depth: 0(.github/workflows/ci.yml), sogit merge-base --is-ancestor <gitHead> HEADcan be evaluated in the job without an extra fetch. assertRegistryNightlyPredecessorin the same module already demonstrates the shape of a fence — it refuses a predecessor that is no longer current — but it is used by the release flow, not by the forward roll.
A fence would establish the baseline as an ancestor and fail loudly when it is not, rather than qualifying in whichever direction the registry happens to point.
Origin
Raised during review of the durable-state forward roll in #4427. That pull request moves the forward roll onto the main CI lane, which widens how often this baseline is resolved; it does not introduce the gap. Fixing it is a separate change with its own verification, so it is tracked here rather than folded in.
Activity
bytelazy commentedon Sep 3, 2026
Following up on @Flandern1211's question and @Astro-Han's review of #4604 (closed): the
gitHeadpremise in the issue is false for this package, so I want to correct the record before anyone else builds on it.gitHeadis not availableThe issue states npm version metadata carries
gitHeadfor packages published from a repository. That is generally true, but not for this publication path. I queried the registry: every recentmaka-agentnightly carriesgitHead: null.npm-publication.ymlpublishes withnpm publish "$NIGHTLY_TARBALL", and npm only injectsgitHeadwhen publishing from a git working directory. A tarball publish never carries it. This is structural, not intermittent — a fence built ongitHeadthrows on every run.Provenance is the candidate authority
The SLSA provenance attestation does carry the source commit. Decoding the attestation for the current nightly:
So the fence is buildable — just on a second registry endpoint (the attestation lookup) plus attestation parsing, not on a field that already exists in the version metadata.
The window is narrower than the issue implies
Worth putting on the table, as @Astro-Han raised on the PR: under
pull_request,actions/checkouttakesrefs/pull/N/merge(the PR head merged with the base tip). In the issue's main scenario — a branch that has not rebased and is well behindmain— the nightly's source commit is therefore already in the tree under test, and an ancestor fence neither fires nor needs to. What it actually catches is a nightly published tomainafter the run started. That is a real window, but a small one, and the failure it prevents is a CI-visible false failure, not a wrong artifact.Suggested next step
Decide whether that narrow window justifies the machinery before building more on it. If it does, the path is: provenance-based commit lookup,
fetch-depth: 0on the validation lane (currently depth 1, somerge-base --is-ancestorexits 128 on a genuine ancestor), and distinguishing exit 128 (cannot tell) from exit 1 (genuinely not an ancestor) in the fence.Happy to take either direction once there's a call on whether the fence is worth having at all.
(Verification: registry and attestation queries executed by me against the live registry; workflow reachability from reading
ci.yml,cli-package-validation.yml,npm-publication.yml. AI-assisted — drafted with Claude Code, the registry and attestation checks are mine.)github-actions commentedon Oct 9, 2026
This issue has had no human activity for 30 days and has been marked stale. It will be closed in 30 days unless someone comments.
If the issue is still current, please confirm it against the latest
mainand add any information that would help move it forward. Assigned issues and issues labelledpinnedare exempt from this policy.