From 164feff88eef2a884d7127ba5883ddd89d2a6487 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Wed, 1 Jul 2026 18:08:10 +0200 Subject: [PATCH] chore(publish): verify with cargo package, not cargo publish --dry-run (#146) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `scripts/publish.rs::verify` already used `cargo package` per-crate, but that still hits the crates.io chicken-and-egg for dependent crates on a first publish of a new version: cargo verifies each packaged crate outside the workspace, downloads its deps from the index, and fails when the new version isn't published yet. Empirically confirmed at an unpublished 0.99.0 — this is why PR #192 removed the CI verify step. Fix: package the whole publishable set in ONE `cargo package -p a -p b …` invocation. Cargo writes every member to target/package/*.crate first, then verify-builds each against those local tarballs at the new version, never touching the index. Confirmed at 0.99.0: dependents (synth-synthesis etc.) compile against the just-packaged deps with nothing on crates.io. Full verify build retained (no --no-verify), so non-compiling code is still caught. Also: - Add synth-backend-aarch64 to CRATES_TO_PUBLISH. It is a hard, always-on dep of synth-cli (#538) but was never published — publish would (and the restored verify does) fail on synth-cli's unresolved dep. The restored fail-fast check is exactly what surfaced this latent gap. - Restore the `./publish verify` pre-flight step in publish-to-crates-io.yml and rewrite the now-false comment (it asserted cargo package cannot avoid the chicken-and-egg; it can, when the whole set is packaged together). - Update docs/release-process.md: describe the token-free verify pre-flight and add synth-backend-aarch64 to the published-crate table. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/publish-to-crates-io.yml | 28 +++++----- docs/release-process.md | 13 ++++- scripts/publish.rs | 62 ++++++++++++++-------- 3 files changed, 66 insertions(+), 37 deletions(-) diff --git a/.github/workflows/publish-to-crates-io.yml b/.github/workflows/publish-to-crates-io.yml index a42e789f..778d9bfd 100644 --- a/.github/workflows/publish-to-crates-io.yml +++ b/.github/workflows/publish-to-crates-io.yml @@ -82,19 +82,21 @@ jobs: - name: Build publish helper run: rustc --edition 2024 scripts/publish.rs -o publish - # NOTE: there is intentionally NO pre-flight `./publish verify` step. - # Both `cargo publish --dry-run` AND `cargo package` resolve the path-dep - # *version requirements* (`synth-core = "^0.11.x"`) against the crates.io - # index when preparing the upload, so a dependent crate fails with - # "failed to select a version for synth-core = ^0.11.x" on the FIRST - # publish of any new version (the deps aren't on the registry yet). This - # is the same chicken-and-egg that sank the v0.7.0 verify step (dropped in - # #144). #146 proposed `cargo package` to avoid it, but it does NOT — it - # only passes when the version is already published. Re-tracked in #146. - # Compile errors are already caught by the CI test/build (path deps, no - # chicken-and-egg); `./publish publish` below validates each crate's - # metadata as cargo's own pre-upload check, with a 10-attempt retry loop - # (40s sleep) that rides out registry index propagation between deps. + # Pre-flight fail-fast (#146). `cargo publish --dry-run` resolves every + # path-dep *version requirement* against the crates.io index, so on the + # FIRST publish of any new version a dependent fails with "failed to + # select a version for synth-core = ^x.y" (the deps aren't on the + # registry yet) — the chicken-and-egg that sank the v0.7.0 verify step + # (dropped in #144). A *per-crate* `cargo package -p ` fails the + # same way. `./publish verify` avoids it by packaging the whole + # publishable set in ONE `cargo package -p a -p b ...` invocation: + # cargo writes every member to target/package/*.crate first, then + # verify-builds each against those local tarballs at the NEW version, + # never touching the index (empirically confirmed at an unpublished + # 0.99.0). This catches broken metadata / missing README / bad + # include-exclude / code that doesn't compile before any upload. + - name: Verify publishable crates package cleanly + run: ./publish verify - name: Publish workspace to crates.io run: ./publish publish env: diff --git a/docs/release-process.md b/docs/release-process.md index 0d38ab1c..34db0ee0 100644 --- a/docs/release-process.md +++ b/docs/release-process.md @@ -215,10 +215,18 @@ The maintainer asked for the rollout to be staged. The committed is in `scripts/publish.rs` (compiled at workflow start by `rustc`). Final user-visible result: `cargo install synth-cli` works after the first release that runs this pipeline. +- **Pre-flight check (#146).** Before publishing, the workflow runs + `./publish verify`, which packages the whole publishable set in one + `cargo package -p a -p b …` invocation. Packaging every member together + lets cargo verify-build each against the locally packaged tarballs at + the new version, so it sidesteps the crates.io chicken-and-egg that a + per-crate `cargo package` (or `cargo publish --dry-run`) hits on a first + publish, while still catching broken metadata and non-compiling code. + This needs no registry token. - **Trust model:** an org-wide `CRATES_IO_TOKEN` secret on the `pulseengine` GitHub organization, inherited by this repo. The - workflow exports it as `CARGO_REGISTRY_TOKEN` for `./publish verify` - and `./publish publish`. Migration to OIDC trusted publishing (to + workflow exports it as `CARGO_REGISTRY_TOKEN` for `./publish publish`. + Migration to OIDC trusted publishing (to match sigil) is tracked as future work — see "Phase 4 — auth model" below; we chose the token path first because the org secret already exists and OIDC requires per-crate trusted-publisher registration @@ -239,6 +247,7 @@ The maintainer asked for the rollout to be staged. The committed | `synth-frontend` | yes | dep of CLI | | `synth-synthesis` | yes | dep of CLI | | `synth-backend` | yes | dep of CLI | + | `synth-backend-aarch64`| yes | hard, always-on dep of CLI (#538) | | `synth-backend-awsm` | yes | optional dep of CLI | | `synth-backend-wasker` | yes | optional dep of CLI | | `synth-backend-riscv` | yes | default optional dep of CLI | diff --git a/scripts/publish.rs b/scripts/publish.rs index dcac8620..ce623398 100644 --- a/scripts/publish.rs +++ b/scripts/publish.rs @@ -1,8 +1,10 @@ //! Helper script for publishing the synth workspace to crates.io. //! //! Modes: -//! ./publish verify — `cargo package` every publishable crate (#146) -//! in dependency order, fail on first error. +//! ./publish verify — `cargo package` all publishable crates in ONE +//! invocation (#146), so inter-member deps resolve +//! against the locally packaged tarballs, not the +//! crates.io index. Fails if any crate fails. //! ./publish publish — `cargo publish` every publishable crate in //! dependency order; retries up to 10 times to //! ride out crates.io index-propagation lag. @@ -40,6 +42,11 @@ const CRATES_TO_PUBLISH: &[&str] = &[ "synth-synthesis", // Depend on the above. "synth-backend", + // Host-native AArch64 backend (#538). A hard, always-on dependency of + // synth-cli, so it must be published too — otherwise synth-cli fails + // to resolve `synth-backend-aarch64` against the index. The restored + // `./publish verify` step (#146) is what surfaced this omission. + "synth-backend-aarch64", "synth-backend-awsm", "synth-backend-wasker", "synth-backend-riscv", @@ -102,28 +109,39 @@ fn read_workspace_version() -> String { } fn verify(crates: &[Krate]) { - // #146: use `cargo package` rather than `cargo publish --dry-run`. - // `--dry-run` resolves every workspace dep against crates.io, so for a - // first publish of any new version the dependents always fail with the - // chicken-and-egg "dep not yet published" error (this sank the v0.7.0 - // verify step, dropped in PR #144). `cargo package` resolves through the - // local path deps, builds the `.crate` end-to-end, and still catches the - // surface a pre-flight check should: missing README/license, bad - // include/exclude, broken metadata, and code that doesn't compile. - let mut failed = Vec::new(); + // #146: use `cargo package` rather than `cargo publish --dry-run`, and + // package the whole publishable set in ONE invocation (`cargo package + // -p a -p b ...`) instead of once per crate. + // + // `cargo publish --dry-run` resolves every workspace dep against + // crates.io, so on a first publish of any new version the dependents + // fail with the chicken-and-egg "dep not yet published" error (this + // sank the v0.7.0 verify step, dropped in PR #144). + // + // A *per-crate* `cargo package -p ` has the SAME problem for + // dependents: cargo verifies the packaged crate outside the workspace, + // so it downloads each dep from crates.io and fails when the new + // version isn't there yet. Empirically confirmed at an unpublished + // 0.99.0 (see PR for #146). + // + // Packaging the whole set in one invocation fixes it: cargo packages + // every selected member into `target/package/*.crate` first, then + // verify-builds each against those local tarballs — so `synth-cli` + // compiles against the just-packaged `synth-core` etc. at the NEW + // version, never touching the index. Full verify build is retained + // (no `--no-verify`), so "code that doesn't compile" is still caught. + let mut args: Vec = vec!["package".to_string()]; for k in crates { - println!("--- cargo package -p {} ---", k.name); - let status = Command::new("cargo") - .args(["package", "-p", &k.name]) - .status() - .expect("invoke cargo"); - if !status.success() { - println!("FAIL: {} package failed: {status}", k.name); - failed.push(k.name.clone()); - } + args.push("-p".to_string()); + args.push(k.name.clone()); } - if !failed.is_empty() { - eprintln!("\nverify failed for: {}", failed.join(", ")); + println!("--- cargo {} ---", args.join(" ")); + let status = Command::new("cargo") + .args(&args) + .status() + .expect("invoke cargo"); + if !status.success() { + eprintln!("\nverify failed: cargo package exited with {status}"); std::process::exit(1); } println!("\nAll {} crates passed cargo package.", crates.len());