Skip to content

Add opt-in manual prerelease publish for library NuGet packages - #851

Merged
Alexandre Zollinger Chohfi (azchohfi) merged 8 commits into
mainfrom
zateutsch-zt/nuget-prerelease-pipeline
Sep 21, 2026
Merged

Alexandre Zollinger Chohfi (azchohfi) merged 8 commits into
mainfrom
zateutsch-zt/nuget-prerelease-pipeline

Conversation

@zateutsch

@zateutsch Zach Teutsch (zateutsch) commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds an opt-in, manual way to publish prerelease versions of the three source-built library NuGet packages to nuget.org, so they can be validated before the first official release:

  • Microsoft.Windows.SDK.BuildTools.WinApp.UIAutomation
  • Microsoft.Windows.SDK.BuildTools.WinApp.UIAutomation.Recording
  • Microsoft.Windows.SDK.BuildTools.WinUIAnalyzer

How

Queue WinDevCLI - Release manually with the new publishPrereleaseNugets parameter ticked (optionally set prereleaseNugetVersion; blank auto-computes version.json + -prerelease.<build>). This adds two stages — Prerelease_Nugets_Build and Prerelease_Nugets_Publish — that pack only those three packages (package-nuget.ps1 -SkipCliPackage, so the CLI tools package is never built), ESRP-sign them, and push only them via the NuGet-WinAppCLI service connection.

The two stages are self-contained in release.yml with inline ESRP signing; build.yaml is untouched.

Why a parameter on the existing pipeline (not a new pipeline or branch)

  • Reuses the release pipeline's service connections, variable groups, and 1ES onboarding — the only ADO wiring needed is allowing the run branch on the Branch control checks for NuGet-WinAppCLI + the signing connection.
  • Stable library publishing is unchanged — it still ships through the normal rel/v* Release_NuGet stage.

Safety model

The stages are gated at compile time so they can't misfire:

  • eq(parameters.publishPrereleaseNugets, true) — defaults false, so the stages are pruned from the graph on every scheduled rehearsal and every real rel/v* release. They exist only when a human explicitly ticks the box.
  • eq(parameters.DoEsrp, true) — the path is pruned entirely when signing is off, so it can never publish unsigned packages.
  • not(startsWith(Build.SourceBranch, 'refs/heads/rel/v')) — belt-and-suspenders: even a manual queue on a release branch publishes nothing, so it can never ride a real release.

The prereleaseNugetVersion value is passed via an environment variable (not spliced into the inline script, avoiding PowerShell injection from a queue-time value) and validated against a strict SemVer-prerelease pattern, so a stable version — including stable-with-build-metadata like 0.7.0+build-1 — cannot be published through this path.

Prerelease_Nugets_Build also runs the targeted test suites for exactly these packages (WinApp.UIAutomation.Tests and the WinUI analyzer tests) before packing, so a test-failing commit can't be signed or published — while staying decoupled from the full ~60-min stable Build.

Incidental fix: over-broad assertion in the winui-app sample test

samples/winui-app/test.Tests.ps1:154 asserted Should -Not -Match 'release-' on the whole CLI output to check that the release publish profile wasn't selected. That scans the banner too, and on a branch build the banner prints a version that embeds the branch name (here 0.6.3-...-prerelease-pipeline.1), which contains release- and false-fails the test. Tightened it to match the resolved PublishProfile=release- token specifically (mirroring the positive assertion on the line above), so it still catches a real release-profile regression without false-matching the version string. This is a latent bug that would fail any branch whose name contains release-; it is unrelated to the pipeline change but was surfacing as red CI on this branch.

Review notes

Went through the repo's pr-review skill. It surfaced two HIGH findings (unsigned-publish reachable with DoEsrp=false; version-parameter injection + weak prerelease guard); both are fixed here and confirmed clean on re-review by an independent different-model cross-check. A reviewer also flagged that the decoupled build skipped tests — addressed with the targeted test gate above.

Not yet validated: an actual ADO template compile / preview and a real nuget.org push — api.nuget.org is unreachable from the corp dev environment. A dry compile in Azure DevOps on this branch is worth doing before first real use.

Files

  • .pipelines/release.yml — two parameters + the two self-contained prerelease stages (with the library test gate)
  • .pipelines/README.md — "Publishing a prerelease of the library packages" section
  • samples/winui-app/test.Tests.ps1 — tighten the release-profile assertion (incidental CI fix)

Adds a publishPrereleaseNugets parameter to release.yml that, when ticked on a
manual, non-rel/v* run, packs the three source-built library packages
(UIAutomation, UIAutomation.Recording, WinUIAnalyzer) at a prerelease version,
ESRP-signs them, and pushes only them to nuget.org via NuGet-WinAppCLI. This
enables validating those packages before the first official release.

The two stages are self-contained in release.yml (signing inline; build.yaml is
untouched). They are gated on the parameter (default false), so they are pruned
at compile time on the weekly rehearsal and every real rel/v* release. The gate
also requires DoEsrp=true, so the path can never publish unsigned packages, and
not(rel/v*), so it can never ride a real release. Stable library publishing is
unchanged: it still ships through the normal rel/v* Release_NuGet stage.

The prereleaseNugetVersion parameter is passed via an environment variable rather
than spliced into the inline script (avoiding PowerShell injection from a
queue-time value) and is validated against a strict SemVer prerelease pattern, so
a stable version - including stable-with-build-metadata like 0.7.0+build-1 -
cannot be published through this path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 15, 2026 18:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The production publishing path requires an Azure DevOps template preview before approval.

Pull request overview

Adds an opt-in Azure DevOps workflow for signing and publishing prerelease library NuGet packages without affecting stable releases.

Changes:

  • Adds guarded prerelease build and publish stages.
  • Validates prerelease versions and signs binaries/packages.
  • Documents operation and safety constraints.
File summaries
File Description
.pipelines/release.yml Adds parameters and prerelease publishing stages.
.pipelines/README.md Documents the manual publishing workflow.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Build Metrics Report

Validation passed. All required build and validation jobs succeeded.

Binary Sizes

Artifact Baseline Current Delta
CLI (ARM64) 57.03 MB 57.03 MB ✅ 0.0 KB (0.00%)
CLI (x64) 57.07 MB 57.07 MB ✅ 0.0 KB (0.00%)
MSIX (ARM64) 23.72 MB 23.72 MB 📉 -0.7 KB (-0.00%)
MSIX (x64) 25.16 MB 25.16 MB 📈 +0.2 KB (+0.00%)
NPM Package 49.44 MB 49.44 MB 📉 -0.7 KB (-0.00%)
NuGet Package 49.55 MB 49.55 MB 📉 -0.0 KB (-0.00%)

.NET Test Results (TRX reports)

Other suites are reflected in the overall validation status above.

✅ 7598 passed, 37 skipped out of 7635 tests in 1241.2s (+30.7s vs. baseline)

Test Coverage

✅ 85.5% line coverage, 79.8% branch coverage · ✅ no change vs. baseline

CLI Startup Time

51ms median (x64, winapp --version) · 📉 -11ms vs. baseline

Try This Build

Installs the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing.

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) 851
Switching between builds often?

Put the tool on your PATH once:

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) -AddToPath

Then this build is just:

winapp-pr 851

Run winapp-pr with no arguments to pick from a list of open PRs.


Updated 2026-09-21 19:07:56 UTC · commit 99cd610 · workflow run

@azchohfi

Copy link
Copy Markdown
Collaborator

One thing worth flagging before this ships: the prerelease publish path bypasses the tested Build stage.

Prerelease_Nugets_Build is declared with dependsOn: [], and Prerelease_Nugets_Publish depends only on it — not on Build, which is where the test suite runs. So a manual prerelease queue will sign and push the three library packages to nuget.org even if the library tests would fail on that commit. Since a nuget.org push is immutable and Microsoft-signed, that means a test-catchable regression could go out publicly (only recoverable by bumping the prerelease number).

I get that decoupling is intentional to avoid waiting on the full ~60-min NativeAOT/MSIX build. If that tradeoff is deliberate, no change needed — just want to make sure it's a conscious choice. If not, the smallest fix is to add Build to the publish gate (Prerelease_Nugets_Publish.dependsOn: [Prerelease_Nugets_Build, Build]), or run just the library tests inside Prerelease_Nugets_Build.

Prerelease_Nugets_Build was decoupled from the stable Build stage (dependsOn: [])
to avoid the ~60-min NativeAOT/MSIX build, but that also skipped the tests that
run inside Build - so a manual prerelease queue could sign and immutably publish
the three library packages even if their tests failed on that commit.

Run the targeted suites for exactly those packages (WinApp.UIAutomation.Tests via
Microsoft.Testing.Platform, and the WinUI analyzer xUnit tests) inside
Prerelease_Nugets_Build before packing, mirroring build-cli.ps1's runners. A
failing suite fails the stage, so the publish stage that depends on it never runs.
This keeps the path decoupled and fast while validating what actually ships.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@zateutsch

Copy link
Copy Markdown
Contributor Author

Good catch Alexandre Zollinger Chohfi (@azchohfi) — this was a gap, not a conscious tradeoff, so I fixed it in 4621476.

You're right that decoupling from Build was intentional (to skip the ~60-min NativeAOT/MSIX build), but skipping the tests along with it wasn't the goal. Rather than gate on the full Build (whose suite also runs CLI tests irrelevant to these three packages), Prerelease_Nugets_Build now runs the targeted suites for exactly what ships — WinApp.UIAutomation.Tests (engine + recording) and the WinUI analyzer xUnit tests — before packing, mirroring build-cli.ps1's runners. A failing suite fails the stage, so the dependent publish stage never runs. That keeps the path decoupled and fast while making sure a test-failing commit can't be signed and pushed to nuget.org.

test.Tests.ps1 asserted `Should -Not -Match 'release-'` against the entire CLI
output to confirm the release publish profile was not selected. That also scans
the CLI banner, which prints the version - and on a branch build the version
embeds the branch name (e.g. 0.6.3-<branch>-prerelease-pipeline.1), so a branch
containing "release-" false-fails the test.

Match the resolved `PublishProfile=release-` token specifically, mirroring the
positive assertion on the line above, so it still catches a real release-profile
regression without false-matching the version banner.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolves two conflicts:
- .pipelines/release.yml: main removed the -ServiceConnections argument from the
  check-release-credentials step and added two federated-token verification tasks
  (AzureCLI@2 / AzurePowerShell@5) to the verify_credentials job. Kept main's
  version of that job and re-appended the prerelease library-package stages after
  the Verify_Release_Credentials stage.
- samples/winui-app/test.Tests.ps1: main independently made the same fix (scope
  the release-profile assertion to the PublishProfile token). Took main's version;
  the branch's own equivalent fix is now redundant.
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) merged commit 7bcebd6 into main Sep 21, 2026
39 checks passed
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) deleted the zateutsch-zt/nuget-prerelease-pipeline branch September 21, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants