Skip to content

Exercise the federated release connections instead of inspecting them - #860

Merged
Nikola Metulev (nmetulev) merged 7 commits into
mainfrom
azchohfi-exercise-release-connections
Sep 17, 2026
Merged

Nikola Metulev (nmetulev) merged 7 commits into
mainfrom
azchohfi-exercise-release-connections

Conversation

@azchohfi

Copy link
Copy Markdown
Collaborator

The question this answers

"Isn't there a way to use the connections without actually publishing?"

Yes — for two of the four, and they're the two most likely to break.

What was wrong

The check read service-endpoint metadata over REST. That needs the build identity to have View Service Connection — a permission nothing else requires and which it doesn't have — so the list came back empty and produced four false "Not found" failures against connections that were all present and authorized (builds 20260907.1, 20260914.1).

The underlying confusion, now verified both ways:

Mechanism State Consequence
Use (pipeline authorization) Pipeline 182109 authorized on all four Releases work
View (ACL bit 16, REST read) No build-service ACE at all Endpoint list returns empty

They're unrelated. Releases can work perfectly while the list comes back empty — so the empty list means nothing.

Granting the permission would have made the check run, but it would still only confirm a connection exists — never that its credential works, which is the failure that actually blocks a release.

What it does now

Both ESRP signing and symbol publishing are azurerm / WorkloadIdentityFederation, so AzureCLI@2 can acquire a token from them. That proves three things metadata never could:

  • the connection exists,
  • this pipeline is authorized to use it,
  • and the federation credential still works — the part that silently rotates or gets de-authorized under ES policy churn.

Nothing is signed. Nothing is published.

(This also settles an earlier review worry that the ESRP connection might be "ESRP-typed" and unusable by AzurePowerShell@5 — it's azurerm, so it is usable.)

What deliberately isn't covered

github-service-connection is consumed only by GitHubRelease@1, whose every action mutates. NuGet-WinAppCLI carries an API key that is only validated on push. Neither can be exercised without actually releasing, so they're documented as uncovered rather than given a check that proves nothing. A real release covers them.

Also

No permission grant is needed any more — the README previously asked for one; that guidance is removed and replaced with why it was the wrong idea.

Net −250 lines: 98 lines of script and 181 lines of tests removed, replaced by two task invocations that test strictly more than the code they replace.

Validation

82 Pester tests pass, YAML validates, all 11 rel/v* publishing gates intact, and the trimmed checker smoke-tested live.

Note: the two new steps only run in the rehearsal stage (main), which is already restricted, so a real rel/v* release is unaffected. They'll be exercised for real on the next Monday run — or sooner if you want me to queue one.

The service-connection check read endpoint metadata over REST. That needed the build identity to
have View Service Connection - a permission nothing else requires and which it does not have - so
it returned an empty list and produced four false "Not found" failures two weeks running (builds
20260907.1 and 20260914.1) against connections that were all present and authorized.

The underlying confusion: pipelines are authorized to USE a connection through pipeline
authorization, which is entirely separate from the ACL governing READING it over REST. Verified
both: pipeline 182109 is explicitly authorized on all four connections, while the ACL contains no
build-service entry at all. So releases work while the endpoint list comes back empty, and the
empty list means nothing.

Granting the permission would have made the check run, but it would still only confirm a
connection exists - never that its credential works, which is the failure that actually blocks a
release. Acquiring a token proves both.

So: drop the REST lookup and instead exercise the two connections that can be exercised. Both
ESRP signing and symbol publishing are azurerm/WorkloadIdentityFederation, so AzureCLI@2 can
acquire a token from them - proving the connection exists, this pipeline is authorized, and the
federation credential still works. Nothing is signed and nothing is published.

github-service-connection and NuGet-WinAppCLI genuinely cannot be verified without publishing:
GitHubRelease@1 has no non-mutating action, and a NuGet API key is only validated on push. They
are documented as uncovered rather than given a check that proves nothing.

Net: 98 lines of script and 181 lines of tests removed, replaced by two task invocations that
test more than the code they replace.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dbafe00e-a7b1-48f9-8065-453bd752ac9b
Copilot AI balanced review requested due to automatic review settings September 16, 2026 19:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved pipeline probe conditions and parameter bindings can prevent complete or accurate validation.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Replaces unreliable service-connection metadata checks with live federation authentication probes.

Changes:

  • Removes obsolete REST checks and tests.
  • Adds AzureCLI probes for ESRP and symbol publishing.
  • Updates pipeline documentation and limitations.
File summaries
File Summary
scripts/tests/check-release-credentials.Tests.ps1 Removes obsolete endpoint-check tests.
scripts/check-release-credentials.ps1 Retains GitHub credential validation only.
.pipelines/release.yml Adds federation probes. Moderate (1 vote): probes should use succeededOrFailed(). Moderate (2 votes): symbol probe must use symbolsIdentity. Moderate (2 votes): ESRP probe must use signingIdentity.
.pipelines/README.md Documents revised rehearsal behavior. Nit (2 votes): update the stale local-usage note.
Review details

Suppressed comments (1)

.pipelines/release.yml:985

  • These tasks inherit Azure Pipelines' default condition: succeeded(). If Check release credentials fails (for example because the GitHub PAT is expired), neither probe runs; if the ESRP probe fails, the symbol probe is skipped as well. The rehearsal then reports only the first failure instead of exercising both connections as documented. Add condition: succeededOrFailed() to each probe so both independent failures are surfaced while the job still fails.
          - task: AzureCLI@2
            displayName: Verify ESRP signing connection (token only, signs nothing)
            inputs:
              azureSubscription: $(SigningServiceName)
              scriptType: pscore
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .pipelines/release.yml Resolved
Comment thread .pipelines/release.yml Resolved
Comment thread .pipelines/README.md Resolved
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

⏳ Build in progress — metrics below are from a previous commit and will update when the current build finishes.

Build Metrics Report

Binary Sizes

Artifact Baseline Current Delta
CLI (ARM64) 48.44 MB 48.44 MB ✅ 0.0 KB (0.00%)
CLI (x64) 48.29 MB 48.29 MB ✅ 0.0 KB (0.00%)
MSIX (ARM64) 19.96 MB 19.96 MB 📈 +0.1 KB (+0.00%)
MSIX (x64) 21.16 MB 21.16 MB 📉 -0.4 KB (-0.00%)
NPM Package 41.53 MB 41.53 MB 📈 +0.0 KB (+0.00%)
NuGet Package 41.66 MB 41.66 MB 📈 +0.8 KB (+0.00%)

Test Results

✅ 6819 passed, 37 skipped out of 6856 tests in 912.0s (-163.9s vs. baseline)

Test Coverage

✅ 85.9% line coverage, 79.8% branch coverage · ✅ no change vs. baseline

CLI Startup Time

71ms median (x64, winapp --version) · ✅ +7ms vs. baseline

Try This Build

Installs the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing.

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) 860
Switching between builds often?

Put the tool on your PATH once:

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) -AddToPath

Then this build is just:

winapp-pr 860

Run winapp-pr with no arguments to pick from a list of open PRs.


Updated 2026-09-17 00:52:08 UTC · commit 9c947ce · workflow run

…e release does

Two defects in the connection probes, both found by the pr-review skill.

Both probes inherited the default succeeded() step condition, so a failure in
the GitHub credential check skipped them entirely. The operator would fix the
PAT, wait a week for the next rehearsal, and only then discover a connection was
also dead - the two-week loop this pipeline exists to end. Both now run on
succeededOrFailed().

The symbol probe used AzureCLI@2 with `az account get-access-token`, while the
real symbol stage uses AzurePowerShell@5 with Get-AzAccessToken. It therefore
could not catch a break in that task or in the Az modules, and its comment
claiming it exercised "the exact token exchange" was false. It now runs the same
task and cmdlet as the release, and keeps the token as a SecureString rather than
converting it to plaintext.

Deliberately not done: probing the Key Vault certificates behind the ESRP
connection. A data-plane read can fail on permissions the ESRP identity holds but
the pipeline context does not, which would recreate the false-failure bug this
branch removes.

Also corrects two stale spots in the runbook: the local-run note still described
service-connection checks the script no longer contains, and the triage table
named steps whose displayName has since changed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dbafe00e-a7b1-48f9-8065-453bd752ac9b

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Only a minor documentation nit remains; no blocking issues were identified.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

Comment thread .pipelines/README.md Resolved
Switching the symbol probe to AzurePowerShell@5 left the runbook claiming
AzureCLI@2 performed both checks, pointing an operator at the wrong task.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dbafe00e-a7b1-48f9-8065-453bd752ac9b
The probes resolved $(SigningServiceName) and $(SymbolPublishingServiceConnection)
directly, while the real ESRP and symbol tasks resolve them through the
signingIdentity and symbolsIdentity parameters. Identical by default, but a run
that overrode either parameter would verify one connection and publish with
another - the probe would report green for a connection the release never used.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dbafe00e-a7b1-48f9-8065-453bd752ac9b

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved blocking issues were identified.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

The test asserted the run output must not match 'release-', meaning "winapp did
not pick release-$(Platform).pubxml". But it matched the whole console output,
which includes winapp's version banner - and that banner carries the branch name.
Any branch containing 'release-' failed a test about publish profiles:

  Version 0.6.3-azchohfi-exercise-release-connections.1
                          ^^^^^^^^

Scoping the match to 'PublishProfile=release-' mirrors the positive assertion on
the line above and still fails if the release profile is ever selected.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: dbafe00e-a7b1-48f9-8065-453bd752ac9b

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Release credential validation changes warrant final human review.

Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0 new
  • Review effort level: Lite (auto)

Note

Copilot is running an experiment and ran this review at Lite.

@nmetulev
Nikola Metulev (nmetulev) merged commit e62435c into main Sep 17, 2026
13 checks passed
@nmetulev
Nikola Metulev (nmetulev) deleted the azchohfi-exercise-release-connections branch September 17, 2026 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants