Skip to content

Reserve proxyless endpoint ports process-wide to avoid cross-app port collisions - #21533

Merged
David Negstad (danegsta) merged 4 commits into
microsoft:mainfrom
davidfowl:fix-20009-port-allocator-cross-app
Nov 1, 2026
Merged

David Negstad (danegsta) merged 4 commits into
microsoft:mainfrom
davidfowl:fix-20009-port-allocator-cross-app

Conversation

@davidfowl

@davidfowl David Fowler (davidfowl) commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Description

DashboardUrlTests.GetDashboardUrlAsyncAuthenticatesDashboardBrowser intermittently fails with Expected: Found, Actual: RedirectKeepVerb on the /login?t=... request (CI log).

Root cause

Over plain HTTP the dashboard only issues 302 Found from its login and redirect middleware. A 307 can't come from the dashboard. It comes from TestingAppHost1's mywebapp1, which calls UseHttpsRedirection().

Target ports for proxied endpoints that don't specify a TargetPort come from ProxylessEndpointPortAllocator. That allocator is a per-application DI singleton. It probes a port by binding it and then releasing it, and the workload binds the port later. Aspire.Hosting.Testing.Tests runs several AppHosts in the same process in parallel, so two applications' allocators can hand out the same target port. In that case:

  • one app's dashboard fails with address already in use, or
  • one app's DCP proxy for aspire-dashboard forwards to another app's mywebapp1, which returns 307.

GetDashboardUrlAsync only waits for the dashboard to be running, so the test then hits the wrong backend.

Fix

Port availability is now tracked in one process-wide ProxylessEndpointPortReservations array, indexed directly by port. Every allocator uses it as the single availability authority. Each byte entry holds:

  • 0: available for dynamic allocation.
  • 1-254: the number of live reservation references.
  • 255: permanently excluded after a failed probe.

How each operation uses it:

  • Dynamic allocation atomically claims a 0 entry before probing (the probe runs outside the shared lock).
    • If the probe succeeds, the allocator registers the reference for cleanup.
    • If the probe fails, the port stays permanently excluded and no cleanup is registered, so no allocator in the process probes it again.
    • If the probe throws, the claim is released and the exception propagates.
  • ExcludePort (fixed, persisted and target ports) adds a reference to the same shared array. Repeated exclusions, or exclusions of ports the allocator already owns, are deduplicated. A permanently excluded port stays permanent.
  • Disposal releases only the references that allocator registered, so overlapping exclusions held by other allocators survive. Adding a reference past 254 throws rather than wrapping or saturating.

Allocators for different applications in one process can no longer return the same port. Assertions, retries and timeouts are unchanged.

DcpExecutorTests builds allocators that are never disposed, so it now passes an isolated reservation set instead of leaking into the shared one.

Evidence

Before (functional repro). I ran the full Aspire.Hosting.Testing.Tests assembly with ASPIRE_PROXYLESS_ENDPOINT_PORT_RANGE=20000-20015 to force overlap between parallel AppHosts:

System.IO.IOException: Failed to bind to address http://127.0.0.1:20004: address already in use.
 ---> Microsoft.AspNetCore.Connections.AddressInUseException: Address already in use

This happened 4 times in aspire-dashboard. GetDashboardUrlAsyncReturnsBaseUrlWhenDashboardAllowsAnonymousAccess failed with "aspire-dashboard ... failed to start".

After. The same run with the fix had 0 address already in use errors and no dashboard start failures. I only have log evidence here: the full assembly doesn't finish in my sandbox because of unrelated environment issues (no trusted dev cert, no Playwright browsers).

Regression test. AllocatorsForDifferentApplicationsInTheSameProcessDoNotShareAPort uses the production constructor with a single-port range:

  • Before the fix: Assert.Throws() Failure: No exception was thrown, because the second app's allocator received the same port.
  • After the fix: it passes.
  • New coverage: exclusions are shared and survive one allocator's disposal, repeated or owned exclusions are deduplicated, permanent exclusions stay permanent, probe exceptions release their claim, and the reference count throws instead of overflowing.
  • ProxylessEndpointPortAllocatorTests (13/13) and DcpExecutorTests (310/310) pass.

Fixes #20009

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
    • No

🤖 Posted by AspireAgent (AI Agent)

ProxylessEndpointPortAllocator is a per-application singleton that probes a
port (bind + release) and hands it out; the workload binds it later. When
multiple AppHosts run in the same process (e.g. Aspire.Hosting.Testing tests
running in parallel), two allocators could hand out the same target port, so
one app's dashboard/proxy ends up pointing at another app's workload.

Track allocated ports in a process-wide reservation set that is released when
the allocator is disposed.

Fixes microsoft#20009

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21533

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21533"

@aspire-repo-bot
aspire-repo-bot Bot requested a balanced review from Copilot October 31, 2026 06:15
@github-actions github-actions Bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The test helper checks only IPv4 loopback while production probes both IPv4 and IPv6 wildcard addresses, so an IPv6-only listener can cause a false regression failure.

1 open issue
What changed in this PR

This PR addresses cross-application port collisions behind the flaky dashboard login test in #20009 by sharing port reservations across AppHosts in the same process.

Changes:

  • Reserve ports before probing, releasing unsuccessful probes immediately and successful allocations when the owning allocator is disposed.
  • Add regression tests for cross-application uniqueness and reservation release.
  • Isolate reservations in executor tests whose allocators are not disposed.
File Description
tests/​Aspire.Hosting.Tests/​Dcp/​ProxylessEndpointPortAllocatorTests.cs Adds shared-reservation and disposal regression tests.
tests/​Aspire.Hosting.Tests/​Dcp/​DcpExecutorTests.cs Uses isolated reservations to avoid process-wide test leaks.
src/​Aspire.Hosting/​Dcp/​ProxylessEndpointPortAllocator.cs Adds synchronized process-wide reservations and disposal cleanup.

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the changes addressing port collisions between parallel AppHosts.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copy link
Copy Markdown
Collaborator Author

CI rerun note for head 3da0812c6b6b4a7f19db4877d7f3f045ae12ba6b (run 41177651865):

On the previous head (6f2520398e14b81d5790e4e1f7b86a475629b619), the only failures were a Deno Windows telemetry-fixture timeout (Hosting.JavaScript, windows-latest; known infra flake) and VerifyMongoDBMultiNodeReplicaWithDataShouldWorkAcrossUsages (known flake, #20478/#20512). That run was superseded by the review-feedback push and cancelled. All Hosting and Hosting.Testing jobs passed on both heads.

🤖 Posted by AspireAgent (AI Agent)

Replace the allocator's per-instance visited array with the shared
ProxylessEndpointPortReservations, backed by a directly indexed state
array for the whole port space instead of a HashSet. Allocated ports
stay reserved until the allocator is disposed; ports whose probe failed
are marked unavailable process-wide so no allocator re-probes them.
Ports excluded by the app model remain per-allocator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A failed port probe permanently excludes the port from the process-wide pool, causing later AppHosts to fail allocation even after the port becomes free.

1 open issue

Review effort: Balanced

Copy link
Copy Markdown
Collaborator Author

CI note for head 91ca3c78d26b7076607cb658b566cb7d9b76f6e3: run 41244080988 failed only in two Windows VS Code extension E2E shards. The shards don't exercise the proxyless port allocator change.

  • diagnostic-logs: Selenium ElementNotInteractableError while dismissing a notification (Notification.dismiss).
  • dynamic-debug-configuration: ran for about 47 minutes and finished with no log.

Windows E2E shards have also failed intermittently on recent main runs, for example 41234363151. I've rerun the failed jobs once as a known flake.

🤖 Posted by AspireAgent (AI Agent)

Use the process-wide reservation array as the single availability
authority. Entries count live reservation references (1-254) with 255
as the permanent-exclusion sentinel. ExcludePort now adds shared
references instead of a per-allocator set, deduplicated per allocator,
and disposal releases only the references that allocator registered.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the changes intended to prevent endpoint port collisions between AppHosts in the same process.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copy link
Copy Markdown
Collaborator Author

CI note for head 1736bffff3b3ef2caf27177e6a878316ff941690: I've rerun the failed jobs in run 41249914951 once. None of the failures are related to the port allocator change:

  • Build CLI (win-x64): the Windows native tray smoke test failed with "Windows notification area is not ready" and "Stop did not send its request without a dialog". It's a UI timing failure in Aspire.Tray and passed on the previous head.
  • Hosting-2, Hosting.Keycloak and Hosting.Browsers (windows-latest): all three lost their hosted runners at the same time (about 23:41Z, "The hosted runner lost communication with the server"), with no logs. The same three jobs passed on Linux for this head. All the other Windows Hosting jobs passed, including Hosting, Hosting-1, -3, -4, -5 and -6, which exercise DcpExecutor and the allocator. Concurrent runs on other branches passed Windows Hosting-2 normally.

🤖 Posted by AspireAgent (AI Agent)

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Copy link
Copy Markdown
Collaborator Author

CI note for head 1736bffff3b3ef2caf27177e6a878316ff941690: run 41249914951 is on attempt 3, retrying only the two jobs that failed on attempt 2. The repo's auto-retry workflow started it at 00:32Z. I'm treating it as this head's one additional rerun and won't trigger another.

  • Hosting.Keycloak (windows-latest): the hosted runner lost communication with the server on both earlier attempts, with no test output. These tests don't start an AppHost or allocate proxyless ports.
  • VS Code extension E2E (Linux, tree-actions): failed with "Aspire extension E2E endpoint command could not find a matching endpoint." in the extension's fake-CLI E2E harness. This signature is being root-caused separately and isn't addressed in this PR.

🤖 Posted by AspireAgent (AI Agent)

@danegsta
David Negstad (danegsta) merged commit e5a5acd into microsoft:main Nov 1, 2026
962 of 972 checks passed
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 17.0 milestone Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure] Flaky: DashboardUrlTests.GetDashboardUrlAsyncAuthenticatesDashboardBrowser expects HTTP Found but gets RedirectKeepVerb

3 participants