Repository navigation
Treat FailedToStart executables as stopped in DcpExecutor.StopResourceAsync - #21441
David Fowler (davidfowl) wants to merge 1 commit into
Conversation
…eAsync When a stop request reaches an Executable before DCP has started its process, DCP reports the Executable as FailedToStart. StopResourceAsync only accepted Finished or Terminated, so it retried for about 15 seconds and then threw 'Failed to stop resource', which made the rebuild command fail intermittently on Windows (ExecuteCommandAsync_RebuildCommand_ReturnsBuildOutput). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21441Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21441" |
github-actions
Bot
added
the
needs-area-label
Tests selector52 / 100 PR test projects · 4 PR jobs, from 3 changed files. Selected PR test projects (52 / 100)
Selected PR jobs (4)
How these were chosen — grouped by what changed
🔧 show 46
🧪 📦 affected project 📦 affected project 🧪 Job reasons
Selection computed for commit |
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain with the change to stop handling for failed-to-start executables.
No open issues 🎉
What changed in this PR
Fixes executable stop handling in Aspire hosting so rebuild commands do not fail when an executable enters FailedToStart. This addresses the stop failure separately from the build-count issue handled by #21427.
Changes:
- Treats
FailedToStartexecutables as stopped; container handling remains unchanged. - Updates the Kubernetes test fake to preserve that terminal state.
- Adds a regression test verifying that stopping a failed-to-start executable succeeds.
| File | Description |
|---|---|
tests/Aspire.Hosting.Tests/Dcp/TestKubernetesService.cs |
Preserves FailedToStart when an executable is stopped. |
tests/Aspire.Hosting.Tests/Dcp/DcpExecutorTests.cs |
Adds regression coverage for stopping a failed-to-start executable. |
src/Aspire.Hosting/Dcp/DcpExecutor.cs |
Recognizes FailedToStart as a completed stop condition. |
Review effort: Balanced
|
The first CI attempt at head
I'm keeping this PR as a draft until the required CI is green. 🤖 Posted by AspireAgent (AI Agent) |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
Description
Fixes the "Failed to stop resource" failure mode of
ResourceCommandServiceTests.ExecuteCommandAsync_RebuildCommand_ReturnsBuildOutputonHosting-2 (windows-latest). This is a product bug inDcpExecutor.StopResourceAsync.Root cause. The first rebuild command restarts the project, which deletes and re-creates its DCP Executable. The test then immediately runs the second rebuild, whose first step stops the resource. In the failing runs, the stop patch (
Spec.Stop = true) reached the new Executable before DCP had started its process. DCP then reported the Executable asFailedToStart, a terminal state in which nothing is running.StopResourceAsynctreats onlyFinishedorTerminatedas stopped. It kept polling through the delete retry pipeline (about 15s) and then threwInvalidOperationException: Failed to stop resource '...', so the rebuild command failed andAssert.True(secondResult.Success)failed.Sequence from job 123561387182 (run 40905807668):
15:58:32First rebuild:Build succeeded. Restarting resource..., thenStarting Executable 'myProject-jgbacpwt'15:58:33.03Second rebuild:Stopping resource 'myProject-jgbacpwt'...15:58:33Resource myProject/myProject-jgbacpwt changed state: FailedToStart15:58:33–15:58:50Executable 'myProject-jgbacpwt' is still running; trying again to stop it...× 10, while the watcher reports no further changes to the Executable15:58:50Error executing command 'rebuild'. System.InvalidOperationException: Failed to stop resource 'myProject-jgbacpwt'.Job 123745958478 shows the same
Starting → FailedToStarttransition followed by the same exception.Fix.
StopResourceAsynctreats an Executable inFailedToStartas stopped. The test fakeTestKubernetesServiceused to force every stopped Executable toFinished. It now leaves aFailedToStartExecutable as is, matching DCP. Container stop handling is unchanged, because there is no evidence for that path.Regression test.
DcpExecutorTests.StopResourceAsync_ExecutableThatFailedToStart_IsTreatedAsStoppedputs an Executable inFailedToStartand stops it.InvalidOperationException: Failed to stop resource 'CoolProgram-…'atDcpExecutor.cs:1232after 19s. This is the same exception as CI. With the committed default guard, it fails with aTimeoutException.DcpExecutorTestsandResourceCommandServiceTestspassed.Recurrence. The window is aspire main CI, Oct 23–30, first attempts only, out of 51 failed runs.
Hosting-2 (windows-latest)failed in 7 of those runs:Failed to stop resource(this PR): runs 40905807668 and 40958399480.Related to #20662 (#21427 fixes the build-count mode of the same test).
Checklist
🤖 Posted by AspireAgent (AI Agent)