Add retry to NuGet setup tasks for node24 task-host libuv crash#406
Merged
Conversation
Node 24 task-host wrappers on Windows ADO agents intermittently abort with
Exit code 57005 (Watson bucket FAIL_FAST_FATAL_APP_EXIT_c0000409 in
vss-agent\externals\node24\bin\node.exe). Symbolicated stacks point at a
libuv UV_HANDLE_CLOSING race in uv_async_send() called from Node's
WorkerThreadsTaskRunner::DelayedTaskScheduler at shutdown:
Assertion failed: !(handle->flags & UV_HANDLE_CLOSING)
file deps/uv/src/win/async.c, line 76
Tracked upstream as nodejs/node#56645 (open; pending fix PR nodejs/node#61999)
and in the agent as microsoft/azure-pipelines-agent#5498. Until either fix
ships and the agent's externals/node24 picks it up, the only mitigation
available to consumers is to retry the affected setup tasks.
retryCountOnTaskFailure: 3 brings the residual per-task failure rate from
~9% (observed) to <0.01%. Both NuGetAuthenticate@1 and NuGetToolInstaller@1
are idempotent setup, so retrying on any failure is safe; legitimate
failures still surface after the retries are exhausted.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dcristoloveanu
marked this pull request as ready for review
May 14, 2026 22:37
dcristoloveanu
enabled auto-merge (squash)
May 15, 2026 00:03
parth21999
approved these changes
May 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
retryCountOnTaskFailure: 3to the two Node-handler setup tasks in this template (NuGetAuthenticate@1andNuGetToolInstaller@1).Why
Node 24 task-host wrappers on Windows ADO agents intermittently abort with Exit code 57005 (Watson bucket
FAIL_FAST_FATAL_APP_EXIT_c0000409, faulting invss-agent\externals\node24\bin\node.exe). Symbolicated against the Node 24.13.0 PDB, the stack ends in:…which is libuv's
uv_async_send()invoked from Node'sWorkerThreadsTaskRunner::DelayedTaskSchedulershutdown path -- a lifecycle race between the task host's V8 platform tear-down and an in-flight async wakeup.References:
The agent ships Node 24 (
externals/node24) regardless of any pipelineUseNode@1step -- the task-host runtime is selected purely from each task'starget.runtimeVersion. The only currently available mitigation for consumers is to retry the affected wrapper invocations.In Azure-MessagingStore we measured ~9
NuGetAuthenticate@1/Npm@1/UseNode@1crashes across ~99 builds in the past 7 days (per-task failure rate ~9%). WithretryCountOnTaskFailure: 3the residual rate drops to <0.01%.Why this is safe
NuGetAuthenticate@1is idempotent (env-var setup + credential-provider probe).NuGetToolInstaller@1is idempotent (downloads + cachesnuget.exe); already hadcontinueOnError: true.Rollback
Revert this commit once nodejs/node#61999 merges, Node ships it in a 24.x, the agent bumps
NODE24_VERSIONto that release, and the 1ES Hosted pool image rolls forward.