Skip to content

[test-improver] Add unit tests for RetryArtifactProcessor.ProcessAsync #10578

Closed

Description

@github-actions

Goal and rationale

PR #10542 ("Consolidate reports across retry attempts") added RetryArtifactProcessor.ProcessAsync in Microsoft.Testing.Extensions.Retry — the method that merges per-attempt artifacts (e.g. TRX/CTRF reports) produced across retry attempts by invoking registered IArtifactPostProcessors. Despite being the most complex method in that file (processor selection, completeness validation per attempt, exception handling, replacement-path bookkeeping), it had zero test coverage — RetryTests.cs only covered SnapshotAttemptArtifacts, PublishExternalArtifacts, and MoveArtifactsAsync.

Approach

  • Added a lightweight TestArtifactPostProcessor test double (in Helpers/) implementing IArtifactPostProcessor with a configurable ProcessAsync delegate, following the existing helper-class pattern in this test project.
  • Added 8 new tests to RetryTests.cs using a real ServiceProvider (already used elsewhere in this test suite, e.g. CtrfArtifactPostProcessorTests) to register the fake processor, plus Moq mocks for ILogger/IOutputDevice, matching the existing style in the file.
  • Covered these branches:
    • No eligible processors registered → empty dictionary, no invocation.
    • attemptCount < 2 → empty dictionary, processor never invoked.
    • Incomplete per-attempt coverage (missing an attempt's artifact) → processor skipped, no replacement recorded.
    • Processor returns null → skipped without recording a replacement.
    • Successful merge → replacement recorded for the final-attempt artifact path, output path returned matches the processor's result.
    • Processor throws a non-cancellation exception → warning logged via ILogger, warning displayed via IOutputDevice, no exception propagates.
    • Processor throws OperationCanceledException with cancellation requested → exception rethrown (not swallowed).

Coverage impact

No project-wide coverage report was generated (not part of the existing CI pipeline for this project), but all new tests directly exercise previously-uncovered branches in RetryArtifactProcessor.ProcessAsync (the only public method in that internal class lacking tests before this change).

Trade-offs

The new TestArtifactPostProcessor fake is intentionally minimal (delegate-based ProcessAsync, fixed SupportedModes/SupportsTruncatedRuns) to keep maintenance low while remaining reusable for future IArtifactPostProcessor-related tests in this project.

Reproducibility

export PATH="$PWD/.dotnet:$PATH"
dotnet build test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Microsoft.Testing.Extensions.UnitTests.csproj -c Debug
./artifacts/bin/Microsoft.Testing.Extensions.UnitTests/Debug/net8.0/Microsoft.Testing.Extensions.UnitTests

Test Status

Build succeeded (0 errors/warnings). Full Microsoft.Testing.Extensions.UnitTests suite: 1097 total, 1063 succeeded, 0 failed, 34 skipped (pre-existing Windows-only skips). All 8 new ProcessAsync_* tests pass.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • southcentralus0.in.applicationinsights.azure.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "southcentralus0.in.applicationinsights.azure.com"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Test Improver workflow. · auto · 318.5 AIC · ⌖ 21.8 AIC · ⊞ 19.4K · [◷]( · ◷)
Comment /test-assist to run again

Add this agentic workflow to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/test-improver.md@main

Note

This was originally intended as a pull request, but GitHub Actions is not permitted to create or approve pull requests in this repository.
The changes have been pushed to branch test-assist/retry-artifact-processor-processasync-8b91794256c47360.

Click here to create the pull request

To fix the permissions issue, go to Settings → Actions → General and enable Allow GitHub Actions to create and approve pull requests. See also: gh-aw FAQ

Show patch preview (309 of 309 lines)
From 28b08da93a187641b0295f16adb20818f7f02cbe Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Wed, 12 Aug 2026 23:21:10 +0000
Subject: [PATCH] Add unit tests for RetryArtifactProcessor.ProcessAsync

Cover the merge-branch logic added in #10542: no eligible processors,
too few attempts, incomplete per-attempt coverage, null processor
result, successful merge replacement recording, processor exceptions
(warning logged + displayed), and OperationCanceledException rethrow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 .../Helpers/TestArtifactPostProcessor.cs      |  50 +++++
 .../RetryTests.cs                             | 205 ++++++++++++++++++
 2 files changed, 255 insertions(+)
 create mode 100644 test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Helpers/TestArtifactPostProcessor.cs

diff --git a/test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Helpers/TestArtifactPostProcessor.cs b/test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Helpers/TestArtifactPostProcessor.cs
new file mode 100644
index 0000000..588cffa
--- /dev/null
+++ b/test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Helpers/TestArtifactPostProcessor.cs
@@ -0,0 +1,50 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+
+using Microsoft.Testing.Platform.Extensions.ArtifactPostProcessing;
+
+namespace Microsoft.Testing.Extensions.UnitTests.Helpers;
+
+#pragma warning disable TPEXP // Artifact post-processing is experimental.
+
+internal sealed class TestArtifactPostProcessor : IArtifactPostProcessor
+{
+    private readonly Func<IReadOnlyList<InputArtifact>, string, ArtifactPostProcessingContext, CancellationToken, Task<ProcessedArtifact?>> _processAsync;
+
+    public TestArtifactPostProcessor(
+        IReadOnlyList<string> supportedKinds,
+        Func<IReadOnlyList<InputArtifact>, str
... (truncated)

Metadata

Metadata

Labels

area/agentic-workflowsGitHub agentic workflow definitions under .github/workflows/*.md.type/automationCreated or maintained by an agentic workflow.type/test-gapMissing or insufficient tests.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions