Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
aecbdee
Add artifact post-processing orchestration
Evangelink Jul 20, 2026
108985b
Address artifact processing review feedback
Evangelink Jul 20, 2026
a094fad
Cover repeated tool option validation
Evangelink Jul 20, 2026
c1e0b62
Harden artifact manifest and merge retries
Evangelink Jul 20, 2026
187eb5f
Address merge readiness review feedback
Evangelink Jul 20, 2026
077cc68
Document artifact processing handshake values
Evangelink Jul 20, 2026
cdb6f2c
Finish public tool contract validation
Evangelink Jul 20, 2026
6e304a2
Validate artifact paths and merge input arity
Evangelink Jul 20, 2026
22322e6
Isolate tool validation from test runs
Evangelink Jul 20, 2026
4a3f92a
Validate selected tool option structure
Evangelink Jul 20, 2026
43ff870
Preserve manifest JSON token shape
Evangelink Jul 20, 2026
eab0882
Refine processor conflict warnings
Evangelink Jul 20, 2026
16fd031
Separate artifact conflict domains
Evangelink Jul 20, 2026
391f71b
Validate manifest value types
Evangelink Jul 20, 2026
7cb1fe6
Document artifact kind and fix encodings
Evangelink Jul 20, 2026
e5a1e0c
Cover merge tool success path
Evangelink Jul 20, 2026
92ced72
Simplify manifest string check
Evangelink Jul 20, 2026
ee4b9e1
Validate configured tool options
Evangelink Jul 20, 2026
71dd467
Validate processor capabilities and outputs
Evangelink Jul 20, 2026
20c56fc
Keep processor failures non-fatal
Evangelink Jul 20, 2026
0539838
Document processor retry requirements
Evangelink Jul 20, 2026
2e05e2c
Reject blank processed artifact kinds
Evangelink Jul 20, 2026
ff68296
Advertise processors from test controllers
Evangelink Jul 20, 2026
a24794f
Validate selected tool JSON options
Evangelink Jul 20, 2026
86bc30f
Reject structured manifest metadata
Evangelink Jul 20, 2026
09cf260
Signal dispatcher processor failures
Evangelink Jul 20, 2026
f883fee
Restore tool binary compatibility
Evangelink Jul 21, 2026
09a2d10
Use flow-aware parse result guard
Evangelink Jul 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions docs/mstest-runner-protocol/004-protocol-dotnet-test-pipe.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,15 +334,17 @@ Property IDs (`HandshakeMessagePropertyNames`):
| 2 | `Framework` | yes | `RuntimeInformation.FrameworkDescription`. |
| 3 | `OS` | yes | `RuntimeInformation.OSDescription`. |
| 4 | `SupportedProtocolVersions` | yes | Semicolon-separated list the host supports (see §10). |
| 5 | `HostType` | yes | `TestHost`, `TestHostController`, `ServerTestHost`, or `TestHostOrchestrator`. |
| 5 | `HostType` | yes | `TestHost`, `TestHostController`, `ServerTestHost`, `TestHostOrchestrator`, or `ArtifactPostProcessor`. |
| 6 | `ModulePath` | yes | Full path of the test application. |
| 7 | `ExecutionId` | yes | The Execution ID (from the env var). |
| 8 | `InstanceId` | yes | The per-connection Instance ID. |
| 9 | `IsIDE` | **reply-only** | Consumer requests full discovery details (see §9). |
| 10 | `ExecutionMode` | yes | `run`, `help`, or `discover` — lets the SDK detect mismatches (e.g. `--help` leaking into a run). |
| 10 | `ExecutionMode` | yes | `run`, `help`, `discover`, or `tool` — lets the SDK detect mismatches (e.g. `--help` leaking into a run). |
| 11 | `OrchestratorFeature` | orchestrator only | The orchestrator extension Uid (e.g. retry). |
| 12 | `ServerControlPipeName` | **reply-only** | OS name of the reverse control pipe (see §12). |
| 13 | `AttemptNumber` | test host only | Positive, 1-based retry attempt. Multiple Instance IDs may share one attempt. |
| 14 | `SupportedPostProcessorKinds` | test host, server test host, test host controller, or artifact post-processor | Semicolon-separated reverse-DNS artifact kinds supported by registered post-processors. |
| 15 | `SupportedPostProcessorExtensionsLegacy` | test host, server test host, test host controller, or artifact post-processor | Semicolon-separated lowercase file extensions used as a fallback for untagged artifacts. |

### 8.2 SDK → host: `HandshakeMessage` (reply)

Expand Down Expand Up @@ -468,8 +470,9 @@ currently running (rendered by non-IDE consumers as a "currently running tests"

Reports produced files (per-test artifacts, session artifacts, and standalone `FileArtifact`s). Each
`FileArtifactMessage`: `FullPath`(1), `DisplayName`(2), `Description`(3), `TestUid`(4),
`TestDisplayName`(5), `SessionUid`(6). Test-scoped artifacts fill `TestUid`/`TestDisplayName`; session
and standalone artifacts leave them empty.
`TestDisplayName`(5), `SessionUid`(6), `Kind`(7). `Kind` carries the producer-asserted artifact kind
used for post-processor routing. Test-scoped artifacts fill `TestUid`/`TestDisplayName`; session and
standalone artifacts leave them empty.

### 9.6 `CommandLineOptionMessages` (ID 3)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,50 @@ const Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.TrxArt
const Microsoft.Testing.Platform.IPC.FileArtifactMessageFieldsId.Kind = 7 -> ushort
static Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.Merge(System.Collections.Generic.IReadOnlyList<System.Xml.Linq.XDocument!>! inputReports, System.Guid runId, string! runName) -> System.Xml.Linq.XDocument!
static Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.MergeToFileAsync(System.Collections.Generic.IReadOnlyList<string!>! inputPaths, string! outputPath, System.Guid runId, string! runName, System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.CreateCommandLine() -> Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.CreateTool(Microsoft.Testing.Platform.CommandLine.ICommandLineOptions! commandLineOptions) -> Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.Description.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.DisplayName.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.IsEnabledAsync() -> System.Threading.Tasks.Task<bool>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.ToolTrxMergeFactory() -> void
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.Uid.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.ToolTrxMergeFactory.Version.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.Description.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.DisplayName.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.IsEnabledAsync() -> System.Threading.Tasks.Task<bool>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.ProcessAsync(System.Collections.Generic.IReadOnlyList<Microsoft.Testing.Platform.Extensions.ArtifactPostProcessing.InputArtifact!>! inputs, string! outputDirectory, System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task<Microsoft.Testing.Platform.Extensions.ArtifactPostProcessing.ProcessedArtifact?>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.SupportedFileExtensionsFallback.get -> System.Collections.Generic.IReadOnlyList<string!>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.SupportedKinds.get -> System.Collections.Generic.IReadOnlyList<string!>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.TrxArtifactPostProcessor() -> void
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.Uid.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxArtifactPostProcessor.Version.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.Description.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.DisplayName.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.IsEnabledAsync() -> System.Threading.Tasks.Task<bool>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.Name.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.RunAsync(System.Threading.CancellationToken cancellationToken) -> System.Threading.Tasks.Task<int>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.TrxMergeTool(Microsoft.Testing.Platform.CommandLine.ICommandLineOptions! commandLineOptions, Microsoft.Testing.Platform.Extensions.IExtension! extension) -> void
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.Uid.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.Version.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.Description.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.DisplayName.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.GetCommandLineOptions() -> System.Collections.Generic.IReadOnlyCollection<Microsoft.Testing.Platform.Extensions.CommandLine.CommandLineOption!>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.IsEnabledAsync() -> System.Threading.Tasks.Task<bool>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.ToolName.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.TrxMergeToolCommandLine(Microsoft.Testing.Platform.Extensions.IExtension! extension) -> void
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.Uid.get -> string!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.ValidateCommandLineOptionsAsync(Microsoft.Testing.Platform.CommandLine.ICommandLineOptions! commandLineOptions) -> System.Threading.Tasks.Task<Microsoft.Testing.Platform.Extensions.ValidationResult>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.ValidateOptionArgumentsAsync(Microsoft.Testing.Platform.Extensions.CommandLine.CommandLineOption! commandOption, string![]! arguments) -> System.Threading.Tasks.Task<Microsoft.Testing.Platform.Extensions.ValidationResult>!
Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.Version.get -> string!
const Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeTool.ToolName = "merge-trx" -> string!
const Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.InputOptionName = "input" -> string!
const Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxMergeToolCommandLine.OutputOptionName = "output-trx" -> string!
static Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.CreateMergeRunId(System.Collections.Generic.IReadOnlyList<string!>! inputPaths) -> System.Guid
static Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxReportEngine.CreateMergeRunId(System.Collections.Generic.IReadOnlyList<string!>! inputPaths, System.Collections.Generic.IReadOnlyList<string?>! executionIds) -> System.Guid
const Microsoft.Testing.Platform.Helpers.EnvironmentVariableConstants.TESTINGPLATFORM_PIPE_DIRECTORY = "TESTINGPLATFORM_PIPE_DIRECTORY" -> string!
Microsoft.Testing.Platform.Helpers.DisposeHelper
static Microsoft.Testing.Platform.Helpers.DisposeHelper.DisposeAsync(object? obj) -> System.Threading.Tasks.Task!
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,17 @@
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<data name="TrxArtifactPostProcessorDescription" xml:space="preserve">
<value>Merges TRX reports produced by Microsoft.Testing.Extensions.TrxReport</value>
<comment>Do not localize format name {Locked="TRX"} or assembly name {Locked="Microsoft.Testing.Extensions.TrxReport"}.</comment>
</data>
<data name="InvalidTestApplicationBuilderType" xml:space="preserve">
<value>TRX report generator only works with builders of type 'Microsoft.Testing.Platform.Builder.TestApplicationBuilder'</value>
<comment>Do not localize {Locked="Microsoft.Testing.Platform.Builder.TestApplicationBuilder"}.</comment>
<value>TRX report generation and merge tools require a test application builder that supports artifact post-processing</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxArtifactPostProcessorDisplayName" xml:space="preserve">
<value>TRX report merger</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxComparerToolBaselineFileOptionDescription" xml:space="preserve">
<value>The baseline TRX file</value>
Expand Down Expand Up @@ -149,6 +157,42 @@
<value>Warning: Trx file '{0}' already exists and will be overwritten.</value>
<comment>{0} is the {Locked="TRX"} report file path.</comment>
</data>
<data name="TrxMergeToolDescription" xml:space="preserve">
<value>Merges multiple TRX files into one from the command line</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergeToolDisplayName" xml:space="preserve">
<value>TRX report merge tool</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergeToolInputOptionDescription" xml:space="preserve">
<value>Two or more input TRX file paths</value>
<comment>Do not localize option name {Locked="--input"} or format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergeToolInvalidInputs" xml:space="preserve">
<value>'--input' expects two or more existing TRX file paths</value>
<comment>Do not localize option name {Locked="--input"} or format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergeToolInvalidOutput" xml:space="preserve">
<value>'--output-trx' expects one TRX file path</value>
<comment>Do not localize option name {Locked="--output-trx"} or format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergeToolOutputOptionDescription" xml:space="preserve">
<value>The output TRX file path</value>
<comment>Do not localize option name {Locked="--output-trx"} or format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergedArtifactDescription" xml:space="preserve">
<value>{0} TRX files merged</value>
<comment>{0} is the number of merged files. Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergedArtifactDisplayName" xml:space="preserve">
<value>TRX report (merged)</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxMergedRunName" xml:space="preserve">
<value>Merged TRX report</value>
<comment>Do not localize format name {Locked="TRX"}.</comment>
</data>
<data name="TrxReportArtifactDescription" xml:space="preserve">
<value>Test session</value>
</data>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,19 @@
<file datatype="xml" source-language="en" target-language="cs" original="../ExtensionResources.resx">
<body>
<trans-unit id="InvalidTestApplicationBuilderType">
<source>TRX report generator only works with builders of type 'Microsoft.Testing.Platform.Builder.TestApplicationBuilder'</source>
<target state="translated">Generátor sestav TRX funguje pouze s tvůrci typu Microsoft.Testing.Platform.Builder.TestApplicationBuilder.</target>
<note>Do not localize {Locked="Microsoft.Testing.Platform.Builder.TestApplicationBuilder"}.</note>
<source>TRX report generation and merge tools require a test application builder that supports artifact post-processing</source>
<target state="new">TRX report generation and merge tools require a test application builder that supports artifact post-processing</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxArtifactPostProcessorDescription">
<source>Merges TRX reports produced by Microsoft.Testing.Extensions.TrxReport</source>
<target state="new">Merges TRX reports produced by Microsoft.Testing.Extensions.TrxReport</target>
<note>Do not localize format name {Locked="TRX"} or assembly name {Locked="Microsoft.Testing.Extensions.TrxReport"}.</note>
</trans-unit>
<trans-unit id="TrxArtifactPostProcessorDisplayName">
<source>TRX report merger</source>
<target state="new">TRX report merger</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxComparerToolBaselineFileOptionDescription">
<source>The baseline TRX file</source>
Expand Down Expand Up @@ -42,6 +52,51 @@
<target state="translated">Upozornění: Soubor Trx '{0}' již existuje a bude přepsán.</target>
<note>{0} is the {Locked="TRX"} report file path.</note>
</trans-unit>
<trans-unit id="TrxMergeToolDescription">
<source>Merges multiple TRX files into one from the command line</source>
<target state="new">Merges multiple TRX files into one from the command line</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergeToolDisplayName">
<source>TRX report merge tool</source>
<target state="new">TRX report merge tool</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergeToolInputOptionDescription">
<source>Two or more input TRX file paths</source>
<target state="new">Two or more input TRX file paths</target>
<note>Do not localize option name {Locked="--input"} or format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergeToolInvalidInputs">
<source>'--input' expects two or more existing TRX file paths</source>
<target state="new">'--input' expects two or more existing TRX file paths</target>
<note>Do not localize option name {Locked="--input"} or format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergeToolInvalidOutput">
<source>'--output-trx' expects one TRX file path</source>
<target state="new">'--output-trx' expects one TRX file path</target>
<note>Do not localize option name {Locked="--output-trx"} or format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergeToolOutputOptionDescription">
<source>The output TRX file path</source>
<target state="new">The output TRX file path</target>
<note>Do not localize option name {Locked="--output-trx"} or format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergedArtifactDescription">
<source>{0} TRX files merged</source>
<target state="new">{0} TRX files merged</target>
<note>{0} is the number of merged files. Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergedArtifactDisplayName">
<source>TRX report (merged)</source>
<target state="new">TRX report (merged)</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxMergedRunName">
<source>Merged TRX report</source>
<target state="new">Merged TRX report</target>
<note>Do not localize format name {Locked="TRX"}.</note>
</trans-unit>
<trans-unit id="TrxReportArtifactDescription">
<source>Test session</source>
<target state="translated">Relace testu</target>
Expand Down
Loading
Loading