Skip to content

Fix MTP MSBuild reporting for controller-backed test runs - #12049

Merged
Amaury Levé (Evangelink) merged 2 commits into
mainfrom
dev/amauryleve/mtp-msbuild-regression
Oct 20, 2026
Merged

Amaury Levé (Evangelink) merged 2 commits into
mainfrom
dev/amauryleve/mtp-msbuild-regression

Conversation

@Evangelink

@Evangelink Amaury Levé (Evangelink) commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Pull request

Summary

MTP's MSBuild integration incorrectly suppresses registration in every controller child, so dotnet build -t:Test and InvokeTestingPlatform lose module information and test summaries when TRX, HangDump, or CrashDump starts a controller.

Restrict suppression to actual Windows AppContainer children. Ordinary and full-trust children retain their MSBuild callbacks, while sandboxed children still avoid the inaccessible outer build pipe. Add packed MSTest.Sdk regression cases for both entry points, all three options, and passing/failing tests.

Also initialize the first module's result directory before opening its MSBuild log. Restored full-trust callbacks can announce a default results path before a reporter creates it; otherwise the task fails fast before cooperative cancellation completes.

Fixes: #12032

Design and decision record

Investigation, alternatives, and paired-run evidence.

A controller PID does not imply a sandbox, and package identity does not imply AppContainer isolation. Query the actual process token, guarded for Windows 8+, without allocating a native handle. Native errors surface explicitly. Browser/WASI exclusions and the MSBuild task's required-handshake checks remain unchanged. No public API, dependency, CLI option, or IPC format changes are needed.

Demonstration

A text comparison is clearer than screenshots for this IPC regression. The same 16 package-consuming cases produced:

Packed implementation Controller-backed cases No-controller controls Outer exit
Before 12 failed at missing module information or summary assertions 4 passed 2
Fixed 12 passed 4 passed 0

The passing-test regression reports MSBuild extension failed to receive test application information before the fix. With the fix, passing tests produce the expected summary/TRX and exit 0; intentionally failing tests retain their identity/summary and fail the build with exit 1.

Validation and regression proof

Tested on Windows x64 with pinned SDK 12.0.100-alpha.1.26509.101, Release packs/runners, and Debug/net11.0 generated consumers. Packages: MSTest.Sdk/TestAdapter/TestFramework 4.6.0-dev; MTP/MSBuild/TRX/HangDump/CrashDump 2.6.0-dev.

Baseline: c83be2a87a95cdd8db8ce526d1827c5d480875f7 with the new regression tests retained. Fixed source: a46ec0d7576422c3bf0969490924b397fa4b101f. The committed diff matches tested patch SHA-256 360D9A0ECA36F48E9670D3CFD1CA7D7E5833E7617D995D86F49DC1E157E47731.

Commands from the worktree root ($results is a separate results directory per run):

.\build.cmd -pack -c Release -bl -projects NonWindowsTests.slnf

$acceptance = 'test\IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj'
$filter = 'FullyQualifiedName~InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results'
.\.dotnet\dotnet.exe test --project $acceptance -c Release --no-build --no-launch-profile --filter $filter --minimum-expected-tests 16 -p:UsingDotNetTest=true -p:TestRunnerAdditionalArguments= "-p:ArtifactsTestResultsDir=$results" "-bl:$results\test.binlog" --show-test-results all

$unit = (Resolve-Path 'test\UnitTests\Microsoft.Testing.Platform.MSBuild.UnitTests\Microsoft.Testing.Platform.MSBuild.UnitTests.csproj').Path
$unitFilter = 'FullyQualifiedName~TestingPlatformBuilderHookTests|FullyQualifiedName~MSBuildLifecycleCallbacksTests|FullyQualifiedName~MSBuildConsumerTests|FullyQualifiedName~InvokeTestingPlatformTaskTests'
.\.dotnet\dotnet.exe test --project $unit -c Release --no-build --no-launch-profile --filter $unitFilter --minimum-expected-tests 568 -p:UsingDotNetTest=true -p:TestRunnerAdditionalArguments= "-p:ArtifactsTestResultsDir=$results" "-bl:$results\test.binlog" --show-test-results failed

Pack: baseline and fixed passed, zero warnings/errors. Acceptance: 12 failed/4 passed before; 16 passed after, exits 2/0. Units: 568 passed, 142 each for net8.0/net9.0/net10.0/net11.0 targets, exit 0. No skipped cases. Independent discovery plans and fresh TRX reports were reconciled with the repository execution verifier. All 32 generated consumers matched their variant's packed runtime DLL; effective caches differed between variants.

Also exercised a real packaged AppContainer WinUI consumer through the shipping MSBuild/AppModel controller:

.\.dotnet\dotnet.exe msbuild $generatedProject -t:InvokeTestingPlatform -p:Configuration=Release -p:Platform=x64 -p:RuntimeIdentifier=win-x64 "-p:TestingPlatformPackagedAppTargetPath=$stagedExecutable" "-p:TestingPlatformCommandLineArguments=--report-trx --report-trx-filename $reportName --results-directory $results --timeout 1m" "-bl:$binlog"

The same consumer/driver passed five named package/token/UI tests with each runtime variant, parent exits 0/0. The fixed runtime was taken from the freshly packed net8.0 payload; both DLL hashes were verified. Short NTFS staging avoided unrelated local XAML/AppX path limits. Exact owned registration/staging/cache/alias resources were cleaned up.

The issue evidence artifact contains the detailed assertions, component hashes, and command results; raw binlogs/TRX/provenance are retained in the session. Linux/macOS, the reporter's exact SDK 10.0.402/released 2.5.x setup, and CI were not run locally. A rel/4.5 servicing backport remains separate work.

Test-value evidence

Contract / test Counterfactual Result
Controller handshake/results: InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results Actual old hook, same new tests/assertions and packed consumers Executed: detected. All 12 controller cases fail for the intended missing-information/summary defect before the fix; all 16 cases pass with it.
Ordinary PID handling: AddExtensions_ControllerPidOptionAtAnyPosition_RegistersFullMSBuildIntegration Head-only unit check; sensitivity is established by the packed acceptance row above Three PID-position cases pass on every unit-test target; included in the 568 passing related checks.
Sandbox preservation: PlainTestMethod_HasExpectedTokenIsolation and four other real WinUI tests Same shipping consumer/driver with archived baseline versus fixed runtime DLL Green/green preservation. Five passed before and five after; actual AppContainer token isolation and successful TRX output verified.

CI follow-up: 780b5e31fa605ff8d04be0075abc95270d746026

Build 1642134 passed Windows Debug, Linux Release, and real Windows app-model acceptance, but Windows Release exposed DirectoryNotFoundException in the module log writer. The repair creates the directory inside the existing first-module ownership lock; it does not weaken cancellation, timeout, or handshake checks.

Tested this commit against parent a46ec0d7576422c3bf0969490924b397fa4b101f, retaining identical tests/assertions/selectors. Tested and committed patch SHA-256: 500349E9D35EE83B4B45CAE55355D4DE1A7BFFC5F064788C02C7BB3FB29EF692.

Contract / exact test Before repair Repaired implementation
Execute_ModuleInfoCreatesResultsDirectoryBeforeOpeningLog (missing/existing directory) 4 missing-directory cases fail with DirectoryNotFoundException; 4 existing-directory controls pass across net8-net11; exit 2 Same 8 cases pass; exit 0
ExplicitConsumer_ControllerTimeoutCancelsHostWithoutFalseSuccess Archived parent MSBuild package reproduces the missing-directory exception and unchanged cancellation-marker assertion failure; 1 executed failure, exit 2 Same selector passes, exit 0; marker is still required to contain True

Exact paired selectors, with separate results directories for each variant:

.\.dotnet\dotnet.exe test --project $unit -c Release --no-build --no-launch-profile --filter 'FullyQualifiedName~Execute_ModuleInfoCreatesResultsDirectoryBeforeOpeningLog' --minimum-expected-tests 8 -p:UsingDotNetTest=true -p:TestRunnerAdditionalArguments= "-p:ArtifactsTestResultsDir=$results" "-bl:$results\test.binlog" --show-test-results all

$mstest = (Resolve-Path 'test\IntegrationTests\MSTest.Acceptance.IntegrationTests\MSTest.Acceptance.IntegrationTests.csproj').Path
.\.dotnet\dotnet.exe test --project $mstest -c Release --no-build --no-launch-profile --filter 'FullyQualifiedName~ExplicitConsumer_ControllerTimeoutCancelsHostWithoutFalseSuccess' --minimum-expected-tests 1 -p:UsingDotNetTest=true -p:TestRunnerAdditionalArguments= "-p:ArtifactsTestResultsDir=$results" "-bl:$results\test.binlog" --show-test-results all

Repacked with the same Release command above: zero warnings/errors. Broader validation passed 576 focused units, the original 16 controller regressions, and 12 packed consumer cases covering cancellation, mixed multi-target results, empty selection, suppressed launcher, and custom executable behavior. All executed, with zero skips; fresh TRX reports matched independently discovered plans.

The archived and repaired netstandard MSBuild task payloads have SHA-256 2A02794ECB169FEE916B1983C3FCAE9FEF69FB0035F69035EA89D2C7EF1B8301 and 96944FD644A208372ACC998EFD0F434233147BB27D78280BDF867097AE018E2C. Local long-path launch errors were excluded from proof; the packed consumer runs used an owned short checkout alias and fresh caches. The fixed package was restored after the counterfactual, and the alias was removed. Results and binlogs remain in the session; this local proof does not claim CI success for the new head.

Limit callback suppression to actual AppContainer children so ordinary TRX, HangDump, and CrashDump controller children report module information and test results to MSBuild.

Add package-consuming regression cases for both MSBuild entry points and passing/failing test runs, and update hook registration coverage.

Fixes #12032

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3a559d41-a07c-4671-9391-a28b09697ffc
Copilot AI balanced review requested due to automatic review settings October 20, 2026 13:08
@github-actions github-actions Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Oct 20, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. labels Oct 20, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12049

Compared merge base c83be2a87a95cdd8db8ce526d1827c5d480875f7 with head
a46ec0d7576422c3bf0969490924b397fa4b101f. Evidence applies to this head, not
every intermediate commit.

Test-value evidence

This is a bug fix: AddExtensions previously suppressed MSBuild registration for any controller-backed child (matched only on --internal-testhostcontroller-pid), losing module-info/test-summary IPC for TRX/HangDump/CrashDump-triggered controllers. The fix narrows suppression to actual Windows AppContainer children via a new IsAppContainer() P/Invoke check.

Contract / test Counterfactual Result
AddExtensions on a controller PID in a non-sandboxed (ordinary CI) process → full MSBuild registration: AddExtensions_ControllerPidOptionAtAnyPosition_RegistersFullMSBuildIntegration Pre-fix behavior (if (_.Contains(...)) with no IsAppContainer() guard) would register only MSBuildCommandLineProvider, failing AssertFullRegistration's checks for host/orchestrator lifetimes and the data consumer. Static prediction: detected. The renamed test's assertion set (AssertFullRegistration) is the exact contract that the old code violated; on a non-AppContainer runner (the repo's Linux/macOS/Windows-non-sandboxed CI), this test fails against the pre-fix hook and passes against the fix. Not independently executed by this review.
MSBuild-node controller end-to-end handshake/summary reporting across Test target, InvokeTestingPlatform target, TRX/HangDump/CrashDump options, and pass/fail outcomes: InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results Pre-fix hook (same new test, same packed consumer asset) Static prediction: detected. The test asserts the exact pre-fix defect strings ("MSBuild extension failed to receive test application information" / "...test run summary information") are absent, plus exit code, summary-line, failure-identity, and TRX-outcome content — all of which the PR description reports as failing (12/16 cases) before the fix and passing after. This review did not execute the suite; it relies on the PR's reported red/green evidence, which is methodologically sound but unverified here.
IsAppContainer() false-path / pre-Windows-8 / non-Windows guard No test exercises this branch directly Not applicable / unverified. No unit test targets IsAppContainer() itself (it is private static and platform-gated); coverage is indirect via the two tests above, which only exercise the "not an AppContainer" branch on non-Windows-sandboxed CI. The genuine-AppContainer branch (isAppContainer != 0) and the GetTokenInformation failure/Win32Exception path are untested in this repository and were only validated manually per the PR description (WinUI consumer run, not part of this diff).

Only two distinct contracts are introduced/changed by source; both are covered by the two tests above. The IsAppContainer() true-branch and native-call-failure branch remain an acknowledged gap (manual-only verification, per the PR's own validation section) — flagged as an evidence gap, not an inline defect, since testing real AppContainer token state from this repo's CI is impractical.

🧵 Parallel safety

STATUS: CLEAN (delegated to the parallel-safety specialist)

Test assembly Scope Workers
Microsoft.Testing.Platform.Acceptance.IntegrationTests MethodLevel (Workers = 0) CPU count
Microsoft.Testing.Platform.MSBuild.UnitTests MethodLevel (Workers = 0) CPU count

No cross-test hazards found. The new data-driven acceptance test calls TestAsset.GenerateAssetAsync per data-row instance, which roots each asset under a fresh randomly-named temp subdirectory, so the derived results directory is unique per instance; useMultithreadedMSBuild: false matches every sibling test in the class. RegistrationRecorder in the unit tests is a fresh per-test field instance (never static), so no mutable state crosses test methods. No [DoNotParallelize]/[ResourceLock] was needed, added, or removed.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results.

ResultQualityTestNotesHow to improve
Pass A (90–100) new MSBuildTests_Test.
InvokeTestingPlatform_
Target_With_
AutoRegistered_
Extensions_Should_
Report_Test_
Results
Asserts absence of both pre-fix defect strings, exit code, pass/fail summary, failure identity, and TRX outcome across 16 packed-consumer data rows. None
Pass A (90–100) mod TestingPlatformBuilderHookTests.
AddExtensions_
ControllerPidOptionAtAnyPosition_
RegistersFullMSBuildIntegration
Renamed to match new full-registration contract; AssertFullRegistration verifies provider, host/orchestrator lifetimes, and shared consumer. None

Pseudo-mutation evidence

  • InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results: Mutating the fix back to the unconditional if (_.Contains(TestHostControllerPidOption, ...)) (no IsAppContainer() guard) would route every controller-backed invocation through MSBuildCommandLineProvider only, which — per the PR's own reported baseline run — reproduces the exact "MSBuild extension failed to receive test application information" output the test asserts is absent. Likely killed (inferred); not independently executed by this review.
  • AddExtensions_ControllerPidOptionAtAnyPosition_RegistersFullMSBuildIntegration: Same mutation (dropping && IsAppContainer()) would make AssertFullRegistration's Assert.HasCount(1, registrations.HostLifetimes) and Assert.AreSame(...DataConsumers..., ...SessionLifetimes...) checks fail, since only the command-line provider would be registered. Likely killed (inferred); not independently executed.
  • IsAppContainer() true-branch (isAppContainer != 0) and the GetTokenInformation failure → Win32Exception path: N/A / unverified — no test in this diff exercises these branches from the repository; the PR description reports only a separate manual WinUI-consumer run outside this test suite.

This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety
specialist with /parallel-audit.

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · 214.1 AIC · ⌖ 9.79 AIC · ⊞ 22.3K · ◷

@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Oct 20, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The native sandbox-routing boundary requires human confirmation of runtime evidence and cross-platform validation.

No open issues 🎉

What changed in this PR

Fixes #12032 by restoring MTP’s MSBuild reporting for ordinary controller-backed runs while preserving AppContainer suppression.

Changes:

  • Restricts suppression to Windows AppContainer children using process-token detection.
  • Adds 16 packed acceptance cases covering both MSBuild entry points, controller options, and passing/failing tests.
  • Updates unit expectations for ordinary controller children.

Test-value evidence: The PR reports 12 regression failures before the fix and 16 passing acceptance cases afterward (c83be2a → a46ec0d), plus AppContainer preservation checks. Execution artifacts were not independently verified.

🟡 Confidence: registration and regression coverage Human confirmation remains needed for native AppContainer behavior and cross-platform validation.
File Description
test/​UnitTests/​Microsoft.Testing.Platform.MSBuild.UnitTests/​TestingPlatformBuilderHookTests.cs Expects full registration for ordinary controller children.
test/​IntegrationTests/​Microsoft.Testing.Platform.Acceptance.IntegrationTests/​MSBuildTests.Test.cs Adds packed controller-reporting regression cases.
src/​Platform/​Microsoft.Testing.Extensions.MSBuild/​TestingPlatformBuilderHook.cs Limits registration suppression to AppContainer children.

Review effort: Balanced

@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026
@github-actions github-actions Bot removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Oct 20, 2026
@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Oct 20, 2026
@github-actions

This comment has been minimized.

Restored controller-child registration can report its default results directory before any reporter creates it. Initialize the module log directory in the first-module ownership block so MSBuild does not fail fast before cooperative cancellation completes.

Add real-filesystem coverage for missing and existing directories, preserve first-module ownership, and keep the cancellation acceptance assertion unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3a559d41-a07c-4671-9391-a28b09697ffc
Copilot AI balanced review requested due to automatic review settings October 20, 2026 15:23
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/approved Proposal approved; ready for implementation. labels Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human review should confirm repaired-head CI and Windows AppContainer evidence for the native-token and cross-process reporting paths.

No open issues 🎉

Review effort: Balanced

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12049

Compared merge base c83be2a87a95cdd8db8ce526d1827c5d480875f7 with head
780b5e31fa605ff8d04be0075abc95270d746026. Evidence applies to this head, not
every intermediate commit.

Test-value evidence

This diff contains three distinct behavioral contracts: the Windows-AppContainer-gated MSBuild suppression fix (also present at a prior reviewed commit a46ec0d) and a new follow-up fix — create the first module's results directory before opening its log — added between a46ec0d and 780b5e3.

Contract / test Counterfactual Result
AddExtensions on a controller PID in a non-sandboxed process → full MSBuild registration: AddExtensions_ControllerPidOptionAtAnyPosition_RegistersFullMSBuildIntegration Pre-fix if (_.Contains(...)) with no IsAppContainer() guard would register only MSBuildCommandLineProvider, failing AssertFullRegistration's host/orchestrator/consumer checks. Static prediction: detected. On the repo's non-AppContainer CI, this test fails against the pre-fix hook and passes against the fix. Not independently executed by this review.
MSBuild-node controller end-to-end handshake/summary reporting across both entry points, TRX/HangDump/CrashDump options, and pass/fail outcomes: InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results Pre-fix hook, same packed-consumer assertions Static prediction: detected, consistent with the PR's own reported 12-failed/16-passed before/after acceptance run. Not independently executed here.
First module's results directory is created before the log stream opens: Execute_ModuleInfoCreatesResultsDirectoryBeforeOpeningLog (new) and the strengthened directory assertions in Execute_FirstModuleOwnsOutputFileAndCompleteHandshakeAllowsSuccess / Execute_FailedToolPersistsOutputInDefaultOrModuleOwnedLog Without _fileSystem.CreateDirectory(_moduleInfo.TestResultFolder); in HandleRequestAsync, a module announcing a not-yet-existing nested results directory (real Windows Release builds per the PR's own CI follow-up evidence, Build 1642134) throws DirectoryNotFoundException when StreamWriter/CreateNew tries to open the log file there. Static prediction: detected, corroborated by the author's own reported CI red/green (DirectoryNotFoundException before, pass after) — an author claim, not independently executed by this review. The new test uses a real FileSystem (not the no-op-recording StubFileSystem), which is the right choice: a stub's CreateDirectory wouldn't have reproduced the original bug.
IsAppContainer() true-branch (isAppContainer != 0) and the GetTokenInformation failure → Win32Exception path No test in this diff exercises these branches Not applicable / unverified (unchanged from prior review) — coverage is only indirect via the "not an AppContainer" branch on non-sandboxed CI; the true-AppContainer path was validated only by the author's separate manual WinUI-consumer run, outside this test suite.

All three source-level contracts changed since the merge base are covered by at least one test in this diff; the acknowledged gap (genuine-AppContainer / native-call-failure branches) remains unchanged from the prior review and is impractical to cover from this repository's CI.

🧵 Parallel safety

STATUS: CLEAN (delegated to the parallel-safety specialist)

Test assembly Scope Workers
Microsoft.Testing.Platform.Acceptance.IntegrationTests MethodLevel ([assembly: Parallelize]) 0 (CPU count)
Microsoft.Testing.Platform.MSBuild.UnitTests MethodLevel ([assembly: Parallelize]) 0 (CPU count)

No cross-test hazards found. The new acceptance test's TestAsset.GenerateAssetAsync call allocates a uniquely-named temp directory per data-row instance, and workingDirectory is passed explicitly to the child dotnet process (no Directory.SetCurrentDirectory). The unit-test fixtures (AppHostTaskFixture, RegistrationRecorder, StubFileSystem, the new real FileSystem instance) are all freshly constructed per test with no static/shared state. No parallelization configuration changed between the merge base and head. No [DoNotParallelize]/[ResourceLock] was needed, added, or removed.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results.

ResultQualityTestNotesHow to improve
Pass B (80–89) new InvokeTestingPlatformTaskTests.
Execute_
ModuleInfoCreatesResultsDirectoryBeforeOpeningLog
Uses a real FileSystem to prove the directory is created and the log content is exact, across both existing/missing-directory data rows. None
Pass A (90–100) mod InvokeTestingPlatformTaskTests.
Execute_
FailedToolPersistsOutputInDefaultOrModuleOwnedLog
Simplified to one unconditional directory assertion, matching the fix's now-unconditional directory creation. None
Pass A (90–100) mod InvokeTestingPlatformTaskTests.
Execute_
FirstModuleOwnsOutputFileAndCompleteHandshakeAllowsSuccess
Added assertion confirms the module's results directory is created exactly once before the log is opened. None
Pass A (90–100) new MSBuildTests_Test.
InvokeTestingPlatform_
Target_With_
AutoRegistered_
Extensions_Should_
Report_Test_
Results
Asserts absence of both pre-fix defect strings, exit code, pass/fail summary, failure identity, and TRX outcome across 16 packed-consumer data rows. None
Pass A (90–100) mod TestingPlatformBuilderHookTests.
AddExtensions_
ControllerPidOptionAtAnyPosition_
RegistersFullMSBuildIntegration
Renamed to match the new full-registration contract; AssertFullRegistration verifies provider, host/orchestrator lifetimes, and shared consumer. None

Pseudo-mutation evidence

  • Execute_ModuleInfoCreatesResultsDirectoryBeforeOpeningLog: Removing _fileSystem.CreateDirectory(_moduleInfo.TestResultFolder); from the fix would make the real-FileSystem-backed resultsDirectoryExists: false data row throw DirectoryNotFoundException instead of returning true from Execute(). Likely killed (inferred); not independently executed by this review.
  • InvokeTestingPlatform_Target_With_AutoRegistered_Extensions_Should_Report_Test_Results: Reverting the fix to the unconditional if (_.Contains(TestHostControllerPidOption, ...)) (no IsAppContainer() guard) would reproduce the "MSBuild extension failed to receive test application information" output the test asserts is absent. Likely killed (inferred); not independently executed.
  • AddExtensions_ControllerPidOptionAtAnyPosition_RegistersFullMSBuildIntegration: The same mutation would make AssertFullRegistration's host-lifetime-count and shared-consumer checks fail, since only the command-line provider would be registered. Likely killed (inferred); not independently executed.
  • IsAppContainer() true-branch and the GetTokenInformation failure → Win32Exception path: N/A / unverified — no test in this diff exercises these branches from the repository.

This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety
specialist with /parallel-audit.

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · 274 AIC · ⌖ 14.9 AIC · ⊞ 22.3K · ◷

@microsoft-github-policy-service microsoft-github-policy-service Bot added the state/needs-review Awaiting review from the team. label Oct 20, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. state/approved Proposal approved; ready for implementation. labels Oct 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Final test-triage resolution — Azure Pipelines build 1642265

This is the final resolution for the completed microsoft.testfx build, superseding any earlier preliminary comment for this build.

Build: 1642265 (\Microsoft\testfx), PR head 780b5e31fa605ff8d04be0075abc95270d746026, merge 8905434219e12f80401418bc35fc7a7d382fdd4d.

Finding: Not a test-product failure. Every failed leg (Windows Debug, Windows Release, Linux, Linux Release, Windows application-model acceptance, Build UWP and package-contract acceptance tests) failed at compilation with the identical error:

src/Adapter/MSTestAdapter.PlatformServices/Helpers/TestDataSourceHelpers.cs(106,25): error CS0111: Type 'TestDataSourceHelpers' already defines a member called 'IsSingleTupleArgumentWithExplicitCancellationToken' with the same parameter types

No tests executed on any leg (results.json is empty, candidateCount is 0, failureOrRetryCount is 0), and the corresponding "No test result files matching '*.trx'" warnings confirm the build never reached the test stage. This is a duplicate-member compile error in the current PR's source change, not a flaky test, crash/hang, or duration regression, so it is out of scope for test triage and is not actionable for an issue here — please route to build-failure analysis / fix the duplicate method definition at TestDataSourceHelpers.cs:106.

Next step: Remove or rename the duplicate IsSingleTupleArgumentWithExplicitCancellationToken overload (or the earlier conflicting member) in TestDataSourceHelpers.cs so the type has only one member with that signature, then re-run the pipeline.

No issue created — this is a one-off compile defect tied to the current PR revision, not a durable/test-infrastructure signal.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · copilot · auto · 57.8 AIC · ⌖ 6.94 AIC · ⊞ 9.8K

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Build Failure Analysis

Summary — The Windows app-model acceptance build fails in MSTestAdapter.PlatformServices because the synthetic merge contains two identical IsSingleTupleArgumentWithExplicitCancellationToken declarations; this regression is in the updated base, not this PR's changes.

Root cause 1: overlapping base-branch helper extractions

#12043 added the first named tuple predicate, then #12045 added another predicate with the same name and parameter types. The analyzed merge commit therefore cannot compile and reports CS0111. The PR head itself does not contain either declaration, and this file is outside the PR diff, so an inline suggestion or append-only fix commit on this PR would target the wrong revision.

Affected file / error

Proposed fix

Remove the second duplicate declaration from the base branch, retaining the first implementation already used by IsDataConsideredSingleArgumentValue:

-    // A tuple that supplies the trailing token is a full-arity row, not one object[] value.
-    private static bool IsSingleTupleArgumentWithExplicitCancellationToken(object?[] data, ParameterInfo[] parameters)
-        => parameters.Length == 2
-            && data.Length == 1
-            && TryHandleTupleDataSource(data[0], parameters, out object?[] tupleArguments)
-            && IsDataWithExplicitCancellationToken(tupleArguments, parameters);

Once main carries that correction, refresh/re-run this PR's merge build; no PR-local source change is indicated by this binlog.


Build overview
  • Leg: Windows application-model acceptance, Release
  • Result: failed after 124.6s
  • Failed compilation project: src/Adapter/MSTestAdapter.PlatformServices/MSTestAdapter.PlatformServices.csproj
  • Compiler target/task: CoreCompile / Csc
  • Errors: 1 compiler error plus the aggregate Build failed. message
All actionable MSBuild errors (1)
Code Project File:Line Message
CS0111 MSTestAdapter.PlatformServices TestDataSourceHelpers.cs:106 Duplicate IsSingleTupleArgumentWithExplicitCancellationToken member with identical parameter types.

🤖 Generated by the Build Failure Analysis workflow using (a href="(dev.azure.com/redacted) · analyzed build [1642265]((dev.azure.com/redacted) · PR commit 780b5e31fa605ff8d04be0075abc95270d746026

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · copilot · gpt56 · 97 AIC · ⌖ 9.23 AIC · ⊞ 43.2K · [◷]( · ◷)

@Evangelink
Amaury Levé (Evangelink) merged commit 00c2b84 into main Oct 20, 2026
22 of 27 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/mtp-msbuild-regression branch October 20, 2026 18:04
@github-actions github-actions Bot removed the state/approved Proposal approved; ready for implementation. label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants