Skip to content

Backport MSTest and MTP servicing fixes to rel/4.5 - #12287

Closed
Amaury Levé (Evangelink) wants to merge 4 commits into
rel/4.5from
evangelink-4-5-backport-audit
Closed

Amaury Levé (Evangelink) wants to merge 4 commits into
rel/4.5from
evangelink-4-5-backport-audit

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Bring the remaining selected MSTest and Microsoft.Testing.Platform servicing fixes to rel/4.5: avoid unnecessary fixture-discovery assembly loads, publish each Retry TRX once at its final path, and preserve cleanup ownership and failure diagnostics. Also include the IPC disconnect test's deterministic EOF rendezvous.

Target and scope: rel/4.5, based on a39964ce2cf09c4abcbaa7bb8f4e5818638a9de3. The live release comparison is four commits and 44 files; this is not the much larger comparison against main.

Backports

Upstream Included correction Local commit
#12269 Metadata-first fixture discovery, retaining passive/transitive providers, runtime binding authority, local-hook precedence and diagnostics. 039c61b6c880b7a54b9bafb5cf9c6d67ab7005d0
#12241 Coordinated Core/Retry/TRX/PackagedApp artifact publication. Remove only identified raw final-attempt TRX after successful relocation, then advertise existing final paths. Preserve earlier attempts, attachments, other report formats and failure handling. 6be7eec3169d5324fdcf75c425b5e0e16a46fadf
#12225 Later cleanup review corrections only: retain owned cleanup completion and original deadlines/run cancellation, preserve independent failures, and format nested Framework aggregate messages. The original isolation fix is already in the branch via #12220. 601527fac8ea3e03811f0a433093b7a10099e3ce
#12252, IPC portion only Wait for exclusive server-side transport EOF while the reply remains gated. Five test-setup lines; no production change or weaker assertions. 826ef0f11a51e59861ceeb6e994917587add82d5

These are adapted servicing ports, not wholesale main-branch cherry-picks. Keep the release's .NET 8/.NET 9, Framework, UWP and WinUI targets and existing public/shipped API baselines. Fixture metadata inspection adds System.Reflection.Metadata 8.0.0 for the older targets that need it. Retry compatibility overloads remain; additive internal members are tracked.

Rationale and binding/compatibility decisions are in the fixture discovery decision record and the originating PRs above. Main-only retargeting, F# changes, coverage protocol changes, the controller-host fixture portion of #12252, and optional CI changes #12237/#12251 are excluded.

Validation

Product validation is bound to 601527fac8ea3e03811f0a433093b7a10099e3ce. The final head 826ef0f11a51e59861ceeb6e994917587add82d5 adds only the separately validated IPC test setup; shipping sources and packages are unchanged.

  • Full repository Release build/pack: exit 0, zero warnings/errors, all retained targets and package metadata/content checks.
  • Focused per-backport unit matrices: 65 fixture, 102 Retry/Core, and 732 cleanup executions passed across their release targets, with no skips.
  • Freshly packed combined consumers: 95 MSTest fixture/cleanup cases and 17 MTP Retry/report cases passed, exit 0, no skips. Source-planned names were checked against independent discovery and parsed TRX results.
  • Actual shipping SDK 10.0.401, using a fresh-cache rebuilt ordinary net10 consumer: recovery exit 0 and persistent-failure expected exit 2. Each run has exactly two physical reports, four unique executions, exact associated names/outcomes, and advertised paths equal to existing final paths.
  • Both controller layouts: all eight embedded producer DLL hashes match the correct current NuGet assets; package repository metadata matches the product-validation commit.
  • IPC: the full class passes 48 Windows executions across net8.0/net9.0/net462 and 32 Ubuntu executions across net8.0/net9.0, plus 20 Ubuntu net9.0 repetitions. TRX/JUnit/CTRF reports reconcile. Existing 32-worker method-level scheduling and per-test pipe ownership are unchanged.

Test-value evidence

Requirement Evidence
Discover valid providers without loading unmarked dependencies or overriding runtime resolution. PassiveProvidersAreDiscoveredWithoutExecutingReferenceMethods, UnmarkedDependenciesRemainUnloadedAndDeletable, custom-ALC/Framework binding cases and existing provider controls: 82 packaged cases pass. The eager-loading counterfactual executes the same 82 cases and fails 24 intended loading/intermediary assertions; 58 compatibility controls pass.
Publish each attempt's TRX once, with correct paths, identities and attachments. RetryTrx_PersistentFailure, RetryTrx_Recovery, both RetryTrx_Native* cases, threshold/disabled/absolute-path/attachment cases and CTRF/HTML/JUnit controls: 17 pass. Copy-only behavior fails eight duplicate-report assertions. Independently restoring old Core notification behavior fails both native stale-path assertions.
Complete owned cleanup, preserve deadlines and retain both failures/method identities. TestMethodInfoTests, TestResultTests, GlobalCleanup_PreservesLocalTimeoutWhenGlobalCleanupThrows and NetFrameworkCleanupFailureMessagesRetainBothMethodIdentities: 732 focused unit executions and 13 packaged consumers pass. Actual pre-review production with the same tests compiles/packs and fails 12 lifecycle and four consumer assertions; remaining controls pass.
Stabilize disconnect-before-reply without weakening the diagnostic or shutdown oracle. SingleConnectionNamedPipeServer_ClientDisconnectsBeforeReply_LogsConciseClosure: unchanged Windows baseline passes 48 cases. Repaired tests pass 80 Windows/Ubuntu class executions and 20 Linux repetitions. Holding tests byte-identical and appending a diagnostic marker produces the intended exact-equality failure on both platforms, exit 2; restoring production makes the full class pass again on both, exit 0. This is diagnostic fault sensitivity, not reproduction of the original intermittent timeout.

Counterfactuals retained the regression tests and their dependencies. Restore/build/launch failures, zero-test runs and skipped controls are not counted as behavioral proof. Detailed patches, hashes, binlogs and reports are retained locally, not publicly attached.

Executed commands and results

All commands used Release configuration. Repository builds used the bootstrapped pinned SDK 12.0.100-alpha.1.26480.103. Machine-local results roots and unique binlog filenames are parameterized below; the consumer filters and runner arguments are preserved.

$dotnet = '.\.dotnet\dotnet.exe'
$results = '<task-owned-results-directory>'
$binlog = '<unique-binlog-path>'

# Product-validation head: passed, exit 0, zero warnings/errors.
.\build.cmd -pack -c Release -msbuildEngine vs "/bl:$binlog"

# 95 passed, 0 failed/skipped; exit 0.
& $dotnet artifacts\bin\MSTest.Acceptance.IntegrationTests\Release\net11.0\MSTest.Acceptance.IntegrationTests.dll `
    --filter 'FullyQualifiedName~AssemblyFixtureProviderDiscoveryTests|FullyQualifiedName~AssemblyFixtureProvider_FromReferencedLibrary_RunsAssemblyInitializeAndCleanup|FullyQualifiedName~CleanupCancellationIsolationTests' `
    --report-trx --report-trx-filename final-mstest.trx --results-directory $results `
    --no-ansi --progress off --show-test-results failed

# 17 passed, 0 failed/skipped; exit 0.
& $dotnet artifacts\bin\Microsoft.Testing.Platform.Acceptance.IntegrationTests\Release\net11.0\Microsoft.Testing.Platform.Acceptance.IntegrationTests.dll `
    --filter 'FullyQualifiedName~RetryFailedTests_TrxReports_ArePublishedOncePerAttempt|FullyQualifiedName~RetryFailedTests_CtrfPreservesTrxNamedAttachments|FullyQualifiedName~RetryFailedTests_NativeSdkPublishesExistingTrxPaths|FullyQualifiedName~RetryFailedTests_CtrfReports_ExhaustedRetriesPreserveIdentities|FullyQualifiedName~RetryFailedTests_CtrfAbsoluteFileName_PublishesConsolidatedReport|FullyQualifiedName~RetryFailedTests_HtmlAndJUnitReports_AreConsolidatedWithRetryHistory' `
    --report-trx --report-trx-filename final-retry.trx --results-directory $results `
    --no-ansi --progress off --show-test-results failed

# IPC fixed/fault/restored builds: each passed, exit 0, zero warnings/errors.
# Each build used a different binlog.
.\build.cmd -c Release -projects test\UnitTests\Microsoft.Testing.Platform.UnitTests\Microsoft.Testing.Platform.UnitTests.csproj `
    -msbuildEngine vs /p:EnableCodeCoverage=false "/bl:$binlog"

# Windows net8.0/net9.0: 16 passed each; net462 EXE: 16 passed; all exit 0.
# Same built portable DLLs under Ubuntu .NET 8.0.30/9.0.12: 16 passed each, exit 0.
$module = '<built-module-path-for-the-selected-TFM>'
& $dotnet $module --filter 'FullyQualifiedName~Microsoft.Testing.Platform.UnitTests.IPCTests.' `
    --report-trx --report-trx-filename unit.trx `
    --report-junit --report-junit-filename unit.xml `
    --report-ctrf --report-ctrf-filename unit.json `
    --results-directory $results --no-ansi --progress off --show-test-results failed

The separate real SDK 10 run used SDK 10.0.401 selected from an isolated SDK root/working directory, a generated ordinary consumer rebuilt against fresh final-package bits, and METHOD1=1, RESULTDIR=<results>. FAIL=0 passed with exit 0; FAIL=1 produced the expected persistent-failure exit 2:

dotnet test --test-modules RetryFailedTests.dll --root-directory <consumer-bin> --results-directory <results> --report-trx --retry-failed-tests 1 --no-ansi --no-progress

Remaining validation gaps

Actual AppX activation is blocked locally by the non-elevated environment and an existing preflight installationPath error. Controller package/layout verification is not activation proof. Hosted Azure DevOps/IDE rendering and CI for the published head have not been verified. No auto-merge is requested.

No visual media is attached: the relevant evidence is executable assembly-loading, lifecycle, report-identity and path assertions rather than a UI/layout change. Temporary comparison worktrees, the isolated native consumer/cache and temporary SDK were removed safely; shared caches and the parent toolchain were left intact.

Adapt #12269 (127798d) while retaining release target assets and package dependencies. Include the same binding/loading regressions and verified release red/green evidence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adapt #12241 (f76d452) across Core, Retry, TRX and both release PackagedApp controller layouts. Preserve compatibility overloads, release framework assets and existing SDK wire models; retain focused upstream failure-path and package-consumer regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Apply the servicing follow-up from #12225 (upstream b3f3c98) on top of the original #12220 release isolation fix. Preserve release file layout and targets, cleanup ownership/deadlines, independent failures and Framework aggregate messages.

Retained tests execute against actual prior production: 12 of 136 lifecycle cases and 4 of 13 packed consumers fail for the intended missing corrections. Corrected code passes 732 focused unit executions, all 13 packaged consumers, and both 10-case source-generation opt-out plans.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Port only the IPC test portion of #12252 (upstream 19c516f). Observe server-side transport EOF while the reply is gated; retain exact request, closure diagnostic and server-loop completion assertions. No production, controller-host fixture, CI, API, package or scheduling changes.

The full IPC class passes 48 Windows executions across net8/net9/net462 and 32 Ubuntu executions across net8/net9, plus 20 Ubuntu repetitions. Byte-identical repaired tests reject an intentionally wrong closure diagnostic on both platforms and pass after production restoration. This is retained fault sensitivity, not reproduction of the original intermittent timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the state/needs-review Awaiting review from the team. label Nov 9, 2026
@Evangelink

Copy link
Copy Markdown
Member Author

Superseded at the author's request by four independent, single-commit PRs so each backport can be squash-merged separately. All target rel/4.5 directly (not each other):

  • #12291: metadata-first fixture discovery.
  • #12290: coordinated Retry/TRX artifact publication.
  • #12292: cleanup cancellation review corrections.
  • #12289: IPC disconnect test EOF rendezvous.

Verified the four independent commit trees and sole release parents; their disjoint 10/27/6/1-file scopes exactly preserve this PR's 44-file bundle. Each replacement has fresh standalone validation and documented remaining gaps. Closing this combined PR without merging; its branch and commit history are retained. No PR was merged or put on auto-merge.

@microsoft-github-policy-service microsoft-github-policy-service Bot removed the state/needs-review Awaiting review from the team. label Nov 9, 2026
@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Final test-triage resolution — Azure Pipelines build 1671144

Conclusion: deterministic test failure tied to this PR (high confidence); no issue created.

Signature: Assert.IsNotNull(adapterPayload) in CleanupCancellationIsolationTests.cs (line 49) in MSTest.Acceptance.IntegrationTests (net11.0).

  • Affected legs: Linux Release and Windows Release.
  • Affected tests: the same 12 GlobalCleanup_* tests fail on both legs (24 failures): Normal, UnrelatedLocalFailure, LocalCancellationWithoutException, LocalCancellationException, CancellationWithUnrelatedLocalFailure, LocalCooperativeTimeout, PreservesLocalTimeoutWhenGlobalCleanupThrows, LocalCooperativeTimeout_NonCooperativeTest, TerminalCancellation_NonCooperativeTest and the three _SourceGeneration variants.
  • Retries: the GlobalCleanupAfterLocalCancellationReleasesOwnedResource test failed on every attempt I could see (attempts 1–8 in the TRX/JUnit report). That is a stable failure, not flakiness. I did not see a fail-then-pass recovery.
  • Likely cause: the test host's adapter payload is missing, so the test could not read the cleanup result. This points at the backported cleanup/cancellation behavior or its test assets not matching on rel/4.5. I did not inspect the PR diff in detail, so this is not a confirmed root cause.

Not actionable:

  • Windows App Model Diagnostics: RetryFailsFirstAttempt, NativeTests.Outcome and RetryTestMethod_PassesOnSecondAppContainerHost failures come from deliberate probe and diagnostic fixtures. Their messages say they fail the first attempt on purpose ("expected host failure"). They are not product defects.
  • Slowness: 0 slow regressions.
  • Crash/hang: no diagnostics were found.

Evidence completeness: the evidence is complete. History covered 0 prior builds, so I can't say whether this signature has occurred before on main. A durable issue would need recurrence on main or scheduled builds.

Next step: run CleanupCancellationIsolationTests locally against this branch. Check whether the backport carries the matching test host and adapter changes from main, then fix the backport or its test assets.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · copilot · auto · ~32 AIC · ⌖ ~1 AIC · ⊞ 10K

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants