Bug: console-setout-races-under-class-parallelism #520

• Work Mode: full-bug

Summary

UtilitiesCS.Test runs under [assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)], and 29 test files in that assembly redirect the process-global Console.Out via Console.SetOut(...) and then restore it in a finally. Because Console.Out is process-global and the redirecting classes are not serialized against one another, two such classes running concurrently on different workers interleave their set/restore pairs: one class's restore clobbers the other's redirect, so the second class's StringWriter captures nothing and its content assertion fails.

Observed failure: PrintTree_WritesIndentedTreeToConsole failed once in a full-suite run and passed on an immediate re-run of the same assembly with no code change.

This is the same defect class as issue #508 (a test whose precondition is unarranged process/thread-global ambient state under class-level parallelism), but with a different global (Console.Out rather than a WPF Dispatcher).

Environment

• OS/version: Windows 11 Pro 10.0.26200
• Runtime: .NET Framework 4.8.1, MSTest via vstest.console.exe (VS18 test platform)
• Command/flags used: pwsh -NoProfile -File scripts/vscode/Invoke-MSTestWithCoverage.ps1 -Configuration Debug
• Data source or fixture: none

Steps to Reproduce

1. Build the solution in Debug.
2. Run the full MSTest suite with coverage (class-level parallelization, 24 workers).
3. Repeat. The failure is probabilistic, not every-run.

Observed on a branch whose only source change was two unrelated files in UtilitiesCS/OutlookObjects/Folder/:

• Full instrumented suite: Total tests: 6397, Passed: 6396, Failed: 1 - PrintTree_WritesIndentedTreeToConsole.
• Immediate re-run of UtilitiesCS.Test alone: Total tests: 4688, Passed: 4688, Failed: 0.

Expected Behavior

Tests produce the same result on every run regardless of which other tests execute concurrently, per .claude/rules/general-unit-test.md Core Principles 1 (Independence) and 4 (Determinism). A test must not depend on, or mutate, process-global state that a concurrently-executing test also mutates.

Actual Behavior

The exact method name PrintTree_WritesIndentedTreeToConsole is defined twice, in two different classes in two different namespaces, and both bodies redirect Console.Out:

• UtilitiesCS.Test/OutlookObjects/DASLFilterParser_Tests.cs:95 - class DASLFilterParser_Tests, namespace UtilitiesCS.Test.OutlookObjects
• UtilitiesCS.Test/OutlookObjects/Filter DASL/DASLFilterParserTests.cs:95 - class DASLFilterParserTests, namespace UtilitiesCS.Test.OutlookObjects.FilterDASL

Both use this shape:

using var writer = new StringWriter();
var originalOut = Console.Out;
Console.SetOut(writer);
try { parser.PrintTree(tree, 0); }
finally { Console.SetOut(originalOut); }
writer.ToString().Should().Contain("AND").And.Contain("  A").And.Contain("  B");

Neither class carries [DoNotParallelize]. Under class-level parallelism the two classes are eligible to run at the same time, so this interleaving is reachable:

1. Class A captures originalOut (the real console) and sets writer_A.
2. Class B captures originalOut (now writer_A) and sets writer_B.
3. Class A finishes and restores the real console - writer_B is now detached.
4. Class B's PrintTree output goes to the real console; writer_B.ToString() is empty; the Contain("AND") assertion fails.

The duplicate method name is a symptom of a broader duplication: DASLFilterParser_Tests.cs and Filter DASL/DASLFilterParserTests.cs appear to be mirrored copies of the same suite.

Logs / Screenshots

• Attached minimal logs or snippet
• Snippet:

  Failed PrintTree_WritesIndentedTreeToConsole [215 ms]
Test Run Failed.
Total tests: 6397
     Passed: 6396
     Failed: 1

Immediate re-run of the same assembly, no code change:

Total tests: 4688
     Passed: 4688

Impact / Severity

• Blocker
• High
• Medium
• Low

Same rationale as #508 and #511: a suite that is not reliably green at baseline prevents anyone from distinguishing a real regression from noise, and trains reviewers and agents to re-run until green. This instance is worse than a single flaky test because the mechanism is shared by 29 files, so the failure can surface in any of them and will be attributed to whichever one loses the race.

Source

From: docs/features/potential/2026-08-08-console-setout-races-under-class-parallelism.md