Repository navigation
feat: Add WebAssembly (Blazor) browser test runner with dotnet test integration - #110
Conversation
….Targets Introduces a new MSBuild targets NuGet package that makes dotnet test work for Android and iOS MAUI test apps with a single command: dotnet test MyTests.csproj -f net10.0-android New: DeviceRunners.Testing.Targets NuGet package - build/DeviceRunners.Testing.Targets.props injects DEVICE_RUNNERS_AUTORUN, DEVICE_RUNNERS_PORT, DEVICE_RUNNERS_HOST_NAMES into the app bundle at build time via AndroidEnvironmentVariables / iOSEnvironmentVariables MSBuild items - build/DeviceRunners.Testing.Targets.targets overrides the VSTest target for Android/iOS to invoke the bundled CLI tool; selects the correct self-contained binary for the host RID (win-x64, linux-x64, osx-x64, osx-arm64) - PublishCliTools MSBuild target publishes the CLI self-contained + trimmed for all 4 RIDs before packing the NuGet package New: UseTestRunnerEnvironment() extension method Replaces the compile-time #if MODE_NON_INTERACTIVE_VISUAL block with a runtime env var check. When DEVICE_RUNNERS_AUTORUN is set (injected by the .targets file), the app auto-enables headless mode with TCP result streaming. Updated: sample test project - Removes #if MODE_NON_INTERACTIVE_VISUAL and TestingMode/MODE_NON_INTERACTIVE_VISUAL - Calls UseTestRunnerEnvironment() instead (no-op when env var is absent) - Imports DeviceRunners.Testing.Targets .props and .targets directly for in-repo use Updated: CI workflows (test-tcp-android-linux, test-tcp-ios) - Replaces 2-step dotnet publish + device-runners CLI with a single dotnet test call - Adds PublishCliTools step to build bundled native binaries before dotnet test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dotnet pack triggers PublishCliTools via BeforeTargets=GenerateNuspec, which is the natural way to prepare the NuGet package and its bundled binaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… support - Replace non-existent AndroidEnvironmentVariables ItemGroup with correct _GeneratedAndroidEnvironment items in a BeforeTargets=_GenerateEnvironmentFiles target. The Android SDK compiles these into the native libxamarin-app.so so that Mono makes them available via Environment.GetEnvironmentVariable() at app startup. - Use %3B to escape semicolons in HOST_NAMES value, preventing MSBuild from treating the list separator as an item separator. - Update iOS LaunchAppAsync to accept environmentVariables parameter and inject them using the xcrun simctl SIMCTL_CHILD_* mechanism when provided. - Update iOS TestCommand to pass DEVICE_RUNNERS_* env vars at app launch time (matching how Windows and macOS CLI commands already inject them). - Add DeviceRunners.Cli FDD publish output to tools/ for in-repo testing. Android dotnet test now works end-to-end: 44 tests, 36 pass, 5 intentional fail, TRX written, correct non-zero exit code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix inaccurate comment in MacOSService: 'open --env' does support env vars; direct launch is used for process handle retention and simplicity - Read stderr on xcrun simctl launch failure for better CI diagnostics - Make iOS CI dotnet pack use 'release' consistently with macOS CI - Fix .app bundle discovery: MSBuild Include glob (*.app/) does not reliably match directories on macOS; use Directory.GetDirectories instead — verified working for both iOS and macOS Catalyst - Remove committed binary DLLs from tools/ and add .gitignore so they are regenerated by dotnet pack (PublishCliTools target) - Validated end-to-end: dotnet test works for macOS Catalyst (44 tests, TRX generated) and iOS simulator (44 tests, TRX generated) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rgets Mirror the same simplification already applied to GitHub Actions: - Replace 'dotnet tool install + dotnet publish + device-runners <plat> test' with 'dotnet pack Testing.Targets + dotnet test' - Remove TestingMode=NonInteractiveVisual (replaced by env var auto-config) - Remove manual .app/.apk/.exe discovery steps - All four platforms updated: Android, iOS, macOS, Windows Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- global.json: pin SDK to 10.0.100 with rollForward=latestFeature so any 10.0.* SDK satisfies the requirement. - Directory.Build.props: set ValidateXcodeVersion=false so minor Xcode version mismatches don't break builds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of the raw MSB3073 'command exited with code 1' error:
- All Exec tasks now use IgnoreExitCode=true and capture the exit code
- New _DeviceRunnersReportResults target parses the TRX via XmlPeek
and emits a dotnet-test-style summary line, e.g.:
Failed! - Failed: 5, Passed: 36, Skipped: 3, Total: 44
- On failure, emits a clean Error task message instead of MSB3073
- Fallback error for the case where CLI crashes before writing TRX
- Fix: count skipped via UnitTestResult[@outcome='NotExecuted'] elements
because xunit/nunit skipped tests don't increment Counters/@notExecuted
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…gle Exec
Replace CallTarget with proper DependsOnTargets chains and eliminate
duplicate Exec blocks. New target hierarchy:
VSTest
→ _DeviceRunnersCheckCli (fail fast before building)
→ Build
→ _DeviceRunnersRunTests
→ _DeviceRunnersPrepareArgs
→ _DeviceRunnersCommonArgs (TRX path, device flag, shared CLI flags)
→ _DeviceRunnersAndroidArgs (Condition=android)
→ _DeviceRunnersiOSArgs (Condition=ios)
→ _DeviceRunnersMacOSArgs (Condition=maccatalyst)
→ _DeviceRunnersWindowsArgs (Condition=windows)
→ _DeviceRunnersExecTests (single Exec, consumes $(_DeviceRunnersCliArgs))
→ _DeviceRunnersReportResults (XmlPeek TRX, emit summary)
Each platform arg target sets $(_DeviceRunnersCliArgs); the optional
$(_DeviceRunnersDeviceArg) is built once in _DeviceRunnersCommonArgs and
appended where the CLI supports it (Android, iOS). No duplicate Exec pairs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace CallTarget usage with proper DependsOnTargets chains - VSTest -> _DeviceRunnersCheckCli;Build;_DeviceRunnersRunTests - _DeviceRunnersRunTests -> _DeviceRunnersPrepareArgs;_DeviceRunnersExecTests;_DeviceRunnersReportResults - _DeviceRunnersPrepareArgs -> _DeviceRunnersCommonArgs + 4 platform targets - Introduce _DeviceRunnersPlatform and _DeviceRunnersIsSupportedPlatform properties so all conditions use property comparisons instead of repeated function calls - Single _DeviceRunnersExecTests consumes fully-assembled $(_DeviceRunnersCliArgs) - Single _DeviceRunnersDeviceArg with leading space for clean concatenation - ASCII-only comments (no box-drawing chars, no -- inside XML comments) Validated: dotnet test macOS Catalyst - 44 tests (36 pass, 5 fail, 3 skip) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Emit the standard dotnet test two-line format: Results File: /path/to/test-results.trx Test summary: total: 44, failed: 5, succeeded: 36, skipped: 3, duration: 9.3s Changes: - Output 'Results File:' line first (before summary, not after) - Use 'succeeded' instead of 'passed' to match dotnet test terminology - Read scalar TRX counters as PropertyName (not ItemName) for clean interpolation - Compute duration from TRX Times/@start and Times/@finish attributes - Keep bare 'Test run failed.' Error as the MSBuild failure signal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds Code="TESTERROR" to the two test-failure Error tasks so the output matches what dotnet test emits for failing tests: error TESTERROR: Test run failed. Tools that parse dotnet test output keyed on the TESTERROR code will now correctly recognise device test failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The dotnet CLI terminal logger (default in .NET 8+) filters MSBuild
Message tasks, so Results File and Test summary were invisible at
default verbosity. Fix by embedding the full summary in the TESTERROR
error message which is always surfaced:
error TESTERROR: Test run failed: 5 of 44 tests failed
(36 succeeded, 3 skipped, duration: 9.7s). Results File: /path/...
The Message tasks are kept for --tl:off / --verbosity normal users
where they render as the clean two-line dotnet test format:
Results File: /path/to/test-results.trx
Test summary: total: 44, failed: 5, succeeded: 36, skipped: 3, duration: 9.7s
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use %0A (MSBuild newline escape) to split the error into two lines matching the standard dotnet test output format: error TESTERROR: Results File: /path/to/test-results.trx error TESTERROR: Test summary: total: 44, failed: 5, succeeded: 36, skipped: 3, duration: 18.5s Both lines are always visible via the terminal logger. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Test classes are discovered via reflection at runtime. Without explicit trimmer roots the iOS/Android linker strips the referenced test assemblies (MauiLibrary.XunitTests, Library.NUnitTests), causing only tests in the main DeviceTests assembly to run (~8 instead of ~44). TrimmerRootAssembly tells the linker to keep the full assembly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The event stream protocol sends begin/end events to bracket the test run. If the TCP connection closes after receiving results but without the end event, the app crashed or was killed mid-run. Previously this produced partial results with no indication of a crash. Now the CLI explicitly reports: The application appears to have crashed during the test run. Only 8 test result(s) were received before the connection was lost. Check the device log for crash details. And forces a non-zero exit code even if all received tests passed, so CI pipelines correctly fail. Changes: - EventStreamService: add HasStarted and HasEnded tracking - BaseTestCommand: check for begin-without-end after listener completes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CLI exit codes: 0 = all tests passed 1 = test failures 2 = app crashed (begin event received, results streamed, but no end event) MSBuild targets check the exit code directly (no file reading needed): exit code 2 + TRX: 'Test summary: ... (incomplete: app crashed)' exit code 2 + no TRX: 'The application crashed before producing any test results' exit code 1 + TRX: 'Test summary: ...' (normal failure) Validated: iOS: 8 tests, crash detected, '(incomplete: app crashed)' shown macOS: 44 tests, no crash, normal TESTERROR shown Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the (int testFailures, string? testResults) tuple with a protected record TestListenerResult(int FailedCount, string? ResultsFile, bool Crashed). This makes the crash detection explicit via a bool rather than a sentinel value, and gives clear property names at all call sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Centralise exit code mapping and success logic on the record itself: - ToExitCode(): 0 = success, 1 = test failures, 2 = app crashed - Success: true when no failures and no crash All callers now use listener.ToExitCode() and listener.Success instead of duplicating the conditional logic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MAUI 10.0.20 had a NullReferenceException in Button.LayoutButton when the platform button was null during a layout pass on the UI thread. This crashed the app mid-test-run on iOS, producing only partial results. The fix (dotnet/maui#35284, fixing dotnet/maui#31048) adds a null guard in LayoutButton. MAUI 10.0.60 includes this fix. Results: iOS now completes all 44 tests (previously crashed at 8). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the existing AppleDev predicate parameter on GetLogsPlainAsync to
filter logs to just the app process:
predicate: 'process == "DeviceTestingKitApp.DeviceTests"'
Results:
Before: 17,000 lines of OS-level noise, no managed exceptions
After: 190 lines with all PASS/FAIL output and full managed stack
traces when the app crashes (NullReferenceException, etc.)
Also restores MAUI 10.0.60 (was temporarily reverted for testing).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AppleDev 0.8.10 adds SimCtl.LaunchAppAsync overload with env vars, so we can replace the manual ProcessStartInfo + SIMCTL_CHILD_* workaround with the native API call. AndroidSdk 0.35.1 is the latest stable release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Testing.Targets .props defaults WindowsPackageType=None (for dotnet test unpackaged mode). The TCP Windows packaged and XHarness Windows workflows need MSIX packaging, so pass -p:WindowsPackageType=MSIX explicitly. Fixes: TCP Windows (GH+AzDO), XHarness Windows (GH+AzDO) CI failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New articles: - using-dotnet-test.md: Main guide covering setup, configuration, output format, crash detection, MSBuild properties, and how it works - dotnet-test-android.md: Emulator setup, env var injection, APK config - dotnet-test-ios.md: Simulator setup, device logs, crash troubleshooting - dotnet-test-macos.md: Direct launch, no simulator needed - dotnet-test-windows.md: Unpackaged EXE mode, WindowsPackageType Updated: - toc.yml: 'dotnet test (Recommended)' section at top, CLI renamed to 'Advanced', XHarness renamed to 'Legacy' (from 'CLI Runners') - index.md: Getting Started leads with dotnet test - README.md: dotnet test as primary, links to docs site - technical-architecture-overview.md: Testing.Targets package section with MSBuild target chain, exit code protocol, package structure - ci-pipeline.md: dotnet test CI examples for GH Actions and AzDO Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove WindowsPackageType=None default from .props entirely. Instead, the _DeviceRunnersWindowsArgs target detects what the build produced: - If .exe exists (unpackaged): uses 'windows test --app path.exe' - If .msix exists (packaged): uses 'windows test --app path.msix' - If neither: clear error message This means dotnet publish for MSIX packaging is no longer broken by the Testing.Targets package. CI workflows no longer need -p:WindowsPackageType overrides. Also search both artifacts/ and sample/ for .msix in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Create DeviceRunners.VisualRunners.Core (net9.0;net10.0 only) with the cross-platform testing types that the CLI needs: Moved from VisualRunners: - Testing/ITestResultInfo, ITestCaseInfo, ITestAssemblyInfo, ITestAssemblyConfiguration, TestResultStatus - Testing/Channels/IResultChannel, IResultChannelFormatter, FileResultChannel, FileResultChannelOptions, TestResultEvent, EventStreamFormatter, TextResultChannelFormatter, TrxResultChannelFormatter, TextWriterResultChannel The CLI now references VisualRunners.Core instead of VisualRunners, breaking the dependency on multi-target mobile TFMs. This unblocks self-contained and AOT publish (no more Mono runtime pack errors). VisualRunners references VisualRunners.Core transitively, so all existing consumers are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add TrimmerRootAssembly for Spectre.Console.Cli and DeviceRunners.Cli to preserve types discovered via reflection (command settings classes). Spectre.Console.Cli is explicitly not trim-safe (spectreconsole/spectre.console#1690) so the entire assembly must be rooted. Suppress trim analysis warnings since Spectre.Console.Cli generates unavoidable IL2026/IL2090 warnings that are covered by the rooting. Result: 19MB self-contained single-file binary (down from 21MB multi-file). dotnet publish -r osx-arm64 --self-contained -p:PublishSingleFile=true Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CLI csproj now has SelfContained, PublishSingleFile, PublishTrimmed, and RuntimeIdentifiers (osx-arm64;win-x64) set by default. Plain 'dotnet publish -r osx-arm64' produces a 19MB single-file binary. The NuGet package PublishCliTools target overrides these back to framework-dependent (SelfContained=false, PublishSingleFile=false, PublishTrimmed=false) since the bundled tool runs via 'dotnet <dll>'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The NuGet package now ships self-contained, trimmed, single-file native binaries for each platform in tools/<rid>/: tools/osx-arm64/DeviceRunners.Cli (~20 MB) tools/win-x64/DeviceRunners.Cli.exe (~20 MB) tools/linux-x64/DeviceRunners.Cli (~20 MB) The MSBuild targets detect the host OS and select the right binary. Falls back to framework-dependent DLL at tools/DeviceRunners.Cli.dll for in-repo dev or older packages. Changes: - CLI csproj: add JsonSerializerIsReflectionEnabledByDefault, linux-x64 RID - Testing.Targets csproj: PublishCliTools publishes 3 RIDs (self-contained) - .targets: detect host OS -> pick tools/<rid>/DeviceRunners.Cli[.exe] Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only the per-RID single-file native binary is used now. The fallback to tools/DeviceRunners.Cli.dll is removed since it will never exist. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds first-class WebAssembly (Blazor) browser test support to DeviceRunners. It introduces a Blazor visual runner, a reflection-based xunit discoverer/runner that can run threadlessly in WASM, and CLI commands (wasm test / wasm serve) that serve a published bundle from a built-in static-file server, drive headless Chrome via the Chrome DevTools Protocol, capture NDJSON console output, and convert it into TRX results. CI (GitHub Actions and Azure DevOps) and the DeviceRunners.Testing.Targets MSBuild integration are extended to cover the browser platform, and a new sample browser test project plus shared Blazor library demonstrate end-to-end usage.
Changes:
- New
DeviceRunners.VisualRunners.BlazorRazor Class Library + reflection-based xunit runner (XunitReflectionTestDiscoverer/XunitReflectionTestRunnerplusXunitYielding…Runnercooperative-scheduling stack). - New CLI surface:
wasm test/wasm servecommands withWasmWebServerService(HttpListener static server) andBrowserService(CDP over WebSocket), plus MSBuild targets/CI templates for thebrowserplatform. - Sample apps reorganised: new
DeviceTestingKitApp.BlazorLibrary,DeviceTestingKitApp.BrowserTests,DeviceTestingKitApp.BlazorLibrary.XunitTests; the MAUI library xunit test project is dropped from the solution.
Reviewed changes
Copilot reviewed 79 out of 79 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| src/DeviceRunners.VisualRunners.Xunit/Xunit{Reflection,Yielding…}*.cs, DiagnosticMessageSink.cs, EmptySourceInformationProvider.cs, XunitTestRunner.cs | New reflection-based discoverer/runner stack and yielding runners for threadless xunit execution |
| src/DeviceRunners.VisualRunners.Xunit/VisualTestRunnerConfigurationBuilderExtensions.cs | AddXunit(useReflection:) toggle to select the new runner |
| src/DeviceRunners.VisualRunners.Core/Testing/Channels/{Console,Debug}ResultChannel.cs, src/DeviceRunners.VisualRunners/Testing/Channels/ConsoleResultChannel.cs, …Builder/Extensions.cs | Promote ConsoleResultChannel into Core, add formatter overloads, and new AddResultChannel extension overloads |
| src/DeviceRunners.VisualRunners.Blazor/** | New Blazor Razor Class Library: pages/components, JS interop, builder extensions, app terminator, CSS |
| src/DeviceRunners.Cli/Program.cs, Commands/Wasm/{WasmTestCommand,WasmServeCommand}.cs | Register wasm branch and implement test/serve commands |
| src/DeviceRunners.Cli/Services/{WasmWebServerService,BrowserService}.cs | Static-file server and headless Chrome/CDP driver |
| src/DeviceRunners.Testing.Targets/build/*.{props,targets} | Add browser platform support and _DeviceRunnersWasmArgs target |
| sample/src/DeviceTestingKitApp.BlazorLibrary/**, sample/src/DeviceTestingKitApp.Library/*.csproj, sample/src/DeviceTestingKitApp.MauiLibrary/Features/MauiSemanticAnnouncer.cs | New Blazor sample library, library multi-targets net9/net10, null-arg validation tweak |
| sample/test/DeviceTestingKitApp.BrowserTests/, DeviceTestingKitApp.BlazorLibrary.XunitTests/, DeviceTestingKitApp.DeviceTests/Tests/, DeviceTestingKitApp.MauiLibrary.XunitTests/ | New WASM/browser test project, new bUnit-based component tests, additional xunit tests |
| test/DeviceRunners.VisualRunners.Tests/Testing/{TestDiscovererTests,TestRunnerTests}.XunitReflection.cs | Test fixtures for the new reflection discoverer/runner |
| DeviceRunners.slnx, Directory.Packages.props | Solution/package additions (Blazor library, BrowserTests, bUnit, ASP.NET Core WASM); MAUI xunit test project removed from solution |
| .github/workflows/ci.yml, .github/workflows/test-wasm-browser/action.yml, .azure/azure-pipelines.yml, .azure/templates/test-wasm-browser.yml | New WASM browser test job/stage in GitHub Actions and Azure DevOps |
EmbedAssembliesIntoApk was always set to true when the Testing.Targets package was referenced, disabling Fast Deployment for ALL Android builds. This made normal debug builds from the IDE slow. Now it's set in _DeviceRunnersMarkTestRun (which only runs during dotnet test), so normal dotnet build / dotnet run / IDE builds keep Fast Deployment enabled for fast iteration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace anonymous types and JsonDocument with typed models: - ChromeDebugPage: Chrome /json endpoint response - CdpCommand / CdpCommandWithParams<T>: commands we send - CdpEvent / CdpEventParams / CdpConsoleArg: events we receive - CdpNavigateParams: Page.navigate parameters - BrowserJsonContext: STJ source generator context Eliminates all IL2026 trim warnings without suppression attributes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…st-mobile' into mattleibow/wasm-support
New: - cli-device-runner-for-wasm-using-devicerunners-cli.md - dotnet-test-wasm.md Updated: - technical-architecture-overview.md: add Browser (WASM) platform - ci-pipeline.md: add WASM Browser Tests section - using-devicerunners-cli.md: add WASM commands - index.md: add WASM entries - visual-runner-in-the-ide.md: add Blazor visual runner Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve 16 merge conflicts from main (which now includes PR #106). Fix CS0067 warning-as-error for CanExecuteChanged in ViewModelBindingTests. Fix reflection discoverer test count (PreEnumerateTheories=false yields 6 not 8). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nd docs
A. dotnet test end-to-end for WASM:
- Fix platform detection: use UsingMicrosoftNETSdkBlazorWebAssembly
for Blazor WASM projects (plain net9.0 has no -browser TFM)
- Add Testing.Targets import to BrowserTests.csproj
- Add dotnet-test-wasm composite action (GitHub Actions)
- Add test-dotnet-test-wasm.yml template (Azure DevOps)
- Add dotnet-test-wasm jobs to both CI pipelines
F. Browser console log capture:
- WasmTestCommand writes all console messages to browser-console.log
in the results directory (like logcat.txt for Android)
G. Expanded WASM CLI documentation:
- Added Command Options tables for wasm test and wasm serve
- Added dotnet test section with usage example
- Added troubleshooting sections (Chrome, timeout, logs, headed mode)
- Updated dotnet-test-wasm.md to reflect full dotnet test support
- Updated architecture overview: WASM dotnet test status now ✅
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Restore MauiLibrary.XunitTests to solution (accidentally removed) - Add BlazorLibrary.XunitTests to solution - Fix TestAssemblyPage: move TestResultSelected subscription to OnInitialized to prevent duplicate handlers on re-render - Fix path traversal in WasmWebServerService: append directory separator to root before StartsWith check - Fix silent exception swallowing in WasmWebServerService: log to Debug.WriteLine and Console.Error - Fix BrowserService: clean up temp Chrome user-data directory in DisposeAsync - Fix BrowserService: capture stderr and exit code on Chrome launch failure for better diagnostics - Fix WasmServeCommand: unsubscribe CancelKeyPress handler to prevent leak on repeated invocations - Fix WasmTestCommand: catch OperationCanceledException instead of TaskCanceledException for robustness Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The MSBuild targets previously hardcoded --logger "trx;LogFileName=test-results.trx" in the CLI args. Plain `dotnet test` does NOT produce TRX by default — only when the user passes --logger. This change matches standard dotnet test behavior: - Forward $(VSTestLogger) to the CLI --logger arg when set - Forward $(VSTestResultsDirectory) to --results-directory when set - Omit --logger entirely when not specified (no TRX produced) - Report target already handles missing TRX gracefully All CI dotnet-test workflows now explicitly pass --logger since they use test result reporters (dorny/test-reporter, PublishTestResults) that require TRX files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ReadDevToolsUrlAsync method's CancellationToken caused an OperationCanceledException that bypassed the descriptive error message with stderr capture. Now catches the exception and falls through to build the diagnostic message. Also increased timeout from 15s to 30s since CI runners can be slow to start Chrome. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove duplicate DeviceRunnersTestRun property assignment - Use async CopyToAsync for file serving (avoids blocking on large WASM DLLs) - Fix tab indentation in AdvancedTests.cs and DisposableTests.cs - Improve TotalCount==0 error message with diagnostic guidance - Remove --disable-gpu flag to enable WebGL in headless Chrome (--headless=new supports WebGL via SwiftShader without it) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix Windows XHarness status in architecture overview (❌ → ✅) - Fix WASM troubleshooting advice in dotnet-test-wasm.md - Add Browser (WASM) to all tables in using-dotnet-test.md - Add DeviceRunnersWasmTimeout property to docs - Standardize TFMs to net10.0 across all CLI docs - Standardize platform ordering in index.md - Use "Browser (WASM)" naming consistently in toc.yml - Add iOS and WASM to CLI command patterns - Add cross-links and clarify no -f flag for WASM - Fix basic example to include --logger flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the URL-based AddCliConfiguration(string currentUrl) overload from DeviceRunners.VisualRunners.Blazor to DeviceRunners.VisualRunners where it sits alongside the env-var/CLI-args overload. The method has no Blazor dependencies — it's pure URL parsing + EnableAutoStart + AddResultChannel. The Blazor project still calls it via the JS interop in UseVisualTestRunner. Also remove the duplicate WASM section from 'Additional Resources' in using-devicerunners-cli.md — WASM is already a first-class platform in the Supported Platforms table and Common Command Patterns section. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Each platform has a fundamentally different CLI configuration strategy: - MAUI (MauiCliConfigurationExtensions): reads env vars (Android, iOS, macOS, Windows EXE) and CLI args (Windows MSIX) → adds TCP result channel for native platform communication - Blazor (BlazorCliConfigurationExtensions): parses URL query string (?device-runners-autorun=1) → adds console result channel for browser communication via Chrome DevTools Protocol These are platform concerns, not shared concerns. The shared VisualRunners library keeps the building blocks (EnableAutoStart, AddTcpResultChannel, AddConsoleResultChannel, AddResultChannel). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rm CLI config Rename MauiCliConfigurationExtensions and BlazorCliConfigurationExtensions to VisualTestRunnerConfigurationBuilderExtensions since they are extensions for IVisualTestRunnerConfigurationBuilder. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…late The parameter was 'expected' but validate-arch.yml declares 'expectedArch'. This mismatch caused AzDO to reject the entire pipeline at parse time, preventing all CI jobs from running. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Android: add Free Disk Space step to prevent disk full failures - Android: add Display Android Tool Logs step for debugging - iOS: add architecture validation (validate-arch.yml) - macOS: add architecture validation (validate-arch.yml) These steps were present in the TCP templates but missing from the dotnet-test templates, causing the Android job to fail with 'No connected devices found' due to 95% disk usage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Job-level variables with $(HOME) don't expand correctly for the emulator — it couldn't find DotnetTestEmulator.ini in ANDROID_AVD_HOME. Switch to ##vso[task.setvariable] at runtime (matching the TCP template) so $HOME resolves to the actual home directory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Merge main (Testing.Targets #106, Blazor browser runner #110) - Add browser/WASM platform detection using UsingMicrosoftNETSdkBlazorWebAssembly - Add index.html.template for Blazor WebAssembly projects - Update BrowserTests sample to use Bootstrap for index.html generation - Fix platform detection to use GetTargetPlatformIdentifier() function - Condition icons/splash to MAUI-only (not browser) - Remove unused AndroidManifest property (SDK provides default + overlay) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
What this PR does
This PR adds first-class WebAssembly (Blazor) support to DeviceRunners — the same level of test infrastructure that already exists for Android, iOS, macOS, and Windows. After this PR, you can:
dotnet test— standarddotnet testworkflow, same as every other platformdevice-runners wasm testfor scripted/CI scenariosNo XHarness, Playwright, Selenium, or npm required. Just .NET and Chrome.
How it works
Tests run inside the browser as a Blazor WebAssembly app. The test app discovers and executes xunit tests via reflection (since
XunitFrontControllerrequires filesystem access, which is unavailable in WASM). Results stream as NDJSON throughConsole.Out.On the host side, the CLI launches a local web server, opens headless Chrome via Chrome DevTools Protocol, captures console output, and converts it to TRX.
For
dotnet test, MSBuild targets handle everything automatically — publish the WASM app, launch the CLI, capture results.Quick start
Using
dotnet test(recommended):Using the CLI directly:
dotnet publish MyBlazorTests.csproj -c release device-runners wasm test --app ./artifacts/publish/.../wwwrootUsing the visual runner (development):
Just run the Blazor WASM app — it opens in your browser with an interactive test UI.
What was added
Blazor Visual Runner (
DeviceRunners.VisualRunners.Blazor)A Razor Class Library that provides the same interactive test runner UI as the MAUI visual runner, but rendered in Blazor. Reuses the existing
HomeViewModel,TestAssemblyViewModel, etc. — same ViewModels, different UI layer.WASM Xunit Integration (
DeviceRunners.VisualRunners.Xunit/Wasm/)Reflection-based test discoverer and runner designed for the single-threaded WASM environment:
WasmXunitDiscoverer— usesSynchronousMessageBus(no threads in WASM)WasmXunitAssemblyRunner— cooperative execution viaTask.Yield()to avoid blocking the UICLI Commands
device-runners wasm test— end-to-end: serve app → headless Chrome → capture results → TRXdevice-runners wasm serve— serve a published WASM app for manual browser testingCLI Services
BrowserService— launches headless Chrome, connects via CDP WebSocket, capturesRuntime.consoleAPICalledevents. Cleans up temp directories, handles stderr, 30s launch timeout. WebGL works out of the box (--headless=new+ SwiftShader).WasmWebServerService—HttpListenerstatic file server with WASM MIME types (application/wasm) and cross-origin isolation headers (COOP/COEP) required forSharedArrayBuffer.dotnet testIntegrationMSBuild targets in
DeviceRunners.Testing.Targetsdetect Blazor WASM projects viaUsingMicrosoftNETSdkBlazorWebAssemblyand wire up the publish → CLI → TRX pipeline. Forwards$(VSTestLogger)and$(VSTestResultsDirectory)fromdotnet testto match standard behavior.Platform-Specific
AddCliConfigurationSplit the CLI configuration into platform-specific extensions:
VisualRunners.Maui→AddCliConfiguration()— reads env vars/CLI args → TCP result channelVisualRunners.Blazor→AddCliConfiguration(string url)— reads URL query string → Console result channelVisualRunners— building blocks only (AddTcpResultChannel,AddConsoleResultChannel, etc.)Sample Apps
sample/src/DeviceTestingKitApp.BlazorLibrary/— Blazor component library (Counter, Login)sample/test/DeviceTestingKitApp.BlazorLibrary.XunitTests/— bUnit component testssample/test/DeviceTestingKitApp.BrowserTests/— Blazor WASM visual test runner appDocumentation
CI/CD
WASM Browser+dotnet test WASM BrowserjobsWASM_Browser_Tests+dotnet_test_WASM_Browserjobsdotnet-testtemplates to match TCP templates (disk space cleanup, env var handling, arch validation)Design decisions
XunitFrontControllerrequires filesystem — unavailable in browser WASMSynchronousMessageBus+Task.Yield()$(VSTestLogger)forwardingdotnet testbehavior instead of hardcoding TRX--disable-gpu--headless=newsupports WebGL via SwiftShader automaticallyHttpListener(not Kestrel)Console.OutEventStreamServicepipelineWebGL support
Works out of the box. Headless Chrome with
--headless=newuses SwiftShader for GPU emulation. The--disable-gpuflag was intentionally removed.