Repository navigation
Fix WASM test runner to use connection/data timeouts like other platforms - #158
Open
mattleibow wants to merge 2 commits into
Open
mattleibow wants to merge 2 commits into
mattleibow wants to merge 2 commits into
Conversation
…latforms The WASM runner used a single 300s total wall-clock cap from browser launch that never reset, so a healthy suite that ran longer than 300s was killed mid-stream and (because the aborted tests were never recorded) looked green. Every other platform (android/ios/macos/windows) already inherits and honours `--connection-timeout` (wait for first sign of life) and `--data-timeout` (an inactivity timeout that resets on every event). WASM inherited both but ignored them. This wires WASM up to the same two timeouts: - connection-timeout: wait for the first browser message (app booting/reporting) - data-timeout: inactivity timeout, reset on every browser console message, so a long healthy run keeps going as long as it keeps producing output A run that hits either timeout is now treated as a crash (exit code 2) with a loud red "TEST RUN TIMED OUT" message, instead of a quiet yellow line + Failed=0. MSBuild's exit-2 report wording is updated to mention timeouts. Docs (dotnet-test-wasm, WASM CLI, using-dotnet-test, CLI README) updated and a WasmTimeoutTests unit test added for the timeout-reason messaging. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a97a5ab5-e10b-41a0-bf6a-a85f37b465d3
… path The device (TCP) runner just prints a single generic "timed out" line and has no connection-vs-data distinction. DescribeWasmTimeout made WASM inconsistently more verbose for little benefit, so remove it and print a plain line to match the other platforms. Also drops the now-unused anyMessageReceived flag and the helper's unit tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: a97a5ab5-e10b-41a0-bf6a-a85f37b465d3
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When running large test suites on WASM, the run was killed at exactly 300s even though tests were actively passing — and it didn't fail the run, so it was easy to miss.
The WASM runner used a single total wall-clock cap from browser launch (
--timeout, default 300s) that never reset. So a healthy suite that legitimately runs longer than 300s gets cancelled mid-stream, and because the aborted/never-run tests are simply not recorded, the results showedFailed=0and looked green.Root cause
Every other platform (
android/ios/macos/windows) already inherits and honours two timeouts from the base settings:--connection-timeout(120s) — wait for the first sign of life.--data-timeout(30s) — an inactivity timeout that resets on every event, so a long healthy run keeps going as long as it keeps producing output.WASM inherited both options but ignored them, using only its own total cap. It was just wrong.
Fix
Wire WASM up to the same two timeouts as everything else:
A single resettable
CancellationTokenSourcemodels the connection-then-inactivity window. The old total--timeout/DeviceRunnersTimeoutknob is removed entirely.A run that hits either timeout is now treated as a crash → exit code 2, with a loud red
Test execution TIMED OUT: <reason>message instead of a quiet yellow line followed byFailed=0. MSBuild's exit-2 report wording now mentions timeouts too.Changes
WasmTestCommand— replace the 300s total cap with the connection + data (inactivity) model; loud failure message; disposal-race guard on the reset.BaseTestCommand— reverted the exploratory overall---timeoutoption (kept the existing connection/data timeouts).props/targets— WASM invocation now passes--connection-timeout/--data-timeout; exit-2 error text says "crashed or timed out".dotnet-test-wasm.md, WASM CLI doc,using-dotnet-test.md, CLIREADME.md.WasmTimeoutTestscovering the connection vs. inactivity timeout messaging.Testing
dotnet test DeviceRunners.Cli.Tests— 72 passed.