[browser][MT] various WebWorkerTest CI failures #96628

Activity

1. stephentoub

   added

   arch-wasmWebAssembly architectureWebAssembly architecture

   blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'

   Known Build ErrorUse this to report build issues in the .NET Helix tabUse this to report build issues in the .NET Helix tab

   on Jan 8, 2024
3. ghost added

   area-Infrastructure-libraries

   on Jan 8, 2024
4. ghost added

   untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

   on Jan 8, 2024
5. stephentoub commented on Jan 8, 2024

   

   stephentoub

   on Jan 8, 2024 · Hidden as resolved

   Last edited by pavelsavara

   Author

   show commentMore actions
6. radical commented on Jan 8, 2024

   

   radical

   on Jan 8, 2024

   Member

   More actions

   cc @pavelsavara
7. pavelsavara

   self-assigned this

   on Jan 8, 2024
8. pavelsavara commented on Jan 8, 2024

   

   pavelsavara

   on Jan 8, 2024

   Member

   More actions

   thanks
9. radical

   added

   area-System.Runtime.InteropServices.JavaScript

   and removed

   area-Infrastructure-libraries

   on Jan 8, 2024
10. pavelsavara

    added this to the 9.0.0 milestone on Jan 8, 2024
11. ghost removed

    untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

    on Jan 8, 2024
12. pavelsavara commented on Jan 14, 2024

    

    pavelsavara

    on Jan 14, 2024 · Hidden as resolved

    show commentMore actions
13. 6 remaining items

    Load more

14. pavelsavara commented on Jan 18, 2024

    

    pavelsavara

    on Jan 18, 2024

    Member

    More actions

    Log

    [15:03:57] info: [2024-01-17T15:03:57.680Z] [FAIL] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WebSocketClient_ResponseCloseInDifferentThread(executor1: JSWebWorker, executor2: JSWebWorker)
    [15:03:57] info: System.OperationCanceledException : The operation was canceled.
    [15:03:57] info:    at System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.<ActionsInDifferentThreads>d__20`1[[System.Net.WebSockets.ClientWebSocket, System.Net.WebSockets.Client, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
    [15:03:57] info: --- End of stack trace from previous location ---
    
15. pavelsavara commented on Jan 19, 2024

    

    pavelsavara

    on Jan 19, 2024

    Member

    More actions

    Log
    Build

    [06:40:32] info: [0x55fc058-worker] MONO_WASM [0x55fc058-worker]: forceDisposeProxies done: 0 imports, 0 exports, 0 GCHandles, 0 JSHandles.
    [06:40:33] info: [2024-01-19T06:40:33.266Z] [FAIL] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WebSocketClient_ResponseCloseInDifferentThread(executor1: Main, executor2: NewThread)
    [06:40:33] info: System.OperationCanceledException : The operation was canceled.
    [06:40:33] info:    at System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.<ActionsInDifferentThreads>d__20`1[[System.Net.WebSockets.ClientWebSocket, System.Net.WebSockets.Client, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
    [06:40:33] info: --- End of stack trace from previous location ---
    [06:40:33] info: [0x5c30040-worker] MONO_WASM [0x5c30040-worker]: forceDisposeProxies done: 0 imports, 0 exports, 0 GCHandles, 0 JSHandles.
    
16. pavelsavara commented on Jan 19, 2024

    

    pavelsavara

    on Jan 19, 2024

    Member

    More actions

    Log
    Build

    [19:21:02] warn: [2024-01-18T19:21:02.523Z] [ERROR] FATAL UNHANDLED EXCEPTION: System.OperationCanceledException: The operation was canceled.
    [19:21:02] warn:    at System.Runtime.InteropServices.JavaScript.JSSynchronizationContext.Post(SendOrPostCallback , Object )
    [19:21:02] warn:    at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.PostAction(Object state)
    [19:21:02] warn:    at System.Threading.Tasks.AwaitTaskContinuation.RunCallback(ContextCallback , Object , Task& )
    [19:21:02] warn: --- End of stack trace from previous location ---
    [19:21:02] warn:    at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
    [19:21:02] warn:    at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi)
    [19:21:02] warn:    at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[QueueUserWorkItemCallback](ExecutionContext , Action`1 , QueueUserWorkItemCallback& )
    [19:21:02] warn:    at System.Threading.QueueUserWorkItemCallback.Execute()
    [19:21:02] warn:    at System.Threading.ThreadPoolWorkQueue.Dispatch()
    [19:21:02] warn:    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject)
    [19:21:02] warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore , WaitEntry )
    [19:21:02] warn:    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData)
    [19:21:02] info: [0x5000368-worker] MONO_WASM [0x5000368-worker]: forceDisposeProxies done: 0 imports, 0 exports, 0 GCHandles, 0 JSHandles.
    [19:21:02] info: [2024-01-18T19:21:02.666Z] [FAIL] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WebSocketClient_CancelInDifferentThread(executor1: NewThread, executor2: JSWebWorker)
    [19:21:02] info: System.OperationCanceledException : The operation was canceled.
    [19:21:02] info:    at System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.<ActionsInDifferentThreads>d__20`1[[System.Net.WebSockets.ClientWebSocket, System.Net.WebSockets.Client, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
    [19:21:02] info: --- End of stack trace from previous location ---
    [19:21:02] info: [0x49e0000-worker] MONO_WASM [0x49e0000-worker]: forceDisposeProxies done: 6 imports, 0 exports, 0 GCHandles, 0 JSHandles.
    
17. radekdoulik commented on Jan 19, 2024

    

    radekdoulik

    on Jan 19, 2024 · Hidden as resolved

    show commentMore actions
18. pavelsavara commented on Jan 20, 2024

    

    pavelsavara

    on Jan 20, 2024

    Member

    More actions

    This is probably timeout in mono_exit while waiting for all managed threads to join.

    Log
    Build

    [10:55:09] info: === TEST EXECUTION SUMMARY ===
    [10:55:09] info: Total: 534, Errors: 0, Failed: 0, Skipped: 1, Time: 24.0933498s
    [10:55:09] info: 
    [10:55:10] info: Received expected 147597 of /root/helix/work/workitem/uploads/xharness-output/testResults.xml
    [10:55:10] info: Finished writing 147597 bytes of RESULTXML
    [10:55:10] info: Xml file was written to the provided writer.
    [10:55:10] info: Tests run: 534 Passed: 533 Inconclusive: 0 Failed: 0 Ignored: 0 Skipped: 1
    [10:55:10] info: test-main.js exiting WasmTestRunner.dll System.Runtime.InteropServices.JavaScript.Tests.dll with result 0
    ...
    [10:55:10] info: MONO_WASM [0xdcd5c-main]: forceDisposeProxies done: 242 imports, 0 exports, 38 GCHandles, 107 JSHandles.
    [11:24:42] fail: Tests timed out. Killing driver service pid 79
    [11:24:42] fail: Application has finished with exit code TIMED_OUT but 0 was expected
    XHarness exit code: 71 (GENERAL_FAILURE)
    
19. pavelsavara

    added a commit that references this issue on Jan 20, 2024

    ActiveIssue dotnet#96628

    3e989ca
20. pavelsavara commented on Jan 21, 2024

    

    pavelsavara

    on Jan 21, 2024 · Hidden as resolved

    show commentMore actions
21. pavelsavara commented on Jan 24, 2024

    

    pavelsavara

    on Jan 24, 2024

    Last edited by pavelsavara

    Member

    More actions

    This is ves_icall_System_Environment_FailFast -> "Process terminated."

    Log
    Build

    [21:30:38] info: [2024-01-23T21:30:38.609Z] [STRT] System.Runtime.InteropServices.JavaScript.Tests.JSImportExportTest.JsImportTaskTypes
    [21:30:38] warn: [0x1f76248-worker][2024-01-23T21:30:38.688Z] [MONO] /__w/1/s/src/mono/mono/metadata/icall.c:6171 <disabled>
    [21:30:38] warn: [0x1f76248-worker][2024-01-23T21:30:38.689Z] [MONO] /__w/1/s/src/mono/mono/metadata/icall.c:6180 <disabled>
    [21:30:38] warn: [0x1f76248-worker] Aborted()
    [21:30:38] info: WASM EXIT 1
    

    This seems to be finalizer thread after mono_exit
22. pavelsavara commented on Jan 26, 2024

    

    pavelsavara

    on Jan 26, 2024

    Member

    More actions

      info: [STRT] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.JSDelay_ConfigureAwait_True(executor: JSWebWorker)
      warn: worker sent an error! http://127.0.0.1:61746/_framework/dotnet.native.worker.js:176: Uncaught Error: Assert failed: mono runtime didn't start yet
      warn: [0x04440000-jsww 15:50:44.309] MONO_WASM: JSSynchronizationContext is still installed on worker 0x4440000.
      warn: [0x000f1d10 15:50:44.309] MONO_WASM: mono_wasm_exit failed unwind
      fail: Error: [MONO] * Assertion at C:/Dev/runtime/src/mono/mono/mini/interp/interp.c:155, condition `stack->current == stack->first && stack->current->pos == (guint8*)&stack->current->data' not met
    
          at mono_wasm_stringify_as_error_with_stack (http://127.0.0.1:61746/_framework/dotnet.runtime.js:1181:18)
          at mono_wasm_trace_logger (http://127.0.0.1:61746/_framework/dotnet.runtime.js:1200:27)
          at wasm_trace_logger (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[18498]:0x35af37)
          at eglib_log_adapter (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[959]:0x5e3fc)
          at monoeg_g_logstr (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[835]:0x5a074)
          at monoeg_g_logv_nofree (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[833]:0x59fc9)
          at monoeg_assertion_message (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[837]:0x5a0f4)
          at mono_assertion_message (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[838]:0x5a152)
          at interp_free_context (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[201]:0x28a3b)
          at free_jit_tls_data (http://127.0.0.1:61746/_framework/dotnet.native.wasm:wasm-function[9157]:0x1fb495)
      warn: worker sent an error! http://127.0.0.1:61746/_framework/dotnet.native.worker.js:176: Uncaught unwind
      fail: [out of order message from the browser]: http://127.0.0.1:61746/_framework/dotnet.native.worker.js 175 Uncaught Error: Assert failed: mono runtime didn't start yet
      fail: [out of order message from the browser]: http://127.0.0.1:61746/_framework/dotnet.native.worker.js 175 Uncaught unwind
      info: WASM EXIT 1
    

    locally on my box
23. pavelsavara commented on Jan 26, 2024

    

    pavelsavara

    on Jan 26, 2024

    Last edited by pavelsavara

    Member

    More actions

    [STRT] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WebSocketClient_CancelInDifferentThread(executor1: JSWebWorker, executor2: ThreadPool)
    dotnet.runtime.js:1124 MONO_WASM: forceDisposeProxies done: 2 imports, 0 exports, 1 GCHandles, 0 JSHandles.
    dotnet.native.js:993 Unexpected test case WebSocketClient_CancelInDifferentThread timeout after 00:00:05.0040000 ManagedThreadId:495
    dotnet.native.js:1009 [ERROR] FATAL UNHANDLED EXCEPTION: System.OperationCanceledException: The operation was canceled.
    dotnet.native.js:1009    at System.Runtime.InteropServices.JavaScript.JSSynchronizationContext.Post(SendOrPostCallback , Object ) in C:\Dev\runtime\src\libraries\System.Runtime.InteropServices.JavaScript\src\System\Runtime\InteropServices\JavaScript\JSSynchronizationContext.cs:line 174
    dotnet.native.js:1009    at System.Threading.Tasks.SynchronizationContextAwaitTaskContinuation.PostAction(Object state) in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\TaskContinuation.cs:line 428
    dotnet.native.js:1009    at System.Threading.Tasks.AwaitTaskContinuation.RunCallback(ContextCallback , Object , Task& ) in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\TaskContinuation.cs:line 697
    dotnet.native.js:1009    at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state) in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\Task.cs:line 1925
    dotnet.native.js:1009    at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi) in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ThreadPoolWorkQueue.cs:line 1197
    dotnet.native.js:1009    at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[QueueUserWorkItemCallback](ExecutionContext , Action`1 , QueueUserWorkItemCallback& ) in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ExecutionContext.cs:line 307
    dotnet.native.js:1009    at System.Threading.QueueUserWorkItemCallback.Execute() in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ThreadPoolWorkQueue.cs:line 1213
    dotnet.native.js:1009    at System.Threading.ThreadPoolWorkQueue.Dispatch() in C:\Dev\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ThreadPoolWorkQueue.cs:line 913
    dotnet.native.js:1009    at System.Threading.PortableThreadPool.WorkerThread.WorkLoopSemaphoreSuccess(LowLevelLifoAsyncWaitSemaphore semaphore, Object stateObject) in C:\Dev\runtime\src\mono\System.Private.CoreLib\src\System\Threading\PortableThreadPool.WorkerThread.Browser.Threads.Mono.cs:line 84
    dotnet.native.js:1009    at System.Threading.LowLevelLifoAsyncWaitSemaphore.InternalAsyncWaitSuccess(LowLevelLifoAsyncWaitSemaphore , WaitEntry ) in C:\Dev\runtime\src\mono\System.Private.CoreLib\src\System\Threading\LowLevelLifoAsyncWaitSemaphore.Browser.Threads.Mono.cs:line 159
    dotnet.native.js:1009    at System.Threading.LowLevelLifoAsyncWaitSemaphore.SuccessCallback(IntPtr lifoSemaphore, IntPtr userData) in C:\Dev\runtime\src\mono\System.Private.CoreLib\src\System\Threading\LowLevelLifoAsyncWaitSemaphore.Browser.Threads.Mono.cs:line 211
        at WebSocket.local_on_error (dotnet.runtime.js:5816:23)
    

    another local
24. pavelsavara commented on Jan 31, 2024

    

    pavelsavara

    on Jan 31, 2024 · Hidden as resolved

    show commentMore actions
25. pavelsavara commented on Feb 2, 2024

    

    pavelsavara

    on Feb 2, 2024

    Last edited by pavelsavara

    Member

    More actions

    Log

    [02:57:19] info: [STRT] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WaitAssertsOnJSInteropThreads(executor: JSWebWorker)
    [02:57:19] info: [0x0452ffe8-jsww 02:57:19.193] MONO_WASM: forceDisposeProxies done: 0 imports, 0 exports, 0 GCHandles, 0 JSHandles.
    [02:57:19] fail: [FAIL] System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WaitAssertsOnJSInteropThreads(executor: JSWebWorker)
    [02:57:19] info: Assert.NotNull() Failure: Value is null
    [02:57:19] info:    at System.Runtime.InteropServices.JavaScript.Tests.Executor.AssertBlockingWait(Exception exception)
    [02:57:19] info:    at System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.<>c__DisplayClass16_0.<WaitAssertsOnJSInteropThreads>b__0()
    [02:57:19] info:    at System.Runtime.InteropServices.JavaScript.Tests.Executor.<>c__DisplayClass7_0.<Execute>g__wrapExecute|0()
    [02:57:19] info:    at System.Runtime.InteropServices.JavaScript.JSWebWorker.<>c__DisplayClass5_0.<<RunAsync>b__0>d.MoveNext()
    [02:57:19] info: --- End of stack trace from previous location ---
    [02:57:19] info:    at System.Runtime.InteropServices.JavaScript.Tests.WebWorkerTest.WaitAssertsOnJSInteropThreads(Executor executor)
    [02:57:19] info: --- End of stack trace from previous location ---
    
    

    Filled #97914
26. pavelsavara commented on Feb 9, 2024

    

    pavelsavara

    on Feb 9, 2024

    Member

    More actions

    closing this catch-all and we will continue creating individual issues with more granularity.
27. pavelsavara

    closed this as completedon Feb 9, 2024
28. github-actions

    locked and limited conversation to collaborators on Mar 12, 2024