System.Threading.Tasks.Tests timed out on net5.0-Linux-Debug-arm64-Mono_release #42024

Activity

1. Dotnet-GitSync-Bot

   added

   area-System.Threading

   untriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner

   on Sep 9, 2020
2. danmoseley commented on Sep 9, 2020

   

   danmoseley

   on Sep 9, 2020

   ContributorAuthor

   More actions

   #42014
3. jkotas

   added

   blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'

   on Sep 9, 2020
4. steveisok commented on Sep 9, 2020

   

   steveisok

   on Sep 9, 2020

   Member

   More actions

   According to Kusto, there are only 13 records of this method taking longer than 1 second.

   Is there a good way to query how many hangs?
5. danmoseley commented on Sep 9, 2020

   

   danmoseley

   on Sep 9, 2020

   ContributorAuthor

   More actions

   @MattGal
6. MattGal commented on Sep 9, 2020

   

   MattGal

   on Sep 9, 2020

   Member

   More actions

   taking a look.
7. MattGal commented on Sep 9, 2020

   

   MattGal

   on Sep 9, 2020

   Member

   More actions

   Given it's only timed out 5x in mono and 3x in not-mono for the history of everything in the DB (less than .1%) I'd go with "rare testcase issue that hasn't shown up in other configurations yet."

   let monoJobs = Jobs
   | extend props=parse_json(Properties) 
   | where props["runtimeFlavor"] == "mono" and QueueName == "ubuntu.1804.armarch.open"
   | project JobId;
   WorkItems
   | where JobId in (monoJobs)
   | where FriendlyName == "System.Threading.Tasks.Tests"
   | summarize count() by Status
   
   let librariesJobs = Jobs
   | extend props=parse_json(Properties) 
   | where props["runtimeFlavor"] =~ "coreclr" and QueueName == "ubuntu.1804.armarch.open"
   | project JobId;
   WorkItems
   | where JobId in (librariesJobs)
   | where FriendlyName == "System.Threading.Tasks.Tests"
   | summarize count() by Status
   
8. danmoseley commented on Sep 9, 2020

   

   danmoseley

   on Sep 9, 2020

   ContributorAuthor

   More actions

   According to Kusto, there are only 13 records of this method taking longer than 1 second.

   Sounds like then this isn't the case of getting unlucky at the tail end of a distribution of test durations. It sounds like if the tests/product work correctly, even on a heavily loaded machine it will take no more than a second or two. In this case, it did not finish in several minutes. That suggests to me that either the tests, the library or the runtime has a flaw that occasionally causes a genuine hang. And the key thing we need is a dump file. So +1 for migrating to vstest, and I guess we wait meantime.

   👍React with 👍1Reacted by Steve Pfister
9. mangod9 commented on Sep 10, 2020

   

   mangod9

   on Sep 10, 2020

   Member

   More actions

   Moving to area-Infrastructure, since there is no System.Threading action at the moment.
10. mangod9

    added

    area-Infrastructure

    and removed

    area-System.Threading

    on Sep 10, 2020
12. ericstj commented on Sep 11, 2020

    

    ericstj

    on Sep 11, 2020

    Member

    More actions

    This bug is tracking a test failure in Threading not an infrastructure issue. Infrastructure can help debug but this isn’t tracking any infra work.
13. ericstj

    removed

    area-Infrastructure

    on Sep 11, 2020
14. 25 remaining items

    Load more

15. jaredpar commented on Sep 30, 2020

    

    jaredpar

    on Sep 30, 2020

    Member

    More actions

    So essentially you want to know when the only source of errors in a build were these 429s?
16. MattGal commented on Sep 30, 2020

    

    MattGal

    on Sep 30, 2020

    Member

    More actions

    So essentially you want to know when the only source of errors in a build were these 429s?

    If you can figure out how to do that that'd be rad. I'm not sure how easy or worth it it is. A hotfix to more than double the quota per minute is rolling out presently and should be live in the next 30 m.
17. lambdageek commented on Oct 9, 2020

    

    lambdageek

    on Oct 9, 2020

    Member

    More actions

    A couple of days ago, I ran this testsuite locally about 300 times in a loop and couldn't get it to hang on OSX. (runtime and libs built with ./build.sh -s Mono+Libs+Libs.Test -c Release -rc Debug)
18. stephentoub

    removed

    blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'

    on Jul 26, 2021
19. stephentoub commented on Aug 4, 2021

    

    stephentoub

    on Aug 4, 2021

    Member

    More actions

    Looks like this test failed frequently for a few weeks and then hasn't failed in over six months. I'm going to close this for now.
20. stephentoub

    closed this as completedon Aug 4, 2021
21. jaredpar commented on Aug 4, 2021

    

    jaredpar

    on Aug 4, 2021

    Last edited by jaredpar

    Member

    More actions

    It's failed 21 times in the last three days

    https://runfo.azurewebsites.net/search/tests/?q=started%3A%7E3+definition%3Aruntime+name%3ASystem.Threading.Tasks.Tests

    On non-pure PR it failed 4 times

    https://runfo.azurewebsites.net/search/tests/?q=started%3A%7E3+definition%3Aruntime+name%3ASystem.Threading.Tasks.Tests+kind%3A%21pr

    At least this one failed with the same error message as the original bug

    https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-56768-merge-0e3e3f8f2bb544f5b0/System.Threading.Tasks.Tests/1/console.615257c0.log?%3Fsv%3D2019-07-07%26se%3D2021-08-23T19%253A25%253A11Z%26sr%3Dc%26sp%3Drl%26sig%3DpgnEOTia%252FqffpW%252FHwagJTQ2s%252FCsvHw1r3ZvGo4mVIA4%253D
22. stephentoub

    reopened this on Aug 4, 2021
23. stephentoub commented on Aug 4, 2021

    

    stephentoub

    on Aug 4, 2021

    Last edited by stephentoub

    Member

    More actions

    Ok, my runfo search skills are apparently quite lacking. Not sure what I searched for that yielded an empty set.
24. stephentoub commented on Aug 4, 2021

    

    stephentoub

    on Aug 4, 2021

    Last edited by stephentoub

    Member

    More actions

    Actually, no, I was right the first time. None of those failures are this test. A couple of them are RunContinueWithStressTestsNoState, which is also in the System.Threading.Tasks test suite... but it's supposed to be disabled:
    

    runtime/src/libraries/System.Threading.Tasks/tests/Task/TaskContinueWithTests.cs

    Line 45 in 57bfe47

    [ActiveIssue("https://github.com/dotnet/runtime/issues/2271")]

    
    so I don't know why it's showing up in test results from the last few days. And that one is tracked by a different issue, #2271.

    👍React with 👍1Reacted by Jared Parsons
25. stephentoub

    closed this as completedon Aug 4, 2021
26. jaredpar commented on Aug 4, 2021

    

    jaredpar

    on Aug 4, 2021

    Member

    More actions

    Actually, no, I was right the first time. None of those failures are this test

    My bad. I was looking at the title of the issue which had the test group.

    👍React with 👍1Reacted by Stephen Toub
27. akoeplinger commented on Aug 4, 2021

    

    akoeplinger

    on Aug 4, 2021

    Last edited by akoeplinger

    Member

    More actions

    @stephentoub the only instance of the RunContinueWithStressTestsNoState I can find is from a release/5.0 build and the test doesn't have ActiveIssue there.
28. stephentoub commented on Aug 4, 2021

    

    stephentoub

    on Aug 4, 2021

    Member

    More actions

    Ah, that explains that then.
29. ghost locked as resolved and limited conversation to collaborators on Sep 3, 2021