Skip to content

System.Net.Security.Tests failed with System.IO.IOException #52362

Closed
Closed
System.Net.Security.Tests failed with System.IO.IOException#52362
Labels
JitStressCLR JIT issues involving JIT internal stress modesarch-arm64area-System.Net.Securityos-windowstest-run-coreTest failures in .NET Core test runs
Milestone
7.0.0

Description

@VincentBu
VincentBu
opened on May 6, 2021

Run: runtime-coreclr libraries-jitstress 20210505.1

Failed tests:

net6.0-windows-Release-arm64-CoreCLR_checked-no_tiered_compilation-Windows.10.Arm64v8.Open
 -System.Net.Security.Tests.ClientAsyncAuthenticateTest.ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success(serverProtocol: Ssl3)
 -System.Net.Security.Tests.ServerAsyncAuthenticateTest.ServerAsyncAuthenticate_EachSupportedProtocol_Success(protocol: Ssl3)

Error message:

System.IO.IOException :  Received an unexpected EOF or 0 bytes from the transport stream.


Stack trace
   at System.Net.Security.SslStream.<FillHandshakeBufferAsync>g__InternalFillHandshakeBufferAsync|181_0[TIOAdapter](TIOAdapter adap, ValueTask`1 task, Int32 minSize) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 953
   at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 456
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 358
   at System.Net.Security.Tests.ClientAsyncAuthenticateTest.ClientAsyncSslHelper(EncryptionPolicy encryptionPolicy, SslProtocols clientSslProtocols, SslProtocols serverSslProtocols, RemoteCertificateValidationCallback certificateCallback) in /_/src/libraries/System.Net.Security/tests/FunctionalTests/ClientAsyncAuthenticateTest.cs:line 155
   at System.Net.Security.Tests.ClientAsyncAuthenticateTest.ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success(SslProtocols serverProtocol) in /_/src/libraries/System.Net.Security/tests/FunctionalTests/ClientAsyncAuthenticateTest.cs:line 102
--- End of stack trace from previous location ---

Activity

  1. VincentBu
    added
    arch-arm64
    area-System.Net.Security
    os-windows
    JitStressCLR JIT issues involving JIT internal stress modes
    on May 6, 2021
  2. ghost added
    untriagedNew issue has not been triaged by the area owner
    on May 6, 2021
  4. karelz
    added this to the Future milestone on May 6, 2021
  5. karelz
    removed
    untriagedNew issue has not been triaged by the area owner
    on May 6, 2021

  6. VincentBu commented on May 20, 2021

    @VincentBu
    VincentBu
    on May 20, 2021 · Hidden as duplicate
    Author

  7. karelz commented on May 20, 2021

    @karelz
    karelz
    on May 20, 2021
    Member

    Kusto -- methods to query:

      or Method == 'ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success'
  or Method == 'ServerAsyncAuthenticate_EachSupportedProtocol_Success'

    Failures 3/18-9/2 (incl. PRs):

    Date Build OS Test
    5/5 no_tiered_compilation Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    5/19 jitstress2_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    5/22 jitstress2_jitstressregs0x80 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/9 jitstress2_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/13 zapdisable Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/14 no_tiered_compilation Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/15 zapdisable Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/22 jitstress2_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/26 jitstress2_jitstressregs4 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    6/30 Official run Windows.Server.Core.1909.Amd64.Open ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/4 tailcallstress Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/5 jitstress1 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/8 PR #55298 Windows.10.Amd64.Client21H1.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/11 jitstressregs0x1000 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/21 no_tiered_compilation Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/23 PR #55298 Windows.10.Amd64.Client21H1.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    7/31 tailcallstress Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/1 jitstressregs0x80 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/4 tailcallstress Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/13 PR #57282 - jitstress1_tiered Windows.10.Arm64v8.Open ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/15 jitstress2 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/15 defaultpgo Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/15 dynamicpgo Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    8/15 nopgo Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    9/1 jitstress1_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    9/2 jitstress2 Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    9/4 jitstress1_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success
    9/6 jitstress2_tiered Windows.10.Arm64v8.Open ClientAsyncAuthenticate_IndividualServerVsAllClientSupportedProtocols_Success
    ServerAsyncAuthenticate_EachSupportedProtocol_Success

    It is suspicious that the tests fail only with JitStress and all tests at once, also always together with test failures in #52361 -- perhaps some kind of environmental problem?

  8. VincentBu commented on Jun 16, 2021

    @VincentBu
    VincentBu
    on Jun 16, 2021 · Hidden as duplicate
    Author

  9. VincentBu commented on Jul 5, 2021

    @VincentBu
    VincentBu
    on Jul 5, 2021 · Hidden as duplicate
    Author

  10. VincentBu commented on Jul 12, 2021

    @VincentBu
    VincentBu
    on Jul 12, 2021 · Hidden as duplicate
    Author

  11. VincentBu commented on Jul 22, 2021

    @VincentBu
    VincentBu
    on Jul 22, 2021 · Hidden as duplicate
    Author

  12. VincentBu commented on Aug 2, 2021

    @VincentBu
    VincentBu
    on Aug 2, 2021 · Hidden as duplicate
    Author

  13. VincentBu commented on Aug 16, 2021

    @VincentBu
    VincentBu
    on Aug 16, 2021 · Hidden as duplicate
    Author

  14. danmoseley commented on Aug 16, 2021

    @danmoseley
    danmoseley
    on Aug 16, 2021
    Contributor

    @BruceForstall could you help hypothesize why this only fails for jitstress? Is this a particularly slow form of jitstress?

  15. karelz
    added
    test-run-coreTest failures in .NET Core test runs
    on Aug 24, 2021

  16. BruceForstall commented on Aug 30, 2021

    @BruceForstall
    BruceForstall
    on Aug 30, 2021
    Contributor

    @BruceForstall could you help hypothesize why this only fails for jitstress? Is this a particularly slow form of jitstress?

    Based on the table above, the test fails in many different types of jitstress. Some are slower (like zapdisable), but many should be much faster than normal, e.g., "no_tiered_compilation" which just says "turn on the JIT optimizer; don't allow tiering". Possibly some timing issue is exacerbated by the code running faster (due to the JIT optimizing the code)?

    Note that one problem with most libraries tests is that the default run style is with tiered compilation enabled. That means that the test runs with no JIT optimization until it runs enough times to be "tiered up" / optimized. But most test code never reaches that level. However, the jitstress modes all disable tiering, forcing JIT optimization.

    Has anyone tried reproducing these locally?

  17. VincentBu commented on Sep 2, 2021

    @VincentBu
    VincentBu
    on Sep 2, 2021 · Hidden as duplicate
    Author

  18. danmoseley commented on Sep 2, 2021

    @danmoseley
    danmoseley
    on Sep 2, 2021
    Contributor

    Again coincident with #52361. @karelz probably one could dupe against the other.

  19. karelz commented on Sep 2, 2021

    @karelz
    karelz
    on Sep 2, 2021
    Member

    @danmoseley the exceptions are different, so keeping 2 different issues isn't bad. No need to dupe them.

  20. wfurt commented on Feb 8, 2022

    @wfurt
    wfurt
    on Feb 8, 2022
    Member

    I think this duplicate of #52361. Depending on timing we can surface the server exception but if the client wins, we would see connection close as the server aborted and the exception is still to be processed.

    await TestConfiguration.WhenAllOrAnyFailedWithTimeout(t1, t2);
  21. wfurt
    closed this as completedon Feb 8, 2022
  22. ghost locked as resolved and limited conversation to collaborators on Mar 11, 2022
  23. karelz
    modified the milestones: Future, 7.0.0 on Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    JitStressCLR JIT issues involving JIT internal stress modesarch-arm64area-System.Net.Securityos-windowstest-run-coreTest failures in .NET Core test runs

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions