Skip to content

CI failure: system.security.cryptography.openssl.tests missing libssl #73210

Closed

Description

@runfoapp

Runfo shows a bunch of failures, but looking at the first few none of them looked similar to the missing libssl error shown here.
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-d48929e09acb474baa/System.Security.Cryptography.OpenSsl.Tests/1/console.a5cb0446.log?%3Fhelixlogtype%3Dresult

/private/tmp/helix/working/B16B09B5/w/AB2909A1/e /private/tmp/helix/working/B16B09B5/w/AB2909A1/e
  Discovering: System.Security.Cryptography.OpenSsl.Tests (method display = ClassAndMethod, method display options = None)
No usable version of libssl was found
./RunTests.sh: line 168: 59278 Abort trap: 6           "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json xunit.console.dll System.Security.Cryptography.OpenSsl.Tests.dll -xml 

Runfo Tracking Issue: system.security.cryptography.openssl.tests missing libssl

Build Definition Kind Run Name Console Core Dump Test Results Run Client
1927756 runtime PR 73520 net7.0-Linux-Debug-arm64-Mono_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20220427172132-97d8652 console.log core dump runclient.py
1916354 runtime PR 73185 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1916354 runtime PR 73185 net7.0-Linux-Release-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open console.log core dump runclient.py
1916354 runtime PR 73185 net7.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 console.log core dump runclient.py
1916354 runtime PR 73185 net7.0-Linux-Debug-arm64-CoreCLR_checked-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20210531091519-97d8652 console.log core dump runclient.py
1916354 runtime PR 73185 net7.0-Linux-Debug-x64-CoreCLR_checked-(Alpine.314.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19 console.log core dump runclient.py
1915309 runtime PR 72934 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1915051 runtime PR 49576 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1915051 runtime PR 49576 net7.0-Linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7-20210910135806-8a6f4f3 console.log core dump runclient.py
1915051 runtime PR 49576 net7.0-Linux-Debug-arm64-CoreCLR_checked-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20210531091519-97d8652 console.log core dump runclient.py
1915051 runtime PR 49576 net7.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 console.log core dump runclient.py
1915051 runtime PR 49576 net7.0-Linux-Release-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open console.log core dump runclient.py
1915051 runtime PR 49576 net7.0-Linux-Debug-x64-CoreCLR_checked-(Alpine.314.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19 console.log core dump runclient.py
1914778 runtime PR 49576 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1914778 runtime PR 49576 net7.0-Linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7-20210910135806-8a6f4f3 console.log core dump runclient.py
1914778 runtime PR 49576 net7.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 console.log core dump runclient.py
1914778 runtime PR 49576 net7.0-Linux-Debug-arm64-CoreCLR_checked-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20210531091519-97d8652 console.log core dump runclient.py
1914778 runtime PR 49576 net7.0-Linux-Release-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open console.log core dump runclient.py
1914778 runtime PR 49576 net7.0-Linux-Debug-x64-CoreCLR_checked-(Alpine.314.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19 console.log core dump runclient.py
1914601 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1914164 runtime PR 73120 net7.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 console.log core dump runclient.py
1914164 runtime PR 73120 net7.0-Linux-Debug-arm64-CoreCLR_checked-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20210531091519-97d8652 console.log core dump runclient.py
1914164 runtime PR 73120 net7.0-Linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7-20210910135806-8a6f4f3 console.log core dump runclient.py
1914164 runtime PR 73120 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1914164 runtime PR 73120 net7.0-Linux-Release-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open console.log core dump runclient.py
1914164 runtime PR 73120 net7.0-Linux-Debug-x64-CoreCLR_checked-(Alpine.314.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19 console.log core dump runclient.py
1913265 runtime PR 73105 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-arm64-CoreCLR_checked-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20210531091519-97d8652 console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-arm64-Mono_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8-20220427172132-97d8652 console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-Mono_release-(Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Release-x64-CoreCLR_checked-Ubuntu.1804.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-Mono_release-Ubuntu.1804.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Release-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm32v7-20210910135806-8a6f4f3 console.log runclient.py
1913236 runtime PR 73095 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-CoreCLR_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20220601183719-dde38af console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-CoreCLR_release-RedHat.7.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-CoreCLR_release-(Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-CoreCLR_release-Ubuntu.1804.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-Mono_release-RedHat.7.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-Mono_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20220601183719-dde38af console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Release-arm64-CoreCLR_checked-(Alpine.314.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-arm64v8-20210910135810-8a6f4f3 console.log runclient.py
1913236 runtime PR 73095 net7.0-OSX-Debug-x64-Mono_release-OSX.1200.Amd64.Open console.log runclient.py
1913236 runtime PR 73095 net7.0-Linux-Debug-x64-mono_interpreter_release-(Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 console.log runclient.py
1913058 runtime PR 73037 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
0 1 10

Activity

  1. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Aug 2, 2022
  2. added
    blocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
    on Aug 2, 2022
  3. changed the title [-]system.security.cryptography.openssl.tests missing libssl[/-] [+]CI failure: system.security.cryptography.openssl.tests missing libssl[/+] on Aug 2, 2022
  4. bartonjs commented on Aug 2, 2022

    @bartonjs
    Member

    @dotnet/dnceng Looks like some macOS images are bad again.

  5. andriipatsula commented on Aug 2, 2022

    @andriipatsula
    Member

    Hello @bartonjs , we have a known issue dotnet/arcade#10139 for that. It's related to ARM64 machines, but I believe the problem is the same for amd64 too. I'll link this one to arcade issue.

  6. added
    tracking-external-issueThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly
    on Aug 2, 2022
  7. added this to the 7.0.0 milestone on Aug 2, 2022
  8. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 2, 2022
  9. MattGal commented on Aug 2, 2022

    @MattGal
    Member

    As I was working on dotnet/arcade#10139, I will take a look however that's not likely particularly related as that problem was purely with different, erroneous, setup steps on M1 Mac machines.

  10. MattGal commented on Aug 2, 2022

    @MattGal
    Member

    @bartonjs I fixed up the ARM64 machines yesterday, it would seem we have the same problem on recently imaged 11.00 and 12.00 AMD64 machines as well. dotnet/arcade#10139 is still open and I'll investigate and get those machines fixed today.

    I would also note that the OSX failures linked in this issue are all over the place and many are not LibSSL related... that certainly throws a wrench in things.

  11. wfurt commented on Aug 2, 2022

    @wfurt
    Member

    This is nature of Runfo to bundle unrelated failures together ;(. I trained myself to only look at rolling builds -> there is one hit @MattGal

    Console log: 'System.Security.Cryptography.OpenSsl.Tests' from job d48929e0-9acb-474b-aa57-c93c7d1e9f43 workitem 3da839f0-191a-427b-95a1-0dbc7aa55df9 (osx.1200.amd64.open) executed on machine dci-mac-build-103
    + ./RunTests.sh --runtime-path /tmp/helix/working/B16B09B5/p
    ----- start Sun Jul 31 16:44:59 EDT 2022 =============== To repro directly: =====================================================
    pushd .
    /tmp/helix/working/B16B09B5/p/dotnet exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json xunit.console.dll System.Security.Cryptography.OpenSsl.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
    popd
    ===========================================================================================================
    /private/tmp/helix/working/B16B09B5/w/AB2909A1/e /private/tmp/helix/working/B16B09B5/w/AB2909A1/e
      Discovering: System.Security.Cryptography.OpenSsl.Tests (method display = ClassAndMethod, method display options = None)
    No usable version of libssl was found
    ./RunTests.sh: line 168: 59278 Abort trap: 6           "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json 
    
  12. MattGal commented on Aug 2, 2022

    @MattGal
    Member

    This is nature of Runfo to bundle unrelated failures together ;(. I trained myself to only look at rolling builds -> there is one hit @MattGal

    Console log: 'System.Security.Cryptography.OpenSsl.Tests' from job d48929e0-9acb-474b-aa57-c93c7d1e9f43 workitem 3da839f0-191a-427b-95a1-0dbc7aa55df9 (osx.1200.amd64.open) executed on machine dci-mac-build-103
    + ./RunTests.sh --runtime-path /tmp/helix/working/B16B09B5/p
    ----- start Sun Jul 31 16:44:59 EDT 2022 =============== To repro directly: =====================================================
    pushd .
    /tmp/helix/working/B16B09B5/p/dotnet exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json xunit.console.dll System.Security.Cryptography.OpenSsl.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
    popd
    ===========================================================================================================
    /private/tmp/helix/working/B16B09B5/w/AB2909A1/e /private/tmp/helix/working/B16B09B5/w/AB2909A1/e
      Discovering: System.Security.Cryptography.OpenSsl.Tests (method display = ClassAndMethod, method display options = None)
    No usable version of libssl was found
    ./RunTests.sh: line 168: 59278 Abort trap: 6           "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Security.Cryptography.OpenSsl.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.OpenSsl.Tests.deps.json 
    

    Yup, I found the one bad machine about 20 minutes ago and added notes in dotnet/arcade#10139. It should be clean everywhere and I have chatted with the DDFUN folks who set up macs, as well as updated their docs.

  13. jeffhandley commented on Aug 11, 2022

    @jeffhandley
    Member

    Yup, I found the one bad machine about 20 minutes ago and added notes in dotnet/arcade#10139. It should be clean everywhere and I have chatted with the DDFUN folks who set up macs, as well as updated their docs.

    Thanks, @MattGal!

  14. ghost locked as resolved and limited conversation to collaborators on Sep 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-System.Securityblocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'tracking-external-issueThe issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions