Skip to content

Build errors under libraries/native when building with GCC #2188

Closed
#34211
Closed
Build errors under libraries/native when building with GCC#2188
#34211
Labels
area-Infrastructure-librariesuntriagedNew issue has not been triaged by the area owner

Description

@Meai
Meai
opened on Jan 25, 2020

gcc 9.2.1
clearlinux
dotnetcore 3.1.1

./libraries.sh

First error (happens all over):

[ 2%] Building C object System.Globalization.Native/CMakeFiles/System.Globalization.Native-Static.dir/pal_calendarData.c.o
In file included from /usr/include/assert.h:35,
from /home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/AnyOS/zlib/pal_zlib.c:5:
/usr/include/features.h(382,4): error G90377845: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Werror=cpp] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]

Then a whole bunch of this:

cc1 : error : unrecognized command line option ‘-Wno-unknown-warning-option’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-extra-semi-stmt’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-documentation-unknown-command’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-documentation’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-reserved-id-macro’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-covered-switch-default’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-c11-extensions’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-typedef-redefinition’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-empty-translation-unit’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1 : error : unrecognized command line option ‘-Wno-disabled-macro-expansion’ [-Werror] [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Native/build-native.proj]
cc1: all warnings being treated as errors

Activity

  1. Dotnet-GitSync-Bot
    added
    area-Infrastructure-coreclrOnly use for closed issues
    untriagedNew issue has not been triaged by the area owner
    on Jan 25, 2020
  2. jkotas
    added
    area-Infrastructure-libraries
    and removed
    area-Infrastructure-coreclrOnly use for closed issues
    on Jan 25, 2020
  3. jkotas
    changed the title [-]Build errors[/-] [+]Build errors under libraries/native when building with GCC[/+] on Jan 25, 2020

  4. am11 commented on Jan 26, 2020

    @am11
    am11
    on Jan 26, 2020
    Member

    In the stock Clear Linux docker, these environment variables are set:

    root@c042f6b156ed/runtime_x # export
declare -x AR="gcc-ar"
declare -x CC="gcc"
declare -x CFFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize  -Wl,-sort-common -Wl,--enable-new-dtags "
declare -x CFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects  -fno-trapping-math -Wl,-sort-common -Wl,--enable-new-dtags -mtune=skylake  -Wa,-mbranches-within-32B-boundaries"
declare -x CXX="g++"
declare -x CXXFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -m64  -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects  -fno-trapping-math -Wl,-sort-common -Wl,--enable-new-dtags -mtune=skylake  -Wa,-mbranches-within-32B-boundaries -fvisibility-inlines-hidden -Wl,--enable-new-dtags "
declare -x EDITOR="/usr/bin/vi"
declare -x FFLAGS="-g -O3 -feliminate-unused-debug-types  -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition -ftree-vectorize  -ftree-loop-vectorize -Wl,--enable-new-dtags  -Wa,-mbranches-within-32B-boundaries "
declare -x GTK_IM_MODULE="ibus"
...

    we just need to make sure to not pass that CFlags to build script, so instead of:

    CLR_CC=$(command -v gcc) ./libraries.sh

    do:

    # pass empty CFLAGS inline
CFLAGS= CLR_CC=$(command -v gcc) ./libraries.sh

    and the build will succeed.

    we can unset fortification and other flags explicitly which cause build failure if inherited by the environment to avoid such issues.

    cc @janvorli

    Edit - this issue is not specific to gcc, clang on Clear Linux also fail to compile unless we clear CFLAGS defined in environment.

  5. Meai commented on Feb 6, 2020

    @Meai
    Meai
    on Feb 6, 2020
    Author

    @am11 I tried that and now I still get errors but different ones:

    (I git pulled first, now on commit 04e3169)

    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Text.Encoding.CodePages/ref/System.Text.Encoding.CodePages.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Threading.Timer/ref/System.Threading.Timer.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Security.Cryptography.Primitives/ref/System.Security.Cryptography.Primitives.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Runtime.InteropServices.WindowsRuntime/ref/System.Runtime.InteropServices.WindowsRuntime.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Runtime.Numerics/ref/System.Runtime.Numerics.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Runtime.Handles/ref/System.Runtime.Handles.csproj]
    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.targetframework.sdk/5.0.0-beta.20103.5/build/BinPlace.targets(93,5): error MSB4062: The "SaveItems" task could not be loaded from the assembly /home/pc/.nuget/packages/microsoft.dotnet.arcade.sdk/5.0.0-beta.20104.2/tools/netcoreapp2.1/Microsoft.DotNet.Arcade.Sdk.dll. Assembly with same name is already loaded Confirm that the declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/Microsoft.Win32.Primitives/ref/Microsoft.Win32.Primitives.csproj]
    0 Warning(s)
    7 Error(s)

  6. am11 commented on Feb 6, 2020

    @am11
    am11
    on Feb 6, 2020
    Member

    There are two parts: managed and native. The native build uses gcc/clang. In the current master we can also pass --clang or --gcc from top level scripts: runtime/build.sh --gcc or runtimte/libraries.sh --clang9 etc.

    error MSB4062: The "SaveItems" task could not be loaded from the assembly

    this sounds like a separate issue with managed build failing, because some conflicting assemblies trying to load in the same appdomain. Could you ps uax | grep dotnet and kill -KILL <space-separated list of pids> all the pids you find in ps output, then git clean -xdf before reattempting to build?

    command for complete product release build + packages using gcc will look like this:

    cd dotnet-runtime
./build.sh --configuration release --gcc
./build.sh --configuration release --subset PkgProj

    the packages will end up at ./artifacts/packages/Release/Shipping/.

  7. Meai commented on Feb 6, 2020

    @Meai
    Meai
    on Feb 6, 2020
    Author

    Ok I did that and then:

    CFLAGS= CLR_CC=$(command -v gcc) ./build.sh --configuration debug --gcc

    still some errors:

    /home/pc/workarea/test/dotnet-runtime-src/eng/referenceFromRuntime.targets(90,5): error : Could not resolve ReferenceFromRuntime item(s) 'System.Private.CoreLib' from ''. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Buffers/src/System.Buffers.csproj]
    /home/pc/workarea/test/dotnet-runtime-src/eng/referenceFromRuntime.targets(90,5): error : Could not resolve ReferenceFromRuntime item(s) 'System.Private.CoreLib' from ''. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.AppContext/src/System.AppContext.csproj]
    /home/pc/workarea/test/dotnet-runtime-src/eng/referenceFromRuntime.targets(90,5): error : Could not resolve ReferenceFromRuntime item(s) 'System.Private.CoreLib' from ''. [/home/pc/workarea/test/dotnet-runtime-src/src/libraries/System.Private.Uri/src/System.Private.Uri.csproj]
    [...]

    /home/pc/workarea/test/dotnet-runtime-src/src/libraries/restore/runtime/runtime.depproj(92,5): error : Could not locate CoreCLR IL files.
    [...]

    /home/pc/.nuget/packages/microsoft.dotnet.build.tasks.packaging/5.0.0-beta.20104.2/build/Packaging.targets(1260,5): error : Error when creating nuget lib package from /home/pc/workarea/test/dotnet-runtime-src/artifacts/packages/Debug/specs/Microsoft.NETCore.App.Internal.nuspec. NuGet.Packaging.Core.PackagingException: File not found: '/home/pc/workarea/test/dotnet-runtime-src/artifacts/obj/netcoreapp/Debug/netcoreapp5.0/linux-x64/Microsoft.NETCore.App.versions.txt'. [/home/pc/workarea/test/dotnet-runtime-src/src/installer/pkg/projects/netcoreapp/pkg/legacy/Microsoft.NETCore.App.Internal.pkgproj]

  8. jkotas commented on Mar 16, 2020

    @jkotas
    jkotas
    on Mar 16, 2020
    Member

    #33446 added CI testing for GCC build. The master should be always buildable with GCC now.

  9. jkotas
    closed this as completedon Mar 16, 2020

  10. am11 commented on Mar 17, 2020

    @am11
    am11
    on Mar 17, 2020
    Member

    For completeness, this issue is actually about build failure on official Clear Linux image, where CFLAGS and CXXLAGS are already set in the environment. Build fails with Clang the same way it fails with GCC. I think we should fix it by setting the values of these flags explicitly and conciously; more importantly this tells us that CoreCLR is not _FORTIFY_SOURCE compatible: https://access.redhat.com/blogs/766093/posts/1976213, so it would be good to set flags value in base cmake script.

    Here is a docker file @Meai, which completely builds the current master (I tested d6bf13a) on Clear Linux (relevant line is ENV CFLAGS...)

    $ docker build --tag dotnet-runtime-clearlinux - << EOF
FROM       clearlinux

RUN        swupd bundle-add git c-basic devpkg-icu4c devpkg-krb5 devpkg-openssl devpkg-e2fsprogs devpkg-lttng-ust python-basic

RUN        git clone https://github.com/dotnet/runtime --single-branch --branch master --depth 1

ENV        CFLAGS= CXXFLAGS=

WORKDIR    runtime

RUN        ./build.sh --gcc --subsetCategory coreclr
RUN        ./build.sh --gcc --subsetCategory libraries
RUN        ./build.sh --gcc --subsetCategory installer
EOF
  11. jkotas
    reopened this on Mar 17, 2020

  12. jkotas commented on Mar 17, 2020

    @jkotas
    jkotas
    on Mar 17, 2020
    Member

    it would be good to set flags value in base cmake script.

    @am11 Do you plan to submit a PR for this?

  13. am11 commented on Mar 17, 2020

    @am11
    am11
    on Mar 17, 2020
    Member

    Sure, I will try to make build work on Clear Linux without needing to unset C{XX}FLAGS.

  14. jkotas
    closed this as completedin #34211on Apr 8, 2020
  15. ghost locked as resolved and limited conversation to collaborators on Dec 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-Infrastructure-librariesuntriagedNew issue has not been triaged by the area owner

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions