Skip to content

LLVM bindings have become incorrect in places. #17795

Open

Description

@eddyb

Because of x86/x86_64 specifics, certain LLVM functions that used to return unsigned int and now return unsigned long long continue to be usable, ending up in truncated return values.

pub fn LLVMABISizeOfType(TD: TargetDataRef, Ty: TypeRef) -> c_uint;
unsigned long long LLVMABISizeOfType(LLVMTargetDataRef TD, LLVMTypeRef Ty);

This is just one example of such incorrectness arising over time, there could be more.
Would be nice if we could use bindgen every LLVM upgrade to verify our bindings (or just generate new ones).

Activity

  1. added
    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
    on Oct 5, 2014
  2. ahmedcharles commented on Oct 6, 2014

    @ahmedcharles
    Contributor

    Just curious, what's bindgen? Are there docs on how to use it?

  3. eddyb commented on Oct 6, 2014

    @eddyb
    ContributorAuthor

    @ahmedcharles I've updated the description with a link to bindgen.

  4. steveklabnik commented on Dec 31, 2015

    @steveklabnik
    Contributor

    I spoke about this with @eddyb on IRC today. This is especially pernicious, because we could be accidentally getting UB in corner cases.

    A tactic to close this bug:

    1. Generate headers from LLVM with bindgen
    2. Run diff
    3. see if any of it is a particular issue.

    Humans looking at it will probably be needed, given that bindgen isn't perfect.

  5. added
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    on Jan 5, 2016
  6. added
    C-bugCategory: This is a bug.
    E-help-wantedCall for participation: Help is requested to fix this issue.
    on Jul 22, 2017
  7. tamird commented on Sep 13, 2017

    @tamird
    Contributor

    I took a look at this, starting with the rather naive:

    #!/usr/bin/env sh
    
    set -eu
    
    LLVM_CONFIG_PATH=$(find build -name llvm-config -type f | head -n1)
    CXX_FLAGS=$($LLVM_CONFIG_PATH --cxxflags)
    
    echo CXX_FLAGS: $CXX_FLAGS
    
    RUST_BACKTRACE=1 bindgen $@ src/rustllvm/rustllvm.h -o src/librustc_llvm/ffi.rs -- $CXX_FLAGS

    which produces the output:

    $ ./bindgen.sh
    CXX_FLAGS: -I/Users/tamird/src/rust/build/x86_64-apple-darwin/llvm/include -ffunction-sections -fdata-sections -fPIC -m64 -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Werror=date-time -std=c++11 -O3 -DNDEBUG -fno-exceptions -fno-rtti -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
    error: invalid argument '-std=c++11' not allowed with 'C/ObjC', err: true
    thread 'main' panicked at 'Unable to generate bindings: ()', src/libcore/result.rs:906:4
    stack backtrace:
       0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::_print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::begin_panic
       6: std::panicking::begin_panic_fmt
       7: rust_begin_unwind
       8: core::panicking::panic_fmt
       9: core::result::unwrap_failed
      10: std::panicking::try::do_call
      11: __rust_maybe_catch_panic
      12: bindgen::main
      13: __rust_maybe_catch_panic
      14: std::rt::lang_start
    
  8. tamird commented on Sep 13, 2017

    @tamird
    Contributor

    Using --cflags rather than --cxxflags produces c++ parsing failures:

    $ ./bindgen.sh
    C_FLAGS: -I/Users/tamird/src/rust/build/x86_64-apple-darwin/llvm/include -ffunction-sections -fdata-sections -fPIC -m64 -stdlib=libc++ -fPIC -Wall -W -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wdelete-non-virtual-dtor -Wstring-conversion -Werror=date-time -O3 -DNDEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:263:9: error: unknown type name '__char16_t'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:264:9: error: unknown type name '__char32_t'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:1: error: unknown type name 'namespace'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:14: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:1: error: unknown type name 'template'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:10: error: expected identifier or '('
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:1: error: unknown type name 'template'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:10: error: expected identifier or '('
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: unknown type name 'namespace'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: unknown type name 'namespace'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:135:38: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:30: error: variable has incomplete type 'void'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:38: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:30: error: variable has incomplete type 'void'
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:38: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:144:38: error: expected ';' after top level declarator
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:149:38: error: expected ';' after top level declarator
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:263:9: error: unknown type name '__char16_t', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:264:9: error: unknown type name '__char32_t', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:1: error: unknown type name 'namespace', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:14: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:1: error: unknown type name 'template', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:10: error: expected identifier or '(', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:1: error: unknown type name 'template', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:10: error: expected identifier or '(', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: unknown type name 'namespace', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: unknown type name 'namespace', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:135:38: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:30: error: variable has incomplete type 'void', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:38: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:30: error: variable has incomplete type 'void', err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:38: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:144:38: error: expected ';' after top level declarator, err: true
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:149:38: error: expected ';' after top level declarator, err: true
    fatal error: too many errors emitted, stopping now [-ferror-limit=], err: true
    thread 'main' panicked at 'Unable to generate bindings: ()', src/libcore/result.rs:906:4
    stack backtrace:
       0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
       1: std::sys_common::backtrace::_print
       2: std::panicking::default_hook::{{closure}}
       3: std::panicking::default_hook
       4: std::panicking::rust_panic_with_hook
       5: std::panicking::begin_panic
       6: std::panicking::begin_panic_fmt
       7: rust_begin_unwind
       8: core::panicking::panic_fmt
       9: core::result::unwrap_failed
      10: std::panicking::try::do_call
      11: __rust_maybe_catch_panic
      12: bindgen::main
      13: __rust_maybe_catch_panic
      14: std::rt::lang_start
    
  9. tamird commented on Sep 13, 2017

    @tamird
    Contributor

    Ah, renaming rustllvm.h to rustllvm.hpp gets this further, but we hit rust-lang/rust-bindgen#492:

    ./bindgen.sh 2>bindgen.stderr
    

    gist containing bindgen.sh & bindgen.stderr

  10. FreezyLemon commented on Dec 10, 2022

    @FreezyLemon

    If someone's interested in this (like me, originally), I'll save you some time:

    The above PR #46353 was rejected due to A) bindgen needing an unwanted dependency and B) the team preferring a validation of the bindings over re-generating the bindings automatically via bindgen. (This is understandable, as the hand-written bindings are way nicer than bindgens output)

    ctest was proposed as a crate that allows testing the existing bindings. However, after some research, ctest is pretty much abandoned and even the fork ctest2 suffers from outdated dependencies, which results in the crate practically being unable to handle rustc_codegen_llvm which is the crate that would need to be parsed. I tried.

    The way I see it, the validation can't be reliably automated at the moment unless ctest2 gets a major update/rewrite or an alternative pops up (maybe with syn-based parsing).

    Edit: Actually, it might be possible. For example. libz-sys uses ctest2. I guess the code in libz-sys is simple enough to be parsed by the outdated syntex crate. Technically, the bindings could be moved to a separate crate (e.g. rustc_llvm_bindings) and if the code is simple/"old" enough, syntex and ctest should be able to handle it.

  11. cuviper commented on Dec 10, 2022

    @cuviper
    Member

    I think the bindings should probably move to rustc_llvm, since that's the one that actually prepares the link library, like a typical *-sys crate. That's also where we define our C++ shims, then we would have all LLVM ffi in one place.

  12. FreezyLemon commented on Dec 11, 2022

    @FreezyLemon

    Oh, I didn't know a crate like that already existed. I'll take a look, maybe it's possible to get ctest to work with it.

  13. FreezyLemon commented on Dec 11, 2022

    @FreezyLemon

    ctest still fails at parsing even after moving ffi.rs to rustc_llvm. It might be possible by refactoring the code, but it's not trivial. It's honestly not something I'm willing to spend more time on at the moment. It might make more sense to "just wait" and hope ctest gets an update

  14. added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness
    on Apr 26, 2023
  15. added
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Apr 26, 2023
  16. removed
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Apr 27, 2023
  17. added a commit that references this issue on Aug 13, 2024
  18. added
    I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness
    and removed
    I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness
    on Mar 1, 2025
  19. added
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Mar 1, 2025
  20. removed
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Mar 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.A-testsuiteArea: The testsuite used to check the correctness of rustcC-bugCategory: This is a bug.E-help-wantedCall for participation: Help is requested to fix this issue.I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions