Skip to content

Serde with serde_derive fails to build in "alt" compiler on OS X #42967

Closed

Description

@SimonSapin

Steps to reproduce:

cargo new a
cd a
echo 'serde = {version = "1.0", features = ["serde_derive"]}' >> Cargo.toml
cargo build

Alternative steps to reproduce that will provides a suitable compiler:

# This branch is temporary,
# it will disappear once this issue is resolved and later be re-purposed
git clone https://github.com/servo/servo/ --branch rustup
cd servo
./mach cargo build -p serde

On OS X, with LLVM assertions disabled (like in rustc builds added in #39754, which are the default in Servo’s mach), rustc 1.20.0-nightly (69c65d2 2017-06-28) fails with:

error: dlopen(/private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
  Referenced from: /private/tmp/a/target/debug/deps/libserde_derive-73114808ba8f6d5d.dylib
  Reason: image not found
   --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
    |
261 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `serde`.

This problem does not occur with nightly-2017-06-29 from rustup, which has LLVM assertions enabled. (But perhaps that’s not the only difference between the two kinds of builds?)

Activity

  1. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    perhaps that’s not the only difference between the two kinds of builds?

    What’s RUST_CONFIGURE_ARGS="--enable-extended"?
    https://github.com/rust-lang/rust/pull/39754/files#diff-354f30a63fb0907d4ad57269548329e3R75

  2. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    It’s also enabled in the "non-alt" build, so probably not relevant.

  3. kennytm commented on Jun 29, 2017

    @kennytm
    Member

    Have you tried building the project in the same disk partition where you've installed rustc, i.e. inside /Users/...?

  4. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    Yes. With the second set of steps (and -v added), the error message looks like this:

       Compiling serde v1.0.8
         Running `rustc --crate-name serde /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs --crate-type lib --emit=dep-info,metadata -C codegen-units=4 -C debuginfo=2 --cfg 'feature="std"' --cfg 'feature="rc"' --cfg 'feature="derive"' --cfg 'feature="default"' --cfg 'feature="serde_derive"' -C metadata=b12802d89eb13761 -C extra-filename=-b12802d89eb13761 --out-dir /Users/simon/projects/servo/target/debug/deps -L dependency=/Users/simon/projects/servo/target/debug/deps --extern serde_derive=/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib --cap-lints allow -W unused-extern-crates`
    error: dlopen(/Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib, 1): Library not loaded: @rpath/libstd-82587804d4836e27.dylib
      Referenced from: /Users/simon/projects/servo/target/debug/deps/libserde_derive-87e93d0afba53041.dylib
      Reason: image not found
       --> /Users/simon/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.8/src/lib.rs:261:1
        |
    261 | extern crate serde_derive;
        | ^^^^^^^^^^^^^^^^^^^^^^^^^^
    
    error: Could not compile `serde`.
  5. kennytm commented on Jun 29, 2017

    @kennytm
    Member

    Does DYLD_PRINT_RPATHS=1 cargo build show anything special?

  6. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    @kennytm not as far I can tell.

  7. kennytm commented on Jun 29, 2017

    @kennytm
    Member

    Okay... There are many other debugging options for dyld but I doubt if they are useful.

    The problem is that @rpath/libstd-82587804d4836e27.dylib is not found, meaning @rpath/ is not expanded properly. I've checked libserde_derive-******.dylib does not have an LC_RPATH load command, meaning the @rpath is only supplied through the environment variable DYLD_LIBRARY_PATH set by Cargo.

    I don't think the alt-build's and normal build's Cargo has any difference though.

    Verify if rustc --print=sysroot is correct?

  8. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    ./mach rustc --print=sysroot with and without llvm-assertions = true in .servobuild:

    /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc
    /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc
    

    Both contain:

    bin		lib		manifest.in	share
    
  9. kennytm commented on Jun 29, 2017

    @kennytm
    Member

    Both contains libstd-82587804d4836e27.dylib directly inside lib/? (not lib/rustlib/<target>/lib/)

  10. SimonSapin commented on Jun 29, 2017

    @SimonSapin
    ContributorAuthor

    Almost, the -alt one has a different hash:

    $ ls /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c*/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd*
    /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-5e0ec722ea8df7e6.dylib
    /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-82587804d4836e27.dylib
    
  11. kennytm commented on Jun 30, 2017

    @kennytm
    Member

    Huh, that's even stranger. The @rpath should be resolved correctly. But the alt- rustc linked serde_derive using the libstd from the non-alt-build??

  12. SimonSapin commented on Jun 30, 2017

    @SimonSapin
    ContributorAuthor

    Well it sounds unlikely to me that “from the other build” is really what’s going on, I’d bet on incorrect hashing. How is the hexadecimal part of libstd-82587804d4836e27.dylib determined? What base values are being hashed?

  13. kennytm commented on Jun 30, 2017

    @kennytm
    Member

    IIRC rustc will just search for all libstd-*, it doesn't recompute the hash. Would it work if you move /Users/simon/tmp/servo-share/rust/69c65d29615c391c958ebf75dd65258ec23e175c/ out of /Users/simon/tmp, and then do a cargo clean + build?

  14. SimonSapin commented on Jun 30, 2017

    @SimonSapin
    ContributorAuthor

    I first hit the issue before the "non-alt" build was downloaded.

  15. kennytm commented on Jun 30, 2017

    @kennytm
    Member

    I've tried to reproduce it myself and it seems $PATH plays a role in finding which libstd is linked. I can make it link to the correctly hashed libstd by:

    1. "Install" it
    2. Make sure $PATH does not contain ~/.cargo/bin, and includes «installed-path»/usr/local/bin
    3. Build it. It should work.

    Overriding $PATH without installing causes rustc not able to find libcore.

  16. SimonSapin commented on Jun 30, 2017

    @SimonSapin
    ContributorAuthor

    What is "it" and what does "installing" mean?

  17. kennytm commented on Jun 30, 2017

    @kennytm
    Member

    Run ./install.sh --destdir=«installed-path» after extracting the tarball (sorry was on mobile so did not check the exact instruction).

  18. SimonSapin commented on Jun 30, 2017

    @SimonSapin
    ContributorAuthor

    What does this do besides copying the files?

    Regardless, this is a regression, it only happens on OS X, and only with "alternate" builds…

  19. Mark-Simulacrum commented on Jun 30, 2017

    @Mark-Simulacrum
    Member

    It looks like the problem is presumably that ./.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/libstd-5e0ec722ea8df7e6.dylib and ./.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/lib/rustlib/x86_64-apple-darwin/lib/libstd-82587804d4836e27.dylib don't have the same hash.

    One fairly clear problem is that Servo for some reason doesn't download rust-std from alt by-default AFAICT; having added some quick prints to the util script. That would lead to incompatibilities; this should be fixed.

    looking for rustc at /Users/mark/Edit/servo/.servo/rust/69c65d29615c391c958ebf75dd65258ec23e175c-alt/rustc-nightly-x86_64-apple-darwin/rustc/bin/rustc
    Downloading {} https://s3.amazonaws.com/rust-lang-ci/rustc-builds-alt/69c65d29615c391c958ebf75dd65258ec23e175c/rustc-nightly-x86_64-apple-darwin.tar.gz
    Downloading Rust compiler...
    Downloading Rust compiler: 100.0%
    Extracting Rust compiler...
    Rust compiler ready.
    Downloading {} https://s3.amazonaws.com/rust-lang-ci/rustc-builds/69c65d29615c391c958ebf75dd65258ec23e175c/rust-std-nightly-x86_64-apple-darwin.tar.gz
    Downloading Host rust library for target x86_64-apple-darwin...
    Downloading Host rust library for target x86_64-apple-darwin:  49.2%^Cmach interrupted by signal or user action. Stopping.
    
  20. Mark-Simulacrum commented on Jun 30, 2017

    @Mark-Simulacrum
    Member

    I've filed servo/servo#17575 which should fix this.

  21. SimonSapin commented on Jun 30, 2017

    @SimonSapin
    ContributorAuthor

    One fairly clear problem is that Servo for some reason doesn't download rust-std from alt by-default

    Good catch! This fixes the problem, thank you.

  22. added a commit that references this issue on Jul 11, 2017
    a148f5b
  23. added a commit that references this issue on Jul 13, 2017
    591d8a1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions