Repository navigation
Unable to find crate proc_macro on musl target #40174
Description
Using procedural derive macros on musl seems to work just fine for the most of the cases, but there is one pitfall; in some cases like this dtolnay/proc-macro-hack#6 build fails because not only the crate that implements the macro (crate marked with proc-macro = true), but also another, normal crate links to the proc_macro crate. "Normal" crates are unable to find proc_macro when built on musl target:
Compiling proc-macro-hack v0.3.0
error[E0463]: can't find crate for `proc_macro`
--> /root/.cargo/registry/src/github.life-white.uk-1ecc6299db9ec823/proc-macro-hack-0.3.0/src/lib.rs:1:1
|
1 | extern crate proc_macro;
| ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
error: aborting due to previous error
error: Could not compile `proc-macro-hack`.
Activity
alexcrichton commentedon Mar 1, 2017
Yes this is currently intentional. The proc_macro crate is only a dynamic library which the musl target currently doesn't support. I unfortunately don't think this will be easy to fix...
golddranks commentedon Mar 1, 2017
I understand why the proc macro crates need to be dynamic: they are loaded and run dynamically by the compiler. However what I don't understand is why the
proc_macrocrate isn't allowed to be linked both dynamically and statically? I'd imagine someone may want to use the types defined inproc_macroin some helper crate, outside the dynamically linked macro crate, as is the case here.golddranks commentedon Mar 1, 2017
Is there some technical hurdle that makes it hard to do, or is it just not done because it's such a minor case?
golddranks commentedon Mar 1, 2017
The simplest test for this seems to be:
test.rswith just single line:build with (succeeds):
and with (fails):
When
RUST_LOG=infois enabled, the library probes can be seen.golddranks commentedon Mar 1, 2017
Okay, I did some testing.
I
git cloned the Rust source, edited away thecrate-type = ["dylib"]annotation inlibproc_macro,librustc_data_structures,librustc_errors,libsyntaxandlibsyntax_pos. (Transitive deps oflibproc_macro)After that, I built a
rliboflibproc_macrowithcargo build --target=x86_64-unknown-linux-musl. (This is on the newest nightly.)After that, I copied the build artefacts from
~/rust/src/target/x86_64-unknown-linux-musl/debug/deps/to~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib.After that, I try to compile the things that failed before with musl target → it works.
So, it seems that there isn't any real technical hurdles. (or if there is, I haven't encountered them yet)
@alexcrichton, what do you think of distributing
rlibs of aforementioned libraries with the musl target?golddranks commentedon Mar 1, 2017
Btw. I did this work on a Linux host. Tried to do the same on macOS (with musl target of course), but on macOS the linking fails with
ld: unknown option: --as-needed. The linker is passed a flag--as-neededwhich seems to be supported on GNU linker only.golddranks commentedon Mar 1, 2017
Ah, okay, the
--as-neededbehaviour is explained here: #34282alexcrichton commentedon Mar 2, 2017
Yes there are technical hurdles in terms of build system of doing that. This is also rarely what you want because there's no reason to have a full Rust parser linked into a binary (when it's not used).
golddranks commentedon Mar 2, 2017
I see, that's a convincing argument.
drozdziak1 commentedon Jan 10, 2018
Hello,
sorry to dig up a closed issue, but I'm experiencing the same problem during an OpenWrt cross build (
i686-unknown-linux-muslis the Rust target and VirtualBox is the OpenWrt one). My compiler isrustc 1.25.0-nightly (61452e506 2018-01-09). I've got a couple questions regarding this:Remove serde-derive dependency
lisbakke commentedon Nov 21, 2018
Hello! I've got this issue, as well -- deps relying on
proc-macro2which do not usedefault-features = falsesoproc-macrocrate is not found because my target does not have it.I'm currently compiling rustc for my target and it seems easiest to create
proc-macrolib when I'm building. Is there any documentation on how to do this?mmastrac commentedon Oct 6, 2019
For those following various links to this issue, it is important to note that you can compile code using
proc_macroto statically-linkedmuslbinaries, BUT you need to cross-compile from a platform that does support dynamic linking.cargo build --target x86_64-unknown-linux-muslshould be sufficient, but you'll need to run that on something like Ubuntu, Fedora, or another full system.20 remaining items
Cross-build from Ubuntu to Alpine
Ekleog commentedon Apr 16, 2020
There are likely much better solutions, but… I've just come across https://github.com/dtolnay/watt ; and… would integrating this into rustc as a secondary backend for use only on platforms where regular proc macros aren't supported be a solution? It wouldn't support all the use cases I think, due to regular proc macros being allowed to eg. interact with the FS, but it should already support a number of use cases. And maybe, with time, the API could be extended so that proc macros could all be compiled with wasm, and the current dylib approach could be dropped, thus making porting rustc to a new architecture a lot easier?
mati865 commentedon Apr 16, 2020
@Ekleog is it still an issue after #69519?
Ekleog commentedon Apr 16, 2020
@mati865 I'm not sure, I've pinged here following a message from IRC that said this happened just today -- don't know for sure the version of the compiler used, so I've just pinged them
breard-r commentedon Apr 24, 2020
@mati865 If it's part of the brand new Rust 1.43, then yes, it is still an issue.
On Alpine, the heavily patched rust package (1.39) can compile without any trouble. However, on the very same machine, any version of Rust (including 1.39 and 1.43) installed via rustup (and therefore without those patches) is subject to this issue.
12101111 commentedon Apr 24, 2020
@breard-r Rust 1.43 don't include this patch and you can try 1.44 beta from rustup.
breard-r commentedon Apr 24, 2020
@12101111 Thank you! I've just installed 1.44.0-beta via rustup and it builds! 😃
jonas-schievink commentedon Apr 24, 2020
Great! Closing as fixed then.
jyn514 commentedon May 23, 2020
This is still broken for me using
rustc 1.44.0-beta.3 (0f0d70055 2020-05-11).Full linker arguments
12101111 commentedon May 23, 2020
It seems you don't install musl-dev
jyn514 commentedon May 23, 2020
That fixed it, thank you!
WhyNotHugo commentedon Nov 24, 2021
If you're bumping into this in 2021, it's likely that you need to run
apk add --no-cache musl-dev.Revert "Dockerfile: Switch to Debian-based images and simplify build …
Add Docker support and update README (#13)