Repository navigation
Add shared toolchain directories #4967
Description
Activity
Although it is possible to manually link a system toolchain, there is no mechanism to keep it synchronized with updates from the package manager.
@Rx1513 Hello! My apologies but I'm not quite sure what are you looking for with this proposal.
As I understand it, since it's a link, a linked system toolchain from the package manager is automatically synchronized with updates from the package manager.
rustup should automatically discover toolchains from the shared directory.
Why would batch-assigning a directory of linked toolchains simplify your workflow? Could you please elaborate?
To solve these issues, a separate shared directory managed by external tools could be added.
Should it be the responsibility of the external tools, whatever they may be, to add such links?
PS: We are avoiding adding more layered configurations so I would prefer being more careful with that part of your proposal, and it's vital that we reach an understanding of what exactly is needed in your use case.
PPS: If you are referring to using project-specific sets of tools under a specific name, I have this proposal that may suit your needs: #4636
Although it is possible to manually link a system toolchain, there is no mechanism to keep it synchronized with updates from the package manager.
@Rx1513 Hello! My apologies but I'm not quite sure what are you looking for with this proposal.
As I understand it, since it's a link, a linked system toolchain from the package manager is automatically synchronized with updates from the package manager.
The problem is that each link is handled manually.
- If the package is removed, rustup still lists the linked toolchain until the user manually removes the link.
- When the package manager installs a package, user must manually link the distributed toolchain and then select it. It may sound simple but in practice it looks something like this:
apt-get install rust Look where package places its toolchain rustup toolchain link <user-specific name> <generally long absolute toolchain path> rustup default <user-specific name> || rustup override set <user-specific name> # DoneTwo middle steps seems very unnecessary as it requires new useless information for user and some extra steps, especially when compared to official toolchains installation:
rustup default # DoneWhat I want is to make usage of system's toolchains is similar to usage of official toolchains:
apt-get install rust rustup default <distro specific name> || rustup override set <distro specific name> # Doneor even:
apt-get install rust # DoneIn my opinion the additional lookup directory should create much smoother user experience when dealing with toolchains from other external sources.
Why would batch-assigning a directory of linked toolchains simplify your workflow? Could you please elaborate?
It doesn't require from each user doing manual tedious work. It helps separate responsibilities: external tools manage external directories under administrator control, while the user's home directory remains solely the user's responsibility.
To solve these issues, a separate shared directory managed by external tools could be added.
Should it be the responsibility of the external tools, whatever they may be, to add such links?
As far as I know many Linux distributions discourage when package manager messes with home directory so it's not the way. The only way is if user manually links every external toolchain or if administrator does it for them, which yet again messes with responsibility.
PS: We are avoiding adding more layered configurations so I would prefer being more careful with that part of your proposal, and it's vital that we reach an understanding of what exactly is needed in your use case.
I think this feature keeps the configuration largely unchanged. It only adds an additional lookup directory for externally managed toolchains, while the existing per-user links continue to work unchanged.
PPS: If you are referring to using project-specific sets of tools under a specific name, I have this proposal that may suit your needs: #4636
I don't think this is related to the problems I'm addressing here, but thanks for help.
@Rx1513 Thanks for your detailed reply! As I roughly understand it, you are referring to adding support for a new toolchain directory for automatic toolchain discovery (but read-only)?
When the package manager installs a package, user must manually link the distributed toolchain and then select it.
IRRC rustup treats every folder under the toolchain directory as a valid toolchain, so I am curious about how we could probably make it work with your case.
What is your specific distro/external tool that installs externally-managed Rust toolchains in a way that is compatible with your new directory proposed here? For Homebrew (what I'm using) for example, the system Rust toolchain is installed alongside other packages. Is rustup supposed to distinguish between those?
I'm not opposed to adding a new folder in the current rustup installation layout of some sort that you can possibly link a local directory to, IMHO the most important thing here is still what tool you are trying to make rustup work with.
I think this feature keeps the configuration largely unchanged. It only adds an additional lookup directory for externally managed toolchains, while the existing per-user links continue to work unchanged.
Not quite; the
alt-nightly-1.95 (shadowed)in your previous example might indicate thatalt-nightly-1.95has different semantics under different situations, that is exactly the kind of extra complexity we would like to avoid.@Rx1513 Thanks for your detailed reply! As I roughly understand it, you are referring to adding support for a new toolchain directory for automatic toolchain discovery (but read-only)?
When the package manager installs a package, user must manually link the distributed toolchain and then select it.
IRRC rustup treats every folder under the toolchain directory as a valid toolchain, so I am curious about how we could probably make it work with your case.
What is your specific distro/external tool that installs externally-managed Rust toolchains in a way that is compatible with your new directory proposed here? For Homebrew (what I'm using) for example, the system Rust toolchain is installed alongside other packages. Is rustup supposed to distinguish between those?
ALT Linux which uses apt-rpm. Currently we package our rust distribution in /usr/lib64/rust-toolchains and provide symlinks to it. So if you install rust and rustup and then replace /usr/bin/ toolchain symlinks with rustup proxies and then link toolchains you will be able to use both official and system's toolchain.
I'm not opposed to adding a new folder in the current rustup installation layout of some sort that you can possibly link a local directory to, IMHO the most important thing here is still what tool you are trying to make rustup work with.
Package manager, nfs, rsync directories or any other ways to distribute compiled artifacts which rustup is able to treat as a toolchain.
I think this feature keeps the configuration largely unchanged. It only adds an additional lookup directory for externally managed toolchains, while the existing per-user links continue to work unchanged.
Not quite; the
alt-nightly-1.95 (shadowed)in your previous example might indicate thatalt-nightly-1.95has different semantics under different situations, that is exactly the kind of extra complexity we would like to avoid.I understand the concern about added complexity. However, the (shadowed) tag is simply an indication that a shared toolchain has a conflicting name with a user's toolchain, meaning the user's toolchain will take precedence. Without this indication, it wouldn't be obvious why the shared toolchain isn't being run. Although I may be wrong here, since I had forgotten that the active toolchain is already marked as (active), which may already provide enough context.
@Rx1513 Thanks for sharing the context!
I think in this case I can allow say linking
toolchain/+systo/usr/lib64/rust-toolchains, and then you can call your new toolchain assys/alt-nightly-1.95. Then an extra CLI interface can be provided, sayrustup toolchain link --dir sys /usr/lib64/rust-toolchainsThis is possible because the
+prefix has been allowed for toolchain names in #4650 plus/is not allowed in regular custom toolchain names, so that will not cause name clashes with existing toolchains.Do you think that would be a better compromise?
However, the (shadowed) tag is simply an indication that a shared toolchain has a conflicting name with a user's toolchain, meaning the user's toolchain will take precedence.
That is correct. What I want to say here is that I would prefer not to create precedence in new features, unless proven absolutely necessary.
@Rx1513 Would you mind elaborating? IIRC relative links means that the link targets are relative, but in this case link targets are absolute.
foo/barjust looks for a toolchain located at$RUSTUP_HOME/toolchains/+foo/bar, and$RUSTUP_HOME/toolchains/+foomust link to an absolute target.I don't think we will allow other kinds of components;
/is just a separator and is only allowed to appear once.I meant that rustup should reject any path traversal like
+foo/../../malicious/baror+foo/bar/../../malicious/barso rustup behaves exactly like you described it.
Problem you are trying to solve
Hello!
As a package maintainer, I want to package different Rust toolchain distributions, such as stable, nightly, specific stable versions, or custom forks (for example, the ESP toolchain). To make this practical, some kind of toolchain multiplexer is needed. While I could write a minimal multiplexer myself, it would lack many useful features.
On the other hand, I could use rustup, which is widely adopted and well supported. However, its main drawback is poor integration with OS package managers. There is no automatic way to make a system-provided Rust toolchain available through rustup. Although it is possible to manually link a system toolchain, there is no mechanism to keep it synchronized with updates from the package manager.
As a user, I want to be able to use both the system-provided toolchain and the official Rust toolchain. For example, I may use one for day-to-day development and the other for feature testing or validation or in some cases I may want to use local distribution infrastructure. Which is possible but with some extra work.
Solution you'd like
To solve these issues, a separate shared directory managed by external tools could be added.
rustup should automatically discover toolchains from the shared directory.
Toolchains in
RUSTUP_HOMEshould take precedence over toolchains from the shared directory. If a local toolchain shadows a shared toolchain with the same name, this should be clearly indicated. For example:This feature should be opt-in. It should be possible to enable it through user or system configuration, or with an environment variable.
rustup toolchain listshould clearly indicate which toolchains belong to which mechanism. For example:Notes
Related to #313, #1085, and #2383.
Unresolved problem:
Should this feature provide a mechanism for specifying multiple shared directories, or should it support only a single shared directory?
I have some initial work in progress, so I would like to implement this feature.