Overrides that can be checked in to version control

[yazaddaruvala]

When working with a team of developers, it is most effective to be building/testing an application consistently.

Hardcoding dependency versions "in code" (managed/shared through version control) has worked out well for all other dependency management. Primarily for reproducibility locally, and across a team.

I can't think of a reason why this same strategy wouldn't work well for toolchain dependency management.

[Diggsey]

What exactly are you proposing? I can see some benefit to being able to pin the rustc version, although I'd hope that the backwards compatibility guarantees would make that unnecessary.

However, if you plan to pin the toolchain, that seems very unhelpful for anyone trying to build the project on a different host.

[yazaddaruvala]

I'm proposing to move the configuration of "overrides" into source control.

The only difference between the current overrides and this, would be that every developer working on the library or package, are forced to conform and build consistently.

that seems very unhelpful for anyone trying to build the project on a different host.

Different hosts can have conforming build environments; Hence, Docker et al.

Meanwhile you're right:

that seems very unhelpful for anyone trying to build the project on a different build environment.

1. Given that I've never had to worry about this for any of my projects (I've always had conformed build environments) -- I'm probably the wrong person to propose a solution here, although if needed I could take a stab at it.
2. Given the workaround is simple: Locally change the Rustup.toml, add Rustup.toml to .gitignore -- Is there even a need to solve this?

[Diggsey]

If you're going to use some form of virtualization/containerization (eg. docker) to get a consistent build environment regardless of the host then it seems like it would make sense to include the rustup/rust installation in that container?

Given that I've never had to worry about this for any of my projects (I've always had conformed build environments)

Pretty much any open source project will have people contributing from a variety of build environments (with the exception of virtualized/containerized builds) and in the remaining cases you can always have a setup script which installs the appropriate override.

Basically I don't see this being widely applicable, and I think it's more likely to be misunderstood as a feature, with people inadvertently checking in their rustup.toml files and causing havoc for anyone else trying to build the project. Given how little effort it is to add an override in the first place, it just doesn't seem worthwhile.

[yazaddaruvala]

Pretty much any open source project will have people contributing from a variety of build environments

Totally agreed!

Is rustup only meant for open source projects?

[Diggsey]

No, hence the second part of my comment: there are a select few cases where this would be useful (closed source projects which support only a single build target but don't use a containerized/virtualized build system), and in those cases there are already readily applicable solutions that aren't really any more effort (for example, checking in a setup script or post-checkout script that adds the relevant rustup override).

[yazaddaruvala]

for example, checking in a setup script or post-checkout script that adds the relevant rustup override

Sure, this is very possible. However, why should this setup script not be solved by the tool meant to do Rust toolchain management? Why should this relatively common usecase require each team to create a custom script?

Would you also be in favor of a setup script that managed your crate dependencies (it would be extremely simple to implement - curl into the appropriate dir), rather than Cargo.toml?

I very much agree, a script to setup Rustup overrides is a lot easier to maintain than a script to set up Crate dependencies. However, it is still one more thing each team has to maintain. At a fundamental level the same reasons to have a consistent way to manage crate dependencies across a team, should apply to a consistent way to manage toolchain dependencies across a team.

Does that make sense? If not, why do we have a disconnect? I'm assuming you prefer Cargo.toml to a custom setup script. If thats true, I'm very curious, why do you prefer Rustup users build a custom script, when Cargo users shouldn't?

[brson]

Setting the override declaratively is something supported by rbenv by writing to ./rbenv/version. It's a desirable feature, but for rustup it's a project-wide policy issue. So far we have on several occasions informally decided not to enable e.g. cargo to pin Rust versions. Needs wider discussion before changing rustup to allow projects to be pinned.

[Diggsey]

why do you prefer Rustup users build a custom script, when Cargo users shouldn't?

Because the Cargo.toml is completely platform independent, whereas a rustup.toml would be extremely platform specific. I feel there's a strong likelihood of it being used improperly. Additionally, it would result in some new rust version being silently installed on your system, which is not something I'd want to happen automatically.

If we do this, I'd at least like to see it broken down by host architecture, so that for each host target, you specify a corresponding toolchain to use: if your host architecture is not present then it would not try to override your default toolchain. The host architecture it would match against would be that configured by the "default host" setting.

[brson]

The std-aware cargo RFC looks like it might incidentally add a type of version pinning: if you write std = "x.y.z" cargo will effectively require a specific compiler.

[emk]

I find myself needing to re-invent this particular wheel for emk/heroku-buildpack-rust, as discussed in emk/heroku-buildpack-rust#11, which needs to know what Rustup channel to use for deploying a given git repo as a Heroku application. I was considering using Cargo.toml's [metadata] section to implement something like:

[metadata]
rustup_channel="stable"

Obviously, if multiple tools are going to need something like this, it would be better to standardize it in one place rather than re-inventing it everywhere.

[wagenet]

My big concern is making sure that different developer machines are running consistent versions. Suppose a bug is fixed in a version of Rust and I update my local copy of Rust to avoid it. Then I push out some code changes and another developer on the team accidentally forgets to update it. Then they make a build with the buggy version of Rust. I'd like to be able to at least enforce a minimum Rust version.

[yazaddaruvala]

@brson kind-of off topic. I know we have had this conversation before, but given the increasing interest in version pinning in general, and version pinning through cargo, are you reconsidering having rustup and cargo be separate tools[0]?

[0] I know, rustup has a lot to do with toolchains and not just std (or compiler) version. While, the linked RFC doesn't currently discuss pinning toolchains, nor does it discuss toolchains at all. However, for me as a user: toolchain version, compiler version, and stdlib version (or picking the lack of std) feel like very similar concepts that I'd prefer to manage in one place.

[metajack]

Overrides with a local file would solve many issues with Servo and related projects.

cc @aturon @paulrouget

[Diggsey]

My comment from the duplicate issue:

Something similar was brought up before. At the time I was against it because I didn't want to see people pinning specific compiler host triples to their projects, and making life difficult for people trying to just build it.

The other factor was that the stable compiler should be... stable, so it's unclear whether that should be handled by rustup, or by cargo (say by depending on at least a certain libstd version).

However, another important case is pinning a specific version of nightly, without pinning the host triple, which is quite important for development against nightly where there are frequent breakages. I'd be in favour of something like that, maybe with precedence lower than the existing override system, so that it's still easy to bypass if need be.

In short:

• Only allow pinning nightly version (and/or commit if that would be useful for servo)
• Versions specified in dependencies are ignored
• Existing directory override system has higher precedence