How to upgrade Rust safely without breaking CI

CI “helpfully” upgraded Rust. Then your laptop failed the same tests.

I’ve watched teams burn a morning on ghosts that disappeared the second everyone used the same rustc. You do not need a new debugging ritual. You need one source of truth for the toolchain and one place to catch drift fast.

Safe means you can rebuild yesterday

Reproducible wins.

I mean this literally: check out a commit from last week, run cargo test, and get the same compiler and the same dependency graph. If you cannot do that, you are gambling with every “minor” Rust upgrade.

Here’s the thing. rustup will pick a toolchain from multiple places, and the winner often is not the file you just added. I’ve tripped on this in CI more than once: the repo had rust-toolchain.toml, but a stale RUSTUP_TOOLCHAIN env var still forced a different compiler.

Pin the Rust toolchain in the repo (first)

Do this first.

Add rust-toolchain.toml at the repo root so every machine reads the same instruction. I keep it boring and explicit because “whatever stable is today” does not help when you need to bisect a regression next month.

• Exact pin for serious repos: set channel = “1.75.0” (or whatever you run) so rustup selects that exact toolchain until you change the file.
• Moving channel for throwaway work: set channel = “stable” only if you accept churn. Stable moves. Your environment still needs to run updates or reinstall to pick it up.
• Keep CI lean: set profile = “minimal” so rustup does not drag docs and extra components into every runner.

Then verify it like you mean it. Run rustup show. If rustup says “active toolchain: stable,” you did not pin what you think you pinned. Read the “active because” line and follow it up the directory tree.

Make Cargo refuse the wrong compiler

Fail early.

This bit me when a teammate pulled the repo on an older laptop and Cargo tried to build until it hit a weird error deep in a dependency. Add rust-version so Cargo stops before it wastes your time.

• Put this in Cargo.toml: under [package], add rust-version = “1.75” (match your policy).
• What you get: Cargo errors out immediately on an older compiler, instead of compiling half the graph and exploding later.

In most cases, Cargo tooling also behaves better around MSRV when you set rust-version, but it depends on your resolver and your dependency mix. I have not tested every corner here, especially with older workspaces that still carry legacy settings.

Lock dependencies on purpose (and know when not to)

Pinning Rust alone won’t save you.

I’ve seen builds change under the same rustc because somebody ignored Cargo.lock. Cargo wrote that file because resolution changes. For an app or service, I usually commit Cargo.lock and treat updates as a deliberate event: run cargo update, run tests, commit the diff. For a library, you might skip committing Cargo.lock so downstream users exercise wider version ranges. Some teams commit it anyway. I do not love that for libraries, but I get why they do it when CI flakiness gets old.

Never hand-edit Cargo.lock. Seriously. If you “fix” it in a text editor, you will ship a mess and you will not notice until a clean runner breaks.

Force CI to follow the same rules (no guessing)

Print the versions.

I do not trust a Rust pipeline that runs tests before it proves which compiler it used. Make CI install the toolchain from the repo file, then print versions every run. Yes, even for a patch-only change.

• Install and verify step: run rustup show, then rustc -V, then cargo -V before cargo test.
• Don’t get tricked by caching: cache downloads if you want, but still print versions so you can see drift when it happens.

If your CI image already ships with Rust, you can still do this, but you need to watch for “helpful” preinstalls that bypass rustup. That’s when version printing turns into your smoke alarm.

Upgrade Rust on purpose (small branch, fast feedback)

Keep it small.

Create a branch. Bump the pinned channel in rust-toolchain.toml. Install it locally, run tests, then let CI confirm it sees the same toolchain. Rust stable releases land about every six weeks, so a calendar reminder beats a surprise failure on a Monday.

Other stuff you might want to do: bump clippy/rustfmt components, tweak caches, update the base image, the usual.

When rustup ignores your pin

This almost always comes from an override you forgot.

Check the obvious offenders in this order: a script that runs cargo +nightly, a RUSTUP_TOOLCHAIN env var in CI, or a directory override you set months ago and never cleaned up. Remove directory overrides with rustup override unset. Then run rustup show again and actually read the “active because” line.

If rustup disagrees with you, it is probably right.

There’s probably a cleaner checklist for this, but slowing down and reading rustup show has saved me every time.