Sample `shoo.d` file prior to this change is below, note the `build.rs`
at the end, which was not from my package.
From booping the debugger, I found this was coming from
`compiler_builtins`. This is not really their bug though: if a build.rs
asks for rerun-if-changed on some crate relative path, this will happen
in general. So I've fixed it in Cargo and added a test to prevent it
regressing.
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s build.rs
```
This change fixes it so it's like:
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.39/build.rs /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.14/build.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s
```
Restore crates.io's `SourceId` hash value to before
This commit restores the hash value of the crates.io `SourceId` to what
it was before #9384. In #9384 the enum variants of `SourceKind` were
reordered which accidentally changed the hash value of the `SourceId`
for crates.io. A change here means that users with a new version of
Cargo will have to redownload the index and all crates, which is
something that we strive to avoid forcing.
In changing this, though, it required a manual implementation of `Ord`
to still contain the actual fix from #9384 which is to sort `SourceKind`
differently from how it's defined. I was curious as to why this was
necessary since it wasn't ever necessary in the past and this led to an
odd spelunking which turned up some interesting information. Turns out
Rust 1.47 and after had a breaking change where Cargo would sort
dependencies differently. This means that #9334 *could* have been opened
up much earlier, but it never was. We ironically only saw an issue when
we fixed this regression (although we didn't realize we were fixing a
regression). This means that we are now permanently codifying the
regression in Cargo.
This commit restores the hash value of the crates.io `SourceId` to what
it was before #9384. In #9384 the enum variants of `SourceKind` were
reordered which accidentally changed the hash value of the `SourceId`
for crates.io. A change here means that users with a new version of
Cargo will have to redownload the index and all crates, which is
something that we strive to avoid forcing.
In changing this, though, it required a manual implementation of `Ord`
to still contain the actual fix from #9384 which is to sort `SourceKind`
differently from how it's defined. I was curious as to why this was
necessary since it wasn't ever necessary in the past and this led to an
odd spelunking which turned up some interesting information. Turns out
Rust 1.47 and after had a breaking change where Cargo would sort
dependencies differently. This means that #9334 *could* have been opened
up much earlier, but it never was. We ironically only saw an issue when
we fixed this regression (although we didn't realize we were fixing a
regression). This means that we are now permanently codifying the
regression in Cargo.
Fix loading `branch=master` patches in the v3 lock transition
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
Fix build-std updating the index on every build.
https://github.com/rust-lang/rust/pull/83776 has caused a problem where build-std will update the index on every build. That PR added `std_detect` from the `stdarch` submodule as a path dependency of `std`. However, since `stdarch` has a workspace of its own, an exclusion had to be added to `Cargo.toml` so that it does not treat `std_detect` as a workspace member (because nested workspaces are not supported).
The problem is that the std `Cargo.lock` file is built thinking that `std_detect` is *not* a workspace member. This means that its dev-dependencies are not included. However, when cargo resolves the std workspace, it doesn't know that `std_detect` should be excluded, so it considers it a workspace member (because it is a path dependency). This means that it expects the dev-dependencies to be in Cargo.lock. Because they are missing, it ends up poisoning the registry and triggering an update:
> poisoning registry `https://github.com/rust-lang/crates.io-index` because std_detect v0.1.5 (/Users/eric/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/stdarch/crates/std_detect) looks like it changed auxv
The solution here is to skip dev-dependencies if they are not actively being resolved, even if the package is a workspace member.
This has happened before (#8962), so I have updated the test to check for it.
There are some alternative solutions I considered:
* Add support for nested workspaces. 😄
* Use a symlink to `std_detect` in the `rust-lang/rust` repository so that it appears to cargo as-if it is "outside" of the stdarch workspace, and thus can be treated like a normal workspace member (and remove the "exclude"). That seems a little hacky.
Fixes#9390
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
Fix disagreement about lockfile ordering on stable/nightly
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name
Fixes https://github.com/rust-lang/cargo/issues/9362.
The whole point of https://github.com/rust-lang/rust/pull/77802/ was to allow specifying this granularly, giving a hard error defeats the point.
I didn't know how to check what targets were reverse-dependencies of build.rs, so I just unconditionally use the library name (and give a hard error for anything else, even if it's the name of one of the binaries). End-users can still opt-in with RUSTC_BOOTSTRAP=1, and no public binaries use RUSTC_BOOTSTRAP=1, so I don't think this a big deal in practice.
<details><summary>Script to verify all crates using RUSTC_BOOTSTRAP=1 have a library</summary>
```sh
curl https://pastebin.com/raw/fGQ97xP6 | cut -d / -f1 | grep -v shnatsel | grep -v cargo- | sed 's#-\([0-9]\)#/\1#' | xargs -i curl -s -I -L "https://docs.rs/{}/" -w "%{http_code}\n" -o/dev/null
```
It should output 20 200s in a row.
</details>
r? `@ehuss` cc `@mark-simulacrum`
I don't know what cargo's policy is for backports, but this should be backported to 1.52.
Fix#9350 (cargo build -Z help is missing options)
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Handle man pages better on Windows.
If a user has `man` installed on Windows via msys/mingw/etc, then `cargo help <subcommand>` would fail with `No manual entry for C:\Users\User\AppData\Local\Temp\cargo-manlSKwTQ`. This is because the cygwin universe does not handle windows-style paths and does not auto-translate in this scenario.
The solution here is to run the command from within the temp directory and use a relative path which *should* work on all platforms and environments. I tested on windows (powershell, cmd, and mingw), macos, and linux.
Fixes#9197
Don't re-use rustc cache when RUSTC_WRAPPER changes
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
This doesn't work because it uses the name of the build script, which is
always build_script_build. I'm not sure what to change it to - the name
of the library crate could be different than the name of the package,
and there could be multiple different crates being compiled in the same
package.
refactor: remove `CargoResultExt`
All `chain_err` -> `with_context` are done by IDE refactoring.
- Remove `CargoResultExt`.
- Call `format!` instead of `anyhow::format_err!` to reduce unnecessary macro expansions.
e870eac996/src/cargo/util/errors.rs (L12-L18)
Track "CARGO" in environment fingerprint.
There is an issue where if a package includes an `env!("CARGO")`, that value is not tracked in the fingerprint. If different cargos are used, it will not rebuild. This changes it so that the path to cargo will be tracked in the fingerprint if the CARGO env var is in the dep-info file.
This came up with rust's build system where it [tracks the env](60158f4a7c/src/bootstrap/config.rs (L574)). If you build rust once, and then change the `cargo` value in `config.toml` and try building again, it would not pick up the change which caused me some confusion.
In theory, cargo could fingerprint this every time, but I figure that could be disruptive and trigger needlessly in some situations.
This diff is a little bigger than I would like for such an obscure case. As an alternative, I think rustbuild could just print `cargo:rerun-if-env-changed=CARGO`, but I figure this could potentially be useful for other projects.
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
Update clippy lint allow set.
This updates the clippy lints to default allow. We would prefer not to take clippy lint PRs at this time as there are a number of false positives and subjective style changes that we would rather not review.
I left a couple lints as `warn` that I have found useful when refactoring.
