Update Windows env uppercase key check.
This is the followup for #9646. Rust now correctly spawns processes with preserved casing on nightly on Windows, so the exceptions here are no longer needed.
When selecting among several versions of a paackage, prefer versions
from `[patch]` sections over other versions, similar to how locked
versions are preferred.
Patches come in the form of a Dependency and not a PackageId, so this
preference is expressed with `prefer_patch_deps`, distinct from
`try_to_use`.
This works by introspecting rustc's error output, using the JSON format
to determine whether it's a warning or error, then skipping it
altogether if it's a summary of the diagnostics printed.
Before:
```
src/main.rs:1:10: warning: trait objects without an explicit `dyn` are deprecated
src/main.rs:1:1: error[E0601]: `main` function not found in crate `wrong`
src/main.rs:1:9: error[E0038]: the trait `Clone` cannot be made into an object
error: aborting due to 2 previous errors; 1 warning emitted
error: could not compile `wrong`
```
After:
```
$ cargo check --message-format short
src/main.rs:1:10: warning: trait objects without an explicit `dyn` are deprecated
src/main.rs:1:1: error[E0601]: `main` function not found in crate `wrong`
src/main.rs:1:9: error[E0038]: the trait `Clone` cannot be made into an object
error: could not compile `wrong` due to 2 previous errors; 1 warning emitted
```
Include the linker in the fingerprint.
This adds the linker from the `[target]` config table to the fingerprint. Previously, changing the value would not trigger a rebuild.
Temporarily disable windows env test.
This temporarily disables the `target_in_environment_contains_lower_case` test on Windows until https://github.com/rust-lang/rust/pull/85270 gets into nightly. That PR changes it so that env vars are case-preserved on Windows. My intent is that after that is in nightly, we can remove the windows-specific code in `Config`, and this test should work the same on all platforms.
Closes#9630
Handle git deleted files with dirty worktree.
When listing git files for things like `cargo package`, it was including unstaged deleted files. This is because the file is still in the index, so it was included in the list. `cargo package --allow-dirty` would then fail with a confusing "file not found" error.
This fixes it by keeping a set of deleted files, and skipping those. This allows `cargo package --allow-dirty` to work.
Closes#9580
Fix `BorrowMutError` when calling `cargo doc --open`
~~I'm not sure why the existing test suite didn't catch this, it definitely calls `cargo doc --open`.~~
I had
```toml
[doc.extern-map]
std = "local"
```
in my `.cargo/config.toml`. Will write a test case that sets that and then tries to run `cargo doc --open`.
Closes#9530
Error when packaging with git dependencies without version
If `cargo package` is run on a package that specifies a git dependency
without a version, cargo will error. This should help with clarifying
that git dependencies will be stripped when packaging, and that the
dependency has to be fetched from a registry.
Closes#9442
If `cargo package` is run on a package that specifies a git dependency
without a version, cargo will error. This should help with clarifying
that git dependencies will be stripped when packaging, and that the
dependency has to be fetched from a registry.
Unify weak and namespaced features.
This unifies weak and namespaced features in order to simplify the syntax and semantics. Previously there were four different ways to specify the feature of a dependency:
* `package-name/feature-name` — Enables feature `package-name` on self and enables `feature-name` on the dependency. (Today's behavior.)
* `package-name?/feature-name` — Only enables `feature-name` on the given package if it that package is enabled and will also activates a feature named `package-name` (which must be defined implicitly or explicitly).
* `dep:package-name/feature-name` — Enables dependency `package-name`, and enables `feature-name` on that dependency. This does NOT enable a feature named "package-name".
* `dep:package-name?/feature-name` — Only enables `feature-name` on the given package if it that package is enabled. This does NOT enable a feature named "package-name".
This changes it so there are only two:
* `package-name/feature-name` — Today's behavior.
* `package-name?/feature-name` — Only enables `feature-name` on the given package if it that package is enabled. This does NOT enable a feature named "package-name" (the same behavior as `dep:package-name?/feature-name` above).
This is a fairly subtle change, and in most cases probably won't be noticed. However, it simplifies things which helps with writing documentation and explaining how it works.
Change `rustc-cdylib-link-arg` error to a warning.
In #9523, an error was added if `cargo:rustc-cdylib-link-arg` was issued in a build script without actually having a cdylib target. This uncovered that there was an unintentional change in #8441 to cause those link args to be applied to transitive dependencies.
This changes it so that the error is now a warning, with a note that this may become an error in the future. It also changes it so that the unstable `rustc-link-arg*` instructions only apply to the package that emitted them.
Don't allow config env to modify vars set by cargo
This changes how the `[env]` variables are applied. They will no longer replace any variables that are being set by cargo itself, whether or not the `force` flag is set. The `force` flag still determines whether existing variables from the environment can be overridden or not.
Addresses unresolved issue in #9539
Fix dep-info files including non-local build script paths.
I derped in #8177 and accidentally used the wrong unit when iterating over the dependencies when writing the `.d` file. The consequence here is that all the `rerun-if-changed` paths from a unit's dependencies are included in the `.d` file. This fixes it so that it does not include non-local dependencies.
Fixes#9445
Enhancements to testsuite error output.
This includes several changes to the testsuite error reporting in theory to help make it easier to see differences when the test output changes. The key change is to add a Myers diff with a little colored output to highlight the differences. Here is an example:
<img width="666" alt="image" src="https://user-images.githubusercontent.com/43198/122311381-e8d43580-cec6-11eb-81b4-e2675f10d6ba.png">
The rest of the changes here are various refactorings to try to clean up the diffing code. It ended up being far more changes than I was intending, but I did try to split things into commits to separate them.
I don't trust that all these transformations won't have unintended
consequences on other platforms. It is nice to verify there aren't any
backslash shenanigans on other platforms.
Enable support for fix --edition for 2021.
This adds support for using `cargo fix --edition` to migrate to 2021.
This also uses the new, currently unstable, `--force-warns` flag. This was added because there were a significant number of crates that were "allow"ing lints that are required for migrating the edition. This wasn't a problem for 2018, because its lints were new, and all "allow" to start. For 2021, several older "warn" lints are becoming hard errors. "allow"ing them would cause the migration to fail.
The output was checking the `targets`, whose order depends on the
filesystem order. Instead of checking all the output, just
check the one field this test is for.
Fix rustc/rustdoc config values to be config-relative.
The `rustc`, `rustdoc`, `rustc_wrapper`, and `rustc_workspace_wrapper` config values (in the `[build]` table) were being interpreted literally. This caused a problem if you used a relative path like `foo/rustc`. This would be interpreted as a relative path from whatever cwd cargo launches rustc from, which changes for different scenarios, making it essentially unusuable (since crates.io dependencies wouldn't be buildable).
Additionally, due to https://github.com/rust-lang/rust/issues/37868, it is a bad idea to use relative paths.
This changes it so that those paths are config-relative. Bare names (like "my-rustc-program") still use PATH as before.
This also includes a commit to centralize the rustc-wrapper program used by several tests so that it isn't built multiple times (and to allow several tests to work on windows).
Fixes#8202
If a combination of --bins, --benches, --examples, --tests flags have
been specified, but we didn't match on anything after resolving the unit-list,
we emit a warning to make it clear that cargo didn't do anything and that the
code is unchecked.
Closes#9536
Add `--prune` option for cargo-tree
Part of #8105
Prune the given package from the display of the dependency tree. Also providing a nice suggestion if the package is not within the resolved dependency graph.
Configure hosts separately from targets when --target is specified.
This prevents target configs from accidentally being picked up when cross compiling from hosts that have the same architecture as their targets.
closes#3349
Add some validation to rustc-link-arg
This adds some validation, so that if a `cargo:rustc-link-arg-*` build script instruction specifies a target that doesn't exist, it will generate an error. This also changes a parse warning to an error if the `=` is missing from BIN=ARG.
I intentionally did not bother to add the validation to config overrides, as it is a bit trickier to do, and that feature is very rarely used (AFAIK), and I'm uncertain if rustc-link-arg is really useful in that context.
cc #9426
`cargo tree -e no-proc-macro` to hide procedural macro dependencies
Probably resolves#9165
`cargo tree -e no-proc-macro` now hides procedural macro dependencies.
Note that when passed with `--invert <spec>`, the spec's reverse proc-macro dependencies are also hidden. Is this desired result?
Also, since I want `-p <spec>` and `-i <spec>` works with any valid package-id in the project, the filter logic is written in print stage instead of graph build stage
Update to semver 1.0.0
I am working on a 1.0.0 of the `semver` crate some time this week. It would be good to confirm Cargo will be able to use it, beforehand!
It's a from-scratch rewrite, but https://github.com/dtolnay/semver/issues/237 has code to compare against 0.10.0 (currently used by Cargo) how every possible version requirement currently published to crates.io matches against every possible crate version. The differences are all broken syntax like `^0-.11.0` previously parsing with ".11.0" as a pre-release string (which is invalid, because pre-release are not allowed to contain empty dot-separated identifiers) and `~2.0-2.2` previously parsing with "2.2" as a pre-release string, when the user almost certainly meant `>=2.0, <=2.2`. I'm not sure how much of those you want to add code into Cargo to preserve behavior, but I would be happy to do it.
Add `cargo:rustc-link-arg-bin` flag.
This PR implements a `cargo:rustc-link-arg-bin` command to specify per-binary link args from build scripts. This follows the suggestion from the tracking issue #9426.
Syntax is `cargo:rustc-link-arg-bin=BIN_NAME=ARG`
This was previously possible to do using the `#[link_args=".."]` attribute, but it was removed in rust-lang/rust#83820 in favor of the Cargo extra-link-args feature, which can currently not specify different link args for different bins in the same crate. This PR adds back the ability to do that.
Add a cargo-doc.browser config option
The idea of this option is to allow cargo to use a separate browser from
the rest of the system. My motivation in doing this is that I want to
write a script that adds a symbolic link in some web root on my system
such that I can access my docs via the http protocol to avoid the
limitations of the file protocol that are accessibility problems for me.
For instance, zoom is not retained across multiple pages and Stylus
filters don't work well.
The idea of this option is to allow cargo to use a separate browser from
the rest of the system. My motivation in doing this is that I want to
write a script that adds a symbolic link in some web root on my system
such that I can access my docs via the http protocol to avoid the
limitations of the file protocol that are accessibility problems for me.
For instance, zoom is not retained across multiple pages and Stylus
filters don't work well.
Improve two error messages
The first error message saying "an integer" is confusing because if you give it `4` it's an integer but it will still complain that it must be an integer. So it's more specific now and tells you that it actually needs to be `1`, `2` or `3`.
In the second error there was a space missing. It says `is not a valid setting,must be`.
The variable is set to $target_dir/$config/tmp
This is a directory where tests & benches can place testcasei-related data
for use by the tests.
cargo makes sure the directory exists when building tests/benches.
Add `report` subcommand.
This renames the `cargo describe-future-incompatibilities` subcommand to `cargo report future-incompatibilities`. The intent here is to have a general `report` subcommand that can be used for other reports in the future. This helps avoid the proliferation of top-level subcommands, and to help keep the CLI a little more organized.
Respect Cargo.toml `[package.exclude]` even not in a git repo.
May resolves#9054
This bug (or feature?) has been lingering for a while. #7680 fixed the `cargo package` part but `cargo vendor` is still affected by the heuristic rule of ignoring dotfiles. ~~I propose to drop the rule and include dotfiles by default even if the package is not under git-controlled~~. See below.
## Updated: Changes Summary
`cargo vendor` vendors dependencies without git-controlled but `cargo package` often runs under a VCS like git. [These lines](https://github.com/rust-lang/cargo/blob/1ca930b/src/cargo/sources/path.rs#L161-L168) are where they diverges: `fn list_files_walk_except_dot_files_and_dirs` builds [its own ignore instance], which cannot merge with other filter rules from `[package.exclude]`. This causes some patterns to not work as expected, such as re-including file after ignoring dotfiles `[.*, !negated_file]`.
To make re-include (negate) rule works, this patch adds the excluding dotfiles rule directly into the `package.exclude` ignore instance if **_no include option nor git repo exists_**. Other old behaviors should not change in this patch.
[its own ignore instance]: https://github.com/rust-lang/cargo/blob/1ca930b6/src/cargo/sources/path.rs#L364-L366
Fix collision doc tests randomly failing.
This fixes some tests that were randomly failing on CI. The cause is that #9419 added a remove_dir_all on the `doc` directory. However, if two jobs are trying to write to that directory at the same time, this can cause errors. The failure rate is low (a little over 1%), and I was unable to reproduce locally (only on GitHub's CI and only on the Windows job).
The solution is to run the jobs with -j1 so they run serially.
I only saw errors for `collision_doc_sources`, but to be on the safe side I added j1 to similar tests.
Fix dep-info files emitting paths relative to deps' roots
Sample `shoo.d` file prior to this change is below, note the `build.rs`
at the end, which was not from my package.
From booping the debugger, I found this was coming from
`compiler_builtins`. This is not really their bug though: if a build.rs
asks for rerun-if-changed on some crate relative path, this will happen
in general. So I've fixed it in Cargo and added a test to prevent it
regressing.
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s build.rs
```
This change fixes it so it's like:
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.39/build.rs /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.14/build.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s
```
Fix rebuild issues with rustdoc.
This fixes two issues related to rebuilds with rustdoc:
* Switching features when running `cargo doc` would result in Cargo not rebuilding the documentation. This is because it was keeping the fingerprints in separate directories based on the features used. However, the rustdoc output isn't keyed off the metadata hash, so although the old fingerprint seemed "up to date", in reality the documentation was rewritten and needs to be rebuilt. The solution is to use a simplified hash for the fingerprint directory name.
* Removing items does not remove the files from the doc directory. This changes it to clear the package's doc directory before running rustdoc, to ensure any stale files are removed.
I'm a little concerned about potential performance impact of running `remove_dir_all`, but I think it shouldn't be too bad?
Fixes#7370
Sample `shoo.d` file prior to this change is below, note the `build.rs`
at the end, which was not from my package.
From booping the debugger, I found this was coming from
`compiler_builtins`. This is not really their bug though: if a build.rs
asks for rerun-if-changed on some crate relative path, this will happen
in general. So I've fixed it in Cargo and added a test to prevent it
regressing.
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s build.rs
```
This change fixes it so it's like:
```
target/riscv64imac-mu-shoo-elf/release/shoo: /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.39/build.rs /home/jade/.cargo/registry/src/github.com-1ecc6299db9ec823/log-0.4.14/build.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/core_arch_docs.md /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/macros.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/mod.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd.rs /home/jade/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/../../stdarch/crates/core_arch/src/simd_llvm.rs crates/build_bits/src/lib.rs shoo/src/main.rs shoo/src/task.rs shoo/src/vectors.s
```
This prevents target configs from accidentally being picked up when
cross compiling from hosts that have the same architecture as their
targets.
closes#3349
Expose build.target .cargo/config setting as packages.target in Cargo.toml
Hey!
I'm trying to do my first cargo contribution by implementing per-crate target settings as per [the irlo thread](https://internals.rust-lang.org/t/proposal-move-some-cargo-config-settings-to-cargo-toml/13336) ; and I think I have a draft that looks good-ish (the root units returned by `generate_targets` have the right kinds set).
Closes#7004
**_Edit: the below problem description is now solved in the latest version of this PR, please ignore_**
But for some reason running on a test project now blocks on `Blocking waiting for file lock on build directory` and I have literally no idea how my changes could trigger this… would anyone have an idea of how the changes could lead to infinitely blocking there? (I already tried cargo clean just in case and it didn't appear to help)
FWIW, the output that looks hopeful to me is, on my testbed workspace:
```
Root units [out of generate_targets] are [...]:
- package ‘smtp-client’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-server’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-rpc’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config-example’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config-example’ with kind ‘Target(CompileTarget { name: "wasm32-unknown-unknown" })’
- package ‘smtp-queue’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-message’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-server-fuzz’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘yuubind-config’ with kind ‘Target(CompileTarget { name: "wasm32-unknown-unknown" })’
- package ‘yuubind’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
- package ‘smtp-queue-fs’ with kind ‘Target(CompileTarget { name: "x86_64-unknown-linux-gnu" })’
```
(where both `yuubind-config` and `yuubind-config-example` are being configured to be `wasm32-unknown-unknown` and the other ones stay as host).
Interestingly enough, if I remove the `target` setting from `yuubind-config` (and leave it on `yuubind-config-example`) then it does no longer block on waiting for file lock on build directory, even though it does not actually compile with `wasm32-unknown-unknown`. And it does appear to correctly build yuubind-config-example as wasm32.
My investigation shows that it appears to happen iff there is a package with `package.target` being set that has dependencies.
This most likely is a bug in my code (eg. I build only the root units and not the whole unit graph maybe?), and am going to keep investigating it as such, but maybe someone would already know how dependency resolution could interact with build lock acquisition and give me hints?
Anyway, thank you all for all you do cargo!
Fix loading `branch=master` patches in the v3 lock transition
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
This commit fixes an issue pointed out during #9352 where in the v2->v3
lock file transition (currently happening on nightly) Cargo will not
correctly use the previous lock file entry for `[patch]` directives that
point to git dependencies using `branch = 'master'` explicitly. The
reason for this is that Cargo previously, with the v2 format, considered
`branch=master` and `DefaultBranch` to be equivalent dependencies. Now
that Cargo treats those as distinct resolve nodes we need to load lock
files that use `DefaultBranch` and transparently use those for
`branch=master` dependencies.
These lock file nodes do not naturally unify so we have to go out of our
way to get the two to line up in modern Cargo. This was previously done
for the lock file at large, but the previous logic didn't take `[patch]`
into account. Unfortunately almost everything to do with `[patch]` and
lock files is pretty complicated, and this is no exception. The fix here
is wordy, verbose, and quite subtle in how it works. I'm pretty sure it
does work though and I think that this should be good enough to at least
transition most users off the v2 lock file format. Once this has baked
in Cargo for some time (on the scale of a year) I would hope that we
could just remove this logic since it's only really here for a
transitionary period.
Closes#9352
Fix disagreement about lockfile ordering on stable/nightly
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name
Fixes https://github.com/rust-lang/cargo/issues/9362.
The whole point of https://github.com/rust-lang/rust/pull/77802/ was to allow specifying this granularly, giving a hard error defeats the point.
I didn't know how to check what targets were reverse-dependencies of build.rs, so I just unconditionally use the library name (and give a hard error for anything else, even if it's the name of one of the binaries). End-users can still opt-in with RUSTC_BOOTSTRAP=1, and no public binaries use RUSTC_BOOTSTRAP=1, so I don't think this a big deal in practice.
<details><summary>Script to verify all crates using RUSTC_BOOTSTRAP=1 have a library</summary>
```sh
curl https://pastebin.com/raw/fGQ97xP6 | cut -d / -f1 | grep -v shnatsel | grep -v cargo- | sed 's#-\([0-9]\)#/\1#' | xargs -i curl -s -I -L "https://docs.rs/{}/" -w "%{http_code}\n" -o/dev/null
```
It should output 20 200s in a row.
</details>
r? `@ehuss` cc `@mark-simulacrum`
I don't know what cargo's policy is for backports, but this should be backported to 1.52.
Fix#9350 (cargo build -Z help is missing options)
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
> Do not merge yet, some options are still undocumented.
Fix#9350 (cargo build -Z help is missing options)
Add a procedural macro to declare `CliUnstable` struct and provide help messages instead of hard-coding help in `src/bin/cargo/cli.rs`
> Flags documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html
Feedback welcome
This commit fixes an issue where the order of packages serialized into a
lock file differs on stable vs nightly. This is due to a bug introduced
in #9133 where a manual `Ord` implementation was replaced with a
`#[derive]`'d one. This was an unintended consequence of #9133 and means
that the same lock file produced by two different versions of Cargo only
differs in what order items are serialized.
With #9133 being reverted soon on the current beta channel this is
intended to be the nightly fix for #9334. This will hopefully mean that
those projects which don't build with beta/nightly will remain
unaffected, and those affected on beta/nightly will need to switch to
the new nightly ordering when it's published (which matches the current
stable). The reverted beta will match this ordering as well.
Closes#9334
Don't re-use rustc cache when RUSTC_WRAPPER changes
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
This doesn't work because it uses the name of the build script, which is
always build_script_build. I'm not sure what to change it to - the name
of the library crate could be different than the name of the package,
and there could be multiple different crates being compiled in the same
package.
We cache initial `rustc --version` invocations, to speed up noop builds.
We check the mtime of `rustc` to bust the cache if the complier changed.
However, before this PR, we didn't look at mtimes of `RUSTC_WRAPPER` /
`RUSTC_WORKSPACE_WRAPPER`, so we could've re-use old cache with new
wrapper.
"exit code" is wrong terminology on Unix. I am trying to fix this in
Rust stdlib in
https://github.com/rust-lang/rust/pull/83462
but this currently breaks the cargo test suite.
See that MR for full explanation of the change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Default macOS targets to `unpacked` debuginfo
This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
This commit fixes an issue where packages which specify `resolver = '2'`
cannot be packaged if they also have a `package.metadata` table. The
issue is that the `toml` serialization implementation serializes fields
in order which requires that tables be emitted last.
Refactor feature handling, and improve error messages.
This changes the way feature strings are handled with an eye towards fixing some improper handling and to improve error messages. The key change is to stop treating all features as free-form strings and instead try to handle them as typed values. This helps avoid needing to deal with parsing different feature syntax (like `dep:` or `foo/bar`) or forgetting to handle it properly.
Overview of refactoring changes:
* `RequestedFeatures` changed to an enum to differentiate between features coming from the command-line, and those that are from a dependency.
* Moved parsing of CLI features to an earlier stage (now stored in `CompileOptions`), and ensures that they are properly handled as `FeatureValue` instead of strings.
* Pushed some feature validation earlier. For example, `DetailedTomlDependency` now validates things so you can see the location for the errant `Cargo.toml` (previously some validation was deep in the resolver, which provided poor errors).
This is a pretty large PR, but at the core it is just changing `RequestedFeatures` and then dealing with the fallout from that. Hopefully this is an improvement overall.
List of user-visible changes:
* Fix handling in resolver V2 of `--features bar?/feat` and `--features dep:bar`
* Better error handling for `bar/feat` and `dep:bar` in dependency declarations.
* Feature keys in the `[features]` table can no longer contain slashes.
* Fixed a minor issue with `cargo tree -e features --all-features -Z namespaced-features`
* Fixed a panic with `cargo tree` involving `-Z weak-dep-features`
I did a small amount of benchmarking, and I wasn't able to record much of a difference.
Allow `cargo update` to operate with the --offline flag
Closes#9248
Limiting to `--precise` operations as the package should be cached locally prior to the usage of offline.
Added an additional test to [offline.rs](tests/testsuite/offline.rs).
Add report if `cargo fix --edition` changes features.
This adds a small report if using `cargo fix --edition` to transition from 2018 to 2021 to determine if the new resolver will result in different features.
There is a gauntlet of checks that must pass before it even tries to show the differences:
* If the resolver is already specified, skip.
* If in a virtual workspace, skip (no way to specify global edition).
* If not migrating the root package, skip. The edition only changes the resolver in the root package.
* If not migrating all targets, skip. It's uncertain if the user intends to set the package edition or not.
Closes#9048
Fix --feature pkg/feat for V1 resolver for non-member.
#8997 had an unintended regression where `-p foo --feature foo/feat` syntax where `foo` is an **optional non-member** fails with an error that `foo` did not match any packages. The issue is that the member/feature selection routine needed to slot this into the features for the package in the current working directory (it was incorrectly treating `foo` as a workspace member).
V2 outright does not allow specifying features for non-workspace members.
Fixes#9265
Support [patch] in .cargo/config files
This patch adds support for `[patch]` sections in `.cargo/config.toml`
files. Patches from config files defer to `[patch]` in `Cargo.toml` if
both provide a patch for the same crate.
The current implementation merge config patches into the workspace
manifest patches. It's unclear if that's the right long-term plan, or
whether these patches should be stored separately (though likely still
in the manifest). Regardless, they _should_ likely continue to be
parsed when the manifest is parsed so that errors and such occur in the
same place regardless of where a patch is specified.
Fixes#5539.
Package ID specification urls must contain a host
Resolves https://github.com/rust-lang/cargo/issues/9041
Not sure which commit breaks this. Cargo shipped with rust 1.46 didn't unwrap on a `None` but 1.47 did. Even checkouted to 149022b1d8 (cargo of rust 1.46), it still unwrap unexpectedly.
So I ended up following the [Specification grammer](https://doc.rust-lang.org/cargo/reference/pkgid-spec.html#specification-grammar) to make sure there is a `host` in the pkgid urls.
<details>
<summary>See console output</summary>
cargo of rust 1.46
```console
$ cargo +1.46 -vV
cargo 1.46.0 (149022b1d 2020-07-17)
release: 1.46.0
commit-hash: 149022b1d8
commit-date: 2020-07-17
$ cargo +1.46 pkgid /path
error: package ID specification `/path` matched no packages
```
cargo of rust 1.47
```console
$ cargo +1.47 -vV
cargo 1.47.0 (f3c7e066a 2020-08-28)
release: 1.47.0
commit-hash: f3c7e066ad
commit-date: 2020-08-28
$ cargo +1.47 pkgid /path
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/tools/cargo/src/cargo/core/package_id_spec.rs:234:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
cargo on commit 149022b1d8
```console
$ git checkout 149022b1d8
$ cargo run -- pkgid /path
Compiling cargo-platform v0.1.1 ([..]/cargo/crates/cargo-platform)
Compiling crates-io v0.31.1 ([..]/cargo/crates/crates-io)
Compiling cargo v0.47.0 ([..]/cargo)
Finished dev [unoptimized + debuginfo] target(s) in 22.90s
Running `target/debug/cargo pkgid /path`
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/cargo/core/package_id_spec.rs:234:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
</details>
cc rust-lang/rust#71249
This implements the Cargo side of 'Cargo report future-incompat'
Based on feedback from alexcrichton and est31, I'm implemented this a
flag `--future-compat-report` on `cargo check/build/rustc`, rather than
a separate `cargo describe-future-incompatibilities` command. This
allows us to avoid writing additional information to disk (beyond the
pre-existing recording of rustc command outputs).
This PR contains:
* Gating of all functionality behind `-Z report-future-incompat`.
Without this flag, all user output is unchanged.
* Passing `-Z emit-future-incompat-report` to rustc when
`-Z report-future-incompat` is enabled
* Parsing the rustc JSON future incompat report, and displaying it
it a user-readable format.
* Emitting a warning at the end of a build if any crates had
future-incompat reports
* A `--future-incompat-report` flag, which shows the full report for
each affected crate.
* Tests for all of the above.
At the moment, we can use the `array_into_iter` to write a test.
However, we might eventually get to a point where rustc is not currently
emitting future-incompat reports for any lints. What would we want the
cargo tests to do in this situation?
This functionality didn't require any significant internal changes to
Cargo, with one exception: we now process captured command output for
all units, not just ones where we want to display warnings. This may
result in a slightly longer time to run `cargo build/check/rustc` from
a full cache. since we do slightly more work for each upstream
dependency. Doing this seems unavoidable with the current architecture,
since we need to process captured command outputs to detect
any future-incompat-report messages that were emitted.
Add the path to the manifest in json output
This allows consumers of the json messages to avoid guessing where
exactly the package root is. Having access to the package root is
difficult by virtue of requiring logic to guess its location by e.g.
walking filesystem from the source file.
This guessing logic becomes further complicated in presence of
workspaces and nigh impossible to implement correctly in instances where
artifacts end up produced from paths above the package root (e.g.
`../foo.rs`).
Since Cargo has access to this data in the first place, there doesn't
seem to be much reason to force consumers to invent their own, possibly
flawed, logic.
This allows consumers of the json messages to avoid guessing where
exactly the package root is. Having access to the package root is
difficult by virtue of requiring logic to guess its location by e.g.
walking filesystem from the source file.
This guessing logic becomes further complicated in presence of
workspaces and nigh impossible to implement correctly in instances where
artifacts end up produced from paths above the package root (e.g.
`../foo.rs`).
Since Cargo has access to this data in the first place, there doesn't
seem to be much reason to force consumers to invent their own, possibly
flawed, logic.
Explain `cargo install` is not for libraries
On a few occasions I've seen novice users assume that `cargo install` works like `npm install` or `apt install`, and they're confused that they can't use library dependencies this way.
I've expanded the error message to hopefully clarify the misconception.
Forbid setting `RUSTC_BOOTSTRAP` from a build script on stable
Instead, recommend `RUSTC_BOOTSTRAP=crate_name`. If cargo is using a nightly toolchain, or if RUSTC_BOOTSTRAP was set in *cargo*'s build environment, the error is downgraded to a warning, since the variable won't affect the build.
This is mostly the same as suggested in https://github.com/rust-lang/cargo/issues/7088#issuecomment-713867773, except that `RUSTC_BOOTSTRAP=` values other than 1 are treated the same as `RUSTC_BOOTSTRAP=1`. My reasoning was that https://github.com/rust-lang/rust/pull/77802 is now on 1.50 stable, so some crates may have started using it, and I would still prefer not to give hard errors when there's no workaround.
Closes https://github.com/rust-lang/cargo/issues/7088.
r? `@joshtriplett`
Don't panic when printing JSON with non-utf8 paths
Before:
λ cd \Xff/foo/ && cargo verify-project && cargo metadata
{"success":"true"}
warning: please specify `--format-version` flag explicitly to avoid compatibility problems
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("path contains invalid UTF-8 characters", line: 0, column: 0)', /rustc/a5a775e3f9e8043dad405e00aee0ae60882a7b71/src/tools/cargo/src/cargo/core/shell.rs:346:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
After:
λ cd \Xff/foo/ && $cargo verify-project && $cargo metadata
{"success":"true"}
warning: please specify `--format-version` flag explicitly to avoid compatibility problems
error: path contains invalid UTF-8 characters
I am pretty sure that this has zero real-world impact, but the diff is
small, so why not handle it?
Before:
λ cd \Xff/foo/ && cargo verify-project && cargo metadata
{"success":"true"}
warning: please specify `--format-version` flag explicitly to avoid compatibility problems
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("path contains invalid UTF-8 characters", line: 0, column: 0)', /rustc/a5a775e3f9e8043dad405e00aee0ae60882a7b71/src/tools/cargo/src/cargo/core/shell.rs:346:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
After:
λ cd \Xff/foo/ && $cargo verify-project && $cargo metadata
{"success":"true"}
warning: please specify `--format-version` flag explicitly to avoid compatibility problems
error: path contains invalid UTF-8 characters
I am pretty sure that this has zero real-world impact, but the diff is
small, so why not handle it?
Minor update to registry API error messages.
This is a minor update to the registry API errors, trying to make them a little clearer and more helpful. My concerns were:
* `api errors (status 200 OK): some error message` — why was there both an error and 200 OK?
* `api errors` — What is an "api error" anyways? The user is probably not familiar with the "registry API".
* Adds the URL of the server it is actually talking to, in case you may be confused when using a registry other than crates.io. This also tries to make it clearer that the message is coming from the remote server, and not cargo itself.
Throw error if CARGO_TARGET_DIR is an empty string
This pull request makes the target dir to be target/ if `CARGO_TARGET_DIR` is `` with spaces trimmed and not delete the current project.
Fixes: #8866
testsuite: Use split debuginfo on macos.
This switches the testsuite to use "unpacked" debuginfo on macos, which is a substantial performance boost. On my system, the testsuite runs 1.55 times faster with this change. Along with #9206, total testsuite time is 3.1 times faster.
