Extend the existing CARGO_BIN_NAME environment variable to be set when
building binary example targets, additional to "normal" binary targets.
Closes#11689.
Deny warnings in CI, not locally
### What does this PR try to resolve?
The problem with #![deny(warnings)] is it makes iteration more difficult as you might have intermediate states with warnings. Its slightly better that we defer this to cargo test --lib but that still means you can't run a subset of tests against your experiment until you've cleaned up all of your warnings. This can lead to users [working around the problem which could accidentally slip in](d92d04840c).
### How should we test and review this PR?
The first round for this PR includes a warning in the `cargo` crate to ensure it breaks CI. It will then be reverted.
### Additional information
See also https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/.60.23!.5Bcfg_attr.28test.2C.20deny.28warnings.29.29.5D.60
Add '-C' flag for changing current dir before build
This implements the suggestion in #10098 to make cargo change cwd before
completing config processing and starting the build. It is also an
alternative to `--manifest-path` that resolves the issue described
in #2930.
The behavior of this new flag makes cargo build function exactly the same when run at the root of the project as if run elsewhere outside of the project. This is in contrast to `--manifest-path`, which, for example, results in a different series of directories being searched for `.cargo/config.toml` between the two cases.
Fixes#10098
Reduces impact of #2930 for many, possibly all impacted, by switching to this new cli argument.
### How should we test and review this PR?
The easiest way to reproduce the issue described in #2930 is to create an invalid `.cargo/config.toml` file in the root of a cargo project, for example `!` as the contents of the file. Running cargo with the current working directory underneath the root of that project will quickly fail with an error, showing that the config file was processed. This is correct and expected behavior.
Running the the same build with the current working directory set outside of the project, e.g. /tmp, and passing the `--manifest-path /path/to/project/Cargo.toml`. The build will proceed without erroring as a result of reading the project's `.cargo/config.toml`, because config file searching is done from cwd (e.g. in `/tmp` and `/`) without including the project root, which is a surprising result in the context of the [cargo config documentation](https://doc.rust-lang.org/cargo/reference/config.html), which suggests that a `.cargo/config.toml` file checked into the root of a project's revision control will be processed during the build of that project.
Finally to demonstrate that this PR results in the expected behavior, run cargo similar to the previous run, from /tmp or similar, but instead of `--manifest-path /path/to/project/Cargo.toml`, pass `-C/path/to/project` to cargo (note the missing `Cargo.toml` at the end). The build will provide the exact same (expected error) behavior as when running it within the project root directory.
### Additional information
~Passing a path with a trailing '/' will result in failure. It is unclear whether this is a result of improper input sanitization, or whether the config.cwd value is being improperly handled on output. In either case this needs to be resolved before this PR is merge-ready.~
(the above issue appears to have been a side effect of local corruption of my rustup toolchain, and unrelated to this change)
Because a `struct Config` gets created before command line arguments are processed, a config will exist with the actual cwd recorded, and it must then be replaced with the new value after command line arguments are processed but before anything tries to use the stored cwd value or any other value derived from it for anything. This change effectively creates a difficult-to-document requirement during cargo initialization regarding the order of events. For example, should a setting stored in a config file discovered via cwd+ancestors search be wanted before argument processing happens, this could result in unpleasant surprises in the exact use case this feature is being added to fix.
A long flag was deferred out to not block this on deciding what to name it. A follow up issue will be created.
This implements the suggestion in #10098 to make cargo change cwd before
completing config processing and starting the build. It is also an
alternative to --manifest-path that resolves the issue described
in #2930.
When running `cargo doc -Zrustdoc-scrape-example`, it performs
additional type checks that a plain old `cargo doc` doesn't.
That leads to some extra failure when adopting scrape-example for docs.rs.
In de34d60 we introduced `unit_can_fail_for_docscraping` in order to
make `-Zrustdoc-scrape-example` not fail whenever `cargo doc` builds.
Build script executions were accidentally included in the list of
fallible units. A plain `cargo doc` does fail when a build script
fails. `-Zrustdoc-scrape-example` should follow that.
Any directory entry ending with `.rs`, including directories, were
previously assumed to be files, and could end up as targets.
Now only regular files and symbolic links are inferred.
Turn off debuginfo for build dependencies v2
This PR is an alternative to #10493, fixing its most important issue: the unit graph optimization to reuse already built artifacts for dependencies shared between the build-time and runtime subgraphs is preserved (most of the time).
By deferring the default debuginfo level in `dev.build-override` until unit graph sharing, we check whether re-use would happen. If so, we use the debuginfo level to ensure reuse does happen. Otherwise, we can avoid emitting debuginfo and improve compile times (on clean, debug and check builds -- although reuse only happens on debug builds).
I've kept the message explaining how to bump the debuginfo level if an error occurs while running a build script (and backtraces were requested) that was in the previous PR.
I've ran benchmarks on 800 crates, at different parallelism levels, and published the (surprisingly good) results with visualizations, summaries, and raw data [here](https://github.com/lqd/rustc-benchmarking-data/tree/main/experiments/cargo-build-defaults).
Opening this PR as discussed in [Eric's message](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Defaults.20for.20faster.20compilation.20of.20.22for.20host.22.20units/near/304236576l) as draft since 2 tests won't pass. That fixes the `cargo-crev` case we saw as a blocker last time, but doesn't fix all such cases of reuse, like the 2 failing tests:
- [`optional_build_dep_and_required_normal_dep`](642a0e625d/tests/testsuite/build_script.rs (L4449))
- and [`proc_macro_ws`](bd5db301b0/tests/testsuite/features2.rs (L1051))
These failures make sense, since the debuginfo optimization is done during traversal, it depends on the contents of the unit graph. These tests ensure that sharing happens even on finer-grained invocations: respectively, with an optional shared dependency that is later enabled, and building shared dependencies by themselves first and later as part of a complete workspace build.
In both these situations, there is no unit that is shared in the graph during the first invocation, so we do the optimization and remove the debuginfo. When the graph changes in the following invocation, sharing is present and we have to build the shared units again with debuginfo.
These cases feel rarer than `cargo-crev`'s case, but I do wonder if there's a way to fix them, or if it's acceptable to not support them.
This test dynamically enables a shared build/runtime dependency, and
therefore doesn't trigger the build/runtime sharing reuse optimization,
as the build dep is initially built without debuginfo for optimization
purposes.
Add some assertions to ensure that debuginfo is not used to compile
build dependencies, in a way that differs between the old and new
defaults: some of the assert elision could match the previous defaults
with debuginfo. These new assertions break if `-C debuginfo` is present
in the commands cargo ran.
Make cargo install report needed features
### What does this PR try to resolve?
The problem described in issue #11617 where cargo tells the user, that no binaries are available
because of the chosen features, but does not tell which features could be enabled to fix the
situation.
Fixes#11617.
### How should we test and review this PR?
Try to cargo install a crate, where all binaries need some features enabled, without enabling
the features and check the error message.
The test suite already contains tests for this.
This enum will be used to model the current Option<u32> value in
profiles, as the explicitly set value, and also allow to model a
deferred value: one that can be ignored for optimization purposes,
and used in all other cases.
This allows to have a default debuginfo for build dependencies, that can
change:
- if a dependency is shared between the build and runtime subgraphs, the
default value will be used, to that re-use between units will
continue.
- otherwise, a build dependency is only used in a context where
debuginfo is not very useful. We can optimize build times in this
situation by not asking rustc to emit debuginfo.
Make cargo aware of dwp files.
When using -Csplit-debuginfo=packed on Linux, rustc will produce a dwp file. Unlike the dwo files, whose paths are embedded into the binary, there's no information in the binary to help a debugger locate a dwp file. By convention, the dwp file for `<EXE>` is given the name `<EXE>.dwp` and placed next to `<EXE>`.
When cargo hardlinks the executable file rustc put in target/debug/deps into target/debug, it also needs to hardlink the dwp file along with it. Failing to do this prevents the debugger from finding the dwp file when the binary is executed from target/debug, as there's no way for the debugger to know to look in the deps subdirectory.
The split_debuginfo option is passed down into file_types to make this possible. For cargo clean manual handling is added to match the other split_debuginfo files. bin_link_for_target also passes in None because it won't care about the dwp file.
refactor(toml): Move `TomlWorkspaceDependency` out of `TomlDependency`
**This should not be merged until after #11409**
In #11523 it was noted that you could use `{}.workspace = true` in `[patch.{}]`, but it would cause a panic. This panic was caused by an oversight on my part when implementing [workspace inheritance](https://github.com/rust-lang/rfcs/blob/master/text/2906-cargo-workspace-deduplicate.md). Before this PR any field that had the type `TomlDependency` could specify `{}.workspace = true` and `cargo` would allow it to be a `TomlWorkspaceDependency`. While it could be `TomlWorkspaceDependency` it would never be resolved since only:
> Dependencies in the `[dependencies]`, `[dev-dependencies]`, [`build-dependencies]`, and `[target."...".dependencies]` sections will support the ability to reference the `[workspace.dependencies]` definition of dependencies.[^1]
This PR makes it so that only those fields can pull from `[workspace.dependencies]`, while still sharing `TomlDependency` everywhere it is needed. It does this by making `MaybeWorkspace` generic over both `Defined` and `Workspace`, then moving `TomlWorkspaceDependency` out of `TomlDependency` and into a `MaybeWorkspace` that the correct fields can use.
[^1]: [rfc2906-new-dependency-directives](https://github.com/rust-lang/rfcs/blob/master/text/2906-cargo-workspace-deduplicate.md#new-dependency-directives)
Closes: #11523
`cargo add` check `[dependencies]` order without considering the dotted item
### What does this PR try to resolve?
Try to close https://github.com/rust-lang/cargo/issues/11584
`cargo check` check `[dependencies]` order without considering the dotted item.
### How should we test and review this PR?
See the unit test.
Improve CI caching by skipping mtime checks for paths in $CARGO_HOME
Skip mtime checks for paths pointing into `$CARGO_HOME` to avoid rebuilds when only caching $CARGO_HOME/registry/{index, cache} and $CARGO_HOME/git/db and some of the dependencies have `rerun-if-changed=directory` in their `build.rs`
I considered skipping mtime checking only on `$CARGO_HOME/registry/src` but it looks like other functionality (like downloading a newer version of dependency) is unaffected by this and this way we also cover the same issue with git based dependencies (except the part where `cargo` is forced to re-fetch submodules if `$CARGO_HOME/git/checkouts` is missing) and it is more in line with the discussion in #9455Fix#11083
Credit `@weihanglo` for the test (I did add a case of checking that dependency update still triggers a rebuild but they are the original author of the rest of the test)
feat: stabilize auto fix note
A note that some warnings could be fixed by running a `cargo fix` command was added in #10989 and made to work with `clippy` in #11399. It has only been turned on for `nightly` builds so far; this PR would make it show on `stable`.
The original motivation for making this note `nightly` only, was to [allow for iteration](https://github.com/rust-lang/cargo/issues/10976#issuecomment-1289297978) on the message output. There has yet to be any feedback on the message format in the time that it has been on `nightly`. This was brought up in a recent cargo team meeting and it was thought that we should move forward with showing this on `stable`.
close#10976
fix(toml): Add `default-features` to `TomlWorkspaceDependency`
In #11329 it was noted that `default-features` is ignored when used in a dependency that inherits from a workspace i.e.
```toml
[workspace]
members = []
[workspace.dependencies]
dep = "0.1"
[package]
name = "bar"
version = "0.2.0"
authors = []
[dependencies]
dep = { workspace = true, default-features = false }
```
This problem is caused by problems with deserializing a `TomlDependency` and not emitting an unused manifest key correctly. When discussed in a recent Cargo team meeting we felt the best course of action was to allow `default-features = false` when inheriting a dependency, but it does not change actually set `default-features`. It will be used to warn when there is a difference between the definition in `[workspace.dependencies]` and `[dependencies]` i.e.
```toml
[package]
name = "bar"
version = "0.2.0"
[dependencies]
dep = { workspace = true, default-features = false }
[workspace]
members = []
[workspace.dependencies]
dep = { version = "0.1", default-features = true }
```
This does not entirely resolve the problem with unused manifest keys. A follow up PR and issue will be created to address those concerns.
close#11329
Stabilize sparse-registry
### What does this PR try to resolve?
Stabilize `sparse-registry`
* Does not change the default protocol for accessing crates.io
### How should we test and review this PR?
Set environment variable `REGISTRIES_CRATES_IO_PROTOCOL=sparse` or set in `config.toml`:
```
[registries.crates-io]
protocol = 'sparse'
```
cc #9069
Ignore `workspace.default-members` when running `cargo install` on root package of a non-virtual workspace
### What does this PR try to resolve?
* Fixes#11058
Two observable behaviors are fixed:
1. When running `cargo install` with `--path` or `--git` and specifically requesting the root package of a non-virtual workspace, `cargo install` will accidentally build `workspace.default-members` instead of the requested root package.
2. Further, if that `default-members` does not include the root package, it will install binaries from those other packages (the `default-members`) and claim they were the binaries from the root package! There is no way, actually, to install the root package binaries.
These two behaviors have the same root cause:
* `cargo install` effectively does `cargo build --release` in the requested package directory, but when this is the root of a non-virtual workspace, that builds `default-members` instead of the requested package.
### How should we test and review this PR?
I have included a test exhibiting this behavior. It currently fails in the manner indicated in the test, and passes with the changes included in this PR.
I'm not sure the solution in the PR is the _best_ solution, but the alternative I am able to come up with involves much more extensive changes to how `cargo install` works, to produce a distinct `CompileOptions` for every package built. I tried to keep the new workspace "API" `ignore_default_members()` as narrowly-scoped in its effect as possible.
### Additional information
The only way I could think this behavior change could impact someone is if they were somehow using `cargo install --path` (or `--git`) and wanting it to actually install the binaries from all of `default-members`. However, I don't believe that's possible, since if there are multiple packages with binaries, I believe cargo requires the packages to be specified. So someone would have to be additionally relying on specifying just the root package, but then wanting the binaries from more than just the root. I think this is probably an acceptable risk for merging!
Add network container tests
This adds some tests which use Docker containers to provide HTTPS and SSH servers. This should help with validating that Cargo's networking and security are working correctly. It can also potentially be used in the future for other tests that require more complex setups.
These tests are only run on Linux in CI. macOS does not have Docker there, and the Windows Docker does not support Linux containers. The tests should work on macOS if you run them locally with Docker Desktop installed. The SSH tests do not work on Windows due to issues with ssh-agent, but the HTTPS tests should work with Docker Desktop.
These tests require an opt-in environment variable to run:
* `CARGO_PUBLIC_NETWORK_TESTS=1` — This is for tests that contact the public internet.
* `CARGO_CONTAINER_TESTS=1` — This is for tests that use Docker.
Show progress of crates.io index update even `net.git-fetch-with-cli` option enabled
### What does this PR try to resolve?
This PR fixes#11574 .
### How should we test and review this PR?
Please run `cargo` with `net.git-fetch-with-cli` option enabled.
It should show `git fetch`'s progress output like below.
```
❯ CARGO_NET_GIT_FETCH_WITH_CLI=true ./target/debug/cargo b
Updating crates.io index
remote: Enumerating objects: 139821, done.
remote: Counting objects: 100% (2226/2226), done.
remote: Compressing objects: 100% (769/769), done.
receiving objects: 0% (1/139821)
```
Previous, `cargo metadata` allows a dependency with different renamed
co-exist. However, its `resolve.nodes.deps` will miss that dependency,
which is wrong. After this commit, `cargo metadata starts erroring out
for that situation.
