Fix publishing to crates.io with -Z sparse-registry
Attempting to publish a crate to crates.io with `-Z sparse-registry` failed with the following error:
```
error: failed to publish to registry at https://crates.io
Caused by:
the remote server responded with an error: Dependency `serde` is hosted on another registry. Cross-registry dependencies are not permitted on crates.io.
```
The check in `registry.rs` `dep_registry_id != registry_id` caused the `publish` operation include the crates.io index url in the HTTP request because the id was replaced. The crates.io API seems to require that the `registry` field is not present.
This change fixes the issue by making the `registry` function return the non-replaced crates.io `source_id` only for this case. Other replacement indices of crates.io continue to include the registry URL when publishing.
Tested manually by publishing `arlosi-cargo-test` to crates.io with `-Z sparse-registry`
Fixes#10828
r? `@Eh2406`
Make `is_yanked` return `Poll<>`
The `is_yanked` check performed by `cargo install` and `cargo package` was running sequentially (calling `block_until_ready` after every check).
This change makes `is_yanked` return `Poll<>` and runs the check in parallel, which gives better performance for `cargo install --locked` and `cargo package` when using a sparse registry.
fixes#10821
r? `@ehuss`
Fix corrupted git checkout recovery.
This fixes an issue where cargo would not recover from a corrupted git checkout correctly when using `net.git-fetch-with-cli`.
Git dependencies have two clones, the "db" and the "checkout". The "db" is shared amongst multiple checkout revisions from the same repository. The "checkout" is each individual revision. There was some code in `copy_to` which creates the "checkout" that tries to recover from an error. The "checkout" can be invalid if cargo was interrupted while cloning it, or if there is fs corruption. However, that code was failing when using the git CLI. For reasons I did not dig into, the "db" does not have a HEAD ref, so that special-case fetch was failing with a `couldn't find remote ref HEAD` error from `git`.
This changes it so that if the "checkout" is invalid, it just gets blown away and a new clone is created (instead of calling `git fetch` from the "db").
I believe there is some long history for this `copy_to` code where it made more sense in the past. Previously, the "checkout" directories used the `GitReference` string as-is. So, for example, a branch would checkout into a directory with that branch name. At some point, it was changed so that each checkout uses a short hash of the actual revision. Rebuilding the checkout made sense when it was possible for that checkout revision to change (like if new commits were pushed to a branch). That recovery is no longer necessary since a checkout is only ever one revision.
Fixes#10826
add a cache for discovered workspace roots
## History
`@ehuss` [noticed that](https://github.com/rust-lang/cargo/pull/10736#discussion_r894071933) workspace inheritance caused a significant increase in startup times when using workspace inheritance. This brought up the creation of #10747.
When using a similar test setup [to the original](https://github.com/rust-lang/cargo/pull/10736#discussion_r894822022) I got
```
Benchmark 1: cd rust; ../../../target/release/cargo metadata
Time (mean ± σ): 149.4 ms ± 3.8 ms [User: 105.9 ms, System: 31.7 ms]
Range (min … max): 144.2 ms … 162.2 ms 19 runs
Benchmark 2: cd rust-ws-inherit; ../../../target/release/cargo metadata
Time (mean ± σ): 191.6 ms ± 1.4 ms [User: 145.9 ms, System: 34.2 ms]
Range (min … max): 188.8 ms … 193.9 ms 15 runs
```
This showed a large increase in time per cargo command when using workspace inheritance.
During the investigation of this issue, other [performance concerns were found and addressed](https://github.com/rust-lang/cargo/pull/10761). This resulted in a drop in time across the board but heavily favored workspace inheritance.
```
Benchmark 1: cd rust; ../../../target/release/cargo metadata
Time (mean ± σ): 139.3 ms ± 1.7 ms [User: 99.8 ms, System: 29.4 ms]
Range (min … max): 137.1 ms … 144.5 ms 20 runs
Benchmark 2: cd rust-ws-inherit; ../../../target/release/cargo metadata
Time (mean ± σ): 161.7 ms ± 1.4 ms [User: 120.4 ms, System: 31.2 ms]
Range (min … max): 158.0 ms … 164.6 ms 18 runs
```
## Performance after changes
`hyperfine --warmup 10 "cd rust; ../../../target/release/cargo metadata" "cd rust-ws-inherit; ../../../target/release/cargo metadata" --runs 40`
```
Benchmark 1: cd rust; ../../../target/release/cargo metadata
Time (mean ± σ): 140.1 ms ± 1.5 ms [User: 99.5 ms, System: 30.7 ms]
Range (min … max): 137.4 ms … 144.0 ms 40 runs
Benchmark 2: cd rust-ws-inherit; ../../../target/release/cargo metadata
Time (mean ± σ): 141.8 ms ± 1.6 ms [User: 100.9 ms, System: 30.9 ms]
Range (min … max): 138.4 ms … 145.4 ms 40 runs
```
[New Benchmark](https://github.com/rust-lang/cargo/pull/10754)
`cargo bench -- workspace_initialization/rust`
```
workspace_initialization/rust
time: [14.779 ms 14.880 ms 14.997 ms]
workspace_initialization/rust-ws-inherit
time: [16.235 ms 16.293 ms 16.359 ms]
```
## Changes Made
- [Pulled a commit](bbd41a4dca) from `@ehuss` that deduplicated finding a workspace root to make the changes easier
- Added a cache in `Config` to hold found `WorkspaceRootConfig`s
- This makes it so manifests should only be parsed once
- Made `WorkspaceRootConfig` get added to the cache when parsing a manifest
## Testing Steps
To check the new benchmark:
1. `cd benches/benchsuite`
2. `cargo bench -- workspace_initialization/rust`
Using `hyperfine`:
1. run `cargo build --release`
2. extract `rust` and `rust-ws-inherit` in `benches/workspaces`
3. cd `benches/workspaces`
4. Prime the target directory with a cache of `rustc` info. In `rust` and `rust-ws-inherit`, run: `cargo +nightly c -p linkchecker`. Otherwise it would be measuring `rustc` overhead.
4. run `hyperfine --warmup 10 "cd rust; ../../../target/release/cargo metadata" "cd rust-ws-inherit; ../../../target/release/cargo metadata" --runs 40`
closes#10747
fix(add): Don't panic with `--offline`
For some reason, I defined my own `--offline` flag and it didn't get
updated with the global `--offline` flag, so it started failing.
The new test previously paniced and now it doesn't.
Fixes#10814
Fix zsh completions for add and locate-project
Currently, trying to tab-complete either the `cargo add` or `cargo locate-project` subcommands results in output such as this:
```
cargo add _cargo:67: command not found: --default-features[enable the default features]
_cargo:68: command not found: --no-default-features[don't enable the default features]
_cargo:69: command not found: --optional[mark the dependency as optional]
```
This is because some line continuations are missing from `src/etc/_cargo`. This PR adds these line continuations.
This file gets packaged for Rust releases by acdcdfb61b/src/bootstrap/dist.rs (L956) and Rustup ultimately places it in `~/.rustup/toolchains/*/share/zsh/site-functions/_cargo`. `rustup completions zsh cargo` outputs a script which sources this filepath.
The easier approach to testing this is probably to manually copy `_cargo` to its proper location under `~/.rustup`, assuming Rust is already installed via Rustup and Zsh completions for Cargo are already installed, but however you choose to install this, testing is as simple as observing that Zsh can correctly tab complete `cargo add` and `cargo locate-project`.
The zsh tab-completion script had missing line continuations which would
cause `command not found:` errors when trying to complete the
`cargo add` or `cargo locate-project` subcommands.
Bump cargo-util version.
#10546 made a semver-incompatible change to the API of `ProcessBuilder::get_args`. Unfortunately we did not catch that until it was published. This bumps the version of cargo-util to 0.2.1 to accommodate that change. Stable will get version 0.2.0 so that the changes on beta can be released as 0.2.1 in their own time.
cc #10803
Fix deserialization of check-cfg in config.toml
When improving the check-cfg implementation in https://github.com/rust-lang/cargo/pull/10566 I changed the internal representation of `check_cfg` from multiple `bool` options to one `Option<(bool, bool, bool, bool)>` but I didn't realize until https://github.com/rust-lang/rust/issues/82450#issuecomment-1169836928 that the internal representation is actually somewhat public as it's used in the `[unstable]` in `.cargo/config.toml`.
And because TOML cannot represent tuples there is no way to set it from the `[unstable]` section. This PR fix this oversight by using a custom deserializer method similar to what was already done for `build-std`.
Fetch GitHub commits by long hash more efficiently
Closes#10078.
**Tested with the following Cargo.toml:**
```toml
[package]
name = "repro"
version = "0.0.0"
edition = "2021"
publish = false
[dependencies]
cargo = { git = "https://github.com/rust-lang/cargo", rev = "b30694b4d9b29141298870b7993e9aee10940524" }
```
```console
$ rm -rf ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-*
$ time $CARGO generate-lockfile
$ du -shc ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-*
```
Using current cargo from the most recent nightly, the `generate-lockfile` command downloads 69704 git objects in 7.0 seconds, consuming 41 MB on disk.
Using cargo built from this PR by `cargo build --release`, the same command downloads 21481 objects in 2.2 seconds, consuming 17 MB on disk.
Once libgit2 is able to do shallow clones (https://github.com/libgit2/libgit2/issues/3058) this can be even more of a speedup. Using command-line git (which does not use libgit2) and `time git fetch --depth=1 https://github.com/rust-lang/cargo b30694b4d9b29141298870b7993e9aee10940524` indicates that it downloads just 262 objects in 1.1 seconds.
refactor(test): Clarify asserts are for UI
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
Fix tests due to change in dead_code diagnostic.
https://github.com/rust-lang/rust/pull/97853 changed some diagnostics which is causing some tests to fail on the latest nightly. This updates the tests to work on both stable and nightly.
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
remove unused dependency from benchsuite
In #10754 I added a new benchmark to the benchsuite. While figuring out the best way to add the new benchmark, I added `cargo-test-support` as a dependency. It appears I missed removing it before making the PR.
This PR removes `cargo-test-support` since it is not needed
docs(contrib): Add documentation for ui tests
### What does this PR try to resolve?
This only adds information about snapshot testing using `snapbox` and
keeps the functional testing documentation focused on the existing
facilities. We can updated this as our use of `snapbox` matures.
### How should we test and review this PR?
I did not generate and verify the HTML
In writing this, I did notice that we define
`cargo_test_support::compare::assert` but only use it for filesystem
asserts and not binary asserts. We should probably add our own function
that wraps `snapbox::cmd::Command::cargo()` and passes in
`cargo_test_support::compare::assert`. I've left that out of this PR to
keep things focused.
