Fix deserialization of check-cfg in config.toml
When improving the check-cfg implementation in https://github.com/rust-lang/cargo/pull/10566 I changed the internal representation of `check_cfg` from multiple `bool` options to one `Option<(bool, bool, bool, bool)>` but I didn't realize until https://github.com/rust-lang/rust/issues/82450#issuecomment-1169836928 that the internal representation is actually somewhat public as it's used in the `[unstable]` in `.cargo/config.toml`.
And because TOML cannot represent tuples there is no way to set it from the `[unstable]` section. This PR fix this oversight by using a custom deserializer method similar to what was already done for `build-std`.
Fetch GitHub commits by long hash more efficiently
Closes#10078.
**Tested with the following Cargo.toml:**
```toml
[package]
name = "repro"
version = "0.0.0"
edition = "2021"
publish = false
[dependencies]
cargo = { git = "https://github.com/rust-lang/cargo", rev = "b30694b4d9b29141298870b7993e9aee10940524" }
```
```console
$ rm -rf ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-*
$ time $CARGO generate-lockfile
$ du -shc ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-*
```
Using current cargo from the most recent nightly, the `generate-lockfile` command downloads 69704 git objects in 7.0 seconds, consuming 41 MB on disk.
Using cargo built from this PR by `cargo build --release`, the same command downloads 21481 objects in 2.2 seconds, consuming 17 MB on disk.
Once libgit2 is able to do shallow clones (https://github.com/libgit2/libgit2/issues/3058) this can be even more of a speedup. Using command-line git (which does not use libgit2) and `time git fetch --depth=1 https://github.com/rust-lang/cargo b30694b4d9b29141298870b7993e9aee10940524` indicates that it downloads just 262 objects in 1.1 seconds.
refactor(test): Clarify asserts are for UI
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
Fix tests due to change in dead_code diagnostic.
https://github.com/rust-lang/rust/pull/97853 changed some diagnostics which is causing some tests to fail on the latest nightly. This updates the tests to work on both stable and nightly.
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
remove unused dependency from benchsuite
In #10754 I added a new benchmark to the benchsuite. While figuring out the best way to add the new benchmark, I added `cargo-test-support` as a dependency. It appears I missed removing it before making the PR.
This PR removes `cargo-test-support` since it is not needed
docs(contrib): Add documentation for ui tests
### What does this PR try to resolve?
This only adds information about snapshot testing using `snapbox` and
keeps the functional testing documentation focused on the existing
facilities. We can updated this as our use of `snapbox` matures.
### How should we test and review this PR?
I did not generate and verify the HTML
In writing this, I did notice that we define
`cargo_test_support::compare::assert` but only use it for filesystem
asserts and not binary asserts. We should probably add our own function
that wraps `snapbox::cmd::Command::cargo()` and passes in
`cargo_test_support::compare::assert`. I've left that out of this PR to
keep things focused.
Add a benchmark for workspace initialization
It [was suggested](https://github.com/rust-lang/cargo/pull/10736#discussion_r894822022) that a benchmark for workspace initialization should be added. This was suggested because there were issues with the performance of [workspace inheritance](https://github.com/rust-lang/cargo/issues/10747) as well as a general way to track the workspace initialization time across cargo changes
### Changes
- Moved common functions out of `resolve.rs` to a shared `lib.rs`
- Added a new struct to be used when creating a new benchmark
- This was done because `env!("CARGO_TARGET_TMPDIR")` would fail to compile when put inside of the new `lib.rs`
- Added a new workspace test for workspace inheritance
- This new workspace does not have a repo that it was built from and if one needs to be made I can change that
Use specific terminology for sparse HTTP-based registry
Before the options is popularized, I'd like to take opportunity to give it a unique name used consistently. It's been called "http" registry, but that's a rather generic term, especially that existing git-based registry also uses HTTP as its transport.
New registry URLs use `sparse+https://` prefix, so calling it "sparse" registry seems more appropriate.
#9069
This only adds information about snapshot testing using `snapbox` and
keeps the functional testing documentation focused on the existing
facilities. We can updated this as our use of `snapbox` matures.
In writing this, I did notice that we define
`cargo_test_support::compare::assert` but only use it for filesystem
asserts and not binary asserts. We should probably add our own function
that wraps `snapbox::cmd::Command::cargo()` and passes in
`cargo_test_support::compare::assert`. I've left that out of this PR to
keep things focused.
chore: Upgrade to clap 3.2
I decided to use cargo as a test case for upgrading to clap 3.2 prior to release. From the builder API's perspective, a lot has change. While the changelog summarizes it, the release announcement is still pending. In short, the API is now typed. You declare what type an `Arg` is and access it by that type. flags (both `is_present` and `occurrences_of`) are also now specified through `ArgAction` and the result gets stored like any other arg value. I made a `ArgMatchesExt::flag` and `command_prelude::flag` functions to make working with these easier.
Now that clap exposes non-panicking variants of its functions, I switched from a "look before you leap" approach with `is_arg_valid` to a "better to ask forgiveness than permission" with checking the error variant. I decided to just make a convenience to swallow that error type. Not a fan of how loose things are but I think this will just be something we iterate on over time.
Improve testing framework for http registries
Improve integration of the http server introduced by the http-registry feature.
The same HTTP server is used for serving downloads, the index, and the API within the test framework.
This makes it easier to write tests that deal with authentication and http registries.
Most of this change is pulled from #10592. Getting it in separately should make the other change easier to review.
r? `@Eh2406`
Improve integration of the http server introduced by the http-registry feature.
Now the same HTTP server is used for serving downloads, the index, and
the API.
This makes it easier to write tests that deal with authentication and
http registries.
Make -Z http-registry use index.crates.io when accessing crates-io
Use `sparse+https://index.crates.io/` to access crates.io when `-Z http-registry` is set.
* `Cargo.lock` files still emit the github URL `https://github.com/rust-lang/crates.io-index`.
* Allows publishing to a source-replaced `crates-io` only for `index.crates.io`. Other source-replacements of `crates-io` are still blocked.
Fixes#10722
r? `@Eh2406`
Respect submodule update=none strategy in .gitmodules
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.
If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:
1. *foo* is a big git repo
```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 big
```
2. *bar* is a repo with a big submodule with `update=none`
```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
path = foo
url = file:///tmp/foo
update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
```
3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped
```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```
Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.
4. *bar* is now a lib with a big submodule with update disabled
```
/tmp $ cargo init --lib bar
Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
3 files changed, 18 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 src/lib.rs
/tmp $
```
5. *qux* depends on *bar*, notice *bar*'s submodules are fetched
```
/tmp $ cargo init qux && cd qux
Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
Updating git repository `file:///tmp/bar`
Updating git submodule `file:///tmp/foo`
real 0m22.182s
user 0m20.402s
sys 0m1.714s
/tmp/qux $
```
Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.
6. With the change applied, submodules with `update=none` are skipped
```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
Updating git repository `file:///tmp/bar`
Skipping git submodule `file:///tmp/foo`
real 0m0.029s
user 0m0.021s
sys 0m0.008s
/tmp/qux $
```
Fixes#4247.