Improve testing framework for http registries
Improve integration of the http server introduced by the http-registry feature.
The same HTTP server is used for serving downloads, the index, and the API within the test framework.
This makes it easier to write tests that deal with authentication and http registries.
Most of this change is pulled from #10592. Getting it in separately should make the other change easier to review.
r? `@Eh2406`
Improve integration of the http server introduced by the http-registry feature.
Now the same HTTP server is used for serving downloads, the index, and
the API.
This makes it easier to write tests that deal with authentication and
http registries.
Make -Z http-registry use index.crates.io when accessing crates-io
Use `sparse+https://index.crates.io/` to access crates.io when `-Z http-registry` is set.
* `Cargo.lock` files still emit the github URL `https://github.com/rust-lang/crates.io-index`.
* Allows publishing to a source-replaced `crates-io` only for `index.crates.io`. Other source-replacements of `crates-io` are still blocked.
Fixes#10722
r? `@Eh2406`
Respect submodule update=none strategy in .gitmodules
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.
If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:
1. *foo* is a big git repo
```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 big
```
2. *bar* is a repo with a big submodule with `update=none`
```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
path = foo
url = file:///tmp/foo
update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
```
3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped
```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```
Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.
4. *bar* is now a lib with a big submodule with update disabled
```
/tmp $ cargo init --lib bar
Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
3 files changed, 18 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 src/lib.rs
/tmp $
```
5. *qux* depends on *bar*, notice *bar*'s submodules are fetched
```
/tmp $ cargo init qux && cd qux
Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
Updating git repository `file:///tmp/bar`
Updating git submodule `file:///tmp/foo`
real 0m22.182s
user 0m20.402s
sys 0m1.714s
/tmp/qux $
```
Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.
6. With the change applied, submodules with `update=none` are skipped
```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
Updating git repository `file:///tmp/bar`
Skipping git submodule `file:///tmp/foo`
real 0m0.029s
user 0m0.021s
sys 0m0.008s
/tmp/qux $
```
Fixes#4247.
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.
If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:
1. *foo* is a big git repo
```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 big
```
2. *bar* is a repo with a big submodule with `update=none`
```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
path = foo
url = file:///tmp/foo
update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
```
3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped
```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```
Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.
4. *bar* is now a lib with a big submodule with update disabled
```
/tmp $ cargo init --lib bar
Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
3 files changed, 18 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 src/lib.rs
/tmp $
```
5. *qux* depends on *bar*, notice *bar*'s submodules are fetched
```
/tmp $ cargo init qux && cd qux
Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
Updating git repository `file:///tmp/bar`
Updating git submodule `file:///tmp/foo`
real 0m22.182s
user 0m20.402s
sys 0m1.714s
/tmp/qux $
```
Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.
6. With the change applied, submodules with `update=none` are skipped
```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
Updating git repository `file:///tmp/bar`
Skipping git submodule `file:///tmp/foo`
real 0m0.029s
user 0m0.021s
sys 0m0.008s
/tmp/qux $
```
Fixes#4247.
Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
Expose rust-version through env var
This adds another env var that is exposed by cargo. In Clippy we would like to use that in order to efficiently check if a rust-version is set for the current package: https://github.com/rust-lang/rust-clippy/pull/8774
Currently we either have to parse the `Cargo.toml` file ourselves or use the `cargo_metadata` crate which has a notable performance impact when running `clippy-driver` on single files.
add validation for string "true"/"false" in lto profile
### What does this PR try to resolve?
Adds a special-cased error message for when `lto` is set to the _string_ `"true"`/`"false"` which is surprisingly (I was surprised anyway) not allowed and the error message is ambiguous. The new error message makes it clear what values are accepted.
Fixes https://github.com/rust-lang/cargo/issues/10572
### How should we test and review this PR?
<!-- Demonstrate how you test this change and guide reviewers through your PR.
With a smooth review process, a pull request usually gets reviewed quicker.
If you don't know how to write and run your tests, please read the guide:
https://doc.crates.io/contrib/tests -->
Uh I've not actually tested yet that's the WIP part. But put
```
[profile.dev]
lto="false"
```
in your TOML and run `cargo build`, check that you get the new error message and that it makes sense and is helpful.
### Additional information
It's worth noting that as per https://github.com/rust-lang/rust/pull/97051 this doesn't fix the _real_ problem here IMO which is that [rust's `opt_parse_bool` cli parsing](491f619f56/compiler/rustc_session/src/options.rs (L456)) doesn't accept true/false which certainly seems an ad-hoc historical choice to me on first glance but also it's a much bigger change to change those semantics than this error message.
- Rephrase doctest exec model as "not guranteed and may change" instead
- Mention `#[bench]` in what cargo-bench automatically runs
- Make it clear for build/rustc when mentioning bin targets auto-built
- Mention that in `src/` Cargo also collect doc tests.
- Remove outdated statement: Cargo no longer tests examples by default.
- Add a link to "Cargo Targets: Tests" to help people learn about it.
Clear disk space on CI.
Cargo's testsuite uses a considerable amount of disk space. On windows-gnu, the target directory can get over 12GB, and there is only 13GB free. We're starting to run out of disk space, so this is a stop-gap that clears out the test data before running the smoke test which uses a fair bit of space itself.
We will probably need to think about addressing #9701 somehow, otherwise we will start running out of space as we add more tests. See the linked issues in https://github.com/rust-lang/cargo/pull/9701#issuecomment-881765517 for additional context.
Guide new users to add use `super::*;` to `mod test`
### What does this PR try to resolve?
Currently, `cargo init --lib` produces examples for unit tests. However, new users will find that they are unable to use functions they defined above. This should resolve the confusion.
Fixes#10559
### How should we test and review this PR?
This PR does not create new features. Test this PR using the already-existing tests.
### Additional information
I didn't think this was a major change, so I did not open a RFC for it. Please let me know if I should have!
Document how to debug change detection events
### What does this PR try to resolve?
I noticed that my build would sometimes seemingly randomly rebuild other crates. I figured this must be the build script detecting a change in some external files. In order to debug this, I figured I'd look at the Cargo sources whether something like this was already being logged. Thankfully, the logging for this was already in place but I didn't find it documented anyway so I thought it might be rather helpful in such scenarios.
I believe it's a common enough scenario that inclusion into the official documentation on this topic should be considered.
### How should we test and review this PR?
Build/view documentation.
### Additional information
fix(publish): add more check when use `publish -p <SPEC>`
### Main issue
As issue say #10536 , we need add more check when user use `cargo publish -p <SPEC>`
>`@ehuss` point outs:
>From a behavior standpoint, here are some things to check:
> - In the root of a virtual workspace, it should be an error to run without -p.
>- It should be an error to pass -p for a non-workspace member.
>- It should be an error for -p to match multiple packages.
>- When using -p, it should publish that package, not the one in the current directory (which can be different).
fix key formatting when switching to a dotted `WorkspaceSource`
This fell out of #10697 see [this comment](https://github.com/rust-lang/cargo/pull/10697#discussion_r882691624)
There was a small issue where changing the source of a `cargo_add::Dependency` to a `WorkspaceSource` would cause the dotted version to have extra space.
```toml
dep .workspace = true
dep.workspace = true
```
This makes sure the key is formatted as well as adds a unit test to make sure this doesn't come back up in the future.
r? `@epage`
doc: discuss build script instruction order
### What does this PR try to resolve?
It is currently not documented that the order of build script `cargo:` instructions may be relevant to linking.
This has caused issues such as: https://github.com/rust-lang/rust/issues/96328
### How should we test and review this PR?
Build/view documentation.
### Additional information
- Cargo maintainers should fact check my wording.
- We may need to discuss if this should also be documented for `rustc`
- Maintainers should ensure that this change does not prevent a change in what is currently unspecified behavior. Perhaps `cargo` will want to rearrange link arguments itself to resolve issues in the future?
Require http-registry URLs to end with a '/'
The `url` crate normalizes URLs with no path component to end in a trailing slash. This causes the current implementation to use urls containing two slashes for registries without a path component (such as `https://index.crates.io//config.json`).
This PR resolves the issue by requiring http registry URLs to end in a slash and generating paths by concatenating. A new error message is emitted for http registry URLs that do not end in a slash.
r? `@Eh2406`
No printing executable names when running tests and benchmarks with json message format
Fixes: #10684
I added a new test for this, though I'm not sure if it's necessary. Let me know if I should delete it.
