feat(publish): Support 'publish.timeout' config behind '-Zpublish-timeout'
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (https://github.com/rust-lang/cargo/issues/1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
This introduces unstable support for blocking by setting
`publish.timeout` to non-zero value.
A step towards #9507
Add missing edition
### What does this PR try to resolve?
When I read this doc, I found that we are missing the edition key in this manifest. I think we should add it because `cargo new` always does this.
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (#1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
This introduces unstable support for blocking by setting
`publish.timeout` to non-zero value.
A step towards #9507
refactor(publish): Clarify which SourceId is being used
Ran into confusion on this when updating #11062 to be on top of #10907. This now threads both types of sources through which comments explaining each so callers will always have to explicitly acknowledge which they are dealing with.
Fix deadlock when build scripts are waiting for input on stdin
### What does this PR try to resolve?
#10738 introduced a regression where build scripts did not close stdin, causing deadlocks for build scripts waiting for stdin.
Fixes#11196 by not piping `stdin` unless requested by the `ProcessBuilder`.
### How should we test and review this PR?
Run a build script that reads from stdin and check that it no longer hangs. See the POC in the linked issue. The included test hangs without the fix.
r? `@ehuss`
Fix rustdoc warning about unclosed HTML tag
nightly rustdoc is warning about `<name>` being an unclosed HTML tag. Fix by using backticks instead of single quotes.
Ran into confusion on this when updating #11062 to be on top of #10907.
This now threads both types of sources through which comments explaining
each so callers will always have to explicitly acknowledge which they
are dealing with.
refactor(tests): Prepare for wait-for-publish test changes
In #11062, we are updating `cargo publish` to wait until a package is published. The problem is a lot of our tests will block until the timeout. In finding the tests to update, I was originally relying on test failures from the extra output when timing out. The problem is not all tests verify the test output so they don't fail.
This tries to update the tests to make the introduction of a timeout more obvious.
- Adding `with_stderr` where it wasn't before
- Moving away from `with_stderr_contains` for publish tests
To help with that, I made the predicates on cargo commands more consistent.
I also moved descriptions of tests to be outside of the test so I can more easily document the `registry::init` calls with what we are doing.
Add configuration option for controlling crates.io protocol
### What does this PR try to resolve?
Currently, if `-Z sparse-registry` is passed, then Cargo will access crates.io using the sparse protocol. Since we want to stabilize this feature soon, we need a stable way to control which protocol is used.
This adds a config option `registries.crates-io.protocol` that can be set to either `sparse` or `git`. If the option is unset, it will be `sparse` if `-Z sparse-registry` is enabled, otherwise it will be `git`.
This PR does not stabilize `sparse-registry`. Using `registries.crates-io.protocol` without `-Z sparse-registry` will result in an error.
The next steps after PR are to:
* stabilize `sparse-registry`
* make `sparse` the default protocol
### Additional information
The config option is based on the discussion in this note: https://hackmd.io/`@rust-cargo-team/B13O52Zko`
Add completions for `cargo remove`
### What does this PR try to resolve?
This PR continues the deferred work of #11099 by adding bash and zsh completions for the cargo remove subcommand.
### How should we test and review this PR?
There doesn't seem to be much in the way of testing for these completions automatically, so I would suggest verifying that they work in practice and sufficiently cover the subcommand's surface area.
### Additional Information
I will also soon post a PR for cargo remove's documentation.
Config file loaded via CLI takes priority over env vars
### What does this PR try to resolve?
Fixes#10992
Behaviour changes: After this commit, config files loaded via CLI take
priority over env vars. Config files loaded via [`config-include`]
feature also get a higher priority over env vars, if the initial config
file is being loaded via CLI.
Cargo knows why it loads a specific config file with `WhyLoad` enum,
and store in the field of `Definition::Cli(…)`. With this additional data,
config files loaded via `--config <path>` get a `Definition::Cli(Some(…))`.
In contrast, `--config <value>` with ordinary values become `Definition::Cli(None)`.
The error message of the `Definition::Cli(Some(…))` is identical to
`Definition::Path(…)`, so we won't lose any path information to debug.
[`config-include`]: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#config-include
### How should we test and review this PR?
Reviewing commit by commit is probably fine. The first two commits adding tests to `config-cli` and `config-include`, which exercises the expected behaviour we are going to fix in the next commits.
To check the precedence, set up a project with an extra config file, such as
```
# Expect to have a target-dir named `foo`
CARGO_BUILD_TARGET_DIR='env' cargo c --config 'build.target-dir = "cli"' --config .cargo/foo.toml
# Inside .cargo/foo.toml
[build]
target-dir = "foo"
```
### Additional information
This is a bit hacky IMO. I don't like leaking the path info to `Definition::Cli`. However, it is really tricky to provide information for deserialization before values get merged.
As the existence of `FeaturesFor` is acting as a key that discriminate
activated features for the same package with different circumstances.
Artifact dependencies deserve its own variant and it makes things clear to me.
- `NormalOrDevOrArtifactTarget(None)` -> `NormalOrDev`
- `NormalOrDevOrArtifactTarget(Some(Target))` -> `ArtifactDep(Target)`
Implement RFC 3289: source replacement ambiguity
### Implements [RFC 3289](https://github.com/rust-lang/rfcs/pull/3289)
* When the crates-io source is replaced, the user needs to specify `--registry <NAME>` when running an API operation to disambiguate which registry to use. Otherwise, cargo will issue a new error.
* In source replacement, the `replace-with` key can reference the name of an alt registry in the `[registries]` table.
* Publishing to source-replaced crates.io is no longer permitted using the crates.io token (`registry.token`). We have had a deprecation warning in place since #7973 (1.45.0).
### Testing
* Tests that interacting with crates.io use the new `replace_crates_io` function, which internally sets an environment variable to change the URL of crates.io.
Changes are insta-stable.
cc #10894
r? `@Eh2406`
Use correct version of cargo in test
Fix `cargo_remove::offline` test using wrong version of cargo in test, the local version and calling instance of cargo instead of the one being tested.
Issue discovered in #10907 after merge of #11099.
Check empty input for login
### What does this PR try to resolve?
close https://github.com/rust-lang/cargo/issues/11144
Check empty input for login.
### How should we test and review this PR?
- unit test
- cargo login and enter
Add retry support to sparse registries
Sparse (HTTP) registries currently do not respect Cargo's retry policy for http requests.
This change makes sparse registries use the same retry system as package downloads.
fix(test): Distinguish 'testname' from escaped arguments
When working on clap v4, it appeared that `last` and `trailing_var_arg`
are mutually exclusive, so I called that out in the debug asserts in
#4187. Unfortunately, I didn't document my research on this
as my focus was elsewhere.
When updating cargo to use clap v4 in #11159, I found `last` and
`trailing_var_arg` were used together. I figured this was what I was
trying to catch with above and I went off of my intuitive sense of these
commands and assumed `trailing_var_arg` was intended, not knowing about
the `testname` positional that is mostly just a forward to `libtest`,
there for documentation purposes, except for a small optimization.
So it looks like we just need the `last` call and not the
`trailing_var_arg` call.
This restores us to behavior from 531ce1321d which is what I originally
wrote the tests against.
It looks like #11159 was merged after the last beta branch was made, so we shouldn't
need to cherry-pick this into any other release.
For reviewing this, I made the test updates in the first commit, showing the wrong behavior. The following commit fixes the behavior and updates the tests to expected behavior.
Fixes#11191
When working on clap v4, it appeared that `last` and `trailing_var_arg`
are mutually exclusive, so I called that out in the debug asserts in
clap-rs/clap#4187. Unfortunately, I didn't document my research on this
as my focus was elsewhere.
When updating cargo to use clap v4 in #11159, I found `last` and
`trailing_var_arg` were used together. I figured this was what I was
trying to catch with above and I went off of my intuitive sense of these
commands and assumed `trailing_var_arg` was intended, not knowing about
the `testname` positional that is mostly just a forward to `libtest`,
there for documentation purposes, except for a small optimization.
So it looks like we just need the `last` call and not the
`trailing_var_arg` call.
This restores us to behavior from 531ce1321 which is what I originally
wrote the tests against.
Fix sparse registry lockfile urls containing 'registry+sparse+'
The `Cargo.lock` file for alternative sparse registries incorrectly lists the url as `registry+sparse+` rather than `sparse+`.
Fixes#10963
