Add test for deleted index entry
This adds a test for when an entry is deleted from the index. This is done on crates.io occasionally for things like copyright takedown requests.
This behavior was tripping a debug assert which this removes. I'm not entirely certain why the debug assert is there, but I think it is not correct, since this is obviously a scenario where there might be a cached file, but the index responds with NotFound.
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (#1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
So now `cargo` will block until it sees the package in the index.
- This is checking via the index instead of server APIs in case there
are propagation delays. This has the side effect of being noisy
because of all of the "Updating index" messages.
- This is done unconditionally because cargo used to block and that
didn't seem to be a problem, blocking by default is the less error
prone case, and there doesn't seem to be enough justification for a
"don't block" flag.
The timeout was 5min but I dropped it to 1m. Unfortunately, I don't
have data from `cargo-release` to know what a reasonable timeout is, so
going ahead and dropping to 60s and assuming anything more is an outage.
Fixes#9507
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (#1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
This introduces unstable support for blocking by setting
`publish.timeout` to non-zero value.
A step towards #9507
refactor(tests): Prepare for wait-for-publish test changes
In #11062, we are updating `cargo publish` to wait until a package is published. The problem is a lot of our tests will block until the timeout. In finding the tests to update, I was originally relying on test failures from the extra output when timing out. The problem is not all tests verify the test output so they don't fail.
This tries to update the tests to make the introduction of a timeout more obvious.
- Adding `with_stderr` where it wasn't before
- Moving away from `with_stderr_contains` for publish tests
To help with that, I made the predicates on cargo commands more consistent.
I also moved descriptions of tests to be outside of the test so I can more easily document the `registry::init` calls with what we are doing.
Config file loaded via CLI takes priority over env vars
### What does this PR try to resolve?
Fixes#10992
Behaviour changes: After this commit, config files loaded via CLI take
priority over env vars. Config files loaded via [`config-include`]
feature also get a higher priority over env vars, if the initial config
file is being loaded via CLI.
Cargo knows why it loads a specific config file with `WhyLoad` enum,
and store in the field of `Definition::Cli(…)`. With this additional data,
config files loaded via `--config <path>` get a `Definition::Cli(Some(…))`.
In contrast, `--config <value>` with ordinary values become `Definition::Cli(None)`.
The error message of the `Definition::Cli(Some(…))` is identical to
`Definition::Path(…)`, so we won't lose any path information to debug.
[`config-include`]: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#config-include
### How should we test and review this PR?
Reviewing commit by commit is probably fine. The first two commits adding tests to `config-cli` and `config-include`, which exercises the expected behaviour we are going to fix in the next commits.
To check the precedence, set up a project with an extra config file, such as
```
# Expect to have a target-dir named `foo`
CARGO_BUILD_TARGET_DIR='env' cargo c --config 'build.target-dir = "cli"' --config .cargo/foo.toml
# Inside .cargo/foo.toml
[build]
target-dir = "foo"
```
### Additional information
This is a bit hacky IMO. I don't like leaking the path info to `Definition::Cli`. However, it is really tricky to provide information for deserialization before values get merged.
Use correct version of cargo in test
Fix `cargo_remove::offline` test using wrong version of cargo in test, the local version and calling instance of cargo instead of the one being tested.
Issue discovered in #10907 after merge of #11099.
Check empty input for login
### What does this PR try to resolve?
close https://github.com/rust-lang/cargo/issues/11144
Check empty input for login.
### How should we test and review this PR?
- unit test
- cargo login and enter
Add retry support to sparse registries
Sparse (HTTP) registries currently do not respect Cargo's retry policy for http requests.
This change makes sparse registries use the same retry system as package downloads.
fix(test): Distinguish 'testname' from escaped arguments
When working on clap v4, it appeared that `last` and `trailing_var_arg`
are mutually exclusive, so I called that out in the debug asserts in
#4187. Unfortunately, I didn't document my research on this
as my focus was elsewhere.
When updating cargo to use clap v4 in #11159, I found `last` and
`trailing_var_arg` were used together. I figured this was what I was
trying to catch with above and I went off of my intuitive sense of these
commands and assumed `trailing_var_arg` was intended, not knowing about
the `testname` positional that is mostly just a forward to `libtest`,
there for documentation purposes, except for a small optimization.
So it looks like we just need the `last` call and not the
`trailing_var_arg` call.
This restores us to behavior from 531ce1321d which is what I originally
wrote the tests against.
It looks like #11159 was merged after the last beta branch was made, so we shouldn't
need to cherry-pick this into any other release.
For reviewing this, I made the test updates in the first commit, showing the wrong behavior. The following commit fixes the behavior and updates the tests to expected behavior.
Fixes#11191
When working on clap v4, it appeared that `last` and `trailing_var_arg`
are mutually exclusive, so I called that out in the debug asserts in
clap-rs/clap#4187. Unfortunately, I didn't document my research on this
as my focus was elsewhere.
When updating cargo to use clap v4 in #11159, I found `last` and
`trailing_var_arg` were used together. I figured this was what I was
trying to catch with above and I went off of my intuitive sense of these
commands and assumed `trailing_var_arg` was intended, not knowing about
the `testname` positional that is mostly just a forward to `libtest`,
there for documentation purposes, except for a small optimization.
So it looks like we just need the `last` call and not the
`trailing_var_arg` call.
This restores us to behavior from 531ce1321 which is what I originally
wrote the tests against.
Fix sparse registry lockfile urls containing 'registry+sparse+'
The `Cargo.lock` file for alternative sparse registries incorrectly lists the url as `registry+sparse+` rather than `sparse+`.
Fixes#10963
Reduce references to `[project]` within cargo
There was an issue (#11129) with `[project]` being put in the docs but nothing else stating what `[project]` is. This was an oversight by me when writing the docs as `[project]` is still supported in cargo. [Back in 2014](86b2a2a432) `[project]` was [renamed to `[package]`](https://github.com/rust-lang/cargo/issues/3388#issuecomment-266504735), since then it has kinda sat as something that exists and is supported but not really. I brought this up in a [zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/removal.20of.20.60.5Bproject.5D.60), and it was suggested that we could show a warning when `[project]` is used in a manifest of a _top-level_ crate.
To go slightly further than this I tried to change as many references from `[project]` to `[package]` as possible. This should hopefully help with confusion between `[project]` and `[package]` in the future.
This PR also includes a cherry-picked commit from #11131 as it was having issues with bors and fits well with the changes that are being made in this PR.
This should ideally be reviewed commit by commit
Iteratively construct target cfg
When setting target features via rustflags via `[build]` config key, cargo correctly propagates them to rustc (via -C flag) and to build.rs (via CARGO_CFG_TARGET_FEATURE env var).
However, if `[target.cfg]` is used instead, build.rs doesn't get the flags (rustc still gets them though).
This changes it so that cargo will call `rustc` up to two times to collect the `cfg` values, updating which flags to use on the second call based on the cfg discovered in the first call.
Closes#6858.
Http publish not noop
Currently the `cargo-test-support` `HttpServer` is noop on publish. This was causing issues with #11062 as there is [not function registry to pull from](https://github.com/rust-lang/cargo/pull/11062#issuecomment-1241220565). [A suggested fix](https://github.com/rust-lang/cargo/pull/11062#issuecomment-1241349110) to this was to have the test `HttpServer` act like a real registry and write to the filesystem. This would allow for tests to be run over the HTTP API and not fail since there was nothing to pull from.
This PR implements that suggestion by adding a body field to `Request`, and when hitting the publish endpoint it will try and write the `.crate` and manifest information to the filesystem.
Improve errors for TOML fields that support workspace inheritance
Fixes#10997
This also addresses the issue with `MaybeWorkspace<VecStringOrBool>` mentioned in #10942:
```
Caused by:
invalid type: string "foo", expected a boolean or vector of strings for key `package.publish`
```
I removed the `maybe_workspace_vec_string` deserializer in 7a50c0c718 because the error message from the inner `Vec<String>` is now surfaced, but I can revert that if it's preferable to keep those changes.
I tried to base the `deserialize` implementation off of the derived impl for an untagged enum from `cargo expand`. This approach [ultimately led me](https://github.com/serde-rs/serde/blob/v1.0.144/serde/src/private/de.rs#L218) to adding the `serde-value` dependency.
When setting target features via rustflags via `[build]` config key,
cargo correctly propagates them to rustc (via -C flag) and to build.rs
(via CARGO_CFG_TARGET_FEATURE env var).
However, if `[target.cfg]` is used instead, build.rs doesn't get the
flags (rustc still gets them though).
This will only show the messaeg if we didn't already show a version req
with full precision specified ... mostly. We'll also skip it if its a
local or git dependency but we never show the version in those cases
because it doesn't matter.
The `precise_version` logic came from cargo-upgrade.
This gives a hint to users that we might not be showing the feature list
for the latest version but the earliest version.
Also when using a workspace dependency or re-using an existing
dependency, this is a good reminder of what the version requirement is
that was selected.
However, when the user or add (the common case) selected a full
precision requirement, this is redundant.
I'm also mixed on whether the meta version should show up.
Fixes#11073
Improve error message for an array value in the manifest
Fixes https://github.com/rust-lang/cargo/issues/10942
The error message will be:
```
error: failed to parse manifest at `/Users/..`
Caused by:
invalid type: string "Ky", expected an array for key `package.authors`
```
Enable two windows tests
These two tests were disabled on Windows a long time ago. AFAICT, the reasons are no longer relevant, and it should be safe to enable these tests. See each commit for a more detailed exposition.
Improve error msg for get target runner
Actually, we'll get this config from three places. So this msg may be confusing when you set it up in `.cargo/config.toml` or pass it by `--config`.
We already printed the location of the config, so I think it's OK to change it to `configurations`.
Only override published resolver when the workspace is different
### What does this PR try to resolve?
Ensures when publishing a package that uses an implicit `resolver = "1"` to maintain an MSRV before the `resolver` key was stabilized the implicitness is retained rather than being turned into an explicit setting.
fixes#10954 (assuming that the workspace and its packages are configured with a consistent resolver)
Packages in the local workspace can't get updated this way; the user
just needs to point to a different source, or otherwise update the
package themselves.
People encountering a dependency with a newer `rust-version` requirement
may not know about `cargo update --precise`, or may consider alternate
approaches that may be harmful (such as pinning with a `=` dependency).
Provide specific guidance in the error message.
If the user is using `cargo install`, suggest `cargo install --locked` instead.
Test if reserved filenames are allowed in Windows
Recent versions of Windows have removed the limitation on filenames like `aux` or `con`. This change allows the `package::reserved_windows_name` to still pass by first trying to create a file with a reserved name to see if Windows supports it. If so, it skips the rest of the test. Otherwise, we keep the same behavior as before.
The previous commit tests whether the current machine supports Windows
restricted names by creating a file and checking whether that succeeds.
This commit reworks this testto use GetFullPathNameW, which can be done
without having to create and remove new files.
Recent versions of Windows have removed the limitation on filenames like
`aux` or `con`. This change allows the `package::reserved_windows_name`
to still pass by first trying to create a file with a reserved name to
see if Windows supports it. If so, it skips the rest of the test.
Otherwise, we keep the same behavior as before.
Add reasons to all ignored tests.
This adds a reason string to all `#[ignore]` attributes. This will be displayed when running the test (since 1.61), which can help quickly see and identify why tests are being ignored. It looks roughly like:
```
test basic ... ignored, requires nightly, CARGO_RUN_BUILD_STD_TESTS must be set
test build::simple_terminal_width ... ignored, --diagnostic-width is stabilized in 1.64
test check_cfg::features_with_cargo_check ... ignored, --check-cfg is unstable
test plugins::panic_abort_plugins ... ignored, requires rustc_private
```
This test was ignored in https://github.com/rust-lang/cargo/pull/3189
without much discussion of explaining why.
AFAICT, this test works fine on Windows on both MSVC and GNU.
Empty paths do the expected behavior (preventing cargo from running
rustc). There are some special rules on Windows about discovering the
process to run (such as searching the app's launch directory), but
I do not think that is relevant here. Confirmed by trying to run
`cargo check` in this test fails to find `rustc`.
Fix formats_source test requiring rustfmt.
The requirements added in #9892 that `rustfmt` must be present doesn't work in the `rust-lang/rust` environment. There are two issues:
* Cargo is run without using rustup. If you also have rustup installed, the test will fail because the `rustfmt` binary found in `PATH` will fail to choose a toolchain because HOME points to the sandbox home which does not have a rustup configuration.
* rust-lang/rust CI uninstalls rustup, and does not have rustfmt in PATH at all. It is not practical to make rustfmt available there.
The solution here is to just revert the behavior back to where it was where it checks if it can run `rustfmt` in the sandbox. This should work for anyone who has a normal rustup installation (including Cargo's CI). If running the testsuite without rustup, then the test will be skipped.
This also includes a small enhancement to provide better error information when rustfmt fails.
This is done in the command, rather than in the op,
- To consistently construct the `Workspace`
- It is more composable as an API
A downside is we update the git dependencies a second time.
We are not rolling back on error.
- For some errors, the user might want to debug what went wrong
- Rollback adds its own complications and risks, including since its
non-atomic
Fixes#10901
This is prep for #10901 to avoid the most common failure case for the
lock file. We are assuming if the users action caused a change in the
manifes, then it will cause a change in the lock file. This isn't
entirely true but close enough and I think these semantics can make
sense.
add a reason to `masquerade_as_nightly_cargo` so it is searchable
When I was working on the stabilization for workspace inheritance, it was very tedious to find all of the places to remove `.masquerade_as_nightly_cargo()`. I [suggested](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/problems.20finding.20.60.2Emasquerade_as_nightly_cargo.28.29.60) to add a reason to `.masquerade_as_nightly_cargo()` so that it would be easier to find all of the places to remove it. By adding the reason it makes it easy to search for all places with the features name. This PR adds the reason(s) to all of the places `.masquerade_as_nightly_cargo()` is called, as well as updates the documentation so it talks about adding a reason when making the call.
Add a test for regressions in selecting the correct workspace root
This adds a test to check for regressions in selecting the correct workspace when there are nested workspaces.
#10846 solved a problem with nested workspace resolution that was caused by #10776. `@ehuss` [suggested](https://github.com/rust-lang/cargo/pull/10846#issuecomment-1183754728) that a test should be added to ensure that this issue does not pop up again.
I ensured that this worked by testing against commit before #10846. Sporadically I would get an error that was the same as described in #10846.
```
error: package `{path}/cargo/target/tmp/cit/t0/foo/sub/foo/Cargo.toml` is a member of the wrong workspace
expected: {path}/cargo/target/tmp/cit/t0/foo/sub/Cargo.toml
actual: {path}/cargo/target/tmp/cit/t0/foo/Cargo.toml
```
I then tested it on the commit with the fix and the test passed every time.
---
While this does add a test to catch any regression I am worried that it will not catch it every time. It was noted in #10846 that this error would sometimes happen but not every time, in my testing I found this to be true as well. Since this is caused by the `HashMap` order changing each run, switching to something ordered like `BTreeMap` **_should_** catch any regressions every run (if the implementation were to ever change). I'm not sure if this is necessary so I figured I would note the concern here.
A comment on killercup/cargo-edit#15 had me worried that `cargo add` was
deleting comments now. It appears that isn't the case for inline
tables.
Standard tables however do delete comments. The work to make sure they
don't conflicts with another need. When changing the source, we delete
the old source fields and append the new which can cause some formatting
to be carried over unnecessarily.
For example, what would normally look like
```toml
cargo-list-test-fixture-dependency = { optional = true, path = "../dependency", version = "0.0.0" }
```
When fixed to preserve comments with my naive solution looks like
```toml
cargo-list-test-fixture-dependency = { optional = true , path = "../dependency", version = "0.0.0" }
```
Note that `optional = true` used to be last, so space separating it and
`}` was kept, now separating it and `,`.
More work will be needed to get this into an ideal state but we can at
least have confidence with inline tables for now.
