Accurately show status when downgrading dependencies
### What does this PR try to resolve?
A few times now I've ran into issues where Cargo ends up downgrading a dependency in order to satisfy a pinned dependency somewhere in the dependency graph. Unfortunately this is not clear in the output of `cargo update`, which just shows all the changed dependencies as `Updating`.
References:
* #5702
* [Finding a pinned dependency edge](https://users.rust-lang.org/t/finding-a-pinned-dependency-edge/81157)
* https://github.com/tokio-rs/axum/issues/1814
### How should we test and review this PR?
This is a small change that tries to make dependency downgrades stand out more in the output of `cargo update`. I have not added any new tests since the existing tests seem to cover this functionality.
Some tests still fail, these refer to Git dependencies. I'm honestly not sure how to handle these, so I'd like to get some feedback on this before I fix those tests up. Git commits form a DAG, so one option is to see if the new commit is an ancestor of the old one (mark as "updating"), if the old commit is an ancestor of the new one (mark as "downgrading"), or neither (could be from parallel branches) we could compare based on timestamp in that case.
### To do
* Fix up tests for Git dependency updates
The proper fix is in https://github.com/Byron/gitoxide/releases/tag/gix-v0.41.0
which unfortunately can't be used as it also comes with the latest `tempfile` v3.4
which causes other issues when compiling on some platforms.
Thus we first disable the flaky tests, and re-enable them with the `gix` upgrade
which should be possible once `tempfile` doesn't hinder `cargo` on some platforms
anymore.
Related to https://github.com/rust-lang/cargo/issues/11821
Add `CARGO_PKG_README`
Fixes#11597
This environment variable shows the path to the README file of your package. From #11597:
> Cargo may rewrite the package’s `Cargo.toml` and move the README file around, relative to the manifest. I would like to `include_str!()` this README in my `lib.rs`, but am unable to do so right now, because if I specify `include_str!("../../README")` it works for development, but I can’t package my crate. Conversely if I specify `include_str!("../README")` it works when packaged, but not during development.
Adding display of which target failed to compile
Closes#10834
Attached example.zip is the same as the one mentioned on the issue. You could use it to test the fix by using the new built cargo:
```./cargo build --manifest-path C:\Rust\example_cargo_error\Cargo.toml```
Before fixing:
```
error: could not compile `example` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
```
After fixing:
```
error: could not compile `example` (build script) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
```
[example.zip](https://github.com/rust-lang/cargo/files/10522342/example.zip)
Fix `CARGO_CFG_` vars for configs defined both with and without value
When a rustc cfg is defined both with and without value, the environment variable should provide all the values. Before this change, it ended up being empty.
Fixes: #11789
When a rustc cfg is defined both with and without value, the
environment variable should provide all the values. Before this change,
it ended up being empty.
Fixes: #11789
Make `sparse` the default protocol for crates.io
Changes the default protocol for accessing crates.io to `sparse`.
The protocol can be switched back to `git` via the configuration key `registries.crates-io.protocol` = `git`
Closes#10965
This allows to use `gitoxide` for all fetch operations, boosting performance
for fetching the `crates.io` index by a factor of 2.2x, while being consistent
on all platforms.
For trying it, nightly builds of `cargo` can specify `-Zgitoxide=fetch`.
It's also possible to set the `__CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2=1` environment
variable (value matters), which is when `-Zgitoxide=none` can be used
to use `git2` instead.
Limitations
-----------
Note that what follows are current shortcomings that will be addressed in future PRs.
- it's likely that authentication around the `ssh` protocol will work differently in practice
as it uses the `ssh` program.
- clones from `file://` based crates indices will need the `git` binary to serve the locatl repository.
- the progress bar shown when fetching doesn't work like the orgiinal, but should already feel 'faster'.
patch can conflict on not activated packages
### What does this PR try to resolve?
In the resolver there is a data structure called a `conflicting_activations`, which records all the reasons a "better" version was not picked for the package being resolved. Normally, these are packages that are currently active that for one reason or another block one of the other versions. Several optimizations assumed that this was always true, even going so far as to use `.expect`. This was a mistake because when there's a `patch` involved there can be conflicts on a version that is not selected. So the correct behavior is to fall back to skip the optimizations and try all versions when a `conflicting_activations` are not active.
### How should we test and review this PR?
This adds two automated tests based on the reproductions in #7463 and #11336. So the test suite now covers this case. It can also be tested by reconstructing the repositories originally reported in those issues.
### Additional information
It could be that in the long term the correct fix is to figure out how to support patch without having a conflicting activation that is not activated. But that would be a much bigger change. And for now this assumption is only used in optimizations, so this gets people unstuck.
fix(toml): Provide a way to show unused manifest keys for dependencies
Dependencies have not been able to show unused manifest keys for some time, this problem partially resulted in #11329.
This problem is caused by having an `enum` when deserializing. To get around this you can use:
```rust
#[serde(flatten)]
other: BTreeMap<String, toml::Value>,
```
This collects any unused keys into `other` that can later be used to show warnings. This idea was suggested in a thread I cannot find but is mentioned in [serde#941](https://github.com/serde-rs/serde/issues/941).
Improve error for missing crate in --offline mode for sparse index
This changes sparse registries to instead return `NotFound` when a non-cached crate is requested in `--offline` mode.
The resolver can then suggest removing the `--offline` flag if resolution fails, which is a more helpful error than the one currently issued: `attempting to make an HTTP request, but --offline was specified`.
With this change, the behavior matches what is already done for git-based registries.
Closes#11276
This changes sparse registries to instead return not found when a non-cached crate is requested in offline mode.
The resolver can then suggest removing the --offline flag if resolution
fails.
feat(resolver): `-Zdirect-minimal-versions`
This is an alternative to `-Zminimal-versions` as discussed in #5657.
Problems with `-Zminimal-versions` includes
- Requires the root most dependencies to verify it and we then percolate that up the stack. This requires a massive level of cooperation to accomplish and so far there have been mixed results with it to the point that cargo's unstable
documentation discourages its use.
- Users expect `cargo check -Zminimal-versions` to force resolving to minimal but it doesn't as the default maximal resolve is compatible and requires `cargo update -Zminimal-versions`
- Different compatible versions might be selected, breaking interop between crates, changing feature unification, and breaking `-sys` crates without bad `links`
`-Zdirect-minimal-versions` instead only applies this rule to your
direct dependencies, allowing anyone in the stack to immediately adopt
it, independent of everyone else.
Special notes
- Living up to the name and the existing design, this ignores yanked
crates. This makes sense for `^1.1` version requirements but might
look weird for `^1.1.1` version requirements as it could select
`1.1.2`.
- This will error if an indirect dependency requires a newer version.
Your version requirement will need to capture what you use **and** all
of you dependencies. An alternative design would have tried to merge
the result of minimum versions for direct dependencies and maximum
versions for indirect dependencies. This would have been complex and
led to weird corner cases, making it harder to predict. I also suspect
the value gained would be relatively low as you can't verify that
version requirement in any other way.
- This also means discrepancies between `dependencies` and `dev-dependencies` are errors
- The error could be improved to call out that this was from minimal
versions but I felt getting this out now and starting to collect
feedback was more important.
One advantage of this approach over `-Zminimal-versions` is that it removes most of the problems that [cargo-minimal-versions](https://github.com/taiki-e/cargo-minimal-versions) tried to workaround.
As for the implementation, this might not be the most elegant solution but it works and we can always iterate and improve on it in the future.
- We keep the state as a `bool` throughout but compensate for that by explicitly creating a variable to abstract away constants
- The name changes depending on the context, from `direct_minimal_version` when dealing with the unstable flag to `first_minimal_version` when the concept of "direct" is lost to `first_version` when we split off the ordering concept into a separate variable
- Packages that respect `direct_minimal_versions` are determined by whether they are the top-level `summaries` that get past into `resolve`
### What does this PR try to resolve?
The primary use case is verifying version requirements to avoid depending on something newer than might be available in a dependent
For this to help the MSRV use case, the crate author must directly depend on all indirect dependencies where their latest release has too new of an MSRV but at least they can do so with the `^` operator, rather than `<` and breaking the ecosystem.
### How should we test and review this PR?
The first two commits add tests using `-Zminimal-versions`. The commit that adds `-Zdirect-minimal-versions` updates the tests, highlighting the differences in behavior.
### Additional information
Potential areas of conversation for stablization
- Flag name
- Handling of yanked (pick first non-yanked, pick yanked, error)
- Quality of error message
- Should the package have a "memory" of this flag being set by writing it to the lockfile?
Potential future work
- Stablize this
- Remove `-Zminimal-versions`
- Update `cargo publish`s `--verify` step to use this.
- The challenge is this won't be using the packaged `Cargo.lock` which probably should also be verified.
Error message for transitive artifact dependencies with targets the package doesn't directly interact with
Address #11260. Produces an error message like described by `@weihanglo` [here](https://github.com/rust-lang/cargo/issues/11260#issuecomment-1400455203):
```
error: could not find specification for target "x86_64-windows-msvc"
Dependency `bar v0.1.0` requires to build for target "x86_64-windows-msvc".
```
Note that this is not a complete fix for #11260.
Suggest cargo add when installing library crate
### What does this PR try to resolve?
When using `cargo install` instead of `cargo use` the error message is the following:
```
error: there is nothing to install in `foo v0.0.1`, because it has no binaries
`cargo install` is only for installing programs, and can't be used with libraries.
To use a library crate, add it as a dependency in a Cargo project instead.
```
It would be good to suggest to the user to use `cargo add`.
### How should we test and review this PR?
The `no_binaries` test from `tests/testsuite/install.rs` covers that case.
Switch some tests from `build` to `check`
#11341 brought up issues with cargo's `testsute` size and speed. One of the suggested fixes was switching most tests to `check` instead of `build`. This PR did that.
Before size on `nightly`: 4.4GB
After size on `nightly`: 4.2GB
Regex to find `build` in `tests/testsuite`: `cargo\(".*build.*\)`
Before: 1607
After: 626
Note I did not remove all `build` I only did the easy ones that required minimal changes. I also tried not to touch systems I was unsure about. There could be other uses of `build` I missed as well.
I still need to play around with `opt-level`, `debug=0`, and a few other tweaks, but there should be more time/memory to drop.
Each test file changed is in a commit of its own, so you should look commit by commit.
* moved `is_empty` check into `check_token`
* improved error message (is quite long now but should explain the error
well)
* removed one helper function from new test
When using registry operations with authentication there will be now an
error if the given token is not valid.
This is a technically a breaking change because a registry might give
some tokens which will be denied by these new checks.
In practice these tokens cause issues with HTTP so no registry should
generate them.
