Make cargo forward pre-existing CARGO if set
Currently, Cargo will always set `$CARGO` to point to what it detects its own path to be (using `std::env::current_exe`). Unfortunately, this runs into trouble when Cargo is used as a library, or when `current_exe` is not actually the binary itself (e.g., when invoked through Valgrind or `ld.so`), since `$CARGO` will not point at something that can be used as `cargo`. This, in turn, means that users can't currently rely on `$CARGO` to do the right thing, and will sometimes have to invoke `cargo` directly from `$PATH` instead, which may not reflect the `cargo` that's currently in use.
This patch makes Cargo re-use the existing value of `$CARGO` if it's already set in the environment. For Cargo subcommands, this will mean that the initial invocation of `cargo` in `cargo foo` will set `$CARGO`, and then Cargo-as-a-library inside of `cargo-foo` will inherit that (correct) value instead of overwriting it with the incorrect value `cargo-foo`. For other execution environments that do not have `cargo` in their call stack, it gives them the opportunity to set a working value for `$CARGO`.
One note about the implementation of this is that the test suite now needs to override `$CARGO` explicitly so that the _user's_ `$CARGO` does not interfere with the contents of the tests. It _could_ remove `$CARGO` instead, but overriding it seemed less error-prone.
Fixes#10119.
Fixes#10113.
Clean up workspace dependencies after cargo remove
### What does this PR try to resolve?
After successful removal of an inherited dependency from a workspace member, clean up the root workspace manifest.
This PR is part of the continued working on cargo remove (#11099, see deferred work).
### How should we test and review this PR?
Make sure the tests cover all possible use cases. After posting this PR, I will post a short self-review regarding some design concerns.
### Additional information
#11194 is currently blocked on this feature.
The bug was `rustflags_from_target` trying to learn rustdocflags from
`target.cfg(…).rustflags`, which is definitely wrong.
As of this writing, either `target.cfg(…).rustdocflags` or
`target.<triple>.rustdocflags` is not supported.
Update the outdated link for rust-semverver
### What does this PR try to resolve?
Update the outdated link for rust-semverver. It works but better to keep it updated.
Only remove fingerprints and build script artifacts of the requested package
Fixes#10069
This is my first PR to cargo. Let me know if you want me to add tests, or if there are any other changes you would like to see :)
This PR changes the globs used to remove fingerprints and build artifacts when running `cargo clean -p <pkid>`. The glob used was `<package_name>-*` which would match artifacts for packages that are prefixed by `<package_name>-` (e.g. `cargo clean -p sqlx` would also remove artifacts for `sqlx-{core,macros,etc.}`). This problem is not seen with other artifacts since those use the crate name instead of package name which normalize hyphens to underscores.
The changes follow the naive approach mentioned in #10069 where some basic string manipulation is done to strip off the last hyphen, hash, and potential extension to get the original package name which can be used to determine if the artifact is actually for the desired package. This means that this **does not** handle trying to resolve the package to determine the artifacts, so it still ignores the url and version that may be passed in the pkgid
Newer anyhow features are required
anyhow doesn't follow semver rules for new features, and Cargo won't compile with anyhow older than this version due to dependence on captured variables in format strings.
Clean stale git temp files
### What does this PR try to resolve?
When cargo is interrupted while libgit2 is indexing the pack file, it will leave behind a temp file that never gets deleted. These files can be very large. This checks for these stale temp files and deletes them.
### How should we test and review this PR?
There is a simulated test here. To test with the actual behavior:
1. Run `CARGO_HOME=chome cargo fetch`
2. While it is "resolving deltas", hit Ctrl-C.
3. Notice that there is a 200MB file in `chome/registry/index/github.com-1ecc6299db9ec823/.git/objects/pack/`
4. Do that several times if you want, each time adds another 200MB file.
5. Build this PR: `cargo b -r`
6. Run `CARGO_HOME=chome CARGO_LOG=cargo::sources::utils=debug ./target/release/cargo fetch`
7. Notice that it deletes the `pack_git2_*` files.
Report crate size on package and publish
### Motivation
Fixes#11251.
This adds a line like `Packaged 42 files, 727.0KiB (143.8KiB compressed)` to the output of `cargo package` and `cargo publish`. See the associated issue for more details.
### Test info
I've updated associated tests to account for the new line in the output, including the file count where relevant. I've also added a few additional tests specifically to address the uncompressed and compressed file sizes.
If you'd like to test this manually, simply run `cargo package` or `cargo publish` within a project of your choice. The new `Packaged` line will appear at the end of the stderr output, or directly before the `Uploaded` line, respectively.
### Review info
This PR touches many test files to account for the additional line of stderr output. For convenience, I've split the PR into 4 commits.
1. contains the actual implementation
2. updates existing tests unrelated to `cargo package` and `cargo publish`
3. updates existing tests related to `cargo package` and `cargo publish`, including file counts where relevant
4. adds new tests specifically for the file sizes, which are generally not covered by existing tests
### Additional information
The feature was discussed [on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Report.20crate.20size.20when.20packaging) prior to implementation.
Potential future extensions to explore include:
- Report size per file when using `--verbose`
- Compare to the size of the previously uploaded version to warn if the increase is sufficiently large
- Consider designs that can draw more attention to the most important information
- Warn if large binary files are included ([#9058](https://github.com/rust-lang/cargo/issues/9058))
add a note that some warnings (and/or errors) can be auto-fixed
This adds a note that some warnings (and/or errors) can be auto-fixed by running `cargo --fix`. This only supports `cargo check` as we can't show the same note for `cargo clippy` since it is an external subcommand.
The message for the note was taken from [#10976 (comment)](https://github.com/rust-lang/cargo/issues/10976#issuecomment-1212517765).
Update libcurl
This updates to the latest libcurl from 7.83.1 to 7.86.0. There are extensive changes documented thoroughly at https://curl.se/changes.html. There are 5 CVEs as documented at https://curl.se/docs/security.html, though none of them look particularly serious for our use case.
This adds a limit of TLS 1.2 for Windows. There have been some issues with TLS 1.3 (which was recently added), and I'm not confident enabling it, yet. Perhaps some day in the future when it looks like it is more reliable, the limit can be removed.
fix(publish): Block until it is in index
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (#1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
So now `cargo` will block until it sees the package in the index.
- This is checking via the index instead of server APIs in case there
are propagation delays. This has the side effect of being noisy
because of all of the "Updating index" messages.
- This blocks by default but there is an unstable `publish.timeout` config field that will disable blocking when set to 0. See #11222 for stablization
Blocking is opt-out as that is the less error prone case for casual users while those doing larger integrations are also likely to do the testing needed to make more complicated scenarios work where blocking is disabled.
Right now we block after the publish. An alternative would be to block until all dependencies are in the index which makes the blocking only happen when needed
- Blocking on dependencies can be imprecise to detect when to block vs propagate an error up
- This is the less error prone case for users. For example I recently publish a crate in one tab and immediately switched to another tab to use it and this only worked because `cargo-release` blocked until it was ready to use
In reviewing this change, be sure to look at the individual commits
- The first makes it possible to write the tests for this
- The second adds a test that shows the current behavior
- The third updates the test to the expected behavior, showing all of this works
In addition to the publish tests:
- We want to maximize the nightly-to-stable time to collect feedback
- We will put this in TWiR's testing section to raise visibility
Fixes#9507
Add Accept-Encoding request header to enable compression
### What does this PR try to resolve?
Cargo does not request compression from servers. Enabling compression can save bandwidth and improve performance.
### How should we test and review this PR?
I validated locally that the header was being sent using a local proxy (Fiddler). It sent `Accept-Encoding: deflate, gzip` on Windows.
Update contrib docs for highfive transition
As of #11293, this repo is now using triagebot (AKA `@rustbot)` for auto-assignment of PRs. This PR updates the contributor documentation to accommodate this change.
Previously, `is_dep_activated` depends on `activated_dependencies`,
which is a map of `PackageFeaturesKey` to its own activated `DepFeature`s.
`PackageFeaturesKey` in feature resolver is always comprised of
* A `PackageId` of a given dependency, and
* A enum value that helps decoupling activated features for that dependency.
Currently depending on the type of given dependency.
Looking into issue 10526, it has an `activated_dependencies` of
```
{
(mycrate, NormalOrDevOrArtifactTarget(None)) -> [dep:mybindep]
}
```
However, this [line][1] accidentally use a parent's `pkgid`
and a dependency's `FeaturesFor` to compose a key:
```
(mycrate, NormalOrDevOrArtifactTarget("x86_64-unknown-linux-gnu"))
```
That is never a valid key to query features for artifacts dependency.
A valid key should be comprised of components both from the parent:
```
(mycrate, NormalOrDevOrArtifactTarget(None))
```
Or both from the dependency itself:
```
(mybindep, NormalOrDevOrArtifactTarget("x86_64-unknown-linux-gnu"))
```
As aforementioned `activated_dependencies` only stores parent packages
and their activated features. Those informations are included in
`activated_features` as well, so this commit goes with the route that
removes `activated_dependencies` and uses only dependency's infomation
to query if itself is activated.
[1]: 0b84a35c2c/src/cargo/core/compiler/unit_dependencies.rs (L1097-L1098)
Fix confusing error messages when using -Zsparse-registry
Built-in replacements of crates.io have confusing a description:
> crates.io index (which is replacing registry `crates-io`)
This adds a special case for built-in source replacements of `crates-io`. User-defined replacements of crates.io continue to use the existing description.
Also fixes the test framework `__CARGO_TEST_CRATES_IO_URL_DO_NOT_USE_THIS` variable to strip the `sparse+` prefix when checking if a url `is_crates_io`.
Fixes#11277
Fix 410 gone response handling
This changes the sparse-registry support for the 410 "Gone" HTTP response code. This is out of sync with the [load function](8adf1df292/src/cargo/sources/registry/http_remote.rs (L375)) mentioned in the comment. I assume it is supposed to be 410 and not 401 since 401 is "Unauthorized", and that doesn't signify that the resource is "not found".
r? `@arlosi`
Currently, Cargo will always set `$CARGO` to point to what it detects
its own path to be (using `std::env::current_exe`). Unfortunately, this
runs into trouble when Cargo is used as a library, or when `current_exe`
is not actually the binary itself (e.g., when invoked through Valgrind
or `ld.so`), since `$CARGO` will not point at something that can be used
as `cargo`. This, in turn, means that users can't currently rely on
`$CARGO` to do the right thing, and will sometimes have to invoke
`cargo` directly from `$PATH` instead, which may not reflect the `cargo`
that's currently in use.
This patch makes Cargo re-use the existing value of `$CARGO` if it's
already set in the environment. For Cargo subcommands, this will mean
that the initial invocation of `cargo` in `cargo foo` will set `$CARGO`,
and then Cargo-as-a-library inside of `cargo-foo` will inherit that
(correct) value instead of overwriting it with the incorrect value
`cargo-foo`. For other execution environments that do not have `cargo`
in their call stack, it gives them the opportunity to set a working
value for `$CARGO`.
One note about the implementation of this is that the test suite now
needs to override `$CARGO` explicitly so that the _user's_ `$CARGO` does
not interfere with the contents of the tests. It _could_ remove `$CARGO`
instead, but overriding it seemed less error-prone.
Fixes#10119.
Fixes#10113.
Fix inequality in "stale mtime" log messages
### What does this PR try to resolve?
If `CARGO_LOG=cargo::core::compiler::fingerprint=info` is set, cargo will print log messages that are useful for finding out why something is being recompiled. E.g. if the modification time (mtime) of a source file is newer than the cached build result, it will print something like:
```
stale: changed "/host//home/runner/.cargo/registry/src/github.com-1285ae84e5963aae/proc-macro2-1.0.47/build.rs"
(vs) "/host/home/runner/target/release/build/proc-macro2-45f04ea9067a46ed/output"
FileTime { seconds: 1666559031, nanos: 16426033 } != FileTime { seconds: 1666559080, nanos: 324117075 }
```
However, the `!=` in the log message is misleading, as equality is not the [criterion that's actually used](071eeaf210/src/cargo/core/compiler/fingerprint.rs (L1761)) to determine when to rebuild.
This PR fixes that by changing `!=` to `<`, corresponding to the actual criterion.
### How should we test and review this PR?
Look at the [criterion](071eeaf210/src/cargo/core/compiler/fingerprint.rs (L1761)) I linked and trace `stale_mtime` in the log message back to `path_mtime` in said criterion to see why it has to be `<`.
