Commit graph

3388 commits

Author SHA1 Message Date
Eric Huss a2ed96df6a Fix self_signed_should_fail for macOS. 2023-05-06 12:41:34 -07:00
Eric Huss d6c20cf109 Fix redacting tokens in http debug. 2023-05-06 12:41:06 -07:00
bors e2d14882a5 Auto merge of #12083 - Eh2406:names, r=weihanglo
do not try an exponential number of package names

re #11934, and as discussed in the cargo team meeting, this changes the strategy to "the original, all underscore, and all dashes".

I was excessively proud of the `hyphen_combination_num` based implementation when I came up with it. But it's always been a hack. I'm glad to be the one to remove it.
2023-05-05 17:18:08 +00:00
Jacob Finkelman d6021c9c34 do not try an exponential number of package names 2023-05-05 17:15:06 +00:00
Weihang Lo 3df35edca7
fix: hack around libsysroot instead of libtest 2023-05-05 14:48:37 +01:00
bors 2d693e20ea Auto merge of #11917 - ehuss:rustup-circumvent, r=weihanglo
Optimize usage under rustup.

Closes #10986

This optimizes cargo when running under rustup to circumvent the rustup proxies. The rustup proxies introduce overhead that can make a noticeable difference.

The solution here is to identify if cargo would normally run `rustc` from PATH, and the current `rustc` in PATH points to something that looks like a rustup proxy (by comparing it to the `rustup` binary which is a hard-link to the proxy). If it detects this situation, then it looks for a binary in `$RUSTUP_HOME/toolchains/$TOOLCHAIN/bin/$TOOL`. If it finds the direct toolchain executable, then it uses that instead.

## Considerations

There have been some past attempts in the past to address this, but it has been a tricky problem to solve. This change has some risk because cargo is attempting to guess what the user and rustup wants, and it may guess wrong. Here are some considerations and risks for this:

* Setting `RUSTC` (as in https://github.com/rust-lang/rustup/pull/2958) isn't an option. This makes the `RUSTC` setting "sticky" through invocations of different toolchains, such as a cargo subcommand or build script which does something like `cargo +nightly build`.

* Changing `PATH` isn't an option, due to issues like https://github.com/rust-lang/rustup/issues/3036 where cargo subcommands would be unable to execute proxies (so things like `+toolchain` shorthands don't work).

* Setting other environment variables in rustup (as in https://github.com/rust-lang/rustup/pull/3207 which adds `RUSTUP_TOOLCHAIN_DIR` the path to the toolchain dir) comes with various complications, as there is risk that the environment variables could get out of sync with one another (like with `RUSTUP_TOOLCHAIN`), causing tools to break or become confused.

  There was some consideration in that PR for adding protections by using an encoded environment variable that could be cross-checked, but I have concerns about the complexity of the solution.

  We may want to go with this solution in the long run, but I would like to try a short term solution in this PR first to see how it turns out.

* This won't work for a `rustup-toolchain.toml` override with a [`path`](https://rust-lang.github.io/rustup/overrides.html#path) setting. Cargo will use the slow path in that case. In theory it could try to detect this situation, which may be an exercise for the future.

* Some build-scripts, proc-macros, or custom cargo subcommands may be doing unusual things that interfere with the assumptions made in this PR. For example, a custom subcommand could call a `cargo` executable that is not managed by rustup. Proc-macros may be executing cargo or rustc, assuming it will reach some particular toolchain. It can be difficult to predict what unusual ways cargo and rustc are being used. This PR (and its tests) tries to make extra sure that it is resilient even in unusual circumstances.

* The "dev" fallback in rustup can introduce some complications for some solutions to this problem. If a rustup toolchain does not have cargo, such as with a developer "toolchain link", then rustup will automatically call either the nightly, beta, or stable cargo if they are available. This PR should work correctly, since rustup sets the correct `RUSTUP_TOOLCHAIN` environment variable for the *actual* toolchain, not the one where cargo was executed from.

* Special care should be considered for dynamic linking. `LD_LIBRARY_PATH` (linux), `DYLD_LIBRARY_PATH` (macos), and `PATH` (windows) need to be carefully set so that `rustc` can find its shared libraries. Directly executing `rustc` has some risk that it will load the wrong shared libraries. There are some mitigations for this. macOS and Linux use rpath, and Windows looks in the same directory as `rustc.exe`. Also, rustup configures the dyld environment variables from the outer cargo. Finally, cargo also configures these (particularly for the deprecated compiler plugins).

* This shouldn't impact installations that don't use rustup.

* I've done a variety of testing on the big three platforms, but certainly nowhere exhaustive.
    * One of many examples is making sure Clippy's development environment works correctly, which has special requirements for dynamic linking.

* There is risk about future rustup versions changing some assumptions made here. Some assumptions:
    * It assumes that if `RUSTUP_TOOLCHAIN` is set, then the proxy *always* runs exactly that toolchain and no other. If this changes, cargo could execute the wrong version. Currently `RUSTUP_TOOLCHAIN` is the highest priority [toolchain override](https://rust-lang.github.io/rustup/overrides.html) and is fundamental to how toolchain selection becomes "sticky", so I think it is unlikely to change.
    * It assumes rustup sets `RUSTUP_TOOLCHAIN` to a value that is exactly equal to the name of the toolchain in the `toolchains` directory. This works for user shorthands like `RUSTUP_TOOLCHAIN=nightly`, which gets converted to the full toolchain name. However, it does not work for `path` overrides (see above).
    * It assumes the `toolchains` directory layout is always `$RUSTUP_HOME/toolchains/$TOOLCHAIN`. If this changes, then I think the only consequence is that cargo will go back to the slow path.
    * It assumes downloading toolchains is not needed (since cargo running from the toolchain means it should already be downloaded).
    * It assumes there is no other environment setup needed (such as the dyld paths mentioned above).

  My hope is that if assumptions are no longer valid that the worst case is that cargo falls back to the slow path of running the proxy from PATH.

## Performance

This change won't affect the performance on Windows because rustup currently alters PATH to point to the toolchain directory. However, https://github.com/rust-lang/rustup/pull/3178 is attempting to remove that, so this PR will be required to avoid a performance penalty on Windows. That change is currently opt-in, and will likely take a long while to roll out since it won't be released until after the next release, and may be difficult to get sufficient testing.

I have done some rough performance testing on macOS, Windows, and Linux on a variety of different kinds of projects with different commands. The following attempts to summarize what I saw.

The timings are going to be heavily dependent on the system and the project. These are the values I get on my systems, but will likely be very different for everyone else.

The Windows tests were performed with a custom build of rustup with https://github.com/rust-lang/rustup/pull/3178 applied and enabled (stock rustup shows no change in performance as explained above).

The data is summarized in this spreadsheet: https://docs.google.com/spreadsheets/d/1zSvU1fQ0uSELxv3VqWmegGBhbLR-8_KUkyIzCIk21X0/edit?usp=sharing

`hello-world` has a particularly large impact of about 1.68 to 2.7x faster. However, a large portion of this overhead is related to running `rustc` at the start to discover its version and querying it for information. This is cached after the first run, so except for first-time builds, the effect isn't as noticeable. The "check with info" row is an benchmark that removes `target/debug/deps` but keeps the `.rustc_info.json` file.

Incremental builds are a bit more difficult to construct since it requires customizing the commands for each project. I only did an incremental test for cargo itself, running `touch src/cargo/lib.rs` and then `cargo check --lib`.

These measurements excluded the initial overhead of launching the rustup proxy to launch the initial cargo process. This was done just for simplicity, but it makes the test a little less characteristic of a typical usage, which will have some constant overhead for running the proxy.

These tests were done using [`hyperfine`](https://crates.io/crates/hyperfine) version 1.16.1. The macOS system was an M2 Max (12-thread). The Windows and Linux experiments were run on a AMD Ryzen Threadripper 2950X (32-thread). Rust 1.68.2 was used for testing. I can share the commands if people want to see them.
2023-05-04 23:51:38 +00:00
Eric Huss d4b0b494f1 Add an optimization when running under rustup. 2023-05-03 13:45:54 -07:00
Eric Huss 52317aa8f8 Add some tests for simulating behavior under rustup. 2023-05-03 13:45:54 -07:00
charles-r-earp 61df6bb418 Fixed test update::update_precise_first_run. 2023-05-03 13:08:51 -07:00
charles-r-earp f43e6d947f Added workspace_default_members to more tests. 2023-05-03 12:44:08 -07:00
charles-r-earp e421345d22 Added workspace_default_members to tests. 2023-05-03 12:44:08 -07:00
charles-r-earp 8c96a28ff7 Added workspace_default_members. 2023-05-03 12:44:08 -07:00
bors 2f06c80bd0 Auto merge of #11840 - Byron:shallow-support, r=weihanglo
support for shallow clones and fetches with `gitoxide`

This PR makes it possible to enable shallow clones and fetches for git dependencies and crate indices independently with the `-Zgitoxide=fetch,shallow_deps` and `-Zgitoxide=fetch,shallow_index` respectively.

### Tasks

* [x] setup the shallow option when fetching, differentiated by 'registry' and 'git-dependency'
* [x] validate registries are cloned shallowly *and* fetched shallowly
* [x] validate git-dependencies are cloned shallowly *and* fetched shallowly
* [x] a test to show what happens if a shallow index is opened with `git2` (*it can open it and fetch like normal, no issues*)
* [x] assure that `git2` can safely operate on a shallow clone - we unshallow it beforehand, both for registries and git dependencies
* [x] assure git-deps with revisions are handled correctly (they should just not be shallow, and they should unshallow themselves if they are)
* [x] make sure shallow index clones aren't seen by older cargo's
* [x] make sure shallow git dependency clones aren't seen by older cargo's
* [x] shallow.lock test and more test-suite runs with shallow clones enabled for everything
* [x] release new version of `gix` with full shallow support and use it here
* [x] check why `shallow` files remain after unshallowing. Should they not rather be deleted if empty? - Yes, `git` does so as well, implemented [with this commit](2cd5054b0a)
* ~~see if it can be avoided to ever unshallow an existing `-shallow` clone by using the right location from the start. If not, test that we can go `shallow->unshallow->shallow` without a hitch.~~ Cannot happen anymore as it can predict the final location perfectly.
* [x] `Cargo.lock` files don't prevent shallow clones
* [x] assure all other tests work with shallow cloning enabled (or fix the ones that don't with regression protection)
* [x] can the 'split-brain' issue be solved for good?

### Review Notes

* there is a chance of 'split brain' in git-dependencies  as the logic for determining whether the clone/fetch is shallow is repeated in two places. This isn't the case for registries though.

### Notes

* I am highlighting that this is the `gitoxide` version of shallow clones as the `git2` version [might soon be available](https://github.com/libgit2/libgit2/pull/6396) as well. Having that would be good as it would ensure interoperability remains intact.
* Maybe for when `git2` has been phased out, i.e. everything else is working, I think (unscientifically) there might be benefits  in using worktrees for checkouts. Admittedly I don't know the history of why they weren't used in the first place. Also: `gitoxide` doesn't yet support local clones and might not have to if worktrees were used instead.
2023-05-03 07:12:20 +00:00
Weihang Lo 7fb35c9c4e
test(install): correct the term workspace -> local packagd 2023-05-03 01:04:18 +01:00
Sebastian Thiel 3bd08d934b
Move helper functions to the bottom for consistency; fix name 2023-05-02 18:57:42 +02:00
Ted Kaminski 79bb2d7143 Update pkg-spec comment, and add 2 more test cases 2023-05-01 17:23:17 -05:00
Ted Kaminski 31d679e81a Build by PackageIdSpec, not name, to avoid ambiguity 2023-05-01 17:23:17 -05:00
Sebastian Thiel 2aaebcbcaa
Only fetch single object in shallow mode for compatibility 2023-04-29 14:29:11 +02:00
Sebastian Thiel d4b8fc68ab
Allow fetching a single object, shallow or not.
Doing so seems cleaner as there should be no logical difference between
shallow or not-shallow when fetching. We want a specific object, and should
get it with the refspec. `git` will assure we see all objects we need,
handling shallow-ness for us.

Note that one test needed adjustments due to the different mechanism used
when fetching local repositories, requiring more changes to properly 'break'
the submodule repo when `gitoxide` is used.
2023-04-29 12:21:45 +02:00
Sebastian Thiel 8b3508cad7
Move all tests for shallow fetching into their own module
Note that those which test both with and without shallow still go
into the shallow module.
2023-04-29 09:52:08 +02:00
bors a285008c8e Auto merge of #12044 - jrose-signal:cargo-tree-no-proc-macro, r=epage
cargo-tree: Handle -e no-proc-macro when building the graph

### What does this PR try to resolve?

Makes `-e no-proc-macro` more useful when combined with `-i` or `-d`. Fixes #12030.

### How should we test and review this PR?

The new and existing tests should cover this, I hope!

### Additional information

Pruning proc-macro crates during graph construction is closer to how the edge-based filters work (`[no-]build` etc.), so even though `no-proc-macro` isn't technically filtering on edges, it's following a well-established code path.
2023-04-27 21:04:25 +00:00
Jordan Rose a0576d1767 cargo-tree: Handle -e no-proc-macro when building the graph
This is closer to how the edge-based filters work ([no-]build etc.),
and results in a more useful behavior when combined with -i or -d.
2023-04-27 10:54:51 -07:00
Jordan Rose 747a5924a0 test: Add a test for cargo tree --duplicates with a proc-macro 2023-04-27 10:49:22 -07:00
Sebastian Thiel 2f45cb54fa
Assure that git-dependencies will get the correct storage, without ever unshallowing them.
This is an improvement over the previous version which would use unshallowing that effectively
makes a shallow repo *not* shallow.

Furthermore, we will now only fetch a single commit, each time we fetch, which should be faster
for the server as well as for the client.

We also make it possible to fetch individual commits that would be specified via Cargo.lock.
2023-04-27 18:49:59 +02:00
Sebastian Thiel 4b93f095c8
Various improvements to address the PR review.
https://github.com/rust-lang/cargo/pull/11840#pullrequestreview-1383844802
2023-04-27 14:19:01 +02:00
Sebastian Thiel 38d92dd9a8
place shallow git dependencies into a different directory.
That way, we avoid any danger with older cargo's not being able
to handle such a repository correctly.
2023-04-27 14:18:27 +02:00
Sebastian Thiel d4af571686
name the shallow registry differently
A couple of test expectations are adjusted accordingly.

Is this desirable behaviour? Unfortunately, there is no alternative
as adding shallow to an existing index most definitely breaks backwards
compatibility.
2023-04-27 14:18:26 +02:00
Sebastian Thiel c7ff94fce8
Enable shallow clones and fetches for registry and git dependencies.
The implementation hinges on passing information about the kind of clone
and fetch to the `fetch()` method, which then configures the fetch accordingly.

Note that it doesn't differentiate between initial clones and fetches as
the shallow-ness of the repository is maintained nonetheless.
2023-04-27 14:18:22 +02:00
cassaundra 5554889f88
Include rust-version in publish request
crates.io reads rust-version from the tarball directly, but we can include it in
the publish request for the sake of consistency for third-party registries.
2023-04-26 11:59:29 -07:00
Kyle Matsuda 90c7d3b9b8 update the publish_with_missing_readme test with new warning 2023-04-25 12:27:39 -06:00
Kyle Matsuda 27e95997ce failing tests on empty readme and license-path fields 2023-04-25 12:27:39 -06:00
Weihang Lo 00484fe00a
fix: apply [env] to target info discovery rustc 2023-04-24 20:30:44 +01:00
Weihang Lo e1c7ce8903
test: [env] isn't applied to target info discovery rustc 2023-04-24 20:28:20 +01:00
bors b5177c6d46 Auto merge of #12001 - hi-rustin:rustin-patch-dep-empty, r=weihanglo
Improve error message for empty dep
2023-04-24 09:20:41 +00:00
bors 77069b43f0 Auto merge of #11958 - jyn514:named-debuginfo, r=Muscraft
Allow named debuginfo options in Cargo.toml
2023-04-22 15:08:08 +00:00
bors 738c699342 Auto merge of #11997 - hi-rustin:rustin-patch-dep-message, r=weihanglo
Better error message when getting an empty dep table
2023-04-20 09:16:30 +00:00
hi-rustin af0cd9fc5a Improve error message for empty dep
Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-04-20 09:32:24 +08:00
hi-rustin e0276cae19 Update error message to not start with capital letters
Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-04-20 09:16:45 +08:00
jyn a982bccfe8 add more debuginfo tests 2023-04-19 20:01:04 -05:00
hi-rustin 4d401bd0b9 Add test for empty dep table error
Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-04-20 08:49:30 +08:00
Eric Huss 6fa758e83c Fix flaky not_found_permutations test. 2023-04-14 08:01:34 -07:00
bors b0742b2145 Auto merge of #11882 - hi-rustin:rustin-patch-clippy-fix, r=weihanglo
Correct the bug report for `cargo clippy --fix`
2023-04-14 07:05:37 +00:00
hi-rustin 08169fd015 Add rustc_shim_for_cargo_fix and wrapped_clippy_driver to reuse code
Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-04-14 11:23:41 +08:00
hi-rustin 47f6e2ddc9 Add broken_clippy_fixes_backed_out
Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-04-14 10:46:53 +08:00
bors 84b7041fd2 Auto merge of #11950 - ehuss:stabilize-logout, r=epage
Stabilize `cargo logout`

This stabilizes the `cargo logout` command.

Closes #11910
Closes #11884
2023-04-13 20:08:40 +00:00
jyn 8a9db7a020 Address review comments
- Update the documentation and doc-comments
- Improve the error message for parsing Cargo.toml
2023-04-12 12:05:14 -05:00
Eric Huss a70c108961 Stabilize cargo logout 2023-04-12 09:56:43 -07:00
Eric Huss c00a633bd6 Don't display headers in spurious warning message.
The headers can significantly contribute to noise in the output,
drowning out the rest of the output. Most investigation will likely be
focused on the case where cargo completely fails to download, so this
only shows the full detail in the final error message.
2023-04-12 09:38:53 -07:00
Eric Huss 4702fa3ad3 Include the IP address in HTTP errors. 2023-04-12 09:38:53 -07:00
Eric Huss c7c9b8f32b Show some HTTP headers in error messages. 2023-04-12 09:38:52 -07:00
bors 96f8d6ced8 Auto merge of #11949 - ehuss:logout-default, r=epage
Use registry.default for login/logout

This changes `cargo login` and `cargo logout` to use the registry configured at `registry.default` as the registry instead of crates.io. For `cargo login`, this was an unintentional regression from #6466. The documentation has always stated that it will use the default registry.

This makes the command more in line with other registry-involving commands. There are still some inconsistencies.

These commands use the default if not specified:

* `cargo init`
* `cargo new`
* `cargo owner`
* `cargo search`
* `cargo yank`
* `cargo publish` uses the default, but will also look at the `publish` field `Cargo.toml` and use that if the registry is not specified.

These commands would always use crates.io if `--registry` is not specified:

* `cargo login`
* `cargo logout`
* `cargo install`

I'm a bit uncertain how to proceed, since this is technically a breaking change particularly if someone has scripted it. I suspect that the number of users that use `registry.default` is very small, and those that script `cargo login` are even smaller, and thus the intersection is probably small or nonexistent. However, there is some risk here.
2023-04-12 13:45:57 +00:00
Eric Huss 3da2b3c67b Mark -C as unstable 2023-04-11 19:46:05 -07:00
Joshua Nelson d861dcf792 Allow named debuginfo options in Cargo.toml
Rustc supports these since rust-lang/rust#109808. It's technically
possible to set a named debuginfo level through `RUSTFLAGS`, but in
practice cargo always passes its own opinion of what the debuginfo level
is, so allow it to be configured through cargo too.
2023-04-11 15:06:02 -05:00
Eric Huss a9e0b505d6 Update auth error message to specify args for cargo login. 2023-04-10 10:20:21 -07:00
bors c429784fd5 Auto merge of #11951 - ehuss:check-token-tab, r=hi-rustin
Fix credential token format validation.

The existing validation incorrectly excluded tab because of a missing backslash. This updates to add the backslash.

This also rewords the comments. I found the current comments to be a little confusing.

This also extends the test to verify more valid inputs.

cc #11600
2023-04-10 04:02:43 +00:00
Eric Huss 35f5862979 Validate token on publish. 2023-04-09 12:15:40 -07:00
Eric Huss 57264ed433 Fix credential token format validation. 2023-04-09 11:53:46 -07:00
Eric Huss 117eb4b33f Use registry.default for login/logout 2023-04-09 09:30:36 -07:00
Eric Huss 0e13f667c8 Add tests for registry.default for login/logout 2023-04-09 09:22:28 -07:00
Eric Huss 91d39bc742 Share the check_token function between login and logout tests. 2023-04-09 09:11:22 -07:00
Arlo Siemsen a888c94052 Stop using UncanonicalizedIter for QueryKind::Exact 2023-04-05 15:52:01 -05:00
Eric Huss 1ee340c0a7 Don't query permutations of the path prefix. 2023-04-04 18:49:42 -07:00
Arlo Siemsen a1cba8fe49 Fix Cargo warning about unused sparse configuration key
When doing a credential lookup, Cargo deserializes the registry configuration and detects the
registries.crates-io.protocol key as unused and issues a warning.

This fixes the issue by adding the field to the struct
2023-04-03 11:27:11 -05:00
Eric Huss 78970bd4a8 Update git2 2023-04-02 15:37:53 -07:00
bors 0e474cfd7b Auto merge of #11881 - ehuss:http-retry, r=epage
Add delays to network retries.

This adds a delay to network retries to help guard against short-lived transient errors.

The overview of how this works is: Requests that fail are placed into a queue with a timestamp of when they should be retried. The download code then checks for any requests that need to be retried, and re-injects them back into curl.

Care must be taken in the download loops to track which requests are currently in flight *plus* which requests are waiting to be retried.

This also adds jitter to the retry times so that multiple failed requests don't all flood the server when they are retried. This is a very primitive form of jitter which suffers from clumping, but I think it should be fine.

The max number of retries is raised to 3 in order to bring the total retry time to around 10 seconds. This is intended to address Cloudfront's default negative TTL of 10 seconds. The retry schedule looks like 0.5seconds ± 1 second, then 3.5 seconds then 6.5 seconds, for a total of 10.5 to 11.5 seconds.  If the user configures additional retries, each attempt afterwards has a max delay of 10 seconds.

The retry times are currently not user-configurable. This could potentially be added in the future, but I don't think is particularly necessary for now.

There is an unfortunate amount of duplication between the crate download and HTTP index code. I think in the near future we should work on consolidating that code. That might be challenging, since their use patterns are different, but I think it is feasible.
2023-03-31 23:15:58 +00:00
Eric Huss 6bd1209a55 Add delays to network retries. 2023-03-31 14:04:48 -07:00
Eric Huss f393f96d7f Add a note to cargo logout that it does not revoke the token. 2023-03-31 08:03:51 -07:00
Eric Huss 5860cd23a2 Disable test_profile test on windows-gnu 2023-03-30 19:18:10 -07:00
Ed Page 6feea34f8a chore: Upgrade to clap v4.2
Tests in `master` are currently failing because its building with clap
v4.2 but the tests have snapshots from v4.1
2023-03-28 05:00:49 -05:00
bors c9faf70061 Auto merge of #11824 - kylematsuda:windows-config-env-fix, r=ehuss
Handle case mismatches when looking up env vars in the Config snapshot

### What does this PR try to resolve?

Fixes #11814.

Windows environment variables are case-insensitive, which causes problems when looking them up in the `Config` env snapshot.

This PR adds another member (`case_insensitive_env`) in `Config` that maps upper-cased keys to their original values in the env (for example, `"PATH" => "Path"`). If lookup in `self.env` fails, this PR converts the key to upper case and looks it up in `self.case_insensitive_env` to obtain the correct key name if it exists (on Windows only).

### How should we test and review this PR?

Please see the new tests in `testsuite/config.rs` and `testsuite/cargo_command.rs`.

### Additional information

Currently, this uses `str::to_uppercase` to uppercase the keys. This requires key to be valid UTF-8, and may disagree with how the OS uppercases things (see the link in [this comment](https://github.com/rust-lang/cargo/issues/11814#issuecomment-1462870983) for details).
2023-03-17 01:06:49 +00:00
bors 94394ebf2a Auto merge of #11855 - robjtede:target-ignore, r=weihanglo
align semantics of generated vcs ignore files

### What does this PR try to resolve?

The currently generated `^target/` spec in a hg ignore will only ignore dirs of that name at the root.

This change matches the behavior of the gitignore spec next to it, by only ignoring both files/symlinks and dirs of name "target".

### How should we test and review this PR?

Run `cargo new`
2023-03-15 17:28:40 +00:00
Rob Ede 8183c31eec
chore: fix target in generated hg ignore file 2023-03-15 16:39:24 +00:00
Eric Huss 3c295cf5ec Update tests for publish text changes 2023-03-15 08:50:11 -07:00
Eric Huss 7b317f3624 Remove "up to 60 seconds" from waiting message.
This level of detail isn't particularly important to the user and
just clutters up the output.
2023-03-15 08:15:00 -07:00
Eric Huss f60666ca6e Reword published/completed to uploaded/published 2023-03-15 08:15:00 -07:00
Eric Huss 7e4764a56b Add more information to wait-for-publish 2023-03-15 08:15:00 -07:00
Eric Huss 7b19a6e8ef Add some more publish timeout tests 2023-03-15 08:15:00 -07:00
bors 9282cf74a3 Auto merge of #11839 - djc:downgrading-status, r=weihanglo
Accurately show status when downgrading dependencies

### What does this PR try to resolve?

A few times now I've ran into issues where Cargo ends up downgrading a dependency in order to satisfy a pinned dependency somewhere in the dependency graph. Unfortunately this is not clear in the output of `cargo update`, which just shows all the changed dependencies as `Updating`.

References:

* #5702
* [Finding a pinned dependency edge](https://users.rust-lang.org/t/finding-a-pinned-dependency-edge/81157)
* https://github.com/tokio-rs/axum/issues/1814

### How should we test and review this PR?

This is a small change that tries to make dependency downgrades stand out more in the output of `cargo update`. I have not added any new tests since the existing tests seem to cover this functionality.

Some tests still fail, these refer to Git dependencies. I'm honestly not sure how to handle these, so I'd like to get some feedback on this before I fix those tests up. Git commits form a DAG, so one option is to see if the new commit is an ancestor of the old one (mark as "updating"), if the old commit is an ancestor of the new one (mark as "downgrading"), or neither (could be from parallel branches) we could compare based on timestamp in that case.

### To do

* Fix up tests for Git dependency updates
2023-03-14 01:01:48 +00:00
Dirkjan Ochtman 20a5d2b82f Accurately show status when downgrading dependencies 2023-03-13 13:29:30 +01:00
Dom Slee 3363931f7f redundant escapes 2023-03-13 18:57:26 +11:00
Dom Slee ea27499a2e review comments 2023-03-13 17:54:45 +11:00
Dom Slee dc449bce6d cargo clippy + fmt 2023-03-12 23:52:45 +11:00
Dom Slee a668956dd7 improve error message and add related tests 2023-03-12 23:52:06 +11:00
Sebastian Thiel c890c64080
Disable flaky tests for now (#11821)
The proper fix is in https://github.com/Byron/gitoxide/releases/tag/gix-v0.41.0
which unfortunately can't be used as it also comes with the latest `tempfile` v3.4
which causes other issues when compiling on some platforms.

Thus we first disable the flaky tests, and re-enable them with the `gix` upgrade
which should be possible once `tempfile` doesn't hinder `cargo` on some platforms
anymore.

Related to https://github.com/rust-lang/cargo/issues/11821
2023-03-10 19:45:22 +01:00
Kyle Matsuda 8353396132 add unit test for config and integration test with cargo --list 2023-03-09 16:54:01 -07:00
Weihang Lo f3778f9193
Revert "#11738" - Use test name for dir when running tests
This reverts commit 64b0e793ce, reversing
changes made to 958078633e.
2023-03-08 15:17:48 +00:00
bors c1334b059c Auto merge of #11645 - chansuke:issue-11597, r=weihanglo
Add `CARGO_PKG_README`

Fixes #11597

This environment variable shows the path to the README file of your package. From #11597:

> Cargo may rewrite the package’s `Cargo.toml` and move the README file around, relative to the manifest. I would like to `include_str!()` this README in my `lib.rs`, but am unable to do so right now, because if I specify `include_str!("../../README")` it works for development, but I can’t package my crate. Conversely if I specify `include_str!("../README")` it works when packaged, but not during development.
2023-03-07 19:21:50 +00:00
bors 66789e531a Auto merge of #11636 - aortizpimentel:fix/10834, r=weihanglo
Adding display of which target failed to compile

Closes #10834

Attached example.zip is the same as the one mentioned on the issue. You could use it to test the fix by using the new built cargo:
```./cargo build --manifest-path C:\Rust\example_cargo_error\Cargo.toml```

Before fixing:
```
error: could not compile `example` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
```

After fixing:
```
error: could not compile `example` (build script)  due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
```

[example.zip](https://github.com/rust-lang/cargo/files/10522342/example.zip)
2023-03-07 07:59:14 +00:00
Adrián Ortiz db457d950f Fixing issue https://github.com/rust-lang/cargo/issues/10834:
- Adding display of which target failed to compile
- Consistent messages for warnings/errors.
- Fixing assertions on related tests.
2023-03-07 06:45:26 +01:00
bors 87a997ab4a Auto merge of #11790 - vadorovsky:fix-cfg-target-env, r=arlosi
Fix `CARGO_CFG_` vars for configs defined both with and without value

When a rustc cfg is defined both with and without value, the environment variable should provide all the values. Before this change, it ended up being empty.

Fixes: #11789
2023-03-06 16:47:35 +00:00
jofas feef14e9d7 breaking endless loop on cyclic features in cargo-add 2023-03-06 13:04:38 +01:00
Michal Rostecki ccd991547b Fix CARGO_CFG_ vars for configs defined both with and without value
When a rustc cfg is defined both with and without value, the
environment variable should provide all the values. Before this change,
it ended up being empty.

Fixes: #11789
2023-03-06 10:30:47 +01:00
bors 234d9f6b86 Auto merge of #11791 - arlosi:sparse-stable, r=ehuss
Make `sparse` the default protocol for crates.io

Changes the default protocol for accessing crates.io to `sparse`.

The protocol can be switched back to `git` via the configuration key `registries.crates-io.protocol` = `git`

Closes #10965
2023-03-05 23:10:49 +00:00
hi-rustin 814ded6ae9 Check publish_to_alt_registry publish content 2023-03-04 09:11:37 +08:00
chansuke fb9be455cd Add tests for CARGO_PKG_README 2023-03-03 23:21:16 +09:00
Arlo Siemsen 78d4f2cb84 Make sparse the default protocol for crates.io 2023-03-02 10:50:22 -06:00
Sebastian Thiel cfffda9ae5
add -Zgitoxide=fetch feature toggle and implementation.
This allows to use `gitoxide` for all fetch operations, boosting performance
for fetching the `crates.io` index by a factor of 2.2x, while being consistent
on all platforms.

For trying it, nightly builds of `cargo` can specify `-Zgitoxide=fetch`.
It's also possible to set the `__CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2=1` environment
variable (value matters), which is when `-Zgitoxide=none` can be used
to use `git2` instead.

Limitations
-----------
Note that what follows are current shortcomings that will be addressed in future PRs.

- it's likely that authentication around the `ssh` protocol will work differently in practice
  as it uses the `ssh` program.
- clones from `file://` based crates indices will need the `git` binary to serve the locatl repository.
- the progress bar shown when fetching doesn't work like the orgiinal, but should already feel 'faster'.
2023-03-02 12:35:50 +01:00
bors c61b0f0680 Auto merge of #11770 - Eh2406:non-active-conflict, r=weihanglo
patch can conflict on not activated packages

### What does this PR try to resolve?

In the resolver there is a data structure called a `conflicting_activations`, which records all the reasons a "better" version was not picked for the package being resolved. Normally, these are packages that are currently active that for one reason or another block one of the other versions. Several optimizations assumed that this was always true, even going so far as to use `.expect`. This was a mistake because when there's a `patch` involved there can be conflicts on a version that is not selected. So the correct behavior is to fall back to skip the optimizations and try all versions when a `conflicting_activations` are not active.

### How should we test and review this PR?

This adds two automated tests based on the reproductions in #7463 and #11336. So the test suite now covers this case. It can also be tested by reconstructing the repositories originally reported in those issues.

### Additional information

It could be that in the long term the correct fix is to figure out how to support patch without having a conflicting activation that is not activated. But that would be a much bigger change. And for now this assumption is only used in optimizations, so this gets people unstuck.
2023-03-02 06:49:44 +00:00
Jacob Finkelman 2c712d5d46 patch can conflict on not activated packages 2023-03-01 20:56:00 -05:00
bors 0942fc72d6 Auto merge of #11630 - Muscraft:fix-unused-manifest-keys, r=ehuss
fix(toml): Provide a way to show unused manifest keys for dependencies

Dependencies have not been able to show unused manifest keys for some time, this problem partially resulted in #11329.

This problem is caused by having an `enum` when deserializing.  To get around this you can use:
```rust
#[serde(flatten)]
other: BTreeMap<String, toml::Value>,
```
This collects any unused keys into `other` that can later be used to show warnings. This idea was suggested in a thread I cannot find but is mentioned in [serde#941](https://github.com/serde-rs/serde/issues/941).
2023-03-01 22:20:41 +00:00
bors dbb2d67b0c Auto merge of #11783 - arlosi:sparse-offline, r=ehuss
Improve error for missing crate in --offline mode for sparse index

This changes sparse registries to instead return `NotFound` when a non-cached crate is requested in `--offline` mode.

The resolver can then suggest removing the `--offline` flag if resolution fails, which is a more helpful error than the one currently issued: `attempting to make an HTTP request, but --offline was specified`.

With this change, the behavior matches what is already done for git-based registries.

Closes #11276
2023-03-01 21:30:50 +00:00
Arlo Siemsen 65e449ecda Improve error for missing crate in --offline mode for sparse index
This changes sparse registries to instead return not found when a non-cached crate is requested in offline mode.

The resolver can then suggest removing the --offline flag if resolution
fails.
2023-03-01 14:45:09 -06:00
Scott Schafer a32af2fff1 fix(toml): Provide a way to show unused manifest keys for workspace inheritance 2023-03-01 13:43:57 -06:00
bors 80f1a5d0f7 Auto merge of #11688 - epage:minimal, r=Eh2406
feat(resolver): `-Zdirect-minimal-versions`

This is an alternative to `-Zminimal-versions` as discussed in #5657.

Problems with `-Zminimal-versions` includes
- Requires the root most dependencies to verify it and we then percolate that up the stack.  This requires a massive level of cooperation to accomplish and so far there have been mixed results with it to the point that cargo's unstable
 documentation discourages its use.
- Users expect `cargo check -Zminimal-versions` to force resolving to minimal but it doesn't as the default maximal resolve is compatible and requires `cargo update -Zminimal-versions`
- Different compatible versions might be selected, breaking interop between crates, changing feature unification, and breaking `-sys` crates without bad `links`

`-Zdirect-minimal-versions` instead only applies this rule to your
direct dependencies, allowing anyone in the stack to immediately adopt
it, independent of everyone else.

Special notes
- Living up to the name and the existing design, this ignores yanked
  crates.  This makes sense for `^1.1` version requirements but might
  look weird for `^1.1.1` version requirements as it could select
  `1.1.2`.
- This will error if an indirect dependency requires a newer version.
  Your version requirement will need to capture what you use **and** all
  of you dependencies.  An alternative design would have tried to merge
  the result of minimum versions for direct dependencies and maximum
  versions for indirect dependencies.  This would have been complex and
  led to weird corner cases, making it harder to predict.  I also suspect
  the value gained would be relatively low as you can't verify that
  version requirement in any other way.
  - This also means discrepancies between `dependencies` and `dev-dependencies` are errors
  - The error could be improved to call out that this was from minimal
    versions but I felt getting this out now and starting to collect
    feedback was more important.

One advantage of this approach over `-Zminimal-versions` is that it removes most of the problems that [cargo-minimal-versions](https://github.com/taiki-e/cargo-minimal-versions) tried to workaround.

As for the implementation, this might not be the most elegant solution but it works and we can always iterate and improve on it in the future.
- We keep the state as a `bool` throughout but compensate for that by explicitly creating a variable to abstract away constants
- The name changes depending on the context, from `direct_minimal_version` when dealing with the unstable flag to `first_minimal_version` when the concept of "direct" is lost to `first_version` when we split off the ordering concept into a separate variable
- Packages that respect `direct_minimal_versions` are determined by whether they are the top-level `summaries` that get past into `resolve`

### What does this PR try to resolve?

The primary use case is verifying version requirements to avoid depending on something newer than might be available in a dependent

For this to help the MSRV use case, the crate author must directly depend on all indirect dependencies where their latest release has too new of an MSRV but at least they can do so with the `^` operator, rather than `<` and breaking the ecosystem.

### How should we test and review this PR?

The first two commits add tests using `-Zminimal-versions`.  The commit that adds `-Zdirect-minimal-versions` updates the tests, highlighting the differences in behavior.

### Additional information

Potential areas of conversation for stablization
- Flag name
- Handling of yanked (pick first non-yanked, pick yanked, error)
- Quality of error message
- Should the package have a "memory" of this flag being set by writing it to the lockfile?

Potential future work
- Stablize this
- Remove `-Zminimal-versions`
- Update `cargo publish`s `--verify` step to use this.
  - The challenge is this won't be using the packaged `Cargo.lock` which probably should also be verified.
2023-03-01 19:26:59 +00:00
Scott Schafer 019aeedeb4 feat: Use test name for dir when running tests 2023-03-01 11:38:58 -06:00
bors 9880b408a3 Auto merge of #11767 - weihanglo:jobserver-style-fifo, r=ehuss
bump jobserver to respect `--jobserver-auth=fifo:PATH`
2023-02-28 19:39:39 +00:00
Weihang Lo b9bfda596f
chore: bump jobserver to respect --jobserver-auth=fifo:PATH'
See https://github.com/alexcrichton/jobserver-rs/pull/49
2023-02-28 16:08:12 +00:00
Eric Huss ab726e7e7f Fix test for Windows 2023-02-25 18:07:49 -08:00
Arlo Siemsen 16ad1f2945 Fix Cargo removing the sparse+ prefix from sparse URLs in .crates.toml 2023-02-25 18:07:46 -08:00
Eric Huss 736fb2f62a Fix warning with tempfile 2023-02-25 15:35:12 -08:00
bors 524231f43f Auto merge of #11643 - jofas:11260-fix, r=weihanglo
Error message for transitive artifact dependencies with targets the package doesn't directly interact with

Address #11260. Produces an error message like described by `@weihanglo` [here](https://github.com/rust-lang/cargo/issues/11260#issuecomment-1400455203):

```
error: could not find specification for target "x86_64-windows-msvc"
  Dependency `bar v0.1.0` requires to build for target "x86_64-windows-msvc".
```

Note that this is not a complete fix for #11260.
2023-02-25 07:39:08 +00:00
Eric Huss 1c4651e5f3 Fix tests with nondeterministic ordering 2023-02-24 17:28:28 -08:00
bors 65cab34dc7 Auto merge of #11650 - hi-rustin:rustin-patch-tests, r=epage
Make some blocking tests non-blocking

Signed-off-by: hi-rustin <rustin.liu@gmail.com>
2023-02-24 03:29:06 +00:00
bors 0625b29d84 Auto merge of #11410 - TrAyZeN:master, r=epage
Suggest cargo add when installing library crate

### What does this PR try to resolve?

When using `cargo install` instead of `cargo use` the error message is the following:
```
error: there is nothing to install in `foo v0.0.1`, because it has no binaries
`cargo install` is only for installing programs, and can't be used with libraries.
To use a library crate, add it as a dependency in a Cargo project instead.
```
It would be good to suggest to the user to use `cargo add`.

### How should we test and review this PR?

The `no_binaries` test from `tests/testsuite/install.rs` covers that case.
2023-02-23 18:44:31 +00:00
hi-rustin 5751e17c9b Change the download output order 2023-02-23 09:11:52 +08:00
hi-rustin 0b06a456f2 Make blocking tests non blocking 2023-02-23 09:11:52 +08:00
hi-rustin a8233d4df5 Support store public request body in the HTTP mock server 2023-02-23 09:11:52 +08:00
hi-rustin 04d592c7ad Make some blocking tests non-blocking by using API server 2023-02-23 09:11:52 +08:00
bors 0c331721d9 Auto merge of #11725 - Muscraft:reduce-build-in-tests, r=ehuss
Switch some tests from `build` to `check`

#11341 brought up issues with cargo's `testsute` size and speed. One of the suggested fixes was switching most tests to `check` instead of `build`. This PR did that.

Before size on `nightly`: 4.4GB

After size on `nightly`: 4.2GB

Regex to find `build` in `tests/testsuite`: `cargo\(".*build.*\)`
Before: 1607
After: 626

Note I did not remove all `build` I only did the easy ones that required minimal changes. I also tried not to touch systems I was unsure about. There could be other uses of `build` I missed as well.

I still need to play around with `opt-level`, `debug=0`, and a few other tweaks, but there should be more time/memory to drop.

Each test file changed is in a commit of its own, so you should look commit by commit.
2023-02-21 23:27:36 +00:00
Scott Schafer 45c9c8e905 chore: update workspaces tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer 221938a2a0 chore: update vendor tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer b974cacb50 chore: update update tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer 866daf4f22 chore: update rustflags tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer 43f9b8ea83 chore: update rust_version tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer 6282282fb0 chore: update replace tests to use check 2023-02-20 12:22:29 -06:00
Scott Schafer ab1794bda4 chore: update registry_auth tests to use check 2023-02-20 12:22:24 -06:00
Scott Schafer 98c746629b chore: update registry tests to use check 2023-02-20 12:22:24 -06:00
Scott Schafer 1b9af485c1 chore: update pub_priv tests to use check 2023-02-20 12:22:24 -06:00
Scott Schafer 41738bea7e chore: update progress tests to use check 2023-02-20 12:22:24 -06:00
Scott Schafer 8d66a0265d chore: update profile_overrides tests to use check 2023-02-20 12:21:55 -06:00
Scott Schafer a726b8d682 chore: update proc_macro tests to use check 2023-02-20 12:21:35 -06:00
Scott Schafer 341c400487 chore: update paths tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 577594770e chore: update path tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 730e2196b7 chore: update patch tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 7b14e72cc6 chore: update package_features tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer de2e97403f chore: update package tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer e3d3bb6b04 chore: update offline tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer fb755a3dd1 chore: update net_config tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 1dd8536935 chore: update metabuild tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 8f4f77e894 chore: update message_format tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 9a46caf1ed chore: update lockfile_compat tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer de704f5093 chore: update local_registry tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 9d85c01994 chore: update jobserver tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer df909b938c chore: update inheritable_workspace_fields tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer e8f9ccf9af chore: update git_gc tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer cb20cbaae9 chore: update git_auth tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 29cd1ce7f2 chore: update git tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 4934dffa5a chore: update future_incompat_report tests to use check 2023-02-20 12:21:27 -06:00
Scott Schafer 9c14193764 chore: update fix tests to use check 2023-02-20 12:21:15 -06:00
Scott Schafer a03660d5ee chore: update features_namespaced tests to use check 2023-02-20 12:21:15 -06:00