Retry command invocation with argfile
### What does this PR try to resolve?
Fixes#10381
The goal is to invoke `rustc` and `rustdoc` with ``@path`` arg file as a fallback.
Since the `-Zcheck-cfg-features`[^1] has already been implemented, in the foreseeable future cargo will probably hit the “**command line too big**” more often on Windows. `rustdoc` and `rustc` both support ``@path`` argfile [^2][^3], so we can leverage the feature for the fallback logic.
The idea behind this PR is that we put the **retry logic** in `ProcessBuilder::exec*` functions and reuse them as much as possible.
### How should we test and review this PR?
Review it commit by commit is recommended.
Argfile fallback is hard to tested with integration tests, so I added some unit tests for `cargo_util::ProcessBuilder` and `cargo::ops::fix::FixArgs`.
If you do want to test the support of argfile manually, a new environment variable `__CARGO_TEST_FORCE_ARGFILE` is added for the sake of forcing cargo to use argfile for rustc invocations. For example, `__CARGO_TEST_FORCE_ARGFILE cargo test --test testsuite -- fix::` is usually a good start to test this feature.
[^1]: https://doc.rust-lang.org/beta/cargo/reference/unstable.html?highlight=check#check-cfg-features
[^2]: https://doc.rust-lang.org/rustc/command-line-arguments.html#path-load-command-line-flags-from-a-path
[^3]: https://doc.rust-lang.org/rustdoc/command-line-arguments.html#path-load-command-line-flags-from-a-path
Part 3 of RFC2906 - Add support for inheriting `license-path`, and `depednency.path`
Tracking issue: #8415
RFC: rust-lang/rfcs#2906
[Part 1](https://github.com/rust-lang/cargo/pull/10497)
[Part 2](https://github.com/rust-lang/cargo/pull/10517)
This PR focuses on adding support for inheriting `license-path`, and `depednency.path`:
- To adjust the relative paths from being workspace-relative to package-relative, we use `pathdiff` which `cargo-add` is also going to be using for a similar purpose
- `ws_path` was added to `InheritableFields` so we can resolve relative paths from workspace-relative to package-relative
- Moved `resolve` for toml dependencies from `TomlDependency::<P>` to `TomlDependency`
- This was done since resolving a relative path should be a string
- You should never inherit from a `.cargo/config.toml` which is the reason `P` was added
Remaining implementation work for the RFC
- Relative paths for `readme`
- Path dependencies infer version directive
- Lock workspace dependencies and warn when unused
- Optimizations, as needed
- Evaluate any new fields for being inheritable (e.g. `rust-version`)
Update git2 dependencies
This pull-request update git2 to 0.14.1 and git2-curl to 0.15.0 and libgit2-sys to 0.13.1.
This fix a memory corruption that I have locally when running the testsuite:
```
==2338650== Uninitialised value was created by a stack allocation
==2338650== at 0x11FE3A0: git2::remote::Remote::fetch (remote.rs:276)
```
`std:🧵:available_parallelism` has been stabilized since 1.59.0.
Also, we don't want to block timing data output, so if parallelism
data is not available the table will display `ncpu=n/a` instead.
`toml_edit` fixed a bug in 0.13.4 that this test was relying on that is
meant to help with rust-lang/cargo#10349. This basically restores us
back to the pre-toml_edit behavior for published manifests.
Port cargo from toml-rs to toml_edit
Benefits:
- A TOML 1.0 compliant parser
- Unblock future work
- Have `cargo init` add the current crate to the workspace, rather
than error
- #5586: Upstream `cargo-add`
TODO
- [x] Analyze performance and address regressions
- [x] Identify and resolve incompatibiies
- [x] Resolve remaining test failures, see
https://github.com/ordian/toml_edit/labels/cargo
- [x] ~~Switch the code from https://github.com/rust-lang/cargo/pull/10176 to only parse once~~ (this PR is being merged first)
Benefits:
- A TOML 1.0 compliant parser
- Unblock future work
- Have `cargo init` add the current crate to the workspace, rather
than error
- #5586: Upstream `cargo-add`
Stabilize namespaced and weak dependency features.
This stabilizes the namespaced and weak dependency features. Support is now enabled on crates.io, so this should be ready to go.
As a part of this change, the new feature resolver is now enabled all of the time. This is fairly risky, since there are likely edge cases that haven't been exercised.
NOTE: Projects using `resolver="1"` *should* continue to have the same behavior, the old resolver behavior is emulated.
Closes#8813Closes#8832
- One parser change found by `cargo_config::includes` is that clap 2
would ignore any values after a `=` for flags.
`cargo config --show-origin` is a flag but the test passed `--show-origin=yes` which
happens to give the desired result for that test but is the same as
`--show-origin=no` or `--show-origin=alien-invasion`.
- The parser now panics when accessing an undefined attribute but clap
takes advantage of that for sharing code across commands that have
different subsets of arguments defined. I've extended clap so we can
"look before you leap" and put the checks at the argument calls to
start off with so its very clear what is tenuously shared. This
allows us to go in either direction in the future, either addressing
how we are sharing between commands or by moving this down into the
extension methods and pretending this clap feature doesn't exist
- On that topic, a test found clap-rs/clap#3263. For now, there is a
hack in clap. Depending on how we fix that in clap for clap 4.0, we
might need to re-address things in cargo.
- `value_of_os` now requires setting `allow_invalid_utf8`, otherwise it
asserts. To help catch this, I updated the argument definitions
associated with lookups reported by:
- `rg 'values?_os' src/`
- `rg 'values?_of_os' src/`
- clap now reports `2` for usage errors, so we had to bypass clap's
`exit` call to keep the same exit code.
BREAKING CHANGE: API now uses clap3
Update cargo-platform to 0.1.2
This preps cargo-platform for a release. The only substantial change is that this should include the license files into the archive that is uploaded to crates.io.
Closes#9758
This preps cargo-platform for a release. The only substantial change is that
this should include the license files into the archive that is uploaded to
crates.io.
Closes#9758
Update opener requirement from 0.4 to 0.5
Updates the requirements on [opener](https://github.com/Seeker14491/opener) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/Seeker14491/opener/blob/master/CHANGELOG.md">opener's changelog</a>.</em></p>
<blockquote>
<h2>[0.5.0] - 2021-06-11</h2>
<h3>Added</h3>
<ul>
<li><code>open_browser()</code>, which uses the <code>$BROWSER</code> environment variable before falling back to <code>open()</code>.</li>
<li>WSL-specific implementation. Previously, WSL used the same implementation as Linux. Now the strategy on WSL is to use
the system's <code>wslview</code> command from <a href="https://github.com/wslutilities/wslu"><code>wslu</code></a> if available, falling back to the
system <code>xdg-open</code>, if available.</li>
</ul>
<h3>Changed</h3>
<ul>
<li>On Linux (non-WSL), the system <code>xdg-open</code> is now used if present. Otherwise, the bundled version is used, as before.</li>
<li>Avoid blocking the thread on Linux and WSL.</li>
</ul>
<h3>Removed</h3>
<ul>
<li><code>impl From<io::Error> for OpenError</code>.</li>
</ul>
<h2>[0.4.1] - 2019-09-30</h2>
<h3>Changed</h3>
<ul>
<li>Update <code>xdg-open</code>.</li>
</ul>
<h2>[0.4.0] - 2019-05-02</h2>
<h3>Added</h3>
<ul>
<li><code>OpenError</code> now implements <code>std::error::Error</code>.</li>
</ul>
<h3>Changed</h3>
<ul>
<li><code>OpenError</code>'s <code>failure::Fail</code> impl was removed from this crate, but the failure crate provides a blanket impl of
<code>failure::Fail</code> for types implementing <code>std::error::Error</code>, so this shouldn't break anything.</li>
</ul>
<h2>[0.3.0] - 2018-08-18</h2>
<h3>Added</h3>
<ul>
<li><code>stderr</code> field to <code>OpenError::ExitStatus</code> variant, which captures anything the failed process wrote to stderr.</li>
</ul>
<h2>[0.2.0] - 2018-08-08</h2>
<h3>Removed</h3>
<ul>
<li>The <code>open_sys</code> function, which was erroneously pub on non-Windows builds.</li>
</ul>
<h2>[0.1.0] - 2018-08-08</h2>
<ul>
<li>Initial release.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="86e0a154d2"><code>86e0a15</code></a> Fix Drakefile.ts</li>
<li><a href="4261c2be99"><code>4261c2b</code></a> Release v0.5.0</li>
<li><a href="5d24271a65"><code>5d24271</code></a> Revert command name in error type to <code>&'static str</code></li>
<li><a href="9a36c6e656"><code>9a36c6e</code></a> Remove dependency on <code>wsl</code> crate</li>
<li><a href="165b933775"><code>165b933</code></a> Avoid blocking</li>
<li><a href="5c1e99a1c1"><code>5c1e99a</code></a> Update changelog</li>
<li><a href="2da976eb48"><code>2da976e</code></a> Refactor <code>linux_and_more</code> module</li>
<li><a href="f5c5564a9c"><code>f5c5564</code></a> Fix newline being included after WSL path conversion</li>
<li><a href="bd30011d61"><code>bd30011</code></a> [WSL] Try <code>wslview</code> first instead of second</li>
<li><a href="e16875d797"><code>e16875d</code></a> Misc formatting changes</li>
<li>Additional commits viewable in <a href="https://github.com/Seeker14491/opener/compare/v0.4.1...v0.5.0">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Update to semver 1.0.0
I am working on a 1.0.0 of the `semver` crate some time this week. It would be good to confirm Cargo will be able to use it, beforehand!
It's a from-scratch rewrite, but https://github.com/dtolnay/semver/issues/237 has code to compare against 0.10.0 (currently used by Cargo) how every possible version requirement currently published to crates.io matches against every possible crate version. The differences are all broken syntax like `^0-.11.0` previously parsing with ".11.0" as a pre-release string (which is invalid, because pre-release are not allowed to contain empty dot-separated identifiers) and `~2.0-2.2` previously parsing with "2.2" as a pre-release string, when the user almost certainly meant `>=2.0, <=2.2`. I'm not sure how much of those you want to add code into Cargo to preserve behavior, but I would be happy to do it.
cc rust-lang/rust#71249
This implements the Cargo side of 'Cargo report future-incompat'
Based on feedback from alexcrichton and est31, I'm implemented this a
flag `--future-compat-report` on `cargo check/build/rustc`, rather than
a separate `cargo describe-future-incompatibilities` command. This
allows us to avoid writing additional information to disk (beyond the
pre-existing recording of rustc command outputs).
This PR contains:
* Gating of all functionality behind `-Z report-future-incompat`.
Without this flag, all user output is unchanged.
* Passing `-Z emit-future-incompat-report` to rustc when
`-Z report-future-incompat` is enabled
* Parsing the rustc JSON future incompat report, and displaying it
it a user-readable format.
* Emitting a warning at the end of a build if any crates had
future-incompat reports
* A `--future-incompat-report` flag, which shows the full report for
each affected crate.
* Tests for all of the above.
At the moment, we can use the `array_into_iter` to write a test.
However, we might eventually get to a point where rustc is not currently
emitting future-incompat reports for any lints. What would we want the
cargo tests to do in this situation?
This functionality didn't require any significant internal changes to
Cargo, with one exception: we now process captured command output for
all units, not just ones where we want to display warnings. This may
result in a slightly longer time to run `cargo build/check/rustc` from
a full cache. since we do slightly more work for each upstream
dependency. Doing this seems unavoidable with the current architecture,
since we need to process captured command outputs to detect
any future-incompat-report messages that were emitted.
Update crossbeam-utils requirement from 0.7 to 0.8
Updates the requirements on [crossbeam-utils](https://github.com/crossbeam-rs/crossbeam) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md">crossbeam-utils's changelog</a>.</em></p>
<blockquote>
<h1>Version 0.8.0</h1>
<ul>
<li>Bump the minimum supported Rust version to 1.36.</li>
<li>Bump <code>crossbeam-channel</code> to <code>0.5</code>.</li>
<li>Bump <code>crossbeam-deque</code> to <code>0.8</code>.</li>
<li>Bump <code>crossbeam-epoch</code> to <code>0.9</code>.</li>
<li>Bump <code>crossbeam-queue</code> to <code>0.3</code>.</li>
<li>Bump <code>crossbeam-utils</code> to <code>0.8</code>.</li>
</ul>
<h1>Version 0.7.3</h1>
<ul>
<li>Fix breakage with nightly feature due to <a href="https://github-redirect.dependabot.com/rust-lang/rust/issues/65214">rust-lang/rust#65214</a>.</li>
<li>Bump <code>crossbeam-channel</code> to <code>0.4</code>.</li>
<li>Bump <code>crossbeam-epoch</code> to <code>0.8</code>.</li>
<li>Bump <code>crossbeam-queue</code> to <code>0.2</code>.</li>
<li>Bump <code>crossbeam-utils</code> to <code>0.7</code>.</li>
</ul>
<h1>Version 0.7.2</h1>
<ul>
<li>Bump <code>crossbeam-channel</code> to <code>0.3.9</code>.</li>
<li>Bump <code>crossbeam-epoch</code> to <code>0.7.2</code>.</li>
<li>Bump <code>crossbeam-utils</code> to <code>0.6.6</code>.</li>
</ul>
<h1>Version 0.7.1</h1>
<ul>
<li>Bump <code>crossbeam-utils</code> to <code>0.6.5</code>.</li>
</ul>
<h1>Version 0.7.0</h1>
<ul>
<li>Remove <code>ArcCell</code>, <code>MsQueue</code>, and <code>TreiberStack</code>.</li>
<li>Change the interface of <code>ShardedLock</code> to match <code>RwLock</code>.</li>
<li>Add <code>SegQueue::len()</code>.</li>
<li>Rename <code>SegQueue::try_pop()</code> to <code>SegQueue::pop()</code>.</li>
<li>Change the return type of <code>SegQueue::pop()</code> to <code>Result</code>.</li>
<li>Introduce <code>ArrayQueue</code>.</li>
<li>Update dependencies.</li>
</ul>
<h1>Version 0.6.0</h1>
<ul>
<li>Update dependencies.</li>
</ul>
<h1>Version 0.5.0</h1>
<ul>
<li>Update <code>crossbeam-channel</code> to 0.3.</li>
<li>Update <code>crossbeam-utils</code> to 0.6.</li>
<li>Add <code>AtomicCell</code>, <code>SharedLock</code>, and <code>WaitGroup</code>.</li>
</ul>
<h1>Version 0.4.1</h1>
<ul>
<li>Fix a double-free bug in <code>MsQueue</code> and <code>SegQueue</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d9dfc9e1ff"><code>d9dfc9e</code></a> Merge <a href="https://github-redirect.dependabot.com/crossbeam-rs/crossbeam/issues/581">#581</a></li>
<li><a href="2a3d84b8db"><code>2a3d84b</code></a> Prepare for the next release</li>
<li><a href="5ea6e705ea"><code>5ea6e70</code></a> Merge <a href="https://github-redirect.dependabot.com/crossbeam-rs/crossbeam/issues/580">#580</a></li>
<li><a href="b363b3dcd7"><code>b363b3d</code></a> Fix UB in destroy_array test</li>
<li><a href="e08b21cc08"><code>e08b21c</code></a> Merge <a href="https://github-redirect.dependabot.com/crossbeam-rs/crossbeam/issues/577">#577</a></li>
<li><a href="6217abf4a9"><code>6217abf</code></a> Ignore clippy::match_like_matches_macro lint</li>
<li><a href="9cbfae7380"><code>9cbfae7</code></a> Update cfg-if to 1</li>
<li><a href="24447495c9"><code>2444749</code></a> Merge <a href="https://github-redirect.dependabot.com/crossbeam-rs/crossbeam/issues/574">#574</a></li>
<li><a href="619f7db2df"><code>619f7db</code></a> Fixed a few typos</li>
<li><a href="7cc8377263"><code>7cc8377</code></a> Better plot.py with Python 3 & remove hard code (<a href="https://github-redirect.dependabot.com/crossbeam-rs/crossbeam/issues/569">#569</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-utils-0.7.0...crossbeam-utils-0.8.0">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` use these labels` will set the current labels as the default for future PRs for this repo and language
- ``@dependabot` use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- ``@dependabot` use these assignees` will set the current assignees as the default for future PRs for this repo and language
- ``@dependabot` use this milestone` will set the current milestone as the default for future PRs for this repo and language
- ``@dependabot` badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
</details>
This stops using `to_string` as a proxy for this now-provided precise API.
This reverts commit b71927224f and bupms the
dependency version in Cargo.toml.
Updates to path source walking.
This is a collection of loosely related changes to path source walking:
* Add more context to error messages.
* Allow `package.exclude` patterns to match directories. Previously, the walker would recurse into the directory, and skip every file. Instead, just skip the whole directory. This can be helpful if the directory is not readable, or otherwise want to avoid walking.
* Don't require `Cargo.toml` to be in root of a git repo in order to use git to guide the selection. I'm not sure I understand the original reasoning that (any) `Cargo.toml` had to reside next to the `.git` directory.
The last is a moderately risky change, since it's hard to predict how this might affect more complex project layouts or new interactions with `.gitignore` that didn't exist before. Also, I'm wondering if it should just ignore if it fails to open the repo instead of emitting an error?
Closes#1729Closes#6188Closes#8092
