This is to help prepare for checking for doc warnings across the entire
workspace being created in rust-lang/cargo#11851
`Mutation` was made `pub`, along with its fields, but they aren't
actually usable with anything, so I went and made it private to match
what its documentation references
This allows to use `gitoxide` for all fetch operations, boosting performance
for fetching the `crates.io` index by a factor of 2.2x, while being consistent
on all platforms.
For trying it, nightly builds of `cargo` can specify `-Zgitoxide=fetch`.
It's also possible to set the `__CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2=1` environment
variable (value matters), which is when `-Zgitoxide=none` can be used
to use `git2` instead.
Limitations
-----------
Note that what follows are current shortcomings that will be addressed in future PRs.
- it's likely that authentication around the `ssh` protocol will work differently in practice
as it uses the `ssh` program.
- clones from `file://` based crates indices will need the `git` binary to serve the locatl repository.
- the progress bar shown when fetching doesn't work like the orgiinal, but should already feel 'faster'.
feat(resolver): `-Zdirect-minimal-versions`
This is an alternative to `-Zminimal-versions` as discussed in #5657.
Problems with `-Zminimal-versions` includes
- Requires the root most dependencies to verify it and we then percolate that up the stack. This requires a massive level of cooperation to accomplish and so far there have been mixed results with it to the point that cargo's unstable
documentation discourages its use.
- Users expect `cargo check -Zminimal-versions` to force resolving to minimal but it doesn't as the default maximal resolve is compatible and requires `cargo update -Zminimal-versions`
- Different compatible versions might be selected, breaking interop between crates, changing feature unification, and breaking `-sys` crates without bad `links`
`-Zdirect-minimal-versions` instead only applies this rule to your
direct dependencies, allowing anyone in the stack to immediately adopt
it, independent of everyone else.
Special notes
- Living up to the name and the existing design, this ignores yanked
crates. This makes sense for `^1.1` version requirements but might
look weird for `^1.1.1` version requirements as it could select
`1.1.2`.
- This will error if an indirect dependency requires a newer version.
Your version requirement will need to capture what you use **and** all
of you dependencies. An alternative design would have tried to merge
the result of minimum versions for direct dependencies and maximum
versions for indirect dependencies. This would have been complex and
led to weird corner cases, making it harder to predict. I also suspect
the value gained would be relatively low as you can't verify that
version requirement in any other way.
- This also means discrepancies between `dependencies` and `dev-dependencies` are errors
- The error could be improved to call out that this was from minimal
versions but I felt getting this out now and starting to collect
feedback was more important.
One advantage of this approach over `-Zminimal-versions` is that it removes most of the problems that [cargo-minimal-versions](https://github.com/taiki-e/cargo-minimal-versions) tried to workaround.
As for the implementation, this might not be the most elegant solution but it works and we can always iterate and improve on it in the future.
- We keep the state as a `bool` throughout but compensate for that by explicitly creating a variable to abstract away constants
- The name changes depending on the context, from `direct_minimal_version` when dealing with the unstable flag to `first_minimal_version` when the concept of "direct" is lost to `first_version` when we split off the ordering concept into a separate variable
- Packages that respect `direct_minimal_versions` are determined by whether they are the top-level `summaries` that get past into `resolve`
### What does this PR try to resolve?
The primary use case is verifying version requirements to avoid depending on something newer than might be available in a dependent
For this to help the MSRV use case, the crate author must directly depend on all indirect dependencies where their latest release has too new of an MSRV but at least they can do so with the `^` operator, rather than `<` and breaking the ecosystem.
### How should we test and review this PR?
The first two commits add tests using `-Zminimal-versions`. The commit that adds `-Zdirect-minimal-versions` updates the tests, highlighting the differences in behavior.
### Additional information
Potential areas of conversation for stablization
- Flag name
- Handling of yanked (pick first non-yanked, pick yanked, error)
- Quality of error message
- Should the package have a "memory" of this flag being set by writing it to the lockfile?
Potential future work
- Stablize this
- Remove `-Zminimal-versions`
- Update `cargo publish`s `--verify` step to use this.
- The challenge is this won't be using the packaged `Cargo.lock` which probably should also be verified.
* moved `is_empty` check into `check_token`
* improved error message (is quite long now but should explain the error
well)
* removed one helper function from new test
When using registry operations with authentication there will be now an
error if the given token is not valid.
This is a technically a breaking change because a registry might give
some tokens which will be denied by these new checks.
In practice these tokens cause issues with HTTP so no registry should
generate them.
- Revert most of changes to expected test output from commit 2a4ec9f2.
- Keep later changes to expected test output from commit 0263ef43.
- Change test input that's converted to trigger similar output as previously.
This is an alternative to `-Zminimal-versions` as discussed in #5657.
The problem with `-Zminimal-versions` is it requires the root most
dependencies to verify it and we then percolate that up the stack. This
requires a massive level of cooperation to accomplish and so far there
have been mixed results with it to the point that cargo's unstable
documentation discourages its use.
`-Zdirect-minimal-versions` instead only applies this rule to your
direct dependencies, allowing anyone in the stack to immediately adopt
it, independent of everyone else.
Special notes
- Living up to the name and the existing design, this ignores yanked
crates. This makes sense for `^1.1` version requirements but might
look weird for `^1.1.1` version requirements as it could select
`1.1.2`.
- This will error if an indirect dependency requires a newer version.
Your version requirement will need to capture what you use **and** all
of you dependencies. An alternative design would have tried to merge
the result of minimum versions for direct dependencies and maximum
versions for indirect dependencies. This would have been complex and
led to weird corner cases, making it harder to predict. I also suspect
the value gained would be relatively low as you can't verify that
version requirement in any other way.
- The error could be improved to call out that this was from minimal
versions but I felt getting this out now and starting to collect
feedback was more important.
Deny warnings in CI, not locally
### What does this PR try to resolve?
The problem with #![deny(warnings)] is it makes iteration more difficult as you might have intermediate states with warnings. Its slightly better that we defer this to cargo test --lib but that still means you can't run a subset of tests against your experiment until you've cleaned up all of your warnings. This can lead to users [working around the problem which could accidentally slip in](d92d04840c).
### How should we test and review this PR?
The first round for this PR includes a warning in the `cargo` crate to ensure it breaks CI. It will then be reverted.
### Additional information
See also https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/.60.23!.5Bcfg_attr.28test.2C.20deny.28warnings.29.29.5D.60
Replace `winapi` with `windows-sys` crate.
### What does this PR try to resolve?
replace `winapi` with `windows-sys` crate.
It's officially maintained.
### How should we test and review this PR?
I just do the replacement of API. I think it is quite clear.
And I found a `cfg` for `windows` may miss checked error.
### Additional information
No one.
Make cargo aware of dwp files.
When using -Csplit-debuginfo=packed on Linux, rustc will produce a dwp file. Unlike the dwo files, whose paths are embedded into the binary, there's no information in the binary to help a debugger locate a dwp file. By convention, the dwp file for `<EXE>` is given the name `<EXE>.dwp` and placed next to `<EXE>`.
When cargo hardlinks the executable file rustc put in target/debug/deps into target/debug, it also needs to hardlink the dwp file along with it. Failing to do this prevents the debugger from finding the dwp file when the binary is executed from target/debug, as there's no way for the debugger to know to look in the deps subdirectory.
The split_debuginfo option is passed down into file_types to make this possible. For cargo clean manual handling is added to match the other split_debuginfo files. bin_link_for_target also passes in None because it won't care about the dwp file.
When using -Csplit-debuginfo=packed on Linux, rustc will produce a dwp file.
Unlike the dwo files, whose paths are embedded into the binary, there's no
information in the binary to help a debugger locate a dwp file. By convention,
the dwp file for <EXE> is given the name <EXE>.dwp and placed next to <EXE>.
When cargo hardlinks the executable file rustc put in target/debug/deps into
target/debug, it also needs to hardlink the dwp file along with it. Failing to
do this prevents the debugger from finding the dwp file when the binary is
executed from target/debug, as there's no way for the debugger to know to look
in the deps subdirectory.
chore: Fix typos
Seeing several typo PRs, like #11560, I figured I'd run my source code spell checker on cargo to help catch a lot of these earlier, in one big batch.
There is a soundness issue with atty when building on Windows with a
custom allocator.
This PR switches direct dependencies on atty to is-terminal. New semver
compatible versions of clap and snapbox remove atty. #11417 upgrades
env_logger to remove it from there.
Fixes#11415
Change rustdoc-scrape-examples to be a target-level configuration
This PR addresses issues raised in rust-lang/cargo#9525. Specifically:
1. It enables examples to be scraped from `#[test]` functions, by passing additional flags to Rustdoc to ensure that these functions aren't ignored by rustc.
2. It moves the `arg` from `-Z rustdoc-scrape-examples={arg}` into a target-level configuration that can be added to Cargo.toml.
The added test `scrape_examples_configure_target` shows a concrete example. In short, examples will be default scraped from Example and Lib targets. Then the user can enable or disable scraping like so:
```toml
[lib]
doc-scrape-examples = false
[[test]]
name = "my_test"
doc-scrape-examples = true
```
chore: Upgrade dependencies
This upgrades several dependencies to the latest "major" release. The original intent was just to get onto the latest snapbox but I noticed several others that could be updated and thought "why not".
- `snapbox` broke compatibility on less used APIs
- Its unclear from the docs if `miow` or `remove_dir_all` even broke compatibility
I did not touch `mdman` as that has several large, stale deps
Make cargo forward pre-existing CARGO if set
Currently, Cargo will always set `$CARGO` to point to what it detects its own path to be (using `std::env::current_exe`). Unfortunately, this runs into trouble when Cargo is used as a library, or when `current_exe` is not actually the binary itself (e.g., when invoked through Valgrind or `ld.so`), since `$CARGO` will not point at something that can be used as `cargo`. This, in turn, means that users can't currently rely on `$CARGO` to do the right thing, and will sometimes have to invoke `cargo` directly from `$PATH` instead, which may not reflect the `cargo` that's currently in use.
This patch makes Cargo re-use the existing value of `$CARGO` if it's already set in the environment. For Cargo subcommands, this will mean that the initial invocation of `cargo` in `cargo foo` will set `$CARGO`, and then Cargo-as-a-library inside of `cargo-foo` will inherit that (correct) value instead of overwriting it with the incorrect value `cargo-foo`. For other execution environments that do not have `cargo` in their call stack, it gives them the opportunity to set a working value for `$CARGO`.
One note about the implementation of this is that the test suite now needs to override `$CARGO` explicitly so that the _user's_ `$CARGO` does not interfere with the contents of the tests. It _could_ remove `$CARGO` instead, but overriding it seemed less error-prone.
Fixes#10119.
Fixes#10113.
Currently, Cargo will always set `$CARGO` to point to what it detects
its own path to be (using `std::env::current_exe`). Unfortunately, this
runs into trouble when Cargo is used as a library, or when `current_exe`
is not actually the binary itself (e.g., when invoked through Valgrind
or `ld.so`), since `$CARGO` will not point at something that can be used
as `cargo`. This, in turn, means that users can't currently rely on
`$CARGO` to do the right thing, and will sometimes have to invoke
`cargo` directly from `$PATH` instead, which may not reflect the `cargo`
that's currently in use.
This patch makes Cargo re-use the existing value of `$CARGO` if it's
already set in the environment. For Cargo subcommands, this will mean
that the initial invocation of `cargo` in `cargo foo` will set `$CARGO`,
and then Cargo-as-a-library inside of `cargo-foo` will inherit that
(correct) value instead of overwriting it with the incorrect value
`cargo-foo`. For other execution environments that do not have `cargo`
in their call stack, it gives them the opportunity to set a working
value for `$CARGO`.
One note about the implementation of this is that the test suite now
needs to override `$CARGO` explicitly so that the _user's_ `$CARGO` does
not interfere with the contents of the tests. It _could_ remove `$CARGO`
instead, but overriding it seemed less error-prone.
Fixes#10119.
Fixes#10113.
Originally, crates.io would block on publish requests until the publish
was complete, giving `cargo publish` this behavior by extension. When
crates.io switched to asynchronous publishing, this intermittently broke
people's workflows when publishing multiple crates. I say interittent
because it usually works until it doesn't and it is unclear why to the
end user because it will be published by the time they check. In the
end, callers tend to either put in timeouts (and pray), poll the
server's API, or use `crates-index` crate to poll the index.
This isn't sufficient because
- For any new interested party, this is a pit of failure they'll fall
into
- crates-index has re-implemented index support incorrectly in the past,
currently doesn't handle auth, doesn't support `git-cli`, etc.
- None of these previous options work if we were to implement
workspace-publish support (#1169)
- The new sparse registry might increase the publish times, making the
delay easier to hit manually
- The new sparse registry goes through CDNs so checking the server's API
might not be sufficient
- Once the sparse registry is available, crates-index users will find
out when the package is ready in git but it might not be ready through
the sparse registry because of CDNs
This introduces unstable support for blocking by setting
`publish.timeout` to non-zero value.
A step towards #9507
Bump git2 to 0.15 and libgit2-sys to 0.14
This will allow cargo to avoid vendored builds of git2 in up-to-date
environments going forward, and brings in the [libgit2 1.4.4 CVE fix].
[libgit2 1.4.4 CVE fix]: https://github.com/libgit2/libgit2/releases/tag/v1.4.4
This will allow cargo to avoid vendored builds of git2 in up-to-date
environments going forward, and brings in the [libgit2 1.4.4 CVE fix].
[libgit2 1.4.4 CVE fix]: https://github.com/libgit2/libgit2/releases/tag/v1.4.4
Add reasons to all ignored tests.
This adds a reason string to all `#[ignore]` attributes. This will be displayed when running the test (since 1.61), which can help quickly see and identify why tests are being ignored. It looks roughly like:
```
test basic ... ignored, requires nightly, CARGO_RUN_BUILD_STD_TESTS must be set
test build::simple_terminal_width ... ignored, --diagnostic-width is stabilized in 1.64
test check_cfg::features_with_cargo_check ... ignored, --check-cfg is unstable
test plugins::panic_abort_plugins ... ignored, requires rustc_private
```
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
Improve integration of the http server introduced by the http-registry feature.
Now the same HTTP server is used for serving downloads, the index, and
the API.
This makes it easier to write tests that deal with authentication and
http registries.
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.
If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:
1. *foo* is a big git repo
```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 big
```
2. *bar* is a repo with a big submodule with `update=none`
```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
path = foo
url = file:///tmp/foo
update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
```
3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped
```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```
Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.
4. *bar* is now a lib with a big submodule with update disabled
```
/tmp $ cargo init --lib bar
Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
3 files changed, 18 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 src/lib.rs
/tmp $
```
5. *qux* depends on *bar*, notice *bar*'s submodules are fetched
```
/tmp $ cargo init qux && cd qux
Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
Updating git repository `file:///tmp/bar`
Updating git submodule `file:///tmp/foo`
real 0m22.182s
user 0m20.402s
sys 0m1.714s
/tmp/qux $
```
Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.
6. With the change applied, submodules with `update=none` are skipped
```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
Updating git repository `file:///tmp/bar`
Skipping git submodule `file:///tmp/foo`
real 0m0.029s
user 0m0.021s
sys 0m0.008s
/tmp/qux $
```
Fixes#4247.
Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
When running tests in the `rust-lang/cargo` repo, `file!` is relative to
the crate root and tests are run relative to the crate root and
everything is fine.
When running tests in the `rust-lang/rust` repo, `file!` is relative to
the workspace root and tests are run relative to the crate root and
there is much sadness.
If we are compiling relative to the crate root, we could make the path
absolute and everything would be dandy but this needs to happen at
compile time. Didn't see a way to do this.
We could stop using `curr_dir` but that makes the tests a bit noisier
with more overhead for creating a new tests from an existing case.
Since we can reasonly know what all roots will be used for `file!`, we
can just hard code-in support for those two roots. Much happiness
ensues as everything works with this surgical hack.
This is something the existing test infrastructure supports, so I
figured I'd make it mirror it for snapbox. I'm mixed.
- It reads more like what a user would type, making it easier to run a
test locally or take a manual test case and automate it
- It can make it harder to parse the arguments when scanning tests
- Without using a crate like `shlex`, the syntax support is unclear
This was written for `cargo_add.rs`, based on `snapbox`. This allows
creating a test from a known reproduction case or easily debugging on an
existing test case.
Mark .cargo/git and .cargo/registry as cache dirs
Fixes#10457
### What does this PR try to resolve?
As we previously discussed in #10457 this marks `~/.cargo/git` and `~/.cargo/registry` as not to be included in backups, and not subject to content indexing. These directories can be fairly large and frequently changed, and should only contain content that can be re-downloaded if necessary.
### How should we test and review this PR?
I did two manual tests:
1. Using the binary built from this tree, run `cargo update` in a source tree that has a git dependency, and observe that afterwards, there is a `CACHEDIR.TAG` in `~/.cargo/git`.
2. Similarly, run `cargo update` and observe that there's a `CACHEDIR.TAG` in `~/.cargo/registry`.
(I ran this on Linux. This code should also trigger OS-specific behavior on macOS and Windows that's the same as is currently applied to `target/`.)
I added some test assertions.
- Add `ProcessBuilder::output` and ProcessBuilder::status`, which are
unopinionated version of `exec_*` (won't error out when exitcode > 0)
- Add `ProcessBuilder::retry_with_argfile` to enable trying with argfile
when hitting the "command line too big" error.
Registry functions return Poll to enable parallel fetching of index data
Adds `Poll` as a return type for several registry functions to enable parallel fetching of crate metadata with a future http-based registry.
Work is scheduled by calling the `query` and related functions, then waited on with `block_until_ready`.
This PR is based on the draft PR started by eh2406 here [#8985](https://github.com/rust-lang/cargo/pull/8985).
r? `@Eh2406`
cc `@alexcrichton`
cc `@jonhoo`
Implement "artifact dependencies" (RFC-3028)
Tracking issue: #9096
#### Tasks
* [x] -Z unstable option called `bindeps`
* ✅ **(config)** allow parsing of newly introduced 'artifact' fields
* [x] into `TomlManifest`
* [x] into `Manifest`
- [x] ~~abort~~ warn if artifacts are used on stable
* ✅ **resolver** : assure artifact dependencies are part of the resolution process and unified into the dependency tree
* 🔬**compiler**: make it understand 'artifact' dependencies and pass new environment variables to the crate being build
* [x] `lib=false` should not be considered a rust library for the dependent, in unit and possibly resolve graph
* [x] assure profile settings are applied correctly
* [x] target overrides work
* [x] `target = "target"` in build deps
* [x] other targets on build deps
* [x] other targets on non-build deps
* [x] 'no-cross doc tests' seems like a constraint we should apply as well maybe
* [x] more confidence with `resolver = "2"`
* [x] assure artifact placement is correct (bin and various forms of lib)
* ✅ **serialization**: rewriting manifests (i.e. for publishing) does not discard artifact information
* [x] publishing keeps `artifact` and `lib` values
* **Other cargo subcommands**
* [x] `cargo metadata`
* leave unchanged
* [x] artifacts work with `cargo check`
* [x] artifacts work with rustdoc, such that it doesn't document them unless `lib=true`
* [x] `cargo tree` maybe?
* [x] `cargo clean` should clean artifacts - even though it's more complex, ultimately it deletes the `target` directory.
* [x] artifacts work with `cargo test` (and dev-dependencies)
* [x] doctests
* [x] try `reproducible` repository as well.
* 🧪 **tests** for more subtle RFC constraints
- [x] build scripts cannot access artifact environment variables at compile time, only at runtime)
- [x] assure 'examples' which also support crate-type fields like [[lib]] won't become artifacts themselves.
- [x] assure `--out-dir` does not leak artifacts - tested manually, it seemed to niche to add a test.
- [x] try `target="foo"` in artifact and assure it sees a decent error message
- [x] Assure RFC 3176 _doesn't_ work
* 🧹cleanup and finalization
- [x] assure no `TODO(ST)` markers are left in code
- [x] assure no tests are ignored
- [x] use `resolver = "1"` once to assert everything also works with the previous resolver, but leave it on "2".
#### Implementation and review notes
- artifacts and unstable options are only checked when transforming them from `TomlManifest` to `Manifest`, discarding artifact information if the unstable flag is not set. Nowhere else in code will the CLI options be checked again.
- `If no binaries are specified, all the binaries in the package will be built and made available.` - this should only refer to `[[bin]]` targets, not examples for instance.
- artifact binaries won't be uplifted, hence won't be present outside of their output directory
- ❗️We don't know how [package links](00e925f61f/src/cargo/core/compiler/unit_dependencies.rs (L380)) will affect artifacts for build dependencies. Should probably be thought through.
- ❗️The location of artifacts is only tested roughly to avoid having to deal with different output names on the four platforms that seem to matter (gnu, macos, windows msvc, windows gnu).
- `cargo tree` doesn't handle artifacts specifically, and it might be interesting to make clear if an artifact is only an artifact, or both artifact and dependency.
- Most error and warning messages can probably be more cargo-matic.
#### Questions
* Does `cargo` without the feature enabled have to complain about the "artifact" field in a dependency, like it does right now? It doesn't look like machinery for that exists in `do_read_manifest()`.
- ✔️It warns now
* Should parsing of artifact values, like "bin" be case sensitive?
- ✔️ It's case sensitive now, which should help with serde roundtripping.
#### Review Progress
* [x] address Josh's review notes one by one
* [x] introduce `IsArtifact` (see [this answer](https://github.com/rust-lang/cargo/pull/9992#discussion_r774928102)) (76cd48a2d62d74e043a1a482199c5bb920f50311)
* [x] prefer uplifting artifact deps that were written into the `deps` directory, but prefer to do that in [this PR instead](https://github.com/rust-lang/cargo/pull/9992)
* [x] add extra-tests as described in Josh's comment: " features get unified between a Rust library and a binary, and one that confirms features don't get unified between a Rust library and a binary for a different target?"
* [x] Make target-based artifact splitting work by porting parts of RFC-3176
* [x] test-support/cross-compile
* [x] namespace separation
* [x] re-create RFC-3176 what's not in RFC-3028, namely multidep support and all related tests
* [x] Address Eh2406 's review comments
* [x] Address Eric's comments
* [x] Unstable features need to be documented in unstable.md.
* [x] sort out [`target_data`](https://github.com/rust-lang/cargo/pull/9992#discussion_r787316229)
* [x] figure out [cargo metadata](https://github.com/rust-lang/cargo/pull/9992#discussion_r787320923)
* [x] See if target-data can work without an [index format update](https://github.com/rust-lang/cargo/pull/9992#issuecomment-1021697124).
* [x] truncate comments at 80-90 lines and remove unused methods, remove -Z unstable-options, use `cfg!(target_env = "msvc")`
* [x] add missing doc comments to newly added methods and funtions
* [x] simplify method parameters and inline some functions
* [x] add test and extend test to show additional issues
* [x] assure current set of tests works consistently, also on windows
Sponsored by [Profian](https://www.profian.com)
Tracking issue: https://github.com/rust-lang/cargo/issues/9096
Original PR: https://github.com/rust-lang/cargo/pull/9992
Add 'bindeps' -Z flag for later use
A test to validate artifact dependencies aren't currently parsed.
Parse 'artifact' and 'lib' fields.
Note that this isn't behind a feature toggle so 'unused' messages will
disappear.
Transfer artifact dependencies from toml- into manifest-dependencies
There are a few premises governing the operation.
- if unstable features are not set, warn when 'artifact' or 'lib' is
encountered.
- bail if 'lib' is encountered alone, but warn that this WOULD happen
with nightly.
- artifact parsing checks for all invariants, but some aren't tested.
Assure serialization of 'artifact' and 'lib' fields produces suitable values during publishing
This should be the only place were these fields matter and where a cargo
manifest is actually produced. These are only for internal use, no user
is typically going to see or edit them.
Place all artifact dependency tests inta their own module
This facilitates deduplication later and possibly redistribution into
other modules if there is a better fit.
Represent artifacts that are rust libraries as another ArtifactKind
This is more consistent and probably simpler for later use.
No need to reflect the TOML data structure.
Add tests to assure only 'lib = true' artifact deps are documented
RFC-3028 doesn't talk about documentation, but for lib=true it's clear
what the desired behaviour should be.
If an artifact isn't a library though, then for now, it's transparent,
maybe.
Many more tests, more documentation, mild `Artifact` refactor
The latter seems to be a better fit for what being an artifact
really means within cargo, as it literally turns being a library
on or off, and thus only optionally becoming a normal library.
refactor to prepare for artifact related checks
Don't show a no-lib warning for artifact dependencies (with lib = false)
Tests for more artifact dependency invariants
These are merely a proof of concept to show that we are not in
a position to actually figure out everything about artifacts
right after resolution.
However, the error message looks more like a fatal error and less
like something that can happen with a more elaborate error message
with causes.
This might show that these kind of checks might be better done later
right before trying to use the information for create compile units.
Validate that artifact deps with lib=true still trigger no-lib warnings
This triggers the same warning as before, for now without any
customization to indicate it's an artifact dependency.
Use warnings instead of errors
------------------------------
This avoids the kind of harsh end of compilation in favor of something
that can be recovered from. Since warnings are annoying, users will
probably avoid re-declaring artifact dependencies.
Hook in artifact dependencies into build script runs
Even though we would still have to see what happens if they have a lib
as well. Is it built twice?
Also
----
- fly-by refactor: fix typo; use ? in method returning option
- Propagate artifact information into Units; put artifacts into place
This means artifacts now have their own place in the 'artifact'
directory and uplifts won't happen for them.
- refactor and fix cippy suggestion
- fix build after rebasing onto master
Create directories when executing the job, and not when preparing it.
also: Get CI to work on windows the easy way, for now.
Set directories for artifact dependencies in build script runtimes
Test remaining kinds of build-script runtime environment variables
Also
----
- Fix windows tests, the quick way.
- Try to fix windows assertions, and generalize them
- Fix second test for windows, hopefully
test for available library dependency in build scripts with lib = true
probably generally exclude all artifact dependencies with lib=false.
Pass renamed dep names along with unit deps to allow proper artifact env names
Test for selective bin:<name> syntax, as well as binaries with dashes
Test to assure dependency names are transformed correctly
assure advertised binaries and directories are actually present
This wouldn't be the case if dependencies are not setup correctly,
for instance.
Also
----
- make it easier to see actual values even on failure
This should help figure out why on CI something fails that works
locally no matter what.
Turns out this is a race condition, with my machine being on the good
side of it so it doesn't show in testing. Fortunately it still can be
reproduced and easily tested for.
- refactor test; the race condition is still present though
- Force CI to pass here by avoiding checks triggering race.
- Fix windows build, maybe?
More tolerant is_file() checks to account for delay on CI
This _should_ help CI to test for the presence which is better than
not testing at all.
This appears to be needed as the output file isn't ready/present in time
for some reason.
The root cause of this issue is unknown, but it's definitely a race
as it rarely happens locally. When it happened, the file was always
present after the run.
Now we will learn if it is truly not present, ever, or if it's maybe
something very else.
Validate libs also don't see artifact dependencies as libraries with lib=false
Also
----
- Add prelimiary test for validating build-time artifacts
- Try to fix CI on gnu windows
Which apparently generates paths similar to linux, but with .exe suffix.
The current linux patterns should match that.
- refactor
Help sharing code across modules
allow rustc to use artifact dep environment variables, but…
…it needs some adjustments to actually setup the unit dependency graph
with artifacts as well.
Right now it will only setup dependencies for artifacts that are libs,
but not the artifacts themselves, completely ignoring them when they
are not libs.
Make artifact dependencies available in main loop
This is the commit message #2:
------------------------------
rough cut of support for artifact dependencies at build time…
…which unfortunately already shows that the binary it is supposed to
include is reproducibly not ready in time even though the path is
correct and it's present right after the run.
Could it be related to rmeta?
This is the commit message #3:
------------------------------
Fix test expectations as failure is typical than the warning we had before…
…and add some tolerance to existing test to avoid occasional failures.
This doesn't change the issue that it also doens't work at all for
libraries, which is nicely reproducable and hopefully helps to fix
this issue.
This is the commit message #4:
------------------------------
Probably the fix for the dependency issue in the scheduler
This means that bin() targets are now properly added to the job graph
to cause proper syncing, whereas previously apparently it would
still schedule binaries, but somehow consider them rmeta and thus
start their dependents too early, leading to races.
This is the commit message #5:
------------------------------
Don't accidentally include non-gnu windows tests in gnu windows.
Support cargo doc and cargo check
The major changes here are…
- always compile artifacts in build mode, as we literally want the
build output, always, which the dependent might rely on being present.
- share code between the rather similar looking paths for rustdoc and
rustc.
Make artifact messages appear more in line with cargo by using backticks
Also: Add first test for static lib support in build scripts
build-scripts with support for cdylib and staticlib
- Fix windows msvc build
No need to speculate why the staticlib has hashes in the name even
though nothing else.
staticlib and cdylib support for libraries
test staticlib and cdylibs for rustdoc as well.
Also catch a seemingly untested special case/warning about the lack
of linkable items, which probably shouldn't be an issue for artifacts
as they are not linkable in the traditional sense.
more useful test for 'cargo check'
`cargo check` isn't used very consistently in tests, so when we use it
we should be sure to actually try to use an artifact based feature
to gain some coverage.
verify that multiple versions are allowed for artifact deps as well.
also: remove redundant test
This is the commit message #2:
------------------------------
Properly choose which dependencies take part in artifact handling
Previously it would include them very generously without considering
the compatible dependency types.
This is the commit message #3:
------------------------------
a more complex test which includes dev-dependencies
It also shows that doc-tests don't yet work as rustdoc is run outside of
the system into which we integrate right now.
It should be possible to write our environment variable configuration
in terms of this 'finished compilation' though, hopefully with
most code reused.
This is the commit message #4:
------------------------------
A first stab at storing artifact environment variables for packages…
…however, it seems like the key for this isn't necessarily correct
under all circumstances. Maybe it should be something more specific,
don't know.
This is the commit message #5:
------------------------------
Adjust key for identifying units to Metadata
This one is actually unique and feels much better.
This is the commit message #6:
------------------------------
Attempt to make use of artifact environment information…
…but fail as the metadata won't match as the doctest unit is, of course,
its separate unit. Now I wonder if its possible to find the artifact
units in question that have the metadata.
Properly use metadata to use artifact environment variables in doctests
This is the commit message #2:
------------------------------
Add test for resolver = "2" and build dependencies
Interestingly the 'host-features' flag must be set (as is seemingly
documented in the flags documentation as well), even though I am not
quite sure if this is the 100% correct solution. Should it rather
have an entry with this flag being false in its map? Probably not…
but I am not quite certain.
This is the commit message #3:
------------------------------
set most if not all tests to use resolver = "2"
This allows to keep it working with the most recent version while
allowing to quickly test with "1" as well (which thus far was working
fine).
All tests I could imagine (excluding target and profiles) are working now
Crossplatform tests now run on architecture aarm64 as well.
More stringent negative testing
Fix incorrect handling of dependency directory computation
Previously it would just 'hack' the deps-dir to become something very
different for artifacts.
This could easily be fixed by putting the logic for artifact output
directories into the right spot.
A test for cargo-tree to indicate artifacts aren't handled specifically
Assure build-scripts can't access artifacts at build time
Actual doc-tests with access to artifact env vars
All relevant parsing of `target = [..]`
Next step is to actually take it into consideration.
A failing test for adjusting the target for build script artifacts using --target
Check for unknown artifact target triple in a place that exists for a year
The first test showing that `target="target"` deps seemingly work
For now only tested for build scripts, but it won't be much different
for non-build dependencies.
build scripts accept custom targets unconditionally
Support target setting for non-build dependencies
This is the commit message #2:
------------------------------
Add doc-test cross compile related test
Even though there is no artifact code specific to doc testing, it's
worth to try testing it with different target settings to validate
it still works despite doc tests having some special caseing around
target settings.
This is the commit message #3:
------------------------------
A test to validate profiles work as expected for build-deps and non-build deps
No change is required to make this work and artifact dependencies 'just work'
based on the typical rules of their non-artifact counterarts.
This is the commit message #4:
------------------------------
Adjust `cargo metadata` to deal with artifact dependencies
This commit was squashed and there is probably more that changed.
This is the commit message #5:
------------------------------
Show bin-only artifacts in "resolve" of metadata as well.
This is the commit message #6:
------------------------------
minor refactoring during research for RFC-3176
This will soon need to return multiple extern-name/dep-name pairs.
This is the commit message #7:
------------------------------
See if opt-level 3 works on win-msvc in basic profile test for artifacts
This is the same value as is used in the other test of the same name,
which certainly runs on windows.
This is the commit message #8:
------------------------------
refactor
Assure the type for targets reflect that they cannot be the host target,
which removes a few unreachable!() expressions.
Put `root_unit_compile_kind` into `UnitFor`
Previously that wasn't done because of the unused `all_values()`
method which has now been deleted as its not being used anyomre.
This allows for the root unit compile kind to be passed as originally
intended, instead of working around the previous lack of extendability
of UnitFor due to ::all_values().
This is also the basis for better/correct feature handling once
feature resolution can be depending on the artifact target as well,
resulting in another extension to UnitFor for that matter.
Also
----
- Fix ordering
Previously the re-created target_mode was used due to the reordering
in code, and who knows what kind of effects that might have
(despite the test suite being OK with it).
Let's put it back in place.
- Deactivate test with filename collision on MSVC until RFC-3176 lands
Avoid clashes with binaries called 'artifact' by putting 'artifact/' into './deps/'
This commit addresses review comment https://github.com/rust-lang/cargo/pull/9992#discussion_r772939834
Don't rely on operator precedence for boolean operations
Now it should be clear that no matter what the first term is,
if the unit is an artifact, we should enqueue it.
Replace boolean and `/*artifact*/ <bool>` with `IsArtifact::(Yes/No)`
fix `doc::doc_lib_false()` test
It broke due to major breakage in the way dependencies are calculated.
Now we differentiate between deps computation for docs and for building.
Avoid testing for doctest cross-compilation message
It seems to be present on my machine, but isn't on linux and it's
probably better to leave it out entirely and focus on the portions
of consecutive output that we want to see at least.
A test to validate features are unified across libraries and those in artifact deps in the same target
Allow aarch64 MacOS to crosscompile to an easily executable alternative target
That way more tests can run locally.
Support for feature resolution per target
The implementation is taken directly from RFC-3176 and notably lacks
the 'multidep' part.
Doing this definitely has the benefit of making entirely clear
'what is what' and helps to greatly reduce the scope of RFC-3176
when it's rebuilt based on the latest RF-3028, what we are implementing
right now.
Also
----
- A test which prooves that artifact deps with different target don't have a feature namespace yet
- Add a test to validate features are namespaced by target
Previously it didn't work because it relies on resolver = "2".
- 'cargo metadata' test to see how artifact-deps are presented
- Missed an opportunity for using the newly introduced `PackageFeaturesKey`
- Use a HashMap to store name->value relations for artifact environment variables
This is semantically closer to what's intended.
also: Remove a by now misleading comment
Prevent resolver crash if `target = "target"` is encountered in non-build dependencies
A warning was emitted before, now we also apply a fix.
Previously the test didn't fail as it accidentally used the old
resolver, which now has been removed.
Abort in parsing stage if nightly flag is not set and 'artifact' is used
There is no good reason to delay errors to a later stage when code
tries to use artifacts via environment variables which are not present.
Change wording of warning message into what's expected for an error message
remove unnecessary `Result` in `collect()` call
Improve logic to warn if dependencie are ignored due to missing libraries
The improvement here is to trigger correctly if any dependency of a
crate is potentially a library, without having an actual library target
as part of the package specification.
Due to artifact dependencies it's also possible to have a dependency
to the same crate of the same version, hence the package name
isn't necessarily a unique name anymore. Now the name of the actual
dependency in the toml file is used to alleviate this.
Various small changes for readability and consistency
A failing test to validate artifacts work in published crates as well
Originally this should have been a test to see target acquisition works
but this more pressing issue surfaced instead.
Make artifacts known to the registry data (backwards compatible)
Now artifacts are serialized into the registry on publish (at least
if this code is actually used in the real crates-io registry) which
allows the resolve stage to contain artifact information.
This seems to be in line with the idea to provide cargo with all
information it needs to do package resolution without downloading
the actual manifest.
Pick up all artifact targets into target info once resolve data is available
Even though this works in the test at hand, it clearly shows there
is a cyclic dependency between the resolve and the target data.
In theory, one would have to repeat resolution until it settles
while avoiding cycles.
Maybe there is a better way.
Add `bindeps`/artifact dependencies to `unstsable.md` with examples
Fix tests
Various small improvements
Greatly simplify artifact environment propagation to commands
Remove all adjustments to cargo-metadata, but leave tests
The tests are to record the status quo with the current code
when artifact dependencies are present and assure the information
is not entirely non-sensical.
Revert "Make artifacts known to the registry data (backwards compatible)"
This reverts commit adc5f8ad04840af9fd06c964cfcdffb8c30769b0.
Ideally we are able to make it work without altering the registry
storage format. This could work if information from the package
set is added to the resolve information.
Enrich resolves information with additional information from downloaded manifests
Resolve information comes from the registry, and it's only as rich as
needed to know which packages take part in the build.
Artifacts, however, don't influence dependency resolution, hence it
shouldn't be part of it.
For artifact information being present nonetheless when it matters,
we port it back to the resolve graph where it will be needed later.
Collect 'forced-target' information from non-workspace members as well
This is needed as these targets aren't present in the registry and
thus can't be picked up by traversing non-workspce members.
The mechanism used to pick up artifact targets can also be used
to pick up these targets.
Remove unnecessary adjustment of doc test
refactor `State::deps()` to have filter; re-enable accidentally disabled test
The initial rebasing started out with a separted `deps_filtered()`
method to retain the original capabilities while minimizing the chance
for surprises. It turned out that the all changes combined in this PR
make heavy use of filtering capabilities to the point where
`deps(<without filter>)` was unused. This suggested that it's required
to keep it as is without a way to inline portions of it.
For the original change that triggered this rebase, see
bd45ac81ba
The fix originally made was reapplied by allowing to re-use the
required filter, but without inlining it.
Always error on invalid artifact setup, with or without enabled bindeps feature
Clarify how critical resolver code around artifact is working
Remove workaround in favor of deferring a proper implementation
See https://github.com/rust-lang/cargo/pull/9992#issuecomment-1033394197
for reference and the TODO in the ignored test for more information.
truncate comments at 80-90c; cleanup
- remove unused method
- remove '-Z unstable-options'
- improve error message
- improve the way MSVC special cases are targetted in tests
- improve how executables are found on non MSVC
Avoid depending on output of rustc
There is cyclic dependency between rustc and cargo which makes it
impossible to adjust cargo's expectations on rustc without leaving
broken commits in rustc and cargo.
Add missing documentation
fix incorrect removal of non-artifact libs
This is also the first step towards cleaning up the filtering logic
which is still making some logic harder to understand than needs be.
The goal is to get it to be closer to what's currently on master.
Another test was added to have more safety regarding the overall
library inclusion logic.
inline `build_artifact_requirements_to_units()`
Simplify filtering
This adds a default filter to `state.deps(…)` making it similar to
what's currently in master, while creating another version of it
to allow setting a custom filter. This is needed as the default filter
won't allow build dependencies, which we need in this particular case.
`calc_artifact_deps(…)` now hard-codes the default filter which is
needed due to the use of `any` here:
c0e6abe384/src/cargo/core/compiler/unit_dependencies.rs (L1119)
.
Simplify filtering.
Port cargo from toml-rs to toml_edit
Benefits:
- A TOML 1.0 compliant parser
- Unblock future work
- Have `cargo init` add the current crate to the workspace, rather
than error
- #5586: Upstream `cargo-add`
TODO
- [x] Analyze performance and address regressions
- [x] Identify and resolve incompatibiies
- [x] Resolve remaining test failures, see
https://github.com/ordian/toml_edit/labels/cargo
- [x] ~~Switch the code from https://github.com/rust-lang/cargo/pull/10176 to only parse once~~ (this PR is being merged first)
Benefits:
- A TOML 1.0 compliant parser
- Unblock future work
- Have `cargo init` add the current crate to the workspace, rather
than error
- #5586: Upstream `cargo-add`
Display alias target on 'cargo help <alias>`
```
Previously, `cargo help <alias>` resolved the alias and displayed the
help for the targeted subcommand. For example, if `br` were aliased to
`build --release`, `cargo help br` would display the manpage for
cargo-build.
With this patch, it will print "'br' is aliased to 'build --release'".
```
Addresses issue #10138.
This is my first patch to Cargo. I attempted to follow the style of the surrounding code. Please let me know if any changes are required: happy to make them. In particular, I wasn't sure if any tests exist for this path.
Implement glob escaping for clean -p
Add pattern escape for glob matching cargo clean files
Implement correct solution for #10068
Removed superfluous formatting changes
Update rm_rf_glob()'s error handling
Remove dir_glob reference for non-glob function
Added test
Satisfy clippy
Add MSVC support for test
Upgrade Cargo to the 2021 edition
While I was poking around in the manifests I also went ahead and removed the `authors` entries since at least the one for Cargo itself is quite outdated and Cargo otherwise doesn't use these any more.
Otherwise symbolic links may also accidentally be resolved which may
lead to unexpected results in the case of 'coreutils', a binary
that depends on the executable name being a symbolic link.
This means a path like /bin/echo being canonicalized to /bin/coreutils
will loose all information about the desired functionality.
Test failures will occur if 'echo' is resolved that way and it's
not trivial to find the cause of it in the provided error messages.
There are already similar preexisting test helpers which
also work on windows. Port over the executable-creation
helper into the file() helper and things appear to pass.
This preps cargo-platform for a release. The only substantial change is that
this should include the license files into the archive that is uploaded to
crates.io.
Closes#9758
This concentrates all of the "prefer this version" logic previously
handled with `try_to_use` and `prefer_patch_deps` parameters into a
struct that hides both the reason a package version might be preferred
and the form that preference took (Dependency or PackageId). Besides
simplifying `RegistryQuerier::query` slightly, this invites further
refinements to version preferences to support new cargo features.
Prefer patched versions of dependencies
When selecting among several versions of a paackage, prefer versions
from `[patch]` sections over other versions, similar to how locked
versions are preferred.
Patches come in the form of a Dependency and not a PackageId, so this
preference is expressed with `prefer_patch_deps`, distinct from
`try_to_use`.
Fixes#9535
When selecting among several versions of a paackage, prefer versions
from `[patch]` sections over other versions, similar to how locked
versions are preferred.
Patches come in the form of a Dependency and not a PackageId, so this
preference is expressed with `prefer_patch_deps`, distinct from
`try_to_use`.
When searching for newlines in a process output keep track which part of
buffer was already examined to avoid processing the same data again and
again.
I don't trust that all these transformations won't have unintended
consequences on other platforms. It is nice to verify there aren't any
backslash shenanigans on other platforms.
This is intended to help with adding more usage of anyhow in the
testsuite, which can help show context for errors.
This also includes some small improvements to the error messages to
provide more information.
Fix rustdoc warnings
Change some small parts of the unit documentation in order to resolve warnings emitted when running `cargo doc` from the root of this project. It should help reduce the noise when checking that new or updated documentation builds correctly.
See the commit messages for details about the modifications themselves, although they should be rather simple.
Cheers,
Paul.
This commit continues the work from #9112 to enable `unpacked` split
debuginfo on macOS targets by default. This has been discussed on [internals]
for awhile now and no breakage has emerged while significant speedups
have. This is expected to be a compile-time and `target`-directory size
win for almost all macOS Rust projects.
While breakage is possible it's possible to mitigate this with
project-local or global cargo configuration of the `dev` and `test` profiles.
[internals]: https://internals.rust-lang.org/t/help-test-faster-incremental-debug-macos-builds-on-nightly/14016/9
Split out cargo-util package for cargo-test-support.
This splits out code from `cargo` that was being used by `cargo-test-support` in order to remove the dev-dependency cycle on the `cargo` crate. The intent here is to improve development build times. On my system, `touch src/cargo/lib.rs ; cargo check --profile=test` goes from 6.4 seconds to 3.5. I suspect more substantial changes (more than just `touch`) should exhibit even more improvements.
This has a substantial downside that it is another package to manage publishing, particularly with making sure the version is updated correctly for crates.io. I'm uncertain if on balance it is worth it, but I lean towards moving forward with it.
cargo-test-support wasn't using any of the caching or other logic from
Rustc, so this just swaps with a very basic implementation in order to
remove the dependency on `cargo`.
Forbid setting `RUSTC_BOOTSTRAP` from a build script on stable
Instead, recommend `RUSTC_BOOTSTRAP=crate_name`. If cargo is using a nightly toolchain, or if RUSTC_BOOTSTRAP was set in *cargo*'s build environment, the error is downgraded to a warning, since the variable won't affect the build.
This is mostly the same as suggested in https://github.com/rust-lang/cargo/issues/7088#issuecomment-713867773, except that `RUSTC_BOOTSTRAP=` values other than 1 are treated the same as `RUSTC_BOOTSTRAP=1`. My reasoning was that https://github.com/rust-lang/rust/pull/77802 is now on 1.50 stable, so some crates may have started using it, and I would still prefer not to give hard errors when there's no workaround.
Closes https://github.com/rust-lang/cargo/issues/7088.
r? `@joshtriplett`
testsuite: Use split debuginfo on macos.
This switches the testsuite to use "unpacked" debuginfo on macos, which is a substantial performance boost. On my system, the testsuite runs 1.55 times faster with this change. Along with #9206, total testsuite time is 3.1 times faster.
`nightly_features_allowed()` is no longer doing any work, so it can be
accessed directly. This also renames the `enable_nightly_features` field
to `nightly_features_allowed`.
Previously, since `ENABLE_NIGHTLY_FEATURES` and
`NIGHTLY_FEATURES_ENABLED` were thread locals, reading them in any other
thread would always say nightly features were disabled. Now, they are
tied to the `Context` itself, so it is both more clear how the variables
are being set and fixes the behavior within work threads.
Note that `Context` is not thread-safe, so this passes a boolean through
to `BuildOutput::parse`.
Updates to edition handling.
This introduces some updates for edition handling (split into commits for review). In short:
* `cargo-features = ["edition2021"]` can be used to opt-in to 2021 support without needing to pass around `-Z unstable-options`. I tried to emphasize in the docs that this is only for testing and experimentation.
* Make `"2"` the default resolver for 2021 edition.
* Make `cargo fix --edition` mean the "next" edition from the present one, and support 2021. Also, if already at the latest edition, generate a warning instead an error.
* I decided to allow `cargo fix --edition` from 2018 to 2021 on the nightly channel without an explicit opt-in. It's tricky to implement with an opt-in (see comment in diff).
Partial for #9048.
Fixes#9047.
What was previously "Fixing" was a message for after the fixes had
been applied. I think it would be clearer if it said "Fixed",
to indicate that the fixes had actually finished.
The new "Fixing" is posted just before it starts. This is verbose-only
since it is a little noisy.
Add split-debuginfo profile option
This commit adds a new `split-debuginfo` option to Cargo compilation
profiles which gets forwarded to the `-Csplit-debuginfo` codegen option
in rustc. This commit also sets the default, only on macOS, to be
`-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage
rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds
while also preserving a pleasant debugging experience by default. This
should lead to much faster incremental build times on macOS since
`dsymutil` isn't exactly the speediest tool in the world.
This is technically a breaking change in Cargo because we're no longer
by-default producing the `*.dSYM` folders on macOS. If those are still
desired, however, authors can always run `dsymutil` themselves or
otherwise configure `split-debuginfo = 'packed'` in their
manifest/profile configuration.
This commit updates the rustc info cache to cache failures to execute
rustc as well as successes. This fixes a weird issue where if you're
probing for flags the `rustc_info_cache` test fails on channels which
don't have the flag since previously a failure to execute rustc resulted
in never caching the result.
This commit adds a new `split-debuginfo` option to Cargo compilation
profiles which gets forwarded to the `-Csplit-debuginfo` codegen option
in rustc. This commit also sets the default, only on macOS, to be
`-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage
rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds
while also preserving a pleasant debugging experience by default. This
should lead to much faster incremental build times on macOS since
`dsymutil` isn't exactly the speediest tool in the world.
This is technically a breaking change in Cargo because we're no longer
by-default producing the `*.dSYM` folders on macOS. If those are still
desired, however, authors can always run `dsymutil` themselves or
otherwise configure `split-debuginfo = 'packed'` in their
manifest/profile configuration.
Implement external credential process. (RFC 2730)
This adds a config setting for an external process to run to fetch the token for a registry. See `unstable.md` for more details.
As part of this, it adds a new `logout` command. This is currently gated on nightly with the appropriate `-Z` flag.
I have included four sample wrappers that integrate with the macOS Keychain, Windows Credential Manager, GNOME libsecret, and 1password. I'm not sure if we'll ultimately ship these, but I would like to. Primarily this provided a proof-of-concept to see if the design works.
**Patch Walkthrough**
This is a brief overview of the changes:
- Adds the `logout` command. With `cargo logout -Z unstable-options`, this allows removing the `token` from `.cargo/credentials`. With `cargo logout -Z credential-process`, this launches the process with the `erase` argument to remove the token from storage.
- Credential-process handling is in the `ops/registry/auth.rs` module. I think it is pretty straightforward, it just launches the process with the appropriate store/get/erase argument.
- `ops::registry::registry()` now returns the `RegistryConfig` to make it easier to pass the config information around.
- `crates/credential/cargo-credential` is a helper crate for writing credential processes.
- A special shorthand of the `cargo:` prefix for a credential process will launch the named process from the `libexec` directory in the sysroot (or, more specifically, the `libexec` directory next to the `cargo` process). For example `credential-process = "cargo:macos-keychain"`. My intent is to bundle these in the pre-built rust-lang distributions, and this should "just work" when used with rustup. I'm not sure how that will work with other Rust distributions, but I'm guessing they can figure it out. This should make it much easier for users to get started, but does add some integration complexity.
**Questions**
- I'm on the fence about the name `credential-process` vs `credentials-process`, which sounds more natural? (Or something else?)
- I'm uneasy about the behavior when both `token` and `credential-process` is specified (see `warn_both_token_and_process` test). Currently it issues a warning and uses `token`. Does that make sense? What about the case where you have `registries.foo.token` for a specific registry, but then have a general `registry.credential-process` for the default (it currently warns and uses the token, maybe it should not warn?)?
- I am still pretty uneasy with writing FFI wrappers, so maybe those could get a little extra scrutiny? They seem to work, but I have not extensively tested them (I tried login, publish, and logout). I have not previously used these APIs, so I am not familiar with them.
- Testing the wrappers I think will be quite difficult, because some require TTY interaction (and 1password requires an online account). Or, for example in the macOS case, it has GUI dialog box where I can use my fingerprint scanner. Right now, I just build them in CI to make sure they compile.
- 1password is a little weird in that it passes the token on the command-line, which is not very secure on some systems (other processes can see these sometimes). The only alternative I can think of is to not support `cargo login` and require the user to manually enter the token in the 1password GUI. I don't think the concern is too large (1password themselves seem to think it is acceptable). Should this be OK?
- I'm a little uneasy with the design of `cargo login`, where it passes the token in stdin. If the wrapper requires stdin for user interaction (such as entering a password to unlock), this is quite awkward. There is a hack in the 1password example that demonstrates using `/dev/tty` and `CONIN$`, which *seems* to work, but I'm worried is fragile. I'm not very comfortable passing the token in environment variables, because those can be visible to other processes (like CLI args), but in some situations that shouldn't be too risky. Another option is to use a separate file descriptor/handle to pass the token in. Implementing that in Rust in a cross-platform way is not terribly easy, so I wanted to open this up for discussion first.
