reset lockfile information between resolutions
#8249 pointed out that some kind of lockfile data was leaking between calls to the resolver. @ehuss made a reproducing test case. This PR resets the `LockedMap` data structure when calling `register_previous_locks`.
lets see if CI likes it.
fix#8249
This stops using `to_string` as a proxy for this now-provided precise API.
This reverts commit b71927224f and bupms the
dependency version in Cargo.toml.
CARGO_BIN_NAME is added for binary compilation units, CARGO_CRATE_NAME is added for binary and library units.
The `crate_env_vars` test was updated to test for this. The test is currently only checking behavior for the binary compile unit.
Fix pdb uplift when executable has dashes.
Windows `.pdb` files were not being uplifted for executables with dashes in their name. `rustc` calls the linker with the crate name (with underscores), which creates a pdb with underscores. Cargo renames the executable (`foo_bar.exe` to `foo-bar.exe`), and it was expecting the pdb to have the same form, but it doesn't.
Note: There shouldn't be any effect for using a debugger. Because the pdb path is embedded in the executable, the debugger was already looking in the `deps/` folder. Uplifting is only useful if you want to copy the exe/pdb pair to some other machine. In that case, it looks in the same directory as the `exe` for the pdb file.
Fixes#8117
Add "build-finished" JSON message.
This adds a JSON message when a build is finished. This is useful for tools to know when to stop parsing JSON, which is particularly useful for commands like `cargo test` or `cargo run` where additional output may follow.
Closes#7978
This commit is the Cargo half of support necessary for
rust-lang/rust#70458. Today the compiler emits embedded bytecode in
rlibs by default, but compresses it. This is both extraneous disk space
and wasted build time for almost all builds, so the PR in question there
is changing rustc to have a `-Cembed-bitcode` flag which, when enabled,
places the bitcode in the object file rather than an auxiliary file (no
extra compression), but also enables `-Cembed-bitcode=no` to disable
bitcode emission entirely.
This Cargo support changes Cargo to pass `-Cembed-bitcode=no` for almost
all compilations. Cargo will keep `lto = true` and such working by not
passing this flag (and thus allowing bitcode to get embedded), but by
default `cargo build` and `cargo build --release` will no longer have
any bitcode in rlibs which should result in speedier builds!
Most of the changes here were around the test suite and various
assertions about the `rustc` command lines we spit out. One test was
hard-disabled until we can get `-Cembed-bitcode=no` into nightly, and
then we can make it a nightly-only test. The test will then be stable
again once `-Cembed-bitcode=no` hits stable.
Note that this is intended to land before the upstream `-Cembed-bitcode`
change. The thinking is that we'll land everything in rust-lang/rust all
at once so there's no build time regressions for anyone. If we were to
land the `-Cembed-bitcode` PR first then there would be a build time
regression until we land Cargo changes because rustc would be emitting
uncompressed bitcode by default and Cargo wouldn't be turning it off.
Credentials are always loaded, even if these are not used. If
access to confidential files such as credentials is not given,
`cargo build` fails despite not using credentials.
Fixes#7624.
A test when argument is negative is added. In addition,
`default_cargo_config_jobs` and `good_cargo_config_jobs` is moved from
`testsuite/bad_config.rs` to `testsuite/build.rs` because these tests
are not for `bad config`.
Stabilize cache-messages
This stabilizes the -Zcache-messages feature, making it always enabled.
## What is stabilized?
This feature is intended to redisplay previous warnings on a "fresh" build instead of displaying no output.
Users have occasionally indicated frustration when they know there are warnings, but no output is displayed when the build is fresh. This also improves the interaction between `cargo check` and `cargo clippy-preview`. This also simplifies the code, and opens more possibilities for `rustc` to send side-channel messages to Cargo.
Cargo will now use JSON output from `rustc` and `rustdoc` 100% of the time (`rustdoc --test` does not use JSON). Previously Cargo would only use JSON for pipelined crates.
Cargo will save the JSON output into a file named `output` in the `.fingerprint` directory. This file is only created when the compiler outputs a diagnostic message.
If a crate is being recompiled, and Cargo considers it to be "fresh", it will replay the output file to the console.
## Notable changes in this PR
- Fixed a bug where replays were erroneously including pipeline rmeta artifact json messages.
- clippy-preview is now included in the metadata hash, to force its artifacts to be separate from `cargo check`.
- clippy-preview is no longer force-enabled, under the assumption that caching and fingerprinting is accurate, and the cached messages will be replayed.
- clippy-preview's arguments are included in the fingerprint now that it is not force-enabled.
- Rustdoc colors and short messages were fixed when pipelining was stabilized, so updated tests.
Closes#6986Closes#6848Closes#6664Closes#2458
## Concerns
The only notable issue with this is that switching between short and long human messages only replays the format from the first invocation. That is, if you do `cargo build` and it generates warnings, then running again with `--message-format=short` will still show the full length human messages. I'm personally fine with that behavior, even though it is not ideal. I think this feature overall improves the situation (where before *no* output was displayed). Being able to re-render between short/long is a very difficult problem, and unlikely to be fixable in the foreseeable future.
There was some concern expressed about being able to disable this. I think that would only be necessary if a severe bug is discovered. I do not feel that this change is risky enough to warrant a configurable option. If it does cause a problem, it can be quickly reverted with a one-line change to set `OutputOptions::cache_cell` to `None`. Since pipelining has been using JSON output for a while now without complaints, I feel pretty confident in it.
First reported in rust-lang/rust#65014 it looks like our error message
on cyclic dependencies may be confusing at times. It looks like this is
an issue because there are multiple paths through a graph for a
dependency, so using the generic `path_to_top` function isn't producing
the most useful path for this purpose.
We're already walking the graph though, so this commit adds an extra
parameter which collects the list of packages we've visited so far to
produce a hopefully always-accurate error message showing the chain of
dependencies end-to-end for what depends on what.
We were sporadically but persistently seeing errors like
failed to link or copy `.../target/debugs/deps/bin-264030cd6c8a02be.dSYM` to `.../target/debug/bin.dSYM`
Caused by:
the source path is not an existing regular file
while running `cargo build`. Once the error occurs once, `cargo build`
will fail forever with the same error until `target/debug/bin.dSYM` is
manually unlinked.
After some investigation, I've determined that this situation arises
when the target of `bin.dSYM` goes missing. For example, if bin.dSYM is
pointing at `deps/bin-86908f0fa7f1440e.dSYM`, and
`deps/bin-86908f0fa7f1440e.dSYM` does not exist, then this error will
occur. I'm still not clear on why the underlying dSYM bundle
sporadically goes missing--perhaps it's the result of pressing Ctrl-C at
the wrong moment?--but Cargo should at least be able to handle this
situation better.
It turns out that Cargo was getting confused by the broken symlink. When
it goes to install the new `target/debug/bin.dSYM` link, it will remove
the existing `target/debug/bin.dSYM` file, if it exists. Unfortunately,
Cargo was checking whether the *target* of that symlink existed (e.g.,
`deps/bin-86908f0fa7f1440e.dSYM`, which in the buggy case would not
exist), rather than the symlink itself, deciding that there was no
existing symlink to remove, and crashing with EEXIST when trying to
install the new symlink.
This commit adjusts the existence check to evaluate whether the symlink
itself exists, rather than its target.
Note that while the symptoms are the same as #4671, the root cause is
unrelated.
Cargo has of #7143 enabled pipelined compilation by default which
affects how the compiler is invoked, especially with respect to JSON
messages. This, in some testing, has proven to cause quite a few issues
with rustbuild's current integration with Cargo. This commit is aimed at
adding features to Cargo to solve this issue.
This commit adds the ability to customize the stream of JSON messages
coming from Cargo. The new feature for Cargo is that it now also mirrors
rustc in how you can configure the JSON stream. Multiple
`--message-format` arguments are now supported and the value specified
is a comma-separated list of directives. In addition to the existing
`human`, `short`, and `json` directives these new directives have been
added:
* `json-render-diagnostics` - instructs Cargo to render rustc
diagnostics and only print out JSON messages for artifacts and Cargo
things.
* `json-diagnostic-short` - indicates that the `rendered` field of rustc
diagnostics should use the "short" rendering.
* `json-diagnostic-rendered-ansi` - indicates that the `rendered` field of rustc
diagnostics should embed ansi color codes.
The first option here, `json-render-diagnostics`, will be used by
rustbuild unconditionally. Additionally `json-diagnostic-short` will be
conditionally used based on the input to rustbuild itself.
This should be enough for external tools to customize how Cargo is
invoked and how all kinds of JSON diagnostics get printed, and it's
thought that we can relatively easily tweak this as necessary to extend
it and such.
Some tools can support rust doctests (e.g. highlighting
or launching). So it should be possible to find out if
doctests are enabled for a target or not. This commit
adds `doctest` field to `cargo metadata` output.
Implement the Cargo half of pipelined compilation (take 2)
This commit starts to lay the groundwork for #6660 where Cargo will
invoke rustc in a "pipelined" fashion. The goal here is to execute one
command to produce both an `*.rmeta` file as well as an `*.rlib` file
for candidate compilations. In that case if another rlib depends on that
compilation, then it can start as soon as the `*.rmeta` is ready and not
have to wait for the `*.rlib` compilation.
Initially attempted in #6864 with a pretty invasive refactoring this
iteration is much more lightweight and fits much more cleanly into
Cargo's backend. The approach taken here is to update the
`DependencyQueue` structure to carry a piece of data on each dependency
edge. This edge information represents the artifact that one node
requires from another, and then we a node has no outgoing edges it's
ready to build.
A dependency on a metadata file is modeled as just that, a dependency on
just the metadata and not the full build itself. Most of cargo's backend
doesn't really need to know about this edge information so it's
basically just calculated as we insert nodes into the `DependencyQueue`.
Once that's all in place it's just a few pieces here and there to
identify compilations that *can* be pipelined and then they're wired up
to depend on the rmeta file instead of the rlib file.
Closes#6660
This commit fixes an issue when pipelining mode is used in handling
recompilations. Previously a sequence of compilations could look like:
* Crate A starts to build
* Crate A produces metadata
* Crate B, which depends on A, starts
* Crate B finishes
* Crate A finishes
In this case the mtime for B is before that of A, which fooled Cargo
into thinking that B needed to be recompiled. In this case, however, B
doesn't actually need to be recompiled because it only depends on the
metadata of A, not the final artifacts.
This unfortunately resulted in some duplication in a few places, but not
really much moreso than already exists between fingerprinting and compilation.
This commit updates Cargo to process JSON directives emitted by rustc
when we're pipelining compilation. In this mode Cargo will attempt to
start subsequent compilations of crates as soon as possible, fully
completing the features of pipelined compilations in Cargo!
This commit starts to lay the groundwork for #6660 where Cargo will
invoke rustc in a "pipelined" fashion. The goal here is to execute one
command to produce both an `*.rmeta` file as well as an `*.rlib` file
for candidate compilations. In that case if another rlib depends on that
compilation, then it can start as soon as the `*.rmeta` is ready and not
have to wait for the `*.rlib` compilation.
Initially attempted in #6864 with a pretty invasive refactoring this
iteration is much more lightweight and fits much more cleanly into
Cargo's backend. The approach taken here is to update the
`DependencyQueue` structure to carry a piece of data on each dependency
edge. This edge information represents the artifact that one node
requires from another, and then we a node has no outgoing edges it's
ready to build.
A dependency on a metadata file is modeled as just that, a dependency on
just the metadata and not the full build itself. Most of cargo's backend
doesn't really need to know about this edge information so it's
basically just calculated as we insert nodes into the `DependencyQueue`.
Once that's all in place it's just a few pieces here and there to
identify compilations that *can* be pipelined and then they're wired up
to depend on the rmeta file instead of the rlib file.
Display environment variables for rustc commands
This picks up on the work done in PR #5683.
The extra output is only displayed with `-vv`.
The Windows output has the form `set FOO=foo && BAR=bar rustc ...` instead of
the form that suggested in #5683 to make escaping easier and since it's
simpler.
This picks up on the work done in PR #5683.
The extra output is only displayed with `-vv`.
The Windows output has the form `set FOO=foo && BAR=bar rustc ...` instead of
the form that suggested in #5683 to make escaping easier and since it's
simpler.
Use "test" profile for `cargo build` benchmarks.
When using `cargo build` (without `--release`), build benchmarks using the "test" profile. This was causing some confusion where the benchmark is placed in the `target/debug` directory, and also causing some duplicates that may not be expected. It also makes it easier to debug benchmarks (previously you had to edit the `[profile.bench]` profile).
Closes#5575, closes#6301, closes#4240, closes#4929.
If rustc exits with a signal, previously all you saw was:
```
Compiling foo ...
error: Could not compile `foo`.
To learn more, run the command again with --verbose.
```
Now it shows the complete error by default:
```
Compiling foo ...
error: Could not compile `foo`.
Caused by:
process didn't exit successfully: `rustc ...` (signal: 6, SIGABRT: process abort signal)
```
build: export repository information
The repository string is made available via the `CARGO_PKG_REPOSITORY`
environment variable similar to other metadata.
Validate that the package name contains no invalid characters.
Fixes#2388.
Invalid characters are currently defined as alphanumeric, _, and -. This matches the rustc restrictions but is not as restrictive as `cargo new` or crates.io.
Mostly this is just so there will be better error messages in the case where characters in the package name aren't valid path characters.
Invalid characters are currently defined as alphanumeric, _, and -. This matches the rustc restrictions but is not as restrictive as `cargo new` or crates.io.
Mostly this is just so there will be better error messages in the case where characters in the package name aren't valid path characters.
Don't print crates.io-index URL when Updating
Following the lead from PackageId's Display, only display the registry's
URL if it's not the default registry (aka crates.io).
Before:
$ cargo install lazy_static
Updating registry `https://github.com/rust-lang/crates.io-index`
After:
$ dcargo install lazy_static
Updating crates.io index
Fixes#4208
This change ensures cargo will output file paths in the expected format
(C:\foo\... on Windows, /foo/... elsewhere). Previously it would output
file:// URLs instead.
To support this change, additional changes were made to the test suite
string processing such that [ROOT] is now replaced with the appropriate
file path root for the platform.
The CWD template was also updated to use [CWD] like other replacement
templates and to do the replacement on the expected value rather than
the actual value to avoid replacing things we don't expect with CWD.
This commit stabilizes the `edition` key in `Cargo.toml`, both in the
`[package]` section and inside subtargets. Additionally the `cargo new` and
`cargo init` subcommands have been enhanced with a `--edition` flag to allow
explicitly specifying the edition to be generated.
This commit does not yet change the default edition that's generated.
Closes#5980
.. with mutliple calls of:
fastmod --accept-all '\.cargo\("([^"]+)"\)\.arg\("([^"]+)"\)' '.cargo("${1} ${2}")' tests/testsuite/
until no changes are left.
This changes it so that filters like `--bin`, `--test`, `--example`,
`--bench`, `--lib` will work in a workspace. Today, these filters
require that they match *every* package in a workspace which makes
them not very useful. This change makes it so that they only must
match at least one package.
* Collapse the nested cargotest::support module into the cargotest
module (merge the mod.rs's)
* Rename the cargotest module to support
* Nest the top-level hamcrest module into support
Generally that means either switching "foo" and "bar" around (reversing
the arrow), or it means push "foo" to "bar" (and sometimes "bar" to
"baz", etc..) to free up "foo".
For trivia that leaves 80/1222 outliers, therefore 93.4% of test
project use the default. :)
By rewriting the tests, with rerast (https://github.com/google/rerast),
to use the newly introduced "at" method.
First I added the following temporary function to cargotest::support:
pub fn project_foo() -> ProjectBuilder {
project("foo")
}
Then I defined the following rewrite.rs:
use cargotest::support::{ project, project_foo };
fn rule1(a: &'static str) {
replace!(project("foo") => project_foo());
replace!(project(a) => project_foo().at(a));
}
Then I ran rerast:
cargo +nightly rerast --rules_file=rewrite.rs --force --targets tests --file tests/testsuite/main.rs
Finally I searched and replaced the references to project_foo with
argument-less project (a little awkardly on macOS with a git clean).
find tests -type f -exec sed -i -e 's/project_foo/project/g' {} +
git clean -d tests
This commit imports the `cargo fix` subcommand in rust-lang-nursery/rustfix
directly into Cargo as a subcommand. This should allow us to ease our
distribution story of `cargo fix` as we prepare for the upcoming 2018 edition
release.
It's been attempted here to make the code as idiomatic as possible for Cargo's
own codebase. Additionally all tests from cargo-fix were imported into Cargo's
test suite as well. After this lands and is published in nightly the `cargo-fix`
command in rust-lang-nursery/rustfix will likely be removed.
cc rust-lang/rust#52272
In Rust 2015 absence of the configuration makes it default to not
include auto-discovered targets (i.e false), with a warnings message.
In Rust 2018 absence makes it default to include auto-discovered
targets (i.e true).
Fixes#5330
Stricter need_dev_deps behaviour
The previous PR (#5012) contained an unnecessary work-around for behaviour of `--all-targets` that was misunderstood. This PR removes that work-around and adds some tests and comments to clarify the behaviour for future contributors, which may help to make easier a future fix for #5177 and #5178.
Do not allow crate-type or proc-macro for [[bin]]-targets
Fixes#5199
This simply disallows `proc-macro` and `crate-type` to be set to anything for binary targets. Is this the best way to go or does a warning about the unused setting suffice?
One historical annoyance I've always had with Cargo that I've found surprising
is that in some situations when you bump version numbers you'll have to end up
running `cargo update` later on to get everything to build. You get pretty wonky
error messages in this case as well saying a package doesn't exist when it
clearly does at a particular location!
I've had difficulty historically nailing down a test case for this but it looks
like we ironically already had one in our test suite and I also jury-rigged up
one from a case I ran into in the wild today.
Depending on the OS, there might be an additional artifacts for
debuginfo (dSYM folder for macs, .pbd file for windows). Given that we
can't disable `.pdb` for windows[1], let's just ignore all filenames!
[1]: https://github.com/rust-lang/rust/pull/28505)
`.pdb` files are for windows debug info (unlike on linux, debug info is
in a separate file). Windows executable actually hard-code paths to
`.pdb` files, so debugging mvsc rust programs works even without this
patch. However, if you want to distribute the executable to other
machines, you'd better distribute both `foo.exe` and `foo.pdb`, because
absolute paths won't work on another machine. Having same-named .pdb
file alongside the binary would work though.
closes#4960
Clap
Reopening of #5129
So, looks like all tests are 🍏 on my machine!
I definitely want to refactor it some more, and also manually checked that we haven't regressed any help messages, but all the major parts are in place already.
Error message for package "depends on itself" lists the packages in the cycle.
I got a cycle while trying to build someone else's code and cargo's error message didn't point me in the right direction, just mentioned there was a cycle. I thought we could be a bit more helpful.
Don't know what you think of {:#?} as the display but it seemed minimal code so I thought I'd start with that. I could compress the output to one package per line if preferred.
* remove Workspace::current_manifest
* remove incorrect logging
* move empty check to Packages::into_package_id_specs
* add test case mentioned in issue
