Fix freshness when linking is interrupted.
Fixes a scenario where hitting Ctrl-C while linking would leave a corrupted executable, but Cargo would think it is "fresh" and fail to rebuild it.
This also includes a separate commit which adds more documentation on fingerprinting.
Fixes#7767
Add `cargo tree` command.
This migrates [cargo-tree](https://github.com/sfackler/cargo-tree/) into Cargo as a built-in command. This is based on a recent master (4108d216ec), and should be mostly similar in functionality. There are a variety changes:
* `--all-targets` renamed to `--no-filter-targets` to avoid confusion with the `--all-targets` flag used in other Cargo commands with a different meaning.
* `--all`/`-a` renamed to `--no-dedupe` to avoid confusion with the `-all` flag which means "all workspace crates" in other Cargo commands.
* `--duplicate` renamed to `--duplicates` (with alias), just a personal preference.
* Added support for multiple roots (workspace support).
* Added the `--graph-features` flag for including features in the graph (to "explain" why a feature is enabled).
* Added `{f}` to format string to show features.
* Handles new feature resolver.
* Handles cyclical dev dependencies.
* Added a test suite.
* Dropped the dependency on petgraph, in favor of a simpler custom graph.
Closes#7286.
Add "build-finished" JSON message.
This adds a JSON message when a build is finished. This is useful for tools to know when to stop parsing JSON, which is particularly useful for commands like `cargo test` or `cargo run` where additional output may follow.
Closes#7978
Extend -Zpackage-features with more capabilities.
This is a proposal to extend `-Zpackage-features` with new abilities to change how features are selected on the command-line. See `unstable.md` for documentation on what it does.
I've contemplated a variety of ways we could transition this to stable. I tried a few experiments trying to make a "transition with warnings" mode, but I'm just too concerned about breaking backwards compatibility. The current way is just fundamentally different from the new way, and I think it would be a bumpy ride to try to push it.
The stabilization story is that the parts of this that add new functionality (feature flags in virtual worskpaces, and `member/feat` syntax) can be stabilized at any time. The change for `cargo build -p member --features feat` in a different member's directory can maybe be part of `-Zfeatures` stabilization, which will need to be opt-in. I've been trying to come up with some transition plan, and I can't think of a way without making it opt-in, and making it part of `-Zfeatures` is an opportunity to simplify things. One concern is that this might be confusing (`--features` flag could behave differently in different workspaces, and documenting the differences), but that seems hard to avoid.
Closes#6195Closes#4753Closes#5015Closes#4106Closes#5362
Disallow invalid dependency names through crate renaming
resolves#6656
As suggested in the issue, I simply checked the dep names by calling `validate_package_name` on the dependencies during the TOML deserialization process.
It might be a bit too strict (and sudden) to error out in this case, so it might be best to convert this into a warning instead. However, this _is_ pretty invalid behavior so I'm not too sure really.
Use the same filename hash for pre-release channels.
This changes it so that filenames do not hash the entire verbose version from rustc if they are a pre-release version. The intent is to avoid leaving stale artifacts in the target directory whenever someone updates a nightly or beta release. This should help reduce disk space usage for someone who updates these toolchains frequently.
I tested with the rustc repo, and it seems to be OK. It keeps everything in separate target directories, so I think it should be generally safe. This should only affect someone switching between different nightlies and wanting to avoid recompiling when switching back. I suspect that is a rare use case, though if there are complaints this can be easily reverted (or made a config option). cargo-bisect-rustc should also be safe since it uses a different target directory for each toolchain.
One concern here for me was incremental support. It looks like ([src](6387b09153/src/librustc_incremental/persist/file_format.rs (L88-L100))) the incremental cache includes the detailed rustc version, so I think that is safe. It also looks like it is [smart enough](6387b09153/src/librustc_incremental/persist/load.rs (L40-L49)) to delete stale files.
We will need to be more careful in the future when changing the target directory structure or the format of files. We previously relied on the fact that each new nightly will use different filenames. If we change the structure in a backwards-incompatible way, we will need to be careful to update the version (`1.hash` in `compute_metadata`).
This commit is the Cargo half of support necessary for
rust-lang/rust#70458. Today the compiler emits embedded bytecode in
rlibs by default, but compresses it. This is both extraneous disk space
and wasted build time for almost all builds, so the PR in question there
is changing rustc to have a `-Cembed-bitcode` flag which, when enabled,
places the bitcode in the object file rather than an auxiliary file (no
extra compression), but also enables `-Cembed-bitcode=no` to disable
bitcode emission entirely.
This Cargo support changes Cargo to pass `-Cembed-bitcode=no` for almost
all compilations. Cargo will keep `lto = true` and such working by not
passing this flag (and thus allowing bitcode to get embedded), but by
default `cargo build` and `cargo build --release` will no longer have
any bitcode in rlibs which should result in speedier builds!
Most of the changes here were around the test suite and various
assertions about the `rustc` command lines we spit out. One test was
hard-disabled until we can get `-Cembed-bitcode=no` into nightly, and
then we can make it a nightly-only test. The test will then be stable
again once `-Cembed-bitcode=no` hits stable.
Note that this is intended to land before the upstream `-Cembed-bitcode`
change. The thinking is that we'll land everything in rust-lang/rust all
at once so there's no build time regressions for anyone. If we were to
land the `-Cembed-bitcode` PR first then there would be a build time
regression until we land Cargo changes because rustc would be emitting
uncompressed bitcode by default and Cargo wouldn't be turning it off.
Checking for binary that is built as an implicit dependency of an integration test.
When a project containing binary with required-features, the binary will only be built with the given dependency's feature is enabled. But this feature syntax is not checked when the binary is built as an implicit dependency of an integration test. This pr addresses this issue.
More info is in the issue page: this pr fixes#7853.
Re-implement proc-macro feature decoupling.
This is essentially a rewrite of #8003. Instead of adding proc-macro to the index, it uses a strategy of downloading all packages before doing feature resolution. Then the package can be inspected for the proc-macro field.
This is a fairly major change. A brief overview:
- `PackageSet` now has a `download_accessible` method which tries to download a minimal set of every accessible package. This isn't very smart right now, and errs on downloading too much. In most cases it should be the same (or nearly the same) as before. It downloads extra in the following cases:
- The check for `[target]` dependencies checks both host and target for every dependency. I could tighten that up a little so build dependencies only check for the host, but it would add some complexity and I wanted to get feedback first.
- Optional dependencies disabled by the new feature resolver will get downloaded.
- Removed the loop in computing unit dependencies where downloading used to reside.
- When downloading starts, it should now show a more accurate count of how many crates are to be downloaded. Previously the count would fluctuate while the graph is being built.
Add proc-macro to index, and new feature resolver.
This adds the "pm" field to the index so that Cargo can detect which packages contain a proc-macro without downloading the package.
The second commit builds on that to support proc-macros in the new "de-unification" of the new feature resolver. This prevents dependencies shared between proc-macros and other dependency kinds from having features unified.
cc #7915
Avoid buffering large amounts of rustc output.
If `rustc` prints out a lot of information (such as with `RUSTC_LOG`, or a huge number of diagnostics), cargo would buffer up large amounts of that in memory. For normal builds, this would happen if the terminal does not print fast enough. For "fresh" replay, *everything* was being buffered.
There are two issues:
1. There is no back-pressure on the mpsc queue. If messages come in faster than they can be processed, it grows without bounds.
2. The cache-replay code runs in the "fresh" code path which does not spawn a thread. Thus the main thread was blocked and unable to process `Message`s while the replay is happening.
The solution here is to use a bounded queue, and to always spawn a thread for the "fresh" case.
The main concern here is performance. Previously the "fresh" jobs avoided spawning a thread to improve performance. I did a fair bit of profiling to understand the impact, using projects with anywhere from 100 to 500 units. On my macOS machine, I found spawning a thread to be slightly faster (1-5%). On Linux and Windows, it was generally about 0 to 5% slower. It might be helpful for others to profile it on their own system.
I'm on the fence for the cost/benefit here. It seems generally good to reduce memory usage, but the slight performance hit is disappointing. I tried several other approaches to fix this, all with worse trade offs (I can discuss them if interested).
Fixes#6197
