review and remove ignored tests in rustfix
### What does this PR try to resolve?
review ignored tests in rustfix crate per #13034.
### How should we test and review this PR?
CI testing
### Additional information
* Removed unproductive test in `parse_and_replace`
* un-ignore proptests, and reduce runtime from ~2s to ~<.25s
Migrate rustfix to the cargo repo
This migrates the `rustfix` crate from https://github.com/rust-lang/rustfix/ to the cargo repo. The cargo team has been responsible for the client-side of `cargo fix`, and it can make it easier to maintain with all our tooling and tracking here. This crate is used by some external parties (like the compiler), so it will need to be maintained like an "ecosystem" package, but hopefully there shouldn't be any outside requirements (I haven't seen any in several years).
After merging, I'll follow up with some things to address in the future, such as:
- Migrating issues from the other repo.
- Opening new issues for some cleanup tasks, such as adding documentation, fixing the `#[ignore]` annotations, fixing testing on windows, maybe migrating the test code to use different dependencies, various code cleanup.
- Archiving the repo.
This fixes an issue where `--quiet` doesn't work with commands that have
subcommands. This is because `config_configure` only looks at the global
and top-level subcommand, and not deeper subcommands. The issue was that
`--quiet` was not defined as a global flag. This was changed in
https://github.com/rust-lang/cargo/pull/6358 in order to give a better
help message for `cargo test --quiet`. I don't remember if clap just
didn't support overriding at the time, or if we just didn't know how it
worked. Anyways, it seems to work to override it now, so I think it
should be fine to mark it as global.
This should bring in `--quiet` more in-line with how `--verbose` works.
This means that `--quiet` is now accepted with `cargo report`,
`cargo help`, and `cargo config`.
This also fixes `--quiet` with `cargo clean gc`.
This should also help with supporting `--quiet` with the new `cargo
owner` subcommands being added in
https://github.com/rust-lang/cargo/pull/11879.
Fixes#12957
This adds a garbage collector which will remove old files from cargo's
global cache.
A general overview of the changes here:
- `cargo::core::global_cache_tracker` contains the `GlobalCacheTracker`
which handles the interface to a sqlite database which stores
timestamps of the last time a file was used.
- `DeferredGlobalLastUse` is a type that implements an optimization for
collecting last-use timestamps so that they can be flushed to disk all
at once.
- `cargo::core::gc` contains the `Gc` type which is the interface for
performing garbage collection. It coordinates with the
`GlobalCacheTracker` for determining what to delete.
- Garbage collection can either be automatic or manual. The automatic
garbage collection supports some config options for defining when
it runs and how much it deletes.
- Manual garbage collection can be performed via options to `cargo
clean`.
- `cargo clean` uses the new package cache locking system to coordinate
access to the package cache to prevent interference with other cargo
commands running concurrently.
This adds a very primitive `du` function for determining the disk usage
in a directory. This should probably be improved or replaced at some
point in the future, this is just a start for supporting tracking sizes
of cache data.
For `cargo install` we'll now show a more specific parse error for
semver, much like other parts of cargo.
This came out of my work on #12801. I was looking at what might be
appropriate to put in a `cargo-util-semver` crate and realized we have
the `ToSemver` trait that exists but doesn't do much, so I dropped it.
I've wanted something like this myself. I dislike using `--open`
because I tend to move up to re-run my `cargo doc` run but then have to
edit it to remove `--open`.
Also makes it annoying when opening docs when `cargo doc` is wrapped by
a tool like `make`.
This was previously attempted in #5592:
- Unlike the request in #5562, this aligns with #5592 in always printing
rather than using a flag as this seems generally useful
- Unlike #5592, this prints as an alternative to "Opening" to keep
things light
- Unlike #5592, this prints afterwards as the link is only valid then
Fixes#5562
add detailed message when target folder path is invalid
# What does this PR try to resolve?
close https://github.com/rust-lang/cargo/issues/12789
add more detailed message when target folder path is invalid
# How should we test and review this PR?
Before this PR, if the target folder refer to broken symbolic link like /a/b/c, then run cargo build, the output is:
```
error: Not a directory (os error 20)
```
the detailed error message is missing.
This PR will add the error context for it, the finall output will be
```
cargo build
error: failed to create directory `/root/workspace/playground/target`
Caused by:
Not a directory (os error 20)
```
Add new package cache lock modes
The way locking worked before this PR is that only one cargo could write to the package cache at once (otherwise it could cause corruption). However, it allowed cargo's to read from the package cache while running a build under the assumption that writers are append-only and won't affect reading. This allows multiple builds to run concurrently, only blocking on the part where it is not possible to run concurrently (downloading to the cache).
This introduces a new package cache locking strategy to support the ability to safely modify existing cache entries while other cargos are potentially reading from the cache. It has different locking modes:
- `MutateExclusive` (new) — Held when cargo wants to modify existing cache entries (such as being introduced for garbage collection in #12634), and ensures only one cargo has access to the cache during that time.
- `DownloadExclusive` (renamed) — This is a more specialized name for the lock that was before this PR. A caller should acquire this when downloading into the cache and doing resolution. It ensures that only one cargo can append to the cache, but allows other cargos to concurrently read from the cache.
- `Shared` (new) — This is to preserve the old concurrent build behavior by allowing multiple concurrent cargos to hold this while a build is running when it is reading from the cache
**Reviewing suggestions:**
There are a few commits needed to help with testing which are first. The main commit has the following:
- `src/cargo/util/cache_lock.rs` is an abstraction around package cache locks, and is the heart of the change. It should have comments and notes which should guide what it is doing. The `CacheLocker` is stored in `Config` along with all our other global stuff.
- Every call to `config.acquire_package_cache_lock()` has been changed to explicitly state which lock mode it wants to lock the package cache in.
- `Context::compile` is the key point where the `Shared` lock is acquired, ensuring that no mutation is done while the cache is being read.
- `MutateExclusive` is not used in this PR, but is being added in preparation for #12634.
- The non-blocking `try_acquire_package_cache_lock` API is not used in this PR, but is being added in preparation for #12634 to allow automatic gc to skip running if another cargo is already running (to avoid unnecessary blocking).
- `src/cargo/util/flock.rs` has been updated with some code cleanup (removing unused stuff), adds support for non-blocking locks, and renames some functions to make their operation clearer.
- `tests/testsuite/cache_lock.rs` contains tests for all the different permutations of ways of acquiring locks.
This introduces a new `CacheLocker` which manages locks on the package
cache. Instead of either being "locked" or "not locked", the new locker
supports multiple modes:
- Shared lock: Cargo can read from the package sources, along with any
other cargos reading at the same time.
- Download exclusive lock: Only one cargo can perform downloads.
Download locks do not interfere with Shared locks, since it is
expected that downloading does not modify existing files (only adds
new ones).
- Mutate exclusive lock: Only one cargo can have this lock, and it also
prevents shared locks. This is so that the cargo can modify the
package cache (such as deleting files) without breaking concurrent
processes.
