Previously Cargo would attempt to work as much as possible with a
previously filled out CARGO_HOME, even if it was mounted as read-only.
In #6880 this was regressed as a few global locks and files were always
attempted to be opened in writable mode.
This commit fixes these issues by correcting two locations:
* First the global package cache lock has error handling to allow
acquiring the lock in read-only mode inaddition to read/write mode. If
the read/write mode failed due to an error that looks like a readonly
filesystem then we assume everything in the package cache is readonly
and we switch to just acquiring any lock, this time a shared readonly
one. We in theory aren't actually doing any synchronization at that
point since it's all readonly anyway.
* Next when unpacking package we're careful to issue a `stat` call
before opening a file in writable mode. This way our preexisting guard
to return early if a package is unpacked will succeed before we open
anything in writable mode.
Closes#6928
This accidentally regressed in the previous caching PR for Cargo.
Invalid lines of JSON in the registry are intended to be skipped over,
but when skipping we forgot to update some indices which meant that all
future versions would fail to parse as well!
This commit tweaks the wording for when versions aren't found in a
source, notably including more location information other than the
source id but rather also taking into account replaced sources and such.
It's hoped that this gives more information to make it a bit more clear
what's going on.
Closes#6111
This commit is intended to lay the groundwork for downloading crates in parallel
from crates.io. It starts out by lifting up the download operation from deep
within the `Source` trait to the `PackageSet`. This should allow us to maintain
a queue of downloads specifically in a `PackageSet` and arbitrate access through
that one type, making it easier for us to implement parallel downloads.
The `Source` trait's `download` method may now fail saying "go download this
data", and then the data is fed back into the trait object once it's complete to
actually get the `Package` out.
Fix publishing renamed dependencies to crates.io
This commit fixes publishing crates which contain locally renamed dependencies
to crates.io. Previously this lack of information meant that although we could
resolve the crate graph correctly it wouldn't work well with respect to optional
features and optional dependencies. The fix here is to persist this information
into the registry about the crate being renamed in `Cargo.toml`, allowing Cargo
to correctly deduce feature names as it does when it has `Cargo.toml` locally.
A dual side of this commit is to publish this information to crates.io. We'll
want to merge the associated PR (link to come soon) on crates.io first and make
sure that's deployed as well before we stabilize the crate renaming feature.
The index format is updated as well as part of this change. The `name` key for
dependencies is now unconditionally what was written in `Cargo.toml` as the
left-hand-side of the dependency specification. In other words this is the raw
crate name, but only for the local crate. A new key, `package`, is added to
dependencies (and it can be `None`). This key indicates the crates.io package is
being linked against, an represents the `package` key in `Cargo.toml`.
It's important to consider the interaction with older Cargo implementations
which don't support the `package` key in the index. In these situations older
Cargo binaries will likely fail to resolve entirely as the renamed name is
unlikely to exist on crates.io. For example the `futures` crate now has an
optional dependency with the name `futures01` which depends on an older version
of `futures` on crates.io. The string `futures01` will be listed in the index
under the `"name"` key, but no `futures01` crate exists on crates.io so older
Cargo will generate an error. If the crate does exist on crates.io, then even
weirder error messages will likely result.
Closes#5962
This change ensures cargo will output file paths in the expected format
(C:\foo\... on Windows, /foo/... elsewhere). Previously it would output
file:// URLs instead.
To support this change, additional changes were made to the test suite
string processing such that [ROOT] is now replaced with the appropriate
file path root for the platform.
The CWD template was also updated to use [CWD] like other replacement
templates and to do the replacement on the expected value rather than
the actual value to avoid replacing things we don't expect with CWD.
This commit fixes publishing crates which contain locally renamed dependencies
to crates.io. Previously this lack of information meant that although we could
resolve the crate graph correctly it wouldn't work well with respect to optional
features and optional dependencies. The fix here is to persist this information
into the registry about the crate being renamed in `Cargo.toml`, allowing Cargo
to correctly deduce feature names as it does when it has `Cargo.toml` locally.
A dual side of this commit is to publish this information to crates.io. We'll
want to merge the associated PR (link to come soon) on crates.io first and make
sure that's deployed as well before we stabilize the crate renaming feature.
The index format is updated as well as part of this change. The `name` key for
dependencies is now unconditionally what was written in `Cargo.toml` as the
left-hand-side of the dependency specification. In other words this is the raw
crate name, but only for the local crate. A new key, `package`, is added to
dependencies (and it can be `None`). This key indicates the crates.io package is
being linked against, an represents the `package` key in `Cargo.toml`.
It's important to consider the interaction with older Cargo implementations
which don't support the `package` key in the index. In these situations older
Cargo binaries will likely fail to resolve entirely as the renamed name is
unlikely to exist on crates.io. For example the `futures` crate now has an
optional dependency with the name `futures01` which depends on an older version
of `futures` on crates.io. The string `futures01` will be listed in the index
under the `"name"` key, but no `futures01` crate exists on crates.io so older
Cargo will generate an error. If the crate does exist on crates.io, then even
weirder error messages will likely result.
Closes#5962
* Collapse the nested cargotest::support module into the cargotest
module (merge the mod.rs's)
* Rename the cargotest module to support
* Nest the top-level hamcrest module into support
Generally that means either switching "foo" and "bar" around (reversing
the arrow), or it means push "foo" to "bar" (and sometimes "bar" to
"baz", etc..) to free up "foo".
For trivia that leaves 80/1222 outliers, therefore 93.4% of test
project use the default. :)
By rewriting the tests, with rerast (https://github.com/google/rerast),
to use the newly introduced "at" method.
First I added the following temporary function to cargotest::support:
pub fn project_foo() -> ProjectBuilder {
project("foo")
}
Then I defined the following rewrite.rs:
use cargotest::support::{ project, project_foo };
fn rule1(a: &'static str) {
replace!(project("foo") => project_foo());
replace!(project(a) => project_foo().at(a));
}
Then I ran rerast:
cargo +nightly rerast --rules_file=rewrite.rs --force --targets tests --file tests/testsuite/main.rs
Finally I searched and replaced the references to project_foo with
argument-less project (a little awkardly on macOS with a git clean).
find tests -type f -exec sed -i -e 's/project_foo/project/g' {} +
git clean -d tests
Currently Cargo can deadlock itself via file locks in somewhat obscure
scenarios. One way to trigger this scenario is by causing the index to fail
being created, for example through an invalid `init.templatedir` configuration.
This commit takes the strategy of solving two bugs:
* First, the deadlock is fixed. This is done by manually ensuring that the
current deadlock doesn't happen by scheduling cached work to happen outside
the scope of a lock.
* Second, the initialization of the registry's index is fixed. We turn off the
usage of external templates as we don't want to use them for this internally
managed repository anyway.
Closes#5551
This commit is directed at fixing #4127, allowing the resolver to automatically
perform transitive updates when required. A few use casese and tagged links are
hanging off #4127 itself, but the crux of the issue happens when you either add
a dependency or update a version requirement in `Cargo.toml` which conflicts
with something listed in your `Cargo.lock`. In this case Cargo would previously
provide an obscure "cannot resolve" error whereas this commit updates Cargo to
automatically perform a conservative re-resolution of the dependency graph.
It's hoped that this commit will help reduce the number of "unresolvable"
dependency graphs we've seen in the wild and otherwise make Cargo a little more
ergonomic to use as well. More details can be found in the source's comments!
Closes#4127Closes#5182
