At least one important I/O related perf fix has landed on nightly (reading
`DT_DIR` for `DirEntry::file_type`) and there are a few other assorted bug fixes
which are being picked up.
Prior to this commit the entire dependency tree would be traversed many, many
times as the short circuits weren't applying fast enough. This commit modifies
the recursion to only happen if the target in question was actually compiled.
This commit saw a noop build time for Servo drop from 4.5s to 2.5s.
This commit adds support for caching the resolved value of a fingerprint among
calls to `resolve`. Due to the recursive nature of a fingerprint, it means that
`resolve` is currently executed a very large number of times for packages which
at transitively dependend up on many times. By caching the returned value of a
fingerprint we're able to prevent extraneous calls into the filesystem or
extraneous hashing.
This also adds an `Arc` to share a `Fingerprint` among all its dependencies as
each fingerprint stores a list of fingerprints that it depends on, and if
they're not shared then the cache isn't exactly the most helpful!
For a noop `cargo build` on Servo (e.g. everything was already built), this
decreased Cargo's runtime from 5s to ~4.5s
This branch includes a change to dl-snapshot.py that makes it less directly linked to the format of snapshots.txt, and also adds an initial FreeBSD x86_64 cargo snapshot. The original URL for my snapshot is:
http://people.freebsd.org/~will/public_distfiles/cargo-nightly-x86_64-unknown-freebsd.tar.gz
SHA256: de678e858de5d2923cc206b7aa1cc84ddc67be00748160f66fc9b7325409e7c7
- Read the file into a nested dictionary instead of variables.
- Only apply special cases where they are really needed.
- Apply a transform to the triple argument to get the platform name to
determine whether there is a snapshot available for it.
- Apply a similar (but slightly different) transform to get the snapshot's
file name. It would be better if the file name matched the platform name
in snapshots.txt, but it's too late for that now (older versions of this
script will certainly expect the existing name format).
cargo build --verbose does output some command lines that cannot be
simply copy and pasted into the shell again. The problem is the
arguments which are output exactly like this: --feature="foo"
When pasted back into the shell, the shell will parse and remove the
double quotes. To counteract this, escape special shell characters when
printing commandlines. Cargo will print --feature=\"foo\" instead, which
can be pasted back into the shell.
Tests are often flaky because the resolution of the OSX filesystem timestamps
appears to be 1s, so we often have to manually sleep for 1s to ensure that
enough time has passed to guarantee that the timestamp is different (used in
freshness calculations).
Tests are often flaky because the resolution of the OSX filesystem timestamps
appears to be 1s, so we often have to manually sleep for 1s to ensure that
enough time has passed to guarantee that the timestamp is different (used in
freshness calculations).
OK, this should fix https://github.com/rust-lang/cargo/issues/1655. It worked on my local. I am not entirely sure how to write a test for this. Can I get some help and be pointed in the right direction?
Unfortunately paths are getting a little too long on Windows and libgit2 is
given us back the error:
Failed to rename lockfile to 'C:/bot/slave/nightly-dist-cargo-win-gnu-64-64/build/target/x86_64-pc-windows-gnu/debug/cit/test-155/home/.cargo/git/checkouts/meta-dep-27104ee87001fd00/08c585f6f3927118da012a2d682b67a789c35211/.git/objects/pack/pack-ba5fd201f4c1be9792e6d54eef194e6b733ae719.idx': The data area passed to a system call is too small.
Windows isn't always the best at handling long paths, so this commit changes the
location of the `cit` output directory where tests are located as well as
shortening the name of each test's folder to hopefully get the folder sizes a
little under the limit.
I believe this test is only failing on the nightly bots because the folder name
for the nightly bots is longer than the folder name for the auto bots. Hurray!
Unfortunately paths are getting a little too long on Windows and libgit2 is
given us back the error:
Failed to rename lockfile to 'C:/bot/slave/nightly-dist-cargo-win-gnu-64-64/build/target/x86_64-pc-windows-gnu/debug/cit/test-155/home/.cargo/git/checkouts/meta-dep-27104ee87001fd00/08c585f6f3927118da012a2d682b67a789c35211/.git/objects/pack/pack-ba5fd201f4c1be9792e6d54eef194e6b733ae719.idx': The data area passed to a system call is too small.
Windows isn't always the best at handling long paths, so this commit changes the
location of the `cit` output directory where tests are located as well as
shortening the name of each test's folder to hopefully get the folder sizes a
little under the limit.
I believe this test is only failing on the nightly bots because the folder name
for the nightly bots is longer than the folder name for the auto bots. Hurray!
This commit adds support to allow specifying a custom output directory to Cargo.
First, the `build.target-dir` configuration key is checked, and failing that the
`CARGO_TARGET_DIR` environment variable is checked, and failing that the root
package's directory joined with the directory name "target" is used.
There are a few caveats to switching target directories, however:
* If the target directory is in the current source tree, and the folder name is
not called "target", then Cargo may walk the output directory when determining
whether a tree is fresh.
* If the target directory is not called "target", then Cargo may look inside it
currently for `Cargo.toml` files to learn about local packages.
* Concurrent usage of Cargo will still result in badness (#354), and this is now
exascerbated because many Cargo projects can share the same output directory.
* The top-level crate is not cached for future compilations, so if a crate is
built into directory `foo` and then that crate is later used as a dependency,
it will be recompiled.
The naming limitations can be overcome in time, but for now it greatly
simplifies the crawling routines and shouldn't have much of a negative impact
other than some Cargo runtimes (which can in turn be negated by following the
"target" name convention).
Closes#482
The bots are trying to build something which isn't needed and is failing on
nightlies, so the build script has been updated to not bulid this unnecessary
artifact as part of libssh2
Updating a path source can be a possibly expensive operation (lots of
directories that need to be traversed), and currently the root path source is
updated three times:
* Once when the root package is initially loaded.
* Again when the first resolution pass happens over a graph.
* Finally a third when the second resolution pass happens over the graph.
This commit pushes through the original `Source` trait object into the
`PackageRegistry` and removes the implicit call to `add_sources`, pushing that
call up to the stack a bit.
Addresses #1591.
I believe this issue only affects `cargo bench` and `cargo test`. If it's applicable to any of the other commands, let me know and I'll amend this.
The bots are trying to build something which isn't needed and is failing on
nightlies, so the build script has been updated to not bulid this unnecessary
artifact as part of libssh2
The wrong kind was mistakenly used in the case of having a dependency on a
plugin.
Unfortunately I don't know of a great way to test this, so no unit test is included for now.
Closes#1612
When walking over entries in the index of a git repository the file type can be
known without a syscall (in the form of a libc constant), but walking over
untracked files still requires a syscall to determine if it's a directory.
When iterating over `DirEntry` values the file type of the entry can typically
be determined without a syscall, so use the new unstable APIs in `std::fs` to
avoid an extra call to `fs::metadata`. When traversing large directory trees the
improvement is quite noticeable!
Updating a path source can be a possibly expensive operation (lots of
directories that need to be traversed), and currently the root path source is
updated three times:
* Once when the root package is initially loaded.
* Again when the first resolution pass happens over a graph.
* Finally a third when the second resolution pass happens over the graph.
This commit pushes through the original `Source` trait object into the
`PackageRegistry` and removes the implicit call to `add_sources`, pushing that
call up to the stack a bit.
These commits add two vectors through which the compiler run can be configured:
* A `RUSTC` environment variable
* A `build.rustc` configuration variable in a `.cargo/config` file
In addition to global RUSTC/RUSTDOC env vars, this commit recognizes
`build.rustc` and `build.rustdoc` as configuration keys for Cargo to instruct
what tools should be used instead of the default.
Closes#967
