mirror of
https://github.com/rust-lang/cargo
synced 2024-09-16 06:20:49 +00:00
659f824406
This commit started out identifying a relatively simple bug in Cargo. A recent change made it such that the resolution graph included all target-specific dependencies, relying on the structure of the backend to filter out those which don't need to get built. This was unfortunately not accounted for in the portion of the backend that schedules work, mistakenly causing spurious rebuilds if different runs of the graph pulled in new crates. For example if `cargo build` didn't build any target-specific dependencies but then later `cargo test` did (e.g. a dev-dep pulled in a target-specific dep unconditionally) then it would cause a rebuild of the entire graph. This class of bug is certainly not the first in a long and storied history of the backend having multiple points where dependencies are calculated and those often don't quite agree with one another. The purpose of this rewrite is twofold: 1. The `Stage` enum in the backend for scheduling work and ensuring that maximum parallelism is achieved is removed entirely. There is already a function on `Context` which expresses the dependency between targets (`dep_targets`) which takes a much finer grain of dependencies into account as well as already having all the logic for what-depends-on-what. This duplication has caused numerous problems in the past, an unifying these two will truly grant maximum parallelism while ensuring that everyone agrees on what their dependencies are. 2. A large number of locations in the backend have grown to take a (Package, Target, Profile, Kind) tuple, or some subset of this tuple. In general this represents a "unit of work" and is much easier to pass around as one variable, so a `Unit` was introduced which references all of these variables. Almost the entire backend was altered to take a `Unit` instead of these variables specifically, typically providing all of the contextual information necessary for an operation. A crucial part of this change is the inclusion of `Kind` in a `Unit` to ensure that everyone *also* agrees on what architecture they're compiling everything for. There have been many bugs in the past where one part of the backend determined that a package was built for one architecture and then another part thought it was built for another. With the inclusion of `Kind` in dependency management this is handled in a much cleaner fashion as it's only calculated in one location. Some other miscellaneous changes made were: * The `Platform` enumeration has finally been removed. This has been entirely subsumed by `Kind`. * The hokey logic for "build this crate once" even though it may be depended on by both the host/target kinds has been removed. This is now handled in a much nicer fashion where if there's no target then Kind::Target is just never used, and multiple requests for a package are just naturally deduplicated. * There's no longer any need to build up the "requirements" for a package in terms of what platforms it's compiled for, this now just naturally falls out of the dependency graph. * If a build script is overridden then its entire tree of dependencies are not compiled, not just the build script itself. * The `threadpool` dependency has been replaced with one on `crossbeam`. The method of calculating dependencies has quite a few non-static lifetimes and the scoped threads of `crossbeam` are now used instead of a thread pool. * Once any thread fails to execute a command work is no longer scheduled unlike before where some extra pending work may continue to start. * Many functions used early on, such as `compile` and `build_map` have been massively simplified by farming out dependency management to `Context::dep_targets`. * There is now a new profile to represent running a build script. This is used to inject dependencies as well as represent that a library depends on running a build script, not just building it. This change has currently been tested against cross-compiling Servo to Android and passes the test suite (which has quite a few corner cases for build scripts and such), so I'm pretty confident that this refactoring won't have at least too many regressions!
260 lines
7.1 KiB
Rust
260 lines
7.1 KiB
Rust
use std::fs::{self, File};
|
|
use std::io::prelude::*;
|
|
use std::thread;
|
|
|
|
use support::{project, execs, path2url};
|
|
use support::COMPILING;
|
|
use support::paths::CargoPathExt;
|
|
use hamcrest::{assert_that, existing_file};
|
|
|
|
fn setup() {}
|
|
|
|
test!(modifying_and_moving {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
authors = []
|
|
version = "0.0.1"
|
|
"#)
|
|
.file("src/main.rs", r#"
|
|
mod a; fn main() {}
|
|
"#)
|
|
.file("src/a.rs", "");
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0).with_stdout(format!("\
|
|
{compiling} foo v0.0.1 ({dir})
|
|
", compiling = COMPILING, dir = path2url(p.root()))));
|
|
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0).with_stdout(""));
|
|
p.root().move_into_the_past().unwrap();
|
|
p.root().join("target").move_into_the_past().unwrap();
|
|
|
|
File::create(&p.root().join("src/a.rs")).unwrap()
|
|
.write_all(b"fn main() {}").unwrap();
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0).with_stdout(format!("\
|
|
{compiling} foo v0.0.1 ({dir})
|
|
", compiling = COMPILING, dir = path2url(p.root()))));
|
|
|
|
fs::rename(&p.root().join("src/a.rs"), &p.root().join("src/b.rs")).unwrap();
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(101));
|
|
});
|
|
|
|
test!(modify_only_some_files {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
authors = []
|
|
version = "0.0.1"
|
|
"#)
|
|
.file("src/lib.rs", "mod a;")
|
|
.file("src/a.rs", "")
|
|
.file("src/main.rs", r#"
|
|
mod b;
|
|
fn main() {}
|
|
"#)
|
|
.file("src/b.rs", "")
|
|
.file("tests/test.rs", "");
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0).with_stdout(format!("\
|
|
{compiling} foo v0.0.1 ({dir})
|
|
", compiling = COMPILING, dir = path2url(p.root()))));
|
|
assert_that(p.cargo("test"),
|
|
execs().with_status(0));
|
|
thread::sleep_ms(1000);
|
|
|
|
assert_that(&p.bin("foo"), existing_file());
|
|
|
|
let lib = p.root().join("src/lib.rs");
|
|
let bin = p.root().join("src/b.rs");
|
|
|
|
File::create(&lib).unwrap().write_all(b"invalid rust code").unwrap();
|
|
File::create(&bin).unwrap().write_all(b"fn foo() {}").unwrap();
|
|
lib.move_into_the_past().unwrap();
|
|
|
|
// Make sure the binary is rebuilt, not the lib
|
|
assert_that(p.cargo("build")
|
|
.env("RUST_LOG", "cargo::ops::cargo_rustc::fingerprint"),
|
|
execs().with_status(0).with_stdout(format!("\
|
|
{compiling} foo v0.0.1 ({dir})
|
|
", compiling = COMPILING, dir = path2url(p.root()))));
|
|
assert_that(&p.bin("foo"), existing_file());
|
|
});
|
|
|
|
test!(rebuild_sub_package_then_while_package {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
authors = []
|
|
version = "0.0.1"
|
|
|
|
[dependencies.a]
|
|
path = "a"
|
|
[dependencies.b]
|
|
path = "b"
|
|
"#)
|
|
.file("src/lib.rs", "extern crate a; extern crate b;")
|
|
.file("a/Cargo.toml", r#"
|
|
[package]
|
|
name = "a"
|
|
authors = []
|
|
version = "0.0.1"
|
|
[dependencies.b]
|
|
path = "../b"
|
|
"#)
|
|
.file("a/src/lib.rs", "extern crate b;")
|
|
.file("b/Cargo.toml", r#"
|
|
[package]
|
|
name = "b"
|
|
authors = []
|
|
version = "0.0.1"
|
|
"#)
|
|
.file("b/src/lib.rs", "");
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0));
|
|
|
|
File::create(&p.root().join("b/src/lib.rs")).unwrap().write_all(br#"
|
|
pub fn b() {}
|
|
"#).unwrap();
|
|
|
|
assert_that(p.cargo("build").arg("-pb"),
|
|
execs().with_status(0));
|
|
|
|
File::create(&p.root().join("src/lib.rs")).unwrap().write_all(br#"
|
|
extern crate a;
|
|
extern crate b;
|
|
pub fn toplevel() {}
|
|
"#).unwrap();
|
|
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0));
|
|
});
|
|
|
|
test!(changing_features_is_ok {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
authors = []
|
|
version = "0.0.1"
|
|
|
|
[features]
|
|
foo = []
|
|
"#)
|
|
.file("src/lib.rs", "");
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0)
|
|
.with_stdout("\
|
|
[..]Compiling foo v0.0.1 ([..])
|
|
"));
|
|
|
|
assert_that(p.cargo("build").arg("--features").arg("foo"),
|
|
execs().with_status(0)
|
|
.with_stdout("\
|
|
[..]Compiling foo v0.0.1 ([..])
|
|
"));
|
|
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0)
|
|
.with_stdout("\
|
|
[..]Compiling foo v0.0.1 ([..])
|
|
"));
|
|
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0)
|
|
.with_stdout(""));
|
|
});
|
|
|
|
test!(rebuild_tests_if_lib_changes {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
version = "0.0.1"
|
|
authors = []
|
|
"#)
|
|
.file("src/lib.rs", "pub fn foo() {}")
|
|
.file("tests/foo.rs", r#"
|
|
extern crate foo;
|
|
#[test]
|
|
fn test() { foo::foo(); }
|
|
"#);
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0));
|
|
assert_that(p.cargo("test"),
|
|
execs().with_status(0));
|
|
|
|
File::create(&p.root().join("src/lib.rs")).unwrap();
|
|
p.root().move_into_the_past().unwrap();
|
|
p.root().join("target").move_into_the_past().unwrap();
|
|
|
|
assert_that(p.cargo("build"),
|
|
execs().with_status(0));
|
|
assert_that(p.cargo("test").arg("-v"),
|
|
execs().with_status(101));
|
|
});
|
|
|
|
test!(no_rebuild_transitive_target_deps {
|
|
let p = project("foo")
|
|
.file("Cargo.toml", r#"
|
|
[package]
|
|
name = "foo"
|
|
version = "0.0.1"
|
|
authors = []
|
|
|
|
[dependencies]
|
|
a = { path = "a" }
|
|
[dev-dependencies]
|
|
b = { path = "b" }
|
|
"#)
|
|
.file("src/lib.rs", "")
|
|
.file("tests/foo.rs", "")
|
|
.file("a/Cargo.toml", r#"
|
|
[package]
|
|
name = "a"
|
|
version = "0.0.1"
|
|
authors = []
|
|
|
|
[target.foo.dependencies]
|
|
c = { path = "../c" }
|
|
"#)
|
|
.file("a/src/lib.rs", "")
|
|
.file("b/Cargo.toml", r#"
|
|
[package]
|
|
name = "b"
|
|
version = "0.0.1"
|
|
authors = []
|
|
|
|
[dependencies]
|
|
c = { path = "../c" }
|
|
"#)
|
|
.file("b/src/lib.rs", "")
|
|
.file("c/Cargo.toml", r#"
|
|
[package]
|
|
name = "c"
|
|
version = "0.0.1"
|
|
authors = []
|
|
"#)
|
|
.file("c/src/lib.rs", "");
|
|
|
|
assert_that(p.cargo_process("build"),
|
|
execs().with_status(0));
|
|
assert_that(p.cargo("test").arg("--no-run"),
|
|
execs().with_status(0)
|
|
.with_stdout(&format!("\
|
|
{compiling} c v0.0.1 ([..])
|
|
{compiling} b v0.0.1 ([..])
|
|
{compiling} foo v0.0.1 ([..])
|
|
", compiling = COMPILING)));
|
|
});
|