cargo/tests/testsuite/jobserver.rs

204 lines
4.7 KiB
Rust
Raw Normal View History

2019-11-25 02:42:45 +00:00
//! Tests for the jobserver protocol.
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
use std::net::TcpListener;
use std::process::Command;
use std::thread;
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
use cargo_test_support::{cargo_exe, project};
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
#[cargo_test]
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
fn jobserver_exists() {
let p = project()
2018-03-14 15:17:44 +00:00
.file(
"build.rs",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
use std::env;
fn main() {
let var = env::var("CARGO_MAKEFLAGS").unwrap();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
let arg = var.split(' ')
.find(|p| p.starts_with("--jobserver"))
.unwrap();
let val = &arg[arg.find('=').unwrap() + 1..];
validate(val);
}
#[cfg(unix)]
fn validate(s: &str) {
use std::fs::File;
use std::io::*;
use std::os::unix::prelude::*;
let fds = s.split(',').collect::<Vec<_>>();
println!("{}", s);
assert_eq!(fds.len(), 2);
unsafe {
let mut read = File::from_raw_fd(fds[0].parse().unwrap());
let mut write = File::from_raw_fd(fds[1].parse().unwrap());
let mut buf = [0];
assert_eq!(read.read(&mut buf).unwrap(), 1);
assert_eq!(write.write(&buf).unwrap(), 1);
}
}
#[cfg(windows)]
fn validate(_: &str) {
// a little too complicated for a test...
}
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file("src/lib.rs", "")
.build();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
p.cargo("build").run();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
}
#[cargo_test]
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
fn makes_jobserver_used() {
2018-03-14 15:17:44 +00:00
let make = if cfg!(windows) {
"mingw32-make"
} else {
"make"
};
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
if Command::new(make).arg("--version").output().is_err() {
2018-03-14 15:17:44 +00:00
return;
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
}
let p = project()
2018-03-14 15:17:44 +00:00
.file(
"Cargo.toml",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
[package]
name = "foo"
version = "0.0.1"
authors = []
[dependencies]
d1 = { path = "d1" }
d2 = { path = "d2" }
d3 = { path = "d3" }
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file("src/lib.rs", "")
2018-03-14 15:17:44 +00:00
.file(
"d1/Cargo.toml",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
[package]
name = "d1"
version = "0.0.1"
authors = []
build = "../dbuild.rs"
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file("d1/src/lib.rs", "")
2018-03-14 15:17:44 +00:00
.file(
"d2/Cargo.toml",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
[package]
name = "d2"
version = "0.0.1"
authors = []
build = "../dbuild.rs"
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file("d2/src/lib.rs", "")
2018-03-14 15:17:44 +00:00
.file(
"d3/Cargo.toml",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
[package]
name = "d3"
version = "0.0.1"
authors = []
build = "../dbuild.rs"
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file("d3/src/lib.rs", "")
2018-03-14 15:17:44 +00:00
.file(
"dbuild.rs",
r#"
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
use std::net::TcpStream;
use std::env;
use std::io::Read;
fn main() {
let addr = env::var("ADDR").unwrap();
let mut stream = TcpStream::connect(addr).unwrap();
let mut v = Vec::new();
stream.read_to_end(&mut v).unwrap();
}
2018-03-14 15:17:44 +00:00
"#,
2018-12-08 11:19:47 +00:00
)
.file(
2018-03-14 15:17:44 +00:00
"Makefile",
"\
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
all:
\t+$(CARGO) build
2018-03-14 15:17:44 +00:00
",
2018-12-08 11:19:47 +00:00
)
.build();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
let l = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = l.local_addr().unwrap();
let child = thread::spawn(move || {
let a1 = l.accept().unwrap();
let a2 = l.accept().unwrap();
l.set_nonblocking(true).unwrap();
for _ in 0..1000 {
assert!(l.accept().is_err());
thread::yield_now();
}
drop(a1);
l.set_nonblocking(false).unwrap();
let a3 = l.accept().unwrap();
drop((a2, a3));
});
2018-08-28 20:38:26 +00:00
p.process(make)
.env("CARGO", cargo_exe())
.env("ADDR", addr.to_string())
.arg("-j2")
.run();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
child.join().unwrap();
}
#[cargo_test]
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
fn jobserver_and_j() {
2018-03-14 15:17:44 +00:00
let make = if cfg!(windows) {
"mingw32-make"
} else {
"make"
};
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
if Command::new(make).arg("--version").output().is_err() {
2018-03-14 15:17:44 +00:00
return;
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
}
let p = project()
.file("src/lib.rs", "")
2018-03-14 15:17:44 +00:00
.file(
"Makefile",
"\
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
all:
\t+$(CARGO) build -j2
2018-03-14 15:17:44 +00:00
",
2018-12-08 11:19:47 +00:00
)
.build();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
2018-08-28 20:38:26 +00:00
p.process(make)
.env("CARGO", cargo_exe())
.arg("-j2")
.with_stderr(
2018-03-14 15:17:44 +00:00
"\
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
warning: a `-j` argument was passed to Cargo but Cargo is also configured \
with an external jobserver in its environment, ignoring the `-j` parameter
[COMPILING] [..]
[FINISHED] [..]
2018-03-14 15:17:44 +00:00
",
2018-12-08 11:19:47 +00:00
)
.run();
Add a GNU make jobserver implementation to Cargo This commit adds a GNU make jobserver implementation to Cargo, both as a client of existing jobservers and also a creator of new jobservers. The jobserver is actually just an IPC semaphore which manifests itself as a pipe with N bytes of tokens on Unix and a literal IPC semaphore on Windows. The rough protocol is then if you want to run a job you read acquire the semaphore (read a byte on Unix or wait on the semaphore on Windows) and then you release it when you're done. All the hairy details of the jobserver implementation are housed in the `jobserver` crate on crates.io instead of Cargo. This should hopefully make it much easier for the compiler to also share a jobserver implementation eventually. The main tricky bit here is that on Unix and Windows acquiring a jobserver token will block the calling thread. We need to either way for a running job to exit or to acquire a new token when we want to spawn a new job. To handle this the current implementation spawns a helper thread that does the blocking and sends a message back to Cargo when it receives a token. It's a little trickier with shutting down this thread gracefully as well but more details can be found in the `jobserver` crate. Unfortunately crates are unlikely to see an immediate benefit of this once implemented. Most crates are run with a manual `make -jN` and this overrides the jobserver in the environment, creating a new jobserver in the sub-make. If the `-jN` argument is removed, however, then `make` will share Cargo's jobserver and properly limit parallelism. Closes #1744
2017-05-30 04:09:53 +00:00
}