Skip to content

Commit

Permalink
Auto merge of #7344 - alexcrichton:jobserver, r=ehuss
Browse files Browse the repository at this point in the history
Create a jobserver with N tokens, not N-1

I recently added `jobserver` support to the `cc` crate and ended up
running afoul of a `jobserver` quirk on Windows. Due to how it's
implemented, on Windows you can't actually add more than the intial
number of tokens to the jobserver (it uses an IPC semaphore). On Unix,
however, you can since you're just writing bytes into a pipe.

In `cc`, however, I found it convenient to control parallelism by simply
releasing a token before the parallel loop, then reacquiring the token
after the loop. That way the loop just has to acquire a token for each
job it wants to spawn and then release it when the job finishes. This is
a bit simpler than trying to juggle the "implicit token" all over the
place as well as coordinating its use. It's technically invalid because
it allows a brief moment of `N+1` parallelism since we release a token
and then do a bit of work to acquire a new token, but that's hopefully
not really the end of the world.

In any case this commit updates Cargo's creation of a jobserver to create
it with `N` tokens instead of `N-1`. The same semantics are preserved
where Cargo then immediately acquires one of the tokens, but the
difference is that this "implicit token" can be released back to the
jobserver pool, unlike before.
  • Loading branch information
bors committed Sep 11, 2019
2 parents 651b1c5 + cab8640 commit a0be578
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions src/cargo/core/compiler/context/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,16 @@ impl<'a, 'cfg> Context<'a, 'cfg> {
// all share the same jobserver.
//
// Note that if we don't have a jobserver in our environment then we
// create our own, and we create it with `n-1` tokens because one token
// is ourself, a running process.
// create our own, and we create it with `n` tokens, but immediately
// acquire one, because one token is ourself, a running process.
let jobserver = match config.jobserver_from_env() {
Some(c) => c.clone(),
None => Client::new(bcx.build_config.jobs as usize - 1)
.chain_err(|| "failed to create jobserver")?,
None => {
let client = Client::new(bcx.build_config.jobs as usize)
.chain_err(|| "failed to create jobserver")?;
client.acquire_raw()?;
client
}
};

let pipelining = bcx
Expand Down

0 comments on commit a0be578

Please sign in to comment.