--jobs not respected in Bazel 7

[chiragramani]

Description of the bug:

We have used --jobs to control the number of jobs that should be executed concurrently. This works as expected in Bazel 6 however, in Bazel 7, this functionality does not work as expected.

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. This is a modified version of the official bazel examples.
   JobsRepro.zip
2. With bazel 7,

build :hello-time :hello-time1 :hello-time2 :hello-time3 :hello-time4

even though jobs=1 is specified in the .bazelrc, we still see parallel jobs > 1.

3. But the same example, with Bazel 6.4, respects --jobs=1.

4. With Bazel 7, if I use --local_cpu_resources=1, then it seems to work as expected. But from the documentation, --jobs is a global override to control such parallelism, wondering if this is a regression or if this behavior is driven differently in Bazel 7 intentionally.

Which operating system are you running Bazel on?

macOS

What is the output of bazel info release?

7.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[coeuvre]

It's due to the upgrade to JDK21 where the implementation of ForkJoinPool has changed -- the number of core threads is larger than --jobs. I can get the old behaviour by using --experimental_use_semaphore_for_jobs with your repro. We probably should flip and remove this flag.

[coeuvre]

@bazel-io fork 7.1.0

[chiragramani]

It's due to the upgrade to JDK21 where the implementation of ForkJoinPool has changed -- the number of core threads is larger than --jobs. I can get the old behaviour by using --experimental_use_semaphore_for_jobs with your repro. We probably should flip and remove this flag.

Thanks for looking into it. Yes, using --experimental_use_semaphore_for_jobs helped. Thanks for sharing this workaround!

[iancha1992]

A fix for this issue has been included in Bazel 7.1.0 RC1. Please test out the release candidate and report any issues as soon as possible. Thanks!