Auto merge of #10383 - dtolnay-contrib:keepgoing, r=ehuss

Unstable --keep-going flag

## Summary

This PR adds an unstable `--keep-going` flag documented as follows:

> `cargo build --keep-going` (and similarly for `check`, `test` etc) will build as many crates in the dependency graph as possible, rather than aborting the build at the first one that fails to build.
>
> For example if the current package depends on dependencies `fails` and `works`, one of which fails to build, `cargo check -j1` may or may not build the one that succeeds (depending on which one of the two builds Cargo picked to run first), whereas `cargo check -j1 --keep-going` would definitely run both builds, even if the one run first fails.
>
> The `-Z unstable-options` command-line option must be used in order to use `--keep-going` while it is not yet stable:
>
> ```console
> cargo check --keep-going -Z unstable-options
> ```

## Prior art

[Buck](https://buck.build/) and [Bazel](https://bazel.build/) and Make all have this flag (though Bazel calls it `--keep_going` 🤮) with exactly this behavior.

## Motivation

I need this in order to make https://github.com/dtolnay/trybuild not super slow.

Trybuild wants to run Cargo on a bunch of test cases, each of which is a bin crate. The bad options currently available are:

- Give each test case its own target dir and run build on them in parallel. This is bad because all the test cases have the same dependencies in common (whatever `dev-dependencies` are declared by the project). If there are 100 test cases, all the dependencies would end up getting built 100 times, which is 100x slower than necessary despite the parallelism.

- Reuse a single target dir for all the test cases. Two Cargos can't operate in parallel on the same target directory, so this forces the test cases to be built serially. This is much slower than necessary on a many-core system, and compounds all of the overheads in Cargo because the project structure must be reloaded by each invocation.

The good option I'd like to switch to is:

- Run `cargo build --bins --keep-going --message-format=json` to build *all* the test cases in parallel. Use the filepaths in the JSON messages to ascribe diagnostics to which bin they're from.

src/bin/cargo/commands/package.rs

@@ -53,6 +53,7 @@ pub fn exec(config: &mut Config, args: &ArgMatches) -> CliResult {
             to_package: specs,
             targets: args.targets(),
             jobs: args.jobs()?,
+            keep_going: args.keep_going(),
             cli_features: args.cli_features()?,
         },
     )?;

src/bin/cargo/commands/publish.rs

@@ -45,6 +45,7 @@ pub fn exec(config: &mut Config, args: &ArgMatches) -> CliResult {
             to_publish: args.packages_from_flags()?,
             targets: args.targets(),
             jobs: args.jobs()?,
+            keep_going: args.keep_going(),
             dry_run: args.is_present("dry-run"),
             registry,
             cli_features: args.cli_features()?,

src/cargo/core/compiler/build_config.rs

@@ -15,6 +15,8 @@ pub struct BuildConfig {
     pub requested_kinds: Vec<CompileKind>,
     /// Number of rustc jobs to run in parallel.
     pub jobs: u32,
+    /// Do not abort the build as soon as there is an error.
+    pub keep_going: bool,
     /// Build profile
     pub requested_profile: InternedString,
     /// The mode we are compiling in.
@@ -56,6 +58,7 @@ impl BuildConfig {
     pub fn new(
         config: &Config,
         jobs: Option<u32>,
+        keep_going: bool,
         requested_targets: &[String],
         mode: CompileMode,
     ) -> CargoResult<BuildConfig> {
@@ -84,6 +87,7 @@ impl BuildConfig {
         Ok(BuildConfig {
             requested_kinds,
             jobs,
+            keep_going,
             requested_profile: InternedString::new("dev"),
             mode,
             message_format: MessageFormat::Human,

src/cargo/core/compiler/job_queue.rs

@@ -825,13 +825,14 @@ impl<'cfg> DrainState<'cfg> {
         //
         // After a job has finished we update our internal state if it was
         // successful and otherwise wait for pending work to finish if it failed
-        // and then immediately return.
+        // and then immediately return (or keep going, if requested by the build
+        // config).
         let mut errors = ErrorsDuringDrain { count: 0 };
         // CAUTION! Do not use `?` or break out of the loop early. Every error
         // must be handled in such a way that the loop is still allowed to
         // drain event messages.
         loop {
-            if errors.count == 0 {
+            if errors.count == 0 || cx.bcx.build_config.keep_going {
                 if let Err(e) = self.spawn_work_if_possible(cx, jobserver_helper, scope) {
                     self.handle_error(&mut cx.bcx.config.shell(), &mut errors, e);
                 }

src/cargo/ops/cargo_compile.rs

@@ -85,8 +85,10 @@ pub struct CompileOptions {
 
 impl CompileOptions {
     pub fn new(config: &Config, mode: CompileMode) -> CargoResult<CompileOptions> {
+        let jobs = None;
+        let keep_going = false;
         Ok(CompileOptions {
-            build_config: BuildConfig::new(config, None, &[], mode)?,
+            build_config: BuildConfig::new(config, jobs, keep_going, &[], mode)?,
             cli_features: CliFeatures::new_all(false),
             spec: ops::Packages::Packages(Vec::new()),
             filter: CompileFilter::Default {

src/cargo/ops/cargo_fetch.rs

@@ -20,8 +20,15 @@ pub fn fetch<'a>(
     let (packages, resolve) = ops::resolve_ws(ws)?;
 
     let jobs = Some(1);
+    let keep_going = false;
     let config = ws.config();
-    let build_config = BuildConfig::new(config, jobs, &options.targets, CompileMode::Build)?;
+    let build_config = BuildConfig::new(
+        config,
+        jobs,
+        keep_going,
+        &options.targets,
+        CompileMode::Build,
+    )?;
     let data = RustcTargetData::new(ws, &build_config.requested_kinds)?;
     let mut fetched_packages = HashSet::new();
     let mut deps_to_fetch = ws.members().map(|p| p.package_id()).collect::<Vec<_>>();

src/cargo/ops/cargo_package.rs

@@ -30,6 +30,7 @@ pub struct PackageOpts<'cfg> {
     pub allow_dirty: bool,
     pub verify: bool,
     pub jobs: Option<u32>,
+    pub keep_going: bool,
     pub to_package: ops::Packages,
     pub targets: Vec<String>,
     pub cli_features: CliFeatures,
@@ -177,6 +178,7 @@ pub fn package(ws: &Workspace<'_>, opts: &PackageOpts<'_>) -> CargoResult<Option
                 allow_dirty: opts.allow_dirty,
                 verify: opts.verify,
                 jobs: opts.jobs,
+                keep_going: opts.keep_going,
                 to_package: ops::Packages::Default,
                 targets: opts.targets.clone(),
                 cli_features: cli_features,
@@ -755,7 +757,13 @@ fn run_verify(
     ops::compile_with_exec(
         &ws,
         &ops::CompileOptions {
-            build_config: BuildConfig::new(config, opts.jobs, &opts.targets, CompileMode::Build)?,
+            build_config: BuildConfig::new(
+                config,
+                opts.jobs,
+                opts.keep_going,
+                &opts.targets,
+                CompileMode::Build,
+            )?,
             cli_features: opts.cli_features.clone(),
             spec: ops::Packages::Packages(Vec::new()),
             filter: ops::CompileFilter::Default {

src/cargo/ops/registry.rs

@@ -80,6 +80,7 @@ pub struct PublishOpts<'cfg> {
     pub verify: bool,
     pub allow_dirty: bool,
     pub jobs: Option<u32>,
+    pub keep_going: bool,
     pub to_publish: ops::Packages,
     pub targets: Vec<String>,
     pub dry_run: bool,
@@ -147,6 +148,7 @@ pub fn publish(ws: &Workspace<'_>, opts: &PublishOpts<'_>) -> CargoResult<()> {
             to_package: ops::Packages::Default,
             targets: opts.targets.clone(),
             jobs: opts.jobs,
+            keep_going: opts.keep_going,
             cli_features: cli_features,
         },
     )?

src/cargo/util/command_prelude.rs

@@ -71,6 +71,10 @@ pub trait AppExt: Sized {
                 .short('j')
                 .value_name("N"),
         )
+        ._arg(opt(
+            "keep-going",
+            "Do not abort the build as soon as there is an error (unstable)",
+        ))
     }
 
     fn arg_targets_all(
@@ -353,6 +357,10 @@ pub trait ArgMatchesExt {
         self.value_of_u32("jobs")
     }
 
+    fn keep_going(&self) -> bool {
+        self._is_present("keep-going")
+    }
+
     fn targets(&self) -> Vec<String> {
         self._values_of("target")
     }
@@ -506,7 +514,13 @@ pub trait ArgMatchesExt {
             }
         }
 
-        let mut build_config = BuildConfig::new(config, self.jobs()?, &self.targets(), mode)?;
+        let mut build_config = BuildConfig::new(
+            config,
+            self.jobs()?,
+            self.keep_going(),
+            &self.targets(),
+            mode,
+        )?;
         build_config.message_format = message_format.unwrap_or(MessageFormat::Human);
         build_config.requested_profile = self.get_profile_name(config, "dev", profile_checking)?;
         build_config.build_plan = self.is_valid_and_present("build-plan");
@@ -540,6 +554,11 @@ pub trait ArgMatchesExt {
             }
         }
 
+        if build_config.keep_going {
+            config
+                .cli_unstable()
+                .fail_if_stable_opt("--keep-going", 10496)?;
+        }
         if build_config.build_plan {
             config
                 .cli_unstable()

src/doc/man/cargo-bench.md

@@ -139,6 +139,7 @@ Rust test harness runs benchmarks serially in a single thread.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 {{> section-environment }}

src/doc/man/cargo-build.md

@@ -91,6 +91,7 @@ See <https://github.com/rust-lang/cargo/issues/5579> for more information.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{> options-future-incompat }}
 {{/options}}
 

src/doc/man/cargo-check.md

@@ -76,6 +76,7 @@ they have `required-features` that are missing.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{> options-future-incompat }}
 {{/options}}
 

src/doc/man/cargo-doc.md

@@ -110,6 +110,7 @@ and supports common Unix glob patterns.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 {{> section-environment }}

src/doc/man/cargo-fix.md

@@ -156,6 +156,7 @@ When no target selection options are given, `cargo fix` will fix all targets
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 {{> section-environment }}

src/doc/man/cargo-install.md

@@ -181,6 +181,7 @@ See also the `--profile` option for choosing a specific profile by name.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 ### Display Options

src/doc/man/cargo-package.md

@@ -112,6 +112,7 @@ Allow working directories with uncommitted VCS changes to be packaged.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 ### Display Options

src/doc/man/cargo-publish.md

@@ -91,6 +91,7 @@ which defaults to `crates-io`.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 ### Display Options

src/doc/man/cargo-run.md

@@ -90,6 +90,7 @@ Run the specified example.
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 {{> section-environment }}

src/doc/man/cargo-rustc.md

@@ -102,6 +102,7 @@ See the [the reference](../reference/profiles.html) for more details on profiles
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{> options-future-incompat }}
 {{/options}}
 

src/doc/man/cargo-rustdoc.md

@@ -98,6 +98,7 @@ if its name is the same as the lib target. Binaries are skipped if they have
 
 {{#options}}
 {{> options-jobs }}
+{{> options-keep-going }}
 {{/options}}
 
 {{> section-environment }}

src/doc/man/cargo-test.md

@@ -155,6 +155,7 @@ includes an option to control the number of threads used:
 {{#options}}
 
 {{> options-jobs }}
+{{> options-keep-going }}
 {{> options-future-incompat }}
 
 {{/options}}

src/doc/man/generated_txt/cargo-bench.txt

@@ -361,6 +361,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
 ENVIRONMENT
        See the reference
        <https://doc.rust-lang.org/cargo/reference/environment-variables.html>

src/doc/man/generated_txt/cargo-build.txt

@@ -307,6 +307,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
        --future-incompat-report
            Displays a future-incompat report for any future-incompatible
            warnings produced during execution of this command

src/doc/man/generated_txt/cargo-check.txt

@@ -301,6 +301,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
        --future-incompat-report
            Displays a future-incompat report for any future-incompatible
            warnings produced during execution of this command

src/doc/man/generated_txt/cargo-doc.txt

@@ -272,6 +272,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
 ENVIRONMENT
        See the reference
        <https://doc.rust-lang.org/cargo/reference/environment-variables.html>

src/doc/man/generated_txt/cargo-fix.txt

@@ -374,6 +374,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
 ENVIRONMENT
        See the reference
        <https://doc.rust-lang.org/cargo/reference/environment-variables.html>

src/doc/man/generated_txt/cargo-install.txt

@@ -257,6 +257,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
    Display Options
        -v, --verbose
            Use verbose output. May be specified twice for "very verbose" output

src/doc/man/generated_txt/cargo-package.txt

@@ -189,6 +189,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
    Display Options
        -v, --verbose
            Use verbose output. May be specified twice for "very verbose" output

src/doc/man/generated_txt/cargo-publish.txt

@@ -156,6 +156,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
    Display Options
        -v, --verbose
            Use verbose output. May be specified twice for "very verbose" output

src/doc/man/generated_txt/cargo-run.txt

@@ -217,6 +217,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
 ENVIRONMENT
        See the reference
        <https://doc.rust-lang.org/cargo/reference/environment-variables.html>

src/doc/man/generated_txt/cargo-rustc.txt

@@ -296,6 +296,11 @@ OPTIONS
            <https://doc.rust-lang.org/cargo/reference/config.html>. Defaults to
            the number of CPUs.
 
+       --keep-going
+           Build as many crates in the dependency graph as possible, rather
+           than aborting the build on the first one that fails to build.
+           Unstable, requires -Zunstable-options.
+
        --future-incompat-report
            Displays a future-incompat report for any future-incompatible
            warnings produced during execution of this command