check subcommand should force rebuild

[JoshMcguigan]

This builds on the work from #5944, and pushed forward rust-lang/rust-clippy#2604. If ensures cargo check always does a rebuild.

The one thing I'm not sure of is whether this should be optional behind a command line flag? My initial thoughts are that most users would expect by default that cargo check always worked in this way (rebuilding and showing warnings every time it is ran), but that users running cargo check on a file system watcher might have some issues.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[JoshMcguigan]

It looks like three tests failed, but only on nightly. I'm going to look into this, but any ideas of what could be the cause here?

profile_targets::profile_selection_check_all_targets
profile_targets::profile_selection_check_all_targets_release
profile_targets::profile_selection_check_all_targets_test

-- edit --

I see now that all three of those tests have a code block which returns early if not running nightly. So I suspect these tests would have failed for all versions of Rust if not for that. I'll work on fixing them up. As a side note, they say they should be removed once 1.27 is stable. Should we just remove these early returns now?

if !is_nightly() {
    // This can be removed once 1.27 is stable, see below.
    return;
}

[JoshMcguigan]

The latest commit should fix these tests, assuming we want to remove the early return for !nightly(), and also assuming we want the default behavior of cargo check to force a rebuild. Some of this would need to be reverted if the new cargo check behavior goes behind a command line option.

[dwijnand]

most users would expect by default that cargo check always worked in this way (rebuilding and showing warnings every time it is ran)

I don't think most users expect cargo check to rebuild, quite the opposite I think they expect it to be fast and avoid rebuilding. They might want to see warnings every time (I'm with them) but, again, I don't think they expect it, given how long cargo check has existed by now.

If we agree cargo check should redisplay warnings then we should find a way to cache and redisplay the warnings every time. If not this should be behind a flag. (--rebuild?)

[BatmanAoD]

@dwijnand I was pretty surprised to discover that cargo check doesn't re-display errors. But since it doesn't "build" in the first place, I wouldn't expect it to "rebuild" either way!

As long as this PR doesn't change the behavior of cargo build (i.e. in cargo build; cargo check; cargo build, the second build would still be a no-op), then I don't understand why this change would be surprising to users.

[dwijnand]

As long as this PR doesn't change the behavior of cargo build (i.e. in cargo build; cargo check; cargo build, the second build would still be a no-op), then I don't understand why this change would be surprising to users.

It's surprising to users not because of any interactions with cargo build but because now every time they ran cargo check their entire crate would recompiled, making cargo check worst than cargo build, given a big enough crate.

[JoshMcguigan]

If we agree cargo check should redisplay warnings then we should find a way to cache and redisplay the warnings every time.

Yes, I should have said users expect to see the error messages each time, so I think cacheing these warnings would be great. Any tips on implementing this cacheing? I'm new to the cargo codebase so I'm not sure how cacheing is typically handled.

[dwijnand]

I don't know exactly but cargo process calls rustc, and does so via a rustc wrapper, so finding that will get you half way to before it executes it.

Then you need to go to the place where it uses the fact that the build artefacts are "fresh" to avoid compile to display the warnings you cached.

If I can find more specific code locations I'll recomment.

[sanmai-NL]

@dwijnand: cargo check shouldn't compile, it should one perform the first few steps in the build pipeline, as documented at https://rust-lang-nursery.github.io/edition-guide/rust-2018/cargo-and-crates-io/cargo-check-for-faster-checking.html. Maybe those first steps take a significant amount of time, but insofar that's unacceptable that'd be an issue in itself.

[sanmai-NL]

@BatmanAoD:

cargo build; cargo check; cargo build

should behave differently, cargo check shouldn't reuse cargo build results at all. That's why it's called 'check': the main use case for the check-subcommand is to check e.g. syntax again, irrespective of whether some build succeeded earlier and whether the source code/build config has changed in the meantime.

[dwijnand]

@dwijnand: cargo check shouldn't compile, it should one perform the first few steps in the build pipeline, as documented at rust-lang-nursery.github.io/edition-guide/rust-2018/cargo-and-crates-io/cargo-check-for-faster-checking.html. Maybe those first steps take a significant amount of time, but insofar that's unacceptable that'd be an issue in itself.

Right, I said "compile" as short-hand, but you're right it stops earlier in the pipeline, running (what's commonly referred to as) the frontend of the compiler.

@BatmanAoD:

cargo build; cargo check; cargo build

should behave differently, cargo check shouldn't reuse cargo build results at all. That's why it's called 'check': the main use case for the check-subcommand is to check e.g. syntax again, irrespective of whether some build succeeded earlier and whether the source code/build config has changed in the meantime.

cargo check actually doesn't reuse cargo build's results, which is tracked in #3501. I disagree that 'check' shouldn't make use of previous results, but I agree it should redisplay warnings, either by default or behind a flag. (--show-warnings? --show-lints?)

[sanmai-NL]

@dwijnand: I agree it should redisplay warnings then, but by default, since I claim it'd be the expected behavior. Since running the first steps in the compilation pipeline is supposed to be low-cost, the desired default behavior would be easily implemented by not reusing previous build results.

[dwijnand]

Since running the first steps in the compilation pipeline is supposed to be low-cost, the desired default behavior would be easily implemented by not reusing previous build results.

I think it's low-cost compared to a full build, but it would be slow compared to the status quo in which cargo uses freshness checks.

[sanmai-NL]

But it's called 'check', when you run that explicitly, it's reasonable some cost is incurred. To the contrary, it's not reasonable e.g. cargo check doesn't redisplay its messages because of caching.

[dwijnand]

But it's called 'check', when you run that explicitly, it's reasonable some cost is incurred. To the contrary, it's not reasonable e.g. cargo check doesn't redisplay its messages because of caching.

While I can agree with the later, I'd consider it a performance regression if cargo check in a big codebase where no file was touched to take too long.

I'm optimistic we can find a way to cache and uncache warnings without changing the freshness behaviour.

[BatmanAoD]

@JoshMcguigan :

Any tips on implementing this cacheing? I'm new to the cargo codebase so I'm not sure how cacheing is typically handled.

it looks like Eric Huss has some pointers; see the links in his comment here: https://internals.rust-lang.org/t/clippy-behavior-affected-by-incremental-compilation/9436/2?u=batmanaod

@sanmai-NL : Yes, I was asking specifically about the last part of the sequence, the second cargo build. My point was that cargo check should never invalidate the cargo build cache, regardless of whether or not the build cache has any effect on the check behavior. (I don't know if this PR actually would cause the build cache to be invalidated, but it's something I'd want to check to be sure.)

@dwijnand :

cargo check actually doesn't reuse cargo build's results

...hold on, that's surprising to me. If cargo check doesn't reuse the build results, why doesn't it re-analyze files that have already been compiled?

[dwijnand]

It uses its own results not build's results.

[JoshMcguigan]

After further review of the way clippy integrates with cargo (https://github.com/rust-lang/rust-clippy/blob/master/src/main.rs#L103-L111), it looks like clippy actually does need a way to ensure rustc will actually be invoked. This means cacheing the results of a rustc invocation within cargo and re-displaying them does not solve the clippy use case (clippy team, please let me know if I am misunderstanding here).

I propose this PR can add a --force-rebuild CLI flag to cargo check to solve the clippy use-case, and a later PR can implement cacheing/re-displaying warning messages when the user runs something like cargo check; cargo check.

[dwijnand]

LGTM. The Cargo.toml isn't needed in the test (one is generated automatically). Also, maybe we should add --force-rebuild to cargo build. But I'm happy to land this as is.

I'd like to hear what @alexcrichton or @ehuss think, though, before landing this.

[JoshMcguigan]

The Cargo.toml isn't needed in the test (one is generated automatically)

Thanks, I'll remove this.

Also, maybe we should add --force-rebuild to cargo build

My primary concern is in the name of the arg. It only rebuilds the crate, but not its dependencies. It seems like that may not be intuitive if applied to cargo build, but I'm not sure what the use case would be for cargo build --force-rebuild so I can't say for sure. Along the same lines, perhaps cargo check --force-recheck makes more sense than cargo check --force-rebuild.

Not to muddy the waters here I just want to make sure this is considered.

[Nemo157]

I just tested this in a complex workspace of mine and there seems to be some bugs in how the force_rebuild flag is actually handled by build config. Probably not a blocker for getting it exposed in the CLI, but I'll try and remember to open a bug ticket in a few days.

There are 11 crates in this workspace, 2 are excluded so there should be 9 being forced to rebuild, but the only ones that I have seen consistently included are embrio-async-dehygiene and local. These two are the only crates that aren't dependencies of other crates in the workspace, so it seems like the logic that excludes dependencies from being forced is also able to exclude packages that were explicitly selected:

> ../check-force-rebuild/target/debug/cargo check --all --exclude embrio-nrf51 --exclude pca10031 --force-rebuild
    Checking embrio-native v0.1.0 (/Users/nemo157/sources/embrio-rs/embrio-native)
    Checking embrio-async-dehygiene v0.1.0 (/Users/nemo157/sources/embrio-rs/embrio-async/dehygiene)
    Checking embrio v0.1.0 (/Users/nemo157/sources/embrio-rs/embrio)
    Checking hello v0.0.0 (/Users/nemo157/sources/embrio-rs/examples/apps/hello)
    Checking local v0.0.0 (/Users/nemo157/sources/embrio-rs/examples/local)
    Finished dev [unoptimized + debuginfo] target(s) in 1.17s

> ../check-force-rebuild/target/debug/cargo check --all --exclude embrio-nrf51 --exclude pca10031 --force-rebuild
    Checking embrio-native v0.1.0 (/Users/nemo157/sources/embrio-rs/embrio-native)
    Checking embrio-async-dehygiene v0.1.0 (/Users/nemo157/sources/embrio-rs/embrio-async/dehygiene)
    Checking local v0.0.0 (/Users/nemo157/sources/embrio-rs/examples/local)
    Finished dev [unoptimized + debuginfo] target(s) in 0.99s

[JoshMcguigan]

The test failure is in the meta_test_deep_trees_from_strategy test, so I think it will pass if retried.

[JoshMcguigan]

@Nemo157 For some reason I am not able to duplicate your issue under cargo fix on the master branch. Somehow it only seems to affect cargo check as demonstrated in the failing test above (or I misconfigured the test for cargo fix).

Given that, perhaps we want to reconsider whether the fix merges with this PR? For now I've pushed the fix in a commit to a different branch (9b3138c), but if everyone agrees I can merge that commit into this branch/PR.

[Nemo157]

cargo fix always working seems to be a side-effect of it implying --all-targets, maybe because it will include the cfg(test) mode build of each package which is not a dependency of the other targets. Using check --force-rebuild --all --all-targets in my local workspace correctly builds all local crates.

[alexcrichton]

I think it's fine for clippy to use "unstable" features of Cargo, although perhaps not literally unstable features as they are today. In any case clippy, as an official tool now, should be fine to have support in Cargo which isn't intended for stable consumption elsewhere. (note that this interface, should it want to exist, would probably need to be designed rather than tacked on to something else existing).

I also think that moving cargo clippy into Cargo itself is fine so long as cargo clippy doesn't get enough development to put a strain on this repository.

[Manishearth]

Yeah, cargo clippy is a very thin shim so it wouldn't change much.

…

On Wed, Feb 20, 2019, 9:07 PM Alex Crichton ***@***.***> wrote: I think it's fine for clippy to use "unstable" features of Cargo, although perhaps not literally unstable features as they are today. In any case clippy, as an official tool now, should be fine to have support in Cargo which isn't intended for stable consumption elsewhere. (note that this interface, should it want to exist, would probably need to be designed rather than tacked on to something else existing). I also think that moving cargo clippy into Cargo itself is fine so long as cargo clippy doesn't get enough development to put a strain on this repository. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#6664 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABivSF3S4oQNyZAbt_8DWnyafvxQsw8mks5vPWvMgaJpZM4a7eRq> .

[JoshMcguigan]

Let me know if there needs to be any additional work here before merging. The test failure is in the meta_test_deep_trees_from_strategy test, so I think it will pass if retried.

[benesch]

CI seems happy with this—is there anything preventing it from being merged?

[ehuss]

I don't think adding a flag of this nature is ultimately the direction that we want to go. Caching shouldn't take very long to add once it is ready upstream. cargo-clippy is in the process of moving into this repo where we will have some more flexibility for the clippy side of things.

I would like to propose the following: Merge this PR without any command-line flags and immediately stable. The intent would be that a caching implementation would remove this change in the near future. I'm not convinced that "running rustc again" is significantly painful. On cargo's source, for me with a warm cache, takes only a couple seconds. On servo, it takes about 7s. Clippy is a bit slower, but still reasonable.

@rfcbot fcp merge

If this fcp does not pass, I propose closing this PR instead and waiting for aforementioned items. Adding an unstable flag here will likely have the same timeframe as the caching implementation.

[rfcbot]

Team member @ehuss has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Eh2406
• @alexcrichton
• @dwijnand
• @ehuss
• @joshtriplett
• @nrc
• @withoutboats
• @wycats

Concerns:

• habitual-check-performance (check subcommand should force rebuild #6664 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[withoutboats]

I'm not sure about merging this. I have the experience (and maybe many others do) of habitually running commands in a sort of impulsive & subconscious way as I do my work. git status for example. If cargo check is one of these for someone (because normally it finishes instantaneously and only doesn't when theyve actually done something they want to check), this would be really disruptive for their work flow as they are now taken out of the flow by this command.

[joshtriplett]

@rfcbot reviewed

Note that while I do want to see the effect of this, I'd like to see the caching as well (both in the normal and clippy-enabled cases, and in an incremental way). I regularly do clippy/edit/clippy cycles and would like those to go as quickly as possible.

[ehuss]

Sounds like a reasonable concern.
@rfcbot concern habitual-check-performance

[joshtriplett]

@ehuss @withoutboats I do think this is a great argument for caching. No good reason to rebuild just to display results. (And ideally, "fix one bug and recheck to get the rest" should go as quickly as possible and not take the same amount of time as the first check.)

[Nemo157]

It has been 5 months since the last update, has there been any movement on getting caching of warnings?

I regularly do clippy/edit/clippy cycles and would like those to go as quickly as possible.

(And ideally, "fix one bug and recheck to get the rest" should go as quickly as possible and not take the same amount of time as the first check.)

Are these use-cases actually significantly affected by these changes? If you are editing the lib then cargo check will have to recheck every target anyway, if you are editing a bin/test/other target and running cargo check --all-targets then you will unnecessarily be rechecking the lib and other unchanged targets, but that doesn't seem like such a massive downside to block making cargo check more usable from the command-line (IMO).

[mati865]

It has been 5 months since the last update, has there been any movement on getting caching of warnings?

Clippy issue can be considered as fixed by #6759. Regular cargo clippy is not fixed yet (it shows error only for the first time) but cargo clippy-preview -Z unstable-options which will internally replace cargo clippy at some point works well and shows warnings/error every time.

[Nemo157]

From my testing #6759 appears to essentially do what this change does and just recheck the targets every time without any caching of warnings (unless there's caching happening inside clippy-driver rather than in cargo)?

[bors]

☔ The latest upstream changes (presumably #7350) made this pull request unmergeable. Please resolve the merge conflicts.

[BatmanAoD]

🎉