Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure flycheck using workspace.discoverConfig #18043

Open
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

cormacrelf
Copy link
Contributor

@cormacrelf cormacrelf commented Sep 4, 2024

Recap of recent developments with non-Cargo build systems

  1. We have workspace.discoverConfig, which configures a command to (basically) spit out a rust-project.json on stdout and treats it like you put a rust-project.json file on disk.
  2. JSON projects have a nice section for runnables, but they're currently limited to RunnableKind { Check, Run, TestOne, }. You cannot have the JSON project configure its own flychecks. Which means that configuring a non-Cargo build system is a complicated dance of workspace.discoverConfig + check.overrideCommand + "check.workspace": false.
  3. The {label} substitution (available for runnables) was not usable for the flycheck command. I think because the only users of discoverConfig have been content with $saved_file so far. So we currently have build labels in the JSON project that can't even be used for flychecking.

What this does

This PR adds this functionality:

  1. Allows the discoverConfig to describe how to flycheck crates, using a new runnable kind of "flycheck".
  2. Allows all types of flycheck override commands to use {label}. If you are not using discoverConfig / rust-project.json and therefore don't have build labels attached to crates, then {label} means the cargo canonical name. Previously we only supported $saved_file here.
  3. The flycheck begin notification now includes a pretty-print of the actual command in use (if you're not just letting RA run cargo). Super useful to know that things got configured ok, at the bottom of your VSCode window.

And fixes:

  1. Flychecks were being run on random downstream crates instead of the crate containing the file you saved, which would randomly result in cargo check-ing a bigger downstream dependency, leading to unnecessarily slow flychecks and wasted CPU cycles. See the comments in notification.rs. I noticed this because the buck2 rust-project check command does not give diagnostics for dependencies of the target you ask for.
Notes for users

For people actually using this, it works well in VSCode if you configure rust-analyzer.workspace.discoverConfig in your settings.json, as per the manual. However, because rust-analyzer.toml support is still in its infancy, to get this going in Neovim, you will need my PR to rustaceanvim to get the settings into the initialization request, and I advise having rustaceanvim autoload settings from a file called rust-analyzer.json.

Tagging: @alibektas @davidbarsky

Say you had A <- B <- C. If you save a file in A, the variable
`crate_root_paths` is populated with all three crates in that order.
However the one we actually ran flycheck on, was simply the first one of
A, B, or C to appear in `project.crates()`. Which is unordered.

This affected
- `[check] workspace = false` config where flycheck runs on individual
  crates
- non-Cargo build environments especially

Usually it was invisible because cargo prints out the warnings found
in A if you build B or C. But this is wasteful if B or C is a big crate.
So it was probably showing up as rust-analyzer constantly trying to
build random parts of your dependency graph on save.

Not every build tool does what cargo does either. Buck2's `[diag.json]`
subtargets do not contain warnings from upstream crates. That's how
I discovered this problem; warnings weren't showing up at all.
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 4, 2024
@cormacrelf cormacrelf force-pushed the feature/rust-project-discovery branch from f1dbc97 to 52354dc Compare September 4, 2024 04:08
@cormacrelf cormacrelf changed the title Configure flycheck using discovered using [workspace.discoverConfig] Configure flycheck using workspace.discoverConfig Sep 4, 2024
@cormacrelf cormacrelf force-pushed the feature/rust-project-discovery branch from e5548f5 to 5bf5e0d Compare September 5, 2024 04:03
@cormacrelf cormacrelf force-pushed the feature/rust-project-discovery branch from 5bf5e0d to 09ef79a Compare September 5, 2024 04:53
Copy link
Contributor

@davidbarsky davidbarsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some assorted comments. I'll do another pass later.

Comment on lines +326 to +328
/// Template for checking a target, emitting rustc JSON diagnostics.
/// May include {label} which will get the label from the `build` section of a crate.
Flycheck,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that you need to add a new enum variant called Flycheck—I always intended the RunnableKind::Check variant to be used for Flycheck, but I never actually wired it up. Happy to have Check be renamed to Flycheck if Check can be (briefly) deserialized.

// Trigger flychecks for the only crate which the target belongs to
Some((_, krate)) => vec![krate],
None => {
tracing::debug!("flycheck scope: {:?}", scope);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tracing::debug!("flycheck scope: {:?}", scope);
tracing::debug!(?scope);

TargetSpec::ProjectJson(p) => (p.label, p.crate_id),
};
#[derive(Debug)]
enum FlycheckScope {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny style nit: maybe move this declaration out of this scope and maybe into the flycheck crate?

// A <- B <- C
//
// [1]: But see FIXME above where we flatten.
crate_root_paths.iter().find_map(|root| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i'll have further comments on this section later once i understand it a little more.

!world.config.flycheck_workspace(source_root_id)
|| flycheck.cannot_run_workspace()
// No point flychecking the whole workspace when you edited a
// main.rs. It cannot have dependencies.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean "reverse dependencies"?

Comment on lines +180 to +183

/// Bit hacky, but this lets us force the use of restart_for_package when the flycheck
/// configuration does not support restart_workspace.
cannot_run_workspace: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why this is necessary, as it appears to only be used in crates/rust-analyzer/src/handlers/notification.rs for binary crates. it seems like the absence of any reverse dependencies should be sufficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I didn't add this, then if check.workspace was enabled (as it is by default), then we would never run any ProjectJson.runnables flychecks if they included {label}. The overall effect here is that having {label} in your flycheck template implies check.workspace = false.

The concept was that the discovery tool should be the only thing you need to configure. I spent a while trying to track down why nothing was working only to find I had deleted my check.workspace config -- and I am working on this code!

Comment on lines +663 to +668
PackageToRestart::Package(PackageSpecifier::BuildInfo { label: _ }) => {
// No way to flycheck this single package. All we have is a build label.
// There's no way to really say whether this build label happens to be
// a cargo canonical name, so we won't try.
return None;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a review comment, but more resignation: it's unfortunate that it is possible to have a PackageSpecifier::BuildInfo in FlycheckConfig::CargoCommand branch.

Copy link
Contributor Author

@cormacrelf cormacrelf Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename it FlycheckConfig::Automatic. Because it is! In cargo workspaces it does cargo stuff; in json workspaces it does JSON stuff.

Comment on lines +380 to +382
/// If you have a runnable, and it has {label} in it somewhere, treat it as a template that
/// may be unsatisfied if you do not provide a label to substitute into it. Returns None in
/// that situation. Otherwise performs the requested substitutions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// If you have a runnable, and it has {label} in it somewhere, treat it as a template that
/// may be unsatisfied if you do not provide a label to substitute into it. Returns None in
/// that situation. Otherwise performs the requested substitutions.
/// A basic template engine for runnables.
///
/// If a runnable has the string `{label}`, the whole runnable will be treated as a template.

@@ -232,13 +367,62 @@ enum FlycheckStatus {
Finished,
}

pub(crate) const SAVED_FILE_PLACEHOLDER: &str = "$saved_file";
/// This is stable behaviour. Don't change.
const SAVED_FILE_PLACEHOLDER: &str = "$saved_file";
Copy link
Contributor

@davidbarsky davidbarsky Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const SAVED_FILE_PLACEHOLDER: &str = "$saved_file";
const SAVED_FILE_PLACEHOLDER_DOLLAR: &str = "$saved_file";
const SAVED_FILE_PLACEHOLDER: &str = "{saved_file}";

@@ -232,13 +367,62 @@ enum FlycheckStatus {
Finished,
}

pub(crate) const SAVED_FILE_PLACEHOLDER: &str = "$saved_file";
/// This is stable behaviour. Don't change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, we can remove $saved_file with a bit of notice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, no, I lied we need to keep this for while. the main user of this is Fuchsia and until they're using Bazel, I don't think it's feasible to remove $saved_file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(cc: @P1n3appl3)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the ping. We'd actually be ok with removing $saved_file if we could use {label} in the flycheck override. Today we actually do that mapping ourselves (wrapper script checks a global file list to find all labels that include the saved file and runs a check build/clippy on all those in the build system).

I'd be happy to migrate us over to label replacement when this lands.

.for_each(|rev_dep| worklist.push(rev_dep));
krate_rev_deps.iter().copied().filter(|&rev_dep| visited.insert(rev_dep)).for_each(
|rev_dep| {
rev_deps.push(rev_dep);
Copy link
Contributor Author

@cormacrelf cormacrelf Sep 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ayyy, that is a very sus line in retrospect, pretty sure this is wrong. This needs tests.

@davidbarsky
Copy link
Contributor

Just to confirm my understanding: when you say:

Allows the discoverConfig to describe how to flycheck crates, using a new runnable kind of "flycheck".

The code changes here seem to imply that it'd be relying on the runnables in a rust-project.json; not that you're adding a few field to the configuration object/value. Is that correct? Do you intend to add a new field to discoverConfig...?

@cormacrelf
Copy link
Contributor Author

Correct. However now that I think about it, the runnable templating would be a useful thing to expose to end users to override. They can then have custom test commands, debug commands, etc.

(Noting that Meta's internal test harness has different flags to the OSS one. Would we want to have another configuration layer for rust-project to pick up and change its output? I would hope not!)

@davidbarsky
Copy link
Contributor

Correct.

Sorry, I realized I phrased my question ambiguously: are you planning on adding another field to workspace.discoverConfig for flycheck-specific? If the answer is yes, I'd push back on that and ask for a more wholistic design of flycheck's configuration: it's pretty messy and I don't want to make it worse.

However now that I think about it, the runnable templating would be a useful thing to expose to end users to override. They can then have custom test commands, debug commands, etc.

I agree! I think it's be nice to expose that via the standard rust-analyzer configuration mechanism.

Noting that Meta's internal test harness has different flags to the OSS one.

Oi, I didn't realize that fact, but good to know. Lemme see if I can try and unify the two.

Would we want to have another configuration layer for rust-project to pick up and change its output? I would hope not!

We should probably move this discussion over to the buck2 repo, but I've wanted a rust-project.toml for ages. There a lot of stuff in rust-project that's #[cfg(fbcode)]'d that I'd want to be controlled through configuration. It also means that setting up Buck with rust-analyzer could be as simple as placing a predefined .rust-project.toml and .rust-analyzer.toml at the root of a Buck workspace. Heck, we could even make the buck2 init command do that.

@bors
Copy link
Contributor

bors commented Sep 24, 2024

☔ The latest upstream changes (presumably #18123) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants