-
Notifications
You must be signed in to change notification settings - Fork 302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Plugin fails to get metadata if java/scala compilation fails #1167
Comments
There are a few reasons for the current situation:
Instead, we assume that the 'resolve' build will succeed initially (for instance, because the user is checking out code without local modifications). Any subsequent builds for which some targets fail to compile will result in that target's 'metadata' and compilation output remaining unchanged, which is generally the desired outcome. We'd need a very good reason to consider changing this approach. |
Thanks for responding! I think part of the problem in the above assumption is that it's a bit too clean. Sometimes someone will rebase onto master they'll have a non compiling file. If this file is in the "middle" of the graph then all dependent targets will be messed up even if they are new from master. This is something that's not trivial to track down without deeply investigating the tool or really understanding Bazel. |
I did combine the two build steps recently (a few months ago) for performance reasons, but I can't think of a situation where that would actually cause a change in behavior like you're seeing. We request all the build outputs in the same build invocation, but we still handle the 'info' and 'resolve' outputs separately -- that hasn't changed. If individual targets fail to build, we should still be updating their metadata in the project data. It might help to have a small reproducer here (ideally in java if possible). |
I'll add a repro no problem. Main issue is where |
repro repo: https://github.com/ittaiz/bazel-ij-info-with-resolve |
The current behavior seems to fit my intuition again. It all boils down to this:
However, if you add This has nothing to do with the difference between Interestingly, even though
So the fix here should be adding |
Thanks for the repro project, I see the problem. The part I was missing is that the intellij-info.txt output artifact is produced (expected), but isn't declared in the BEP output (unexpected). This feels like a bug in BEP. Prior to BEP we relied on parsing stdout/stderr, with the --experimental_show_artifacts flag set. The intellij-info outputs for B are declared just fine there, regardless of whether the jars are successfully compiled. |
@brendandouglas are you saying the root cause is that these two lists aren't consistent with each other?
If we omit
|
Yes, if that second list corresponds to the BEP contents
…On Fri, Sep 20, 2019, 4:22 PM Jin ***@***.***> wrote:
@brendandouglas <https://github.com/brendandouglas> are you saying the
root cause is that these two lists aren't consistent with each other?
$ tree -fi bazel-bin/ | grep intellij-info.txt
bazel-bin/A-65.intellij-info.txt
bazel-bin/B-66.intellij-info.txt
bazel-bin/C-67.intellij-info.txt
bazel-bin/external/bazel_tools/tools/jdk/current_java_toolchain--1286701614.intellij-info.txt
bazel-bin/external/com_google_code_findbugs_jsr305/com_google_code_findbugs_jsr305-1219275564.intellij-info.txt
bazel-bin/external/com_google_errorprone_error_prone_annotations/com_google_errorprone_error_prone_annotations--1327292041.intellij-info.txt
bazel-bin/external/com_google_guava_guava/com_google_guava_guava-46378293.intellij-info.txt
bazel-bin/external/google_java_format/google_java_format--2076037202.intellij-info.txt
bazel-bin/external/remote_java_tools_linux/toolchain--412636119.intellij-info.txt
$ cat /tmp/f.txt | grep intellij-info.txt
name: "external/remote_java_tools_linux/toolchain--412636119.intellij-info.txt"
uri: "file:///usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/af0112346abd86e45930298bb524e045/execroot/main/bazel-out/k8-fastbuild/bin/external/remote_java_tools_linux/toolchain--412636119.intellij-info.txt"
name: "external/bazel_tools/tools/jdk/current_java_toolchain--1286701614.intellij-info.txt"
uri: "file:///usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/af0112346abd86e45930298bb524e045/execroot/main/bazel-out/k8-fastbuild/bin/external/bazel_tools/tools/jdk/current_java_toolchain--1286701614.intellij-info.txt"
name: "C-67.intellij-info.txt"
uri: "file:///usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/af0112346abd86e45930298bb524e045/execroot/main/bazel-out/k8-fastbuild/bin/C-67.intellij-info.txt"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1167?email_source=notifications&email_token=AE3KDEOVXIH5T7KHOZ25MNTQKUWHBA5CNFSM4IWRKUL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7HY7DI#issuecomment-533696397>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AE3KDEI3STS4N63GE5HEVUDQKUWHBANCNFSM4IWRKULQ>
.
|
Thanks, filed bazelbuild/bazel#9413 |
Interesting, thanks! |
Also, |
Thanks for filing the Bazel-side bug @jin. @ittaiz the intellij-info outputs are not dependent on the compilation outputs, so in theory they could be implemented with different graphs. In practice I doubt they are though. There's no cheap workaround, so I'll wait to hear back from the Bazel team. If they're willing to support this use case, I'll leave as is and wait for a Bazel-side fix. If not, I'll conditionally separate the 'info' and 'resolve' steps. |
Technically they are different subgraphs in the Skyframe-sense, since the intellij-info outputs were requested by a different output group from the compilation output group. I'll look into this issue. It's a particularly specific but important edge case for IDE use cases. |
Digging into this a little deeper, this issue only seems to happen if you're doing an initial sync and that the Java code doesn't compile. If you've successfully performed an initial sync, and then you sync again and a compilation error happens (e.g. B in the repro project), the metadata of the rest of the files don't get affected. This seems WAI to me. @ittaiz @or-shachar could you please confirm about my diagnosis in your repro project? |
@jin I'm not sure I understand. Is the scenario you're describing about whether or not this issue also discards existing metadata of previously resolved and working files ( |
@brendandouglas @jin if Bazel won't support this use-case what do you think about running the |
The info command always runs before the command that builds the aspect. Do you mean separating out |
Yes, exactly |
@brendandouglas wdyt about adding a fallback
What we'd like is:
For the user it's to go from 5 operations to 2 where often people have a hard time because they forget to comment out the symbol and so the sync fails and they waste some time on this. We'd be really interested in contributing this change if it's ok with you. |
I'd prefer to push for a Bazel-side fix here, rather than adding more complexity to the sync process. I've pinged the bazel-side bug and filed an internal bug for this issue -- let's see what the response is. In the meantime, there's one part of that situation I don't understand. If both the source file and target deps are updated in step 1, why does the build fail in step 3 (without step 2)? |
Good question (and sorry for missing the notification)- what we add in step 1 is merely the symbol in the editor. |
Thanks, that workflow makes sense. Though I don't understand how the ide-info data helps. I'd expect that the target is already synced prior to step 1, so adding 'Foo' to the source file then syncing wouldn't change anything (you'd still be missing both the import and dep and syncing wouldn't add them) |
The difference would have been in IJ-Bazel plugin "discovering" about If you think I'm missing something I'd love to know (I'm fairly certain this used to work for us before the unification to one invocation) |
Also a separate use-case (and the one that caused us to look at this issue) is when you're working on prod code A and it doesn't compile for some reason and you're switching to the test target ATest to add some code that requires a new dependency you need to "blindly" code in the editor because "new-test-dep" won't be identified until you stash changes to A and probably to ATest as well. |
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
…ith info only This is to make sure we have the needed metadata even if build broke Relates to: bazelbuild#1167 bazelbuild/bazel#9413 Fix test
Thank you for contributing to the IntelliJ repository! This issue has been marked as stale since it has not had any activity in the last 2 years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-maintainer". Please reach out to the triage team ( |
This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team ( |
Using bazel scala project we noticed that if compilation fails the plugin fails to get the metadata.
When we try to sync currently the plugin runs 2 bazel commands:
We noticed that if we remove the
intellij-resolve-java
the second operation will succeed.We should probably break the second bazel invocation to two phases so we will still get the metadata even if the compilation fails.
cc: @jin @chaoren @ittaiz
The text was updated successfully, but these errors were encountered: