Shaded Checker Framework dataflow library can crash for JDK12-JDK15 (which are among the versions it doesn't support) #3540
Comments
copybara-service bot
pushed a commit
to google/guava
that referenced
this issue
Nov 15, 2022
(Guava is already _usable_ under plenty of verions. This change affects only people who build it themselves.) And run CI under JDK17. Maybe this will make CI painfully slow, but we'll see what happens. If we want to drop something, we should consider whether to revert 17 or to drop 11 instead (so as to maintain coverage at the endpoints of \[8, 17\]). ## Notes on some of the versions ### JDK9 I expected Error Prone to work, but I saw `invalid flag: -Xep:NullArgumentForNonNullParameter:OFF`, even though that flag is [already](https://github.com/google/guava/blob/166d8c0d8733d40914fb24f368cb587a92bddfe0/pom.xml#L515) part of [the same `<arg>`](google/error-prone#1086 (comment)), which works fine for other JDK versions. So I disabled Error Prone for that version. Then I had a Javadoc problem with the `--no-module-directories` configuration from cl/413934851 (the fix for #5457). After reading [JDK-8215582](https://bugs.openjdk.org/browse/JDK-8215582) more carefully, I get the impression that that flag might not have been added until 11: "addressed in JDK 11, along with an option to revert to the old layout in case of need." So I disabled it for 9-10. Then I ran into a problem similar to bazelbuild/bazel#6173 / [JDK-8184940](https://bugs.openjdk.java.net/browse/JDK-8184940). I'm not sure exactly what tool produced a file with a month of 0, but it happened only when building `guava-tests`. At that point, I gave up, though I left the 2 above workarounds in place. ### JDK10 This fails with some kind of problem finding a Guice dependency inside Maven. I didn't investigate. ### JDK15 and JDK16 These fail with [the `TreeMap` bug](https://bugs.openjdk.org/browse/JDK-8259622) that [our collection testers had detected](#5801 (comment)) but we never got around to reporting. Thankfully, it got reported and [fixed](openjdk/jdk@2c8e337) for JDK17. We could consider suppressing the tests under that version. ### JDK18, JDK19, and JDK20-early-access These fail with [`SecurityManager` trouble](#5801 (comment)). ## Notes on the other actual changes ### `maven-javadoc-plugin` I set up `maven-javadoc-plugin` to use `-source ${java.specification.version}`. Otherwise, it would [take the version from `maven-compiler-plugin`](#5801 (comment)). That's typically fine: Guava's source code targets Java 8, so `-source 8` "ought" to work. But it doesn't actually work because we also pass Javadoc the _JDK_ sources (so that `{@inheritdoc}` works better), which naturally can target whichever version of the JDK we're building with. ### Error Prone While Error Prone is mostly usable [on JDK11+](https://errorprone.info/docs/installation), some of its checks have [problems under some versions](google/error-prone#3540), at least when they're reporting warnings. This stems from its use of part of the Checker Framework, which [doesn't support JDKs in the gap between 11 and 17](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/framework/src/main/java/org/checkerframework/framework/source/SourceChecker.java#L553-L554). And specifically, it looks like the Checker Framework is [trying to look up `BindingPatternTree` under any JDK12+](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/javacutil/src/main/java/org/checkerframework/javacutil/TreeUtils.java#L131-L144). But `BindingPatternTree` (besides not being present at all [until JDK14](openjdk/jdk@229e0d1#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R41)) didn't declare that method [until JDK16](openjdk/jdk@18bc95b#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R39). Anyway, the problem we saw was [a `NoSuchMethodException` during the `AbstractReferenceEquality` call to `NullnessAnalysis.getNullness`](https://oss-fuzz-build-logs.storage.googleapis.com/log-a9d04aa2-8b5a-47ca-8066-7e6b38548064.txt), which uses Checker Framework dataflow. To address that, I disabled Error Prone for the versions under which I'd expect the `BindingPatternTree` code to be a problem. (I also disabled it for JDK10: As noted above, Error Prone [supports JDK11+](https://errorprone.info/docs/installation). And as noted further above, Maven doesn't get far enough with JDK10 to even start running Error Prone.) Fixes #5801 RELNOTES=n/a PiperOrigin-RevId: 488700624
copybara-service bot
pushed a commit
to google/guava
that referenced
this issue
Nov 15, 2022
(Guava is already _usable_ under plenty of verions. This change affects only people who build it themselves.) And run CI under JDK17. Maybe this will make CI painfully slow, but we'll see what happens. If we want to drop something, we should consider whether to revert 17 or to drop 11 instead (so as to maintain coverage at the endpoints of \[8, 17\]). ## Notes on some of the versions ### JDK9 I expected Error Prone to work, but I saw `invalid flag: -Xep:NullArgumentForNonNullParameter:OFF`, even though that flag is [already](https://github.com/google/guava/blob/166d8c0d8733d40914fb24f368cb587a92bddfe0/pom.xml#L515) part of [the same `<arg>`](google/error-prone#1086 (comment)), which works fine for other JDK versions. So I disabled Error Prone for that version. Then I had a Javadoc problem with the `--no-module-directories` configuration from cl/413934851 (the fix for #5457). After reading [JDK-8215582](https://bugs.openjdk.org/browse/JDK-8215582) more carefully, I get the impression that that flag might not have been added until 11: "addressed in JDK 11, along with an option to revert to the old layout in case of need." So I disabled it for 9-10. Then I ran into a problem similar to bazelbuild/bazel#6173 / [JDK-8184940](https://bugs.openjdk.java.net/browse/JDK-8184940). I'm not sure exactly what tool produced a file with a month of 0, but it happened only when building `guava-tests`. At that point, I gave up, though I left the 2 above workarounds in place. ### JDK10 This fails with some kind of problem finding a Guice dependency inside Maven. I didn't investigate. ### JDK15 and JDK16 These fail with [the `TreeMap` bug](https://bugs.openjdk.org/browse/JDK-8259622) that [our collection testers had detected](#5801 (comment)) but we never got around to reporting. Thankfully, it got reported and [fixed](openjdk/jdk@2c8e337) for JDK17. We could consider suppressing the tests under that version. ### JDK18, JDK19, and JDK20-early-access These fail with [`SecurityManager` trouble](#5801 (comment)). ## Notes on the other actual changes ### `maven-javadoc-plugin` I set up `maven-javadoc-plugin` to use `-source ${java.specification.version}`. Otherwise, it would [take the version from `maven-compiler-plugin`](#5801 (comment)). That's typically fine: Guava's source code targets Java 8, so `-source 8` "ought" to work. But it doesn't actually work because we also pass Javadoc the _JDK_ sources (so that `{@inheritdoc}` works better), which naturally can target whichever version of the JDK we're building with. ### Error Prone While Error Prone is mostly usable [on JDK11+](https://errorprone.info/docs/installation), some of its checks have [problems under some versions](google/error-prone#3540), at least when they're reporting warnings. This stems from its use of part of the Checker Framework, which [doesn't support JDKs in the gap between 11 and 17](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/framework/src/main/java/org/checkerframework/framework/source/SourceChecker.java#L553-L554). And specifically, it looks like the Checker Framework is [trying to look up `BindingPatternTree` under any JDK12+](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/javacutil/src/main/java/org/checkerframework/javacutil/TreeUtils.java#L131-L144). But `BindingPatternTree` (besides not being present at all [until JDK14](openjdk/jdk@229e0d1#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R41)) didn't declare that method [until JDK16](openjdk/jdk@18bc95b#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R39). Anyway, the problem we saw was [a `NoSuchMethodException` during the `AbstractReferenceEquality` call to `NullnessAnalysis.getNullness`](https://oss-fuzz-build-logs.storage.googleapis.com/log-a9d04aa2-8b5a-47ca-8066-7e6b38548064.txt), which uses Checker Framework dataflow. To address that, I disabled Error Prone for the versions under which I'd expect the `BindingPatternTree` code to be a problem. (I also disabled it for JDK10: As noted above, Error Prone [supports JDK11+](https://errorprone.info/docs/installation). And as noted further above, Maven doesn't get far enough with JDK10 to even start running Error Prone.) Fixes #5801 RELNOTES=n/a PiperOrigin-RevId: 488700624
copybara-service bot
pushed a commit
to google/guava
that referenced
this issue
Nov 16, 2022
(Guava is already _usable_ under plenty of verions. This change affects only people who build it themselves.) And run CI under JDK17. Maybe this will make CI painfully slow, but we'll see what happens. If we want to drop something, we should consider whether to revert 17 or to drop 11 instead (so as to maintain coverage at the endpoints of \[8, 17\]). ## Notes on some of the versions ### JDK9 I expected Error Prone to work, but I saw `invalid flag: -Xep:NullArgumentForNonNullParameter:OFF`, even though that flag is [already](https://github.com/google/guava/blob/166d8c0d8733d40914fb24f368cb587a92bddfe0/pom.xml#L515) part of [the same `<arg>`](google/error-prone#1086 (comment)), which works fine for other JDK versions. So I disabled Error Prone for that version. Then I had a Javadoc problem with the `--no-module-directories` configuration from cl/413934851 (the fix for #5457). After reading [JDK-8215582](https://bugs.openjdk.org/browse/JDK-8215582) more carefully, I get the impression that that flag might not have been added until 11: "addressed in JDK 11, along with an option to revert to the old layout in case of need." So I disabled it for 9-10. Then I ran into a problem similar to bazelbuild/bazel#6173 / [JDK-8184940](https://bugs.openjdk.java.net/browse/JDK-8184940). I'm not sure exactly what tool produced a file with a month of 0, but it happened only when building `guava-tests`. At that point, I gave up, though I left the 2 above workarounds in place. ### JDK10 This fails with some kind of problem finding a Guice dependency inside Maven. I didn't investigate. ### JDK15 and JDK16 These fail with [the `TreeMap` bug](https://bugs.openjdk.org/browse/JDK-8259622) that [our collection testers had detected](#5801 (comment)) but we never got around to reporting. Thankfully, it got reported and [fixed](openjdk/jdk@2c8e337) for JDK17. We could consider suppressing the tests under that version. ### JDK18, JDK19, and JDK20-early-access These fail with [`SecurityManager` trouble](#5801 (comment)). ## Notes on the other actual changes ### `maven-javadoc-plugin` I set up `maven-javadoc-plugin` to use `-source ${java.specification.version}`. Otherwise, it would [take the version from `maven-compiler-plugin`](#5801 (comment)). That's typically fine: Guava's source code targets Java 8, so `-source 8` "ought" to work. But it doesn't actually work because we also pass Javadoc the _JDK_ sources (so that `{@inheritdoc}` works better), which naturally can target whichever version of the JDK we're building with. ### Error Prone While Error Prone is mostly usable [on JDK11+](https://errorprone.info/docs/installation), some of its checks have [problems under some versions](google/error-prone#3540), at least when they're reporting warnings. This stems from its use of part of the Checker Framework, which [doesn't support JDKs in the gap between 11 and 17](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/framework/src/main/java/org/checkerframework/framework/source/SourceChecker.java#L553-L554). And specifically, it looks like the Checker Framework is [trying to look up `BindingPatternTree` under any JDK12+](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/javacutil/src/main/java/org/checkerframework/javacutil/TreeUtils.java#L131-L144). But `BindingPatternTree` (besides not being present at all [until JDK14](openjdk/jdk@229e0d1#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R41)) didn't declare that method [until JDK16](openjdk/jdk@18bc95b#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R39). Anyway, the problem we saw was [a `NoSuchMethodException` during the `AbstractReferenceEquality` call to `NullnessAnalysis.getNullness`](https://oss-fuzz-build-logs.storage.googleapis.com/log-a9d04aa2-8b5a-47ca-8066-7e6b38548064.txt), which uses Checker Framework dataflow. To address that, I disabled Error Prone for the versions under which I'd expect the `BindingPatternTree` code to be a problem. (I also disabled it for JDK10: As noted above, Error Prone [supports JDK11+](https://errorprone.info/docs/installation). And as noted further above, Maven doesn't get far enough with JDK10 to even start running Error Prone.) Fixes #5801 RELNOTES=n/a PiperOrigin-RevId: 488700624
copybara-service bot
pushed a commit
to google/guava
that referenced
this issue
Nov 16, 2022
(Guava is already _usable_ under plenty of verions. This change affects only people who build it themselves.) And run CI under JDK17. Maybe this will make CI painfully slow, but we'll see what happens. If we want to drop something, we should consider whether to revert 17 or to drop 11 instead (so as to maintain coverage at the endpoints of \[8, 17\]). ## Notes on some of the versions ### JDK9 I expected Error Prone to work, but I saw `invalid flag: -Xep:NullArgumentForNonNullParameter:OFF`, even though that flag is [already](https://github.com/google/guava/blob/166d8c0d8733d40914fb24f368cb587a92bddfe0/pom.xml#L515) part of [the same `<arg>`](google/error-prone#1086 (comment)), which works fine for other JDK versions. So I disabled Error Prone for that version. Then I had a Javadoc problem with the `--no-module-directories` configuration from cl/413934851 (the fix for #5457). After reading [JDK-8215582](https://bugs.openjdk.org/browse/JDK-8215582) more carefully, I get the impression that that flag might not have been added until 11: "addressed in JDK 11, along with an option to revert to the old layout in case of need." So I disabled it for 9-10. Then I ran into a problem similar to bazelbuild/bazel#6173 / [JDK-8184940](https://bugs.openjdk.java.net/browse/JDK-8184940). I'm not sure exactly what tool produced a file with a month of 0, but it happened only when building `guava-tests`. At that point, I gave up, though I left the 2 above workarounds in place. ### JDK10 This fails with some kind of problem finding a Guice dependency inside Maven. I didn't investigate. ### JDK15 and JDK16 These fail with [the `TreeMap` bug](https://bugs.openjdk.org/browse/JDK-8259622) that [our collection testers had detected](#5801 (comment)) but we never got around to reporting. Thankfully, it got reported and [fixed](openjdk/jdk@2c8e337) for JDK17. We could consider suppressing the tests under that version. ### JDK18, JDK19, and JDK20-early-access These fail with [`SecurityManager` trouble](#5801 (comment)). ## Notes on the other actual changes ### `maven-javadoc-plugin` I set up `maven-javadoc-plugin` to use `-source ${java.specification.version}`. Otherwise, it would [take the version from `maven-compiler-plugin`](#5801 (comment)). That's typically fine: Guava's source code targets Java 8, so `-source 8` "ought" to work. But it doesn't actually work because we also pass Javadoc the _JDK_ sources (so that `{@inheritdoc}` works better), which naturally can target whichever version of the JDK we're building with. ### Error Prone While Error Prone is mostly usable [on JDK11+](https://errorprone.info/docs/installation), some of its checks have [problems under some versions](google/error-prone#3540), at least when they're reporting warnings. This stems from its use of part of the Checker Framework, which [doesn't support JDKs in the gap between 11 and 17](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/framework/src/main/java/org/checkerframework/framework/source/SourceChecker.java#L553-L554). And specifically, it looks like the Checker Framework is [trying to look up `BindingPatternTree` under any JDK12+](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/javacutil/src/main/java/org/checkerframework/javacutil/TreeUtils.java#L131-L144). But `BindingPatternTree` (besides not being present at all [until JDK14](openjdk/jdk@229e0d1#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R41)) didn't declare that method [until JDK16](openjdk/jdk@18bc95b#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R39). Anyway, the problem we saw was [a `NoSuchMethodException` during the `AbstractReferenceEquality` call to `NullnessAnalysis.getNullness`](https://oss-fuzz-build-logs.storage.googleapis.com/log-a9d04aa2-8b5a-47ca-8066-7e6b38548064.txt), which uses Checker Framework dataflow. To address that, I disabled Error Prone for the versions under which I'd expect the `BindingPatternTree` code to be a problem. (I also disabled it for JDK10: As noted above, Error Prone [supports JDK11+](https://errorprone.info/docs/installation). And as noted further above, Maven doesn't get far enough with JDK10 to even start running Error Prone.) Fixes #5801 RELNOTES=n/a PiperOrigin-RevId: 488700624
copybara-service bot
pushed a commit
to google/guava
that referenced
this issue
Nov 16, 2022
(Guava is already _usable_ under plenty of verions. This change affects only people who build it themselves.) And run CI under JDK17. Maybe this will make CI painfully slow, but we'll see what happens. If we want to drop something, we should consider whether to revert 17 or to drop 11 instead (so as to maintain coverage at the endpoints of \[8, 17\]). ## Notes on some of the versions ### JDK9 I expected Error Prone to work, but I saw `invalid flag: -Xep:NullArgumentForNonNullParameter:OFF`, even though that flag is [already](https://github.com/google/guava/blob/166d8c0d8733d40914fb24f368cb587a92bddfe0/pom.xml#L515) part of [the same `<arg>`](google/error-prone#1086 (comment)), which works fine for other JDK versions. So I disabled Error Prone for that version. Then I had a Javadoc problem with the `--no-module-directories` configuration from cl/413934851 (the fix for #5457). After reading [JDK-8215582](https://bugs.openjdk.org/browse/JDK-8215582) more carefully, I get the impression that that flag might not have been added until 11: "addressed in JDK 11, along with an option to revert to the old layout in case of need." So I disabled it for 9-10. Then I ran into a problem similar to bazelbuild/bazel#6173 / [JDK-8184940](https://bugs.openjdk.java.net/browse/JDK-8184940). I'm not sure exactly what tool produced a file with a month of 0, but it happened only when building `guava-tests`. At that point, I gave up, though I left the 2 above workarounds in place. ### JDK10 This fails with some kind of problem finding a Guice dependency inside Maven. I didn't investigate. ### JDK15 and JDK16 These fail with [the `TreeMap` bug](https://bugs.openjdk.org/browse/JDK-8259622) that [our collection testers had detected](#5801 (comment)) but we never got around to reporting. Thankfully, it got reported and [fixed](openjdk/jdk@2c8e337) for JDK17. We could consider suppressing the tests under that version. ### JDK18, JDK19, and JDK20-early-access These fail with [`SecurityManager` trouble](#5801 (comment)). ## Notes on the other actual changes ### `maven-javadoc-plugin` I set up `maven-javadoc-plugin` to use `-source ${java.specification.version}`. Otherwise, it would [take the version from `maven-compiler-plugin`](#5801 (comment)). That's typically fine: Guava's source code targets Java 8, so `-source 8` "ought" to work. But it doesn't actually work because we also pass Javadoc the _JDK_ sources (so that `{@inheritdoc}` works better), which naturally can target whichever version of the JDK we're building with. ### Error Prone While Error Prone is mostly usable [on JDK11+](https://errorprone.info/docs/installation), some of its checks have [problems under some versions](google/error-prone#3540), at least when they're reporting warnings. This stems from its use of part of the Checker Framework, which [doesn't support JDKs in the gap between 11 and 17](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/framework/src/main/java/org/checkerframework/framework/source/SourceChecker.java#L553-L554). And specifically, it looks like the Checker Framework is [trying to look up `BindingPatternTree` under any JDK12+](https://github.com/typetools/checker-framework/blob/c2d16b3409000ac2e2ca95b8b81ae11e42195308/javacutil/src/main/java/org/checkerframework/javacutil/TreeUtils.java#L131-L144). But `BindingPatternTree` (besides not being present at all [until JDK14](openjdk/jdk@229e0d1#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R41)) didn't declare that method [until JDK16](openjdk/jdk@18bc95b#diff-3db4b0ce4411c851bcf75d92ef4dadc7351debcf0f9b2c2623dc513923b45867R39). Anyway, the problem we saw was [a `NoSuchMethodException` during the `AbstractReferenceEquality` call to `NullnessAnalysis.getNullness`](https://oss-fuzz-build-logs.storage.googleapis.com/log-a9d04aa2-8b5a-47ca-8066-7e6b38548064.txt), which uses Checker Framework dataflow. To address that, I disabled Error Prone for the versions under which I'd expect the `BindingPatternTree` code to be a problem. (I also disabled it for JDK10: As noted above, Error Prone [supports JDK11+](https://errorprone.info/docs/installation). And as noted further above, Maven doesn't get far enough with JDK10 to even start running Error Prone.) Fixes #5801 RELNOTES=n/a PiperOrigin-RevId: 488902996
dkfellows
added a commit
to SpiNNakerManchester/JavaSpiNNaker
that referenced
this issue
Jun 15, 2023
This isn't well documented in the slightest, but google/guava#6243 points to the right place and google/error-prone#3540 is the core issue.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Since no one else seems to have hit this, I don't see much need to address it. But if it's worth a look someday, we could try skipping dataflow for JDK 12, 13, 14, and 15. Of course, that works fine for a check like
AbstractReferenceEqualitythat uses dataflow only to generate simpler fixes—but it wouldn't be a good option if a check relies on dataflow.Mostly I'm filing this bug because I feel like this stack trace should appear somewhere on the Internet for anyone who might see it again. I also put some notes about Error Prone at the end of google/guava#6243.
The text was updated successfully, but these errors were encountered: