Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-16928. Make javadoc work on Java 17 #6976

Merged
merged 2 commits into from
Sep 4, 2024

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Aug 2, 2024

Description of PR

HADOOP-15902 added -html4 options for javadoc on Java 11 and above, while javadoc -html4 option has been removed since Java 13 (https://bugs.openjdk.java.net/browse/JDK-8215578).

To make the javadoc work on Java 17, I must remove the -html4 and fix all errors for -html5(this is default and is the only option for Java 17 javadoc)

How was this patch tested?

macOS Java 17, aarch64

$ java -version
openjdk version "17.0.8" 2023-07-18 LTS
OpenJDK Runtime Environment Zulu17.44+15-CA (build 17.0.8+7-LTS)
OpenJDK 64-Bit Server VM Zulu17.44+15-CA (build 17.0.8+7-LTS, mixed mode, sharing)

this works

$ mvn package -Pdocs -DskipTests

this also works

$ mvn clean install -DskipTests
$ mvn javadoc:javadoc

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

* <caption>Memory usage by blocksize</caption>
* <colgroup> <col width="33%" > <col width="33%" > <col width="33%" >
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed/replaced all attribute/tag that were complained by the javadoc and were removed in HTML5, this might slightly impact the final HTML style but I think it should be okay.

@pan3793
Copy link
Member Author

pan3793 commented Aug 2, 2024

@slfan1989
Copy link
Contributor

slfan1989 commented Aug 2, 2024

@pan3793 I will attempt to verify compatibility with JDK 17/JDK 11/JDK 8. If compatibility is confirmed, I will +1.

The Hadoop-Yetus report indicates issues with blanks, checkstyle, and sputbug that need to be addressed.
I have submitted a pr(#6977) with fixes for sputbug. The problem of non-existent Java doc parameters is outside the scope of this pr.

@hadoop-yetus

This comment was marked as outdated.

@@ -185,22 +185,22 @@
* &lt;/property&gt;
* </code></pre>
*
* <p>When <tt>conf.get("tempdir")</tt> is called, then <tt>${<i>basedir</i>}</tt>
* <p>When <code>conf.get("tempdir")</code> is called, then <code>${<i>basedir</i>}</code>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's also the {@code } clause, but given search and replace is easier. your PR is better

@pan3793
Copy link
Member Author

pan3793 commented Aug 7, 2024

To reviewers: part of this PR is separated to #6975, please take a look at that first.

@hadoop-yetus

This comment was marked as outdated.

Copy link
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lots of OOM problems here, hopefully unrelated

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 4s Maven dependency ordering for branch
+1 💚 mvninstall 19m 55s trunk passed
+1 💚 compile 9m 3s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚 compile 8m 8s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚 checkstyle 2m 21s trunk passed
+1 💚 mvnsite 10m 3s trunk passed
+1 💚 javadoc 8m 43s trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚 javadoc 8m 54s trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗 spotbugs 0m 31s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌ spotbugs 0m 48s /branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core in trunk has 1 extant spotbugs warnings.
+1 💚 shadedclient 20m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for patch
+1 💚 mvninstall 5m 32s the patch passed
+1 💚 compile 8m 40s the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚 javac 8m 40s the patch passed
+1 💚 compile 8m 0s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚 javac 8m 0s the patch passed
-1 ❌ blanks 0m 0s /blanks-eol.txt The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 2m 9s /results-checkstyle-root.txt root: The patch generated 5 new + 1701 unchanged - 2 fixed = 1706 total (was 1703)
+1 💚 mvnsite 9m 51s the patch passed
-1 ❌ javadoc 0m 35s /results-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-client-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt hadoop-hdfs-project_hadoop-hdfs-client-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 1062 new + 99 unchanged - 0 fixed = 1161 total (was 99)
-1 ❌ javadoc 0m 47s /results-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 3017 new + 99 unchanged - 0 fixed = 3116 total (was 99)
-1 ❌ javadoc 0m 28s /results-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt hadoop-mapreduce-project_hadoop-mapreduce-examples-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 42 new + 99 unchanged - 0 fixed = 141 total (was 99)
-1 ❌ javadoc 0m 30s /results-javadoc-javadoc-hadoop-tools_hadoop-streaming-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt hadoop-tools_hadoop-streaming-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 74 new + 99 unchanged - 0 fixed = 173 total (was 99)
-1 ❌ javadoc 0m 32s /results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚 javadoc 9m 8s the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗 spotbugs 0m 26s hadoop-project has no data from spotbugs
+1 💚 shadedclient 20m 51s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 26s hadoop-project in the patch passed.
+1 💚 unit 16m 33s hadoop-common in the patch passed.
+1 💚 unit 2m 3s hadoop-hdfs-client in the patch passed.
+1 💚 unit 199m 49s hadoop-hdfs in the patch passed.
+1 💚 unit 0m 57s hadoop-yarn-api in the patch passed.
+1 💚 unit 4m 46s hadoop-yarn-common in the patch passed.
+1 💚 unit 2m 57s hadoop-yarn-server-common in the patch passed.
+1 💚 unit 6m 32s hadoop-mapreduce-client-core in the patch passed.
+1 💚 unit 7m 5s hadoop-mapreduce-client-app in the patch passed.
+1 💚 unit 112m 5s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 unit 0m 43s hadoop-yarn-server-timelineservice-hbase-common in the patch passed.
+1 💚 unit 20m 3s hadoop-yarn-services-core in the patch passed.
+1 💚 unit 0m 58s hadoop-mapreduce-examples in the patch passed.
+1 💚 unit 6m 2s hadoop-streaming in the patch passed.
+1 💚 unit 14m 33s hadoop-gridmix in the patch passed.
+1 💚 unit 2m 22s hadoop-aws in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
612m 39s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6976/3/artifact/out/Dockerfile
GITHUB PR #6976
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux d930f0603ab6 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1b12f2d
Default Java Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6976/3/testReport/
Max. process+thread count 4009 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-streaming hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-aws U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6976/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Member

@aajisaka aajisaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thank you for supporting Java 17.

@steveloughran steveloughran merged commit 9486844 into apache:trunk Sep 4, 2024
2 of 5 checks passed
KeeProMise pushed a commit to KeeProMise/hadoop that referenced this pull request Sep 9, 2024
Hexiaoqiao pushed a commit to Hexiaoqiao/hadoop that referenced this pull request Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants