HDFS-16714: Replace okhttp by apache http client #4684

pan3793 · 2022-08-03T11:40:17Z

Description of PR

Repalce okhttp by apache http client, and remove kotlin dependencies

How was this patch tested?

Existing test.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

pan3793 · 2022-08-03T11:50:15Z

Hi @zhongjun2, currently, esdk-obs-java:3.20.4.2 still depends on okhttp, I checked the latest https://mvnrepository.com/artifact/com.huaweicloud/esdk-obs-java/3.22.3.1 removed (or shaded?) it, could you please update it?

hadoop-yetus · 2022-08-03T19:40:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 25s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 30s		trunk passed
+1 💚	compile	23m 17s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	20m 45s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	4m 25s		trunk passed
+1 💚	mvnsite	19m 23s		trunk passed
+1 💚	javadoc	8m 18s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 23s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 35s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 34s		branch/hadoop-client-modules/hadoop-client no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	55m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 40s		Maven dependency ordering for patch
-1 ❌	mvninstall	1m 30s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	compile	22m 24s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	22m 24s		the patch passed
+1 💚	compile	20m 42s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	20m 42s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 1 new + 2653 unchanged - 3 fixed = 2654 total (was 2656)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 13s	/results-checkstyle-root.txt	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	18m 30s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 5s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 31s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 34s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	2m 51s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html	hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+0 🆗	spotbugs	0m 34s		hadoop-client-modules/hadoop-client has no data from spotbugs
-1 ❌	spotbugs	33m 33s	/new-spotbugs-root.html	root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	shadedclient	38m 7s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	154m 54s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	1m 16s		ASF License check generated no output?
		474m 1s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs-client
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
SpotBugs	module:root
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
Failed junit tests	hadoop.hdfs.TestParallelUnixDomainRead
	hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy
	hadoop.hdfs.TestHDFSFileSystemContract
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestReadStripedFileWithDNFailure
	hadoop.hdfs.TestGetBlocks
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.TestLeaseRecovery
	hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy
	hadoop.hdfs.TestDFSClientExcludedNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/4/artifact/out/Dockerfile
GITHUB PR	#4684
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 47ed3315595c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b5e1ed5`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/4/testReport/
Max. process+thread count	3277 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs-client hadoop-client-modules/hadoop-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-03T19:49:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 29s		Maven dependency ordering for branch
+1 💚	mvninstall	27m 8s		trunk passed
+1 💚	compile	26m 4s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	22m 16s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	4m 50s		trunk passed
+1 💚	mvnsite	20m 24s		trunk passed
+1 💚	javadoc	8m 33s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 54s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 34s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 30s		branch/hadoop-client-modules/hadoop-client no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	54m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
-1 ❌	mvninstall	1m 32s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	compile	22m 21s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	22m 21s		the patch passed
+1 💚	compile	20m 46s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	20m 46s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 1 new + 2650 unchanged - 1 fixed = 2651 total (was 2651)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 10s	/results-checkstyle-root.txt	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	18m 42s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	javadoc	8m 12s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 24s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 34s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	2m 49s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html	hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+0 🆗	spotbugs	0m 34s		hadoop-client-modules/hadoop-client has no data from spotbugs
-1 ❌	spotbugs	33m 32s	/new-spotbugs-root.html	root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	shadedclient	38m 5s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	160m 25s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	1m 4s		ASF License check generated no output?
		487m 47s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs-client
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
SpotBugs	module:root
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
Failed junit tests	hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
	hadoop.hdfs.server.datanode.TestDataNodeLifeline
	hadoop.hdfs.server.datanode.TestBlockRecovery2
	hadoop.hdfs.server.datanode.TestDataNodeMetricsLogger
	hadoop.hdfs.server.datanode.TestDataNodeReconfiguration
	hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
	hadoop.hdfs.server.datanode.TestDataNodeVolumeMetrics
	hadoop.hdfs.server.datanode.checker.TestDatasetVolumeCheckerTimeout

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/1/artifact/out/Dockerfile
GITHUB PR	#4684
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 0752119b5afb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1c3e894`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/1/testReport/
Max. process+thread count	2889 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs-client hadoop-client-modules/hadoop-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-04T06:51:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 32s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 16s		trunk passed
+1 💚	compile	24m 27s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	20m 55s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 47s		trunk passed
+1 💚	mvnsite	19m 25s		trunk passed
+1 💚	javadoc	8m 20s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 23s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 22s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 21s		branch/hadoop-client-modules/hadoop-client no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	55m 21s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 36s		Maven dependency ordering for patch
-1 ❌	mvninstall	1m 27s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	compile	23m 41s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	23m 41s		the patch passed
+1 💚	compile	20m 35s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	20m 35s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 4 new + 2647 unchanged - 1 fixed = 2651 total (was 2648)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 37s	/results-checkstyle-root.txt	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	18m 59s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 14s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 21s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 21s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	2m 40s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html	hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+0 🆗	spotbugs	0m 20s		hadoop-client-modules/hadoop-client has no data from spotbugs
-1 ❌	spotbugs	35m 34s	/new-spotbugs-root.html	root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	shadedclient	39m 24s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	801m 0s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 28s		The patch does not generate ASF License warnings.
		1121m 40s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs-client
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
SpotBugs	module:root
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
Failed junit tests	hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/5/artifact/out/Dockerfile
GITHUB PR	#4684
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 8a65fdb51ce2 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3 10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0056f0d`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/5/testReport/
Max. process+thread count	3577 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs-client hadoop-client-modules/hadoop-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-04T07:21:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 35s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 29s		trunk passed
+1 💚	compile	26m 3s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	23m 6s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	4m 29s		trunk passed
+1 💚	mvnsite	20m 32s		trunk passed
+1 💚	javadoc	8m 57s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 45s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 29s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 26s		branch/hadoop-client-modules/hadoop-client no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	56m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 40s		Maven dependency ordering for patch
-1 ❌	mvninstall	1m 32s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	compile	25m 8s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	25m 8s		the patch passed
+1 💚	compile	23m 20s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	23m 20s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 2 new + 2649 unchanged - 2 fixed = 2651 total (was 2651)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 19s	/results-checkstyle-root.txt	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	20m 12s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 54s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 44s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 27s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	2m 59s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html	hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+0 🆗	spotbugs	0m 29s		hadoop-client-modules/hadoop-client has no data from spotbugs
-1 ❌	spotbugs	37m 28s	/new-spotbugs-root.html	root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	shadedclient	41m 41s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	836m 3s		root in the patch passed.
+1 💚	asflicense	2m 39s		The patch does not generate ASF License warnings.
		1177m 31s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs-client
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
SpotBugs	module:root
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/2/artifact/out/Dockerfile
GITHUB PR	#4684
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux c5af723301a9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b5e1ed5`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/2/testReport/
Max. process+thread count	3137 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs-client hadoop-client-modules/hadoop-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-04T07:26:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 18s		trunk passed
+1 💚	compile	26m 4s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	23m 20s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	4m 9s		trunk passed
+1 💚	mvnsite	20m 40s		trunk passed
+1 💚	javadoc	8m 58s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 49s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 29s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 29s		branch/hadoop-client-modules/hadoop-client no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	60m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 40s		Maven dependency ordering for patch
-1 ❌	mvninstall	1m 41s	/patch-mvninstall-root.txt	root in the patch failed.
+1 💚	compile	25m 18s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javac	25m 18s		the patch passed
+1 💚	compile	22m 46s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	22m 46s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 2 new + 2653 unchanged - 2 fixed = 2655 total (was 2655)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 24s	/results-checkstyle-root.txt	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	mvnsite	20m 7s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 55s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	7m 46s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗	spotbugs	0m 30s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	2m 54s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html	hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+0 🆗	spotbugs	0m 30s		hadoop-client-modules/hadoop-client has no data from spotbugs
-1 ❌	spotbugs	37m 16s	/new-spotbugs-root.html	root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	shadedclient	41m 43s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	836m 8s		root in the patch passed.
+1 💚	asflicense	2m 43s		The patch does not generate ASF License warnings.
		1181m 14s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs-client
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]
SpotBugs	module:root
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.ConfRefreshTokenBasedAccessTokenProvider.refresh() At ConfRefreshTokenBasedAccessTokenProvider.java:[line 145]
	Exception is caught when Exception is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:is not thrown in org.apache.hadoop.hdfs.web.oauth2.CredentialBasedAccessTokenProvider.refresh() At CredentialBasedAccessTokenProvider.java:[line 139]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/3/artifact/out/Dockerfile
GITHUB PR	#4684
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 93bb3266aa07 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b5e1ed5`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/3/testReport/
Max. process+thread count	3137 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-hdfs-project/hadoop-hdfs-client hadoop-client-modules/hadoop-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4684/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran · 2022-08-04T10:27:45Z

esdk-obs-java:3.20.4.2 still depends on okhttp,

at least that would restrict the use to a subset of the code, not hdfs

pan3793 · 2022-08-04T10:42:08Z

cc @Yikun, I'm not familiar w/ Huawai OBS, does it make sense and safe to upgrade esdk-obs-java to get rid of okhttp deps?

iwasakims · 2022-08-09T14:48:03Z

@pan3793
I got following error on building trunk with the patch applied.

$ mvn clean install -DskipTests
...
[WARNING]
Dependency convergence error for org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.0:test paths to dependency are:
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT
  +-com.squareup.okhttp3:mockwebserver:jar:4.9.3:test
    +-com.squareup.okhttp3:okhttp:jar:4.9.3:test
      +-com.squareup.okio:okio:jar:2.8.0:test
        +-org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.0:test
and
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT
  +-com.squareup.okhttp3:mockwebserver:jar:4.9.3:test
    +-com.squareup.okhttp3:okhttp:jar:4.9.3:test
      +-org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.10:test
and
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT
  +-com.squareup.okhttp3:mockwebserver:jar:4.9.3:test
    +-org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.4.10:test
      +-org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.10:test
and
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT
  +-com.squareup.okhttp3:mockwebserver:jar:4.9.3:test
    +-org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.4.10:test
      +-org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:1.4.10:test
        +-org.jetbrains.kotlin:kotlin-stdlib:jar:1.4.10:test

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
Failed while enforcing releasability. See above detailed error message.

pan3793 · 2024-07-09T10:24:33Z

Completed by HADOOP-18890, except for transitive deps coming from Huawei OBS client.

pan3793 added 4 commits August 3, 2022 19:38

Repalce okhttp by apache http client

d56fbd9

nit

7d67c60

nit

1c3e894

nit

b5e1ed5

nit

0056f0d

pan3793 changed the title ~~HDFS-16714: Repalce okhttp by apache http client~~ HDFS-16714: Replace okhttp by apache http client Aug 4, 2022

pan3793 closed this Jul 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16714: Replace okhttp by apache http client #4684

HDFS-16714: Replace okhttp by apache http client #4684

pan3793 commented Aug 3, 2022

pan3793 commented Aug 3, 2022

hadoop-yetus commented Aug 3, 2022

hadoop-yetus commented Aug 3, 2022

hadoop-yetus commented Aug 4, 2022

hadoop-yetus commented Aug 4, 2022

hadoop-yetus commented Aug 4, 2022

steveloughran commented Aug 4, 2022

pan3793 commented Aug 4, 2022

iwasakims commented Aug 9, 2022

pan3793 commented Jul 9, 2024

HDFS-16714: Replace okhttp by apache http client #4684

HDFS-16714: Replace okhttp by apache http client #4684

Conversation

pan3793 commented Aug 3, 2022

Description of PR

How was this patch tested?

For code changes:

pan3793 commented Aug 3, 2022

hadoop-yetus commented Aug 3, 2022

hadoop-yetus commented Aug 3, 2022

hadoop-yetus commented Aug 4, 2022

hadoop-yetus commented Aug 4, 2022

hadoop-yetus commented Aug 4, 2022

steveloughran commented Aug 4, 2022

pan3793 commented Aug 4, 2022

iwasakims commented Aug 9, 2022

pan3793 commented Jul 9, 2024