HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile #3722

virajjasani · 2021-11-25T11:46:17Z

Description of PR

Upgrade HBase version to 1.7.1 for hbase1 profile. The current version is quite old.
Also, replace old deprecated APIs of Scan with better alternatives.

How was this patch tested?

Verified with some local testing. All CRUD unit tests are tested.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

virajjasani · 2021-11-25T14:06:28Z

TimelineService tests will pass after PR #3712 is merged.

jojochuang · 2021-11-29T09:19:38Z

...main/java/org/apache/hadoop/yarn/server/timelineservice/storage/reader/EntityTypeReader.java

-    Scan scan = new Scan(startPrefix, endPrefix);
+    Scan scan = new Scan();
+    scan.withStartRow(startPrefix);
+    scan.withStopRow(endPrefix);


nit: it would be the best it make it as the following:
scan.withStartRow()
.withStopRow()
.setFilter()
.setSmall()

jojochuang

+1

hadoop-yetus · 2021-11-30T04:05:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	pathlen	0m 0s	/results-pathlen.txt	The patch appears to contain 3 files with names longer than 240
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 38s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 36s		trunk passed
+1 💚	compile	21m 58s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 23s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 46s		trunk passed
+1 💚	mvnsite	26m 32s		trunk passed
+1 💚	javadoc	8m 8s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 26s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 18s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	54m 46s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 53s		Maven dependency ordering for patch
+1 💚	mvninstall	22m 53s		the patch passed
+1 💚	compile	22m 13s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 13s		root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1916 unchanged - 13 fixed = 1916 total (was 1929)
+1 💚	compile	19m 14s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 14s		root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 0 new + 1795 unchanged - 13 fixed = 1795 total (was 1808)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 35s		the patch passed
+1 💚	mvnsite	21m 47s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 47s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 57s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 20s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	55m 49s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	765m 43s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 39s		The patch does not generate ASF License warnings.
		1117m 0s

Reason	Tests
Failed junit tests	hadoop.yarn.csi.client.TestCsiClient
	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/artifact/out/Dockerfile
GITHUB PR	#3722
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname	Linux 5850570d138a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `476b777`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/testReport/
Max. process+thread count	2939 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-30T04:32:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	pathlen	0m 0s	/results-pathlen.txt	The patch appears to contain 3 files with names longer than 240
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 42s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 6s		trunk passed
+1 💚	compile	22m 5s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 19s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 40s		trunk passed
+1 💚	mvnsite	25m 19s		trunk passed
+1 💚	javadoc	8m 1s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 19s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	53m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 55s		Maven dependency ordering for patch
+1 💚	mvninstall	22m 15s		the patch passed
+1 💚	compile	21m 29s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	21m 29s		root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1917 unchanged - 13 fixed = 1917 total (was 1930)
+1 💚	compile	19m 8s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 8s		root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 0 new + 1791 unchanged - 13 fixed = 1791 total (was 1804)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 33s		the patch passed
+1 💚	mvnsite	21m 31s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 49s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 3s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 21s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	54m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	795m 50s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 35s		The patch does not generate ASF License warnings.
		1141m 21s

Reason	Tests
Failed junit tests	hadoop.yarn.csi.client.TestCsiClient
	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/artifact/out/Dockerfile
GITHUB PR	#3722
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname	Linux 11794c51c104 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `476b777`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/testReport/
Max. process+thread count	3426 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-30T08:41:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	pathlen	0m 0s	/results-pathlen.txt	The patch appears to contain 3 files with names longer than 240
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 27s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 33s		trunk passed
+1 💚	compile	25m 43s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	22m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	4m 19s		trunk passed
+1 💚	mvnsite	28m 7s		trunk passed
+1 💚	javadoc	9m 2s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	10m 57s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 16s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	61m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 50s		Maven dependency ordering for patch
+1 💚	mvninstall	26m 56s		the patch passed
+1 💚	compile	24m 11s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	24m 11s		root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1921 unchanged - 13 fixed = 1921 total (was 1934)
+1 💚	compile	22m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	22m 9s		root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 0 new + 1796 unchanged - 13 fixed = 1796 total (was 1809)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 54s		the patch passed
+1 💚	mvnsite	25m 1s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 3s		The patch has no ill-formed XML file.
+1 💚	javadoc	9m 22s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 15s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	62m 16s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	995m 23s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 37s		The patch does not generate ASF License warnings.
		1388m 46s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
	hadoop.yarn.csi.client.TestCsiClient
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/artifact/out/Dockerfile
GITHUB PR	#3722
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname	Linux 431d216ad92f 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `476b777`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/testReport/
Max. process+thread count	2390 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

virajjasani · 2021-11-30T10:54:31Z

Thanks for the review @jojochuang. The build has run for the entire project, all modules combined. Test failures are known (irrelevant to this PR) failures.

jojochuang · 2021-12-02T02:40:39Z

Merged. Thanks @virajjasani !

…he#3722)

virajjasani added 2 commits November 25, 2021 17:12

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile

1e92752

replace deprecate APIs with better alternatives

aa0b228

virajjasani mentioned this pull request Nov 26, 2021

HADOOP-18022. Add restrict-imports-enforcer-rule for Guava Preconditions and remove remaining usages #3712

Merged

1 task

virajjasani added 2 commits November 29, 2021 12:40

update license

6c89765

Merge branch 'trunk' of github.com:apache/hadoop into HADOOP-18025-trunk

7169be6

jojochuang reviewed Nov 29, 2021

View reviewed changes

address comment

476b777

jojochuang approved these changes Nov 29, 2021

View reviewed changes

jojochuang merged commit b34dcb5 into apache:trunk Dec 2, 2021

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile (apac…

858cddb

…he#3722)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile #3722

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile #3722

Uh oh!

virajjasani commented Nov 25, 2021 •

edited

Loading

Uh oh!

virajjasani commented Nov 25, 2021

Uh oh!

jojochuang Nov 29, 2021

Uh oh!

jojochuang left a comment

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

jojochuang commented Dec 2, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile #3722

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile #3722

Uh oh!

Conversation

virajjasani commented Nov 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

virajjasani commented Nov 25, 2021

Uh oh!

jojochuang Nov 29, 2021

Choose a reason for hiding this comment

Uh oh!

jojochuang left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

jojochuang commented Dec 2, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

virajjasani commented Nov 25, 2021 •

edited

Loading