HDDS-1884. Support Bucket ACL operations for OM HA. #1202

bharatviswa504 · 2019-08-01T00:58:39Z

In this PR implemented add/set/remove ACL for the bucket.

xiaoyuyao

LGTM overall. A few comments added inline.

hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHA.java

...-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/util/ObjectParser.java

...nager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/acl/OMBucketAddAclRequest.java

bharatviswa504 · 2019-08-02T00:45:21Z

Now it is ready for review.

xiaoyuyao

Thanks @bharatviswa504 for the patch. It LGTM overall. A few comments added inline.

hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketInfo.java

hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/BiFunction.java

hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOzoneManagerHA.java

bharatviswa504 · 2019-08-07T18:20:31Z

Thank You @xiaoyuyao for the review.
I have addressed review comments.

xiaoyuyao · 2019-08-08T17:03:42Z

@bharatviswa504 The latest update LGTM. Let's rebase the PR to get a clean CI.

bharatviswa504 · 2019-08-08T17:07:13Z

/retest

bharatviswa504 · 2019-08-08T17:24:54Z

Thank You @xiaoyuyao for the review.
Rebased the PR with the latest trunk.

bharatviswa504 · 2019-08-08T17:27:11Z

/retest

bharatviswa504 · 2019-08-08T22:29:49Z

/retest

bharatviswa504 · 2019-08-08T23:55:30Z

/retest

hadoop-yetus · 2019-08-09T00:44:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	43	Docker mode activated.
		_ Prechecks _
+1	dupname	1	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	15	Maven dependency ordering for branch
+1	mvninstall	592	trunk passed
+1	compile	374	trunk passed
+1	checkstyle	75	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	878	branch has no errors when building and testing our client artifacts.
+1	javadoc	162	trunk passed
0	spotbugs	420	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	618	trunk passed
-0	patch	472	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
0	mvndep	22	Maven dependency ordering for patch
+1	mvninstall	545	the patch passed
+1	compile	381	the patch passed
+1	cc	381	the patch passed
+1	javac	381	the patch passed
-0	checkstyle	43	hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	mvnsite	0	the patch passed
+1	whitespace	1	The patch has no whitespace issues.
+1	shadedclient	681	patch has no errors when building and testing our client artifacts.
+1	javadoc	164	the patch passed
+1	findbugs	636	the patch passed
		_ Other Tests _
+1	unit	324	hadoop-hdds in the patch passed.
-1	unit	2226	hadoop-ozone in the patch failed.
+1	asflicense	40	The patch does not generate ASF License warnings.
		8037

Reason	Tests
Failed junit tests	hadoop.ozone.om.TestKeyManagerImpl
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
	hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures
	hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.om.TestScmSafeMode
	hadoop.ozone.om.TestOzoneManagerHA
	hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures
	hadoop.ozone.client.rpc.TestSecureOzoneRpcClient

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/19/artifact/out/Dockerfile
GITHUB PR	#1202
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname	Linux b985ae6f5a93 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6ad9a11`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/19/artifact/out/diff-checkstyle-hadoop-ozone.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/19/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/19/testReport/
Max. process+thread count	4685 (vs. ulimit of 5500)
modules	C: hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/19/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-08-09T00:54:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	121	Docker mode activated.
		_ Prechecks _
+1	dupname	1	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 9 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	12	Maven dependency ordering for branch
+1	mvninstall	618	trunk passed
+1	compile	368	trunk passed
+1	checkstyle	67	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	855	branch has no errors when building and testing our client artifacts.
+1	javadoc	161	trunk passed
0	spotbugs	443	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	645	trunk passed
-0	patch	481	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
0	mvndep	18	Maven dependency ordering for patch
+1	mvninstall	573	the patch passed
+1	compile	368	the patch passed
+1	cc	368	the patch passed
+1	javac	368	the patch passed
+1	checkstyle	68	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	692	patch has no errors when building and testing our client artifacts.
+1	javadoc	153	the patch passed
-1	findbugs	442	hadoop-ozone generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
		_ Other Tests _
+1	unit	403	hadoop-hdds in the patch passed.
-1	unit	2925	hadoop-ozone in the patch failed.
+1	asflicense	45	The patch does not generate ASF License warnings.
		8913

Reason	Tests
FindBugs	module:hadoop-ozone
	Unread field:OzoneBucket.java:[line 145]
	Unwritten field:BucketArgs.java:[line 88]
Failed junit tests	hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures
	hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
	hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory
	hadoop.ozone.client.rpc.TestCommitWatcher
	hadoop.ozone.om.TestOzoneManagerHA
	hadoop.ozone.TestMiniOzoneCluster
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.om.TestScmSafeMode
	hadoop.ozone.client.rpc.TestContainerStateMachine
	hadoop.ozone.client.rpc.TestReadRetries
	hadoop.ozone.om.TestKeyManagerImpl

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/18/artifact/out/Dockerfile
GITHUB PR	#1202
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname	Linux 9eca47fd4302 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6ad9a11`
Default Java	1.8.0_222
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/18/artifact/out/new-findbugs-hadoop-ozone.html
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/18/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/18/testReport/
Max. process+thread count	3946 (vs. ulimit of 5500)
modules	C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/objectstore-service hadoop-ozone/integration-test U: hadoop-ozone
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/18/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-08-09T01:48:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	44	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
0	mvndep	14	Maven dependency ordering for branch
+1	mvninstall	581	trunk passed
+1	compile	367	trunk passed
+1	checkstyle	76	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	879	branch has no errors when building and testing our client artifacts.
+1	javadoc	188	trunk passed
0	spotbugs	422	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	624	trunk passed
-0	patch	464	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
0	mvndep	20	Maven dependency ordering for patch
+1	mvninstall	545	the patch passed
+1	compile	377	the patch passed
+1	cc	377	the patch passed
+1	javac	377	the patch passed
+1	checkstyle	74	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	633	patch has no errors when building and testing our client artifacts.
+1	javadoc	148	the patch passed
+1	findbugs	612	the patch passed
		_ Other Tests _
+1	unit	293	hadoop-hdds in the patch passed.
-1	unit	2016	hadoop-ozone in the patch failed.
+1	asflicense	48	The patch does not generate ASF License warnings.
		7690

Reason	Tests
Failed junit tests	hadoop.ozone.om.TestKeyManagerImpl
	hadoop.ozone.TestStorageContainerManager
	hadoop.ozone.om.TestScmSafeMode
	hadoop.ozone.client.rpc.TestBCSID

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/20/artifact/out/Dockerfile
GITHUB PR	#1202
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc
uname	Linux 91af5857cbb7 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `aa5f445`
Default Java	1.8.0_212
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/20/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/20/testReport/
Max. process+thread count	4953 (vs. ulimit of 5500)
modules	C: hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1202/20/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

bharatviswa504 · 2019-08-09T04:28:43Z

Test failures are not related to this PR.
Thank You @xiaoyuyao for the review.
I will commit this to the trunk.

bharatviswa504 requested review from xiaoyuyao and arp7 August 1, 2019 00:58

bharatviswa504 force-pushed the HDDS-1884 branch from 5f544ef to baa72a6 Compare August 1, 2019 01:02

bharatviswa504 added the ozone label Aug 1, 2019

bharatviswa504 self-assigned this Aug 1, 2019

bharatviswa504 force-pushed the HDDS-1884 branch from baa72a6 to 120aae5 Compare August 1, 2019 01:11

This comment has been minimized.

Sign in to view

xiaoyuyao reviewed Aug 1, 2019

View reviewed changes

This comment has been minimized.

Sign in to view

bharatviswa504 changed the title ~~HDDS-1884. Support Bucket addACL operations for OM HA.~~ HDDS-1884. Support Bucket ACL operations for OM HA. Aug 2, 2019

This comment has been minimized.

Sign in to view

apache deleted a comment from hadoop-yetus Aug 2, 2019

bharatviswa504 added 10 commits August 5, 2019 16:10

HDDS-1884. Support Bucket addACL operations for OM HA.

44fc1dd

fix find bug.

b84b7a4

HDDS-1884. Support Bucket ACL operations for OM HA.

3f85616

fix checkstyle.

82fcd1f

unneeded change.

094ad24

add new line at end of file.

e67922a

move BiFunction to ozone common.

8a4ccdd

fix check in OmBucketAclResponse.

b5b93bd

fix onSuccess in addAcl.

f27f196

minor changes.

dbddced

bharatviswa504 force-pushed the HDDS-1884 branch from 4effbff to dbddced Compare August 5, 2019 23:10

This comment has been minimized.

Sign in to view

xiaoyuyao reviewed Aug 7, 2019

View reviewed changes

bharatviswa504 added 2 commits August 7, 2019 11:14

address comments.

d48e48a

fix minor comment.

8c206eb

Merge branch 'trunk' into HDDS-1884

3612f59

fix issue caused by HDDS-1619.

0e7273c

bharatviswa504 force-pushed the HDDS-1884 branch from 3b92faa to 0e7273c Compare August 8, 2019 22:29

apache deleted a comment from hadoop-yetus Aug 8, 2019

fix checkstyle

3893fdd

bharatviswa504 merged commit 91f41b7 into apache:trunk Aug 9, 2019

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019

HDDS-1884. Support Bucket ACL operations for OM HA. (apache#1202)

ad1630d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDDS-1884. Support Bucket ACL operations for OM HA. #1202

HDDS-1884. Support Bucket ACL operations for OM HA. #1202

bharatviswa504 commented Aug 1, 2019 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

xiaoyuyao left a comment

This comment has been minimized.

bharatviswa504 commented Aug 2, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

xiaoyuyao left a comment

bharatviswa504 commented Aug 7, 2019

xiaoyuyao commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

hadoop-yetus commented Aug 9, 2019

hadoop-yetus commented Aug 9, 2019

hadoop-yetus commented Aug 9, 2019

bharatviswa504 commented Aug 9, 2019

HDDS-1884. Support Bucket ACL operations for OM HA. #1202

HDDS-1884. Support Bucket ACL operations for OM HA. #1202

Conversation

bharatviswa504 commented Aug 1, 2019 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

xiaoyuyao left a comment

Choose a reason for hiding this comment

This comment has been minimized.

bharatviswa504 commented Aug 2, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

xiaoyuyao left a comment

Choose a reason for hiding this comment

bharatviswa504 commented Aug 7, 2019

xiaoyuyao commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

bharatviswa504 commented Aug 8, 2019

hadoop-yetus commented Aug 9, 2019

hadoop-yetus commented Aug 9, 2019

hadoop-yetus commented Aug 9, 2019

bharatviswa504 commented Aug 9, 2019

bharatviswa504 commented Aug 1, 2019 •

edited

Loading