HBASE-26985 check permission for SecureBulkLoadManager #4379

skysiders · 2022-04-28T04:07:39Z

HBASE-26985 check permission for SecureBulkLoadManager

SecureBulkLoadManager will create baseStagingDir if not exist. start method use
fs.mkdirs(baseStagingDir, PERM_HIDDEN); to create directory with permission -rwx–x–x.BUT if umask is too strict such as 077 ,this directory will create with 0700 so it too strict for GROUP and OTHER user to own execute permission.In this modification, we check whether permissions are affected by umask, and if so, correct permissions to -rwx–x–x

Apache-HBase · 2022-04-28T04:34:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 3s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 21s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	3m 39s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
-1 ❌	mvninstall	1m 12s	root in the patch failed.
-1 ❌	compile	0m 26s	hbase-server in the patch failed.
-0 ⚠️	javac	0m 26s	hbase-server in the patch failed.
-1 ❌	shadedjars	2m 39s	patch has 16 errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	0m 27s	hbase-server in the patch failed.
		13m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 89463bdc7082 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `90d42f2`
Default Java	AdoptOpenJDK-1.8.0_282-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/testReport/
Max. process+thread count	60 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-28T04:34:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 21s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 50s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	3m 41s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
-1 ❌	mvninstall	1m 25s	root in the patch failed.
-1 ❌	compile	0m 29s	hbase-server in the patch failed.
-0 ⚠️	javac	0m 29s	hbase-server in the patch failed.
-1 ❌	shadedjars	2m 41s	patch has 16 errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	0m 30s	hbase-server in the patch failed.
		14m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c64109f955f8 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `90d42f2`
Default Java	AdoptOpenJDK-11.0.10+9
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/testReport/
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-28T04:35:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 40s	master passed
+1 💚	compile	2m 12s	master passed
+1 💚	checkstyle	0m 36s	master passed
+1 💚	spotbugs	1m 14s	master passed
		_ Patch Compile Tests _
-1 ❌	mvninstall	1m 4s	root in the patch failed.
-1 ❌	compile	0m 26s	hbase-server in the patch failed.
-0 ⚠️	javac	0m 26s	hbase-server in the patch failed.
+1 💚	checkstyle	0m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
-1 ❌	hadoopcheck	1m 8s	The patch causes 16 errors with Hadoop v3.1.2.
-1 ❌	hadoopcheck	2m 21s	The patch causes 16 errors with Hadoop v3.2.2.
-1 ❌	hadoopcheck	3m 33s	The patch causes 16 errors with Hadoop v3.3.1.
-1 ❌	spotbugs	0m 22s	hbase-server in the patch failed.
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		15m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 1c7935df5e8c 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `90d42f2`
Default Java	AdoptOpenJDK-1.8.0_282-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-javac-3.2.2.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-javac-3.3.1.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-30T04:08:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 54s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 23s	master passed
+1 💚	compile	2m 13s	master passed
+1 💚	checkstyle	0m 35s	master passed
+1 💚	spotbugs	1m 15s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 1s	the patch passed
+1 💚	compile	2m 9s	the patch passed
+1 💚	javac	2m 9s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 55s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	1m 27s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		33m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 0a511a64637e 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f5a566e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-30T07:01:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 49s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	3m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	186m 6s	hbase-server in the patch passed.
		206m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cab3365b0c59 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f5a566e`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/testReport/
Max. process+thread count	2616 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-30T07:03:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 24s	master passed
+1 💚	compile	0m 38s	master passed
+1 💚	shadedjars	3m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 17s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	3m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	191m 32s	hbase-server in the patch passed.
		208m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4379
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1cb097cfa56a 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b0c2832`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/testReport/
Max. process+thread count	2835 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4379/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

skysiders · 2022-05-01T04:43:32Z

Hi @maoling ,can you please take a look?

skysiders · 2022-05-06T09:16:29Z

Hi @virajjasani ,could you please have a look this?

skysiders · 2022-05-08T08:21:02Z

Hi @virajjasani , this patch is similar as #4391 , and I patch it with check permission is set as we expected, if not equal we need to set it with setPermission.Could you please have a look at this please?

Apache9 · 2022-06-02T13:20:29Z

@virajjasani Can we merge this?

Signed-off-by: Viraj Jasani <vjasani@apache.org>

…e#4379)" This reverts commit b43a0ea.

Signed-off-by: Viraj Jasani <vjasani@apache.org> Change-Id: I28b4b118a7b2aeeb942dcf73a1eb443c0b57337a

check permission for SecureBulkLoadManager

968a0d5

fix some mistake

447d42f

skysiders mentioned this pull request May 6, 2022

HBASE-26994 MasterFileSystem create directory without permission check #4391

Merged

virajjasani approved these changes May 8, 2022

View reviewed changes

virajjasani merged commit c0e8243 into apache:master Jun 2, 2022

virajjasani pushed a commit that referenced this pull request Jun 2, 2022

HBASE-26985 check permission for SecureBulkLoadManager (#4379)

286c9c3

Signed-off-by: Viraj Jasani <vjasani@apache.org>

virajjasani pushed a commit that referenced this pull request Jun 2, 2022

HBASE-26985 check permission for SecureBulkLoadManager (#4379)

0c5d660

Signed-off-by: Viraj Jasani <vjasani@apache.org>

virajjasani pushed a commit that referenced this pull request Jun 2, 2022

HBASE-26985 check permission for SecureBulkLoadManager (#4379)

2e08c69

Signed-off-by: Viraj Jasani <vjasani@apache.org>

wenwj0 pushed a commit to wenwj0/hbase that referenced this pull request Jun 14, 2022

HBASE-26985 check permission for SecureBulkLoadManager (apache#4379)

b43a0ea

Signed-off-by: Viraj Jasani <vjasani@apache.org>

wenwj0 added a commit to wenwj0/hbase that referenced this pull request Jun 14, 2022

Revert "HBASE-26985 check permission for SecureBulkLoadManager (apach…

ce1407f

…e#4379)" This reverts commit b43a0ea.

vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024

HBASE-26985 check permission for SecureBulkLoadManager (apache#4379)

56c1fd7

Signed-off-by: Viraj Jasani <vjasani@apache.org> Change-Id: I28b4b118a7b2aeeb942dcf73a1eb443c0b57337a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26985 check permission for SecureBulkLoadManager #4379

HBASE-26985 check permission for SecureBulkLoadManager #4379

skysiders commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 30, 2022

Apache-HBase commented Apr 30, 2022

Apache-HBase commented Apr 30, 2022

skysiders commented May 1, 2022

skysiders commented May 6, 2022

skysiders commented May 8, 2022

Apache9 commented Jun 2, 2022

HBASE-26985 check permission for SecureBulkLoadManager #4379

HBASE-26985 check permission for SecureBulkLoadManager #4379

Conversation

skysiders commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 28, 2022

Apache-HBase commented Apr 30, 2022

Apache-HBase commented Apr 30, 2022

Apache-HBase commented Apr 30, 2022

skysiders commented May 1, 2022

skysiders commented May 6, 2022

skysiders commented May 8, 2022

Apache9 commented Jun 2, 2022