HADOOP-18257. Analyze S3A Audit Logs

[steveloughran]

Description of PR

HADOOP-18257. Analyze S3A Audit Logs

This is #6000 rebased and squashed with a couple of changes on top.

• tool is invoked through hadoop s3guard command
• which can now also be invoked as "hadoop s3a"! (not documented, maybe we should isolate?)
• tests are improved.
• OperationDuration implements DurationTracker for bit more completeness

TODO

• split out the record parsing into a hadoop MR record read/write
• so support large scale parsing
• cli tool just glues that together either for small parallelised extraction
  or for aggregation to one file.
• but a bulk job would work with a larger dataset

How was this patch tested?

new unit test cases.

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 21s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 18s		trunk passed
+1 💚	compile	8m 38s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	8m 2s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	2m 3s		trunk passed
+1 💚	mvnsite	14m 36s		trunk passed
+1 💚	javadoc	4m 51s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 53s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
-1 ❌	spotbugs	17m 6s	/branch-spotbugs-root-warnings.html	root in trunk has 2 extant spotbugs warnings.
+1 💚	shadedclient	36m 36s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	18m 35s		the patch passed
+1 💚	compile	8m 24s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	8m 24s		the patch passed
+1 💚	compile	8m 6s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	8m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 0s	/results-checkstyle-root.txt	root: The patch generated 7 new + 2 unchanged - 0 fixed = 9 total (was 2)
+1 💚	mvnsite	8m 41s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	javadoc	4m 45s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 54s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
-1 ❌	spotbugs	0m 46s	/new-spotbugs-hadoop-tools_hadoop-aws.html	hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌	spotbugs	17m 27s	/new-spotbugs-root.html	root generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shadedclient	37m 11s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	645m 36s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 4s		The patch does not generate ASF License warnings.
		868m 4s

Reason	Tests
SpotBugs	module:hadoop-tools/hadoop-aws
	Found reliance on default encoding in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream):in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream): new java.io.PrintWriter(OutputStream) At AuditTool.java:[line 122]
SpotBugs	module:root
	Found reliance on default encoding in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream):in org.apache.hadoop.fs.s3a.audit.AuditTool.run(String[], PrintStream): new java.io.PrintWriter(OutputStream) At AuditTool.java:[line 122]
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestDatanodeManager

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/artifact/out/Dockerfile
GITHUB PR	#6901
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 346db37e4260 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / f3e1bd5
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/testReport/
Max. process+thread count	4268 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 25s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 22s		trunk passed
+1 💚	compile	9m 4s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	8m 22s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	2m 17s		trunk passed
+1 💚	mvnsite	18m 35s		trunk passed
+1 💚	javadoc	5m 3s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	5m 0s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
-1 ❌	spotbugs	17m 35s	/branch-spotbugs-root-warnings.html	root in trunk has 2 extant spotbugs warnings.
+1 💚	shadedclient	37m 33s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	18m 53s		the patch passed
+1 💚	compile	8m 41s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	8m 41s		the patch passed
+1 💚	compile	8m 24s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	8m 24s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	2m 2s	/results-checkstyle-root.txt	root: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2)
+1 💚	mvnsite	11m 36s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	4m 54s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	5m 5s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	19m 48s		the patch passed
+1 💚	shadedclient	37m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	639m 13s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		867m 41s

Reason	Tests
Failed junit tests	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/artifact/out/Dockerfile
GITHUB PR	#6901
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 913d0845eac3 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / fe0322a
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/testReport/
Max. process+thread count	4010 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 24s		trunk passed
+1 💚	compile	10m 1s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	9m 13s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	2m 25s		trunk passed
+1 💚	mvnsite	16m 47s		trunk passed
+1 💚	javadoc	5m 19s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	5m 3s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
-1 ❌	spotbugs	18m 8s	/branch-spotbugs-root-warnings.html	root in trunk has 2 extant spotbugs warnings.
+1 💚	shadedclient	40m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	21m 50s		the patch passed
+1 💚	compile	9m 56s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	9m 56s		the patch passed
+1 💚	compile	9m 27s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	9m 27s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	2m 15s	/results-checkstyle-root.txt	root: The patch generated 8 new + 2 unchanged - 0 fixed = 10 total (was 2)
+1 💚	mvnsite	9m 49s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	4m 24s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
-1 ❌	javadoc	4m 33s	/patch-javadoc-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt	root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.
-1 ❌	spotbugs	0m 49s	/new-spotbugs-hadoop-tools_hadoop-aws.html	hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌	spotbugs	17m 55s	/new-spotbugs-root.html	root generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shadedclient	37m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	639m 16s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 8s		The patch does not generate ASF License warnings.
		875m 29s

Reason	Tests
SpotBugs	module:hadoop-tools/hadoop-aws
	org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76]
SpotBugs	module:root
	org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76]
Failed junit tests	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
	hadoop.fs.s3a.audit.TestAuditTool

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/artifact/out/Dockerfile
GITHUB PR	#6901
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 48468eb1eff9 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7303563
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/testReport/
Max. process+thread count	4787 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 34s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 19s		trunk passed
+1 💚	compile	8m 57s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	compile	8m 22s		trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚	checkstyle	2m 4s		trunk passed
+1 💚	mvnsite	14m 58s		trunk passed
+1 💚	javadoc	5m 20s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	4m 58s		trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚	spotbugs	19m 17s		trunk passed
+1 💚	shadedclient	37m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 14s	/patch-mvninstall-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
-1 ❌	mvninstall	18m 5s	/patch-mvninstall-root.txt	root in the patch failed.
-1 ❌	compile	8m 23s	/patch-compile-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.
-1 ❌	javac	8m 23s	/patch-compile-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.
-1 ❌	compile	8m 10s	/patch-compile-root-jdkPrivateBuild-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.txt	root in the patch failed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.
-1 ❌	javac	8m 10s	/patch-compile-root-jdkPrivateBuild-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.txt	root in the patch failed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 31s	/results-checkstyle-root.txt	root: The patch generated 6 new + 2 unchanged - 0 fixed = 8 total (was 2)
-1 ❌	mvnsite	2m 56s	/patch-mvnsite-root.txt	root in the patch failed.
+1 💚	shellcheck	0m 0s		No new issues.
-1 ❌	javadoc	4m 55s	/patch-javadoc-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.
-1 ❌	javadoc	4m 33s	/patch-javadoc-root-jdkPrivateBuild-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.txt	root in the patch failed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.
-1 ❌	spotbugs	0m 21s	/patch-spotbugs-hadoop-tools_hadoop-aws.txt	hadoop-aws in the patch failed.
-1 ❌	spotbugs	15m 50s	/patch-spotbugs-root.txt	root in the patch failed.
+1 💚	shadedclient	21m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	647m 48s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	1m 6s		The patch does not generate ASF License warnings.
		860m 43s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.TestDecommission

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/artifact/out/Dockerfile
GITHUB PR	#6901
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs
uname	Linux a5a51b1cb4b9 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / fb2a10e
Default Java	Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/testReport/
Max. process+thread count	4350 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 38s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 40s		trunk passed
+1 💚	compile	10m 36s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	compile	8m 49s		trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚	checkstyle	2m 20s		trunk passed
+1 💚	mvnsite	14m 54s		trunk passed
+1 💚	javadoc	5m 24s		trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	4m 53s		trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚	spotbugs	19m 18s		trunk passed
+1 💚	shadedclient	37m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	18m 46s		the patch passed
+1 💚	compile	8m 43s		the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚	javac	8m 43s		the patch passed
+1 💚	compile	8m 30s		the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚	javac	8m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	2m 4s	/results-checkstyle-root.txt	root: The patch generated 6 new + 2 unchanged - 0 fixed = 8 total (was 2)
+1 💚	mvnsite	9m 58s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
-1 ❌	javadoc	4m 48s	/patch-javadoc-root-jdkUbuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04.
-1 ❌	javadoc	4m 32s	/patch-javadoc-root-jdkPrivateBuild-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.txt	root in the patch failed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga.
-1 ❌	spotbugs	0m 49s	/new-spotbugs-hadoop-tools_hadoop-aws.html	hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌	spotbugs	17m 32s	/new-spotbugs-root.html	root generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	37m 33s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	647m 39s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	0m 52s		The patch does not generate ASF License warnings.
		871m 52s

Reason	Tests
SpotBugs	module:hadoop-tools/hadoop-aws
	org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76]
SpotBugs	module:root
	org.apache.hadoop.fs.s3a.audit.mapreduce.S3AAuditLogMergerAndParser.FIELDS_OF_TYPE_LONG is a mutable collection which should be package protected At S3AAuditLogMergerAndParser.java:which should be package protected At S3AAuditLogMergerAndParser.java:[line 76]
Failed junit tests	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.TestDecommission
	hadoop.fs.s3a.audit.TestAuditTool

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/artifact/out/Dockerfile
GITHUB PR	#6901
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint shellcheck shelldocs
uname	Linux 81b7addc1e9b 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e012694
Default Java	Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/testReport/
Max. process+thread count	4598 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6901/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.