HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 #7524

stoty · 2025-03-20T06:35:19Z

Description of PR

JIRA: HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3.

Remove unused commons-beanutils dependency

How was this patch tested?

CI

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-03-20T08:36:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 10s		trunk passed
+1 💚	compile	8m 21s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	7m 20s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	1m 22s		trunk passed
+1 💚	javadoc	1m 13s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 59s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	64m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 39s		the patch passed
+1 💚	compile	7m 59s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	7m 59s		the patch passed
+1 💚	compile	7m 9s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	7m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	1m 25s		the patch passed
+1 💚	javadoc	1m 5s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 58s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	24m 26s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 20s		hadoop-project in the patch passed.
+1 💚	unit	12m 24s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		119m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/1/artifact/out/Dockerfile
GITHUB PR	#7524
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux e8529aaf0604 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `11dd6f0`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/1/testReport/
Max. process+thread count	3105 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-03-20T19:48:47Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 32s		Maven dependency ordering for branch
+1 💚	mvninstall	18m 48s		trunk passed
+1 💚	compile	8m 15s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	7m 19s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	1m 19s		trunk passed
+1 💚	javadoc	1m 13s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 56s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	64m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 41s		the patch passed
+1 💚	compile	8m 3s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	8m 3s		the patch passed
+1 💚	compile	7m 16s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	7m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	1m 23s		the patch passed
+1 💚	javadoc	1m 6s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 0s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	25m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 22s		hadoop-project in the patch passed.
+1 💚	unit	12m 22s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		121m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/2/artifact/out/Dockerfile
GITHUB PR	#7524
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux e1184219d589 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `53437e8`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/2/testReport/
Max. process+thread count	3151 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7524/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

pan3793 · 2025-03-21T07:28:58Z

update LICENSE too?

stoty · 2025-03-21T09:40:49Z

update LICENSE too?

While the current hadoop-aluyin module no longer has this transitive dependecy, hadoop-yarn-server-timelineservice-documentstore still depends on it transitively via azure-cosmosdb-commons.

My understanding is that LICENSE-binary is supposed to include all transitive dependencies, @pan3793 , but please correct me if I'm wrong, and I will remove it.

pan3793 · 2025-03-21T09:50:02Z

... LICENSE-binary is supposed to include all transitive dependencies

strictly speaking, the LICENSE shipped in artifacts(either jar or tarball) should exactly match the content, so keeping or removing depends on whether Hadoop binary distribution contains it.

stoty · 2025-03-21T11:44:08Z

I've just built the assembly, and it's there, @pan3793 .
So no need to touch LICENSE-binary.

slfan1989 · 2025-03-24T00:52:15Z

@stoty Thanks for the contribution! @pan3793 Thanks for the review!

…e#7524) Contributed by Istvan Toth. Reviewed-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

#7778) Contributed by Istvan Toth. Reviewed-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3

11dd6f0

github-actions bot added build trunk Common labels Mar 20, 2025

also remove exclusion from aliyun-sdk-oss

53437e8

slfan1989 approved these changes Mar 20, 2025

View reviewed changes

pan3793 approved these changes Mar 21, 2025

View reviewed changes

slfan1989 merged commit 2af914b into apache:trunk Mar 24, 2025
1 of 3 checks passed

jamie-albert mentioned this pull request Jun 25, 2025

spark-4.0: add pending-upstream-fix advisories for transitive dependency CVEs wolfi-dev/advisories#20540

Merged

steveloughran mentioned this pull request Jul 4, 2025

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 (#7524) #7778

Merged

3 tasks

steveloughran added a commit that referenced this pull request Jul 4, 2025

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 (#7524) (

d9f8d13

#7778) Contributed by Istvan Toth. Reviewed-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

steveloughran added a commit that referenced this pull request Jul 4, 2025

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 (#7524) (

276526d

#7778) Contributed by Istvan Toth. Reviewed-by: Cheng Pan <chengpan@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 #7524

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 #7524

Uh oh!

stoty commented Mar 20, 2025 •

edited by slfan1989

Loading

Uh oh!

hadoop-yetus commented Mar 20, 2025

Uh oh!

hadoop-yetus commented Mar 20, 2025

Uh oh!

pan3793 commented Mar 21, 2025

Uh oh!

stoty commented Mar 21, 2025

Uh oh!

pan3793 commented Mar 21, 2025

Uh oh!

stoty commented Mar 21, 2025

Uh oh!

slfan1989 commented Mar 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 #7524

HADOOP-18991 Remove commons-beanutils dependency from Hadoop 3 #7524

Uh oh!

Conversation

stoty commented Mar 20, 2025 • edited by slfan1989 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Mar 20, 2025

Uh oh!

hadoop-yetus commented Mar 20, 2025

Uh oh!

pan3793 commented Mar 21, 2025

Uh oh!

stoty commented Mar 21, 2025

Uh oh!

pan3793 commented Mar 21, 2025

Uh oh!

stoty commented Mar 21, 2025

Uh oh!

slfan1989 commented Mar 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

stoty commented Mar 20, 2025 •

edited by slfan1989

Loading