Fail with "duplicate dependency" since 4.0.0-rc3

[pan3793]

Affected version

4.0.0-rc3, 4.0.0-rc4

Bug description

I face the issue when trying to upgrade Maven 4 for Spark apache/spark#51230

Maven 4.0.0-rc-2 works, but it starts to fail since RC3 with

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:3.1.4:install (default-install) on project spark-tags_2.13: Execution default-install of goal org.apache.maven.plugins:maven-install-plugin:3.1.4:install failed: Duplicate dependency: Dependency {groupId=org.scalatest, artifactId=scalatest_2.13, version=3.2.19, type=jar} -> [Help 1]

Looks like the issue is related to maven-shade-plugin, I created a minimal project to reproduce it, mvn clean install works with 4.0.0-rc-2 but fails with rc-3 and the rc-4

https://github.com/pan3793/SPARK-51224

[Bukama]

Maven 4 (not sure since which RC) has a more strict validation of dependency and plugin configuration (which "only" resulted in warnings with Maven 3) so this is intended in general.

But @cstamas could add more details and maybe an site effect when using shade-plugin?

[cstamas]

I cannot reproduce with reproducer, I have "all good":

[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Reactor Summary for SPARK-51224 1-SNAPSHOT:
[INFO] 
[INFO] SPARK-51224 .......................................................................................... SUCCESS [  0.440 s]
[INFO] module-a ............................................................................................. SUCCESS [  0.718 s]
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] --------------------------------------------------------------------------------------------------------------------------
[INFO] Total time:  1.398 s
[INFO] Finished at: 2025-07-01T22:59:18+02:00
[INFO] --------------------------------------------------------------------------------------------------------------------------
[cstamas@infinity SPARK-51224 (master)]$ 

Please provide information how to reproduce.

EDIT: mvn install does it

[cstamas]

Created IT apache/maven-shade-plugin#740
@gnodet also, this is related to POM transformation, with -Dmaven.maven3Personality reproducer does not reproduce. This has to be related that a rogue dep enters the stage (via wrongly dep reduced POM)

[pan3793]

@cstamas, thanks for your help diagnosing this issue, I have a new discovery that may narrow down the problem
pan3793/SPARK-51224@1bee7f4