[MSHADE-36] Use reduced dependencies #25

nielsbasjes · 2019-08-27T09:43:20Z

This is a proposed fix for the issue https://issues.apache.org/jira/browse/MSHADE-36 which has been open for over 11 years.

The implementation is activated when the setting "useDependencyReducedPomInJar" is set to true.
It then essentially injects two ResourceTransformers from code

Remove the original "META-INF/maven/${project.groupId}/${project.artifactId}/pom.xml".
Include the dependency-reduced-pom.xml under the name of the original pom.xml into the final jar file.

This includes an integration test to verify this actually happens.

Following this checklist to help us incorporate your
contribution quickly and easily:

Make sure there is a JIRA issue filed
for the change (usually before you start working on it).
https://issues.apache.org/jira/browse/MSHADE-36
Each commit in the pull request should have a meaningful subject line and body.
Format the pull request title like [MSHADE-XXX] - Fixes bug in ApproximateQuantiles,
where you replace MSHADE-XXX with the appropriate JIRA issue. Best practice
is to use the JIRA issue title in the pull request title and in the first line of the
commit message.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Run mvn clean verify to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
You have run the integration tests successfully (mvn -Prun-its clean verify).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004
In any other case, please file an Apache Individual Contributor License Agreement. I'm a committer and PMC for Apache Avro.

nielsbasjes · 2019-08-27T11:00:26Z

I noticed that there will be a conflict with #22 because I moved the creation of the dpr to a slightly different place (more forward instead of more backward).

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/verify.bsh

struberg · 2019-12-17T20:28:15Z

What is the status of this ticket? @Tibor17 was there something missing still?

rfscholte · 2019-12-17T21:11:53Z

I talked some time ago with @hboutemy about shading and poms. Which is the right pom? Think about licenses, etc. I can image we need a shaded scope, which keeps all original information and respects the shading-aspects. But that is another topic.

elharo

Is this still under consideration/development or should we close it?

nielsbasjes · 2020-04-10T22:25:10Z

@elharo I would like to see this problem fixed. If there is anything I need to change then I'm happy to do that. Just say what the desired solution is.

elharo · 2020-04-11T00:03:22Z

Looks like for this issue to move forward it needs a proposal (not a PR) that can be discussed on the dev mailing list. If you can get consensus from the project maintainers on what a fix should look like--i.e. what would go into a project POM and what the output would be--then it could be implemented. But right now it doesn't look like there's consensus on the form of the solution.

felfert · 2021-03-17T21:28:26Z

Looks like for this issue to move forward it needs a proposal (not a PR) that can be discussed on the dev mailing list.

Perhaps, there could be a pragmatic solution to this: Instead of directly using the dependency-reduced pom directly, there
could be an option to use a user-supplied pom instead. Then the user could copy the generated pom, edit it (adding licence info etc) and then in a second run, that pom could be used in the jar?

Snipx · 2021-04-29T06:53:08Z

Dear Maintainers/Contributors,
I think this is a terribly useful feature - is there any chance it can be given a bit more priority?

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/invoker.properties

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/pom.xml

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/src/main/java/nl/example/Main.java

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java

michael-o · 2021-06-13T12:14:10Z

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java

@@ -437,6 +447,24 @@ public void execute()

            List<ResourceTransformer> resourceTransformers = getResourceTransformers();

+            if ( createDependencyReducedPom )


You have moved this before the Shader works. No changes in behavior?

With the tests present in the project today 2 fail with my changes in place.
So this does indeed need some rework.

Apparently the problem that failed the tests made two files.
Fix that now all tests pass.

Still the question "No changes in behavior?" remains a valid one.

For now: Let's assume there IS a change the current tests did not pick up.
I'll look into it.

My verification steps:

All integration tests pass.

Checkout both master and my version into separate directories.

Run in both: MAVEN_OPTS=-Dinvoker.parallelThreads=2 mvn clean verify -Prun-its under javac 11.0.11

Get this tool to compare the jars https://github.com/scala/jardiff

Run these magical incantations on my Linux machine:
find target/it -type f -name '*.jar' | while read JAR ; do java -jar ../jardiff.jar ${JAR} ../maven-shade-plugin-master/${JAR} ; done > jar-changes.log
find target/it -type f -name '*pom*' | while read POM ; do diff ${POM} ../maven-shade-plugin-master/${POM} ; done > pom-changes.log

Manually go through the output.
Differences logged:

Files like surefirebooter2116138492290729396.jar are unique (Ignore)

The test files I created are only present in mine (Ignore)

All files have a difference in the timestamps when the files were generated (pom.properties) (Ignore)

Some tests include the plugin itself and show differences because of the new feature (Ignore)

@michael-o Is this sufficient verification?

Will verify!

src/main/java/org/apache/maven/plugins/shade/resource/UseDependencyReducedPomXml.java

michael-o · 2021-06-13T12:16:37Z

I am inclined to merge this after a few small changes since this is optional and does not affect default behavior. Respecing licenses and other attributions is solely a dev's task. Not ours.

michael-o · 2021-06-13T12:18:38Z

@nielsbasjes Please also rebase on top of master to resolve merge conflicts.

felfert · 2021-06-13T12:49:24Z

Wow. Finally some progress.
@michael-o Please would you be so kind to have a look at #26 as well? Those two PR kind of go hand-in-hand and finally would make shading usable in multi-module projects. Thanks.

michael-o · 2021-06-13T17:59:19Z

@felfert As far as I can see #26 requires this PR, doesn't it? If so, I would like to complete this one and have a look at the other. I need to understand the problem first.

felfert · 2021-06-13T18:43:17Z

@felfert As far as I can see #26 requires this PR, doesn't it? If so, I would like to complete this one and have a look at the other. I need to understand the problem first.

No, as far as I interpret the comments in #26 (See the very first line in the initial text), #26 includes #25. Also, if you click on the commits tab of #26 you see TWO commits, where one is identical to this PR.
As for understanding the problem, maybe a simple use case shows the dilemma:

Have a look at https://github.com/jenkinsci/jclouds-plugin (A plugin which uses jclouds which in turn uses guava) and
https://github.com/felfert/jclouds-shaded (A shaded version of jclouds, where the guava dependency is shaded, because jenkins itself uses a different guava version). The latter exists for the sole purpose of shading.
Normally, I never would publish a separate project only to facilitate shading. Instead, I would make the shaded jclouds a submodule of the plugin. However maven currently forces me to do this, because even with a workaround for #25, maven still uses the unshaded pom during reactor build.

Also, maybe have a look at @nielsbasjes remarks here: https://yauaa.basjes.nl/NOTES-shading-dependencies.html

Cheers
-Fritz

michael-o · 2021-06-13T19:39:32Z

@felfert As far as I can see #26 requires this PR, doesn't it? If so, I would like to complete this one and have a look at the other. I need to understand the problem first.

No, as far as I interpret the comments in #26 (See the very first line in the initial text), #26 includes #25. Also, if you click on the commits tab of #26 you see TWO commits, where one is identical to this PR.

This is a contradiction, isn't it? If #26 includes this PR then it requires it. I will first resolve this one waiting for @nielsbasjes then continue with the rest.

nielsbasjes · 2021-06-15T19:25:01Z

I'm picking up the reviews and comments.
It has been a while and several things have changed.

michael-o · 2021-06-15T19:29:59Z

I'm picking up the reviews and comments.
It has been a while and several things have changed.

Thanks, looking forward too. If all goes well we'll have a release this month.

nielsbasjes · 2021-06-16T09:06:57Z

Regarding #26: That one indeed requires this fix to be in place first.
Once this one (#25) has been merged I'll update #26 for a renewed review round.

michael-o · 2021-06-16T09:22:01Z

@nielsbasjes Something is wrong, GitHub shows 97 commits to be merged. Did you rebase incorrectly?

nielsbasjes · 2021-06-16T09:39:06Z

@nielsbasjes Something is wrong, GitHub shows 97 commits to be merged. Did you rebase incorrectly?

Yes, something went wrong here.
I'm fixing it.

… the final shaded jar

michael-o · 2021-06-16T09:57:10Z

@nielsbasjes Let me explicitly know when you are done from your PoV and I will review and test everything.

nielsbasjes · 2021-06-16T09:57:29Z

@michael-o I have fixed it. CI builds all pass now.
Pushing a squased version for cleaner merging.

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/invoker.properties

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/pom.xml

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/src/main/java/nl/example/Main.java

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java

michael-o · 2021-06-16T12:53:35Z

Going through now...

michael-o

A few issues

src/it/projects/MSHADE-36-inject-dep-reduced-pom-in-final/verify.bsh

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java

src/main/java/org/apache/maven/plugins/shade/resource/UseDependencyReducedPom.java

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java

michael-o · 2021-06-17T19:18:03Z

Works for me on:

$ mvn -v
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T20:33:14+02:00)
Maven home: /usr/local/apache-maven-3.5.4
Java version: 1.8.0_292, vendor: OpenJDK BSD Porting Team, runtime: /usr/local/openjdk8/jre
Default locale: de_DE, platform encoding: UTF-8
OS name: "freebsd", version: "12.2-stable", arch: "amd64", family: "unix"

Waiting for ASF Jenkins to complete

michael-o · 2021-06-17T19:54:37Z

@nielsbasjes Please continue (rebase) the next PR and let me know when you are done.

nielsbasjes · 2021-06-17T20:22:10Z

Yes, will do

Snipx · 2021-06-18T06:37:22Z

Excellent cooperation, thank you folks! So glad to see this fixed

michael-o · 2021-06-18T09:03:02Z

Excellent cooperation, thank you folks! So glad to see this fixed

Took way too long to process, but people are busy.

nielsbasjes changed the title ~~[MSHADE 36] Use reduced dependencies~~ [MSHADE-36] Use reduced dependencies Aug 27, 2019

This was referenced Aug 27, 2019

Relocated shaded dependencies are still transitive. nielsbasjes/yauaa#171

Closed

[MSHADE-326] Hide shaded dependencies from the rest of the reactor build #26

Open

nielsbasjes force-pushed the MSHADE-36-Use-Reduced-Dependencies branch from 9b4e29b to 6fd4581 Compare August 30, 2019 08:15

Tibor17 reviewed Aug 30, 2019

View reviewed changes

src/it/MSHADE-36-inject-dep-reduced-pom-in-final/verify.bsh Outdated Show resolved Hide resolved

nielsbasjes force-pushed the MSHADE-36-Use-Reduced-Dependencies branch 2 times, most recently from 08d08a6 to 6c25ac7 Compare August 30, 2019 11:57

elharo reviewed Apr 10, 2020

View reviewed changes

michael-o requested changes Jun 13, 2021

View reviewed changes

nielsbasjes force-pushed the MSHADE-36-Use-Reduced-Dependencies branch from 031ec4a to 0cf7fc5 Compare June 16, 2021 09:37

[MSHADE-36] Support for injecting the dependency-reduced-pom.xml into…

3d1dbbc

… the final shaded jar

nielsbasjes force-pushed the MSHADE-36-Use-Reduced-Dependencies branch from 13b12cb to 3d1dbbc Compare June 16, 2021 09:57

nielsbasjes requested a review from michael-o June 16, 2021 09:57

michael-o requested changes Jun 16, 2021

View reviewed changes

Missing review changes

7644ae8

nielsbasjes requested a review from michael-o June 16, 2021 11:45

michael-o requested changes Jun 16, 2021

View reviewed changes

nielsbasjes added 2 commits June 16, 2021 16:15

Review comments

cdcd5af

Do not fail if not DRP generated

134c523

nielsbasjes requested review from rfscholte and michael-o June 16, 2021 19:19

michael-o requested changes Jun 17, 2021

View reviewed changes

src/main/java/org/apache/maven/plugins/shade/mojo/ShadeMojo.java Show resolved Hide resolved

Improve Javadoc

5506508

michael-o closed this in 83c123d Jun 17, 2021

nielsbasjes deleted the MSHADE-36-Use-Reduced-Dependencies branch June 17, 2021 21:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MSHADE-36] Use reduced dependencies #25

[MSHADE-36] Use reduced dependencies #25

nielsbasjes commented Aug 27, 2019 •

edited

Loading

nielsbasjes commented Aug 27, 2019

struberg commented Dec 17, 2019

rfscholte commented Dec 17, 2019

elharo left a comment

nielsbasjes commented Apr 10, 2020

elharo commented Apr 11, 2020

felfert commented Mar 17, 2021

Snipx commented Apr 29, 2021

michael-o Jun 13, 2021

nielsbasjes Jun 15, 2021

nielsbasjes Jun 16, 2021

nielsbasjes Jun 16, 2021

michael-o Jun 16, 2021

michael-o commented Jun 13, 2021

michael-o commented Jun 13, 2021

felfert commented Jun 13, 2021

michael-o commented Jun 13, 2021

felfert commented Jun 13, 2021 •

edited

Loading

michael-o commented Jun 13, 2021

nielsbasjes commented Jun 15, 2021

michael-o commented Jun 15, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

michael-o left a comment

michael-o commented Jun 17, 2021

michael-o commented Jun 17, 2021

nielsbasjes commented Jun 17, 2021

Snipx commented Jun 18, 2021

michael-o commented Jun 18, 2021

		@@ -437,6 +447,24 @@ public void execute()

		List<ResourceTransformer> resourceTransformers = getResourceTransformers();

		if ( createDependencyReducedPom )

[MSHADE-36] Use reduced dependencies #25

[MSHADE-36] Use reduced dependencies #25

Conversation

nielsbasjes commented Aug 27, 2019 • edited Loading

nielsbasjes commented Aug 27, 2019

struberg commented Dec 17, 2019

rfscholte commented Dec 17, 2019

elharo left a comment

Choose a reason for hiding this comment

nielsbasjes commented Apr 10, 2020

elharo commented Apr 11, 2020

felfert commented Mar 17, 2021

Snipx commented Apr 29, 2021

michael-o Jun 13, 2021

Choose a reason for hiding this comment

nielsbasjes Jun 15, 2021

Choose a reason for hiding this comment

nielsbasjes Jun 16, 2021

Choose a reason for hiding this comment

nielsbasjes Jun 16, 2021

Choose a reason for hiding this comment

michael-o Jun 16, 2021

Choose a reason for hiding this comment

michael-o commented Jun 13, 2021

michael-o commented Jun 13, 2021

felfert commented Jun 13, 2021

michael-o commented Jun 13, 2021

felfert commented Jun 13, 2021 • edited Loading

michael-o commented Jun 13, 2021

nielsbasjes commented Jun 15, 2021

michael-o commented Jun 15, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

nielsbasjes commented Jun 16, 2021

michael-o commented Jun 16, 2021

michael-o left a comment

Choose a reason for hiding this comment

michael-o commented Jun 17, 2021

michael-o commented Jun 17, 2021

nielsbasjes commented Jun 17, 2021

Snipx commented Jun 18, 2021

michael-o commented Jun 18, 2021

nielsbasjes commented Aug 27, 2019 •

edited

Loading

felfert commented Jun 13, 2021 •

edited

Loading