Add DeduplicatingResourceTransformer to deduplicate on path and content
#1859
+360
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
snazy
force-pushed
the
dedup-content-transformer
branch
from
a26f539 to
4ed3bf3
Compare
snazy
changed the title
Goooler
reviewed
Nov 18, 2025
src/testKit/kotlin/com/github/jengelman/gradle/plugins/shadow/testkit/JarPath.kt
Outdated
Show resolved
Hide resolved
.../github/jengelman/gradle/plugins/shadow/transformers/DeduplicatingResourceTransformerTest.kt
Outdated
Show resolved
Hide resolved
snazy
force-pushed
the
dedup-content-transformer
branch
3 times, most recently
from
c23e4ab to
5353511
Compare
Goooler
changed the title
DeduplicatingResourceTransformer to deduplicate on path and content
snazy
force-pushed
the
dedup-content-transformer
branch
from
5353511 to
7ee24d4
Compare
Goooler
reviewed
Nov 19, 2025
.../com/github/jengelman/gradle/plugins/shadow/transformers/DeduplicatingResourceTransformer.kt
Outdated
Show resolved
Hide resolved
Goooler
reviewed
Nov 19, 2025
.../com/github/jengelman/gradle/plugins/shadow/transformers/DeduplicatingResourceTransformer.kt
Outdated
Show resolved
Hide resolved
snazy
force-pushed
the
dedup-content-transformer
branch
from
7ee24d4 to
cec54aa
Compare
Copilot finished reviewing on behalf of
Goooler
November 19, 2025 13:19
snazy
force-pushed
the
dedup-content-transformer
branch
from
d94d362 to
2c7c252
Compare
Goooler
reviewed
Nov 20, 2025
src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/BaseTransformerTest.kt
Outdated
Show resolved
Hide resolved
Adds a new `DeduplicatingResourceTransformer` that works different than `PreserveFirstFoundResourceTransformer`. `PreserveFirstFoundResourceTransformer` is to preserve the first resource that matches the configured paths and ignore all other ones. `DeduplicatingResourceTransformer` preserves resources by path _and_ identical content and fails for all not explicitly allowed (excluded) resources with different content. It works intentionally against all resources. The new one is intended to guard a couple of unexpected situations: * A (transitive) dependency brings a non-relocated version of a dependency that is also included elsewhere but with a different version. This could normally lead to unexpected exceptions during runtime. * Unintended inclusion or removal or legally important license information, see also `MergeLicenseResourceTransformer` (GradleUp#1858). * Unintended removal or (false) inclusion of shaded dependency information via `META-INF/x/y/pom.xml`/`.properties` files, which can be important for dependency/license analyzation tools. Adding the functionality of `DeduplicatingResourceTransformer` to `PreserveFirstFoundResourceTransformer` became a bit too difficult without breaking the existing behavior of the latter.
…nsformers/DeduplicatingResourceTransformer.kt Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…shadow/transformers/TransformersTest.kt Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…testkit/JarPath.kt Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…testkit/JarPath.kt Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
snazy
force-pushed
the
dedup-content-transformer
branch
from
2c7c252 to
58c927a
Compare
Goooler
reviewed
Nov 20, 2025
...ionalTest/kotlin/com/github/jengelman/gradle/plugins/shadow/transformers/TransformersTest.kt
Outdated
Show resolved
Hide resolved
| * Multiple files with the same path but different content lead to an error. | ||
| * | ||
| * Some scenarios for duplicate resources in a shadow jar: | ||
| * * Duplicate `.class` files |
Member
There was a problem hiding this comment.
Suggested change
| * * Duplicate `.class` files | |
| * | |
| * - Duplicate `.class` files |
| * Having duplicate `.class` files with different content is a situation indicating that the resulting jar is | ||
| * built with _incompatible_ classes, likely leading to issues during runtime. | ||
| * This situation can happen when one dependency is (also) included in an uber jar. | ||
| * * Duplicate `META-INF/<group-id>/<artifact-id>/pom.properties`/`xml` files. |
Member
There was a problem hiding this comment.
Suggested change
| * * Duplicate `META-INF/<group-id>/<artifact-id>/pom.properties`/`xml` files. | |
| * | |
| * - Duplicate `META-INF/<group-id>/<artifact-id>/pom.properties`/`xml` files. |
Comment on lines
+39
to
+40
| * ```kotlin | ||
| * tasks.named<ShadowJar>("shadowJar").configure { |
Member
There was a problem hiding this comment.
Suggested change
| * ```kotlin | |
| * tasks.named<ShadowJar>("shadowJar").configure { | |
| * | |
| * ```kotlin | |
| * tasks.shadowJar { |
.../com/github/jengelman/gradle/plugins/shadow/transformers/DeduplicatingResourceTransformer.kt
Outdated
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new
DeduplicatingResourceTransformerthat works different thanPreserveFirstFoundResourceTransformer.PreserveFirstFoundResourceTransformeris to preserve the first resource that matches the configured paths and ignore all other ones.DeduplicatingResourceTransformerpreserves resources by path and identical content and fails for all not explicitly allowed (excluded) resources with different content. It works intentionally against all resources.The new one is intended to guard a couple of unexpected situations:
MergeLicenseResourceTransformer(AddMergeLicenseResourceTransformerto merge licenses #1858).META-INF/x/y/pom.xml/.propertiesfiles, which can be important for dependency/license analyzation tools.Adding the functionality of
DeduplicatingResourceTransformertoPreserveFirstFoundResourceTransformerbecame a bit too difficult without breaking the existing behavior of the latter.Refs #1848.