Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for service providers #22

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@ lazy val jarjar = project
.value
.map(_.data).mkString(System.getProperty("path.separator"))
)

assemblyMergeStrategy := {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I tried to run sbt jarjar/assembly I got:

[error] 1 error was encountered during merge
[error] stack trace is suppressed; run last assembly for the full output
[error] (assembly) deduplicate: different file contents found in the following:
[error] /Users/gabriel.russo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-analysis/9.2/asm-analysis-9.2.jar:module-info.class
[error] /Users/gabriel.russo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-commons/9.2/asm-commons-9.2.jar:module-info.class
[error] /Users/gabriel.russo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-tree/9.2/asm-tree-9.2.jar:module-info.class
[error] /Users/gabriel.russo/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm/9.2/asm-9.2.jar:module-info.class
[error] Total time: 2 s, completed 21 Oct 2021, 15:12:28

These module-info.class all clash when being packed into the uber jar and ignoring them seemed reasonable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been discussed here: sbt/sbt-assembly#391

case PathList("module-info.class") => MergeStrategy.discard
case x if x.endsWith("/module-info.class") => MergeStrategy.discard
case x =>
val oldStrategy = (ThisBuild / assemblyMergeStrategy).value
oldStrategy(x)
}
})

lazy val core = project
Expand Down
53 changes: 49 additions & 4 deletions jarjar/src/main/java/com/eed3si9n/jarjar/ResourceProcessor.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,67 @@

package com.eed3si9n.jarjar;

import com.eed3si9n.jarjar.util.*;
import com.eed3si9n.jarjar.util.EntryStruct;
import com.eed3si9n.jarjar.util.JarProcessor;

import java.io.IOException;
import java.util.*;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.stream.Collectors;

class ResourceProcessor implements JarProcessor
{
private final static String META_INF_SERVICES = "META-INF/services/";
private PackageRemapper pr;

public ResourceProcessor(PackageRemapper pr) {
this.pr = pr;
}

public boolean process(EntryStruct struct) throws IOException {
if (!struct.name.endsWith(".class"))
struct.name = pr.mapPath(struct.name);
switch (identify(struct)) {
case CLASS_FILE:
break;
case SERVICE_PROVIDER_CONFIGURATION:
struct.name = remapService(struct.name);
struct.data = remapServiceProviders(struct.data);
break;
case OTHER:
struct.name = pr.mapPath(struct.name);
break;
}
return true;
}

private String remapService(String serviceFile) {
int idx = serviceFile.lastIndexOf('/');
return META_INF_SERVICES + pr.mapValue(serviceFile.substring(idx + 1));
}

private byte[] remapServiceProviders(byte[] providers) {
// Provider configuration is encoded in UTF-8
// The file can also have comments and whitespaces
String s = new String(providers, StandardCharsets.UTF_8);

String mapped = Arrays.stream(s.split(System.lineSeparator()))
.map(l -> (String) pr.mapValue(l.split("#")[0].trim()))
.collect(Collectors.joining(System.lineSeparator()));
return mapped.getBytes(StandardCharsets.UTF_8);
}

private Resource identify(EntryStruct struct) {
if (struct.name.endsWith(".class"))
return Resource.CLASS_FILE;
// https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html
if (struct.name.startsWith(META_INF_SERVICES) && !struct.name.equals(META_INF_SERVICES))
return Resource.SERVICE_PROVIDER_CONFIGURATION;
return Resource.OTHER;
}

private enum Resource {
CLASS_FILE,
SERVICE_PROVIDER_CONFIGURATION,
OTHER
}
}

Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public static List<PatternElement> parse(String value) throws IOException {

private static String stripComment(String in) {
int p = in.indexOf("#");
return p < 0 ? in : in.substring(0, p);
return p < 0 ? in.trim() : in.substring(0, p).trim();
}

private static List<PatternElement> parse(Reader r) throws IOException {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
package com.eed3si9n.jarjar;

import com.eed3si9n.jarjar.util.EntryStruct;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.List;

import static org.junit.Assert.*;

public class ResourceProcessorTest {

private ResourceProcessor processor;

@Before
public void setUp() throws Exception {
String rules = "rule org.example.** something.shaded.@0";
List<Rule> parsed = (List<Rule>)(List<?>) RulesFileParser.parse(rules);
processor = new ResourceProcessor(new PackageRemapper(parsed, true));
}

@Test
public void testClassFile() throws IOException {
EntryStruct entryStruct = new EntryStruct();
entryStruct.name = "org/example/Object.class";
entryStruct.data = new byte[]{0x10};

assertTrue(processor.process(entryStruct));
assertEquals(entryStruct.name, "org/example/Object.class");
assertArrayEquals(entryStruct.data, new byte[]{0x10});
}

@Test
public void testServiceProviderConfig() throws IOException {
String original = "org.example.Impl # comment" + System.lineSeparator()
+ "org.example.AnotherImpl" + System.lineSeparator()
+ System.lineSeparator()
+ " org.another.Impl";
String expected = "something.shaded.org.example.Impl" + System.lineSeparator()
+ "something.shaded.org.example.AnotherImpl" + System.lineSeparator()
+ System.lineSeparator()
+ "org.another.Impl";
EntryStruct entryStruct = new EntryStruct();
entryStruct.name = "META-INF/services/org.example.Service";
entryStruct.data = original.getBytes(StandardCharsets.UTF_8);

assertTrue(processor.process(entryStruct));
assertEquals(entryStruct.name, "META-INF/services/something.shaded.org.example.Service");
assertArrayEquals(entryStruct.data, expected.getBytes(StandardCharsets.UTF_8));
}

@Test
public void testOtherResource() throws IOException {
EntryStruct entryStruct = new EntryStruct();
entryStruct.name = "org/example/file.txt";
entryStruct.data = new byte[]{0x10};

assertTrue(processor.process(entryStruct));
assertEquals(entryStruct.name, "something/shaded/org/example/file.txt");
assertArrayEquals(entryStruct.data, new byte[]{0x10});

EntryStruct entryStruct2 = new EntryStruct();
entryStruct2.name = "another_org/example/file.txt";
entryStruct2.data = new byte[]{0x10};

assertTrue(processor.process(entryStruct2));
assertEquals(entryStruct2.name, "another_org/example/file.txt");
assertArrayEquals(entryStruct.data, new byte[]{0x10});
}
}
1 change: 1 addition & 0 deletions project/plugins.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ addSbtPlugin("com.eed3si9n" % "sbt-nocomma" % "0.1.0")
addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.4.3")
addSbtPlugin("org.scala-sbt" % "sbt-contraband" % "0.4.6")
addSbtPlugin("com.jsuereth" % "sbt-pgp" % "2.0.1")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "1.1.0")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about this line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @er1c thanks for fixing the java8 issue

I added this line because we are not (yet) using jarjar-abrams but only the jarjar cli. To give a bit more context, we build our code with Bazel and we have a jarjar rule that basically invokes java -jar jarjar.jar [args]. This means we need an uber jar, and that's why I included the sbt-assembly plugin. Alternatively, I could port the jarjar build to bazel and build it on demand, but that's probably too much.

I'm not very familiar with sbt, is there another way to get an uber jar?