Maven Build Issue from Setting Class-Path in the jar manifest for grpc-netty-shaded

[michaelli321]

What version of gRPC-Java are you using?

1.41.0

What is your environment?

JDK 16, Maven version 3.8.1, MacOS Catalina 10.15.7

What did you expect to see?

a clean build of my project

What did you see instead?

[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/grpc-core-1.41.0.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/grpc-api-1.41.0.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/guava-30.1-android.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/error_prone_annotations-2.9.0.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/perfmark-api-0.23.0.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/failureaccess-1.0.1.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/jsr305-3.0.2.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/checker-compat-qual-2.5.5.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/j2objc-annotations-1.3.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/gson-2.8.6.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/annotations-4.1.1.4.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/animal-sniffer-annotations-1.19.jar": no such file or directory
[WARNING] [path] bad path element "/Users/mli/.m2/repository/io/grpc/grpc-netty-shaded/1.41.0/grpc-context-1.41.0.jar": no such file or directory

Steps to reproduce the bug

I added some other dependencies to an existing project in my pom.xml such as

 <dependency>
      <groupId>org.grpcmock</groupId>
      <artifactId>grpcmock-junit5</artifactId>
      <version>${grpc-mock.version}</version>
</dependency>

This issue has happened with other projects that we take a dependency on, see here. Taking a look at the MANIFEST.MF of the grpc-netty-shaded jar, the Class-Path attribute is set which seems to be causing the bug.

Manifest-Version: 1.0
Implementation-Title: grpc-netty-shaded
Implementation-Version: 1.41.0
Class-Path: grpc-core-1.41.0.jar grpc-api-1.41.0.jar guava-30.1-androi
 d.jar error_prone_annotations-2.9.0.jar perfmark-api-0.23.0.jar failu
 reaccess-1.0.1.jar listenablefuture-9999.0-empty-to-avoid-conflict-wi
 th-guava.jar jsr305-3.0.2.jar checker-compat-qual-2.5.5.jar j2objc-an
 notations-1.3.jar gson-2.8.6.jar annotations-4.1.1.4.jar animal-sniff
 er-annotations-1.19.jar grpc-context-1.41.0.jar

[dapengzhang0]

Can you try by adding

 <compilerArgs><arg>-Xlint:-path</arg></compilerArgs>

in your pom.xml as described in this document?

[dapengzhang0]

@ejona86 Is the particular Class-Path attribute in grpc-netty-shaded MANIFEST.MF necessary or does it have any value?

[ejona86]

It isn't necessary, but I'm suspicious that michaelli321's using some broken tooling for it to be causing problems. I don't understand how it could cause problems, unless you are doing java -jar grpc-netty-shaded.jar but that won't work no matter what you do.

[dapengzhang0]

I tried by adding

 <dependency>
      <groupId>org.grpcmock</groupId>
      <artifactId>grpcmock-junit5</artifactId>
      <version>0.6.0</version>
</dependency>

in the grpc helloworld example https://github.com/grpc/grpc-java/blob/v1.41.0/examples/pom.xml as described in "Steps to reproduce the bug" above,
but did not reproduce the issue. @michaelli321 Can you provide a complete reproducible project?

[michaelli321]

@dapengzhang0 sorry I'm not able to provide a complete reproducible project since the project is for my company.

I'm able to get around the warnings using <arg>-Xlint:-path</arg> but I was hoping that this would be a temporary work around. My understanding is that if you were planning on others to use your jar as a library jar as opposed to a stand-alone application jar, then you shouldn't be setting the Class-Path attribute in the manifest since those dependencies could come elsewhere versus solely from the grpc-netty-shaded/ directory.

references:

• https://stackoverflow.com/questions/1344202/bad-path-warning-where-is-it-coming-from
• https://stackoverflow.com/questions/41944939/maven-build-failure-involving-manifest-mf-classpath-in-dependency-jars
• http://samskivert.com/blog/2010/09/why-maven-sucks-act-ii/

[ejona86]

Something doesn't add up, because the Class-Path attribute is only used with java -jar, to my knowledge, and it isn't transitive. javac for other users isn't noticing this issue, so there's something special about your environment. I suspect some plugin is trying to be "helpful" and messing things up...

FWIW, we didn't explicitly add Class-Path to grpc-netty-shaded. It was added there automatically by a plugin (which was also trying to be helpful). We can try to find how to disable that feature in the plugin, but it isn't all that wrong of the plugin since it should have no effect.

[michaelli321]

I see thanks for checking on that!

because the Class-Path attribute is only used with java -jar

Based off what I've been reading, I don't think this is true.

• https://stackoverflow.com/questions/5900196/when-is-the-class-path-attribute-in-a-jar-file-examined-and-used
• https://docs.oracle.com/javase/6/docs/technotes/tools/findingclasses.html#jarclass
• https://docs.oracle.com/javase/tutorial/deployment/jar/downman.html

but yes I agree something weird is happening because we weren't running into this issue before and it just now surfaced

[ejona86]

As a side note to all this, you really shouldn't have grpc-netty-shaded in the compile scope. We'd expect it to be runtime scope.

https://docs.oracle.com/javase/tutorial/deployment/jar/downman.html

I don't really see anything there that would disagree with what I said. The language is simply not precise enough.

https://docs.oracle.com/javase/6/docs/technotes/tools/findingclasses.html#jarclass

But this definitely does. (I'm not ignoring the stack overflow link, but it isn't authoritative.)

I ran a test myself, and I am also surprised. I know I tested this long ago, so I am as surprised as that stack overflow answer author and the other commenter. Either I messed up that test or the behavior changed. I probably originally tested it with Java 1.4, but I don't have a JDK that old laying around. I did test with Java 5 and saw the behavior you linked to. TIL.

$ java -version
java version "1.5.0_22"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_22-b03)
Java HotSpot(TM) Server VM (build 1.5.0_22-b03, mixed mode)
$ cat > A.java
final class A {
  public static void doit() {
    System.out.println("class A");
  }
}
$ cat > B.java
public final class B {
  public static void main(String[] args) {
    A.doit();
  }
}
$ cat > manifest.txt
Class-Path: a.jar
Main-Class: B
$ javac B.java
$ jar cf a.jar A.class
$ jar cfm b.jar manifest.txt B.class
$ java -jar b.jar
class A
$ java -cp b.jar B
class A

And then with some renaming, I was able to reproduce the warning with -Xlint:all.

Our build has had -Xlint:-path for 4.5 years but removing it now doesn't cause any warnings. Interestingly even the Maven example uses -path. It seems I added it back then to avoid lots of these useless warnings: [path] bad path element "grpc-java/auth/build/resources/main": no such file or directory.

[ejona86]

I looked into the shadow plugin some, and it appears this is difficult/annoying the solve. The plugin sets this configuration in a doFirst, which makes it hard to override.
https://github.com/johnrengelman/shadow/blob/829647a06971ebc96c7d3df717f9a4e92811b602/src/main/groovy/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.groovy#L76-L82

We may need to "manually" rewrite the JAR file/manifest after the JAR has been created by the shadow plugin.

[danielnorberg]

@ejona86 Hey, I found this issue debugging the same "bad path element" warning in one of our internal projects where we're using grpc-netty-shaded and noticed the above comment that grpc-netty-shaded should only be in runtime scope. Is that documented somewhere? The usage example in the readme doesn't seem to be specifying any scope.

[dapengzhang0]

Is that documented somewhere? The usage example in the readme doesn't seem to be specifying any scope.

In the helloworld example
https://github.com/grpc/grpc-java/blob/v1.43.1/examples/build.gradle#L37
https://github.com/grpc/grpc-java/blob/v1.43.1/examples/pom.xml#L39
we do specify grpc-netty-shaded in runtime scope since the commit 3202fcc. The REAME seems not in sync.

[danielnorberg]

@dapengzhang0 Got it, thank you. Took the liberty of submitting a PR to add the runtime scope to the readme.