Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproduction test case with an shading issue #8

Closed
wants to merge 1 commit into from

Conversation

daddykotex
Copy link

Hi thanks for the library, it's currently saving our asses (I'm sad to say, but we have to shade some libraries).

While working with the library, we noticed one issue with it. This pull request only includes a reproduction case and hopefully the CI can run and show the error.

I would be more than happy to provide a fix for it. I would just like to ask if that's possible at all (I'm not sure because the issue seems to come from a low level compiler behaviour).

Here is the stack (explanation follows):

[info] [info] Non-compiled module 'compiler-bridge_2.13' for Scala 2.13.1. Compiling...
[info] 8 warnings found
[info] [info]   Compilation completed in 8.449s.
[info] [info] Done compiling.
[info] [info] running example.ApplicationMain
[info] [error] (run-main-0) java.lang.AbstractMethodError: example.DependsOn.example$ATraitWithAVal$_setter_$thing_$eq(Lshaded/example/Thing;)V
[info] [error] java.lang.AbstractMethodError: example.DependsOn.example$ATraitWithAVal$_setter_$thing_$eq(Lshaded/example/Thing;)V
[info] [error] 	at shaded.example.ATraitWithAVal.$init$(Test.scala:6)
[info] [error] 	at example.DependsOn.<init>(Test.scala:5)
[info] [error] 	at example.ApplicationMain$.<clinit>(Test.scala:8)
[info] [error] nat example.ApplicationMain.main(Test.scala)
[info] [error] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[info] [error] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[info] [error] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[info] [error] 	at java.lang.reflect.Method.invoke(Method.java:498)
[info] [error] stack trace is suppressed; run 'last use / Compile / bgRun' for the full output
[info] [error] Nonzero exit code: 1
[info] [error] (use / Compile / run) Nonzero exit code: 1
[info] [error] Total time: 10 s, completed 7-Oct-2020 10:15:38 AM

Let's say I'm trying to shade com.example:some-library. This library is a Scala library and it has a trait defined like so:

package example

final case class Thing(params: String)

trait ATraitWithAVal {
  val thing = Thing("value")
}

In this instance, the val inside the trait is compiled in a very specific way to be a Java interface. A setter will be introduced and called during initialization. The setter will look like:

public abstract ATraitWithAVal$_setter_$thing$eq(example.Thing s);

When we shade, I believe we should rename those as well, because otherwise, extending the shaded trait raise an error like shown above. (more info at https://stackoverflow.com/questions/43446423/using-val-in-scala-trait).

Do you think it's possible to fix this, if yes, let me know, I'll look into it more carefully and try to fix it.

@daddykotex
Copy link
Author

I forgot one folder outside of git. So the first build is completely irrelevant, but the latest one is:

https://travis-ci.com/github/eed3si9n/jarjar-abrams/jobs/396423322#L488

@eed3si9n
Copy link
Owner

eed3si9n commented Oct 8, 2020

@daddykotex Thanks for the reproduction.

I would be more than happy to provide a fix for it. I would just like to ask if that's possible at all (I'm not sure because the issue seems to come from a low level compiler behaviour).

I'm not really sure since I've only packaged the code contributed by @jeroentervoorde and others. I'd suggest you look into the output of scalap to pinpoint the exact bytecode in your trait. My guess is that all classes are renamed, but maybe some references to classes may not be handled by jarjar? If so I think it's possible to provide a visitor that would rewrite the bytecode. See test case in jarjar (https://github.com/pantsbuild/jarjar/blob/master/src/test/java/org/pantsbuild/jarjar/MethodRewriterTest.java).

@daddykotex
Copy link
Author

Thank you, I will look into it

@daddykotex
Copy link
Author

I see that the project is now in read-only, I may have to fork and publish a new version.

@daddykotex
Copy link
Author

Dropping this here as pieces of my investigation. I must say I'm learning as I go here.

I've added a way to run scalap from the project itself (so long as you have scalap on your PATH: cs install scalap was useful for me).

[info] Loading settings for project jarjar-abrams from build.sbt ...
[info] Set current project to some-random-library (in build file:/Users/david/workspace/jarjar-abrams/)
[info] Running [scalap -verbose -classpath /Users/david/workspace/jarjar-abrams/some-random-library/target/scala-2.13/classes:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/scala-library-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.13.1/scalap-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.1/scala-compiler-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.1/scala-reflect-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.12/jansi-1.12.jar example.ATraitWithAVal]
[info] Result:
[info] CLASSPATH = /Users/david/workspace/jarjar-abrams/some-random-library/target/scala-2.13/classes:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/scala-library-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.13.1/scalap-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.1/scala-compiler-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.1/scala-reflect-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.12/jansi-1.12.jar
[info] FILENAME = /Users/david/workspace/jarjar-abrams/some-random-library/target/scala-2.13/classes/example/ATraitWithAVal.class
[info] package example
[info] trait ATraitWithAVal extends scala.AnyRef {
[info]   def $init$(): scala.Unit = { /* compiled code */ }
[info]   val thing: example.Thing = { /* compiled code */ }
[info] }
[info]
[success] Total time: 1 s, completed 14-Oct-2020 9:51:04 PM

@daddykotex
Copy link
Author

Running javap gives me this:

sbt:some-random-library> runJavaP example.ATraitWithAVal
[info] Running [javap -verbose -classpath /Users/david/workspace/jarjar-abrams/some-random-library/target/scala-2.13/classes:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/scala-library-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.13.1/scalap-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.1/scala-compiler-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.1/scala-reflect-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.12/jansi-1.12.jar example.ATraitWithAVal]
[info] Result:
[info] Classfile /Users/david/workspace/jarjar-abrams/some-random-library/target/scala-2.13/classes/example/ATraitWithAVal.class
[info]   Last modified 7-Oct-2020; size 860 bytes
[info]   MD5 checksum 644ee94ab52f7cd12022dd4a1e195fcb
[info]   Compiled from "Test.scala"
[info] public interface example.ATraitWithAVal
[info]   minor version: 0
[info]   major version: 520s
[info]   flags: ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT
[info] Constant pool:
[info]    #1 = Utf8               example/ATraitWithAVal
[info]    #2 = Class              #1             // example/ATraitWithAVal
[info]    #3 = Utf8               java/lang/Object
[info]    #4 = Class              #3             // java/lang/Object
[info]    #5 = Utf8               Test.scala
[info]    #6 = Utf8               Lscala/reflect/ScalaSignature;
[info]    #7 = Utf8               bytes
[info]    #8 = Utf8               e1qaqCqC\tI
                                             #&/Y5u/&.,bY*\tQ!Afql\3M\t1iA)1oY1mC&QB:L(+4\rJg.%)CA\t\"BV]&$!;iS:<W#A
Y9R\"\na!!!+iS:<
[info]    #9 = Utf8               example$ATraitWithAVal$_setter_$thing_$eq
[info]   #10 = Utf8               (Lexample/Thing;)V
[info]   #11 = Utf8               x$1
[info]   #12 = Utf8               thing
[info]   #13 = Utf8               ()Lexample/Thing;
[info]   #14 = Utf8               $init$
[info]   #15 = Utf8               (Lexample/ATraitWithAVal;)V
[info]   #16 = Utf8               $this
[info]   #17 = Utf8               example/Thing
[info]   #18 = Class              #17            // example/Thing
[info]   #19 = Utf8               value
[info]   #20 = String             #19            // value
[info]   #21 = Utf8               <init>
[info]   #22 = Utf8               (Ljava/lang/String;)V
[info]   #23 = NameAndType        #21:#22        // "<init>":(Ljava/lang/String;)V
[info]   #24 = Methodref          #18.#23        // example/Thing."<init>":(Ljava/lang/String;)V
[info]   #25 = NameAndType        #9:#10         // example$ATraitWithAVal$_setter_$thing_$eq:(Lexample/Thing;)V
[info]   #26 = InterfaceMethodref #2.#25         // example/ATraitWithAVal.example$ATraitWithAVal$_setter_$thing_$eq:(Lexample/Thing;)V
[info]   #27 = Utf8               Lexample/ATraitWithAVal;
[info]   #28 = Utf8               MethodParameters
[info]   #29 = Utf8               Code
[info]   #30 = Utf8               LineNumberTable
[info]   #31 = Utf8               LocalVariableTable
[info]   #32 = Utf8               SourceFile
[info]   #33 = Utf8               RuntimeVisibleAnnotations
[info]   #34 = Utf8               ScalaInlineInfo
[info]   #35 = Utf8               ScalaSig
[info] {
[info]   public abstract void example$ATraitWithAVal$_setter_$thing_$eq(example.Thing);
[info]     descriptor: (Lexample/Thing;)V
[info]     flags: ACC_PUBLIC, ACC_ABSTRACT
[info]     MethodParameters:
[info]       Name                           Flags
[info]       x$1                            final
[info]
[info]   public abstract example.Thing thing();
[info]     descriptor: ()Lexample/Thing;
[info]     flags: ACC_PUBLIC, ACC_ABSTRACT
[info]
[info]   public static void $init$(example.ATraitWithAVal);
[info]     descriptor: (Lexample/ATraitWithAVal;)V
[info]     flags: ACC_PUBLIC, ACC_STATIC
[info]     Code:
[info]       stack=4, locals=1, args_size=1
[info]          0: aload_0
[info]          1: new           #18                 // class example/Thing
[info]          4: dup
[info]          5: ldc           #20                 // String value
[info]          7: invokespecial #24                 // Method example/Thing."<init>":(Ljava/lang/String;)V
[info]         10: invokeinterface #26,  2           // InterfaceMethod example$ATraitWithAVal$_setter_$thing_$eq:(Lexample/Thing;)V
[info]         15: return
[info]       LineNumberTable:
[info]         line 6: 0
[info]         line 5: 15
[info]       LocalVariableTable:
[info]         Start  Length  Slot  Name   Signature
[info]             0      16     0 $this   Lexample/ATraitWithAVal;
[info]     MethodParameters:
[info]       Name                           Flags
[info]       $this                          final synthetic
[info] }
[info] SourceFile: "Test.scala"
[info] RuntimeVisibleAnnotations:
[info]   0: #6(#7=s#8)
[info] Error: unknown attribute
[info]   ScalaInlineInfo: length = 0x13
[info]    01 00 00 03 00 0E 00 0F 01 00 09 00 0A 00 00 0C
[info]    00 0D 00
[info] Error: unknown attribute
[info]   ScalaSig: length = 0x3
[info]    05 02 00
[success] Total time: 0 s, completed 14-Oct-2020 10:05:40 PM

@daddykotex
Copy link
Author

javap of the shaded class:

[info] Running [javap -verbose -classpath /Users/david/.ivy2/local/com.example/trait-test_2.13/0.1.0-SNAPSHOT/jars/trait-test_2.13.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.1/scala-library-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scalap/2.13.1/scalap-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.13.1/scala-compiler-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.13.1/scala-reflect-2.13.1.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar:/Users/david/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/fusesource/jansi/jansi/1.12/jansi-1.12.jar shaded.example.ATraitWithAVal]
[info] Result:
[info] Classfile jar:file:/Users/david/.ivy2/local/com.example/trait-test_2.13/0.1.0-SNAPSHOT/jars/trait-test_2.13.jar!/shaded/example/ATraitWithAVal.class
[info]   Last modified 14-Oct-2020; size 916 bytes
[info]   MD5 checksum 0ccaf3168720b88b44f245d1e95a2d80
[info]   Compiled from "Test.scala"
[info] public interface shaded.example.ATraitWithAVal
[info]   minor version: 0
[info]   major version: 52
[info]   flags: ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT
[info] Constant pool:
[info]    #1 = Utf8               shaded/example/ATraitWithAVal
[info]    #2 = Class              #1             // shaded/example/ATraitWithAVal
[info]    #3 = Utf8               java/lang/Object
[info]    #4 = Class              #3             // java/lang/Object
[info]    #5 = Utf8               Test.scala
[info]    #6 = Utf8               Lscala/reflect/ScalaSignature;
[info]    #7 = Utf8               bytes
[info]    #8 = Utf8               m1qaqCqC\tI
                                             #&/Y5u/&.,bY*QAKb
[info] W7f\"%aQ\"
                 -\tQa]2bYL!!\rsPU3g!N\5uIQ\t\n#%!C+:LG/AuQ&tw-F!\t1r#D\tABAAUQ&tw-tQ$W\r3
[info]    #9 = Utf8               example$ATraitWithAVal$_setter_$thing_$eq
[info]   #10 = Utf8               (Lshaded/example/Thing;)V
[info]   #11 = Utf8               x$1
[info]   #12 = Utf8               thing
[info]   #13 = Utf8               ()Lshaded/example/Thing;
[info]   #14 = Utf8               $init$
[info]   #15 = Utf8               (Lshaded/example/ATraitWithAVal;)V
[info]   #16 = Utf8               $this
[info]   #17 = Utf8               shaded/example/Thing
[info]   #18 = Class              #17            // shaded/example/Thing
[info]   #19 = Utf8               value
[info]   #20 = String             #19            // value
[info]   #21 = Utf8               <init>
[info]   #22 = Utf8               (Ljava/lang/String;)V
[info]   #23 = NameAndType        #21:#22        // "<init>":(Ljava/lang/String;)V
[info]   #24 = Methodref          #18.#23        // shaded/example/Thing."<init>":(Ljava/lang/String;)V
[info]   #25 = NameAndType        #9:#10         // example$ATraitWithAVal$_setter_$thing_$eq:(Lshaded/example/Thing;)V
[info]   #26 = InterfaceMethodref #2.#25         // shaded/example/ATraitWithAVal.example$ATraitWithAVal$_setter_$thing_$eq:(Lshaded/example/Thing;)V
[info]   #27 = Utf8               Lshaded/example/ATraitWithAVal;
[info]   #28 = Utf8               MethodParameters
[info]   #29 = Utf8               Code
[info]   #30 = Utf8               LineNumberTable
[info]   #31 = Utf8               LocalVariableTable
[info]   #32 = Utf8               SourceFile
[info]   #33 = Utf8               RuntimeVisibleAnnotations
[info]   #34 = Utf8               ScalaInlineInfo
[info]   #35 = Utf8               ScalaSig
[info] {
[info]   public abstract void example$ATraitWithAVal$_setter_$thing_$eq(shaded.example.Thing);
[info]     descriptor: (Lshaded/example/Thing;)V
[info]     flags: ACC_PUBLIC, ACC_ABSTRACT
[info]     MethodParameters:
[info]       Name                           Flags
[info]       x$1                            final
[info]
[info]   public abstract shaded.example.Thing thing();
[info]     descriptor: ()Lshaded/example/Thing;
[info]     flags: ACC_PUBLIC, ACC_ABSTRACT
[info]
[info]   public static void $init$(shaded.example.ATraitWithAVal);
[info]     descriptor: (Lshaded/example/ATraitWithAVal;)V
[info]     flags: ACC_PUBLIC, ACC_STATIC
[info]     Code:
[info]       stack=4, locals=1, args_size=1
[info]          0: aload_0
[info]          1: new           #18                 // class shaded/example/Thing
[info]          4: dup
[info]          5: ldc           #20                 // String value
[info]          7: invokespecial #24                 // Method shaded/example/Thing."<init>":(Ljava/lang/String;)V
[info]         10: invokeinterface #26,  2           // InterfaceMethod example$ATraitWithAVal$_setter_$thing_$eq:(Lshaded/example/Thing;)V
[info]         15: return
[info]       LineNumberTable:
[info]         line 6: 0
[info]         line 5: 15
[info]       LocalVariableTable:
[info]         Start  Length  Slot  Name   Signature
[info]             0      16     0 $this   Lshaded/example/ATraitWithAVal;
[info]     MethodParameters:
[info]       Name                           Flags
[info]       $this                          final synthetic
[info] }
[info] SourceFile: "Test.scala"
[info] RuntimeVisibleAnnotations:
[info]   0: #6(#7=s#8)
[info] Error: unknown attribute
[info]   ScalaInlineInfo: length = 0x13
[info]    01 00 00 03 00 0E 00 0F 01 00 09 00 0A 00 00 0C
[info]    00 0D 00
[info] Error: unknown attribute
[info]   ScalaSig: length = 0x3
[info]    05 02 00
[success] Total time: 0 s, completed 14-Oct-2020 10:59:08 PM

@eed3si9n eed3si9n changed the base branch from master to develop March 28, 2021 21:03
@ericdotdata
Copy link
Contributor

@daddykotex you might try testing this against #15 and see if the update of the newer jarjar fixes this issue

@er1c
Copy link
Collaborator

er1c commented Aug 25, 2021

This works if you don't reference the "shaded" in the original Test.scala:

diff --git a/sbtplugin/src/sbt-test/actions/trait/use/src/main/scala/Test.scala b/sbtplugin/src/sbt-test/actions/trait/use/src/main/scala/Test.scala
index 072aadc..3075135 100644
--- a/sbtplugin/src/sbt-test/actions/trait/use/src/main/scala/Test.scala
+++ b/sbtplugin/src/sbt-test/actions/trait/use/src/main/scala/Test.scala
@@ -2,7 +2,7 @@ package example

 import shaded.example._

-class DependsOn extends shaded.example.ATraitWithAVal
+class DependsOn extends example.ATraitWithAVal

 object ApplicationMain extends App {
   new DependsOn

Branch against master:

Running actions / trait
[error] Getting org.scala-sbt sbt 1.3.10  (this may take some time)...
[info] :: retrieving :: org.scala-sbt#boot-app
[info] 	confs: [default]
[info] 	81 artifacts copied, 0 already retrieved (38284kB/233ms)
[error] Getting Scala 2.12.10 (for sbt)...
[info] :: retrieving :: org.scala-sbt#boot-scala
[info] 	confs: [default]
[info] 	6 artifacts copied, 0 already retrieved (20113kB/55ms)
[info] [info] Loading settings for project trait-build from plugins.sbt ...
[info] [info] Loading project definition from /private/var/folders/sn/qngm85fs09j3v3q9rqb5k5rw0000gn/T/sbt_7e143fd9/trait/project
[info] [info] Loading settings for project trait from build.sbt ...
[info] [info] Set current project to trait (in build file:/private/var/folders/sn/qngm85fs09j3v3q9rqb5k5rw0000gn/T/sbt_7e143fd9/trait/)
[info] [info] Wrote /private/var/folders/sn/qngm85fs09j3v3q9rqb5k5rw0000gn/T/sbt_7e143fd9/trait/shadedLibrary/target/scala-2.13/shaded-json4s-ast_2.13-0.1.0-SNAPSHOT.pom
[info] [info] /private/var/folders/sn/qngm85fs09j3v3q9rqb5k5rw0000gn/T/sbt_7e143fd9/trait/shadedLibrary/target/scala-2.13/shaded-json4s-ast_2.13-0.1.0-SNAPSHOT.pom
[info] [success] Total time: 0 s, completed Aug 25, 2021 12:41:22 PM
[info] [info] Compiling 1 Scala source to /private/var/folders/sn/qngm85fs09j3v3q9rqb5k5rw0000gn/T/sbt_7e143fd9/trait/use/target/scala-2.13/classes ...
[info] [info] Non-compiled module 'compiler-bridge_2.13' for Scala 2.13.1. Compiling...
[info] 8 warnings found
[info] [info]   Compilation completed in 10.574s.
[info] [info] Done compiling.
[info] [info] running example.ApplicationMain
[info] [success] Total time: 12 s, completed Aug 25, 2021 12:41:35 PM
[info] + actions / trait

@er1c
Copy link
Collaborator

er1c commented Aug 25, 2021

Going to close for now, please re-open if you think in error

@er1c er1c closed this Aug 25, 2021
@daddykotex daddykotex deleted the trait-val-issue branch August 31, 2021 18:24
@daddykotex
Copy link
Author

Amazing work @er1c thanks a lot.

@daddykotex
Copy link
Author

Do you know when the next release will be published?

Thanks again

@er1c
Copy link
Collaborator

er1c commented Aug 31, 2021

@daddykotex hopefully later tonight

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants