IllegalAccessError with mixed Scala/Java due to trait encoding

[scabug]

This issue is present on both trunk and 2.7.x.

j/J.java:

package j;

public class J {
  protected void foo() {
    System.out.println("J.foo()");
  }
}

s/S.scala:

package s

import j.J

trait S extends J {
  def bar() {
    foo()
  }
}

class SC extends J with S

object Test {
  def main(args : Array[String]) {
    (new SC).bar()
  }
}

These compile without error (either compiling the Java first then the Scala; or compiling both the Scala and Java with scalac first, then the Java with javac). On running the following stacktrace is generated,

java.lang.IllegalAccessError: tried to access method j.J.foo()V from class s.S$$class
	at s.S$$class.bar(S.scala:7)
	at s.SC.bar(S.scala:11)
	at s.Test$$.main(S.scala:15)
	at s.Test.main(S.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at scala.util.ScalaClassLoader$$$$anonfun$$run$$1.apply(ClassLoader.scala:54)
	at scala.util.ScalaClassLoader$$class.asContext(ClassLoader.scala:21)
	at scala.util.URLClassLoader.asContext(ClassLoader.scala:58)
	at scala.util.ScalaClassLoader$$class.run(ClassLoader.scala:54)
	at scala.util.URLClassLoader.run(ClassLoader.scala:58)
	at scala.tools.nsc.ObjectRunner$$.run(ObjectRunner.scala:33)
	at scala.tools.nsc.MainGenericRunner$$.main(MainGenericRunner.scala:138)
	at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

Viewing the bytecode for s.S$$class shows why this is the case:

public abstract class s.S$$class extends java.lang.Object{
public static void $$init$$(s.S);
  Code:
   0:	return

public static void bar(s.S);
  Code:
   0:	aload_0
   1:	checkcast	SI-12; //class j/J
   4:	invokevirtual	SI-16; //Method j/J.foo:()V
   7:	return
}

J.foo is invoked from s.S$$class which does not extend j.J despite trait s.S extending j.J.

It would be great if this could be fixed, but presumably that would involve a significant change to the encoding of traits, so I'd settle for a compile time error for this case.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-2296?orig=1
Reporter: @milessabin
Other Milestones: 2.10.0-M3, 2.10.0

[scabug]

@paulp said:
I just ran into this bug too, working with classloaders. Interestingly it does not manifest if you explicitly qualify with super. Change trait S to this:

  def bar() {
    super.foo()
  }

and scala s.Test says "J.foo()". The bytecode looks like this, as a result of the mangling in SuperAccessors. Perhaps this is as simple as identifying this situation during that phase and treating it like a super invocation?

public static void bar(s.S);
  Code:
   0:	aload_0
   1:	invokeinterface	SI-16,  1; //InterfaceMethod s/S.s$$S$$$$super$$foo:()V
   6:	return

[scabug]

@paulp said:
I just ran into this again. We should really figure out a way to at minimum make it a compile time error. VerifyErrors at runtime are poison.

[scabug]

Antony Stubbs (astubbs) said:
Been struggling with this for a few days now... Someone in the channel pointed me to this issue. At least I can stop banging my head against the wall...

Weird thing is. Idea runs the code just fine :? as retronym points out:
retronym: antonystubbs: After IDEA has compiled the .java file once, it might not be passed to scalac as a source file, but rather as a .class file.
[5:29pm] retronym: The bug only manifests when the .java and .scala files are processed in a single invocation of scalac.

[scabug]

@dragos said:
Partial fix in r22219. A compile-time error is issued, describing an implementation restriction.

[scabug]

@paulp said:
Noting for the record that the r22219 only fails at compile time if scala source is compiled against java bytecode. If both sources are given to scalac it still fails at runtime.

[scabug]

@hubertp said:
Another example of this bug in Scala plugin development:
http://www.assembla.com/code/scala-ide/git/changesets/e49527c46411dbafc3ef0fa712a00550ce7cf724
(search for 'SI-2296'). The problem manifests in runtime due to the mixture of Java/Scala libraries - the culprit is protected updateApplyButton so I had to add the forwarder to it, which is ugly.

[scabug]

Matthew Pocock (drdozer) said:
Just ran into this with 2.8.1 trying to access a protected field in a mix-in trait. I didn't notice any warnings from the compiler. I got the run-time IllegalAccessError symptom. It happens for me regardless of if I access the protected field via this, naked or through self:

trait SessionStore {
  self : GenericAutowireComposer =>

  // also tried this.session and session
  def getUser: Option[User] = Option(self.session.getAttribute("user").asInstanceOf[User]

}

class FooCtrl extends GenericAutowireComposer with SessionStore { ... }

At run-time, I get an IllegalArgumentException:

java.lang.IllegalAccessError: tried to access field org.zkoss.zk.ui.util.GenericAutowireComposer.session from class FooCtrl$$class
  at FooCtrl$$class.setUser

I can see that 'fixing' this so that it actually works could be a royal pain. However, generating bytecode that attempts to access protected fields in other classes can't be a Good Thing. Shouldn't this kind of access violation be caught by something in the compiler guts?

[scabug]

@paulp said:
Hubert, that link doesn't work anymore. Can you (or anyone) tell me if after f708b87e55 you still need to manually add a forwarder?

[scabug]

@paulp said:
Tentatively claiming fixed in f708b87e55 .

[scabug]

@hubertp said:
The commit I mentioned is here: scala-ide/scala-ide@e49527c#diff-2
In order for the ide people to be happy it would have to be ported to 2.9.x branch. Is that possible?

[scabug]

@paulp said:
Sure, once you or someone can confirm it fixes the manifestation(s) of the problem in the IDE.

[scabug]

@hubertp said:
Seems to be fine. I will ask Iulian to double check.

[scabug]

@hubertp said:
I think I have seen it again, but I failed to minimize it to a sensible code snippet. The general picture of my bug was:
bug/A.java

package bug;
class A {
  public void addAction(bug.action.Action a) {
     ...
  }
}

bug/Display.java

package bug;
class Display {
  protected A sth;
}

bug/action/Action.java

package bug.action;
public abstract class Action {
  protected A sth;
  public abstract void run();
}

and scala code
test/MyDisplay.scala

package test;

abstract class MyDisplay extends bug.Display {
  def init() {
    sth.addAction(ScalaAction)
  }

  object ScalaAction extends action.Action {
    def run() {
      sth. // call some method on A, crashes on runtime with illegal access
    }
  }
  
}

The surprising thing is that, if I make ScalaAction a class (and change addAction(...) to addAction(new ScalaAction())) this thing works again.
As I said, I have it reproducible but on a pretty large java and scala library. I will try to dig it more when I find some time.

[scabug]

@hubertp said:
Got it (I think this is the same bug but maybe I am wrong):
bug/Global.java

package bug;

import bug.action.Action;
import java.util.List;
import java.util.LinkedList;

public class Global {
  public int items() {
    return 0;
  }

  public int items(int i) {
    return i + ls.size();
  }

  private List<Action> ls = new LinkedList<Action>();

  public void putAction(Action a) {
    a.setGlobal(this);
    ls.add(a);
  }

  public void runActions() {
    for (Action action:  ls) {
      System.out.println("RUNNING ACTION");
      action.run(0);
    }
  }
}

bug/Display.java

package bug;

public class Display {
  protected Global m_glob;

  public void start() {
    m_glob.runActions();
  }
}

bug/action/Action.java

package bug.action;

import bug.Global;

public abstract class Action {
  protected Global m_glob;

  public Action(Global glob0) {
    m_glob = glob0;
  }
  
  public Action() {
    this(null);
  }

  public abstract void run(int v);

  public void setGlobal(Global g) {
    m_glob = g;
  }
}

test/ScalaActivity.scala

package test

import bug.Display
import bug.action.Action

abstract class Outer extends Display {

  def init() {
    m_glob.putAction(ScalaActivity)
  }

  object ScalaActivity extends Action {
    def run(v: Int) {
      val testSet = List(1,2,3)
      testSet.map(p => m_glob.items(p)) // crash with illegal access
    }
  }
}

test/Test.scala

package test

import bug.Global

object Test {
  def main(args: Array[String]) {
    val m = new Main()
    m.init()
    m.start()
  }
}

class Main extends Outer {
  m_glob = new Global()
}

Making ScalaAction a class instead of an object fixed the problem. Closure in run method is necessary to trigger the illegal access error.

[scabug]

@paulp said:
I can see this crash up to m2, but it works in m3, which makes me think the original fix was in. I checked in this as a test anyway in b0dd0452fd .

[scabug]

@dotta said:
Please, let us know the moment this is backported to the 2.9.x stream, as we have an open ticket on our issue tracker that is related to this issue: http://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001090

This is of course no high priority (it might as well never be backported).

[scabug]

@hubertp said:
Mirco, this has been backported long time ago (see merge in 2.9.x around 1st of June).

[scabug]

@dotta said (edited on Jun 19, 2012 5:03:48 PM UTC):
I see. Then it looks like another bug is lying around somewhere, as we can still see this issue in the IDE.

In short, I tried to merge your pull-request (scala-ide/scala-ide#80) but had to rollback right after because of http://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001083. If you want to investigate, I'd be delighted ;)

[scabug]

@hubertp said:
Paul, found a counterexample for you: #6245.