Accessing nested objects from java

[prolativ]

Not sure if this is simply a bug or for some reason it was designed that way but accessing nested objects from java code is very tricky.

Having such scala code

object Foo {
  def foo = println("foo")

  object Bar {
    def bar = println("bar")
  }
}

one can call foo from java simply as Foo.foo(). So one could expect that bar can be similarly called as Foo.Bar.bar(). Unfortunately this doesn't work. The workaround that I've found is not very intuitive: (new Foo.Bar$()).bar().
In case of foo this works because we synthesize static forwarders for methods defined in top level objects. The behaviour is common with scala 2 and I've seen this discussion scala/scala-dev#573 regarding whether generating static forwarders is the best solution but it didn't address accessing nested objects. Doing it like this new Foo.Bar$() is not only slightly verbose and unintuitive (a java programmer using an API defined in scala needs to know how scala objects are represented in JVM) but even misleading (normally new should return a new instance of a class but here the same reference is returned all the time - which can be seen if you introduce some mutability into the object).

Should we then generate static forwarders for accessing inner objects so that they can be accessed from java like Foo.Bar
or at least Foo.Bar()? I guess generating static forwarders directly for methods of a nested object is not an option, at least if the object is defined inside a class, which can have multiple instances.
@smarter @odersky any ideas?

[smarter]

(new Foo.Bar$()).bar()

Making a new instance of an object is a bad idea since everything assumes that it's a singleton. It also shouldn't be needed: one can access the singleton through the MODULE$ field: Foo.Bar$.MODULE$.bar() should work.

Should we then generate static forwarders for accessing inner objects so that they can be accessed from java like Foo.Bar
or at least Foo.Bar()?

Scala 2 never did that and we just copied that behavior, since this is such a long-established behavior, changing it would require a very careful analysis of the impact of that change (and digging up past discussions on this topic in Scala 2).

[prolativ]

Indeed Foo.Bar$.MODULE$ is a slightly nicer workaround than new Foo.Bar$() (actually I'm quite surprised the latter works at all as the constructor of Bar$ seems to be private in the bytecode).
However this still seems quite weird. If someone wanted to call foo directly rather than using the static forwarder, the invocation would look like Foo$.MODULE$.foo().
But if there was one more level of nesting

object Foo {
  def foo = println("foo")

  object Bar {
    def bar = println("bar")

    object Baz {
      def baz = println("baz")
    }
  }
}

how can one invoke baz?
Following the pattern
Foo$.MODULE$.foo()
Foo.Bar$.MODULE$.bar()
I would expect
Foo.Bar.Baz$.MODULE$.baz()
but this gives a compilation error

[error] cannot find symbol
[error]     symbol:   variable Bar
[error]    location: class Foo
[error] Foo.Bar

which is somehow understandable as there is no such generated JVM class as Foo.Bar - there's only Foo.Bar$ mapping to Foo$Bar$.class.
However
Foo.Bar$.Baz$.MODULE$.baz()
doesn't compile either:

[error] cannot access Foo.Bar$.Baz$
[error]     class file for Foo$Bar$$Baz$ not found
[error] Foo.Bar$.Baz$

The existing class file is Foo$Bar$Baz$.class.

Is this a bug then? I'm getting the same errors for scala 2.

[smarter]

However Foo.Bar$.Baz$.MODULE$.baz() doesn't compile either: [...] I'm getting the same errors for scala 2.

This goes beyond my knowledge of inner classes, maybe @lrytz can help? At least in Scala 2 the InnerClasses table of Foo.Bar$ seems to list Baz$ with the correct bytecode class name:

InnerClasses:
     public static lampepfl/dotty#8= lampepfl/dotty#2 of lampepfl/dotty#7; //Bar$=class Foo$Bar$ of class Foo
     public static lampepfl/dotty#11= lampepfl/dotty#10 of lampepfl/dotty#2; //Baz$=class Foo$Bar$Baz$ of class Foo$Bar$

But maybe javac doesn't rely on this information? I have no idea.

[smarter]

As a workaround, one can write Foo$Bar$Baz$.MODULE$ in Java.

[prolativ]

The problem seems to be explained here https://stackoverflow.com/questions/30809070/accessing-scala-nested-classes-from-java
Interestingly Foo.Bar$.MODULE$.Baz.baz() also compiles but throws a NPE

[lrytz]

Some background here...
https://github.com/scala/scala/blob/cf8680d1e2212b8dc2e3f5528d63f43ec521e9d6/src/compiler/scala/tools/nsc/backend/jvm/BTypes.scala#L492-L514

Basically the recommendation is to make nested objects accessible to Java by adding explicit getters for them in the Scala source. It's hard enough to generate the (mandatory) InnerClasses table and have it adhere to the spec, I don't know if we could change it in a valid way to improve Java interop here.

[lrytz]

Interestingly Foo.Bar$.MODULE$.Baz.baz() also compiles but throws a NPE

That's interesting, comparing the bytecode generated by 2.13.5 and dotc:

-public final class p/Foo$ {
+public final class p/Foo$ implements java/io/Serializable  {

+  // access flags 0x19
+  public final static Lp/Foo$Bar$; Bar

-public class p/Foo$Bar$ {
+public final class p/Foo$Bar$ implements java/io/Serializable  {

+  // access flags 0x19
+  public final static Lp/Foo$Bar$Baz$; Baz

what are those fields for?

[smarter]

Thanks, so if I understand this correctly, to be able to reference Foo.Bar.Baz from java we'd need to generate a nested mirror class Bar in Foo and put Baz$ in its InnerClass table, which brings us back to @prolativ's original question: do you know if there is any particular reason (besides complexity of the implementation) for only having mirror classes for top-level objects and not nested objects?

[lrytz]

No, I don't know... My guess is it's about the tradeoff between complexity of the implementation and the value it would provide.

[liufengyun]

A workaround is to use @static in Dotty:

// Foo.scala
object Foo {
  def foo = println("foo")

  class Bar

  object Bar {
    @scala.annotation.static
    def bar = println("bar")
  }
}

// Test.java
class Test {
  void foo() { Foo.Bar.bar(); }
}

[prolativ]

@liufengyun this doesn't seem to scale with more levels of nesting

// Foo.scala
object Foo {
  def foo = println("foo")

  object Bar {
    def bar = println("bar")

    class Baz

    object Baz {
      @scala.annotation.static
      def baz = println("baz")
    }
  }
}

public class Test {
  public static void main(String[] args) {
    Foo.Bar.Baz.baz();
  }
}

[error] Test.java:3:1: cannot find symbol
[error]   symbol:   variable Bar
[error]   location: class Foo
[error] Foo.Bar

[liufengyun]

@prolativ Indeed, @static does not solve the fundamental problem.

For any non-top level static object A enclosed in object B, if we synthesize a proxy field var A inside B, then it might be able to solve the interop problem.

[prolativ]

@liufengyun yeah, it was my initial intuition that synthesizing proxies should solve the problem.
By emphasizing static you mean that there are some situations when synthesizing a proxy to an object nested inside a top level object could cause some problems? And in general we would also need proxies to objects nested in non-top-level objects. And there's also the problem of objects nested in classes although I guess the issue would get slightly more complicated here.

[liufengyun]

By emphasizing static you mean that there are some situations when synthesizing a proxy to an object nested inside a top level object could cause some problems?

By "static object" I mean objects that are not nested inside any class (thus no need for outer).

And in general we would also need proxies to objects nested in non-top-level objects.

Yes, static objects could be nested deeply.

there's also the problem of objects nested in classes

They are not static, thus can be ignored for the moment.

There might be issues with initialization ordering. But it's worth trying and see how it works.