build.sc exceeds JVM code size limits

[kongo2002]

Hi,

I just tried to migrate a rather large SBT project to mill. The experience is great so far! However I do run into problems now that I am about to add all project's modules (approx 100 modules).

I tried to move some methods and stuff around but now I cannot continue without running into errors like these:

Could not write class ammonite/$file/build$ because it exceeds JVM code size limits. Method ammonite/main/Router$Overrides's code too large!
Compilation Failed

Could not write class ammonite/$file/build$ because it exceeds JVM code size limits. Method null's code too large!
Compilation Failed

Could not write class ammonite/$file/build$ because it exceeds JVM code size limits. Method mill/util/Router$EntryPoint's code too large!
Compilation Failed

As you can see the method names do vary.

I will try to come up with a reproducible example but for now I can say the modules are at max 3 levels deep and are basically plain SbtModule instances:

object base extends Module {
  object sub extends Module {
    object inner1 extends SbtModule {
      def moduleDeps = Seq(some.other.module, some.more.module)
    }
    object inner2 extends SbtModule {
      def moduleDeps = Seq(base.sub.inner1, even.more.module)
    }
    // ...
  }
}

Cheers
Gregor

[lihaoyi]

Geez must be a pretty big Mill build! Could you try import $fileing a script file that sets the compiler flag -Ydelambdafy=inline to see if that avoids the issue for now? Here's the docs for how to configure compiler flags in .sc files:

http://ammonite.io/#CompilerFlags

You just need to put it in another conf.sc file next to build.sc, put the configureCompiler call in conf.sc, then put import $file.conf in your build.sc.

Not sure if this will solve your issue, but worth a shot

[kongo2002]

Thanks for the quick response! Sadly this does not resolve the issue:

Could not write class ammonite/$file/build$ because it exceeds JVM code size limits. Method millDiscover$lzycompute's code too large!
Compilation Failed

I've added this to the conf.sc like you mentioned:

interp.configureCompiler(_.settings.Ydelambdafy.tryToSetColon(List("inline")))

[lihaoyi]

Is it a different exception than before? Might still be progress even if it still doesn't work! Now we just need to break up millDiscover method rather than breaking up all the other methods your original thing reported

[kongo2002]

That's true indeed. I did try a couple of times and now it's always millDiscover$lzycompute in the exception.

[lihaoyi]

The "big" code for millDiscover all lives in this macro here

• https://github.com/lihaoyi/mill/blob/master/main/core/src/define/Discover.scala#L87

Maybe you could try chunking that big list of things into sub-lists of fixed size, and put some immediately-executing lambdas (() => ...)() around each chunk? Together with -Ydelambdafy inline, that should shove a bunch of the bytecode into separate methods/classes, and make the JVM bytecode limits happy

[lihaoyi]

there are probably other ways that code can be optimized to not generate so much stuff, but wrapping stuff in lambdas would be a nice short-term workaround and let us see if there are any other problems elsewhere

[kongo2002]

I am not sure if my scala macro skills are solid enough to fix this one by myself ;-)

However I was able to come up with a minimal reproducible example (simply put in an empty directory):

// build.sc
import $file.conf
import mill._, scalalib._

trait TModule extends SbtModule {
  def scalaVersion = "2.12.7"
}

object foo extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq(foo.common.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(foo.common.two)
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq(foo.common.one)
    }
    object two extends TModule {
      def moduleDeps = Seq(foo.domain.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(foo.domain.two)
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq(foo.domain.three)
    }
    object two extends TModule {
      def moduleDeps = Seq(foo.server.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(foo.server.two)
    }
  }
}

object bar extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq(foo.common.three)
    }
    object two extends TModule {
      def moduleDeps = Seq(bar.common.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(bar.common.two)
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq(foo.domain.three)
    }
    object two extends TModule {
      def moduleDeps = Seq(bar.domain.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(bar.domain.two)
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq(foo.server.one)
    }
    object two extends TModule {
      def moduleDeps = Seq(bar.server.one)
    }
    object three extends TModule {
      def moduleDeps = Seq(bar.server.two)
    }
  }
}

object ham extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq(bar.common.one)
    }
    object two extends TModule {
      def moduleDeps = Seq(bar.common.two)
    }
    object three extends TModule {
      def moduleDeps = Seq(bar.common.three)
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq(bar.domain.three)
    }
    object two extends TModule {
      def moduleDeps = Seq(bar.domain.two, ham.common.three)
    }
    object three extends TModule {
      def moduleDeps = Seq(bar.domain.two)
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}

object eggs extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}

object salt extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}

object pepper extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}

object oregano extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}

object rosmarin extends Module {
  object common extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object domain extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
  object server extends Module {
    object one extends TModule {
      def moduleDeps = Seq()
    }
    object two extends TModule {
      def moduleDeps = Seq()
    }
    object three extends TModule {
      def moduleDeps = Seq()
    }
  }
}