Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid use of Java Reflection (restructure part 5) #870

Merged
merged 61 commits into from
Apr 13, 2023
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
68678d3
v5.x-restructure: abstract from Java reflection
aslesarenko Oct 4, 2022
f142baf
v5.x-restructure: optimize reflection.scala
aslesarenko Oct 5, 2022
b5df752
v5.x-restructure: collect classes, fields and methods
aslesarenko Oct 5, 2022
94f8ea4
v5.x-restructure: introduce JavaImpl.scala and StaticImpl.scala
aslesarenko Oct 6, 2022
f52b936
v5.x-restructure: towards implementing ReflectionData.scala
aslesarenko Oct 6, 2022
f7e1cf1
v5.x-restructure: StaticImpl.scala first implementation
aslesarenko Oct 6, 2022
5bceb19
v5.x-restructure: create JRConstructors lazily
aslesarenko Oct 6, 2022
ef05f18
v5.x-restructure: removed overloadded methods of SigmaProp && and ||
aslesarenko Oct 6, 2022
ae72e2d
v5.x-restructure: removed all usages of java.lang.reflect
aslesarenko Oct 7, 2022
4d380d3
v5.x-restructure: register some simple SRClasses
aslesarenko Oct 7, 2022
bb8f592
v5.x-restructure: generate reflection constructors
aslesarenko Oct 9, 2022
26a34a9
v5.x-restructure: generate reflection methods
aslesarenko Oct 9, 2022
0697f81
v5.x-restructure: generate reflection methods (part 2)
aslesarenko Oct 12, 2022
ec38797
v5.x-restructure: more registerClassEntry
aslesarenko Oct 14, 2022
028f812
v5.x-restructure: more registerClassEntry (2)
aslesarenko Oct 15, 2022
20a1d2c
v5.x-restructure: move registrations to Reflection.scala
aslesarenko Oct 15, 2022
0630471
v5.x-restructure: move registrations to LibaryReflection.scala
aslesarenko Oct 15, 2022
062226d
v5.x-restructure: renamed ReflectionData -> CommonReflection
aslesarenko Oct 15, 2022
1b302f9
v5.x-restructure: removed library-api module
aslesarenko Oct 15, 2022
a234f7b
v5.x-restructure: removed sigma-api module
aslesarenko Oct 15, 2022
4f3bc44
v5.x-restructure: removed sigma-library module (2 tests fail)
aslesarenko Oct 15, 2022
5c3246e
v5.x-restructure: comment out reflection data for a while
aslesarenko Oct 16, 2022
4ec502d
v5.x-restructure: library-ir module renamed to graph-ir
aslesarenko Oct 16, 2022
f75de67
v5.x-restructure: core module merged into graph-ir
aslesarenko Oct 16, 2022
37f81e4
v5.x-restructure: library-impl module renamed to core-lib
aslesarenko Oct 16, 2022
3a15f51
v5.x-restructure: sigmastate module renamed to interpreter
aslesarenko Oct 16, 2022
0bd0e17
v5.x-restructure: Reflection object for each module
aslesarenko Oct 16, 2022
7c05056
v5.x-restructure: new generator for reflection data
aslesarenko Oct 16, 2022
2a15c9a
v5.x-restructure: reflection data for SigmaProp
aslesarenko Oct 17, 2022
ce8ea19
v5.x-restructure: reflection data for BigInt, CollBuilder + cleanups
aslesarenko Oct 17, 2022
5b29925
v5.x-restructure: cleanup PairColl and CollBuilder IR
aslesarenko Oct 17, 2022
bf259a3
v5.x-restructure: cleanup AnyValue from IR
aslesarenko Oct 17, 2022
a3d83b7
v5.x-restructure: cleanup WOptions + reflection for OptionWrapSpec
aslesarenko Oct 17, 2022
71ded79
v5.x-restructure: cleanup CollsImpl.scala + reflection for Coll
aslesarenko Oct 18, 2022
b84d40f
v5.x-restructure: reflection data
aslesarenko Oct 19, 2022
a278907
v5.x-restructure: reflection data for AvlTree + cleanup
aslesarenko Oct 19, 2022
4f391e3
v5.x-restructure: reflection data for Box, Context + cleanup
aslesarenko Oct 19, 2022
d0116bd
v5.x-restructure: reflection data for GroupElement, Header, PreHeader…
aslesarenko Oct 19, 2022
6c1e089
v5.x-restructure: reflection data for SigmaDslBuilder + cleanup
aslesarenko Oct 20, 2022
22e40de
v5.x-restructure: reflection data for WRType, WSpecialPredef + cleanup
aslesarenko Oct 20, 2022
a0bb16c
v5.x-restructure: cleanup WSpecialPredefs
aslesarenko Oct 20, 2022
8566262
v5.x-restructure: more cleanup in graph-ir
aslesarenko Oct 20, 2022
f4cca0c
v5.x-restructure: reflection data for CollElem
aslesarenko Oct 20, 2022
022fd86
v5.x-restructure: exclude GraphVizExport from Scalan cake
aslesarenko Oct 20, 2022
fb47766
v5.x-restructure: move GraphVizExport to Test scope
aslesarenko Oct 20, 2022
c1a97b4
v5.x-restructure: move BenchmarkUtil and ScalaNameUtil to Test scope
aslesarenko Oct 20, 2022
aa12d79
v5.x-restructure: move more classes from common to downstream modules
aslesarenko Oct 20, 2022
829ec1a
Merge remote-tracking branch 'origin/v5.0.6-RC' into restructure-part5
aslesarenko Mar 7, 2023
df11cb8
restructure-part5: fixes after merge
aslesarenko Mar 7, 2023
f965e41
restructure-part5: ScalaDoc for reflection + cleanups
aslesarenko Mar 8, 2023
55a7f93
restructure-part5: removed SpecialPredef.scala
aslesarenko Mar 8, 2023
a411a12
restructure-part5: more ScalaDocs
aslesarenko Mar 8, 2023
e328d15
restructure-part5: introduce printDebug in tests
aslesarenko Mar 8, 2023
a15daf8
restructure-part5: make public compileWithCosting
aslesarenko Mar 10, 2023
0b134b1
restructure-part5: scrypto 2.3.0
aslesarenko Mar 15, 2023
b21dea1
restructure-part5: Platform for reflection
aslesarenko Mar 17, 2023
8dde9c0
restructure-part5: TrieMap for Dupers
aslesarenko Mar 17, 2023
3cd3ec4
restructure-part5: ScalaDocs and cleanups
aslesarenko Mar 17, 2023
71a93af
restructure-part5: addressing review comments
aslesarenko Apr 11, 2023
1eb35c9
restructure-part5: ScalaDoc for ZLastSpec
aslesarenko Apr 11, 2023
c68759c
restructure-part5: comment out debug code
aslesarenko Apr 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
99 changes: 34 additions & 65 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,27 @@ dynverSonatypeSnapshots in ThisBuild := true
dynverSeparator in ThisBuild := "-"

val bouncycastleBcprov = "org.bouncycastle" % "bcprov-jdk15on" % "1.66"
val scrypto = "org.scorexfoundation" %% "scrypto" % "2.3.0-RC1"
val scrypto = "org.scorexfoundation" %% "scrypto" % "2.3.0"
val scorexUtil = "org.scorexfoundation" %% "scorex-util" % "0.2.0"
val debox = "org.scorexfoundation" %% "debox" % "0.10.0"
val spireMacros = "org.typelevel" %% "spire-macros" % "0.17.0-M1"
val fastparse = "com.lihaoyi" %% "fastparse" % "2.3.3"
val scalaCompat = "org.scala-lang.modules" %% "scala-collection-compat" % "2.7.0"

lazy val circeCore211 = "io.circe" %% "circe-core" % "0.10.0"
lazy val circeGeneric211 = "io.circe" %% "circe-generic" % "0.10.0"
lazy val circeParser211 = "io.circe" %% "circe-parser" % "0.10.0"

lazy val circeCore = "io.circe" %% "circe-core" % "0.13.0"
lazy val circeGeneric = "io.circe" %% "circe-generic" % "0.13.0"
lazy val circeParser = "io.circe" %% "circe-parser" % "0.13.0"

def circeDeps(scalaVersion: String) = if (scalaVersion == scala211)
Seq(circeCore211, circeGeneric211, circeParser211)
else
Seq(circeCore, circeGeneric, circeParser)


val testingDependencies = Seq(
"org.scalatest" %% "scalatest" % "3.2.14" % Test,
"org.scalactic" %% "scalactic" % "3.2.14" % Test,
Expand All @@ -101,19 +115,7 @@ libraryDependencies ++= Seq(
scorexUtil,
"org.bouncycastle" % "bcprov-jdk15on" % "1.+",
fastparse, debox, spireMacros, scalaCompat
) ++ testingDependencies ++
(if (scalaVersion.value == scala211)
Seq(circeCore211, circeGeneric211, circeParser211)
else
Seq(circeCore, circeGeneric, circeParser))

lazy val circeCore211 = "io.circe" %% "circe-core" % "0.10.0"
lazy val circeGeneric211 = "io.circe" %% "circe-generic" % "0.10.0"
lazy val circeParser211 = "io.circe" %% "circe-parser" % "0.10.0"

lazy val circeCore = "io.circe" %% "circe-core" % "0.13.0"
lazy val circeGeneric = "io.circe" %% "circe-generic" % "0.13.0"
lazy val circeParser = "io.circe" %% "circe-parser" % "0.13.0"
) ++ testingDependencies ++ circeDeps(scalaVersion.value)

scalacOptions ++= Seq("-feature", "-deprecation")

Expand Down Expand Up @@ -152,76 +154,43 @@ lazy val common = Project("common", file("common"))
))
.settings(publish / skip := true)

lazy val libraryapi = Project("library-api", file("library-api"))
lazy val corelib = Project("core-lib", file("core-lib"))
.dependsOn(common % allConfigDependency)
.settings(libraryDefSettings,
libraryDependencies ++= Seq())
.settings(publish / skip := true)

lazy val libraryimpl = Project("library-impl", file("library-impl"))
.dependsOn(libraryapi % allConfigDependency)
.settings(libraryDefSettings,
libraryDependencies ++= Seq( debox ))
libraryDependencies ++= Seq( debox, scrypto ))
.settings(publish / skip := true)

lazy val core = Project("core", file("core"))
.dependsOn(common % allConfigDependency, libraryapi % allConfigDependency)
.settings(libraryDefSettings,
libraryDependencies ++= Seq( debox ))
.settings(publish / skip := true)

lazy val library = Project("library", file("library"))
.dependsOn(common % allConfigDependency, core % allConfigDependency, libraryapi, libraryimpl)
lazy val graphir = Project("graph-ir", file("graph-ir"))
.dependsOn(common % allConfigDependency, corelib)
.settings(
libraryDefSettings,
libraryDependencies ++= Seq( debox ))
libraryDependencies ++= Seq( debox, scrypto, bouncycastleBcprov ))
.settings(publish / skip := true)

lazy val sigmaapi = Project("sigma-api", file("sigma-api"))
.dependsOn(common, libraryapi)
.settings(libraryDefSettings,
libraryDependencies ++= Seq(
scrypto, bouncycastleBcprov
))
.settings(publish / skip := true)

lazy val sigmalibrary = Project("sigma-library", file("sigma-library"))
.dependsOn(
common % allConfigDependency,
core % allConfigDependency,
libraryapi % allConfigDependency,
libraryimpl % allConfigDependency,
library % allConfigDependency,
sigmaapi % allConfigDependency)
.settings(libraryDefSettings,
libraryDependencies ++= Seq(
scrypto,
bouncycastleBcprov
))
lazy val interpreter = (project in file("interpreter"))
.dependsOn(graphir % allConfigDependency)
.settings(libraryDefSettings)
.settings(libraryDependencies ++=
Seq(scorexUtil, fastparse) ++ circeDeps(scalaVersion.value)
)
.settings(publish / skip := true)

lazy val sigmastate = (project in file("sigmastate"))
.dependsOn(sigmalibrary % allConfigDependency)
lazy val sc = (project in file("sc"))
.dependsOn(graphir % allConfigDependency, interpreter % allConfigDependency)
.settings(libraryDefSettings)
.settings(libraryDependencies ++= Seq(
scorexUtil, fastparse,
if (scalaVersion.value == scala211) circeCore211 else circeCore,
if (scalaVersion.value == scala211) circeGeneric211 else circeGeneric,
if (scalaVersion.value == scala211) circeParser211 else circeParser
))
.settings(libraryDependencies ++=
Seq(scorexUtil, fastparse) ++ circeDeps(scalaVersion.value)
)
.settings(publish / skip := true)

lazy val sigma = (project in file("."))
.aggregate(
sigmastate, common, core, libraryapi, libraryimpl, library,
sigmaapi, sigmalibrary)
.aggregate(common, corelib, graphir, interpreter, sc)
.settings(libraryDefSettings, rootSettings)
.settings(publish / aggregate := false)
.settings(publishLocal / aggregate := false)

lazy val aggregateCompile = ScopeFilter(
inProjects(common, core, libraryapi, libraryimpl, library, sigmaapi,
sigmalibrary, sigmastate),
inProjects(common, corelib, graphir, interpreter, sc),
inConfigurations(Compile))

lazy val rootSettings = Seq(
Expand Down
3 changes: 0 additions & 3 deletions common/src/main/scala/scalan/Typeclass.scala

This file was deleted.

1 change: 0 additions & 1 deletion common/src/main/scala/scalan/package.scala
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@

import scala.language.experimental.macros
import scala.reflect.ClassTag

package object scalan {
Expand Down
70 changes: 70 additions & 0 deletions common/src/main/scala/scalan/reflection/CommonReflection.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package scalan.reflection

import scala.collection.compat.immutable.ArraySeq
import scala.collection.mutable
import scala.collection.immutable

/** Reflection metadata and global dictionaries to access it. */
object CommonReflection {
/** Descriptors of classes. */
val classes = mutable.HashMap.empty[Class[_], SRClass[_]]

/** Registers a class entry in the map of classes.
*
* @tparam T the type of the class to be registered
* @param clazz the class to be registered
* @param constructors the constructors of the class
* @param fields the fields of the class
* @param methods the methods of the class (represented as a map of method names and argument types to the corresponding RMethod instances)
*/
def registerClassEntry[T](clazz: Class[T],
constructors: Seq[SRConstructor[_]] = ArraySeq.empty,
fields: Map[String, SRField] = Map.empty,
methods: Map[(String, Seq[Class[_]]), RMethod] = Map.empty): Unit = classes.synchronized {
classes.put(clazz, new SRClass(clazz, constructors, fields, methods))
}

registerClassEntry(classOf[Boolean])

registerClassEntry(classOf[Byte])

registerClassEntry(classOf[Short])

registerClassEntry(classOf[Int])

registerClassEntry(classOf[Long])

registerClassEntry(classOf[Product2[_, _]])

registerClassEntry(classOf[immutable.$colon$colon[_]],
constructors = Array(
mkConstructor(Array(classOf[java.lang.Object], classOf[immutable.List[_]])) { args =>
new immutable.$colon$colon(args(0).asInstanceOf[java.lang.Object], args(1).asInstanceOf[immutable.List[_]])
}
)
)

{ val clazz = classOf[scala.Option[_]]
registerClassEntry(clazz,
methods = Map(
mkMethod(clazz, "filter", Array(classOf[scala.Function1[_,_]])) { (obj, args) =>
obj.asInstanceOf[Option[Any]].filter(args(0).asInstanceOf[Any => Boolean])
},
mkMethod(clazz, "map", Array(classOf[scala.Function1[_,_]])) { (obj, args) =>
obj.asInstanceOf[Option[Any]].map(args(0).asInstanceOf[Any => Any])
}
)
)
}

{ val clazz = classOf[scala.Some[_]]
registerClassEntry(clazz,
constructors = Array(
mkConstructor(Array(classOf[java.lang.Object])) { args =>
new scala.Some(args(0).asInstanceOf[java.lang.Object])
}
)
)
}

}
159 changes: 159 additions & 0 deletions common/src/main/scala/scalan/reflection/JavaImpl.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
package scalan.reflection

import debox.cfor
import scalan.reflection.memoize

import java.lang.reflect.{Field, Constructor, Method}
import scala.collection.concurrent.TrieMap
import scala.collection.mutable

/**
* A class that represents a Java class of type `T`.
*
* @constructor creates a new instance of `JRClass` with the given value.
* @param value the Java class of type `T`.
*/
class JRClass[T](val value: Class[T]) extends RClass[T] {

/** A mutable map that stores the fields of this class. */
val fields = TrieMap.empty[String, RField]

override def getField(name: String): RField =
memoize(fields)(name, JRField(value.getField(name)))

/** A mutable map that stores the methods of this class. */
val methods = TrieMap.empty[(String, Seq[Class[_]]), RMethod]

override def getMethod(name: String, parameterTypes: Class[_]*): RMethod = {
memoize(methods)((name, parameterTypes), JRMethod(this, value.getMethod(name, parameterTypes:_*)))
}

override def getSimpleName: String = value.getSimpleName
override def getName: String = value.getName

/** A sequence that stores the constructors of this class. */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just "Constructors of this class" ? see https://docs.scala-lang.org/style/scaladoc.html#general-style

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

var constructors: Seq[RConstructor[_]] = _

override def getConstructors(): Seq[RConstructor[_]] = {
if (constructors == null) {
synchronized {
if (constructors == null) {
val cs = value.getConstructors.asInstanceOf[Array[Constructor[Any]]]
val buf = mutable.ArrayBuilder.make[RConstructor[Any]]
cfor(0)(_ < cs.length, _ + 1) { i =>
val c = cs(i)
buf += JRConstructor[Any](i, c)
}
constructors = buf.result().asInstanceOf[Array[RConstructor[_]]]
}
}
}
constructors
}

/** Helper method that returns a sequence of `JRConstructor` objects that were at least
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"a sequence of" seems to be excessive here as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

* once used at runtime.
*/
def getUsedConstructors(): Seq[JRConstructor[_]] =
getConstructors().collect { case c: JRConstructor[_] if c.wasUsed => c }

override def isPrimitive(): Boolean = value.isPrimitive

override def getSuperclass(): RClass[_ >: T] = RClass(value.getSuperclass)

override def isAssignableFrom(cls: Class[_]): Boolean = value.isAssignableFrom(cls)

override def getDeclaredMethods(): Array[RMethod] = value.getDeclaredMethods.map(JRMethod(this, _))

override def equals(other: Any): Boolean = (this eq other.asInstanceOf[AnyRef]) || (other match {
case that: JRClass[_] =>
val eq = value == that.value
if (!eq)
assert(this.getName != that.getName) // sanity check
eq
case _ => false
})

override def hashCode(): Int = value.hashCode()

override def toString: String = s"JRClass(${value.getName})"
}


/** Implements [[RField]] using Java reflection.
*
* @param value The [[java.lang.reflect.Field]] object to wrap.
*/
class JRField private (val value: Field) extends RField {
override def getType: Class[_] = value.getType

override def equals(other: Any): Boolean = (this eq other.asInstanceOf[AnyRef]) || (other match {
case that: JRField => value == that.value
case _ => false
})
override def hashCode(): Int = value.hashCode()
override def toString: String = s"JRField($value)"
}
object JRField {
private[reflection] def apply(field: Field): RField = new JRField(field)
}

/** Implements [[RConstructor]] using Java reflection.
*
* @tparam T The type of the class that declares this constructor.
* @param index The index of the constructor in the sequence of all constructors of the class.
* @param value The [[java.lang.reflect.Constructor]] to be wrapped.
*/
class JRConstructor[T] private (val index: Int, val value: Constructor[T]) extends RConstructor[T] {
@volatile var wasUsed: Boolean = false
override def newInstance(initargs: AnyRef*): T = {
wasUsed = true
value.newInstance(initargs:_*)
}
override def getParameterTypes(): Array[Class[_]] = {
wasUsed = true
value.getParameterTypes
}

override def equals(other: Any): Boolean = (this eq other.asInstanceOf[AnyRef]) || (other match {
case that: JRConstructor[_] => value == that.value
case _ => false
})
override def hashCode(): Int = value.hashCode()
override def toString: String = s"JRConstructor($index, $value)"
}
object JRConstructor {
private[reflection] def apply[T](index: Int, value: Constructor[T]): RConstructor[T] = new JRConstructor[T](index, value)
}

/**
* Implements [[RMethod]] using Java reflection.
*
* @param declaringClass The JRClass that declares this method.
* @param value The [[java.lang.reflect.Method]] instance that this JRMethod represents.
*/
class JRMethod private (declarigClass: JRClass[_], val value: Method) extends RMethod {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

declaringClass

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

override def invoke(obj: Any, args: AnyRef*): AnyRef = {
val name = value.getName
val parameterTypes: Seq[Class[_]] = value.getParameterTypes
memoize(declarigClass.methods)((name, parameterTypes), this)
value.invoke(obj, args:_*)
}

override def getName: String = value.getName

override def getDeclaringClass(): Class[_] = value.getDeclaringClass

override def getParameterTypes(): Seq[Class[_]] = value.getParameterTypes

override def equals(other: Any): Boolean = (this eq other.asInstanceOf[AnyRef]) || (other match {
case that: JRMethod => value == that.value
case _ => false
})

override def hashCode(): Int = value.hashCode()
override def toString: String = s"JRMethod($value)"
}
object JRMethod {
private[reflection] def apply(clazz: JRClass[_], value: Method): RMethod = new JRMethod(clazz, value)
}
Loading