Support both Scala 2.11 and Scala 2.12 in performance evaluation #26
Comments
xeno-by
added a commit
to xeno-by/rsc
that referenced
this issue
Nov 24, 2017
This commit splits project `bench` into multiple projects: `benchJavac18`, `benchRsc`, `benchScalac211` and `benchScalac212`. This provides additional flexibility and fine-grained control over target compiler versions. The main motivation for this change is the fact that neither Scala 2.11 nor Scala 2.12 clearly dominate one another on re2s. Scala 2.11 is faster in cold mode, whereas Scala 2.12 is faster in hot mode. While refactoring the benchmarking infrastructure to accommodate this change, I've also retired CliBench. It is confusing to have multiple benchmarks that test the same thing (ColdXxxYyy and CliXxxYyy), so I picked the one that's simpler, i.e. ColdXxxYyy powered by JMH. RscNativeTypecheck is still powered by CliBench-like logic, but it has been merged directly into that particular benchmark. I am not completely happy with the astonishing degree of duplication between benchScalac211 and benchScalac212, but my sbt skills don't allow me to remove this duplication in a satisfying manner. (I'm aware of ++, but I want Scala versions to be part of the project name, not an optional prefix to an sbt invocation.)
xeno-by
added a commit
to xeno-by/rsc
that referenced
this issue
Nov 25, 2017
This commit splits project `bench` into multiple projects: `benchJavac18`, `benchRsc`, `benchScalac211` and `benchScalac212`. This provides additional flexibility and fine-grained control over target compiler versions. The main motivation for this change is the fact that neither Scala 2.11 nor Scala 2.12 clearly dominate one another on re2s. Scala 2.11 is faster in cold mode, whereas Scala 2.12 is faster in hot mode. While refactoring the benchmarking infrastructure to accommodate this change, I've also retired CliBench. It is confusing to have multiple benchmarks that test the same thing (ColdXxxYyy and CliXxxYyy), so I picked the one that's simpler, i.e. ColdXxxYyy powered by JMH. RscNativeTypecheck is still powered by CliBench-like logic, but it has been merged directly into that particular benchmark. I am not completely happy with the astonishing degree of duplication between benchScalac211 and benchScalac212, but my sbt skills don't allow me to remove this duplication in a satisfying manner. (I'm aware of ++, but I want Scala versions to be part of the project name, not an optional prefix to an sbt invocation.)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
By the looks of it, neither Scala 2.11 nor Scala 2.12 clearly dominate one another on re2s. Scala 2.11 is faster in cold mode, whereas Scala 2.12 is faster in hot mode. Let's include both in our benchmarks.
The text was updated successfully, but these errors were encountered: