Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizations in escape hatch #898

Merged
merged 4 commits into from
Oct 19, 2018

Conversation

marcelocenerine
Copy link
Contributor

@marcelocenerine marcelocenerine commented Oct 7, 2018

This PR implements some optimizations in EscapeHatch (see details in #894):

  • short circuit the parsing logic when there is no suppression in a given file (each suppression method is short circuited individually). This avoids the cost of traversing the scalameta Tree.
  • Make atomic empty patches be considered empty. This will potentially maximize the cases where parts or the entire EscapeHatch is short circuited
  • A few other minor improvements

The tables below compare the running time of what is on master vs. this PR + scalameta/scalameta#1793. These figures are the median of 5 consecutive runs of Scalafix against the Spark code base (3201 files; 103181 lines). The setup is the same as explained here:

The rule utilized for this exercise is purposely simple to reduce noise and make EscapeHatch stand out. Different rules will give different results.

  • Without suppression:
Scalafix (total) EscapeHatch (total) :ok :on|:off @ filter
Before 50354ms 17454ms (34%) 10326ms (20%) 252ms (0.5%) 6795ms (13%) 0ms (0%)
After 26066ms 61ms (0%) 0ms (0%) 0ms (0%) 18ms (0%) 0ms (0%)
  • With suppression (all flavors put together in every source file):
Scalafix (total) EscapeHatch (total) :ok :on|:off @ filter
Before 55094ms 27479ms (50%) 9885ms (18%) 342ms (0.6%) 6470ms (11%) 10718ms (19%)
After 30598ms 11456ms (37%) 6330ms (20%) 356ms (1%) 2364ms (7%) 2362ms (7%)

I believe that most projects using Scalafix don't use suppressions often (only exceptionally as it should be). For those, the cost of EscapeHatch is now zero. Any source file containing suppressions only pay the price of the chosen suppression method.

As highlighted here, other parts of Scalafix got faster due to the improved tree traverser currently under review in Scalameta. Rules will also benefit from that.


if (hasDisabledPatch) EmptyPatch
else loop(name, underlying)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it shouldn't be necessary to check whether the underlying patches are disabled if it was already verified that the parent (AtomicPatch) contains a disabled patch. This should prevent some expensive tree traversals

}
private object SuppressWarningsArgs {
def unapply(mods: List[Mod]): Option[List[Term]] =
mods.collectFirst {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to traverse all elements since the annotation can only appear once

Copy link
Contributor

@olafurpg olafurpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @marcelocenerine this is a big contribution to Scalafix that I am sure many users with large codebases will love.

I have only one blocking comment that is unrelated to the optimizations: I would prefer to hardcode SuppressWarnings over classOf[SuppressWarnings].getSimpleName. There are cases when runtime reflection is warranted but I don't think it's necessary in this case.

def apply(tree: Tree, associatedComments: AssociatedComments): EscapeHatch =
def apply(
input: Input,
tree: => Tree,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: my experience with by-name is that it's easy to accidentally trigger the value. You can also use () => Tree or the LazyValue[T] in https://github.com/scalacenter/scalafix/blob/18650f572178ab1fa461846604bd82f9fed996ac/scalafix-core/src/main/scala/scalafix/internal/v1/LazyValue.scala which has the benefit of being a normal type and avoids accidental double computations. I prefer explicit tree.value or tree() to make it clearer when the function is called.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -180,17 +184,22 @@ object EscapeHatch {
}

private object AnnotatedEscapes {
private val SuppressWarnings = "SuppressWarnings"
private val SuppressWarnings = classOf[SuppressWarnings].getSimpleName
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? I would avoid runtime reflection when possible even if that means hardcoding a string literal, the name is not going to change in the JDK anytime soon anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough. Reverted

import scala.meta.contrib.AssociatedComments
import scala.meta.inputs.Input

class EscapeHatchSuite extends FunSuite {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍 nice tests

test("Patch.empty") {
val empty = Patch.empty
assert(empty.isEmpty)
assert(empty.atomic.isEmpty)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@olafurpg olafurpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Fantastic work @marcelocenerine Combined with scalameta/scalameta#1793 I am excited to try out the new speedups 😁

@olafurpg olafurpg merged commit 1f2cff7 into scalacenter:master Oct 19, 2018
@olafurpg
Copy link
Contributor

We can release scalafix v0.9.1 together with scalameta v4.1.0

@olafurpg olafurpg added this to the 0.9.1 milestone Nov 29, 2018
@marcelocenerine marcelocenerine deleted the optimized_escape_hatch branch November 23, 2023 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants