Improve performance of implicit search #422

jvican · 2016-05-09T12:59:30Z

These are the times we are looking for an implicit of type T in the implicit search for our test suite (which, btw, is huge):

Type	Times
Any	1420
String	1456
Int	1677
Ref	2338

These stats lay the ground for the upcoming improvement in the implicit search that hopefully will address the performance issues that we face right now, especially large compilation times. I think that there's room for improvement and I'm working tirelessly on it.

The text was updated successfully, but these errors were encountered:

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

@jsuereth

* Change all the implicit definitions to return `AbstractPicklerUnpickler` instead of `Pickler[T] with Unpickler[T]`. The former one has more precedence than the latter one. This avoids to call the generator macro EVERY TIME we ask for an implicit pickler/unpickler. * Make sure that the `LowPriorityImplicits` are placed in the right location of the `Defaults` trait. * This addresses a big deal of scala#422 although more changes will come, especially affecting `preferringAlternativeImplicits`. * Do some minor cleanup and formatting changes (trying out scalafmt) in the project. * Remove explicit registerings of picklers that were not necessary because of `AutoRegister`. * Register myself to the TODOs that are concerned with the creation of runtime picklers/unpicklers. @jsuereth, I've stolen you some TODOs ;)

jvican · 2016-05-09T21:56:02Z

Current:

Type	Times
Any	168
String	0
Int	0
Ref	0

I still have to figure out why Any is still being invoked through the macro generation. This hints that the compiler cannot find it in scope. This is probably caused by some test cases not importing AnyPicklerUnpickler from the whole Defaults (we have some tests that cherry-pick the imports) and therefore pickling has to create it from scratch.

My last changes (in my private repo) decrease the compilation time by a half (in my computer the speedup is ~3x). There are still some more improvements coming that will improve preferringAlternativeImplicits.

@jsuereth

* Change all the implicit definitions to return `AbstractPicklerUnpickler` instead of `Pickler[T] with Unpickler[T]`. The former one has more precedence than the latter one. This avoids to call the generator macro EVERY TIME we ask for an implicit pickler/unpickler. * Make sure that the `LowPriorityImplicits` are placed in the right location of the `Defaults` trait. * This addresses a big deal of scala#422 although more changes will come, especially affecting `preferringAlternativeImplicits`. * Do some minor cleanup and formatting changes (trying out scalafmt) in the project. * Remove explicit registerings of picklers that were not necessary because of `AutoRegister`. * Register myself to the TODOs that are concerned with the creation of runtime picklers/unpicklers. @jsuereth, I've stolen you some TODOs ;)

jvican · 2016-05-09T22:16:03Z

It also improves #342. I don't own a MBP, but in my Thinkpad x201 which is about 5 years old, the example in the issue consumes ~2s to compile. I think I could cut it even more with the upcoming changes.

jvican · 2016-05-13T13:50:47Z

I'm going to put off the next improvements in the implicit search for the next PR, since it's not high priority.

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

@jsuereth

* Change all the implicit definitions to return `AbstractPicklerUnpickler` instead of `Pickler[T] with Unpickler[T]`. The former one has more precedence than the latter one. This avoids to call the generator macro EVERY TIME we ask for an implicit pickler/unpickler. * Make sure that the `LowPriorityImplicits` are placed in the right location of the `Defaults` trait. * This addresses a big deal of scala#422 although more changes will come, especially affecting `preferringAlternativeImplicits`. * Do some minor cleanup and formatting changes (trying out scalafmt) in the project. * Remove explicit registerings of picklers that were not necessary because of `AutoRegister`. * Register myself to the TODOs that are concerned with the creation of runtime picklers/unpicklers. @jsuereth, I've stolen you some TODOs ;)

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

@jsuereth

* Change all the implicit definitions to return `AbstractPicklerUnpickler` instead of `Pickler[T] with Unpickler[T]`. The former one has more precedence than the latter one. This avoids to call the generator macro EVERY TIME we ask for an implicit pickler/unpickler. * Make sure that the `LowPriorityImplicits` are placed in the right location of the `Defaults` trait. * This addresses a big deal of scala#422 although more changes will come, especially affecting `preferringAlternativeImplicits`. * Do some minor cleanup and formatting changes (trying out scalafmt) in the project. * Remove explicit registerings of picklers that were not necessary because of `AutoRegister`. * Register myself to the TODOs that are concerned with the creation of runtime picklers/unpicklers. @jsuereth, I've stolen you some TODOs ;)

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

@jsuereth

* Change all the implicit definitions to return `AbstractPicklerUnpickler` instead of `Pickler[T] with Unpickler[T]`. The former one has more precedence than the latter one. This avoids to call the generator macro EVERY TIME we ask for an implicit pickler/unpickler. * Make sure that the `LowPriorityImplicits` are placed in the right location of the `Defaults` trait. * This addresses a big deal of scala#422 although more changes will come, especially affecting `preferringAlternativeImplicits`. * Do some minor cleanup and formatting changes (trying out scalafmt) in the project. * Remove explicit registerings of picklers that were not necessary because of `AutoRegister`. * Register myself to the TODOs that are concerned with the creation of runtime picklers/unpicklers. @jsuereth, I've stolen you some TODOs ;)

jvican added enhancement in progress benchmark labels May 9, 2016

jvican added a commit to jvican/pickling that referenced this issue May 9, 2016

Add benchmark utility that helps address scala#422

a471582

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

jvican mentioned this issue May 13, 2016

Improve implicit search drastically #424

Merged

jvican added a commit to jvican/pickling that referenced this issue May 23, 2016

Add benchmark utility that helps address scala#422

a1a9f4d

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

jvican added a commit to jvican/pickling that referenced this issue May 24, 2016

Add benchmark utility that helps address scala#422

4098b7c

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

jvican added a commit to jvican/pickling that referenced this issue May 24, 2016

Add benchmark utility that helps address scala#422

e4aef7e

* Add just a memo that will tell us how many times we are looking for the pickler/unpickler of a type `T`.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve performance of implicit search #422

Improve performance of implicit search #422

jvican commented May 9, 2016 •

edited

Loading

jvican commented May 9, 2016 •

edited

Loading

jvican commented May 9, 2016

jvican commented May 13, 2016

Improve performance of implicit search #422

Improve performance of implicit search #422

Comments

jvican commented May 9, 2016 • edited Loading

jvican commented May 9, 2016 • edited Loading

jvican commented May 9, 2016

jvican commented May 13, 2016

jvican commented May 9, 2016 •

edited

Loading

jvican commented May 9, 2016 •

edited

Loading