Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid double evaluation of pattern matchers in Chunk.collect #3364

Merged
merged 1 commit into from
Dec 23, 2023

Conversation

flipp5b
Copy link
Contributor

@flipp5b flipp5b commented Dec 20, 2023

fs2.Chunk#collect evaluates pattern matchers and guards of a passed partial function twice because it calls pf.isDefinedAt and then pf.apply:

// ...
foreach(o => if (pf.isDefinedAt(o)) b += pf(o))
// ...

Scala collections use pf.applyOrElse/pf.runWith (depending on scala version) instead to avoid double evaluation. This PR implements the same approach in a Chunk.

It should be noted that this may change observable behavior for those who use a partial function with side-effecting matchers/guards. But hey, side-effecting matchers/guards is considered a bad practice anyway, and I doubt that there's an fs2 user relying on that double evaluation "feature."

@flipp5b flipp5b force-pushed the optimize-chunk-collect branch from 7df3227 to 1c3ed3d Compare December 20, 2023 08:39
@@ -87,7 +87,8 @@ abstract class Chunk[+O] extends Serializable with ChunkPlatform[O] with ChunkRu
def collect[O2](pf: PartialFunction[O, O2]): Chunk[O2] = {
val b = makeArrayBuilder[Any]
b.sizeHint(size)
foreach(o => if (pf.isDefinedAt(o)) b += pf(o))
val f = pf.runWith(b += _)
foreach { o => f(o); () }
Copy link
Contributor Author

@flipp5b flipp5b Dec 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this looks rather ugly. pf.runWith returns O => Boolean so we have to transform it to O => Unit to avoid warnings. This could be rewritten as follows (at the cost of extra lambda call per Chunk item):

foreach(pf.runWith(b += _).andThen(_ => ()))

@armanbilge, which one would you prefer?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugly but performant is the right choice 😅

@flipp5b
Copy link
Contributor Author

flipp5b commented Dec 20, 2023

Here is some benchmark results.

1. Master

foreach(o => if (pf.isDefinedAt(o)) b += pf(o))
Benchmark               (chunkSize)   Mode  Cnt        Score       Error  Units
ChunkBenchmark.collect           16  thrpt   25  2828318.069 ± 72556.887  ops/s
ChunkBenchmark.collect          256  thrpt   25   191183.476 ±   581.751  ops/s
ChunkBenchmark.collect         4096  thrpt   25    11461.704 ±   504.370  ops/s

2. This PR

    val f = pf.runWith(b += _)
    foreach { o => f(o); () }
Benchmark               (chunkSize)   Mode  Cnt        Score       Error  Units
ChunkBenchmark.collect           16  thrpt   25  3373807.015 ± 25730.144  ops/s
ChunkBenchmark.collect          256  thrpt   25   257101.963 ±  7552.952  ops/s
ChunkBenchmark.collect         4096  thrpt   25    15417.548 ±   333.196  ops/s

3. This PR (prettified)

foreach(pf.runWith(b += _).andThen(_ => ()))
Benchmark               (chunkSize)   Mode  Cnt        Score       Error  Units
ChunkBenchmark.collect           16  thrpt   25  3420711.048 ± 90333.134  ops/s
ChunkBenchmark.collect          256  thrpt   25   242847.544 ±  4885.911  ops/s
ChunkBenchmark.collect         4096  thrpt   25    14369.581 ±   442.678  ops/s

Option 2 (this PR) shows slightly better score than the other two options.

@flipp5b flipp5b force-pushed the optimize-chunk-collect branch from 1c3ed3d to 4e146de Compare December 20, 2023 11:25
Copy link
Member

@armanbilge armanbilge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the very thorough PR, this is very nice work 🙏

@flipp5b
Copy link
Contributor Author

flipp5b commented Dec 20, 2023

@armanbilge Thank you for the prompt and kind feedback :)

@mpilquist mpilquist merged commit f5d3587 into typelevel:main Dec 23, 2023
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants