Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelise JVM backend - Scala 2 port #15392

Merged
merged 2 commits into from
Oct 10, 2023

Conversation

WojciechMazur
Copy link
Contributor

@WojciechMazur WojciechMazur commented Jun 7, 2022

This PR ports scala/scala#6124 introduces backend parallelism, on top of the previously ported changes introduced in #15322
Adds Scala2 flags:

  • -Yjar-compression-level:N
  • -Ybackend-parallelism:N
  • -Ybackend-worker-queue:N

@WojciechMazur

This comment was marked as resolved.

@dottybot
Copy link
Member

dottybot commented Jun 7, 2022

performance test scheduled: 1 job(s) in queue, 1 running.

1 similar comment
@dottybot
Copy link
Member

dottybot commented Jun 7, 2022

performance test scheduled: 1 job(s) in queue, 1 running.

@dottybot
Copy link
Member

dottybot commented Jun 7, 2022

Performance test finished successfully:

Visit https://dotty-bench.epfl.ch/15392/ to see the changes.

Benchmarks is based on merging with main (76a0b29)

@WojciechMazur WojciechMazur force-pushed the port/scala-6124-parBackend branch 2 times, most recently from 25c997d to 6ed7b94 Compare March 28, 2023 20:01
@WojciechMazur WojciechMazur marked this pull request as ready for review March 28, 2023 20:03
@WojciechMazur WojciechMazur requested a review from sjrd March 28, 2023 20:03
@WojciechMazur WojciechMazur assigned sjrd and unassigned WojciechMazur Mar 29, 2023
Copy link
Member

@sjrd sjrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't say I have carefully reviewed everything, but it seems to be a reasonable enough port of the Scala 2 PR. I only have a few comments to make it a bit safer to evolve later.

compiler/src/dotty/tools/backend/jvm/CoreBTypes.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/backend/jvm/GenBCode.scala Outdated Show resolved Hide resolved
@WojciechMazur WojciechMazur added this to the 3.4.0 milestone Apr 11, 2023
@He-Pin
Copy link

He-Pin commented Sep 3, 2023

@WojciechMazur Will this feature be backported to the 3.3.2 release?

@WojciechMazur
Copy link
Contributor Author

Will this feature be backported to the 3.3.2 release?

I don't think it will or should be it backported. In the Scala 2 JVM backend contained additional postprocessing (optimizer, inliner) of the Bytecode which actually benefited from fact of its paralysation. However, currently in Scala 3 we would not see any reasonable performance gains as we would be able to only write generated classfiles in parallel - typically we would be limited by I/O, and not the CPU.
This might change when we'd add the bytecode optimizer, but the effort in this area has just began, so it probably would not target 3.3.x soon or at all.

@bishabosha bishabosha merged commit db7e033 into scala:main Oct 10, 2023
18 checks passed
@WojciechMazur WojciechMazur deleted the port/scala-6124-parBackend branch October 10, 2023 09:58
ornicar added a commit to lichess-org/lila that referenced this pull request Mar 15, 2024
scala/scala3#15392

before:
```
[success] Total time: 102 s (01:42), completed Mar 15, 2024, 8:03:47 AM
./lila 'clean;compile'  677.02s user 24.42s system 607% cpu 1:55.56 total
```
after:
```
[success] Total time: 99 s (01:39), completed Mar 15, 2024, 8:08:23 AM
./lila 'clean;compile'  652.09s user 23.96s system 628% cpu 1:47.53 total
```
WojciechMazur added a commit that referenced this pull request Jul 11, 2024
Backports #15392 to the LTS branch.

PR submitted by the release tooling.
[skip ci]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants