Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improve turbine cold-start performance #8006

Closed
benjaminp opened this issue Apr 11, 2019 · 10 comments
Closed

improve turbine cold-start performance #8006

benjaminp opened this issue Apr 11, 2019 · 10 comments
Labels
area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository not stale Issues or PRs that are inactive but not considered stale P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-Java Issues for Java rules type: feature request

Comments

@benjaminp
Copy link
Collaborator

It seems to me that using a persist worker for turbine Java header compilations would be beneficial for the same reasons as running javac as a persistent worker is. Has this been considered?
@aiuto aiuto added team-Local-Exec Issues and PRs for the Execution (Local) team team-Rules-Java Issues for Java rules untriaged and removed team-Local-Exec Issues and PRs for the Execution (Local) team labels Apr 17, 2019
@iirina
Copy link
Contributor

iirina commented May 31, 2019

cc: @cushon

@iirina iirina added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels May 31, 2019
@cushon
Copy link
Contributor

cushon commented May 31, 2019

I considered it, but decided to use a graalvm native-image for turbine instead. That approach has shown good startup performance without the overhead of additional workers.

@benjaminp
Copy link
Collaborator Author

I see. So maybe this issue is actually feature request to graal turbine in Bazel's java tools. :)

@cushon cushon changed the title persistent worker for turbine improve turbine cold-start performance May 31, 2019
@meisterT
Copy link
Member

meisterT commented Aug 4, 2020

cc @larsrc-google

@larsrc-google
Copy link
Contributor

That, or use multiplex workers.

@comius comius added area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed P2 We'll consider working on this in future. (Assignee optional) labels Nov 21, 2020
benjaminp referenced this issue in benjaminp/bazel Dec 2, 2020
@benjaminp
Allow G1 for javac workers.
36965ae 
G1 has a smoother memory profile than parallelGC, which is desirable for workers
that users run locally like javac.

Unfortunately, completely getting rid of the parallelGC option regresses Java
build performance. This is because Turbine does not run as a worker—see
https://github.com/bazelbuild/bazel/issues/8006—and G1 seems to do worse than
parallelGC for short-lived programs. So, I simply moved the parallelGC option to
the Turbine-specific jvm opts.

I ran bazel-bench to test the performance this change building //src:bazel_nojdk:
```
  metric          mean                median          stddev      pval
    wall:  475.842s  ( -0.05%)  474.997s  ( -0.09%)   1.203s     0.00000
     cpu:   73.063s  ( -2.28%)   72.980s  ( -2.21%)   1.499s     0.40000
  system:   19.427s  ( +0.66%)   19.450s  ( +1.09%)   0.111s     0.40000
  memory:   89.667MB ( -0.74%)   90.000MB ( +0.00%)   0.471MB    0.00000
```
@benjaminp benjaminp mentioned this issue Dec 2, 2020
Closed
bazel-io referenced this issue Dec 4, 2020
@benjaminp @copybara-github
Allow G1 for javac workers.
d1f6b01 
G1 has a smoother memory profile than parallelGC, which is desirable for workers
that users run locally like javac.

Unfortunately, completely getting rid of the parallelGC option regresses Java
build performance. This is because Turbine does not run as a worker—see
https://github.com/bazelbuild/bazel/issues/8006—and G1 seems to do worse than
parallelGC for short-lived programs. So, I simply moved the parallelGC option to
the Turbine-specific jvm opts.

I ran bazel-bench to test the performance this change building //src:bazel_nojdk:
```
  metric          mean                median          stddev      pval
    wall:  475.842s  ( -0.05%)  474.997s  ( -0.09%)   1.203s     0.00000
     cpu:   73.063s  ( -2.28%)   72.980s  ( -2.21%)   1.499s     0.40000
  system:   19.427s  ( +0.66%)   19.450s  ( +1.09%)   0.111s     0.40000
  memory:   89.667MB ( -0.74%)   90.000MB ( +0.00%)   0.471MB    0.00000
```

Closes #12598.

PiperOrigin-RevId: 3456162
@benjaminp benjaminp mentioned this issue Aug 29, 2022
Closed
@sgowroji sgowroji added the stale Issues or PRs that are stale (no activity for 30 days) label Feb 16, 2023
@sgowroji
Copy link
Member

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.

@sgowroji sgowroji closed this as not planned Won't fix, can't repro, duplicate, stale Feb 16, 2023
@meisterT meisterT removed the stale Issues or PRs that are stale (no activity for 30 days) label Feb 16, 2023
@meisterT
Copy link
Member

While there is no immediate work planned, I think this is something we want to keep open for the time being.

@fmeum
Copy link
Collaborator

fmeum commented Feb 16, 2023

@sgowroji Could you reopen?

@sgowroji sgowroji added the not stale Issues or PRs that are inactive but not considered stale label Feb 16, 2023
@sgowroji sgowroji reopened this Feb 16, 2023
@cushon cushon mentioned this issue Jun 8, 2023
Open
@benjaminp
Copy link
Collaborator Author

#19361

@fmeum
Copy link
Collaborator

fmeum commented Feb 7, 2024

Fixed by #19361, which makes turbine a native image. I have further performance improvements planned.

@fmeum fmeum closed this as completed Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-java-toolchains javabase, java_toolchain flags, JDK selection, java_toolchain rules, java_tools repository not stale Issues or PRs that are inactive but not considered stale P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Rules-Java Issues for Java rules type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@benjaminp @meisterT @cushon @iirina @comius @aiuto @lberki @fmeum @larsrc-google @sgowroji