Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update embedded JDK to 11 to fix a host of docker related issues #6592

Closed
Globegitter opened this issue Nov 5, 2018 · 6 comments
Closed

Update embedded JDK to 11 to fix a host of docker related issues #6592

Globegitter opened this issue Nov 5, 2018 · 6 comments
Assignees
@meisterT
Labels
area-EngProd Bazel CI, infrastructure, bootstrapping, release, and distribution tooling P2 We'll consider working on this in future. (Assignee optional) team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website team-Rules-Java Issues for Java rules type: feature request

Comments

@Globegitter
Copy link

Description of the problem / feature request:

As far as I can tell bazel is currently coming embedded with openjdk 9 and it is having issues with easily running out of memory in docker containers (most prominently CircleCI it seems), or more generically not respecting cgroups for resource limits.

issues I could find that seem releated:
#3886
#4616
#5180
#3645
#5389

bazelbuild/rules_docker#520

Feature requests: what underlying problem are you trying to solve with this feature?

It turns out OpenJDK 10 comes with a host of fixes that would improve that situation and for us also just seemed to have solved an issue with using CircleCI. I installed bazel pointing at jdk 11 using startup --server_javabase /usr/lib/jvm/java-11-openjdk-amd64/" in the bazelrc, removed any flags around local_resources and builds that would continuously fail before are now working, in fact I had a build that would even fail with local_resources and -Xmx3072m set, but pointing bazel to use jdk 11 fixed that.

Here the links to the openjdk issues:
https://bugs.openjdk.java.net/browse/JDK-8196595
https://bugs.openjdk.java.net/browse/JDK-8146115
https://bugs.openjdk.java.net/browse/JDK-8170888
https://bugs.openjdk.java.net/browse/JDK-8140793

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Probably not that easy to reproduce, but I know rules_nodejs use circleci and had similar issues in the past that where solved with setting the same flags here: https://github.com/bazelbuild/rules_nodejs/blob/master/.circleci/bazel.rc#L17

So maybe they would be a good test case, trying to remove the flags and see if the situation improves. cc @alexeagle

What operating system are you running Bazel on?

Ubuntu 18.04, the docker images are running on debian stretch

What's the output of bazel info release?

release 0.19.0

Have you found anything relevant by searching the web?

Yep, see issues linked above.
@jin jin added type: feature request area-EngProd Bazel CI, infrastructure, bootstrapping, release, and distribution tooling untriaged labels Nov 6, 2018
@meisterT
Copy link
Member

meisterT commented Nov 7, 2018

Cc @lberki @iirina

bazel-io pushed a commit that referenced this issue Nov 30, 2018
@meisterT
Remove --incompatible_use_remotejdk_as_host_javabase flag.
f195d55 
Now we always use the remote JDK if host javabase is not explicitly set (and if
we need a JDK at all).

Progress on #6656 and #6592 and #6313.

RELNOTES: None
PiperOrigin-RevId: 223524214
@Monnoroch
Copy link

Related: #6661.

@meisterT
Copy link
Member

meisterT commented Dec 6, 2018

@Monnoroch There's no requirement anymore to have the same embedded JDK version as we use for the host javabase so #6661 is talking about updating the JDK version, it's a different JDK.

Independently, we also plan to upgrade the embedded JDK after we have submitted some other changes and hopefully it'll land before we cut 0.22.

If I understand the new java container support correctly, you probably still want to set a JVM flag in the container. The JVM uses by default 1/4 of the physical available memory for the heap. In a container, you may have less memory available than what's physically in your machine. Now, if the heap grows too large, Bazel gets killed. With the new container support, the "physical available memory" is limited to the memory available in the container. Thus, the heap will not grow too large, but it's using 1/4 of the available memory. That may not be enough and you might want to set MaxRamFraction in your CI environment.

@davido davido mentioned this issue Jan 7, 2019
Closed
3 tasks
@davido
Copy link
Contributor

davido commented Jan 8, 2019

Independently, we also plan to upgrade the embedded JDK after we have submitted some other changes and hopefully it'll land before we cut 0.22.

I tried 0.22rc2 yesterday and it is still using JDK 9.

@meisterT meisterT added P2 We'll consider working on this in future. (Assignee optional) team-Rules-Java Issues for Java rules and removed untriaged labels Jan 8, 2019
@meisterT
Copy link
Member

meisterT commented Jan 8, 2019

David, that's correct. I wasn't able to properly test and land the changes mostly because of holidays. I'll definitely do it before 0.23.

@davido
Copy link
Contributor

davido commented Jan 8, 2019

@meisterT Thanks. Let me know, if you have a commit somewhere. I would be happy to build Bazel from scratch and test it against Gerrit Code Review and related projects.

meisterT added a commit to meisterT/bazel that referenced this issue Jan 18, 2019
@meisterT
Add //src:bazel-bin_dev_jdk target.
09aac9c 
This builds the minimal JDK locally and uses it as embedded JDK.
We typically use the cached one on mirror.bazel.build, because the jlink
is heavily non-determistic. However, this will make local testing
easier, e.g. when we do upgrades from one JDK to another.

Progress on bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None
@meisterT meisterT mentioned this issue Jan 18, 2019
Closed
laszlocsomor pushed a commit to laszlocsomor/bazel that referenced this issue Jan 18, 2019
@meisterT
Add //src:bazel-bin_dev_jdk target.
5d2d06c 
This builds the minimal JDK locally and uses it as embedded JDK.
We typically use the cached one on mirror.bazel.build, because the jlink
is heavily non-determistic. However, this will make local testing
easier, e.g. when we do upgrades from one JDK to another.

Progress on bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None

Closes bazelbuild#7175.

PiperOrigin-RevId: 229912262
meisterT added a commit to meisterT/bazel that referenced this issue Jan 24, 2019
@meisterT
Vendor jaxb library in our third_party.
d5178a8 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None
@meisterT meisterT mentioned this issue Jan 24, 2019
Closed
meisterT added a commit to meisterT/bazel that referenced this issue Jan 24, 2019
@meisterT
Vendor jaxb library in our third_party.
ab644b7 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None
meisterT added a commit to meisterT/bazel that referenced this issue Jan 28, 2019
@meisterT
Vendor jaxb library in our third_party.
a64fd25 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None
meisterT added a commit to meisterT/bazel that referenced this issue Jan 28, 2019
@meisterT
Vendor jaxb library in our third_party.
09aab11 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None
meisterT added a commit to meisterT/bazel that referenced this issue Jan 28, 2019
@meisterT
Vendor jaxb library in our third_party.
c4ac104 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None
bazel-io pushed a commit that referenced this issue Jan 28, 2019
@meisterT @philwo
Vendor jaxb library in our third_party.
1770366 
We want to upgrade the embedded JDK to version 11 (see #6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None

Partial commit for third_party/*, see #7240.

Signed-off-by: Philipp Wollermann <philwo@google.com>
meisterT added a commit to meisterT/bazel that referenced this issue Jan 29, 2019
@meisterT
Add target to build the allmodules JDK from the vanilla JDK.
c31fd0b 
With this we can also build the allmodules JDK in the buildkite
pipeline.

This is a necessary step for the upgrade to JDK 11, see bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None
@meisterT meisterT mentioned this issue Jan 29, 2019
Closed
meisterT added a commit to meisterT/bazel that referenced this issue Jan 29, 2019
@meisterT
Add target to build the allmodules JDK from the vanilla JDK.
2d6141e 
With this we can also build the allmodules JDK in the buildkite
pipeline.

This is a necessary step for the upgrade to JDK 11, see bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None
bazel-io pushed a commit that referenced this issue Jan 29, 2019
@meisterT
Add target to build the allmodules JDK from the vanilla JDK.
a87451a 
With this we can also build the allmodules JDK in the buildkite
pipeline.

This is a necessary step for the upgrade to JDK 11, see #6592 and #6314.

RELNOTES: None

Closes #7287.

PiperOrigin-RevId: 231398365
meisterT added a commit to meisterT/bazel that referenced this issue Jan 30, 2019
@meisterT
Upgrade vanilla JDK to 11.
38cfd35 
Note that this does NOT yet update the embbedded JDKs, this will be one
of the next steps after we actually build it on buildkite.

Part of bazelbuild#6592.

RELNOTES: None
bazel-io pushed a commit that referenced this issue Jan 30, 2019
@meisterT
Upgrade vanilla JDK to 11.
5a8be80 
Note that this does NOT yet update the embbedded JDKs, this will be one
of the next steps after we actually build it on buildkite.

Part of #6592.

Closes #7284.

PiperOrigin-RevId: 231578167
weixiao-huang pushed a commit to weixiao-huang/bazel that referenced this issue Jan 31, 2019
@meisterT @weixiao-huang
Add //src:bazel-bin_dev_jdk target.
e77a6d3 
This builds the minimal JDK locally and uses it as embedded JDK.
We typically use the cached one on mirror.bazel.build, because the jlink
is heavily non-determistic. However, this will make local testing
easier, e.g. when we do upgrades from one JDK to another.

Progress on bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None

Closes bazelbuild#7175.

PiperOrigin-RevId: 229912262
weixiao-huang pushed a commit to weixiao-huang/bazel that referenced this issue Jan 31, 2019
@meisterT @weixiao-huang
Vendor jaxb library in our third_party.
0058cce 
We want to upgrade the embedded JDK to version 11 (see bazelbuild#6592). However,
some Java EE modules were removed from 10 to 11:
http://openjdk.java.net/jeps/320

From these modules, we need java.xml.bind for the android_common
dependency which is used by the android_sdk_repository rule.
This will be starlark-ified some time in the future and then will also
move the tools to an external repository making it possible to remove
this and the android_common dependency from Bazel. But since the
migration is not imminent, this commit is the work around to unblock the
upgrade to JDK11.

RELNOTES: None

Partial commit for third_party/*, see bazelbuild#7240.

Signed-off-by: Philipp Wollermann <philwo@google.com>
weixiao-huang pushed a commit to weixiao-huang/bazel that referenced this issue Jan 31, 2019
@meisterT @weixiao-huang
Add target to build the allmodules JDK from the vanilla JDK.
5c92d9f 
With this we can also build the allmodules JDK in the buildkite
pipeline.

This is a necessary step for the upgrade to JDK 11, see bazelbuild#6592 and bazelbuild#6314.

RELNOTES: None

Closes bazelbuild#7287.

PiperOrigin-RevId: 231398365
weixiao-huang pushed a commit to weixiao-huang/bazel that referenced this issue Jan 31, 2019
@meisterT @weixiao-huang
Upgrade vanilla JDK to 11.
9ca277f 
Note that this does NOT yet update the embbedded JDKs, this will be one
of the next steps after we actually build it on buildkite.

Part of bazelbuild#6592.

Closes bazelbuild#7284.

PiperOrigin-RevId: 231578167
bazel-io pushed a commit that referenced this issue Feb 1, 2019
@meisterT
Use freshly built allmodules JDK for the jdeps test.
144912e 
Otherwise we have a cliff every time we bump the embedded JDK version.

Next step of #6592.

RELNOTES: None

Closes #7326.

PiperOrigin-RevId: 231997465
bazel-io pushed a commit that referenced this issue Feb 3, 2019
@meisterT
Bump vanilla JDK from 11.0.1 to 11.0.2.
0068055 
While testing Bazel with the upgraded embedded JDK we hit a bug which
was fixed by 11.0.2:
See
https://www.oracle.com/technetwork/java/javase/11-0-2-relnotes-5188746.html
for the release notes which lists
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8211883
as fixed.

The problem showed e.g. in our integration tests where we download
protobuf as external repository from github and we would show an
handshake error:
ERROR: [...]/BUILD:1:1: every rule of type proto_library implicitly depends upon the target '@com_google_protobuf//:protoc', but this target could not be found because of: no such package '@com_google_protobuf//': java.io.IOException: Error downloading [https://github.com/google/protobuf/archive/7b28271a61a3da0a37f6fda399b0c4c86464e5b3.zip] to [...].zip: Received fatal alert: handshake_failure

Note that this commit does not the update of the embedded JDK yet.

Part of #6592.

RELNOTES: None

Closes #7335.

PiperOrigin-RevId: 232175766
bazel-io pushed a commit that referenced this issue Feb 3, 2019
@meisterT
Explicitly add jdk.crypto.ec to the list of modules.
9075514 
JDK 11 adds support for TLSv1.3 but the handshake while downloading from
github fails while download external repositories.

RELNOTES: Explicitly set https.protocols and exclude TLSv1.3.

Partly address #6592.

Closes #7337.

PiperOrigin-RevId: 232191351
@philwo philwo added the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label Jun 15, 2020
@nicknezis nicknezis mentioned this issue Feb 27, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@meisterT meisterT

Labels
area-EngProd Bazel CI, infrastructure, bootstrapping, release, and distribution tooling P2 We'll consider working on this in future. (Assignee optional) team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website team-Rules-Java Issues for Java rules type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@davido @jin @meisterT @philwo @Monnoroch @Globegitter