Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable C2 from using AVX-512 on JDK 10 #32138

Merged
merged 1 commit into from
Jul 17, 2018
Merged

Disable C2 from using AVX-512 on JDK 10 #32138

merged 1 commit into from
Jul 17, 2018

Conversation

jasontedor
Copy link
Member

@jasontedor jasontedor commented Jul 17, 2018 •
edited
Loading

The C2 compiler in JDK 10 appears to have an issue compiling to AVX-512 instructions (on hardware that supports such). As a workaround, this commit adds a JVM flag on JDK 10+ to disable the use of AVX-512 instructions until a fix is introduced to the JDK. Instead, we use a flag to enable AVX and AVX2 only.

Note: Based on my reading of the C2 code, this flag does not appear to have any impact on hardware that does not support AVX2. I have tested this manually on an Intel Atom C2538 processor that supports neither AVX nor AVX2. I have also tested this manually on an Intel i5-3317U processor that supports AVX but not AVX2.

Relates #31425

@jasontedor
Disable C2 from using AVX-512 on JDK 10
c19ad97 
The C2 compiler in JDK 10 appears to have an issue compiling to AVX-512
instructions (on hardware that supports such). As a workaround, this
commit adds a JVM flag on JDK 10+ to disable the use of AVX-512
instructions until a fix is introduced to the JDK. Instead, we use a
flag to enable AVX and AVX2 only.

Note: Based on my reading of the C2 code, this flag does not appear to
have any impact on hardware that does not support AVX2. I have tested
this manually on an Intel Atom C2538 processor that supports neither AVX
nor AVX2. I have also tested this manually on an Intel i5-3317U
processor that supports AVX but not AVX2.
@jasontedor jasontedor requested review from jpountz and dliappis July 17, 2018 17:40
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

jpountz
jpountz approved these changes Jul 17, 2018
View reviewed changes
Copy link
Contributor

@jpountz jpountz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Let's add the bug url to the comment when we have one.

@jasontedor jasontedor merged commit d170ab3 into elastic:master Jul 17, 2018
jasontedor added a commit that referenced this pull request Jul 17, 2018
@jasontedor
Disable C2 from using AVX-512 on JDK 10 (#32138)
da949fa 
The C2 compiler in JDK 10 appears to have an issue compiling to AVX-512
instructions (on hardware that supports such). As a workaround, this
commit adds a JVM flag on JDK 10+ to disable the use of AVX-512
instructions until a fix is introduced to the JDK. Instead, we use a
flag to enable AVX and AVX2 only.

Note: Based on my reading of the C2 code, this flag does not appear to
have any impact on hardware that does not support AVX2. I have tested
this manually on an Intel Atom C2538 processor that supports neither AVX
nor AVX2. I have also tested this manually on an Intel i5-3317U
processor that supports AVX but not AVX2.
jasontedor added a commit that referenced this pull request Jul 17, 2018
@jasontedor
Disable C2 from using AVX-512 on JDK 10 (#32138)
00085bd 
The C2 compiler in JDK 10 appears to have an issue compiling to AVX-512
instructions (on hardware that supports such). As a workaround, this
commit adds a JVM flag on JDK 10+ to disable the use of AVX-512
instructions until a fix is introduced to the JDK. Instead, we use a
flag to enable AVX and AVX2 only.

Note: Based on my reading of the C2 code, this flag does not appear to
have any impact on hardware that does not support AVX2. I have tested
this manually on an Intel Atom C2538 processor that supports neither AVX
nor AVX2. I have also tested this manually on an Intel i5-3317U
processor that supports AVX but not AVX2.
@jasontedor jasontedor deleted the disable-avx-512 branch July 17, 2018 19:50
@jasontedor jasontedor mentioned this pull request Jul 17, 2018
Closed
dnhatn added a commit that referenced this pull request Jul 19, 2018
@dnhatn
Merge branch '6.x' into ccr-6.x
bca77c6 
* 6.x:
  Fix rollup on date fields that don't support epoch_millis (#31890)
  Revert "Introduce a Hashing Processor (#31087)" (#32179)
  [test] use randomized runner in packaging tests (#32109)
  Painless: Fix caching bug and clean up addPainlessClass. (#32142)
  Fix BwC Tests looking for UUID Pre 6.4 (#32158) (#32169)
  Call setReferences() on custom referring tokenfilters in _analyze (#32157)
  Add more contexts to painless execute api (#30511)
  Add EC2 credential test for repository-s3 (#31918)
  Fix CP for namingConventions when gradle home has spaces (#31914)
  Convert Version to Java - clusterformation part1 (#32009)
  Fix Java 11 javadoc compile problem
  Improve docs for search preferences (#32098)
  Configurable password hashing algorithm/cost(#31234) (#32092)
  [DOCS] Update TLS on Docker for 6.3
  ESIndexLevelReplicationTestCase doesn't support replicated failures but it's good to know what they are
  Switch distribution to new style Requests (#30595)
  Build: Skip jar tests if jar disabled
  Build: Move shadow customizations into common code (#32014)
  Painless: Add PainlessClassBuilder (#32141)
  Fix accidental duplication of bwc test for script behavior
  Handle missing values in painless (#30975) (#31903)
  Build: Make additional test deps of check (#32015)
  Painless: Fix Bug with Duplicate PainlessClasses (#32110)
  Adjust translog after versionType removed in 7.0 (#32020)
  Disable C2 from using AVX-512 on JDK 10 (#32138)
  [Rollup] Add new capabilities endpoint for concrete rollup indices (#32111)
  Mute :qa:mixed-cluster indices.stats/10_index/Index - all’
  [ML] Wait for aliases in multi-node tests (#32086)
  Ensure to release translog snapshot in primary-replica resync (#32045)
  Docs: Fix missing example script quote (#32010)
  Add Index UUID to `/_stats` Response (#31871) (#32113)
  [ML] Move analyzer dependencies out of categorization config (#32123)
  [ML][DOCS] Add missing 6.3.0 release notes (#32099)
  Updates the build to gradle 4.9 (#32087)
  Update monitoring template version to 6040099 (#32088)
  Fix put mappings java API documentation (#31955)
  Add exclusion option to `keep_types` token filter (#32012)
dnhatn added a commit that referenced this pull request Jul 20, 2018
@dnhatn
Merge branch 'master' into ccr
d0f3ed5 
* master:
  Painless: Simplify Naming in Lookup Package (#32177)
  Handle missing values in painless (#32207)
  add support for write index resolution when creating/updating documents (#31520)
  ECS Task IAM profile credentials ignored in repository-s3 plugin (#31864)
  Remove indication of future multi-homing support (#32187)
  Rest test - allow for snapshots to take 0 milliseconds
  Make x-pack-core generate a pom file
  Rest HL client: Add put watch action (#32026)
  Build: Remove pom generation for plugin zip files (#32180)
  Fix comments causing errors with Java 11
  Fix rollup on date fields that don't support epoch_millis (#31890)
  Detect and prevent configuration that triggers a Gradle bug (#31912)
  [test] port linux package packaging tests (#31943)
  Revert "Introduce a Hashing Processor (#31087)" (#32178)
  Remove empty @return from JavaDoc
  Adjust SSLDriver behavior for JDK11 changes (#32145)
  [test] use randomized runner in packaging tests (#32109)
  Add support for field aliases. (#32172)
  Painless: Fix caching bug and clean up addPainlessClass. (#32142)
  Call setReferences() on custom referring tokenfilters in _analyze (#32157)
  Fix BwC Tests looking for UUID Pre 6.4 (#32158)
  Improve docs for search preferences (#32159)
  use before instead of onOrBefore
  Add more contexts to painless execute api (#30511)
  Add EC2 credential test for repository-s3 (#31918)
  A replica can be promoted and started in one cluster state update (#32042)
  Fix Java 11 javadoc compile problem
  Fix CP for namingConventions when gradle home has spaces (#31914)
  Fix `range` queries on `_type` field for singe type indices (#31756)
  [DOCS] Update TLS on Docker for 6.3 (#32114)
  ESIndexLevelReplicationTestCase doesn't support replicated failures but it's good to know what they are
  Remove versionType from translog (#31945)
  Switch distribution to new style Requests (#30595)
  Build: Skip jar tests if jar disabled
  Painless: Add PainlessClassBuilder (#32141)
  Build: Make additional test deps of check (#32015)
  Disable C2 from using AVX-512 on JDK 10 (#32138)
  Build: Move shadow customizations into common code (#32014)
  Painless: Fix Bug with Duplicate PainlessClasses (#32110)
  Remove empty @param from Javadoc
  Re-disable packaging tests on suse boxes
  Docs: Fix missing example script quote (#32010)
  [ML] Wait for aliases in multi-node tests (#32086)
  [ML] Move analyzer dependencies out of categorization config (#32123)
  Ensure to release translog snapshot in primary-replica resync (#32045)
  Handle TokenizerFactory  TODOs (#32063)
  Relax TermVectors API to work with textual fields other than TextFieldType (#31915)
  Updates the build to gradle 4.9 (#32087)
  Mute :qa:mixed-cluster indices.stats/10_index/Index - all’
  Check that client methods match API defined in the REST spec (#31825)
  Enable testing in FIPS140 JVM (#31666)
  Fix put mappings java API documentation (#31955)
  Add exclusion option to `keep_types` token filter (#32012)
  [Test] Modify assert statement for ssl handshake (#32072)
@QingsongYao QingsongYao mentioned this pull request Aug 20, 2018
Closed
@nabadger
Copy link

nabadger commented Aug 29, 2018 •
edited
Loading

We've just run into the following issue related to this:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=1
Error: Could not find or load main class `2
Caused by: java.lang.ClassNotFoundException: `2

This puts our pod (we run in kubernetes) into a crash-state, so it won't start.

Our instances are provided by GCP. Today we added some additional instances, and our new ES master just happened to get scheduled onto one of the new instances.

We can workaround this by specifying a minCpuPlatform on Google when creating instances (assuming we can figure out what family we need to run), but just wanted to add a note to say that this can happen in a rather unexpected way :)

Also, looks like the instances that it provisioned us was an older model (Sandy Bridge).

@jasontedor
Copy link
Member Author

@nabadger I think there is something else going on here, but it's not clear what it is. This message is fine:

OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=1

It is expected when you run on an x64 machine that does not support AVX-2. In your case, the Sandy Bridge processor supports AVX-1 so the JVM falls back to that. In my case, I have a Atom processor that does not support any AVX instructions so I have:

OpenJDK 64-Bit Server VM warning: UseAVX=2 is not supported on this CPU, setting it to UseAVX=0

Yet, Elasticsearch starts fine.

This:

Error: Could not find or load main class `2
Caused by: java.lang.ClassNotFoundException: `2

Indicates to me that something else happens, like you have a set an environment variable ES_JAVA_OPTS or something else that is interfering with the CLI for starting the JVM for Elasticsearch. Would you please double check your configuration. If not, can you give exact steps to reproduce.

@nabadger
Copy link

@jasontedor Ah yes, apologies. It was typo in JAVA_OPTS.

@jefflill jefflill mentioned this pull request Dec 5, 2018
Closed
@bai
Copy link

bai commented Apr 3, 2019

This issue has been resolved in JDK 11 and above: http://hg.openjdk.java.net/jdk/jdk11/rev/7339b9e38182. Would it make sense to revert this change?

@jasontedor jasontedor mentioned this pull request Apr 4, 2019
Merged
@jpountz jpountz mentioned this pull request Dec 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jpountz jpountz jpountz approved these changes

@dliappis dliappis Awaiting requested review from dliappis

Assignees
No one assigned
Labels
>bug :Core/Infra/Core Core issues without another label jvm bug v6.3.2 v6.4.0 v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jasontedor @elasticmachine @nabadger @bai @jpountz @colings86