Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the default number of cached queries. #26949

Merged

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented Oct 10, 2017

Memory usage of queries can't be properly accounted, which can be an issue when
large queries are cached since the actual memory usage will be much higher than
what the cache thinks. This problem is very hard if not impossible to fix so as
a workaround I would like to decrease the maximum number of cached queries so
that this problem is

For the record, this problem is more likely to occur in envirenments that have
small shards or don't give much memory to the JVM.

Closes #26938

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I think your commit message got truncated, as the explanation ends with "...this problem is"

@jpountz
Copy link
Contributor Author

jpountz commented Oct 13, 2017

@elasticmachine test it please

Memory usage of queries can't be properly accounted, which can be an issue when
large queries are cached since the actual memory usage will be much higher than
what the cache thinks. This problem is very hard if not impossible to fix so as
a workaround I would like to decrease the maximum number of cached queries so
that this problem is less likely to cause trouble in practice.

For the record, this problem is more likely to occur in envirenments that have
small shards or don't give much memory to the JVM.

Closes elastic#26938
@jpountz jpountz force-pushed the fix/reduce_default_number_of_cached_queries branch from 2be732f to dda9730 Compare October 23, 2017 12:11
@jpountz jpountz merged commit d0104c2 into elastic:master Oct 23, 2017
jpountz added a commit that referenced this pull request Oct 23, 2017
Memory usage of queries can't be properly accounted, which can be an issue when
large queries are cached since the actual memory usage will be much higher than
what the cache thinks. This problem is very hard if not impossible to fix so as
a workaround I would like to decrease the maximum number of cached queries so
that this problem is less likely to cause trouble in practice.

For the record, this problem is more likely to occur in envirenments that have
small shards or don't give much memory to the JVM.

Closes #26938
jpountz added a commit that referenced this pull request Oct 23, 2017
Memory usage of queries can't be properly accounted, which can be an issue when
large queries are cached since the actual memory usage will be much higher than
what the cache thinks. This problem is very hard if not impossible to fix so as
a workaround I would like to decrease the maximum number of cached queries so
that this problem is less likely to cause trouble in practice.

For the record, this problem is more likely to occur in envirenments that have
small shards or don't give much memory to the JVM.

Closes #26938
jpountz added a commit that referenced this pull request Oct 23, 2017
Memory usage of queries can't be properly accounted, which can be an issue when
large queries are cached since the actual memory usage will be much higher than
what the cache thinks. This problem is very hard if not impossible to fix so as
a workaround I would like to decrease the maximum number of cached queries so
that this problem is less likely to cause trouble in practice.

For the record, this problem is more likely to occur in envirenments that have
small shards or don't give much memory to the JVM.

Closes #26938
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Oct 24, 2017
* master:
  Timed runnable should delegate to abstract runnable
  Expose adaptive replica selection stats in /_nodes/stats API
  Remove dangerous `ByteBufStreamInput` methods (elastic#27076)
  Blacklist Gradle 4.2 and above
  Remove duplicated test (elastic#27091)
  Update numbers to reflect 4-byte UTF-8-encoded characters (elastic#27083)
  test: avoid generating duplicate multiple fields (elastic#27080)
  Reduce the default number of cached queries. (elastic#26949)
  removed unused import
  ingest: date processor should not fail if timestamp is specified as json number
@lcawl lcawl added v6.0.0-rc2 and removed v6.0.0 labels Oct 30, 2017
@lcawl lcawl removed the v6.1.0 label Dec 12, 2017
@jpountz jpountz added the :Search/Search Search-related issues that do not fall into other categories label Jan 29, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories v5.6.4 v6.0.0-rc2 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

query cache used memory calculating is not correct, which cause non-stopping old gc.
5 participants