Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Logging flood on search rejections #7830

Closed
jainankitk opened this issue May 30, 2023 · 1 comment
Closed

[BUG] Logging flood on search rejections #7830

jainankitk opened this issue May 30, 2023 · 1 comment
Labels
bug Something isn't working distributed framework good first issue Good for newcomers Performance This is for any performance related enhancements or bugs

Comments

@jainankitk
Copy link
Collaborator

Describe the bug
Search rejection for every shard is being logged even when the debug logging is not enabled. Logging an exception is very expensive as the resulting message can be big wreaking havoc on memory, CPU (due to GC) and disk throughput. Regression commit came from ES 7.10 here.

Search rejection is a mechanism for ensuring resiliency and preventing the cluster from getting overwhelmed. Because of this issue, the cluster degrades further once we see search thread pool rejections

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
The logging should only happen when the debug log is enabled

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@jainankitk jainankitk added bug Something isn't working untriaged labels May 30, 2023
@anasalkouz anasalkouz added Performance This is for any performance related enhancements or bugs low hanging fruit good first issue Good for newcomers and removed untriaged low hanging fruit labels Jun 6, 2023
@jainankitk
Copy link
Collaborator Author

Closing this issue as PR was merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working distributed framework good first issue Good for newcomers Performance This is for any performance related enhancements or bugs
Projects
Development

No branches or pull requests

3 participants