Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Another attempt to fix o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow #2051

Merged
merged 1 commit into from
Feb 4, 2022

Conversation

reta
Copy link
Collaborator

@reta reta commented Feb 3, 2022

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

This time no guesses, I was able to reproduce the issue. The Netty dumps raw messages (in hex) into logs and for some specific rare patterns (I have an example), the appender filter blows with StrackOverflowError. The change to appender filter to watch only for single line log statements (expressed in expectations) fixed the issue.

Issues Resolved

Closes #1767

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@reta reta requested a review from a team as a code owner February 3, 2022 22:04
@reta
Copy link
Collaborator Author

reta commented Feb 3, 2022

@nknize @andrross the StackOveflowError should be fixed now, the exact cause was found

@opensearch-ci-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4e0224656872e50e582fe8a33ccefd764b94003c
Log 2223

Reports 2223

…T fails w/ stack overflow

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
@reta reta force-pushed the issue-1767.attempt.2 branch from 4e02246 to bdaeeb4 Compare February 3, 2022 22:56
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success bdaeeb4
Log 2225

Reports 2225

@dblock
Copy link
Member

dblock commented Feb 4, 2022

Nice debugging! For posterity you might want to post a repro into #1767.

@dblock dblock merged commit 1e5d983 into opensearch-project:main Feb 4, 2022
github-actions bot pushed a commit that referenced this pull request Feb 4, 2022
…T fails w/ stack overflow (#2051)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 1e5d983)
@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2022

The backport to 1.2 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.2 1.2
# Navigate to the new working tree
cd .worktrees/backport-1.2
# Create a new branch
git switch --create backport/backport-2051-to-1.2
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1e5d98329eaa76d1aea19306242e6fa74b840b75
# Push it to GitHub
git push --set-upstream origin backport/backport-2051-to-1.2
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.2

Then, create a pull request where the base branch is 1.2 and the compare/head branch is backport/backport-2051-to-1.2.

dblock pushed a commit that referenced this pull request Feb 7, 2022
…T fails w/ stack overflow (#2051) (#2055)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 1e5d983)

Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] o.o.transport.netty4.OpenSearchLoggingHandlerIT fails w/ stack overflow
3 participants