Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a bunch of go routine leaks and add a test to check for leaks #8184

Merged
merged 5 commits into from
Jun 25, 2020

Conversation

mkeeler
Copy link
Member

@mkeeler mkeeler commented Jun 24, 2020

The test is in a sub package to isolate the go runtime from other tests as each package runs in its own binary.

We needed to pass a cancellable context into the limiter.Wait instead of context.Background. So I made the func take a context instead of a chan as most places were just passing through a Done chan from a context anyways.

Fix go routine leak in the gateway locator
@mkeeler mkeeler requested a review from a team June 24, 2020 16:44
Copy link
Contributor

@dnephin dnephin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Changes look good to me. One question about the new leak test.

agent/routine-leak-checker/leak_test.go Show resolved Hide resolved
@dnephin
Copy link
Contributor

dnephin commented Jun 24, 2020

(nomad integration test failure)

unit_test.go:890: expected shutdown to take >1s and <3s but took: 999.69662ms

Not bad!

@mkeeler mkeeler force-pushed the bugfix/goroutine-leaks branch 8 times, most recently from 388c128 to 2941e7b Compare June 24, 2020 21:00
This is in its own separate package so that it will be a separate test binary that runs thus isolating the go runtime from other tests and allowing accurate go routine leak checking.

This test would ideally use goleak.VerifyTestMain but that will fail 100% of the time due to some architectural things (blocking queries and net/rpc uncancellability).

This test is not comprehensive. We should enable/exercise more features and more cluster configurations. However its a start.
Copy link
Member

@banks banks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ Great job, been meaning to do this for years 😄

@mkeeler mkeeler merged commit 7041f69 into master Jun 25, 2020
@mkeeler mkeeler deleted the bugfix/goroutine-leaks branch June 25, 2020 13:22
@hashicorp-ci
Copy link
Contributor

🍒❌ Cherry pick of commit 7041f69 onto release/1.8.x failed! Build Log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants