test: Ensure that all databases are always closed on exit #1187
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AndrewSisley
added
bug
Codecov Report
@@ Coverage Diff @@
## develop #1187 +/- ##
===========================================
+ Coverage 68.76% 68.79% +0.02%
===========================================
Files 180 180
Lines 17046 17046
===========================================
+ Hits 11722 11727 +5
+ Misses 4372 4368 -4
+ Partials 952 951 -1 |
AndrewSisley
force-pushed
the
sisley/fix/I1186-change-detector-leak
branch
from
4d6ff7a to
4af5447
Compare
This is really important for file based datastores, and failure to close even a couple will have a highly significant performance impact due to a long-term defra bug, where if the directory hosting defra is deleted whilst a database instance is active the instance will hang/infinate-loop chewing up a huge amount of resources. As the file based directories are deleted by `go test` at the end of each test, this magnifies the effect of any lingering instances. In the case of this issue I believe only a very small number of databases were leaked, yet it nearly doubled the execution time of the change detector (the only potential leaks I found were a couple of panics, and a couple of subscription tests, it could be that as few as 3-4 leaked databases were responsible for the loss of performance).
|
Sorry about the holdup, I moved the defer function from an inline declaration to a proper function shortly before opening the PR and my local runtimes have shot up again. Still trying to figure out why |
Branch without the breaking commit is now taking 15min locally... I'm just going to merge this and see if it sorts it. Is a good enough change to have anyway |
shahzadlone
pushed a commit
that referenced
this pull request
Apr 13, 2023
This is really important for file based datastores, and failure to close even a couple will have a highly significant performance impact due to a long-term defra bug, where if the directory hosting defra is deleted whilst a database instance is active the instance will hang/infinate-loop chewing up a huge amount of resources. As the file based directories are deleted by `go test` at the end of each test, this magnifies the effect of any lingering instances. In the case of this issue I believe only a very small number of databases were leaked, yet it nearly doubled the execution time of the change detector (the only potential leaks I found were a couple of panics, and a couple of subscription tests, it could be that as few as 3-4 leaked databases were responsible for the loss of performance).
shahzadlone
pushed a commit
to shahzadlone/defradb
that referenced
this pull request
Feb 23, 2024
…ork#1187) This is really important for file based datastores, and failure to close even a couple will have a highly significant performance impact due to a long-term defra bug, where if the directory hosting defra is deleted whilst a database instance is active the instance will hang/infinate-loop chewing up a huge amount of resources. As the file based directories are deleted by `go test` at the end of each test, this magnifies the effect of any lingering instances. In the case of this issue I believe only a very small number of databases were leaked, yet it nearly doubled the execution time of the change detector (the only potential leaks I found were a couple of panics, and a couple of subscription tests, it could be that as few as 3-4 leaked databases were responsible for the loss of performance).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relevant issue(s)
Resolves #1186
Description
Ensures that all databases are always closed on exit.
This is really important for file based datastores, and failure to close even a couple will have a highly significant performance impact due to a long-term defra bug, where if the directory hosting defra is deleted whilst a database instance is active the instance will hang/infinite-loop chewing up a huge amount of resources. As the file based directories are deleted by
go testat the end of each test, this magnifies the effect of any lingering instances. In the case of this issue I believe only a very small number of databases were leaked, yet it nearly doubled the execution time of the change detector (the only potential leaks I found were a couple of panics, and a couple of subscription tests, it could be that as few as 3-4 leaked databases were responsible for the loss of performance).Tested by executing the change detector by targeting this branch. The change detector will likely be slow in this PR and the problem will only be resolved upon merge to develop (as the issue is most visible in change detector startup).