-
-
Notifications
You must be signed in to change notification settings - Fork 177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
End iterator on GC or db close #601
Conversation
Gonna take a break from this for a few days. I'm close and the code is fairly clean now, but I feel like I'm stuck in a bike shed and need a breather to regain clarity of the big picture. |
Wow, great work! I've thought about this for a long time but never realized the complexity involved in actually implementing it. I'll try to play around with this and see if I can break it. |
@chjj Don't dive too deep yet because I plan to simplify it (but haven't had much time) |
Gonna open it up for review (and @chjj go! kick the tires!) though there's some cleanup left (reverting the repeated tests and then squashing). I prefer to keep repeating tests when changes are requested. |
/** | ||
* Worker class for ending an iterator | ||
*/ | ||
struct EndWorker final : public BaseWorker { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the idea of ending on GC, but it will be nice if we keep the ability to manually end iterators as well.
One may not want to fully rely on GC on a busy server if there is a clean way to eagerly release an iterator (and the underlying snapshot and resources).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer either/or. Explicit resource management, or GC. Supporting both is not impossible (the EndWorker
could do the same for a single iterator as what the CloseWorker
does for all iterators now), but it raises the question of what we recommend as best practice - and whether modules that wrap iterators should end explicitly or not.
Would be nice to do some real world benchmarks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 for benchmarks. I thought ending on GC was purely for better promise support as @chjj mentioned in #597 (comment).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what to benchmark. Ending on GC will have some cost, but what cost is acceptable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One of my private project use iterators for prefix searching, with lots of concurrent seek
next
and (early) end
. I would like to benchmark the new GC-version with my project (especially on write intensive workloads), but I'm pretty busy for the next few days...
Shall we release v5.0.0 first and work on this PR later?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Already released it :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My main concern is about the underlying snapshot. I remember that holding snapshots might affect compaction of new keys after the SequenceNumber, so LevelDB suggests to release snapshots as soon as possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll assign you to this PR, no rush.
🤔 In order to benchmark the two implementations, I've routed some real world workload to a test server. I started with the original v5.0.0, but it crashed on segmentation fault (status=11/SEGV) after about an hour. Now I'm trying to reproduce the issue in local environment, I think it should be top priority since v5.0.0+ is already on NPM. I'll open a new issue or PR after locating the cause, and go back to this PR later. |
👍 |
@peakji Can you share some details about the test you did? What kind of operations, open/close states, a stack trace if you have it, etc? |
@vweevers It was a segment fault so no useful stack trace. I've enabled core dump yesterday but unfortunately it didn't break (till now), and I have a feeling that it will break tonight 😜. I'll do a back trace with In addition, I can confirm that the issue is not related to |
Does the software open and close the database during this phase? If not, that rules out an obvious suspect |
No, the software opens a database on start and uses it forever, and I'm sure it opened successfully. |
The software is still up and running, no sign of segment fault, monitor info looks perfect 🙄. I know its kinda weird to wish your own software to crash, but nothing could be fixed by simply enabling core dump and reboot... In fact what worries me this most is that the segfault is NOT related to
@vweevers With your knowledge of the current code base, if you could review and confirm there is no obvious illegal memory access (e.g. out-of-bound, dereferencing NULL, ...), then I'd rather suspect the segfault was caused by the old |
Will hunt for issues at the end of this week. Do your operations have a predefined order (e.g. a get followed by a del) or is it random? Are you using I will deploy |
Yes, its always
I am using the chained form, with mixed
I will keep core dump enabled for the next ... umm... forever! until it breaks! |
Thanks! I made an edit to my questions just a minute ago before you answered:
|
I'm using buffers for both key and value in all the operations. |
👍 I'll deploy the new patched version on a separated server. |
Released 5.0.2 I'll rebase this branch later. |
1506628
to
e546575
Compare
Rebased. I'll also see if I can move some of the smaller refactorings I did here into separate PRs. |
e546575
to
cca5004
Compare
@vweevers Absolutely good news! This solved my doubts, in fact I thought it might be related to BTW, what do you think is the best practice for empty batches (created but unused)? Call |
There's no benefit to calling |
Just reviewed the new PRs, nice work @vweevers ! These should make a new semver-patch? I'm now working on a standalone benchmark script for this PR (finally!). |
Yup! Will try to get that out today or tomorrow.
Awesome! |
cca5004
to
ad9e102
Compare
(Be advised, GitHub shows the commits in the wrong order due to rebasing) |
This comment has been minimized.
This comment has been minimized.
ad9e102
to
3406bba
Compare
@peakji @chjj @ralphtheninja A summary so we're all on the same page:
Unless there are other reasons for wanting to end on GC, I propose we close this. One possible reason is "safety" - i.e. when the user forgets to call |
👍 not worth the added code complexity. Someone forgetting to call |
Removing the need for the user to do
iterator.end()
, following the suggestion by @chjj in #597 (comment).For early review; needs a CI run, some light refactoring and comments to clarify the order of things.