feat(rust): post commit hook (v2), create checkpoint hook #2391
Conversation
ion-elgreco
changed the title
ion-elgreco
requested review from
wjones127,
fvaleye,
roeap and
rtyler
as code owners
| #[derive(Clone, Debug, Copy)] | ||
| /// Properties for post commit hook. | ||
| pub struct PostCommitHookProperties { | ||
| create_checkpoint: bool, | ||
| } | ||
|
|
There was a problem hiding this comment.
I think we can remove this struct and just keep the config in the commit builder and commit properties for now.
There was a problem hiding this comment.
I was planning to add support for log retention cleanup, auto compact and auto vacuüm afterwards. So perhaps still good to keep this then?
|
Overall it looks good to me. In the future if we are going to expand on post commit hooks then we should maybe refactor so the commit code is responsible for returning the snapshot version of that delta table that was successfully committed. This would remove the burden of updating the table from each operation too. |
|
@Blajda that's a good idea! That would essentially make it easier to solve the issue for concurrent writers where sometimes it tries to advance the state after a commit, but the delta of the commit version and the table version is greater than 1 |
ion-elgreco
force-pushed
the
feat/post_commit_hook_v2
branch
from
68f3d6e
to
e68c19e
Compare
# Description We advance the state in the post commit now, so it's done in a single location as per suggestion from @Blajda here: #2391 (comment) This PR also supersedes this one: #2280 # Related Issue(s) - fixes #2279 - fixes #2262
Description
Introduces a post commit, which can do additional actions before returning the FinalizedCommit.
Current commit hook will creates a checkpoint if it meets the condition of the interval.
Also bumping the default interval to 100 commits. 10 commits can be a bit aggressive
Related Issue(s)