-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Storm v4 #290
Open
asdine
wants to merge
6
commits into
master
Choose a base branch
from
v4
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
[WIP] Storm v4 #290
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
houndci-bot
reviewed
May 11, 2021
Repository owner
deleted a comment from
codecov-commenter
May 14, 2021
Codecov Report
@@ Coverage Diff @@
## master #290 +/- ##
=========================================
Coverage ? 33.33%
=========================================
Files ? 2
Lines ? 132
Branches ? 0
=========================================
Hits ? 44
Misses ? 75
Partials ? 13 Continue to review full report at Codecov.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After v3, I realized Storm was becoming hard to maintain. It was relying too much on reflection, felt bloated, and was performing poorly. There were a lot of ongoing discussions on the subject, like #211 or #222, that led me to realize what was the actual problem: Storm design was good enough for a simple toolkit but not for an actual database.
If fixing some of the performance issues can be done by tweaking a loop there and there, any other addition was a real pain and my motivation wasn't high enough to handle that level of plumbing.
Instead, I decided to write an actual database as a separate project, Genji, with the goal of, someday, building Storm on top of it.
However, Genji's API is still not yet stable and requires a little bit more work before being able to be relied upon. But as a first step, here is a draft of what it would look like.
Another aspect I wanted to fix was the Storm API. I believe the current API is error-prone and requires too much magic, as we can see in these issues. I think this is the opportunity to improve that design as well.
Storm now uses Genji. What does it mean?
Genji is a document-oriented, embedded SQL database written in Go.
It uses BoltDB as a storage engine by default: Nothing changes for Storm, it's still a toolkit for BoltDB. What changes though, is how that data is encoded, stored, and queried.
It supports other backends: This means that Storm can also be used to store data in-memory or in Badger. The Badger engine is still experimental though, but this is possible in the future.
It supports SQL: This means that if users need advanced queries, they can use SQL directly.
It's a document database: This means that we can now query nested fields as well
It supports schemas: This means that we can now apply constraints on some of the fields of the table/bucket
It is designed for performance: Even though Genji is still in an early phase and lacks optimizations, it was designed to scale and it's much easier to apply changes there.
What are the big changes in the API?
Take a look at the README file.
How can I contribute?
Inputs are welcome! Don't hesitate to comment on the PR or creating an issue.
Next steps