Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ristretto caching for bloomfilters in memory #791

Closed
jarifibrahim opened this issue May 2, 2019 · 3 comments · Fixed by #1204
Closed

Ristretto caching for bloomfilters in memory #791

jarifibrahim opened this issue May 2, 2019 · 3 comments · Fixed by #1204
Assignees
@jarifibrahim
Labels
area/performance Performance related issues. kind/enhancement Something could be better. priority/P3 Low priority, something to be done once everything else is fixed. status/accepted We accept to investigate or work on it.
Milestone
Future ideas

Comments

@jarifibrahim
Copy link
Contributor

While debugging #772 (comment) we realized that bloomfilter size increases as the number of table increases (we store all the bloom filters in memory)

So, we should maintain a clock based cache for the tables in memory and allow them to be evicted.
@jarifibrahim jarifibrahim added kind/enhancement Something could be better. priority/P0 Critical issue that requires immediate attention. labels May 2, 2019
@jarifibrahim
Copy link
Contributor Author

Related to #663

@campoy campoy added area/performance Performance related issues. priority/P1 Serious issue that requires eventual attention (can wait a bit) status/accepted We accept to investigate or work on it. and removed priority/P0 Critical issue that requires immediate attention. labels May 28, 2019
@campoy
Copy link
Contributor

campoy commented May 28, 2019

Downgraded to p1 since no action has been taken since this was filed.

Should this still be worked on? If so, let's chat about how and when to implement this.

@campoy campoy added this to the Unplanned milestone May 28, 2019
@campoy campoy added priority/P3 Low priority, something to be done once everything else is fixed. and removed priority/P1 Serious issue that requires eventual attention (can wait a bit) labels Jun 13, 2019
@campoy campoy modified the milestones: Unplanned, Future ideas Jun 13, 2019
@manishrjain manishrjain changed the title Clock based cache for tables in memory Ristretto caching for tables in memory Nov 19, 2019
@manishrjain
Copy link
Contributor

We should do something to avoid bloom filter size growing linearly as the number of tables grow.

@jarifibrahim jarifibrahim changed the title Ristretto caching for tables in memory Ristretto caching for bloomfilters in memory Jan 22, 2020
@jarifibrahim jarifibrahim self-assigned this Jan 22, 2020
@jarifibrahim jarifibrahim mentioned this issue Jan 22, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jarifibrahim jarifibrahim

Labels
area/performance Performance related issues. kind/enhancement Something could be better. priority/P3 Low priority, something to be done once everything else is fixed. status/accepted We accept to investigate or work on it.
Milestone
Future ideas
Development

Successfully merging a pull request may close this issue.

Add support for caching bloomfilters hypermodeinc/badger
3 participants
@manishrjain @campoy @jarifibrahim