Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is there a level-blob-store ? #19

Open
jbenet opened this issue Dec 16, 2015 · 5 comments
Open

is there a level-blob-store ? #19

jbenet opened this issue Dec 16, 2015 · 5 comments
Labels

Comments

@jbenet
Copy link

jbenet commented Dec 16, 2015

is there a level-blob-store that allows using any level instance as a blob-store? i know that's very much not the goal of blob-store (big blobs instead of small values) but sometimes useful if you already have the data there and just want to read it with same interface

@daviddias
Copy link
Contributor

I started one yesterday https://github.com/diasdavid/level-blob-store (will release when it complete, hopefully not later than today), since I didn't find any at the list: https://github.com/maxogden/abstract-blob-store#some-modules-that-use-this

@mafintosh
Copy link
Collaborator

I wrote this a while back before abstract-blob-store was a thing but it actually shares the same interface, https://github.com/mafintosh/level-blobs

@daviddias
Copy link
Contributor

ohh by 3 minutes xD https://github.com/diasdavid/level-blob-store

Nice, thank you @mafintosh, now we have two! :D Seems yours is already set in a way that the level storage is passed afterwards so that it runs in the browser and in Node.js, awesome!

@max-mapper
Copy link
Owner

@diasdavid I dont wanna discourage use of a level blob store module, but it would be worth mentioning two things in your readme:

  • LevelDB doesn't support streaming file writes, so values must fit in memory.
  • In Google LevelDB compaction is notoriously inefficient in heavy write scenarios, so writing large quantities of data will likely result in lots of CPU thrashing.

@daviddias
Copy link
Contributor

Thank you, @maxogden :) I'll add that to the README, it was one of the things I found, but fortunately for our scenario, levelDB will be most to store DHT records and we are in fact moving out of levelDB for that too, it is just to be able to support legacy repos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants