-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support getting async and sync handles for existing blob store items. #60
Comments
There are a number of questions about this:
I guess in the spirit of content addressing the first one would be the one to implement first. You only ever get data that is consistent with the hash you have opened, or io errors. However, the second version is much faster since it does not need to validate anything. |
The API I would expect to find (which may or may not be realistic) is two functions:
The completed variants are basically pass throughs for file readers The in progress variant would actually just wait until the bytes are available, indefinitely. Seeking around might inform the underlying blobs machinery to prioritize certain chunks. There may be other error conditions as well. |
Perhaps the read_in_progress needs to be split into a seekable but unverified version and a non seekable verified version |
Here is a very WIP PR that adds the ability to get an impl Read + Seek for a complete or incomplete file. It can also do tokio AsyncRead and AsyncSeek. What it does not do however is to trigger downloads on seek - that would be another thing. |
We should have a way to get a handle from the store that allows you to treat a blob as a file in a sync context (implement
Read
andSeek
traits) or in a (tokio-flavoured) async context (tokio::io::AsyncSeek and tokio::io::AsyncRead).The text was updated successfully, but these errors were encountered: