Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last modified timestamp for DBBlob #100

Closed
jkawamoto opened this issue Aug 11, 2020 · 5 comments
Closed

Last modified timestamp for DBBlob #100

jkawamoto opened this issue Aug 11, 2020 · 5 comments

Comments

@jkawamoto
Copy link
Contributor

jkawamoto commented Aug 11, 2020

It'd be helpful if DBBlob remembers the last modified timestamp because we need to sync new blob objects among gateways.

Relate to #67.

@lukechampine
Copy link
Owner

I guess there's a broader question here of how much metadata should be attached to blobs. MetaFile has Mode, for example, but blobs aren't necessarily files, so should they have a Mode or not?

Timestamp tracking could certainly be handled by a custom wrapper around MetaDB, but I realize that this is annoying, especially if a timestamp is the only piece of metadata you're missing. Let's revisit this once we know exactly what metadata you need to store for each blob.

@jkawamoto
Copy link
Contributor Author

We need the last modified timestamp and addresses where the blob is stored. Addresses seem to be retrievable with shards. So, the timestamp is the missing one for us.

Perhaps, we can store other metadata such as mode in a blob if we need it. However, since timestamp is needed to synchronize metadata between clients and referred frequently, it'd be useful if it's stored as a part of blob metadata.

@jkawamoto
Copy link
Contributor Author

This commit doesn't add the last modified timestamp 4ce2d49, and I'll reopen this issue.

@jkawamoto jkawamoto reopened this Aug 19, 2020
@jkawamoto
Copy link
Contributor Author

Since we could implement this by ourselves, I'd close this issue.

@jkawamoto
Copy link
Contributor Author

I found AddMetadata and Metadata don't work to store the last modified timestamp of a blob. The problem is AddBlob and AddMetadata (and Blob and Metadata) aren't executed in the same transaction. As a result, if two goroutines add a blob each at the same time, the last modified timestamp stored in the DB can be associated with the other blob. It can also happen that one goroutine read and another goroutine write a blob, the reader can get the newest blob with the previous time samp.

@jkawamoto jkawamoto reopened this Oct 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants