add a Size field to Query's Result

[MichaelMure]

Rational is that some datastore can easilty get the size of a value when listing only keys, so we might as well have a way to pass that. One way this could be useful is when implementing a cache for Has/GetSize.

[Stebalien]

Hm. I just wanted this when messing with ipfs repo stat (I wanted to be able to report sizes for various record types).

[MichaelMure]

I changed the semantic so that:

• the calling side can opt-in to always have the size set, even with KeysOnly, even it that implies a performance cost
• a datastore implementation can optionally always return the size if it doesn't imply a performance cost

I feel like that the best compromise. What do you think ?

[Stebalien]

LGTM!

[Stebalien]

Could you file pull requests against the other datastores before we merge this?

• go-ds-badger
• go-ds-bolt broken
• go-ds-flatfs
• go-ds-leveldb
• go-ds-redis
• go-ds-s3

[Stebalien]

Actually, this needs to update the mount datastore. We don't need to forward the RequestSize to child queries.

While we're at it, we should add tests to test/suite.go.

[MichaelMure]

Actually, this needs to update the mount datastore. We don't need to forward the RequestSize to child queries.

Not sure I follow, It doesn't at the moment, but it should, right ?

[Stebalien]

Ah, sorry, yes, we _need_ to forward ReturnsSize. Also, I noticed the variable is currently `ReturnsSize`, it should be `ReturnSizes` (we're _asking_ the datastore to return the sizes).

[MichaelMure]

• fixed the mount datastore
• renamed ReturnsSize to ReturnsSizes
• added a test for ReturnsSizes

Should be good to go now

[MichaelMure]

Could you file pull requests against the other datastores before we merge this?

• go-ds-badger : issue opened, query: always return the size go-ds-badger#76
• go-ds-bolt : can't file issue there, first step: drop PutMany now that batching exist go-ds-bolt#4
• go-ds-flatfs : issue opened, query: deny ReturnsSizes and ReturnExpirations instead of returning wrong result go-ds-flatfs#59
• go-ds-leveldb : issue opened, query: always return the size go-ds-leveldb#35
• go-ds-redis : issue opened, but also doesn't support query to begin with
• go-ds-s3: Returns sizes go-ds-s3#45

[Stebalien]

Unfortunately, I can't merge this until we actually have PRs for those issues. Otherwise, all those datastores will return invalid results.

[MichaelMure]

Any chance the respective maintainers could help here ?

[Stebalien]

Unfortunately, I am the de facto maintainer.

[MichaelMure]

I did a pass on all of them but go-ds-bolt is problematic as it's broken at the moment.

[Stebalien]

All merged and updated.