[Feature Request]: Avoid data corruption in slow `Get`

[nickyc975]

Describe your feature request

Currently Mooncake Store relies on lease to protect objects from eviction and removing during data transferring. However, if data transfer takes too long (because of huge object or poor network), objects may be evicted or removed before the transfer is completed, causing the client to read corrupted data.

A simple solution is to check whether the objects are still valid after completing a long-time transmission, then discard read data of invalid objects. This may slightly degrade the performance, so maybe we should make it configurable.

Before submitting a new issue...

• Make sure you already searched for relevant issues and read the documentation

[ykwd]

That’s a great suggestion. In fact, this work is already in progress: #778. It was just delayed due to some other tasks, but I’ll make sure to complete it soon.

[friday0513]

"Double check" may not be a perfect solution: between transferring the the post check, many things may happen.
i.e., that slice of memory may be allocated for another token, or the evicted KVCache may be rewrite to a new address by other clients, both may cause mistakes for the checking.

[ykwd]

Thanks for the suggestion! This feature has now been implemented #778