Server returns Blob_Deleted for deleted blob out of retention #2821
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2821 +/- ##
============================================
+ Coverage 64.24% 70.21% +5.96%
- Complexity 10398 11756 +1358
============================================
Files 840 842 +2
Lines 71755 72362 +607
Branches 8611 8709 +98
============================================
+ Hits 46099 50806 +4707
+ Misses 23004 18896 -4108
- Partials 2652 2660 +8 ☔ View full report in Codecov by Sentry. |
| * This option indicates that the store needs to return the message as long as the message | ||
| * has not been physically deleted from the store. | ||
| */ | ||
| Store_Include_Compaction_Ready |
There was a problem hiding this comment.
QQ: currently we have several user who enable Store_Include_Deleted to get the blob as long as it not been physically deleted. So seems like this is not a backward compatible change. Will this affect any of their current use cases?
| Store_Include_Deleted | ||
| Store_Include_Deleted, | ||
| /** | ||
| * This option indicates that the store needs to return the message as long as the message |
There was a problem hiding this comment.
QQ. Do we need this new option? Would it be easier if we tell customers that Store_Include_Deleted only works if they issue GET within the delete retention window?
There was a problem hiding this comment.
The reason to add this option is to make sure that Include_All get option would continue to work for compaction_ready blobs.
…linkedin#2821)" This reverts commit 1d989b0.
Summary
When issuing a get request with include_deleted get option, frontend would send a get request to server to fetch the content of the blob. However, if any chunk blob is deleted and out of retention window, even if we have metadata chunk still in the server, we can't finish downloading the entire blob.
This PR fixes this issue by not returning content for metadata. Now if client issues a get request with include_deleted get option and trying to download a blob that was already deleted and out of retention window, server would just return ID_Deleted error back to frontend so frontend would just return 410 back to client right away.
To download deleted and out of retention blobs, we can issue a get request with include_all.
Details
Add a new StoreGetOptions
Store_Include_Compaction_ReadyChange the meaning of
Store_Include_DeletedTest
Unit test