Umbrella issue for RW coordination for object storage.

[bwplotka]

Agreed proposal: https://thanos.io/proposals/201901-read-write-operations-bucket.md/

Action Items:

• Ensure "consistency delay" (aka syncDelay) is implemented on all components (handling fresh blocks) with the reasonable minimum value
  • Related: Unified fetching/syncing block metadata everywhere  #1394
• Ensure Store gateway handles toDelete blocks (detecting those and unloading before they are removed)
• Implement handling of HealthyOverlapped block based on source ULIDs from meta file.
• Delete should start from deleting meta.json (done)
• Add ScheduleDelete that will delete adding deletion-mark.json
• Add deleteDelay support for compactor on apply on toDelete blocks.
• [Low] Save successful compactor block uploads to persistent file, same as shipper to avoid unnecessary duplicated compactions/downsamplings
• [Low] Add ignore_delay parameter that will ignore syncDelay for sudden repairs.
• [Low] Ignore HealthyOverlapped to reduce resource consumption for store gateway.

[bwplotka]

cc @krasi-georgiev @mjudeikis

[jojohappy]

@bwplotka I'm interested to implement the proposal, could I start to do that?

[bwplotka]

Anyone is welcome to help. cc @abursavich as we talked about this today at KubeCon. Feel free to read the proposal: https://thanos.io/proposals/201901-read-write-operations-bucket.md/ It should be quite explanatory, even though a bit long. This issue description contains all tasks related to this. If you wish to start working on it I think we could join forces and finish it quickly, happy to jump on it finally as well.

[bwplotka]

Fixes: #564

[palash25]

Hi @bwplotka I am also interested in the project. Since there are only two days left in the application period how are you going to select the mentee for this? Does one need to have previous contributions to thanos or a proposal on how to implement the project (like we do for GSoC) or both?

I am quite comfortable with Go with almost 2 years of experience writing it but distributed systems is something I have only begun to scratch the surface of and would like to get more experience in it. This seems like a perfect project for it.

Here is what I have done so far

• Built thanos locally
• Explored it a bit using the Katacode tutorial
• Went through the project proposal

Haven't been through the code yet, will do it in the next few hours.
I am looking for some guidance here on what to do next and proceed further with my application for community bridge any help would be appreciated. Thanks

[bwplotka]

Hi, Thanks for this @Masquerade0097 @palash25 ((:

However, I am not the one who decides. The full project list will be finalized on Monday: https://github.com/cncf/soc/blob/master/communitybridge/README.md#timeline

The CNCF will select projects and once the project will be selected, you will see it on the website and you will be able to apply as a mentee. Also cc @GiedriusS

Have a nice weekend. (:

[palash25]

thanks

[bwplotka]

@palash25 @Masquerade0097 please apply for this as described here: https://docs.linuxfoundation.org/display/DOCS/Mentees

[palash25]

I did apply through the community bridge site @bwplotka

[bwplotka]

@khyatisoneji our Community Bridge student is on it (:

[stale]

This issue/PR has been automatically marked as stale because it has not had recent activity. Please comment on status otherwise the issue will be closed in a week. Thank you for your contributions.

[bwplotka]

Still in doing.

…

On Sun, 8 Mar 2020 at 11:24, stale[bot] ***@***.***> wrote: This issue/PR has been automatically marked as stale because it has not had recent activity. Please comment on status otherwise the issue will be closed in a week. Thank you for your contributions. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1528?email_source=notifications&email_token=ABVA3OYBTEUJ6I2N6E5V6W3RGN6ARA5CNFSM4IXFIEH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOETK7I#issuecomment-596194685>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVA3O5VTPI5WWYZEAGBMRLRGN6ARANCNFSM4IXFIEHQ> .

[bwplotka]

This should be now done, thanks to @khyatisoneji #2136