Zone aware replication

[brancz]

Is your proposal related to a problem?

Allow not just replication factor and quorum to be a deciding factor, but also that this quorum must be across availability zones.

Describe the solution you'd like

Make Thanos aware of the availability zones receive nodes are in, and allow quorum to pass if and only if ceil(replication-factor/2) availability zones were successfully replicated to.

Describe alternatives you've considered

Leaving as is, or not using Thanos. Leaving as is does not tolerate an entire zone outage though, which is not included in most cloud provider's SLOs (as in cloud providers say, that there is nothing wrong with an entire availability zone being unavailable if the rest works).

@thanos-io/thanos-maintainers

[kakkoyun]

We could immediately benefit from it in our setup. We have been planning to support multi-cluster deployments.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[kakkoyun]

Still valid.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[singularitatem]

+1, multi-az is a highly available requirement for many companies

[GiedriusS]

This has been implemented already, I think: https://thanos.io/tip/components/receive.md/#az-aware-ketama-hashring-experimental.