*: Flexible Multi-tenancy Data Model (?)

[kakkoyun]

Opening this issue to start discussing the details for supporting multi-tenancy (and other possible use cases) in a flexible way in Thanos using labels. This could evolve into a design document.

As @brancz introduced in several issues (#3834, #3822, #3819, #3820), we try to support multi-tenancy. I'd like to discuss whether this could be achievable flexibly using labels and label matchers throughout all components in Thanos.

To make implementation flexible, rather than focusing on a single label could we achieve tenant enforcement using a set of labels?

I would like to hear your opinions before jumping on the details? Do you see any major blockers?

Some YOLO questions:

• What kind of configurations we should provide depending on each component?
• What do we need for each component to make it tenant-aware?
• What do we need to include in StoreAPI? QueryAPI?

@thanos-io/thanos-maintainers

P.S: We can also include discussion concerning metadata vs data labels in this issue. cc @brancz @bwplotka

[brancz]

I've thought about identifying tenants by a set of labels as well for a while, and while it sounds like it fits our ecosystem well, I have never found an actually good use case why we should do this over a flat opaque string. I think there's a reason why the flat opaque string is the standard everywhere, it's because everyone can make it whatever they want which ultimately means any system can integrate with it.

[bwplotka]

I don't think we talk about the same thing @brancz. It's not about identifying tenants, it's rather identifying ... everything. Every resource has tenancy granularity (so really anything: data, configuration, APIs, request coming in, the response coming out, limits, gates, instrumentation, permissions etc).

So to be specific, our worry for @kakkoyun is that suddenly all of those things have to have some unique tenancy attribute and all operations to list, print, edit, search, enforce on this attribute tenant.

Now if we suddenly want some resources to be zone aware we will need to again duplicate code and just add complexity, because suddenly we need to add all operations to list, print, edit, search, enforce on this attribute zone... and so on.

That's why we are wondering with @kakkoyun if it makes sense rather to talk about attribute or label or external labels... so every resources like: data, configuration, APIs, request coming in, the response coming out, limits, gates, instrumentation, permissions has it's label that could be tenant or zone or literally anything else whatever user need.

WDYT?

[brancz]

We are talking about the same thing and I'm saying I don't think tenants need to be this way just for consistency sake.

I 100% agree that infrastructure details should be labels, but these should not be mixed with data. A tenant is neither data nor an infrastructure detail, which is why I think it should be an entirely separate thing. I do agree we should have a distinct set of labels to describe infrastructure details. External labels are data. And tenants describe data isolation. I think we need all three, but I see no reason why data isolation needs to be a set of labels.

[bwplotka]

Makes sense 👍🏽 So labels as mechanisms is agreed, what matters is the separation vs data vs infra details

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗