gRPC: Isolate Info to separate gRPC API so we can reuse discovery for different RPCs: Store, Rules and Targets

[bwplotka]

Something decided on our public meeting: https://docs.google.com/document/d/165L9DVKHW0sS2PfKYsUCDikwq5XngKkCqTOVjzC6qQE/edit#heading=h.7dcto2wgvh4q

Something to do in later step. This will avoid us to have all those weird flags like separate SD, separate store etc.
We can disable certain APIs if needed on API level (e.g ruler can disable RulesAPI if needed) (:

cc @s-urbaniak

[povilasv]

I think I would vote for new flag and StoreInfo endpoint returning types supported, like previous proposal mentioned.

Plus on the sidecar you could do --rule-api=disabled , --store-api=disabled or smth similiar.

Thoughts? Maybe we can write a mini proposal for this?

I'm also thinking that potentially we could do TargetsAPI which would show leaf node Targets scraped (I mean Prometheus /targets endpoint) and dedup them? Thoughts?

[bwplotka]

I think I would vote for new flag

New flag? what flag?

and StoreInfo endpoint returning types supported, like previous proposal mentioned.

Hm, yea we wanted to different info per type so its kind of equivalent. So we can detect things either by grpc reflection or info... what would be better? 🤔

[bwplotka]

Also for target API there is issue already: #1375

[bwplotka]

Ok we discussed offline that it would be amazing to actually split discovery as separate gRPC API.

I think we can do it after Rules API is landed, but overall idea is that Info endpoint is shared and Querier will only wait for --endpoint flag or something which will then ask for info API and THEN discover what APIs are there (: This will ensure things like #2407 (comment) and per store TLS will be much easier to implement.

cc @brancz @s-urbaniak @kakkoyun @squat @povilasv @GiedriusS

[povilasv]

👍 I like --endpoint

[bwplotka]

Changed title.

[stale]

Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[GiedriusS]

Still valid.

[stale]

Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[bwplotka]

We need that! (:

[stale]

Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[matej-g]

Just to keep tabs on this, since now #4282 has been merged, what's still missing? Some points:

• Implement Info API for the remaining components (taken by @matej-g)
• Revert the change to always get metadata through fallback (Endpointset: Do not use info client to obtain metadata (for now) #4714)
• Add --endpoint-strict similarly to --store-strict (taken by @hitanshu-mehta: query: add --endpoint-strict flag to statically configure endpoints #4874 )
• Add feature to disable selected APIs
• Deprecate and remove old flags

cc @hitanshu-mehta - as I said if you'd like a hand with these, happy to help

[hitanshu-mehta]

There was also a request to add a feature to disable certain APIs.

cc @hitanshu-mehta - as I said if you'd like a hand with these, happy to help

That would be great :) Is it okay if we divide tasks?

[matej-g]

@hitanshu-mehta I updated the list, thanks 👍 Sure, let's divide! Feel free to take what you prefer first.

[hitanshu-mehta]

@matej-g I can start working on adding new endpoint-strict flag.

[GiedriusS]

Just to keep tabs on this, since now #4282 has been merged, what's still missing? Some points:

• Implement Info API for the remaining components (taken by @matej-g)
• Revert the change to always get metadata through fallback (Endpointset: Do not use info client to obtain metadata (for now) #4714)
• Add --endpoint-strict similarly to --store-strict (taken by @hitanshu-mehta: query: add --endpoint-strict flag to statically configure endpoints #4874 )
• Add feature to disable selected APIs
• Deprecate and remove old flags

cc @hitanshu-mehta - as I said if you'd like a hand with these, happy to help

Only the last two points are left, I think 💪

[matej-g]

So I believe for the API disabling we have overlap with #4785 which should take care of it (one of TODOs - https://github.com/thanos-io/thanos/pull/4785/files#diff-5d0883e9f6db360a29dd686b1ba6dbf152d0ab9549a0506aeffdf532f5ae0591)

@Namanl2001 any updates on the PR progress? Any way we can support you? 🙂

[Namanl2001]

@Namanl2001, any updates on the PR progress? Any way we can support you? 🙂

Many changes are proposed in this commit, so I definitely need support to proceed further.

In PR #4785, we are trying to have a single endpoint set in cmd/thanos/query.go. For context: #4389 (comment)

Currently, I'm blocked from implementing this function . Help needed.
Thanks

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[GiedriusS]

Endpoint config is a WIP by our mentee #5585 💪

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.