Installation Configuration

[maleck13]

What

Look into ways to allow a user to configure a set of subcomponents of Kuadrant in order to achieve a goal without needing to expose the innards of the different APIs. Examples of a goal a user deploying and configuring Kuadrant may want to achieve are:

• zero-trust
• resilient-data-plane

Some imagined use cases

As a platform engineer I want to ensure communications between the gateway and the kuadrant components are secured so that I can comply with my orgs encryption policies.

As a platform engineer, I want to be disable and re-enable zero-trust so that I can run performance tests to understand the cost I am incurring by using this feature

As a platform engineer, I need to be able to set a custom CA to use for mTLS so that I can use one provided by my security team.

Additional Context

There is a lot of context in the linked slack channel. But we want to reduce the burden on the user when they want to setup this type of configuration while still allowing the user to "go under the hood" and override the configuration if needed. At a top level Kuadrant object level we want to avoid leaking up the underlying API. The concept put forward in the slack channel is to start with just a boolean property:

zero-trust : 
      enable: true
observability:
     enable: true
resilientDataPlane:
    enable: true
...

Slack Thread
Community Call Item
RFC
Diagram

We then also want to think about how a user might "tinker" with some of this configuration if needed. Perhaps they want to use a different CA for mTLS for example.

Finally we also want to decide on the behaviour if we set this value to false.

The outcome of this should feed into the RFC(s) around this type of configuration
Kuadrant/architecture#112

[maleck13]

related #798

Examples of the type of user experience we want to remove

https://github.com/Kuadrant/kuadrant-operator/blob/a1ca1d0ee2ca7dac79ad921b0ff30ded2c986612/doc/install/mtls-configuration.md?plain=1
https://docs.kuadrant.io/dev/install-olm/#resilient-deployment-of-data-plane-components

[alexsnaps]

Key here I guess is that a feature or something alike exposed at the "highest level", e.g. the kuadrant CR, doesn't carry any "how" semantics, and is by itself "useless". i.e. it only is used as input to a function to configure "lower level" components.

So it might be so that zero-trust: true eventually means configuring mTLS between all components, but that would need to be explicitly configured at the "lower-level" component explicitly and carries that symmetry: fn(feature) -> (configA, configB), so that (configA, configB) now also implies feature enabled. (I hope this makes some sense)

[maleck13]

One thing that worries me is that there will be things in these areas that we cannot know or configure. Example with observability, there maybe a need to configure a remote write location or the otlp_collector and a credential to use. The observability use case could be a good first one as it is high on our priority list and is very well understood by members of the team

spec:
    observability:
        enabled: true

@david-martin @R-Lawton

Open questions would be of the following type for me:

• I have set the config, what else do I need to do now to get everything working?
• I want to set the otlp host and credentials how do I do that now?
• I want to change the default log level for all components how do I do that?

[alexsnaps]

I wasn't saying all these need to be booleans but rather convey the minimal set of "config". so in the case of tracing, I'd have a single config for the collector for the entirety of Kuadrant's components. If then someone would want to change where the traces from e.g. limitador would be sent, then they'd override that part of the config with Limitador.

wrt "default" log level tho, I think that might be better inferred from some other setting. e.g. production: false. which might mean higher log level than running in "production-mode", but possibly many other things. The question I'd ask is "why/when do you need to change that log level?" and use that to define how to best expose that at the "highest abstraction level of our API"

[maleck13]

@alexsnaps Yes agreed minimum needed config 👍

[david-martin]

Issue with details on the changes in the kuadrant operator for observability #1065

[maleck13]

@david-martin may be useful to know but GH allows you to set parent issues now. Which I think is very helpful.

With observability the first goal of this it might make sense to create a sub issue of type feature that covers the observability work with a goal of completing that feature in the next iteration? It doesn't mean that we are done with configuring observability at that point but instead we will have something that provides value available. This issue can be the top level parent IMO and each configuration option become a sub feature of this.