Mandatory modules for Kyma instances

[janmedrek]

Description
Some components in our ecosystem need to be deployed in every runtime, there should be a possibility to enable such "mandatory" modules in Kyma. The requirements for such modules are as follows:

• Manifest of the module should be aligned with the original, any changes should be reverted back to the previous state
• There is no per-Kyma configuration for the module, it keeps the same config across all the runtimes
• Configuration of the module is not exposed to the customer, any changes should be reverted back to the previous state
• Such modules should not appear on the list of modules in the SKR Kyma

This is very similar to the previous behaviour of KCP Kyma CRs - enabling a module in KCP to enforce its existence in the SKR.

This feature would additionally require differentiation between mandatory and regular modules (since the KCP list now behaves as an initial module list) and hiding them from the user (so they do not pop up in the Kyma SKR module list).

Unfortunately, this will most likely require some minor tweaks to the Kyma API.

Reasons
We want to provide a way to enforce some modules and their configuration in the runtimes.

Acceptance Criteria

• Describe the Mandatory Module behaviour
  • Given an SKR Cluster
    • And a KCP Kyma CR is created on the KCP Cluster
    • And a Kubeconfig Secret is created on the KCP Cluster
  • When the Mandatory Module is created (for every SKR Cluster simultaneously)
    • Then the Manifest CR is created in the KCP Cluster
      • And the Manifest CR contains status for the Mandatory Module
      • And the Module Manifest is installed in the SKR Cluster
      • And the User cannot enable or disable the Mandatory Module
      • And the User cannot change the configuration of the Mandatory Module
      • And any changes to the deployed Module Manifest are reverted
      • And the ModuleTemplate CR for the Mandatory Module is not synced into the SKR Cluster
      • And the Mandatory Module is not listed in the Kyma CR
      • And the Kyma CR Status does not have an entry for the Mandatory Module
      • And the monitoring dashboard is created for the Mandatory Module

[janmedrek]

Please have a look at this comment first, the specialized implementation may not be needed.

[kwiatekus]

Is a module template CR of a mandatory module present on the (skr) runtime?
Is the submission process for mandatory modules any different comparing to optional ones?

[janmedrek]

@kwiatekus The ModuleTemplate for such modules should not be synced to the SKR, the idea is to completely hide it from the users.

As for the submission - I guess we will go with the same flow. It may vary though due to decisions and the final implementation of the feature.

[ruanxin]

There is no per-Kyma configuration for the module, it keeps the same config across all the runtimes.

does it mean for mandatory modules, there is no Module CR? Or there is Module CR, but the content is identical for all SKR, and LM must maintain consistency of them (revert user change)

[janmedrek]

@ruanxin there will be no ModuleCR for the Mandatory Modules - it does not need it since there is no configuration to be exposed to the user

[kwiatekus]

@janmedrek Would the mandatory module need to be marked somehow specially via label?
How should we adopt the module-config.yaml file when promoting new versions of mandatory warden module?
See the experimental PR we created for warden mandatory module

[janmedrek]

@kwiatekus there will be an additional flag to be passed in the module config used in the submission pipeline, no details yet, depends on the neighbours implementation.

We don't have the doc for CLI yet unfortunately, it will be implemented in: #1053

For now you can include mandatory:true/false in your module config yaml: https://github.com/kyma-project/cli/blob/505c3f193c9017f6b77cf6e64a75b9bd16978d32/cmd/kyma/alpha/create/module/moduleconfig.go#L22