Allow to check configuration of all resources like ingresses

[fredgate]

I use starboard on a pre-production k8s cluster, where all our development teams deploy their Helm charts. So I would to check that all deployed resources respect some best practices.
For that I use starboard with conftest and several selected policies. One thing that I want to be sure is that no resources use some deprecated k8s API. For that I use policies from deprek8ion project.

K8s 1.19 deprecated some API that are removed in k8s 1.22. These deprek8ion policies check all these deprecations that concern for example ingresses.

The problem is that the ConfigAuditReportReconciler controller does not support ingresses:

starboard/pkg/operator/controller/configauditreport.go

Lines 49 to 65 in 7ef9060

	resources := []struct {
	kind kube.Kind
	forObject client.Object
	ownsObject client.Object
	}{
	{kind: kube.KindPod, forObject: &corev1.Pod{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindReplicaSet, forObject: &appsv1.ReplicaSet{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindReplicationController, forObject: &corev1.ReplicationController{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindStatefulSet, forObject: &appsv1.StatefulSet{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindDaemonSet, forObject: &appsv1.DaemonSet{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindCronJob, forObject: &batchv1beta1.CronJob{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindJob, forObject: &batchv1.Job{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindService, forObject: &corev1.Service{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindConfigMap, forObject: &corev1.ConfigMap{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindRole, forObject: &rbacv1.Role{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	{kind: kube.KindRoleBinding, forObject: &rbacv1.RoleBinding{}, ownsObject: &v1alpha1.ConfigAuditReport{}},
	}

Could we add more supported resources, like ingresses...

[danielpacak]

When it comes to configuration audits we definitely want to support all resource kinds!

Currently (v0.13.0) the Conftest plugin has a predefined list or resource kinds, but short term we can extend it easily by adding Ingress there.

Eventually we'd like to make the list of discovered resources configurable, but this requires thinking about RBAC config. For example, adding Ingress or RuntimeClass object must be synced with RBAC permissions granted to a service account used run a Starboard Operator pod. This is also somehow related to another ask from the community to run the Conftest plugin with a dedicated service account, which may be a bit simpler to configure overall.

[fredgate]

Very interesting. What is the roadmap? Can we participate?

[danielpacak]

For the upcoming release v0.14 we are focusing or vulnerability scanning, especially scanning workloads that run containers from private or managed registries such as ECR where you don't really create imagePullSecrets but authorize a service account or cluster node to pull down container images.

Anyway, we can start thinking about configuration checkers in parallel. I created #836 as a starting point to write a minimal design document. Please let me know if anyone would like to pick it up.