Support readOnlyRootFilesystem=true in KEDA metric adapter

[joebowbeer]

Proposal

Configure keda-metrics-apiserver to run with readOnlyRootFilesystem=true, or at least document how to configure this.

See suggested solution in discussion #2880

Relates to #2938 which omitted this setting from keda-metrics-apiserver

Use-Case

Enable KEDA to pass security audit.

While readOnlyRootFilesystem=true is not required by PSS/restricted, it is a recommended security best practice and may be required by other policies.

Anything else?

No response

[tomkerkhove]

I will let @zroubalik determine if we want to do this, because from what I'm seeing on #2880 this is something that the end-user has to do and we should provide better docs on.

We cannot run KEDA with full readOnlyRootFilesytem as far as I know.

[joebowbeer]

@tomkerkhove in #2880 @zroubalik asked for someone to create a doc PR.

IIUC, @ygnr was able to accomplish this by setting mountPath to /apiserver.local.config/certificates/

--set volumes.metricsApiServer.extraVolumes[0].name=keda-volume
--set volumes.metricsApiServer.extraVolumeMounts[0].name=keda-volume
--set volumes.metricsApiServer.extraVolumeMounts[0].mountPath=/apiserver.local.config/certificates/

and then setting readOnlyRootFilesystem to true in the securityContext of metricServer

[zroubalik]

Yeah, it is partly covered in FAQ: https://keda.sh/docs/2.7/faq/

Though we can definitely improve it, @joebowbeer mind opening a RP on https://github.com/kedacore/keda-docs with changes that would make it more clear to you?

[joebowbeer]

Created PR kedacore/keda-docs#830 to improve FAQ answer