Run Consul servers and clients as non-root and make it configurable.

[ishustava]

Based on work done in #311

Changes proposed in this PR

Add new values server.securityContext and client.securityContext to allow configuring securityContext setting for clients and servers. The map directly to pod's security context in Kubernetes.

We also default these settings to running both servers and clients as non-root and with the consul uid and gid, which are created in the Consul docker image.

When openshift is enabled, we don't want these settings to be there since openshift generates random user and group and sets them for each pod.

Testing

No need for the reviewers to test since passing acceptance tests is enough of a validation

TODO

• Changelog entry

[thisisnotashwin]

This looks great!! 🥳

[lkysow]

Nice!!

[lawliet89]

Is there any guidance for upgrading Consul servers to the new version of the chart wrt this change?

When I ls the /consul/data directory, I get:

/ # ls -al /consul/data/
total 44
drwxrwsr-x    6 root     consul        4096 Mar 26  2019 .
drwxr-xr-x    1 consul   consul        4096 Dec  9 22:07 ..
-rw-rw-r--    1 root     consul         394 Mar 26  2019 checkpoint-signature
drwxrws---    2 root     consul       16384 Mar 26  2019 lost+found
-rw-rw----    1 root     consul          36 Mar 26  2019 node-id
drwxrws---    2 root     consul        4096 Oct 17  2019 proxy
drwxrwsr-x    3 root     consul        4096 Mar 26  2019 raft
drwxrwsr-x    2 root     consul        4096 Nov 25 03:58 serf

Based on my understanding, GID 1000 is consul and UID 100 is also consul, so theoretically the server pods can continue to read and write to /consul/data.

EDIT: Did a test upgrade and it seems like the servers were able to read/write to the data files fine. I guess this was expected as the GID owner of the files were set to 1000.