Add ability to add Kubernetes Secrets and SecretKeyRefs

[nicknezis]

Added support for dynamically adding Secrets as VolumeMounts and as ENV variables. The design mimics the design found in Apache Spark which is documented here: https://spark.apache.org/docs/latest/running-on-kubernetes.html#secret-management

This feature is important for loading TLS artifacts (i.e. certs, keys, certificate authority files)

[nicknezis]

Currently testing. I was able to add a secret as a volume mount, but the submission failed when I tried to add an ENV variable. Will try to fix that code next.

[nicknezis]

Apparently Array.asList produces a fixed sized array. This code existed for the two already existing EnvVars. This was causing my code to fail when trying to add a third. I was finally able to test. Can now mount Secrets as files and as ENV variables.

[nicknezis]

To test you can create a secret like this:

kubectl create secret generic dev-user-pass \
  --from-literal=username=devuser \
  --from-literal=password='S!B\*d$zDsb='

Can then use it with the following config-properties:
--config-property heron.kubernetes.pod.secret.db-user-pass=/heron/db-pass
This will mount a secret to a path. db-user-pass secret to /heron/db-pass volume
Can also make an ENV variable:
--config-property heron.kubernetes.pod.secretKeyRef.DBPASS=db-user-pass:password
This will create a DBPASS ENV variable which has the password key from the db-user-user secret.

[joshfischer1108]

Thanks @nicknezis . Could you add this feature to the documentation somewhere? I'd hate for it to get lost.

[nicknezis]

Thanks @nicknezis . Could you add this feature to the documentation somewhere? I'd hate for it to get lost.

Yeah we currently have two pages dedicated to the Kubernetes scheduler. I'd like to consolidate them and have a good location for general config information like this. Was planning to do that as another ticket related to all of these changes I'm making. I'll try to add that soon.