-
Notifications
You must be signed in to change notification settings - Fork 9
Deploying to Git
Marc Boorshtein edited this page Nov 16, 2020
·
2 revisions
The change from direct API calls to creating manifests in a Git repo changes the way OpenUnison communicates with the API server. The process for GitOps is:
- In a workflow, add objects to a
Secret - Once ready to commit, launch a
Jobthat will checkout the repo, add the files from theSecret, and push to the repo - An OpenUnison scheduled task will cleanup
JobandSecretobjects
From an audit perspective, objects are logged against the k8s target, just as before.
- Git repo with SSH support
- An SSH public and private key, with the public key added as a "Deploy key" to the git repo
- Create a new secret in the
openunisonnamespace namedgit-ssh-keyfrom the deploy key's private key. Name the private keyid_rsa - Import
docker.io/tremolosecurity/git-push:latestinto you local container registry
First, create a Secret for storing the objects. This should be done at the beginning of the workflow:
<customTask className="com.tremolosecurity.provisioning.tasks.CreateK8sObject">
<param name="targetName" value="k8s"/>
<param name="template" >
<![CDATA[
---
kind: Secret
apiVersion: v1
metadata:
name: git-secret-$nameSpace$
labels:
deleteAfterJob: "true"
forJob: push-to-git-$nameSpace$
data: {}
]]>
</param>
<param name="kind" value="Secret" />
<param name="srcType" value="yaml" />
<param name="url" value="/api/v1/namespaces/openunison/secrets" />
</customTask>
Note the label forJob must be present.
Each CreateK8sObject task in a workflow needs to be updated to support writing to git. As an example, the existing creation of the Namespace looks like:
<customTask className="com.tremolosecurity.provisioning.tasks.CreateK8sObject">
<param name="targetName" value="k8s"/>
<param name="template" value="{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"$nameSpace$","labels":{"name":"$nameSpace$"}}}" />
</param>
<param name="kind" value="Namespace" />
<param name="url" value="/api/v1/namespaces" />
</customTask>
it needs to be converted to:
<customTask className="com.tremolosecurity.provisioning.tasks.CreateK8sObject">
<param name="targetName" value="k8s"/>
<param name="template" >
<![CDATA[
---
kind: Namespace
apiVersion: v1
metadata:
name: "$nameSpace$"
labels:
name: "$nameSpace$"
]]>
</param>
<param name="kind" value="Namespace" />
<param name="url" value="/api/v1/namespaces" />
<param name="srcType" value="yaml" />
<param name="writeToSecret" value="true" />
<param name="secretName" value="git-secret-$nameSpace$" />
<param name="path" value="src/main/ns/$nameSpace$/namespace-$nameSpace$.yaml" />
<param name="secretNamespace" value="openunison" />
</customTask>
First, convert the value of the template parameter by reversing the XML escaping the JSON and converting to YAML. The param tag now supports embedding the yaml directly in the value of the tag instead of an attribute using the <![CDATA[ ]]> notation. Also, add the the following parameters to each task being written to git:
<!-- tells openunison to interpret the template as YAML instead of JSON -->
<param name="srcType" value="yaml" />
<!-- tells openunison to write to the Secret instead of directly to the API server -->
<param name="writeToSecret" value="true" />
<!-- The secret to write to, created at the beginning of the workflow -->
<param name="secretName" value="git-secret-$nameSpace$" />
<!-- The path, relative to the root of the git repo, to write the yaml to -->
<param name="path" value="src/main/ns/$nameSpace$/namespace-$nameSpace$.yaml" />
<!-- The namespace where the Secret created at the beginning of the workflow is stored -->
<param name="secretNamespace" value="openunison" />
At then end of the workflow, when you wish to provision the objects to git, add a Job:
<customTask className="com.tremolosecurity.provisioning.tasks.CreateK8sObject">
<param name="targetName" value="k8s"/>
<param name="template" >
<![CDATA[
---
kind: Job
apiVersion: batch/v1
metadata:
name: push-to-git-$nameSpace$
namespace: openunison
labels:
jobtype: gitpush
forNamespace: $nameSpace$
spec:
parallelism: 1
completions: 1
backoffLimit: 1
selector:
matchLabels:
job-name: push-to-git-$nameSpace$
template:
metadata:
labels:
job-name: push-to-git-$nameSpace$
spec:
volumes:
- name: ssh-keys
secret:
secretName: git-ssh-key
defaultMode: 420
- name: ssh-dir
emptyDir: {}
- name: git-dir
emptyDir: {}
- name: secret-dir
secret:
secretName: git-secret-$nameSpace$
defaultMode: 420
containers:
- name: push-to-git
image: 'docker.io/tremolosecurity/git-push:latest'
command:
- /usr/local/openunison/pushtogit.sh
- /usr/src
- /usr/git
- git@github.com:repo/test-git-push.git
resources: {}
volumeMounts:
- name: ssh-keys
readOnly: true
mountPath: /etc/ssh-keys
- name: ssh-dir
mountPath: /usr/local/openunison/.ssh
- name: secret-dir
mountPath: /usr/src
- name: git-dir
mountPath: /usr/git
env:
- name: GIT_EMAIL
value: user@domain.com
- name: GIT_USERNAME
value: user
- name: GIT_COMMIT_MSG
value: For workflow $WORKFLOW.id$
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
restartPolicy: Never
serviceAccountName: default
serviceAccount: default
]]>
</param>
<param name="kind" value="Job" />
<param name="url" value="/apis/batch/v1/namespaces/openunison/jobs" />
<param name="srcType" value="yaml" />
<param name="writeToSecret" value="false" />
</customTask>
Update the Job per your needs (ie pointing to a local docker repository).
Add the following job to unison.xml inside of the <scheduler> tag:
<job className="com.tremolosecurity.provisioning.jobs.ClearJobs" name="clearJobs" group="management">
<!-- When to run the job -->
<cronSchedule
seconds="0"
minutes="*"
hours="*"
dayOfMonth="*"
month="*"
dayOfWeek="?"
year="*"
/>
<param name="target" value="k8s" />
<param name="uri" value="/apis/batch/v1/namespaces/openunison/jobs" />
<param name="labels" value="jobtype=gitpush" />
<param name="workflow" value="deleteNewNSJob" />
<param name="runWorkflowAsUsername" value="system" />
<param name="runWorkflowAsUsernameAttribute" value="sub" />
</job>
This will delete completed Job objects and their associated Secret once a minute. Add the following workflow as src/man/webapp/WEB-INF/workflows/deleteNewNSJob.xml:
<workflow name="deleteNewNSJob" label="deleteNewNsJob" description="Delete new namespace job" inList="false" orgid="687da09f-8ec1-48ac-b035-f2f182b9bd1e">
<tasks>
<customTask className="com.tremolosecurity.provisioning.tasks.DeleteK8sObject">
<param name="targetName" value="k8s"/>
<param name="kind" value="Secret" />
<param name="url" value="/api/v1/namespaces/openunison/secrets/git-secret-$job_labels_forNamespace$" />
</customTask>
<customTask className="com.tremolosecurity.provisioning.tasks.DeleteK8sObject">
<param name="targetName" value="k8s"/>
<param name="kind" value="Job" />
<param name="url" value="/apis/batch/v1/namespaces/openunison/jobs/$job_name$" />
</customTask>
</tasks>
</workflow>