Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Introduce new MultiContainerRun function to run pods with two containers #3095

Merged
merged 7 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions docs/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,82 @@ Example:
- |
echo "Example"

MultiContainerRun
-----------------

MultiContainerRun spins up a new pod with two containers connected
via shared `emptyDir`_ volume.
It's similar to KubeTask, but allows using multiple images to move backup data.
"background" container is one responsible for generating data, while "output" container
should export it to destination.
The main difference between these containers is that phase outputs can only be generated
from the "output" container.
The function also supports an optional init container to set up the volume contents.

.. csv-table::
:header: "Argument", "Required", "Type", "Description"
:align: left
:widths: 5,5,5,15

`namespace`, No, `string`, namespace in which to execute (the pod will be created in controller's namespace if not specified)
`backgroundImage`, Yes, `string`, image to be used in "background" container
`backgroundCommand`, Yes, `[]string`, command list to execute in "background" container
Comment on lines +171 to +172
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we call this container snapshot or backup container? or maybe data container? just a suggestion, if you think background is ok, I am good.

`outputImage`, Yes, `string`, image to be used in "output" container
`outputCommand`, Yes, `[]string`, command list to execute in "output" container
Comment on lines +173 to +174
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be called upload container?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same note as a note before, trying to name things for what they do rather than what they can be used for.

`initImage`, No, `string`, image to be used in init container of the pod
`initCommand`, No, `[]string`, command list to execute in init container of the pod
`podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
`podAnnotations`, No, `map[string]string`, custom annotations for the temporary pod that gets created
`podLabels`, No, `map[string]string`, custom labels for the temporary pod that gets created
`sharedVolumeMedium`, No, `string`, medium setting for shared volume. See `emptyDir`_.
`sharedVolumeSizeLimit`, No, `string`, sizeLimit setting for shared volume. See `emptyDir`_.
`sharedVolumeDir`, No, `string`, directory to mount shared volume. Defaults to `/tmp`

.. _emptyDir: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir

Example:

.. code-block:: yaml
:linenos:

- func: MultiContainerRun
name: examplePhase
args:
namespace: "{{ .Deployment.Namespace }}"
podOverride:
containers:
- name: export
imagePullPolicy: IfNotPresent
podAnnotations:
annKey: annValue
podLabels:
labelKey: labelValue
sharedVolumeMedium: Memory
sharedVolumeSizeLimit: 1Gi
sharedVolumeDir: /tmp/
initImage: ubuntu
initCommand:
- bash
- -c
- |
mkfifo /tmp/pipe-file
backgroundImage: ubuntu
backgroundCommand:
- bash
- -c
- |
for i in {1..10}
do
echo $i
sleep 0.1
done > /tmp/pipe-file
outputImage: ubuntu
outputCommand:
- bash
- -c
- |
cat /tmp/pipe-file

ScaleWorkload
-------------

Expand Down
3 changes: 3 additions & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,6 @@ webhook
Kopia
kopia
hostname
emptyDir
sizeLimit
init
66 changes: 66 additions & 0 deletions docs_new/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,72 @@ Example:
echo "Example"
```

### MultiContainerRun

MultiContainerRun spins up a new pod with two containers connected via shared [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) volume.
It's similar to KubeTask, but allows using multiple images to move backup data.
"background" container is one responsible for generating data, while "output" container
should export it to destination.
The main difference between these containers is that phase outputs can only be generated from the
"output" container.
The function also supports an optional init container to set up the volume contents.


| Argument | Required | Type | Description |
| ----------- | :------: | ----------------------- | ----------- |
| namespace | No | string | namespace in which to execute (the pod will be created in controller's namespace if not specified) |
| backgroundImage | Yes | string | image to be used in "background" container |
| backgroundCommand | Yes | []string | command list to execute in "background" container |
| outputImage | Yes | string | image to be used in "output" container |
| outputCommand | Yes | []string | command list to execute in "output" container |
| initImage | No | string | image to be used in init container of the pod |
| initCommand | No | []string | command list to execute in init container of the pod |
| podOverride | No | map[string]interface{} | specs to override default pod specs with |
| podAnnotations | No | map[string]string | custom annotations for the temporary pod that gets created |
| podLabels | No | map[string]string | custom labels for the temporary pod that gets created |
| sharedVolumeMedium | No | string | medium setting for shared volume. See [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir). |
| sharedVolumeSizeLimit | No | string | sizeLimit setting for shared volume. See [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir). |
| sharedVolumeDir | No | string | directory to mount shared volume, defaults to `/tmp` |


Example:

``` yaml
- func: MultiContainerRun
name: examplePhase
args:
namespace: "{{ .Deployment.Namespace }}"
podOverride:
containers:
- name: export
imagePullPolicy: IfNotPresent
podAnnotations:
annKey: annValue
podLabels:
labelKey: labelValue
sharedVolumeMedium: Memory
sharedVolumeSizeLimit: 1Gi
sharedVolumeDir: /tmp/
backgroundImage: ubuntu
backgroundCommand:
- bash
- -c
- |
mkfifo /tmp/pipe-file
for i in {1..10}
do
echo $i
sleep 0.1
done > /tmp/pipe-file
outputImage: ubuntu
outputCommand:
- bash
- -c
- |
while [ ! -e /tmp/pipe-file ]; do sleep 1; done
cat /tmp/pipe-file
```

### ScaleWorkload

ScaleWorkload is used to scale up or scale down a Kubernetes workload.
Expand Down
Loading