Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support autoBuild and deployByDefault on Image and Kubernetes/OpenShift components #6654

Merged
merged 23 commits into from
Apr 5, 2023
Merged

Support autoBuild and deployByDefault on Image and Kubernetes/OpenShift components #6654

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
dddb2dc
Add sample Devfile with multiple autoBuild/deployByDefault combinations
rm3l Mar 10, 2023
d3a9420
Add helper function to update a given Devfile Command Group
rm3l Mar 11, 2023
35a6e50
Add test cases for 'odo dev'
rm3l Mar 10, 2023
125ff54
Add test cases for 'odo deploy'
rm3l Mar 10, 2023
510c5db
Add test cases for 'odo build-images'
rm3l Mar 10, 2023
344879a
Display the spinner when creating or updating Kubernetes resources
rm3l Mar 10, 2023
cbbd52a
Handle deployByDefault on K8s and OpenShift components
rm3l Mar 10, 2023
399b42c
Add 'devfile.GetImageComponentsToPush' functions that returns the lis…
rm3l Mar 10, 2023
669b6a8
Handle automatic image component creation for 'odo dev' on Kubernetes
rm3l Mar 10, 2023
b14213a
Handle automatic image component creation for 'odo dev' on Podman
rm3l Mar 10, 2023
3058f5a
Handle automatic image and K8s/OpenShift component creation for 'odo …
rm3l Mar 10, 2023
035e826
Bump Devfile library to the latest commit at this time (c1b23d2)
rm3l Mar 31, 2023
0760872
Do not set default values when parsing a Devfile
rm3l Mar 31, 2023
228c704
Add documentation in the Devfile reference page
rm3l Apr 3, 2023
5ef6655
Revert "Display the spinner when creating or updating Kubernetes reso…
rm3l Apr 3, 2023
d027561
Avoid re-applying Image components multiple times
rm3l Apr 4, 2023
2ad44e8
Move GetK8sAndOcComponentsToPush and GetImageComponentsToPush to libd…
rm3l Apr 4, 2023
bba3c88
fixup! Handle automatic image and K8s/OpenShift component creation fo…
rm3l Apr 4, 2023
f6887b5
fixup! Handle automatic image component creation for 'odo dev' on Podman
rm3l Apr 4, 2023
71e0ba2
fixup! Avoid re-applying Image components multiple times
rm3l Apr 4, 2023
7a1f1a0
Apply suggestions from code review
rm3l Apr 4, 2023
3673d01
fixup! Do not set default values when parsing a Devfile
rm3l Apr 4, 2023
dc0fe61
Fix devfile-deploy-functional-pods.yaml (used in 'odo logs' tests)
rm3l Apr 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 155 additions & 2 deletions docs/website/docs/development/devfile.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ For this reference, we use one singular (working) Node.js example which can be r

When `odo dev` is ran, `odo` creates the following Kubernetes resources from `container` (component) and `exec` (command) sections of the Devfile:
* Deployment for running the containers
* Service for accessibility
* Service for accessibility

See [How odo dev translates a container component into Kubernetes resources](./architecture/how-odo-works.md#how-odo-dev-translates-a-container-component-into-kubernetes-resources) for more details.

### File Synchronization

Expand Down Expand Up @@ -90,7 +92,7 @@ The most common deploy scenario is the following:
1. Use the `image` component to build a container
2. Deploy a `kubernetes` component(s) with a Kubernetes resource defined

### How odo runs exec commands in Deploy mode
### How `odo` runs exec commands in Deploy mode
```yaml
commands:
- id: deploy-db
Expand Down Expand Up @@ -140,6 +142,157 @@ Commands defined by the `exec` command are run inside a container started by a [
4. The Kubernetes Job is created with appropriate annotations and labels so that it can be deleted by `odo delete component --name <name>` command if the need be.
5. If the Kubernetes Job fails, a new pod is created to re-run the job instead of re-running it inside the same pod's container, this ensures that we can fetch the logs when needed.

### How `odo` determines components that are applied automatically

When running `odo dev` or `odo deploy`, `odo` determines components that need to be applied automatically by looking at several things:
- whether the component is referenced by any [`apply` commands](https://devfile.io/docs/2.2.0/adding-an-apply-command);
- and looking at dedicated fields that allow to explicitly control this behavior:
- `autoBuild` on [Image components](https://devfile.io/docs/2.2.0/adding-an-image-component)
- `deployByDefault` on [Kubernetes or OpenShift components](https://devfile.io/docs/2.2.0/adding-a-kubernetes-or-openshift-component)

#### `autoBuild` for Image Components

An Image component is applied automatically if **any** of the following conditions are met:
- `autoBuild` is `true`
- `autoBuild` is not set, **and** this Image component is not referenced in any `apply` commands

If the Image component is referenced by an `apply` command, it will be applied when this command is invoked.

For example, given the following excerpt:
```yaml
variables:
CONTAINER_IMAGE_REPO: localhost:5000/odo-dev/node

components:
# autoBuild true => automatically created on startup
- image:
autoBuild: true
dockerfile:
buildContext: .
uri: Dockerfile
imageName: "{{ CONTAINER_IMAGE_REPO }}:autobuild-true"
name: "autobuild-true"

# autoBuild false, referenced in apply command => created when apply command is invoked
- image:
autoBuild: false
dockerfile:
buildContext: .
uri: Dockerfile
imageName: "{{ CONTAINER_IMAGE_REPO }}:autobuild-false-and-referenced"
name: "autobuild-false-and-referenced"

# autoBuild not set, not referenced in apply command => automatically created on startup
- image:
dockerfile:
buildContext: .
uri: Dockerfile
imageName: "{{ CONTAINER_IMAGE_REPO }}:autobuild-not-set-and-not-referenced"
name: "autobuild-not-set-and-not-referenced"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be helpful to add one more component with autoBuild:false and not referenced?

Suggested change
# autoBuild set to false, not referenced in apply command => never created
- image:
autoBuild: false
dockerfile:
buildContext: .
uri: Dockerfile
imageName: "{{ CONTAINER_IMAGE_REPO }}:autobuild-false-and-not-referenced"
name: "autobuild-false-and-not-referenced"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9060596 (#6720)

- container:
image: "{{ CONTAINER_IMAGE_REPO }}:autobuild-not-set-and-not-referenced"
name: runtime
commands:
- apply:
component: autobuild-false-and-referenced
id: image-autobuild-false-and-referenced

- composite:
commands:
- image-autobuild-false-and-referenced
- start-app
group:
isDefault: true
kind: run
id: run

- composite:
commands:
- image-autobuild-false-and-referenced
- deploy-k8s
group:
isDefault: true
kind: deploy
id: deploy
```

When running `odo`, the following Image components will be applied automatically, before applying any other components and invoking any commands:
- `autobuild-true` because it has `autoBuild` set to `true`.
- `autobuild-not-set-and-not-referenced` because it doesn't set `autoBuild` and is not referenced by any `apply` commands.

Because the `image` component `autobuild-false-and-referenced` is referenced by the `apply` command `image-autobuild-false-and-referenced`, it will be applied when this command
is invoked, that is, in this example, when the `run` or `deploy` commands are invoked.


#### `deployByDefault` for Kubernetes/OpenShift Components

A Kubernetes or OpenShift component is applied automatically if **any** of the following conditions are met:
- `deployByDefault` is `true`
- `deployByDefault` is not set, **and** this component is not referenced in any `apply` commands

If the component is referenced by an `apply` command, it will be applied when this command is invoked.

For example, given the following excerpt (simplified to use only Kubernetes components, but the same behavior applies to OpenShift ones):
```yaml
variables:
CONTAINER_IMAGE_REPO: localhost:5000/odo-dev/node

components:
# deployByDefault true => automatically created on startup
- kubernetes:
deployByDefault: true
inlined: |
[...]
name: "k8s-deploybydefault-true"

# deployByDefault false, referenced in apply command => created when apply command is invoked
- kubernetes:
deployByDefault: false
inlined: |
[...]
name: "k8s-deploybydefault-false-and-referenced"

# deployByDefault not set, not referenced in apply command => automatically created on startup
- kubernetes:
inlined: |
[...]
name: "k8s-deploybydefault-not-set-and-not-referenced"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if you can add an example where deployByDefault is false, and not referenced in apply command.

Suggested change
# deployByDefault is set to false, not referenced in apply command => never created
- kubernetes:
deployByDefault: false
inlined: |
[...]
name: "k8s-deploybydefault-false-and-not-referenced"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9060596 (#6720)

- container:
image: "{{ CONTAINER_IMAGE_REPO }}:autobuild-not-set-and-not-referenced"
name: runtime
commands:
- apply:
component: k8s-deploybydefault-false-and-referenced
id: apply-k8s-deploybydefault-false-and-referenced

- composite:
commands:
- apply-k8s-deploybydefault-false-and-referenced
- start-app
group:
isDefault: true
kind: run
id: run

- composite:
commands:
- apply-k8s-deploybydefault-false-and-referenced
- deploy-k8s
group:
isDefault: true
kind: deploy
id: deploy
```

When running `odo`, the following Kubernetes components will be applied automatically, before applying any other components and invoking any commands:
- `k8s-deploybydefault-true` because it has `deployByDefault` set to `true`.
- `k8s-deploybydefault-not-set-and-not-referenced` because it doesn't set `deployByDefault` and is not referenced by any `apply` commands.

Because the `kubernetes` component `k8s-deploybydefault-false-and-referenced` is referenced by the `apply` command `apply-k8s-deploybydefault-false-and-referenced` , it will be applied when this command
is invoked, that is, in this example, when the `run` or `deploy` commands are invoked.


## File Reference

Expand Down
11 changes: 1 addition & 10 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ require (
github.com/Xuanwo/go-locale v1.1.0
github.com/blang/semver v3.5.1+incompatible
github.com/devfile/api/v2 v2.2.0
github.com/devfile/library/v2 v2.2.1-0.20230323124903-d36e409ff94f
github.com/devfile/library/v2 v2.2.1-0.20230330160000-c1b23d25e652
github.com/devfile/registry-support/index/generator v0.0.0-20230322155332-33914affc83b
github.com/devfile/registry-support/registry-library v0.0.0-20221201200738-19293ac0b8ab
github.com/fatih/color v1.14.1
Expand Down Expand Up @@ -130,7 +130,6 @@ require (
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-version v1.4.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hinshun/vt10x v0.0.0-20220301184237-5011da428d02 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
Expand All @@ -143,14 +142,12 @@ require (
github.com/klauspost/compress v1.15.1 // indirect
github.com/kr/pty v1.1.8 // indirect
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
github.com/magiconair/properties v1.8.5 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2 // indirect
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect
github.com/mitchellh/go-wordwrap v1.0.0 // indirect
github.com/mitchellh/mapstructure v1.4.1 // indirect
github.com/mitchellh/reflectwalk v1.0.1 // indirect
github.com/moby/buildkit v0.10.6 // indirect
github.com/moby/locker v1.0.1 // indirect
Expand All @@ -165,7 +162,6 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799 // indirect
github.com/openshift/library-go v0.0.0-20220210170159-18f172cff934 // indirect
github.com/pelletier/go-toml v1.9.4 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
Expand All @@ -179,11 +175,7 @@ require (
github.com/sergi/go-diff v1.2.0 // indirect
github.com/sirupsen/logrus v1.8.1 // indirect
github.com/skeema/knownhosts v1.1.0 // indirect
github.com/spf13/cast v1.3.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/viper v1.8.1 // indirect
github.com/stretchr/testify v1.8.1 // indirect
github.com/subosito/gotenv v1.2.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
Expand All @@ -205,7 +197,6 @@ require (
google.golang.org/grpc v1.49.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.62.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
k8s.io/apiserver v0.26.1 // indirect
k8s.io/component-base v0.26.1 // indirect
Expand Down
Loading