Make Devfile the default deployment mechanism for odo #3634

cdrage · 2020-07-23T19:55:24Z

What type of PR is this?

Uncomment only one /kind line, and delete the rest.
For example, > /kind bug would simply become: /kind bug

/kind feature

What does does this PR do / why we need it:

Makes Devfile the default deployment mechanism, removing S2I in favour
of Devfile deployment.

Which issue(s) this PR fixes:

Closes #3550

How to test changes / Special notes to the reviewer:

Run:

odo preference set experimental false
odo create --starter nodejs
odo push

codecov · 2020-07-24T14:16:25Z

Codecov Report

Merging #3634 into alpha will decrease coverage by 0.06%.
The diff coverage is 59.25%.

@@            Coverage Diff             @@
##            alpha    #3634      +/-   ##
==========================================
- Coverage   44.62%   44.56%   -0.07%     
==========================================
  Files         124      124              
  Lines       12524    12522       -2     
==========================================
- Hits         5589     5580       -9     
- Misses       6384     6390       +6     
- Partials      551      552       +1

Impacted Files	Coverage Δ
pkg/catalog/catalog.go	`53.74% <ø> (ø)`
pkg/devfile/adapters/kubernetes/utils/utils.go	`52.22% <0.00%> (-0.59%)`	⬇️
pkg/util/util.go	`51.81% <ø> (ø)`
pkg/devfile/adapters/docker/component/utils.go	`64.43% <33.33%> (-2.34%)`	⬇️
pkg/devfile/adapters/common/command.go	`96.44% <100.00%> (+0.08%)`	⬆️
pkg/devfile/adapters/common/utils.go	`88.46% <100.00%> (+0.79%)`	⬆️
pkg/sync/adapter.go	`81.73% <100.00%> (+0.73%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2956ce...2ed2d01. Read the comment docs.

cdrage · 2020-07-27T15:23:18Z

Current blocked on: #3545 which will need to be merged in before we can merge this PR.

cdrage · 2020-07-30T20:27:45Z

Ping @dharmit @girishramnani @kadel

See below for commands that have been tested. There are a lot left out. But below can at least get the basic functionality of alpha out.

Files that have been modified / default switches to Devfile:

List of commands confirmed working (with odo create nodejs --starter in the directory):

NOT WORKING:

Unconfirmed (please test @girishramnani & @dharmit):

odo catalog describe
odo catalog list services
odo catalog list components (make odo catalog list components work outside of experimental mode #3669) <- opened a PR
odo catalog delete

Preliminary steps:

Make sure that you have set your experimental preferences as false.

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo preference set experimental false
? experimental is already set. Do you want to override it in the config Yes
Global preference was successfully updated

Proof of concept:

`odo`

`odo create`

Works as intended

▶ odo create --starter nodejs
Validation
 ✓  Checking devfile existence [48701ns]
 ✓  Checking devfile compatibility [91890ns]
 ✓  Creating a devfile component from registry: DefaultDevfileRegistry [115381ns]
 ✓  Validating devfile component [273144ns]

Project
 ✓  Downloading project from https://github.com/odo-devfiles/nodejs-ex.git [289ms]

Please use `odo push` command to create the component with source deployed

`odo delete`

Works as intended

~/openshift/test                                                                                                                                                                     
▶ odo delete
? Are you sure you want to delete the devfile component: nodejs? Yes
 ✓  Deleting devfile component nodejs [105ms]
 ✓  Successfully deleted component

`odo exec`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo exec -- ls -lah
total 28K
drwxrwxr-x 1 default root 4.0K Jul 30 19:50 .
drwxrwxr-x 1 default root 4.0K Jun  4 10:38 ..
drwx------ 3 root    root 4.0K Jul 30 19:50 .config
drwxr-xr-x 4 default root 4.0K Jul 30 19:50 .npm
drwxr-xr-x 2 default root 4.0K Jul 30 19:50 .npm-global
drwxrwxr-x 1 default root 4.0K Jun  3 15:55 .pki

`odo link`

Unable to confirm functionality

`odo log`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo log
Dtime="2020-07-30T19:50:42Z" level=info msg="create process:devrun" 
Ftime="2020-07-30T19:50:42Z" level=info msg="create process:debugrun" 
@time="2020-07-30T19:50:45Z" level=debug msg="no auth required" 
Ptime="2020-07-30T19:50:45Z" level=info msg="stop the program" program=debugrun 
Ntime="2020-07-30T19:50:45Z" level=info msg="stop the program" program=devrun 
Ytime="2020-07-30T19:50:45Z" level=info msg="force to kill the program" program=debugrun 
Wtime="2020-07-30T19:50:45Z" level=info msg="force to kill the program" program=devrun 
@time="2020-07-30T19:50:45Z" level=debug msg="no auth required" 
Ntime="2020-07-30T19:50:45Z" level=debug msg="succeed to find process:devrun" 
Rtime="2020-07-30T19:50:45Z" level=info msg="try to start program" program=devrun 
Vtime="2020-07-30T19:50:45Z" level=info msg="success to start program" program=devrun 
ODO_COMMAND_RUN is npm start
Changing directory to /project
+Executing command cd /project && npm start

&> nodejs-starter@1.0.0 start /project
> node server.js

App started on PORT 3000
Ptime="2020-07-30T19:50:46Z" level=debug msg="wait program exit" program=devrun

`odo push`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo push

Validation
 ✓  Validating the devfile [24568ns]

Initialization
 ✓  Initializing the component [3s]

Creating Docker resources for component nodejs
 ✓  Pulling image registry.access.redhat.com/ubi8/nodejs-12:1-45 [2s]
 ✓  Starting container for registry.access.redhat.com/ubi8/nodejs-12:1-45 [532ms]

Syncing to component nodejs
 ✓  Checking files for pushing [871259ns]
 ✓  Syncing files to the component [407ms]

Executing devfile commands for component nodejs
 ✓  Executing install command "npm install" [3s]
 ✓  Executing run command "npm start" [1s]

Pushing devfile component nodejs
 ✓  Changes successfully pushed to component

`odo update`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo update
 ✗  'odo update' command is not available for Devfile based components.

`odo watch`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo watch
Waiting for something to change in /home/wikus/openshift/test
File /home/wikus/openshift/test/foo.txt changed
Pushing files...

Validation
 ✓  Validating the devfile [29845ns]

Creating Kubernetes resources for component nodejs
 ✓  Waiting for component to start [34ms]

Applying URL changes
 ✓  URLs are synced with the cluster, no changes are required.

Syncing to component nodejs
 ✓  Syncing files to the component [746ms]

Executing devfile commands for component nodejs
 ✓  Executing install command "npm install" [1s]
 ✓  Executing run command "npm start" [3s]
Waiting for something to change in /home/wikus/openshift/test

`odo debug`

`odo debug info`

`odo debug port-forward`

DOES NOT WORK.

▶ odo debug port-forward
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x80 pc=0x1790e86]

goroutine 1 [running]:
github.com/openshift/odo/pkg/debug.(*DefaultPortForwarder).ForwardPorts(0xc0004a63f0, 0xc000a14a60, 0x9, 0xc00053e300, 0xc00053e360, 0x0, 0x0, 0x22)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/pkg/debug/portforward.go:60 +0x56
github.com/openshift/odo/pkg/odo/cli/debug.PortForwardOptions.Run(0xc0005e3910, 0x6, 0x0, 0x0, 0xc0005e3928, 0x7, 0xc000a14a60, 0x9, 0x16e2, 0x0, ...)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/pkg/odo/cli/debug/portforward.go:168 +0x31c
github.com/openshift/odo/pkg/odo/genericclioptions.GenericRun(0x2005e80, 0xc00063e3f0, 0xc000163b80, 0x30b8240, 0x0, 0x0)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/pkg/odo/genericclioptions/runnable.go:31 +0x13c
github.com/openshift/odo/pkg/odo/cli/debug.NewCmdPortForward.func1(0xc000163b80, 0x30b8240, 0x0, 0x0)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/pkg/odo/cli/debug/portforward.go:181 +0x5e
github.com/openshift/odo/vendor/github.com/spf13/cobra.(*Command).execute(0xc000163b80, 0x30b8240, 0x0, 0x0, 0xc000163b80, 0x30b8240)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/vendor/github.com/spf13/cobra/command.go:830 +0x2aa
github.com/openshift/odo/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc00042c780, 0x1df24a8, 0xc0006bc6c0, 0x2)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/vendor/github.com/spf13/cobra/command.go:914 +0x2fb
github.com/openshift/odo/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/vendor/github.com/spf13/cobra/command.go:864
main.main()
        /home/wikus/syncthing/files/dev/go/src/github.com/openshift/odo/cmd/odo/odo.go:67 +0x330

`odo service`

`odo service create`

Unable to confirm functionality. Please confirm @dharmit

`odo service delete`

Unable to confirm functionality. Please confirm @dharmit

`odo service list`

Unable to confirm functionality. Please confirm @dharmit

`odo url`

`odo url create`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                           ⍉
▶ odo url create
 ✓  URL nodejs-3000 created for component: nodejs

To apply the URL configuration changes, please use `odo push`

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo push

Validation
 ✓  Validating the devfile [142132ns]

Creating Kubernetes resources for component nodejs
 ✓  Waiting for component to start [40ms]

Applying URL changes
 ✓  URL nodejs-3000: http://nodejs-3000-testing.e8ca.engint.openshiftapps.com created

Syncing to component nodejs
 ✓  Checking file changes for pushing [3ms]
 ✓  Syncing files to the component [692ms]

Executing devfile commands for component nodejs
 ✓  Executing install command "npm install" [2s]
 ✓  Executing run command "npm start" [3s]

Pushing devfile component nodejs
 ✓  Changes successfully pushed to component

Proof of curl:

▶ curl http://nodejs-3000-testing.e8ca.engint.openshiftapps.com
Hello from Node.js Starter Application!%

`odo url delete`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ odo url delete nodejs-3000
? Are you sure you want to delete the url nodejs-3000 Yes
 ✓  URL nodejs-3000 removed from the env file

To delete the URL on the cluster, please use `odo push`

~/openshift/test                                                                                                                                                                                                                                                                                                                                                            
▶ push

~/openshift/test                                                                                                                                                                                                                                                                                                                                                           ⍉
▶ odo push

Validation
 ✓  Validating the devfile [144494ns]

Creating Kubernetes resources for component nodejs
 ✓  Waiting for component to start [43ms]

Applying URL changes
 ✓  URL nodejs-3000 successfully deleted

Syncing to component nodejs
 ✓  Checking file changes for pushing [2ms]
 ✓  No file changes detected, skipping build. Use the '-f' flag to force the build.

Pushing devfile component nodejs
 ✓  Changes successfully pushed to component

`odo url describe`

Works as intended

~/openshift/test                                                                                                                                                                                                                                                                                                                                                           ⍉
▶ odo url describe nodejs-3000
NAME            STATE      URL                                                          PORT     SECURE     KIND
nodejs-3000     Pushed     http://nodejs-3000-testing.e8ca.engint.openshiftapps.com     3000     false      route

`odo url list`

~/openshift/test                                                                                                                                                                                                                                                                                                                                                           ⍉
▶ odo url list
Found the following URLs for component nodejs
NAME            STATE      URL                                                          PORT     SECURE     KIND
nodejs-3000     Pushed     http://nodejs-3000-testing.e8ca.engint.openshiftapps.com     3000     false      route

cdrage · 2020-07-30T20:51:40Z

/retest

girishramnani · 2020-07-31T10:03:22Z

opened a PR for odo catalog list components #3669 @cdrage

**What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` > Documentation changes: Please include [skip ci] in your commit message as well /kind documentation [skip ci] **What does does this PR do / why we need it**: This PR adds documentation regarding each section of Devfile and what we support / do not support **Which issue(s) this PR fixes**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com>

**What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind documentation [skip ci] **What does does this PR do / why we need it**: Updates the file reference documentation to not have the introductory section and move unsupported features to the bottom. **Which issue(s) this PR fixes**: N/A **PR acceptance criteria**: N/A **How to test changes / Special notes to the reviewer**: N/A Signed-off-by: Charlie Drage <charlie@charliedrage.com>

cdrage · 2020-07-31T19:34:10Z

/retest

cdrage · 2020-07-31T19:36:05Z

This is ready for a quick test @girishramnani @kadel @dharmit @dev-gaur etc.

List of commands that DO and DO NOT work are located here: #3550

Remember that this PR is intended not to be merged into master, but into the alpha branch.

…eveloper#3649)

…per#3666)

* Make the file sync destination more consistent Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update tests to use PROJECTS_ROOT Signed-off-by: John Collier <John.J.Collier@ibm.com> * Fix workdir in tests Signed-off-by: John Collier <John.J.Collier@ibm.com> * Remove more references to nodejs-starter Signed-off-by: John Collier <John.J.Collier@ibm.com> * Update more sample devfiles Signed-off-by: John Collier <John.J.Collier@ibm.com>

…edhat-developer#3669) * make odo catalog list components work outside of experimental mode * moved the logic to cli * Ignore error while checking for imagestream support * rearranged the catalog list

* Validate exec subcommands in composite * Add integration test

**What type of PR is this?** > Uncomment only one ` /kind` line, and delete the rest. > For example, `> /kind bug` would simply become: `/kind bug` /kind feature **What does does this PR do / why we need it**: Makes Devfile the default deployment mechanism, removing S2I in favour of Devfile deployment. **Which issue(s) this PR fixes**: Closes redhat-developer#3550 **How to test changes / Special notes to the reviewer**: Run: ```sh odo preference set experimental false odo create --starter nodejs odo push ```

openshift-ci-robot · 2020-08-04T14:39:26Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign cdrage
You can assign the PR to them by writing /assign @cdrage in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-08-04T16:13:29Z

@cdrage: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/v4.5-integration-e2e	`5e1dc68`	link	`/test v4.5-integration-e2e`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot added the kind/feature Categorizes issue as a feature request. For PRs, that means that the PR is the implementation label Jul 23, 2020

openshift-ci-robot requested review from adisky and johnmcollier July 23, 2020 19:55

cdrage marked this pull request as draft July 23, 2020 19:55

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jul 23, 2020

cdrage force-pushed the devfile-default branch from 4676a3b to 5e30032 Compare July 24, 2020 13:54

cdrage mentioned this pull request Jul 27, 2020

Move the devfile feature set out of the experimental mode #3550

Closed

3 tasks

cdrage force-pushed the devfile-default branch from 5e30032 to 231bacd Compare July 30, 2020 18:47

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Jul 30, 2020

cdrage force-pushed the devfile-default branch from 231bacd to 2d61fdc Compare July 30, 2020 19:07

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Jul 30, 2020

cdrage marked this pull request as ready for review July 30, 2020 19:12

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jul 30, 2020

cdrage force-pushed the devfile-default branch 2 times, most recently from 4e7ead6 to 99ffd22 Compare July 30, 2020 19:24

cdrage force-pushed the devfile-default branch from 99ffd22 to 00bdf4a Compare July 30, 2020 20:37

cdrage force-pushed the devfile-default branch 2 times, most recently from 09a76bc to 262c6ea Compare July 31, 2020 14:09

cdrage added 2 commits July 31, 2020 10:45

cdrage force-pushed the devfile-default branch from 262c6ea to 2504a05 Compare July 31, 2020 19:33

girishramnani and others added 2 commits August 1, 2020 03:15

make odo delete work on s2i components in experimental mode (redhat-d…

380c21f

…eveloper#3649)

Fixes GetRunningPodNameByComponent for the CLI runners (redhat-develo…

63c22a7

…per#3666)

increase ResponseHeaderTimeout (redhat-developer#3674)

56f860b

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Aug 3, 2020

johnmcollier and others added 3 commits August 3, 2020 09:13

make odo catalog list components work outside of experimental mode (r…

eec1f66

…edhat-developer#3669) * make odo catalog list components work outside of experimental mode * moved the logic to cli * Ignore error while checking for imagestream support * rearranged the catalog list

Validate exec subcommands in a composite (redhat-developer#3658)

db5a56a

* Validate exec subcommands in composite * Add integration test

cdrage force-pushed the devfile-default branch from 2504a05 to 5e1dc68 Compare August 4, 2020 14:34

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. Required by Prow. label Aug 4, 2020

cdrage changed the base branch from master to alpha August 4, 2020 14:35

devang-gaur and others added 5 commits August 4, 2020 10:38

added --s2i flag for odo create command

3abf6cb

corrected error messages

134d7c7

Corrected flag description and validation

4e32656

added --s2i flag for odo create command

4c2bea3

cdrage force-pushed the devfile-default branch from 5e1dc68 to 2ed2d01 Compare August 4, 2020 14:39

cdrage merged commit 04cc724 into redhat-developer:alpha Aug 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make Devfile the default deployment mechanism for odo #3634

Make Devfile the default deployment mechanism for odo #3634

cdrage commented Jul 23, 2020

codecov bot commented Jul 24, 2020 •

edited

Loading

cdrage commented Jul 27, 2020

cdrage commented Jul 30, 2020 •

edited by girishramnani

Loading

cdrage commented Jul 30, 2020

girishramnani commented Jul 31, 2020

cdrage commented Jul 31, 2020

cdrage commented Jul 31, 2020

openshift-ci-robot commented Aug 4, 2020

openshift-ci-robot commented Aug 4, 2020

Make Devfile the default deployment mechanism for odo #3634

Make Devfile the default deployment mechanism for odo #3634

Conversation

cdrage commented Jul 23, 2020

codecov bot commented Jul 24, 2020 • edited Loading

Codecov Report

cdrage commented Jul 27, 2020

cdrage commented Jul 30, 2020 • edited by girishramnani Loading

Files that have been modified / default switches to Devfile:

Preliminary steps:

Proof of concept:

odo

odo create

odo delete

odo exec

odo link

odo log

odo push

odo update

odo watch

odo debug

odo debug info

odo debug port-forward

odo service

odo service create

odo service delete

odo service list

odo url

odo url create

odo url delete

odo url describe

odo url list

cdrage commented Jul 30, 2020

girishramnani commented Jul 31, 2020

cdrage commented Jul 31, 2020

cdrage commented Jul 31, 2020

openshift-ci-robot commented Aug 4, 2020

openshift-ci-robot commented Aug 4, 2020

codecov bot commented Jul 24, 2020 •

edited

Loading

cdrage commented Jul 30, 2020 •

edited by girishramnani

Loading

`odo`

`odo create`

`odo delete`

`odo exec`

`odo link`

`odo log`

`odo push`

`odo update`

`odo watch`

`odo debug`

`odo debug info`

`odo debug port-forward`

`odo service`

`odo service create`

`odo service delete`

`odo service list`

`odo url`

`odo url create`

`odo url delete`

`odo url describe`

`odo url list`