Dynamic registry support

[GeekArthur]

Signed-off-by: jingfu wang jingfu.j.wang@ibm.com

What type of PR is this?
/kind feature

What does does this PR do / why we need it:
Currently we only support static registry, which means user only can create component with devfile that is hosted on our two predefined registries:
Che devfile registry: https://github.com/eclipse/che-devfile-registry/tree/master/devfiles
Private devfile registry: https://github.com/elsony/devfile-registry/tree/master/devfiles

This PR is to implement dynamic registry support, with this PR users can easily configure their own registries with the following operations:

• Add registry to the registry list
• List registry in the registry list
• Delete registry from the registry list
• Update registry in the registry list
• List component with corresponding registry
• Create component that is hosted by specific registry

Which issue(s) this PR fixes:
Fixes #2635

How to test changes / Special notes to the reviewer:

# Add registry to the registry list
odo registry add <registry name> <registry URL>

# List registry in the registry list
odo registry list

# Delete registry from the registry list
odo registry delete <registry name>

# Update registry in the registry list
odo registry update <registry name> <registry URL>

# List component with corresponding registry
odo catalog list components

# Create component that is hosted by specific registry
odo create <component type> --registry <registry name>

Special notes:

• For testing purpose, you can use my personal registry https://raw.githubusercontent.com/GeekArthur/registry/master (eg. odo registry add MyRegistry https://raw.githubusercontent.com/GeekArthur/registry/master)

[GeekArthur]

I will add unit tests and integration tests in my following commit of this PR.

[GeekArthur]

/assign

[GeekArthur]

/area devfile

[codecov]

Codecov Report

Merging #2940 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2940   +/-   ##
=======================================
  Coverage   45.50%   45.50%           
=======================================
  Files         109      109           
  Lines       10501    10501           
=======================================
  Hits         4778     4778           
  Misses       5264     5264           
  Partials      459      459           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0fc8488...0fc8488. Read the comment docs.

[elsony]

We also need to handle gracefully if the user already have a preference.yaml file but have no registry entry, i.e. a preference file created in previous version of odo. In that case, we can just default back to the default list of repos (instead of empty); otherwise, the user will need to workaround the case to manually add the default list of repos before they can get odo catalog list components to work again after they update to the latest version of odo.

[GeekArthur]

We also need to handle gracefully if the user already have a preference.yaml file but have no registry entry, i.e. a preference file created in previous version of odo. In that case, we can just default back to the default list of repos (instead of empty); otherwise, the user will need to workaround the case to manually add the default list of repos before they can get odo catalog list components to work again after they update to the latest version of odo.

Yup, my latest commit will handle the following two cases regarding new odo version migration and add default registries for user:

• User doesn't have preference.yaml file
• User has preference.yaml file but doesn't use dynamic registry feature before

[groeges]

If we are adding the capability to add registries, should we still be providing a default that points to a personal git repository (ie Elson's)?

I can see how having this temporarily to provide the newer registry index.json might have been a good thing but with the new functionality I don't think having a personal git repository listed in the code is OK.

[GeekArthur]

If we are adding the capability to add registries, should we still be providing a default that points to a personal git repository (ie Elson's)?

I can see how having this temporarily to provide the newer registry index.json might have been a good thing but with the new functionality I don't think having a personal git repository listed in the code is OK.

The original idea regarding default registry was to let it host on personal git repository for tech preview as it's easier to manage and manipulate for both testing and development. Given we have plan for support devfile 2.0, so one option is we can do the default registry migration with devfile 2.0 support together. I can bring this up in our status meeting and discuss:

• Where we should host default registry publicly
• Do we need to create corresponding account or we can use the existing one
• How long it takes to do the default registry migration

[GeekArthur]

@groeges I open an issue to track the default registry migration: #2961

[GeekArthur]

/retest

[GeekArthur]

/retest

[GeekArthur]

/retest

[dharmit]

@GeekArthur Thanks for the discussions and accommodating the requested changes.

/lgtm

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[GeekArthur]

/retest

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-ci-robot]

@GeekArthur: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/v4.4-integration-e2e-benchmark	6eb07c2	link	/test v4.4-integration-e2e-benchmark

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.