odo default registry migration/update

[maysunfaisal]

/kind user-story

User Story

As a user I should be able to get the updated default registry whenever the default registries change in odo.

This was first noticed when adding the default devfile v2 registry https://github.com/elsony/devfile2-registry

Case 1: User does not have registries in preference. Automatically Updated!

No registries in preference:

$ cat ~/.odo/preference.yaml 
kind: Preference
apiversion: odo.dev/v1alpha1
OdoSettings:
  Experimental: true

Before adding a new default registry in odo:

$ odo registry list
NAME                         URL
CheDevfileRegistry           https://che-devfile-registry.openshift.io
DefaultDevfileRegistry       https://raw.githubusercontent.com/elsony/devfile-registry/master
DefaultDevfileV2Registry     https://raw.githubusercontent.com/elsony/devfile2-registry/master

After adding a new default registry DefaultDevfileV2Registry2222 in odo:

$ odo registry list
NAME                             URL
CheDevfileRegistry               https://che-devfile-registry.openshift.io
DefaultDevfileRegistry           https://raw.githubusercontent.com/elsony/devfile-registry/master
DefaultDevfileV2Registry2222     https://raw.githubusercontent.com/elsony/devfile2-registry/master

The change is picked up automatically 👍

Case 2: User does have registries in preference. Needs Manual Update!

Registries in preference:

$ cat ~/.odo/preference.yaml 
kind: Preference
apiversion: odo.dev/v1alpha1
OdoSettings:
  Experimental: true
  RegistryList:
  - Name: CheDevfileRegistry
    URL: https://che-devfile-registry.openshift.io
  - Name: DefaultDevfileRegistry
    URL: https://raw.githubusercontent.com/elsony/devfile-registry/master
  - Name: DefaultDevfileV2Registry2222
    URL: https://raw.githubusercontent.com/elsony/devfile2-registry/master
  - Name: maysunv2
    URL: https://raw.githubusercontent.com/maysunfaisal/devfile2-registry/master

Before adding a new default registry in odo:

$ odo registry list
NAME                             URL
CheDevfileRegistry               https://che-devfile-registry.openshift.io
DefaultDevfileRegistry           https://raw.githubusercontent.com/elsony/devfile-registry/master
DefaultDevfileV2Registry2222     https://raw.githubusercontent.com/elsony/devfile2-registry/master
maysunv2                         https://raw.githubusercontent.com/maysunfaisal/devfile2-registry/master

After adding a new default registry DefaultDevfileV2Registry3333 in odo:

$ odo registry list
NAME                             URL
CheDevfileRegistry               https://che-devfile-registry.openshift.io
DefaultDevfileRegistry           https://raw.githubusercontent.com/elsony/devfile-registry/master
DefaultDevfileV2Registry2222     https://raw.githubusercontent.com/elsony/devfile2-registry/master
maysunv2                         https://raw.githubusercontent.com/maysunfaisal/devfile2-registry/master

It still has the old registry. The change is not picked up automatically 👎

Workaround to update:

Delete the preference from home dir

$ rm -rf ~/.odo/

Set up new preference

$ odo preference set experimental true
Global preference was successfully updated

New default registry is now reflected

$ odo registry list
NAME                             URL
CheDevfileRegistry               https://che-devfile-registry.openshift.io
DefaultDevfileRegistry           https://raw.githubusercontent.com/elsony/devfile-registry/master
DefaultDevfileV2Registry3333     https://raw.githubusercontent.com/elsony/devfile2-registry/master

Acceptance Criteria

• It should update the default registry automatically when odo changes its internal default

Links

• Feature Request:
• Related issue: Support group in devfile commands (application lifecycle) #2938

[johnmcollier]

This one’s tricky because it can be hard to tell if a missing default registry is due to a new one being added in odo, or if the user chose to remove it. We don’t want to be adding back default registries that the user explicitly removed.

Maybe we need to consider not ever storing the default registries in the preferences.yaml (but still providing a way to disable them). Then we wouldn’t hit the issue of a user not getting new default registries?

@maysunfaisal WDYT?

[maysunfaisal]

Not sure if we want to allow users to disable default but not storing default in preference.yaml is one way of solving this 🤔

[johnmcollier]

We allow users today to remove/disable the default registries, don’t we?

[GeekArthur]

Exactly, this one is tricky. We do have a mechanism to cover the cases below:

1. User never uses dynamic feature -> add default registries to preference.yaml automatically
2. User already use dynamic feature -> use the existing registries in preference.yaml directly

But we don't cover the case where users missing one new default registry for devfile v2 (either user doesn't have it or user has it but remove it), if we want to still use the same mechanism above for supporting migration we can provide a one time migration script/way to force user to only add the new devfile v2 registry, then we don't need to change the existing code base.

Another alternative is that we can merge devfile v1 registry and devfile v2 registry to let one GitHub repo hosts two registries, then we don't have this migration issue.

Regarding @johnmcollier 's question

We allow users today to remove/disable the default registries, don’t we?

Yes, we allow

[johnmcollier]

Another alternative is that we can merge devfile v1 registry and devfile v2 registry to let one GitHub repo hosts two registries, then we don't have this migration issue.

Merging the devfile registries won't prevent the issue from occurring again if the default devfile registry URLs are changed (or new default ones are added)

[maysunfaisal]

Btw, yes we allow delete on default registries. Ignore my prev comment regd it.

[GeekArthur]

Another alternative is that we can merge devfile v1 registry and devfile v2 registry to let one GitHub repo hosts two registries, then we don't have this migration issue.

Merging the devfile registries won't prevent the issue from occurring again if the default devfile registry URLs are changed (or new default ones are added)

For that alternative, definitely there are some limitations. If want to make it work, we have to always use the same URL and only provide one default registry. All changes will be made on upstream/GitHub side such as creating separate folders to host v1 and v1 devfiles under one GitHub repo, and no new default registry can be added in the future.

[kadel]

Maybe we need to consider not ever storing the default registries in the preferences.yaml (but still providing a way to disable them). Then we wouldn’t hit the issue of a user not getting new default registries?

I think that this might be a good way forward. We would just have a preference that would indicate if the default registry should be used or not, but the actual URL would be in the code.
But default the default registry would be used, and optionally user could do something like odo preference set UseDefaultRegistry false to disable it.

To migrate existing users to this new approach, we could simply delete DefaultDevfileRegistry entry from the preference if it exists.

BTW: I've forked @elsony's devfile v2 registry into https://github.com/odo-devfiles/registry we could start using that to avoid using personal github namespaces.

[kadel]

There is already another issue related to this. #3249
We might want to merge discussion into one single issue.

[GeekArthur]

Maybe we need to consider not ever storing the default registries in the preferences.yaml (but still providing a way to disable them). Then we wouldn’t hit the issue of a user not getting new default registries?

I think that this might be a good way forward. We would just have a preference that would indicate if the default registry should be used or not, but the actual URL would be in the code.
But default the default registry would be used, and optionally user could do something like odo preference set UseDefaultRegistry false to disable it.

To migrate existing users to this new approach, we could simply delete DefaultDevfileRegistry entry from the preference if it exists.

BTW: I've forked @elsony's devfile v2 registry into https://github.com/odo-devfiles/registry we could start using that to avoid using personal github namespaces.

This is a good approach, one disadvantage I can think of is that user couldn't remove specific default registry as we don't store the URL on disk, IIUC we only provide option to let user enable/disable all default registries.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[kadel]

/remove-lifecycle rotten
/lifecycle frozen

this is still valid

[kadel]

This is no longer valid in odo v3.

/close

[openshift-ci]

@kadel: Closing this issue.

In response to this:

This is no longer valid in odo v3.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.