odo service create should add operator CR to the devfile

[girishramnani]

/kind user-story

User Story

As a developer working with odo, when I add a new service to the application (like DB), I want to be able to easily share service definitions with my team so they can start the application with the service in the same way as me.

Acceptance Criteria

• odo service create should not create any services on cluster (Creates service on cluster upon "odo push" #4650)
• odo service create should store service definition in devfile as kubernetes component (ServiceInstance for SericeCatalog or CR for operator backed service) (Add and remove service info from devfile #4465)
• odo service delete should delete the service from devfile (remove appropriate kubernetes component). Deletion from the cluster will happen once odo push is executed (similar to how we handle URLs).
• odo service list should list services and their status. Important is to differentiate between statues like DeletedLocally, DeletedRemotely etc... (Similarly to what odo does for URLs)
  odo link should add Service Binding information to devfile.yaml and odo unlink should remove it from the file. Subsequent odo push should create/delete the actual link (a Service Binding). (being tracked by odo link should store link information in devfile #4208)

Links

• Feature Request: Support Kubernetes Devfile component #4159
• Related issue:

/kind user-story

[dharmit]

It seems the Uri holds a link to the yaml or we can provide the yaml as a string in Inlined. both seem mutually exclusive.

I'm not sure what purpose does having both Inlined and Uri serve. When I read this, I think of Inlined as a string representation of the CRD used to spin up a CR. It would be the same as what we have right now when doing odo service create <operator-name>/<cr-name> --dry-run. Then what is Uri/reference in this case?

What is odo supposed to do with name? At the moment, we have odo service create etcdoperator.v0.9.4/EtcdCluster newetcd which creates a new service from the CR EtcdCluster provided by the Operator etcdoperator.v0.9.4 and names it as newetcd. Should odo use the value in name for the newly created service?

devfile/api#36

This is a proposal, IIUC. Do we have to implement things based on this proposal? Or do we use Kubernetes/OpenShift lingo instead? If we should be using Kubernetes/OpenShift based on current devfile schema, what happens when this change goes in? We make changes to work with the new terminology?

Few other questions (more along the lines of "thinking out loud"):

• What's the timeline for this? It seems more like an epic than a user story to me.
• What about Service Catalog backed services? Are we planning to support those in the new approach?
• What about the combination of "s2i components" + "Service Catalog services"? Are we planning to support that using existing tooling using if..else that we had for experimental mode?
• How does this play with devfile parser that's being moved out to a separate project? What parts are supposed to be handled by devfile parser and what is to be handled by odo?

[girishramnani]

Then what is Uri/reference in this case?

I think inline and Uri/reference are mutually exclusive. i.e. if you provide inline then you then the uri would be ignored.

• What's the timeline for this? It seems more like an epic than a user story to me.

well I was hoping for this to be done in this sprint? only the devfile + operator backed service and that too just adding the CR into the devfile and still do that original work so we dont break operator backed services. Essentially
do what odo service create does and also store the CR in the devfile for now.

• What about Service Catalog backed services? Are we planning to support those in the new approach?

Yeah that is something I was thinking alot about and right now I would say no.

Some more thinking on what kind of combination we are looking at

• devfile component + operator backed service - scoped in this issue

• devfile component + service catalog services - we could store the service template in the devfile directly

• s2i component + operator backed service - we dont have a devfile for s2i component so we might need to add the CR in the config.yaml. So imagine an interface ServiceStorageProvider that can be implemented by a config.yaml and the devfile?

• s2i component + service catalog service - again store the service template in the config.yaml ( many other things are possible so think about this point )

I bet there is a newer service catalog CR someone would have implemented to move the service catalog into an operator but I haven't searched about it.

[dharmit]

Sorry about writing an essay. I have a lot of questions around this and I couldn't find a better way to put it.

I think inline and Uri/reference are mutually exclusive. i.e. if you provide inline then you then the uri would be ignored.

OK. And what would be a valid value for URI in this case? What would we like to store as URI?

About storing the CRD in devfile. Currently, we don't support passing parameters for odo service create when the user wants to spin up an Operator backed service. I think, storing CRD in devfile would be a lot more valuable when we have this support.

Then there's also this notion of keeping the information in devfile compatible with what Che might need.

well I was hoping for this to be done in this sprint? only the devfile + operator backed service and that too just adding the CR into the devfile and still do that original work so we dont break operator backed services. Essentially do what odo service create does and also store the CR in the devfile for now.

If we put the CRD (the definition used to bring up the CR, a.k.a. service, provided by an Operator) in the devfile when doing odo service create, and if odo push fails to bring up the service for some reason, what do we do about the info that's gone into devfile? Should we delete it. What if this happens when user2 is cloning the code provided by user1 and odo push fails to spin up the service because the required Operator doesn't exist on their environment? Deleting the CRD from devfile might be a bad idea in that case. Also, does thinking of failure scenarios from odo watch's perspective add any other challenges?

Also, if I understand correctly, you mean to only add the CRD into devfile as Step 1 and not changing the flow to create the service when doing odo push. Sounds doable.

Yeah that is something I was thinking alot about and right now I would say no.

OK. So what would be our approach to handle it? Separate code (as is the case right now)? Do we want to provide different UX to the users when they spin up a service using Operators vs. Service Catalog? By different approach, I mean that if we store CRD info in devfile, we eventually want to provide the ability to create a component+service in one odo push when, say, user2 clones the repo on their system. But this wouldn't hold true for Service Catalog based services.

From k8s point of view, I don't think we will/should be deprecating support for Service Catalog in a rush. OTOH, we could also deprecate it since it's no longer available on OCP and focus solely on Operators. This reduces the scope of bugs in odo code. IMO, we should think this hard because supporting both requires considering a lot of permutations and combinations.

This piece is confusing and I'd prefer we think of reaching a conclusion sooner rather than later.

• devfile component + operator backed service - scoped in this issue

This means storing the CRD in the devfile. It doesn't mean actually spinning up a service when doing odo push after odo service create stores the info in devfile. Should we add a few unambiguous acceptance criteria to this issue?

• devfile component + service catalog services - we could store the service template in the devfile directly

At the moment, this scenario is not supported. I think I need to come up with a matrix about all the possibilities and their support status. Otherwise, this is quickly going to become insane.

• s2i component + operator backed service - we dont have a devfile for s2i component so we might need to add the CR in the config.yaml. So imagine an interface ServiceStorageProvider that can be implemented by a config.yaml and the devfile?
• s2i component + service catalog service - again store the service template in the config.yaml ( many other things are possible so think about this point )

We discussed under the hood conversion of s2i components to devfile on the Cabal. So I think this needs a rethink?

I bet there is a newer service catalog CR someone would have implemented to move the service catalog into an operator but I haven't searched about it.

I'm not so sure about the CR/Operator for Service Catalog. But I think we can give a try to the steps mentioned here on a minikube setup. But again, I'd prefer we discuss how long we intend to support it. And I am seeking this info only because OCP doesn't support it anymore.

[girishramnani]

Sorry about writing an essay. I have a lot of questions around this and I couldn't find a better way to put it.

I think inline and Uri/reference are mutually exclusive. i.e. if you provide inline then you then the uri would be ignored.

OK. And what would be a valid value for URI in this case? What would we like to store as URI?

It would be a public link to a kubernetes manifest and when there is something inline then the uri would be blank.
reference: https://.../mongo.yaml

About storing the CRD in devfile. Currently, we don't support passing parameters for odo service create when the user wants to spin up an Operator backed service. I think, storing CRD in devfile would be a lot more valuable when we have this support.

Agreed but for now lets not add the functionality of passing parameters - first step is simple and that is store the CRD in the devfile then maybe we can think about this

Then there's also this notion of keeping the information in devfile compatible with what Che might need.

well I was hoping for this to be done in this sprint? only the devfile + operator backed service and that too just adding the CR into the devfile and still do that original work so we dont break operator backed services. Essentially do what odo service create does and also store the CR in the devfile for now.

If we put the CRD (the definition used to bring up the CR, a.k.a. service, provided by an Operator) in the devfile when doing odo service create, and if odo push fails to bring up the service for some reason, what do we do about the info that's gone into devfile? Should we delete it. What if this happens when user2 is cloning the code provided by user1 and odo push fails to spin up the service because the required Operator doesn't exist on their environment? Deleting the CRD from devfile might be a bad idea in that case. Also, does thinking of failure scenarios from odo watch's perspective add any other challenges?

I asked the same question to tomas and he mentioned that the person who added the service operator in the devfile knows that their application wont function correctly without it so the person cloning the application should have the operator present and if not - its a good thing that push fails. We need to provide good error messages here though

Also, if I understand correctly, you mean to only add the CRD into devfile as Step 1 and not changing the flow to create the service when doing odo push. Sounds doable.

Yes as this flow of storing the CRD in the devfile is still a smaller part of the whole puzzle we need to keep the original work intact until the new flow is completely implemented

Yeah that is something I was thinking alot about and right now I would say no.

OK. So what would be our approach to handle it? Separate code (as is the case right now)? Do we want to provide different UX to the users when they spin up a service using Operators vs. Service Catalog? By different approach, I mean that if we store CRD info in devfile, we eventually want to provide the ability to create a component+service in one odo push when, say, user2 clones the repo on their system. But this wouldn't hold true for Service Catalog based services.

From k8s point of view, I don't think we will/should be deprecating support for Service Catalog in a rush. OTOH, we could also deprecate it since it's no longer available on OCP and focus solely on Operators. This reduces the scope of bugs in odo code. IMO, we should think this hard because supporting both requires considering a lot of permutations and combinations.

This piece is confusing and I'd prefer we think of reaching a conclusion sooner rather than later.

• devfile component + operator backed service - scoped in this issue

This means storing the CRD in the devfile. It doesn't mean actually spinning up a service when doing odo push after odo service create stores the info in devfile. Should we add a few unambiguous acceptance criteria to this issue?

• devfile component + service catalog services - we could store the service template in the devfile directly

At the moment, this scenario is not supported. I think I need to come up with a matrix about all the possibilities and their support status. Otherwise, this is quickly going to become insane.

• s2i component + operator backed service - we dont have a devfile for s2i component so we might need to add the CR in the config.yaml. So imagine an interface ServiceStorageProvider that can be implemented by a config.yaml and the devfile?
• s2i component + service catalog service - again store the service template in the config.yaml ( many other things are possible so think about this point )

We discussed under the hood conversion of s2i components to devfile on the Cabal. So I think this needs a rethink?

I bet there is a newer service catalog CR someone would have implemented to move the service catalog into an operator but I haven't searched about it.

I'm not so sure about the CR/Operator for Service Catalog. But I think we can give a try to the steps mentioned here on a minikube setup. But again, I'd prefer we discuss how long we intend to support it. And I am seeking this info only because OCP doesn't support it anymore.

[dharmit]

OK. And what would be a valid value for URI in this case? What would we like to store as URI?

It would be a public link to a kubernetes manifest and when there is something inline then the uri would be blank.
reference: https://.../mongo.yaml

👍 makes sense.

About storing the CRD in devfile. Currently, we don't support passing parameters for odo service create when the user wants to spin up an Operator backed service. I think, storing CRD in devfile would be a lot more valuable when we have this support.

Agreed but for now lets not add the functionality of passing parameters - first step is simple and that is store the CRD in the devfile then maybe we can think about this

Ah, I didn't mean to stall this till we have parameters support. Totally agree about adding CRD info into devfile as a first step.

If we put the CRD (the definition used to bring up the CR, a.k.a. service, provided by an Operator) in the devfile when doing odo service create, and if odo push fails to bring up the service for some reason, what do we do about the info that's gone into devfile? Should we delete it. What if this happens when user2 is cloning the code provided by user1 and odo push fails to spin up the service because the required Operator doesn't exist on their environment? Deleting the CRD from devfile might be a bad idea in that case. Also, does thinking of failure scenarios from odo watch's perspective add any other challenges?

I asked the same question to tomas and he mentioned that the person who added the service operator in the devfile knows that their application wont function correctly without it so the person cloning the application should have the operator present and if not - its a good thing that push fails. We need to provide good error messages here though

OK. So if the push fails for user2, what do we want the state of the component to be? Only the service creation on the cluster fails for them or does the entire push become invalid? I get it that this is not a part of the first step we just agreed on above, but this is going to be the next step, IIUC.

Other questions remain unanswered. Can we discuss this as well because there are a lot of possible scenarios that we need to think of? cc @kadel

In the meantime, I'll start looking into adding just the CRD to the devfile.

[kadel]

The first thing that needs to be addressed is #4159
Otherwise, services stored in Devfile can't be created.

Once we have that odo service create should be quite simple.
Instead of creating Kubernetes resources in cluster it will save it into devfile as inlined kubernetes component

For example command odo service create mongodb-enterprise.v1.8.0/MongoDB mymongo will add a new component into Devfile that looks like this:

components:
  - name: mymongo
    kubernetes:
      inlined: >
        apiVersion: mongodb.com/v1
        kind: MongoDB
        metadata:
          name: mymongo
        spec:
          credentials: my-credentials
          members: 3
          opsManager:
            configMapRef:
              name: my-project
          persistent: true
          type: ReplicaSet
          version: 4.4.0-ent
        status:
          phase: Running
          type: ReplicaSet

odo service create etcdoperator.v0.9.4-clusterwide/EtcdCluster myetcd --dry-run > myetcd.yaml
odo service create --from-file myetcd.yaml

Will save the content of myetcd.yaml as a new kubernetes component in Devfile

components:
  - name: myetcd
    kubernetes:
      inlined: >
        apiVersion: etcd.database.coreos.com/v1beta2
        kind: EtcdCluster
        metadata:
          annotations:
            etcd.database.coreos.com/scope: clusterwide
          name: myetcd
        spec:
          size: 3
          version: 3.2.13

Similarly with service catalog
odo service create mariadb --plan 10-3-22 --parameters param-1=value-1 --parameters param-2=value-2

should result in adding the following kubernetes component into the devfile

components:
  - name: mymaria
    kubernetes:
      inlined: >
        apiVersion: servicecatalog.k8s.io/v1beta1
        kind: ServiceInstance
        metadata:
          name: mymaria
          namespace: tkral-test
        spec:
          clusterServiceClassExternalName: mariadb
          clusterServicePlanExternalName: 10-3-22
          parameters:
            param-1: value-1
            param-2: value-2

But again, before we tackle this we need to make sure that odo understands kubernetes components
#4159. This needs to be the first step.

[dharmit]

odo service create etcdoperator.v0.9.4-clusterwide/EtcdCluster myetcd --dry-run > myetcd.yaml
odo service create --from-file myetcd.yaml

We can get rid of the --dry-run and --from-file entirely because odo service create will store the CR in the devfile so a user can open the devfile and edit it before doing a odo push if they want to. And --from-file makes no sense either once we start storing the CRD in devfile.

But how/where do we want to store link related information?

[kadel]

But how/where do we want to store link related information?

We need to create a separate issue for this. (just created #4208)

The link will come as next after we have #4159 and this one.

It will work similarly as odo service create. odo link will store ServiceBinding CR in devfile as a kubernetes component. And then it will be created later by running odo push.

[dharmit]

This will not be possible to do before #4159 which is in turn dependent on devfile/api#216.

[dharmit]

odo push should create the service on the cluster instead of odo service create doing it right now.

I'm considering adding the code to create service right before https://github.com/openshift/odo/blob/16946d4068f30f1e51cecc64877e535338b043b2/pkg/devfile/adapters/kubernetes/component/adapter.go#L175-L178

Thinking along these lines because there's a call to the function WaitForDeploymentRollout. And I'm taking it literally as a function that will wait for something to complete.

@kadel @mik-dass @girishramnani @valaparthvi if you have a differing opinion/suggestion, let me know.

[dharmit]

Scope of this issue for Sprint 201:

• odo service list should show output like below from the component directory of component-1:

  $ odo service list
  NAME                       MANAGED BY ODO    STATE
  EtcdCluster/etcd-1         component-1       Pushed
  EtcdCluster/etcd-2         component-2       Pushed
  EtcdCluster/etcd-3         component-1       Not Pushed
  EtcdCluster/etcd-4         component-1       Deleted Locally

• odo service delete should delete the service only from devfile.yaml; odo push should delete the service from the cluster.
• Tests

I'll defer documentation till we have properly defined structure via #4664.

[valaparthvi]

  ```shell
  $ odo service list
  NAME                       MANAGED BY ODO    STATE
  EtcdCluster/etcd-1         component-1       Pushed
  EtcdCluster/etcd-2         component-2       Pushed
  EtcdCluster/etcd-3         component-1       Not Pushed
  EtcdCluster/etcd-4         component-1       Deleted Locally
  ```

@feloy I noticed that the state doesn't show up when running this command outside of a component directory. I don't think I have a question here, just an fyi, or is this expected for now and we'll be iterating over it?

➜  odo git:(pr/4819) odo service list
NAME               MANAGED BY ODO     STATE     AGE
Database/bkddb     Yes (bkd)                    15m22s