Inline Che Theia plugins and Che Editors definitions in devfile v2

[sunix]

Is your enhancement related to a problem? Please describe.

At the moment, adding a cheplugin is requiring the user to have a plugin registry and point the meta.yaml to use to another external http available resource.

Describe the solution you'd like

It would be nice if we could inline the definition of a Che Theia plugin or of a Che editor in a v2 devfile (syntax details are in this other issue):

schemaVersion: 2.1.0  
metadata:  
  name: my-example  
attributes:  
  .che/che-theia-plugins.yaml: |  
    - id: redhat/java
  .vscode/extensions.json: |  
    {  
      "recommendations": [  
        "ms-vscode.vscode-github-pullrequest"  
      ]  
    }
  .che/che-editor.yaml: |  
      id: che-incubator/che-code/insiders
components:
(...)

Once we have that, we could even imagine composing new workspace by adding custom cheplugins from the dashboard: would just choose a container image and 1+ vsx extensions from openvsx marketplace

[l0rd]

@sunix this is a good suggestion. We need to think if it's worth adding it to the v1.0 spec or we include it in the devfile 2.0 spec.

[sunix]

@l0rd I would say ASAP so both ... this is a high value enhancement and low cost as we have almost everything in place to achieve that. Would make testing new vscode extensions or sharing unsupported vscode extension or che plugin in draft mode a lot easier.
To me this is important as it will help to show the value of Che, running vscode exts in sidecar containers, to users and futur contributors.

[sunix]

devfile 2.0 devfile/api#95

[themr0c]

I agree it would be really a very nice feature. It feels awkward to have the creation of a custom plug-in registry (administrator task, out of the hands of an end-user) as a dependency to add an IDE extension (routine end-user task).

[ericwill]

I agree it would be really a very nice feature. It feels awkward to have the creation of a custom plug-in registry (administrator task, out of the hands of an end-user) as a dependency to add an IDE extension (routine end-user task).

There is another way, and that is putting a meta.yaml somewhere (GH branch, or GH gist) and pointing to that in your devfile.

https://www.eclipse.org/che/docs/che-7/administration-guide/using-a-visual-studio-code-extension-in-che/#adding-the-vs-code-extension-using-the-workspace-configuration_adding-che-plug-in-registry-vs-code-extension-to-a-workspace

[sunix]

I agree it would be really a very nice feature. It feels awkward to have the creation of a custom plug-in registry (administrator task, out of the hands of an end-user) as a dependency to add an IDE extension (routine end-user task).

There is another way, and that is putting a meta.yaml somewhere (GH branch, or GH gist) and pointing to that in your devfile.

https://www.eclipse.org/che/docs/che-7/administration-guide/using-a-visual-studio-code-extension-in-che/#adding-the-vs-code-extension-using-the-workspace-configuration_adding-che-plug-in-registry-vs-code-extension-to-a-workspace

Ok it works but hosting meta.yaml somewhere else is still painful. It is adding x2 confusion and complexity compared to this proposal.

[ericwill]

I agree, I was more suggesting that in the meantime we shouldn't encourage people to deploy an entire registry just for one extension if the hosted meta.yaml way also works. At least until we have a fix for this ticket.

[themr0c]

https://www.eclipse.org/che/docs/che-7/administration-guide/using-a-visual-studio-code-extension-in-che/#adding-the-vs-code-extension-using-the-workspace-configuration_adding-che-plug-in-registry-vs-code-extension-to-a-workspace I read:

"Change the current registry to the registry in which the VS Code extension was added."

So this is an erroneous statement.

[themr0c]

I'll try to address that in eclipse-che/che-docs#1633

[mickaelistria]

+1, I think that would be consistent with other IDEs if we want devfile to address them. The devfile could then contain for each IDE a description of the recommended extensions to install in a format that's relatively native to them. For Che, it would be content of a meta.yaml, for Eclipse IDE or VSCode it'd be some of their marketplace entries, for Theia some vsix URLs and so on...

[l0rd]

@sunix I have updated the description/title of this issue to make it clear that we will implement it in the context of the devfile v2.

[sunix]

@l0rd ok feedback for the format:

schemaVersion: 2.0.0
attributes:
  che-editor.yaml:
    <yaml content>
  che-theia-plugins.yaml:
    <yaml content>

As a che user, I am confused seeing file names in the devfile: "che-editor.yaml" or "che-theia-plugins.yaml" what are these file names referring to ?

[l0rd]

@sunix those are the new meta.yaml (c.f. the issue where those files are described)

[sunix]

@l0rd
My point is that there is too many indirections to files and registries etc. If we could make it simpler to read and to compose.

As I understand it is not a replacement of meta.yaml as those would stay in the registries. We are just adding an extra layer and more files to deal with.

I would love to see a comparaison with some projects like: https://github.com/sunix/scratch-kubernetes or https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app. If a 2.0 devfile is harder to read and maintain and if it is even bigger. I don't see the point.

[l0rd]

I would love to see a comparaison with some projects like: sunix/scratch-kubernetes or redhat-developer-demos/quarkus-reactjs-postit-app. If a 2.0 devfile is harder to read and maintain and if it is even bigger. I don't see the point.

Are we discussing inline here or are we discussing how to reference plugins when using a devfile v2?
I am asking because your examples are not about inlining and this issue is about inlining.

[sunix]

these are all related, I am just giving my feedback here because I see the file names in your proposal and it looks too complex having 2 more files references. But maybe I am wrong, and that would be fine for users that will use Che and other editors. So I hope it will work if we go that way.

[sunix]

For vscode, if we say that we use .vscode it means that the devfile with inlining could override .vscode? same for .idea

[sunix]

also at some point, Some may just want to inline 1 plugin and not all. So that would be harder with this proposal.

[ericwill]

also at some point, Some may just want to inline 1 plugin and not all. So that would be harder with this proposal.

I don't believe you have to inline all entries, if you want to inline just one plugin you only inline the che-theia-plugins.yaml file with one entry.

[sunix]

Let's say you have these plugins: https://github.com/redhat-developer-demos/quarkus-reactjs-postit-app/blob/master/devfile.yaml#L99-L118
You have all of these in a che-theia-plugins.yaml file. And you decide to inline the openshift-connector plugin. With this format, the logical way to do is to copy paste the whole che-theia-plugins.yaml in the devfile and modify just the openshift-connector part providing overrides.
The original goal of this issue was to inline what we had in a meta.yaml file. But the new proposal is to inline che-theia-plugins.yaml. The original proposal is covered with the override capabilities in che-theia-plugins.yaml.

This is all to say that it is not simplifying but making more complex for user to understand and to use.

We used to have meta.yaml and devfile.yaml. Now we will have meta.yaml, devfile.yaml, che-theia-plugins.yaml, etc ... It will be hard not to get confused.

[benoitf]

I guess it will be what you're finding there for a VS Code extension:

https://github.com/eclipse/che-plugin-registry/blob/master/che-theia-plugins.yaml#L422-L433

except that directory will be replaced by image

[sunix]

To me it is unclear how we deal with the usecase:

• I am using che-theia-plugins.yaml in my repo with 3 plugins
• I want to add a new one, but inline meta.yaml of this new one and keep the 3 other plugins.

[benoitf]

I think it's about merge strategy (like it could use both input)

But I suspect it might be easier to just update che-theia-plugins.yaml (to bring plugin 4) of the repo in a dedicated branch for your usecase.

[sunix]

I guess it will be what you're finding there for a VS Code extension:

https://github.com/eclipse/che-plugin-registry/blob/master/che-theia-plugins.yaml#L422-L433

except that directory will be replaced by image

So like that ?

schemaVersion: 2.0.0
attributes:
  che-theia-plugins.yaml: |
    - repository:
        url: 'https://github.com/camel-tooling/camel-lsp-client-vscode'
        revision: 0.0.31
      sidecar:
        image: eclipse/che-remote-plugin-runner-java11:7.0.0-rc-3.0
        name: vscode-apache-camel
        memoryLimit: 768Mi
        cpuLimit: 500m
        cpuRequest: 30m
      extensions:
        - >-
          https://download.jboss.org/jbosstools/vscode/stable/vscode-apache-camel/vscode-apache-camel-0.0.31-143.vsix
     

[sunix]

But I suspect it might be easier to just update che-theia-plugins.yaml (to bring plugin 4) of the repo in a dedicated branch for your usecase.

The idea of this issue is to make things easier when you want to quickly add a new chePlugin (extension + sidecar) in a devfile. The pain point was that we have to create an additional gist file and reference that one. I don't see what are the benefit if finally we have to edit the files in a fork branch repo.

Also merge strategy could be complex and leading to more bugs that we could avoid having a proper format.

[benoitf]

For now top level attributes is not in devfile specification, so che-server is failing at validation level with such devfile

[l0rd]

For now top level attributes is not in devfile specification, so che-server is failing at validation level with such devfile

Top level attributes are not in the spec yet. The following PR need to be merged devfile/api#352

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[l0rd]

/remove-lifecycle stale

[sleshchenko]

Well, there is nothing to do on devfile/v2 format with that.
Dashboard already supports che-editor.yaml.
Theia lib already supports che-plugins.yaml (name is not exact). So, assigning Theia to check the state and possible close.
@benoitf If I'm not mistaken it's you who implemented that part.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[l0rd]

It's currently possible to specify theia plugins inline in a devfile but the missing piece is the possibility to specify the editor. That should probably be a dashboard responsibility.

implemented:

schemaVersion: 2.1.0  
metadata:  
  name: my-example  
attributes:  
  .che/che-theia-plugins.yaml: |  
    - id: redhat/java
  .vscode/extensions.json: |  
    {  
      "recommendations": [  
        "ms-vscode.vscode-github-pullrequest"  
      ]  
    }
components:
  (...)

not implemented yet:

schemaVersion: 2.1.0  
metadata:  
  name: my-example  
attributes:  
  .che/che-editor.yaml: |  
     id: che-incubator/che-code/insiders
components:
  (...)

Here is a full devfile example.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.