Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt ws.next model to new requirements #9895

Closed
skabashnyuk opened this issue May 31, 2018 · 19 comments
Closed

Adapt ws.next model to new requirements #9895

skabashnyuk opened this issue May 31, 2018 · 19 comments
Labels
kind/task Internal things, technical debt, and to-do tasks to be performed.

Comments

@skabashnyuk
Copy link
Contributor

skabashnyuk commented May 31, 2018

Description

  1. We need to have an ability to install Theia plugins in workspace runtime from Theia IDE
  2. We need to have an ability to define language syntax.
@skabashnyuk skabashnyuk added kind/task Internal things, technical debt, and to-do tasks to be performed. team/platform labels May 31, 2018
@skabashnyuk
Copy link
Contributor Author

skabashnyuk commented May 31, 2018

@benoitf @gorkem @garagatyi @vparfonov @ashumilova @tsmaeder
Until now I've considered Theia container as a thing that has nothing to do with workspace lifecycle.
But with requirement [1] it is quite hard to do. I guess we need to do the opposite.

  1. Theia container on start gets a list of ChePlugins from workspace configuration.
  2. From Plugins registry it gets all ChePlugins configurations with links to Theia plugins archives.
  3. Download archives and start Theia with plugins.
  4. In case if a user wants to add new plugins Che Theia is obligated to reflect this changes in workspace configuration.

Also, I don't see anymore the need to have ChePlugin and CheFeature as a separate thing. I guess we can combine it together.
I want to propose to have two things ChePlugin and CheService.

Examples of ChePlugin

Githib plugin, it has no additional containers. The content of the plugin is packaged in che-theia-github.tar.gz

version: "0.0.1"
name: "che-theia-github"
title: "Github Client"
homepage: "https://github.com/eclipse/che-theia-github"
description: "Eclipse Che Plugin for Github"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
services:
  - name : org.eclipse.che.theia-ide
    version : 0.0.1
files: 
  - che-theia-github.tar.gz

Typescript language support with sidecar ls, theia plugin in che-theia-typescript.tar.gz and syntax config.

version: "0.0.1"
name: "che-theia-typescript"
title: "Typescript language support"
homepage: "https://github.com/eclipse/che-theia-typescript"
description: "Eclipse Che Theia Typescript support"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
services:
  - name : org.eclipse.che.che-theia
    version: 0.0.1
  - name : org.eclipse.che.ls.typescript
    version: 1.2.1
language:
  id: TYPESCRIPT_LANGUAGE_ID
  extenstions: [ ".ts", ".tsx"]
  aliases: ["TYPESCRIPT_LANGUAGE_NAME", "ts" , "typescript"]
  mimetypes: [ 'text/typescript' ]
  tokenprovider: 'typescript-tokenprovider.ts'
files: 
  - che-theia-typescript.tar.gz

@slemeur
Copy link
Contributor

slemeur commented May 31, 2018

Is "4." force a restart of the workspace ?

@skabashnyuk
Copy link
Contributor Author

Is "4." force a restart of the workspace ?

This is a good question that we have to answer.
Also, it contains questions.

  • how to determine if a plugin is required workspace restart
  • how and who will initiate workspace restart.

@gorkem
Copy link
Contributor

gorkem commented May 31, 2018

I am OK with having just one ChePlugin. How will this work if a workspace does not include the Theia service? I am trying to understand how we would handle dependencies and common services between different ChePlugins.

@tsmaeder
Copy link
Contributor

tsmaeder commented Jun 1, 2018

About 4. above: I think we should think the other way around: che server should update the workspace config and tell Theia to add some extensions. This would decouple Theia from knowing about workspace configuration. To this purpose, Theia would have to run a service that the workspace can talk to. Maybe we could even get rid of the env variables at startup altogether and use the same control mechanism for everything.

@tsmaeder
Copy link
Contributor

tsmaeder commented Jun 1, 2018

About the configuration of the language and typescrip-tokenprovider.ts: why do we need to do that? I don't understand the use case.

@skabashnyuk
Copy link
Contributor Author

@tsmaeder
basic idea is to add language syntax support without codding
https://github.com/eclipse/che/blob/master/ide/che-core-ide-api/src/main/java/org/eclipse/che/ide/api/editor/filetype/ExtensionFileTypeIdentifier.java#L55-L167
https://github.com/theia-ide/theia/blob/44c36d90f91615b5eed6591640c5e0fdf589feb8/packages/typescript/src/browser/typescript-language-config.ts#L86-L87
second is a theia use case
but I'm not sure how it is planned to do in theia
Are we planning always have a ts code to define language
so typescript coding is mandatory
I mean to have theia plugin is mandatory to add new language

@tsmaeder
Copy link
Contributor

tsmaeder commented Jun 1, 2018

I think there is no use case anymore for allowing contribution of language server processes in workspace configuration (and no associated code).
We introduced that because it was not possible for third parties to add plugins in Che classic without creating their own assembly. The Theia extensibility mechanism is plugins, let's not introduce a second one.

So the idea would be to have a Theia plugin that reads the workspace config at startup and registers Languages? Where would the file 'typescript-tokenprovider.ts' be found if not inside a client-side Theia plugin? But if we have a client-side Theia plugin for typescript, why not configure the syntax there?

Also, you're defining a language, but you're not configuring a language server. There may be multiple language servers for Java files (Spring LS, Java LS, Coverage LS, etc.) If you don't allow for separation of language configuration and langauge SERVER configuration, the will possibly have conflicting definitions of the Java language.

The last issue I see is:
files: - che-theia-typescript.tar.gz
Does that mean that workspace.next will create a container and automatically run that file?

In general, we should strive to reuse existing Theia language plugins as much as possible. The consequence of this is that we should use the same configuration concepts as Theia, meaning language descriptions and separate language server configuration based on DocumentFilters and langauge id's.

@skabashnyuk
Copy link
Contributor Author

What I understand from your comment that we should do all syntax configuration from code (Theia plugin) and I should not worry about language support.

@tsmaeder
Copy link
Contributor

tsmaeder commented Jun 1, 2018

That is my opinion. I don't think Joe Programmer will have much need to manually add language servers in the WS configuration. If it's for people contributing into the feature store, I think it's easier to require them to do a very simple Theia plugin. If we do decide to implement it, we should make such that it closely mirrors the Theia model of LS configuration and make sure the model plays nice with LS plugins installed via the feature store.

@skabashnyuk
Copy link
Contributor Author

skabashnyuk commented Jun 4, 2018

I am OK with having just one ChePlugin.

@gorkem I'm sorry. I guess I've added some confusion on terminology. I'll try to make things more clear below.

How will this work if a workspace does not include the Theia service?

This is an IDE loader concern. It's trying to find a link to a server with ide attribute inside. In case if other ISV wants to use Che with another ide they have to use service with a server with ide attribute inside as a part of a workspace.

I am trying to understand how we would handle dependencies and common services between different ChePlugins.

I think that we can take versioning model from npm in a connection between ChePlugins(CheFeatures) and (CheServices).
Like: it can be strong reference - org.eclipse.che.theia-ide:0.0.1 or weak org.eclipse.che.theia-ide:latest

Terminology

  1. Plugin or ChePlugin - I'm proposing to define everything with word plugin as Theia plugin.
  2. As discussed with @l0rd and @garagatyi here Workspace.Next/Phase 1 - CheService/CheFeature model #9292 (comment) . Rename CheService to DevTool
    DevTool - defines development tool container or set of containers with servers that are providing some functionality.
  3. Feature - Main point of user interaction on dashboard or marketplace. Provide a connection between DevTools.
    Contains all necessary information to be published in the marketplace.

Examples:

DevTool

apiVersion: v1
kind: DevTool
metadata:
  name: io.typefox.theia-ide.che-service
spec:
 version: 0.0.1 
 containers:
    - image: eclipse/che-theia:nightly
      resources:
        requests:
          memory: 200Mi
      env:
        - name: THEIA_PLUGINS
          value: ${THEIA_PLUGINS}
      volumes:
        - mountPath: "/projects"
          name: projects
      servers:
        - name: theia
          attributes:
            type: ide
            internal : true
          port: 3000
          protocol: http

Questions.

  1. Does everybody ok with name DevTool?
  2. Should we keep using Kubernetes notation for this object?

Feature

version: "0.0.1"
name: "che-theia-github"
title: "Github Client"
homepage: "https://github.com/eclipse/che-theia-github"
description: "Eclipse Che Plugin for Github"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
keywords:
 - github
 - theia
 - git
 - vcs 
devtools:
  - name : org.eclipse.che.theia-ide
    version : 0.0.1
files: 
  - rel: theia/plugin
    name: che-theia-github.tar.gz
  - rel: marketplace/icon  
    name: github.png

Whats new:

  • Combined marketplace metadata and CheFeature object.
  • keywords section. To be able to select only theia base features
  • files section. Add rel and name to be able to generate appropriate links in responses.

WDYT?

@benoitf
Copy link
Contributor

benoitf commented Jun 4, 2018

question
what part is responsible to compute

        value: ${THEIA_PLUGINS}

metadata

About

name: io.typefox.theia-ide.che-service

it should be renamed to org.eclipse.theia as theia is moving to Eclipse.

about the Feature example
is it possible to include external files ? and how they'll be listed

For example can I just link an external plugin from a feature.
Let say a theia plugin is stored on the registry/marketplace.
Could I link this theia plugin from the che feature instead of downloading the file and wrapping it inside the che archive

sum up:
let say I have on marketplace theia plugins plugin1 and plugin2
Can I reference these two plugins from the che feature metadata.

How will they appear ?

   rel: theia/plugin
    name: che-theia-github.tar.gz

will the name will contain marketplace link ?
Will they appear on files section ? (or it's in another section)

Also in your example, how the files can be retrieved (not the name but the content) ?

keywords section. To be able to select only theia base features

how can I list "pure theia plugins" ?

@skabashnyuk
Copy link
Contributor Author

what part is responsible to compute
value: ${THEIA_PLUGINS}
metadata

The idea was to combine values from Feature
Example:

version: "0.0.1"
name: "che-theia-github"
title: "Github Client"
homepage: "https://github.com/eclipse/che-theia-github"
description: "Eclipse Che Plugin for Github"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
keywords:
 - github
 - theia
 - git
 - vcs 
devtools:
  - name : org.eclipse.che.theia-ide
    version : 0.0.1
    parameters:
       - name: THEIA_PLUGINS
         value: org.eclipse.che.terminal@latest
files: 
  - rel: theia/plugin
    name: che-theia-github.tar.gz
  - rel: marketplace/icon  
    name: github.png

This part was originally designed to send a list of Theia plugins to Theia DevTool (Service) container. Recently we discussed that this functionality is no longer needed. So we may omit it when we are discussing Theia plugins.

it should be renamed to org.eclipse.theia as theia is moving to Eclipse.

ok.

For example can I just link an external plugin from a feature.
Let say a theia plugin is stored on the registry/marketplace.
Could I link this theia plugin from the che feature instead of downloading the file and wrapping it inside the che archive

Initially, I didn't plan that.
We should consider both pros and cons. For example

  • What happens if the external plugin would not be accessible anymore.
  • What happens if the external plugin would be changed.
  • Is there any legal issues with such referencing.

I wouldn't say that this is a big issue. But I would like to listen @gorkem and @slemeur opinion how important this functionality on this stage.

One of the alternative I may propose to automatically publish all Theia plugins as Che Features on a marketplace.

sum up:
let say I have on marketplace theia plugins plugin1 and plugin2
Can I reference these two plugins from the che feature metadata.

How will they appear ?

rel: theia/plugin
name: che-theia-github.tar.gz
will the name will contain marketplace link ?
Will they appear on files section ? (or it's in another section)

Also in your example, how the files can be retrieved (not the name but the content) ?

Was planning that user in Feature's yaml will define set of Theia's plugins like this:

files: 
  - rel: theia/plugin
    name: che-theia-github.tar.gz
  - rel: theia/plugin
    name: che-theia-bitbucket.tar.gz  

This files will appear in Feature response as a links.

as a links

{
  "name": "che-theia-git",
  "version": "0.0.1",
  "title": "GIT Support Client",
  "created_at": "2015-11-22T05:40:57Z",
  "updated_at": "2018-04-22T00:27:28Z",
  "installation_count": 88,
  "description": "Eclipse Che Plugin for GIT",
  "owner": {
    "name": "redhat",
    "url": "https://redhat.com"
  },
  "links": {
    "theia/plugin": "http://somehost/dfsdfsd/che-theia-github.tar.gz",
    "theia/plugin": "http://somehost/dfsdfsd/che-theia-bitbucket.tar.gz",
	...
  },
}

how can I list "pure theia plugins" ?

To be able to have a proper answer for this question we should clarify use case.
What I see is:

From dashboard

  1. When user start selecting features we should be able to get from marketplace list of all available features.
  2. When a user wants to add second or more features we should be able to get from marketplace list of all features that are compatible we already selected.

On Theia container entry point

  1. On Theia container start we should be able to get a list of URLs to Theia plugins for features of the current workspace.

In Theia

When a user wants to add more features to already running workspace

  1. We should be able to get from marketplace list of all features that are compatible with features of a current workspace.
  2. To get a list of URLs to Theia plugins for new features of the current workspace.

@benoitf Am I understand your question correctly?

@benoitf
Copy link
Contributor

benoitf commented Jun 6, 2018

@skabashnyuk correct me if I'm wrong, but workspace.next is a client of the market place like theia could be (or other clients), correct ?
so I'm not sure to follow why the registry/market place is not decouple ?
It seems for me that you're trying to map everything to some che internal stuff while the two are linked but it's not the same. (linking + decoupling )

One of the alternative I may propose to automatically publish all Theia plugins as Che Features on a marketplace.

Why would theia would grab che feature ??

  • marketplace is something to store/expose data
  • workspace.next is a consumer of data that are inside a marketplace. It can grab theia plugins, che features and it could grab in a future maybe other stuff from other marketplace or other data provided by a market place.

so AFAIK, marketplace should expose what is submitted. (no transformation, no generic auto publication, no random stuff).

workspace.next is a client, then it can transform a theia.plugin into a che feature model internally if it may help internally but that's "internal". Or it can transform anything into the workspace.next model but marketplace is still something highly extensible/versatile

@slemeur
Copy link
Contributor

slemeur commented Jun 6, 2018

@skabashnyuk:

1- What do you mean by "get from marketplace list of all features that are compatible with features already selected". How is that suppose to work. Is that something that plugin developer will have to care about?

2- I don't get why the plugin registry needs to know about "how" the plugins are getting packaged/added to a workspace configuration. Why the marketplace would need to know about workspace.next?

3- This terminology "Plugin or ChePlugin - I'm proposing to define everything with word plugin as Theia plugin." I don't get that. A Theia plugin is not necessarily a Che plugin and vice-versa. Example: A plugin with a server part, will not be configured the same way between Che and a standalone Theia. But the marketplace should be able to register and deliver those different types of plugins.

@skabashnyuk
Copy link
Contributor Author

To @benoitf

@skabashnyuk correct me if I'm wrong, but workspace.next is a client of the market place like theia could be (or other clients), correct ?

Not sure that do you mean by "workspace.next is a client". In case if you mean our che server(workspace master) then it will just reference features in workspace definition.

so I'm not sure to follow why the registry/market place is not decouple ?

Me too. It's a same thing.

Why would theia would grab che feature ??

I never said that. I was talking only about Theia feature that is running as a part of Che workspace.

workspace.next is a consumer of data that are inside a marketplace. It can grab theia plugins, che features and it could grab in a future maybe other stuff from other marketplace or other data provided by a market place.

Looks like by "workspace.next is a consumer" you have something different than in my mind. Could you clarify that?

so AFAIK, marketplace should expose what is submitted. (no transformation, no generic auto publication, no random stuff).

Ok. Let's forget about "automatically publish"

workspace.next is a client, then it can transform a theia.plugin into a che feature model internally if it may help internally but that's "internal". Or it can transform anything into the workspace.next model but marketplace is still something highly extensible/versatile

here is something that is not very clear. I would expect that Feature will have

version: "0.0.1"
name: "che-theia-github"
title: "Github Client"
homepage: "https://github.com/eclipse/che-theia-github"
description: "Eclipse Che Plugin for Github"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
keywords:
 - github
 - theia
 - git
 - vcs 
devtools:
  - name : org.eclipse.che.theia-ide
    version : 0.0.1
files: 
  - rel: theia/plugin
    name: che-theia-github.tar.gz
  - rel: marketplace/icon  
    name: github.png

Where che-theia-github.tar.gz contains pure original Theia plugin.

To @slemeur

1- What do you mean by "get from marketplace list of all features that are compatible with features already selected". How is that suppose to work. Is that something that plugin developer will have to care about?

This is something that should be handled between che clients like a dashboard, Che based Theia from one hand and marketplace/registry from another hand. For example, if a user chooses Feature A:0.0.1 that is required devtool D:5.2.x we can select Feature C:0.2 that is required devtool D:6.x. Our flow should handle a situation when different
features require same devtool containers but with different versions.

2- I don't get why the plugin registry needs to know about "how" the plugins are getting packaged/added to a workspace configuration. Why the marketplace would need to know about workspace.next?

registry/marketplace don't care about compatibility of different features. It's just a storage. But someone has to help our UI that will provide user functionality of selecting new features to show users only compatible features that we can run simultaneously.

3- This terminology "Plugin or ChePlugin - I'm proposing to define everything with word plugin as Theia plugin." I don't get that. A Theia plugin is not necessarily a Che plugin and vice-versa. Example: A plugin with a server part, will not be configured the same way between Che and a standalone Theia. But the marketplace should be able to register and deliver those different types of plugins.

"A Theia plugin is not necessarily a Che plugin and vice-versa" - this type of confusing I was intended to avoid with my definitions. Where

  • Plugin - is only Theia plugin.
  • "A plugin with a server part, will not be configured the same way between Che and a standalone Theia" - this is a Feature, che Feature.
    A feature may have only Theia part(will be referencing Theia devtool), only server part when it referencing some devtool, or both.
  • But the marketplace should be able to register and deliver those different types of plugins" - on marketplace we are going to register only Features.

@benoitf
Copy link
Contributor

benoitf commented Jun 6, 2018

To @benoitf

@skabashnyuk correct me if I'm wrong, but workspace.next is a client of the market place like theia could be (or other clients), correct ?

Not sure that do you mean by "workspace.next is a client". In case if you mean our che
server(workspace master) then it will just reference features in workspace definition.

yes, workspace master is a client of marketplace.

so I'm not sure to follow why the registry/market place is not decouple ?
Me too. It's a same thing.

It's not. Marketplace is kind of hosting services with statistics, metadata, nice UI, etc.
But you can imagine you can store a long list of different files/type on it.
A che plugin, a theia plugin, a Docker image (it's just an example. We can store almost anything, just need to add the support of a new type)

Why would theia would grab che feature ??
I never said that. I was talking only about Theia feature that is running as a part of Che workspace.

I was referencing this line:

One of the alternative I may propose to automatically publish all Theia plugins as Che Features on a marketplace.

workspace.next is a consumer of data that are inside a marketplace. It can grab theia plugins, che features and it could grab in a future maybe other stuff from other marketplace or other data provided by a market place.

Looks like by "workspace.next is a consumer" you have something different than in my mind. Could you clarify that?

I mean that workspace master is a client of market place (like any other client)
It can list what is stored on the market place, and if engine is supporting a specific type, it means we can deploy this as part of a workspace.
The "how" is internal stuff of workspace.next engine.

so AFAIK, marketplace should expose what is submitted. (no transformation, no generic auto publication, no random stuff).

Ok. Let's forget about "automatically publish"
ok

workspace.next is a client, then it can transform a theia.plugin into a che feature model internally if it may help internally but that's "internal". Or it can transform anything into the workspace.next model but marketplace is still something highly extensible/versatile

here is something that is not very clear. I would expect that Feature will have

version: "0.0.1"
name: "che-theia-github"
title: "Github Client"
homepage: "https://github.com/eclipse/che-theia-github"
description: "Eclipse Che Plugin for Github"
created_at: "2015-11-22T05:40:57Z"
updated_at: "2018-04-22T00:27:28Z"
installation_count: 88
license: 
 key: mit
 name: "MIT License"
 url: "https://api.github.com/licenses/mit"
owner: 
 name: redhat
 avatar_url: ""
 gravatar_id: ""
 url: "https://redhat.com"
keywords:
 - github
 - theia
 - git
 - vcs 
devtools:
  - name : org.eclipse.che.theia-ide
    version : 0.0.1
files: 
  - rel: theia/plugin
    name: che-theia-github.tar.gz
  - rel: marketplace/icon  
    name: github.png

Where che-theia-github.tar.gz contains pure original Theia plugin.

The main difference, I think is that for me marketplace is serving what has been submitted so in your example I think it's missing the nature (For me the market place is not only hosting che feature)) of what is provided.

like :
type: che-feature
and a specification version for this feature like 1.0
or : che-feature:1.0 (combo of the type of what is provided by the marketplace data)

But if I submit for example a .theia plugin I would expect to have something like

type: theia-plugin:1.0

and I can ask marketplace to grab only theia-plugin or che-feature or another registered type. I could also filter based upon the specification version.

devtools could be more transformed into a requirement or engine.

so for a theia plugin, we could require:

engine: 
-  name: org.eclipse.theia
- version: 0.0.1

and for a che feature it would be the same

engine:
  - name : org.eclipse.theia
  -  version : 0.0.1

(and it is was a docker image, we could as well specify the engine to be : docker or anything else)
(but it's a very minor issue right now)

so workspace.next engine will know

  • for which specification of che a feature is
  • if we want to deploy a theia plugin, then it can transform it internally to a che feature or other stuff (it can do what is required by the workspace.next engine, as long as it's only managed by the workspace.next engine)
  • if the feature is not supported for the current engine, if it needs to be converted, etc.

UI clients could list all stuff possible to install.
For example, inside a running Theia inside Che, we could propose to users the list of all 'theia plugins' and all 'che features'
It could also be possible to some clients to only list che feature, etc

@l0rd
Copy link
Contributor

l0rd commented Jun 7, 2018

FYI I have created a new issue here to refactor/simplify current CheService model (renamed DevTool). This adjustment has been discussed with @garagatyi and @skabashnyuk in the last days to simplify current model.

@skabashnyuk
Copy link
Contributor Author

Closing since I think we will go in this direction #10123

@gorkem gorkem mentioned this issue Jun 21, 2018
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/task Internal things, technical debt, and to-do tasks to be performed.
Projects
None yet
Development

No branches or pull requests

6 participants