Add TypeScript Language Server - previously typescript syntax

[cmelion]

Is syntax highlighting for these languages on the roadmap?

[TylerJewell]

Hi @cmelion. Che embeds the Eclipse Orion editor, and we inherit the syntax highlighting for the included languages. @benoitf Can you comment on how Orion pulls in syntax highlighting and what is available? I know that Orion is very strong in JavaScript and CSS, but not sure if that extends to ES6.

[cmelion]

@TylerJewell thanks for the quick response. Is the editor a pluggable component?

example of typescript support in a web editor:
https://github.com/hi104/typescript-playground-on-ace

[TylerJewell]

The editor is pluggable, but not cleanly so. Over the years of experimenting with Che we started with Ace editor, and then migrated to CodeMirror, and then migrated to Orion now, as it has the most stability, best internationalization keyboard support, and some other important features.

So, the IDE editor is a plug-in that itself embeds the Orion editor. And the IDE editor does inherit certain features directly from Orion such as keybindings and color syntax highlighting. Though I am not the engineer that has worked on that, so we'll need to get an engineer familiar with Orion to explain where those components are configured (and what other elements are directly inherited).

But we say that the editor is not easily pluggable meaning that you can replace the Orion editor with Ace, you would have to then associate it with certain files. In our world, we do a lot of work with Java files. So if the project has been detected to kick in the JDT services running embedded in the workspace, there is a whole layer of value added intellisense that gets injected and bound into special hooks in the Orion editor. Swapping out Orion for another editor would cause the IDE to lose all of these added value editing capabilities.

If you want to know more, we can get the engineers that oversaw this work to comment more.

[cmelion]

It would be interesting to find out more about how new file types can be supported by the Orion Editor.
Any suggestions on where to start? orionhub.org keeps coming up in google, but the site has been down for at least a day or two.

[TylerJewell]

They have a lot of content on their project wiki. http://wiki.eclipse.org/Orion. I haven't reviewed this yet but I would guess it is in here. Our engineers will comment too.

[TylerJewell]

Looks like Orion uses Textmate internally and allows for the incorporation of new styles. http://wiki.eclipse.org/Orion/How_Tos/Code_Edit#Theming_the_syntax_highlighting

[cmelion]

thanks @TylerJewell that link looks promising.

[TylerJewell]

I think this is the matching file inside of Che. https://github.com/eclipse/che/blob/master/plugins/plugin-orion/che-plugin-orion-editor/src/main/resources/org/eclipse/che/ide/editor/orion/client/orion-codenvy-theme.css.

If this file is edited, you can then rebuild the plug-in, and it is pulled into Che. Though in reading it briefly, it's unclear to me how TextMate has an associated syntax highlighter for different languages.

[cmelion]

test.js

test.ts

Currently ts/jsx files are treated as plain text when parsed, changing the file extension to .js produces more complex markup with associated css styles:

We can't simply change all ts/es6/jsx files to .js because the intellisense(?) doesn't know how to handle the newer syntax:

But it's pretty cool how much of the desktop IDE look and feel Che has achieved within the browser!

[TylerJewell]

@ashumilova and @slemeur: Will you consider adding an investigation issue into your next development sprint starting Thursday to provide some recommendations on how @cmelion can apply a series of patches to enable Che to properly handle their extension types? We should have this documentation added to the Che docs site as well, as I think it would be a common exercise for people to ask about how to add new file formats into the editor.

@cmelion - that is pretty cool what you did. Thank you for the comments about the desktop feel. @slemeur has made the design and details a labor of love. Oddly, we think there is a lot more we can do to improve the responsiveness and the little details that make the experience feel more natural. We are striving for big performance, minimal design, and a way to add layers of powerful tools.

[ddementieva]

Fixed by 5458966

[cmelion]

@ddementieva This is awesome, my thanks to all involved!

[TareqElMasriDev]

Not solved, still having problem with ES6.

[TylerJewell]

@ashumilova @ddementieva - can you follow up. Was this documented in another thread?

[ashumilova]

@TareqElMasri previously we solved the problem that files with ".es6" extension were not considered as javascript ones. If you are facing the problem, that ecma6 specific stuff is marked as errors - it should be solved by updating Orion to v.12 (http://planetorion.org/news/2016/06/announcing-orion-12/). We have issue for that #1900 and it is in our nearest plans.

[masaeedu]

@ddementieva A bit confused here; have the appropriate transpilers/type checkers for ES6 and TypeScript been integrated with the editor? I'm not super familiar with the internals, but from what I understand this would involve integrating language servers for ES6 and TypeScript.

If full support for these languages has been added, is there a place where the feature is documented? If the languages aren't fully supported yet, shouldn't the issue remain open?

[TylerJewell]

This issue was specific to having the editor support syntax highlighting. This just involves adding the appropriate configuration to the Orion editor, which was done previously. Adding intellisense for TypeScript / ES6 is not tracked here.

For what it is worth - Microsoft has not open sourced their TypeScript language servers. So either the community needs to build one or we have to wait for Microsoft to provide an open server that can be integrated.

[masaeedu]

@TylerJewell Thanks for clarifying. Since VS Code and TypeScript can be built from source and configured to work with each other, it seems to me that there is either an implementation of the language server protocol buried somewhere in the VS Code or TypeScript source, or that VS Code is interfacing directly with the TypeScript language service and not using the protocol at all.

[TylerJewell]

Eclipse Che now supports the language server protocol. But each language server is an independent product with its own license restrictions. Microsoft has not open sourced their TypeScript intellisense libraries in a license that would allow them to be packaged and bundled by the community. The TypeScript language server written by Microsoft is a JSON wrapper that adheres to the protocol and some of that is open source, but the underlying language server depends upon TypeScript intellisense libraries which have not been opened.

[masaeedu]

The TypeScript language server written by Microsoft is a JSON wrapper that adheres to the protocol and some of that is open source

@TylerJewell I've done some more digging in the VS Code and TypeScript issue trackers, and I don't think this is correct. It looks like there is no TypeScript language server adhering to the protocol specification in existence.

Instead, both Visual Studio and Visual Studio Code directly utilize the TypeScript compiler APIs for their intellisense. While Visual Studio is proprietary, both TypeScript (including the compiler APIs) and Visual Studio Code (including the code interfacing with the TS compiler API) are open source and are available under fairly permissive licenses (Apache 2.0 and MIT respectively). This means the TypeScript compiler API can be used, for example, to implement VS Code grade TypeScript intellisense in a vim plugin or a different JS-based text editor.

For reference:

• Here is the source for the VSC TypeScript extension (which implements type checking, intellisense, etc.)
• Here is the compiler API documentation, and here for the language service.
  • If consuming from JS, you need to implement LanguageServiceHost, then instantiate a single LanguageService instance per TypeScript project
  • If your editor is not native JS, or needs to host the language service out-of-process, you can use tsserver and the wire protocol. This is less well-documented, but it is possible to suss it out from the various client implementations (e.g. Sublime, emacs), and the protocol interface here
  • There are simpler wrappers around tsserver, e.g. https://github.com/clausreinke/typescript-tools allows you to interface with an instance of the language service over stdin/stdout, which is useful for command line tools

So while there isn't already a language server out there, it doesn't look like there are any license restrictions blocking intellisense and type-checking support for TypeScript. It should be possible to interface directly with the TypeScript compiler API (available as an NPM package), or push for the existing TypeScript language service to be updated to support the protocol specification, or implement a specification compliant wrapper around the compiler API.

microsoft/TypeScript#11274 currently tracks adding conformance to the official language server protocol specification to tsserver. From discussion concerning that issue, it doesn't appear there are any licensing roadblocks. Quoting one of the maintainers:

@mhegazy do you think you will attention to support Language Server for TypeScript microsoft/TypeScript#11274

I do not think we have talked about this specifically. A bit of a historical perspective. the TS LS protocol came first :) and then VSCode elaborated on it, and changed some things.

having said that, i do not see why not implement the protocol if there are consensuses around it. again, we will need to support this along with the current protocol, either through a configuration message to select the protocol version to use or some other mean.

[mhegazy]

Microsoft has not open sourced their TypeScript intellisense libraries in a license that would allow them to be packaged and bundled by the community.

The tsserver, along with other parts of the typescript compiler and language services layer, is and has always been open source. tsserver has been published as part of the standard typescript npm package. all the type script offerings are under the Apache 2.0 licence. Please find more documentation about the server in https://github.com/Microsoft/TypeScript/wiki/Standalone-Server-(tsserver)

The TypeScript language server written by Microsoft is a JSON wrapper that adheres to the protocol

The JSON protocol that the tsserver implements is slightly different from that of VSCode's. Issue microsoft/TypeScript#11274 tracks addressing the differences. regardless of that the two protocols are fairly similar and shimming one to the other is straight forward.

Instead, both Visual Studio and Visual Studio Code directly utilize the TypeScript compiler APIs for their intellisense. While Visual Studio is proprietary, both TypeScript (including the compiler APIs) and Visual Studio Code (including the code interfacing with the TS compiler API) are open source and are available under fairly permissive licenses (Apache 2.0 and MIT respectively).

That is not accurate. TypeScript has had tsserver since TS 1.5. You can find more documentation at https://github.com/Microsoft/TypeScript/wiki/Standalone-Server-(tsserver). That is what VSCode, Sublime text plugin and newer version of VS use. I would not recommend using the compiler/language service APIs directly, there are a lot of work that will need to be duplicated. instead i would recommend using the server.

[TylerJewell]

That is good news. Thanks for cross linking the tracking issue for what it will take to get the JSON protocols aligned. If tsserver can implement the same protocol, then we will package it up as an agent for deployment in the cloud workspaces that we manage.

[kylecordes]

In the months since the most recent comment here, Che version 5 shipped, release notes:

https://che.eclipse.org/release-notes-eclipse-che-5-0-fde5626194f1#.lps7l5iqp

"Today, Che supports JSON, C#, Java, PHP and TypeScript language servers."

Could someone in the know, add a comment here that reconciles the discussion in this item with that announcement?

[TylerJewell]

@bmicklea @slemeur @evidolob @JamesDrummond - can one of you comment on this question about TypeScript?

I also do not think the labels and status are accurate at this point.

[masaeedu]

@TylerJewell It appears Che is now using https://github.com/sourcegraph/javascript-typescript-langserver now, which wraps the LanguageService API I mentioned earlier. This doesn't really support the full protocol yet, but any remaining work can probably be negotiated between TypeScript and that language server's maintainers.

[TylerJewell]

Thanks @masaeedu - I would like the PMs that look out for this to also confirm that we have a good agent package of the language server, registrations to all of the appropriate file extensions / types, and that we are using the latest one as well.

[felixfbecker]

Hi folks, I'm currently reworking the README of https://github.com/sourcegraph/javascript-typescript-langserver. Can I add Eclipse Che to the "Used By" section?