Autocompletion server for editors and IDEs

[rexim]

Derived from morganey-lang/morganey-vscode-extension#1 (comment)

[06needhamt]

Started developing prototype at https://github.com/06needhamt/morganey-integration-server

[rexim]

I'm really glad for you, but how is that related to this not yet approved proposal?

[06needhamt]

@rexim It may be useful when we come to develop the actual server if this proposal gets approved

[rexim]

@06needhamt cool

[rexim]

@keddelzz it feels like let-morganey-rest is already the autocompletion server for morganey in some sense. :D Maybe it's just a matter of generalizing it. We need to research that.

[06needhamt]

@rexim @keddelzz My sever should also have the functionality to call Morganey APIs is Typescript as mentioned here does let-morganey-rest also have this functionality

[rexim]

What do you mean by "Morganey API"? Morganey API is not even defined yet #245. What do you want to call? Actual JVM methods?

[rexim]

Autocompletion server should provide sufficient functionality without mentioning underlying JVM methods in any way.

[06needhamt]

@rexim Yes sorry i mean methods within Morganey such as this So I can implement features such as raw term display in my vscode extension

[rexim]

@06needhamt LambdaParser class should NEVER be exposed directly in any APIs for interaction with Morganey (even though it's used outside of Morganey in some places, but this is because Morganey is still unstable). Such things should be implemented as for instance a REST call /raw which receives a term and returns a raw representation. And if I understand correctly let-morganey-rest can already do that via the REPL. @keddelzz please correct me if I'm wrong.

[rexim]

@06needhamt It's cool that you started prototyping something. But it doesn't look like a great idea doing that without discussing anything with other people involved into Morganey development. I mean I'm really glad that you're doing something. But if you don't discussing anything with anyone, everyone will do the same with you. And at the end, please don't be surprised if nobody needs your prototype.

[keddelzz]

And if I understand correctly let-morganey-rest can already do that via the REPL.

let-morganey-rest implements REST calls for

• autocompletion
• evaluating a term
• conversion from any term to a raw term

@keddelzz please correct me if I'm wrong.

So, @rexim you are totally right!

[rexim]

@keddelzz cool! Thank you very much! Are you ok with starting to move let-morganey-rest to a more general thing? So it can be used not only for Web REPL, but also for integration with editors and IDEs.

[keddelzz]

Are you ok with starting to move let-morganey-rest to a more general thing?

I have absolutely no problem with that.

[rexim]

@keddelzz great! I'm gonna take a look at how let-morganey-rest can be reused in other places and will make proposal to your project if something is needed. Thanks!

[06needhamt]

@rexim @keddelzz Keep me posted for updates

[rexim]

@06needhamt sure! I'm gonna keep the progress captured in this issue. :)

[rexim]

After a brief research I concluded that unfortunately I don't see let-morganey-rest suitable as a server for Morganey development tools integration. Here are the reasons:

1. Sessions. It meant to work in multiuser environment where each user has their own isolated session. It's a complete overkill in case of our autocompletion server and absolutely not needed. We can introduce some sort of a single user mode but it's a maintenance cost.
2. Database. It requires a database. Even though it's h2 an embedded one. It's still a maintenance cost.
3. Separate Application. It's actually not good from both of the sides. As a user I don't wanna download and configure an extra dependency just for a basic autocompletion in my Emacs (which I actually have to do for many modern languages these days for some reason, and those tools sometimes reimplement lots of stuff from the compiler or interpreter itself). Also as a user I have to make sure that the server and interpreter are compatible to each other. As a Morganey developer I don't wanna mess with discrepancies that will arise between APIs of those two closely related projects.

Right now Morganey has two modes: REPL and Execution. I suggest to introduce a third one: Server mode. The use case is the following. If you have a Morganey fatjar you start it like:

java -jar morganey.jar server

1. Morganey creates an execution context as if it's REPL mode.
2. Then Morganey opens a port (for instance 8080, but should be configurable) and listens for local HTTP requests.
3. By responinding HTTP requests Morganey answers all sorts of question (autocompletion, raw terms, etc) and also allows to modify its current context (loading modules, defining bindings, etc.). It becomes sort of an HTTP REPL mode.

If I implement for example an Emacs extension for Morganey I don't even have to ask the user to install or configure anything. I can simply download one jar on first run (or make it a part of the extension distribution), and start Morganey as Server in the background when the user asks to perform autocompletion. The same jar can be also used for executing Morganey programs in Execution mode with IO.

And as far as I can see, it's quite easy to introduce such mode. Everything we need for that is already there, we just have to implement a REST interface. We can use [http4s](the same library used by [let-morganey-rest], so it's quite reliable). It comes with its own small HTTP server based on NIO which is perfect for our single user environment.

What do you guys think?

[06needhamt]

@rexim Sounds Really Good To Me Especially Because No Other Applications Are Required

[keddelzz]

I'm not sure if I like the idea of the new mode in morganey.

Pros of the new mode:

• It only provides functionality, which is required (no sessions, no database)
• No separate application

Cons of the new mode:

• Replication of code from let-morganey-rest
• It also introduces additional maintenance cost in Morganey

I don't really see the advantage to implement the autocompletion server in Morganey over the implementation of it in let-morganey-rest. Both strategies will create maintenance cost or code duplication.

[ForNeVeR]

Well, ideally we should either extract the editing support from let-morganey-rest to Morganey, or put the functionality described by @rexim into let-morganey-rest (or a new, independent module?)

What will be more convenient for us? From the code base point of view I think that it'll be slightly better to store that in a core module (i.e. Morganey itself).

In any case, there should be no code duplication.

[rexim]

@keddelzz @ForNeVeR if both approaches are almost the same from the developer's point of view, what about the user's one? I mean just a single jar for REPL, Execution and Autocompletion server is rather convenient and @06needhamt pretty much confirmed that.

[rexim]

Alright, since all of the stakeholder +1-ed my previous comment I consider the decision of introducing the Server Mode to Morganey Interpreter accepted.

[rexim]

Btw, what do you guys think about this thing? https://github.com/Microsoft/language-server-protocol

Maybe we should implement this protocol instead of using HTTP.

[rexim]

Oh, I was corrected that it's already HTTP! :) Which probably makes it even easier.

Ignore that! They're trying to confuse me!

[06needhamt]

This looks good to me because it seems like it handles all relevant file events for you

[ForNeVeR]

Well, it seems like the protocol is supported by VSCode, some Eclipse plugin / mod and Atom, and also for example Rust guys are building their tooling for it. Also there's a Java "binding" (whatever that means).

So I think it's a good idea.