[RFC] Ide generation and Guidance Proposition

[rvion]

This PR is just here to see if you agree the philosophy :)
(The code is not ready to be merged)
I mostly did 3 things:

I changed the readme

(see here: https://github.com/haskell/haskell-ide-engine/tree/invite-people-to-programatically-generate-plugins#haskell-ide-engine-hie)

• call for help in a more structures way
• identify more clearly the 3 possible way of contributing

I added new ide-xxx folders

• rename elisp/.. to ide-emacs
• add ide-sublime/
• add ide-atom/
  (related to the discussion at the bottom of PluginDescriptor no longer serialisable. #93)

I suggested one place to add plugin-generation code in hie

• move plugin array outside of Main module to access them in..
• plugin generators (example initial SublimeText python plugin generator)

---

The advantage of generating code from hie is that:

• hie executable can register plugins by himself if wanted
• encourage people to use code generating techniques
• ability to really share code between different plugins
• ability to reason easilly on user interface :)
• ability to generate 1-file plugin (weird, but needed for simple reload in sublime text for example)

[rvion]

ping @gracjan @alanz @cocreature @JPMoresmau

[alanz]

Not commenting fully, but please note also #25

[cocreature]

I like the nicer Readme, however I don’t think we should encourage generated IDE integrations.

Instead I would like IDEs to be able to figure out everything by API introspection at runtime. We already have some commands for that such as base:plugins and base:commandDetail and if you take a look at the emacs integration we use that to figure out which commands exist, add them to a nice menu and figure out what parameters we need to pass them.

I'd be interested to hear why you prefer code generation over api introspection. Obviously there are still things missing there, but we can just add whatever the people writing the IDE integrations feel necessary.You can still use the same code for a lot of plugins if you get it via introspection.

Edit: I somehow missed your last paragraph. Imho the first four points are easily achievable via introspection. I am not sure what you mean by "reason easily on user interface" and I don’t know anything about sublimetext.

[rvion]

I thought a little bit about HIE components over the night.
Let's imagine we have the following cabal libraries and executable architecture:
(with stack, it seems easily manageable to me, and I think it will speed compilation a lot)

I am not sure what you mean by "reason easily on user interface"

basically, for each ide, we want to

• generate one function per command
• generate some menu with all commands
• generate key bindings

and each command-function basically:

• gather required params
• send request to hie
• wait answer
• according to the response, does something for the user:
  • Visual info - popup, status-bar, etc.
  • text change - add type signature, etc.

if we have in ide-generator-base expose data-types like

data Visual = Popup Text | Status Bar | CodeInsertion | ...

with corresponding helper code, then each ide backend, only has to write very few custom things, such as

• how to gather context to send to hie
• how to display a popup
• how to change the status bar
• etc.

I'd be interested to hear why you prefer code generation over api introspection.

in addition to above text, I really prefer code generation (== api introspection in haskell) to api introspection in plugins because:

• We only do plugin introspection once, and in haskell, instead of many times and in each other non-typed languages
• we can't easily do plugin api introspection easily in some languages and ides:
  • Scripting can be limited
  • Commands may tied to files
  • We may have to write an interpreter and loose type safety
  • etc.
• we can explore the generated code and look for function names to debug more easilly what is generated. (all documentation generation code available, ability to leverage existing code analysis tools)
• All contributors to this repo will know haskell, so we have more power and less work so more productivity.
• I have read lot of things about code generation being fun and productive: we'll probably learn a lot from this experience and it will be useful for our future lives (well, okay, this one is a bit esoteric)

Not commenting fully, but please note also #25

With that design, choosing a different set of plugins becomes quite easy by swapping the all-plugins package.

[rvion]

again, mini-precision about the PR content: the demo code I wrote in the PR doens't reflect the level of widget gui abstraction we could achieve, the package split suggested is not done, and the readme can still be tweaked more to drive more people. This PR is more here to get us discuss than anything.

[rvion]

ping @lukexi, as author of stack-ide-sublime interested in switching to hie when it's ready : do you have any opinion on plugin code generation vs available commands introspection from within the plugin ?

[alanz]

This is an interesting approach, and I think it may be beneficial. It is similar to the one adopted by Wrangler

Can you put together a small POC?

[cocreature]

One disadvantage of code generation is that every update which changes something about a function also requires an update to the bindings. In particular the bindings need to be always kept in sync with the executable. While that’s obviously doable it is not completely trivial since the package management systems of editors typically don’t deal with executables so it’s up to the user to not mess up.

[alanz]

Maybe we should split this up, so on the one hand hie exe can emit a json
format of the commands it knows, which can then be used by a
plugin-generator for a particular IDE, which may or may not be written in
haskell.

If an IDE prefers to go the fully-dynamic route that is fine.

Once things stabilise, I expect the IDE-specific plugin to be delivered via
its standard plugin management mechanism, and it will have to match a given
set of versions of hie.

Much like ghc-mod now, with its atom, vim, etc plugins.

On Fri, Nov 27, 2015 at 1:13 PM, Moritz Kiefer notifications@github.com
wrote:

One disadvantage of code generation is that every update which changes
something about a function also requires an update to the bindings. In
particular the bindings need to be always kept in sync with the executable.
While that’s obviously doable it is not completely trivial since the
package management systems of editors typically don’t deal with executables
so it’s up to the user to not mess up.

—
Reply to this email directly or view it on GitHub
#107 (comment)
.

[rvion]

Once things stabilise, I expect the IDE-specific plugin to be delivered via
its standard plugin management mechanism,

One disadvantage of code generation is that every update which changes something about a function also requires an update to the bindings. In particular the bindings need to be always kept in sync with the executable. While that’s obviously doable it is not completely trivial since the package management systems of editors typically don’t deal with executables so it’s up to the user to not mess up.

@alanz @cocreature I agree
In fact, I think deploying plugins via their standard plugin management platform is the best way.

@cocreature to keep things in sync, it would be to our resort to generate binding and publish them on each editor package system for each releases. This way, people still access them the usual way.

it will have to match a given set of versions of hie.

Indeed, this is an other downside against my proposal:

since we'll probably have to release plugins via traditional package management platforms anyway, we will have no simple ways but to error out when api don't mach with generated plugin :/

[cocreature]

@rvion I wasn’t afraid of us not updating the bindings, in fact the opposite: since their is package management for editor plugins users are likely to update their editor bindings without updating hie. In particular some plugin management systems (at least melpa for emacs) simply update from git so users will also need to update hie fairly often. Of course we can tell users not to use update or use some other mechanism (for emacs that would be melpa-stable) but at some point it will probably break.

In the end I think we should just leave it up to the people developing the individual integrations whether they prefer code generation or runtime introspection. Personally I don’t see a huge value in code generation for the emacs bindings and I like the idea of being able to add,change & remove things in hie while not changing the bindings, so unless someone else involved on the emacs side ( @gracjan ) brings forward strong reasons for why we should generate the bindings, I'd prefer to stick with the current approach there.

I don’t think we gain a lot by forcing everybody into one camp here.

[rvion]

@cocreature You've convinced me :)

[cocreature]

@rvion I wasn’t trying to convince you, if you prefer code generation for the sublime text bindings go for it! I just was trying to explain why I don’t think that it is always superior.

[alanz]

I agree with @cocreature that for some IDEs it makes sense to make dynamic bindings, and for others not.

We need to be able to support both.

[rvion]

@cocreature We're on the same page.
You convinced me that forcing people to go the code generation route is not a good idea.

I wanted to push code generation because I didn't see enough drawbacks, and I thought it might turns out to be a good idea, but situation is not so clear as I thought.

I may still try that for sublime or atom plugin, since I'm not a python expert :)

@alanz yes, I agree

[tobiasgwaaler]

If we provided a hie --upgrade, like stack does, the editor-plugins could initiate an upgrade if needed. But being able to upgrade itself obviously comes at a cost. Just wanted to mention it

[mgsloan]

Despite liking code generation, I also lean towards using API introspection as much as possible here. Code generation does make quite a bit of sense in the following contexts:

• JSON serialization / deserialization
• When doing the necessary introspection / command generation isn't supported nicely by the editor's scripting / plugin language.

[cocreature]

What do we do with this PR? The discussion seems to have stagnated and the actual PR has become outdated. Can we just close this? Some of the readme changes would be nice to have but we can also do that in a separate PR.

[alanz]

@rvion can you update this to reflect the non-contentious changes, so we can land them?

[DanielG]

it's spelled ghc-mod ;)

[rvion]

Yes, I'll make an other PR soon to replace this one.
Let's close this for now, I'll rework the readme later.

For info, I am working on some side projects around haskell-ide-engine (a web based ide + some missing tooling I'd like to have), I hope to have something to show soon.

[alanz]

Sounds good