HIE and current working directory

[alanz]

This is a summary of a discussion with @DanielG on #ghc-mod last night

1. In general, given that GHC is non-reentrant, and we are running it in the same process as HIE via the GHC API, it will not be possible/easy to have more than one GHC session at the same time.
2. Certain plugins make use of the GHC API to typecheck and more, so the GHC version in HIE must match the GHC version for the project. So we are likely to end up with hie-7.10, hie-8.0, etc as executables

So the conclusion is that if an IDE is going to support multiple haskell projects simultaneously, it wiil need to run multiple instances of HIE, one per project.

An eventual future component may sit in front of the running HIE processes, to manage the communications from the IDE to them.

---

Furthermore, the monad we are using wraps a ghc-mod session, and this is started in the main programme, and launches the dispatcher. Up to 5.4 ghc-mod would throw an exception if it is not in the root directory of a file being loaded into a ghc session, and we used this to (transparently) change to that directory, run the command, and then change back to what it was, $HOME by convention.

The initial entry into runGhcModT sets up the environment, identifies the project root directory, and ensures that the cabal information is cached ready for use. This should happen once per project, when the ghc-mod / GHC session is initiated. Think of this as firing up ghci.

So, given than we are likely to run a separate HIE instance per project, it would make more sense to initiate it in the project directory, so that the initial ghc-mod session startup actually sets up the right thing, which is then cached for speedy operation of commands sent from the IDE.

We will have to make some change to manage this for the imminent release of ghc-mod 5.5, the question is whether we change to specifying a project root on startup (which ghc-mod can work out from any project file), or whether we try to panel beat what we have to work somehow, with all the pain of trying to get the ghc-mod caching/session management to work properly.

[cocreature]

I’m fine with just using different instances of hie for separate projects. There is not a lot to be gained by using the same instance since caching and similar performance improvements are also limited to a single project.

What I am not quite sure about is what you mean by "specifying a project root on startup". If ghc-mod can figure that out do we really need the ide to supply that? I don’t like the idea of forcing that burden on the ide. If you just mean that ghc-mod figures that out on startup, then I’m all for it.

[alanz]

That is the part we need to figure out. The question is at what point does an IDE start hie? If it is in the context of a file in a project of interest, then it gets launched at that point and we are fine.

Thinking about it further though, if you are just wanting basic error checking and e.g. going through a bunch of cabal unpacked source to look at stuff then this won't work so well at all, at least not as a daemon.

[cocreature]

I think it’s reasonable to expect the ide to start with a file in a project, what we might not want to expect from the ide is to figure out if a file belongs to an already running instance or if it requires a new instance. So a single haskell daemon which would then proxy the requests to individual processes for each project is the best approach I can see.

I am not quite sure why going through a bunch of cabal unpacked source should be a problem. Sure we might spawn a bunch of processes if you go through a lot of projects, but apart from the resource requirements I don’t see a problem with that.

[gracjan]

I'd like to point out the issue that current directory poses. Look at the code in Emacs' comint that is devoted to managing consistent view of what the shell considers pwd and what Emacs' session considers that the shell considers is pwd:

https://github.com/emacs-mirror/emacs/blob/master/lisp/comint.el#L1764

I'd like to avoid code like that in Emacs.

So the problem is changing the current working directory. On the other hand starting in a directory that both Emacs and ghc-mod agree on is pretty okay in my book.

[gracjan]

One more link:

https://github.com/emacs-mirror/emacs/blob/master/lisp/shell.el#L506

Note how much of shell.el is devoted just to manage current directory view.

[alanz]

@gracjan I think this could only work if the directory is sent from the IDE, or is immediately returned on startup and thereafter never changes.

[glittershark]

Gonna throw my weight behind having hie manage the potential different instances per project, and figure out which one to use based on the filename given.

[alanz]

Which I interpret as keep the initial minimal version with a single project/exe only, and add the manager later. It will have implications on the transport, as we will end up with an internal serialisation choke point, potentially.

[JPMoresmau]

As a random additional point, the manager will also need to manage restarting instances when needed. In the past some flags from the cabal file could only be loaded once in the GHC API (static flags) so any change to them involved restarting the executable to get a new session with new flags. this may have been improved in recent versions, though.

[alanz]

Ping @mgsloan do you have an opinion?

[mgsloan]

CWD is indeed tricky, due to stuff like TH depending on it. I'm in favor of:

• Ignoring the issue for now and having a HIE instance per project.
• For Haskell projects in general, it'd be great to shift to writing TH which is agnostic of CWD, possibly by writing a library which makes this convenient.

If a shift towards CWD agnostic TH doesn't happen, then possibly packages can be marked "CWD sensitive", and HIE can fork a process for that.

[alanz]

What is the issue with TH and CWD? That it should never change, or that it
should be a specific value? What would a library do to help?
On 08 Jan 2016 3:59 AM, "Michael Sloan" notifications@github.com wrote:

CWD is indeed tricky, due to stuff like TH depending on it. I'm in favor
of:

Ignoring the issue for now and having a HIE instance per project.

For Haskell projects in general, it'd be great to shift to writing TH
which is agnostic of CWD, possibly by writing a library which makes this
convenient.

—
Reply to this email directly or view it on GitHub
#161 (comment)
.

[mgsloan]

The specific issue is when TH depends on some file in the project. It's normal to use relative paths with TH, and addDependentFile accepts relative file paths.

A library solution to this would be to have readFile variants (String, Text, ByteString, Handle, etc) which can handle these relative paths. This would work by using .cabal file finding logic by walking up the directory tree from the source file location. Then, files would get loaded relative to the .cabal file rather than the project.

This isn't 100% satisfying, though, as it means that TH relative files would be tied to having a .cabal file in the expected place. Still, much better than using relative paths like they are currently..

AFAIK, the only two scenarios where CWD matters for a project is when TH cares about it, and when running user code (forkProcess is probably ideal for this anyway, due to issues with ensuring resource cleanup).

[DanielG]

@JPMoresmau ghc-mod handles all that mess it just creates a whole new ghc session when any flags change.

[JPMoresmau]

@DanielG but wasn't there some "static flags" that you could parse only once? I seem to remember that in scion and buildwrapper we had to restart the executable because GHC would use some global variable. Has that been all removed?

[DanielG]

Looking at the source they seem to still exist but they're options that I've never seen used anywhere: http://downloads.haskell.org/~ghc/latest/docs/html/libraries/ghc-7.10.3/src/StaticFlags.html so I assume it's not a problem anymore.

[JPMoresmau]

OK, nice to see that.