Parsing problem with ghc-8.2.1

[ruomad]

It seems the messages when loading a file have changed with ghc-8.2.1.
I got the error "Unexpected response from haskell process".
After looking into it, I fixed it by changing the regex in haskell-load.el. My patch is :

--- Source/hs/haskell-load.el 2017-09-06 12:24:28.341913600 +0200
+++ .emacs.d/elpa/haskell-mode-20170903.948/haskell-load.el 2017-09-06 12:30:54.954209500 +0200
@@ -107,11 +107,11 @@
(let* ((ok (cond
((haskell-process-consume
process

•            "Ok, modules loaded: \\(.+\\)\\.$")
  

•            "Ok, \\([[:digit:]]+ \\)?modules? loaded[.:]\\(.*\\)$")
            t)
           ((haskell-process-consume
             process
  

•            "Failed, modules loaded: \\(.+\\)\\.$")
  

•            "Failed, \\([[:digit:]]+ \\)?modules? loaded[.:]\\(.*\\)$")
            nil)
           (t
            (error (message "Unexpected response from haskell process.")))))
  

I didn't check it's working on previous ghc versions, but it's meant to do it.

[gracjan]

Relevant #1168.

[alios]

I'am working on making interactive working again as well. The 2 things I've done so far are:

Added --ghci-options=-fno-diagnostics-show-caret" to my haskell-process-args-stack-ghci in .emacs.

 '(haskell-process-args-stack-ghci
   (quote
    ("--ghci-options=-ferror-spans" "--ghci-options=-fno-diagnostics-show-caret" "--no-build" "--no-load")))

Thia sisables the new (very nice) ghc8.2 error messages with source code snippets, as it seems not to be supported by the error parser yet.

In #1555
I added a config option to disable the issuing the :set prompt2 command to ghci when starting up interactive as it creates a warning in ghc-8.2.1.

Next step would be to fix/extend the error parsers.

[alios]

As mentioned by @ruomad the problem sits in haskell-load.el.

To illustrate the changes between before/after ghc 8.2. I tested the load on the shell with both versions and both cases (one or multiple modules loaded).

Behavior ghc-8.0.2:

[~/src/alios-cats]$ stack ghci --resolver=lts-9.4 --ghci-options=-ferror-spans                                                                           
alios-cats-0.1.0.0: configure (lib)
Configuring alios-cats-0.1.0.0...
alios-cats-0.1.0.0: initial-build-steps (lib)
Configuring GHCi with the following packages: alios-cats
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Lib              ( /Users/alios/src/alios-cats/src/Lib.hs, interpreted )
Ok, modules loaded: Lib.
Loaded GHCi configuration from /private/var/folders/rq/lpz047951cb5jqzl0x67mjrc0000gn/T/ghci42813/ghci-script
*Lib Lib> 
Leaving GHCi.

[~/src/alios-cats]$ stack ghci --resolver=lts-9.4 --ghci-options=-ferror-spans                                                                           
alios-cats-0.1.0.0: initial-build-steps (lib)
Configuring GHCi with the following packages: alios-cats
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling SHA256           ( /Users/alios/src/alios-cats/src/SHA256.hs, interpreted )
[2 of 2] Compiling Lib              ( /Users/alios/src/alios-cats/src/Lib.hs, interpreted )
Ok, modules loaded: Lib, SHA256.
Loaded GHCi configuration from /private/var/folders/rq/lpz047951cb5jqzl0x67mjrc0000gn/T/ghci42986/ghci-script
*Lib Lib> 

Behavior ghc-8.2.1:

[~/src/alios-cats]$ stack ghci --ghci-options=-ferror-spans                                                                                              
alios-cats-0.1.0.0: initial-build-steps (lib)
Configuring GHCi with the following packages: alios-cats
GHCi, version 8.2.1: http://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Lib              ( /Users/alios/src/alios-cats/src/Lib.hs, interpreted )
Ok, 1 module loaded.
Loaded GHCi configuration from /private/var/folders/rq/lpz047951cb5jqzl0x67mjrc0000gn/T/ghci43629/ghci-script
*Lib Lib>

[~/src/alios-cats]$ stack ghci --ghci-options=-ferror-spans                                                                                              
alios-cats-0.1.0.0: initial-build-steps (lib)
Configuring GHCi with the following packages: alios-cats
GHCi, version 8.2.1: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling SHA256           ( /Users/alios/src/alios-cats/src/SHA256.hs, interpreted )
[2 of 2] Compiling Lib              ( /Users/alios/src/alios-cats/src/Lib.hs, interpreted )
Ok, 2 modules loaded.
Loaded GHCi configuration from /private/var/folders/rq/lpz047951cb5jqzl0x67mjrc0000gn/T/ghci43024/ghci-script
*Lib Lib> 
Leaving GHCi.

Try to fix:

The change from 8.02 to 8.2.1 are:

• from Ok, modules loaded: Lib, SHA256. to Ok, 2 modules loaded. for 2 modules
• from Ok, modules loaded: Lib. to Ok, 1 module loaded. for 1 module

So I added another condition to haskell-load.el, the one with the regexp "Ok, ([[:digit:]]+) modules? loaded.$":

(defun haskell-process-load-complete (session process buffer reload module-buffer &optional cont)
  "Handle the complete loading response. BUFFER is the string of
text being sent over the process pipe. MODULE-BUFFER is the
actual Emacs buffer of the module being loaded."
  (when (get-buffer (format "*%s:splices*" (haskell-session-name session)))
    (with-current-buffer (haskell-interactive-mode-splices-buffer session)
      (erase-buffer)))
  (let* ((ok (cond
              ((haskell-process-consume
                process
                "Ok, modules loaded: \\(.+\\)\\.$")
               t)
              ((haskell-process-consume
                process
                "Ok, ([[:digit:]]+) modules? loaded.$")
               t)
              ((haskell-process-consume
                process
                "Failed, modules loaded: \\(.+\\)\\.$")
               nil)
              (t
               (error (message "Unexpected response from haskell process.")))))
         (modules (haskell-process-extract-modules buffer))
         (cursor (haskell-process-response-cursor process))
         (warning-count 0))

But existing regex is used match a group (the comma seperated list of module names), which no longer exists in ghci 8.2. So my matcher works But I don't have the string with the modules which seems to be used further down in the chain.

[zenspider]

@alios your code is missing 2 close parens and your regexp is incorrect. Remove the parens in the regexp or double backslash them. Since you're capturing the count of modules and that isn't used, you can drop them.

I'm still trying to figure out how to get the modules printing so they can be introspected properly... but I don't know ghc's flags at all.

[zenspider]

Actually you need the parens so it has SOMETHING to process as module names, even if it is just a number.

[treblacy]

The module list seems to be only important if you turn on haskell-process-reload-with-fbytecode or you turn on haskell-process-auto-import-loaded-modules.

I turn them off (never needed them) so I'm doing OK. But I can't speak for other people's use cases.

[swilde]

The relevant change in ghc is
https://phabricator.haskell.org/D3651
so this functionality is lost and there is no command line option to
get it back.

But it should be possible to get the necessary information from
":show modules" which might be the better solution anyways...

[swilde]

Btw, the way, the changes in haskell-load.el need to match the case of failure, too.

[peti]

Is anyone aware of a solution to the problems with GHC 8.2.1, or at least of a way to work around them? It's rather unpleasant that haskell-mode doesn't support that compiler properly at the moment since many projects switch to the new GHC version and it's getting more and more difficult (for me) to stick to 8.0.x. Any help remedying this issue would be very much appreciated.

[ksjogo]

Did someone talk to some upstream developers/has connections? Some machine readable output would be interesting to have behind some flag.

[ryukinix]

Fixing the @alios solution I got my code loaded again... But I don't know if will work always and the modules names are missing because the breaking change of new ghc, as people above commented.

(defun haskell-process-load-complete (session process buffer reload module-buffer &optional cont)
  "Handle the complete loading response. BUFFER is the string of
text being sent over the process pipe. MODULE-BUFFER is the
actual Emacs buffer of the module being loaded."
  (when (get-buffer (format "*%s:splices*" (haskell-session-name session)))
    (with-current-buffer (haskell-interactive-mode-splices-buffer session)
      (erase-buffer)))
  (let* ((ok (cond
              ((haskell-process-consume
                process
                "Ok, modules loaded: \\(.+\\)\\.$")
               t)
              ((haskell-process-consume  ;; new condition
                process
                "Ok, \\([[:digit:]]+\\) modules? loaded.$")
               t)
              ((haskell-process-consume
                process
                "Failed, modules loaded: \\(.+\\)\\.$")
               nil)
              (t
               (error (message "Unexpected response from haskell process.")))))
         (modules (haskell-process-extract-modules buffer))
         (cursor (haskell-process-response-cursor process))
         (warning-count 0))
    (haskell-process-set-response-cursor process 0)
    (haskell-check-remove-overlays module-buffer)
    (while
        (haskell-process-errors-warnings module-buffer session process buffer)
      (setq warning-count (1+ warning-count)))
    (haskell-process-set-response-cursor process cursor)
    (if (and (not reload)
             haskell-process-reload-with-fbytecode)
        (haskell-process-reload-with-fbytecode process module-buffer)
      (haskell-process-import-modules process (car modules)))
    (if ok
        (haskell-mode-message-line (if reload "Reloaded OK." "OK."))
      (haskell-interactive-mode-compile-error session "Compilation failed."))
    (when cont
      (condition-case-unless-debug e
          (funcall cont ok)
        (error (message "%S" e))
        (quit nil)))))

@peti I think you can at least silent the useless errors with it...

[peti]

@ryukinix, thank you very much for the updated patch. It does remedy the parser errors when loading a module, indeed, but another significant problem remains, i.e. Emacs still fails with a "parse error" in case the loaded Haskell module contains syntax errors. With ghc 8.0.x and before, Haskell mode would display those error messages in the ghci process buffer, but with 8.2.x it no longer does. This makes development with 8.2 pretty much unpractical, still. 😞

[ryukinix]

I'm glad that this helped in someway. Sad about the syntax errors, I understand that this is not good enough... would be nice if the GHC developers instead of just removing features because they THINK that no ones use, just disable by default and keep some flag to be possible enable again... as the commit that @swilde attached on this thread, seems that doesn't exists such thing. They just removed all the shit :/

We need to figure out another approach to handling this cases... more hacks? 😆

[Mistuke]

Raise an issue on the ghc bug tracker for this. It's trivial to bring this back and hide the new behavior behind a flag. Which in my opinion is what should have been done in the first place so I agree with you there.

I can assure you that all these "quiet the UI" patches have all been quite controversial but a good way to raise a stink and get it fixed before release is to try the RC builds.

I don't think you guys should reduce the quality of your code with hacks for something that can be easily corrected upstream.

Anyways, raise a ticket. There's still time to correct it for 8.4

[Mistuke]

Alternatively, go to https://phabricator.haskell.org/rGHC14ff644c45e04ac6441f2faa3d2a518a7201ab11 log in (you can do so with your GitHub credentials if you don't have an account) and scroll to the bottom, you'll find an box with actions, pick Raise a concern and explain what's happening.

The original author an reviewers will be notified of this then.

[bgamari]

Did someone talk to some upstream developers/has connections? Some machine readable output would be interesting to have behind some flag.

Please do come raise this idea with us either on IRC (#ghc on Freenode), Trac (http://ghc.haskell.org) or ghc-devs (https://mail.haskell.org/mailman/listinfo/ghc-devs). The more concrete the proposal the better.

We would love to put in place some machine readable output to make consumers like haskell-mode more robust. However, we can't do so without knowing what your needs are.

[bgamari]

To summarize: Sven Panne kindly brought this issue to my attention this morning on ghc-devs. Given that we need a solution quickly on account of the coming 8.2.2 release, I quickly added a flag (-show-loaded-modules) as suggested by Sven.

However, in general I'm not at all fond of adding complexity to the compiler to allow a downstream project to continue using a REPL as a language server. In fact, I would like to deprecate this flag as soon as possible. The ultimate solution here, as mentioned in this thread, is to design a machine-readable interface. Let's do this. I've opened #1570 to track this.

[peti]

So ... what is the current state of affairs? Can I use ghc 8.2.2 with a current version of haskell-mode?

[treblacy]

The good news is that -fshow-loaded-modules restores listing loaded modules.

But there are bad news too.

In the GHC User's Guide, The release note says "-show-mods-loaded" erroneously (probably because hastily). (The flag reference has it right, though.)

If you don't use -fshow-loaded-modules, then there is also a change: Whereas 8.2.1 gives "Ok, 5 modules loaded.", 8.2.2 gives "Ok, five modules loaded.". I have only tested up to five, I don't know where it ends.

[mschristiansen]

For anyone looking for a quick fix. M-x customize-option enter haskell-process-args-stack-ghci or haskell-process-args-cabal-repl. Add -fshow-loaded-modules to the list.

[heptahedron]

To further elaborate on @mschristiansen's quick fix: in my case, I had to add -fshow-loaded-modules to the --ghci-options argument being passed to stack ghci. The list item was previously --ghci-options=-ferror-spans, I just had to add it to the end of this item as in --ghci-options=-ferror-spans -fshow-loaded-modules. I imagine you could simply add -fshow-loaded-modules to the end of the list if you were only using ghci and not stack ghci or cabal repl.

[dmjio]

To further elaborate again, for GHC 8.2.2. Calling M-x customize-option, enter, then haskell-process-args-cabal-repl, enter. And making the path look like --ghc-options=-ferror-spans -fshow-loaded-modules, saving the changes and then calling M-x haskell-process-restart did the trick. @mschristiansen is a hero.

[raxod502]

For people like me who find it difficult to follow the above instructions:

    (setq haskell-process-args-ghci
          '("-ferror-spans" "-fshow-loaded-modules"))

    (setq haskell-process-args-cabal-repl
          '("--ghc-options=-ferror-spans -fshow-loaded-modules"))

    (setq haskell-process-args-stack-ghci
          '("--ghci-options=-ferror-spans -fshow-loaded-modules"
            "--no-build" "--no-load"))

    (setq haskell-process-args-cabal-new-repl
          '("--ghc-options=-ferror-spans -fshow-loaded-modules"))

[redfish64]

raxod502, you need to add "--ghc-options" twice:

(setq haskell-process-args-cabal-repl
'("--ghc-options -fshow-loaded-modules --ghc-options -ferror-spans"))

[k-bx]

With current haskell-mode and GHC 8.4.3, this works for me:

  (setq haskell-process-args-stack-ghci '("--ghci-options=-ferror-spans -fshow-loaded-modules"))

[lehins]

For those who still want their haskell-mode and stack ghci to work together with ghc-8.2.2 and newer, while not breaking ghc versions that don't know about -fshow-loaded-modules flag, such as ghc-8.0.2 and older, here is a little trick that I did to use that flag only for ghc versions that do support it.

Change you ghci scripts for all ghc-8.2.2 version and newer to include -fshow-loaded-modules flag, eg. for ghc-8.2.2:

$HOME/.stack/programs/x86_64-linux/ghc-8.2.2/bin/ghci :

#!/bin/bash
exec "/home/user/.stack/programs/x86_64-linux/ghc-8.2.2/bin/ghc-8.2.2" --interactive -fshow-loaded-modules "$@"

By default stack doesn't call ghci, but ghc --interactive instead, so we also need to let stack know to use ghci:

  (setq haskell-process-args-stack-ghci
        '("--with-ghc=ghci"
          "--ghci-options=-ferror-spans"
          "--no-build" "--no-load" "--test" "--bench"))

I am sure similar approach will work for cabal as well, except that system ghci script would have to be modified instead.
CC @owickstrom We've discussed a workaround for this problem before.

[raxod502]

Is there a plan to fix this issue? If I understand correctly, the interactive Haskell integration has been completely broken since last September. What??

I totally get that open-source takes time to maintain, but I'm just wondering what the status is. Are we waiting for a pull request to be submitted? Are we aiming to get something changed upstream? Are we still trying to figure out what the correct approach is?

[bgamari]

Generally the solution should come in the form of #1570. GHC added the -fshow-loaded-modules flag as a temporary workaround. However, we would really like to remove sometime in the next few years.

Someone just needs to take ownership of haskell-mode's REPL interaction and design an interface which GHCi can expose. @alanz has thought about this a fair amount in the past; I'm sure he has interesting thoughts to offer.

[bgamari]

For the record, I have opened a GHC ticket where we can collect design ideas. Please do chime in; we would love to improve GHC's tooling story but we need your input to do so.

[raxod502]

Thanks! In the meantime (before all of this gets fixed properly) would it make sense to put my workaround #1553 (comment) or something similar in the documentation of haskell-mode since it's now required in order to use haskell-mode with recent GHCI versions? Or perhaps to make it happen by default?