Build HIE with cabal via install.hs

[fendor]

Closes #1047

Discovers GHC installations that can be found via System.Directory.findExecutable. Found GHC installations are shown with cabal-ghcs target.
Adapts the target cabal-build and cabal-docs to only install HIE for GHCs that have already been found.

HIE executables are installed to $HOME/.local/bin. The main executable hie will be symlinked to that location, as well as hie-wrapper, while the other executables will be copied. We could also symlink them, but this may cause Problems for Windows. On the other hand, we could also replace the symlinked hie and hie-wrapper with real copies, or wait until the respective flag lands in cabal.

Adds more helpful error messages such as, "GHC not found!".

Blockers
Tested on the systems:

• Windows, does currently not work since cabal new-install does not work for windows, see Task: Implement v2-install --bindir-method={copy,symlink} cabal#5837
• Fedora
• MacOS
• NixOS
• Readme updated

The documentation needs some changes. If someone uses cabal new-build without our smart wrapper, a .ghc.environment file is generated. Almost any tool seems to read that and consequentially, the script install.hs can not be run because its dependencies are not added to the local .ghc.environment file. We can not include a fix for this, like deleting the file if present, in the install.hs since it can not be run at all. Thus, I propose we add this to the FAQ.

Discussion:

• Is the cabal target needed for installation via cabal new-build? I removed it for now.
• We require that $HOME/.cabal/bin is in the path instead of $HOME/.local/bin in the current implementation. We no longer require $HOME/.cabal/bin to be on the $PATH.

[lukel97]

I'm getting an error when building with ghc-8.4.4 via ./install.sh cabal-build-all on macOS:

Error when running Shake build system:
* cabal-build-all
* cabal-build
* cabal-hie-8.4.4
user error (Development.Shake.command_, system command failed
Command: cabal new-build --write-ghc-environment-files=never
Exit code: 1
Stderr:
<command line>: can't load .so/.DLL for: /Users/luke/.cabal/store/ghc-8.4.4/lib/libHShddck-p-2.20.0-12cd95e5-ghc8.4.4.dylib (dlopen(/Users/luke/.cabal/store/ghc-8.4.4/lib/libHShddck-p-2.20.0-12cd95e5-ghc8.4.4.dylib, 5): Symbol not found: _timezm1zi8zi0zi2_DataziTimeziClockziInternalziCTimespec_zdwclockGetTime_closure
  Referenced from: /usr/local/opt/ghc/lib/ghc-8.4.4/ghc-8.4.4/libHSghc-8.4.4-ghc8.4.4.dylib
  Expected in: /Users/luke/.stack/programs/x86_64-osx/ghc-8.4.4/lib/ghc-8.4.4/bin/../time-1.8.0.2/libHStime-1.8.0.2-ghc8.4.4.dylib
 in /usr/local/opt/ghc/lib/ghc-8.4.4/ghc-8.4.4/libHSghc-8.4.4-ghc8.4.4.dylib)
cabal: Failed to build haskell-ide-engine-0.6.0.0 (which is required by
exe:hie-wrapper from haskell-ide-engine-0.6.0.0, lib:hie-test-utils from
haskell-ide-engine-0.6.0.0 and others).

Not sure why both stack and my system ghc are shown in the paths above

Here are my GHCs installed:

Found the following GHC paths: 
ghc-8.6.3: /usr/local/bin/ghc-8.6.3
ghc-8.6.2: /usr/local/bin/ghc-8.6.2
ghc-8.4.4: /Users/luke/.stack/programs/x86_64-osx/ghc-8.4.4/bin/ghc-8.4.4

[fendor]

Hmpf, I think stack script changes the path such that the GHC of the global project is on the PATH. Not sure what causes the error though.

Maybe there is a difference between GHCs installed by stack and those installed from source/package manager? At least for MacOS, since it worked for NixOs, Windows and Fedora with GHCs installed by stack.

[lukel97]

Cleaning up my dist and dist-newstyle seemed to help, I presume something funny went wrong in the configure step, probably not the fault of the install script.

[fendor]

Mainly because then we know where the locations of the executables are located. It then also has the same behaviour as build.
Moreover, I don't know how else I can obtain the path to the executable

[lukel97]

I believe cabal new-install symlinks into ~/.cabal/bin/ by default, but that makes sense in terms of keeping the artefacts consistent with the stack installation

[fendor]

I believe cabal new-install symlinks into ~/.cabal/bin/ by default, but that makes sense in terms of keeping the artefacts consistent with the stack installation

Yes, but I am not sure where it is on windows and also it might break if someone has a custom cabal configuration.

[Anrock]

@fendor do i understand correctly that this PR adds\alters cabal targets, while stack targets are unaffected? Also, if those cabal targets aren't properly working on Windows - will user get a clear message about it if he tries to run it on Windows machine?

[fendor]

@Anrock That is right, it adds targets to build HIE via cabal new-build. Stack targets are not affected except that some implementation details have changed.
Regarding windows, yes and no, it will build hie as expected but when trying to install, cabal will terminate with a message that it does not work on windows atm. So, you have the worst of both: It will build and fail just before the finish line.

[Anrock]

@fendor borderline acceptable, imo. Could you either hide broken cabal targets or fail them instantly with error message specifically for Windows? If no/too much effort, then LGTM.

[fendor]

@Anrock You are right, I will add a check whether the script is executed on mingw32 and error out accordingly.

[Anrock]

@fendor shouldn't it be mingw32 || win32? I'm using stack via cmd.exe instead of mingw mintty and i'm not exatcly sure if in this case host will be reported as mingw32.

[alanz]

I am out of my depth on this one, but if you guys are happy with it, it can merge.

[fendor]

@fendor shouldn't it be mingw32 || win32? I'm using stack via cmd.exe instead of mingw mintty and i'm not exatcly sure if in this case host will be reported as mingw32.

On my windows, I tested it with
stack ghci on cmd and powershell, the result was mingw32
compiled with stack ghc --stack-yaml shake.yaml -- -O2 install.hs; ./install.exe and the result was always reported as mingw32. Not sure if win32 is actually a possible result of System.Info.os

[fendor]

Should show now a nice error message when trying to use one of the cabal targets.

[fendor]

A fitting readme is missing.

[alanz]

I am worried about the first-time user experience with this. Running the bare command gives a wall of text, with so many options it is not clear what is going on.

$ ./install.hs 

Usage:
    stack install.hs <target>

Targets:
    help                      Show help
    cabal                     Makes sure that Cabal the lib is available for cabal-helper-wapper, to speed up project start
    dist                      Creates a tarball containing all the hie binaries
    build                     Builds hie for all supported GHC versions (8.2.1, 8.2.2, 8.4.2, 8.4.3, 8.4.4, 8.6.1, 8.6.2 and 8.6.3)
    build-all                 Builds hie and hoogle databases for all supported GHC versions
    build-docs                Builds the Hoogle database for all supported GHC versions
    test                      Runs hie tests with stack
    hie-8.2.1                 Builds hie for GHC version 8.2.1 only with stack
    hie-8.2.2                 Builds hie for GHC version 8.2.2 only with stack
    hie-8.4.2                 Builds hie for GHC version 8.4.2 only with stack
    hie-8.4.3                 Builds hie for GHC version 8.4.3 only with stack
    hie-8.4.4                 Builds hie for GHC version 8.4.4 only with stack
    hie-8.6.1                 Builds hie for GHC version 8.6.1 only with stack
    hie-8.6.2                 Builds hie for GHC version 8.6.2 only with stack
    hie-8.6.3                 Builds hie for GHC version 8.6.3 only with stack
    build-doc-8.2.1           Builds the Hoogle database for GHC version 8.2.1 only with stack
    build-doc-8.2.2           Builds the Hoogle database for GHC version 8.2.2 only with stack
    build-doc-8.4.2           Builds the Hoogle database for GHC version 8.4.2 only with stack
    build-doc-8.4.3           Builds the Hoogle database for GHC version 8.4.3 only with stack
    build-doc-8.4.4           Builds the Hoogle database for GHC version 8.4.4 only with stack
    build-doc-8.6.1           Builds the Hoogle database for GHC version 8.6.1 only with stack
    build-doc-8.6.2           Builds the Hoogle database for GHC version 8.6.2 only with stack
    build-doc-8.6.3           Builds the Hoogle database for GHC version 8.6.3 only with stack
    cabal-ghcs                Show all GHC versions that can be installed via `cabal-build` and `cabal-build-all`.
    cabal-build               Builds hie with cabal with all installed GHCs. Target `cabal-ghcs` shows all GHC versions that will be installed
    cabal-build-all           Builds hie and hoogle databases for all installed GHC versions with cabal
    cabal-build-docs          Builds the Hoogle database for all installed GHC versions with cabal
    cabal-test                Runs hie tests with cabal
    cabal-hie-8.2.1           Builds hie for GHC version 8.2.1 only with cabal new-build
    cabal-hie-8.2.2           Builds hie for GHC version 8.2.2 only with cabal new-build
    cabal-hie-8.4.2           Builds hie for GHC version 8.4.2 only with cabal new-build
    cabal-hie-8.4.3           Builds hie for GHC version 8.4.3 only with cabal new-build
    cabal-hie-8.4.4           Builds hie for GHC version 8.4.4 only with cabal new-build
    cabal-hie-8.6.1           Builds hie for GHC version 8.6.1 only with cabal new-build
    cabal-hie-8.6.2           Builds hie for GHC version 8.6.2 only with cabal new-build
    cabal-hie-8.6.3           Builds hie for GHC version 8.6.3 only with cabal new-build
    cabal-build-doc-8.2.1     Builds the Hoogle database for GHC version 8.2.1 only with cabal
    cabal-build-doc-8.2.2     Builds the Hoogle database for GHC version 8.2.2 only with cabal
    cabal-build-doc-8.4.2     Builds the Hoogle database for GHC version 8.4.2 only with cabal
    cabal-build-doc-8.4.3     Builds the Hoogle database for GHC version 8.4.3 only with cabal
    cabal-build-doc-8.4.4     Builds the Hoogle database for GHC version 8.4.4 only with cabal
    cabal-build-doc-8.6.1     Builds the Hoogle database for GHC version 8.6.1 only with cabal
    cabal-build-doc-8.6.2     Builds the Hoogle database for GHC version 8.6.2 only with cabal
    cabal-build-doc-8.6.3     Builds the Hoogle database for GHC version 8.6.3 only with cabal
    icu-macos-fix             Fixes icu related problems in MacOS

Build completed in 0:01m

[fendor]

I agree with that, however, in the readme there are simple instructions
what targets do you think are not required?
Or maybe, help shows all targets, while no target just shows a clear overview?

[alanz]

Or maybe, help shows all targets, while no target just shows a clear overview?

This sounds like a good idea.

[fendor]

Introduced short help message that only shows the commands absolutely necessary, example output:

> stack install.hs  # equal to `stack install.hs short-help` and `stack install.hs all`

Usage:
    stack install.hs <target>

Targets:
    help                      Show help message including all targets

    build                     Builds hie for all supported GHC versions (8.2.1, 8.2.2, 8.4.2, 8.4.3, 8.4.4, 8.6.1, 8.6.2 and 8.6.3)
    build-all                 Builds hie and hoogle databases for all supported GHC versions
    hie-8.6.3                 Builds hie for GHC version 8.6.3 only with stack
    build-doc-8.6.3           Builds the Hoogle database for GHC version 8.6.3 only with stack
    hie-8.4.4                 Builds hie for GHC version 8.4.4 only with stack
    build-doc-8.4.4           Builds the Hoogle database for GHC version 8.4.4 only with stack
                              
    cabal-ghcs                Show all GHC versions that can be installed via `cabal-build` and `cabal-build-all`.
    cabal-build               Builds hie with cabal with all installed GHCs.
    cabal-build-all           Builds hie and hoogle databases for all installed GHC versions with cabal
    cabal-hie-8.6.3           Builds hie for GHC version 8.6.3 only with cabal new-build
    cabal-build-doc-8.6.3     Builds the Hoogle database for GHC version 8.6.3 only with cabal
    cabal-hie-8.4.4           Builds hie for GHC version 8.4.4 only with cabal new-build
    cabal-build-doc-8.4.4     Builds the Hoogle database for GHC version 8.4.4 only with cabal

Build completed in 0.00s

[fendor]

In my opinion, ready to merge.
I would ask for a quick proof read of the changed README.

[alanz]

I picked up one tiny typo in the README, once that is fixed I am happy for this to land.

[alanz]

LGTM, will land it once CI completes

[tyrak]

Do I understand it correctly that this PR show to make it possible to install HIE without stack? Because I tried to do that on my system (x86_64 linux, w/o stack, with ghc 8.6.3) and it failed. Specifically, I compiled install.hs with cabal, and then ran ./install cabal-hie-8.6.3, but it failed after a couple of minutes with the following error message:

# stack (for cabal)
Error when running Shake build system:
  at want, called at src/Development/Shake/Internal/Args.hs:83:69 in shake-0.17.6-2Y524cW7xdJLdvFCU0NUxw:Development.Shake.Internal.Args
* Depends on: cabal-hie-8.6.3
  at need, called at install.hs:121:9 in main:Main
* Depends on: cabal
* Raised the exception:
stack: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

which indicates that it needs stack to continue.

[fendor]

No, it means you can build hie with cabal, we still require stack to ensure that cabal is installed and to know the stack path --local-bin location. It is one of the design goals of install.hs to only require stack and git.