Support cabal/stack/nix shebangs

[anka-213]

All three tools support writing a self-contained haskell script with a shebang at the top which describes
what package dependencies the Haskell script has.

Cabal syntax

Cabal

Cabal syntax looks like this: https://github.com/nrolland/helloNixShebang/blob/master/cabal_hello.hs

#! /usr/bin/env cabal
{- cabal:
index-state: 2019-01-02T10:01:10Z
with-compiler: ghc-8.0.1
build-depends: base, type-level-sets
-}

and is documented ... here i guess?

Stack syntax

Stack

Stack syntax looks like this

#!/usr/bin/env stack
-- stack --resolver lts-14.20 script

-- or
{- stack
  script
  --resolver lts-14.20
  --package turtle
  --package "stm async"
  --package http-client,http-conduit
-}

-- this script makes use of the http-client library, which is in stackage

or like this

#!/usr/bin/env stack
-- stack --resolver lts-14.20 script --package type-level-sets
-- (the package type-level-sets is not in stackage)

and is documented here: https://docs.haskellstack.org/en/latest/GUIDE/#script-interpreter

Nix syntax

Nix

Nix syntax looks like this: https://github.com/nrolland/helloNixShebang/blob/master/nix_hello.hs

#! /usr/bin/env nix-shell
#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages(p: with p; [type-level-sets])"
#! nix-shell -I nixpkgs=channel:nixos-18.03

and is documented here (and here)

---

I would love to have support for any of the three formats in ghcide, whichever is easiest to implement.

The nix support could probably be implemented by having the process relaunch itself (or the ghci instance) in a nix-shell with the parameters from the nix-shell shebang line, excluding the -i runghc part. I don't know if stack/cabal support would require any extra support from stack/cabal or if everything we need is already available.

[fendor]

Cabal

Firstly, iirc, the fields index-state and with-compiler are ignored by cabal.
What we need is cabal to support giving us the compilation flag for scripts. This can be done by implementing any of these issues/prs

• cabal v2-repl scriptfile.hs cabal#6149
• Extend show-build-info (exe:cabal part plus tests) cabal#6241 to work on scripts

Stack

That used to work, but apparently there was some regression? E.g. when stack repl ./Script.hs works (it used to, but with 2.1.3.1 it seems not to), hie-bios can load it.
We would need proper support from stack to support scrtips.

Nix

Is there a way to get the ghci arguments from nix that are used to compile this script?

[jneira]

I would love to have support for any of the three formats in ghcide, whichever is easiest to implement.

Could we implement it first for one that works natively in all operating systems, please? 😄

[anka-213]

@fendor

Nix

Is there a way to get the ghci arguments from nix that are used to compile this script?

I think by default it doesn't use any arguments to ghci. It just creates a ghc installation with the relevant packages installed, sets the environment variables to point to that ghcWithPackages and puts it in path and runs ghci scriptname.hs or runghc scriptname.hs. Normal shebangs don't allow passing arguments to the interpreter and I don't think the nix-shell syntax supports that either.

Another option for nix could be to fetch the environment variables that nix creates the same way that direnv does it.

[anka-213]

@jneira What do you mean with "natively"? Is WSL ok? If so, I would assume that all three should work in all three major environments (Linux, Mac, Windows), but I haven't tested it on Windows yet. Does shebangs work on windows at all without WSL?

[jneira]

What do you mean with "natively"? Is WSL ok?

In fact i wanted to exclude it. 😉

Does shebangs work on windows at all without WSL?

Yeah, in a msys2 console, for example.
But it does not support nix and it seems it never will be.

[fendor]

I think by default it doesn't use any arguments to ghci.

Probably true, this makes it very complicated for us to know how to compile the given file.

Without further support from nix, I dont think this will be easy to integrate at all.

[anka-213]

@fendor Why does not needing command line arguments make it difficult? I am not familiar with the internals of how hie-bios works. Why is not environment variables sufficient?

If all we need is a ghci with the right environment, this should work:

nix-shell -p "haskellPackages.ghcWithPackages(p: with p; [type-level-sets])" --run "ghci NameOfFile.hs"

But I assume something more is needed for the ide to work?

[fendor]

Why does not needing command line arguments make it difficult?

Because hie-bios by design asks the build-tool how it should compile a given file. It tells these options to ghcide/hls, which use them to load and compile the given file. I dont really see how this workflow can integrate a nix-shell, since hie-bios would need to open the shell, but ghcide / hls will eventually use the options to compile the flag and they are not within the nix-shell anymore.

Why is not environment variables sufficient?

What information is given in environment variables?
We need the exact flags that are necessary to compile the given file.

[anka-213]

What information is given in environment variables?
We need the exact flags that are necessary to compile the given file.

I was going to say "no flags are needed" again, but apparently there are flags, they are just sneakily applied through a wrapper shell script.

The only relevant environment variable seems to be PATH, which includes a link to the generated GHC installation. This installation contains wrapper shell scripts for all the ghc* binaries. Here is for example the generated ghc wrapper for the example above.

$ cat /nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/bin/ghc
#! /nix/store/nfd458cwwymmqpy5qb1q7a92zqp7mqsz-bash-4.4-p23/bin/bash -e
export NIX_GHC='/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/bin/ghc'
export NIX_GHCPKG='/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/bin/ghc-pkg'
export NIX_GHC_DOCDIR='/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/share/doc/ghc/html'
export NIX_GHC_LIBDIR='/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/lib/ghc-8.8.3'
exec "/nix/store/02v06008rx2kw6rjp149y3gf822hmwgx-ghc-8.8.3/bin/ghc"  "-B$NIX_GHC_LIBDIR" "$@"

and here is the generated ghc-pkg

$ cat /nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/bin/ghc-pkg
#! /nix/store/nfd458cwwymmqpy5qb1q7a92zqp7mqsz-bash-4.4-p23/bin/bash -e
exec "/nix/store/02v06008rx2kw6rjp149y3gf822hmwgx-ghc-8.8.3/bin/ghc-pkg"  --global-package-db=/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/lib/ghc-8.8.3/package.conf.d "$@"

I guess we could parse these generated files to extract the arguments from that, or I could see if there is a more direct way to get the arguments.

[anka-213]

Here is the file that generates the wrapper-scripts: https://github.com/NixOS/nixpkgs/blob/f5b6ea126f0f85cd2126984607e95151daf9a859/pkgs/development/haskell-modules/with-packages-wrapper.nix

But maybe we can see nix-shell as a build tool and simply forward nix-shell -p "haskellPackages.ghcWithPackages(p: with p; [type-level-sets])" --run ghci to ghcide/hie? I don't know if that would break a ton of assumptions? I guess I can try that with for myself cradle: {bios: {shell: "build-tool flags $HIE_BIOS_ARG"}} and see what happens.

Edit: Nope, it seems like at least ghcide doesn't support that option. So I guess the command line flags are needed.

[fendor]

simply forward nix-shell -p "haskellPackages.ghcWithPackages(p: with p; [type-level-sets])" --run ghci to ghcide/hie?

No, because this runs the shell, but HLS/ghcide itself is responsible for the ghci session.

---

Some of these options seem helpful. Maybe we can craft a bios cradle that does what we want.

[anka-213]

Ah, right. We ask cabal/stack about the command line flags by feeding it a fake ghc that just prints its arguments here: https://github.com/mpickering/hie-bios/blob/2ac11025a109e5f09693c3f328cf05994f9fbb9d/src/HIE/Bios/Cradle.hs#L436-L439

I see. Yes, the same trick wouldn't quite work with nix-shell since the ghc it uses is hard coded in the script. So I guess we would either have to parse the shell script or just extract the base path and assume the relative path stays the same (except for ghc-version number). Or add some tool to ghcWithPackages in nix which just prints out the command line flags needed.

---

Btw, I tried using cradle: {direct: { arguments: ["-B","/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/lib/ghc-8.8.3"]} }, but it complained that ghcide: target ‘-B’ is not a module name or a source file. What am I doing wrong?

[fendor]

You probably need:

cradle: 
  direct: 
    arguments: ["-B/nix/store/c36yvssdnv05sgcszsmm5fj9iky7impx-ghc-8.8.3-with-packages/lib/ghc-8.8.3"]

since it is a single option, not two.

[anka-213]

No. That is not the problem. It seems like it doesn't accept flags for some reason. Only a "module name or a source file". :/

[wraithm]

A thing that I think would work for stack at least is to detect if there's a stack project and then use stack ghc or stack exec ghc instead of whatever ghc is on the path.

[fendor]

@wraithm If there is a stack project, then we basically do this already.

[wraithm]

@fendor In my experience, if there isn't an explicit component listed in the hie.yaml, then the file won't get loaded.

For example, I can do stack repl <my file>, but I get the following from hie-bios check:

$ hie-bios check ./<my file>
hie-bios: ghc: readCreateProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

I don't have a ghc installed globally. I'm only using stack. But even if I put the stack installed ghc into my path, it can't find the libraries I reference.

It seems like it's trying to open the file with the "direct" cradle, but that doesn't work for my case.

$ hie-bios version
hie-bios version 0.5.1 compiled by GHC 8.8.3

[wraithm]

Maybe you've fixed this in a newer version?

[fendor]

I see what you mean and indeed, we do not default to the stack cradle ever, except when we find a "stack.yaml" (assuming stack can be found).
Out of interest, does it work if you have a hie.yaml such as:

cradle:
  stack:
    component: "<path/to/File.hs>"

I think it opens it a "Default" Cradle which also needs a "ghc" on the $PATH.

[wraithm]

So, firstly, I'm opening this file inside of a stack.yaml containing project. This haskell file is actually in the same directory as the stack.yaml and it's still trying to open the file with the direct cradle.

If I do the hie.yaml as you specified above, I get:

hie-bios: AesonException "Error in $.cradle: Expected an object with path and component keys"

I assume this means that the path field is required. If I use this as my hie.yaml:

cradle:
  stack:
    - path: "."
      component: "File.hs"

I get:

hie-bios: ghc: readCreateProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

Which again seems to be trying to use the direct cradle.

The only thing that works is making a cabal file and registering that package with the stack.yaml.