new-* seem to ignore --with-hsc2hs and --hsc2hs-options=

[angerman]

As such it picks up a random hsc2hs binary in PATH, and without --with-hsc2hs being honored (as @hvr explains to me) by design, there is no way around this. Not respecting --hsc2hs-options=, e.g. --hsc2hs-options=--cross-compile, cross compiling with new-build seems impossible.

[angerman]

I would like to challenge the idea that --with-hsc2hs should not be supported by design. I do understand the rational of waning to pull in build tools on it's own, however a manual override seems useful in special cases.

[23Skidoo]

/cc @dcoutts

[hvr]

Let me elaborate a bit about some of the issues current issues with build-tools: hsc2hs and --with-hsc2hs. For starters, what is --with-hsc2hs even supposed to mean? One possibility is to impose a hard constraint cabal to only ever use the provided hsc2hs binary, which would effectively imply a hard version constraint on hsc2hs to the solver, which would otherwise have the liberty of picking different versions of hsc2hs in the same install-plan.

A related issue is if hsc2hs happens to occur on $PATH. If such a hsc2hs found on $PATH is forced to be used, it would effectively regress cabal to the pre-qualified-goals situation for hsc2hs, and would result in less reachable install-plans than if there's no hsc2hs on $PATH at all.

In the build-tool-happens-to-be-on-$PATH case however I'd argue that we have a comparable situation, insofar as this is to build-tools-depends, like global pkg-db is to libraries declared via build-depends. Except it's a bit unfortunate that we'd reintroduce the statefulness we've just gotten rid of for libraries with the nix-style local builds, as the reproducibility of your install-plans now depends on what you placed on your $PATH (and if/when you ran cabal install hsc2hs, and whether ${HOME}/.cabal/bin happens to be on your $PATH).

Consequently IMO cabal should ignore $PATH for build-tools it can install itself into the nix-style store (which includes hsc2hs) for the sake of consistency with the nix-style paradigm we're aiming for with cabal new-build.

[angerman]

--with-hsc2hs would mean "use this hsc2hs" regardless. Assume it is any version you expect it to be. (E.g. I really know what I'm doing).

I would also argue that if we assume we build build-tools (e.g. hsc2hs), we should never use the one found on $PATH, but look it up in the location cabal expects it to be. However if --with-hsc2hs is passed, with a non-absolute path, it should be looked up in $PATH. Assuming i pass --with-hsc2hs=my-hsc2hs, and my-hsc2hs is in $PATH, that should work, without having to specify the absolute path to my-hsc2hs.

As such I agree with your final conclusion with the exception, that I do want that escape hatch that allows me to specify a specific program to use, in which case cabal just assumes I know what I'm doing.

[angerman]

I've just built a cabal from head, and both --with-hsc2hs and --hsc2hs-options=--cross-compile do work. Now I'm thoroughly confused.

for the record

blake2 $ ../cabal/dist-newstyle/build/x86_64-osx/ghc-8.2.1/cabal-install-2.1.0.0/build/cabal/cabal new-build -v --hsc2hs-option=--cross-compile --with-hsc2hs=false
...
/usr/bin/false '--cc=/usr/bin/clang' '--ld=/usr/bin/clang' '--cflag=-m64' '--cflag=-fno-stack-protector' '--cflag=-m64' '--lflag=-m64' '--lflag=-fno-stack-protector' '--lflag=-m64' '--cflag=-D__GLASGOW_HASKELL__=802' '--cflag=-Ddarwin_BUILD_OS=1' '--cflag=-Dx86_64_BUILD_ARCH=1' '--cflag=-Ddarwin_HOST_OS=1' '--cflag=-Dx86_64_HOST_ARCH=1' '--cflag=-Iinclude/sse' '--cflag=-std=c99' '--cflag=-I/Users/angerman/Projects/zw3rk/blake2/dist-newstyle/build/x86_64-osx/ghc-8.2.1/blake2-0.2.0/build/autogen' '--cflag=-I/Users/angerman/Projects/zw3rk/blake2/dist-newstyle/build/x86_64-osx/ghc-8.2.1/blake2-0.2.0/build/global-autogen' '--cflag=-include' '--cflag=/Users/angerman/Projects/zw3rk/blake2/dist-newstyle/build/x86_64-osx/ghc-8.2.1/blake2-0.2.0/build/autogen/cabal_macros.h' '--cflag=-I/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/bytestring-0.10.8.2/include' '--cflag=-I/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/base-4.10.0.0/include' '--cflag=-I/usr/local/Cellar/ghc/8.2.1/libexec/integer-gmp/include' '--cflag=-I/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/integer-gmp-1.0.1.0/include' '--cflag=-I/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/include' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/bytestring-0.10.8.2' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/deepseq-1.4.3.0' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/array-0.5.2.0' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/base-4.10.0.0' '--lflag=-liconv' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/integer-gmp-1.0.1.0' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/libexec/integer-gmp/lib' '--lflag=-lgmp' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/ghc-prim-0.5.1.0' '--lflag=-L/usr/local/Cellar/ghc/8.2.1/lib/ghc-8.2.1/rts' '--lflag=-lm' '--lflag=-ldl' -o /Users/angerman/Projects/zw3rk/blake2/dist-newstyle/build/x86_64-osx/ghc-8.2.1/blake2-0.2.0/build/Crypto/Hash/BLAKE2/BLAKE2b.hs src/Crypto/Hash/BLAKE2/BLAKE2b.hsc --cross-compile
...

[jprider63]

I'm experiencing this when attempting to cross-compile with head.hackage. Here are the full logs of the failing compilation. --cross-compile is passed to basement and foundation, but not memory. FYI memory depends on foundation, which depends on basement.

Here is my cabal.project:

-- generated by head.hackage.sh

with-compiler: /usr/local/ghc-mobile/aarch64-apple-ios-ghc

packages: .
  /Users/james/Documents/Projects/foundation/basement
  /Users/james/Documents/Projects/foundation
  /Users/james/Documents/PKAuth/Projects/crypto-simple-haskell
  /Users/james/Documents/PKAuth/Projects/pkauth-haskell
-- foundation crypto-simple-haskell pkauth-haskell

package cryptonite
  flags: -integer-gmp

repository head.hackage
   url: http://head.hackage.haskell.org/
   secure: True
   root-keys: 07c59cb65787dedfaef5bd5f987ceb5f7e5ebf88b904bbd4c5cbdeb2ff71b740
              2e8555dde16ebd8df076f1a8ef13b8f14c66bad8eafefd7d9e37d0ed711821fb
              8f79fd2389ab2967354407ec852cbe73f2e8635793ac446d09461ffb99527f6e
   key-threshold: 3

-- end of generated content

Checking out memory and adding it to the packages list does not seem to solve the issue. @angerman thinks this might have something to do with the depth of the dependency graph (recursive descent is messing up the flags?).

[angerman]

@jprider63 as outlined in #4939, this probably worked for foundation, but not for memory, because you have foundation as a local package in your project and as such it gets the --with-PROG and --PROG-option(s) treatment. To get memory to compile, I would suggest that you ask cabal unpack memory and thenn add the memory-<version> path to the packages part of your cabal.project.

If this comes up with more / other packages, you'll likely need to do the same. I would even think you might have to basically unpack all (even transitive) dependencies for this to work reliable with cabal right now :(

[angerman]

This was fixed with #5018