haskellPackages.shellFor with hoogle enabled fails with null dependencies

[alexpeits]

Describe the bug
On the nixpkgs-unstable branch, when there are no haskell dependencies other than base (and other common ones like text) in build-depends in the cabal file, starting a shell with hoogle enabled gives the error:

error: value is null while a set was expected, at /nix/store/<hash>-source/pkgs/development/haskell-modules/hoogle.nix:69:43

To Reproduce
Steps to reproduce the behavior:

1. Create the following default.nix:

{ pkgs ? import <nixpkgs> {} }:

let

  nixpkgs-src = pkgs.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "nixpkgs-unstable";
    sha256 = "06755h2cxrsgrsiclxbk3h5d7yqajvjjq3y4pl1dqbxfbjhq0h67";
  };
  nixpkgs = import nixpkgs-src {};

  haskellPackages = nixpkgs.haskellPackages.override {
    overrides = self: super:
      { testpkg = self.callCabal2nix "testpkg" ./. {}; };
  };

  shell = haskellPackages.shellFor {
    packages = ps: [ ps.testpkg ];
    buildInputs = [ haskellPackages.cabal-install ];
    withHoogle = true;
  };

in

if nixpkgs.lib.inNixShell then shell else haskellPackages.testpkg

2. The following testpkg.cabal:

cabal-version: 2.2

name:           testpkg
version:        0.1.0.0
description:    testpkg
build-type:     Simple

library
  exposed-modules:
  other-modules:
      Paths_testpkg
  hs-source-dirs:
      src
  build-depends:
      base >=4.7 && <5
      -- uncomment the following to make it work
      -- , lens 
  default-language: Haskell2010

3. run nix-shell

Expected behavior
The nix shell is initialized, and running hoogle server --local runs a hoogle server. Adding another dependency in build-depends, e.g. lens, results in the expected behaviour.

Additional context
The issue seems similar to this closed issue

Metadata

these paths will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/4cr2bqya0an66mpqpj195z49453qm8f6-nix-info
copying path '/nix/store/4cr2bqya0an66mpqpj195z49453qm8f6-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 4.15.0-88-generic, Ubuntu, 16.04.6 LTS (Xenial Xerus)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.1`
 - channels(alex): `"nixpkgs-20.03pre211462.0c960262d15"`
 - nixpkgs: `/<redacted>/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: 
  - haskellPackages.shellFor

[cdepillabout]

Ping @Ericson2314.

It is possible this was introduced in either #80845 or #76349.

Also ping @jmininger and @matthewbauer, who did reviews on these issues.

---

Also, @alexpeits, if you'd like to look into this, I'm hoping the fix wouldn't be too tricky.

[alexpeits]

Surprisingly what causes the issue is this warning message

nixpkgs/pkgs/development/haskell-modules/hoogle.nix

Lines 66 to 69 in 001be89

	${lib.optionalString (packages != [] -> docPackages == [])
	("echo WARNING: localHoogle package list empty, even though"
	+ " the following were specified: "
	+ lib.concatMapStringsSep ", " (p: p.name) packages)}

(if I remove the concatMapStringsSep call it works). Seems like packages is a list which contains a null for some reason that I can't figure out, but I'll look into it a bit more

[JosephLucas]

I had the same issue, previous suggestion worked.

[Ericson2314]

https://github.com/NixOS/nixpkgs/pull/42368/files this revert would seem to reintroduce the issue, but the issue was not reopened.

[bolt12]

Today I experienced this error with nixos-20.03, which is weird given that by reading this thread I got the idea that this was already fixed.

Edit: I think this is because the package I was using does not have an executable, but the package only depends on base. How can I work around this?
Here's my derivation:

  shell = myHaskellPackages.shellFor {
    packages = p: [
      p."laop"
    ];
    buildInputs = with pkgs.haskellPackages; [
      myHaskellPackages.cabal-install
      pkgs.neo4j
      (import sources.niv {}).niv
      pkgs.nixpkgs-fmt
    ];
    withHoogle = true;
  };

[stale]

I marked this as stale due to inactivity. → More info

[chuahou]

This bug still occurs for me on nixos-20.09.

[jul1u5]

I have also encountered the same issue. Are there any workarounds?

[cdepillabout]

@jul1u5 I haven't taken a good look at this, but from reading the thread it sounds like these are your options:

1. Don't enable hoogle. (You arguably don't need hoogle if you only depend on base. You may be able to just use https://hoogle.haskell.org/.)
2. Explicitly add a dependency so your library has at least one dependency. (Does this actually get rid of the problem?)
3. Use developPackage instead of shellFor. (Does this actually work?)
4. Investigate if the problem really is haskellPackages.shellFor with hoogle enabled fails with null dependencies #82245 (comment) and send a PR fixing it to Nixpkgs.

Everyone would probably appreciate if you did 4., but that is probably the most time-consuming solution in this list :-)

[jul1u5]

Thanks for the quick response!

2. Yes, adding any dependency makes it work. Thanks, I haven't realised this.
3. I tried developPackage, but enabling withHoogle alone didn't seem to do anything, hoogle was not in the shell environment. I enabled returnShellEnv but the original error still occurs (value is null). Here is what I tried:

haskellPackages.developPackage {
  name = "project-name";
  root = ./.;

  returnShellEnv = true;
  withHoogle = true;
}

4. I will try to investigate, but I am not that familiar with nixpkgs codebase.

[cdepillabout]

Also see #108100 which may have some additional information about this bug.

[expipiplus1]

Although this doesn't tackle the root cause of a null package sneaking in (via executableHaskellDepends), it does effectively treat the symptom by just ignoring any null packages.

diff --git a/pkgs/development/haskell-modules/hoogle.nix b/pkgs/development/haskell-modules/hoogle.nix
index cfa06b045de..fb2f893943c 100644
--- a/pkgs/development/haskell-modules/hoogle.nix
+++ b/pkgs/development/haskell-modules/hoogle.nix
@@ -63,10 +63,11 @@ buildPackages.stdenv.mkDerivation {
   passAsFile = ["buildCommand"];

   buildCommand = ''
-    ${lib.optionalString (packages != [] -> docPackages == [])
+    ${let packages' = lib.filter (p: p != null) packages;
+      in lib.optionalString (packages' != [] -> docPackages == [])
        ("echo WARNING: localHoogle package list empty, even though"
        + " the following were specified: "
-       + lib.concatMapStringsSep ", " (p: p.name) packages)}
+       + lib.concatMapStringsSep ", " (p: p.name) packages')}
     mkdir -p $out/share/doc/hoogle

     echo importing builtin packages