Couldn't build hmatrix due to a curl build error

[rihardsk]

Tried to build hamtrix with

NIX_PATH=nixpkgs=nixpkgs nix-build --no-link survey/default.nix -A haskellPackages.hmatrix

and got a build error for curl. Here's the last bit of the build output:

/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_ucs4_to_utf8'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_os_mutex_init'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_null_create_val'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_get_tid'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_path_split'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_object_count'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_array_add'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_string_create'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_string_unbase64'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_utf8_to_utf16le'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_open_plugin_dirs'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_once'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_utf8_to_ucs4'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_key_delete'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_get_space'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_free_filenames'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `add_key'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_set_error_info_callout_fn'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_open_plugin'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_vset_error'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_init_dynamic'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_hashtab_get'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_get_keyring_ID'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_encode'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_key_register'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_free'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_array_get'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_getspecific'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_release'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_object_create'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_unlink'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_gmt_mktime'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_hashtab_create'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_add'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_object_set'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_number_create'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_array_create'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_get_plugin_dir_data'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_clear'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_get_error'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_os_mutex_lock'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_path_join'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_number_value'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_add_fmt'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_get_persistent'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_get_plugin_func'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_getnameinfo'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_utf8_mintab'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_os_mutex_unlock'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_set_timeout'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_string_create_len'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_add_len'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_init_dynamic_zap'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_set_error'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_dir_filenames'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_bcmp'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_utf16le_to_utf8'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_free_plugin_dir_data'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_read_alloc'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_close_plugin'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_clear_error'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_getaddrinfo'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_string_utf8'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_describe_alloc'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_hashtab_add'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_close_plugin_dirs'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_os_mutex_destroy'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_buf_status'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_freeaddrinfo'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_link'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_decode'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_bool_value'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_setspecific'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_free_error'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `keyctl_search'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_object_get'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `krb5int_utf8_lentab'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_array_fmt'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_hashtab_remove'
/nix/store/ljx5sayk6w0bpd1ahcf0rm5fj55cqafd-binutils-2.31.1/bin/ld: ../lib/.libs/libcurl.so: undefined reference to `k5_json_array_length'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:923: curl] Error 1
make[2]: Leaving directory '/build/curl-7.64.1/src'
make[1]: *** [Makefile:1967: all-recursive] Error 1
make[1]: Leaving directory '/build/curl-7.64.1/src'
make: *** [Makefile:923: all-recursive] Error 1
builder for '/nix/store/rlj4m1xj45wrispidf7719gpf9j2gp4w-curl-7.64.1.drv' failed with exit code 2
building '/nix/store/5my8sj5z1ah7nzx06518n25qg4c0f5c3-expand-response-params.drv'...
cannot build derivation '/nix/store/07s2vw3jd0fclh9d7i55axq9326fnmhn-hmatrix-Allow-disabling-random_r-usage-manually.patch.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yyg8qq8dh3b7129zdn7ya8nzs00a1ym5-hmatrix-0.19.0.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/yyg8qq8dh3b7129zdn7ya8nzs00a1ym5-hmatrix-0.19.0.0.drv' failed

[nh2]

Is this on latest static-haskell-nix master, commit eb48d1e, and with the nixpkgs submodule updated appropriately with git submodule update --init --recursive?

This kind of failure looks exactly like something I'm currently working on (updating to latest nixpkgs master), so I suspect you're doing something similar.

[rihardsk]

Yes I'm on eb48d1e and I've updated the submodule.

I suspected that it might be related to PR #45. Is that the case?

[nh2]

I'm not sure what's going on yet but I found some things I don't understand in NixOS/nixpkgs#66598, maybe this one is similar.

[nh2]

OK, this is indeed the same problem as in NixOS/nixpkgs#66598, a fixed-output problem:

The way I overrode curl, it wouldn't compile when pulled in via fetchpatch (see points in #6 (comment)), but because hmatrix-Allow-disabling-random_r-usage-manually.patch is a fixed-output derivation, it didn't try to build that curl. Only when one doesn't have downloaded hmatrix-Allow-disabling-random_r-usage-manually.patch at all yet (I had, you hadn't) did this issue surface.

I reproduced it on a from-scratch throwaway AWS NixOS machine as mentioned in NixOS/nixpkgs#66598, and could reproduce.

So indeed it's #45 as you said, and should be fixed with my next update to nixpkgs master that I'll hopefully finish soon (progress in #6 (comment)).

[nh2]

Yep, my test shows that this is fixed with commit 4f57531 (currently on branch nixpkgs-master-2019-08-10-issue-66490-zlib-cleanup), can you try that (also needs a submodules update)?

[rihardsk]

Yup, that helped, hmatrix now builds successfully. Thank you!

And more generally - thank you for this repo. It feels almost magical to be able to do these things :)

[nh2]

And more generally - thank you for this repo. It feels almost magical to be able to do these things :)

Just restoring some normality in the world; in theory and originally, static linking was the default -- as it is much simpler than the fancy hack we call dynamic linking :)