support Bazel 7 cc_shared_library

[malt3]

This change is based on #2180 and adds support for cc_shared_library. This is needed for macOS and Bazel 7+, since cc_library no longer emits dynamic libraries on macOS (feature removal, linker flag removal).

This implementation differs from the one in cc_binary in some ways: cc_binary uses a separate attribute dynamic_deps for CcSharedLibraryInfo, while haskell_library / haskell_binary use deps for both CcInfo and CcSharedLibraryInfo.
We also do not implement the same deduplication logic as cc_binary where we filter CcInfo deps which are already provided by a CcSharedLibraryInfo. This may lead to duplicate symbols. A more correct implementation would need to use aspects and might require the use of a separate attribute (dynamic_deps).

Open issues:

• Fix //tests/recompilation:recompilation_test_nixpkgs_bazel_7, which depends on the macOS Foundation framework. Probably broken by the removal of --undefined dynamic_lookup was broken by -fobjc-link-runtime but only worked accidentally (needed nixpkgs cc_toolchain).
• Make tests work on Bazel 6 and 7 (use cc_shared_library only conditionally and make //tests/repl-targets:hs_lib_repl_test_nixpkgs_bazel_6 work again)

Supersedes #2107 and #2180

[aherrmann]

We can probably refactor the testing code so that cc_shared_library is used iff we are on Bazel 7 or greater.

Perhaps bazel_features can help. It doesn't seem to list a feature for cc_shared_library, yet. But, we could add one.

A more correct implementation would need to use aspects and might require the use of a separate attribute (dynamic_deps).

I think that's true. Nonetheless, I think this is a good step forward.

[malt3]

Adding back the linker flags --undefined dynamic_lookup did not fix the missing Foundation framework at link time. I'll try to investigate why we depend on the framework in the first place and what other things might have changed.

Here is the remaining diff between the two Bazel versions for the failing target:

--- /tmp/linker_flags_bazel_6   2024-08-29 19:06:26.401508228 +0200
+++ /tmp/linker_flags_bazel_7   2024-08-29 19:06:34.941537070 +0200
@@ -1,5 +1,6 @@
--optl-undefined
--optldynamic_lookup
+-optl-mmacosx-version-min=10.11
+-optl-no-canonical-prefixes
+-optl-fobjc-link-runtime
 -optl-headerpad_max_install_names
 -optl-lc++
 -optl-lm

Those might be responsible..

The latter two come from here, as far as I can tell: https://cs.opensource.google/bazel/bazel/+/refs/tags/7.3.1:tools/cpp/unix_cc_toolchain_config.bzl;l=1412-1427

[avdv]

Adding back the linker flags --undefined dynamic_lookup did not fix the missing Foundation framework at link time.

I think that issue occurs when using an automatically configured local cc toolchain which is picking up a C compiler from inside a nix shell. The nix C compiler requires some environment variables to be set in order to locate the frameworks and these are not available inside of the Bazel actions (that is why we write these flags to files in the nix-support directory for the cc toolchain from rules_nixpkgs: https://github.com/tweag/rules_nixpkgs/blob/a11818b2173100ab8122712ae3694b711f956cbd/toolchains/cc/cc.nix#L30C1-L50C7).

The failing test uses haskell_register_ghc_nixpkgs but does not configure a nixpkgs cc toolchain. That is probably not something that should be supported, but it worked with Bazel 6 / older rules_cc versions.

[malt3]

The failing test uses haskell_register_ghc_nixpkgs but does not configure a nixpkgs cc toolchain. That is probably not something that should be supported, but it worked with Bazel 6 / older rules_cc versions.

So a better fix would be to add the nixpkgs cc toolchain to that WORKSPACE, correct?

[avdv]

So a better fix would be to add the nixpkgs cc toolchain to that WORKSPACE, correct?

Yes. 🤞

[avdv]

Nice work, thank you! ❤️