Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ocamlPackages: init a bunch of libraries for the MirageOS xen target #118066

Merged
merged 4 commits into from
Apr 25, 2021
Merged

ocamlPackages: init a bunch of libraries for the MirageOS xen target #118066

merged 4 commits into from
Apr 25, 2021

Conversation

sternenseemann
Copy link
Member

Motivation for this change

#23955

cc @Ericson2314 as well, I'd love to hear your thoughts on the pkg-config specific changes to the findlib setup hook.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@sternenseemann sternenseemann force-pushed the mirage-xen branch 2 times, most recently from 518d02f to c2344a4 Compare March 30, 2021 13:34
@sternenseemann
Copy link
Member Author

@ofborg build ocamlPackages.mirage-net-xen ocamlPackages.mirage-bootvar-xen

@r-rmcgibbo
Copy link

r-rmcgibbo commented Mar 30, 2021 •
edited
Loading

Result of nixpkgs-review pr 118066 at c2344a44 run on x86_64-linux 1

1 package marked as broken and skipped:
  • hhvm
53 packages skipped due to time constraints:
  • abella
  • beluga
  • coccinelle
  • compcert
  • coqPackages.contribs.zorns-lemma
  • coq_8_10
  • coq_8_12
  • coq_8_13
  • coq_8_5
  • coq_8_6
  • ...
36 packages built successfully:
  • acgtk
  • alt-ergo
  • coq_8_11
  • dot-merlin-reader
  • dune_1
  • dune_2
  • eff
  • google-drive-ocamlfuse
  • haxe (haxe_4_2)
  • hxcpp (haxePackages.hxcpp)
  • haxePackages.hxcs
  • haxePackages.hxjava
  • haxePackages.hxnodejs_4
  • hevea
  • libguestfs
  • liquidsoap
  • obelisk
  • ocaml-crunch
  • ocaml-top
  • ocamlformat (ocamlformat_0_17_0)
  • ocamlformat_0_11_0
  • ocamlformat_0_12
  • ocamlformat_0_13_0
  • ocamlformat_0_14_0
  • ocamlformat_0_14_1
  • ocamlformat_0_14_2
  • ocamlformat_0_14_3
  • ocamlformat_0_15_0
  • ocamlformat_0_15_1
  • ocamlformat_0_16_0
  • ocsigen-i18n
  • opaline
  • opam-installer
  • reason
  • virt-top
  • why3

Result of nixpkgs-review pr 118066 at c2344a44 run on aarch64-linux 1

18 packages marked as broken and skipped:
  • coccinelle
  • coq_8_5
  • coq_8_6
  • cubicle
  • haxe_3_2
  • haxe_3_4
  • libvmi
  • mldonkey
  • monotoneViz
  • obliv-c
  • ...
7 packages failed to build:
26 packages skipped due to time constraints:
  • abella
  • coqPackages.contribs.zorns-lemma
  • coq_8_10
  • coq_8_11
  • coq_8_12
  • coq_8_13
  • coq_8_7
  • coq_8_8
  • coq_8_9
  • flow
  • ...
33 packages built successfully:
  • acgtk
  • alt-ergo
  • beluga
  • dot-merlin-reader
  • dune-release
  • dune_1
  • dune_2
  • eff
  • fstar
  • google-drive-ocamlfuse
  • haxe (haxe_4_2)
  • hxcpp (haxePackages.hxcpp)
  • haxePackages.hxcs
  • haxePackages.hxjava
  • haxePackages.hxnodejs_4
  • hevea
  • libguestfs
  • liquidsoap
  • obelisk
  • ocaml-crunch
  • ocaml-top
  • ocamlformat (ocamlformat_0_17_0)
  • ocamlformat_0_14_3
  • ocamlformat_0_15_0
  • ocamlformat_0_15_1
  • ocamlformat_0_16_0
  • opaline
  • opam-installer
  • orpie
  • patdiff
  • reason
  • satysfi
  • virt-top

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement.

Ericson2314
Ericson2314 reviewed Mar 30, 2021
View reviewed changes
pkgs/development/ocaml-modules/mirage-xen/pkg-config.patch Outdated
-prefix=${pcfiledir}/../..
-exec_prefix=${prefix}
-libdir=${exec_prefix}/lib
+libdir=${pcfiledir}/../
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm normally we keep prefix and friends? And prefix is set to $out by default as a best effort.

I went into the rabbit hole a bit, looking at ocaml/dune#1253 and linked issues like:

It looks like perhaps the pieces are there and those issues can be resolved?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, I completely forgot about that. I quickfixed this on a local branch and was meaning to follow up. I'll see what this is all about :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue boils down to the following: The pkg config file in the repository has been manually created and assumes the default relation between dune's prefix and libdir (which would be "${prefix}/lib" for the latter). However for ocamlPackages we install libraries into a version specific site-lib, so that doesn't work out.

This is probably fixable in mirage-xen's build system, but they probably have little interest in fixing that before the upcoming rework.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be better now!

@Ericson2314
Copy link
Member

mirage/mirage#969 mirage/mirage#1195 also make me begin to understand the pkg-config might duplicate functionality for the OCaml -> OCaml case (vs OCaml -> C or C -> OCaml case), and why they thus might want to get rid of it.

@sternenseemann
Copy link
Member Author

mirage/mirage#969 mirage/mirage#1195 also make me begin to understand the pkg-config might duplicate functionality for the OCaml -> OCaml case (vs OCaml -> C or C -> OCaml case), and why they thus might want to get rid of it.

Yep, I talked to them on IRC and basically they want to use dune for everything in MirageOS 4 including stub libs for the freestanding stuff. That should be no problem as dune is pretty well-behaved in my experience. Apparently the use of pkg-config was only introduced because dune used to automatically pass some “default” CFLAGS including -fPIC, so they pkg-config was introduced as an alternative way of passing CFLAGS around

@ehmry ehmry added the 6.topic: exotic Exotic hardware or software platform label Apr 15, 2021
@sternenseemann sternenseemann requested a review from vbgl April 20, 2021 11:46
@vbgl
Copy link
Contributor

vbgl commented Apr 21, 2021

I don’t understand why the PKG_CONFIG_PATH environment variable is set within the findlib hook: the two things look unrelated.

@sternenseemann
Copy link
Member Author

I don’t understand why the PKG_CONFIG_PATH environment variable is set within the findlib hook: the two things look unrelated.

The location of the .pc files is OCaml-specific unfortunately, but I don't think we'll need this in the long term (since MirageOS 4 promises to get rid of pkg-config), so I think I'll move it into an ocamlPkgConfigHook package, so we don't have to have it everywhere (and removing it doesn't trigger a huge rebuild).

@sternenseemann
ocamlPackages.shared-memory-ring(-lwt): init at 3.1.0
b5f7ed7
@sternenseemann
ocamlPackages.mirage-xen: init at 6.0.0
980a2be 
Unfortunately this requires a bit of trickery with pkg-config to get to
work. The root issue is that the mirage-xen assumes that we use the
default libdir of dune ($out/lib) whereas we install to an OCaml
version-specific site-lib directory. Thus the manually created
pkg-config file makes wrong assumptions (which warrants a patch) and the
.pc file is installed to the wrong location (which is fixed with a mv
invocation).
@sternenseemann
ocamlPackages.{mirage-net-xen,netchannel}: init at 2.0.0
2fe8239
@sternenseemann
ocamlPackages.mirage-bootvar-xen: init at 0.8.0
4f7a661
@sternenseemann
Copy link
Member Author

I believe, I've cleaned this up now: We don't need a new setup hook or the likes, we can just install the .pc file to its standard location in $out/lib/pkgconfig. This is probably the intention of mirage-xen anyways: From the .pc file I inferred that it assumes that dune's default libdir is used (which is $prefix/lib).

@sternenseemann sternenseemann requested a review from ehmry April 25, 2021 11:18
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux and removed 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Apr 25, 2021
@Ericson2314
Copy link
Member

Was waiting for you to say you think it's ready. LGTM!

@Ericson2314 Ericson2314 merged commit a86418a into NixOS:master Apr 25, 2021
@sternenseemann sternenseemann deleted the mirage-xen branch April 25, 2021 14:50
@sternenseemann sternenseemann restored the mirage-xen branch July 24, 2021 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Ericson2314 Ericson2314 Ericson2314 left review comments

@vbgl vbgl Awaiting requested review from vbgl

@ehmry ehmry Awaiting requested review from ehmry

Assignees
No one assigned
Labels
6.topic: exotic Exotic hardware or software platform 6.topic: ocaml 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sternenseemann @r-rmcgibbo @Ericson2314 @vbgl @ehmry