Add new aports ghc, cabal, and stack #205
Conversation
|
Updates should cover it, in autosquash format, I'll squash them and force push if they pass muster. Should have covered most of the comments, thanks for the reviews @jirutka ! |
|
After looking at that travis build failure, I think it would work if i updated the ghc-bootstrap package to be compiled with -fPIC. Not entirely sure how travis is trying to run things but if you need build logs from an alpine system I can provide them. These do work outside of travis. |
|
I don’t think that it’s just Travis-specific problem, it fails for the same reason even on my machine (x86_64, hardened kernel). |
jirutka
added
the
S-changes-requested
|
More than likely the ghc-bootstrap package needs to force everything in it to fPIC and nopie as well. |
|
I've managed to get this working but broke ghci in the process and debugging why that is broken is leading me to believe this is going to take a lot more time than I budgeted for getting ghc upstreamed. Also chasing down an arm build failure with changed in place. But since I can't easily replicate it I really don't know if i'm just chasing waterfalls to be honest. |
There’s no requirement that every package must work on every supported architecture. I recommend to focus only on x86_64 for now. |
|
Ok, long time no update but finally have some more time to poke about at this. This doesn't address the issues with ghci with armhf, but things seem to compile and work there its just the REPL that has issues. Basically things appear to be functional just not fully usable, which while lamentable I could fix later. But the changes I made had to hit the bootstrap compiler itself first, so we'll need to have that regenerated first for this to work. I included a change in that last commit to testing/ghc-bootstrap/APKBUILD which points to the gcc-bootstrap compilers I built and threw up on my own site. That bit we wouldn't want. I can rip it out but I left it more as a note. Should I make the gcc-bootstrap changes in a different PR than this to keep them separate? They're interrelated but ultimately someone would have to rebuild the gcc-bootstrap apk for things to move forward. Sorry for the late response and no rush on the PR. |
That’s great! 😸
Not necessary, you can keep it in this PR.
You have nothing to apologize, we know that ghc is very hard to port! |
jirutka
added
S-needs-review
|
Ok cool sounds good, so how about I clean up this branch a bit by squashing and condense this down to just:
That way you can just cherry-pick the bootstrap changes and I can validate that everything still builds as expected post changes? |
mitchty
force-pushed
the
ghc
branch
2 times, most recently
from
6181b32 to
f76dd0a
Compare
|
Ok think I have this sorted out after a long time of testing. What I've done to date:
But what it means is that the ghc-bootstrap source tar.xz files will need to be built again and uploaded. I tested they work via: I restricted the ghc-bootstrap process for now to x86_64, arm is still there but just takes forever to test builds. It should still work it did the last time I tried at least but untested so I'm leaving it out of the final build product. And If this can get in place soon, it'll be just in time for 8.0.2. :) |
|
Looks like armhf still works. I'll update the PR tonight with that arch as well. And it looks like ghc 8.0.2 will be released soon as it just got tagged in git. |
|
Ok, everything updated to also work with armhf as well. So whenever anyone gets a chance to run the snapshot function on a docker machine to produce the binaries listed in ghc-bootstrap I can validate that everything still works with those and update the ghc-bootstrap APKBUILD with that and get this upstreamed. Otherwise if desired I can give sha1sums of what I tested against for x86_64 and armhf and put them on an s3 bucket. |
|
Updated the PR to use ghc 8.0.2 and cabal 1.24.0.2 and stack etc.. to build off ghc 8.0.2. |
|
While testing 8.0.2 building stack on arm, I encountered this bug: Which basically means gobs of ram to build via 8.0.2 vs 8.0.1 Roughly 8GiB, I can port the patches from 8.0.2 that I need to make the 8.0.1 port work right on Alpine Linux 3.5 if we want. It should also work fine to bootstrap 8.0.2 in the future if so (I'll need to test/validate this assumption). Regardless I need to update the ghc-bootstrap port either way so just need to know which way to go. Note 8.0.2 works fine, it just has issues with that one module compiling causing huge memory use. |
|
Ok 8.0.1 is a no show, getting some really odd ghc-pkg behavior and it can't find one of its bootstrap packages. So 8.0.2 it is mostly because if needed we can constrain stack to just x86_64 for now. |
The bootstrap port now sets up needed settings at apk build time instead of at docker build time. Should be less brittle for changes.
|
Just did a quick update to the ghc-bootstrap package to remove a line that pointlessly caused the x86_64 bootstrap to use llvm for its stage 0/1 compilers. Reduced time to make the bootstraps by a few hours. |
|
Urgh. This was accidental commit. I will revert and get back on how to do it. |
|
So saw your note from git, can you elaborate further on what you're looking for exactly from the port? Note this version is based on the last input from my prior 2 attempts at getting this upstreamed, having to contend with alpine linux versions changing and other moving targets along with this being a part time thing has meant it takes a fair amount of effort to test this especially with the compile times on arm. Another note, using stage 2 compilers like I do with the Dockerfiles are a largely untested/on your own type of thing in GHC. Outside of the recent rebootstrap project in Debian this port was the first/only user of stage 2 ghc compilers as it is. Presuming I understood the irc comment, you are looking for:
I'm not against doing that but after about 2 years of working on refining this port I'm a bit hesitant to put a ton more work into it if each review stage causes me to invest too much more time with yak shaving. Testing this port is rather time consuming due to it taking so long to produce all these cross compilers. In either case if the above is correct let me know, and if you could point to an example apkbuild that does the above that I can copy^Wmimic/use to understand the cross compilation support in alpine linux that would be helpful. I'll just close this issue out since it doesn't appear it'll get upstreamed as is. |
|
Please don't close it, we're definitely not rejecting this PR! Just give us a little more time please. @fabled, do you already have some feedback what you'd like to improve? |
|
Doesn't it require |
|
@wiz The current dependencies are setup to keep the install size to an absolute extreme minimum. This was requested originally for things relating to docker containers which use ghc to build. Technically all you need is gcc for linking. Though just installing ghc you can get away with running ghci as that interprets things and doesn't compile anything for repl support. Realistically you'd need gcc, llvm (for arm hard requirement), linux-headers, zlib-devel, gmp-devel, likely alpine-sdk, amongst others. But in the interest of keeping the default install size down I pared the dependencies to an absolute minimum. |
|
I've launched Anyway, the GHC build slams a whooping 1.2 GB upon the base image. |
|
ghc-bootstrap and ghc are now applied with minor changes. |
|
I'd argue thats a case of stack setup needing build-base but only tangentially. Adding it to ghc would be fine. And 1.2GiB is pretty normal, the debian install is around 700MiB, with stripping I could probably get this closer to the debian size, but have had fun with stripping the profiled runtime where once done, I can't debug anything via the profiled runtime once stripped. I've punted on fully debugging size reduction until later. The majority of the size is the profiled runtimes' static archives. And changing the cabal build involves a bit of work with this script, it was never a priority before (note stack will need similar involvement) As a note however, the ghc package infrastructure is hardened via (pertains for the stack build mostly): Most of the effort in getting this ported involved far too much time with objdump and other fun commands debugging linker shenanigans between ghc->opt/llc->linker. Fortunately thats mostly a thing of the past thanks to this commit (in 8.0.2 only): |
|
Testing out the cabal package bootstrap.sh changes, should I update this branch or do that in a different pr once tested? |
|
Now that the base work is committed, and ghc works let's close this one. Please submit new PR for further work, so there's no merge conflicts and CI can do test runs. Please submit with arch="x86_64" until the ghc is available on other arches. I hope to do the cross-building feature to ghc aport soon. Thanks! |
|
Sounds good thanks @fabled for all the help! As for the cross building support is there anything I should look at specifically? Or should I just ask in irc how that is expected to be done? Note I can't guarantee how well that will/won't work, but I can speak to using the armhf version of the port for a few years now fine. |
This finishes off my patch series for adding ghc to testing. I validated that the llvm3.7 and gcc-bootstrap ports in testing build all three packages successfully.
This adds the native ghc aport, aka the ghc haskell compiler. It also adds the cabal and stack build tools.
With these in place more packages that use haskell can start to get added in if desired. I have a port of the idris programming language that is waiting in the wings as an initial example of that.
Updated a url and clarified a bit on the ghc-bootstrap ports purpose.