RFC: dynamic linkage of libraries and their inter-dependencies

[ViralBShah]

I have made a number of build related modifications, that will make it easier to integrate into various packaging systems and such:

1. Avoids multiple linking of BLAS and LAPACK into other libraries. The shared BLAS and LAPACK libraries are loaded by others that need them. (eg. ARPACK, SUITESPARSE).
2. Link the various julia binaries to the the libjulia shared library. No LD_LIBRARY_PATH mucking around should be needed. libjulia now lives with all the other libraries in lib/
3. Add Rmath to the default build
4. Move GLPK building to external and enable it in the default build
5. Disable LAPACK build, and instead make OpenBLAS build LAPACK, which replaces some routines with faster ones.
6. Change the way suitesparse is compiled. Now, each individual suitesparse library is compiled into its own shared library, rather than one monolithic libsuitesparse - which is more consistent with how various distributions handle it.
7. Version upgrades for some libraries.
8. Turn on threads in openblas and fftw build, although not yet used by the drivers

[ViralBShah]

This commit addresses parts of #353, #387. This pull request should also address #726 before merging.

[ViralBShah]

Would appreciate if anyone has a better way to do in the Makefile what I am doing with external/install-name-$(OS).sh scripts. They basically are just fixing paths for shared libraries using the install_name_tool command on Darwin.

[nolta]

Sorry Viral, mucked about w/ the arpack dependencies and broke stuff. Hopefully fixed now.

[ViralBShah]

No worries - it was easy enough. I just figured out a way to work around the build issues in arpack 3.1.0, which is why I was updating it. I am done for now - so feel free to muck as necessary. :-)

-viral

On 29-Apr-2012, at 11:13 PM, Mike Nolta wrote:

Sorry Viral, mucked about w/ the arpack dependencies and broke stuff. Hopefully fixed now.

---

Reply to this email directly or view it on GitHub:
#773 (comment)

[StefanKarpinski]

@nolta: 9f13c27 is broken for me whereas 54d99f0 worked.

[nolta]

(oops, butterfingers)

How is it broken?

[StefanKarpinski]

I've accidentally closed pull requests many times — the GitHub button placesment is lousy.

Ah, weird. I tried doing "make testall" again and now it works :-/

[StefanKarpinski]

This patch renames external to deps on this branch: https://gist.github.com/edff3ebd6756f8da3048. However, it requires deleting your entire external directory and rebuilding from scratch. It also makes hard to reasonably switch branches, so I didn't want to just push it without asking people first. Let me know if I can push this change.

[nolta]

Go for it.

[ViralBShah]

I am merging master into local_blas now (all the libuv stuff). Also, the symlinks for lib and include in $JULIAHOME are not achieving anything. I suggest that instead of installing all libraries to $JULIAHOME/deps/root/, we install them in $JULIAHOME/root, outside of deps. Julia binaries, sys.ji, etc. should also be installed inside $JULIAHOME/root, with symlinks in $JULIAHOME. This will make it much easier to create a packages for distribution.

I am going to take a crack at this, and after that we can merge this back into master, so that people have to rebuild only once.

[Keno]

The symlinks right now are runtime related as that is where Julia is looking for the libraries. I agree though that we should change that.

[ViralBShah]

Crap, I did it this time. How do I reopen this?

[ViralBShah]

Crap, I seem to have merged into master.

[Keno]

Just making sure, did all the local_blas stuff actually make it into master, as it says "ViralBShah merged 2 commits into master from local_blas about 8 hours ago" a and I think there were more than 2 commits (I don't even know why github let you merge it as it shouldn't have allowed that until the merge conflicts were fixed)

[StefanKarpinski]

Yeah, I'm not sure exactly how that all happened — I had just gotten home, saw Viral's "Oh crap..." comments, realized we were in the middle of serious breakage and started trying to fix stuff. Is it possible there were actually no merge conflicts? Or maybe Viral did a manual merge and just got the direction wrong? That kind of thing is a bit unintuitive in git.

[StefanKarpinski]

But, yes, if you look at the commit history and state of current affairs, it certainly does look like we got everything.

[ViralBShah]

Yes, I do believe it got everything, since I merged local_blas into master instead of the other way around. Everything also builds and links as intended.

-viral

On 30-Apr-2012, at 9:32 PM, Stefan Karpinski wrote:

But, yes, if you look at the commit history and state of current affairs, it certainly does look like we got everything.

---

Reply to this email directly or view it on GitHub:
#773 (comment)