julia 1.6.1 (new formula) #76527

carlocab · 2021-05-03T19:01:01Z

Have you followed the guidelines for contributing?
Have you checked that there aren't other open pull requests for the same formula update/change?
Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
Does your build pass brew audit --strict <formula> (after doing brew install <formula>)?

~~This fails the audit because of the curl and openlibm dependencies. Julia seems to need a libcurl and libopenlibm that live in the file system, but those don't exist on Big Sur.~~

Note that this will also vendor its own libLLVM and utf8proc. We can use brewed llvm and utf8proc, but this requires patches (already upstreamed). We can therefore do one of three things:

Add the formula as-is, and then add dependencies on llvm and utf8proc when those patches land in a stable release. (Might be a while though.)
Use upstreamed patches to allow julia to build with brewed llvm and utf8proc.
Wait until the necessary patches are in a stable release and add the formula then.

Last proposed in Homebrew/legacy-homebrew#11482.

carlocab · 2021-05-03T19:21:51Z

That said, I'm not convinced that openlibm is provided by macOS. These are the contents of openlibm's lib directory:

Permissions Size User     Date Modified Name
.rw-r--r--  181k carlocab 27 Apr 14:38  libopenlibm.3.0.dylib
lrwxr-xr-x    21 carlocab 17 Feb 17:04  libopenlibm.3.dylib -> libopenlibm.3.0.dylib
.r--r--r--  390k carlocab 17 Feb 17:04  libopenlibm.a
lrwxr-xr-x    21 carlocab 17 Feb 17:04  libopenlibm.dylib -> libopenlibm.3.0.dylib
drwxr-xr-x     - carlocab 27 Apr 14:38  pkgconfig

If you try to dlopen a libopenlibm.dylib, you get nothing.

Bo98 · 2021-05-03T19:51:07Z

Julia seems to need a libcurl and libopenlibm that live in the file system, but those don't exist on Big Sur.

What error do you get when you try use system libcurl?

but this requires patches (already upstreamed)

Can you link the patches?

carlocab · 2021-05-03T19:57:03Z

What error do you get when you try use system libcurl?

System library symlink failure: Unable to locate libcurl.dylib on your system!
make[2]: *** [/private/tmp/julia-20210503-8184-m1lldo/julia-1.6.1/usr/lib/julia/libcurl.dylib] Error 1

https://github.com/carlocab/homebrew-personal/pull/78/checks?check_run_id=2493787747#step:8:574

It wants its library dependencies to live in lib/"julia". If you want it to use system versions of those libraries, then it will create symlinks to system libraries, but it has no symlink to create when you want it to use system libcurl.

Can you link the patches?

For utf8proc:

JuliaLang/julia@ba653ec

For llvm:

JuliaLang/julia@8675648

The llvm patch has not yet been merged, but I have an open PR for it: JuliaLang/julia#40680

Bo98 · 2021-05-03T20:17:33Z

If you want it to use system versions of those libraries, then it will create symlinks to system libraries

Considering an unversioned libcurl.dylib does exist in the default dyld search path, I wonder what would happen if this symlink just wasn't created.

For llvm:

Ah yes, this all links back to the ABI versioning thing. llvm-config actually expects libLLVM-11.dylib to exist!

carlocab · 2021-05-03T20:20:49Z

Considering an unversioned libcurl.dylib does exist in the default dyld search path, I wonder what would happen if this symlink just wasn't created.

Bad things, because it then points all library searches to lib/"julia" instead of relying on the dynamic linker to find them.

Bo98 · 2021-05-03T20:21:11Z

Formula/julia.rb

+      prefix=#{prefix}
+      USE_SYSTEM_CSL=1
+      USE_SYSTEM_PCRE=1
+      USE_SYSTEM_OPENLIBM=1


What's the benefit of using openlibm over the system libm (USE_SYSTEM_LIBM)?

openlibm is optimised for Julia (and vice-versa). openlibm is actually maintained by Julia maintainers. I don't have specific benchmarks, though.

It also looks like Julia relies on some symbols that aren't necessarily in a given libm implementation: JuliaLang/julia#26434 (comment)

It seems like it'll also download its own at some point if you don't give it an openlibm: JuliaLang/julia#26434 (comment)

carlocab · 2021-05-03T20:38:41Z

Considering an unversioned libcurl.dylib does exist in the default dyld search path, I wonder what would happen if this symlink just wasn't created.

Bad things, because it then points all library searches to lib/"julia" instead of relying on the dynamic linker to find them.

That said, it seems to manage just fine with zlib. I could try to patch out the libcurl symlinking and see if it breaks anything.

carlocab · 2021-05-04T11:08:53Z

Ok, this should pass the audit now. Do we want this to use brewed llvm and utf8proc? The patches will make it fail the audit again though.

Or an alternative fix in Homebrew/core would be to create a libLLVM-12.dylib symlink to libLLVM.dylib in the llvm formula, which would allow julia to use brewed llvm without any patching.

carlocab · 2021-05-04T17:45:33Z

#76646 will also allow use of brewed llvm.

`llvm-config` relies on a versioned `libLLVM` (e.g. `libLLVM-12`), but their build system does not install one for some reason. With the symlink: ❯ llvm-config --libs -lLLVM-12 Without: ❯ llvm-config --libs -lLLVMWindowsManifest -lLLVMXRay -lLLVMLibDriver -lLLVMDlltoolDriver ... -lLLVMRemarks -lLLVMBitstreamReader -lLLVMBinaryFormat -lLLVMTableGen -lLLVMSupport -lLLVMDemangle This deviates from the intended behaviour, as seen on Linux, where `llvm-config --libs` returns the output shown above with the symlink. Attempting to link against the libraries shown by `llvm-config` without this fix is also leads to build failures due to missing symbols. This will also allow Julia to use brewed `llvm` instead of a vendored one. (Homebrew#76527) Closes Homebrew#76798.

`llvm-config` relies on a versioned `libLLVM` (e.g. `libLLVM-12`), but their build system does not install one for some reason. With the symlink: ❯ llvm-config --libs -lLLVM-12 Without: ❯ llvm-config --libs -lLLVMWindowsManifest -lLLVMXRay -lLLVMLibDriver -lLLVMDlltoolDriver ... -lLLVMRemarks -lLLVMBitstreamReader -lLLVMBinaryFormat -lLLVMTableGen -lLLVMSupport -lLLVMDemangle This deviates from the intended behaviour, as seen on Linux, where `llvm-config --libs` returns the output shown above with the symlink. Attempting to link against the libraries shown by `llvm-config` without this fix is also leads to build failures due to missing symbols. This will also allow Julia to use brewed `llvm` instead of a vendored one. (#76527) Closes #76798. Closes #76646. Signed-off-by: Mike McQuaid <mike@mikemcquaid.com> Signed-off-by: BrewTestBot <1589480+BrewTestBot@users.noreply.github.com>

carlocab · 2021-05-15T07:58:57Z

Switched this back to brewed curl. Encountered some libcurl-related errors locally with system curl.

BrewTestBot · 2021-05-15T08:04:54Z

@carlocab has triggered a merge.

BrewTestBot added new formula PR adds a new formula to Homebrew/homebrew-core perl Perl use is a significant feature of the PR or issue labels May 3, 2021

carlocab force-pushed the julia branch from 58289da to 0dc66bd Compare May 3, 2021 19:14

Bo98 reviewed May 3, 2021

View reviewed changes

carlocab force-pushed the julia branch 4 times, most recently from 599a59d to 90d6639 Compare May 4, 2021 11:06

carlocab mentioned this pull request May 4, 2021

llvm: fix llvm-config #76646

Closed

5 tasks

carlocab force-pushed the julia branch 4 times, most recently from 8faebe9 to b110e72 Compare May 7, 2021 14:29

carlocab force-pushed the julia branch 4 times, most recently from 4216201 to f27b396 Compare May 11, 2021 01:10

carlocab force-pushed the julia branch from f27b396 to 64e8e13 Compare May 14, 2021 12:26

carlocab force-pushed the julia branch 6 times, most recently from 3c7c835 to f26d4b8 Compare May 15, 2021 02:40

julia 1.6.1 (new formula)

fd22648

carlocab force-pushed the julia branch from f26d4b8 to fd22648 Compare May 15, 2021 06:31

BrewTestBot closed this in 96fca6d May 15, 2021

carlocab deleted the julia branch May 15, 2021 08:46

github-actions bot added the outdated PR was locked due to age label Jun 15, 2021

github-actions bot locked as resolved and limited conversation to collaborators Jun 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

julia 1.6.1 (new formula) #76527

julia 1.6.1 (new formula) #76527

carlocab commented May 3, 2021 •

edited

Loading

carlocab commented May 3, 2021

Bo98 commented May 3, 2021

carlocab commented May 3, 2021 •

edited

Loading

Bo98 commented May 3, 2021

carlocab commented May 3, 2021

Bo98 May 3, 2021

carlocab May 3, 2021

carlocab May 3, 2021 •

edited

Loading

carlocab commented May 3, 2021

carlocab commented May 4, 2021 •

edited

Loading

carlocab commented May 4, 2021

carlocab commented May 15, 2021

BrewTestBot commented May 15, 2021

julia 1.6.1 (new formula) #76527

julia 1.6.1 (new formula) #76527

Conversation

carlocab commented May 3, 2021 • edited Loading

carlocab commented May 3, 2021

Bo98 commented May 3, 2021

carlocab commented May 3, 2021 • edited Loading

Bo98 commented May 3, 2021

carlocab commented May 3, 2021

Bo98 May 3, 2021

Choose a reason for hiding this comment

carlocab May 3, 2021

Choose a reason for hiding this comment

carlocab May 3, 2021 • edited Loading

Choose a reason for hiding this comment

carlocab commented May 3, 2021

carlocab commented May 4, 2021 • edited Loading

carlocab commented May 4, 2021

carlocab commented May 15, 2021

BrewTestBot commented May 15, 2021

carlocab commented May 3, 2021 •

edited

Loading

carlocab commented May 3, 2021 •

edited

Loading

carlocab May 3, 2021 •

edited

Loading

carlocab commented May 4, 2021 •

edited

Loading