v0.9.0 breaks MacOS

[bhanukumar314]

v0.9.0 seems to have broken the library's functionality on MacOS (at least on Monterey, where I tested it). The library fails to find the Apple OpenCL platform. The same behavior is seen both on the Apple Silicon build of Julia (tested on an M1 Pro 16 inch MacBook Pro) as well as on the Intel x86-64 build of Julia (tested on an i9 16 inch MacBook Pro).

On v0.9.0:
julia> cl.platforms()
ERROR: CLError(code=-1001, CL_PLATFORM_NOT_FOUND_KHR)
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/OpenCL/BTcrM/src/macros.jl:6 [inlined]
[2] platforms()
@ OpenCL.cl ~/.julia/packages/OpenCL/BTcrM/src/platform.jl:43
[3] top-level scope
@ none:1

On v0.8.1
julia> cl.platforms()
1-element Vector{OpenCL.cl.Platform}:
OpenCL.Platform('Apple' @0x000000007fff0000)

[juliohm]

@bhanukumar314 can you please try to install the underlying OpenCL_jll binary:

https://github.com/JuliaBinaryWrappers/OpenCL_jll.jl

You can ] add OpenCL_jll and then load the package. The package should define a OpenCL_jll.libopencl variable pointing to the library installed.

[bhanukumar314]

No change even with OpenCL_jll loaded.

(@v1.8) pkg> status OpenCL
Status ~/.julia/environments/v1.8/Project.toml
[08131aa3] OpenCL v0.9.0

(@v1.8) pkg> status OpenCL_jll
Status ~/.julia/environments/v1.8/Project.toml
[6cb37087] OpenCL_jll v2022.9.23+1

julia> using OpenCL

julia> using OpenCL_jll

julia> cl.platforms()
ERROR: CLError(code=-1001, CL_PLATFORM_NOT_FOUND_KHR)
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/OpenCL/BTcrM/src/macros.jl:6 [inlined]
[2] platforms()
@ OpenCL.cl ~/.julia/packages/OpenCL/BTcrM/src/platform.jl:43
[3] top-level scope
@ none:1

[juliohm]

That is not what I asked in the comment. Can you please try to access OpenCL_jll.libopencl?

[bhanukumar314]

I am able to access that variable, as well as the other related ones. It displays

julia> OpenCL_jll.libopencl
"/Users/bkumar/.julia/artifacts/11dff7de7b89f38e0ae228668f3e38a76ba86b97/lib/libOpenCL.1.2.dylib"
julia> OpenCL_jll.libopencl_handle
Ptr{Nothing} @0x00000002094d2160
julia> OpenCL_jll.libopencl_path
"/Users/bkumar/.julia/artifacts/11dff7de7b89f38e0ae228668f3e38a76ba86b97/lib/libOpenCL.1.2.dylib"

[juliohm]

Yes, so OpenCL.jl uses OpenCL_jll as the underlying complied C library. The good news is that it is properly installed. Now, can you please confirm that you have a OpenCL driver installed in your machine? I know that MacOS is particularly problematic with OpenCL support.

Try to install the PoCL driver for example, or a more specific one for your own hardware. On Ubuntu for example, you have intel-opencl beignet-opencl, etc as possible drivers to recognize the hardware.

[juliohm]

Also, take a look at this post on Discourse: https://discourse.julialang.org/t/opencl-jl-v0-9/88437

[bhanukumar314]

PoCL installs successfully; also, I had installed clinfo from homebrew and it successfully recognizes the OpenCL platform on my machine. So it's a problem specifically with OpenCL.jl v0.9.0 not being able to recognize it. Here is the clinfo output from the M1 Pro machine.

bkumar ~ % clinfo
Number of platforms 1
Platform Name Apple
Platform Vendor Apple
Platform Version OpenCL 1.2 (Aug 8 2022 21:29:55)
Platform Profile FULL_PROFILE
Platform Extensions cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event

Platform Name Apple
Number of devices 1
Device Name Apple M1 Pro
Device Vendor Apple
Device Vendor ID 0x1027f00
Device Version OpenCL 1.2
Driver Version 1.2 1.0
Device OpenCL C Version OpenCL C 1.2
Device Type GPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 16
Max clock frequency 1000MHz
Device Partition (core)
Max number of sub-devices 0
Supported partition types None
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 256x256x256
Max work group size 256
Preferred work group size multiple (kernel) 32
Preferred / native vector sizes
char 1 / 1
short 1 / 1
int 1 / 1
long 1 / 1
half 0 / 0 (n/a)
float 1 / 1
double 1 / 1 (n/a)
Half-precision Floating-point support (n/a)
Single-precision Floating-point support (core)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (n/a)
Address bits 64, Little-Endian
Global memory size 22906503168 (21.33GiB)
Error Correction support No
Max memory allocation 4294967296 (4GiB)
Unified memory for Host and Device Yes
Minimum alignment for any data type 1 bytes
Alignment of base address 32768 bits (4096 bytes)
Global Memory cache type None
Image support Yes
Max number of samplers per kernel 32
Max size for 1D images from buffer 268435456 pixels
Max 1D or 2D image array size 2048 images
Base address alignment for 2D image buffers 256 bytes
Pitch alignment for 2D image buffers 256 pixels
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 8
Local memory type Local
Local memory size 32768 (32KiB)
Max number of constant args 31
Max constant buffer size 1073741824 (1024MiB)
Max size of kernel argument 4096 (4KiB)
Queue properties
Out-of-order execution No
Profiling Yes
Prefer user sync for interop Yes
Profiling timer resolution 1000ns
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
printf() buffer size 1048576 (1024KiB)
Built-in kernels (n/a)
Device Extensions cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_image2d_from_buffer cl_khr_depth_images

NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Apple
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [P0]
clCreateContext(NULL, ...) [default] Success [P0]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name Apple
Device Name Apple M1 Pro
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name Apple
Device Name Apple M1 Pro
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) Invalid device type for platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name Apple
Device Name Apple M1 Pro

[juliohm]

The previous version had a path hard-coded in the case of MacOS as you can see here: https://github.com/JuliaGPU/OpenCL.jl/pull/196/files

The idea of using OpenCL_jll is that we install a local library as an artifact and load it instead. Maybe we need to do something else in MacOS to recognize the local library instead of the global one? Can you try to uninstall the global OpenCL or similar experiments?

[bhanukumar314]

Uninstalling the global OpenCL probably isn’t a good idea, since it’s a System library (which potentially other softwares depend on) and these are my main systems which I need to keep functional. There isn’t any straightforward way of uninstalling it anyways on MacOS. Any other ideas? As far as I can tell OpenCL.jl isn’t recognizing any library at all, neither local nor global.

[juliohm]

You need to make sure that the local OpenCL library that is being installed by OpenCl_jll can find the drivers installed on the system. Can you list the drivers you have currently installed? I'm typing from my phone right now... Em seg., 17 de out. de 2022 22:09, Bhanu Kumar ***@***.***> escreveu:

…

Uninstalling the global OpenCL probably isn’t a good idea, since it’s a System library (which potentially other softwares depend on) and these are my main systems which I need to keep functional. There isn’t any straightforward way of uninstalling it anyways on MacOS. Any other ideas? As far as I can tell OpenCL.jl isn’t recognizing any library at all, neither local nor global. — Reply to this email directly, view it on GitHub <#205 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZQW3JUIRK7JZLHMFIOXRLWDX2FTANCNFSM6AAAAAARHPLDRE> . You are receiving this because you commented.Message ID: <JuliaGPU/OpenCL. ***@***.***>

[bhanukumar314]

The OpenCL driver that Macs use is a custom Apple-developed driver compatible with up to OpenCL 1.2. It is a framework included by default on every Mac; I've copied and pasted below the info about it from the System Information application.

OpenCL:

Version: 4.7
Obtained from: Unknown
Last Modified: 8/24/22, 1:59 AM
Get Info String: 4.7, Copyright 2008-2019 Apple Inc.
Location: /System/Library/Frameworks/OpenCL.framework
Private: No

[juliohm]

Got it. I don't have a MacOS to debug it myself. Do you have suggestions on how to make the local ICD loader library recognize this global pre-installed driver? Maybe other people had similar issues on MacOS in the past?

We are trying to avoid hard-coded paths to pre-installed libraries, and I am assuming that there is a way to make this driver "more visible" to the local library that OpenCL_jll installs.

[bhanukumar314]

So I'm not an OpenCL or Julia development expert myself, but searching around on the internet, my best guess is that this problem is related to the fact that OpenCL is a "library" on Linux and a "framework" on MacOS. OpenCL_jll is generated using BinaryBuilder.jl it seems? BinaryBuilder.jl has different product types for libraries vs frameworks; according to its documentation, frameworks have to be encapsulated as FrameworkProduct types instead of LibraryProduct types. See

https://docs.binarybuilder.org/dev/reference/#BinaryBuilderBase.FrameworkProduct

However, according to the README.md for OpenCL_jll.jl, it only generates a LibraryProduct for libopencl, which thus I assume would not work on MacOS.

[bhanukumar314]

Also, on MacOS, when compiling (for instance with clang), one does not refer to "libopencl", but just "OpenCL"; thus, for instance, one uses the option -framework OpenCL during compilation on MacOS instead of -lopencl during compilation. See for instance

https://stackoverflow.com/questions/49656832/what-is-the-lopencl-on-mac

I have zero experience with BinaryBuilder.jl, but perhaps this is enough information for you to understand a potential solution?

[juliohm]

@giordano can you give a hand here?

[juliohm]

@bhanukumar314 I think one way forward is try to generate the PoCL_jll driver with BinaryBuilder.jl and add it here as a dependency so that the local OpenCL_jll installation can find a safe driver always independently of MaCOS, Windows, Linux etc.

[bhanukumar314]

Of course adding PoCL would not hurt, but PoCL as far as I’m aware doesn’t support most AMD GPUs, which is what all the newer Intel Macs have. I also don’t know if it works with Apple Silicon or not; it would be best if the FrameworkProduct for Apple’s OpenCL implementation can just be added to OpenCL_jll.

[juliohm]

If you can help with this framework detail @bhanukumar314 that would be super helpful. I am not a MacOS users myself and am quite busy with other activities currently to stop and try to fix the issue, unfortunately.

[dbl001]

The OpenCL interface in Mathematica 13.2 which uses Apple's OpenCL framework works on my 2021 iMac 27" running MacOS Ventura 13.2 with an AMD Radeon Pro 5700 XT

[maleadt]

What exactly is the incompatibility between the Khronos ICD loader from OpenCL_jll and macOS' libOpenCL.jl? If it's just a missing ICD specification, that can be overridden with environment variables: OCL_ICD_FILENAMES or OCL_ICD_VENDORS. In fact, I recently reworked OpenCL_jll to automatically set these based on a global array, e.g., as done here by pocl_jll: https://github.com/JuliaBinaryWrappers/pocl_jll.jl/blob/6c8509126286a8dcbce574a96871aa633ac3bda1/src/wrappers/aarch64-apple-darwin-llvm_version%2B14.asserts.jl#L28

Could you try if adding the path to macOS' libOpenCL.dylib (or whatever it's called; I have an M-series processor so can't test) to the OpenCL_jll.drivers array makes OpenCL.jl detect the Apple OpenCL implementation?

[maleadt]

Okay, I found out that even my M3 macbook still has a functioning OpenCL framework, see #221. However, with OpenCL being officially deprecated on macOS since 2018 (10.14), I don't think we should keep this open as an issue. If anybody really needs native macOS support, I'd suggest using an older version of OpenCL.jl, or having a look at the linked PR.