FIX: Build Issue on MacOS

[dmsilev]

I'd like to give Fidimag a try, so I downloaded everything to my Mac and started to build it. Running the install scripts for Sundial and FTTW were fine, compiling using GCC 13 (installed via HomeBrew). But when I then went to build the main code set (using either 'make' or 'make build' from the root folder of the distribution), it got caught in an endless loop with the errors below; this chunk of compiler output repeated over and over again for hours.

Unsurprisingly, loading the system from within Python doesn't work.
Exception Error message
-----------------------
No module named 'fidimag.extensions.common_clib'

     System Info
     -----------------------
     Architecture: arm64
     Platform: macOS-13.5.1-arm64-arm-64bit
     Processor: arm

Any suggestions on where to look for troubleshooting would be appreciated.

Output from "make":

Using CC=gcc-13
Building with 10 threads
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/atomistic/lib/clib.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/common/lib/common_clib.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/common/sundials/cvode.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/micro/lib/baryakhtar/baryakhtar_clib.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/micro/lib/micro_clib.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/common/neb_method/nebm_clib.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/common/dipolar/dipolar.pyx because it changed.
Compiling /Users/dmsilev/mag_solve/fidimag/fidimag/atomistic/fmmlib/fmm.pyx because it changed.
Traceback (most recent call last):
File "", line 1, in
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 129, in _main
prepare(preparation_data)
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 240, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 291, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 291, in run_path
File "", line 98, in _run_module_code
File "", line 88, in _run_code
File "/Users/dmsilev/mag_solve/fidimag/setup.py", line 274, in
ext_modules=cythonize(ext_modules, nthreads=nthreads),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1114, in cythonize
pool = multiprocessing.Pool(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/pool.py", line 215, in init
self._repopulate_pool()
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/pool.py", line 306, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/pool.py", line 329, in _repopulate_pool_static
w.start()
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/context.py", line 288, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 158, in get_preparation_data
_check_not_importing_main()
File "/Users/dmsilev/anaconda3/lib/python3.11/multiprocessing/spawn.py", line 138, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

[davidcortesortuno]

Hi, from what I can see, the problem arises from building the C modules in parallel. Particularly from the spawn.py file.
Can you try to build Fidimag in Python 3.10? (or 3.9, where we know it works) You can make a new environment in conda:

conda create -n fidimag python=3.10
conda activate fidimag

From here, compile as usual. I'm not sure if the modules in 3.11 have changed too much.
If this doesn't work we can try to check the compilation process in a macOS environment in Github Actions, and see if we can track the problem.

Also, be sure that you are using the system's GCC for both compiling Sundials and for building the modules (conda has a gcc compiler as well)

[dmsilev]

Hi David,

Thank you for the suggestions. I tried both a 3.9 and a 3.10 Python environment, and they both failed with the same issue. Just to be sure, I rebuilt both FFTW and Sundials in the new environments before running the overall make command. Since you mentioned that parallel processing might be an issue, I also tried editing setup.py to set nthreads=1 (rather than the 10 from counting cores on my machine); no luck there either, I'm afraid.

spawn.py does consistently seem to be the source of the problem.

[davidcortesortuno]

Ok, thanks for clarifying. I was thinking the problem might be the arm64 architecture of your macOS machine, which is quite new.
I'll try to find a solution.
If you really want to try Fidimag, the only option for now might be to use Docker. There is a docker file in the main directory, which you can build using docker build -t fidimag . in the Dockerfile folder. You might want to change Ubuntu 18.04 by Ubuntu 20.04 in the script , as 18.04 is quite old but we know it works though.

[davidcortesortuno]

This seems to be an open issue with cython on macOS NixOS/nixpkgs#253205 :(

cython/cython#3262

[davidcortesortuno]

Can you try adding

if __name__ == "__main__":
    multiprocessing.set_start_method('spawn', force=True)

to setup.py ?

[dmsilev]

Adding the set_start_method call doesn't seem to have changed anything.

I'll give the Docker approach a try when I have a chance.

[dmsilev]

Some progress. In setup.py, I changed the execute block to be this:

if __name__ == "__main__":
	multiprocessing.set_start_method('spawn', force=True)
	nthreads = multiprocessing.cpu_count()
	print('Building with {} threads'.format(nthreads))
	setup(
		name='fidimag',
		version=version,
		description='Finite difference micromagnetic code',
		packages=['fidimag',
              'fidimag.atomistic',
              'fidimag.micro',
              'fidimag.extensions',
              'fidimag.common',
		      ],
		ext_modules=cythonize(ext_modules, nthreads=nthreads),
	)

i.e. wrapping the entire block inside the if main test. That resolved the infinite loop. I'm now getting a bunch of linker warnings and one error,

ld: warning: dylib (/Users/dmsilev/mag_solve/fidimag/local/lib/libsundials_nvecopenmp.dylib) was built for newer macOS version (13.0) than being linked (11.1)
ld: warning: dylib (/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/libgomp.dylib) was built for newer macOS version (13.0) than being linked (11.1)
ld: warning: dylib (/Users/dmsilev/mag_solve/fidimag/local/lib/libsundials_cvodes.dylib) was built for newer macOS version (13.0) than being linked (11.1)
clang: error: unsupported option '-fopenmp'
error: command '/usr/bin/clang++' failed with exit code 1

Something presumably was built with Clang rather than GCC-13 (the latter is specified with the CC environmental variable). I'll have to dig into that, but at least there's some hope.

Edit: The version warnings go away by setting an appropriate environmental variable: "export MACOSX_DEPLOYMENT_TARGET=13.0". Clang++ is still being called for some reason though.

[davidcortesortuno]

Maybe https://apple.stackexchange.com/questions/245891/installed-gcc-with-homebrew-now-how-to-use-that-gcc-instead-of-clang

[dmsilev]

After following that suggestion and symlinking both cc and c++ to GCC, same issue. I verified that the symlinks work and Homebrew GCC is what should be called by default

(base) fidimag % c++ --version
c++ (Homebrew GCC 13.2.0) 13.2.0

(base) fidimag % cc --version
cc (Homebrew GCC 13.2.0) 13.2.0

and the shell Path variable is set correctly to see the Homebrew directory before /usr/bin, but something in the build routine is still trying to call /usr/bin/clang++. Going through the whole routine in a fresh conda environment (to try to make sure there weren't any stealth calls to Clang during the environmental setup process) didn't help either.

[davidcortesortuno]

mm, interesting. The setup.py file searches for the compiler in these lines:

if 'CC' in os.environ:
    print("Using CC={}".format(os.environ['CC']))
else:
    os.environ["CC"] = "gcc"
    print("Using CC={} (set by setup.py)".format(os.environ['CC']))

Maybe you need to set export CC=/usr/bin/gcc or so, but you already set that before.

Can you try adding these at the beginning?

os.environ["CC"] = "/usr/bin/gcc";
os.environ["CXX"] = "/usr/bin/g++"

(or wherever you have gcc and g++ installed)

[dmsilev]

Got it to build! To summarize for future reference...

Install GCC via Homebrew. Make two changes to setup.py. At the top, adding

os.environ["CC"] = "/usr/local/bin/gcc"
os.environ["CXX"] = "/usr/local/bin/c++"

where gcc and c++ are symlinks to Homebrew (in my case /opt/homebrew/bin/gcc-13 and g++), and changing the bottom to be

if name == "main":
multiprocessing.set_start_method('spawn', force=True)
nthreads = multiprocessing.cpu_count()
print('Building with {} threads'.format(nthreads))
setup(
name='fidimag',
version=version,
description='Finite difference micromagnetic code',
packages=['fidimag',
'fidimag.atomistic',
'fidimag.micro',
'fidimag.extensions',
'fidimag.common',
],
ext_modules=cythonize(ext_modules, nthreads=nthreads),
)

Two environmental variables set:

export CC = gcc-13
export MACOSX_DEPLOYMENT_TARGET=13.0

Thank you for all of the help and suggestions. Now to try actually using the software...

[davidcortesortuno]

Awesome, I will probably add this to the documentation soon.
Thanks to you for reporting the issue :D

[dmsilev]

It looks like there are still some issues, but now we're on to more subtle problems. For some reason, it's not picking up the LD_LIBRARY_PATH variable and when I run 'import fidimag' from within Python, it complains about not being able to find sundials_cvodes.2.dylib (which exists, as expected, in local/lib ).

If I just switch over to the local/lib directory and run Python from there, it then works, so that's enough for me to get started.

[davidcortesortuno]

I see, maybe check https://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s
And set the other related library paths: DYLD_LIBRARY_PATH in macOS
Also, be sure to not run fidimag from the main directory, as it will try to import the local Python libraries. Running it from a different directory, for instance, from the test or the other folders, is ok

[dmsilev]

Yeah, running it from the root folder seems to be the issue. If I run 'make test' from the root folder, it spams a whole bunch of 'can't find sundials' errors. Going to the tests folder and running python -m pytest instead did work correctly.