parallel cython build ends up in infinite recursion on macOS in yt-4.0

[jzuhone]

Bug report

Bug summary

We started using all available cores to build yt-4.0 from source in PR #2438. I recently tried to build on macOS and ended up in an infinite recursion. The huge traceback is here:

http://paste.yt-project.org/show/225/

Code for reproduction

Simply trying to install yt-4.0 from source in python setup.py develop or pip install -e . triggers this. If I comment out lines 22 and 23 here (disabling the parallel build) the problem goes away:

yt/setup.py

Lines 21 to 24 in d99f816

	def _get_cpu_count():
	if platform.system() != "Windows":
	return os.cpu_count()
	return 0

I have no idea if we have to treat macOS like Windows or if there's some way we can make it work.

Version Information

• Operating System: macOS 10.15.3
• Python Version: 3.8.2
• yt version: 4.0

[Xarthisius]

I can't reproduce on OSX I have access to. Which compiler are you using?

[jzuhone]

clang:

gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.3 (clang-1103.0.32.59)
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[jzuhone]

This seems to be a Python 3.8 thing also, as I tried this on my 3.7 installation and it worked fine.

[Xarthisius]

Could you try adding import multiprocessing at the top of the setup.py and see if that changes anything?

[jzuhone]

Sadly no. What seems to happen is that multiple copies of everything in setup.py runs.