Use cython distutils directives

[matthewturk]

This is the start of a process to make all the modules use cythonize and distutils directives. In the first PR, it just converts yt/geometry/.

Putting the configuration information into the files themselves should make it easier to update them. But, if we decide this isn't what we want, then it isn't what we want!

At present this is not working for in-place builds, which I'm hoping to address soon. Basically, the question before us is: would this be an improvement for developers' lives? I think all of the conditional compilations can be handled in this way.

[matthewturk]

So I think this is ready to be looked at. I know that the conditional compiles need to be addressed (just pyembree; gperftools is addressed in #2510 ) and that right now the parallelized build has to be turned back on, but otherwise this should be OK.

[matthewturk]

I've ticked off the boxes for conditional compilation of embree, and I've reverted to using the overridden built_ext function, so I think this is ready for review and maybe being included. I've tested the results of sdist and they seem to be consistent, as do the results of building in-place and via installation.

[matthewturk]

I should note that for the purposes of review, nearly 100% of the changes that are meaningful happened in setup.py. I moved some embree stuff around so that it wouldn't get caught up in the glob patterns and I added the appropriate # distutils: directives to the top of all the cython files that needed them.

[munkm]

I'm happy to review this but it's going to be a week or so until after the 3.6 release. I want to comment about it here so you know it's not being forgotten! 🙂

[matthewturk]

I'm hoping that once this is in, we can do header-level cython directives instead of function-level, which should make debugging easier as we can toggle them on/off from the command line instead of modifying individual functions. This script implements that.

import glob
prepend = "# cython: language_level=2, bounds_check=False, wraparound=False\n# cython: cdivision=True, initializedcheck=False\n"

for fn in sorted(glob.glob("yt/**/*.pyx", recursive=True)):
    print(f"Prepending for {fn}")
    lines = []
    with open(fn, "r") as f:
        for line in f:
          if line.strip().startswith("@cython"): continue
          lines.append(line)
    with open(fn, "w") as f:
        f.write(prepend)
        f.writelines(lines)

[matthewturk]

I've merged with upstream post-#2513 to run with the new tests.

[Xarthisius]

--- a/setup.py
+++ b/setup.py
@@ -42,7 +42,7 @@ with open('README.md') as file:
 if check_for_openmp() is True:
     omp_args = ['-fopenmp']
 else:
-    omp_args = None
+    omp_args = []
 
 if os.name == "nt":
     std_libs = []

did the trick for me on OSX.

[munkm]

This looks good to me. It looks like there is also some mesh traversal stuff in the test install script:

https://github.com/yt-project/yt/blob/yt-4.0/tests/test_install_script.py#L39

and I think also the gitignore might need to be updated? https://github.com/yt-project/yt/blob/yt-4.0/.gitignore#L61

[matthewturk]

Had to fix the conflicts; ptal

[matthewturk]

@jzuhone can you check that I've correctly ported your std=c++11 here?