pyoculus build is crashing simsopt CI

[landreman]

Starting last week, the simsopt CI's have been failing at the step where pyoculus is pip-installed. Here's one example. The error message is below. Todd Elder is having the same error when trying to install pyoculus on his laptop. It looks like it's an issue with pyoculus's setup.py system. @smiet @zhisong @mbkumar Any idea how to fix this?

Collecting pyoculus
  Downloading pyoculus-0.3.1.tar.gz (844 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 844.2/844.2 kB 58.6 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
        File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/numpy/distutils/core.py", line 24, in <module>
          from numpy.distutils.command import config, config_compiler, \
        File "/tmp/pip-build-env-qhbh[225](https://github.com/hiddenSymmetries/simsopt/actions/runs/10683512465/job/29611749973#step:15:226)t/overlay/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 19, in <module>
          from numpy.distutils.mingw32ccompiler import generate_manifest
        File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
          from distutils.msvccompiler import get_build_version as get_build_msvc_version
      ModuleNotFoundError: No module named 'distutils.msvccompiler'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

[smiet]

Hi all, I know I can fix this, but I don't have much time this week as I'm off before ISHW, then at ISHW. I think It's related to the issues we have been having with new numpy and other packages updating because of this kick-in-the-butt, notably setuptools deprecating its Fortran compilation features. I have a pull request for a build using the 'future proof-er' meson build on pyoculus, @zhisong, could you try it and approve it? #8 Additionally, can someone create a test branch of simsopt to confirm this branch solves the issue? I can dedicate some time on Thursday and will see what I can do. There are future changes to simsopt planned to integrate @aikitzu features related to turnstile calculations, but we will fast-forward the modifications needed to fix the failing ci. Warmest regards, Chris

…

On Tue, Sep 3, 2024, 10:27 PM Matt Landreman ***@***.***> wrote: Starting last week, the simsopt CI's have been failing at the step where pyoculus is pip-installed. Here's one example. <https://github.com/hiddenSymmetries/simsopt/actions/runs/10683512465/job/29611749323> The error message is below. Todd Elder is having the same error when trying to install pyoculus on his laptop. It looks like it's an issue with pyoculus's setup.py system. @smiet <https://github.com/smiet> @zhisong <https://github.com/zhisong> @mbkumar <https://github.com/mbkumar> Any idea how to fix this? Collecting pyoculus Downloading pyoculus-0.3.1.tar.gz (844 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 844.2/844.2 kB 58.6 MB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [21 lines of output] Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module> main() File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 318, in run_setup exec(code, locals()) File "<string>", line 2, in <module> File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/numpy/distutils/core.py", line 24, in <module> from numpy.distutils.command import config, config_compiler, \ File "/tmp/pip-build-env-qhbh[225](https://github.com/hiddenSymmetries/simsopt/actions/runs/10683512465/job/29611749973#step:15:226)t/overlay/lib/python3.9/site-packages/numpy/distutils/command/config.py", line 19, in <module> from numpy.distutils.mingw32ccompiler import generate_manifest File "/tmp/pip-build-env-qhbh225t/overlay/lib/python3.9/site-packages/numpy/distutils/mingw32ccompiler.py", line 28, in <module> from distutils.msvccompiler import get_build_version as get_build_msvc_version ModuleNotFoundError: No module named 'distutils.msvccompiler' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. — Reply to this email directly, view it on GitHub <#10>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFPG2HBKH76E2WG4YUEKQLZUW2K5AVCNFSM6AAAAABNSETXKOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUYDEOBYGU2TENQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[smiet]

@mbkumar @landreman:

The issue is with an unpinned version of setuptools, and changes to distutils, as explained here: pypa/setuptools#3532.

Pointing simsopt to the meson-built pyoculus fixes pyoculus, but now VMEC2000 fails.

There are some suggested fixes, such as: pypa/setuptools#3505 (comment):

However, having setuptools<65 in requirements.txt with pip install --no-use-pep517 does fix it

[smiet]

pinning setuptools to lt. 65, (PR #11 ) as I don't like the setuptools solution (PR #8 ). Confirmed to work in the branch used by Todd Elder.

Re-open if this does not fix.