Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use prefer_ebpythonprefixes flag to use EBPYTHONPREFIXES #3343

Open
wants to merge 5 commits into
base: 5.0.x
Choose a base branch
from
Open

Use prefer_ebpythonprefixes flag to use EBPYTHONPREFIXES #3343

wants to merge 5 commits into from

Conversation

Micket
Copy link
Contributor

@Micket Micket commented May 29, 2024

@Micket Micket added feature EasyBuild-5.0 EasyBuild 5.0 labels May 29, 2024
@Micket Micket added this to the 5.0 milestone May 29, 2024
@Micket Micket changed the title Use new prefer_ebpythonprefixes Use prefer_ebpythonprefixes flag to use EBPYTHONPREFIXES May 29, 2024
@Micket Micket mentioned this pull request Jul 7, 2024
Open
@Micket
Copy link
Contributor Author

Micket commented Aug 5, 2024 •
edited
Loading

special easyblocks that have been directly modified:

  • amber - Amber-22.0-foss-2021b-AmberTools-22.3.eb - no source
  • cplex - CPLEX-22.1.1-GCCcore-11.2.0.eb - no source
  • gurobi - Gurobi-11.0.0-GCCcore-12.3.0.eb
  • lammps - LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb
  • mxnet - MXNet-1.9.1-foss-2022a.eb
  • openbabel - OpenBabel-3.1.1-gompi-2023a.eb
  • opencv - OpenCV-4.6.0-foss-2022a-contrib.eb
  • qscintilla - QScintilla-2.11.6-GCCcore-11.2.0.eb
  • tkinter - Tkinter-3.11.5-GCCcore-13.2.0.eb

PythonPackage or PythonBundle or subclasses indirectly modified:

  • Python - Python-3.11.5-GCCcore-13.2.0.eb
  • PythonPackage - PyYAML-6.0.1-GCCcore-13.2.0.eb
  • PythonBundle - Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb
  • CargoPythonPackage - cryptography-41.0.5-GCCcore-13.2.0.eb
  • CargoPythonBundle - pydantic-2.5.3-GCCcore-12.3.0.eb
  • wxpython - wxPython-4.2.1-foss-2022a.eb
  • dm_reverb - dm-reverb-0.7.0-foss-2021b.eb
  • torchvision - torchvision-0.16.0-foss-2023a.eb
  • tensorrt - TensorRT-8.6.1-foss-2022a-CUDA-11.7.0.eb
  • tensorflow - TensorFlow-2.9.1-foss-2022a.eb
  • tensorflow_compression - tensorflow-compression-2.11.0-foss-2022a-CUDA-11.7.0.eb
  • numpy - part of testing Scipy bundle
  • numexpr - numexpr-2.8.4-foss-2022a.eb
  • netcdf4_python - netcdf4-python-1.6.5-foss-2023b.eb
  • configuremakepythonpackage - PyQt5-5.15.7-GCCcore-12.2.0.eb
  • cmakepythonpackage - vigra-1.11.2-foss-2023a.eb
  • fortranpythonpackage - inherited by scipy easyblock
  • libxml2 - libxml2-2.12.7-GCCcore-13.3.0.eb
  • pybind11 - pybind11-2.11.1-GCCcore-13.2.0.eb
  • pyzmq - PyZMQ-25.1.2-GCCcore-13.2.0.eb (is this even using pyzmq? i don't think so)
  • sepp - SEPP-4.5.1-foss-2022b.eb
  • sympy - sympy-1.12-gfbf-2023b.eb
  • scipy - SciPy-bundle-2023.11-gfbf-2023b.eb
  • eqqlib - EggLib-3.3.0-GCC-13.2.0.eb
  • easybuildmeta - EasyBuild-4.9.2.eb
  • jaxlib - jax-0.4.4-foss-2022a.eb

Nothing to test:

  • vsc_tools - nothing?
  • cryptography - not in use anymore?
  • ufc - nothing
  • nose - nothing? nose3 uses pythonpackage directly.
  • dolfin - DOLFIN-xxx.eb (nothing remains)
  • versionindependentpythonpackage - vsc-base (nothing remains)
  • pyquante - PyQuante (nothing remains)
  • python_meep - python-meep (nothing remains)

(todo: make list of specific easyconfigs to test for the generic easyblocks)

@Micket
Copy link
Contributor Author

Micket commented Aug 15, 2024

Slight issue with libxml2; the easyblock inherits from PythonPackage, and PythonPackage has always unconditionally added PYTHONPATH/EBPYTHONPREFIXES, so that behavior is retained here.

I could change this; always let the framework add the environment variable if it detects the corresponding path, but this is not done here (yet?)

@Micket Micket force-pushed the ebpythonprefixes branch 3 times, most recently from b6f0388 to 2ed73d5 Compare August 15, 2024 19:12
@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS Gurobi-11.0.0-GCCcore-12.3.0.eb
  • SUCCESS LAMMPS-2Aug2023_update2-foss-2023a-kokkos.eb
  • SUCCESS MXNet-1.9.1-foss-2022a.eb
  • SUCCESS OpenBabel-3.1.1-gompi-2023a.eb
  • SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb
  • SUCCESS QScintilla-2.11.6-GCCcore-11.2.0.eb
  • SUCCESS Tkinter-3.11.5-GCCcore-13.2.0.eb

Build succeeded for 7 out of 7 (7 easyconfigs in total)
alvis-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/9eef93fc27bd9113f744e9a00c06ba64 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS Python-3.11.5-GCCcore-13.2.0.eb
  • SUCCESS PyYAML-6.0.1-GCCcore-13.2.0.eb
  • SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb
  • SUCCESS cryptography-41.0.5-GCCcore-13.2.0.eb
  • SUCCESS pydantic-2.5.3-GCCcore-12.3.0.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
alvis-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/6f25e6b6852273bd08bb082839e84420 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS wxPython-4.2.1-foss-2022a.eb
  • SUCCESS dm-reverb-0.7.0-foss-2021b.eb
  • SUCCESS torchvision-0.16.0-foss-2023a.eb
  • SUCCESS numexpr-2.8.4-foss-2022a.eb
  • SUCCESS PyQt5-5.15.7-GCCcore-12.2.0.eb
  • SUCCESS libxml2-2.12.7-GCCcore-13.3.0.eb
  • SUCCESS pybind11-2.11.1-GCCcore-13.2.0.eb

Build succeeded for 7 out of 7 (7 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/e60e25dd2a7ee72182ebded2138fdf1c for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS netcdf4-python-1.6.5-foss-2023b.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/fe7c70f7a98bf61dfc899fb8cc2cebb2 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS PyZMQ-25.1.2-GCCcore-13.2.0.eb
  • SUCCESS SEPP-4.5.1-foss-2022b.eb
  • SUCCESS sympy-1.12-gfbf-2023b.eb
  • SUCCESS SciPy-bundle-2023.11-gfbf-2023b.eb
  • SUCCESS EggLib-3.3.0-GCC-13.2.0.eb
  • SUCCESS jax-0.4.4-foss-2022a.eb
  • SUCCESS vigra-1.11.2-foss-2023a.eb

Build succeeded for 7 out of 7 (7 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/f65845a0504893de88541d44b48432d1 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS EasyBuild-4.9.2.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/f9a0187ee654cfafbd899010aaf97132 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS TensorRT-8.6.1-foss-2022a-CUDA-11.7.0.eb
  • SUCCESS TensorFlow-2.9.1-foss-2022a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.9, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/d15524a675bdb83242a8c84e2a042c99 for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 22, 2024

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS tensorflow-compression-2.11.0-foss-2022a-CUDA-11.7.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
alvis2-01 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz, 1 x NVIDIA Tesla T4, 550.54.14, Python 3.6.8
See https://gist.github.com/Micket/37055a52691ade6faff20ff9cb4e2e0e for a full test report.

@Micket
Copy link
Contributor Author

Micket commented Aug 23, 2024

All the tests above ran fine and produced the correct PYTHONPATH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
EasyBuild-5.0 EasyBuild 5.0 feature
Projects
EasyBuild v5.0
Status: Nice-to-have
Milestone
5.0
Development

Successfully merging this pull request may close these issues.
1 participant
@Micket