Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pyproject.toml, with pixi as project management tool. add pixi.lock #1459

Merged
merged 13 commits into from
Jan 2, 2025

Conversation

jlnav
Copy link
Member

@jlnav jlnav commented Nov 5, 2024

Helps address #1444 and #1403

This adds a pyproject.toml to replace setup.py, which the Python community dramatically encourages.

For project management I chose Pixi (https://pixi.sh/latest/):

  • Can install/interact with both conda and PyPI packages. For example, I can specify mpich as a dev dependency without having to say "install pydantic with pip and mpich with conda".
  • Can split dependencies/packages into project-inherent "environments". Use pixi install -e dev to use the dev environment. With the lockfile included we're all guaranteed to get a working environment since the lockfile was resolved earlier.
  • Very fast rust backend.
  • Installing a package documents/adds that install in the pyproject.toml.
  • Platform-specific dependencies can be specified to only install on that platform. e.g. us linux people that install the project will get gpcam. As a macOS user I automatically get a different clang from conda-forge I use with mpicc.

This does not mean that users need Pixi, and y'all don't need it either.

@jlnav jlnav marked this pull request as ready for review November 5, 2024 17:57
Copy link

codecov bot commented Nov 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.95%. Comparing base (a45928b) to head (61fb6b8).
Report is 31 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #1459       +/-   ##
============================================
+ Coverage    78.43%   90.95%   +12.52%     
============================================
  Files           76       76               
  Lines         7684     7688        +4     
  Branches      1140     1141        +1     
============================================
+ Hits          6027     6993      +966     
+ Misses        1464      534      -930     
+ Partials       193      161       -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jlnav jlnav requested review from jmlarson1 and shuds13 November 5, 2024 20:43
@jmlarson1
Copy link
Member

I'd check the instances of setup.py being used/referred. From the root libE directory, run

grep -ir --include=*.* "setup\.py"

@jlnav
Copy link
Member Author

jlnav commented Nov 7, 2024

Noting that a very minimal setup.py has been added back. But it'll never need to be updated.

@jlnav
Copy link
Member Author

jlnav commented Nov 7, 2024

More information about pixi.lock:

https://pixi.sh/latest/features/lockfile/#how-to-use-a-lock-file

pyproject.toml Outdated
{name = "Stefan M. Wild"}, {name = "David Bindel"},
{name = "John-Luke Navarro"}]

dependencies = ["pip>=24.3.1,<25", "setuptools>=75.1.0,<76", "numpy>=1.21,<3",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"pip>=24.3.1,<25", "setuptools>=75.1.0,<76",
should prob be below in build systems so separate build from run deps.

default = []
dev = ["dev"]

[tool.pixi.feature.dev.dependencies]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking lower bounds should be based on what we support. Or if we want to be latest, maybe make a comment to that effect.

"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now support Python 3.13 - make sure any updates to setup.py are put in here.

@shuds13
Copy link
Member

shuds13 commented Dec 19, 2024

I suggest we dont have lock file in here for now.

@shuds13
Copy link
Member

shuds13 commented Dec 19, 2024

And as Jeff points out above:

(libe_general) shudson@shuds:libe_toml$ grep -ir --include=*.* "setup\.py"
libensemble/tests/unit_tests_logger/.coveragerc:    */setup.py/*
libensemble/tests/unit_tests_mpi_import/.coveragerc:    */setup.py/*
libensemble/tests/regression_tests/.coveragerc:    */setup.py/*
libensemble/tests/.coveragerc:    */setup.py
libensemble/tests/unit_tests/.coveragerc:    */setup.py/*
libensemble/tests/functionality_tests/.coveragerc:    */setup.py/*
libensemble/tests/unit_tests_nompi/.coveragerc:    */setup.py/*
MANIFEST.in:include setup.py
docs/xSDK_policy_compatibility.md:|**M1.** Support xSDK community GNU Autoconf or CMake options. |N/A| libEnsemble is a Python package and provides a `setup.py` file for installation. This is compatible with Python's built-in installation feature (`python setup.py install`) and with the ubiquitous `pip` installer. libEnsemble is also in the Spack repository and can be installed with `spack install py-libensemble`. GNU Autoconf or CMake are unsuitable for a Python package.
docs/xSDK_policy_compatibility.md:|**R5.** Provide a mechanism to export ordered list of library dependencies. |Full| The dependencies for libEnsemble are given in `setup.py` and when pip install or pip setup.py egg_info are run, a file is created `libensemble.egg-info/requires.txt` containing the list of required and optional dependencies. If installing through pip, these will automatically be installed if they do not exist (`pip install libensemble` installs req. dependencies, while `pip install libensemble[extras]` installs both required and optional dependencies.|
docs/dev_guide/release_management/release_platforms/rel_pypi.rst:In the package directory on the main branch (the one containing setup.py) do
docs/dev_guide/release_management/release_platforms/rel_pypi.rst:    python setup.py sdist
docs/dev_guide/release_management/release_process.rst:- ``MANIFEST.in`` is checked. Locally, try out ``python setup.py sdist``

Update docs as necessary and other things here. Maybe this is done.

Not sure about changing xSDK_policy_compatibility.md though, it might be duplicate of something in xSDK repo, so maybe leave that for now.

@jlnav jlnav requested a review from shuds13 December 19, 2024 20:24
@jlnav jlnav merged commit 5c6d70a into develop Jan 2, 2025
34 checks passed
@jlnav jlnav deleted the refactor/pyprojecttoml branch January 2, 2025 18:30
@shuds13 shuds13 mentioned this pull request Feb 14, 2025
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants