PyPartMC

[slayoo]

Submitting Author: Sylwester Arabas (@slayoo)
All current maintainers: (@zdaq12, @jcurtis2, @nriemer, @mwest1066)
Package Name: PyPartMC
One-Line Description of Package: Python interface to PartMC aerosol-dynamics Monte-Carlo simulation package
Repository Link: https://github.com/open-atmos/PyPartMC/
Version submitted: v1.2.0
EIC: @Batalex
Editor: @cmarmo
Reviewer 1: @simonom
Reviewer 2: @dbaston
Archive: TBD
Version accepted: TBD
Date accepted (month/day/year): TBD

---

Code of Conduct & Commitment to Maintain Package

• I agree to abide by [pyOpenSci's Code of Conduct][PyOpenSciCodeOfConduct] during the review process and in maintaining my package after should it be accepted.
• I have read and will commit to package maintenance after the review as per the [pyOpenSci Policies Guidelines][Commitment].

Description

PyPartMC is a Python interface to PartMC, a particle-resolved Monte-Carlo code for atmospheric aerosol simulation. PyPartMC is implemented mostly in C++ (based on the pybind11 framework) with some C and Fortran boilerplate layers. PyPartMC constitutes an API to the PartMC Fortran internals. Besides empowering Python/Jupyter users, PyPartMC can facilitate using PartMC from other environments - see, e.g., Julia and Matlab examples in the project README.

Scope

- [ ] Data retrieval
- [ ] Data extraction
- [ ] Data processing/munging
- [ ] Data deposition
- [ ] Data validation and testing
- [ ] Data visualization
- [ ] Workflow automation
- [ ] Citation management and bibliometrics
- [x] Scientific software wrappers
- [ ] Database interoperability

Domain Specific

n/a

(atmospheric science)

Community Partnerships

n/a

(we host development as a part of the OpenAtmos initiative: https://github.com/open-atmos)

Target audience and scientific applications

Development of PyPartMC has been intended to remove limitations to the use of Fortran-implemented PartMC software. PyPartMC facilitates the dissemination of computational research results by streamlining independent execution of PartMC simulations, which could prove advantageous during peer review process. Additionally, the ability to easily package examples, simple simulations, and results in a web-based notebook allows PyPartMC to support the efforts of many members of the scientific community, including researchers, instructors, and students, with nominal software and hardware requirements.

Other Python packages with relevant feature scope

• PySDM: particle-based Monte-Carlo aerosol-cloud simulation package (differences: PySDM focuses on growth and breakup processes relevant to cloud droplets; PyPartMC focuses on processes relevant to air pollutants and their chemical and physical transformations)
• DustPy: Python package for modelling dust evolution in protoplanetary disks (differences: focus on astrophysical applications vs. atmospheric aerosol)
• PyBox: aerosol simulation model featuring gas and particle chamistry (differences: PyBox focuses on chemical mechanisms; PyPartMC is an interface to PartMC which focuses on physics - e.g., collisions of aerosol particles - while chemical processes are handled with external software, e.g., CAMP or MOSAIC)

Technical checks

For details about the pyOpenSci packaging requirements, see our [packaging guide][PackagingGuide]. Confirm each of the following by checking the box. This package:

• does not violate the Terms of Service of any service it interacts with.
• uses an [OSI approved license][OsiApprovedLicense].
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions:
  • README file: https://github.com/open-atmos/PyPartMC/blob/main/README.md
  • auto-generated docstring-based docs: https://open-atmos.github.io/PyPartMC/
  • documentation of the wrapped PartMC package: https://lagrange.mechse.illinois.edu/partmc/partmc-2.6.1/doc/html/index.html
• contains a tutorial with examples of its essential functions and uses.
  • Jupyter notebooks with examples: https://github.com/open-atmos/PyPartMC/blob/main/README.md#jupyter-notebooks-with-examples
• has a test suite.
• has continuous integration setup, such as GitHub Actions CircleCI, and/or others.

Publication Options

• Do you wish to automatically submit to the [Journal of Open Source Software][JournalOfOpenSourceSoftware]? If so:

(we have recently published a SoftwareX paper on PyPartMC: https://doi.org/10.1016/j.softx.2023.101613)

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

• Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Confirm each of the following by checking the box.

• I have read the author guide.
• I expect to maintain this package for at least 2 years and can help find a replacement for the maintainer (team) if needed.

Please fill out our survey

• Last but not least please fill out our pre-review survey.

[Batalex]

Hey @slayoo, I'm Alex, currently serving as the Editor-in-Chief. I'm sorry it took me so long to get back to you.

PyPartMC seems to be in scope for us. I truly appreciate the care you put into the submission, especially the comparative section. It's very valuable to a profane such as myself when it comes to evaluating submissions.

Please find below the preliminary checks.

Editor-in-Chief checks

Thank you for submitting your package for pyOpenSci review. Below are the basic checks that your package needs to pass to begin our review. If some of these are missing, we will ask you to work on them before the review process begins.

Please check our Python packaging guide for more information on the elements below.

• Installation The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
  • The package imports properly into a standard Python environment import package.
    The package installation does not install the dependencies
• Fit The package meets criteria for fit and overlap.
• Documentation The package has sufficient online documentation to allow us to evaluate package function and scope without installing the package. This includes:
  • User-facing documentation that overviews how to install and start using the package.
  • Short tutorials that help a user understand how to use the package and what it can do for them.
  • API documentation (documentation for your code's functions, classes, methods and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
• Core GitHub repository Files
  • README The package has a README.md file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
  • Contributing File The package has a CONTRIBUTING.md file that details how to install and contribute to the package.
  • Code of Conduct The package has a CODE_OF_CONDUCT.md file.
  • License The package has an OSI approved license.
    NOTE: We prefer that you have development instructions in your documentation too.
• Issue Submission Documentation All of the information is filled out in the YAML header of the issue (located at the top of the issue template).
• Automated tests Package has a testing suite and is tested via a Continuous Integration service.
• Repository The repository link resolves correctly.
• Package overlap The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly.
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

---

• Initial onboarding survey was filled out
  We appreciate each maintainer of the package filling out this survey individually. 🙌
  Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. 🙌

---

---

Editor comments

I have a few suggestions regarding the documentation:

• Having the links to the binder/collab/etc. is great. Would you consider including those end to end examples in the documentation website? It is a pity that it only has the API reference. I'd like to see more textual content as well, to properly frame the code blocks and fully understand what is going on
• I think it would be great to have a dedicated CONTRIBUTING.md/rst file in the repository with the README's sections about the implementation details and the developers notes.
• We suggest adding a code of conduct document as well. It's always a good idea to set the proper expectations from the start!

[slayoo]

@Batalex, thanks for the feedback!
We've just started addressing the above points with a code-of-conduct PR: open-atmos/PyPartMC#358
I'll report back here after addressing all points

[slayoo]

@Batalex, with the following PRs just merged:

• Create CODE_OF_CONDUCT.md open-atmos/PyPartMC#358
• improve pdoc landing site by rendering the README contents in the index.html. closes #332 open-atmos/PyPartMC#359
• move developer notes into a new CONTRIBUTING.md file open-atmos/PyPartMC#360

the repo and docs are improved following your suggestions:

• https://open-atmos.github.io/PyPartMC
• https://github.com/open-atmos/PyPartMC/tree/main/CODE_OF_CONDUCT.md
• https://github.com/open-atmos/PyPartMC/tree/main/CONTRIBUTING.md

[Batalex]

Thank you, @slayoo, the docs look great!
PyPartMC is a strong submission, I'll get started on finding an editor right now 🫡

[slayoo]

thank you!

[Batalex]

Sorry, I forgot to add something important to my previous message. Since our field of expertise is Python, the significant portion of C++ code in PyPartMC will only be evaluated on the best effort basis (which means that maybe it won't be evaluated). Our focus will be on the Python packaging side of the project.

[cmarmo]

Hello @slayoo, thank you for patience! I'm Chiara, I am following up as Editor in chief.
I am glad to announce that @russbiggs kindly accepted to serve as editor for your submission.
I'll let him introduce himself here and I wish you a happy review process!

[slayoo]

Thank you, @russbiggs and @cmarmo!

[russbiggs]

@slayoo Sorry for the slow start on my part as editor, I am currently seeking reviewers and will update as that progresses.

[cmarmo]

Hello @slayoo , unfortunately Russ couldn't move forward with this submission.

We are working to find a new editor for you.
Thank you for your patience! 🙏

[slayoo]

thank you, @cmarmo

[lwasser]

hey team - i'm checking in on this review. I think we need an editor for it. Is that correct? If so i can put this one on my radar to find someone to edit.

[slayoo]

thank you @lwasser

[cmarmo]

Hello @slayoo, I'm going to take care of PyPartMC as editor.
I have started to look for reviewers let's cross fingers ... 🤞

[lwasser]

thank you @cmarmo ❤️ @slayoo thank you for your patience!!

[slayoo]

thank you @cmarmo

[cmarmo]

Hi @slayoo, I am glad to announce that we have our first reviewer! 🥳
@simonom, thank you for volunteering to review for pyOpenSci!

I am actively looking for a second reviewer, but I think Simon can start to review, as this submission already waited a bit.
So I'm letting him introduce himself here and continuing my quest.

Before beginning your review, please fill out our pre-review survey. This helps us improve all aspects of our review and better understand our community. No personal data will be shared from this survey - it will only be used in an aggregated format by our Executive Director to improve our processes and programs.

The following resources will help you complete your review:

1. Here is the reviewers guide. This guide contains all of the steps and information needed to complete your review.
2. Here is the review template that you will need to fill out and submit
   here as a comment, once your review is complete.

[Kai-Striega]

Hello @slayoo,

@cmarmo commented on the scientific-python discord, asking for reviewers. I'm happy to review this if no one else becomes available soon.

I've done significant work using C++/Fortran + Python. However, I haven't done much with pybind11 or CMake. Happy to put some work in and brush up on those.

Before committing, I’ve signed up to a couple of different things that I’ll need to prioritise. It may take me a while (at the latest, over the Christmas break) to get a quality review done. Is all that acceptable for you? Happy to be superseded if someone else becomes available.

[cmarmo]

@Kai-Striega thank you so much for offering your help!
@slayoo, please let us know if this timeline is ok for you: I'm going to check if someone else is available but without answers before Christmas I may be willing to let Kai take care of the review... 🙂

[simonom]

Dear @slayoo. I am struggling with
pip install PyPartMC
at command line on an Apple M2 Pro (macOS Sonoma). Below I paste the first and final lines of the error message (there are lots of lines in between). Please let me know what further information you need.
first lines:

(PyPartMC) mbexjso4@c-losx7mqvt ~ % pip install PyPartMC
Collecting PyPartMC
  Using cached PyPartMC-1.3.6.tar.gz (6.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy (from PyPartMC)
  Using cached numpy-2.1.3-cp313-cp313-macosx_14_0_arm64.whl.metadata (62 kB)
Using cached numpy-2.1.3-cp313-cp313-macosx_14_0_arm64.whl (5.1 MB)
Building wheels for collected packages: PyPartMC
  Building wheel for PyPartMC (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for PyPartMC (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [9528 lines of output]
      WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
      Traceback (most recent call last):
        File "/private/var/folders/1b/939cjlwj0tv2hfh7hw5_ns3m0000gp/T/pip-build-env-3ai1s42x/normal/lib/python3.13/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject
          section = defn.get("tool", {})[tool_name]
                    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
      KeyError: 'setuptools_scm'
      running bdist_wheel
      running build
      running build_py
      creating build/lib.macosx-11.0-arm64-cpython-313/PyPartMC
      copying PyPartMC/__init__.py -> build/lib.macosx-11.0-arm64-cpython-313/PyPartMC
      running build_ext
      CMake Deprecation Warning at CMakeLists.txt:7 (cmake_minimum_required):
        Compatibility with CMake < 3.10 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      -- The C compiler identification is AppleClang 16.0.0.16000026
      -- The CXX compiler identification is AppleClang 16.0.0.16000026
      -- The Fortran compiler identification is GNU 14.2.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Checking whether Fortran compiler has -isysroot
      -- Checking whether Fortran compiler has -isysroot - yes
      -- Checking whether Fortran compiler supports OSX deployment target flag
      -- Checking whether Fortran compiler supports OSX deployment target flag - yes
      -- Detecting Fortran compiler ABI info
      -- Detecting Fortran compiler ABI info - done
      -- Check for working Fortran compiler: /opt/homebrew/bin/gfortran - skipped
      CMake Warning (dev) at CMakeLists.txt:19 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      This warning is for project developers.  Use -Wno-dev to suppress it.

final lines:

      File "<string>", line 119, in build_extension
        File "/Users/user/miniconda3/envs/PyPartMC/lib/python3.13/subprocess.py", line 419, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', '/private/var/folders/1b/939cjlwj0tv2hfh7hw5_ns3m0000gp/T/pip-install-2q15_bdg/pypartmc_f08100eb62e54cd78bbd0dd90fad68ee', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/1b/939cjlwj0tv2hfh7hw5_ns3m0000gp/T/pip-install-2q15_bdg/pypartmc_f08100eb62e54cd78bbd0dd90fad68ee/build/lib.macosx-11.0-arm64-cpython-313/', '-DPYTHON_EXECUTABLE=/Users/user/miniconda3/envs/PyPartMC/bin/python3.13', '-DCMAKE_BUILD_TYPE=Release', '-DVERSION_INFO=1.3.6']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for PyPartMC
Failed to build PyPartMC
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (PyPartMC)

[slayoo]

@simonom, thank you!

So far, we have not yet produced binary wheels for Python 3.13, please stay tuned, hope to roll these out in the coming days at least for macOS.

Since there are no binary wheels available for your platform/OS setup, pip tries to compile the package from source. The failure in getting it compiled seems to originate from an incompatibility around the setuptools_scm package which we use to automatically generate PyPartMC version strings, will investigate it as well.

Thanks!

[slayoo]

@Kai-Striega thank you so much for offering your help! @slayoo, please let us know if this timeline is ok for you: I'm going to check if someone else is available but without answers before Christmas I may be willing to let Kai take care of the review... 🙂

@Kai-Striega, of course, thank you!

[cmarmo]

Hi @slayoo, I am glad to announce that we have our second reviewer! 🥳
@dbaston, thank you for volunteering to review for pyOpenSci!

Before beginning your review, please fill out our pre-review survey. This helps us improve all aspects of our review and better understand our community. No personal data will be shared from this survey - it will only be used in an aggregated format by our Executive Director to improve our processes and programs.

The following resources will help you complete your review:

1. Here is the reviewers guide. This guide contains all of the steps and information needed to complete your review.
2. Here is the review template that you will need to fill out and submit here as a comment, once your review is complete.

Now that we have the two reviewers assigned I am tentatively setting a deadline for the review of PyPartMC: let's try to give a nice Christmas gift to PyPartMC maintainers and let the review in for December 23th! 🎄

Thank you all for your work and your patience!

[cmarmo]

@simonom I took the liberty to highlight the traceback in your previous comment.

I see you are using a miniconda installation, in order to move forward with the review, may I suggest to create a conda/mamba environment with a 3.12 python version?
So we do not need to wait for a 3.13 wheel: the 3.13 wheel build can be a review recommendation though....

Also @dbaston, @simonom , please let me know if the review deadline is realistic.

Thank you! 🙏

[simonom]

@cmarmo, yes, I will continue with a compatible python version, and yes, deadline can be met!

[slayoo]

@simonom

I am struggling with pip install PyPartMC at command line on an Apple M2 Pro (macOS Sonoma).

So far, we have not yet produced binary wheels for Python 3.13, please stay tuned, hope to roll these out in the coming days at least for macOS.

As of PyPartMC v1.3.10 (released earlier today), there are binary wheels on PyPI for Python 3.13 on macOS. Supported OS versions are: macosx-13 (universal binary) and macosx-14 (ARM binary). Hope it helps! Feedback welcome.

Note that Python 3.13 is not fully supported in auxiliary packages used in PyPartMC example notebooks, notably:

• PyMieScatt used in the mie_optical.ipynb example requires installation from GitHub instead of PyPI (pip install git+https://github.com/bsumlin/PyMieScatt.git) to work due to Python 3.13 implying newer SciPy (reported upstream)
• PySDM used in lognorm_ex.ipynb and in terminal_velocities.ipynb uses Numba which requires pre-release version to be compatible with Python 3.13 (pip install numba==0.61.0rc1), but it anyhow fails on ARM CPUs (reported upstream)