Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use poetry for Python dependency management #19

Merged
merged 1 commit into from
Mar 30, 2023
Merged

Use poetry for Python dependency management #19

merged 1 commit into from
Mar 30, 2023

Conversation

chimaerase
Copy link
Collaborator

@chimaerase chimaerase commented Nov 17, 2022

Poetry is emerging as the best / most updated tool for this purpose, and has achieved enough maturity to adopt. See, for example, this article

  1. Remove setup.py in lieu of pyproject.toml and poetry.lock
    Most of the configuration is pasted, but I made some updates, e.g. to classifiers to work out-of-the-box for most new synbio projects.

  2. Use the poetry-dynamic-versioning plugin
    Enables (as configured) creating a PEP440-compliant release number from tags in the git repo. This plugin automates updating multiple files, including pyproject.toml and __init__.py, which would likely become inconsistent over time using only manual updates.

  3. Update Python version to 3.10
    Included in pyproject.toml, so also updating in related CI files. At the time of writing, Python 3.11 is the latest, but only recently released. Likely that some libraries won't have added support for 3.11 yet. Earlier setup.py and CI configs were inconsistent on the Python version, but poetry will enforce consistency during CI builds.

  4. Update CI to use poetry

@chimaerase chimaerase self-assigned this Nov 17, 2022
@chimaerase chimaerase force-pushed the use-poetry branch 4 times, most recently from 55166ad to 497143a Compare November 18, 2022 00:29
Poetry is emerging as the best / most updated tool for this purpose, and has achieved enough maturity to adopt.
See, for example, https://towardsdatascience.com/packages-part-2-how-to-publish-test-your-package-on-pypi-with-poetry-9fc7295df1a5

1.  Remove setup.py in lieu of `pyproject.toml` and `poetry.lock`
    Most of the configuration is pasted, but I made some updates, e.g. to classifiers to work out-of-the-box for most new synbio projects.

2.  Use the `poetry-dynamic-versioning` plugin
    Enables (as configured) creating a PEP440-compliant release number from tags in the `git` repo.  This plugin automates updating multiple files, including `pyproject.toml` and `__init__.py`, which would likely become inconsistent over time using only manual updates.

3.  Update Python version to 3.10
    Included in pyproject.toml, so also updating in related CI files.  At the time of writing, Python 3.11 is the latest, but only recently released.  Likely that some libraries won't have added support for 3.11 yet.  Earlier `setup.py `and CI configs were inconsistent on the Python version, but poetry will enforce consistency during CI builds.

4.  Update CI to use `poetry`
@chimaerase
Copy link
Collaborator Author

Force pushing to resolve conflicts with recent merge of #20. Some manual changes, so re-review needed if this PR is already partly reviewed. Changes should be visible via the "force" link in the GitHub UI.

@chimaerase
Copy link
Collaborator Author

Ping! I'm not comfortable merging this PR without review. Lots happening of course, and it's now 2 months old.

@chimaerase
Copy link
Collaborator Author

Added Steve as a reviewer in case he has time to help with this. Steve, I initially omitted pyenv from directions here in the hope of keeping scope narrow, but intended to raise that during discussion. Curious to hear your thoughts on that -- maybe just a brief mention and a link out to pyenv? E.g. since poetry includes a Python version and so increases the need for it.

@sdlane-lbl
Copy link

Hi Mark,

(I'm really sorry it took me so long to get to this...)

I'd say that, in general, this looks good to me. If you've tested it / are satisfied with it, then I'd say go ahead (I didn't see anything I'd call out).

Re: pyenv: obv you don't need to use it, and can add it in later if you wish. My thinking on it at this point is that, in general, it provides more flexibility (and maybe more reliability...) than using the poetry-supplied Python, which is why folks seem to like it, but if just using poetry works for you / your project(s), it is one less dependency / thing to deal with, so...

FYI: some links (I came up with after ~5s of searching):

https://python.plainenglish.io/why-you-should-use-pyenv-poetry-for-python-projects-a13f8d234e46

https://blog.jayway.com/2019/12/28/pyenv-poetry-saviours-in-the-python-chaos/ (Note that this post contains a pyramid figure of how different installed versions of Python relate to each other that I think is the best illustration of this issue I've seen so far)

python-poetry/poetry#5252 (This is an... interesting..? discussion of some of the potential issues with poetry Python vs pyenv + poetry Python)

https://www.adaltas.com/en/2021/06/09/pyrepo-project-initialization/

@sdlane-lbl
Copy link

A further comment: given that this is intended to be both a demonstration of how to do things The Right Way (or how things can be done A Right Way), and a prototype / template / cookiecutter-like thing, maybe it would be good to have two branches, one with the python version being managed by poetry, and one with it being managed by pyenv on top of (in front of? somewhere...) poetry.

In which case, it may not make sense to have a main / trunk branch per-se, but rather just the pyenv+poetry and poetry branches (though I could also see an argument for having the main / trunk branch be called that, and have it be the pyenv+poetry branch, and a second, poetry-only branch – actually, I think I like that better at the moment).

@chimaerase
Copy link
Collaborator Author

Thanks for your comments, Steve. And no problem re: delay -- I think we would have heard if someone needed this in the interim, so no harm in taking some time to get it right. On revisiting this myself, my impulse is to insert a plug for pyenv in the documentation, and to avoid documenting that (since it should have its own docs). I agree it's probably best to merge as-is for now and I'll follow up soon with an encouraging pyenv mention as a way of avoiding multiple branches. Also updating to a newer version of poetry would be good.

@chimaerase chimaerase merged commit 9416f89 into main Mar 30, 2023
@chimaerase chimaerase deleted the use-poetry branch March 30, 2023 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants