[WIP] Use poetry #715

bartekpacia · 2020-08-05T22:14:49Z

Description

Migrated to Poetry as our default dependency management tool. It's simple, yet powerful.

added pyproject.toml
removed requirements.txt

Docker is currently broken I'm not sure how to fix this!

mentorship-backend(use_poetry) ➜ docker build -t mentorship-backend:latest .
Sending build context to Docker daemon  673.3kB
Step 1/7 : FROM python:3.7
 ---> fbf9f709ca9f
Step 2/7 : COPY ./pyproject.toml /dockerBuild/pyproject.toml
 ---> Using cache
 ---> 9e4608c4655e
Step 3/7 : WORKDIR /dockerBuild
 ---> Using cache
 ---> ad1effb304dc
Step 4/7 : RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
 ---> Using cache
 ---> dc3855305db0
Step 5/7 : RUN poetry install
 ---> Running in eae19bec558e
/bin/sh: 1: poetry: not found
The command '/bin/sh -c poetry install' returned a non-zero code: 127

fixes #691

Type of Change:

Code
Documentation

Code/Quality Assurance Only

New feature (non-breaking change which adds functionality pre-approved by mentors)

How Has This Been Tested?

Run the test suite, all tests pass.

Checklist:

My PR follows the style guidelines of this project
I have performed a self-review of my own code or materials
I have commented my code or provided relevant documentation, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
Any dependent changes have been merged
Update Postman API at /docs folder
Update Swagger documentation and the exported file at /docs folder
~~Update requirements.txt~~ Not anymore! Updated pyproject.toml haha

Code/Quality Assurance Only

My changes generate no new warnings
My PR currently breaks something (fix or feature that would cause existing functionality to not work as expected)
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been published in downstream modules

bartekpacia · 2020-08-05T22:37:15Z

Currently poetry run start and poetry run test are broken. I'm looking for a solution.

README.md

SanketDG

Couple of comments!

I wrote this a couple of months ago and forgot to update this, but feel free to take some pointers from here if you are struggling with the documentation.

https://github.com/SanketDG/djurl/tree/master/modern-django-tooling#use-poetry-for-dependency-management-or-pip-tools

SanketDG · 2020-08-06T16:36:49Z

pyproject.toml

+    "Anita Borg",
+    "Poetry",
+    "Flask",
+    "Women in Technology"]


I have a hard time digesting poetry's suitable for just dependency management when using with Django, because of the way it enforces package information, which is largely redundant here for the most part.

This is why my mind says to go with pip-tools when working with Django, even though I myself really like Poetry! 😆

I don't have much experience with Django so you're probably right😁Idk how the situation looks like when using Flask

Oh I mean both Django/Flask, apologies, anything that is not a Python package, basically.

SanketDG · 2020-08-06T16:38:03Z

pyproject.toml

+urllib3 = "^1.22"
+wcwidth = "^0.1.7"
+websocket-client = "^0.48.0"
+Werkzeug = "^0.15.3"


Do we need all of the pacakages here? I am pretty sure some of them are actually sub deps of other packages, that have creep'd into here (more reason for doing this 🙂 )

I'll go through them and remove the subdeps:)

SanketDG · 2020-08-06T16:44:52Z

requirements.txt

@@ -1,74 +0,0 @@
-alembic==1.4.2


We do not need to exacly remove this file, this file can still be used, especially with Docker!

Accurate dependency resolution is a hard problem, and poetry does it really well, so it takes a toll on time required for doing poetry install, instead we can make sure to always export our dependencies to a requirements.txt file, which would contain pinned versions and hash checks.

We can then re-use this file for people who absolutely still cannot use poetry for some reason, and we should also use this file for things like CI and deployment.

Would simply auto-generating this file (during a e.g CI) be a good solution?

No, the dev would generate and commit it (there are even git commit hooks for that) whenever the dev adds/updates/removes a package.

But why? If it's the only purpose would be to be utilized by CI, then I think it's better to hide it.

The only reason is because it is simpler, there are enough issues on the Poetry repo about the dependency resolution taking ages sometimes, because it's a costly operation.

I was taking a shower when I realized that my question (above) is stupid and the reason is obvious. I agree totally with this.

SanketDG · 2020-08-06T16:45:34Z

scripts.py

@@ -0,0 +1,11 @@
+# This is a temporary workaround till Poetry supports scripts, see
+# https://github.com/sdispater/poetry/issues/241.


I wish this was done soon enough with poetry, but they have been prioritizing with other things.

SanketDG · 2020-08-06T16:46:34Z

scripts.py

+
+
+def test():
+  check_call(["python", "-m", "unittest", "discover", "tests"])


We could also add scripts for linting / autoformatting 🙂

SanketDG · 2020-08-06T16:48:28Z

README.md

-3. Install all the dependencies in `requirements.txt` file:
-`pip install -r requirements.txt`
+3. Activate the virtual environment (auto-crated by Poetry):
+`poetry shell`


We should also document briefly about poetry run or poetry shell, that would be helpful for people who still use virtualenv driven dev, so to speak 🙂

missing ending Co-authored-by: Isabel Costa <isabelcmdcosta@gmail.com>

bartekpacia · 2020-08-07T14:24:08Z

@isabelcosta @SanketDG Please review the newest changes:)

fixed scripts not working
remove all packages that I identified to be peer dependencies of other packages

bartekpacia · 2020-08-07T15:14:35Z

I also tested this with Python 3.8 and all tests pass, so I think it's a great idea to bump the recommended Python version to 3.8

pyproject.toml

SanketDG · 2020-08-18T18:04:38Z

python-poetry/poetry#366

Dropping this here, seems useful

Edit (for caching): python-poetry/poetry#366 (comment)

bartekpacia · 2020-08-19T20:34:25Z

Thank you @SanketDG. I'm busy right now but will surely find some time in the future to finish this PR

bartekpacia · 2020-10-11T18:01:29Z

I'd like to finish this issue (hey, it's Hacktoberfest!) as soon as I get my laptop back.

Just pinging so it doesn't get closed:)

isabelcosta · 2020-10-11T18:08:28Z

@bartekpacia ack

bartekpacia · 2020-10-18T20:30:25Z

a new PR #914 is up, so I'm closing this:)
Big thank you to everybody who commented and helped me here!

bartekpacia added 4 commits August 5, 2020 23:42

created pyproject.toml and migrated to it

358b3aa

removed requirements.txt and migrated Dockerfile to Poetry

683e9c2

updated README

2dc698d

add start and test scripts, but they don't work :(

7f8471e

isabelcosta changed the title ~~Use poetry~~ [WIP] Use poetry Aug 5, 2020

isabelcosta reviewed Aug 5, 2020

View reviewed changes

README.md Outdated Show resolved Hide resolved

isabelcosta reviewed Aug 5, 2020

View reviewed changes

README.md Outdated Show resolved Hide resolved

SanketDG reviewed Aug 6, 2020

View reviewed changes

bartekpacia and others added 4 commits August 7, 2020 15:22

Update README.md

2171c41

missing ending Co-authored-by: Isabel Costa <isabelcmdcosta@gmail.com>

add Isabel's email to pyproject.toml

6e414d5

add a few basic Poetry commands to README

0581f2d

fix scripts not working, remove some unnecessary packages

1691361

removed all peer dependencies

5692cba

bartekpacia force-pushed the use_poetry branch from a324ae5 to 5692cba Compare August 7, 2020 15:07

bump Python to 3.8

2fa729d

isabelcosta reviewed Aug 8, 2020

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

add AnitaB.org email to pyproject.toml

f3a6576

bartekpacia force-pushed the use_poetry branch from 52f9870 to f3a6576 Compare August 8, 2020 15:04

bartekpacia mentioned this pull request Oct 18, 2020

Use Poetry for package management #914

Closed

13 tasks

bartekpacia closed this Oct 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Use poetry #715

[WIP] Use poetry #715

bartekpacia commented Aug 5, 2020

bartekpacia commented Aug 5, 2020

SanketDG left a comment •

edited

Loading

SanketDG Aug 6, 2020

bartekpacia Aug 7, 2020

SanketDG Aug 7, 2020

SanketDG Aug 6, 2020

bartekpacia Aug 7, 2020

SanketDG Aug 6, 2020

bartekpacia Aug 7, 2020

SanketDG Aug 7, 2020

bartekpacia Aug 7, 2020

SanketDG Aug 7, 2020

bartekpacia Aug 8, 2020

SanketDG Aug 6, 2020

SanketDG Aug 6, 2020

SanketDG Aug 6, 2020

bartekpacia commented Aug 7, 2020 •

edited

Loading

bartekpacia commented Aug 7, 2020

SanketDG commented Aug 18, 2020 •

edited

Loading

bartekpacia commented Aug 19, 2020

bartekpacia commented Oct 11, 2020

isabelcosta commented Oct 11, 2020

bartekpacia commented Oct 18, 2020

		@@ -0,0 +1,11 @@
		# This is a temporary workaround till Poetry supports scripts, see
		# https://github.com/sdispater/poetry/issues/241.



		def test():
		check_call(["python", "-m", "unittest", "discover", "tests"])

[WIP] Use poetry #715

[WIP] Use poetry #715

Conversation

bartekpacia commented Aug 5, 2020

Description

Type of Change:

How Has This Been Tested?

Checklist:

bartekpacia commented Aug 5, 2020

SanketDG left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bartekpacia commented Aug 7, 2020 • edited Loading

bartekpacia commented Aug 7, 2020

SanketDG commented Aug 18, 2020 • edited Loading

bartekpacia commented Aug 19, 2020

bartekpacia commented Oct 11, 2020

isabelcosta commented Oct 11, 2020

bartekpacia commented Oct 18, 2020

SanketDG left a comment •

edited

Loading

bartekpacia commented Aug 7, 2020 •

edited

Loading

SanketDG commented Aug 18, 2020 •

edited

Loading