Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up Python packaging #17

Merged
merged 3 commits into from
Jul 18, 2023
Merged

Set up Python packaging #17

merged 3 commits into from
Jul 18, 2023

Conversation

akx
Copy link
Contributor

@akx akx commented Jun 27, 2023

Sibling of Stability-AI/stablediffusion#269 but for the new fancy repo (and congrats on shipping)!

This makes it possible to build a standard Python .whl from the repo and use it instead of having to e.g. clone the entire repository.

Features

  • Adds __version__ to sgm/ (copying 0.0.1 from setup.py)
  • Sets up packaging using hatch (removing the defunct setup.py)
    • configs/ gets packaged as sgm/configs when using a packaged version
  • Adds sgm.get_configs_path() to make it easy for downstream users to find the configs path

Caveats

  • Unlike #269, I haven't tested this with a downstream package (because I can't summon weights from thin air 😁)
  • Requirements aren't yet handled here (because the current requirements* files are a bit messy), but I would recommend an approach where inference and training requirements are strictly separated, and training reqs would be installed with optional requirements.

The package seems to be installable and the configs are correctly shipped:

$ docker run -it -v (pwd):/a:ro python:3.11 bash
root@a8a617f480ff:/# pip install /a/dist/sgm-0.0.1-py3-none-any.whl
Processing /a/dist/sgm-0.0.1-py3-none-any.whl
Successfully installed sgm-0.0.1
root@a8a617f480ff:/# python
>>> import sgm
...
ModuleNotFoundError: No module named 'torchdata'
>>>
root@a8a617f480ff:/# ls -la /usr/local/lib/python3.11/site-packages/sgm/configs/
total 16
drwxr-xr-x 4 root root 4096 Jun 27 08:53 .
drwxr-xr-x 7 root root 4096 Jun 27 08:53 ..
drwxr-xr-x 4 root root 4096 Jun 27 08:53 example_training
drwxr-xr-x 2 root root 4096 Jun 27 08:53 inference

@benjaminaubin
Copy link
Contributor

Hello @akx ,
I am starting reviewing your PR

@akx
Copy link
Contributor Author

akx commented Jul 17, 2023

@benjaminaubin Cool - let me know if you have any questions.

Copy link
Contributor

@benjaminaubin benjaminaubin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akx thank you for this. Tested your branch, the package installed properly from the hatch wheel.

Could you

  • add in the documentation a section Packaging with commands pip install hatch hatch build . and pip3 install xxxx.whl
  • add **.pyc to gitignore
  • [Optional] split requirements in train / inference as suggested

@akx
Copy link
Contributor Author

akx commented Jul 18, 2023

@benjaminaubin Thank you 🙌

  • Rebased
  • Added *.py[cod] to .gitignore (and sorted the file 🧹)
  • Added a section on packaging with a note that dependencies are not yet specified for the built package
  • I'll take a look at the requirements in a subsequent PR, if that's okay? It will also change the general installation instructions (as it will become something like pip install -e .[train]).

@akx akx requested a review from benjaminaubin July 18, 2023 10:47
@benjaminaubin
Copy link
Contributor

Looks great to me.
Agreed requirements split can be done in another PR

@benjaminaubin benjaminaubin merged commit e5dc966 into Stability-AI:main Jul 18, 2023
@akx
Copy link
Contributor Author

akx commented Jul 18, 2023

@benjaminaubin Awesome, thanks! Can you take a look at #44? It's a requirement for being able to split requirements.

@tubylec8
Copy link

How to install it?

LinearFalcon pushed a commit to LinearFalcon/generative-models that referenced this pull request Jul 6, 2024
* Sort .gitignore; add dist and *.py[cod]

* Use pyproject.toml + Hatch instead of setup.py

Sibling of Stability-AI/stablediffusion#269

* Add packaging documentation
SevanBrodjian pushed a commit to SevanBrodjian/sd-latent-exploration that referenced this pull request Aug 12, 2024
* Sort .gitignore; add dist and *.py[cod]

* Use pyproject.toml + Hatch instead of setup.py

Sibling of Stability-AI/stablediffusion#269

* Add packaging documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants