Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build python pyproject package #1146

Closed
danmur97 opened this issue Sep 5, 2023 · 2 comments
Closed

Build python pyproject package #1146

danmur97 opened this issue Sep 5, 2023 · 2 comments

Comments

@danmur97
Copy link
Contributor

danmur97 commented Sep 5, 2023

Problem to solve

nixpkgs have functions that handle the construction of python packages and python environments i.e. buildPythonPackage, buildEnv. All the python packages from nixpkgs are built from these helper functions.

The core problems by using makePythonPypiEnvironment are:

  • is limited to only pypi dependencies (cannot include local deps)
  • does not safely integrate with other environments.
  • since packages are built using pypi and poetry they relay on packaged binaries bundled on the wheels (that can be a problem since binary dependencies are not handled entirely by nix)

It is true that all of this problems can be overtaken and "fixed" by modifying PATH and other env setup but this makes that:

  • using a python package that depends on binaries is not as straightforward as using the nixpkgs version of the package
  • integrating pypi-environments (e.g. dev and runtime) with local deps and nixpkgs python packages results in posible package version collisions. With nixpkgs buildEnv this can be detected using ignoreCollisions = false;.

Intended users

Python developers

Permissions and Security

Proposal

  • The idea is to enable the building of python pkgs and environments throw the nixpkgs standard functions.
  • Also is desirable to adjust makePythonPypiEnvironment to be usable by this new python builder

Test plan

What does success look like, and how can we measure that?

Links / references

/label ~"type::feature"

@dsalaza4
Copy link
Contributor

dsalaza4 commented Sep 5, 2023

Some thoughts regarding this:

  1. Provided packages for makePythonPypiEnvironment should be fetched from PyPi instead of relying on the ones built on nixpkgs in order to avoid compilations and having to deal with several nixpkgs versions.
  2. If local dependencies are an actual necessity, let's try to reuse the fetch logic used in (1) to fetch them.

danmur97 added a commit to danmur97/makes that referenced this issue Sep 5, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
danmur97 added a commit to danmur97/makes that referenced this issue Sep 5, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
danmur97 added a commit to danmur97/makes that referenced this issue Sep 5, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
danmur97 added a commit to danmur97/makes that referenced this issue Sep 5, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
danmur97 added a commit to danmur97/makes that referenced this issue Sep 5, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils

Signed-off-by: Daniel F. Murcia Rivera <danmur97@outlook.com>
danmur97 added a commit to danmur97/makes that referenced this issue Sep 6, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils

Signed-off-by: Daniel F. Murcia Rivera <danmur97@outlook.com>
danmur97 added a commit to danmur97/makes that referenced this issue Sep 6, 2023
danmur97 added a commit to danmur97/makes that referenced this issue Sep 6, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
- add docs and example

Signed-off-by: Daniel F. Murcia Rivera <danmur97@outlook.com>
danmur97 added a commit to danmur97/makes that referenced this issue Sep 6, 2023
- add make-python-pyproject buiding from nixpkgs functions
- add python-override-utils
- add docs and example

Signed-off-by: Daniel F. Murcia Rivera <danmur97@outlook.com>
dsalaza4 added a commit that referenced this issue Sep 6, 2023
feat(back): #1146 make python pyproject
@danmur97
Copy link
Contributor Author

integration support for pypi will be handled on this issue: #1161

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants