Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create features and environments from extras #1077

Merged
merged 22 commits into from
Apr 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
9191a03
Create features and environments from extras
olivier-lacroix Mar 28, 2024
4728043
Handle self-refencing extras
olivier-lacroix Mar 29, 2024
b6c3f35
Add documentation
olivier-lacroix Mar 29, 2024
46fee3d
lint
olivier-lacroix Mar 29, 2024
bbe7d9e
Simplify project templates
olivier-lacroix Mar 31, 2024
551f627
Add a "default" SolveGroup to the default Environment (if not redefined)
olivier-lacroix Apr 1, 2024
6ed29fb
Merge remote-tracking branch 'upstream/main' into pyproject-extras
olivier-lacroix Apr 2, 2024
5fe6b89
Move environment extraction from extras to pyproject.rs
olivier-lacroix Apr 2, 2024
c16dea9
Merge branch 'main' into pyproject-extras
olivier-lacroix Apr 2, 2024
1959dfd
Merge branch 'main' into pyproject-extras
olivier-lacroix Apr 2, 2024
b7790b9
Merge branch 'main' into pyproject-extras
ruben-arts Apr 4, 2024
bd7a8fc
Update docs/advanced/pyproject_toml.md
olivier-lacroix Apr 4, 2024
9077bdd
Revert "Add a "default" SolveGroup to the default Environment (if not…
olivier-lacroix Apr 4, 2024
8ca31db
Merge remote-tracking branch 'upstream/main' into pyproject-extras
olivier-lacroix Apr 4, 2024
3f116d9
refactor parsing and display info message
olivier-lacroix Apr 5, 2024
b4c6e9d
revert test changes
olivier-lacroix Apr 5, 2024
1c2f5f1
lint
olivier-lacroix Apr 5, 2024
54d6772
Refine message, and update flask example
olivier-lacroix Apr 5, 2024
d09885a
Simplify print
olivier-lacroix Apr 5, 2024
05650ee
Add a message when not runnning init
olivier-lacroix Apr 5, 2024
d6e9833
cleanup
olivier-lacroix Apr 5, 2024
0ee0411
Filter out unused features
olivier-lacroix Apr 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/advanced/pyproject_toml.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,49 @@ matplotlib = "*"
This would result in the conda dependencies being installed and the pypi dependencies being ignored.
As pixi takes the conda dependencies over the pypi dependencies.

## Optional dependencies
If your python project includes groups of optional dependencies, pixi will automatically interpret them as [pixi features](../configuration.md#the-feature-table) of the same name with the associated `pypi-dependencies`.

You can add them to pixi environments manually, or use `pixi init` to setup the project, which will create one environment per feature. Self-references to other groups of optional dependencies are also handled.

For instance, imagine you have a project folder with a `pyproject.toml` file similar to:
```toml
[project]
name = "my_project"
dependencies = ["package1"]

[project.optional-dependencies]
test = ["pytest"]
all = ["package2","my_project[test]"]
```

Running `pixi init` in that project folder will transform the `pyproject.toml` file into:
```toml
[project]
name = "my_project"
dependencies = ["package1"]

[project.optional-dependencies]
test = ["pytest"]
all = ["package2","my_project[test]"]

[tool.pixi.project]
channels = ["conda-forge"]
platforms = ["linux-64"] # if executed on linux

[tool.pixi.environments]
default = {features = [], solve-group = "default"}
test = {features = ["test"], solve-group = "default"}
all = {features = ["all", "test"], solve-group = "default"}
```
In this example, three environments will be created by pixi:

- **default** with 'package1' as pypi dependency
- **test** with 'package1' and 'pytest' as pypi dependencies
- **all** with 'package1', 'package2' and 'pytest' as pypi dependencies
Comment on lines +143 to +145
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the above, is it correct that installing the test environment with pixi would install pytest from pypi rather than conda-forge?

If I wanted my pixi test environment to only install dependencies from conda-forge can I specify that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dhirschfeld you are correct.

Currently, you would have to manually add them to the tool.pixi.feature.test.dependencies table. What you want is also on my wishlist :-) #532

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, if I did manually specify the test dependencies in [tool.pixi.feature.test.dependencies] table, then that would be used instead of mapping pypi dependencies from the [project.optional-dependencies] table?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Pixi currently solves pypi dependencies after the conda ones. And if it detects a package is already provided by one of the conda packages requested, it is skipped.


All environments will be solved together, as indicated by the common `solve-group`, and added to the lock file. You can edit the `[tool.pixi.environments]` section manually to adapt it to your use case (e.g. if you do not need a particular environment).

## Example
As the `pyproject.toml` file supports the full pixi spec with `[tool.pixi]` prepended an example would look like this:
```toml title="pyproject.toml"
Expand Down
2 changes: 2 additions & 0 deletions examples/flask-hello-world-pyproject/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML
2 changes: 2 additions & 0 deletions examples/flask-hello-world-pyproject/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# pixi environments
.pixi
*.egg-info
119 changes: 111 additions & 8 deletions examples/flask-hello-world-pyproject/pixi.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 10 additions & 1 deletion examples/flask-hello-world-pyproject/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ version = "0.1.0"
description = "Example how to get started with flask in a pixi environment."
readme = "README.md"
requires-python = ">=3.11"
dependencies = ["flask==2.*", "pytest"]
dependencies = ["flask==2.*"]

[project.optional-dependencies]
test = ["pytest"]

[build-system]
requires = ["setuptools", "wheel"]
Expand All @@ -17,6 +20,12 @@ platforms = ["linux-64", "osx-arm64", "osx-64", "win-64"]
[tool.pixi.pypi-dependencies]
flask-hello-world-pyproject = { path = ".", editable = true }

[tool.pixi.environments]
default = { features = [], solve-group = "default" }
test = { features = ["test"], solve-group = "default" }

[tool.pixi.tasks]
start = "python -m flask --app flask_hello_world_pyproject.app:app run --port=5050"

[tool.pixi.feature.test.tasks]
test = "pytest -v tests/*"
Loading
Loading