Skip to content

Commit

Permalink
docs: add page to showcase supported dependency managers
Browse files Browse the repository at this point in the history
  • Loading branch information
mkniewallner committed Oct 16, 2024
1 parent 1210c0d commit 77e884b
Show file tree
Hide file tree
Showing 3 changed files with 212 additions and 37 deletions.
191 changes: 191 additions & 0 deletions docs/supported-dependency-managers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
# Supported dependency managers

While most dependency managers support the
standard [PEP 621 format](https://packaging.python.org/en/latest/specifications/pyproject-toml/) for defining
dependencies in `pyproject.toml`, not all of them do. Even those that do often provide additional ways to define
dependencies that are not standardised.

_deptry_ can extract dependencies from most of the package managers that support PEP
621 (e.g. [uv](https://docs.astral.sh/uv/), [PDM](https://pdm-project.org/en/latest/)), including tool-specific
extensions, but also from package managers that do not (or used to not) support PEP
621 (e.g. [Poetry](https://python-poetry.org/), [pip](https://pip.pypa.io/en/stable/reference/requirements-file-format/)).

## PEP 621

_deptry_ fully supports [PEP 621 standard](https://packaging.python.org/en/latest/specifications/pyproject-toml/), and
use the presence of a `[project]` section in `pyproject.toml` to determine that the project uses PEP 621.

By default, _deptry_ extracts, from `pyproject.toml`:

- regular dependencies from:
- `dependencies` entry under `[project]` section
- groups under `[project.optional-dependencies]` section
- development dependencies from groups under `[dependency-groups]` section

For instance, with this `pyproject.toml`:

```toml title="pyproject.toml"
[project]
name = "foo"
dependencies = ["orjson>=3.0.0"]

[project.optional-dependencies]
cli = ["click>=8.0.0"]
http = [
"httpx>=0.27.0",
"uvicorn>=0.32.0",
]

[dependency-groups]
docs = ["mkdocs==1.6.1"]
test = [
"pytest==8.3.3",
"pytest-cov==5.0.0",
]
```

the following dependencies will be extracted:

- regular dependencies: `orjson`, `click`, `httpx`, `uvicorn`
- development dependencies: `mkdocs`, `pytest`, `pytest-cov`

!!! note

Groups under `[project.optional-dependencies]` can be flagged as development dependency groups by
using [`--pep621-dev-dependency-groups`](usage.md#pep-621-dev-dependency-groups) argument (or its
`pep_621_dev_dependency_groups` equivalent in `pyproject.toml`).

### uv

Additionally to PEP 621 dependencies, _deptry_ will
extract [uv development dependencies](https://docs.astral.sh/uv/concepts/dependencies/#development-dependencies) from
`dev-dependencies` entry under `[tool.uv]` section, for instance:

```toml title="pyproject.toml"
[tool.uv]
dev-dependencies = [
"mkdocs==1.6.1",
"pytest==8.3.3",
"pytest-cov==5.0.0",
]
```

### PDM

Additionally to PEP 621 dependencies, _deptry_ will
extract [PDM development dependencies](https://pdm-project.org/en/latest/usage/dependency/#add-development-only-dependencies)
from `[tool.pdm.dev-dependencies]` section, for instance:

```toml title="pyproject.toml"
[tool.pdm.dev-dependencies]
docs = ["mkdocs==1.6.1"]
test = [
"pytest==8.3.3",
"pytest-cov==5.0.0",
]
```

### Setuptools

When using setuptools as a build backend, both `dependencies` and `optional-dependencies` can
be [dynamically read](https://setuptools.pypa.io/en/stable/userguide/pyproject_config.html#dynamic-metadata) from
`requirements.txt`-format files, for instance:

```toml title="pyproject.toml"
[build-backend]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "foo"
dynamic = ["dependencies", "optional-dependencies"]

[tool.setuptools.dynamic]
dependencies = { file = ["requirements.txt"] }

[tool.setuptools.dynamic.optional-dependencies]
cli = { file = ["cli-requirements.txt"] }
```

In this example, regular dependencies will be extracted from both `requirements.txt` and `cli-requirements.txt` files.

!!! note

Groups under `[tool.setuptools.dynamic.optional-dependencies]` can be flagged as development dependency groups by
using [`--pep621-dev-dependency-groups`](usage.md#pep-621-dev-dependency-groups) argument (or its
`pep_621_dev_dependency_groups` equivalent in `pyproject.toml`).

## Poetry

_deptry_ supports
extracting [dependencies defined using Poetry](https://python-poetry.org/docs/pyproject/#dependencies-and-dependency-groups),
and uses the presence of a `[tool.poetry.dependencies]` section in `pyproject.toml` to determine that the project uses
Poetry.

In a `pyproject.toml` file where Poetry is used, _deptry_ will extract:

- regular dependencies from entries under `[tool.poetry.dependencies]` section
- development dependencies from entries under each `[tool.poetry.group.<group>.dependencies]` section (or the
legacy `[tool.poetry.dev-dependencies]` section)

For instance, given the following `pyproject.toml` file:

```toml title="pyproject.toml"
[tool.poetry.dependencies]
python = "^3.10"
orjson = "^3.0.0"
click = { version = "^8.0.0", optional = true }

[tool.poetry.extras]
cli = ["click"]

[tool.poetry.group.docs.dependencies]
mkdocs = "1.6.1"

[tool.poetry.group.test.dependencies]
pytest = "8.3.3"
pytest-cov = "5.0.0"
```

the following dependencies will be extracted:

- regular dependencies: `orjson`, `click`
- development dependencies: `mkdocs`, `pytest`, `pytest-cov`

## `requirements.txt` (pip, pip-tools)

_deptry_ supports extracting [dependencies using
`requirements.txt` format](https://pip.pypa.io/en/stable/reference/requirements-file-format/), which is mostly used
by [pip](https://pip.pypa.io/en/stable/) and [pip-tools](https://pip-tools.readthedocs.io/en/stable/).

By default, _deptry_ will look for:

- regular dependencies in `requirements.txt` (or `requirements.in` if existing, assuming pip-tools is used)
- development dependencies in `dev-requirements.txt` and `requirements-dev.txt`

For instance, given the following `requirements.txt` file:

```python title="requirements.txt"
click>=8.0.0
orjson>=3.0.0
```

and the following `dev-requirements.txt` file:

```python title="dev-requirements.txt"
mkdocs==1.6.1
pytest==8.3.3
pytest-cov==5.0.0
```

the following dependencies will be extracted:

- regular dependencies: `click`, `orjson`
- development dependencies: `mkdocs`, `pytest`, `pytest-cov`

!!! note

If using different files for regular dependencies, [`--requirements-files`](usage.md#requirements-files) (or its
`requirements_files` equivalent in `pyproject.toml`) can be used to instruct _deptry_ about the requirements files
locations. Similarly, [`--requirements-files-dev`](usage.md#requirements-files-dev) (or its `requirements_files_dev`
equivalent in `pyproject.toml`) can be used for requirements files containing development dependencies.
57 changes: 20 additions & 37 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,36 +20,17 @@ If you want to configure _deptry_ using `pyproject.toml`, or if your dependencie

## Dependencies extraction

_deptry_ extracts dependencies into 2 separate groups:
_deptry_ can extract dependencies from a broad range of [dependency managers](supported-dependency-managers.md).

- "production" ones, meant to be used in the codebase
Dependencies are always extracted into two separate groups:

- regular ones, meant to be used in the codebase
- development ones

This is an important distinction, as development dependencies are usually meant to only be used outside the
codebase (for instance in tests, or as CLI tools for type-checking, formatting, etc.). For this reason, _deptry_ will
codebase (e.g. `pytest` to run tests, Mypy for type-checking, or Ruff for formatting). For this reason, _deptry_ will
not run [Unused dependencies (DEP002)](rules-violations.md#unused-dependencies-dep002) for development dependencies.

To determine the project's dependencies, _deptry_ will scan the directory it is run from for files in the following order:

1. If a `pyproject.toml` file with a `[tool.poetry.dependencies]` section is found, _deptry_ will assume it uses Poetry and extract:
- dependencies from `[tool.poetry.dependencies]` section
- development dependencies from `[tool.poetry.group.dev.dependencies]` or `[tool.poetry.dev-dependencies]` section
1. If a `pyproject.toml` file with a `[tool.pdm.dev-dependencies]` section is found, _deptry_ will assume it uses PDM and extract:
- dependencies from `[project.dependencies]` and `[project.optional-dependencies]` sections
- development dependencies from `[tool.pdm.dev-dependencies]` section and from the groups under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
1. If a `pyproject.toml` file with a `[tool.uv.dev-dependencies]` section is found, _deptry_ will assume it uses uv and extract:
- dependencies from `[project.dependencies]` and `[project.optional-dependencies]` sections
- development dependencies from `[tool.uv.dev-dependencies]` section and from the groups under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
1. If a `pyproject.toml` file with a `[project]` section is found, _deptry_ will assume it uses [PEP 621](https://peps.python.org/pep-0621/) for dependency specification and extract:
- dependencies from `[project.dependencies]` and `[project.optional-dependencies]`.
- development dependencies from the groups under `[dependency-groups]`, and the ones under `[project.optional-dependencies]` passed via the [`--pep621-dev-dependency-groups`](#pep-621-dev-dependency-groups) argument.
1. If a `requirements.in` or `requirements.txt` file is found, _deptry_ will:
- extract dependencies from that file.
- extract development dependencies from `dev-dependencies.txt` and `dependencies-dev.txt`, if any exist

_deptry_ can be configured to look for `pip` requirements files with other names or in other directories.
See [Requirements files](#requirements-files) and [Requirements files dev](#requirements-files-dev).

## Imports extraction

_deptry_ will search for imports in Python files (`*.py`, and `*.ipynb` unless [`--ignore-notebooks`](#ignore-notebooks)
Expand Down Expand Up @@ -454,30 +435,32 @@ deptry . --package-module-name-map "foo-python=foo,bar-python=bar"

#### PEP 621 dev dependency groups

PEP 621 does [not define](https://peps.python.org/pep-0621/#recommend-that-tools-put-development-related-dependencies-into-a-dev-extra) a standard convention for specifying development dependencies. However, deptry offers a mechanism to interpret specific optional dependency groups as development dependencies.
Historically, PEP 621
did [not define](https://peps.python.org/pep-0621/#recommend-that-tools-put-development-related-dependencies-into-a-dev-extra)
a standard convention for specifying development dependencies. [PEP 735](https://peps.python.org/pep-0735/) now covers
this, but in the meantime, several projects defined development dependencies under `[project.optional-dependencies]`.
_deptry_ offers a mechanism to interpret specific optional dependency groups as development dependencies.

By default, all dependencies under `[project.dependencies]` and `[project.optional-dependencies]` are extracted as regular dependencies. By using the `--pep621-dev-dependency-groups` argument, users can specify which groups defined under `[project.optional-dependencies]` should be treated as development dependencies instead. This is particularly useful for projects that adhere to PEP 621 but do not employ a separate build tool for declaring development dependencies.
By default, all dependencies under `[project.dependencies]` and `[project.optional-dependencies]` are extracted as
regular dependencies. By using the `--pep621-dev-dependency-groups` argument, users can specify which groups defined
under `[project.optional-dependencies]` should be treated as development dependencies instead. This is particularly
useful for projects that adhere to PEP 621 but do not employ a separate build tool for declaring development
dependencies.

For example, consider a project with the following `pyproject.toml`:

```toml
[project]
...
dependencies = [
"httpx",
]
dependencies = ["httpx"]
[project.optional-dependencies]
test = [
"pytest < 5.0.0",
]
plot = [
"matplotlib",
]
plot = ["matplotlib"]
test = ["pytest"]
```

By default, `httpx`, `pytest` and `matplotlib` are extracted as regular dependencies. By specifying `--pep621-dev-dependency-groups=test`,
the dependency `pytest` will be considered a development dependency instead.
By default, `httpx`, `matplotlib` and `pytest` are extracted as regular dependencies. By specifying
`--pep621-dev-dependency-groups=test`, `pytest` dependency will be treated as a development dependency instead.

- Type: `list[str]`
- Default: `[]`
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ nav:
- Home: index.md
- Usage and Configuration: usage.md
- Rules and Violations: rules-violations.md
- Supported dependency managers: supported-dependency-managers.md
- Changelog: CHANGELOG.md
- Contributing: contributing.md

Expand Down

0 comments on commit 77e884b

Please sign in to comment.