Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce sync command and deprecate install --sync #9801

Merged
merged 2 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 150 additions & 20 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,14 @@ poetry init
The `install` command reads the `pyproject.toml` file from the current project,
resolves the dependencies, and installs them.

{{% note %}}
Normally, you should prefer `poetry sync` to `poetry install` to avoid untracked outdated packages.
However, if you have set `virtualenvs.create = false` to install dependencies into your system environment,
which is discouraged, or `virtualenvs.options.system-site-packages = true` to make
system site-packages available in your virtual environment, you should use `poetry install`
because `poetry sync` will normally not work well in these cases.
{{% /note %}}

```bash
poetry install
```
Expand Down Expand Up @@ -186,21 +194,6 @@ poetry install --only-root
See [Dependency groups]({{< relref "managing-dependencies#dependency-groups" >}}) for more information
about dependency groups.

If you want to synchronize your environment – and ensure it matches the lock file – use the
`--sync` option.

```bash
poetry install --sync
```

The `--sync` can be combined with group-related options:

```bash
poetry install --without dev --sync
poetry install --with docs --sync
poetry install --only dev --sync
```

You can also specify the extras you want installed
by passing the `-E|--extras` option (See [Extras]({{< relref "pyproject#extras" >}}) for more info).
Pass `--all-extras` to install all defined extras for a project.
Expand All @@ -211,7 +204,7 @@ poetry install -E mysql -E pgsql
poetry install --all-extras
```

Extras are not sensitive to `--sync`. Any extras not specified will always be removed.
Any extras not specified will always be removed.

```bash
poetry install --extras "A B" # C is removed
Expand Down Expand Up @@ -258,7 +251,126 @@ poetry install --compile
* `--with`: The optional dependency groups to include.
* `--only`: The only dependency groups to include.
* `--only-root`: Install only the root project, exclude all dependencies.
* `--sync`: Synchronize the environment with the locked packages and the specified groups.
* `--sync`: Synchronize the environment with the locked packages and the specified groups. (**Deprecated**, use `poetry sync` instead)
* `--no-root`: Do not install the root package (your project).
* `--no-directory`: Skip all directory path dependencies (including transitive ones).
* `--dry-run`: Output the operations but do not execute anything (implicitly enables `--verbose`).
* `--extras (-E)`: Features to install (multiple values allowed).
* `--all-extras`: Install all extra features (conflicts with `--extras`).
* `--all-groups`: Install dependencies from all groups (conflicts with `--only`, `--with`, and `--without`).
* `--compile`: Compile Python source files to bytecode.

{{% note %}}
When `--only` is specified, `--with` and `--without` options are ignored.
{{% /note %}}


## sync

The `sync` command makes sure that the project's environment is in sync with the `poetry.lock` file.
It is similar to `poetry install` but it additionally removes packages that are not tracked in the lock file.

```bash
poetry sync
```

If there is a `poetry.lock` file in the current directory,
it will use the exact versions from there instead of resolving them.
This ensures that everyone using the library will get the same versions of the dependencies.

If there is no `poetry.lock` file, Poetry will create one after dependency resolution.

If you want to exclude one or more dependency groups for the installation, you can use
the `--without` option.

```bash
poetry sync --without test,docs
```

You can also select optional dependency groups with the `--with` option.

```bash
poetry sync --with test,docs
```

To install all dependency groups including the optional groups, use the ``--all-groups`` flag.

```bash
poetry sync --all-groups
```

It's also possible to only install specific dependency groups by using the `only` option.

```bash
poetry sync --only test,docs
```

To only install the project itself with no dependencies, use the `--only-root` flag.

```bash
poetry sync --only-root
```

See [Dependency groups]({{< relref "managing-dependencies#dependency-groups" >}}) for more information
about dependency groups.

You can also specify the extras you want installed
by passing the `-E|--extras` option (See [Extras]({{< relref "pyproject#extras" >}}) for more info).
Pass `--all-extras` to install all defined extras for a project.

```bash
poetry sync --extras "mysql pgsql"
poetry sync -E mysql -E pgsql
poetry sync --all-extras
```

Any extras not specified will always be removed.

```bash
poetry sync --extras "A B" # C is removed
```

By default `poetry` will install your project's package every time you run `sync`:

```bash
$ poetry sync
Installing dependencies from lock file

No dependencies to install or update

- Installing <your-package-name> (x.x.x)
```

If you want to skip this installation, use the `--no-root` option.

```bash
poetry sync --no-root
```

Similar to `--no-root` you can use `--no-directory` to skip directory path dependencies:

```bash
poetry sync --no-directory
```

This is mainly useful for caching in CI or when building Docker images. See the [FAQ entry]({{< relref "faq#poetry-busts-my-docker-cache-because-it-requires-me-to-copy-my-source-files-in-before-installing-3rd-party-dependencies" >}}) for more information on this option.

By default `poetry` does not compile Python source files to bytecode during installation.
This speeds up the installation process, but the first execution may take a little more
time because Python then compiles source files to bytecode automatically.
If you want to compile source files to bytecode during installation,
you can use the `--compile` option:

```bash
poetry sync --compile
```

### Options

* `--without`: The dependency groups to ignore.
* `--with`: The optional dependency groups to include.
* `--only`: The only dependency groups to include.
* `--only-root`: Install only the root project, exclude all dependencies.
* `--no-root`: Do not install the root package (your project).
* `--no-directory`: Skip all directory path dependencies (including transitive ones).
* `--dry-run`: Output the operations but do not execute anything (implicitly enables `--verbose`).
Expand Down Expand Up @@ -1024,16 +1136,34 @@ runtime environment.

{{% note %}}
The `self install` command works similar to the [`install` command](#install). However,
is different in that the packages managed are for Poetry's runtime environment.
it is different in that the packages managed are for Poetry's runtime environment.
{{% /note %}}

```bash
poetry self install --sync
poetry self install
```

#### Options

* `--sync`: Synchronize the environment with the locked packages and the specified groups. (**Deprecated**, use `poetry self sync` instead)
* `--dry-run`: Output the operations but do not execute anything (implicitly enables `--verbose`).

### self sync

The `self sync` command ensures all additional (and no other) packages specified
are installed in the current runtime environment.

{{% note %}}
The `self sync` command works similar to the [`sync` command](#sync). However,
it is different in that the packages managed are for Poetry's runtime environment.
{{% /note %}}

```bash
poetry self sync
```

#### Options

* `--sync`: Synchronize the environment with the locked packages and the specified groups.
* `--dry-run`: Output the operations but do not execute anything (implicitly enables `--verbose`).

## export
Expand Down
2 changes: 2 additions & 0 deletions src/poetry/console/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ def _load() -> Command:
"run",
"search",
"show",
"sync",
"update",
"version",
# Cache commands
Expand All @@ -85,6 +86,7 @@ def _load() -> Command:
"self update",
"self show",
"self show plugins",
"self sync",
# Source commands
"source add",
"source remove",
Expand Down
14 changes: 13 additions & 1 deletion src/poetry/console/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ class InstallCommand(InstallerCommand):
"sync",
None,
"Synchronize the environment with the locked packages and the specified"
" groups.",
" groups. (<warning>Deprecated</warning>)",
),
option(
"no-root", None, "Do not install the root package (the current project)."
Expand Down Expand Up @@ -89,6 +89,10 @@ def activated_groups(self) -> set[str]:
else:
return super().activated_groups

@property
def _alternative_sync_command(self) -> str:
return "poetry sync"

def handle(self) -> int:
from poetry.core.masonry.utils.module import ModuleOrPackageNotFoundError

Expand Down Expand Up @@ -147,6 +151,14 @@ def handle(self) -> int:
self.installer.extras(extras)

with_synchronization = self.option("sync")
if with_synchronization:
self.line_error(
"<warning>The `<fg=yellow;options=bold>--sync</>` option is"
" deprecated and slated for removal in the next minor release"
" after June 2025, use the"
f" `<fg=yellow;options=bold>{self._alternative_sync_command}</>`"
" command instead.</warning>"
)

self.installer.only_groups(self.activated_groups)
self.installer.skip_directory(self.option("no-directory"))
Expand Down
4 changes: 4 additions & 0 deletions src/poetry/console/commands/self/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,7 @@ class SelfInstallCommand(SelfCommand, InstallCommand):
@property
def activated_groups(self) -> set[str]:
return {MAIN_GROUP, self.default_group}

@property
def _alternative_sync_command(self) -> str:
return "poetry self sync"
31 changes: 31 additions & 0 deletions src/poetry/console/commands/self/sync.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from __future__ import annotations

from typing import TYPE_CHECKING
from typing import ClassVar

from poetry.console.commands.self.install import SelfInstallCommand


if TYPE_CHECKING:
from cleo.io.inputs.option import Option


class SelfSyncCommand(SelfInstallCommand):
name = "self sync"
description = (
"Sync Poetry's own environment according to the locked packages (incl. addons)"
" required by this Poetry installation."
)
options: ClassVar[list[Option]] = [
opt for opt in SelfInstallCommand.options if opt.name != "sync"
]
help = f"""\
The <c1>self sync</c1> command ensures all additional (and no other) packages \
specified are installed in the current runtime environment.

This is managed in the \
<comment>{SelfInstallCommand.get_default_system_pyproject_file()}</> file.

You can add more packages using the <c1>self add</c1> command and remove them using \
the <c1>self remove</c1> command.
"""
36 changes: 36 additions & 0 deletions src/poetry/console/commands/sync.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from __future__ import annotations

from typing import TYPE_CHECKING
from typing import ClassVar

from poetry.console.commands.install import InstallCommand


if TYPE_CHECKING:
from cleo.io.inputs.option import Option


class SyncCommand(InstallCommand):
name = "sync"
description = "Update the project's environment according to the lockfile."

options: ClassVar[list[Option]] = [
opt for opt in InstallCommand.options if opt.name != "sync"
Secrus marked this conversation as resolved.
Show resolved Hide resolved
]

help = """\
The <info>sync</info> command makes sure that the project's environment is in sync with
the <comment>poetry.lock</> file.
It is equivalent to running <info>poetry install --sync</info>.

<info>poetry sync</info>

By default, the above command will also install the current project. To install only the
dependencies and not including the current project, run the command with the
<info>--no-root</info> option like below:

<info> poetry sync --no-root</info>

If you want to use Poetry only for dependency management but not for packaging,
you can set the "package-mode" to false in your pyproject.toml file.
"""
21 changes: 19 additions & 2 deletions tests/console/commands/self/test_install.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,13 @@


@pytest.fixture
def tester(command_tester_factory: CommandTesterFactory) -> CommandTester:
return command_tester_factory("self install")
def command() -> str:
return "self install"


@pytest.fixture
def tester(command_tester_factory: CommandTesterFactory, command: str) -> CommandTester:
return command_tester_factory(command)


@pytest.mark.parametrize(
Expand Down Expand Up @@ -60,3 +65,15 @@ def test_self_install(

assert tester.io.fetch_output() == expected_output
assert tester.io.fetch_error() == ""


@pytest.mark.parametrize("sync", [True, False])
def test_sync_deprecation(tester: CommandTester, sync: bool) -> None:
tester.execute("--sync" if sync else "")

error = tester.io.fetch_error()
if sync:
assert "deprecated" in error
assert "poetry self sync" in error
else:
assert error == ""
Loading
Loading