Version solving fails with PyTorch +cu118

[jamesWalker55]

• Poetry version: 1.4.2
• Python version: 3.10.7
• OS version and name: Windows 10
• pyproject.toml:

[tool.poetry]
name = "stalp"
version = "0.1.0"
description = ""
authors = ["James Walker <james.chunho@gmail.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"


[[tool.poetry.source]]
name = "pytorch-index"
url = "https://download.pytorch.org/whl/cu118"
default = true
secondary = false

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I'm trying to add the cu118 version of PyTorch to my dependencies with the cu118 version. I am following PyTorch's documentation under the CUDA 11.8 tab. The original pip command I am trying to follow is this:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

With Poetry, I first added PyTorch's cu118 source with:

poetry source add --default pytorch-index https://download.pytorch.org/whl/cu118

Then I try to install torch and related packages:

poetry add torch torchvision torchaudio

I get the following output:

❯ poetry add torch torchvision torchaudio -vvv
Loading configuration file C:\Users\James\AppData\Roaming\pypoetry\config.toml
Adding repository pytorch-index (https://download.pytorch.org/whl/cu118) and setting it as the default one
Deactivating the PyPI repository
Trying to detect current active python executable as specified in the config.
Found: C:\Users\James\AppData\Local\Programs\Python\Python310\python.exe
Using virtualenv: D:\Programming\STALP\.venv
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for download.pytorch.org
[urllib3.connectionpool] Starting new HTTPS connection (1): download.pytorch.org:443
[urllib3.connectionpool] https://download.pytorch.org:443 "GET /whl/cu118/torch/ HTTP/1.1" 200 13162
Source (pytorch-index): 30 packages found for torch *
Using version ^2.0.0+cu118 for torch
[urllib3.connectionpool] https://download.pytorch.org:443 "GET /whl/cu118/torchvision/ HTTP/1.1" 200 20342
Source (pytorch-index): 30 packages found for torchvision *
Using version ^0.15.1+cu118 for torchvision
[urllib3.connectionpool] https://download.pytorch.org:443 "GET /whl/cu118/torchaudio/ HTTP/1.1" 200 30741
Source (pytorch-index): 27 packages found for torchaudio *
Using version ^2.0.1+cu118 for torchaudio

Updating dependencies
Resolving dependencies...
   1: fact: stalp is 0.1.0
   1: derived: stalp
   1: fact: stalp depends on torch (^2.0.0+cu118)
   1: fact: stalp depends on torchvision (^0.15.1+cu118)
   1: fact: stalp depends on torchaudio (^2.0.1+cu118)
   1: selecting stalp (0.1.0)
   1: derived: torchaudio (>=2.0.1+cu118,<3.0.0)
   1: derived: torchvision (>=0.15.1+cu118,<0.16.0)
   1: derived: torch (>=2.0.0+cu118,<3.0.0)
Source (pytorch-index): 1 packages found for torchaudio >=2.0.1+cu118,<3.0.0
Source (pytorch-index): 1 packages found for torchvision >=0.15.1+cu118,<0.16.0
Source (pytorch-index): 1 packages found for torch >=2.0.0+cu118,<3.0.0
   1: fact: torchaudio (2.0.1+cu118) depends on torch (2.0.0)
   1: derived: not torchaudio (==2.0.1+cu118)
   1: fact: no versions of torchaudio match >2.0.1+cu118,<3.0.0
   1: conflict: no versions of torchaudio match >2.0.1+cu118,<3.0.0
   1: ! torchaudio (>2.0.1+cu118,<3.0.0) is partially satisfied by not torchaudio (==2.0.1+cu118)
   1: ! which is caused by "torchaudio (2.0.1+cu118) depends on torch (2.0.0)"
   1: ! thus: torchaudio (>=2.0.1+cu118,<3.0.0) requires torch (2.0.0)
   1: ! not torch (==2.0.0) is satisfied by torch (>=2.0.0+cu118,<3.0.0)
   1: ! which is caused by "stalp depends on torch (^2.0.0+cu118)"
   1: ! thus: torchaudio is forbidden
   1: ! torchaudio (>=2.0.1+cu118,<3.0.0) is satisfied by torchaudio (>=2.0.1+cu118,<3.0.0)
   1: ! which is caused by "stalp depends on torchaudio (^2.0.1+cu118)"
   1: ! thus: version solving failed
   1: Version solving took 0.008 seconds.
   1: Tried 1 solutions.

  Stack trace:

  4  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\puzzle\solver.py:157 in _solve
      155│
      156│         try:
    → 157│             result = resolve_version(self._package, self._provider)
      158│
      159│             packages = result.packages

  3  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\mixology\__init__.py:18 in resolve_version
       16│     solver = VersionSolver(root, provider)
       17│
    →  18│     return solver.solve()
       19│

  2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\mixology\version_solver.py:111 in solve
      109│             next: str | None = self._root.name
      110│             while next is not None:
    → 111│                 self._propagate(next)
      112│                 next = self._choose_package_version()
      113│

  1  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\mixology\version_solver.py:150 in _propagate
      148│                     # where that incompatibility will allow us to derive new assignments
      149│                     # that avoid the conflict.
    → 150│                     root_cause = self._resolve_conflict(incompatibility)
      151│
      152│                     # Back jumping erases all the assignments we did at the previous

  SolveFailure

  Because no versions of torchaudio match >2.0.1+cu118,<3.0.0
   and torchaudio (2.0.1+cu118) depends on torch (2.0.0), torchaudio (>=2.0.1+cu118,<3.0.0) requires torch (2.0.0).
  So, because stalp depends on both torch (^2.0.0+cu118) and torchaudio (^2.0.1+cu118), version solving failed.

  at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\mixology\version_solver.py:349 in _resolve_conflict
      345│             )
      346│             self._log(f'! which is caused by "{most_recent_satisfier.cause}"')
      347│             self._log(f"! thus: {incompatibility}")
      348│
    → 349│         raise SolveFailure(incompatibility)
      350│
      351│     def _choose_package_version(self) -> str | None:
      352│         """
      353│         Tries to select a version of a required package.

The following error occurred when trying to handle this error:


  Stack trace:

  11  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  10  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\application.py:190 in _run
       188│         self._load_plugins(io)
       189│
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│

   9  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│

   8  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│
       475│         return terminate_event.exit_code

   7  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\application.py:457 in _run_command
       455│
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   6  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\commands\base_command.py:119 in run
       117│         io.input.validate()
       118│
     → 119│         status_code = self.execute(io)
       120│
       121│         if status_code is None:

   5  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\cleo\commands\command.py:62 in execute
        60│
        61│         try:
     →  62│             return self.handle()
        63│         except KeyboardInterrupt:
        64│             return 1

   4  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\console\commands\add.py:262 in handle
       260│         self.installer.whitelist([r["name"] for r in requirements])
       261│
     → 262│         status = self.installer.run()
       263│
       264│         if status == 0 and not self.option("dry-run"):

   3  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\installer.py:116 in run
       114│             self._execute_operations = False
       115│
     → 116│         return self._do_install()
       117│
       118│     def dry_run(self, dry_run: bool = True) -> Installer:

   2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\installer.py:263 in _do_install
       261│                 source_root=self._env.path.joinpath("src")
       262│             ):
     → 263│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       264│         else:
       265│             self._io.write_line("Installing dependencies from lock file")

   1  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\puzzle\solver.py:74 in solve
        72│         with self._progress(), self._provider.use_latest_for(use_latest or []):
        73│             start = time.time()
     →  74│             packages, depths = self._solve()
        75│             end = time.time()
        76│

  SolverProblemError

  Because no versions of torchaudio match >2.0.1+cu118,<3.0.0
   and torchaudio (2.0.1+cu118) depends on torch (2.0.0), torchaudio (>=2.0.1+cu118,<3.0.0) requires torch (2.0.0).
  So, because stalp depends on both torch (^2.0.0+cu118) and torchaudio (^2.0.1+cu118), version solving failed.

  at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\puzzle\solver.py:163 in _solve
      159│             packages = result.packages
      160│         except OverrideNeeded as e:
      161│             return self._solve_in_compatibility_mode(e.overrides)
      162│         except SolveFailure as e:
    → 163│             raise SolverProblemError(e)
      164│
      165│         combined_nodes = depth_first_search(PackageNode(self._package, packages))
      166│         results = dict(aggregate_package_nodes(nodes) for nodes in combined_nodes)
      167│

[dimbleby]

what makes you think this is a bug? the error mesage tells you why solving fails.

[jamesWalker55]

The error should not occur. Installing the same package versions using pip works as expected without any errors, i.e. version 2.0.0 should match the version 2.0.0+cu118.

[dimbleby]

2.0.0 should match the version 2.0+cu118.

nope, 2.0.0 is strictly less than 2.0+cu118.

If 2.0.0 is good for you then provide that as a constraint. poetry add torch will always insist on the latest version of the package. (This isn't necessarily what you want, but #707 covers that so there's no need for a new issue saying the same thing)

[jamesWalker55]

Unfortunately 2.0.0 does not work for me. In PyTorch the cu118 versions are incompatible with non-cu118 versions. cu118 versions are compiled for the GPU, while the normal versions are compiled for the CPU. Installing both PyTorch 2.0.0 (for CPU) and torchaudio 2.0.1+cu118 (for GPU) would just cause an error on runtime.

[jamesWalker55]

For reference, here is the documentation from PyTorch where the above package install commands are derived from. The original command when run with pip would be:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

[dimbleby]

There are lots of duplicates for pytorch gpu stuff being difficult, afaik #6409 contains the most recent discussion (and is still an open issue), I'd try the things in there.

Please close

[jamesWalker55]

nope, 2.0.0 is strictly less than 2.0+cu118.

I believe this goes against PEP 440:

If the specified version identifier is a public version identifier (no local version label), then the local version label of any candidate versions MUST be ignored when matching versions.

[dimbleby]

If the specified version identifier is a public version identifier (no local version label),

but it isn't, because poetry add torch adds a dependency on ^2.0.0+cu118 (as the poetry output tells you)

[jamesWalker55]

I think I solved it. The proper way to install PyTorch cuda 11.8 on Poetry is with the following command:

poetry add torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1+cu118

The generated pyproject.toml now contains this:

[tool.poetry.dependencies]
python = "^3.10"
torch = "2.0.0+cu118"
torchvision = "0.15.1+cu118"
torchaudio = "2.0.1+cu118"

[I8dNLo]

For some reason this one does not really work for me:
Because _your_project_name_ depends on torchaudio (2.0.1+cu118) which doesn't match any versions, version solving failed.

[CoderTonyB]

In my .toml file, I added:

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu118"
priority = "explicit"

Then, I ran:

poetry add --source torch torch

and now my .toml file has

torch = {version = "^2.1.0+cu118", source = "torch"}

It was such a pain to get this working!

Previously, I was able to do:

poetry shell
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

which worked, but once I installed anything else, It would decide to "update" to 2.1.0 and remove the +118, which breaks cuda.

[thelinuxkid]

2.1.0+cu118

This worked for me. Thanks!!

[enricoGiga]

add:
[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu121"

or the correct url for you and it will work properly.

[Ashkf]

In my .toml file, I added:

[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu118"
priority = "explicit"

Then, I ran:

poetry add --source torch torch

and now my .toml file has

torch = {version = "^2.1.0+cu118", source = "torch"}

It was such a pain to get this working!

Previously, I was able to do:

poetry shell
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

which worked, but once I installed anything else, It would decide to "update" to 2.1.0 and remove the +118, which breaks cuda.

This successfully solved my problem.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.