[bug] Failure to resolve python-requires required by a python-requires when it only exists on a remote and not the local cache

[samuel-emrys]

Environment details

• Operating System+version: Arch Linux
• Compiler+version: clang 15
• Conan version: 2.0.3
• Python version: 3.11.1

I've also replicated this within a standard conanio docker environment.

Steps to reproduce

Replicate the failure

1. Create a blank project

$ mkdir pyrequire_bug && cd pyrequire_bug
$ conan new cmake_lib -d name=foo -d version=0.1.0

2. Apply the following diff to conanfile.py:

diff --git a/conanfile.py b/conanfile.py
index 745af9c..0355c59 100644
--- a/conanfile.py
+++ b/conanfile.py
@@ -20,6 +20,7 @@ class fooRecipe(ConanFile):
 
     # Sources are located in the same place as this recipe, copy them to the recipe
     exports_sources = "CMakeLists.txt", "src/*", "include/*"
+    python_requires = "cmake-python-deps/[>=0.3.0]@mtolympus/stable"
 
     def config_options(self):
         if self.settings.os == "Windows":

3. Add the mtolympus remote:

$ conan remote add mtolympus-conan https://mtolympus.jfrog.io/artifactory/api/conan/mtolympus-conan

4. Attempt to install

$ conan install .

At which point it fails with the error below.

Circumvent the failure

Now what does work, is if i try to install this as a package

$ conan install --requires cmake-python-deps/0.3.0@mtolympus/stable --build=missing

And then the python-requires is able to be resolved in the local cache so a subsequent conan install works:

$ conan install .

Logs

$ conan install .

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=15
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=15
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 88, in _resolve_py_requires
    py_require = self._cached_py_requires[resolved_ref]
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: cmake-python-deps/0.3.0@mtolympus/stable%1679910243.341

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 272, in main
    cli.run(args)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 171, in run
    command.run(self._conan_api, self._commands[command_argument].parser, args[0][1:])
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/command.py", line 157, in run
    info = self._method(conan_api, parser, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/commands/install.py", line 64, in install
    deps_graph = conan_api.graph.load_graph_consumer(path, args.name, args.version,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/api/subapi/graph.py", line 134, in load_graph_consumer
    root_node = self._load_root_consumer_conanfile(path, profile_host, profile_build,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/api/subapi/graph.py", line 28, in _load_root_consumer_conanfile
    conanfile = app.loader.load_consumer(path,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/loader.py", line 161, in load_consumer
    conanfile = self.load_named(conanfile_path, name, version, user, channel, graph_lock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/loader.py", line 98, in load_named
    conanfile, _ = self.load_basic_module(conanfile_path, graph_lock, remotes=remotes,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/loader.py", line 60, in load_basic_module
    self._pyreq_loader.load_py_requires(conanfile, self, graph_lock, remotes,
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 68, in load_py_requires
    py_requires = self._resolve_py_requires(py_requires_refs, graph_lock, loader, remotes,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 90, in _resolve_py_requires
    pyreq_conanfile = self._load_pyreq_conanfile(loader, graph_lock, resolved_ref,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 115, in _load_pyreq_conanfile
    conanfile, module = loader.load_basic_module(path, graph_lock, update, check_update)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/loader.py", line 60, in load_basic_module
    self._pyreq_loader.load_py_requires(conanfile, self, graph_lock, remotes,
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 68, in load_py_requires
    py_requires = self._resolve_py_requires(py_requires_refs, graph_lock, loader, remotes,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 86, in _resolve_py_requires
    resolved_ref = self._resolve_ref(requirement, graph_lock, remotes, update)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/python_requires.py", line 105, in _resolve_ref
    self._range_resolver.resolve(requirement, "py_require", remotes, update)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/range_resolver.py", line 34, in resolve
    remote_resolved_ref = self._resolve_remote(search_ref, version_range, remotes, update)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conans/client/graph/range_resolver.py", line 75, in _resolve_remote
    for remote in remotes:
TypeError: 'NoneType' object is not iterable

ERROR: 'NoneType' object is not iterable

[samuel-emrys]

The source code for the generator in question can be found here: https://github.com/samuel-emrys/cmake-python-deps

Also, I have a feeling that this might related to nested python_requires. I don't get the same issue when trying to install pyvenv/[>=0.2.2]@mtolympus/stable, which is a python_requires within cmake-python-deps. Actually, this issue is also resolved if I:

1. Ensure that pyvenv is already available in the local cache, i.e.

$ conan install --requires pyvenv/0.2.2@mtolympus/stable --build=missing
$ conan install .

or,
2. Add pyvenv as a co-requirement, i.e.

diff --git a/conanfile.py b/conanfile.py
index 745af9c..0355c59 100644
--- a/conanfile.py
+++ b/conanfile.py
@@ -20,6 +20,7 @@ class fooRecipe(ConanFile):
 
     # Sources are located in the same place as this recipe, copy them to the recipe
     exports_sources = "CMakeLists.txt", "src/*", "include/*"
+    python_requires = "cmake-python-deps/[>=0.3.0]@mtolympus/stable", "pyvenv/[>=0.2.2]@mtolympus/stable"

     def config_options(self):
         if self.settings.os == "Windows":

So it looks like the issue here is that the resolver isn't able to traverse the python_requires dependency tree where a dependency exists solely on a remote

[AbrilRBS]

Hi @samuel-emrys thanks for the detailed report and follow-up, it's really appreciated. I've identified a potential issue that might fix this (Does for a simplified test case locally for me) :)