Can't export lockfiles with dependencies that add extras in dev-requirements

[a-recknagel]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian GNU/Linux 9 (stretch)

• Poetry version: 1.10

• Link of a Gist with the contents of my pyproject.toml file: here

• Failing command: poetry export -f requirements.txt

Issue

This is a very fresh bug. It's due to 1159148, in particular this line. As can be seen in the attached pyproject.toml gist, if a dependency is added as a bare in [poetry.dependencies] and with extras in [poetry.dev-dependencies], find_packages will at some point return an empty list. This leads to the following crash when trying to export a lockfile:

# poetry lock
Updating dependencies
Resolving dependencies... (0.3s)

Writing lock file
# poetry export -f requirements.txt -vvv

  Stack trace:

  6  /usr/local/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  5  /usr/local/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  4  /usr/local/lib/python3.7/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  3  /usr/local/lib/python3.7/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  2  /usr/local/lib/python3.7/site-packages/poetry/console/commands/export.py:73 in handle
      71│             dev=self.option("dev"),
      72│             extras=self.option("extras"),
    → 73│             with_credentials=self.option("with-credentials"),
      74│         )
      75│ 

  1  /usr/local/lib/python3.7/site-packages/poetry/utils/exporter.py:43 in export
       41│             dev=dev,
       42│             extras=extras,
    →  43│             with_credentials=with_credentials,
       44│         )
       45│ 

  IndexError

  list index out of range

  at /usr/local/lib/python3.7/site-packages/poetry/utils/exporter.py:76 in _export_requirements_txt
       72│             if not dev
       73│             else self._poetry.package.all_requires,
       74│             with_nested=True,
       75│         ):
    →  76│             package = repository.find_packages(dependency=dependency)[0]
       77│ 
       78│             # If a package is optional and we haven't opted in to it, continue
       79│             if package.optional and package.name not in extra_package_names:
       80│                 continue

Other things I tried:

• a different set of dependencies that had alpha releases (pyparsing v.3.0.2 with its "diagrams" extra) -> same bug
• extras in [poetry.dependencies] and bare dependency in [poetry.dev-dependencies] -> no bug

Pinging @abn, who might know more about this.

[tony]

FYI: If you need to downgrade temporarily while 1.1 stabilizes, see if poetry self update 1.0 works

[a-recknagel]

Thanks for the quick fix, I just tested the new master and can confirm that it works =)

[SpangleLabs]

I'm getting a slightly different failure when doing poetry lock with poetry 1.1.4, installing pyparsing 3.0.0a2 with diagrams extra, as you mentioned in your issue.

I get this, slightly baffling, error:

Updating dependencies
Resolving dependencies...

  FileNotFoundError

  [Errno 2] No such file or directory: 'C:\\ProgramData\\Anaconda3\\Lib\\venv\\scripts\\nt\\python.exe'

  at C:\ProgramData\Anaconda3\lib\shutil.py:120 in copyfile
       116│
       117│     if not follow_symlinks and os.path.islink(src):
       118│         os.symlink(os.readlink(src), dst)
       119│     else:
    →  120│         with open(src, 'rb') as fsrc:
       121│             with open(dst, 'wb') as fdst:
       122│                 copyfileobj(fsrc, fdst)
       123│     return dst
       124│

Did you check pyparsing+diagrams after this fix?

(My pyproject.toml, for context)

[sinoroc]

@joshcoales

Are you sure it is related to the current topic? What was the command you entered? Can you run the command again with more verbosity (poetry command -vvv ...) and post the output?

Bonus points (maybe it is unrelated):
I recently saw some issues when poetry is running in a conda environment (virtualenv and conda do not really mix so well apparently). Could you maybe try to remove conda (anaconda, miniconda) from the equation and see if you can still recreate the issue?

[SpangleLabs]

I'm not certain whether it is the same issue, no. I just saw that this issue was also discussing issues installing pyparsing with the diagrams extra, which is where I am encountering issues.

I've just been doing poetry lock. With the -vvv argument, I get this:
https://gist.github.com/joshcoales/2b77ba660596d094d2e0c0f7f7c0964c

I had some failed attempts to bypass anaconda's python install, I shall have another go

[SpangleLabs]

@sinoroc

Ah, okay, no issue once I managed to get a non-anaconda python installed. Strange.

[sinoroc]

@joshcoales Thanks for helping with the debug. For information I'll link these other tickets that seem to relate to virtualenv and conda.

• No module named 'virtualenv.seed.via_app_data' #3305
• File not found error when using poetry with conda on windows. #3297

If your problem persist, maybe open a new ticket with focus on virtualenv and conda. Maybe that is a good lead, maybe it is not. A bit difficult to say at this stage.

[SpangleLabs]

That's fair enough. It only seemed to happen with pyparsing+diagrams, but yeah. All seems good now, and I don't really use anaconda, so I'm happy to just uninstall that anyway.
Thanks for the help!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.