Poetry 1.1.0 not able to use repository environment variable with private feed

[ghost]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian GNU/Linux 10

• Poetry version: 1.1.0

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/andrew-zimmerman-rseg/40c7ed84fe8ab521ef2c2b6c3e7ffcf0

Issue

Noticed this originally on our CI tool of Azure Pipelines using internal artifact feed but able to reproduce otherwise. It seems that with the change to poetry 1.1.0 we are unable to set our repository config as environment variables.

Failure on 1.1.0:

root@cdb6ebebc14e:/src# poetry --version
Poetry version 1.1.0
root@cdb6ebebc14e:/src# export POETRY_REPOSITORIES_FEED=https://feed.pkgs.visualstudio.com/_packaging/feed/pypi/simple/
root@cdb6ebebc14e:/src# export POETRY_HTTP_BASIC_FEED_USERNAME=awesomeuser
root@cdb6ebebc14e:/src# export POETRY_HTTP_BASIC_FEED_PASSWORD=coolpassword
root@cdb6ebebc14e:/src# poetry install -vvv
  Stack trace:
  12  /usr/local/lib/python3.6/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1
  11  /usr/local/lib/python3.6/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():
  10  /usr/local/lib/python3.6/site-packages/clikit/api/command/command.py:163 in _do_handle
       161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
       162│             event = PreHandleEvent(args, io, self)
     → 163│             self._dispatcher.dispatch(PRE_HANDLE, event)
       164│
       165│             if event.is_handled():
   9  /usr/local/lib/python3.6/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
        20│
        21│         if listeners:
     →  22│             self._do_dispatch(listeners, event_name, event)
        23│
        24│         return event
   8  /usr/local/lib/python3.6/site-packages/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
        87│                 break
        88│
     →  89│             listener(event, event_name, self)
        90│
        91│     def _sort_listeners(self, event_name):  # type: (str) -> None
   7  /usr/local/lib/python3.6/site-packages/poetry/console/config/application_config.py:116 in set_env
       114│
       115│         io = event.io
     → 116│         poetry = command.poetry
       117│
       118│         env_manager = EnvManager(poetry)
   6  /usr/local/lib/python3.6/site-packages/poetry/console/commands/command.py:10 in poetry
        8│     @property
        9│     def poetry(self):
     → 10│         return self.application.poetry
       11│
       12│     def reset_poetry(self):  # type: () -> None
   5  /usr/local/lib/python3.6/site-packages/poetry/console/application.py:69 in poetry
        67│             return self._poetry
        68│
     →  69│         self._poetry = Factory().create_poetry(Path.cwd())
        70│
        71│         return self._poetry
   4  /usr/local/lib/python3.6/site-packages/poetry/factory.py:72 in create_poetry
        70│         # Configuring sources
        71│         for source in poetry.local_config.get("source", []):
     →  72│             repository = self.create_legacy_repository(source, config)
        73│             is_default = source.get("default", False)
        74│             is_secondary = source.get("secondary", False)
   3  /usr/local/lib/python3.6/site-packages/poetry/factory.py:156 in create_legacy_repository
       154│             config=auth_config,
       155│             cert=get_cert(auth_config, name),
     → 156│             client_cert=get_client_cert(auth_config, name),
       157│         )
       158│
   2  /usr/local/lib/python3.6/site-packages/poetry/repositories/legacy_repository.py:195 in __init__
       193│         )
       194│
     → 195│         username, password = self._authenticator.get_credentials_for_url(self._url)
       196│         if username is not None and password is not None:
       197│             self._authenticator.session.auth = requests.auth.HTTPBasicAuth(
   1  /usr/local/lib/python3.6/site-packages/poetry/installation/authenticator.py:118 in get_credentials_for_url
       116│         if credentials == (None, None):
       117│             if "@" not in netloc:
     → 118│                 credentials = self._get_credentials_for_netloc_from_config(netloc)
       119│             else:
       120│                 # Split from the right because that's how urllib.parse.urlsplit()
  AttributeError
  'str' object has no attribute 'get'
  at /usr/local/lib/python3.6/site-packages/poetry/installation/authenticator.py:155 in _get_credentials_for_netloc_from_config
      151│             )
      152│             if not repository_config:
      153│                 continue
      154│
    → 155│             url = repository_config.get("url")
      156│             if not url:
      157│                 continue
      158│
      159│             parsed_url = urlparse.urlsplit(url)

Successful example on poetry 1.0.10:

root@431cad0df37f:/src# poetry --version
Poetry version 1.0.10
root@431cad0df37f:/src# export POETRY_REPOSITORIES_FEED=https://feed.pkgs.visualstudio.com/_packaging/feed/pypi/simple/
root@431cad0df37f:/src# export POETRY_HTTP_BASIC_FEED_USERNAME=aweseomeuser
root@431cad0df37f:/src# export POETRY_HTTP_BASIC_FEED_PASSWORD=coolpassword
root@431cad0df37f:/src# poetry install
Installing dependencies from lock file
Package operations: 53 installs, 0 updates, 0 removals
  - Installing zipp (3.1.0)
  - Installing importlib-metadata (1.7.0)
  - Installing lazy-object-proxy (1.4.3)
  - Installing pyparsing (2.4.7)
  - Installing six (1.15.0)
  - Installing typed-ast (1.4.1)
  - Installing wrapt (1.12.1)
  - Installing astroid (2.4.2)
  - Installing attrs (19.3.0)
  - Installing isort (4.3.21)
  - Installing locket (0.2.0)
  - Installing mccabe (0.6.1)
  - Installing more-itertools (8.4.0)
  - Installing packaging (20.4)
  - Installing pluggy (0.13.1)
  - Installing py (1.9.0)
  - Installing pycodestyle (2.6.0)
  - Installing pyflakes (2.2.0)
  - Installing toml (0.10.1)
  - Installing toolz (0.10.0)
  - Installing wcwidth (0.2.5)
  - Installing appdirs (1.4.4)
  - Installing argparse (1.4.0)
  - Installing click (7.1.2)
  - Installing cloudpickle (1.5.0)
  - Installing coverage (5.2.1)
  - Installing entrypoints (0.3)
  - Installing flake8 (3.8.3)
  - Installing fsspec (0.8.0)
  - Installing numpy (1.15.4)
  - Installing ordereddict (1.1)
  - Installing partd (1.1.0)
  - Installing pathspec (0.8.0)
  - Installing psutil (5.7.2)
  - Installing pygments (2.6.1)
  - Installing pylint (2.5.3)
  - Installing pytest (5.4.3)
  - Installing python-dateutil (2.8.1)
  - Installing pytz (2020.1)
  - Installing pyyaml (5.2)
  - Installing regex (2020.7.14)
  - Installing termcolor (1.1.0)
  - Installing urllib3 (1.25.10)
  - Installing adr-tools-python (1.0.3)
  - Installing black (19.10b0)
  - Installing dask (2.23.0)
  - Installing flakehell (0.5.0)
  - Installing lasio (0.23)
  - Installing oyaml (0.9)
  - Installing pandas (0.23.4)
  - Installing pyarrow (0.17.1)
  - Installing pytest-cov (2.10.1)
  - Installing taskipy (1.3.0)

[willemt]

I have the same issue. I removed POETRY_REPOSITORIES_XXX and added the following to my pyproject.toml as a workaround (I still use POETRY_HTTP_BASIC_XXX_USERNAME, POETRY_HTTP_BASIC_XXX_PASSWORD succesfully), thus it looks like the bug affects POETRY_REPOSITORIES_XXX (and POETRY_REPOSITORIES_XXX_URL) soley.

[[tool.poetry.source]]
name = "XXX"
url = "https://XXX/dev/simple"

[abn]

@andrew-zimmerman-rseg I believe that your configuration is incorrect. You need to be using POETRY_REPOSITORY_XXX_URL not POETRY_REPOSIOTRY_XXX.

$ env | grep POETRY
POETRY_HTTP_BASIC_FEED_USERNAME=awesomeuser
POETRY_REPOSITORIES_FEED_URL=https://feed.pkgs.visualstudio.com/_packaging/feed/pypi/simple/
POETRY_HTTP_BASIC_FEED_PASSWORD=coolpassword
$ poetry@1.1.0 new foobar
Created package foobar in foobar
$ cd foobar/
$ poetry@1.1.0 publish 
No files to publish. Run poetry build first or use the --build option.
$ poetry@1.1.0 publish --build -r feed
Creating virtualenv foobar in /tmp/foobar/.venv
Building foobar (0.1.0)
  - Building sdist
  - Built foobar-0.1.0.tar.gz
  - Building wheel
  - Built foobar-0.1.0-py3-none-any.whl

Publishing foobar (0.1.0) to feed
 - Uploading foobar-0.1.0-py3-none-any.whl 100%

  UploadError

  HTTP Error 405: Method Not Allowed

  at ~/.local/pipx/venvs/poetry@1.1.0/lib64/python3.8/site-packages/poetry/publishing/uploader.py:216 in _upload
      212│                     self._register(session, url)
      213│                 except HTTPError as e:
      214│                     raise UploadError(e)
      215│ 
    → 216│             raise UploadError(e)
      217│ 
      218│     def _do_upload(
      219│         self, session, url, dry_run=False
      220│     ):  # type: (requests.Session, str, Optional[bool]) -> None

The 405 is is because the URL you have used cannot be used for publishing, that should be something like this.

https://pkgs.dev.azure.com/<your-organization-name>/_packaging/<your-feed-name>/pypi/upload

Reminder that POETRY_REPOSITORIES_XXX_URL configuration is for publishing. The source section, as indicated by @willemt, in your pyproject is what you must use for installing packages from your feed.

I suspect that in 1.0.10 this was working because you already had apip.conf somewhere that added this as a global index. As of 1.1.0 poetry no longer uses pip for discovery and download depednecy artifacts. This is now handled internally by Poetry and hence pip's index configuration is ignored.

Also, note that there is a known bug (#3052) when the source and publish url use the same name. This will be resolved in 1.1.1.

[ghost]

It does look like my issue was due to setting the environment variable like POETRY_REPOSIOTRY_XXX and not the correct way of POETRY_REPOSITORY_XXX_URL.

Does it make sense to make note of this in the documentation? In reading through https://python-poetry.org/docs/configuration/#using-environment-variables it would not be obvious to me that I would need to append "_URL" to this environment variable (though it's entirely possible I'm missing something obvious here 😄). But the setting is repositories.<name>: string, with no url attached.

Really appreciate the help getting me straightened out here and all of the hard work being done on this project.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.