Failing to upload to JFrog Artifactory v1.5.x

[mitakora]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:

System
Platform: darwin
OS:       posix
Python:   /Users/xxx/.pyenv/versions/3.7.4

• Poetry version:

Poetry
Version: 1.0.3
Python:  3.7.4

• Link of a Gist with the contents of your pyproject.toml file:

[tool.poetry]
name = "xxx"
version = "0.1.0"
description = ""
authors = ["XXX"]

[tool.poetry.dependencies]
python = "^3.7"
kfp = "^0.2.5"

[tool.poetry.dev-dependencies]
pytest = "^3.0"
pylama = "^7.7.1"
isort = "^4.3.21"
black = "19.3b0"

[tool.black]
line-length = 120
target-version = ['py37']
exclude = '''
(
  /(
      \.eggs         # exclude a few common directories in the
    | \.git          # root of the project
    | \.hg
    | \.mypy_cache
    | \.tox
    | \.venv
    | _build
    | buck-out
    | build
    | dist
    | venv
    | docs
  )/
)
'''

[tool.isort]
known_third_party = "pytest,kfp"
line_length = 120
lines_between_sections = 1
multi_line_output = 3
include_trailing_comma = true
skip="__pycache__"


[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Issue

When executing

poetry publish --build -r internal -u ${ARTIFACTORY_USER} -p ${ARTIFACTORY_PASS} -vvv

I got this:

Using virtualenv: /Users/xx/yy/venv
Building xxx (0.1.0)
 - Building sdist
 - Adding: 
...
 - Built xxx-0.1.0.tar.gz

 - Building wheel
 - Adding:
....
 - Built xxx-0.1.0-py3-none-any.whl

Publishing xxx(0.1.0) to internal
 - Uploading xxx-0.1.0.tar.gz 100%

[UploadError]
HTTP Error 500: Internal Server Error

Traceback (most recent call last):
  File "/Users/yy/tools/poetry/1.0/lib/poetry/_vendor/py3.7/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/Users/yy/tools/poetry/1.0/lib/poetry/_vendor/py3.7/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/Users/yy/tools/poetry/1.0/lib/poetry/_vendor/py3.7/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/Users/yy/tools/poetry/1.0/lib/poetry/_vendor/py3.7/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/Users/yy/tools/poetry/1.0/lib/poetry/console/commands/publish.py", line 81, in handle
    client_cert,
  File "/Users/yy/tools/poetry/1.0/lib/poetry/masonry/publishing/publisher.py", line 96, in publish
    client_cert=resolved_client_cert,
  File "/Users/yy/tools/poetry/1.0/lib/poetry/masonry/publishing/uploader.py", line 110, in upload
    self._upload(session, url)
  File "/Users/yy/tools/poetry/1.0/lib/poetry/masonry/publishing/uploader.py", line 205, in _upload
    raise UploadError(e)

[mxab]

A 500 Error sounds like there is some bug in artifactory. Do you have access to the logs of the server?

[ToddG]

I am seeing a similar error.

Similar repo. In my case the error is:

415 Client Error: Unsupported Media Type for url: https://artifactory.my.url/artifactory/api/pypi/pypi-shared/simple/

[dhasek00]

I'm getting the same error as well on Poetry 1.0.5.
Trying to publish to Artifactory v5.3.0 rev 50045.

HTTP Error 500: Internal Server Error

I have no access to server side logs to provide.

EDIT:
I printed out the Requests POST response in poetry/masonry/publishing/uploader.py and it provided the 500 error: No enum constant org.jfrog.repomd.pypi.model.PypiMetadata.MetadataVersion.v2_1

Older versions of Artifactory are affected by this issue, resolved in v6.1.0.
https://www.jfrog.com/jira/browse/RTFACT-16189

[ToddG]

Ok. I was able to work around this by removing 'simple' from the path per this post: #708

some app 'bar'
notice the url does not have `/simple here'

poetry config repositories.foo https://artifactory.foo.com/artifactory/api/pypi/pypi-shared/
poetry publish --build -r foo

then in a dependent project, I can add this library via poetry

notice that the url ends with /simple here

//pyproject.toml

[[tool.poetry.source]]
name = "artifactory"
url = "https://artifactory/api/pypi/local-pypi/simple"

$ poetry add bar

[ToddG]

Worth noting, that this seems to be doc'd:

https://www.jfrog.com/confluence/display/JFROG/PyPI+Repositories

PyPI remote repository URL

You should not include /pypi or /simple in the the PyPI remote repository URL. These suffixes are added by Artifactory when accessing the remote repository.

If you use a custom PyPI remote repository, you need to make sure it has a simple index (directory listing style) accessible by <URL>/simple.

[mitakora]

Hello there

After further investigation, the version of our internal artifactory needs to use the old sdist format.

The workaround I've found was to:

• Build the package with poetry build
• Untar the package in the dist folder
• Go to the untared folder and then execute: poetry run python setup.py sdist upload -r internal.

The script I've built is the following:

#!/bin/sh

PACKAGE_NAME=my-package

ARTIFACT_VERSION=$(cat pyproject.toml|\
 grep '^version'|\
 cut -d \" -f2)

SDIST=${PACKAGE_NAME}-${ARTIFACT_VERSION}
TARBALL=${SDIST}.tar.gz


# local ~/.pypirc
echo "\n
[distutils]\n\
index-servers = internal\n\n\
[internal]\n\
repository: ${ARTIFACTORY_REPOSITORY}\n\
username: ${ARTIFACTORY_USERNAME}\n\
password: ${ARTIFACTORY_PASSWORD}\n\
" > ${HOME}/.pypirc 

# Building tarball
poetry build

# Untar to take setup.py
tar xvzf dist/${TARBALL} -C dist

# Pushing
cd dist/${SDIST}
poetry run python setup.py sdist upload -r internal

For now, is working until we migrate to a newer version of JFrog.

[colindean]

While others have gotten this working in #3052 and #3059, I kinda had it working but then it failed on me for reasons I don't understand.

My current solution is to define a repo at runtime inside of the Makefile I'm using:

# the source in pyproject.toml to publish to
PUBLISH_SOURCE = myinternalrepo

PUBLISH_URL ?= $(shell $(POETRY) run python -c "import toml; from urllib.parse import urljoin; print(urljoin(next(filter(lambda s: s['name'] == '$(PUBLISH_SOURCE)', toml.load('pyproject.toml')['tool']['poetry']['source']))['url'], '.'))")

.PHONY: publish
publish: ## Publish a build to the configured repo
	POETRY_REPOSITORIES_PUBLISH_URL=$(PUBLISH_URL) \
	$(POETRY) publish \
		--repository publish

My CI (Vela) runs it like this:

     - name: publish
        image: repo.example.com/python/poetry:latest
        secrets:
          - source: artifactory_username
            target: poetry_http_basic_publish_username
          - source: artifactory_password
            target: poetry_http_basic_publish_password
        ruleset:
          event:
            - tag
        commands:
          - poetry version $(git describe --tags)
          - make build publish

This feels pretty awkward. I wish that Poetry could either detect the need to lop off /simple when accessing Artifactory, that Artifactory was more forgiving with this, or at least that the Poetry user could specify a publish_url in pyproject.toml:

[[tool.poetry.source]]
name = "myinternalrepo"
url = "https://repo.example.com/artifactory/api/pypi/myinternalrepo/simple"
publish_url = "https://repo.example.com/artifactory/api/pypi/myinternalrepo"

[neersighted]

Closing for now as not a Poetry issue -- @colindean if you want to take your proposal and turn it into a concrete FR or a PR, please do!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.