boto3 release process on PyPI is broken

[techtonik]

https://pypi.org/pypi/boto3/1.9.220/json doesn't correctly list botocore as a dependency. As a result https://libraries.io/pypi/boto3 and other tools are unable to detect those dependencies correctly.

[swetashre]

@techtonik - Thank you for your post. It seems like you are trying to use this API : https://warehouse.readthedocs.io/api-reference/json/#get--pypi--project_name--json and this api does not return dependency for that particular project. That's why the page you linked is not showing the dependency for boto3.

Botocore is listed as a dependency for boto3 and you can see here:
8f74a9d

Hope it helps and please let me know if you have any questions.

[techtonik]

Hi @swetashre. Does AWS have any kind of invisible infrastructure support program like https://gitcoin.co or Tidelift? Over the last two days I tracked the problem down to the following issues:

1. https://pypi.org (warehouse) reads dependencies from source and binary packages, but source package for boto3 doesn't contain dependency metadata (2). Luckily, binary package for boto3 contains the correct metadata. https://pypy.org doesn't have any logic to ensure that dependencies are filled correctly - the first uploaded package wins.

quick hack solution for boto3 to publish its dependencies is to upload binary package first. The release process is not documented, so I couldn't send a PR to fix that.

proper solution part 1 is to patch https://pypi.org to fill missing metadata from subsequently uploaded packages with the same version.

2. Another way to fix things is to make sure that dependency metadata is included in source package of boto3. If you're using python setup.py dist to create the package, then setuptools is the tool that needs to be patched.

proper solution part 2 is to implement missing part in setuptools.

[swetashre]

@techtonik - Thanks for sharing your output. Our release process first uploads the source package to pypi then the binary package. After uploading the source package pypi does not look for any dependency so evenif the binary package has all the dependency it never gets updated.

We could first publish the binary package as you suggested to update all those dependency. I would mark this as enhancement.

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. Because it has been longer than one year since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment to prevent automatic closure, or if the issue is already closed, please feel free to reopen it.

[nateprewitt]

Hi @techtonik, we've published today's release using a new process and the metadata is now available for awscli, boto3, and botocore. s3transfer doesn't release on a regular cadence, but it will also see the updates automatically in its next release.

Please let us know if you're still seeing any issues, otherwise we'll plan to close this soon. Thanks!

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.