Piptools compile fails with python 3.8

[cducrest]

piptools compile fails with error AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__' when ran with python3.8 and not with python3.6

Environment Versions

1. OS Type: Ubuntu
2. Python version: Python 3.8.8
3. pip version: pip 21.0.1
4. pip-tools version: pip-tools==5.5.0

Steps to replicate

I run python3.8 -m piptools compile --output-file=requirements.txt constraints.in setup.py with the folllowing setup.py

setup.py

import codecs
import os

from setuptools import find_packages, setup

here = os.path.abspath(os.path.dirname(__file__))

with codecs.open(os.path.join(here, "README.rst"), encoding="utf-8") as f:
    long_description = f.read()


setup(
    name="trustlines-relay",
    setup_requires=["setuptools_scm"],
    use_scm_version=True,
    description="Relay Server for the Trustlines Network",
    long_description=long_description,
    # The project's main homepage.
    url="https://github.com/trustlines-protocol/relay",
    # Author details
    author="Trustlines-Network",
    author_email="contact@brainbot.com",
    license="MIT",
    # See https://pypi.python.org/pypi?%3Aaction=list_classifiers
    classifiers=[
        # How mature is this project? Common values are
        #   3 - Alpha
        #   4 - Beta
        #   5 - Production/Stable
        "Development Status :: 2 - Pre-Alpha",
        # Indicate who your project is intended for
        "Intended Audience :: Developers",
        # Pick your license as you wish (should match "license" above)
        "License :: OSI Approved :: MIT License",
        # Specify the Python versions you support here. In particular, ensure
        # that you indicate whether you support Python 2, Python 3 or both.
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.6",
    ],
    # What does your project relate to?
    keywords="trustlines",
    packages=find_packages("src"),
    package_dir={"": "src"},
    install_requires=[
        "flask",
        "flask_restful",
        "flask_cors",
        "sentry-sdk[flask]",
        "webargs>=5.0,<6.0",
        "gevent",
        "web3>=5.0,<6.0",
        "networkx>=2.0",
        "trustlines-contracts-bin>=1.3.0,<1.4.0",
        "trustlines-contracts-deploy>=1.2.0,<1.3.0",
        "contract-deploy-tools>=0.9.1,<0.10.0",
        "sqlalchemy",
        "eth-utils",
        "tinyrpc",
        "gevent-websocket",
        "marshmallow>=3.0.0b7",
        "marshmallow-oneofschema>=2.0.0",
        "flask-sockets",
        "firebase-admin>=4.3,<5.0",
        "psycopg2",
        "psycogreen",
        "wrapt",
        "attrs",
        "click",
        "toml",
        "cachetools",
        "coverage",
    ],
    python_requires=">=3.6",
    entry_points={"console_scripts": ["tl-relay=relay.boot:main"]},
)

and constraints.in

constraints.in

werkzeug<1.0  # Seems to be incompatible with flask-sockets
contract-deploy-tools>=0.8.0   # to make requirements and dev-requirements compatible

I expect it to generate the requirements.txt file but it raises an error:

error

    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.8 /home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmppkdc6hvg
         cwd: /tmp/pip-resolver-a7b919j_/psycogreen
    Complete output (10 lines):
    Traceback (most recent call last):
      File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 108, in get_requires_for_build_wheel
        backend = _build_backend()
      File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 99, in _build_backend
        obj = getattr(obj, path_part)
    AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__'
    ----------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/__main__.py", line 17, in <module>
    cli()
  File "/home/oizo/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/oizo/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/oizo/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/oizo/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/oizo/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/oizo/.local/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/scripts/compile.py", line 452, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/resolver.py", line 178, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/resolver.py", line 275, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/resolver.py", line 385, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/repositories/local.py", line 76, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 247, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
  File "/home/oizo/.local/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 203, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 385, in _resolve_one
    dist = self._get_dist_for(req_to_install)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 337, in _get_dist_for
    dist = self.preparer.prepare_linked_requirement(req)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 457, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 500, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 66, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 39, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 97, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 177, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 274, in _call_hook
    self._subprocess_runner(
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 289, in runner
    call_subprocess(
  File "/home/oizo/.local/lib/python3.8/site-packages/pip/_internal/utils/subprocess.py", line 258, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /usr/bin/python3.8 /home/oizo/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmppkdc6hvg Check the logs for full command output.

If I run the same command with python3.6, the command runs through and the requirements are generated: python3.6 -m piptools compile --output-file=requirements.txt constraints.in setup.py.

[atugushev]

Hello @cducrest,

Thanks for the report. Is there a pyproject.toml in the project root? Could you also provide its content?

[cducrest]

The content of pyproject.toml is

[tool.black]
target-version = ['py36']

My current goal is to update from python 3.6 to 3.8 and later also support 3.9

[atugushev]

Thanks! Which version of setuptools?

[cducrest]

setuptools 54.0.0

[atugushev]

Could you check also this related pypa/setuptools#1694 (comment) issue?

[cducrest]

I checked the related issue and at first dismissed it (before posting here) because I am not using --system-site-packages or setuptools less than 40.8.0.
But I am indeed using virtualenv. If instead I create a new venv with python3.8 -m venv newvenv and install my dependencies in it, I can then compile the requirements successfully (I guess this is what I'll do from now on).

We can close this issue if you think it is not a problem coming from pip-tools.

[atugushev]

@cducrest could you reproduce the bug with the master branch?

pip install -e git+https://github.com/jazzband/pip-tools#egg=pip-tools

[cducrest]

It did work, I did:

• virtualenv -p python3.8 newvenv
• source newvenv/bin/activate
• pip install -U pip wheel setuptools (pip==21.0.1, wheel==0.36.2, setuptools=54.1.1)
• pip install -e git+https://github.com/jazzband/pip-tools#egg=pip-tools
• python3.8 -m piptools compile --output-file=requirements.txt constraints.in setup.py
  It ran through and generated the requirements as expected.

Thanks for the help

[atugushev]

Cool! I presume it was fixed by #1311. Thanks for the issue report!