pdm add failed in python 3.8

[k4ml]

• [*] I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

pdm add django

Actual behavior

pdm add django
Adding packages to default dependencies: django
✔ 🔒 Lock successful
Changes are written to pdm.lock.
Changes are written to pyproject.toml.
Synchronizing working set with lock file: 1 to add, 0 to update, 0 to remove

Installing the project as an editable package...
  ✖ Install pdmtest 0.0.0 failed
  See /tmp/pdm-install-ybwaqhax.log for detailed debug log.
  [BuildError]: Call command ['/usr/bin/python3.8', '-m', 'pip', 'install', '--prefix', '/tmp/pdm-build-env-riyb4ngo', '-r', '/tmp/pdm-build-reqs-ocm63zat.txt'] return non-zero status.

Expected behavior

pdm add django
Adding packages to default dependencies: django
✔ 🔒 Lock successful
Changes are written to pdm.lock.
Changes are written to pyproject.toml.
Synchronizing working set with lock file: 4 to add, 0 to update, 0 to remove

  ✔ Install asgiref 3.3.1 successful
  ✔ Install pytz 2020.5 successful
  ✔ Install django 3.1.5 successful
  ✔ Install sqlparse 0.4.1 successful

🎉 All complete!

Environment Information

Paste the output of pdm info && pdm info --env below:

pdm info
PDM version:        1.1.0                     
Python Interpreter: /usr/bin/python3.8 (3.8.2)
Project Root:       /home/kamal/pdmtest

pdm info --env
{
  "implementation_name": "cpython",
  "implementation_version": "3.8.2",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.4.0-1035-aws",
  "platform_system": "Linux",
  "platform_version": "#37~18.04.1-Ubuntu SMP Wed Jan 6 22:31:04 UTC 2021",
  "python_full_version": "3.8.2",
  "platform_python_implementaiton": "CPython",
  "python_version": "3.8",
  "sys_platform": "linux"
}

pdm add -v django
Adding packages to default dependencies: django
======== Start resolving requirements ========
	httpie<3.0.0,>=2.3.0
	django
	Adding requirement httpie<3.0.0,>=2.3.0
	Adding requirement django
	Adding requirement requests[socks]>=2.22.0(from httpie 2.3.0)
	Adding requirement Pygments>=2.5.2(from httpie 2.3.0)
	Adding requirement requests-toolbelt>=0.9.1(from httpie 2.3.0)
	Adding requirement colorama>=0.2.4; sys_platform == "win32"(from httpie 2.3.0)
Resolving: httpie 2.3.0
	New pin: httpie 2.3.0
======== Ending round 0 ========
	Adding requirement requests<3.0.0,>=2.0.1(from requests-toolbelt 0.9.1)
Resolving: requests-toolbelt 0.9.1
	New pin: requests-toolbelt 0.9.1
======== Ending round 1 ========
	Adding requirement chardet<5,>=3.0.2(from requests 2.25.1)
	Adding requirement idna<3,>=2.5(from requests 2.25.1)
	Adding requirement urllib3<1.27,>=1.21.1(from requests 2.25.1)
	Adding requirement certifi>=2017.4.17(from requests 2.25.1)
	Adding requirement PySocks!=1.5.7,>=1.5.6(from requests 2.25.1)
	Adding requirement requests==2.25.1(from requests 2.25.1)
Resolving: requests 2.25.1
	New pin: requests 2.25.1
======== Ending round 2 ========
	Adding requirement chardet<5,>=3.0.2(from requests 2.25.1)
	Adding requirement idna<3,>=2.5(from requests 2.25.1)
	Adding requirement urllib3<1.27,>=1.21.1(from requests 2.25.1)
	Adding requirement certifi>=2017.4.17(from requests 2.25.1)
Resolving: requests 2.25.1
	New pin: requests 2.25.1
======== Ending round 3 ========
Resolving: chardet 4.0.0
	New pin: chardet 4.0.0
======== Ending round 4 ========
Resolving: idna 2.10
	New pin: idna 2.10
======== Ending round 5 ========
Resolving: Pygments 2.7.4
	New pin: Pygments 2.7.4
======== Ending round 6 ========
Resolving: PySocks 1.7.1
	New pin: PySocks 1.7.1
======== Ending round 7 ========
Resolving: colorama 0.4.4
	New pin: colorama 0.4.4
======== Ending round 8 ========
Resolving: certifi 2020.12.5
	New pin: certifi 2020.12.5
======== Ending round 9 ========
Resolving: urllib3 1.26.2
	New pin: urllib3 1.26.2
======== Ending round 10 ========
	Adding requirement asgiref<4,>=3.2.10(from django 3.1.5)
	Adding requirement pytz(from django 3.1.5)
	Adding requirement sqlparse>=0.2.2(from django 3.1.5)
Resolving: django 3.1.5
	New pin: django 3.1.5
======== Ending round 11 ========
Resolving: asgiref 3.3.1
	New pin: asgiref 3.3.1
======== Ending round 12 ========
Resolving: sqlparse 0.4.1
	New pin: sqlparse 0.4.1
======== Ending round 13 ========
Resolving: pytz 2020.5
	New pin: pytz 2020.5
======== Ending round 14 ========
Finish resolving
======== Resolution Result ========
Stable pins:
             httpie 2.3.0
  requests-toolbelt 0.9.1
    requests[socks] 2.25.1
           requests 2.25.1
            chardet 4.0.0
               idna 2.10
           pygments 2.7.4
            pysocks 1.7.1
           colorama 0.4.4
            certifi 2020.12.5
            urllib3 1.26.2
             django 3.1.5
            asgiref 3.3.1
           sqlparse 0.4.1
               pytz 2020.5
Extracting package metadata
🔒 Lock successful
Changes are written to pdm.lock.
Changes are written to pyproject.toml.
Synchronizing working set with lock file: 1 to add, 0 to update, 0 to remove

Installing the project as an editable package...
Preparing isolated env for PEP 517 build...
ERROR: Can not combine '--user' and '--prefix' as they imply different installation locations
  Install pdmtest 0.0.0 failed
Traceback (most recent call last):
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/builders.py", line 85, in log_subprocessor
    subprocess.check_call(
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3.8', '-m', 'pip', 'install', '--prefix', '/tmp/pdm-build-env-xtdz7d_4', '-r', '/tmp/pdm-build-reqs-wuvlmjs_.txt']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kamal/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/core.py", line 73, in __call__
    return self.main(*args, **kwargs)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/core.py", line 112, in main
    raise err.with_traceback(traceback)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/core.py", line 108, in main
    f(options.project, options)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/cli/commands/add.py", line 35, in handle
    actions.do_add(
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/cli/actions.py", line 172, in do_add
    do_sync(
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/cli/actions.py", line 119, in do_sync
    handler.synchronize(clean=clean, dry_run=dry_run)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 315, in synchronize
    handlers[install_self[0]](install_self[1])
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 130, in install_candidate
    installer.install(can)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/installers/installers.py", line 44, in install
    candidate.get_metadata(allow_all_wheels=False)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/candidates.py", line 144, in get_metadata
    built = self.environment.build(ireq, self.hashes, allow_all_wheels)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/environment.py", line 363, in build
    ret = builder.build_egg_info(kwargs["build_dir"])
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/builders.py", line 266, in build_egg_info
    self.install(["setuptools"])
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/builders.py", line 243, in install
    self.subprocess_runner(cmd)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/builders.py", line 177, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/home/kamal/.local/pipx/venvs/pdm/lib/python3.8/site-packages/pdm/models/builders.py", line 93, in log_subprocessor
    raise BuildError(f"Call command {cmd} return non-zero status.")
pdm.exceptions.BuildError: Call command ['/usr/bin/python3.8', '-m', 'pip', 'install', '--prefix', '/tmp/pdm-build-env-xtdz7d_4', '-r', '/tmp/pdm-build-reqs-wuvlmjs_.txt'] return non-zero status.

My sys.path:-

>>> import sys
>>> sys.path
['', '/home/kamal/pdmtest', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/kamal/pdmtest/__pypackages__/3.8/lib', '/home/kamal/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']

My pyproject.toml:-

[project]
name = "pdmtest"
version = "0.0.0"
description = ""
authors = [
    {name = "Kamal Mustafa", email = "kamal@xxx.com"},
]
dependencies = [
    "httpie<3.0.0,>=2.3.0",
    "django<4.0.0,>=3.1.5",
]
dev-dependencies = []
requires-python = ">=3.8"
dynamic = ["classifiers"]
license = {text = "MIT"}

[project.urls]
homepage = ""

[tool]
[tool.pdm]

[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"

[frostming]

Did you set the PIP_USER env var? Or any config in pip.conf

[k4ml]

env | grep PIP_USER returns nothing.

ls $HOME/.config/pip/pip.conf
ls: cannot access '/home/kamal/.config/pip/pip.conf': No such file or directory
ls /home/kamal/.local/pipx/venvs/pip.conf
ls: cannot access '/home/kamal/.local/pipx/venvs/pip.conf': No such file or directory
ls $HOME/.local/pipx/venvs/pdm/pip.conf
ls: cannot access '/home/kamal/.local/pipx/venvs/pdm/pip.conf': No such file or directory

So I'm pretty sure I don't have such settings. But pipx was install with:-

python3.8 -mpip install --user pipx

[frostming]

You should also check /etc/pip.conf

Or you can try pip install --prefix ~/temp requests to see how it works

[k4ml]

Aah, turn out my pip also installed with --user. I have to remove that and then upgrade my system wide pip to latest version. Now pdm is working, althought it mentioned version mismatch:-

sudo python3.8 get-pip.py 
WARNING: The directory '/home/kamal/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
  Downloading pip-21.0-py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 23.2 MB/s 
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pdm 1.1.0 requires pip<21,>=20.1, but you have pip 21.0 which is incompatible.
Successfully installed pip-21.0

Maybe add some warning in the docs regarding this - make sure pip is global and not local (installed via --user flag)?