pdm import -f setuppy => ModuleNotFoundError: No module named 'setuptools'

[dev590t]

• I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

run pdm import -f setuppy noExistingFile -v in empty dir

Actual behavior

Traceback (most recent call last):
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/parse_setup.py", line 211, in <module>
    print(json.dumps(parse_setup(sys.argv[1])))
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/parse_setup.py", line 174, in parse_setup
    import setuptools
ModuleNotFoundError: No module named 'setuptools'
Traceback (most recent call last):
  File "/home/dev_1/.local/bin/pdm", line 10, in <module>
    sys.exit(main())
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 254, in main
    return Core().main(args)
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 187, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 182, in main
    f(options.project, options)
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/commands/import_cmd.py", line 35, in handle
    do_import(project, options.filename, options.format, options)
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/actions.py", line 734, in do_import
    project_data, settings = FORMATS[key].convert(project, filename, options)
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/formats/setup_py.py", line 18, in convert
    parsed = parse_setup_py(
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/__init__.py", line 43, in parse_setup_py
    return json.loads(subprocess.check_output(cmd))
  File "/gnu/store/abznl1yzi6isa57mh1mj3fsdylksqd7m-python-3.8.2/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/gnu/store/abznl1yzi6isa57mh1mj3fsdylksqd7m-python-3.8.2/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/test/.venv/bin/python', '-Es', '/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/parse_setup.py', 'noExistingFile']' returned non-zero exit status 1.

Expected behavior

Environment Information

# Paste the output of `pdm info && pdm info --env` below:

$ pdm info && pdm info --env
PDM version:
2.1.3
Python Interpreter:
/tmp/test/.venv/bin/python (3.7)
Project Root:
/tmp/test
Project Packages:
None
{
"implementation_name": "cpython",
"implementation_version": "3.7.4",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "5.18.18",
"platform_system": "Linux",
"platform_version": "#1 SMP PREEMPT_DYNAMIC 1",
"python_full_version": "3.7.4",
"platform_python_implementation": "CPython",
"python_version": "3.7",
"sys_platform": "linux"
}

$ find /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/ -name setuptools
/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/setuptools
$ echo $PYTHONPATH
/home/dev_1/opt/python-dev_3_7/lib/python3.7/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/pep582

[frostming]

Install setuptools, it is required for importing from setup.py.

[dev590t]

setuptools is installed in /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/setuptools, and /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages is in PYTHONPATH:

$ find /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/ -name setuptools
/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/setuptools
$ echo $PYTHONPATH
/home/dev_1/opt/python-dev_3_7/lib/python3.7/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/pep582

It is also installed in my local system by system package manager:

dev_1@dev_1 /tmp/test2$ .venv/bin/python 
Python 3.7.4 (default, Jan  1 1970, 00:00:01) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__path__
['/home/dev_1/opt/python-dev_3_7/lib/python3.7/site-packages/setuptools']

I have successed to run pdm import -f setuppy in a other project dir, but it use setuptools present in .venv:

$ pdm import -f setuppy setup.py 
Changes are written to pyproject.toml.
$ .venv/bin/python -Es
Python 3.7.4 (default, Jan  1 1970, 00:00:01) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__path__
['/tmp/test3/.venv/lib/python3.7/site-packages/setuptools']

I think pdm can't find setuptools because the option '-Es' passed to '/tmp/test/.venv/bin/python'.
According python --help:

-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE

I should in first install setuptools in .venv to allow pdm import -f setuppy runnable. But pdm init don't install that.

[mro-rhansen2]

setuptools is installed in /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/setuptools, and /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages is in PYTHONPATH:

$ find /home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/ -name setuptools
/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/setuptools
$ echo $PYTHONPATH
/home/dev_1/opt/python-dev_3_7/lib/python3.7/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages:/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/pep582

It is also installed in my local system by system package manager:

dev_1@dev_1 /tmp/test2$ .venv/bin/python 
Python 3.7.4 (default, Jan  1 1970, 00:00:01) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__path__
['/home/dev_1/opt/python-dev_3_7/lib/python3.7/site-packages/setuptools']

I have successed to run pdm import -f setuppy in a other project dir, but it use setuptools present in .venv:

$ pdm import -f setuppy setup.py 
Changes are written to pyproject.toml.
$ .venv/bin/python -Es
Python 3.7.4 (default, Jan  1 1970, 00:00:01) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__path__
['/tmp/test3/.venv/lib/python3.7/site-packages/setuptools']

I think pdm can't find setuptools because the option '-Es' passed to '/tmp/test/.venv/bin/python'. According python --help:

-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE

I should in first install setuptools in .venv to allow pdm import -f setuppy runnable. But pdm init don't install that.

The gist is that the default venv created by pdm does not have what it needs to make the pdm documentation make sense. The claim is seemingly that pdm can import setup.py out-of-the-box but as you've noticed that is not entirely accurate.

However, you can shortcut this little issue by setting up the in-project vm using the --with-pip flag. You can also enable that globally with pdm config venv.with-pip True. The setuptools package will install alongside pip without needing a secondary step.

[dev590t]

However, you can shortcut this little issue by setting up the in-project vm using the --with-pip flag.

How I can use --with-pip flag? python interpreter doesn't have this option.

You can also enable that globally with pdm config venv.with-pip True

That doesn't run:

$ pdm config venv.with-pip True
[NoConfigError]: 'No such config item: venv.with-pip'

[frostming]

It's venv.with_pip and make sure your PDM is latest

[dev590t]

It's venv.with_pip

The documentation indicated it is venv.with-pip. I have submitted a PR to fix that #1512

pdm config venv.with_pip True doesn't run for me:

$ pdm config venv.with_pip True
$ pdm import -f setuppy noExistingFile -v
Traceback (most recent call last):
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/parse_setup.py", line 211, in <module>
    print(json.dumps(parse_setup(sys.argv[1])))
  File "/home/dev_1/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/in_process/parse_setup.py", line 174, in parse_setup
    import setuptools
ModuleNotFoundError: No module named 'setuptools'

[mro-rhansen2]

You have to reinstall the venv before it will take effect.

pdm venv remove in-project -y
pdm venv create

And thank you for submitting that PR. That is actually where I pulled that information from. Shame on me for not trying it first.

[dev590t]

@mro-rhansen2 That runs. Thanks to have shared this tips.