pdm add expands submodule relative path to full path

[0liu]

• I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

In the project root /home/user/projects/myproj there are some submodules like /home/user/projects/myproj/submodules/subproj. Add this submodule by pdm add ./submodules/subproj will expand to full path in pyproject.toml like:

dependencies = [
    "subproj @ file:///home/user/projects/myproj/submodules/subproj",
]

or by pdm add -e ./submodules/subproj --dev, it expands the relative path to

dev = [
    "-e file:///home/user/projects/myproj/submodules/subproj#egg=subproj",
]

/home/user/projects/myproj/submo dules/other_proj

The problem is, these full paths can't be committed to vcs. This is a similar issue to jazzband/pip-tools#204, but in pip requirements.in or txt, it can be remedied as shown in this comment, and it seems working with the latest pip tools without this workaround.

Actual behavior

A relative path of git submodule is expanded to full path in pyproject.toml after running pdm add.

Expected behavior

The relative path should not be changed.

Environment Information

__pypackages__ is detected, using the PEP 582 mode
PDM version:
  2.4.5
Python Interpreter:
  /usr/bin/python (3.10)
Project Root:
  /home/user/projects/myproj
Project Packages:
  /home/user/projects/myproj/__pypackages__/3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.9",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.1.12-1-MANJARO",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT_DYNAMIC Tue Feb 14 21:59:10 UTC 2023",
  "python_full_version": "3.10.9",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

[frostming]

Note that if you are using the default backend pdm-pep517, or pdm-backend, or not set in pyproject.toml, it will be correctly converted, while if you are using setuptools, relative paths will be expanded. Look at the build-system table in pyproject.toml

[0liu]

Yes, build-backend is set to setuptools.build_meta to support pybind11 extension. Not sure how that will work if switching to pdm-pep517. With setuptools as backend, if expanding path is by design, then it seems no solution with PEP-582.

[frostming]

With setuptools as backend, if expanding path is by design

Yes, we don't keep relative paths if the backend doesn't support it, because the metadata will be eventually consumed by the backend when building the wheel, without PDM involved.

[0liu]

FWIW, I tried build-backend pdm-pep517 to build pybind11 extension, by following this doc. The setup files are below. It just works! This is a very nice feature, should be added to pybind11 build doc.

# pyproject.toml

[build-system]
requires = ["pdm-pep517>=1.0", "pybind11>=2.10.0"]
build-backend = "pdm.pep517.api"

[tool.pdm]

[tool.pdm.build]
setup-script = "myext.py"
run-setuptools = true

and

# myext.py

from pybind11.setup_helpers import Pybind11Extension

ext_modules = [
    Pybind11Extension(
        name="subproj.transformer",
        sources=["subproj/transformer.cc"],
        extra_compile_args=['-O3'],
        cxx_std=20,
    ),
]

def build(setup_kwargs):
    setup_kwargs.update(ext_modules=ext_modules)