top level module package with submodules - installation fails in external system with poetry built distributable

[aprath1]

• Poetry version: 1.3.1
• Python version: 3.10.8
• OS version and name: Windows 11
• pyproject.toml: top_module pyproject.toml

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
  NOTE: some issues are closely related and discusses this current case, which leads to the current issue.
  Related issues:
  Local path in wheel metadata seems wrong #4709
  Project with path dependency cannot be installed with pip install . (or via requirements file) due to "RequirementParseError: Invalid URL" #5273
• I have consulted the FAQ and blog for any relevant entries or release notes.

Issue

The distributable (wheel) installation fails for the following package design - top_module packages with submodule packages such as the sample base tree file structure below:

│   pyproject.toml
│   README.md
│
└───top_module
    │   __init__.py
    │
    ├───submodule_a
    │   │   pyproject.toml
    │   │   README.md
    │   │   setup.py
    │   │
    │   ├───submodule_a
    │           __init__.py          
    │
    └───submodule_b
        │   pyproject.toml
        │   README.md
        │   setup.py
        │
        ├───submodule_b
                __init__.py
        

when distributable is built with poetry build it has the local system absolute path in the metadata, for example in the above sample tree case the wheel metadata include the following:

...
Requires-Dist: submodule_a @ file:///<absolute_path_local_to_system>/top_module/top_module/submodule_a
Requires-Dist: submodule_b @ file:///<absolute_path_local_to_system>/top_module/top_module/submodule_b
...

which means this will impact the distributable installation in any other external systems, which is failing the purpose of distributable - just like it is mentioned in this #5273 (comment). (Even though cloning the source repo and installing works). dist installation will trigger following error as the installation is looking for absolute paths local to the system which generated the distributable:
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory:

currently the submodules are provided as relative path dependencies in pyproject.toml as below; is there any alternate approach available for this scenario to facilitate dist installation in external systems?

[tool.poetry.dependencies]
python = "^3.10, < 3.11"
submodule_a = { path = "top_module/submodule_a", develop=true }
submodule_b = { path = "top_module/submodule_b", develop=true

Impact:
for projects with sub packages such as those with git submodules, distributable installation will not be possible in any other external systems.

[dimbleby]

Path dependencies cannot do what you want:

• relative paths are simply not valid
• absolute paths can indeed only work on a system that has the corresponding directory structure.

This is not a poetry issue (prove me wrong by using some other tool to build a wheel that does what you want!)

you'll want to find some other way to model this eg perhaps you want three wheels, perhaps it should all have been a single module all along.

[neersighted]

What you want might be covered by #4583 (vendoring in files, essentially). Path dependencies will not do what you want, @dimbleby is right to point out that Poetry is doing exactly what you tell it to (building a package with a dependency on an absolute path). You might also be interested in #1168 if the modules can have a dependency by name outside of development.

Either way, this is a duplicate of existing issues/misapplication of the tool.

[aprath1]

Thanks for your reply.
agreed with the relative paths not being valid, and the second point as being the reason for this issue.
I was looking today for performing the mentioned distributable wheel generation with other tool but haven't actually found any yet.

yes, using separate wheels is what was considered as an alternative.

So this actually leads to a question of why are package designs with git submodules not distributable friendly!
Considering the above I would close this issue for now.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.