Adding a nesting structure so that pip-compile can pin the same versions across nested extras.

[FeryET]

What's the problem this feature will solve?

I am using setup.cfg to manage my dependencies and extra dependencies, and I am nesting some extra dependencies in each other. Currently, pip-compile does not guarantee resolving to the same version when I compile the requirements for nested extras.

[options.extras_require]
server =
    pydantic~=1.10.7, >1.9.0

dev =
    %(server)s
    wget

Describe the solution you'd like

I would like to constrain the extended extras to the versions pinned by the nested extra. Here server should pin a pydantic version that dev must follow, and all should be generated into a single independent requirements.txt for each extra.

This is especially important in building smaller containers, since I can guarantee version pinning and smaller diffs between layers of nested extras.

FROM base as builder-1
pip install --no-cache-dir server-requirements.txt

FROM builder
pip install --no-cache-dir dev-requirements.txt

Alternative Solutions

I can use multiple requirements.txt and nest them in each other and introduce dynamic constraints by symlinking the outputs of base extra.

Additional context

I use a single setup.cfg so that I can nest extra pip dependencies easily using setuptools magic. This gives me great flexibility to add requirements and does not make my repo much more crowded. If I had to migrate to having multiple requirments.in and dynamic constrains, it would add a lot of intermediary files, and it is harder to maintain the repository. I think having a single file in which I can define all packages is way more superior to any alternative IMO.

[atugushev]

@FeryET could you please try this PR (#1936)? I've added support for pip-compile -c requirements.txt, which I believe should resolve the issue.