Files included relatively in requirements.in file gets converted to absolute paths

[rosen-dash]

Originally reported here - pypa/pip#13055

Environment Versions

1. OS Type: MacOS 15.0.1, Ubuntu 24.04
2. Python version: $ python -V Python 3.9.6
3. pip version: $ pip --version pip 24.3.1
4. pip-tools version: $ pip-compile --version 7.4.1

Steps to replicate

Sample repo in referred issue.

With pip 24.2

backoff==2.2.1
    # via -r requirements.in

With pip 24.3.1

backoff==2.2.1
    # via -r /Users/rosen/Personal/sample/requirements.in

Expected result

The relative references should be preserved

Actual result

Relative paths get replaced with absolute paths

[jmac105]

Same happens if specifying a constraint file inside requirements.in, such as -c requirements.txt. However if the constraint file is set via command-line argument instead, like pip-compile -c requirements.txt then the annotation uses the relative path instead.

Can replicate with pip 24.3 too, suspect this is a consequence of this change? https://github.com/pypa/pip/pull/12877/files#diff-e9f0a8e3da86978e2042a5a8073078ae9e37ee041dbc943c066e0c622c52d0d4

[waqas-saeed]

like @jmac105 said I can confirm this is caused by pip 24.3.1. if I use a bit older pip version like 23.0.1 then pip-compile shows relative path.

[AndydeCleyre]

FWIW this is not an issue in my feature/relpaths-post-6.8.0 branch, but there are no plans to integrate those changes here.

[AndydeCleyre]

Alternatively, it's not a problem with uv pip compile, either.

[webknjaz]

@macro1 the CI is now green and working on PRs unblocked.

[johnraz]

@webknjaz is there a consensus for a solution that needs to be worked on?

I can definitely help with a PR but I'd need guidance on what needs to be fixed and I'm confused where to look for more infos.

Thanks !

[tarasinf]

I'm curious if there's an update on the timeline for addressing this issue, or if there's a recommended workaround in the meantime?

[webknjaz]

@johnraz yeah, feel free to send a PR. You'll need to grep the repo for what outputs it and go from there. You'll also need to add tests.

[webknjaz]

@tarasinf a workaround could be using older pip or post-processing the output with some wrapper. You can also figure out how to fix it and send a pull request.

Timeline: whenever someone sends an acceptable PR, it gets merged and releasing's unblocked.

[tarasinf]

Thanks.
Here is the workaround I used. It's to lock the pip version only for pip-tools in pre-commit:

  - repo: https://github.com/jazzband/pip-tools
    rev: 7.3.0
    hooks:
      - id: pip-compile
        name: pip-compile requirements-test.in
        args: [requirements/test.in]
        files: ^requirements/(base|test)\.(txt|in)$
        additional_dependencies: ['pip==24.2']

For the test purpose, to be sure pre-commit uses a proper pip version, I used this custom check:

  - repo: local
    hooks:
    - id: pip-version-check
      name: Check pip version
      entry: pip --version
      language: python
      additional_dependencies: ['pip==24.2']
      pass_filenames: false

and run it by pre-commit run --verbose pip-version-chec.

[zachborboacryptofi]

Timeline: whenever someone sends an acceptable PR, it gets merged and releasing's unblocked.

@webknjaz hi. what's considered an acceptable PR (in addition to the guidelines[1])? does it just need to 1) fix the issue and 2) have a test that demonstrates the issue has been fixed?

[1] https://github.com/jazzband/pip-tools/blob/main/CONTRIBUTING.md

[AndydeCleyre]

FWIW I've just merged latest pip-tools into my relpaths branch (an unacceptable PR), so both that and uv pip compile remain working alternatives. You can test out the branch for example with pipx:

$ pipx run --spec 'pip-tools @ git+https://github.com/AndydeCleyre/pip-tools@feature/relpaths-post-6.8.0' pip-compile requirements.in

[webknjaz]

@zachborboacryptofi I'd require good test coverage (including local testing against the main branch of pip). The CI should be green. If there's any tests marked as xfail, they'll need to be unmarked. Nothing else comes to my mind. Note that this is maintained sporadically by a bunch of volunteers so don't expect immediate reviews.

@AndydeCleyre could you remind me what's unacceptable in your branch? I vaguely recall you trying something but don't remember the context..

[AndydeCleyre]

@webknjaz While I tried to introduce only as much complexity as needed to reliably track and handle paths, given the constraints of maintaining backward compatibility, and not making changes to pip itself, the final judgment can be found in this comment and the following one.

The highlights are:

. . . these changes are too hacky and bring too much complexity to the codebase.
. . .
I'd rather not have this feature rather than maintain unreliable code.
. . . proposed implementation does touch too much code and adds undesired complexity.