[pdm.lock - pdm 2.5.0b1] Cannot add optional-dependencies to lockfile after initial generation of lockfile

[taylorb-syd]

• I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

1. Update to PDM 2.5.0b1
2. Have an optional dependency, in my environment my pyproject.toml has:

[project.optional-dependencies]
build = [
    ...
]

3. Purge your lock-file:

❯ rm pdm.lock

4. Produce a lock file:

❯ pdm -vv lock --skip :post
🔒 Lock successful
Changes are written to pdm.lock.

5. Attempt to export the the optional dependency

❯ pdm -vv export --format requirements --without-hashes --group build
[PdmUsageError]: Requested groups not in lockfile: build

6. Attempt to add the missing optional dependency to the lockfile (or all optional dependencies)

❯ pdm -vv lock --group build --skip :post
[PdmUsageError]: Requested groups not in lockfile: build
❯ pdm -vv lock --group :all --skip :post
[PdmUsageError]: Requested groups not in lockfile: build

7. Purge the lockfile, and install with optional dependencies from the beginning, and attempt same operation:

❯ rm pdm.lock
❯ pdm -vv lock --group :all --skip :post
🔒 Lock successful
Changes are written to pdm.lock.
❯ pdm -vv export --format requirements --without-hashes --group build
The exported requirements file is no longer cross-platform. Using it on other platforms may cause unexpected result.
# This file is @generated by PDM.
# Please do not edit it manually.
...

Actual behavior

Looking a little deeper, it looks like when the groups field is generated in the PDM lock by default it only includes dev-dependencies and the default, and not optional-dependencies. To get it to include optional-dependencies you need to include the relevant --group selector(s) (or :all).

This is fine, except once the lockfile is generated appears to use the lockfile as the source of truth for what groups are valid even when performing a lock operation, not the pyproject.toml. This means without purging the lockfile it is impossible to add a optional-dependency.

Give what I can see of the code, I also suspect that if you were to add an additional entry to dev-dependencies after this point you would similarly be able to add this to the lockfile, but I did not test this.

Expected behavior

When you create a lockfile it should either create the lockfile with the optional-dependencies included, or allow you to add optional-dependencies to the lockfile after initial generation.

Environment Information

❯ pdm info
PDM version:
  2.5.0b0
Python Interpreter:
  /Volumes/workplace/PythonProjects/REDACTED/.venv/bin/python (3.10)
Project Root:
  /Volumes/workplace/PythonProjects/REDACTED
Local Packages:

❯ pdm info --env
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.10",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "21.6.0",
  "platform_system": "Darwin",
  "platform_version": "Darwin Kernel Version 21.6.0: Mon Dec 19 20:44:01 PST 2022; root:xnu-8020.240.18~2/RELEASE_X86_64",
  "python_full_version": "3.10.10",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "darwin"
}

[frostming]

This means without purging the lockfile it is impossible to add a optional-dependency.

Yes, this is a bug on 2.5.0. You can also achieve that with pdm add --group build.