add --dev --group test ignores dev if empty group with that name already exists in optional-dependencies

[aneroid]

• I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

Tl;DR

If a package has previously been added as an optional dependency, and then removed and re-added as a dev dependency with the same group name (which is currently empty), then it gets added back to optional-dependencies instead of dev-dependencies.

1. Create a new project with pdm init, accept defaults.

   • In my case I selected "yes" for 'Do you want to build this project for distribution(such as wheel)?

2. First add pytest (for example) as a test dependency: pdm add --group test pytest. pyproject.toml will now have this:

   [project.optional-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   
   [build-system]
   requires = ["setuptools>=61", "wheel"]
   build-backend = "setuptools.build_meta"
   
   [tool.pdm]
   distribution = true

3. At this point the user (me) realises that instead it should be a dev-only dependency, so I do: pdm remove -v --group test pytest. Resulting section in pyproject.toml is still have the empty group 'test':

   [project.optional-dependencies]
   test = [
   ]

4. Now, I add it as a dev dependency using pdm add -v --dev --group test pytest but in pyproj it's added back to the old section with the pre-existing group 'test':

   [project.optional-dependencies]
   test = [
       "pytest>=8.3.2",
   ]

5. Remove it again from the group using similar-but-not-same command as Ste 3: pdm remove -v --dev --group test pytest and also fully delete the section [project.optional-dependencies] in pyproject.toml

6. Again, add it as a dev dependency using pdm add -v --dev --group test pytest - exactly the same command as in step 4.

   • This time, it goes into a new and correct section with that group name:

   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]

Actual behavior

(Step 4 above.)

Expected behavior

(Step 6 above.) ➕

The documentation states:

The same group name MUST NOT appear in both [tool.pdm.dev-dependencies] and [project.optional-dependencies].
Which is the likely cause of the issue. However, I did remove that dependency and then re-add it using the expected-correct commands. But because the test group name already existed as an empty list in optional-deps, it just got added there again.

Potential fixes:

• Remove empty groups in *-dependencies to allow moving packages & groups between optional & dev deps.
  • Otherwise, it requires the user to manually modify the deps sections in pyproject.toml - which is fine if they realised that was the cause.
• Or, Issue a warning that the group with the same name already exists in the "other-deps" section (whichever other is) and will be added there instead of where the user stated in the command.

Environment Information

PS C:\some\folders\pdm-bug> pdm info && pdm info --env
PDM version:
  2.18.1
Python Interpreter:
  C:\some\folders\pdm-bug\.venv\Scripts\python.exe (3.12)
Project Root:
  C:/some/folders/pdm-bug
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.12.5",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19045",
  "python_full_version": "3.12.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "win32"
}
PS C:\some\folders\pdm-bug>

[aneroid]

Also, just for further verification, I've now also reproduced the issue in reverse: If the group name exists in dev-deps and then is removed & added as regular optional-dep, it still gets added back to dev-deps:

1. After step 6 above, pyproject.toml has:

   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]

2. Then execute: pdm remove -v --dev --group test pytest:

   [tool.pdm.dev-dependencies]
   test = [
   ]

3. After pdm add -v --group test pytest, it's not in opt-deps

   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]

4. pdm-Removing it again; then manually deleting only the test group from dev-dependencies which is now empty; and re-adding it with pdm add -v --group test pytest also works. And notably, it deletes the now-empty tool.pdm.dev-dependencies section too.