[bug] conan remove <ref> -p <query> removes all packages and recipe instead of packages matching query

[SpaceIm]

Environment details

• Operating System+version: Ubuntu 22.04
• Compiler+version: doesn't matter
• Conan version: 2.0.2
• Python version: 3.10.6

Steps to reproduce

Install & build several recipes with clang & gcc

call conan remove "*" -p "compiler=clang" -c. All packages AND recipes themselves are removed, instead of removing only clang packages.

Example:

conan list "zlib/*:*#*"

Local Cache
  zlib
    zlib/1.2.13
      revisions
        13c96f538b52e1600c40b88994de240f (2022-11-02 13:46:53 UTC)
          packages
            4a364544d7d9babcd53ebe990c1079dbfe108a35
              revisions
                a6b5acd767b79a74c5dd3fba4e8ee7f5 (2023-03-16 00:37:52 UTC)
              info
                settings
                  arch: x86_64
                  build_type: Release
                  compiler: gcc
                  compiler.exception: seh
                  compiler.threads: posix
                  compiler.version: 10
                  os: Windows
                options
                  shared: True
            5bc851010eb7b707e5cb2e24cb8ccf0f27989fa9
              revisions
                7d40fc2b6297d11b68b2ae7713b4f7bf (2023-03-27 21:35:12 UTC)
              info
                settings
                  arch: x86_64
                  build_type: Release
                  compiler: gcc
                  compiler.version: 12
                  os: Linux
                options
                  fPIC: True
                  shared: False
            9a7f5466b6926f6dc790c94d617e893533d5c141
              revisions
                1922b2913ada31dbf6bee60a389939e7 (2023-03-27 19:36:50 UTC)
              info
                settings
                  arch: x86_64
                  build_type: Release
                  compiler: gcc
                  compiler.version: 12
                  os: Linux
                options
                  shared: True
            d35143545e409767ebceb1aa13a2c47438de5f09
              revisions
                83642d7a2d370f242844c24b45c971ab (2023-04-02 14:07:36 UTC)
              info
                settings
                  arch: x86_64
                  build_type: Release
                  compiler: clang
                  compiler.version: 14
                  os: Linux
                options
                  shared: True
            e4e2f25cb0478d79981369ba6087b40167e5cb66
              revisions
                00c3d992eb1f0c1899e2f032c672a6a5 (2023-04-02 14:22:45 UTC)
              info
                settings
                  arch: x86_64
                  build_type: Release
                  compiler: clang
                  compiler.version: 14
                  os: Linux
                options
                  fPIC: True
                  shared: False

call conan remove "zlib" -p "compiler=clang" -c

call conan list "zlib/*:*#*" again after above command:

Local Cache
  WARN: There are no matching recipe references

This behavior is clearly not my understanding of conan remove -h:

usage: conan remove [-h] [-v [V]] [-c] [-p PACKAGE_QUERY] [-r REMOTE] reference

Remove recipes or packages from local cache or a remote.

- If no remote is specified (-r), the removal will be done in the local conan cache.
- If a recipe reference is specified, it will remove the recipe and all the packages, unless -p
  is specified, in that case, only the packages matching the specified query (and not the recipe)
  will be removed.
- If a package reference is specified, it will remove only the package.

positional arguments:
  reference             Recipe reference or package reference, can contain * aswildcard at any reference field. e.g: lib/*

options:
  -h, --help            show this help message and exit
  -v [V]                Level of detail of the output. Valid options from less verbose to more verbose: -vquiet, -verror, -vwarning, -vnotice, -vstatus, -v or -vverbose, -vv or
                        -vdebug, -vvv or -vtrace
  -c, --confirm         Remove without requesting a confirmation
  -p PACKAGE_QUERY, --package-query PACKAGE_QUERY
                        Remove all packages (empty) or provide a query: os=Windows AND (arch=x86 OR compiler=gcc)
  -r REMOTE, --remote REMOTE
                        Will remove from the specified remote

Logs

No response

[AbrilRBS]

Hi @SpaceIm, thanks for the report.

I can reproduce your issue, in which

If a recipe reference is specified, it will remove the recipe and all the packages, unless -p
is specified, in that case, only the packages matching the specified query (and not the recipe)
will be removed.

is not what's happening. Note that you can work around this by calling conan remove *:* -p compiler=clang -c which will ensure only packages are removed. But you're right in that the help suggests that conan remove * -p compiler=clang -c would only remove packages and not recipes, which is clearly not what's happening here.

We'll need to figure out if the docs or the code is wrong, and by looking at the implementation with a quick glance, it seems to me like the help message is the misleading one, but not sure if it's the right thing to do. Will add it as a look-into to discuss with the team

[SpaceIm]

conan remove *:* -p compiler=clang -c behaves as expected, but I think it's misleading.

It's worth noting that it may fail (I guess if at least one recipe in cache has no compiler settings, or maybe if compiler is removed from package id):

ERROR: Invalid package query: compiler=clang. 'NoneType' object has no attribute 'get'

(And same error with conan list "*:*" -p "compiler=clang")

[AbrilRBS]

After checking with the team, we came to the conclusion that both the comments and the code were wrong :)

You now need to pass a package pattern (ie *:*) for queries to affect packages, and * -p "..." is now disallowed to avoid the deletion of recipes if a package was found by the query. I'll make sure to improve the documentation around the command to make this more clear.

I've also fixed the ERROR you were seeing, thanks for reporting :)

Closed by #13601

[SpaceIm]

Thanks

[AbrilRBS]

(I forgot to actually push the fix for the ERROR 🤦, it will still fail for now, give me a minute)