[bug] when conan upload conan center package to private artifactory, the source backup step reports error

[Austin1511]

Environment details

• Operating System+version: windows 10
• Compiler+version: msvc 190
• Conan version:2.0.17
• Python version:3.11

Steps to reproduce

1. I have a conan package from conan center "expat/2.5.0" in my local cache.
2. I upload to my private Artifactory in my local server
3. I already set a source backup repository.
4. I got the python error showed in below:

Logs

PS D:\3rdparty_codes\conan-center-index\recipes\itk\all> conan upload expat* -r remevision
Are you sure you want to upload recipe 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9'? (yes/no): yes
Are you sure you want to upload package 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9:10db2609a6a02761afb2699bd6c0d067d0be6015#8acb7b6b7ae0dc94c7bebe2a191e545e'? (yes/no): yes
Are you sure you want to upload package 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9:d80b5d8f20bd194cf53f5729e19d4bdc4df3feb5#8425a32dbf5bef6f82c45919aac0e2e7'? (yes/no): yes

-------- Checking server existing packages --------
expat/2.5.0: Checking which revisions exist in the remote server
expat/2.5.0: Recipe 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9' already in server, skipping upload
expat/2.5.0: Package 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9:10db2609a6a02761afb2699bd6c0d067d0be6015#8acb7b6b7ae0dc94c7bebe2a191e545e' already in server, skipping upload
expat/2.5.0: Package 'expat/2.5.0#91e43e4544923e4c934bfad1fa4306f9:d80b5d8f20bd194cf53f5729e19d4bdc4df3feb5#8425a32dbf5bef6f82c45919aac0e2e7' already in server, skipping upload

-------- Preparing artifacts for upload --------

-------- Uploading artifacts --------
Upload complete

ERROR: Traceback (most recent call last):
File "D:\python311\Lib\site-packages\conan\cli\cli.py", line 280, in main
cli.run(args)
File "D:\python311\Lib\site-packages\conan\cli\cli.py", line 190, in run
command.run(self._conan_api, args[0][1:])
File "D:\python311\Lib\site-packages\conan\cli\command.py", line 126, in run
info = self._method(conan_api, parser, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python311\Lib\site-packages\conan\cli\commands\upload.py", line 109, in upload
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python311\Lib\site-packages\conan\api\subapi\upload.py", line 69, in get_backup_sources
return download_cache.get_backup_sources_files_to_upload(excluded_urls, package_list)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python311\Lib\site-packages\conans\client\downloaders\download_cache.py", line 81, in get_backup_sources_files_to_upload
metadata = json.loads(load(metadata_path))
^^^^^^^^^^^^^^^^^^^
File "D:\python311\Lib\site-packages\conans\util\files.py", line 130, in load
with open(path, 'r', encoding=encoding, newline="") as handle:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'D:/3rdparty_codes/s\da9dec6c12cf2ecf269c31ab65b5de18e8e52b96f35d5bcd08c12b43e6878803.dirty.json'

ERROR: [Errno 2] No such file or directory: 'D:/3rdparty_codes/s\da9dec6c12cf2ecf269c31ab65b5de18e8e52b96f35d5bcd08c12b43e6878803.dirty.json'

[memsharded]

Thanks very much @Austin1511 for reporting.

We are more or less aware of this issue and @RubenRBS did a PR for next release in #15501

However, it seems the PR is avoiding the stacktrace, because it was focused on erronously placed files. But the .dirty files you are reporting are actually created by Conan, and it tracks a broken or interrupted download. So probably we need to explictly handle this, what do you think @RubenRBS ?

[AbrilRBS]

Looking into it, it does seem like there's a gap in how we handle those cases yes! We'll get back to you once we have a proposed solution, but for now, removing the .dirty files should allow the upload to proceed :)

[memsharded]

Closed by #15601, it will be in next 2.1, thanks!