Package-update-job fails because of missing packages field for some sources

[mayaCostantini]

Describe the bug

When checking package availability during package-update-job, some sources used in _check_package_availability are missing a packages field, causing an unhandled KeyError:

{
https://download.pytorch.org/whl/cpu: {
packages: [
'torchserve', 
'torch-cuda80', 
'torch', 
'torchrec-cpu', 
'torchcsprng', 
'torch-tb-profiler', 
'torchrec', 
'torchtext', 
'torchaudio', 
'torchvision'
], 
source: AIOSource(url='https://download.pytorch.org/whl/cpu', name='download-pytorch-org-whl-cpu', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://download.pytorch.org/whl/cu111: {
packages: [
'torchserve', 
'torch-cuda80', 
'torch', 
'torchrec-cpu', 
'torchcsprng', 
'torch-tb-profiler', 
'torchrec', 
'torchtext', 
'torchaudio', 
'torchvision'
], 
source: AIOSource(url='https://download.pytorch.org/whl/cu111', name='download-pytorch-org-whl-cu111', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://download.pytorch.org/whl/cu113: {
packages: [
'torchserve', 
'torch-cuda80', 
'torch', 
'torchrec-cpu', 
'torchcsprng', 
'torch-tb-profiler', 
'torchrec', 
'torchtext', 
'torchaudio', 
'torchvision'
], 
source: AIOSource(url='https://download.pytorch.org/whl/cu113', name='download-pytorch-org-whl-cu113', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://pulp.operate-first.cloud/pypi/gym-donkeycar/simple: {
source: AIOSource(url='https://pulp.operate-first.cloud/pypi/gym-donkeycar/simple', name='pulp-operate-first-cloud-pypi-gym-donkeycar-simple', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://pulp.operate-first.cloud/pypi/hello-world/simple: {
source: AIOSource(url='https://pulp.operate-first.cloud/pypi/hello-world/simple', name='pulp-operate-first-cloud-pypi-hello-world-simple', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://pulp.operate-first.cloud/pypi/test/simple: {
source: AIOSource(url='https://pulp.operate-first.cloud/pypi/test/simple', name='pulp-operate-first-cloud-pypi-test-simple', verify_ssl=True, warehouse=False, warehouse_api_url=None)
}, 
https://pypi.org/simple: {
packages: [
'napari-tools-menu', 
'ftw-linkchecker', 
'lim-cpu', 
'sphinxnotes-lilypond', 
'twinter', 
'cinder-data', 
'dagcheck', 
'pywork', 
'pyquebec', 
'calculo-irrf'
], 
source: AIOSource(url='https://pypi.org/simple', name='pypi-org-simple', verify_ssl=True, warehouse=True, warehouse_api_url=None)
}, 
https://tensorflow.pypi.thoth-station.ninja/index/manylinux2010/AVX2/simple: {
packages: [
'tensorflow-gpu', 
'tensorflow-cpu', 
'tensorflow-serving-api'
], 
source: AIOSource(url='https://tensorflow.pypi.thoth-station.ninja/index/manylinux2010/AV...
}
}

Concerned sources are from https://pulp.operate-first.cloud/pypi/ index.
My first guess would be that the method to get packages for Pulp-based sources fails because of the way it parses the index page: see https://github.com/thoth-station/python/blob/299d6ece8a8392c7fe2d3ef5015684bf355bbfa1/thoth/python/aiosource.py#L251

To Reproduce
See Sentry report for Oct. 19, 2022, 12:00:24 p.m. UTC in ocp4-stage.

Expected behavior
Packages can be gathered for Pulp sources.

[mayaCostantini]

/sig stack-guidance
/kind bug
/priority critical-urgent

[harshad16]

As there has been much packages included to pulp
and the instance seems to be down.

we have disabled those index on our systems:
https://management.stage.thoth-station.ninja/api/v1/python-package-index

  "disabled": [
    {
      "only_if_package_seen": false,
      "url": "https://pulp.operate-first.cloud/pypi/gym-donkeycar/simple",
      "verify_ssl": true,
      "warehouse_api_url": null
    },
    {
      "only_if_package_seen": false,
      "url": "https://pulp.operate-first.cloud/pypi/test/simple",
      "verify_ssl": true,
      "warehouse_api_url": null
    },
    {
      "only_if_package_seen": false,
      "url": "https://pulp.operate-first.cloud/pypi/hello-world/simple",
      "verify_ssl": true,
      "warehouse_api_url": null
    }
  ],

[harshad16]

This should fix the issue for now.
Keeping this issue open to view if we see changes.

[codificat]

Next steps:

• check packages in the pulp index
• re-enable the index
• run package-update job to verify if the issue persists