purl-next: Update PurlDB to support on-demand, live query

[pombredanne]

Update PurlDB to support on-demand, live query of upstream package registries for the PURL types documented in the PURL specification to validate the existence of PURLs.

As a follow up of these earlier tasks, we need to cover more supported ecosystems:

• purl-next: Arch Linux alpm - on demand data collection #697
• purl-next: Alpine apk- on demand, data collection #698
• BitBucket https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/bitbucket.py
• bitnami we cannot do as it is 1. impossible 2. Broadcom is closing it
• Cargo Add on-demand package data collection for cargo #594
• Cocoapods PURLDB: Add on-demand package data collection for Cocoapods #466 BUT "CocoaPods trunk is moving to be read-only. Read more on the blog, there are 16 months to go." .... this is going dark in December 202r
  • See also https://github.com/CocoaPods/Specs/blob/master/Specs/1/1/1/AKTransmission/0.1.1/AKTransmission.podspec.json
  • Add Fetchcode support for cocoapods fetchcode#177
• Composer purl-next: Update PurlDB to support on-demand - composer #645
• Conan https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/conan.py
• purl-next: Conda- on demand, data collection #699
• CRAN purl-next: Update PurlDB to support on-demand - CRAN #646
• CPAN purl-next: Update PurlDB to support on-demand - CPAN #647
• Debian https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/debian.py
• purl-next: Dockerhub- on demand, data collection #700
• Gem Add on-demand package data collection for gem #595
  • https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/rubygems.py
• Generic https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/generic.py
• Github https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/github.py
• ++ Gitlab https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/gitlab.py
• ++ Gnu https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/gnu.py
  • Create PR for a gnu package type @ purl spec #695
• Golang Add on-demand package data collection for golang #596
• purl-next: Hackage- on demand, data collection #704
• purl-next: Hex- on demand, data collection #705
• purl-next: HuggingFace - on demand, data collection #706
• purl-next: luarocks - on demand, data collection #707
• Maven https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/maven.py
• mlflow there is no registry, we cannot collect things
• npm https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/npm.py
• Nuget Add on-demand package data collection for nuget #597 and https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/nuget.py
• OCI - not possible, not a single package registry
• purl-next: Pub - on demand, data collection #703
• PyPi Collect pypi PURL ondemand #468 #592 and https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/pypi.py
• ++ openssl https://github.com/aboutcode-org/purldb/blob/dba43e6e667824d1eec7b9ad2c2f75ce1d9c002a/minecode/collectors/openssl.py
• qpkg the registry is not accessible
• swid no registry
• purl-next: Swift- on demand, data collection #701
• ++ Ubuntu https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/ubuntu.py extended from Debian

Next work:

• some out of this list of 11 types, like about 6 or 7 of these: swift, pub, oci, docker, luarocks, huggingface, hex, hackage, conda, apk, alpm or the list below.
• Apache https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/apache.py
• F-Droid https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/fdroid.py
• Fedora would be starting from scratch https://github.com/aboutcode-org/purldb/blob/main/minecode/miners/fedora.py