Improve debian package detection

[AyanSinhaMahapatra]

• Detect and store more attributes from debian .dsc metadata files
• Also properly detect and create packages from control and md5sums files.

Reference: aboutcode-org/scancode.io#1151
Reference: aboutcode-org/purldb#3

Tasks

• Reviewed contribution guidelines
• PR is descriptively titled 📑 and links the original issue above 🔗
• Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
  Run tests locally to check for errors.
• Commits are in uniquely-named feature branch and has no merge conflicts 📁
• Updated documentation pages (if applicable)
• Updated CHANGELOG.rst (if applicable)

[pombredanne]

Looking good!
Yet I think we can do better.
See https://en.wikipedia.org/wiki/Deb_(file_format)#Implementation
We should support xz, lzma, bz2, gz and zst AND the data.tar and control.tar may be uncompressed too

See also:

• https://gitlab.com/gitlab-org/gitlab/-/issues/422584
• https://gitlab.com/pmhahn/debian-package-registry

@pmhahn do you reckon we should support all of .tar.xz, .tar.lzma, .tar.bz2, .tar.gz, .tar.zst, .tar.zstd AND the plain data.tar and control.tar as formats for the tarballs inside a .deb? We need to support all current but also legacy formats to properly scan and index them!

[pombredanne]

@pmhahn are the packages in https://gitlab.com/pmhahn/debian-package-registry testing all known formats?
Note the .deb for deb-lzma_1.0_all.deb and deb-bzip2_1.0_all.deb both contain a .zst extension and not lzma or bzip2 tarballs.

@AyanSinhaMahapatra to build all these combos in a Ci/CD test (likely from our own clone for stability):

sudo apt-get update
sudo apt-get --assume-yes --no-install-recommends install build-essential dput-ng curl ca-certificates
git clone https://gitlab.com/pmhahn/debian-package-registry
cd debian-package-registry
for f in deb*
  do  pushd $f
  sudo apt-get --assume-yes build-dep .
  dpkg-buildpackage --no-sign --build=source,all
  popd
done

[pombredanne]

@AyanSinhaMahapatra https://wiki.debian.org/Teams/Dpkg/DebSupport gives some visibility on the topic

[pmhahn]

@pmhahn are the packages in https://gitlab.com/pmhahn/debian-package-registry testing all known formats?

As I noted in the Gitlab issue the problem is that newer dpkg-deb no longer supports building some old formats like bzip2 and lzma; so sadly no, my repository is not complete as those two are missing.

But as .deb files are ar-file with .tar-files within, it it trivial to create a testing .deb by using ar and tar yourself.

[pombredanne]

But as .deb files are ar-file with .tar-files within, it it trivial to create a testing .deb by using ar and tar yourself.

@pmhahn good point! Thank you ++ for your swift reply! .... @AyanSinhaMahapatra let's support them all then as this has no downside.

[pombredanne]

LGTM! Thanks