feat: add GitHub attestation discovery

[benmss]

This PR adds support for GitHub attestation discovery. To access GitHub attestations, the SHA256 hash of the repositories artefacts must be generated and submitted via the API. Artefacts may be found in local caches, or downloaded from remote repositories.

• Support for Maven artefacts
• Support for PyPI artefacts

Closes #915

[nicallen]

Is it really necessary to dump it to a temp file then read it back in? Can't we just process the git_attestation directly as a provenance in-memory?

[benmss]

It is not necessary but is in-line with pre-existing handling of attestation. Changing it is doable, it will just require some refactoring to ensure consistency.

[benmss]

This will be addressed as part of a separate issue #1095

[tromai]

Are we assuming that the the first element is the attestation we are looking for ? Or is there a schema for the attestation content from Github that supports this decision 🤔

[benmss]

We are assuming that the first attestation is the one we are looking for. The alternative is to decode the payload of each and compare against the target artifact name. I'm unsure how common it is to find multiple attestations for a given artifact hash.

For the schema, there is an example given in the API. I will add this to the doc comment.

[tromai]

Thanks. Updating the doc comment sounds good to me.

[tromai]

I wonder if we could have some unit tests for this function ?

[tromai]

From my understanding, this function is trying to do 2 things:

• Find the first PyPIPackageJsonAsset instance within pypi_registry_info.metadata and returns it: asset_name and asset_version are not used, pypi_registry_info.metadata is not mutated.
• If we cannot find the asset, we create a new PyPIPackageJsonAsset instance and append to pypi_registry_info.metadata.

I think this function can be refactored so that it only find and process any existing PyPIPackageJsonAsset in pypi_registry_info. The creation of a new PyPIPackageJsonAsset could be done separately, because it's only a constructor call. This way, we can prevent the mutation of a passed in parameter. This is what I currently have in mind:

def find_pypi_asset_from_registry_info(registry_info: PackageRegistryInfo) -> PyPIPackageJsonAsset | None:
    ...

def create_pypi_asset_from_pypi_registry_info(name: str, version: str | None, pypi_registry_info: PyPIRegistry) -> PyPIPackageJsonAsset | None:
    ...

And then the adding of the new asset to pypi_registry_info.metadata can happen outside of those 2 functions.

[benmss]

I disagree with this suggestion. It's true that the function can be split into smaller pieces, however, in the 2 places that it is used, all split pieces would still need to be called sequentially. E.g. If we are about to download an asset we must first check if it already has been downloaded, then if not, create the asset, add it to the registry, etc. Ultimately, this means identical blocks of code calling all 2 to 3 sub functions.

Perhaps we can improve the name and/or documentation instead?

(There is one more place this function should be called. I will add it there instead of the almost identical solution that exists there currently.)

[tromai]

Ah I see. Thanks for the clarification. I think it's good enough to update the doc string to indicate the mutation of the registry infor object when a new PyPIPackageJsonAsset instance is created.

[tromai]

I have finished my first round of review. Thank you.

[behnazh-w]

Why don't we support PyPI packages here?

[benmss]

Assuming packages are installed via pip, the wheel files are not stored within the virtual environment. It might be possible to add support for pip caches though.