RE: design for cachi2 wheels: Report whether dependency has any wheels (in the SBOM)

[ben-alkov]

@brunoapimentel, @ejegrova, @eskultety, @slimreaper35, @taylormadore;

Is

cachi2/cachi2/core/package_managers/pip.py

Lines 122 to 126 in 6225d7a

	def _generate_properties(dependency: dict) -> list[Property]:
	if not dependency["hash_verified"]:
	return [Property(name="cachi2:missing_hash:in_file", value=dependency["requirement_file"])]
	else:
	return []

the right place to implement adding a property to an SBOM component (for Python wheels)?

Note to self: Wheels were implemented in #255 & #319

[eskultety]

IIUC yes, it is.

[ben-alkov]

IIUC yes, it is

Nice. OK, second question - what determines whether a dependency is a wheel or not? I've tried

• dependency["package_type"] == "wheel"
• hacking in (what I thought was) dependency["wheel"]

but neither of those properties exist by the time the dependecy shows up in _generate_properties()...

[ben-alkov]

It looks like the allow_binary flag is actually (finally) used in

cachi2/cachi2/core/package_managers/pip.py

Lines 1459 to 1472 in 6225d7a

	if allow_binary:
	log.info("Downloading %d wheel(s) ...", len(to_download))
	files: dict[str, Union[str, PathLike[str]]] = {pkg.url: pkg.path for pkg in to_download}
	asyncio.run(async_download_files(files, get_config().concurrency_limit))
	for pkg in to_download:
	try:
	if pkg.should_verify_checksums():
	must_match_any_checksum(pkg.path, pkg.checksums_to_verify)
	except PackageRejected:
	pkg.path.unlink()
	log.warning("The %s is removed from the output directory", pkg.path.name)
	return downloaded

but the careful observer will note that to_download is never appended to downloaded, which is the only return from _download_dependencies()

Also, I note with interest that none of the docstrings have been updated to include allow_binary, which is making this all the more difficult.

[ben-alkov]

Oh, and another thing; every place in the cachi2 source where we are assigning to allow_binary, it is set to False. The design doc explicitly says to do the opposite?![1]

[1] Section Option not to download wheels: "The default should be allow_binary: true, since that usually results in less surprising behavior"

[brunoapimentel]

As for the allow_binary defaulting to False for now, I think it's because we didn't actually release the feature yet. Once it's done and documented, we can turn it on to true by default, but we also need to think on the impact this would have on Konflux (as it is the main Cachi2 'user' nowadays).

[chmeliik]

It probably shouldn't default to true until Enterprise Contract can block builds that use wheels (if it should ever default to true)

[ben-alkov]

I'm still blocked on exactly what marks a dependency as a wheel, permanently (i.e. after all downloading etc. is complete)... as I noted above, to_download is never appended to downloaded, which is the only return from _download_dependencies().

[slimreaper35]

I think there is another story that should fix this "problem" and make handling all Python packages more straightforward.

https://issues.redhat.com/browse/STONEBLD-1705

@chmeliik Am I right?

[chmeliik]

Hmm, yeah Ben's story would probably (require / be easier to do after) this refactoring

[ben-alkov]

Hmm, yeah Ben's story would probably (require / be easier to do after) this refactoring

Phew, ok, at least it's not just me 🙄. I was starting to think my brains had turned to oatmeal.

[ben-alkov]

Resolved in #467 and #508