feat: Add cachedimage pulling progress status

[aDisplayName]

Add following property in CachedImage CRD

• spec.progress.available
• spec.progress.total

Update the status.progress.available in CachedImage CRD every 5 seconds during the image pulling in cachedimage controller

This is to replace the PR #400 with all the clutters removed.

This is to implement the feature request #401

The CRD / API version is not changed here, still left as v1alpha1.

[plaffitt]

Thanks for this new PR ! It will be easier for me to review since there are less commits and much less changes too haha

To answer your comment in the previous one (by the way, you could also squash, reword and force-push your old branch instead of creating a new one, but, this way we keep discussions in the same PR, but it's ok for this time don't worry): I think we don't need to bump the version of the CRD, the goal of naming it alpha is to emphasize that it can change over time, moreover you only add fields, so it shouldn't create any friction.

I will review it soon, thansk again for your work!

[plaffitt]

I tried to compile and run your changes on my dev cluster and I'm a bit confused about the values showed in the status of CachedImages. For instance for quay.io/jetstack/cert-manager-cainjector:v1.13.3, it shows a total of 318 (bytes ?) but docker inspect quay.io/jetstack/cert-manager-cainjector:v1.13.3 | jq '.[0].Size' outputs 42445613. So it doesn't really seems to work. Also showing the total size doesn't seems that useful to know the progression, as I would have to download and inspect the image first, or maybe describe the CachedImage in order to compare the status.progress.available vs status.progress.total. It would be much more interesting to have a percentage of completion instead of a total.

PS : conventional commit requires commits message to not start with an uppercase letter

[aDisplayName]

I tried to compile and run your changes on my dev cluster and I'm a bit confused about the values showed in the status of CachedImages. For instance for quay.io/jetstack/cert-manager-cainjector:v1.13.3, it shows a total of 318 (bytes ?) but docker inspect quay.io/jetstack/cert-manager-cainjector:v1.13.3 | jq '.[0].Size' outputs 42445613. So it doesn't really seems to work. Also showing the total size doesn't seems that useful to know the progression, as I would have to download and inspect the image first, or maybe describe the CachedImage in order to compare the status.progress.available vs status.progress.total. It would be much more interesting to have a percentage of completion instead of a total.

PS : conventional commit requires commits message to not start with an uppercase letter

I'm also puzzled by this. Here is the experiment I've done (https://mcr.microsoft.com/en-us/product/dotnet/sdk/tags)

image	reported total / complete value by remote progress update	type
mcr.microsoft.com/dotnet/sdk:8.0.401	318	Manifest List
mcr.microsoft.com/dotnet/sdk:8.0.401-alpine3.19	257371821	Manifest List
mcr.microsoft.com/dotnet/sdk:8.0.401-alpine3.19-amd64	257371503	Docker Image
mcr.microsoft.com/dotnet/sdk:8.0.401-alpine3.20	318	Manifest List
mcr.microsoft.com/dotnet/sdk:8.0.401-alpine3.20-amd64	258143681	Docker Image

Not sure what is happening under the hood from remote.Write and remote.WriteIndex

[plaffitt]

Thanks for the update, but now the code isn't compiling...

Error: internal/controller/kuik/cachedimage_controller.go:350:3: needUpdate declared and not used

[plaffitt]

Not sure what is happening under the hood from remote.Write and remote.WriteIndex

Sorry but I cannot merge something that don't work as expected and I'm a bit worried that you implemented a feature without understanding what you were doing. If you really need this feature please dig a bit more into the functioning of this in order to get a clear understanding of it and make it work as expected.

[aDisplayName]

I would believe the smaller size is the size of the new manifest, when the original layer has already existed, thus why the registry routine does not count it as total written size.
I'll try to find out if the total size of the image can be retrieved from the manifest, and update the total size accordingly.
But the current value does reflect how much data the kuik operator has transferred from the network, which are more close to the reality when using that to estimate the network traffic performance