Preliminary support for HTTP_AUTH.

[mattmoor]

I am still working through a few issues, but I believe the main problem at this point is that we don't support range requests.

[mattmoor]

It looks like Alpine withdrew a package breaking one of the legs?

Error in download: java.io.IOException: Error downloading [https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/libcrypto3-3.1.5-r0.apk] to /home/runner/.bazel/external/_main~apko~examples_multi_arch_and_repo_libcrypto3_aarch64_3.1.5-r0/https%3A%2F%2Fdl-cdn.alpinelinux.org%2Falpine%2Fv3.18%2Fmain/aarch64/libcrypto3-3.1.5-r0/06951e2d641fbcc40718707542c45a30ca1807f6.sig.tar.gz: GET returned 404 Not Found

[imjasonh]

It looks like Alpine withdrew a package breaking one of the legs?

Error in download: java.io.IOException: Error downloading [https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/libcrypto3-3.1.5-r0.apk] to /home/runner/.bazel/external/_main~apko~examples_multi_arch_and_repo_libcrypto3_aarch64_3.1.5-r0/https%3A%2F%2Fdl-cdn.alpinelinux.org%2Falpine%2Fv3.18%2Fmain/aarch64/libcrypto3-3.1.5-r0/06951e2d641fbcc40718707542c45a30ca1807f6.sig.tar.gz: GET returned 404 Not Found

We should probably migrate tests to only pull from Wolfi, where we withdraw less often and have more control in general.

[mattmoor]

This fixes alpine for now by just regenerating the lock, but I agree we should switch to something where we have more control: #85

[sfc-gh-mhazy]

@mattmoor Thank you for the changes. We are quite excited about using this instead of building on top of base image!

I am wondering if we could use similar approach for fetching token within bazel rules as in rules_oci so that the rule don't depend on external env vars. I don't know much details about chainctl, but I am happy to implement the auth in bazel if you think this is feasible.

[mattmoor]

@sfc-gh-mhazy I'm receptive to that. I added HTTP_AUTH mostly for symmetry with apko, melange and apk, but we'd love something better.

I bet @imjasonh and @jonjohnsonjr will have thoughts, but open to ideas for what we'd do here!

[imjasonh]

apko and TF-apko have support for calling chainctl to provide auth for packages, which I think we should support in rules_apko too. They also have code support for doing Chainguard identity auth without chainctl based on env vars, if that's interesting to you.

[sfc-gh-mhazy]

@imjasonh I think ideally the bazel rules would

1. Not depend on chainctl being on host
   • If feasible, fetch the token as in rules_oci, without chainctl command at all. rules_oci seem to work for private chainguard images registries, not sure if this is a good sign here.
   • we could add chainctl toolchain (similar as apko toolchain)
2. Don't fetch the token as part of build, because we want to support "offline" mode.
3. We will need to think how to work with apko_lock which is not hermetic and serves for refreshing purposes.

In rules_apko, the apk fetch happens completely outside of apko, as part of apk.bzl. Perhaps we should fetch token there ("natively" with rctx.download or with chainctl)

Maybe lets move the discussion to some doc?

[imjasonh]

Thanks, this is really great context.

I think the fix will be for apk.bzl to fetch the token (using chainctl or CHAINGUARD_IDENTITY env var, either way), and populate the cache for offline image builds.

Then, when building the image in offline mode, we should (we might already) not attempt to get a token until we're actually about to use it to fetch an apk -- which we won't do if offline mode. I think this might already work this way, or if not, it could fairly easily. This would be a performance benefit even in online-mode, since a token wouldn't be requested if there are no APKs to fetch if they're already in the cache.