Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support apko configs that consist of multiple files. #64

Merged
merged 29 commits into from
May 14, 2024

Conversation

sfc-gh-mhazy
Copy link
Contributor

@sfc-gh-mhazy sfc-gh-mhazy commented Apr 24, 2024

Overview

This PR is paving the way for image configurations that consist of multiple files. Currently an example of such configuration is any include usage, but in the future there will be more, for instance base image that is part of chainguard-dev/apko#806.

Example

a.apko.yaml

contents
  repositories:
    - foo
   packages:
    - bar

b.apko.yaml

include: a.apko.yaml

env:
  FOO=BAR

c.apko.yaml

include: b.apko.yaml

env:
   X=Y

and finally

apko_image(
   config = "c.apko.yaml"
)

Details

RIght now the only file added to the build action in apko_image is the direct config. Therefore, the multi file config will fail to be evaluated by apko.

This PR introduces the apko_config rule that allows setting deps and ApkoConfigInfo provider to support transitive dependencies. We want to support including both source files and generated targets with the path referencing convention that both source and generated files are referenced as paths relative to workspace root.

Additionally I added dedicated apko_lock rule to decouple it from the one generated by apko_image. With the complex configs the restrictions in the latter were becoming problematic.

For the example above it would work as

BUILD

apko_config(
  name = "b",
  config = "b.apko.yaml",
  deps = [":a.apko.yaml"],
)

apko_config(
  name = "c",
  config = "c.apko.yaml",
  deps = [":b"],
)

apko_image(
   config = ":c"
)

c.apko.yaml

include: b

env:
   X=Y

The change is supposed to be fully backwards compatible - single file configs without ApkoConfigInfo providers are still supported.

Alternative considered

Move to the Bazel based apko configs apko_config, which may accept the parameters for every corresponding element of apko config.

How it works for example above

a.apko.yaml - remains as is

b.apko.yaml

env:
  FOO=BAR

and in BUILD

apko_config(
   name = "b"
   include = "a.apko.yaml"
   config = "b.apko.yaml"
)

c.apko.yaml

env:
   X=Y

and in BUILD

apko_config(
   name="c",
   include = ":b",
   config = "c.apko.yaml"

apko/private/apko_run.bzl Outdated Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
e2e/smoke/apko_config.bzl Outdated Show resolved Hide resolved
e2e/smoke/BUILD Outdated Show resolved Hide resolved
e2e/smoke/BUILD Show resolved Hide resolved
e2e/smoke/BUILD Outdated Show resolved Hide resolved
@sfc-gh-mhazy sfc-gh-mhazy changed the title Allow adding additional files that are used by apko config to build and lock. Support apko configs that consist of multiple files. Apr 30, 2024
@sfc-gh-mhazy sfc-gh-mhazy marked this pull request as ready for review May 9, 2024 12:51
Copy link
Collaborator

@sfc-gh-ptabor sfc-gh-ptabor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refresh of examples/wolfi-base/apko.lock.json makes this PR look huge... but it's actually not that big.

apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/image_config.bzl Outdated Show resolved Hide resolved
docs/rules.md Outdated Show resolved Hide resolved
e2e/smoke/test.sh Outdated Show resolved Hide resolved
Copy link
Member

@imjasonh imjasonh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @thesayyn as well, since he might be able to spot issues with this better than @jonjohnsonjr or myself.

Overall this seems to make sense to me, but I'm only ~30% sure I'm reading it right 😄

e2e/smoke/test.sh Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/apko_image.bzl Outdated Show resolved Hide resolved
apko/private/apko_lock.bzl Outdated Show resolved Hide resolved
@sfc-gh-ptabor
Copy link
Collaborator

@sfc-gh-mhazy now the docs require regeneration after the spelling fixes.

sfc-gh-mhazy and others added 19 commits May 14, 2024 09:38
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Co-authored-by: Piotr Tabor (ptab) <132376425+sfc-gh-ptabor@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <158565055+sfc-gh-mhazy@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
sfc-gh-mhazy and others added 9 commits May 14, 2024 09:39
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Co-authored-by: Jon Johnson <jonjohnsonjr@gmail.com>
Signed-off-by: Piotr Tabor (ptab) <132376425+sfc-gh-ptabor@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Co-authored-by: Jon Johnson <jonjohnsonjr@gmail.com>
Signed-off-by: Piotr Tabor (ptab) <132376425+sfc-gh-ptabor@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Co-authored-by: Jon Johnson <jonjohnsonjr@gmail.com>
Signed-off-by: Piotr Tabor (ptab) <132376425+sfc-gh-ptabor@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Co-authored-by: Jon Johnson <jonjohnsonjr@gmail.com>
Signed-off-by: Piotr Tabor (ptab) <132376425+sfc-gh-ptabor@users.noreply.github.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
Signed-off-by: Mateusz Hazy <mateusz.hazy@snowflake.com>
@jonjohnsonjr jonjohnsonjr merged commit 44a98c9 into chainguard-dev:main May 14, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants