Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

helm: factor out logic from controller into package #485

Merged
merged 23 commits into from
Nov 22, 2021
Merged

helm: factor out logic from controller into package #485

merged 23 commits into from
Nov 22, 2021

Commits on Nov 19, 2021

  1. internal/helm: add helpers to load chart metadata 

    This commits adds `LoadChartMetadataFromArchive` and
`LoadChartMetadataFromDir` helpers to the internal `helm` package
to be able to make observations to the Helm metadata file without
loading the chart in full.

The helpers are compatible with charts of the v1 format (with a
separate `requirements.yaml` file), and an additional
`LoadChartMetadata` helper is available to automatically call the
right `LoadChartMetadataFrom*` version by looking at the file
description of the given path.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    8537a0f View commit details
    Browse the repository at this point in the history

  2. internal/helm: add repository cache helpers 

    This commits adds simple caching capabilities to the
`ChartRepository`, which makes it possible to load the `Index` from a
defined `CachePath` using `LoadFromCache()`, and to download the index
to a new `CachePath` using `CacheIndex()`.

In addition, the repository tests have been updated to make use of
Gomega, and some missing ones have been added.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    44c1863 View commit details
    Browse the repository at this point in the history

  3. internal/helm: optimize dependency manager 

    This commit starts with the optimization of the `DepenendencyManager`,
ensuring the chart indexes are lazy loaded, and replacing the
(limitless) concurrency with a configurable number of workers with a
default of 1.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    d60131d View commit details
    Browse the repository at this point in the history

  4. internal/helm: introduce ChartBuilder 

    This commit starts with the creation of a `ChartBuilder` to facilitate
the (conditional) build of a chart outside of the reconciler logic.

The builder can be configured with a set of (modifying) options, which
define together with the type of chart source what steps are taken
during the build.

To better facilitate the builder's needs and attempt to be more
efficient, changes have been made to the `DependencyBuilder` and
`ChartRepository` around (order of) operations and/or lazy-load
capabilities.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    f5f212f View commit details
    Browse the repository at this point in the history

  5. controllers: wire ChartRepository in reconciler 

    This wires the `ChartRepository` changes into the reconciler to ensure
it works.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    d23bcbb View commit details
    Browse the repository at this point in the history

  6. internal/helm: make ChartBuilder an interface 

    This commit refactors the `ChartBuilder` that used to be a do-it-all
struct into an interace with two implementations:

- `LocalChartBuilder`: to build charts from a source on the local
  filesystem, either from a directory or from a packaged chart.
- `RemoteChartBuilder`: to build charts from a remote Helm repository
  index.

The new logic within the builders validates the size of the Helm size
it works with based on the `Max*Size` global variables in the internal
`helm` package, to address the recommendation from the security audit.

In addition, changes `ClientOptionsFromSecret` takes now a directory
argument which temporary files are placed in, making it easier to
perform a garbage collection of the whole directory at the end of a
reconcile run.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    52459c8 View commit details
    Browse the repository at this point in the history

  7. controllers: rough wiring of Helm chart builder 

    This commit starts wiring the factored out Helm chart build logic into
the reconciler to ensure, validating the API capabilities.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    9abbdd8 View commit details
    Browse the repository at this point in the history

  8. internal/helm: divide into subpackages 

    With all the logic that used to reside in the `controllers` package
factored into this package, it became cluttered. This commit tries to
bring a bit more structure in place.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    7d0f79f View commit details
    Browse the repository at this point in the history

  9. controllers: more tidying of wiring 

    Dealing with some loose ends around making observations, and code
style.

The loaded byes of a chart are used as a revision to ensure e.g.
periodic builds with unstable ordering of items do not trigger a false
positive.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    32e19eb View commit details
    Browse the repository at this point in the history

  10. internal/helm: local builder & dep manager test 

    Add more chart local builder and dependency manager tests.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
    @darkowlzz @hiddeco
    darkowlzz authored and hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    7c910e3 View commit details
    Browse the repository at this point in the history

  11. internal/helm: add remote builder tests 

    - For remote builds, if the build option has a version metadata, the
  chart should be repackaged with the provided version.
- Update internal/helm/testdata/charts/helmchart-0.1.0.tgz to include
  value files for testing merge chart values.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
    @darkowlzz @hiddeco
    darkowlzz authored and hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    753abed View commit details
    Browse the repository at this point in the history

  12. internal/helm: add cached chart build tests 

    Cached chart build tests for both local and remote builder.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
    @darkowlzz @hiddeco
    darkowlzz authored and hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    dd3afce View commit details
    Browse the repository at this point in the history

  13. internal/helm: tweak and test chart build summary 

    This makes the string less verbose and deals with the safe handling
of some edge-case build states.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    ef05173 View commit details
    Browse the repository at this point in the history

  14. internal/helm: add more tests 

    Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    4fd6e6e View commit details
    Browse the repository at this point in the history

  15. internal/helm: introduce typed BuildError 

    This commit introduces a typed `BuildError` to be returned by
`Builder.Build` in case of a failure.

The `Reason` field in combination with `BuildErrorReason` can be used
to signal (or determine) the reason of a returned error within the
context of the build process.

At present this is used to determine the correct Condition Reason, but
in a future iteration this can be used to determine the negative
polarity condition that should be set to indicate a precise failure to
the user.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    2b8134c View commit details
    Browse the repository at this point in the history

  16. internal/helm: test load funcs for max size cases 

    This includes a change of the defaults to more acceptible (higher)
values.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
    @darkowlzz @hiddeco
    darkowlzz authored and hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    37ac5a9 View commit details
    Browse the repository at this point in the history

  17. internal/helm: "value files" -> "values files" 

    Previous usage while consistent, was incorrect, and inconsitent with
the field in the API spec.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    a1e9302 View commit details
    Browse the repository at this point in the history

  18. Allow configuration of Helm file limits 

    This allows custom configuration of the Helm file read limits, allowing
a user to overwrite them to their likenings if the defaults are too
restrictive for their specific setup using arguments:

`--helm-{index,chart,chart-file}-max-size`

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    4de8f1f View commit details
    Browse the repository at this point in the history

  19. internal/helm: various nitpicks 

    - Add some more documentation around chart builders
- Ensure correct indentation in some doc comments
- Provide example of using `errors.Is` for typed `BuildError`
- Mention "bytes" in file size limit errors
- Add missing copyright header

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    dcd5dd3 View commit details
    Browse the repository at this point in the history

  20. helm/internal: add ErrChartReference 

    This makes it possible to signal reference (validation) errors
happening before the build process actually starts dealing with
the chart.

At present, this does not have a more specific counterpart in the API,
but this is expected to change when the conditions logic is revised.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    c202ad5 View commit details
    Browse the repository at this point in the history

  21. controllers: set generation as version metadata 

    By providing the Generation of the object that is getting reconciled
as version metadata to the builder if any custom values files are
defined, the Artifact revision changes if the specification does,
ensuring consumers of the Artifact are able to react to changes in
values (and perform a release).

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    472eb12 View commit details
    Browse the repository at this point in the history

  22. internal/helm: ensure cached chart name matches 

    This helps detect e.g. path or chart name reference changes.

Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    88ff049 View commit details
    Browse the repository at this point in the history

  23. internal/helm: doc block nitpicks 

    Signed-off-by: Hidde Beydals <hello@hidde.co>
    @hiddeco
    hiddeco committed Nov 19, 2021
    Configuration menu
    Copy the full SHA
    2392326 View commit details
    Browse the repository at this point in the history