Vendor charts are not downloaded and included in the bundle, even when exported for an air-gapped install

[Steve973]

I thought that helm charts were supposed to be included in a porter-generated CNAB, but I am not seeing them, even when I export the bundle for an air-gapped install of my bundle. This is my porter manifest:

# Optional indicator of the resource type of this file
schemaType: Bundle

# Version of the porter.yaml schema used by this file
schemaVersion: 1.0.1

# Name of the bundle
name: example-app-stack

# Version of the bundle. Change this each time you modify a published bundle
version: 0.1.0

# Description of the bundle and what it does
description: "An example Porter configuration to install an application stack"

# Registry where the bundle is published by default
registry: "docker.io/steve973"

# Declare and optionally configure the mixins used by the bundle
mixins:
  - exec
  - helm3:
      clientVersion: v3.14.4
      repositories:
        arangodb-charts:
          url: "https://arangodb.github.io/kube-arangodb"
        redpanda-charts:
          url: "https://charts.redpanda.com"
        traefik-charts:
          url: "https://traefik.github.io/charts"

install:
  - helm3:
      description: Install ArangoDB from the vendor chart
      name: arangodb
      chart: arangodb-charts/kube-arangodb
      version: $arangodb-helm-chart-version
      set:
        global.name: arangodb
        namespace: $arangodb-namespace
  - helm3:
      description: Install Redpanda from the vendor chart
      name: redpanda
      chart: redpanda-charts/redpanda/redpanda
      version: $redpanda-helm-chart-version
      set:
        global.name: redpanda
        namespace: $redpanda-namespace
  - helm3:
      description: Install Traefik from the vendor chart
      name: traefik
      chart: traefik-charts/traefik/traefik
      version: $traefik-helm-chart-version
      set:
        global.name: traefik
        namespace: $traefik-namespace

upgrade:
  - helm3:
      description: Upgrade ArangoDB from the vendor chart
      name: arangodb
      chart: arangodb-charts/kube-arangodb
      version: $arangodb-helm-chart-version
      set:
        global.name: arangodb
        namespace: $arangodb-namespace
  - helm3:
      description: Upgrade Redpanda from the vendor chart
      name: redpanda
      chart: redpanda-charts/redpanda/redpanda
      version: $redpanda-helm-chart-version
      set:
        global.name: redpanda
        namespace: $redpanda-namespace
  - helm3:
      description: Upgrade Traefik from the vendor chart
      name: traefik
      chart: traefik-charts/traefik/traefik
      version: $traefik-helm-chart-version
      set:
        global.name: traefik
        namespace: $traefik-namespace

uninstall:
  - helm3:
      description: "Uninstall ArangoDB"
      namespace: $arangodb-namespace
      releases:
        - "arangodb"
  - helm3:
      description: "Uninstall Redpanda"
      namespace: $redpanda-namespace
      releases:
        - "redpanda"
  - helm3:
      description: "Uninstall Traefik"
      namespace: $traefik-namespace
      releases:
        - "traefik"

parameters:
  - name: arangodb-helm-chart-version
    type: string
    default: 1.2.40
  - name: arangodb-namespace
    type: string
    default: arangodb
  - name: redpanda-helm-chart-version
    type: string
    default: 5.7.39
  - name: redpanda-namespace
    type: string
    default: redpanda
  - name: traefik-helm-chart-version
    type: string
    default: 27.0.2
  - name: traefik-namespace
    type: string
    default: traefik

Note that this is not complete, because I need to add the parameters for the charts.

I have a shell script that I am using to build and export the bundle, and then create an archive with porter and the exported bundle:

#!/bin/bash

# Download the Porter CLI binary
PORTER_OS=$(uname -s | tr '[:upper:]' '[:lower:]')
PORTER_OS=${PORTER_OS:-linux}
case "$PORTER_OS" in
  linux*)     PORTER_OS=linux;;
  darwin*)    PORTER_OS=darwin;;
  cygwin*)    PORTER_OS=windows;;
  mingw*)     PORTER_OS=windows;;
  *)          echo "Unsupported operating system: $(uname -s)" && exit 1
esac

if [ "$PORTER_OS" == "linux" ]; then
  PORTER_FILENAME="porter-linux-amd64"
elif [ "$PORTER_OS" == "darwin" ]; then
  PORTER_FILENAME="porter-darwin-amd64"
elif [ "$PORTER_OS" == "windows" ]; then
  PORTER_FILENAME="porter-windows-amd64.exe"
else
  echo "Unsupported operating system: $PORTER_OS"
  exit 1
fi

# Get the latest Porter version
PORTER_VERSION=$(porter version | grep -o -E '[0-9]+\.[0-9]+\.[0-9]+')

# Get the Porter bundle name
BUNDLE_NAME=$(cat ./porter.yaml | grep "name:" | head -n 1 | cut -d':' -f2 | xargs)
BUNDLE_VERSION=$(cat ./porter.yaml | grep "version:" | head -n 1 | cut -d':' -f2 | xargs)
BUNDLE_REGISTRY=$(grep -E '^registry:' porter.yaml | sed 's/"//g' | cut -d':' -f2,3 | xargs)

# Create the "dist" directory if nonexistent
mkdir -p dist

# Switch to the distribution directory
pushd dist || exit 1

# Clear any previous distribution files
rm -rf *

# Download the Porter CLI binary for use on systems that do not have Porter installed
curl -L https://cdn.porter.sh/v${PORTER_VERSION}/$PORTER_FILENAME -o ./porter
chmod +x ./porter

# Build the bundle
porter build --dir ../
porter publish --force --dir ../ --reference ${BUNDLE_REGISTRY}/${BUNDLE_NAME}:v${BUNDLE_VERSION}
porter archive ${BUNDLE_NAME}.tgz --reference ${BUNDLE_REGISTRY}/${BUNDLE_NAME}:v${BUNDLE_VERSION}

# Package the CLI and the bundle in a distributable archive
tar -czf ./${BUNDLE_NAME}-bundle.tar.gz porter ${BUNDLE_NAME}.tgz

popd

When I run the script, this is the output that I am seeing:

~/vscode-projects/porter-deployment-example/dist ~/vscode-projects/porter-deployment-example
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    95  100    95    0     0    150      0 --:--:-- --:--:-- --:--:--   150
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 61.3M  100 61.3M    0     0  26.6M      0  0:00:02  0:00:02 --:--:-- 49.8M
Copying porter runtime ===> 
Copying mixins ===> 
Copying mixin exec ===> 
Copying mixin helm3 ===> 
Building invocation image
[+] Building 5.3s (28/28) FINISHED                                                                                                                                                                                                                                                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                                      0.0s
 => => transferring dockerfile: 1.52kB                                                                                                                                                                                                                                                                                                                                                    0.0s
 => resolve image config for docker-image://docker.io/docker/dockerfile-upstream:1.4.0                                                                                                                                                                                                                                                                                                    0.7s
 => [auth] docker/dockerfile-upstream:pull token for registry-1.docker.io                                                                                                                                                                                                                                                                                                                 0.0s
 => CACHED docker-image://docker.io/docker/dockerfile-upstream:1.4.0@sha256:178c4e4a93795b9365dbf6cf10da8fcf517fcb4a17f1943a775c0f548e9fc2ff                                                                                                                                                                                                                                              0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                                         0.0s
 => => transferring context: 227B                                                                                                                                                                                                                                                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/debian:stable-slim                                                                                                                                                                                                                                                                                                                     0.2s
 => [auth] library/debian:pull token for registry-1.docker.io                                                                                                                                                                                                                                                                                                                             0.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                                                                                         0.7s
 => => transferring context: 260.33MB                                                                                                                                                                                                                                                                                                                                                     0.7s
 => [stage-0  1/19] FROM docker.io/library/debian:stable-slim@sha256:2c96e0011b1b2855d8e482e6d4ced6d292ea50750e9e535d09510167ae4858f9                                                                                                                                                                                                                                                     0.0s
 => CACHED [stage-0  2/19] RUN useradd nonroot -m -u 65532 -g 0 -o                                                                                                                                                                                                                                                                                                                        0.0s
 => CACHED [stage-0  3/19] RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache                                                                                                                                                                                                                         0.0s
 => CACHED [stage-0  4/19] RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt     apt-get update && apt-get install -y ca-certificates                                                                                                                                                                                                                   0.0s
 => CACHED [stage-0  5/19] RUN apt-get update && apt-get install -y curl                                                                                                                                                                                                                                                                                                                  0.0s
 => CACHED [stage-0  6/19] RUN curl https://get.helm.sh/helm-v3.14.4-linux-amd64.tar.gz --output helm3.tar.gz                                                                                                                                                                                                                                                                             0.0s
 => CACHED [stage-0  7/19] RUN tar -xvf helm3.tar.gz && rm helm3.tar.gz                                                                                                                                                                                                                                                                                                                   0.0s
 => CACHED [stage-0  8/19] RUN mv linux-amd64/helm /usr/local/bin/helm3                                                                                                                                                                                                                                                                                                                   0.0s
 => CACHED [stage-0  9/19] RUN curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.22.1/bin/linux/amd64/kubectl &&    mv kubectl /usr/local/bin && chmod a+x /usr/local/bin/kubectl                                                                                                                                                                              0.0s
 => CACHED [stage-0 10/19] RUN helm3 repo add arangodb-charts https://arangodb.github.io/kube-arangodb                                                                                                                                                                                                                                                                                    0.0s
 => CACHED [stage-0 11/19] RUN helm3 repo add redpanda-charts https://charts.redpanda.com                                                                                                                                                                                                                                                                                                 0.0s
 => CACHED [stage-0 12/19] RUN helm3 repo add traefik-charts https://traefik.github.io/charts                                                                                                                                                                                                                                                                                             0.0s
 => CACHED [stage-0 13/19] RUN helm3 repo update                                                                                                                                                                                                                                                                                                                                          0.0s
 => [stage-0 14/19] COPY --link . /cnab/app                                                                                                                                                                                                                                                                                                                                               0.7s
 => [stage-0 15/19] RUN rm /cnab/app/porter.yaml                                                                                                                                                                                                                                                                                                                                          0.3s
 => [stage-0 16/19] RUN rm -fr /cnab/app/.cnab                                                                                                                                                                                                                                                                                                                                            0.4s
 => [stage-0 17/19] COPY --link .cnab /cnab                                                                                                                                                                                                                                                                                                                                               0.3s
 => [stage-0 18/19] RUN chgrp -R 0 /cnab && chmod -R g=u /cnab                                                                                                                                                                                                                                                                                                                            0.3s
 => [stage-0 19/19] WORKDIR /cnab/app                                                                                                                                                                                                                                                                                                                                                     0.1s
 => exporting to image                                                                                                                                                                                                                                                                                                                                                                    1.2s
 => => exporting layers                                                                                                                                                                                                                                                                                                                                                                   1.2s
 => => writing image sha256:2d64bdd8f3add4f5ffdab8b2f2fcc33b8ba2c45f4002b0c0f732259ec6d90d6f                                                                                                                                                                                                                                                                                              0.0s
 => => naming to docker.io/steve973/example-app-stack:porter-f17692df40ebc0fbc6d664e2c079791f                                                                                                                                                                                                                                                                                             0.0s
Pushing bundle image...
The push refers to repository [docker.io/steve973/example-app-stack]

The charts do not seem to be included anywhere. I am assuming that this is a job for the helm3 mixin, but please let me know if I am wrong about that. Also, if I am doing something wrong that prevents the charts from being downloaded and included, please let me know. I have read and re-read the documentation for both porter and for the helm3 mixin, but I could have missed something. And if you need more information, I'll provide whatever I can.

As a convenience, I have pushed this up to a git repo to make it easier: https://github.com/Steve973/porter-example-app-stack

[MChorfa]

Thank you for reporting this ... I will take a look

[Steve973]

Thank you for reporting this ... I will take a look

No problem! Please don't hesitate to ask if you need anything from me. I am implementing a workaround where I simply download the charts and package them up with my bundle. But if your plugin could do that, that would be so perfect!