-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
🎉 Create a Helm Chart For Airbyte (#5891)
See number #1868. This creates an initial helm chart for installing Airbyte in Kubernetes to make it easier for users who are more familiar with helm. It also includes GitHub actions to help continually test that the chart works in the most basic case. All of the templates are based off of the kustomize folder, but minio and postgres have been removed in favor of adding the bitnami helm charts as dependencies since they have an active community and allow easily tweaking their install.
- Loading branch information
Showing
27 changed files
with
1,898 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| name: Helm | ||
| on: | ||
| push: | ||
| paths: | ||
| - '.github/workflows/helm.yaml' | ||
| - 'charts/**' | ||
| pull_request: | ||
| paths: | ||
| - '.github/workflows/helm.yaml' | ||
| - 'charts/**' | ||
| jobs: | ||
| lint: | ||
| name: Lint | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Setup Kubectl | ||
| uses: azure/setup-kubectl@v1 | ||
| - name: Setup Helm | ||
| uses: azure/setup-helm@v1 | ||
| with: | ||
| version: '3.6.3' | ||
| - name: Lint Chart | ||
| working-directory: ./charts/airbyte | ||
| run: ./ci.sh lint | ||
|
|
||
| generate-docs: | ||
| name: Generate Docs Parameters | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Checkout bitnami-labs/readme-generator-for-helm | ||
| uses: actions/checkout@v2 | ||
| with: | ||
| repository: 'bitnami-labs/readme-generator-for-helm' | ||
| ref: '55cab5dd2191c4ffa7245cfefa428d4d9bb12730' | ||
| path: readme-generator-for-helm | ||
| - name: Install readme-generator-for-helm dependencies | ||
| working-directory: readme-generator-for-helm | ||
| run: npm install -g | ||
| - name: Test can update README with generated parameters | ||
| working-directory: charts/airbyte | ||
| run: ./ci.sh check-docs-updated | ||
|
|
||
| install: | ||
| name: Install | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Setup Kubectl | ||
| uses: azure/setup-kubectl@v1 | ||
| - name: Setup Helm | ||
| uses: azure/setup-helm@v1 | ||
| with: | ||
| version: '3.6.3' | ||
| - name: Setup Kind Cluster | ||
| uses: helm/kind-action@v1.2.0 | ||
| with: | ||
| version: "v0.11.1" | ||
| image: "kindest/node:v1.21.1" | ||
| - name: Install airbyte chart | ||
| working-directory: ./charts/airbyte | ||
| run: ./ci.sh install | ||
| - if: always() | ||
| name: Print diagnostics | ||
| working-directory: ./charts/airbyte | ||
| run: ./ci.sh diagnostics | ||
| - if: success() | ||
| name: Test airbyte chart | ||
| working-directory: ./charts/airbyte | ||
| run: ./ci.sh test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Charts are downloaded at install time with `helm dep build`. | ||
| charts |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Patterns to ignore when building packages. | ||
| # This supports shell glob matching, relative path matching, and | ||
| # negation (prefixed with !). Only one pattern per line. | ||
| .DS_Store | ||
| # Common VCS dirs | ||
| .git/ | ||
| .gitignore | ||
| .bzr/ | ||
| .bzrignore | ||
| .hg/ | ||
| .hgignore | ||
| .svn/ | ||
| # Common backup files | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| *.orig | ||
| *~ | ||
| # Various IDEs | ||
| .project | ||
| .idea/ | ||
| *.tmproj | ||
| .vscode/ | ||
|
|
||
| ci.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| dependencies: | ||
| - name: common | ||
| repository: https://charts.bitnami.com/bitnami | ||
| version: 1.8.0 | ||
| - name: postgresql | ||
| repository: https://charts.bitnami.com/bitnami | ||
| version: 10.9.4 | ||
| - name: minio | ||
| repository: https://charts.bitnami.com/bitnami | ||
| version: 7.2.0 | ||
| digest: sha256:6b5428c4bffa53a16e1015635d712f28b1c1991eea4d048928e4fbb88974cf4f | ||
| generated: "2021-08-31T22:58:47.835303672-05:00" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| apiVersion: v2 | ||
| name: airbyte | ||
| description: Helm chart to deploy airbyte | ||
|
|
||
| # A chart can be either an 'application' or a 'library' chart. | ||
| # | ||
| # Application charts are a collection of templates that can be packaged into versioned archives | ||
| # to be deployed. | ||
| # | ||
| # Library charts provide useful utilities or functions for the chart developer. They're included as | ||
| # a dependency of application charts to inject those utilities and functions into the rendering | ||
| # pipeline. Library charts do not define any templates and therefore cannot be deployed. | ||
| type: application | ||
|
|
||
| # This is the chart version. This version number should be incremented each time you make changes | ||
| # to the chart and its templates, including the app version. | ||
| # Versions are expected to follow Semantic Versioning (https://semver.org/) | ||
| version: 0.1.0 | ||
|
|
||
| # This is the version number of the application being deployed. This version number should be | ||
| # incremented each time you make changes to the application. Versions are not expected to | ||
| # follow Semantic Versioning. They should reflect the version the application is using. | ||
| # It is recommended to use it with quotes. | ||
| appVersion: "0.29.13-alpha" | ||
|
|
||
| dependencies: | ||
| - name: common | ||
| repository: https://charts.bitnami.com/bitnami | ||
| tags: | ||
| - bitnami-common | ||
| version: 1.x.x | ||
| - condition: postgresql.enabled | ||
| name: postgresql | ||
| version: 10.x.x | ||
| repository: https://charts.bitnami.com/bitnami | ||
| - condition: minio.enabled | ||
| name: minio | ||
| version: 7.x.x | ||
| repository: https://charts.bitnami.com/bitnami |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| # airbyte | ||
|
|
||
| ## Parameters | ||
|
|
||
| ### Global Parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | --------------------- | -------------------------------------------- | ----- | | ||
| | `global.storageClass` | Global StorageClass for Persistent Volume(s) | `""` | | ||
|
|
||
|
|
||
| ### Common Parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | ---------------------------- | ------------------------------------------------------------------------------------------------------------------- | --------------- | | ||
| | `nameOverride` | String to partially override airbyte.fullname template with a string (will prepend the release name) | `""` | | ||
| | `fullnameOverride` | String to fully override airbyte.fullname template with a string | `""` | | ||
| | `serviceAccount.annotations` | Annotations for service account. Evaluated as a template. Only used if `create` is `true`. | `{}` | | ||
| | `serviceAccount.create` | Specifies whether a ServiceAccount should be created | `true` | | ||
| | `serviceAccount.name` | Name of the service account to use. If not set and create is true, a name is generated using the fullname template. | `airbyte-admin` | | ||
|
|
||
|
|
||
| ### Webapp Parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | ---------------------------- | ---------------------------------------------------------------- | ---------------- | | ||
| | `webapp.replicaCount` | Number of webapp replicas | `1` | | ||
| | `webapp.image.repository` | The repository to use for the airbyte webapp image. | `airbyte/webapp` | | ||
| | `webapp.image.pullPolicy` | the pull policy to use for the airbyte webapp image | `IfNotPresent` | | ||
| | `webapp.image.tag` | The airbyte webapp image tag. Defaults to the chart's AppVersion | `0.29.13-alpha` | | ||
| | `webapp.podAnnotations` | Add extra annotations to the scheduler pod | `{}` | | ||
| | `webapp.service.type` | The service type to use for the webapp service | `ClusterIP` | | ||
| | `webapp.service.port` | The service port to expose the webapp on | `80` | | ||
| | `webapp.resources.limits` | The resources limits for the Web container | `{}` | | ||
| | `webapp.resources.requests` | The requested resources for the Web container | `{}` | | ||
| | `webapp.nodeSelector` | Node labels for pod assignment | `{}` | | ||
| | `webapp.tolerations` | Tolerations for webapp pod assignment. | `[]` | | ||
| | `webapp.ingress.enabled` | Set to true to enable ingress record generation | `false` | | ||
| | `webapp.ingress.className` | Specifies ingressClassName for clusters >= 1.18+ | `""` | | ||
| | `webapp.ingress.hosts` | Ingress Hosts configuration | `[]` | | ||
| | `webapp.ingress.annotations` | Ingress annotations done as key:value pairs | `{}` | | ||
| | `webapp.ingress.hosts` | The list of hostnames to be covered with this ingress record. | `[]` | | ||
| | `webapp.ingress.tls` | Custom ingress TLS configuration | `[]` | | ||
|
|
||
|
|
||
| ### Scheduler Parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | ------------------------------ | ------------------------------------------------------------------- | ------------------- | | ||
| | `scheduler.replicaCount` | Number of scheduler replicas | `1` | | ||
| | `scheduler.image.repository` | The repository to use for the airbyte scheduler image. | `airbyte/scheduler` | | ||
| | `scheduler.image.pullPolicy` | the pull policy to use for the airbyte scheduler image | `IfNotPresent` | | ||
| | `scheduler.image.tag` | The airbyte scheduler image tag. Defaults to the chart's AppVersion | `0.29.13-alpha` | | ||
| | `scheduler.podAnnotations` | Add extra annotations to the scheduler pod | `{}` | | ||
| | `scheduler.resources.limits` | The resources limits for the scheduler container | `{}` | | ||
| | `scheduler.resources.requests` | The requested resources for the scheduler container | `{}` | | ||
| | `scheduler.nodeSelector` | Node labels for pod assignment | `{}` | | ||
| | `scheduler.tolerations` | Tolerations for scheduler pod assignment. | `[]` | | ||
|
|
||
|
|
||
| ### Pod Sweeper parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | ------------------------------- | ---------------------------------------------------- | ----------------- | | ||
| | `podSweeper.image.repository` | The image repository to use for the pod sweeper | `bitnami/kubectl` | | ||
| | `podSweeper.image.pullPolicy` | The pull policy for the pod sweeper image | `IfNotPresent` | | ||
| | `podSweeper.image.tag` | The pod sweeper image tag to use | `latest` | | ||
| | `podSweeper.podAnnotations` | Add extra annotations to the podSweeper pod | `{}` | | ||
| | `podSweeper.resources.limits` | The resources limits for the podSweeper container | `{}` | | ||
| | `podSweeper.resources.requests` | The requested resources for the podSweeper container | `{}` | | ||
| | `podSweeper.nodeSelector` | Node labels for pod assignment | `{}` | | ||
| | `podSweeper.tolerations` | Tolerations for podSweeper pod assignment. | `[]` | | ||
|
|
||
|
|
||
| ### Server parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | ------------------------------------------- | ---------------------------------------------------------------- | ---------------- | | ||
| | `server.replicaCount` | Number of server replicas | `1` | | ||
| | `server.image.repository` | The repository to use for the airbyte server image. | `airbyte/server` | | ||
| | `server.image.pullPolicy` | the pull policy to use for the airbyte server image | `IfNotPresent` | | ||
| | `server.image.tag` | The airbyte server image tag. Defaults to the chart's AppVersion | `0.29.13-alpha` | | ||
| | `server.podAnnotations` | Add extra annotations to the server pod | `{}` | | ||
| | `server.livenessProbe.enabled` | Enable livenessProbe on the server | `true` | | ||
| | `server.livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `30` | | ||
| | `server.livenessProbe.periodSeconds` | Period seconds for livenessProbe | `10` | | ||
| | `server.livenessProbe.timeoutSeconds` | Timeout seconds for livenessProbe | `1` | | ||
| | `server.livenessProbe.failureThreshold` | Failure threshold for livenessProbe | `3` | | ||
| | `server.livenessProbe.successThreshold` | Success threshold for livenessProbe | `1` | | ||
| | `server.readinessProbe.enabled` | Enable readinessProbe on the server | `true` | | ||
| | `server.readinessProbe.initialDelaySeconds` | Initial delay seconds for readinessProbe | `10` | | ||
| | `server.readinessProbe.periodSeconds` | Period seconds for readinessProbe | `10` | | ||
| | `server.readinessProbe.timeoutSeconds` | Timeout seconds for readinessProbe | `1` | | ||
| | `server.readinessProbe.failureThreshold` | Failure threshold for readinessProbe | `3` | | ||
| | `server.readinessProbe.successThreshold` | Success threshold for readinessProbe | `1` | | ||
| | `server.resources.limits` | The resources limits for the server container | `{}` | | ||
| | `server.resources.requests` | The requested resources for the server container | `{}` | | ||
| | `server.service.type` | The service type to use for the API server | `ClusterIP` | | ||
| | `server.service.port` | The service port to expose the API server on | `8001` | | ||
| | `server.persistence.accessMode` | The access mode for the airbyte server pvc | `ReadWriteOnce` | | ||
| | `server.persistence.size` | The size of the pvc to use for the airbyte server pvc | `1Gi` | | ||
| | `server.persistence.storageClass` | The storage class to use for the airbyte server pvc | `""` | | ||
| | `server.nodeSelector` | Node labels for pod assignment | `{}` | | ||
| | `server.tolerations` | Tolerations for server pod assignment. | `[]` | | ||
|
|
||
|
|
||
| ### Temporal parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | --------------------------- | --------------------------------------------- | ----------------------- | | ||
| | `temporal.replicaCount` | The number of temporal replicas to deploy | `1` | | ||
| | `temporal.image.repository` | The temporal image repository to use | `temporalio/auto-setup` | | ||
| | `temporal.image.pullPolicy` | The pull policy for the temporal image | `IfNotPresent` | | ||
| | `temporal.image.tag` | The temporal image tag to use | `1.7.0` | | ||
| | `temporal.service.type` | The Kubernetes Service Type | `ClusterIP` | | ||
| | `temporal.service.port` | The temporal port and exposed kubernetes port | `7233` | | ||
| | `temporal.nodeSelector` | Node labels for pod assignment | `{}` | | ||
| | `temporal.tolerations` | Tolerations for pod assignment. | `[]` | | ||
|
|
||
|
|
||
| ### Airbyte Database parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | -------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------ | | ||
| | `postgresql.enabled` | Switch to enable or disable the PostgreSQL helm chart | `true` | | ||
| | `postgresql.postgresqlUsername` | Airbyte Postgresql username | `airbyte` | | ||
| | `postgresql.postgresqlPassword` | Airbyte Postgresql password | `airbyte` | | ||
| | `postgresql.postgresqlDatabase` | Airbyte Postgresql database | `db-airbyte` | | ||
| | `postgresql.existingSecret` | Name of an existing secret containing the PostgreSQL password ('postgresql-password' key) | `""` | | ||
| | `externalDatabase.host` | Database host | `localhost` | | ||
| | `externalDatabase.user` | non-root Username for Airbyte Database | `airbyte` | | ||
| | `externalDatabase.password` | Database password | `""` | | ||
| | `externalDatabase.existingSecret` | Name of an existing secret resource containing the DB password | `""` | | ||
| | `externalDatabase.existingSecretPasswordKey` | Name of an existing secret key containing the DB password | `""` | | ||
| | `externalDatabase.database` | Database name | `db-airbyte` | | ||
| | `externalDatabase.port` | Database port number | `5432` | | ||
|
|
||
|
|
||
| ### Minio parameters | ||
|
|
||
| | Name | Description | Value | | ||
| | -------------------------- | ------------------------------------------------ | ----------- | | ||
| | `minio.enabled` | Switch to enable or disable the Minio helm chart | `true` | | ||
| | `minio.accessKey.password` | Minio Access Key | `minio` | | ||
| | `minio.secretKey.password` | Minio Secret Key | `minio123` | | ||
| | `externalMinio.host` | Minio Host | `localhost` | | ||
| | `externalMinio.port` | Minio Port | `9000` | | ||
|
|
||
|
|
Oops, something went wrong.