Skip to content

Commit

Permalink
Airbyte Pro docs (airbytehq#29360)
Browse files Browse the repository at this point in the history
* add airbyte pro docs

* backlink from deployment
  • Loading branch information
josephkmh authored and harrytou committed Sep 1, 2023
1 parent d20e711 commit f8c6f68
Show file tree
Hide file tree
Showing 6 changed files with 99 additions and 23 deletions.
62 changes: 62 additions & 0 deletions docs/airbyte-pro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Airbyte Pro

Airbyte Pro is a self-hosted version of Airbyte with additional features for enterprise customers. Airbyte Pro is currently in early development. Interested in Airbyte Pro for your organization? [Talk to sales](https://airbyte.com/company/talk-to-sales).

## Airbyte Pro License Key

A valid license key is required for Airbyte Pro. Talk to your Sales Representative to receive your license key before installing Airbyte Pro.

## Single Sign-On (SSO)

Airbyte Pro supports Single Sign-On, allowing an organization to manage user acces to their Airbyte Pro instance through the configuration of an Identity Provider (IdP).

Airbyte Pro currently supports SSO via OIDC with [Okta](https://www.okta.com/) as an IdP.

### Setting up Okta for SSO

You will need to create a new Okta OIDC App Integration for your Airbyte instance. Documentation on how to do this in Okta can be found [here](https://help.okta.com/en-us/Content/Topics/Apps/Apps_App_Integration_Wizard_OIDC.htm).

You should create an app integration with **OIDC - OpenID Connect** as the sign-in method and **Web Application** as the application type:

![Screenshot of Okta app integration creation modal](./assets/docs/okta-create-new-app-integration.png)

#### App integration name

Please choose a URL-friendly app integraiton name without spaces or special characters, such as `my-airbyte-app`:

![Screenshot of Okta app integration name](./assets/docs/okta-app-integration-name.png)

Spaces or special characters in this field could result in invalid redirect URIs.

#### Redirect URIs

In the **Login** section, set the following fields, substituting `<your-airbyte-domain>` and `<app-integration-name>` for your own values:

Sign-in redirect URIs:

```
<your-airbyte-domain>/auth/realms/airbyte/broker/<app-integration-name>/endpoint
```

Sign-out redirect URIs

```
<your-airbyte-domain>/auth/realms/airbyte/broker/<app-integration-name>/endpoint/logout_response
```

![Okta app integration name screenshot](./assets/docs/okta-login-redirect-uris.png)

_Example values_

`<your-airbyte-domain>` should point to where your Airbyte instance will be available, including the http/https protocol.

#### Deploying Airbyte Pro with Okta

Your Okta app is now set up and you're ready to deploy Airbyte with SSO! Take note of the following configuration values, as you will need them to configure Airbyte to use your new Okta SSO app integration:

- Okta domain ([how to find your Okta domain](https://developer.okta.com/docs/guides/find-your-domain/main/))
- App integration name
- Client ID
- Client Secret

Visit [Airbyte Pro deployment](/deploying-airbyte/on-kubernetes-via-helm#alpha-airbyte-pro-deployment) for instructions on how to deploy Airbyte Pro using `kubernetes`, `kubectl` and `helm`.
Binary file added docs/assets/docs/okta-app-integration-name.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/docs/okta-login-redirect-uris.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
56 changes: 33 additions & 23 deletions docs/deploying-airbyte/on-kubernetes-via-helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Alternatively, you can deploy Airbyte on [Restack](https://www.restack.io) to pr

For local testing we recommend following one of the following setup guides:

* [Docker Desktop \(Mac\)](https://docs.docker.com/desktop/kubernetes)
* [Minikube](https://minikube.sigs.k8s.io/docs/start)
* NOTE: Start Minikube with at least 4gb RAM with `minikube start --memory=4000`
* [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/)
- [Docker Desktop \(Mac\)](https://docs.docker.com/desktop/kubernetes)
- [Minikube](https://minikube.sigs.k8s.io/docs/start)
- NOTE: Start Minikube with at least 4gb RAM with `minikube start --memory=4000`
- [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/)

For testing on GKE you can [create a cluster with the command line or the Cloud Console UI](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster).

Expand All @@ -40,7 +40,7 @@ For GKE:
1. Configure `gcloud` with `gcloud auth login`.
2. On the Google Cloud Console, the cluster page will have a `Connect` button, which will give a command to run locally that looks like

`gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE_NAME --project $PROJECT_NAME`.
`gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE_NAME --project $PROJECT_NAME`.

3. Use `kubectl config get-contexts` to show the contexts available.
4. Run `kubectl config use-context $GKE_CONTEXT` to access the cluster from `kubectl`.
Expand All @@ -58,8 +58,8 @@ For EKS:
To install helm simply run:

For MacOS:
`brew install helm`

`brew install helm`

For Linux:

Expand All @@ -82,14 +82,14 @@ After this you can browse all charts uploaded to repository by running `helm sea
It'll produce the output below:

```text
NAME CHART VERSION APP VERSION DESCRIPTION
airbyte-oss/airbyte 0.30.23 0.39.37-alpha Helm chart to deploy airbyte
airbyte-oss/airbyte-bootloader 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-bootloader
NAME CHART VERSION APP VERSION DESCRIPTION
airbyte-oss/airbyte 0.30.23 0.39.37-alpha Helm chart to deploy airbyte
airbyte-oss/airbyte-bootloader 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-bootloader
airbyte-oss/pod-sweeper 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-pod-sweeper
airbyte-oss/server 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-server
airbyte-oss/temporal 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-temporal
airbyte-oss/webapp 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-webapp
airbyte-oss/worker 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-worker
airbyte-oss/server 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-server
airbyte-oss/temporal 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-temporal
airbyte-oss/webapp 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-webapp
airbyte-oss/worker 0.30.23 0.39.37-alpha Helm chart to deploy airbyte-worker
```

## Deploy Airbyte
Expand All @@ -98,7 +98,8 @@ airbyte-oss/worker 0.30.23 0.39.37-alpha Helm chart to de

If you don't intend to customise your deployment, you can deploy airbyte as is with default values.

In order to do so, run the command:
In order to do so, run the command:

```
helm install %release_name% airbyte/airbyte
```
Expand All @@ -117,14 +118,16 @@ helm install --values path/to/values.yaml %release_name% airbyte/airbyte

### (Alpha) Airbyte Pro deployment

Airbyte Pro is in early alpha stages, so this section will likely evolve. That said, if you have an Airbyte Pro license key and wish to install Airbyte Pro via helm, follow these steps:
[Airbyte Pro](/airbyte-pro) is in early alpha stages, so this section will likely evolve. That said, if you have an Airbyte Pro license key and wish to install Airbyte Pro via helm, follow these steps:

1. Checkout the latest revision of the [airbyte-platform repository](https://github.com/airbytehq/airbyte-platform)

2. Add your Airbyte Pro license key and auth configuration details to a file called `airbyte.yml` in the root directory of `airbyte-platform`. You can copy `airbyte.sample.yml` to use as a template:
2. Add your Airbyte Pro license key and [auth configuration details](/airbyte-pro#single-sign-on-sso) to a file called `airbyte.yml` in the root directory of `airbyte-platform`. You can copy `airbyte.sample.yml` to use as a template:

```text
cp airbyte.sample.yml airbyte.yml
```

Then, open up `airbyte.yml` in your text editor to fill in the indicated fields.

:::caution
Expand All @@ -134,16 +137,18 @@ For now, auth configurations aren't easy to modify once initially installed, so
:::

3. Make sure your helm repository is up to date:

```text
helm repo update
```

4. Install Airbyte Pro on helm using the following command:

```text
RELEASE_NAME=<your release name>./tools/bin/install_airbyte_pro_on_helm.sh
```
If unspecified, the default release name is `airbyte-pro`. You can change this by editing the `install_airbyte_pro_on_helm.sh` script.

If unspecified, the default release name is `airbyte-pro`. You can change this by editing the `install_airbyte_pro_on_helm.sh` script.

## Migrate from old charts to new ones

Expand All @@ -158,12 +163,15 @@ Since the latest release of bitnami/minio chart, they've changed the way of sett
Going forward in new version you need to specify the following values in values yaml for user/password instead old one

Before:

```text
minio:
rootUser: airbyte-user
rootPassword: airbyte-password-123
```

After:

```text
minio:
auth:
Expand All @@ -174,9 +182,9 @@ minio:

Before upgrading the chart update values.yaml as stated above and then run:

* Get the old rootPassword by running `export ROOT_PASSWORD=$(kubectl get secret --namespace "default" %release_name%-minio -o jsonpath="{.data.root-password}" | base64 -d)`
* Perform upgrade of chart by running `helm upgrade %release_name% airbyte/airbyte --set auth.rootPassword=$ROOT_PASSWORD`
* If you get an error about setting the auth.rootPassword, then you forgot to update the `values.yaml` file
- Get the old rootPassword by running `export ROOT_PASSWORD=$(kubectl get secret --namespace "default" %release_name%-minio -o jsonpath="{.data.root-password}" | base64 -d)`
- Perform upgrade of chart by running `helm upgrade %release_name% airbyte/airbyte --set auth.rootPassword=$ROOT_PASSWORD`
- If you get an error about setting the auth.rootPassword, then you forgot to update the `values.yaml` file

### Custom logging and jobs configuration

Expand All @@ -192,7 +200,8 @@ global:
%your_jobs_options_here%
```

After updating `values.yaml` simply upgrade your chart by running command:
After updating `values.yaml` simply upgrade your chart by running command:

```shell
helm upgrade -f path/to/values.yaml %release_name% airbyte/airbyte
```
Expand All @@ -209,7 +218,8 @@ If you're using external DB secrets, then provide them in `values.yaml` under gl
port: "5432"
```

And upgrade the chart by running:
And upgrade the chart by running:

```shell
helm upgrade -f path/to/values.yaml %release_name% airbyte/airbyte
```
4 changes: 4 additions & 0 deletions docusaurus/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,10 @@ module.exports = {
type: "doc",
id: "troubleshooting",
},
{
type: 'doc',
id: 'airbyte-pro',
},
sectionHeader("Developer Guides"),
{
type: "doc",
Expand Down

0 comments on commit f8c6f68

Please sign in to comment.