Update examples and docs for backup locations

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
vmware-tanzu · Aug 28, 2018 · 6445dbf · 6445dbf
1 parent 133dc18
commit 6445dbf
Show file tree

Hide file tree

Showing 13 changed files with 201 additions and 95 deletions.
diff --git a/docs/aws-config.md b/docs/aws-config.md
@@ -141,7 +141,11 @@ Specify the following values in the example files:
 
 * In `examples/aws/00-ark-config.yaml`:
 
-  * Replace `<YOUR_BUCKET>` and `<YOUR_REGION>` (for S3, region is optional and will be queried from the AWS S3 API if not provided). See the [Config definition][6] for details.
+  * Replace `<YOUR_REGION>`. See the [Config definition][6] for details.
+
+* In `examples/aws/05-ark-backupstoragelocation.yaml`:
+
+  * Replace `<YOUR_BUCKET>` and `<YOUR_REGION>` (for S3 backup storage, region is optional and will be queried from the AWS S3 API if not provided). See the [BackupStorageLocation definition][21] for details.
 
 * (Optional) If you run the nginx example, in file `examples/nginx-app/with-pv.yaml`:
 
@@ -273,4 +277,5 @@ It can be set up for Ark by creating a role that will have required permissions,
 [5]: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html
 [6]: config-definition.md#aws
 [14]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html
-[20]: faq.md
+[20]: faq.md
+[21]: backupstoragelocation-definition.md#aws
diff --git a/docs/azure-config.md b/docs/azure-config.md
@@ -135,9 +135,13 @@ Now that you have your Azure credentials stored in a Secret, you need to replace
 
 * In file `examples/azure/10-ark-config.yaml`:
 
-  * Replace `<YOUR_BUCKET>` and `<YOUR_TIMEOUT>`. See the [Config definition][8] for details.
+  * Replace `<YOUR_TIMEOUT>`. See the [Config definition][8] for details.
 
-Here is an example of a completed file.
+* In file `examples/azure/05-ark-backupstoragelocation.yaml`:
+
+  * Replace `<YOUR_BUCKET>`. See the [BackupStorageLocation definition][21] for details.
+
+Here is an example of a completed config file.
 
 ```yaml
 apiVersion: ark.heptio.com/v1
@@ -149,9 +153,6 @@ persistentVolumeProvider:
   name: azure
   config:
     apiTimeout: 15m
-backupStorageProvider:
-  name: azure
-  bucket: ark
 backupSyncPeriod: 30m
 gcSyncPeriod: 30m
 scheduleSyncPeriod: 1m
@@ -166,9 +167,10 @@ In the root of your Ark directory, run:
   kubectl apply -f examples/azure/
   ```
 
-  [0]: namespace.md
-  [8]: config-definition.md#azure
-  [17]: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects
-  [18]: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli
-  [19]: https://docs.microsoft.com/en-us/azure/architecture/best-practices/naming-conventions#storage
-  [20]: faq.md
+[0]: namespace.md
+[8]: config-definition.md#azure
+[17]: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects
+[18]: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli
+[19]: https://docs.microsoft.com/en-us/azure/architecture/best-practices/naming-conventions#storage
+[20]: faq.md
+[21]: backupstoragelocation-definition.md#azure
diff --git a/docs/backupstoragelocation-definition.md b/docs/backupstoragelocation-definition.md
@@ -0,0 +1,65 @@
+# Ark Backup Storage Locations
+
+## Backup Storage Location
+
+Ark can store backups in a number of locations. These are represented in the cluster via the `BackupStorageLocation` CRD.
+
+Ark must have at least one `BackupStorageLocation`. By default, this is expected to be named `default`, however the name can be changed by specifying `--default-backup-storage-location` on `ark server`.  Backups that do not explicitly specify a storage location will be saved to this `BackupStorageLocation`.
+
+> *NOTE*: `BackupStorageLocation` takes the place of the `Config.backupStorageProvider` key as of v0.10.0
+
+A sample YAML `BackupStorageLocation` looks like the following:
+
+```yaml
+apiVersion: ark.heptio.com/v1
+kind: BackupStorageLocation
+metadata:
+  name: default
+  namespace: heptio-ark
+spec:
+  provider: aws
+  objectStorage:
+    bucket: myBucket
+    config:
+      region: us-west-2
+```
+
+### Parameter Reference
+
+The configurable parameters are as follows:
+
+#### Main config parameters
+
+| Key | Type | Default | Meaning |
+| --- | --- | --- | --- |
+| `provider` | String (Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.)| Required Field | The name for whichever cloud provider will be used to actually store the backups. |
+| `objectStorage` | ObjectStorageLocation | Specification of the object storage for the given provider. |
+| `objectStorage/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. |
+| `objectStorage/prefix` | String | Optional Field | The directory inside a storage bucket where backups are to be uploaded. |
+| `objectStorage/config` | map[string]string<br><br>(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for backup storage. |
+
+#### AWS
+
+**(Or other S3-compatible storage)**
+
+##### objectStorage/config
+
+| Key | Type | Default | Meaning |
+| --- | --- | --- | --- |
+| `region` | string | Empty | *Example*: "us-east-1"<br><br>See [AWS documentation][3] for the full list.<br><br>Queried from the AWS S3 API if not provided. |
+| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. |
+| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000<br><br>You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, and `bucket`. This field is primarily for local storage services like Minio.|
+| `kmsKeyId` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f" or "alias/`<KMS-Key-Alias-Name>`"<br><br>Specify an [AWS KMS key][10] id or alias to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.|
+
+#### GCP
+
+No parameters required.
+
+#### Azure
+
+No parameters required.
+
+[0]: #aws
+[1]: #gcp
+[2]: #azure
+[3]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions
diff --git a/docs/config-definition.md b/docs/config-definition.md
@@ -31,11 +31,6 @@ persistentVolumeProvider:
   name: aws
   config:
     region: us-west-2
-backupStorageProvider:
-  name: aws
-  bucket: ark
-  config:
-    region: us-west-2
 ```
 
 ### Parameter Reference
@@ -49,24 +44,9 @@ The configurable parameters are as follows:
 | `persistentVolumeProvider` | CloudProviderConfig | None (Optional) | The specification for whichever cloud provider the cluster is using for persistent volumes (to be snapshotted), if any.<br><br>If not specified, Backups and Restores requesting PV snapshots & restores, respectively, are considered invalid. <br><br> *NOTE*: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks. |
 | `persistentVolumeProvider/name` | String<br><br>(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | None (Optional) | The name of the cloud provider the cluster is using for persistent volumes, if any. |
 | `persistentVolumeProvider/config` | map[string]string<br><br>(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for persistent volumes.  |
-| `backupStorageProvider` | CloudProviderConfig | Required Field | The specification for whichever cloud provider will be used to actually store the backups. |
-| `backupStorageProvider/name` | String<br><br>(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | Required Field | The name of the cloud provider that will be used to actually store the backups. |
-| `backupStorageProvider/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. |
-| `backupStorageProvider/config` | map[string]string<br><br>(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for backup storage. |
 
 #### AWS
 
-**(Or other S3-compatible storage)**
-
-##### backupStorageProvider/config
-
-| Key | Type | Default | Meaning |
-| --- | --- | --- | --- |
-| `region` | string | Empty | *Example*: "us-east-1"<br><br>See [AWS documentation][3] for the full list.<br><br>Queried from the AWS S3 API if not provided. |
-| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. |
-| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000<br><br>You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, and `bucket`. This field is primarily for local storage services like Minio.|
-| `kmsKeyId` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f" or "alias/`<KMS-Key-Alias-Name>`"<br><br>Specify an [AWS KMS key][10] id or alias to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.|
-
 ##### persistentVolumeProvider/config (AWS Only)
 
 | Key | Type | Default | Meaning |
@@ -75,20 +55,12 @@ The configurable parameters are as follows:
 
 #### GCP
 
-#### backupStorageProvider/config
-
-No parameters required.
-
 ##### persistentVolumeProvider/config
 
 No parameters required.
 
 #### Azure
 
-##### backupStorageProvider/config
-
-No parameters required.
-
 ##### persistentVolumeProvider/config
 
 | Key | Type | Default | Meaning |

diff --git a/docs/gcp-config.md b/docs/gcp-config.md
@@ -112,9 +112,9 @@ _Note: If you use a custom namespace, replace `heptio-ark` with the name of the
 
 Specify the following values in the example files:
 
-* In file `examples/gcp/00-ark-config.yaml`:
+* In file `examples/gcp/05-ark-backupstoragelocation.yaml`:
 
-  * Replace `<YOUR_BUCKET>`. See the [Config definition][7] for details.
+  * Replace `<YOUR_BUCKET>`. See the [BackupStorageLocation definition][7] for details.
 
 * (Optional) If you run the nginx example, in file `examples/nginx-app/with-pv.yaml`:
 
@@ -130,7 +130,7 @@ In the root of your Ark directory, run:
   ```
 
   [0]: namespace.md
-  [7]: config-definition.md#gcp
+  [7]: backupstoragelocation-definition.md#gcp
   [15]: https://cloud.google.com/compute/docs/access/service-accounts
   [16]: https://cloud.google.com/sdk/docs/
   [20]: faq.md

diff --git a/docs/ibm-config.md b/docs/ibm-config.md
@@ -53,9 +53,9 @@ kubectl create secret generic cloud-credentials \
 
 Specify the following values in the example files:
 
-* In `examples/ibm/00-ark-config.yaml`:
+* In `examples/ibm/05-ark-backupstoragelocation.yaml`:
 
-  * Replace `<YOUR_BUCKET>`, `<YOUR_REGION>` and `<YOUR_URL_ACCESS_POINT>`. See the [Config definition][6] for details.
+  * Replace `<YOUR_BUCKET>`, `<YOUR_REGION>` and `<YOUR_URL_ACCESS_POINT>`. See the [BackupStorageLocation definition][6] for details.
 
 
 
@@ -78,5 +78,5 @@ In the root of your Ark directory, run:
   [3]: https://console.bluemix.net/docs/services/cloud-object-storage/iam/service-credentials.html#service-credentials
   [4]: https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0/kc_welcome_containers.html
   [5]: https://console.bluemix.net/docs/containers/container_index.html#container_index
-  [6]: config-definition.md#aws
+  [6]: backupstoragelocation-definition.md#aws
   [14]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html
diff --git a/examples/aws/00-ark-config.yaml b/examples/aws/00-ark-config.yaml
@@ -20,16 +20,5 @@ metadata:
   name: default
 persistentVolumeProvider:
   name: aws
-  config:
-    region: <YOUR_REGION>
-backupStorageProvider:
-  name: aws
-  bucket: <YOUR_BUCKET>
-  # Uncomment the below line to enable restic integration.
-  # The format for resticLocation is <bucket>[/<prefix>],
-  # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
-  # This MUST be a different bucket than the main Ark bucket
-  # specified just above.
-  # resticLocation: <YOUR_RESTIC_LOCATION>
   config:
     region: <YOUR_REGION>
diff --git a/examples/ibm/00-ark-config.yaml → ...les/aws/05-ark-backupstoragelocation.yaml b/examples/ibm/00-ark-config.yaml → ...les/aws/05-ark-backupstoragelocation.yaml
@@ -14,21 +14,19 @@
 
 ---
 apiVersion: ark.heptio.com/v1
-kind: Config
+kind: BackupStorageLocation
 metadata:
-  namespace: heptio-ark
   name: default
-backupStorageProvider:
-  name: aws
-  bucket: <YOUR_BUCKET>
-  # Uncomment the below line to enable restic integration.
-  # The format for resticLocation is <bucket>[/<prefix>],
-  # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
-  # This MUST be a different bucket than the main Ark bucket
-  # specified just above.
-  # resticLocation: <YOUR_RESTIC_LOCATION>
-  config:
-    region: <YOUR_REGION>
-    s3ForcePathStyle: "true"
-    s3Url:  <YOUR_URL_ACCESS_POINT>
----
+  namespace: heptio-ark
+spec:
+  provider: aws
+  objectStorage:
+    bucket: <YOUR_BUCKET> 
+    config:
+      region: <YOUR_REGION>
+      # Uncomment the below line to enable restic integration.
+      # The format for resticLocation is <bucket>[/<prefix>],
+      # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
+      # This MUST be a different bucket than the main Ark bucket
+      # specified just above.
+      # restic-location: <YOUR_RESTIC_LOCATION>
diff --git a/examples/azure/05-ark-backupstoragelocation.yaml b/examples/azure/05-ark-backupstoragelocation.yaml
@@ -0,0 +1,30 @@
+# Copyright 2018 the Heptio Ark contributors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+---
+apiVersion: ark.heptio.com/v1
+kind: BackupStorageLocation
+metadata:
+  name: default
+  namespace: heptio-ark
+spec:
+  provider: azure
+  objectStorage:
+    bucket: <YOUR_BUCKET> 
+    # Uncomment the below line to enable restic integration.
+    # The format for resticLocation is <bucket>[/<prefix>],
+    # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
+    # This MUST be a different bucket than the main Ark bucket
+    # specified just above.
+    # restic-location: <YOUR_RESTIC_LOCATION>
diff --git a/examples/azure/10-ark-config.yaml b/examples/azure/10-ark-config.yaml
@@ -21,13 +21,4 @@ metadata:
 persistentVolumeProvider:
   name: azure
   config:
-    apiTimeout: <YOUR_TIMEOUT>
-backupStorageProvider:
-  name: azure
-  bucket: <YOUR_BUCKET>
-  # Uncomment the below line to enable restic integration.
-  # The format for resticLocation is <bucket>[/<prefix>],
-  # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
-  # This MUST be a different bucket than the main Ark bucket
-  # specified just above.
-  # resticLocation: <YOUR_RESTIC_LOCATION>
+    apiTimeout: <YOUR_TIMEOUT>
diff --git a/examples/gcp/00-ark-config.yaml b/examples/gcp/00-ark-config.yaml
@@ -19,14 +19,4 @@ metadata:
   namespace: heptio-ark
   name: default
 persistentVolumeProvider:
-  name: gcp
-backupStorageProvider:
-  name: gcp
-  bucket: <YOUR_BUCKET>
-  # Uncomment the below line to enable restic integration.
-  # The format for resticLocation is <bucket>[/<prefix>],
-  # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
-  # This MUST be a different bucket than the main Ark bucket
-  # specified just above.
-  # resticLocation: <YOUR_RESTIC_LOCATION>
-
+  name: gcp
diff --git a/examples/gcp/05-ark-backupstoragelocation.yaml b/examples/gcp/05-ark-backupstoragelocation.yaml
@@ -0,0 +1,30 @@
+# Copyright 2018 the Heptio Ark contributors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+---
+apiVersion: ark.heptio.com/v1
+kind: BackupStorageLocation
+metadata:
+  name: default
+  namespace: heptio-ark
+spec:
+  provider: gcp
+  objectStorage:
+    bucket: <YOUR_BUCKET> 
+    # Uncomment the below line to enable restic integration.
+    # The format for resticLocation is <bucket>[/<prefix>],
+    # e.g. "my-restic-bucket" or "my-restic-bucket/repos".
+    # This MUST be a different bucket than the main Ark bucket
+    # specified just above.
+    # restic-location: <YOUR_RESTIC_LOCATION>