Skip to content

Commit

Permalink
Merge pull request #7839 from braze-inc/BD-3144
Browse files Browse the repository at this point in the history
BD-3144: Clarify SQL example and minor edits
  • Loading branch information
rachel-feinberg authored Aug 7, 2024
2 parents 697184a + da983fe commit 8bed365
Showing 1 changed file with 8 additions and 12 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Before creating a new Cloud Data Ingestion (CDI) integration for [catalogs]({{si
- Create a catalog in the [Braze dashboard]({{site.baseurl}}/user_guide/personalization_and_dynamic_content/catalogs/catalog/#method-2-create-in-browser)
- Create a catalog using the [Create catalog endpoint]({{site.baseurl}}/api/endpoints/catalogs/catalog_management/synchronous/post_create_catalog/)

Any changes to the catalog schema (for example, adding new fields, changing field type) must be made through the catalog dashboard before updated data is synced through CDI. We recommend making these updates when the sync is paused or not scheduled to run to avoid conflicts between your data warehouse data and the schema in Braze.
Any changes to the catalog schema (for example, adding new fields or changing field type) must be made through the catalog dashboard before updated data is synced through CDI. We recommend making these updates when the sync is paused or not scheduled to run to avoid conflicts between your data warehouse data and the schema in Braze.

## Step 2: Integrate Cloud Data Ingestion with catalog data
The setup for a catalog sync closely follows the process for [user-data CDI integrations]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations#product-setup).
Expand Down Expand Up @@ -120,18 +120,18 @@ CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CATALOGS_SYNC`

2. Set up a user and grant proper permissions. If you already have credentials from an existing sync, you can reuse those—just make sure to extend access to the catalog source table.
The service account should have the below permissions:
- BigQuery Connection User: This will allow Braze to make connections
- BigQuery Connection User: This will allow Braze to make connections.
- BigQuery User: This will provide Braze access to run queries, read dataset metadata, and list tables.
- BigQuery Data Viewer: This will provide Braze access to view datasets and their contents.
- BigQuery Job User: This will provide Braze access to run jobs<br><br>After creating the service account and granting permissions, generate a JSON key. Refer to [Keys create and delete](https://cloud.google.com/iam/docs/keys-create-delete) for more information. You will update this to the Braze dashboard later.
- BigQuery Job User: This will provide Braze access to run jobs<br><br>After creating the service account and granting permissions, generate a JSON key. Refer to [Keys create and delete](https://cloud.google.com/iam/docs/keys-create-delete) for more information. You'll update this to the Braze dashboard later.

{:start="3"}
3. If you have network policies in place, you must give Braze network access to your BigQuery instance. For a list of IPs, see the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views).

{% endtab %}
{% tab Databricks %}

1. Set up a source table in Databricks. You can use the names in the following example or choose your own catalog, schema, and table names. You may also use a view or a materialized view instead of a table.
1. Set up a source table in Databricks. You can use the names in the following example or choose your catalog, schema, and table names. You can also use a view or a materialized view instead of a table.

```json
CREATE SCHEMA BRAZE-CLOUD-PRODUCTION.INGESTION;
Expand Down Expand Up @@ -159,16 +159,11 @@ CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CATALOGS_SYNC`
2. Create a personal access token in your Databricks workspace.

- a. Select your Databricks username, then select **User Settings** from the dropdown menu.

- b. On the **Access tokens** tab, select **Generate new token**.

- c. Enter a comment that helps you to identify this token, such as "Braze CDI".

- d. Change the token’s lifetime to no lifetime by leaving the **Lifetime (days)** box blank. Select **Generate**.

- e. Copy the displayed token, and then select **Done**.

- f. Keep the token in a safe place until you need to enter it on the Braze dashboard during the credential creation step.
- f. Keep the token in a safe place until you need to enter it during the credential creation step in the Braze dashboard.

{:start="3"}
3. If you have network policies in place, you must give Braze network access to your Databricks instance. For a list of IPs, see the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views) page.
Expand All @@ -177,7 +172,8 @@ CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CATALOGS_SYNC`
{% endtabs %}

## How the integration works
Each time the sync runs, Braze will pull in all rows where `UPDATED_AT` is after the last timestamp synced. To set up a source table that will fully refresh each time a sync runs, we recommend creating a view from your catalog data. For example, if you have a table of product data (`product_catalog_1`) with `product_id`, `price`, and three additional attributes, you could sync the below view:

Each time the sync runs, Braze will pull in all rows where `UPDATED_AT` is after the last timestamp synced. We recommend creating a view from your catalog data to set up a source table that will fully refresh each time a sync runs. For example, if you have a table of product data (`product_catalog_1`) with `product_id` and three additional attributes, you could sync the below view:

{% tabs %}
{% tab Snowflake %}
Expand Down Expand Up @@ -248,4 +244,4 @@ CREATE view IF NOT EXISTS BRAZE_CLOUD_PRODUCTION.INGESTION.CATALOGS_SYNC AS (SEL

- The data fetched from the integration will be used to create or update items in the target catalog based on the `id` provided.
- If DELETED is set to `true`, the corresponding catalog item will be deleted.
- The sync will not consume data points, but all data synced will count toward your total catalog usage; this usage is measured based on total data stored, so you don’t need to worry about only syncing changed data.
- The sync won't consume data points, but all data synced will count toward your total catalog usage; this usage is measured based on the total data stored, so you don’t need to worry about only syncing changed data.

0 comments on commit 8bed365

Please sign in to comment.