Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(braze): switch to dual region buckets to avoid egress fees #6902

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore(braze): switch to dual region buckets to avoid egress fees #6902

wants to merge 1 commit into from

Conversation

whd
Copy link
Member

@whd whd commented Jan 29, 2025 •
edited by data-sync-user
Loading

Description

Requires https://github.com/mozilla-services/cloudops-infra/pull/6212 and possibly some config changes outside of bqetl

Related Tickets & Documents

Reviewer, please follow this checklist

┆Issue is synchronized with this Jira Task

@whd whd requested a review from chelseybeck January 29, 2025 00:38
@dataops-ci-bot
Copy link

Integration report for "chore(braze): switch to dual region buckets to avoid egress fees"

sql.diff

Click to expand! 
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_braze_currents.py /tmp/workspace/generated-sql/dags/bqetl_braze_currents.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_braze_currents.py	2025-01-29 00:38:17.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_braze_currents.py	2025-01-29 00:40:04.000000000 +0000
@@ -61,7 +61,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_click_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Click",
         ],
@@ -80,7 +80,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_conversion_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.campaigns.Conversion",
         ],
@@ -99,7 +99,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_delivery_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Delivery",
         ],
@@ -118,7 +118,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_global_state_changes_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.behaviors.subscription.GlobalStateChange",
         ],
@@ -137,7 +137,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_hard_bounces_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Bounce",
         ],
@@ -170,7 +170,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_mark_as_spam_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.MarkAsSpam",
         ],
@@ -189,7 +189,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_open_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Open",
         ],
@@ -208,7 +208,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_send_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Send",
         ],
@@ -227,7 +227,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_soft_bounce_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.SoftBounce",
         ],
@@ -246,7 +246,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_state_changes_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.behaviors.subscriptiongroup.StateChange",
         ],
@@ -265,7 +265,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_firefox_unsubscribe_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071",
             "--event-type=users.messages.email.Unsubscribe",
         ],
@@ -298,7 +298,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_click_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.Click",
         ],
@@ -317,7 +317,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_conversion_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.campaigns.Conversion",
         ],
@@ -336,7 +336,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_delivery_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.Delivery",
         ],
@@ -355,7 +355,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_global_state_changes_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.behaviors.subscription.GlobalStateChange",
         ],
@@ -374,7 +374,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_hard_bounces_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.Bounce",
         ],
@@ -407,7 +407,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_mark_as_spam_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.MarkAsSpam",
         ],
@@ -426,7 +426,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_open_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.Open",
         ],
@@ -445,7 +445,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_send_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.Send",
         ],
@@ -464,7 +464,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_soft_bounce_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.messages.email.SoftBounce",
         ],
@@ -483,7 +483,7 @@
             "--destination-project=moz-fx-data-shared-prod",
             "--destination-dataset=braze_external",
             "--destination-table=braze_currents_mozilla_state_changes_v1",
-            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla",
+            "--source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region",
             "--source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366",
             "--event-type=users.behaviors.subscriptiongroup.StateChange",
         ],
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_click_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_click_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_click_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_click_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_click_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Click
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_conversion_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_conversion_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_conversion_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_conversion_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_conversion_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.campaigns.Conversion
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_delivery_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_delivery_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_delivery_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_delivery_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_delivery_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Delivery
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_global_state_changes_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_global_state_changes_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_global_state_changes_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_global_state_changes_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_global_state_changes_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.behaviors.subscription.GlobalStateChange
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_hard_bounces_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_hard_bounces_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_hard_bounces_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_hard_bounces_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_hard_bounces_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Bounce
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_mark_as_spam_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_mark_as_spam_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_mark_as_spam_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_mark_as_spam_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_mark_as_spam_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.MarkAsSpam
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_open_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_open_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_open_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_open_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_open_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Open
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_send_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_send_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_send_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_send_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_send_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Send
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_soft_bounce_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_soft_bounce_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_soft_bounce_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_soft_bounce_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_soft_bounce_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.SoftBounce
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_state_changes_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_state_changes_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_state_changes_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_state_changes_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_state_changes_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.behaviors.subscriptiongroup.StateChange
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_unsubscribe_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_unsubscribe_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_unsubscribe_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_firefox_unsubscribe_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_firefox_unsubscribe_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox
+  - --source-bucket=moz-fx-data-marketing-prod-braze-firefox-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf55eea9932004d7fb071
   - --event-type=users.messages.email.Unsubscribe
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_click_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_click_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_click_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_click_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_click_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.Click
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_conversion_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_conversion_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_conversion_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_conversion_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_conversion_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.campaigns.Conversion
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_delivery_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_delivery_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_delivery_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_delivery_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_delivery_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.Delivery
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_global_state_changes_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_global_state_changes_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_global_state_changes_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_global_state_changes_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_global_state_changes_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.behaviors.subscription.GlobalStateChange
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_hard_bounces_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_hard_bounces_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_hard_bounces_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_hard_bounces_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_hard_bounces_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.Bounce
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_mark_as_spam_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_mark_as_spam_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_mark_as_spam_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_mark_as_spam_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_mark_as_spam_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.MarkAsSpam
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_open_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_open_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_open_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_open_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_open_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.Open
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_send_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_send_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_send_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_send_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_send_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.Send
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_soft_bounce_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_soft_bounce_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_soft_bounce_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_soft_bounce_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_soft_bounce_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.messages.email.SoftBounce
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_state_changes_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_state_changes_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_state_changes_v1/metadata.yaml	2025-01-29 00:34:53.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/braze_external/braze_currents_mozilla_state_changes_v1/metadata.yaml	2025-01-29 00:34:58.000000000 +0000
@@ -14,7 +14,7 @@
   - --destination-project=moz-fx-data-shared-prod
   - --destination-dataset=braze_external
   - --destination-table=braze_currents_mozilla_state_changes_v1
-  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla
+  - --source-bucket=moz-fx-data-marketing-prod-braze-mozilla-dual-region
   - --source-prefix=currents/dataexport.prod-05.GCS.integration.65fdf62e352dc7004cebd366
   - --event-type=users.behaviors.subscriptiongroup.StateChange
 bigquery: null

Link to full diff

@whd whd changed the title [do not merge] chore(braze): switch to dual region buckets to avoid egress fees chore(braze): switch to dual region buckets to avoid egress fees Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chelseybeck chelseybeck Awaiting requested review from chelseybeck

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@whd @dataops-ci-bot