-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SVCSE-1595 Setup import of tables from staging FxA databases #4578
Merged
+1,909
−0
Merged
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0eddeaa
SVCSE-1595 Setup import of tables from staging FxA databases
akkomar 956fbff
Update dataset metadata
akkomar fce026b
Add note on excluded confidential fields to table metadata
akkomar c69df57
Cast hasScopedKeys to BOOL
akkomar fd10e95
More BOOL casting fixes
akkomar 8f48ffa
Merge branch 'main' into fxa_db_sync_all
akkomar b881035
Rename dataset_metadata.yaml
akkomar 4ce742b
Merge branch 'main' into fxa_db_sync_all
akkomar a1f3d21
Merge branch 'main' into fxa_db_sync_all
akkomar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
18 changes: 18 additions & 0 deletions
18
...z-fx-data-shared-prod/accounts_db_nonprod_external/fxa_account_customers_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: accountCustomers table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `accountCustomers` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
17 changes: 17 additions & 0 deletions
17
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_account_customers_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| SELECT | ||
| TO_HEX(uid) AS uid, | ||
| stripeCustomerId, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(createdAt AS INT)) AS createdAt, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(updatedAt AS INT)) AS updatedAt, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| uid, | ||
| stripeCustomerId, | ||
| createdAt, | ||
| updatedAt | ||
| FROM | ||
| fxa.accountCustomers | ||
| """ | ||
| ) |
14 changes: 14 additions & 0 deletions
14
...moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_account_customers_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| fields: | ||
| - name: uid | ||
| type: STRING | ||
| mode: NULLABLE | ||
| description: Account ID in hexadecimal format. | ||
| - name: stripeCustomerId | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: createdAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: updatedAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
18 changes: 18 additions & 0 deletions
18
...x-data-shared-prod/accounts_db_nonprod_external/fxa_account_reset_tokens_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: accountResetTokens table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `accountResetTokens` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| excluding columns containing confidential data, updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
13 changes: 13 additions & 0 deletions
13
...oz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_account_reset_tokens_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| SELECT | ||
| TO_HEX(uid) AS uid, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(createdAt AS INT)) AS createdAt, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| uid, | ||
| createdAt | ||
| FROM | ||
| fxa.accountResetTokens | ||
akkomar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| """ | ||
| ) | ||
8 changes: 8 additions & 0 deletions
8
...-fx-data-shared-prod/accounts_db_nonprod_external/fxa_account_reset_tokens_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| fields: | ||
| - name: uid | ||
| type: STRING | ||
| mode: NULLABLE | ||
| description: Account ID in hexadecimal format. | ||
| - name: createdAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE |
18 changes: 18 additions & 0 deletions
18
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_accounts_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: accounts table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `accounts` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| excluding columns containing confidential data, updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
37 changes: 37 additions & 0 deletions
37
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_accounts_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| SELECT | ||
| TO_HEX(uid) AS uid, | ||
| normalizedEmail, | ||
| email, | ||
| SAFE_CAST(emailVerified AS BOOL) AS emailVerified, | ||
| verifierVersion, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(verifierSetAt AS INT)) AS verifierSetAt, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(createdAt AS INT)) AS createdAt, | ||
| locale, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(lockedAt AS INT)) AS lockedAt, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(profileChangedAt AS INT)) AS profileChangedAt, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(keysChangedAt AS INT)) AS keysChangedAt, | ||
| ecosystemAnonId, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(disabledAt AS INT)) AS disabledAt, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(metricsOptOutAt AS INT)) AS metricsOptOutAt, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| uid, | ||
| normalizedEmail, | ||
| email, | ||
| emailVerified, | ||
| verifierVersion, | ||
| verifierSetAt, | ||
| createdAt, | ||
| locale, | ||
| lockedAt, | ||
| profileChangedAt, | ||
| keysChangedAt, | ||
| ecosystemAnonId, | ||
| disabledAt, | ||
| metricsOptOutAt | ||
| FROM | ||
| fxa.accounts | ||
| """ | ||
| ) |
44 changes: 44 additions & 0 deletions
44
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_accounts_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| fields: | ||
| - name: uid | ||
| type: STRING | ||
| mode: NULLABLE | ||
| description: Account ID in hexadecimal format. | ||
| - name: normalizedEmail | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: email | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: emailVerified | ||
| type: BOOLEAN | ||
| mode: NULLABLE | ||
| - name: verifierVersion | ||
| type: INTEGER | ||
| mode: NULLABLE | ||
| - name: verifierSetAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: createdAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: locale | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: lockedAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: profileChangedAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: keysChangedAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: ecosystemAnonId | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: disabledAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE | ||
| - name: metricsOptOutAt | ||
| type: TIMESTAMP | ||
| mode: NULLABLE |
18 changes: 18 additions & 0 deletions
18
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_db_metadata_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: dbMetadata table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `dbMetadata` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
13 changes: 13 additions & 0 deletions
13
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_db_metadata_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| SELECT | ||
| name, | ||
| value, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| name, | ||
| value | ||
| FROM | ||
| fxa.dbMetadata | ||
| """ | ||
| ) |
7 changes: 7 additions & 0 deletions
7
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_db_metadata_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| fields: | ||
| - name: name | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: value | ||
| type: STRING | ||
| mode: NULLABLE |
18 changes: 18 additions & 0 deletions
18
...-shared-prod/accounts_db_nonprod_external/fxa_device_command_identifiers_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: deviceCommandIdentifiers table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `deviceCommandIdentifiers` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
13 changes: 13 additions & 0 deletions
13
...data-shared-prod/accounts_db_nonprod_external/fxa_device_command_identifiers_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| SELECT | ||
| commandId, | ||
| commandName, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| commandId, | ||
| commandName | ||
| FROM | ||
| fxa.deviceCommandIdentifiers | ||
| """ | ||
| ) |
7 changes: 7 additions & 0 deletions
7
...ta-shared-prod/accounts_db_nonprod_external/fxa_device_command_identifiers_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| fields: | ||
| - name: commandId | ||
| type: INTEGER | ||
| mode: NULLABLE | ||
| - name: commandName | ||
| type: STRING | ||
| mode: NULLABLE |
18 changes: 18 additions & 0 deletions
18
...moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_device_commands_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: deviceCommands table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `deviceCommands` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| excluding columns containing confidential data, updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
15 changes: 15 additions & 0 deletions
15
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_device_commands_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| SELECT | ||
| TO_HEX(uid) AS uid, | ||
| deviceId, | ||
| commandId, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| uid, | ||
| deviceId, | ||
| commandId | ||
| FROM | ||
| fxa.deviceCommands | ||
| """ | ||
| ) |
11 changes: 11 additions & 0 deletions
11
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_device_commands_v1/schema.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| fields: | ||
| - name: uid | ||
| type: STRING | ||
| mode: NULLABLE | ||
| description: Account ID in hexadecimal format. | ||
| - name: deviceId | ||
| type: STRING | ||
| mode: NULLABLE | ||
| - name: commandId | ||
| type: INTEGER | ||
| mode: NULLABLE |
18 changes: 18 additions & 0 deletions
18
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_devices_v1/metadata.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| friendly_name: devices table from nonprod (stage) fxa database | ||
| description: > | ||
| A mirror of the `devices` table from the nonprod (stage) `fxa` CloudSQL database, | ||
| excluding columns containing confidential data, updated daily to match the current state of the table. | ||
|
|
||
| See https://mozilla.github.io/ecosystem-platform/reference/database-structure#database-fxa | ||
| owners: | ||
| - akomar@mozilla.com | ||
| labels: | ||
| application: accounts_backend | ||
| schedule: daily | ||
| scheduling: | ||
| dag_name: bqetl_accounts_backend_external | ||
| # destination is the whole table, not a single partition, | ||
| # so don't use date_partition_parameter | ||
| date_partition_parameter: null | ||
| referenced_tables: [] |
25 changes: 25 additions & 0 deletions
25
sql/moz-fx-data-shared-prod/accounts_db_nonprod_external/fxa_devices_v1/query.sql
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| SELECT | ||
| TO_HEX(uid) AS uid, | ||
| id, | ||
| name, | ||
| nameUtf8, | ||
| type, | ||
| SAFE.TIMESTAMP_MILLIS(SAFE_CAST(createdAt AS INT)) AS createdAt, | ||
| callbackPublicKey, | ||
| SAFE_CAST(callbackIsExpired AS BOOL) AS callbackIsExpired, | ||
| FROM | ||
| EXTERNAL_QUERY( | ||
| "moz-fx-fxa-nonprod.us.fxa-rds-nonprod-stage-fxa", | ||
| """SELECT | ||
| uid, | ||
| id, | ||
| name, | ||
| nameUtf8, | ||
| type, | ||
| createdAt, | ||
| callbackPublicKey, | ||
| callbackIsExpired | ||
| FROM | ||
| fxa.devices | ||
| """ | ||
| ) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we should go to the hassle of making the BigQuery columns snake_case. We're essentially already doing so for the table names, and if any of these tables will be made available in Looker that would impact how the column names would be displayed there (though I suppose we could also implement LookML generator logic to create snake_case dimensions for camelCase columns).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was expecting this data would be exposed in Looker via aggregates and raw tables would rather be used for exploratory analyses. One argument for keeping columns unchanged I can think of is that it makes it a bit easier to find them in FxA codebase. I'm open to discussion though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine punting on this for now. No need to go to all that extra work if there isn't a concrete use case/benefit.