🐛 Source mixpanel: basic normalization fails with duplicate field names

[caiopedroso]

Enviroment

• Airbyte version: 0.29.4-alpha
• OS Version / Instance: macOS
• Deployment: Docker
• Source Connector and version: Mixpanel (0.1.0)
• Destination Connector and version: BigQuery (0.3.12)
• Severity: High
• Step where error happened: Sync job

Current Behavior

I’m trying to sync for the first time and the process fails in the normalization step, apparently, something broke the dbt process, the log is attached.
If I try without normalization (raw data), it runs successfully

Expected Behavior

The normalized data appears at BigQuery without any error

Logs

logs-10-0.txt

**UPDATE - Apparently I have a field called username and another one userName, when I check the destination_catalog.json, I can see both fields, maybe this is causing the ambiguity that BigQuery is not able to resolve?

Steps to Reproduce

1. Create a Mixpanel Source Connection and a BigQuery destination
2. Try to sync with the Basic Normalization selected

[sherifnada]

**UPDATE - Apparently I have a field called username and another one userName, when I check the destination_catalog.json, I can see both fields, maybe this is causing the ambiguity that BigQuery is not able to resolve?

@caiopedroso yup that would do it 😅 is renaming these an option at all?

[caiopedroso]

Sadly Mixpanel don't handle renaming properties/events, they only a have a feature called merge, but I'm not sure that it will hide one of the properties, I can try that.. keep u posted.

[caiopedroso]

No luck doing that, tried to merge and hide the value, but at the export, the userN[n]ame keep coming as different fields..

[marcosmarxm]

@caiopedroso normalization module applies some standard conventions for destination, like lowercase column names and other functions. Maybe you can export the normalization generated by Airbyte and run as a custom operator. I that case you can remove both username from final table or (I don't have full Bigquery knowledge here) use a function to select the column, maybe using quotes.

[caiopedroso]

@marcosmarxm , Got you, I will try that, just waiting for the release of a fix for the M1 version and update here if I come up with a solution.

[misteryeo]

@caiopedroso is this still an issue?

[caiopedroso]

Yes @misteryeo , I was not able to work myself in a custom dbt model to try to solve that, if it's something that can be "out of the box" on airbyte, would appreciate it.

[roman-romanov-o]

@caiopedroso I've added handling of such case into Export stream in connector

Try to update to latest version of connector, that will fix your problem