Destination Redshift: failed normalization with field larger than 65k

[mjkanji]

Environment

• Airbyte version: 0.39.17-alpha
• OS Version / Instance: AWS EC2 Linux
• Deployment: Docker
• Source Connector and version: Zendesk Support 0.2.9
• Destination Connector and version: Redshift 0.3.39
• Step where error happened: Normalization

Current Behavior

The job fails at the normalization stage with the following error:

[error] [MainThread]: Database Error in model tickets_scd (models/generated/airbyte_incremental/scd/zendesk/tickets_scd.sql)
[error] [MainThread]:   Invalid input
[error] [MainThread]:   DETAIL:  
[error] [MainThread]:     -----------------------------------------------
[error] [MainThread]:     error:  Invalid input
[error] [MainThread]:     code:      8001
[error] [MainThread]:     context:   CONCAT() result too long for type varchar(65535)
[error] [MainThread]:     query:     4105572
[error] [MainThread]:     location:  string_ops.cpp:110
[error] [MainThread]:     process:   query0_119_4105572 [pid=17694]
[error] [MainThread]:     -----------------------------------------------
[error] [MainThread]:   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_incremental/scd/zendesk/tickets_scd.sql

Reverting back to destination Redshift version 0.3.28 solves this problem, so I'm guessing it has something to do with the recent changes relating to the SUPER data type (#12064) and the maximum record size (#12940).

Expected Behavior

Normalization should happen successfully.

Logs

error_log.txt

Steps to Reproduce

1. Create a connection between Stripe and Redshift (with the COPY strategy)
2. Sync the tickets stream with sync mode Incremental | Deduped + History
3. Reset and run a sync job.

Are you willing to submit a PR?

No.

[marcosmarxm]

Workaround rollback Redshift to version 0.2.28

[alafanechere]

Duplicate of #14441