concat macro can use alternative_concat on Postgres too #296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
clrcrl
reviewed
Dec 23, 2020
There was a problem hiding this comment.
So it turns out that Postgres has supported this for a long time, and BigQuery has been supporting it since March
I think we should just rip out the "alternative" syntax and make || the default. Though, @jtcohen6, I'm curious whether you think that might break any other non-core adapters (e.g. does Spark support ||?)
clrcrl
force-pushed
the
dev/0.7.0
branch
5 times, most recently
from
bbba960
to
60a3b3c
Compare
clrcrl
pushed a commit
that referenced
this pull request
May 19, 2021
jtcohen6
pushed a commit
that referenced
this pull request
Jun 6, 2021
* Postgres also have an alternative concat binary operation (#296) * Update default implementation of concat macro Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
jtcohen6
added a commit
that referenced
this pull request
Jun 6, 2021
* Tidy up changelog * Add 0.7.0 entry to changelog * Add order_by argument to get_column_values (#349) * Add slugify macro to utils, use in pivot macro (#314) * 0.20.0 compatibility (#371) * Explicitly redefine Redshift -> default * Upgrade generic tests * Rm namespaces macro. New dispatch syntax * Run tests with 0.20.0rc1 * Update changelog, readme Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com> * Simplify concat (#373) * Postgres also have an alternative concat binary operation (#296) * Update default implementation of concat macro Co-authored-by: Christophe Duong <christophe.duong@gmail.com> Co-authored-by: Jeremy Cohen <jeremy@fishtownanalytics.com> Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a:
main)Description & motivation
Using Postgres database, I have a model that needs to produce a surrogate key or hash on a large number of columns.
Unfortunately it breaks because, on postgres, we cannot pass more than 100 arguments to a function...
The current
dbt_utils.surrogate_keyis indeed calling thedbt_utils.concatwhich is using theconcat()function that accepts a list of arguments.However postgres has another alternative to concat string using the binary operator
||instead as we can do on Redshift and on Snowflake and adding this adapter version to the dispatched function for thedbt_utils.concatwould avoir crashes when passing more than 100 strings to concat together.Checklist