Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Column Name Lengths - Column Aliasing With Secondary Calcs Could Break Limit #41

Closed
callum-mcdata opened this issue Jul 11, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@callum-mcdata
Copy link
Contributor

callum-mcdata commented Jul 11, 2022

@joellabes had a great observation during our call today that the additional functionality of prepending metric names onto secondary calculation aliases could cause the generated dataset to hit the character limits for different data warehouse.

The least permissive of the Big 4 connectors is Postgres with its 59 character limit. The second of these is Redshift with 127 bytes (which I'm assuming is synonymous with characters).

We should add additional testing to confirm whether this fails or not. If it does, we should potentially trim the metric name to the maximum number of characters that can be combined with the secondary calculation alias name.

@callum-mcdata callum-mcdata changed the title Column Name Lengths - Column Aliasing With Secondary Calcs could break Limit Column Name Lengths - Column Aliasing With Secondary Calcs Could Break Limit Jul 11, 2022
@callum-mcdata callum-mcdata added the enhancement New feature or request label Jul 11, 2022
@callum-mcdata
Copy link
Contributor Author

So. I've done some testing. Interestingly:

  • Postgres and Redshift already do column name truncation 🤷 . No idea how because that logic isn't encoded into the package so it must be happening in the adapter.
  • BigQuery and Snowflake are NOT doing this. However their limits are 300 and 250 respectively, so this is less a concern then I had previously thought.

@callum-mcdata
Copy link
Contributor Author

I'm going to close out this an issue because its not much of a concern due to the 300/250 character limitation of BQ and Snowflake

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant