Skip to content

Commit

Permalink
Postgres: Prevent temp relation identifiers from being too long
Browse files Browse the repository at this point in the history
Related: dbt-labs#2197 

The currently postgres `make_temp_relation` adds a 29 character suffix to the end of the temp relation identifier (9 from default suffix and 20 from timestamp).  This is a problem now that relations with more than 63 characters raise exceptions. 
The fix is to shorten the suffix and also trim the base_relation identifier so that the total length is always less than 63 characters.

An exception can also be raised if the default suffix is overridden with a value that is too long.
  • Loading branch information
elexisvenator committed Nov 3, 2020
1 parent 4e19e87 commit c3b5b88
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion plugins/postgres/dbt/include/postgres/macros/adapters.sql
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,21 @@
{{ current_timestamp() }}::timestamp without time zone
{%- endmacro %}

{#
Postgres tables have a maximum length off 63 characters, anything longer is silently truncated.
Temp relations add a lot of extra characters to the end of table namers to ensure uniqueness.
To prevent this going over the character limit, the base_relation name is truncated to ensure
that name + suffix + uniquestring is < 63 characters.
#}
{% macro postgres__make_temp_relation(base_relation, suffix) %}
{% set tmp_identifier = base_relation.identifier ~ suffix ~ py_current_timestring() %}
{% set dt = modules.datetime.datetime.now() %}
{% set dtstring = dt.strftime("%H%M%S%f") %}
{% set suffix_length = suffix|length + dtstring|length %}
{% set relation_max_name_length = 63 %}
{% if suffix_length > relation_max_name_length %}
{% do exceptions.raise_compiler_error('Temp relation suffix is too long (' ~ suffix|length ~ ' characters). Maximum length is ' ~ (relation_max_name_length - dtstring|length) ~ ' characters.') %}
{% endif %}
{% set tmp_identifier = base_relation.identifier[:relation_max_name_length - suffix_length] ~ suffix ~ dtstring %}
{% do return(base_relation.incorporate(
path={
"identifier": tmp_identifier,
Expand Down

0 comments on commit c3b5b88

Please sign in to comment.