Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix expect_column_values_to_be_in_type_list for Redshift in 0.5.0 #131

Merged
merged 1 commit into from
Dec 8, 2021
Merged

Fix expect_column_values_to_be_in_type_list for Redshift in 0.5.0 #131

merged 1 commit into from
Dec 8, 2021

Conversation

nicholasyager
Copy link
Contributor

@nicholasyager nicholasyager commented Dec 6, 2021

This PR fixes #130.

Context
In some circumstances, Redshift will fail to execute queries generated by expect_column_values_to_be_in_type_list if it is unable infer the type of the relation_column or relation_column_type fields. This commit enforces a casting of those values to a string type to prevent ambiguity.

Integration Tests (Redshift)

Test Results
$ ./test.sh redshift
18:00:03  Running with dbt=1.0.0
18:00:04  Unable to do partial parsing because config vars, config profile, or config target have changed
18:00:04  Unable to do partial parsing because a project config has changed
18:00:06  Found 11 models, 93 tests, 0 snapshots, 0 analyses, 580 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
18:00:06
18:00:07  Concurrency: 8 threads (target='integration_tests')
18:00:07
18:00:07  1 of 104 START table model dbt_expectations_integration_tests.data_test......... [RUN]
18:00:07  2 of 104 START table model dbt_expectations_integration_tests.data_test_factored [RUN]
18:00:07  3 of 104 START table model dbt_expectations_integration_tests.emails............ [RUN]
18:00:07  4 of 104 START table model dbt_expectations_integration_tests.series_10......... [RUN]
18:00:07  5 of 104 START table model dbt_expectations_integration_tests.series_4.......... [RUN]
18:00:07  6 of 104 START table model dbt_expectations_integration_tests.timeseries_base... [RUN]
18:00:07  7 of 104 START table model dbt_expectations_integration_tests.timeseries_hourly. [RUN]
18:00:10  5 of 104 OK created table model dbt_expectations_integration_tests.series_4..... [SELECT in 2.51s]
18:00:10  6 of 104 OK created table model dbt_expectations_integration_tests.timeseries_base [SELECT in 2.97s]
18:00:10  8 of 104 START table model dbt_expectations_integration_tests.timeseries_data... [RUN]
18:00:14  2 of 104 OK created table model dbt_expectations_integration_tests.data_test_factored [SELECT in 6.39s]
18:00:15  4 of 104 OK created table model dbt_expectations_integration_tests.series_10.... [SELECT in 7.48s]
18:00:15  9 of 104 START table model dbt_expectations_integration_tests.timeseries_data_extended [RUN]
18:00:15  10 of 104 START table model dbt_expectations_integration_tests.timeseries_data_grouped [RUN]
18:00:16  7 of 104 OK created table model dbt_expectations_integration_tests.timeseries_hourly [SELECT in 8.48s]
18:00:16  11 of 104 START table model dbt_expectations_integration_tests.timeseries_hourly_data_extended [RUN]
18:00:16  3 of 104 OK created table model dbt_expectations_integration_tests.emails....... [SELECT in 8.57s]
18:00:16  12 of 104 START test dbt_expectations_expect_column_values_to_match_like_pattern_emails_email_address___ [RUN]
18:00:16  13 of 104 START test dbt_expectations_expect_column_values_to_match_like_pattern_list_emails_email_address______ [RUN]
18:00:16  14 of 104 START test dbt_expectations_expect_column_values_to_match_regex_emails_email_address___ [RUN]
18:00:16  8 of 104 OK created table model dbt_expectations_integration_tests.timeseries_data [SELECT in 5.72s]
18:00:16  15 of 104 START test dbt_expectations_expect_column_values_to_match_regex_list_emails_email_address______ [RUN]
18:00:22  13 of 104 PASS dbt_expectations_expect_column_values_to_match_like_pattern_list_emails_email_address______ [PASS in 6.27s]
18:00:22  16 of 104 START test dbt_expectations_expect_column_values_to_not_match_like_pattern_emails_email_address___ [RUN]
18:00:22  14 of 104 PASS dbt_expectations_expect_column_values_to_match_regex_emails_email_address___ [PASS in 6.46s]
18:00:22  17 of 104 START test dbt_expectations_expect_column_values_to_not_match_like_pattern_list_emails_email_address______ [RUN]
18:00:23  12 of 104 PASS dbt_expectations_expect_column_values_to_match_like_pattern_emails_email_address___ [PASS in 6.77s]
18:00:23  18 of 104 START test dbt_expectations_expect_column_values_to_not_match_regex_emails_email_address___ [RUN]
18:00:23  9 of 104 OK created table model dbt_expectations_integration_tests.timeseries_data_extended [SELECT in 8.04s]
18:00:23  19 of 104 START test dbt_expectations_expect_column_values_to_not_match_regex_list_emails_email_address______ [RUN]
18:00:23  11 of 104 OK created table model dbt_expectations_integration_tests.timeseries_hourly_data_extended [SELECT in 7.39s]
18:00:23  20 of 104 START test dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day [RUN]
18:00:26  15 of 104 PASS dbt_expectations_expect_column_values_to_match_regex_list_emails_email_address______ [PASS in 10.45s]
18:00:26  21 of 104 START test dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_data_date_day__date___dbt_expectations_type_datetime_ [RUN]
18:00:27  21 of 104 PASS dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_data_date_day__date___dbt_expectations_type_datetime_ [PASS in 0.45s]
18:00:27  22 of 104 START test dbt_expectations_expect_column_values_to_be_increasing_timeseries_data_date_day__date_day [RUN]
18:00:28  10 of 104 OK created table model dbt_expectations_integration_tests.timeseries_data_grouped [SELECT in 12.95s]
18:00:28  23 of 104 START test dbt_expectations_expect_column_values_to_be_of_type_timeseries_data_date_day___dbt_expectations_type_datetime_ [RUN]
18:00:28  23 of 104 PASS dbt_expectations_expect_column_values_to_be_of_type_timeseries_data_date_day___dbt_expectations_type_datetime_ [PASS in 0.43s]
18:00:28  24 of 104 START test dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_data_row_value__date_day__6__True [RUN]
18:00:28  16 of 104 PASS dbt_expectations_expect_column_values_to_not_match_like_pattern_emails_email_address___ [PASS in 6.21s]
18:00:28  25 of 104 START test dbt_expectations_expect_column_values_to_be_within_n_stdevs_timeseries_data_row_value__6 [RUN]
18:00:29  17 of 104 PASS dbt_expectations_expect_column_values_to_not_match_like_pattern_list_emails_email_address______ [PASS in 6.41s]
18:00:29  26 of 104 START test dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_date_day__day [RUN]
18:00:30  18 of 104 PASS dbt_expectations_expect_column_values_to_not_match_regex_emails_email_address___ [PASS in 7.14s]
18:00:30  27 of 104 START test dbt_expectations_expect_row_values_to_have_recent_data_timeseries_data_date_day__day__1 [RUN]
18:00:30  19 of 104 PASS dbt_expectations_expect_column_values_to_not_match_regex_list_emails_email_address______ [PASS in 7.43s]
18:00:30  28 of 104 START test dbt_expectations_expect_table_columns_to_match_ordered_list_timeseries_data_date_day__row_value__row_value_log [RUN]
18:00:31  28 of 104 PASS dbt_expectations_expect_table_columns_to_match_ordered_list_timeseries_data_date_day__row_value__row_value_log [PASS in 0.61s]
18:00:31  29 of 104 START test dbt_expectations_expect_column_distinct_count_to_be_greater_than_timeseries_data_extended_row_value__date_day_dbt_date_yesterday___1 [RUN]
18:00:33  22 of 104 PASS dbt_expectations_expect_column_values_to_be_increasing_timeseries_data_date_day__date_day [PASS in 5.67s]
18:00:33  30 of 104 START test dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day__date_day__ref_timeseries_data_extended_ [RUN]
18:00:33  30 of 104 PASS dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day__date_day__ref_timeseries_data_extended_ [PASS in 0.61s]
18:00:33  31 of 104 START test dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day__ref_timeseries_data_extended_ [RUN]
18:00:33  31 of 104 PASS dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day__ref_timeseries_data_extended_ [PASS in 0.23s]
18:00:33  32 of 104 START test dbt_expectations_expect_column_distinct_count_to_equal_timeseries_data_extended_row_value__date_day_dbt_date_yesterday___10 [RUN]
18:00:34  25 of 104 PASS dbt_expectations_expect_column_values_to_be_within_n_stdevs_timeseries_data_row_value__6 [PASS in 5.31s]
18:00:34  33 of 104 START test dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_data_extended_date_day__date___dbt_expectations_type_datetime_ [RUN]
18:00:34  20 of 104 PASS dbt_expectations_expect_column_distinct_count_to_equal_other_table_timeseries_data_date_day [PASS in 10.73s]
18:00:34  34 of 104 START test dbt_expectations_expect_column_values_to_be_of_type_timeseries_data_extended_date_day___dbt_expectations_type_datetime_ [RUN]
18:00:34  33 of 104 PASS dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_data_extended_date_day__date___dbt_expectations_type_datetime_ [PASS in 0.65s]
18:00:34  35 of 104 START test dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_data_extended_row_value_log__cast_date_day_as_dbt_expectations_type_datetime___6__False [RUN]
18:00:34  34 of 104 PASS dbt_expectations_expect_column_values_to_be_of_type_timeseries_data_extended_date_day___dbt_expectations_type_datetime_ [PASS in 0.63s]
18:00:34  36 of 104 START test dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_extended_date_day__day [RUN]
18:00:35  24 of 104 PASS dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_data_row_value__date_day__6__True [PASS in 6.42s]
18:00:35  37 of 104 START test dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_extended_date_day__day__7 [RUN]
18:00:36  26 of 104 PASS dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_date_day__day [PASS in 7.30s]
18:00:36  38 of 104 START test dbt_expectations_expect_row_values_to_have_recent_data_timeseries_data_extended_date_day__day__1 [RUN]
18:00:36  27 of 104 PASS dbt_expectations_expect_row_values_to_have_recent_data_timeseries_data_date_day__day__1 [PASS in 6.54s]
18:00:36  39 of 104 START test dbt_expectations_expect_table_columns_to_match_ordered_list_timeseries_data_extended_date_day__row_value__row_value_log [RUN]
18:00:36  36 of 104 PASS dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_extended_date_day__day [PASS in 1.91s]
18:00:36  40 of 104 START test dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_hourly_data_extended_date_hour___dbt_expectations_type_datetime_ [RUN]
18:00:36  38 of 104 PASS dbt_expectations_expect_row_values_to_have_recent_data_timeseries_data_extended_date_day__day__1 [PASS in 0.43s]
18:00:36  41 of 104 START test dbt_expectations_expect_column_values_to_be_of_type_timeseries_hourly_data_extended_date_hour___dbt_expectations_type_datetime_ [RUN]
18:00:37  1 of 104 OK created table model dbt_expectations_integration_tests.data_test.... [SELECT in 29.46s]
18:00:37  42 of 104 START test dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_hourly_data_extended_row_value_log__cast_date_hour_as_dbt_expectations_type_datetime___hour__6__False__12__48 [RUN]
18:00:37  39 of 104 PASS dbt_expectations_expect_table_columns_to_match_ordered_list_timeseries_data_extended_date_day__row_value__row_value_log [PASS in 0.82s]
18:00:37  43 of 104 START test dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_hourly_data_extended_date_hour__hour [RUN]
18:00:37  41 of 104 PASS dbt_expectations_expect_column_values_to_be_of_type_timeseries_hourly_data_extended_date_hour___dbt_expectations_type_datetime_ [PASS in 0.81s]
18:00:37  44 of 104 START test dbt_expectations_expect_row_values_to_have_recent_data_timeseries_hourly_data_extended_date_hour__hour__24 [RUN]
18:00:37  40 of 104 PASS dbt_expectations_expect_column_values_to_be_in_type_list_timeseries_hourly_data_extended_date_hour___dbt_expectations_type_datetime_ [PASS in 1.00s]
18:00:37  45 of 104 START test dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__1__date_day [RUN]
18:00:38  44 of 104 PASS dbt_expectations_expect_row_values_to_have_recent_data_timeseries_hourly_data_extended_date_hour__hour__24 [PASS in 0.39s]
18:00:38  46 of 104 START test dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__1__group_id_4__date_day [RUN]
18:00:38  32 of 104 PASS dbt_expectations_expect_column_distinct_count_to_equal_timeseries_data_extended_row_value__date_day_dbt_date_yesterday___10 [PASS in 4.30s]
18:00:38  47 of 104 START test dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__row_value__1__date_day [RUN]
18:00:40  35 of 104 PASS dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_data_extended_row_value_log__cast_date_day_as_dbt_expectations_type_datetime___6__False [PASS in 5.48s]
18:00:40  48 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_10 [RUN]
18:00:42  29 of 104 PASS dbt_expectations_expect_column_distinct_count_to_be_greater_than_timeseries_data_extended_row_value__date_day_dbt_date_yesterday___1 [PASS in 11.02s]
18:00:42  49 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_date_day__10 [RUN]
18:00:44  45 of 104 PASS dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__1__date_day [PASS in 6.58s]
18:00:44  47 of 104 PASS dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__row_value__1__date_day [PASS in 6.33s]
18:00:44  50 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_group_id__10000__True [RUN]
18:00:44  51 of 104 START test dbt_expectations_equal_expression_data_test_ref_data_test___sum_col_numeric_a___idx [RUN]
18:00:45  43 of 104 PASS dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_hourly_data_extended_date_hour__hour [PASS in 7.59s]
18:00:45  52 of 104 START test dbt_expectations_equal_expression_data_test_sum_col_numeric_a_5___ref_data_test___sum_col_numeric_a___idx__0_5 [RUN]
18:00:45  48 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_10 [PASS in 5.17s]
18:00:45  53 of 104 START test dbt_expectations_equal_expression_data_test_sum_col_numeric_b___ref_data_test___sum_col_numeric_a_ [RUN]
18:00:48  49 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_date_day__10 [PASS in 6.04s]
18:00:48  54 of 104 START test dbt_expectations_expect_column_distinct_values_to_be_in_set_data_test_col_string_a__True__a__b__c__d [RUN]
18:00:48  46 of 104 PASS dbt_expectations_expect_grouped_row_values_to_have_recent_data_timeseries_data_grouped_day__group_id__1__group_id_4__date_day [PASS in 10.38s]
18:00:48  55 of 104 START test dbt_expectations_expect_column_distinct_values_to_contain_set_data_test_col_string_a__True__a__b [RUN]
18:00:48  42 of 104 PASS dbt_expectations_expect_column_values_to_be_within_n_moving_stdevs_timeseries_hourly_data_extended_row_value_log__cast_date_hour_as_dbt_expectations_type_datetime___hour__6__False__12__48 [PASS in 11.50s]
18:00:48  56 of 104 START test dbt_expectations_expect_column_distinct_values_to_equal_set_data_test_col_string_a__True__a__b__c__c [RUN]
18:00:49  37 of 104 PASS dbt_expectations_expect_row_values_to_have_data_for_every_n_datepart_timeseries_data_extended_date_day__day__7 [PASS in 14.02s]
18:00:49  57 of 104 START test dbt_expectations_expect_column_max_to_be_between_data_test_col_numeric_a__1__1 [RUN]
18:00:49  50 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_timeseries_data_grouped_group_id__10000__True [PASS in 5.42s]
18:00:49  58 of 104 START test dbt_expectations_expect_column_mean_to_be_between_data_test_col_numeric_a__1_5__0 [RUN]
18:00:50  57 of 104 PASS dbt_expectations_expect_column_max_to_be_between_data_test_col_numeric_a__1__1 [PASS in 1.35s]
18:00:50  59 of 104 START test dbt_expectations_expect_column_min_to_be_between_data_test_col_numeric_a__0__0 [RUN]
18:00:51  59 of 104 PASS dbt_expectations_expect_column_min_to_be_between_data_test_col_numeric_a__0__0 [PASS in 0.77s]
18:00:51  60 of 104 START test dbt_expectations_expect_column_most_common_value_to_be_in_set_data_test_col_numeric_a__1__0_5 [RUN]
18:00:51  51 of 104 PASS dbt_expectations_equal_expression_data_test_ref_data_test___sum_col_numeric_a___idx [PASS in 6.88s]
18:00:51  61 of 104 START test dbt_expectations_expect_column_pair_values_A_to_be_greater_than_B_data_test_col_numeric_a_10__col_numeric_a [RUN]
18:00:51  52 of 104 PASS dbt_expectations_equal_expression_data_test_sum_col_numeric_a_5___ref_data_test___sum_col_numeric_a___idx__0_5 [PASS in 6.21s]
18:00:51  62 of 104 START test dbt_expectations_expect_column_pair_values_A_to_be_greater_than_B_data_test_col_numeric_a__col_numeric_a__True [RUN]
18:00:51  53 of 104 PASS dbt_expectations_equal_expression_data_test_sum_col_numeric_b___ref_data_test___sum_col_numeric_a_ [PASS in 6.20s]
18:00:51  63 of 104 START test dbt_expectations_expect_column_pair_values_to_be_equal_data_test_col_numeric_a__col_numeric_a [RUN]
18:00:54  55 of 104 PASS dbt_expectations_expect_column_distinct_values_to_contain_set_data_test_col_string_a__True__a__b [PASS in 5.69s]
18:00:54  64 of 104 START test dbt_expectations_expect_column_pair_values_to_be_in_set_data_test_col_numeric_a__col_numeric_b___0_1____1_0____0_5_0_5____0_5_0_5_ [RUN]
18:00:55  56 of 104 PASS dbt_expectations_expect_column_distinct_values_to_equal_set_data_test_col_string_a__True__a__b__c__c [PASS in 7.06s]
18:00:55  65 of 104 START test dbt_expectations_expect_column_proportion_of_unique_values_to_be_between_data_test_col_numeric_a__0_75__0 [RUN]
18:00:55  54 of 104 PASS dbt_expectations_expect_column_distinct_values_to_be_in_set_data_test_col_string_a__True__a__b__c__d [PASS in 7.56s]
18:00:55  66 of 104 START test dbt_expectations_expect_column_stdev_to_be_between_data_test_col_numeric_a__0__True [RUN]
18:00:56  58 of 104 PASS dbt_expectations_expect_column_mean_to_be_between_data_test_col_numeric_a__1_5__0 [PASS in 6.23s]
18:00:56  67 of 104 START test dbt_expectations_expect_column_stdev_to_be_between_data_test_col_numeric_a__2__0 [RUN]
18:00:57  61 of 104 PASS dbt_expectations_expect_column_pair_values_A_to_be_greater_than_B_data_test_col_numeric_a_10__col_numeric_a [PASS in 5.80s]
18:00:57  68 of 104 START test dbt_expectations_expect_column_sum_to_be_between_data_test_col_numeric_a__3__1 [RUN]
18:00:57  62 of 104 PASS dbt_expectations_expect_column_pair_values_A_to_be_greater_than_B_data_test_col_numeric_a__col_numeric_a__True [PASS in 6.03s]
18:00:57  69 of 104 START test dbt_expectations_expect_column_to_exist_data_test_1__idx... [RUN]
18:00:57  63 of 104 PASS dbt_expectations_expect_column_pair_values_to_be_equal_data_test_col_numeric_a__col_numeric_a [PASS in 5.87s]
18:00:57  70 of 104 START test dbt_expectations_expect_column_to_exist_data_test_3__col_numeric_a [RUN]
18:00:58  69 of 104 PASS dbt_expectations_expect_column_to_exist_data_test_1__idx......... [PASS in 0.80s]
18:00:58  71 of 104 START test dbt_expectations_expect_column_to_exist_data_test_col_numeric_a [RUN]
18:00:58  70 of 104 PASS dbt_expectations_expect_column_to_exist_data_test_3__col_numeric_a [PASS in 0.77s]
18:00:58  72 of 104 START test dbt_expectations_expect_column_unique_value_count_to_be_between_data_test_col_numeric_a__3__3 [RUN]
18:00:58  68 of 104 PASS dbt_expectations_expect_column_sum_to_be_between_data_test_col_numeric_a__3__1 [PASS in 1.22s]
18:00:58  73 of 104 START test dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__1 [RUN]
18:00:58  71 of 104 PASS dbt_expectations_expect_column_to_exist_data_test_col_numeric_a.. [PASS in 0.64s]
18:00:58  74 of 104 START test dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__4 [RUN]
18:00:59  72 of 104 PASS dbt_expectations_expect_column_unique_value_count_to_be_between_data_test_col_numeric_a__3__3 [PASS in 1.04s]
18:00:59  75 of 104 START test dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__4__1 [RUN]
18:00:59  64 of 104 PASS dbt_expectations_expect_column_pair_values_to_be_in_set_data_test_col_numeric_a__col_numeric_b___0_1____1_0____0_5_0_5____0_5_0_5_ [PASS in 5.11s]
18:00:59  76 of 104 START test dbt_expectations_expect_column_value_lengths_to_equal_data_test_col_string_a__1 [RUN]
18:01:01  66 of 104 PASS dbt_expectations_expect_column_stdev_to_be_between_data_test_col_numeric_a__0__True [PASS in 5.92s]
18:01:01  77 of 104 START test dbt_expectations_expect_column_values_to_be_between_data_test_col_numeric_a__0 [RUN]
18:01:02  67 of 104 PASS dbt_expectations_expect_column_stdev_to_be_between_data_test_col_numeric_a__2__0 [PASS in 6.00s]
18:01:02  78 of 104 START test dbt_expectations_expect_column_values_to_be_between_data_test_col_numeric_a__1__0 [RUN]
18:01:02  78 of 104 PASS dbt_expectations_expect_column_values_to_be_between_data_test_col_numeric_a__1__0 [PASS in 0.33s]
18:01:02  79 of 104 START test dbt_expectations_expect_column_values_to_be_in_set_data_test_col_string_a__True__a__b__c [RUN]
18:01:02  65 of 104 PASS dbt_expectations_expect_column_proportion_of_unique_values_to_be_between_data_test_col_numeric_a__0_75__0 [PASS in 6.80s]
18:01:02  80 of 104 START test dbt_expectations_expect_column_values_to_be_increasing_data_test_col_numeric_a__col_numeric_a__False [RUN]
18:01:03  60 of 104 PASS dbt_expectations_expect_column_most_common_value_to_be_in_set_data_test_col_numeric_a__1__0_5 [PASS in 12.70s]
18:01:03  81 of 104 START test dbt_expectations_expect_column_values_to_be_increasing_data_test_idx__idx [RUN]
18:01:04  73 of 104 PASS dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__1 [PASS in 5.81s]
18:01:04  82 of 104 START test dbt_expectations_expect_column_values_to_be_null_data_test_col_null [RUN]
18:01:04  74 of 104 PASS dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__4 [PASS in 6.13s]
18:01:04  83 of 104 START test dbt_expectations_expect_column_values_to_be_unique_data_test_idx [RUN]
18:01:05  83 of 104 PASS dbt_expectations_expect_column_values_to_be_unique_data_test_idx. [PASS in 0.27s]
18:01:05  84 of 104 START test dbt_expectations_expect_column_values_to_not_be_in_set_data_test_col_string_a__True__2__3 [RUN]
18:01:05  75 of 104 PASS dbt_expectations_expect_column_value_lengths_to_be_between_data_test_col_string_b__4__1 [PASS in 6.26s]
18:01:05  85 of 104 START test dbt_expectations_expect_column_values_to_not_be_in_set_data_test_col_string_b__True__a__c [RUN]
18:01:05  76 of 104 PASS dbt_expectations_expect_column_value_lengths_to_equal_data_test_col_string_a__1 [PASS in 6.36s]
18:01:05  86 of 104 START test dbt_expectations_expect_column_values_to_not_be_null_data_test_col_numeric_a [RUN]
18:01:05  86 of 104 PASS dbt_expectations_expect_column_values_to_not_be_null_data_test_col_numeric_a [PASS in 0.30s]
18:01:05  87 of 104 START test dbt_expectations_expect_compound_columns_to_be_unique_data_test_date_col__col_string_b__any_value_is_missing [RUN]
18:01:06  77 of 104 PASS dbt_expectations_expect_column_values_to_be_between_data_test_col_numeric_a__0 [PASS in 5.00s]
18:01:06  88 of 104 START test dbt_expectations_expect_multicolumn_sum_to_equal_data_test_col_numeric_a__col_numeric_b__4 [RUN]
18:01:08  80 of 104 PASS dbt_expectations_expect_column_values_to_be_increasing_data_test_col_numeric_a__col_numeric_a__False [PASS in 5.54s]
18:01:08  89 of 104 START test dbt_expectations_expect_select_column_values_to_be_unique_within_record_data_test_col_string_a__col_string_b__any_value_is_missing [RUN]
18:01:08  79 of 104 PASS dbt_expectations_expect_column_values_to_be_in_set_data_test_col_string_a__True__a__b__c [PASS in 5.80s]
18:01:08  90 of 104 START test dbt_expectations_expect_table_column_count_to_be_between_data_test_1 [RUN]
18:01:08  90 of 104 PASS dbt_expectations_expect_table_column_count_to_be_between_data_test_1 [PASS in 0.50s]
18:01:08  91 of 104 START test dbt_expectations_expect_table_column_count_to_be_between_data_test_10 [RUN]
18:01:09  91 of 104 PASS dbt_expectations_expect_table_column_count_to_be_between_data_test_10 [PASS in 0.49s]
18:01:09  92 of 104 START test dbt_expectations_expect_table_column_count_to_be_between_data_test_10__1 [RUN]
18:01:09  81 of 104 PASS dbt_expectations_expect_column_values_to_be_increasing_data_test_idx__idx [PASS in 5.68s]
18:01:09  93 of 104 START test dbt_expectations_expect_table_column_count_to_equal_data_test_7 [RUN]
18:01:09  92 of 104 PASS dbt_expectations_expect_table_column_count_to_be_between_data_test_10__1 [PASS in 0.57s]
18:01:09  94 of 104 START test dbt_expectations_expect_table_column_count_to_equal_other_table_data_test_ref_data_test_ [RUN]
18:01:10  93 of 104 PASS dbt_expectations_expect_table_column_count_to_equal_data_test_7.. [PASS in 0.46s]
18:01:10  95 of 104 START test dbt_expectations_expect_table_columns_to_contain_set_data_test_col_numeric_b__col_string_a [RUN]
18:01:10  95 of 104 PASS dbt_expectations_expect_table_columns_to_contain_set_data_test_col_numeric_b__col_string_a [PASS in 0.50s]
18:01:10  96 of 104 START test dbt_expectations_expect_table_columns_to_match_ordered_list_data_test_idx__date_col__col_numeric_a__col_numeric_b__col_string_a__col_string_b__col_null [RUN]
18:01:10  94 of 104 PASS dbt_expectations_expect_table_column_count_to_equal_other_table_data_test_ref_data_test_ [PASS in 0.79s]
18:01:10  97 of 104 START test dbt_expectations_expect_table_columns_to_match_set_data_test_idx__date_col__col_numeric_a__col_numeric_b__col_string_a__col_string_b__col_null [RUN]
18:01:11  82 of 104 PASS dbt_expectations_expect_column_values_to_be_null_data_test_col_null [PASS in 6.98s]
18:01:11  98 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_data_test_1 [RUN]
18:01:11  96 of 104 PASS dbt_expectations_expect_table_columns_to_match_ordered_list_data_test_idx__date_col__col_numeric_a__col_numeric_b__col_string_a__col_string_b__col_null [PASS in 0.75s]
18:01:11  99 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_data_test_4 [RUN]
18:01:11  97 of 104 PASS dbt_expectations_expect_table_columns_to_match_set_data_test_idx__date_col__col_numeric_a__col_numeric_b__col_string_a__col_string_b__col_null [PASS in 0.67s]
18:01:11  100 of 104 START test dbt_expectations_expect_table_row_count_to_be_between_data_test_4__1 [RUN]
18:01:11  84 of 104 PASS dbt_expectations_expect_column_values_to_not_be_in_set_data_test_col_string_a__True__2__3 [PASS in 6.08s]
18:01:11  101 of 104 START test dbt_expectations_expect_table_row_count_to_equal_data_test_4 [RUN]
18:01:11  85 of 104 PASS dbt_expectations_expect_column_values_to_not_be_in_set_data_test_col_string_b__True__a__c [PASS in 6.17s]
18:01:11  102 of 104 START test dbt_expectations_expect_table_row_count_to_equal_other_table_data_test_ref_data_test___1_1__1_1 [RUN]
18:01:11  101 of 104 PASS dbt_expectations_expect_table_row_count_to_equal_data_test_4.... [PASS in 0.42s]
18:01:11  103 of 104 START test dbt_expectations_expect_table_row_count_to_equal_other_table_times_factor_data_test_factored_ref_data_test___2 [RUN]
18:01:12  99 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_data_test_4 [PASS in 0.87s]
18:01:12  104 of 104 START test dbt_expectations_expression_is_true_data_test__col_numeric_a_col_numeric_b_1_ [RUN]
18:01:12  100 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_data_test_4__1 [PASS in 1.04s]
18:01:12  103 of 104 PASS dbt_expectations_expect_table_row_count_to_equal_other_table_times_factor_data_test_factored_ref_data_test___2 [PASS in 0.76s]
18:01:12  87 of 104 PASS dbt_expectations_expect_compound_columns_to_be_unique_data_test_date_col__col_string_b__any_value_is_missing [PASS in 6.83s]
18:01:13  88 of 104 PASS dbt_expectations_expect_multicolumn_sum_to_equal_data_test_col_numeric_a__col_numeric_b__4 [PASS in 6.62s]
18:01:16  98 of 104 PASS dbt_expectations_expect_table_row_count_to_be_between_data_test_1 [PASS in 5.40s]
18:01:18  102 of 104 PASS dbt_expectations_expect_table_row_count_to_equal_other_table_data_test_ref_data_test___1_1__1_1 [PASS in 6.62s]
18:01:18  104 of 104 PASS dbt_expectations_expression_is_true_data_test__col_numeric_a_col_numeric_b_1_ [PASS in 6.32s]
18:01:24  89 of 104 PASS dbt_expectations_expect_select_column_values_to_be_unique_within_record_data_test_col_string_a__col_string_b__any_value_is_missing [PASS in 16.73s]
18:01:25
18:01:25  Finished running 11 table models, 93 tests in 78.36s.
18:01:25
18:01:25  Completed successfully
18:01:25
18:01:25  Done. PASS=104 WARN=0 ERROR=0 SKIP=0 TOTAL=104

In some circumstances, Redshift will fail to execute queries generated by `expect_column_values_to_be_in_type_list` if it is unable infer the type of the `relation_column` or `relation_column_type` fields. This commit enforces a casting of those values to a string type to prevent ambiguity.
@nicholasyager nicholasyager changed the title Cast relation_column and relation_column_type to string type Fix expect_column_values_to_be_in_type_list for Redshift Dec 6, 2021
@nicholasyager nicholasyager changed the title Fix expect_column_values_to_be_in_type_list for Redshift Fix expect_column_values_to_be_in_type_list for Redshift in 0.5.0 Dec 6, 2021
@clausherther
Copy link
Contributor

Hi @nicholasyager ! I feel like this Redshift bug keeps coming back, or maybe it's surfacing in different parts of the code base. Thanks for raising this and your PR!

Copy link
Contributor

@clausherther clausherther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@clausherther clausherther merged commit ce3a4ed into calogica:main Dec 8, 2021
@clausherther
Copy link
Contributor

I'll probably do a pass through the repo to find other cases like that, and if I can get a PR out before the end of the week, I'll combine that into a new release (0.5.1), or I'll cut a new release by EOW. Thanks!

@nicholasyager
Copy link
Contributor Author

I'll probably do a pass through the repo to find other cases like that, and if I can get a PR out before the end of the week, I'll combine that into a new release (0.5.1), or I'll cut a new release by EOW. Thanks!

Sounds great 👍 Thanks for the review, @clausherther!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expect column values types to be in type list fails in redshift
2 participants