bug fix #180
bug fix #180
Conversation
|
|
| @@ -46,7 +46,7 @@ | |||
| {% elif existing_relation.can_exchange %} | |||
| -- We can do an atomic exchange, so no need for an intermediate | |||
| {% call statement('main') -%} | |||
| {% do run_query(create_empty_table_from_relation(backup_relation, view_relation)) or '' %} | |||
| {{ create_empty_table_from_relation(backup_relation, view_relation) }} | |||
| {%- endcall %} | |||
| {% do exchange_tables_atomic(backup_relation, existing_relation) %} | |||
There was a problem hiding this comment.
In my opinion, we should modify exchange_tables_atomic for distributed tables. In my case(#179), the SYSTEM SYNC REPLICA on the distributed table bugged.
There was a problem hiding this comment.
Here the error message with the actual version.
...
�[0m12:10:06.320142 [debug] [Thread-1 ]: dbt_clickhouse adapter: Error running SQL: /* {"app": "dbt", "dbt_version": "1.4.6", "profile_name": "stage", "target_name": "clickhouse_stage", "node_id": "model.dbt_prototype_version.event"} */
SYSTEM SYNC REPLICA
ON CLUSTER "default" default_db_test_dbt.event
�[0m12:10:06.321265 [debug] [Thread-1 ]: Timing info for model.dbt_prototype_version.event (execute): 2023-08-18 12:10:05.375159 => 2023-08-18 12:10:06.321126
�[0m12:10:06.364239 [debug] [Thread-1 ]: Database Error in model event (models/event.sql)
Code: 36.
DB::Exception: There was an error on [clickhouse-server..........]: Code: 36. DB::Exception: Table default_db_test_dbt.event (zzzzzzzz.............zzzzzzzz) is not replicated. (BAD_ARGUMENTS) (version 23.3.1.2823 (official build)). Stack trace:
There was a problem hiding this comment.
Thanks for your work. IMO this is not only a bug for distributed_table materialization but for every other materialization using exchange_tables_atomic macro with cluster setting.
I will commit the change with more fixes.
|
I forked the project to make a better versioning on my side. Can you please tell me how do you think about the following modifications? It worked for me (multiple runs: well distributed, correct row counts). |
I think this fix makes sense -- @GUNC do you want to add it, or @zli06160 do you want to open a new PR? |
|
I think @zli06160 you should open a PR for this fix. I will close this PR. I am currently working on |
Summary
bug fix for distributed_table materialization