Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metabase sync fails with Code: 51. DB::Exception: Empty list of columns in SELECT query. (EMPTY_LIST_OF_COLUMNS_QUERIED) #256

Open
Gerrit-K opened this issue Jul 10, 2024 · 11 comments
Labels
blocked-by-clickhouse bug Something isn't working

Comments

@Gerrit-K
Copy link

Describe the bug

Not sure if this is something that the metabase-clickhouse-driver can fix or if it needs to be reported to metabase/metabase.

We recently upgraded ClickHouse from 24.1 to 24.5 and also Metabase from 0.47 to 0.50. After the upgrades, we noticed that the column sync now fails with:

java.sql.BatchUpdateException: Code: 51. DB::Exception: Empty list of columns in SELECT query. (EMPTY_LIST_OF_COLUMNS_QUERIED) (version 24.5.4.49 (official build))

By checking the query log, we identified that this was particularly failing for a parameterized view when running

SELECT * FROM `<database>`.`<some_parameterized_view>` WHERE 1 <> 1 LIMIT 0

This leads us to believe it has something to do with ClickHouse/ClickHouse#65402. Though, by unfolding the order of events, it seemed that this was only happening with the combination of both, the Metabase and the ClickHouse upgrade:

  • Syncing ClickHouse 24.5 on Metabase 0.47 worked
  • Syncing ClickHouse 24.1 on Metabase 0.50 worked
  • Syncing ClickHouse 24.5 on Metabase 0.50 did not work

Steps to reproduce

  1. Create a parameterized view in ClickHouse 24.5
  2. Connect ClickHouse to Metabase (0.50+)
  3. Start a full sync
  4. Observe the sync logs

Expected behaviour

Parameterized views should not cause the full sync to fail

Error log

Stack trace
clojure.lang.ExceptionInfo: Code: 51. DB::Exception: Empty list of columns in SELECT query. (EMPTY_LIST_OF_COLUMNS_QUERIED) (version 24.5.4.49 (official build))

	at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107) ~[clickhouse.metabase-driver.jar:?]
	at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeAny(SqlBasedPreparedStatement.java:219) ~[clickhouse.metabase-driver.jar:?]
	at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeQuery(SqlBasedPreparedStatement.java:246) ~[clickhouse.metabase-driver.jar:?]
	at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:1471) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$fallback_fields_metadata_from_select_query$reify__83213.reduce(describe_table.clj:89) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6886) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$fields_metadata$reify$reify__83237.reduce(describe_table.clj:156) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6886) ~[metabase.jar:?]
	at clojure.core$cat$fn__8851.invoke(core.clj:7694) ~[metabase.jar:?]
	at clojure.lang.PersistentVector.reduce(PersistentVector.java:343) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6886) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$fields_metadata$reify__83232.reduce(describe_table.clj:164) ~[metabase.jar:?]
	at clojure.core$transduce.invokeStatic(core.clj:6947) ~[metabase.jar:?]
	at clojure.core$into.invokeStatic(core.clj:6963) ~[metabase.jar:?]
	at clojure.core$into.invoke(core.clj:6951) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$fn__83253.invokeStatic(describe_table.clj:210) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$fn__83253.invoke(describe_table.clj:208) ~[metabase.jar:?]
	at clojure.lang.MultiFn.invoke(MultiFn.java:244) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table_STAR_.invokeStatic(describe_table.clj:252) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table_STAR_.invoke(describe_table.clj:246) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table_STAR_.invokeStatic(describe_table.clj:248) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table_STAR_.invoke(describe_table.clj:246) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table$fn__83285.invoke(describe_table.clj:264) ~[metabase.jar:?]
	at metabase.driver.clickhouse$fn__112984$fn__112986.invoke(clickhouse.clj:124) ~[clickhouse.metabase-driver.jar:?]
	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invokeStatic(execute.clj:337) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.execute$do_with_resolved_connection.invoke(execute.clj:320) ~[metabase.jar:?]
	at metabase.driver.clickhouse$fn__112984.invokeStatic(clickhouse.clj:84) ~[clickhouse.metabase-driver.jar:?]
	at metabase.driver.clickhouse$fn__112984.invoke(clickhouse.clj:82) ~[clickhouse.metabase-driver.jar:?]
	at clojure.lang.MultiFn.invoke(MultiFn.java:244) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table.invokeStatic(describe_table.clj:259) ~[metabase.jar:?]
	at metabase.driver.sql_jdbc.sync.describe_table$describe_table.invoke(describe_table.clj:256) ~[metabase.jar:?]
	at metabase.driver.clickhouse_introspection$fn__108842.invokeStatic(clickhouse_introspection.clj:160) ~[clickhouse.metabase-driver.jar:?]
	at metabase.driver.clickhouse_introspection$fn__108842.invoke(clickhouse_introspection.clj:158) ~[clickhouse.metabase-driver.jar:?]
	at clojure.lang.MultiFn.invoke(MultiFn.java:239) ~[metabase.jar:?]
	at metabase.sync.fetch_metadata$table_fields_metadata.invokeStatic(fetch_metadata.clj:46) ~[metabase.jar:?]
	at metabase.sync.fetch_metadata$table_fields_metadata.invoke(fetch_metadata.clj:33) ~[metabase.jar:?]
	at metabase.sync.fetch_metadata$describe_fields_using_describe_table$fn__84155.invoke(fetch_metadata.clj:61) ~[metabase.jar:?]
	at clojure.core$map$fn__5931$fn__5932.invoke(core.clj:2759) ~[metabase.jar:?]
	at toucan2.pipeline$with_init$fn__21386.invoke(pipeline.clj:327) ~[metabase.jar:?]
	at clojure.core$completing$fn__8528.invoke(core.clj:6932) ~[metabase.jar:?]
	at clojure.core$map$fn__5931$fn__5932.invoke(core.clj:2759) ~[metabase.jar:?]
	at toucan2.jdbc.result_set$reduce_result_set.invokeStatic(result_set.clj:158) ~[metabase.jar:?]
	at toucan2.jdbc.result_set$reduce_result_set.invoke(result_set.clj:125) ~[metabase.jar:?]
	at toucan2.jdbc.query$reduce_jdbc_query.invokeStatic(query.clj:51) ~[metabase.jar:?]
	at toucan2.jdbc.query$reduce_jdbc_query.invoke(query.clj:22) ~[metabase.jar:?]
	at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invokeStatic(pipeline.clj:19) ~[metabase.jar:?]
	at toucan2.jdbc.pipeline$transduce_execute_with_connection_primary_method_java_sql_Connection_default_default.invoke(pipeline.clj:9) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.doInvoke(core.clj:2639) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:667) ~[metabase.jar:?]
	at clojure.core$apply.invoke(core.clj:662) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$fn__18073$fn__18074$fn__18081.invoke(threaded.clj:79) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043$fn__18047.invoke(threaded.clj:23) ~[metabase.jar:?]
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invokeStatic(protocols.clj:136) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invoke(protocols.clj:124) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19) ~[metabase.jar:?]
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invoke(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6887) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043.invoke(threaded.clj:21) ~[metabase.jar:?]
	at clojure.core$comp$fn__5876.doInvoke(core.clj:2589) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$apply.doInvoke(core.clj:662) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:533) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$combine_with_threader$fn__18053.doInvoke(threaded.clj:46) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:151) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$apply.doInvoke(core.clj:662) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:533) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.doInvoke(standard.clj:47) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:594) ~[metabase.jar:?]
	at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:199) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_execute$with_connection_STAR___21306.invoke(pipeline.clj:78) ~[metabase.jar:?]
	at toucan2.connection$bind_current_connectable_fn$fn__20983.invoke(connection.clj:104) ~[metabase.jar:?]
	at toucan2.connection$bind_current_connectable_fn$fn__20983.invoke(connection.clj:104) ~[metabase.jar:?]
	at toucan2.connection$bind_current_connectable_fn$fn__20983.invoke(connection.clj:104) ~[metabase.jar:?]
	at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invokeStatic(connection.clj:18) ~[metabase.jar:?]
	at toucan2.jdbc.connection$do_with_connection_primary_method_javax_sql_DataSource.invoke(connection.clj:15) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$fn__18073$fn__18074$fn__18075.invoke(threaded.clj:70) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043$fn__18047.invoke(threaded.clj:23) ~[metabase.jar:?]
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invokeStatic(protocols.clj:136) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invoke(protocols.clj:124) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19) ~[metabase.jar:?]
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invoke(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6887) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043.invoke(threaded.clj:21) ~[metabase.jar:?]
	at clojure.core$comp$fn__5876.invoke(core.clj:2587) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$combine_with_threader$fn__18053.invoke(threaded.clj:43) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47) ~[metabase.jar:?]
	at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:193) ~[metabase.jar:?]
	at metabase.db.connection$do_with_connection_primary_method_default.invokeStatic(connection.clj:132) ~[metabase.jar:?]
	at metabase.db.connection$do_with_connection_primary_method_default.invoke(connection.clj:130) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$fn__18073$fn__18074$fn__18075.invoke(threaded.clj:70) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043$fn__18047.invoke(threaded.clj:23) ~[metabase.jar:?]
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invokeStatic(protocols.clj:136) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invoke(protocols.clj:124) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19) ~[metabase.jar:?]
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invoke(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6887) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043.invoke(threaded.clj:21) ~[metabase.jar:?]
	at clojure.core$comp$fn__5876.invoke(core.clj:2587) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$combine_with_threader$fn__18053.invoke(threaded.clj:43) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47) ~[metabase.jar:?]
	at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:193) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_primary_method_.invokeStatic(connection.clj:204) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_primary_method_.invoke(connection.clj:194) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$fn__18073$fn__18074$fn__18075.invoke(threaded.clj:70) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043$fn__18047.invoke(threaded.clj:23) ~[metabase.jar:?]
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invokeStatic(protocols.clj:136) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invoke(protocols.clj:124) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19) ~[metabase.jar:?]
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invoke(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6887) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043.invoke(threaded.clj:21) ~[metabase.jar:?]
	at clojure.core$comp$fn__5876.invoke(core.clj:2587) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$combine_with_threader$fn__18053.invoke(threaded.clj:43) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invokeStatic(connection.clj:118) ~[metabase.jar:?]
	at toucan2.connection$do_with_connection_around_method_toucan2_connection_default.invoke(connection.clj:106) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:436) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.invoke(core.clj:2642) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:421) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:55) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invoke(standard.clj:47) ~[metabase.jar:?]
	at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:193) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_execute.invokeStatic(pipeline.clj:77) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_execute.invoke(pipeline.clj:64) ~[metabase.jar:?]
	at clojure.lang.Var.invoke(Var.java:399) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_compiled_query.invokeStatic(pipeline.clj:244) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_compiled_query.invoke(pipeline.clj:240) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_built_query.invokeStatic(pipeline.clj:252) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_built_query.invoke(pipeline.clj:246) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_query_primary_method_default.invokeStatic(pipeline.clj:272) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_query_primary_method_default.invoke(pipeline.clj:269) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$partial$fn__5908.doInvoke(core.clj:2639) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:667) ~[metabase.jar:?]
	at clojure.core$apply.invoke(core.clj:662) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$fn__18073$fn__18074$fn__18081.invoke(threaded.clj:79) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043$fn__18047.invoke(threaded.clj:23) ~[metabase.jar:?]
	at clojure.lang.ArrayChunk.reduce(ArrayChunk.java:58) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invokeStatic(protocols.clj:136) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8244.invoke(protocols.clj:124) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8204$G__8199__8213.invoke(protocols.clj:19) ~[metabase.jar:?]
	at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invokeStatic(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8236.invoke(protocols.clj:75) ~[metabase.jar:?]
	at clojure.core.protocols$fn__8178$G__8173__8191.invoke(protocols.clj:13) ~[metabase.jar:?]
	at clojure.core$reduce.invokeStatic(core.clj:6887) ~[metabase.jar:?]
	at clojure.core$reduce.invoke(core.clj:6869) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$reducer_fn$fn__18043.invoke(threaded.clj:21) ~[metabase.jar:?]
	at clojure.core$comp$fn__5876.doInvoke(core.clj:2589) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$apply.doInvoke(core.clj:662) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:533) ~[metabase.jar:?]
	at methodical.impl.combo.threaded$combine_with_threader$fn__18053.doInvoke(threaded.clj:46) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:151) ~[metabase.jar:?]
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) ~[metabase.jar:?]
	at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:675) ~[metabase.jar:?]
	at clojure.core$apply.doInvoke(core.clj:662) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:533) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.invokeStatic(standard.clj:65) ~[metabase.jar:?]
	at methodical.impl.standard$invoke_multifn.doInvoke(standard.clj:47) ~[metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:594) ~[metabase.jar:?]
	at methodical.impl.standard.StandardMultiFn.invoke(standard.clj:199) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_query_STAR_.invokeStatic(pipeline.clj:278) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_query_STAR_.invoke(pipeline.clj:274) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_with_model.invokeStatic(pipeline.clj:293) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_with_model.invoke(pipeline.clj:280) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_parsed.invokeStatic(pipeline.clj:309) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_parsed.invoke(pipeline.clj:295) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_unparsed.invokeStatic(pipeline.clj:317) ~[metabase.jar:?]
	at toucan2.pipeline$transduce_unparsed.invoke(pipeline.clj:311) ~[metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:160) ~[metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) ~[metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:669) ~[metabase.jar:?]
	at clojure.core$apply.invoke(core.clj:662) ~[metabase.jar:?]
	at toucan2.pipeline$reducible_fn$reify__21406.reduce(pipeline.clj:386) ~[metabase.jar:?]
	at clojure.core$transduce.invokeStatic(core.clj:6947) ~[metabase.jar:?]
	at clojure.core.Eduction.reduce(core.clj:7751) ~[metabase.jar:?]
	at clojure.core$transduce.invokeStatic(core.clj:6947) ~[metabase.jar:?]
	at clojure.core$transduce.invoke(core.clj:6934) ~[metabase.jar:?]
	at metabase.sync.sync_metadata.fields$sync_fields_BANG_$fn__84217.invoke(fields.clj:82) ~[metabase.jar:?]
	at metabase.sync.util$do_with_error_handling.invokeStatic(util.clj:188) [metabase.jar:?]
	at metabase.sync.util$do_with_error_handling.invoke(util.clj:181) [metabase.jar:?]
	at metabase.sync.sync_metadata.fields$sync_fields_BANG_.invokeStatic(fields.clj:76) [metabase.jar:?]
	at metabase.sync.sync_metadata.fields$sync_fields_BANG_.invoke(fields.clj:71) [metabase.jar:?]
	at clojure.lang.AFn.applyToHelper(AFn.java:154) [metabase.jar:?]
	at clojure.lang.AFn.applyTo(AFn.java:144) [metabase.jar:?]
	at clojure.core$apply.invokeStatic(core.clj:669) [metabase.jar:?]
	at clojure.core$apply.invoke(core.clj:662) [metabase.jar:?]
	at metabase.sync.util$run_step_with_metadata$fn__57417$fn__57420.invoke(util.clj:470) [metabase.jar:?]
	at metabase.models.task_history$do_with_task_history.invokeStatic(task_history.clj:116) [metabase.jar:?]
	at metabase.models.task_history$do_with_task_history.invoke(task_history.clj:105) [metabase.jar:?]
	at metabase.sync.util$run_step_with_metadata$fn__57417.doInvoke(util.clj:464) [metabase.jar:?]
	at clojure.lang.RestFn.invoke(RestFn.java:397) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_logging_STAR_.invokeStatic(util.clj:129) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_logging_STAR_.invoke(util.clj:123) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_debug_logging.invokeStatic(util.clj:147) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_debug_logging.invoke(util.clj:143) [metabase.jar:?]
	at metabase.sync.util$run_step_with_metadata.invokeStatic(util.clj:459) [metabase.jar:?]
	at metabase.sync.util$run_step_with_metadata.invoke(util.clj:454) [metabase.jar:?]
	at metabase.sync.util$run_sync_operation$fn__57459$fn__57467.invoke(util.clj:543) [metabase.jar:?]
	at metabase.sync.util$run_sync_operation$fn__57459.invoke(util.clj:541) [metabase.jar:?]
	at metabase.models.task_history$do_with_task_history.invokeStatic(task_history.clj:116) [metabase.jar:?]
	at metabase.models.task_history$do_with_task_history.invoke(task_history.clj:105) [metabase.jar:?]
	at metabase.sync.util$run_sync_operation.invokeStatic(util.clj:538) [metabase.jar:?]
	at metabase.sync.util$run_sync_operation.invoke(util.clj:533) [metabase.jar:?]
	at metabase.sync.sync_metadata$sync_db_metadata_BANG_$fn__84717.invoke(sync_metadata.clj:70) [metabase.jar:?]
	at metabase.sync.util$do_with_error_handling.invokeStatic(util.clj:188) [metabase.jar:?]
	at metabase.sync.util$do_with_error_handling.invoke(util.clj:181) [metabase.jar:?]
	at clojure.core$partial$fn__5910.invoke(core.clj:2647) [metabase.jar:?]
	at metabase.driver$fn__55255.invokeStatic(driver.clj:801) [metabase.jar:?]
	at metabase.driver$fn__55255.invoke(driver.clj:801) [metabase.jar:?]
	at clojure.lang.MultiFn.invoke(MultiFn.java:239) [metabase.jar:?]
	at metabase.sync.util$sync_in_context$fn__57329.invoke(util.clj:164) [metabase.jar:?]
	at metabase.sync.util$with_db_logging_disabled$fn__57326.invoke(util.clj:156) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_logging_STAR_.invokeStatic(util.clj:129) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_logging_STAR_.invoke(util.clj:123) [metabase.jar:?]
	at metabase.sync.util$with_start_and_finish_logging$fn__57313.invoke(util.clj:141) [metabase.jar:?]
	at metabase.sync.util$with_sync_events$fn__57308.invoke(util.clj:115) [metabase.jar:?]
	at metabase.sync.util$with_duplicate_ops_prevented$fn__57295.invoke(util.clj:87) [metabase.jar:?]
	at metabase.sync.util$do_sync_operation.invokeStatic(util.clj:213) [metabase.jar:?]
	at metabase.sync.util$do_sync_operation.invoke(util.clj:207) [metabase.jar:?]
	at metabase.sync.sync_metadata$sync_db_metadata_BANG_.invokeStatic(sync_metadata.clj:68) [metabase.jar:?]
	at metabase.sync.sync_metadata$sync_db_metadata_BANG_.invoke(sync_metadata.clj:65) [metabase.jar:?]
	at metabase.api.database$fn__94434$fn__94439.invoke(database.clj:1020) [metabase.jar:?]
	at clojure.core$binding_conveyor_fn$fn__5823.invoke(core.clj:2047) [metabase.jar:?]
	at clojure.lang.AFn.call(AFn.java:18) [metabase.jar:?]
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
	at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.sql.BatchUpdateException: Code: 51. DB::Exception: Empty list of columns in SELECT query. (EMPTY_LIST_OF_COLUMNS_QUERIED) (version 24.5.4.49 (official build))

	... 342 more

Configuration

Environment

  • metabase-clickhouse-driver version: 1.50.1
  • metabase-clickhouse-driver configuration: default/unchanged
  • Metabase version: 0.50.9
  • OS:

ClickHouse server

  • ClickHouse Server version: 24.5.4.49
  • ClickHouse Server non-default settings, if any:
  • CREATE TABLE statements for tables involved: any parameterized view, e.g. CREATE VIEW delete_me_param_view AS SELECT * FROM system.clusters WHERE cluster = {cluster:String}
@Gerrit-K Gerrit-K added the bug Something isn't working label Jul 10, 2024
@slvrtrn
Copy link
Collaborator

slvrtrn commented Jul 10, 2024

The only major change in the connection between 0.47 and 0.50 I could think of, aside from the JDBC driver upgrade, is this setting:
https://github.com/ClickHouse/metabase-clickhouse-driver/blob/1.50.1/src/metabase/driver/clickhouse.clj#L78-L79
so that the analyzer is disabled because of the mentioned bug.

The behavior on both versions of the analyzer is slightly different on 24.6 (but it looks like they still indicate the same issue):

I am currently not sure that it is the driver bug, either.

@Gerrit-K
Copy link
Author

Gerrit-K commented Aug 8, 2024

@slvrtrn I see you marked this as "blocked-by-clickhouse". Do you know if there is any movement on this? Or a related issue in ClickHouse that I could follow & bump (apart from the one I just linked)? Thanks!

@slvrtrn
Copy link
Collaborator

slvrtrn commented Aug 8, 2024

I added the flair because I too think that this could be related to ClickHouse/ClickHouse#65402

@novikd
Copy link
Member

novikd commented Sep 2, 2024

select * from delete_me_param_view where 1<>1 limit 0;

@slvrtrn @Gerrit-K The query you provide is not supposed to work. You must specify parameter values to select from parameterized view:

select * from delete_me_param_view(cluster = 'default') where 1<>1 limit 0;

@slvrtrn
Copy link
Collaborator

slvrtrn commented Sep 2, 2024

@novikd, thanks!
Metabase itself generates this default metadata query when it scans the database.

Theoretically, it could be possible to override the entirety of https://github.com/metabase/metabase/blob/v1.50.23.1/src/metabase/driver/sql_jdbc/sync/describe_table.clj#L205-L217, but it contains a significant amount of logic from the private methods, so it does not look like a good idea.

Additionally, we need to get cluster = 'default' param from somewhere, but it won't be possible from the scope of that function.

So I am still not sure how to resolve this on the driver side.

@novikd
Copy link
Member

novikd commented Sep 3, 2024

@slvrtrn Why does Metabase try to query parameterized views? It just doesn't make any sense. Parameterized view is almost a user-defined table function.

Additionally, we need to get cluster = 'default' param from somewhere, but it won't be possible from the scope of that function.

cluster = 'default' was just an example, I didn't mean that it's a solution.

@slvrtrn
Copy link
Collaborator

slvrtrn commented Sep 3, 2024

@novikd

Why does Metabase try to query parameterized views?

This is part of the database introspection. This query allows Metabase to figure out column names and types from the JDBC result set (which is supposed to be always empty, it just provides the column metadata) so that they can be displayed in the available tables list in the UI.

The more I think about it, the more it looks like we should just exclude parametrized views from the database introspection. However, this is strange:

Syncing ClickHouse 24.5 on Metabase 0.47 worked
Syncing ClickHouse 24.1 on Metabase 0.50 worked
Syncing ClickHouse 24.5 on Metabase 0.50 did not work

Especially given that the default Metabase introspection query was never changed on the driver side.

@novikd
Copy link
Member

novikd commented Sep 3, 2024

Syncing ClickHouse 24.1 on Metabase 0.50 worked

This can be explained by the change in parameterized views implementation in 24.2.

Syncing ClickHouse 24.5 on Metabase 0.50 did not work

Something definitely changed in 0.50.

This query allows Metabase to figure out column names and types from the JDBC result set (which is supposed to be always empty, it just provides the column metadata) so that they can be displayed in the available tables list in the UI.

It's strange to use SELECT query to find a table schema. For that purpose, it makes more sense to use DESCRIBE queries, but it wouldn't help in the case of parameterized views.

IMHO, Metabase should handle parametrized views differently.

@slvrtrn
Copy link
Collaborator

slvrtrn commented Sep 11, 2024

It's strange to use SELECT query to find a table schema. For that purpose, it makes more sense to use DESCRIBE queries, but it wouldn't help in the case of parameterized views.

It kind of is, but that's the thing with how JDBC drivers work. With an empty select, you still have ResultSetMetadata, which can be used to fetch the information about the columns and their types (not CH types, but their mapping to the standard JDBC ones which are then used internally). Then, for each specific JDBC type, we provide a function specifying how to process it, for example: https://github.com/ClickHouse/metabase-clickhouse-driver/blob/1.50.6/src/metabase/driver/clickhouse_qp.clj#L441-L449

Using DESCRIBE (or SELECT * FROM system.columns WHERE table = ...) will provide the ClickHouse types as strings in the ResultSet itself and not in ResultSetMetaData, and we will have to do the parsing/mapping on the Metabase driver side (instead of having it done by the JDBC driver). Plus, the override of the introspection logic has to be changed to custom implementation and then be maintained, which is complicated (see this comment).

@novikd
Copy link
Member

novikd commented Sep 16, 2024

@slvrtrn I mean that JDBC drivers should not try to infer a schema for parameterized view.

Let's look at it from another point:
Parameterized views represent user-defined table functions. JDBC driver doesn't try to get the schema for any embedded in the ClickHouse table function because such SELECT queries would fail (e.g., SELECT * FROM numbers() also doesn't work).

So in my opinion, parameterized views should be skipped. I don't know if there's any other way to show them in Metabase UI, but we definitely should treat them the same way as table functions.

@lainiwa
Copy link

lainiwa commented Sep 27, 2024

However, this is strange:
...
Especially given that the default Metabase introspection query was never changed on the driver side.

We started getting EMPTY_LIST_OF_COLUMNS_QUERIED errors in metabase logs
when we upgraded our clickhouse 23.8.9 -> 23.8.16.

Our metabase didn't change then (it was 0.48.8 and the plugin was 1.4.0).

So probably it was some patch tier change in clickhouse that was backported from 24.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked-by-clickhouse bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants