Iceberg Spark Extensions conflict with Paimon #10143
Description
Feature Request / Improvement
The Call syntax is defined in both Iceberg and Paimon, which may cause conflicts when I introduce their SparkSessionExtensions at the same time.
Reproduce:
spark.sql.extensions=org.apache.paimon.spark.extensions.PaimonSparkSessionExtensions,org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
-- create iceberg table
CREATE TABLE iceberg_catalog.sample.iceberg_t1 (
user_id BIGINT,
item_id BIGINT,
behavior STRING,
dt STRING,
hh STRING
) using iceberg;
-- create paimon table
CREATE TABLE paimon_catalog.sample.paimon_t1 (
user_id BIGINT,
item_id BIGINT,
behavior STRING,
dt STRING,
hh STRING
) TBLPROPERTIES (
'primary-key' = 'dt,hh,user_id'
);
-- Successed
CALL iceberg_catalog.system.remove_orphan_files(table => "sample.iceberg_t1");
-- Failed, use iceberg ResolveProcedures
CALL paimon_catalog.sys.remove_orphan_files(table => "sample.paimon_t1");
One idea:
If currentCatalog is not Spark Session catalog and Iceberg Spark Catalog, we first use delegate parser to parse sqlText in IcebergSparkSqlExtensionsParser#parsePlan.
Query engine
Spark