[Bug] paimon-generic catalog options don't agree with flink HiveCatalog's

[nonggialiang]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

0.8.0

Compute Engine

flink 1.17.1

Minimal reproduce step

use the following options to confiure a flink catalog:

• name: xxxx
  type: paimon-generic
  metastore: hive
  uri: thrift://xxxxxx:9093
  warehouse: hdfs://xxxx/warehouse
  default-database: test_fts
  hive-conf-dir: /data/app/hadoop/etc/hadoop
  table-default.metastore.partitioned-table: true

and the flink catalog failed to initialize:

Catalog options are:
'default-database'='test_fts'
'hive-conf-dir'='/data/app/hadoop/etc/hadoop'
'metastore'='hive'
'table-default.metastore.partitioned-table'='true'
'type'='paimon-generic'
'uri'='thrift://xxxxxx:9093'
'warehouse'='hdfs://xxxx/warehouse'
        at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:439) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
		...
Caused by: org.apache.flink.table.api.ValidationException: Unsupported options found for 'hive'.

Unsupported options:

metastore
table-default.metastore.partitioned-table
uri
warehouse

Supported options:

default-database
hadoop-conf-dir
hive-conf-dir
hive-version
property-version
        at org.apache.flink.table.factories.FactoryUtil.validateUnconsumedKeys(FactoryUtil.java:632) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at org.apache.flink.table.factories.FactoryUtil$FactoryHelper.validate(FactoryUtil.java:931) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        at org.apache.flink.table.catalog.hive.factories.HiveCatalogFactory.createCatalog(HiveCatalogFactory.java:69) ~[flink-sql-connector-hive-3.1.3_2.12-1.17.0.jar:1.17.0]
        at org.apache.paimon.flink.FlinkGenericCatalogFactory.createCatalog(FlinkGenericCatalogFactory.java:59) ~[?:?]
        at org.apache.paimon.flink.FlinkGenericCatalogFactory.createCatalog(FlinkGenericCatalogFactory.java:37) ~[?:?]
        at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:436) ~[flink-table-api-java-uber-1.17.1.jar:1.17.1]
        ... 64 more

What doesn't meet your expectations?

paimon-generic catalog options don't agree with flink HiveCatalog's

Anything else?

Maybe we should filtered out the invalid options for flink hive catalog from context before we pass it to HiveCatalogFactory.createCatalog(context) in FlinkGenericCatalogFactory

Are you willing to submit a PR?

• I'm willing to submit a PR!

[xiangyuf]

@nonggialiang same problem in #3738