Tracking issues of Reading Iceberg Table Support

[Xuanwo]

This issue is used to track the progress of reading iceberg table data.

After #11923, databend has the basic functional support for reading iceberg table data. But there are still a lot of work to do.

Powered by icelake

Show time

CREATE CATALOG iceberg_ctl
TYPE=ICEBERG
CONNECTION=(
    URL='s3://testbucket/iceberg_ctl/'
    AWS_KEY_ID='minioadmin'
    AWS_SECRET_KEY='minioadmin'
    ENDPOINT_URL='http://127.0.0.1:9900'
);

SELECT * FROM iceberg_ctl.iceberg_db.iceberg_tbl;

Tasks

• feat: Allow persisit user-defined catalog in metasrv #11948
• feat(catalog): Add get_catalog support and schema api test #11964
• Store catalog in metasrv instead of in-memory
• Implement drop catalog support (with metasrv)
• Implement show catalogs support (from metasrv)

Research

Merge parquet reader

Iceberg table now read data via icelake which uses parquet. Databend has three different implementations of parquet read. Maybe we should find a way to merge the implementations of parquet files.

Remove the global context of iceberg tables

Databend doesn't support init a table in async way, which enforce us to store the table info gloablly in context. We should find a way to remove it.

[Xuanwo]

The iceberg is now ready to be read (without optimization). Let's move on to the next stage.